Skip to content
shimmer/docs
Reference

Python SDK

Install the Shimmer package and connect your Python agent to your workspace.

On this page

Install shimmer-sdk from your Python package manager. The package provides the shimmer CLI, the UserMessage harness input, and integration tools for collecting your agent’s behavior.

Install the package

terminal
pip install shimmer-sdk

Python 3.11 or newer is required. Install your agent’s dependencies in the same environment.

Harness input

Your harness exposes a synchronous run function that accepts a UserMessage:

python
from shimmer import UserMessage
from support_bot import answer


def run(message: UserMessage) -> None:
    answer(message.text)

message.text contains the input from a test case. The return value is ignored; Shimmer collects the instrumented interaction and evaluates that trace.

Point the CLI at this module with --path harness, or set the GitHub Action’s harness input to harness. See Write a Python harness for asynchronous agents and module layout.

Test execution

Use the CLI or the Shimmer GitHub Action to execute a suite. Both invoke your harness once per case and handle suite lookup, execution, and trace collection for you.

Execution and evaluation are separate: your agent can run successfully and still fail a behavioral criterion. Review the run and its assertions in your Shimmer workspace.

The runner reports missing or ambiguous suites, disabled tracing, callback exceptions, collection failures, and unsuccessful evaluation verdicts. See Run and compare for how to read results.

Data models

UserMessage is the input to your harness. Read message.text to get the case’s user input and pass it to your agent.

The model is frozen. Treat the input as immutable; keep agent-specific setup and state inside your harness or application.

Integrations

Use LangSmith instrumentation to capture the activity inside your harness. Set LANGSMITH_TRACING=local to collect for Shimmer without also uploading to LangSmith.

For recorded tool-result preparation with the synchronous OpenAI client, import shim_openai from shimmer.openai.