MockDrift drift replay

Gate 1 intent-binding tests for AI agents — replay schema drift locally, then pull fixtures from hosted watches in CI.

Prerequisites: Python 3.10+, the open-source driftguard repo, and (for cloud replay) a DriftGuard API key with mockdrift_cloud enabled.

What you will build

1. Run a local mockdrift demo against a bundled fixture.
2. Wrap a pytest with @drift_replay and a failure profile.
3. Enable MockDrift Cloud and copy a watch ID.
4. Replay drift from that watch with --simulate-drift in CI.

Step 1 — Install MockDrift

From the OSS monorepo, install the Python package in editable mode with dev extras (LangGraph reference agents):

cd driftguard/packages/mockdrift
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

Step 2 — Run a local demo

Smoke-test the harness without writing a test file. The stripe-required-field fixture simulates a breaking Stripe refund schema change:

mockdrift demo stripe-required-field

A PASS verdict means criteria (schema_valid, no_loop_spiral, etc.) succeeded for the golden proxy entry.

Step 3 — Add a drift-replay pytest

Decorate a test with @drift_replay. Point entry at your LangGraph (or use runner="custom" for plain Python). Pick a failure_profile that matches how your agent should react to tool errors:

from mockdrift import drift_replay

@drift_replay(
    fixture="stripe-required-field",
    runner="langgraph",
    entry="agents.billing.refund_graph:refund_graph",
    failure_profile="bubble_to_orchestrator",
)
def test_refund_bubbles(mockdrift_session):
    result = mockdrift_session.run()
    assert result.verdict == "PASS"

Run with:

mockdrift run --pytest tests/test_refund_drift.py -v

Step 4 — API key and MockDrift Cloud

Cloud replay requires authentication:

1. Copy your API key from /activate or the console Account → API keys panel.
2. Open Console → sidebar Products.
3. Enable MockDrift Cloud (mockdrift_cloud SKU).

export DRIFTGUARD_API_KEY=dg_live_…
export DRIFTGUARD_API_URL=https://driftguard.org

Step 5 — Copy a watch ID

Cloud fixtures are keyed to a hosted watch — the same watch you use for schema monitoring. Open the watch in the console and copy its UUID:

Step 6 — Cloud replay with --simulate-drift

Fetch a fixture from the hosted API, materialize it under .mockdrift/cache/, and run pytest with env vars set for simulate-drift fixture resolution:

mockdrift run --pytest tests/ -v \
  --simulate-drift YOUR_WATCH_ID \
  --cache-fixture

When the cloud fixture is materialized, use fixture="simulate-drift" in @drift_replay instead of a local fixture key.

Equivalent pytest plugin flags:

pytest --simulate-drift YOUR_WATCH_ID --cache-fixture tests/

API reference: POST /v1/mockdrift/fixtures/from-watch (same payload as simulate-drift).

Step 7 — GitHub Actions CI

Add the composite action from the OSS repo. Store DRIFTGUARD_API_KEY as a repository secret. Set MOCKDRIFT_TELEMETRY=0 to skip CI usage events.

jobs:
  mockdrift:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - run: pip install -e "./packages/mockdrift[dev]"
      - uses: ./.github/actions/mockdrift
        env:
          DRIFTGUARD_API_KEY: ${{ secrets.DRIFTGUARD_API_KEY }}
          MOCKDRIFT_TELEMETRY: "0"
        with:
          pytest-args: tests/test_refund_drift.py
          simulate-drift: YOUR_WATCH_ID
          cache-fixture: "true"

Troubleshooting

Next steps

• More how-to guides
• MCP monitoring concepts
• OSS MockDrift README & R3 API spec