OutcomeLab is an open-source CLI for stateful HTTP API testing. It checks webhook idempotency, invoice persistence, and inventory reservation behavior against a disposable local service or CI test environment. For example, it sends the same webhook twice, then reads the ledger to check that the payment was credited only once.
Use Python 3.11 or newer. From the repository root, in an active virtual environment:
python -m pip install -e .
outcomelab catalog
outcomelab demoThe demo runs two bundled SQLite examples. To try the verifier over HTTP, start the disposable reference service in one terminal:
python -m risk_lab.demo_http --variant broken --port 8765In a second terminal with the same environment active, run:
outcomelab verify-http --contract examples/http/v2/webhook-idempotency.json --base-url http://127.0.0.1:8765 --allow-local-writesThe broken service returns HTTP 200 for both deliveries, but the ledger shows two credits. OutcomeLab prints FAIL (7 assertions, 0 skipped). Restart the service with --variant fixed and run the same command. It prints PASS (7 assertions, 0 skipped).
The optional SQLite witness can catch a duplicate ledger row even when the API reports one credit and every HTTP assertion passes. It counts only rows for the current run in an explicitly approved disposable database. See the deceptive-API walkthrough and the evidence matrix.
Copy an stateful example contract and set the routes, test payloads, and expected state for your service. Check it offline with outcomelab validate --contract PATH, then run against a disposable database. The runner accepts literal IPv4 loopback targets and requires --allow-local-writes for POST requests. Add --json to get a receipt with the contract hash, verdict, and assertion outcomes.
- Walkthrough for webhook idempotency and invoice persistence
- Run API regression tests in CI
- Contract format, exit codes, and safety rules
- Production Risk Lab agent skill
A passing result covers the observations in the supplied contract. The bundled examples use fictional inputs and temporary databases. Reproduce the checks independently or propose a scenario.
| Resource | Contents |
|---|---|
| Skills | Portable playbooks for security, architecture, and engineering reviews |
| Compatibility | How to use the playbooks in different agent clients |
| Case studies | Illustrative walkthroughs of the review process |
| Benchmarks and evaluation tooling | Rubrics, comparison fixtures, and reproducibility guidance |
| Contributing and skill standard | How to contribute a contract, fix, or playbook |
Maintained by the LetsCookTech Open Source Team. Releases · Report an issue · MIT license
