Skip to content

About

Open-source CLI for local HTTP API regression testing. Check webhook idempotency and invoice persistence in development and CI.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

7 stars

Watchers

2 watching

Forks

Repository files navigation

OutcomeLab — Local HTTP API Regression Testing

LetsCookTech Open Source Team banner

OutcomeLab is an open-source CLI for stateful HTTP API testing. It checks webhook idempotency, invoice persistence, and inventory reservation behavior against a disposable local service or CI test environment. For example, it sends the same webhook twice, then reads the ledger to check that the payment was credited only once.

Quickstart

Use Python 3.11 or newer. From the repository root, in an active virtual environment:

python -m pip install -e .
outcomelab catalog
outcomelab demo

The demo runs two bundled SQLite examples. To try the verifier over HTTP, start the disposable reference service in one terminal:

python -m risk_lab.demo_http --variant broken --port 8765

In a second terminal with the same environment active, run:

outcomelab verify-http --contract examples/http/v2/webhook-idempotency.json --base-url http://127.0.0.1:8765 --allow-local-writes

The broken service returns HTTP 200 for both deliveries, but the ledger shows two credits. OutcomeLab prints FAIL (7 assertions, 0 skipped). Restart the service with --variant fixed and run the same command. It prints PASS (7 assertions, 0 skipped).

Check the backing store

The optional SQLite witness can catch a duplicate ledger row even when the API reports one credit and every HTTP assertion passes. It counts only rows for the current run in an explicitly approved disposable database. See the deceptive-API walkthrough and the evidence matrix.

Check your own API

Copy an stateful example contract and set the routes, test payloads, and expected state for your service. Check it offline with outcomelab validate --contract PATH, then run against a disposable database. The runner accepts literal IPv4 loopback targets and requires --allow-local-writes for POST requests. Add --json to get a receipt with the contract hash, verdict, and assertion outcomes.

A passing result covers the observations in the supplied contract. The bundled examples use fictional inputs and temporary databases. Reproduce the checks independently or propose a scenario.

Engineering resources

Resource Contents
Skills Portable playbooks for security, architecture, and engineering reviews
Compatibility How to use the playbooks in different agent clients
Case studies Illustrative walkthroughs of the review process
Benchmarks and evaluation tooling Rubrics, comparison fixtures, and reproducibility guidance
Contributing and skill standard How to contribute a contract, fix, or playbook

Maintained by the LetsCookTech Open Source Team. Releases · Report an issue · MIT license

About

Open-source CLI for local HTTP API regression testing. Check webhook idempotency and invoice persistence in development and CI.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

7 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors

Languages