Skip to content

TESTING.md

Latest commit

 

History

History
52 lines (33 loc) · 2.7 KB

TESTING.md

File metadata and controls

52 lines (33 loc) · 2.7 KB
Raw

Testing Strategy

The suite is intentionally layered and acceptance-first.

Invariants

Fast deterministic tests protect serialization, contracts, path security, message replay, evaluation math, receipts, workflow transitions, and other security/durability/accounting boundaries that are more reliable as deterministic matrices than repeated full product journeys.

python -m pytest -q tests/invariants

Acceptance journeys

Installed-product journeys exercise the public CLI, real filesystem behavior, Git worktrees, worker processes, durable messages, workflow resume, evaluation, review, benchmarks, MCP/plugins, preflight, and indexing.

python -m pytest -q tests/acceptance

Consolidation policy

Keep a narrow test only when it protects a distinct security, durability, schema, replay, path, state-machine, or accounting boundary that is difficult to prove economically end to end. Otherwise prefer one assertion-dense installed-product journey over multiple feature-specific journeys.

For the 0.9 simplification work, test-surface growth is not a deliverable. New coverage should normally be an integration/E2E or public-contract journey that protects a real user/agent boundary. Do not add unit/invariant tests merely to measure implementation details, role-file parsing permutations, or efficiency counters. When a stronger installed-product journey subsumes narrow coverage, consolidate or delete the redundant tests rather than keeping both. Static size/context budgets belong in repository audit tooling.

Do not add compatibility tests for removed terminal-era APIs and do not preserve test counts for their own sake.

Release tests

Distribution tests validate wheel contents, installation, parser-derived command catalogs, release evidence, documentation synchronization, and repository/distribution boundaries.

python -m pytest -q tests/release

Headless-core acceptance contract

The 0.8 release gate permanently protects these architectural outcomes:

  • external preparation with no runtime host installed;
  • Agent-Workflow-owned headless process-group lifecycle and terminal sealing;
  • persist-first steer/progress/ack journeys;
  • process/evidence-based supervision;
  • workflow resume through durable Agent Run bindings and sealed predecessor inputs;
  • headless benchmark execution;
  • no interactive-runtime dependency in core configuration/source; and
  • no Herdr dependency in core.

These requirements are enforced by the current invariant, acceptance, release, and repository-audit layers rather than by an implementation-phase checklist.

For a current inventory, count the test tree or use pytest collection in the working revision rather than relying on historical handoff numbers.