User Story
As an OpenShell contributor maintaining the conformance PR stack ending at #3866, I need the conformance tests organized with their shared implementation so I can align the next PR with the final testing-strategy proposal in #3460.
Problem Statement
After #3866 removes the standalone conformance executable, the Cargo entry points live under tests/suites/conformance, but their shared scenarios and runner still live in the product workspace under crates/openshell-conformance. The test implementation and execution guidance remain split across those locations.
Impact / Why This Matters
Contributors must navigate the product workspace to maintain the test suite. Moving the library without updating source checks would silently remove its unit-test and lint coverage from the root workspace. Keeping it in place leaves the first consolidation step of RFC 0016 unfinished.
Proposed Design
Contributors should find the reusable runner, scenario implementations, behavior-grouped CLI entry points, and execution guidance in the conformance suite tree. Existing driver and feature consumers should continue to reuse the runner, and existing Cargo/nextest selection names and artifact execution should continue to work. Keep source checks and gateway-backed execution independently selectable.
Example contributor workflow:
mise run test:conformance-support
cargo test --locked --manifest-path tests/suites/conformance/Cargo.toml \
--package openshell-test-conformance-cli --no-run
Acceptance Criteria
Alternatives Considered
- Retain the library in
crates/: keeps current source checks but leaves suite ownership split after runner removal.
- Duplicate the runner into each suite: avoids cross-workspace dependencies but creates divergent execution and cleanup behavior.
- Add capability discovery and result reporting in the same PR: useful follow-up work, but crosses public API and reporting boundaries beyond this consolidation. Middleware, interceptors, and providers do not address repository test ownership.
Context
Implementation follow-up to #3866 and the consolidation step of RFC PR #3460. Part of #3954; this issue does not cover the full strategy, capability API, reporting, or coverage migrations.
User Story
As an OpenShell contributor maintaining the conformance PR stack ending at #3866, I need the conformance tests organized with their shared implementation so I can align the next PR with the final testing-strategy proposal in #3460.
Problem Statement
After #3866 removes the standalone conformance executable, the Cargo entry points live under
tests/suites/conformance, but their shared scenarios and runner still live in the product workspace undercrates/openshell-conformance. The test implementation and execution guidance remain split across those locations.Impact / Why This Matters
Contributors must navigate the product workspace to maintain the test suite. Moving the library without updating source checks would silently remove its unit-test and lint coverage from the root workspace. Keeping it in place leaves the first consolidation step of RFC 0016 unfinished.
Proposed Design
Contributors should find the reusable runner, scenario implementations, behavior-grouped CLI entry points, and execution guidance in the conformance suite tree. Existing driver and feature consumers should continue to reuse the runner, and existing Cargo/nextest selection names and artifact execution should continue to work. Keep source checks and gateway-backed execution independently selectable.
Example contributor workflow:
mise run test:conformance-support cargo test --locked --manifest-path tests/suites/conformance/Cargo.toml \ --package openshell-test-conformance-cli --no-runAcceptance Criteria
tests/suites/conformanceand is absent from the product workspace.Alternatives Considered
crates/: keeps current source checks but leaves suite ownership split after runner removal.Context
Implementation follow-up to #3866 and the consolidation step of RFC PR #3460. Part of #3954; this issue does not cover the full strategy, capability API, reporting, or coverage migrations.