test: track OpenShell conformance testing

[elezar]

Description

Track the definition, coverage, execution, documentation, and qualification of OpenShell conformance testing.

This issue succeeds #2183 as the active conformance-tracking issue. Issue #2183 remains the completed initial design milestone, and PR #2925 remains the initial implementation and gateway-lane integration.

Focused behavior changes and test implementations remain in their own issues and pull requests.

Tracked Work

Foundation

• spike: driver conformance suite for compute drivers #2183 — define the initial shared driver conformance suite
• test(e2e): run conformance in gateway lanes #2925 — run conformance in gateway-backed E2E lanes

Scenario model and execution

• test(conformance): select leaf scenarios by capability #3768 — select leaf scenarios by capability
• test(conformance): run driver suites with cargo #3866 — run driver conformance through portable Cargo suites
• test(e2e): remove schema parity campaign #3864 — remove the completed schema-parity campaign; stack dependency for test(conformance): run driver suites with cargo #3866

Portable behavioral coverage

• test(e2e): add cross-driver image-to-sandbox conformance #2422 — add cross-driver image-to-sandbox conformance
• Define additional portable lifecycle, policy, networking, provider, and file-operation coverage

Platform and deployment profiles

• ci(windows): report full MXC conformance #3769 — report full mock-MXC conformance on Windows x64 and ARM64
• test(e2e): separate VM gateway provisioning from test execution #2314 — separate VM gateway provisioning from conformance execution
• Clarify default workspace behavior in Kubernetes operator mode #2971 — define default-workspace behavior for Kubernetes operator mode
• feat(ha): harden gateway peer transport and expand failover conformance #3529 — expand HA and failover conformance

CI and release qualification

• ci(qualification): add Ubuntu 24.04 k3s scenario #3455 — compose the Ubuntu k3s qualification environment with conformance
• Define which profiles are required in branch CI
• Define which profiles and outcomes gate release promotion
• Detect required scenarios that are omitted from applicable workflows

Documentation and contributor guidance

• docs(conformance): route agents to portable test guide #3945 — establish the portable-test guide and route contributor agents to it
• Document the supported conformance profiles and coverage matrix
• Document how operators and driver authors run and interpret conformance
• Document how contributors classify, add, and promote scenarios

Related Product and Driver Work

These changes were exposed by, support, or contain relevant regression coverage, but do not directly count as conformance-program work:

• feat(cli): add identity-aware sandbox deletion waiting #3944 — identity-aware sandbox deletion waiting, exposed by lifecycle conformance
• fix(mxc): preserve JSON arguments in PowerShell 5.1 (NVBug 6843165) #3893 — PowerShell argument preservation for MXC runners

Additional product gaps discovered by conformance should be linked here while remaining independently scoped issues.

Completion Criteria

• The portable conformance boundary is documented.
• Supported drivers and deployment configurations have declared profiles.
• Scenario applicability and pass, fail, unsupported, skipped, and advisory outcomes have defined meanings.
• Required coverage is visible in CI and release qualification.
• Contributors have a documented path for adding and promoting scenarios.
• Remaining implementation work is represented by focused linked issues or pull requests.