Strengthen release-plan.yaml validation with cross-field consistency checks

[hdamker]

Problem description

Current release-plan.yaml validation (P-009, P-022, P-023) covers regex-level checks on individual fields. Combinations of fields can still pass these checks while being internally inconsistent — and the same content drives the release at pre-snapshot, so an inconsistency that slips through is no longer just a documentation issue.

Possible evolution

Add a class of cross-field consistency rules running both at PR-time on release-plan.yaml changes and at pre-snapshot on the active release plan.

Example case — target_api_status: public is valid only if either:

• target_release_type is public, or
• the target_api_version was previously published in a public release (e.g. an unchanged stable base API in a cycle that only ships subscription changes).

A merged real-world instance of this inconsistency: WebRTC#140 sets target_api_status: public for a target_api_version that has not previously been published as public.

Further validation requirements welcome as comments on this issue before scoping. Please include the field combination, the failure mode it would catch, and — if known — a published release plan where it would have flagged a real issue.

Alternative solution

Renaming fields (e.g. target_api_version → target_api_base_version, target_api_status → next_api_release_status) so the "next release" semantics are self-evident. Out of scope here: that is a breaking schema change and would need to be handled as a major schema revision.

Additional context

Once the rule list stabilises in comments, scoping moves into the validation framework and a dedicated regression/r4.2-release-plan-validation branch with bogus-tag and exclusivity-violation fixtures.

[hdamker]

What release-plan.yaml validation already covers

Recording the current state so this thread doesn't re-propose checks already in place.

• Schema validation (formats, enums, required fields, apis minItems 1)
• Track / meta-release consistency — meta-release requires meta_release; independent + meta_release warns; meta_release value must be in the allowlist
• Release-type / API-status floor — pre-release-alpha blocks draft APIs; pre-release-rc blocks draft/alpha; public-release and maintenance-release require all public
• API spec file existence — alpha/rc/public API requires the spec file; draft API with orphan files in code/API_definitions/ warns about possible naming mismatch
• Orphan API spec files — YAML files in code/API_definitions/ not declared in the plan are flagged
• Exclusivity — release-plan.yaml changes must be in their own PR
• Dependency tag existence — when commonalities_release or identity_consent_management_release advances in the PR, the declared tag is verified to exist in the upstream repo
• Spec-side rules consume the plan — filename match, server-URL api-name match, version segments derived from the plan
• Pre-snapshot — all server URLs / test feature lines / resource URLs must already be wip / vwip before snapshot transformation

The comments below propose additional rules — cross-field and cross-history checks that go beyond regex/format and beyond per-field semantics. One comment per rule (each may bundle several closely-related sub-checks).

[hdamker]

Rule: Validate the release plan against the published history

The plan must move the repository forward from its published state in a coherent way. The rule fires when one of the following is wrong, against the published-tag list and the latest published release-metadata for the repo:

• Tag already published, but the cohort has moved. The declared target_release_tag already exists as a published release for this repo, and the rest of the plan no longer matches that release 1:1. Codeowner forgot to bump the tag.
• Tag regression. target_release_tag is smaller than the highest published tag reachable from the current branch (maintenance-rX counts only rX.y tags).
• Cycle opening without prior cycle. rX.1 declared with no r(X-1).y published. Skipped for r1.1.
• Null tag with non-none type. Reverse direction (non-null tag under type: none) is legitimate.
• No new content for public-release / maintenance-release. The cohort offers neither an API with a target_api_version not previously published for that api_name, nor a brand-new api_name. Nothing to ship at the declared tag.

Skipped entirely in the "parked after publish" state — target_release_tag matches the latest published release and the cohort matches that release 1:1 (codeowner intentionally hasn't started the next cycle yet).

Severity: error in all cases.

Gated on release-plan.yaml exists. Findings on PRs that don't touch release-plan.yaml carry a "pre-existing condition" prefix.

Rule suggestion: fix must land in a dedicated PR that changes only release-plan.yaml (existing exclusivity rule applies).

[hdamker]

Rule: Per-API target_api_version / target_api_status consistency with the published history

For each API, against published versions of that api_name:

• Locked-version status. Once published as public at X.Y.Z, status must stay public.
• public under pre-release. Allowed only when target_api_version was previously published as public. Real instance: WebRTC#140 — webrtc-call-handling@0.4.0 declared public under pre-release-rc r3.1, never previously published.
• Version regression. target_api_version may not be lower than the highest previously-published version on the current branch lineage.

Content backstop. Whenever the rule decides "this is a republish of an already-public version", the spec file (and matching Test_definitions/{api_name}.feature if applicable) must be byte-equal to the file at source/rX.Y. Without this, the status declaration would be "unchanged" while content silently drifts.

Severity: error for the declaration checks; backstop warn at PR-time → error at /create-snapshot.

Gated on release-plan.yaml exists — i.e. main or maintenance branch. Not extended to Release Review PRs (snapshot already validated at /create-snapshot time, and the Release-Review-PR file-restriction rule blocks anything but doc changes).

[hdamker]

Rule: Release-plan internal consistency

Pure release-plan / latest-release-metadata checks, no further GitHub-API walk required:

• Duplicate api_name — no two API entries may share the same api_name. The schema does not enforce uniqueness on the array; a duplicate produces ambiguous downstream behaviour. Severity: error.
• Status drift on an unchanged-version API — an api_name previously published with target_api_version: X.Y.Z and target_api_status: public may not appear in the plan with a lower target_api_status at the same version. Catches editing accidents that knock a published API back to draft or alpha. Severity: error (a published-public API at the same version cannot legitimately go back to a lower status — that's the lock semantics).

Gated on release-plan.yaml exists. Findings on PRs that don't touch release-plan.yaml carry a "pre-existing condition" prefix. Cheap to implement; intended as the first wave to validate the wiring (release-history artifact, regression-fixture format) before the heavier groups land.

[hdamker]

Rule: Maintenance-release branch and version-bump rules

Two related checks that both depend on the maintenance-branch design coming together (RM#356):

• Branch enforcement. target_release_type: maintenance-release should be valid only on a maintenance-rX branch. Today maintenance releases are sometimes done from main (when no other changes have landed since the public release), so the branch constraint cannot yet be enforced.
• Version-bump severity per component. For each API in a maintenance-release, compare target_api_version to the highest previously-published version for that api_name:
  • Major (X) bump → error: a major bump in a maintenance release is by definition impossible.
  • Minor (Y) bump → warn: usually wrong, but legitimate when maintaining a prior major-version line with a new minor (e.g., maintaining the 1.x line after 2.0 has shipped).
  • Patch (Z) bump → no finding: normal maintenance.

Severity per case as listed.

Both checks are scope-bound to the maintenance-branch design. Once RM#356 has been scoped, this should move to its own backlog issue rather than continuing to live inside this discussion thread.

[hdamker]

Content backstop. Whenever the rule decides "this is a republish of an already-public version", the spec file (and matching Test_definitions/{api_name}.feature if applicable) must be byte-equal to the file at source/rX.Y. Without this, the status declaration would be "unchanged" while content silently drifts.

Discussed within RM meeting on 2026-05-05: both the source as well the bundled API definitions must be byte-equal, otherwise they shall not be published under the same semver version (for reformatting or similar a patch version would be sufficient, changes in the bundled common definitions are a case-by-case decision).

Also discussed: changes within the test definition files are also a change of the API version if they define a new behavior of the API.

[hdamker]

Folding in #63 — block release-plan.yaml edits while a release snapshot is active.

Proposed addition (new group, complements A–H):

Group I — Release-state gating (PR-time only)

I1. Block release-plan.yaml changes while a snapshot is in flight.

• Trigger: PR touches release-plan.yaml.
• PR-time error when:
  • a release-snapshot/* branch exists, OR
  • a canonical release issue carries a non-terminal release-state:* label (e.g. in-flight, release-review)
• Pre-snapshot: not applicable — pre-snapshot validates the active release plan, not file changes; the rule never fires there.
• Rationale: prevents configuration drift between the snapshot's frozen plan and a divergent merge into main.
• Detection: the validation workflow can query the release-snapshot branch refs and the release-issue labels (the surface already used by sync-issue).

Suggest implementing as a small standalone group, independently of the W/T/P waves.