Skip to content

feat(security): mandatory secret patterns floor + qualification closures -> dev - #173

Closed
Rwanbt wants to merge 15 commits into
devfrom
multiforge
Closed

Rwanbt wants to merge 15 commits into
devfrom
multiforge

Conversation

@Rwanbt

@Rwanbt Rwanbt commented Sep 16, 2026

Copy link
Copy Markdown
Owner

Refs #166. Follow-up to PR #172: makes the mandatory secret-pattern floor unremovable (extra_secret_patterns union, no replacement parameter; GitLab token prefixes added in the scanner, the anti-debt owner and the vault-sync fallback), closes qualification scenarios J (GitLab object-storage blob), M (legacy GitLab remote warning), P (cross-command selector conflict E2E) and the EN/FR documentation parity automation, and updates the qualification report and CHANGELOG. Local proof: 314 lifecycle + 245 other suites green, gates green. Targets dev; main is untouched.

)

ADR-0017 makes profiles (governance) and features (optional project-scope
capabilities) orthogonal. This change adds the model and the read side;
switching comes next in the lane.

- data/features.json: forge-github / forge-gitlab — project scope, symmetric
  conflicts, work-forge flag, declared legacy_default (no hard-coded default).
- manifest: Feature, Distribution.features / legacy_default_feature,
  Distribution.feature(); the catalogue refuses a machine-scope entry, an
  asymmetric conflict, a component claimed by two features, an undeclared
  component reference and an undeclared legacy default.
- state: schema V2 with active_features. V1 still loads (no field); a V2 state
  naming a feature this release does not know refuses as
  INSTALL_STATE_CORRUPTED (upgrade the CLI, never downgrade the state), and the
  existing schema guard still refuses a newer schema.
- features.project_install_state(): the single projection every read path will
  share; V1 projects to the declared legacy default; a V2 state activating two
  work forges refuses as STATE_CONFLICTING_WORK_FORGE_FEATURES. The projection
  never writes.
- features.migrate_state(): stamps the current schema and seeds the legacy
  default exactly once, inside the existing install transaction (state-last);
  it plans no file change of its own, so a managed file that is already absent
  stays absent through the migration.
- installer: a fresh state seeds the default feature (a fresh Standard install
  keeps shipping the GitHub templates, as it always has); a no-op install still
  saves the state when it was V1, which is what makes the migration persist.
- gitlab-templates component and templates/gitlab/ (owned by forge-gitlab; the
  .gitlab/ equivalents of the GitHub templates).
- New refusal codes: STATE_CONFLICTING_WORK_FORGE_FEATURES, FEATURE_UNKNOWN.
- CI: tests.test_lifecycle_features runs in the lifecycle job.

Verified: 295 lifecycle tests OK (21 new), machine/CLI suites OK, gates
(conventions, scope, complexity, LOC) green.

Refs #162
`ainative feature enable | disable | switch | status` moves a project between
optional capabilities with the profile path's guarantees unchanged: project
lifecycle lock, reload, project the effective V2 set, plan, apply, verify,
state last, commit. A V1 state migrates inside the same transaction.

- profiles.json: `github-templates` leaves the standard profile; the feature
  `forge-github` owns it now (a component belongs to exactly one owner), so
  `feature switch none` really is Generic Git.
- planner: the wanted set is profile components plus the active features'
  components, computed by one shared function the installer also records from
  (plan and record cannot drift). Feature files are seeded when a feature is
  enabled; ordinary maintenance never re-seeds an absent feature file — a file
  the user removed stays removed until an explicit transition (ADR-0017 §5).
- installer.set_features(): the transition semantics. `enable` refuses a
  conflicting active feature with the remedy (`FEATURE_CONFLICT`, never an
  implicit disable); `switch` replaces the work forge in one plan; `switch
  none` removes every work forge and nothing else. A no-op transition changes
  nothing — including the state bytes — unless it still has something to
  record (migration, moved set, dropped component records).
- feature_cli.py: the command surface, split out of cli.py so the dispatcher
  stays under its LOC budget; status derives from the shared projection.
- New refusal code FEATURE_CONFLICT.
- Tests: switching, disable preserves a modified file, absent feature file
  stays absent through installs and is re-seeded by an explicit transition,
  the migration fixture (a V1 state whose GitHub template is absent stays
  absent), a round trip without user-data loss, CLI status/switch.
- Docs: DISTRIBUTION-LIFECYCLE §18 and CHANGELOG.

Verified: 307 lifecycle tests OK (33 in the features suite), CLI/machine
suites OK, gates green (conventions, scope, complexity, LOC — cli.py back
under 800), five non-vacuity guards re-proved.

Refs #162
…journal (#163)

ADR-0018's local half: the harness observes remote facts, this layer records
them deterministically and never decides by guessing.

- ainative/forge.py: `resolve_observed_work_authority()` — pure (no network,
  no credentials, no writes, no push authorization). Priority: explicit
  reference, then harness declaration, then exactly one compatible observed
  candidate; otherwise WORK_AUTHORITY_UNAVAILABLE / _AMBIGUOUS / _MISMATCH.
  Only github.com and gitlab.com are read as providers; a self-hosted host is
  `unknown`, never inferred from its name; a fork is ambiguity, never
  "prefer origin".
- ainative/claims.py: canonical identities
  (`<provider>:principal:<id>`, `<provider>:<kind>:<id>`), UTC-normalized
  timestamps (a naive one refuses), total winner ordering with CLAIM_CONFLICT
  for unorderable duplicates, and the ClaimAttempt journal under
  `.ai-native/state/claim-attempts/` — PENDING written durably before the
  remote POST, outcomes CONFIRMED/LOST/CONFLICT/UNCERTAIN/ABANDONED, an
  unwritable or corrupt journal fails closed, abandon is an explicit local
  operator transition that keeps the record, and an abandoned record is final.
- ainative/claim_cli.py + `ainative claim-attempt list|inspect|abandon
  --confirm`: the recovery surface. No retry path exists by design.
- New refusal codes: WORK_AUTHORITY_*, CLAIM_INVALID, CLAIM_CONFLICT,
  CLAIM_UNCERTAIN, CLAIM_JOURNAL_UNAVAILABLE.
- Tests: 27 in tests/test_forge_claims.py, including the invariant that a
  pending attempt survives a lifecycle update byte-identically and the CLI
  confirmation flow. CI runs them in the lifecycle job.

Verified: 308 lifecycle tests OK, knowledge/machine/CLI suites OK, gates green
(conventions, scope, complexity, LOC).

Refs #163
The engineering-method component installs templates/AGENTS.md (Work Authority
vocabulary, ADR-0018) instead of the repository's own AGENTS.md, which stays
GitHub-specific. Identical to the root file except the work-management
section; the fixture distribution ships both files.
…sion (#164)

`ainative forge detect|status` observe the project's Git remotes and render the
Work Authority resolution. Read-only by construction: the only subprocess is
`git remote`, the only inputs are the pure resolver, the install state and the
claim journal — zero network, zero credentials, zero writes, zero persistent
trust. A fork renders as AMBIGUOUS (both candidates shown, no preference for
origin) and an unknown host as UNAVAILABLE, both exit 0: detection is a
diagnostic, the refusal belongs to the mutation path.

- ainative/observation.py: remote reading, `forge_picture()` (resolution as
  state), `project_view()` (features + forge + unresolved claim attempts) and
  the doctor text lines. An unreadable claim journal is displayed, not a crash.
- ainative/forge_cli.py: the command surface; `status` adds the effective
  feature set (shared projection) and the unresolved claim count.
- Doctor extension: profile/features/forge/claim summary in both outputs,
  never a credential (`doctor --json` gains features, forge, claim_attempts).
- status.py: reports the effective feature set through the same projection
  (`features`, `features_projected_from_legacy`), closing the read-only parity
  requirement for `status`.
- ainative/cli_support.py: the small shared plumbing (emit/project/report/
  plan text) extracted so cli.py stays the dispatcher under its LOC budget and
  the command modules stop importing render callables through cli.py.
- Tests: 7 new observation tests (resolution, fork rendering, unknown host,
  features+claims in status, byte-identical project across detect/status/
  doctor) and a status/projection parity test for a V1 state.

Verified: 309 lifecycle tests OK, 139 CLI/knowledge/machine suites OK, gates
green (conventions, scope, complexity, LOC: cli.py 757).

Refs #164
#165)

PR-4A. `resolve_release_source()` (ainative/lifecycle/release_source.py) is now
the only function that decides where a release comes from, and `update`,
`update check`, `status` and `doctor` all consume it — so the answer cannot
differ between commands (ADR-0019 sections 1-3).

- Selector rules exactly as frozen: local pair -> mirror; LOCAL_DIR without
  provider=local -> UPDATE_SOURCE_CONFLICT; URL mixed with any selector ->
  UPDATE_SOURCE_CONFLICT; URL alone -> anonymous; named provider -> machine
  config or built-in; nothing -> machine default_provider, else GitHub.com.
  Validation before precedence: nothing is ordered silently, and the old
  "unknown provider" refusal is preserved.
- Machine scope `~/.ai-native/release-providers.json` (schema 1): reserved
  names (github/gitlab/local) cannot be redefined, a named provider is
  anonymous in V1 (a custom auth_origin is refused), a future schema refuses
  rather than guessing. `default_provider: local` requires a configured
  directory. New refusal codes: UPDATE_SOURCE_CONFLICT, RELEASE_CONFIG_INVALID.
- `provider.build()` delegates to the resolver; the selector constants are
  re-exported so existing imports keep working.
- Observability: `status` and `doctor` display the effective source, the
  selection reason, authenticated yes/no and the auth origin — never a secret
  (`describe()` turns a selector conflict into a displayed state, so the
  diagnostics keep diagnosing while the updater refuses).
- Tests: 19 in tests/test_release_source.py (selector matrix, machine config,
  provider construction, status/doctor parity); CI runs them in the lifecycle
  job.

Verified: 309 lifecycle tests OK, 123 update/CLI suites OK, gates green
(conventions, scope, complexity 0 findings after splitting the resolver,
LOC 0 warnings).

Refs #165
…on chain (#165)

PR-4B + PR-4C. ainative/lifecycle/release_v3.py owns the V3 vocabulary and the
fail-closed policies around it (ADR-0019 sections 7-10):

- Candidate policy: installable versions are SemVer without build metadata
  (1.2.3, 1.2.3-rc.1); `1.2.3+build1` refuses as
  RELEASE_BUILD_METADATA_UNSUPPORTED; ordering uses version precedence, never
  string order; two identities on one version refuse as
  RELEASE_DUPLICATE_VERSION (never "first match"); an enumeration that stopped
  at its bounds refuses as RELEASE_ENUMERATION_INCOMPLETE; a complete channel
  with nothing refuses as RELEASE_NO_CANDIDATE.
- The external anchor travels WITH the candidate (provider metadata):
  manifest sha256 + size are verified BEFORE the document is parsed —
  RELEASE_INTEGRITY_METADATA_MISSING / _INVALID for absent/malformed anchors,
  UPDATE_INTEGRITY_FAILED for any mismatch. A hostile manifest can never
  influence whether its own bytes are trusted.
- ReleaseManifest V3: schema, protocol, version, channel, compatibility,
  artifacts, provenance — strictly validated (plain filenames, exact one
  lifecycle artifact, canonical versions). A newer protocol refuses as
  CLI_UPDATE_REQUIRED, mirroring the bridge; anything else is an integrity
  refusal.
- The exact version chain: candidate == manifest == compatibility.runtime_version
  == artifact == artifact filename == lifecycle-protocol.json release_version,
  with the broken link named in the refusal detail. `resolve_manifest()`
  formalizes the order: enumerate -> select -> verify -> parse -> chain.

Tests: 32 in tests/test_release_v3.py (SemVer policy, selection, anchors,
manifest refusals, every broken chain link, resolution order with a fake
provider). CI runs them in the lifecycle job.

Verified: 309 lifecycle tests OK, complexity 0 findings, LOC 0 warnings,
scope within tolerance.

Refs #165
#165)

PR-4D + PR-4E. Each provider implements the `release_v3` contract and nothing
else: enumerate, fetch the manifest, fetch an artifact. Every trust decision
stays in release_v3 — the provider that fetched a manifest never decides
whether to believe it (ADR-0019 sections 9-11).

- GitHubReleaseProvider: bounded enumeration of the releases API (a full page
  says `complete=False`, so selection refuses instead of picking the best of
  what it saw); drafts, non-SemVer tags and releases without an
  `ainative-release-v3.json` asset are not candidates (a V2-era release is not
  a broken V3 one); the manifest asset's digest/size travel as the candidate's
  anchor; artifacts and manifests are fetched through the PR-0A transport
  (asset API locator preferred, `browser_download_url` fallback, octet-stream,
  bearer only at `api.github.com`).
- AnonymousReleaseApiProvider: `AINATIVE_UPDATE_URL` as one GitHub-shaped
  release document, never authenticated; a document without a V3 manifest
  yields no candidate (`RELEASE_NO_CANDIDATE`), never a silent V2 fallback.
- LocalReleaseProvider: `releases.json` channels carry the manifest locator
  with its size and SHA-256; the mirror executes the same logical chain as a
  network source, traversal-checked, bounded, and never trusted for being on
  disk. A V2-era channel entry is not a candidate.
- `ReleaseCandidate` gains `manifest_locator` (provider-internal, opaque to
  the contract); `provider.environment_token()` exposes the environment token
  publicly for the V3 providers under the same origin rule.

Tests: 16 in tests/test_release_providers.py, including an end-to-end
`resolve_manifest` against the local mirror and against a scripted GitHub
server, a tampered-manifest refusal, a bounded-listing refusal and a
path-traversal refusal. CI runs them in the lifecycle job.

Verified: 309 lifecycle tests OK, 67 release tests OK, complexity 0 findings,
LOC 0 warnings, scope within tolerance.

Refs #165
…#165)

PR-4 completion. `update check` and `update` now resolve through
`resolve_v3_candidate()`: the resolved source is enumerated (bounded), the
newest V3 candidate of the channel is selected, and the manifest is fetched
and anchor-verified only when an update is applied — a check needs the version,
nothing more.

- A source that hits its enumeration bounds refuses
  (RELEASE_ENUMERATION_INCOMPLETE): partial results are never a reason to
  change generation.
- A complete channel with no V3 candidate is a V2-era mirror, and the caller
  speaks the existing V2 path to it (deterministic, disclosed fallback — the
  bridge-era releases stay consumable). Everything else propagates.
- apply(): V3 fetch verifies the artifact against the manifest's own
  size+SHA-256 before extraction, then the existing transaction applies the
  payload; the V3 bundle declares lifecycle protocol 3
  (`_distribution_root` gained an `expected_protocol`), and a bundle carrying
  the V2 protocol refuses as UPDATE_VERSION_MISMATCH.
- `provider.build_v3()` is the second patchable seam; the four tests that stub
  the V2 provider now stub it explicitly.

Tests: 5 new end-to-end tests on a local V3 mirror (check, apply + rollback,
tampered manifest refused with zero writes, protocol mismatch refused, CLI
report). 314 lifecycle tests OK; release/CLI suites OK; gates green.

Refs #165
#166)

PR-5 core. The GitLab provider implements the V3 contract (ADR-0019 §11):

- Releases API for discovery only; the Generic Package Registry is the
  canonical distribution and integrity surface. Release Link URLs are never
  integrity roots — the anchor is the package file API's `file_sha256` and
  `size` (absent -> RELEASE_INTEGRITY_METADATA_MISSING, malformed ->
  RELEASE_INTEGRITY_METADATA_INVALID), and the manifest file lookup requires
  exactly one `ainative-release-v3.json` (0 -> RELEASE_MANIFEST_MISSING,
  >1 -> RELEASE_MANIFEST_AMBIGUOUS).
- One project identity (`release_project_ref`) is used for Releases, packages
  and package files — no second project configuration. `release_project_ref`
  is a numeric id or namespace path, never a URL; it is machine-scope config
  (`providers.gitlab`), while `github`/`local` remain non-redefinable.
- Exact package match (`type == generic`, canonical package name, selected
  version) with exactly one record, and bounded pagination (a full page
  refuses as incomplete, never a partial best-of).
- Authentication shapes differ per provider: `ReleaseProviderEndpointConfig`
  gained `auth_header`/`auth_prefix`. GitLab sends `PRIVATE-TOKEN: <token>`
  from `GITLAB_TOKEN`, only at its configured origin; GitHub keeps
  `Authorization: Bearer`, unchanged.
- GitLab is V3-only: `supports_v2_fallback = False`, and a complete channel
  without a V3 candidate refuses as RELEASE_NO_CANDIDATE instead of falling
  back to a generation that does not exist there.

Tests: 10 GitLab provider tests on a scripted GitLab (exact-identity match,
private-token confinement, duplicate packages, every manifest/integrity
refusal, pagination bounds, full chain, anonymous endpoints) plus 3 config
tests; and the three purity suites required by §74
(tests/purity/test_dependency_purity.py, test_contract_purity.py,
test_distributed_policy_purity.py) — neutral modules do not import provider
implementations, the neutral helpers behave identically for both forges, and
the distributed policy carries no GitHub-only authority.

Verified: 314 lifecycle tests OK, release suites OK, purity 7/7, complexity 0
findings, LOC 0 warnings.

Refs #166
…enario closures (#166)

Closes the remaining traceable items from the Multi-Forge qualification report.

- Candidate scanner (ainative/multivault/git_scanner.py): the constructor now
  takes `extra_secret_patterns` and no longer accepts a replacement set;
  the effective patterns are `MANDATORY_SECRET_PATTERNS + extras`, so no
  caller, repository or operator configuration can remove, replace, disable or
  shadow the floor (ADR-0019 section 12). The floor gains the documented
  GitLab token prefixes (`glpat-`, `gldt-`, `glrt-`, `glsoat-`), alongside
  private-key markers, AWS, GitHub and Slack patterns. The anti-debt owner
  (`finding_common.SECRET_PATTERNS`) and the vault-sync fallback carry the
  same GitLab prefixes.
- Scenario J: the GitLab provider test proves an object-storage redirect on a
  package-file download is followed anonymously — the token stays at the API
  origin.
- Scenario M: `doctor` warns when a GitLab remote is observed while the
  legacy `forge-github` default is only projected, naming the explicit
  `feature switch forge-gitlab` transition.
- Scenario P: one end-to-end test asserts `update check`, `update`, `status`
  and `doctor` all surface the same `UPDATE_SOURCE_CONFLICT` for the same
  environment — one resolver, one refusal.
- Documentation parity (plan section 75): `tests/purity/test_docs_parity.py`
  enforces the EN/FR README heading hierarchy, the operational surface
  (commands, env vars, config files) and the critical security statements —
  structurally, never by raw line counts. CI runs it with the purity suite.
- Qualification report and CHANGELOG updated: scenarios J/M/P/Q and parity are
  GREEN; only the GitLab.com LIVE qualification and the release choreography
  remain, both declared.

Verified: 314 lifecycle tests OK, 245 other suites OK (18 scanner, 110
release/claims/purity), gates green.

Refs #166
@Rwanbt

Rwanbt commented Sep 16, 2026

Copy link
Copy Markdown
Owner Author

Superseded by a clean-branch PR: the multiforge history diverged from dev's squash of #172, which made this PR conflicting (and GitHub cannot build the merge ref for a conflicted PR, so the main CI matrix never started). The same increment was cherry-picked onto dev as fix/multiforge-qualification.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant