Skip to content

GT-608: the human-approval seam has now actually run, Runtime → Tracker → human → audit - #312

Open
beyondnetPeru wants to merge 2 commits into
developfrom
feat/gt-608-hitl-end-to-end
Open

GT-608: the human-approval seam has now actually run, Runtime → Tracker → human → audit#312
beyondnetPeru wants to merge 2 commits into
developfrom
feat/gt-608-hitl-end-to-end

Conversation

@beyondnetPeru

Copy link
Copy Markdown
Contributor

Closes GT-608 (P1). Criteria 3 and 4 met, over real HTTP. Criteria 1 and 2 were already met.

The root cause was one line

Wave 2 had already built ManifestSkillRegistryAdapter, deriveSkillsFromManifest and the harnessRoot option on createAgentRuntime — but no shipping composition ever passed it. createRuntimeFromEnv built HarnessProcessAdapter from AGENT_RUNTIME_HARNESS_ROOT and then dropped that root, so the deployed service executed a manifest it had never read: the catalogue stayed the hardcoded 7 skills, nothing routable declared requiresApproval: true, step 4 of the governed pipeline never ran, and evolith_hitl_approvals_total was structurally zero. The CLI factory had the identical gap.

That is why the row's own impact note said two independent readers of the two halves disagreed about whether the opposite endpoint existed. Both halves were real; nothing connected them.

Criterion 3 — pending → approved → executed → audited

Every hop is a real HTTP request. A stubbed TrackerApprovalClient would have reproduced exactly the defect this row records:

test ──HTTP──▶ agent-runtime-api (real AppModule via NestFactory, real port)
                    └─HTTP──▶ Tracker :5100 ──▶ PostgreSQL
test ──HTTP──▶ Tracker /runtime-approvals/{id}/resolve   (the human)
test ──HTTP──▶ agent-runtime-api /metrics

The dev-bypass cannot drive this: /runtime-approvals is bound to the CoreMachine scheme by name precisely so the bypass cannot reach the gate, and granting additionally requires a designated approver authority the bypass principal does not carry. So the human channel runs on the T-053 symmetric-key JWT path.

Negative half (harnessRoot removed, live Tracker still up): 3 failed — the DEPLOYED catalogue honours the manifest → Received: undefined, and both approval paths Expected "blocked", Received "error". Green: 8 suites / 86 tests with the live Tracker.

Rows the seam left in the other repo's ledger — proof it crossed the boundary:

gt608-approve-… | self-improving-loop | approved | 9f3c1a44-… | t
gt608-reject-…  | self-improving-loop | rejected | 9f3c1a44-… | t

Criterion 4 — the counter actually moved

evolith_hitl_approvals_total{decision="approved"} 1, scraped from the service's own /metrics after the run, incremented by AgentMetricsService.recordRun from trace.approvedBy — the field only the Tracker can fill. The rejection test asserts the series does not move on a denial.

Why a CI-runnable guard was added too

The integration spec skips in CI (nothing provisions a Tracker + PostgreSQL), so a regression would be silent. runtime.factory.spec.ts guards the wiring and runs everywhere. Independently falsified by the integrating session: deleting the single harnessRoot, line turns it red (1 failed / 39 passed); restored, 40/40.

Conflict note for PR #310: the change to runtime.factory.ts is one line plus a comment, no new imports, and src/packages/agent-runtime/src/adapters/index.ts was not touched at all.

The row's second finding

Half true, and now closed for both shipping surfaces. The derivation machinery and CI parity test existed, but since nothing passed harnessRoot the runtime posture was unchanged — 9 manifest capabilities invisible, two sources of truth wherever it mattered. Fixed in the runtime service and the CLI, conditional on AGENT_RUNTIME_HARNESS_ROOT being set: the Coolify deploy doc sets it, the Helm values still have it commented out, so a Helm deployment keeps the hardcoded table until someone uncomments it — documented in place rather than flipped, since that is a deploy decision coupled to the corpus mount. LocalSkillRegistryAdapter still defaults to the 7 hardcoded skills correctly: it is the routing/semantics layer, and design rule #5 requires the runtime to boot with no .harness checkout.

Deliberately left out

  • CI cannot run the integration. It skips with an explicit console.warn naming the four required variables, so a skip is never mistaken for a pass. Provisioning is a separate infra item.
  • FileApprovalStore and HttpSlackClient remain unexercised end to end — unit specs only; there is no live Slack workspace, and the Tracker (not Slack) is the seam this row is about.
  • No polling loop. The Core treats pending as a denial and requires re-submission of the same correlation id; the Tracker is idempotent by correlationId and the test exercises that as-is.
  • decision="blocked" is still never emittedAgentMetricsService counts only an approval the trace attests, since a block can come from policy rather than a human. Existing decision, left alone.

Verification hygiene: the worktree had no node_modules at all, so a full npm install ran rather than symlinking the main checkout; readlink -f confirmed worktree-local resolution, and tsc -b ran before every measurement. Full CLI suite green: 101 suites / 1446 tests.

🤖 Generated with Claude Code

…racker

Criteria 3 and 4. The row does not record a missing implementation: both
halves were written and neither had ever spoken to the other, which is why
two independent readers of the two halves disagreed about whether the
opposite endpoint existed.

The root cause of the seam being unreachable in the DEPLOYED service was
one missing line: `createRuntimeFromEnv` built the .harness process executor
from `AGENT_RUNTIME_HARNESS_ROOT` but never passed that root to
`createAgentRuntime`, so the catalogue stayed the hardcoded 7-skill table.
Nothing the runtime could route declared `requiresApproval: true`, step 4 of
the governed pipeline never ran, and `evolith_hitl_approvals_total` was
structurally zero. The same one-line gap existed in the CLI's factory.

- `hitl-approval-tracker.integration.spec.ts` boots the real AppModule via
  NestFactory on a real port and drives pending → approved → executed →
  audited, plus pending → rejected → NOT executed, over real HTTP against a
  live Tracker + PostgreSQL. A stubbed TrackerApprovalClient would have
  reproduced the defect exactly, so every hop is a real request. Env-gated:
  it SKIPS loudly without a Tracker rather than passing vacuously.
- `runtime.factory.spec.ts` gains the CI-runnable guard on the wiring the
  integration depends on (the integration itself cannot run in CI — nothing
  provisions a Tracker yet).
- helm values.yaml records that AGENT_RUNTIME_HARNESS_ROOT now drives the
  catalogue as well as the executor.

Observed: `evolith_hitl_approvals_total{decision="approved"} 1` scraped from
the service's own /metrics after the run, and the approval rows persisted in
the Tracker's `tracker_governance.runtime_approvals` with the approver the
Tracker named.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@beyondnetPeru
beyondnetPeru requested a review from a team as a code owner July 31, 2026 03:21
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@github-actions

github-actions Bot commented Jul 31, 2026

Copy link
Copy Markdown

📊 Bilingual Coverage Impact

PR Changes

  • Paired EN/ES files modified: 4
  • New EN files needing ES translation: 2

Repository Coverage

Metric Value
Total EN files 525
Total ES files 499
Paired files 0
Coverage 0%

⚠️ Action required: 2 new EN file(s) added without ES counterparts.

To create skeletons:

node .harness/scripts/generate-es-skeleton.mjs <file.md>

Generated by GitHub Actions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant