GT-608: the human-approval seam has now actually run, Runtime → Tracker → human → audit - #312
Open
beyondnetPeru wants to merge 2 commits into
Open
GT-608: the human-approval seam has now actually run, Runtime → Tracker → human → audit#312beyondnetPeru wants to merge 2 commits into
beyondnetPeru wants to merge 2 commits into
Conversation
…racker
Criteria 3 and 4. The row does not record a missing implementation: both
halves were written and neither had ever spoken to the other, which is why
two independent readers of the two halves disagreed about whether the
opposite endpoint existed.
The root cause of the seam being unreachable in the DEPLOYED service was
one missing line: `createRuntimeFromEnv` built the .harness process executor
from `AGENT_RUNTIME_HARNESS_ROOT` but never passed that root to
`createAgentRuntime`, so the catalogue stayed the hardcoded 7-skill table.
Nothing the runtime could route declared `requiresApproval: true`, step 4 of
the governed pipeline never ran, and `evolith_hitl_approvals_total` was
structurally zero. The same one-line gap existed in the CLI's factory.
- `hitl-approval-tracker.integration.spec.ts` boots the real AppModule via
NestFactory on a real port and drives pending → approved → executed →
audited, plus pending → rejected → NOT executed, over real HTTP against a
live Tracker + PostgreSQL. A stubbed TrackerApprovalClient would have
reproduced the defect exactly, so every hop is a real request. Env-gated:
it SKIPS loudly without a Tracker rather than passing vacuously.
- `runtime.factory.spec.ts` gains the CI-runnable guard on the wiring the
integration depends on (the integration itself cannot run in CI — nothing
provisions a Tracker yet).
- helm values.yaml records that AGENT_RUNTIME_HARNESS_ROOT now drives the
catalogue as well as the executor.
Observed: `evolith_hitl_approvals_total{decision="approved"} 1` scraped from
the service's own /metrics after the run, and the approval rows persisted in
the Tracker's `tracker_governance.runtime_approvals` with the approver the
Tracker named.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
To create skeletons: node .harness/scripts/generate-es-skeleton.mjs <file.md>Generated by GitHub Actions |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes GT-608 (P1). Criteria 3 and 4 met, over real HTTP. Criteria 1 and 2 were already met.
The root cause was one line
Wave 2 had already built
ManifestSkillRegistryAdapter,deriveSkillsFromManifestand theharnessRootoption oncreateAgentRuntime— but no shipping composition ever passed it.createRuntimeFromEnvbuiltHarnessProcessAdapterfromAGENT_RUNTIME_HARNESS_ROOTand then dropped that root, so the deployed service executed a manifest it had never read: the catalogue stayed the hardcoded 7 skills, nothing routable declaredrequiresApproval: true, step 4 of the governed pipeline never ran, andevolith_hitl_approvals_totalwas structurally zero. The CLI factory had the identical gap.That is why the row's own impact note said two independent readers of the two halves disagreed about whether the opposite endpoint existed. Both halves were real; nothing connected them.
Criterion 3 — pending → approved → executed → audited
Every hop is a real HTTP request. A stubbed
TrackerApprovalClientwould have reproduced exactly the defect this row records:The dev-bypass cannot drive this:
/runtime-approvalsis bound to theCoreMachinescheme by name precisely so the bypass cannot reach the gate, and granting additionally requires a designated approver authority the bypass principal does not carry. So the human channel runs on the T-053 symmetric-key JWT path.Negative half (
harnessRootremoved, live Tracker still up): 3 failed —the DEPLOYED catalogue honours the manifest → Received: undefined, and both approval pathsExpected "blocked", Received "error". Green: 8 suites / 86 tests with the live Tracker.Rows the seam left in the other repo's ledger — proof it crossed the boundary:
Criterion 4 — the counter actually moved
evolith_hitl_approvals_total{decision="approved"} 1, scraped from the service's own/metricsafter the run, incremented byAgentMetricsService.recordRunfromtrace.approvedBy— the field only the Tracker can fill. The rejection test asserts the series does not move on a denial.Why a CI-runnable guard was added too
The integration spec skips in CI (nothing provisions a Tracker + PostgreSQL), so a regression would be silent.
runtime.factory.spec.tsguards the wiring and runs everywhere. Independently falsified by the integrating session: deleting the singleharnessRoot,line turns it red (1 failed / 39 passed); restored, 40/40.Conflict note for PR #310: the change to
runtime.factory.tsis one line plus a comment, no new imports, andsrc/packages/agent-runtime/src/adapters/index.tswas not touched at all.The row's second finding
Half true, and now closed for both shipping surfaces. The derivation machinery and CI parity test existed, but since nothing passed
harnessRootthe runtime posture was unchanged — 9 manifest capabilities invisible, two sources of truth wherever it mattered. Fixed in the runtime service and the CLI, conditional onAGENT_RUNTIME_HARNESS_ROOTbeing set: the Coolify deploy doc sets it, the Helm values still have it commented out, so a Helm deployment keeps the hardcoded table until someone uncomments it — documented in place rather than flipped, since that is a deploy decision coupled to the corpus mount.LocalSkillRegistryAdapterstill defaults to the 7 hardcoded skills correctly: it is the routing/semantics layer, and design rule #5 requires the runtime to boot with no.harnesscheckout.Deliberately left out
console.warnnaming the four required variables, so a skip is never mistaken for a pass. Provisioning is a separate infra item.FileApprovalStoreandHttpSlackClientremain unexercised end to end — unit specs only; there is no live Slack workspace, and the Tracker (not Slack) is the seam this row is about.pendingas a denial and requires re-submission of the same correlation id; the Tracker is idempotent bycorrelationIdand the test exercises that as-is.decision="blocked"is still never emitted —AgentMetricsServicecounts only an approval the trace attests, since a block can come from policy rather than a human. Existing decision, left alone.Verification hygiene: the worktree had no
node_modulesat all, so a fullnpm installran rather than symlinking the main checkout;readlink -fconfirmed worktree-local resolution, andtsc -bran before every measurement. Full CLI suite green: 101 suites / 1446 tests.🤖 Generated with Claude Code