Skip to content

installBrokerShapedAgentRelayBin does not verify the treatment was administered — removing the poisoning leaves the test green #428

Description

@khaliqgant

installBrokerShapedAgentRelayBin is the treatment in the hermetic regression test for the broker-collision fix from #426. It installs a fake broker-shaped agent-relay and empties PATH so a shell-out into the real CLI would fail loudly.

The helper never checks that the treatment was administered. Remove the poisoning and the test stays green — a real agent-relay on PATH masks the shell-out and the assertion passes for the wrong reason.

Evidence

Mutation battery run against 4b51077 during the #426 review, 5 mutations applied and reverted:

mutation result
collapse degradedStallClassFor to always Retryable RED (6 tests)
%w%v in refreshDelegatedCredentials RED (2 tests)
force resetNotice=true RED
reintroduce the shell-out with poisoning present RED — agent-relay CLI >= 8.7.0 required: error: unrecognized subcommand 'cloud'
remove the poisoning from the hermetic arm GREEN — should have been red

With the shell-out reintroduced and the poisoning removed, the test also went green: a real agent-relay on PATH masks the defect entirely.

Why it matters

TestRefreshDelegatedCredentialsCloudRemintIgnoresBrokerShapedAgentRelayBin is the arm that actually runs in CI. The properly self-verifying arm — TestLiveInheritedBrokerDelegatedRemint, whose t.Fatal("AGENT_RELAY_BIN must be inherited from the relay broker") fires correctly under both an unset var and a wrong-binary treatment — is gated on RELAYFILE_LIVE_BROKER_COLLISION=1 and skips in CI. The green "Go Test" check has never executed it.

So the guard that runs cannot detect its own treatment going missing, and the guard that can does not run.

Suggested fix

  1. Hoist a treatment-administered assertion into installBrokerShapedAgentRelayBin itself, so every present and future caller inherits it — assert the resolved agent-relay on PATH is the fake, and that a control invocation fails as expected.
  2. And/or run TestLiveInheritedBrokerDelegatedRemint in CI against a stub broker so the self-verifying arm is actually exercised.

Prefer (1): it is the version that cannot be forgotten when the pattern is copied to the next test.

Definition of done

Deleting the installBrokerShapedAgentRelayBin call from the hermetic test makes that test fail, with a message naming the missing treatment rather than the downstream assertion.


Found by the independent review of #426 (reviewer rf426-independent-review-0816, Check 3); filed by factory-lead. Not dispatched — no readiness label.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions