fix(agent): log-delivery pin table is keyed by stack name but holds per-account state, so it breaks deploys it was meant to protect

[isadeks]

Summary

PINNED_LOG_DELIVERY_BY_STACK (cdk/src/stacks/agent.ts:1469) is keyed on stack name, but the values it holds are per-account deployed state. Two accounts running a stack of the same name have diverged, so a single hard-coded table cannot be correct for both: whichever account it matches deploys cleanly, and the other rolls back with

AWS::Logs::DeliverySource  CREATE_FAILED
  "This ResourceId has already been used in another Delivery Source in this account."
  (HandlerErrorCode: AlreadyExists)

The whole stack update then rolls back.

Background: why the pin exists at all

The AgentCore Runtime L2 auto-creates AWS::Logs::DeliverySource / Delivery / DeliveryDestination per logging config, and names them from the construct path the library uses internally. When that path changes between library versions, the CFN logical IDs change with it — and CloudFormation treats a renamed resource as a new one, creating it before deleting the old.

That is fatal for a DeliverySource specifically: it is unique per (resource ARN, log type) account-wide, and the runtime ARN does not change across the rename. So the new source collides with the live one still pointing at the same runtime.

Worth stating plainly, because it is counter-intuitive: renaming these resources cannot avoid the collision. The conflict is on the ARN they point at, not on their own names. Only holding the logical ID steady avoids it, because that is what makes CloudFormation update in place instead of creating a second source for the same runtime.

pinLogDeliveryLogicalIds() (agent.ts:1529) does that by overriding the logical IDs from a table of values recorded off a live stack.

The bug

The table records one account's state and is selected by stack name:

const PINNED_LOG_DELIVERY_BY_STACK: Record<string, readonly PinnedLogResource[]> = {
  'backgroundagent-dev': [
    { childId: 'ApplicationLogsDeliverySource',
      liveLogicalId: 'RuntimeCDKSourceAPPLICATIONLOGSbackgroundagentdevRuntimeBC0AE9ED96A02E02',
      liveName: 'cdk-applicationlogs-source-backgroundagentdevRuntimeBC0AE9ED' },
    …

Stack name is not a proxy for deployed state. Two accounts, both with a stack named backgroundagent-dev, are in different states as of 2026-07-28:

account	live logical ID	live physical name
A	RuntimeCDKSourceAPPLICATIONLOGS…96A02E02	cdk-applicationlogs-source-…
B	RuntimeApplicationLogsDeliverySource818497BD	backgroundagentdevRuntimeAlicationLogsDeliverySource07205B47

Account B crossed over when a commit that removed the shim was deployed there on 2026-07-28 (the six delivery resources' LastUpdated still read that morning). The shim was restored afterwards but never deployed to B, so B never moved back. Account A never deployed that commit, so it still has the pre-rename naming.

Consequence: main today deploys cleanly to A and rolls back on B. Re-recording the table from B's live stack simply inverts that. There is no single set of literal values that serves both.

Evidence

On B, the failed update emitted CREATE_FAILED with AlreadyExists on
RuntimeCDKSourceAPPLICATIONLOGSbackgroundagentdevRuntimeBC0AE9ED96A02E02.

That event is decisive on its own, independent of any template read: CloudFormation only emits CREATE for a logical ID not present in the deployed template. So B's deployed template does not contain the pinned ID — the pin is introducing the rename there rather than preventing it.

Confirmed on B by three independent probes: aws logs describe-delivery-sources (account-level, no CFN involved), get-template --template-stage Processed, and list-stack-resources. All three agree on the …818497BD naming.

On A, cdk diff against the live stack shows all six delivery resources matching — 0 created, 0 deleted — which is why the same main deploys cleanly there.

Confirmation that the pin is the trigger

On B, deleting the table key so the lookup misses (letting the library name the resources) made the deploy succeed: UPDATE_COMPLETE, zero failed resources, and the six delivery resources untouched — their timestamps still read 2026-07-28. That is the clean outcome, and it is only obtainable there by disabling the pin.

Why not just re-record the table

Because the table is the wrong shape, not the wrong contents. Re-recording:

• fixes B, breaks A;
• has to be redone by hand every time any deployment crosses a library rename;
• silently applies one account's liveName values — which are account-unique — to any other stack sharing the name.

The comment above the table already says the entries are "a record of what CloudFormation already has" and must be re-recorded when they stop matching. The problem is that "what CloudFormation already has" differs per account, and a source-controlled literal cannot express that.

Two account-agnostic fix shapes

1. Derive, don't tabulate. Resolve the existing logical IDs from the deployed stack instead of a literal — a CFN lookup or an equivalent — so the pin is always correct for the account being deployed to, with no table to maintain. Cost: synth needs credentials for that account, which affects offline synth and CI.

2. Own the logical IDs outright. overrideLogicalId these resources to stable, self-chosen IDs at construction, so the library's internal naming never reaches the template and future library renames are inert. This removes the class of bug rather than tracking it. Cost: a one-time migration for each existing state, since both A's and B's current IDs differ from any new scheme — likely a retain-and-import, or a documented one-off.

(2) is the better long-term shape; (1) is closer to the current behaviour.

Constraint on any fix

It must be non-breaking by default, with no opt-in flag. An earlier iteration gated the pin behind -c pinnedLogDeliveryStack=<name>, which inverted the default: the safe path was the one an operator had to already know about, and the failure that teaches them is a mid-update rollback whose message never mentions the flag. That was corrected to apply unconditionally; the same reasoning should apply to whatever replaces the table.

Scope

Affects any deployment that crossed the library rename. Fresh installs are unaffected — they have no pre-existing resources to collide with. The underlying issue is general, though: a template that depends on an auto-generated name from a construct library is one upstream rename away from an un-deployable stack.

Repro

1. Deploy a stack named backgroundagent-dev from a commit before the pin existed, so it takes the library's naming.
2. Deploy main (4357c353) to the same account.
3. The update fails on AWS::Logs::DeliverySource with AlreadyExists and rolls back.

References

• cdk/src/stacks/agent.ts:1469 — PINNED_LOG_DELIVERY_BY_STACK
• cdk/src/stacks/agent.ts:1529 — pinLogDeliveryLogicalIds()
• main @ 4357c353

[isadeks]

Live confirmation that the pin is the trigger, not the guard

Follow-up evidence from the affected account, which closes the loop on the diagnosis above.

Disabling the table lookup for that account — so the library names the resources instead of the pin — made the deploy succeed: UPDATE_COMPLETE, zero failed resources, and the six log-delivery resources untouched (their timestamps still read 2026-07-28). An end-to-end task on the resulting stack also passed: COMPLETED in 43s, 8 turns, opened a PR, and the AgentCore runtime log group showed fresh activity at the run time.

That is the cleanest possible confirmation. On that account the pin is what introduces the rename, and removing it produces exactly the zero-churn outcome the pin exists to guarantee.

It also settles which of the two fix shapes is right. If letting the library name the resources yields zero churn there, then the problem is not that the table holds stale values — it is that a source-controlled literal cannot describe per-account deployed state at all. Re-recording it would just move the breakage to the other account.

Operational note

The affected account is currently running main plus an uncommitted local edit that disables the table lookup. That edit is deliberately not committed, because the other account needs the pins live. Anyone deploying pristine main to that account will hit the rollback again until this is fixed — worth knowing if more than one person deploys there.

Status

Not started, and deliberately so: per ADR-003 this needs the approved label from an admin before implementation. Flagging for triage and priority.