[316] Converge a dashboard's CloudFormation stack on republish - #348
Draft
CarsonDavis wants to merge 1 commit into
Draft
[316] Converge a dashboard's CloudFormation stack on republish#348CarsonDavis wants to merge 1 commit into
CarsonDavis wants to merge 1 commit into
Conversation
Terraform plan preview —
|
Terraform plan preview —
|
This was referenced Aug 27, 2026
CarsonDavis
force-pushed
the
316-dashboard-subpath
branch
from
August 27, 2026 16:15
1b2ccc3 to
9f3ad31
Compare
CarsonDavis
force-pushed
the
316-republish-stack-convergence
branch
from
August 27, 2026 16:30
60c6db9 to
6bb5bf6
Compare
CarsonDavis
force-pushed
the
316-dashboard-subpath
branch
from
August 27, 2026 18:02
9f3ad31 to
078adc9
Compare
The `update` action only re-uploaded files, so a dashboard created before an
edge-function change could never receive it: the stack was created once and
never updated. Update now renders the current template, calls UpdateStack, and
polls to UPDATE_COMPLETE — treating "No updates are to be performed" as an
up-to-date no-op — and the publish action reuses that same describe so a
re-run over a stack mid-operation waits it out instead of dying on
AlreadyExistsException. The publish task role, the Terraform module, and the
permissions boundary each grant cloudformation:UpdateStack plus the
CloudFront read-back/update actions the converge needs; DescribeStackEvents,
which nothing calls, is dropped. A test pins the template's logical IDs and
the bucket's anonymity, since renaming either would make UpdateStack REPLACE
the distribution and mint a new domain.
The wait itself has to survive DescribeStacks' eventual consistency: the first
polls after an UpdateStack can still report the pre-update status, which for
the ordinary republish IS UPDATE_COMPLETE. waitForStack therefore takes the
prior { status, lastUpdatedTime } and treats a poll as stale only while it
matches BOTH — an advanced LastUpdatedTime is positive proof this update
landed, so an update that starts and finishes inside one poll interval is
recognized rather than polled to a false timeout, and a rollback back to the
resting status still fails with its reason. That reason is the last non-empty
StackStatusReason seen while polling, because CloudFormation puts it on the
in-progress rollback and leaves the terminal status empty. UPDATE_FAILED joins
the terminal statuses so a stuck stack throws instead of being polled for
thirty minutes, and CREATE_FAILED — where this code's OnFailure "DO_NOTHING"
leaves a failed first publish — joins the statuses that get the
delete-and-republish guidance.
Two republish clicks start two ECS tasks, and CloudFormation rejects the
loser's UpdateStack outright. Rather than mark that row failed after the other
task succeeded, the update path recognizes the rejection, waits the in-flight
operation out, and carries on to the upload, so a double republish stays
harmlessly last-write-wins. planStackWait() derives the wait parameters for
each of these cases, so the wiring between the two is table-tested rather than
implicit in the call sites.
CarsonDavis
force-pushed
the
316-republish-stack-convergence
branch
from
August 27, 2026 18:08
6bb5bf6 to
1f719bb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #316 — PR 2 of 3, stacked on #330 (PR 3 is #349). Retarget to
developmentafter #330 merges.Before this, a republish (the
updateaction) only re-uploaded files — the dashboard's CloudFormation stack was created once and never touched again, so an existing dashboard would never receive the edge-function change from #330. This makes republish converge the infrastructure too.UpdateStackand polls to completion. Staleness of the eventually-consistent DescribeStacks reads is keyed onLastUpdatedTime(a read is stale only while the status equals the pre-update status and the timestamp hasn't advanced), so a stale pre-updateUPDATE_COMPLETEcan't be mistaken for convergence and a fast update can't be mistaken for a timeout. Failure throws carry the last non-emptyStackStatusReasonseen during the poll.is in UPDATE_IN_PROGRESS state and can not be updated) waits for the other task's update to settle and continues with its own upload instead of marking the row failed;UPDATE_FAILEDis treated as terminal instead of polling a stuck stack for 30 minutes; aCREATE_FAILEDfirst publish gets the same actionable delete-and-republish guidance as the rollback states.planStackWait({action, existing})decides the wait parameters for both the publish settle branch and the update branch, table-tested so dropping any parameter fails the suite.cloudformation:UpdateStackandcloudfront:UpdateFunctiongranted in all three layers (task role, Terraform module, permissions boundary), pluscloudfront:GetDistributionConfigin the task role and Terraform module (the boundary already had it), andcloudfront:UpdateOriginAccessControl/GetOriginAccessControlConfigin all three layers so a future template change to the OriginAccessControl can converge instead of failing mid-update. Also deliberately removescloudformation:DescribeStackEventsfrom all three layers — verified unused by any code on any branch; failure reasons come from DescribeStacks.Verification: 1594 unit tests green. The convergence tests were mutation-audited empirically — each guard was reverted (status-equality-only staleness, ignoring
LastUpdatedTime, the transition-latch alternative, terminal-only reason reads,UPDATE_FAILEDnon-terminal, unwiredplanStackWaitparams) and the corresponding tests observed to fail.Post-merge: one republish in the DEV account to verify live stack convergence end to end (the only piece not verifiable locally).