Skip to content

fix: verify Pantheon push landed; pre-flight SFTP-mode flip; self-pin… - #101

Merged
square360-george merged 4 commits into
mainfrom
fix/v3.2.4-fail-on-pantheon-push-rejection
May 22, 2026
Merged

fix: verify Pantheon push landed; pre-flight SFTP-mode flip; self-pin…#101
square360-george merged 4 commits into
mainfrom
fix/v3.2.4-fail-on-pantheon-push-rejection

Conversation

@square360-george

Copy link
Copy Markdown
Member

… internal refs to v3.2.4

pantheon-systems/push-to-pantheon@0.9.0 doesn't propagate git push's exit code. When Pantheon rejects the push (SFTP-mode env, branch protection, lost-update race), the upstream action reports success and bobheadxi marks the GitHub deployment "success" — while Pantheon still has the old SHA. The PR gets a green check and a "Multidev Environment Ready" comment for a deploy that never happened.

Discovered: YH-571 / yalehealth-yale-edu PR #126. Security fix appeared deployed but drush deploy ran on stale code and re-installed a module we'd intentionally removed.

Two defensive layers added to .github/actions/pantheon-push/action.yml:

  • Pre-flight SFTP-mode flip: terminus connection:set git when the env exists and is in any mode other than git. Catches the most common cause of the silent rejection.
  • Verify-after-push: compare git rev-parse HEAD against git ls-remote refs/heads/ on the Pantheon repo. Fail loudly on mismatch with an actionable error pointing at the previous step's rejection notice. Catch-all for any silent-fail mode in the upstream action.

PR-comment step in reusable-pantheon-deploy-pr-multidev.yml gated with if: success() so the "Multidev Environment Ready" comment doesn't post on a failed deploy.

Self-pin internal @v3.2.3 → @v3.2.4 across all four reusable workflows.

Deferred: the bobheadxi/deployments@v1 "status: success" call lives inside the upstream pantheon-systems/push-to-pantheon@0.9.0 and runs before our verify step. The deployment record will briefly show success before the job goes red. Fixing requires forking upstream or adding a follow-up "reset to failure" step — not blocking the false-positive fix.

… internal refs to v3.2.4

pantheon-systems/push-to-pantheon@0.9.0 doesn't propagate git push's exit
code. When Pantheon rejects the push (SFTP-mode env, branch protection,
lost-update race), the upstream action reports success and bobheadxi
marks the GitHub deployment "success" — while Pantheon still has the
old SHA. The PR gets a green check and a "Multidev Environment Ready"
comment for a deploy that never happened.

Discovered: YH-571 / yalehealth-yale-edu PR #126. Security fix appeared
deployed but drush deploy ran on stale code and re-installed a module
we'd intentionally removed.

Two defensive layers added to .github/actions/pantheon-push/action.yml:

- Pre-flight SFTP-mode flip: terminus connection:set <env> git when
  the env exists and is in any mode other than git. Catches the most
  common cause of the silent rejection.
- Verify-after-push: compare git rev-parse HEAD against git ls-remote
  refs/heads/<env> on the Pantheon repo. Fail loudly on mismatch with
  an actionable error pointing at the previous step's rejection notice.
  Catch-all for any silent-fail mode in the upstream action.

PR-comment step in reusable-pantheon-deploy-pr-multidev.yml gated with
if: success() so the "Multidev Environment Ready" comment doesn't post
on a failed deploy.

Self-pin internal @v3.2.3 → @v3.2.4 across all four reusable workflows.

Deferred: the bobheadxi/deployments@v1 "status: success" call lives
inside the upstream pantheon-systems/push-to-pantheon@0.9.0 and runs
before our verify step. The deployment record will briefly show success
before the job goes red. Fixing requires forking upstream or adding a
follow-up "reset to failure" step — not blocking the false-positive fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 22, 2026 19:26
…Health pre-merge validation

Temporary commit. Flips internal `uses:` self-references from
@v3.2.4 (the tag that doesn't exist yet) to
@fix/v3.2.4-fail-on-pantheon-push-rejection (this branch) so that
Yale Health can consume the reusable workflows from this branch
and exercise the SFTP-rejection fix end-to-end before we merge.

MUST be reverted before merging to main. The pin-drift trap
(see memory: release-pin-drift-trap) requires that when v3.2.4
is tagged, the workflows at that tag contain @v3.2.4 self-refs,
not @branch-name refs. Revert this commit (or interactive-rebase
it out) once Yale Health validation passes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the shared Pantheon deployment workflows against a false-positive success case where Pantheon rejects a git push but the upstream pantheon-systems/push-to-pantheon@0.9.0 action still reports success. It adds defensive checks in the in-repo pantheon-push composite action and prevents “Multidev Environment Ready” PR comments from posting when the deploy actually failed.

Changes:

  • Add a pre-flight “ensure git mode” step and a post-push verification step to .github/actions/pantheon-push/action.yml to fail loudly when Pantheon did not advance to the expected SHA.
  • Gate the PR multidev “Environment Ready” comment on success() so it won’t post after a failed deploy.
  • Bump in-repo uses: self-references from @v3.2.3 to @v3.2.4 and document the incident/fix in repo docs.

Reviewed changes

Copilot reviewed 2 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
docs/todo/pantheon-push-swallows-rejection.md Adds an investigation write-up and records the resolution approach for the silent push-rejection issue.
docs/CHANGELOG-AGENT.md Documents the fix details and the internal self-pin bump as part of the agent changelog.
.github/workflows/reusable-pantheon-deploy-rc-multidev.yml Updates internal uses: references to @v3.2.4.
.github/workflows/reusable-pantheon-deploy-pr-multidev.yml Updates internal uses: references and gates the PR comment step on successful deploys.
.github/workflows/reusable-pantheon-deploy-epic-multidev.yml Updates internal uses: references to @v3.2.4.
.github/workflows/reusable-pantheon-deploy-dev.yml Updates internal uses: references to @v3.2.4.
.github/actions/pantheon-push/action.yml Adds pre-flight connection-mode enforcement and post-push ls-remote verification to detect silent push failures.
Comments suppressed due to low confidence (1)

.github/actions/pantheon-push/action.yml:135

  • MODE=$(terminus connection:info ... 2>/dev/null || echo "") suppresses errors and coerces failures into an empty string, which then triggers a connection mode flip with MODE shown as unknown. If connection:info fails (auth/network), it would be better to fail the step (or retry) rather than proceed and potentially attempt connection:set on incomplete info.
        MODE=$(terminus connection:info "${SITE}.${TARGET_ENV}" --field=connection_mode 2>/dev/null || echo "")
        echo "Current connection mode for ${SITE}.${TARGET_ENV}: ${MODE:-unknown}"
        if [[ "${MODE}" != "git" ]]; then
          echo "::warning::${SITE}.${TARGET_ENV} is in '${MODE:-unknown}' mode. Switching to git so the push isn't rejected by the pre-receive hook."
          terminus connection:set "${SITE}.${TARGET_ENV}" git
        fi

Comment thread .github/actions/pantheon-push/action.yml Outdated
Comment thread .github/actions/pantheon-push/action.yml Outdated
square360-george and others added 2 commits May 22, 2026 15:43
…or Yale Health pre-merge validation"

This reverts commit 00bdffb.
…iew on #101

Three improvements suggested by Copilot's PR review, all valid:

- Replace `terminus env:info >/dev/null 2>&1` with `terminus env:list
  --format=list | grep` for the env-existence probe. The previous form
  swallowed all errors as "env doesn't exist," masking auth/network
  failures. env:list is authoritative and lets set -e fail the step
  on real errors. Matches the existing pattern in
  reusable-pantheon-deploy-rc-multidev.yml for the live-env check.

- Drop the `2>/dev/null || echo ""` fallback on connection:info inside
  the pre-flight step. If that call fails, we want terminus's real
  error rather than attempting connection:set on an empty MODE.

- Add a 3-attempt retry with 5s/10s backoff around the verify-after-push
  ls-remote call. Newly-created multidevs can have a brief window where
  Pantheon accepted the push but the ref isn't yet visible via
  ls-remote (same family of propagation lag the post-deploy drush
  step also defends against). Existing multidevs (the case we tested
  on YH PR #126) wouldn't hit this; new PR opens would.

None of these change the success-path behavior validated against
yalehealth-yale-edu PR #126 (logs_70381879410). They tighten edges
around auth/network failures and freshly-provisioned multidevs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@square360-george
square360-george requested a review from Copilot May 22, 2026 19:47
@square360-george
square360-george merged commit 223a513 into main May 22, 2026
1 check failed
@square360-george
square360-george removed the request for review from Copilot May 22, 2026 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants