feat(pipedream): allow rollback final_pipeline override#99
feat(pipedream): allow rollback final_pipeline override#99joseph-sentry wants to merge 6 commits into
Conversation
Add optional `rollback.final_pipeline` to anchor the rollback material on an earlier group instead of the last pipeline in the chain. Useful when the tail group is flaky and would otherwise starve the rollback target pool. Errors at compile time if the named group doesn't exist. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 86ee098. Configure here.
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
Material is keyed {pipeline}-{final_stage}; final_stage defaults to the
appended pipeline-complete stage, not the deploy stage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Looks good overall, just one issue I forgot about previously. The rollback script ran in the gocd agents will abort the entire rollback if any pipeline is missing a passing run, essentially defeating the purpose of your change. Made a PR here: https://github.com/getsentry/devinfra-deployment-service/pull/874. Please wait to merge until that one is in.
You'll have to modify the PR to pass --allow-missing to gocd-emergency-deploy, e.g. emit a ROLLBACK_ALLOW_PARTIAL env var when final_pipeline is set and have rollback.sh add the flag.
One other small thing, the new jsonnet fixture is missing its golden files. Could you run
rm test/testdata/goldens/pipedream/rollback*.goldennpm run test
then commit the .golden files? Thanks.
- update rollback script to use allow-missing introduced in getsentry/devinfra-deployment-service#874 - update golden files
…S condition Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2 participants
Adds optional
rollback.final_pipelineto the pipedream config. It anchors the rollback pipeline's dependency material on a named group's final stage instead of the default (last pipeline in the chain).When the tail group (e.g.
st) is flaky, the rollback material rarely reaches its final stage, starving the pool of rollback-eligible SHAs. Anchoring on an earlier group (e.g.us) lets a SHA become rollback-eligible sooner.This caused an issue with rolling back during an inc recently, since our last stage hadn't run successfully for a week, meaning that we had to fix forward, which took longer than if we were to just rollback.
Trade-off (documented): an upstream-anchored target may not have been validated on the downstream groups the rollback then deploys it to.
🤖 Generated with Claude Code