As a person responsible for a live workflow, I want retrying a run to run the version that ran, so that a different result tells me something.
Part of #4852 (Sandbox DevX epic). Phase 2.
Outcome
Retrying a run from the editor re-runs its input against the version that run executed, not against whatever is live now.
The question a retry answers is whether a failure was the world or the workflow. Same code, same input, different result means the thing you were calling was down. That only holds if the code is the same. Today the retry takes the old input and runs it against the latest version, so a different result tells you nothing, and someone checking whether an overnight failure was a blip gets an answer to a question they did not ask.
Every comparable platform pins a retry to what ran. GitHub Actions replays the same commit and the same workflow file. CircleCI reruns the same pipeline. Buildkite and GitLab are the same. The one that works our way is Airflow, where clearing a task reruns it against the current definition, and it is a known trap.
Scope
- A retry started from a run in the editor uses that run's own version.
- The bulk retry on the history page keeps today's behaviour. That one is used after a fix, where running the latest version is the whole point. The two are different actions that currently share one path, which is why this is not a one-line change.
- A version that retention has since removed cannot be retried faithfully. Say so and offer to run the input against the latest version instead, rather than failing.
Notes
This changes behaviour for everyone, flag or not, so it needs a decision before any code: is a retry a reproduction, or another attempt at the goal? My reading is that the editor wants the first and the history page wants the second, and they should stop sharing a path. Product's call.
While this is open, the editor's retry says in its tooltip that it runs the latest version, so at least nobody is surprised.
As a person responsible for a live workflow, I want retrying a run to run the version that ran, so that a different result tells me something.
Part of #4852 (Sandbox DevX epic). Phase 2.
Outcome
Retrying a run from the editor re-runs its input against the version that run executed, not against whatever is live now.
The question a retry answers is whether a failure was the world or the workflow. Same code, same input, different result means the thing you were calling was down. That only holds if the code is the same. Today the retry takes the old input and runs it against the latest version, so a different result tells you nothing, and someone checking whether an overnight failure was a blip gets an answer to a question they did not ask.
Every comparable platform pins a retry to what ran. GitHub Actions replays the same commit and the same workflow file. CircleCI reruns the same pipeline. Buildkite and GitLab are the same. The one that works our way is Airflow, where clearing a task reruns it against the current definition, and it is a known trap.
Scope
Notes
This changes behaviour for everyone, flag or not, so it needs a decision before any code: is a retry a reproduction, or another attempt at the goal? My reading is that the editor wants the first and the history page wants the second, and they should stop sharing a path. Product's call.
While this is open, the editor's retry says in its tooltip that it runs the latest version, so at least nobody is surprised.