Skip to content

Warn when a usr1/usr2 restart leaves a config change unapplied - #42

Open
facundofarias wants to merge 1 commit into
adamcooke:masterfrom
deployhq:warn-usr-restart-keeps-environment
Open

facundofarias wants to merge 1 commit into
adamcooke:masterfrom
deployhq:warn-usr-restart-keeps-environment

Conversation

@facundofarias

Copy link
Copy Markdown

Problem

Supervisor#restart reloads the configuration before restarting anything, and Config#reload clears @options, @local_options and @environment_variables — so procodile restart genuinely does pick up Procfile/Procfile.options/Procfile.local changes.

Whether a given process sees them depends entirely on restart_mode:

restart_mode behaviour new environment applied?
start-term, term-start spawns a replacement yes
usr1, usr2 ::Process.kill(mode, @pid) no

usr1/usr2 signal the running process rather than replacing it. The environment is only ever applied at Process.spawn, so such a process keeps the environment it was born with indefinitely — while Procodile reports the restart as successful.

This is genuinely hard to diagnose from outside: the config on disk is correct, status looks healthy, and only the application's behaviour disagrees. We lost time to exactly this — an environment variable added to Procfile.local, procodile restart run, and the value never reached the process.

Change

Record the environment and command each instance was actually spawned with. After signalling a usr1/usr2 process, log a warning if either has since changed:

Warning: environment changed since this process started, but usr1 signals the
existing process rather than replacing it. Stop and start this process for the
change to take effect.

Notes

  • Fires only when something actually changed, so routine restarts stay quiet.
  • Instances this supervisor never spawned record nothing and are skipped, so there are no false warnings after a supervisor restart.
  • No behaviour change — this is diagnostics only. Signalling semantics are untouched.

Specs in spec/specs/instance_spec.rb cover unchanged, environment-changed, variable-added, command-changed, both-changed, and never-spawned. Full suite green (40 examples).

Supervisor#restart reloads the configuration before restarting anything, so
`procodile restart` does pick up changes to the Procfile, Procfile.options and
Procfile.local -- Config#reload clears @options, @local_options and
@environment_variables.

Whether a given process *sees* those changes depends entirely on its
restart_mode. start-term and term-start spawn a replacement, which is given the
freshly computed environment. usr1 and usr2 signal the running process instead:

    ::Process.kill(@process.restart_mode.upcase, @pid)

Nothing is re-spawned, and the environment is only ever applied at
Process.spawn, so a usr1 process keeps the environment it was born with
indefinitely. Procodile reports the restart as successful either way, and
there is no indication that the change did not land. Diagnosing this from the
outside is genuinely hard: the config on disk is correct, `status` looks
healthy, and only the application's behaviour disagrees.

Record the environment and command each instance was actually spawned with,
and after signalling a usr1/usr2 process, log a warning if either has since
changed. Instances that this supervisor never spawned record nothing and are
skipped, so there are no false warnings after a supervisor restart.

The check only fires when something genuinely changed, so routine restarts
are unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P89F451YDRDBj5br14i3GD
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant