Repository navigation
Conversation
If a mold exits right after sending `MoldReady`, `reap_all_workers` can remove it before the monitor reads the message. `Children#update` then falls back to a lookup by `nr`, which `MoldReady` doesn't have, and raises `NoMethodError`. Had it returned nil, the monitor would have failed on `nil.to_log` instead. In the main loop the error is rescued and logged. During shutdown it escapes `stop`, so the monitor exits without killing the children that are still running. Only fall back to `nr` for messages that carry one, and have the monitor log and ignore `MoldReady` for a mold it no longer tracks. The integration test pauses the monitor with SIGSTOP so the mold always exits before its `MoldReady` is read.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why?
If a mold exits right after it reports ready, the monitor can reap it before reading that message. Handling the message then raises
undefined method 'nr' for an instance of Pitchfork::Message::MoldReady (NoMethodError). Normally the monitor just logs this, but during shutdown the error escapes and the monitor exits without killing the children that are still running.How?
The child lookup now falls back to the worker number only for messages that have one, and the monitor logs and ignores a ready message from a mold it no longer tracks.
Notes for reviewers
I found this while working on #235. That PR has the shutdown drain read control messages, which makes this easier to hit, but master has the same bug.
The integration test pauses the monitor with SIGSTOP so the mold always exits before its ready message is read. On master it fails with the error above.