Stop the macOS binary printing a traceback on every diff - #201
Merged
Merged
Conversation
Every diff through the macOS binary printed a traceback to stderr:
ModuleNotFoundError: No module named 'numpy._core._multiarray_umath'
tqdm builds a multiprocessing.RLock for its default write lock the first time a
progress bar is constructed, at __main__.py:357. Registering that lock's
semaphore starts multiprocessing.resource_tracker, which re-executes
sys.executable with the interpreter's own flags. Under PyInstaller
sys.executable is the Graphtage binary, so the helper re-ran Graphtage with
`-B -S -I`, and that re-execution failed importing numpy in isolated mode.
Only macOS was affected, because it spawns rather than forks. Only 0.5.0 showed
it, because graphtage/__init__.py now imports batch_distance, which imports
numpy at module scope, so the re-executed process had numpy on its import path
for the first time. The diff itself was always correct; the noise came entirely
from the helper.
Graphtage never uses multiprocessing, so the lock guards nothing. Installing a
threading lock at import keeps tqdm from building the multiprocessing one.
Measured on a rebuilt macOS binary, for both the smoke test's 2-key diff and a
45-key diff large enough to reach the numpy kernel: stderr goes from 1948 bytes
to 0, exit statuses are unchanged, and the rendered diff stays byte-identical to
the source install.
The existing tests in test_progress.py cannot catch this, because from a source
checkout the helper is a real interpreter and exits quietly. The new test
asserts the tracker never starts. Verified it fails without the fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F2sHz5c5TvMs9tFn2HhwaC
The smoke test checked only the exit status, so the macOS binary shipped a traceback on every diff while the step stayed green: the diff was correct and the traceback came from a helper process. `--no-status` promises an empty stderr, and test_progress.py already asserts that for a source checkout. Hold the frozen binary to the same contract, where re-executing sys.executable is a live hazard. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F2sHz5c5TvMs9tFn2HhwaC
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every diff through the macOS binary built from
masterwrites a traceback to stderr. It is visible in theBuild binary artifactsdry run onmaster(run 35127349018),immediately after the correct diff output:
The smoke test passed anyway, because it only checks the exit status.
Root cause
Not numpy.
graphtage/__main__.py:357constructs the first progress bar, and tqdm builds amultiprocessing.RLockfor its default write lock:Registering that lock's semaphore starts
multiprocessing.resource_tracker, which re-executessys.executablewith the interpreter's own flags. Under PyInstaller
sys.executableis the Graphtage binary, so the helper re-runsGraphtage with
-B -S -I. Locally that surfaces asgraphtage: error: unrecognized arguments: -B -S -I; on the CIrunner the re-execution fails earlier, importing numpy in isolated mode.
Two things make this new and macOS-only:
graphtage/__init__.pynow importsbatch_distance, which imports numpy at module scope, so the re-executedprocess had numpy on its import path for the first time. v0.4.0's binary does not show this.
The diff itself was never wrong. All the noise came from the helper.
Fix
Graphtage never uses multiprocessing, so the lock guards nothing.
graphtage/progress.pyinstalls a threading lockat import, which keeps tqdm from building the multiprocessing one.
Measured on a rebuilt macOS binary
--no-status --format yaml(the smoke test's 2-key diff)--no-status --no-coloron a 45-key diff (reaches the numpy kernel)The rendered diff is byte-identical to the source install before and after, and identical under both
GRAPHTAGE_BATCH_BACKENDsettings.--versionand-dumpversionwere already clean.Testing
test_progress.pygainstest_drawing_a_progress_bar_starts_no_helper_process, which asserts the resource trackernever starts. Its four existing siblings cannot catch this: from a source checkout the helper is a real interpreter
and exits quietly, so stderr stays empty either way. Verified the new test fails without the fix.
The second commit closes the gap that let this ship undetected: the binary smoke test now fails if
--no-statuswrites anything to stderr, which is the contract
test_no_status_suppresses_progressalready holds the sourceinstall to.
Full suite: 273 passed, 1770 subtests. Ruff and actionlint clean.
🤖 Generated with Claude Code
https://claude.ai/code/session_01F2sHz5c5TvMs9tFn2HhwaC