fix(install): survive the rgbmatrix build on low-memory Pis - #430
Merged
Conversation
The one-shot installer failed at Step 6 on a 1GB Pi with "Failed building wheel for rgbmatrix", and told the user to install build tools they already had. The real cause was the kernel OOM killer. Upstream's pyproject.toml declares no [tool.scikit-build] options, so scikit-build-core drives Ninja at its default of nproc+2 jobs -- six concurrent compiles on a 4-core Pi. CMakeLists.txt compiles the same 14 sources three times (~45 translation units), two of them Cython-generated C++ where a single cc1plus peaks near 800MB. That does not fit in 512MB-1GB of RAM. Add scripts/install/lib_lowmem.sh and wire it into the installer: - Cap build parallelism at max(1, min(cores, RAM/768)) via CMAKE_BUILD_PARALLEL_LEVEL, which is what cmake --build actually reads. MAKEFLAGS is ignored by Ninja and is set only as a Makefile-generator fallback. A 4GB Pi 4 still gets 4 jobs; 512MB and 1GB boards get 1. - Add a temporary swapfile sized to bring RAM+swap to 3GB (capped at 2GB), removed once the build finishes. An EXIT trap is the backstop for the error path. Nothing is written to /etc/fstab or /etc/dphys-swapfile. Existing swap is measured excluding zram, which is compressed RAM and so does not help a build OOM. - Keep pip's build tree off tmpfs. Debian 13 mounts /tmp as tmpfs, so the default held the whole C++ build tree in RAM alongside the compiler. - Diagnose OOM failures from the build log and the kernel ring buffer, instead of always blaming missing build tools. The OOM killer writes nothing to pip's output, which is why this was misreported. - Report RAM and the chosen job count in the Step 1 preflight, and emit a heartbeat during the compile so a deliberately serial 15-25 minute build does not look like a hang. New flags --skip-swap and --build-jobs N, with LEDMATRIX_SKIP_SWAP and LEDMATRIX_BUILD_JOBS equivalents. Also skip the duplicate apt-get update that the one-shot installer and first_time_install.sh each ran a minute apart, and complete the dphys-swapfile advice in diagnose_dependencies.sh with the CONF_MAXSWAP line, without which raising CONF_SWAPSIZE above 2048 is silently clamped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdsJs65WnUo8BtKHMAGA1q
Contributor
|
Warning Review limit reached
Next review available in: 19 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe installers now support low-memory boards through memory checks, reduced build parallelism, temporary swap, disk-backed build storage, progress reporting, and OOM diagnostics. Tests cover the helper library. Documentation describes setup, troubleshooting, and swap configuration. ChangesLow-memory installation flow
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant one-shot-install.sh
participant first_time_install.sh
participant lib_lowmem.sh
participant rgbmatrix_build
one-shot-install.sh->>first_time_install.sh: pass LEDMATRIX_APT_UPDATED=1
first_time_install.sh->>lib_lowmem.sh: detect memory, swap, and temporary storage
lib_lowmem.sh-->>first_time_install.sh: build jobs and swap requirements
first_time_install.sh->>rgbmatrix_build: run controlled parallel build
rgbmatrix_build-->>first_time_install.sh: build output and status
first_time_install.sh->>lib_lowmem.sh: remove temporary swap
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Up to standards ✅🟢 Issues
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@first_time_install.sh`:
- Around line 1078-1089: Check the exit status of mkdir -p in the
LOWMEM_AVAILABLE branch before retaining BUILD_TMPDIR. If directory creation
fails, report a clear disk-space/build-directory error and continue without
using the invalid path, preserving the existing run_rgbmatrix_build flow for
successful creation.
- Around line 304-357: Move the existing positive-integer validation for
BUILD_JOBS_OVERRIDE to the beginning of check_memory, before the
LOWMEM_AVAILABLE early return, so invalid --build-jobs or LEDMATRIX_BUILD_JOBS
values always produce the clear validation error. Preserve the current
assignment behavior for valid overrides and avoid duplicating the validation in
the low-memory branch.
In `@scripts/install/one-shot-install.sh`:
- Around line 260-267: Update the non-root installation path around the apt
update in one-shot-install.sh so LEDMATRIX_APT_UPDATED=1 is passed explicitly
through the environment used for the sudo -E child, ensuring
first_time_install.sh receives it even when sudo does not preserve exported
variables. Keep the root path unchanged and preserve the existing apt-update
flow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 865a0bf1-1eac-431e-bf24-e49497ca1c31
📒 Files selected for processing (7)
README.mddocs/TROUBLESHOOTING.mdfirst_time_install.shscripts/diagnose_dependencies.shscripts/install/lib_lowmem.shscripts/install/one-shot-install.shtest/test_install_lowmem.py
Three fixes from PR review: - Validate --build-jobs / LEDMATRIX_BUILD_JOBS before check_memory's fallback return. When lib_lowmem.sh is absent that return also honoured the override, so a non-numeric value skipped validation and instead blew up later in an arithmetic test in Step 6 with a generic error. - Fall back to the default TMPDIR when the disk-backed build directory cannot be created, rather than pointing the build at a path that does not exist. A nearly-full disk is the likely cause on exactly the devices this targets. - Pass LEDMATRIX_APT_UPDATED explicitly to the sudo child instead of relying on -E, which a sudoers env_reset/env_keep policy can strip. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdsJs65WnUo8BtKHMAGA1q
The build progress heartbeat slept for the full 30s report interval before re-checking whether the compile had finished, so every build paid up to 30 seconds of dead wall time -- including fast ones on a Pi 4/5 and every --force-rebuild run. Poll every 2s and report every 30s instead. Measured: 30s of overhead on an instant build drops to 2s, with heartbeats still emitted on the same schedule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdsJs65WnUo8BtKHMAGA1q
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Summary
The one-shot installer failed at Step 6 on a low-memory Pi with
Failed building wheel for rgbmatrix, and told the user to install build tools they already had. The real cause was the kernel OOM killer. This caps build parallelism based on available RAM, adds a temporary swapfile for the compile, keeps pip's build tree off tmpfs, and correctly diagnoses OOM failures instead of always blaming the toolchain.Type of change
Related issues
Reported on Discord by
wahf, who lost about an hour to the misleading error message before diagnosing it as memory and working around it with a 2GBdphys-swapfileswap.Root cause
Upstream
hzeller/rpi-rgb-led-matrix'spyproject.tomldeclares no[tool.scikit-build]options, so scikit-build-core uses the Ninja generator at its own default ofnproc + 2— six concurrent compiles on a 4-core Pi.CMakeLists.txtcompiles the same 14 sources three times (the shared library plus thecoreandgraphicsextensions), about 45 translation units, two of which are Cython-generated C++ where a singlecc1pluspeaks near 800MB at-O3. That does not fit in 512MB–1GB of RAM.Compounding it,
one-shot-install.shexportedTMPDIR=/tmpand Debian 13 mounts/tmpas tmpfs, so the entire C++ build tree was held in RAM alongside the compiler.The failure was then misreported: the OOM killer writes nothing to pip's output, so the handler's fixed "install build tools" message was the only thing the user saw.
What changed
New
scripts/install/lib_lowmem.sh, sourced byfirst_time_install.sh(with a fail-soft fallback if absent):max(1, min(cores, RAM/768))viaCMAKE_BUILD_PARALLEL_LEVEL, which is whatcmake --buildactually reads.MAKEFLAGSis ignored by Ninja and is set only as a Makefile-generator fallback. A 4GB Pi 4 still gets 4 jobs; 512MB and 1GB boards get 1.EXITtrap is the backstop for the error path. Nothing is written to/etc/fstabor/etc/dphys-swapfile. Existing swap is measured excluding zram, since zram is compressed RAM and does not help a build OOM./tmpis memory-backed.New flags
--skip-swapand--build-jobs N, withLEDMATRIX_SKIP_SWAP/LEDMATRIX_BUILD_JOBSequivalents.Two unrelated cleanups found while in here: the one-shot installer and
first_time_install.sheach ranapt-get updatea minute apart, so the second is now skipped; and thedphys-swapfileadvice indiagnose_dependencies.shwas missing theCONF_MAXSWAPline, without which raisingCONF_SWAPSIZEabove 2048 is silently clamped.Test plan
EMULATOR=true python3 run.py)scripts/dev_server.py)pytest)Details:
test/test_install_lowmem.py— 31 tests, all passing, running in the existing pytest CI job. Covers the job-sizing table (512MB→1 job … 8GB→4), the swap-sizing table, zram exclusion, and OOM detection. The last of those is the direct regression test for the misdiagnosis: it asserts a kernel-log-only OOM is detected and that a missing-Python.hfailure is not flagged as OOM.shellcheck -S warningis clean on all four shell scripts;bash -npasses on each.swapon --show→ teardown, plus the refusal paths (enough RAM, disk starvation, stale swapfile from a killed run) and confirmation that theEXITtrap releases swap even when theERRhandler exits.python3onPATH: the build receivesCMAKE_BUILD_PARALLEL_LEVEL,MAKEFLAGSand the redirectedTMPDIR.Not verified on hardware. I have no low-memory Pi in this environment, so the end-to-end claim — that a 512MB/1GB board now completes Step 6 — rests on the unit tests plus the reporter's own confirmation that swap resolved it. Worth one real run on a Pi 3B/3B+ before merging;
sudo LEDMATRIX_FORCE_LOW_RAM=512 ./first_time_install.sh -y --no-reboot-prompt --force-rebuildforces the low-memory path on any device.Documentation
README.mdif user-facing behavior changeddocs/if developer behavior changedREADME.mdgains a RAM note in Hardware Requirements;docs/TROUBLESHOOTING.mdgains an "Installation & Build Issues" section (the file had none) documenting this exact failure.Plugin compatibility
Checklist
CONTRIBUTING.mdCONTRIBUTING.mdandCODE_OF_CONDUCT.mdNotes for reviewer
Judgment calls worth a second opinion:
dphys-swapfileroute. That avoids editing a config file the user owns and limits SD-card wear, but it means a 1GB Pi gets no runtime swap afterwards — arguably useful for running the app. Easy to flip if you'd rather it persist.nprocis the saner default and it keeps one formula across all devices.LEDMATRIX_APT_UPDATEDrelies onsudo -Epreserving the variable. If a sudoers policy strips it, the fallback is simply the redundantapt-get updatewe have today — no breakage.Generated by Claude Code
Summary by CodeRabbit
New Features
--skip-swapand--build-jobsinstaller options.Documentation