configs: full32L bf16-source batch ladder — b192 + b224 probes#945
Open
ocg-goodfire wants to merge 7 commits into
Open
configs: full32L bf16-source batch ladder — b192 + b224 probes#945ocg-goodfire wants to merge 7 commits into
ocg-goodfire wants to merge 7 commits into
Conversation
… sharding The prior device_put-onto-sharding was a confirmed no-op (StandardRestore already honors the sharding spec — verified via an instrumented resume: restored CI-fn was correctly ÷N). The real resume-OOM is a ÷1-scale ENTRY RELAYOUT on the first resumed step: orbax-restored arrays carry a default memory layout that differs from what the jitted step was compiled for, so the same executable that runs fresh at ~150GB materialized a ~103GB buffer on resume. The fresh-init reference is built by the same XLA layout assignment as the step, so its `.format` (layout + sharding) IS the step's expected input layout; coercing the restored tree onto it removes the entry relayout. Unblocks durable auto-resume (transient crashes can now recover). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QvFotbQNtDNsgXJQZuzghR
…attempt) mem_profile-and-exit variants of llama8b_full32L_HSDP_b128_dp32 with source_dtype bfloat16. Bridge task: lp-bf16-source-ab. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…roughput) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCT1aKeNXbYcD8bg4LAJdi
…a hf_cache ~/.cache/huggingface/hub lost its Llama-3.1-8B snapshot at 14:36 today; the shared team cache has a complete world-readable copy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCT1aKeNXbYcD8bg4LAJdi
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCT1aKeNXbYcD8bg4LAJdi
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends #938's b160 cash-in up the batch ladder at full32L/dp32. Configs only — memprobe + 60-step throughput pairs for b192 (6 seq/GPU) and b224 (7 seq/GPU), both with `launch_env.env.HF_HUB_CACHE` pointed at the shared /mnt/data hf_cache (the home HF hub cache lost its Llama-3.1-8B snapshot on 2026-07-03).
Measured (4 nodes scavenge, steady-state steps 30–60, same code lineage):
+31% total over the b128 fp32 baseline. XLA absorbs batch growth into temp scheduling (peak ≈ flat b160→b192), so per-rung peaks don't extrapolate linearly — probe each rung. Ladder stopped at b224 on diminishing returns (~15 GiB headroom remains; b256 projected +2-3%).
Runs: p-e7bf57a7 / p-3d7bb9b8 (b192), p-f35651c5 / p-3aae8b60 (b224). Full writeup: lore 2026-07-03--bf16src-batch-ladder-b224-diminishing-returns. Bridge task: lower-precision-experiments.
🤖 Generated with Claude Code
https://claude.ai/code/session_01GCT1aKeNXbYcD8bg4LAJdi