feat(adversary): optional global-norm grad clip on PPGD source ascent + src-grad-norm metric#937
Open
ocg-goodfire wants to merge 2 commits into
Open
feat(adversary): optional global-norm grad clip on PPGD source ascent + src-grad-norm metric#937ocg-goodfire wants to merge 2 commits into
ocg-goodfire wants to merge 2 commits into
Conversation
… (SRC_STEP) AdamPGDConfig.grad_clip_norm (default None = oracle parity, S13/N1 unchanged): one global norm over the whole per-adversary source bundle, same clip semantics as the main optimizers' clip_by_global_norm_with_eps, applied before the Adam moments in sources_adam_ascend_project — covers warmup and final ascents alike. Logs the pre-clip final-ascend source-grad norm (grad_norms/summary/src/<key>, unscaled by the term coeff) so thresholds can be sized off real runs. SPEC §6 SRC_STEP row amended with the config-gated variation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… 0.01) Clip thresholds sized off the measured grad_norms/summary/src scale (0.027-0.054 over steps 200-1000 of the first baseline attempt p-5d480c2d, killed at step 1000 by the pre-#919 hidden-acts-eval bug). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
AdamPGDConfig.grad_clip_norm: PositiveFloat | None = None— global-norm clip over the whole per-adversary source bundle (all sites jointly, samemin(max_norm/(norm+eps), 1)semantics as the main optimizers'clip_by_global_norm_with_eps), applied to the source grad before the Adam moments inadversary.py::sources_adam_ascend_project. Covers both warmup ascents and the final ascent (S13'/S14').grad_norms/summary/src/<state_key>: the pre-clip final-ascend source-grad global norm, unscaled by the term's coeff — exactly the gradient SRC_STEP sees, for sizing clip thresholds and adversary-health visibility.SRC_STEProw amended with the config-gated variation (default off = oracle parity, N1/S13 unchanged).param_decomp/configs/adv_gradclip/(pile 4L ppgd bsc, 40k, dp=8): base / clip 0.03 / clip 0.01.Related Issue
Bridge task
try-grad-clip-adv-recon.Motivation and Context
Oli's ask: try grad clipping in the adversarial recon term. A/B result (lore
2026-07-03--ppgd-source-grad-clip-ab-no-effect-4l): no effect at 4L pile scale — the source grad is already smooth (whole-bundle norm 0.01–0.07, spike-free through step 35k), so even a continuously-binding clip@0.01 leaves every metric within noise of base (runs p-4515d4dd / p-cddac242 / p-a3386426, wandb groupadv-gradclip). Landing the knob anyway: zero cost when off (bit-identical default), and it's the ready lever if big-run adversary spikes recur; the norm metric is independently useful visibility.How Has This Been Tested?
param_decomp/tests/at default AND 4 sim devices (SLURM jobs 167167 pre-rebase, 170708 post-rebase onto feature/jax): 230 passed ×2, equivalence goldens pass UNMODIFIED (default-None path bit-identical).make typegreen).Does this PR introduce a breaking change?
No. Default
grad_clip_norm: Nonepreserves existing trajectories bit-for-bit; existing configs unaffected. One new metric key family in train logs.🤖 Generated with Claude Code