Skip to content

TU: fix stale legacy_source extensions, promoting Platform and Warp - #1884

Closed
andrewboudreau wants to merge 1 commit into
tu/inline-dtor-orderfrom
tu/legacy-source-extensions
Closed

TU: fix stale legacy_source extensions, promoting Platform and Warp#1884
andrewboudreau wants to merge 1 commit into
tu/inline-dtor-orderfrom
tu/legacy-source-extensions

Conversation

@andrewboudreau

Copy link
Copy Markdown
Collaborator

Stacked on #1882. Review that one first.

tu_promote refused ov002/Platform and ov002/Warp with what looked like a
delinks problem:

src/_ZN4WarpD0Ev.c has 0 entries in config/arm9/overlays/ov002/delinks.txt

The file is on disk as .cpp. The manifest recorded the extension each
function's legacy source had when the entry was written, and .c.cpp
migrations have landed on main since. 41 rows across 20 entries point at a
name that no longer exists. Nothing but tu_promote reads them, which is why
this went unnoticed — and why it reported as a missing delinks entry rather
than a stale manifest.

Corrected only where the sibling extension is actually on disk. Rows whose
file is simply gone (the entry was already promoted, so its legacy sources
were deleted) are left alone.

That unblocks two:

19 more hand-placed structor files retired.

module fidelity: 106/106 exact, 100.000000% of compared bytes
source-built functions: 11,065   reproducing: 11,065   mismatching: 0
ROM data: 378 verified, 206 partial, 54 differ   (unchanged)

layout_check clean · port_refcheck 397/397 · check_dead_references no new
· check_src_tu_compiles 88/88 · pytest tools/test_tubuild.py 36 passed ·
prepush_attribution 0 changed, 0 lost.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VregK5ZWRa2NbUcneaprG6

`tu_promote` refused ov002/Platform and ov002/Warp with

    src/_ZN4WarpD0Ev.c has 0 entries in config/arm9/overlays/ov002/delinks.txt

and the file it wanted is on disk as `.cpp`. The manifest recorded the
extension a function's legacy source had when the entry was written; `.c` to
`.cpp` migrations have landed on main since and nothing updated it. 41 rows
across 20 entries were pointing at a name that no longer exists.

Nothing but tu_promote reads those rows, which is why this sat unnoticed --
and why it read as "the delinks entry is missing" rather than "the manifest
is stale". Corrected only where the sibling extension is actually on disk;
rows whose file is simply gone (the entry was already promoted) are left
alone.

That unblocks two:

* **ov002/Platform**, 11 functions, already in ROM order -- it only needed
  the `_ZN7Vector3D1Ev` duplicate row.
* **ov002/Warp**, 8 functions, which needed the same inline-destructor change
  as the previous commit's twenty, and then matched in ROM order.

    module fidelity: 106/106 exact, 100.000000% of compared bytes
    ROM data: 378 verified, 206 partial, 54 differ -- unchanged

19 more hand-placed structor files retired, 19 attribution overrides added.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VregK5ZWRa2NbUcneaprG6
@tangos-validator

tangos-validator Bot commented Aug 27, 2026

Copy link
Copy Markdown

✅ PR validation — Passed

Committed merge introduces no reconstruction or attribution regression.

Full merge validation

Check Result
Committed test merge yes
Byte-verified functions 10,857 / 11,347 (95.68%, -17)
Byte-verified code bytes 2,027,372 / 2,211,124 (91.69%, -2,004)
Claimed, not byte-verified 342 functions, 74,008 bytes (+17)
Perfect source moves 0 R100
Enrolled ranges (delinks complete) 11,036 functions, 2,047,728 bytes (92.61%, +0) -- differs from byte-verified by +179
Contributor credit 0 added, 0 changed, 0 lost
Relocation check 8 checked; 8 VERIFIED
Port reference check 397 checked; 0 stale
Module fidelity 106/106 exact; 100.000000% compared bytes
Code linked from verified source 11,065 functions, 2,061,728 bytes (93.24%)
Module bytes from source 2,061,728 / 3,049,600 (67.6%); 811,492 (26.6%) are data no delink entry reaches
ROM data reproduced from source 378 symbol(s) exact, 206 partial, 54 differ

Byte-verified means the range carries complete in a delinks.txt, so the ROM build compiled it and compared it to the cartridge. The 342 claimed functions have a src/ file named after the symbol with no NONMATCHING banner, and nothing compiles them -- dsd fills their addresses with the ROM's own bytes. Both together are the 11,199 this project calls matched.

Warnings: 19 address range(s) left the byte-verified set while enrolled totals held steady: ov002:0x020ec388-0x020ec3b8, ov002:0x020ec3b8-0x020ec3fc, ov002:0x020ec3fc-0x020ec404, ov002:0x020ec404-0x020ec408, ov002:0x020ec408-0x020ec410, +14 more; 17 more function(s) now claim a match that nothing compiles; enroll them in a delinks.txt to have the ROM build check them.

Per-file link-check detail

All 1 changed file(s) compile to the ROM byte-for-byte with correct relocation targets.

File Symbol Result Slots checked
src/actors/Warp.cpp _ZN4WarpD1Ev + _ZN4WarpD0Ev + _ZN4Warp16CleanupResourcesEv + _ZN4Warp16OnPendingDestroyEv + _ZN4Warp6RenderEv + _ZN4Warp8BehaviorEv + _ZN4Warp13InitResourcesEv + Warp_Spawn ✅ verified 8

The private worker commits a test merge, builds the stock ROM profile, compares every executable module, measures matched and source-built code, checks contributor lineage, and verifies affected relocations. The mod profile is opt-in and is not part of this merge gate.

@andrewboudreau

Copy link
Copy Markdown
Collaborator Author

Triage: leaving this open. The working assumption was that #1993 (tu/restack-inline-dtors) supersedes this whole chain. It supersedes the legacy_source extension fixes but not the promotions.

#1993 re-derived the 38 stale legacy_source refs across 19 manifest entries from current main (commit a0d9abda0), so that half of this PR is genuinely redundant.

The two promotions are not:

  • src_tu/actors/Platform.cpp -> src/actors/Platform.cpp (config/tu_manifest.d/ov002/Platform.json)
  • src_tu/actors/Warp.cpp -> src/actors/Warp.cpp (config/tu_manifest.d/ov002/Warp.json)

Both are still under src_tu/ on origin/main @ 7869b817d with no src/actors/ counterpart, neither is in #1993's 26-TU promotion set, and no other open PR touches either. #1993's commit a0d9abda0 mentions Platform and Warp only as legacy_source extension repoints — the promotions themselves were dropped in the regenerate with no recorded rationale, unlike the four TUs #1993 documents as deliberately excluded (ov002/Tree, ov009/CastleWater, ov036/daObjRcCarpet_c, arm9/ActorDerived).

This branch is stale: it sits on top of #1882 -> #1880 -> tu/pathlift-promote, all of which predate the tools/ split landed as #1990. Do not rebase. After #1993 lands, re-cut just the Platform and Warp promotions from main with tools/tu_promote.py, gated on a per-symbol rombuild --data-json diff against main. Close this once that lands.

@andrewboudreau

Copy link
Copy Markdown
Collaborator Author

Gatekeeper review — keep open. This is NOT superseded by #1993.

I checked the coverage rather than assuming it. The #1880#1882#1884 chain promotes 66 TUs against #1993's 26, and five promotions are uncovered by anything else in the queue: ov029/ArrowLift and arm9/Scene (#1880), ov045/PoleLift (#1882), ov002/Platform and ov002/Warp (#1884). Closing the chain would silently drop those five, so it stays open.

What I need before this can merge, in order:

  1. Merge current origin/main (bfd908b32). Two things moved under you: Spell the TU-promotion metavariable as <Class>, unbreaking main's references gate #2010 fixed a dead-reference red caused by a src/actors/X.cpp metavariable left in prose, and Re-bank the two daObjAbuku_c paths #1996 absorbed, and check the ratchet on main #2011 re-banked the two daObjAbuku_c paths that TU: promote daObjAbuku_c as genuine C++ #1996 absorbed. If you resolved against an older main, redo it — do not carry a stale count forward.

  2. Resolve config/converted-baseline.json by REGENERATION, not key union. Union is correct for attribution.json and wrong here: score() returns None for an unreadable path, so a single banked path that a promotion absorbed and deleted exits 1 and turns the ratchet red on main.

  3. A correction to guidance I issued earlier, which may be steering you wrong right now. I told agents "the floor is 2565, never go below it." That was wrong. tiers_ratchet.py is a set ratchet: --check computes missing = banked - current and fails only when a path leaves the set. The printed count is informational — there is no floor to defend, and a promotion batch legitimately produces a much smaller count. The real gate is exactly two conditions:

    • every removed path is classified by the tool itself as MOVED -- absorbed into <file> by TU promotion (<tu_id>), i.e. zero genuine backslides; and
    • every named absorbing file exists on the branch and is tracked by git.
      If both hold, regenerate with --update --reason and push. If either fails, stop and report rather than working around it.
  4. Confirm no delinks range is orphaned. Every promoted entry from python tools/tu_manifest.py list must have an existing, tracked promoted_source and exactly one complete .text delinks entry naming it. This is the one failure mode the pipeline cannot self-detect: a .text range that stops being named complete, or names a path with no file there, is refilled by dsd with retail cartridge bytes — the ROM still byte-matches, module fidelity stays 106/106, romdata stays green, every gate passes. Only a drop in bytes-built-from-source reveals it. tools/layout_check.py is the dedicated gate.

Do not run rombuild.py or eligible.pybuild/ is shared across all worktrees and concurrent runs produce fake failures. Read the TU manifest only through python tools/tu_manifest.py, never config/tu_manifest.d/** directly.

If any premise above is wrong, say so on the PR with evidence rather than routing around it — I have been corrected three times today and each correction was right.

@andrewboudreau

Copy link
Copy Markdown
Collaborator Author

Status note, so nobody spends a validator slot chasing this. The red ratchet on this PR is a stale run, not a live regression:

FAILURE   converted ratchet   ratchet   2026-08-27T20:00

That is three days old. It predates #2007's re-bank, #2011, and today's merges (#1991, #1978, #2000, #1985). gh pr checks shows two checks both named ratchetlangmode ratchet (passing) and converted ratchet (this one) — so the bare name is ambiguous; read the workflow column.

This is almost certainly the base-desync shape rather than a real backslide: a PR cut before a re-bank merges against a baseline that predates it and inherits the red. The fix for that is to merge main in, never to lower the pin. I am not doing that yet, deliberately — see below.

Two corrections to what I told people earlier, both mine and both wrong:

  • The converted ratchet is a SET ratchet, not a count ratchet. --check computes missing = banked - current and fails only when a banked path leaves the set. The printed count is informational and there is no floor number to defend — a TU-promotion batch is expected to lower it. I circulated "the floor is 2565, never go below it" to several agents; it was wrong in both the number and the concept. Correct two stale claims in tiers_ratchet.py's docstring #2014 fixes the docstring that said otherwise.
  • The validator no longer restores tools/ from baseValidation now runs the pull request's own tools/ #1994 reversed that this morning. A PR now exercises its own tool change.

Why this is parked rather than refreshed

#1993 is the restack of this chain, and it is the queue chokepoint: it regenerates config/converted-baseline.json wholesale, so it must be the last thing to touch that file. Refreshing this PR now would (a) consume a slot on the serial validator that #1993 and the ov070 stack need, and (b) produce a green that #1993 immediately invalidates.

Order I am walking: #2002#2005#1995#1993, then re-assess what of this chain is still unlanded. Current signal is that the remainder is smaller than the diffstat suggests — a parallel review of #1914 found all 37 of its uniquely-promoted sources already present on origin/main under src_tu/, with the real harvest being a few dozen header inline-destructor flips. I want #1993 landed and measured before deciding whether this is a re-cut or a merge.

Nothing here is closed and nothing is being discarded. Coverage on main is holding at 2,066,772 B built from source, 0 bytes handed back to the cartridge, and I will re-check it after every merge.

@andrewboudreau

Copy link
Copy Markdown
Collaborator Author

Closing — see #1880 for the full reasoning. This sits on #1880/#1882 and inherits the rejected
plan_many mechanism, 411 commits of drift, and a converted ratchet failure that grows to
111 GONE entries once refreshed against main.

But the stale-legacy_source fix in here is still needed, and it is the one clean win in the
stack.
I re-derived it against current main: 38 rows across 20 manifest entries still point
at a .c path whose .cpp sibling is on disk

arm9/Actor.json (×7), arm9/ActorDerived.json, ov002/Platform.json (×2), ov002/Warp.json,
ov009/{CastleWater,Flag,MetalNet}, ov010/{LightBeam,PeachPainting}, ov014/ShutterBob,
ov019/{IceSlideManager,RacingPenguin}, ov020/HauntedChair,
ov029/{CageLift,RotatingPlatformWdw,SwitchActivatedPlank,WaterDiamond}, ov062/KoopaFlag,
ov077/Spiny.

Each one blocks tu_promote on that entry. The change is manifest-JSON only and touches no
tooling, so it is zero-risk and independently landable.

Please re-open it as a fresh PR cut from current main — do not cherry-pick 13bf9222f. That
commit's row set was computed against a 411-commit-old tree, so it will both miss rows main has
since added and rewrite rows main has since fixed. Re-derive from the current manifest instead.
I have the 38-row list above if it helps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant