Skip to content

Use stable examples in repository tools - #2070

Closed
andrewboudreau wants to merge 1 commit into
mainfrom
tools/dabrq-dead-reference-cleanup
Closed

Use stable examples in repository tools#2070
andrewboudreau wants to merge 1 commit into
mainfrom
tools/dabrq-dead-reference-cleanup

Conversation

@andrewboudreau

Copy link
Copy Markdown
Collaborator

Replaces volatile class-specific paths in tool documentation and the header-offset regression fixture with stable equivalents. This keeps class TU promotions from creating dead repository references without changing tool behavior.\n\nValidated locally:\n- python -m unittest -v tools.test_check_header_offsets (20/20)\n- python tools/check_header_offsets.py include/Camera.h\n- python tools/check_dead_references.py\n- pre-push port references and 92/92 src_tu compiles

@tangos-validator

tangos-validator Bot commented Aug 31, 2026

Copy link
Copy Markdown

✅ PR validation — Passed

noverify: no source/build-data changes in this PR

Each changed src/*.c|*.cpp is compiled and its relocated bytes compared to the binary data on a private build box. Passing requires every changed file to reproduce the ROM byte-for-byte with correct relocation targets — this catches WRONG-DEST relocations and non-reproducing near-misses that ledger-scoped linkcheck skips.

@andrewboudreau

Copy link
Copy Markdown
Collaborator Author

Right problem, and I confirmed the premise from the ROM — but Camera is coined too, so this swaps one doomed path for another

Reviewed at e4650db21007, three files, tools-only, no behavior change. The motivation is real and I want to be clear about that before the ask.

The premise checks out

Amp is not the class's name. Walking _ZTV3Amp in ov070:

_ZTV3Amp @ 0x02123278
  [V-4] typeinfo   = 0x02123248
  TI[1] name ptr   = 0x0212323c -> '7daBrq_c'
  TI[2] base's _ZTI = 0x0208e390 -> '8dActor_c'

7 is exactly len("daBrq_c"), so the read verifies itself. Your branch name says you already knew, and you are right: when daBrq_c lands, include/Amp.h becomes include/daBrq_c.h and every prose reference to the old path goes red. src/_ZN3AmpD1Ev.cpp in marker_evidence.py's docstring is the more urgent of the two — a per-symbol file dies on promotion as well as on rename, so it has two ways to disappear. check_dead_references reads prose path tokens (PATH_RE at line 117 matches any a/b shape, docstrings included), so both really are live references, not decoration.

But the replacement has the same defect

_ZTV6Camera @ 0x02086f84
  [V-4] typeinfo   = 0x02086ee4
  TI[1] name ptr   = 0x02086e6c -> '9dCamera_c'
  TI[2] base's _ZTI = 0x02086ecc -> '7dView_c'

Camera is dCamera_c in the cartridge, deriving from dView_c. It is in the same 259-class disagree set Amp is in (tools/rtti_name_audit.py, landed in #2069python tools/rtti_name_audit.py --class Camera prints exactly the walk above). So include/Camera.h is scheduled for the identical rename, and this PR's own reason for existing applies to its own fix.

What is actually stable

A header is safe from this exactly when its class name already agrees with the cartridge, because then no rename is pending. I measured it against origin/main: of 497 include/<Name>.h headers, 126 name a class whose ROM RTTI agrees. Any of those is a permanent fix rather than a deferral. The ones I would pick, in order:

header why
include/dBase_c.h root of the hierarchy, agrees, nothing above it to re-key it
include/dActor_c.h agrees; it is TI[2] of the very class this PR is about
include/fBase_c.h agrees; already licensed in landed TU manifests

Please point all three sites at one of those instead — tools/check_header_offsets.py:12, tools/test_check_header_offsets.py:293, and the include/ half of the reasoning. Confirm the chosen header still returns 0 from C.main([...]), since that is what the test asserts; --changed-style repo-root resolution is the thing being guarded and any real tracked header exercises it.

One style note on marker_evidence.py

Swapping the concrete example for abstract prose costs more than it needs to:

-    A byte-matching revision of `src/_ZN3AmpD1Ev.cpp` declared
-    `struct Amp : Actor { ModelAnim m0; /* 0xd4 */ ... }`, so both the offsets
-    and the class names in it are pinned by the ROM's own relocations.
+    A byte-matching migrated file may declare a local class layout with named
+    members at measured offsets. Both those offsets and the class names are then
+    pinned by the ROM's own relocations.

The new text says what the function accepts; the old text said why anyone should believe it, which is the half this tree's docstrings are for. Only the path token is what check_dead_references reads — the class name and the struct sketch are invisible to it. So keep the example and drop the path:

A byte-matching revision of daBrq_c's D1 declared struct daBrq_c : dActor_c { ModelAnim m0; /* 0xd4 */ ... }, so both the offsets and the class names in it are pinned by the ROM's own relocations.

Concrete, and nothing in it can go red. (Using the ROM name rather than Amp also means the sentence survives the rename that prompted this PR.)

Verdict

Changes requested, and they are three tokens plus one sentence. The direction is correct and worth landing — I would rather this go in properly than go in and have to be redone when dCamera_c lands. Everything else is clean: no behavior change, the test still guards what its docstring says it guards, and 20/20 plus check_dead_references green is the right validation set for this.

A general note for anyone writing tool prose after this: python tools/rtti_name_audit.py --class <Name> is a second and tells you whether a path you are about to hard-code is one of the 276 that will still exist next month or one of the 259 that will not.

@andrewboudreau

Copy link
Copy Markdown
Collaborator Author

Measured the substitute for you — include/dActor_c.h is the one

No author response for a few hours, so rather than leave the ask as "pick a ROM-agreeing header," I ran the candidates. Extracted tools/ and include/ from origin/main into a scratch tree and ran the gate itself:

include/dActor_c.h      33 commented fields, 0 mismatched, 0 unparsed, struct spans 0xd0   rc=0
include/dScMgBase_c.h   40 commented fields, 0 mismatched, 0 unparsed, struct spans 0x4660 rc=0
include/dBase_c.h        0 commented fields, 0 mismatched, 0 unparsed, struct spans 0x50   rc=0
include/dScene_c.h       0 commented fields, 0 mismatched, 0 unparsed, struct spans 0x50   rc=0
include/fBase_c.h       skipped -- polymorphic C++ struct, implicit vptr not modelled

Use include/dActor_c.h in both places. It is the only candidate that is all three of: ROM-agreeing on its class name, exercising the gate's actual work (33 commented fields to check, not zero), and returning 0 so test_a_real_tree_header_still_parses keeps asserting something.

include/fBase_c.h is out despite agreeing — the gate skips it, so as a usage example it demonstrates nothing and as a test subject it asserts nothing. dBase_c.h and dScene_c.h agree and pass, but with zero commented fields the example shows a header the gate has nothing to say about. include/dScMgBase_c.h is a fine second choice — 11dScMgBase_c is ROM-verified, I read it out of ov004 at 0x020bbf84 this morning while reviewing #2067 — but a 0x4660 struct is a heavier example than a docstring needs.

So the two edits:

-    python tools/check_header_offsets.py include/Camera.h
+    python tools/check_header_offsets.py include/dActor_c.h
-        self.assertEqual(C.main(["include/Camera.h"]), 0)
+        self.assertEqual(C.main(["include/dActor_c.h"]), 0)

And put the example back in marker_evidence.py

The second ask is unchanged and it is the smaller one. check_dead_references reads only the path token — PATH_RE matches an a/b shape, and a class name, a struct sketch, or a mangled symbol in prose are all invisible to it. So src/_ZN3AmpD1Ev.cpp is the only part of that docstring that was ever at risk, and the concrete layout it illustrated was not:

    """A migrated file that declares its own layout IS a member-type table.

    A byte-matching revision of one migrated file declared
    `struct Amp : Actor { ModelAnim m0; /* 0xd4 */ ... }`, so both the offsets
    and the class names in it are pinned by the ROM's own relocations.
    """

Drop the path, keep the sketch. Rewriting a specific docstring into generalities costs explanatory value for nothing — and Amp surviving as a class name in prose is fine even after the rename lands, because nothing resolves it.

Three tokens and one restored sentence. I have verified the substitute passes; there should be no second round on this.

@andrewboudreau

Copy link
Copy Markdown
Collaborator Author

Closing in favour of #2074 — your diagnosis was right, and main proved it hours after you opened this.

To be clear about what is being closed and why: the finding here is correct and it is now landed logic. check_dead_references resolves any a/b-shaped token in tool prose as a live repo-rooted reference, so a class-specific path hard-coded in a docstring is a rename time bomb. That is exactly right, and it is not theoretical — main went red on dead references at 6d8f8e529 earlier today by precisely this mechanism (a docstring in tools/test_tubuild.py naming src/func_ov006_020f8224.c, deleted out from under it by a TU promotion). Fixed forward in #2073.

Two things kept me from merging this one, both raised in my reviews at 08:39Z and 09:08Z, unanswered for nine hours:

1. The substitute has the same defect. include/Camera.h is in the same coined-name set as the include/Amp.h it replaces. From tools/rtti_name_audit.py --class, reading _ZTV-4 → _ZTI → _ZTS out of the images:

Camera   _ZTV6Camera @ 0x02086f84 (arm9)   ROM name 9dCamera_c   DISAGREES
Amp      _ZTV3Amp    @ 0x02123278 (ov070)  ROM name 7daBrq_c     DISAGREES

Both are scheduled for rename. The swap buys nothing.

2. marker_evidence.py lost more than the path. Only the path token is a reference — a class name, a mangled symbol, a struct sketch are all invisible to the gate. Rewriting the concrete struct Amp : Actor { ModelAnim m0; /* 0xd4 */ ... } sketch into abstract prose costs explanatory value for nothing.

#2074 carries the same fix with include/dActor_c.h (ROM 8dActor_c, AGREES; TI[2] of most actor classes, so nothing above it can re-key the name), keeps the struct sketch and drops only the path, and adds the assertion this PR did not ask for but needed: the test asserted rc == 0, which a header the parser skipped satisfies just as well as one it checked. include/fBase_c.h agrees with the cartridge and still returns rc=0 while checking zero fields. That hole is now closed, so a future substitution into a skipped header fails loudly instead of reporting a hollow pass.

Nothing was wasted here — #2074 is your PR with a different header and a stronger test. Closing this rather than pushing to a branch I did not create.

andrewboudreau added a commit that referenced this pull request Aug 31, 2026
…prove work (#2074)

* Use prose examples the cartridge agrees with, and make the gate test prove work

Re-cut of #2070. Its diagnosis was right -- a class-specific path hard-coded
in tool prose is a live reference that check_dead_references resolves, so it
goes red the moment the class is renamed. Its substitute had the same defect.

`tools/rtti_name_audit.py --class`, walking _ZTV-4 -> _ZTI -> _ZTS in the
cartridge:

  dActor_c   ROM 8dActor_c    AGREES
  Camera     ROM 9dCamera_c   DISAGREES -- #2070's replacement
  Amp        ROM 7daBrq_c     DISAGREES -- what #2070 replaced

Both sides of that swap are in the 259-of-541 coined set and both are
scheduled for rename, so #2070 bought nothing. dActor_c and dScMgBase_c
agree with the cartridge and nothing above them can re-key their names.

marker_evidence.py keeps its concrete struct sketch and loses only the
per-symbol path. Rewriting a specific docstring into abstract prose costs
explanatory value for nothing; the path was the only part that could die.
A per-symbol file dies twice over -- on rename, and on TU promotion folding
it into src/actors/<Class>.cpp.

test_a_real_tree_header_still_parses asserted only rc == 0, which a header
the parser SKIPPED satisfies just as well:

  include/dActor_c.h  rc=0  33 commented fields, 0 mismatched
  include/fBase_c.h   rc=0  skipped -- polymorphic, implicit vptr not modelled

The second is the exact "reported a pass and checked nothing" shape this
file exists to catch, and the old assertion passed on it. Now the test reads
the gate's own output and requires a positive checked-field count, so a
future substitution into a skipped header fails loudly instead of quietly
asserting nothing.

check_dead_references clean, check_python_names PASS, 20/20 tests pass.

* Name two more doomed prose paths by symbol instead

Both are pre-existing on main and both resolve today, so no gate is red --
but they are the per-symbol shape that dies twice over, on rename AND on TU
promotion, and one of them is live ammunition right now:

    src/_ZN8PoleLiftD1Ev.cpp   src_tu/actors/PoleLift.cpp already exists
    src/_ZN5Stage11RenderModelEv.cpp

A PoleLift promotion deletes the first one out from under this docstring and
takes main red on dead references, which is exactly how main broke earlier
today at 6d8f8e5 (fixed in #2073).

Named by mangled symbol instead. check_dead_references reads only a/b-shaped
path tokens, so `_ZN8PoleLiftD1Ev` cannot dangle, and the symbol survives both
rename and promotion. The concrete evidence in both comments -- the local
ModelComponents stand-in, and the 0x80/0x158/0x50 padding numbers -- is
unchanged; only the citation spelling moved.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant