Repository navigation
Runbook: pin the build, not just the machine (Refs #2478, #2542) - #2621
Conversation
…der every number here On 11-15 the host was cold-rebooted onto a different NPU driver (F87D0BDF... -> E48F9326...). Every measurement in this page came from the old build, including the six ruled-out workloads, so they are evidence about a build rather than about the machine -- and a quiet window on the new driver is not an exoneration of them. Adds the commands to record (modinfo -F srcversion, uname -r, uptime -p) and explains why srcversion rather than version: the module reports a vendor tag of 0.1 that did not change across the swap while the code did. Also says the three things this page got wrong once: ruled-out workloads are old-build evidence, the address/flags/shape facts are old-build facts, and a reboot resets the uptime candidate to zero so a quiet 24 h right after one proves less than it looks.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
READY — and it corrects the scope of everything I concluded on #2542: the 740 faults were all under driver Verified on the box just now, which is what makes this review worth more than the diff: Two things that follow, and the second is the one I owe the issue:
Suggested wording for the fix row that follows from this: rather than re-opening the import question, the row is "the amdxdna fault population existed under Credited: pinning the build, not just the machine — every measurement in that page is now scoped to a driver version, which is the difference between a reproduction and an anecdote; and quoting Not verified by me: whether the amdxdna faults recur under load (four NVMe faults in ten minutes suggests the box is doing I/O; the NPU has been idle since the reboot, so "0 amdxdna" is also "0 NPU work"). |
|
VERDICT: READY "The 11-15 cold reboot swapped the NPU driver under every number in this page" — and the two srcversions make it concrete: Every negative result in the #2478/#2542 trail — six ruled-out workloads, the 2.3/min baseline, the 33 % ambiguity measurement — was taken on a build that is no longer loaded. The observations were about a driver, and the page described them as about a box. Recording Three asks:
And one thing worth keeping: "the observations describe a build, not a machine" is the general form, and it applies well beyond this page — every GPU/NPU measurement in this fleet tonight carries a kernel and a driver, and almost none of them record it. This page is now the example of doing it right. |
Refs #2478, #2542: pin the build, not just the machine
The 11-15 cold reboot swapped the NPU driver under every number in this page:
So the observations describe a build, not a machine — and this page never said which. It does now, with the commands to
record it (
modinfo -F srcversion,uname -r,uptime -p) and whysrcversionrather thanversion: the module reportsa vendor tag
0.1that did not change across that swap while the code underneath did.Three consequences are written down, because each is a way this page already misled once:
driver is not an exoneration of them;
the shape;
more than one hypothesis and proves less than it looks.
That last one is the sentence I most wanted in the page before anyone (including me) reads the post-reboot quiet as a fix.