Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions npu/forward/include/q4nx_head_image.h
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,15 @@
// measured at 10-17 s quiet and minutes contended for the h4096 shapes. Nothing device-side enters that image, so it can be
// produced on the host, written once at import and copied on later serves.
//
// What makes that safe is the identity, not the speed. The image is written with a format version, its geometry, and a hash
// over the payload; `read` recomputes the hash and refuses rather than returning bytes it cannot vouch for. A packer change,
// a geometry change or a truncated write are therefore a loud refusal and never a plausible-looking image -- the same rule the
// .fconst canonical copy follows after engine#1662/#1669.
// What makes that safe is INTEGRITY AND GEOMETRY, not provenance (engine#1802 audit). The image is written with a format
// version, its geometry, and a hash over the payload; `read` recomputes the hash and refuses rather than returning bytes it
// cannot vouch for. So a truncated write, a corrupted byte, or a geometry change is a loud refusal and never a
// plausible-looking image -- the same rule the .fconst canonical copy follows after engine#1662/#1669.
//
// What the hash does NOT prove: that the image was built from THESE weights. An image from another pack with the same kd/nd
// hashes faithfully to its own bytes and passes every check here, so a stale image left beside a different container of the
// same head geometry would be accepted. Nothing calls this API yet; whoever first does must key the path on the container's
// identity (or add a source hash beside the payload hash) before a stale file can be read as this pack's head.
//
// Absence is not an error: a container without an image simply packs as it did before.
#ifndef Q4NX_HEAD_IMAGE_H
Expand Down
20 changes: 18 additions & 2 deletions tools/npu_elf_ref.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,15 @@ def prng_bits(n, seed):
def quantise(weights, axis):
"""int8 codes + per-(check) scale, the way a container holds them: absmax/127 with a 1.0 floor on an all-zero slice.

axis=1 -> a per-COLUMN scale (B, the weight matrix: one scale per output column, floored per 32-column group upstream)
axis=0 -> a per-ROW scale (A, the activations: one scale per token)
axis=0 -> a per-COLUMN scale (B, the weight matrix: `amax` is taken along K, one value per output column)
axis=1 -> a per-ROW scale (A, the activations: one value per token)

Those two lines were swapped until #1805's audit: `axis=1` reduces ALONG axis 1 and broadcasts as `(-1, 1)`, i.e. one scale per ROW, and
`run_case()` agrees with the code (`a_w, axis=1` = per token, `b_w, axis=0` = per column). The self-check now asserts the convention so the
docstring and the code cannot drift again.

`codes = rint(weights / scale)` rounds half to even (numpy's default). The packer's mode is not stated here; if it differs on exact .5
boundaries the codes differ there and nowhere else, which is the edge the saturating case is built to reach.
"""
amax = np.max(np.abs(weights), axis=axis).astype(np.float64)
amax[amax < 1e-12] = 1.0 # the container's floor: an all-zero slice quantises to zeros at scale 1
Expand Down Expand Up @@ -167,6 +174,15 @@ def main():
f"rel={rel:.1e} acc32==acc64={p['int32_matches_int64']} zero_cols={p['zero_b_cols']} "
f"floored={p['floored_cols']} maxcode={p['max_abs_code']}")
bad += 0 if ok else 1
# engine#1805 audit: the docstring's axes were swapped for as long as this tool existed, so assert the convention too -- a per-column
# scale comes from reducing along axis 0, and a 3x4 must give 4 column scales and 3 row scales. Re-swapping either line fails here.
_w = np.ones((3, 4))
_col, _col_s = quantise(_w, axis=0)
_row, _row_s = quantise(_w, axis=1)
_conv = _col_s.shape == (4,) and _row_s.shape == (3,)
print(f" {'ok ' if _conv else 'FAIL'} axis convention: axis=0 -> {_col_s.shape[0]} column scales, "
f"axis=1 -> {_row_s.shape[0]} row scales (3x4 matrix)")
bad += 0 if _conv else 1
print("npu_elf_ref self-check: " + ("ok" if not bad else f"{bad} FAILED"))
return 0 if not bad else 1

Expand Down
Loading