Skip to content

[973.3] A weight-transposing matmul primitive; ops.transpose on packed data becomes a loud error #1096

Description

@michalharakal

Sub-issue of #973 · depends on #1094, #1095 · behaviour change, needs downstream coordination.

Task

ops.transpose on a block-quantized tensor is not a representable operation (#973, "the deeper semantic problem"): blocks quantize runs along the input dimension, so a true transpose needs requantization. What the engine calls a packed transpose is a layout conversion wearing transpose'''s name, and since #969 it is also an O(bytes) copy on every forward call (Linear.onForward does weight.t()).

  • Add the primitive the reference implementations actually have: matmulWT(x, W[out, in]) — ggml'''s mul_mat, BLAS'''s op(B).
  • ops.transpose on packed data becomes a loud error naming the primitive (or an explicit dequantization), after a deprecation cycle.

Acceptance

The packed hot path performs no per-forward copy; a benchmark before/after; the double-transpose hazard becomes impossible rather than untested.

Working rules (every slice)

  • One slice = one branch feature/<this-issue#>-<slug> from develop = one PR (Closes #<this issue>), merged independently.
  • develop stays usable after merge: additive API or behind façade / opt-in; no default behaviour switch without golden-parity evidence in the PR; deprecate with ReplaceWith, never delete before a major. BCV: apiDump, apiCheck clean.
  • Test gate before the PR: scripts/pr-gate.sh (and --golden for anything touching packed bytes).
  • Commits: conventional subject, Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> trailer. Do not edit CHANGELOG.md in slice PRs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    codingImplementation task (DARC: C)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions