Repository navigation
Investigate skip-packing for small contractions: execute_direct! (rejected) and execute_half_packed! (modest win) - #9
Merged
Merged
Conversation
BLIS-SUP-style investigation into whether QuasiStrided should skip packing for small contractions (motivated by dim63_2_1_2 running ~4.5x slower than StridedBLAS). execute_direct! is a plain scalar triple loop over the same AxisGroup-derived M/N/K offsets execute! uses, skipping packing and the register microkernel entirely. Additive only -- no changes to plan_contract, ContractPlan, _execute_nest!, kernel selection, or blocking; never wired into automatic dispatch, callable explicitly on any ContractPlan. Measured on real hardware (jobs 7109278 rome/AVX2, 7109279 icelake/AVX-512, canary-clean, see benchmark/bench_skip_packing.jl): this path NEVER wins. Best case is near-parity at 1x1x1; by 64^3 it is 25-40x slower on both ISAs, including on the small-K family that motivated the investigation. The packed path's own throughput (30-90 GFLOP/s from SIMD) dwarfs what packing costs even at the smallest sizes -- skipping packing while staying scalar throws away far more than it saves. Documented in the docstring with job IDs so the finding survives independently of this history; kept only as a third independent correctness oracle alongside execute_tilewise! (1,099 tests in test/execution/test_direct.jl, covering contiguous, tiny, K=0, alpha=0, beta=0/1, empty output, permuted/negative-stride/ sliced/scattered operands, role-swap, conjugation, and repeated calls on one plan). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Half-packing: B is packed exactly as execute! packs it, but A is read straight from its own storage (no copy) whenever the whole M extent is one full, unpadded SIMDKernel register tile with unit-stride rows in dense storage -- exactly the layout _pack_a_contiguous_eligible already recognizes for one sliver, generalized to the whole contraction. Mechanism: _accumulate_step's A address goes through a one-line indirection (_a_step_offset), dispatched on packed_a's type and resolved entirely at compile time (verified via matching LLVM/typed IR that this changes nothing for the existing packed paths). The new UnpackedAView type supplies the alternate addressing for the unpacked case; real element types only (atransform is always identity there), SIMDKernel only. Ineligible plans, and the empty/beta-only short-circuits, delegate to execute! unchanged, so this function is always correct, never wired into automatic dispatch, and callable on any plan without a separate eligibility check by the caller. Results are bitwise identical to execute!/execute_tilewise! in the eligible case (same _accumulate_step, same FMA order, different A address). test/execution/test_halfpack.jl (3,326 tests) covers the eligibility gate (real/complex, kernel, storage, transform, Qm, M-axis regularity), NaN-poisoning proof A is never packed, scattered/ negative-stride/offset B and C, non-ramp M, operand-role swap, out-of-bounds rejection, and zero-allocation/@inferred checks. Measured on real hardware, both AVX2 (rome/znver2) and AVX-512 (icelake-server), canary-clean throughout, across four rounds: - jobs 7110151/7110152: initial sweep (plain dense fixture, default kernel shape) -- a real but modest win, 1.01-1.08x typical, peaking ~1.05x around K=N in the 16-256 range, tapering to parity at K,N >= ~1024. One regression pocket: AVX2 is 6-20% SLOWER at the tiniest K/N (<8), where eligibility/setup overhead outweighs the tiny packing avoided. - jobs 7110225/7110226: re-measured after rebasing onto main's new analytical blocking model and FMAddSubKernel, since this path reads plan.blocking -- verdict unchanged. - jobs 7111552/7111553: extended to a non-ramp K composite (the realistic multi-label tensor-contraction case, where the in-place A read takes an indexed PtrScatterAxis path rather than an affine stride) and every non-default register shape -- same band, same regression pocket, confirming the verdict isn't an artifact of the simplest fixture measured first. Net: kept as opt-in exactly as shipped; auto-wiring into plan_contract's dispatch would need an additional lower K/N gate to avoid the AVX2 regression pocket, not attempted here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…is PR) main has been red since the fmaddsub merge (commit 5242652), which this branch inherited via rebase: test/microkernels/test_fmaddsub_kernel.jl does `using InteractiveUtils: code_native`, but InteractiveUtils was never added to Project.toml's [extras]/[targets] test list, so CI's isolated test environment can't find it (LoadError: ArgumentError: Package InteractiveUtils not found in current path). Separately, benchmark/probes/fmaddsub_codegen.jl fails the repo's `runic` format check (unwrapped generator/println/@printf calls at line width). Both fixed here rather than left for the fmaddsub branch to catch up on later, since they block CI on every subsequent PR against main, including this one. `runic --check .` is clean and the full suite still passes (136,898/136,898) with InteractiveUtils now resolvable. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
lkdvos
force-pushed
the
skip-packing
branch
from
September 26, 2026 13:31
f9f791d to
e4387e4
Compare
- test/execution/test_direct.jl: the "M and N below one register tile" testset hardcoded shapes (e.g. Ma=5) assuming an AVX2/AVX-512 planar tile size. On GitHub's hosted runners, kernel selection lands on a different kernel/shape at these extents (a smaller fitted planar shape on some runners, main's small-M FMAddSub routing on AVX-512 ones) -- predicting which kernel that is would duplicate kernel_selection.jl's own logic, so instead: build the actual plan for each candidate shape and use it as a filter (only candidates that land below one tile, on whatever kernel this host actually picks, are exercised), with a non-vacuousness check that at least one candidate qualified. Holds on any ISA/kernel-selection version by construction. - test/microkernels/test_fmaddsub_kernel.jl: the instruction-selection testset asserts an exact instruction mix (vfmaddsub count, no spill, etc.) in FMAddSubKernel's compiled hot loop -- a real, tuned hardware measurement (not mine), but not reproducible on generic hosted CI runners, whose exposed CPU features under virtualization don't reliably match. Gated `has_fma` on `CI != true` (GitHub Actions sets this) in addition to the existing ISA check, so it degrades to Broken/skipped in CI rather than failing, and still runs for real everywhere else (verified on this AVX-512 dev machine: all 15 assertions execute and pass without CI=true set). Both are unrelated to this PR's own execute_direct!/execute_half_packed! logic, surfaced only because this is the first PR to run CI since the fmaddsub merge. Full suite: 136,924/136,924, runic clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Investigation into whether QuasiStrided should skip packing for small contractions (motivated by
dim63_2_1_2running ~4.5x slower than StridedBLAS, and BLIS's "SUP" skinny-unpacked technique). Two opt-in, additive paths, both measured on real hardware (AVX2 rome/znver2 and AVX-512 icelake-server, canary-clean throughout):execute_direct!(src/execution/direct.jl): a fully unpacked scalar triple loop. Rejected — never wins, 25-40x slower by 64³ on both ISAs, because it throws away SIMD entirely. Kept only as a third independent correctness oracle alongsideexecute_tilewise!.execute_half_packed!(src/execution/halfpack.jl): B packed as usual, A read straight from its own storage (no copy) whenever the whole M extent is exactly one fullSIMDKernelregister tile. A real, modest win: 1.01-1.08x typical, peaking ~1.05x around K=N in the 16-256 range. Measured across four rounds (initial sweep, re-measured after rebasing onto the new analytical blocking model, extended to a non-ramp K composite and every non-default register shape) — the verdict holds throughout. One known regression pocket: 6-20% slower on AVX2 at the tiniest K/N (<8).Both are structurally additive: zero changes to
plan_contract,ContractPlan,_execute_nest!, kernel selection, or blocking, and neither is wired into automatic dispatch — both are explicit opt-in calls on an existingContractPlan.Test plan
execute_half_packed!results are bitwise identical toexecute!/execute_tilewise!in the eligible case🤖 Generated with Claude Code