Skip to content

Add NEON for av2_update_nbr_diagonal - #5410

Merged
urvangjoshi merged 2 commits into
AOMediaCodec:mainfrom
jjustiss-apple:jjustiss/neon-update-nbr-diagonal
Sep 23, 2026
Merged

urvangjoshi merged 2 commits into
AOMediaCodec:mainfrom
jjustiss-apple:jjustiss/neon-update-nbr-diagonal

Conversation

@jjustiss-apple

Copy link
Copy Markdown
Contributor

The av2_update_nbr_diagonal function updates context and magnitude arrays
along the diagonal scan path in trellis coded quantization. It accounts
for roughly 5% of ARM encode time at cpu-used=1.

This adds a NEON implementation that vectorizes across the 8 trellis
states using 64-bit vectors. Key techniques: vtbl1 for state-chain
table lookups, vrhadd for rounding halving adds, vsli for bit-field
packing, and an identity fast-path to skip redundant table lookups on
the common path.

Also deduplicates kTcqBaseMaxTbl from separate static definitions in the
C and AVX2 files into a single shared definition in trellis_quant.h.

Micro-benchmark results (Apple M-series, P-core):

Kernel C (ns) NEON (ns) Speedup
update_nbr_diagonal 4x4 130.7 9.2 14.2x
update_nbr_diagonal 8x8 144.5 10.6 13.6x
update_nbr_diagonal 16x16 204.4 12.7 16.0x
update_nbr_diagonal 32x32 340.5 20.0 17.0x

CTC Results (RA, cpu-used=1, 33 frames, ARM):

Metric Delta
Encode time -4.9%
BD-rate Y 0.000%
BD-rate Cb 0.000%
BD-rate Cr 0.000%

@jjustiss-apple

Copy link
Copy Markdown
Contributor Author

@yunqingwang1 I don't want to burden you with too many review requests. Whom else would you suggest that might be willing to help review ARM NEON PRs?

@yunqingwang1

Copy link
Copy Markdown
Contributor

@yunqingwang1 I don't want to burden you with too many review requests. Whom else would you suggest that might be willing to help review ARM NEON PRs?

@jjustiss-apple Please feel free to add @jianj-g as a reviewer. He is quite experienced with SIMD optimization.

Comment thread test/trellis_test.cc
Comment thread av2/encoder/arm/neon/trellis_quant_neon.c Outdated
Comment thread av2/encoder/arm/neon/trellis_quant_neon.c Outdated
@jjustiss-apple

Copy link
Copy Markdown
Contributor Author

Thank you @jianj-g for the careful review, I believe I've addressed everything.

@urvangjoshi

Copy link
Copy Markdown
Contributor

@jianj-g : any other comments?

@urvangjoshi

Copy link
Copy Markdown
Contributor

Pls rebase and force-push (don't create merge commit) @jjustiss-apple

@jjustiss-apple
jjustiss-apple force-pushed the jjustiss/neon-update-nbr-diagonal branch from fcefd29 to 534f15b Compare September 23, 2026 17:58
@urvangjoshi
urvangjoshi merged commit 01803db into AOMediaCodec:main Sep 23, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants