Skip to content

Fast two-pass: additional dry-pass simplifications - #5435

Merged
leolzhao merged 1 commit into
AOMediaCodec:av2-encfrom
yeqing-wu:sb_tpp_simplify_dry_pass_submission
Sep 25, 2026
Merged

leolzhao merged 1 commit into
AOMediaCodec:av2-encfrom
yeqing-wu:sb_tpp_simplify_dry_pass_submission

Conversation

@yeqing-wu

Copy link
Copy Markdown
Contributor

Three simplifications to the dry pass of fast two-pass partitions.

  1. Skip deblock-aware distortion. The dry pass ranks shapes only, so reconstructing and filtering to nudge distortion by the deblocking delta is wasted precision.

  2. Reuse the dry pass's simple motion searches in the wet pass. MVs, sse/var, rate/dist/rdcost and start MVs depend only on source, references and geometry, so recomputing them is redundant. A new preserve_sms_cache flag drives av2_reset_sms_cross_pass_state(), which clears only what must not cross the pass boundary.

  3. Cap two transform-side inner loops in search_tx_type, select_tx_partition_type and choose_tx_size_type_from_rd on x->apply_dry_pass_shortcuts: force the (set_idx=0, stx=0) IST/stx baseline, and restrict the tx-partition search to NONE and SPLIT (2 of 8 variants).

STATS_CHANGED

Anchor: commit c5df831
Speed 4 (cpu-used=4): FG16 CTC (33 frames, class A1 and A2, RA)
Speed 5 (cpu-used=5): FG16 CTC (33 frames, class A1 and A2, RA)

1) Speed 4 (cpu-used=4)
  +------------+------+-------+-------+------+------+------+
  | Class      |    Y |    Cb |    Cr | wAvg | Enc% | Dec% |
  +------------+------+-------+-------+------+------+------+
  | A1         | 0.27 | -0.23 | -0.21 | 0.23 |   95 |  100 |
  | A2         | 0.31 | -0.42 | -0.46 | 0.25 |   95 |  100 |
  | Avg w/o B2 | 0.30 | -0.36 | -0.39 | 0.25 |   95 |  100 |
  +------------+------+-------+-------+------+------+------+

2) Speed 5 (cpu-used=5)
  +------------+------+-------+-------+------+------+------+
  | Class      |    Y |    Cb |    Cr | wAvg | Enc% | Dec% |
  +------------+------+-------+-------+------+------+------+
  | A1         | 0.39 |  0.01 |  0.13 | 0.37 |   94 |  100 |
  | A2         | 0.26 | -0.39 | -0.57 | 0.20 |   94 |  100 |
  | Avg w/o B2 | 0.30 | -0.27 | -0.37 | 0.25 |   94 |  100 |
  +------------+------+-------+-------+------+------+------+

@leolzhao

Copy link
Copy Markdown
Contributor

What's the run-time savings for cpu-used = 6? @yeqing-wu

@yeqing-wu

Copy link
Copy Markdown
Contributor Author

What's the run-time savings for cpu-used = 6? @yeqing-wu

I have not tested that since I saw both speed 4 and 5 are within the allowed ratio.

@yunqingwang1
yunqingwang1 enabled auto-merge (rebase) September 24, 2026 18:33
Three simplifications to the dry pass of fast two-pass partitions.

1. Skip deblock-aware distortion. The dry pass ranks shapes only, so
   reconstructing and filtering to nudge distortion by the deblocking
   delta is wasted precision.

2. Reuse the dry pass's simple motion searches in the wet pass. MVs,
   sse/var, rate/dist/rdcost and start MVs depend only on source,
   references and geometry, so recomputing them is redundant. A new
   preserve_sms_cache flag drives av2_reset_sms_cross_pass_state(),
   which clears only what must not cross the pass boundary.

3. Cap two transform-side inner loops in search_tx_type,
   select_tx_partition_type and choose_tx_size_type_from_rd on
   x->apply_dry_pass_shortcuts: force the (set_idx=0, stx=0) IST/stx
   baseline, and restrict the tx-partition search to NONE and SPLIT
   (2 of 8 variants).

STATS_CHANGED

Anchor: commit c5df831
Speed 4 (cpu-used=4): FG16 CTC (33 frames, class A1 and A2, RA)
Speed 5 (cpu-used=5): FG16 CTC (33 frames, class A1 and A2, RA)

```
1) Speed 4 (cpu-used=4)
  +------------+------+-------+-------+------+------+------+
  | Class      |    Y |    Cb |    Cr | wAvg | Enc% | Dec% |
  +------------+------+-------+-------+------+------+------+
  | A1         | 0.27 | -0.23 | -0.21 | 0.23 |   95 |  100 |
  | A2         | 0.31 | -0.42 | -0.46 | 0.25 |   95 |  100 |
  | Avg w/o B2 | 0.30 | -0.36 | -0.39 | 0.25 |   95 |  100 |
  +------------+------+-------+-------+------+------+------+

2) Speed 5 (cpu-used=5)
  +------------+------+-------+-------+------+------+------+
  | Class      |    Y |    Cb |    Cr | wAvg | Enc% | Dec% |
  +------------+------+-------+-------+------+------+------+
  | A1         | 0.39 |  0.01 |  0.13 | 0.37 |   94 |  100 |
  | A2         | 0.26 | -0.39 | -0.57 | 0.20 |   94 |  100 |
  | Avg w/o B2 | 0.30 | -0.27 | -0.37 | 0.25 |   94 |  100 |
  +------------+------+-------+-------+------+------+------+
```
@yeqing-wu
yeqing-wu force-pushed the sb_tpp_simplify_dry_pass_submission branch from 5c0ad72 to 3777003 Compare September 25, 2026 16:40
@leolzhao
leolzhao enabled auto-merge (squash) September 25, 2026 16:41
@leolzhao
leolzhao merged commit e6f0837 into AOMediaCodec:av2-enc Sep 25, 2026
107 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants