Skip to content

Introduction to the code structures

Jump to bottom
Nuo Mi edited this page Feb 12, 2023 · 5 revisions

vvcdec.c

Plugin for ffmpeg. It accepts input packet, output yuv frame. We put each input packet (bitstream for frame) into a frame queue tail. The queue is a circular buffer. When it is full, it will wait until a free slot becomes available.

vvc_thread.c

Decoder split frame to multiple CTUs. Each CTU has 8 stages, each stage has a different task type. ff_vvc_task_ready will handle the dependencies between task types.

  1. VVC_TASK_TYPE_PARSE (P): This stage parses the bitstream at CTU level. A frame may have several parser points. Each point is independent. We can parse them simultaneously. We use EntryPoint to represent parse points. Each slice has at least one Entrypoint. Depending on the tile structure and entropy_coding_sync_enabled_flag, we may have multiple EntryPoints in a slice. This stage may be blocked in temporal_luma_motion_vector when we select the temporal candidates.
  2. VVC_TASK_TYPE_INTER (I): This stage predicts CTU from inter. We have no intra dependency for this stage. Once the inter dependency is met, any CTU in this stage will be decoded at the same time. This stage may be blocked if inter pixels are not ready
  3. VVC_TASK_TYPE_RECON (R): This stage predicts CTU from Intra and does the transformation for the CTU. This stage also performs the CIIP prediction as it needs the result of the intra prediction.In this stage, the current CTU is dependent on the above, left, and above-left CTU. We have to schedule the task in a zigzag order.
  4. VVC_TASK_TYPE_LMCS (L): LMCS(Luma Mapping with Chroma Scaling) stage. This is the first stage for filters. Since the dependency is needed in VVC_TASK_TYPE_RECON, we have to wait until the bottom right CTU is done.
  5. VVC_TASK_TYPE_DEBLOCK_V (V): Vertical deblock stage
  6. VVC_TASK_TYPE_DEBLOCK_H (H): Horizontal deblock stage
  7. VVC_TASK_TYPE_SAO (S): Sample Adaptive Offset(SAO) stage
  8. VVC_TASK_TYPE_ALF (A): Adaptive Loop Filter(ALF) stage

Take the following image as an example. We can decode all the yellow blocks(Maybe more) at the same time (Just informative, the actual situation may be more complex than this, you can check ff_vvc_task_ready for details) CTU

vvc_ctu.c

TBA

vvc_filter.c

TBA

Executor.c

FFmpeg's thread model has limitations, take HEVC for example:

  1. It can't support frame + slice (subframe) threads at the same time. Once you have a slice thread, it will disable the frame thread. This will hurt performance.
  2. It only supports one-way calls. You can only send data to threads, the thread cannot create new tasks for future processes. This means you cannot split the decoder block into multiple stages. This will also affect performance.

We create the executor to solve the above problems. Users call ff_executor_execute to submit tasks. The executor will use the priority_higher callback to reorder the task queue. A thread in the thread pool will wake up to check task readiness using the ready callback. Once a task is ready, it will use the run callback to execute the task.

Clone this wiki locally