Skip to content

Stop the Android backend from holding the executor's workers - #181

Merged
ruccho merged 1 commit into
feature/device-testsfrom
feature/android-mc-fix
Aug 26, 2026
Merged

Stop the Android backend from holding the executor's workers#181
ruccho merged 1 commit into
feature/device-testsfrom
feature/android-mc-fix

Conversation

@ruccho

@ruccho ruccho commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Problem

The end-to-end harness deadlocks on Android whenever the executor has two
workers. It stops immediately after the video encoder accepts its first frame
and never recovers. GitHub's Linux runners have two cores, so the Android job
added by #179 hangs on every run.

unienc_android_mc waits for MediaCodec buffers in three loops, and none of them
releases the executor worker it is running on:

Site Wait
audio::push_impl std::thread::sleep(10 ms) — blocks the worker outright
video::push_video_impl yield_now().await
common::pull_encoded_data_with_codec yield_now().await

yield_now wakes its own waker from inside poll and returns Pending. On
futures::executor::ThreadPool that does not hand the worker to another task.
UnparkMutex::notify sees the task in POLLING, moves it to REPOLL and
returns Err(()), so nothing is queued; Task::run then gets Err(task) back
from mutex.wait and re-polls the same future in place. The task never
re-enters the pool's queue. Three tasks that do nothing but yield_now() in a
loop, on a two-worker pool, poll about 2.9 M times each for the first two and
zero times for the third.

The pipeline runs four tasks — video push, audio push, video pull, audio pull.
With two workers the two push tasks take both and hold them, so the pull tasks
are never polled, releaseOutputBuffer is never called, the codec never frees an
input buffer, and the push loops retry forever. Three workers pass because a pull
task gets a worker too and drains its stream, which lets the matching push task
finish and release a worker for the fourth task.

This is not confined to CI. unienc_c builds the same ThreadPool sized to the
machine, so a device with two usable cores on the CPU readback path meets the
same condition.

What this does

Runs dequeueInputBuffer and dequeueOutputBuffer on the blocking pool through
SpawnBlocking, which the runtime already provides for exactly this. Awaiting a
blocking-pool result parks the task rather than occupying a worker, so the worker
is free for the rest of the pipeline, and the busy loop goes away with it: each
dequeue can now use a real timeout instead of polling with a zero one and
retrying.

MediaCodecAudioEncoder and MediaCodecVideoEncoderOutput gain the Runtime
handle this needs. yield_now is deleted rather than fixed, because there is no
correct self-waking yield on this executor.

The two dequeue helpers return an owned boxed future instead of being async fn,
so that no borrow of the runtime is held across the await; Runtime is Send
but not Sync.

Verification

scripts/android-device-test.sh against an arm64 API 34 emulator with the
google_apis image, at UNIENC_TEST_THREADS of 1, 2 and 3 and with the default
pool. All four pass and produce the expected file:

movie: timescale 10000, duration 9.981 s, 2 track(s)
  audio: mp4a 48000 Hz 2 ch, 469 frames, start 0.000 s, 9.981 s, decoder config: true
  video: avc1 1280x720 (Baseline 4.1), 10 frames, start 0.000 s, 9.000 s

Before this change, two workers hung and three passed, repeatably.

The diagnosis came from debuggerd -b on the hung process: both workers sat in
MediaCodec.dequeueInputBuffer, one below video::push_video_impl and the other
below audio::push_impl, in twelve samples out of twelve, with no pull frame in
any of them.

What this does not cover

The end-of-stream signal in Drop still waits synchronously, because Drop
cannot await. It is bounded and happens once per stream rather than once per
buffer, but it is the same shape, and both sites now say so in a comment.

Stack created with GitHub Stacks CLIGive Feedback 💬

🤖 Generated with Claude Code

@ruccho
ruccho force-pushed the feature/android-mc-fix branch 2 times, most recently from 7a967c2 to dfb4c07 Compare August 25, 2026 07:32
@ruccho
ruccho force-pushed the feature/android-mc-fix branch 2 times, most recently from 069c288 to cf178cc Compare August 25, 2026 08:05
`unienc_android_mc` waited for MediaCodec buffers in three loops, and
none of them released the executor worker it was running on. The audio
push loop slept on the worker outright; the video push loop and the pull
loop called `yield_now().await`, which does not hand the worker to
another task on `futures::executor::ThreadPool`: waking your own waker
from inside `poll` moves the task's `UnparkMutex` from `POLLING` to
`REPOLL`, nothing is sent to the pool's queue, and `Task::run` re-polls
the same future in place. Three tasks that only `yield_now()` in a loop,
on a two-worker pool, poll about 2.9 M times each for the first two and
zero times for the third.

The pipeline runs four tasks: video push, audio push, video pull, audio
pull. Where the pool has two workers, the two push tasks took both and
never gave them up, so the pull tasks were never polled,
`releaseOutputBuffer` was never called, the codec never freed an input
buffer, and the push loops retried forever. `unienc_c` sizes the same
pool to the machine, so a two-core device on the CPU readback path meets
that condition.

Run the dequeues on the blocking pool instead. Awaiting a blocking-pool
result parks the task properly, which frees the worker and removes the
busy loop with it; each dequeue can then use a real timeout rather than
polling with a zero one. `MediaCodecAudioEncoder` and
`MediaCodecVideoEncoderOutput` gain the `Runtime` handle this needs, and
`yield_now` goes away.

The end-to-end harness now passes on an emulator with one, two and three
workers as well as with the default pool. The remaining synchronous wait
is the end-of-stream signal in `Drop`, which cannot await; it is bounded
and happens once per stream, and both sites now say so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ruccho
ruccho force-pushed the feature/android-mc-fix branch from cf178cc to 66d6308 Compare August 25, 2026 08:20
@ruccho
ruccho marked this pull request as ready for review August 26, 2026 02:35
@ruccho
ruccho requested a review from hkmt-mmy August 26, 2026 02:35
@ruccho
ruccho merged commit 0f41028 into main Aug 26, 2026
7 of 14 checks passed
@ruccho
ruccho deleted the feature/android-mc-fix branch August 26, 2026 03:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants