Skip to content

Implement FUTEX_WAIT_BITSET and FUTEX_WAKE_BITSET - #2775

Open
emkey1 wants to merge 1 commit into
ish-app:masterfrom
emkey1:futex_wait_bitset
Open

Implement FUTEX_WAIT_BITSET and FUTEX_WAKE_BITSET#2775
emkey1 wants to merge 1 commit into
ish-app:masterfrom
emkey1:futex_wait_bitset

Conversation

@emkey1

@emkey1 emkey1 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Any program that uses a pthread barrier or an absolute condition-variable timeout dies on master with:

The futex facility returned an unexpected error code.

glibc's pthread_cond_timedwait and pthread_barrier_wait issue FUTEX_WAIT_BITSET | FUTEX_CLOCK_REALTIME, which is op 265. sys_futex only strips FUTEX_PRIVATE_FLAG before its switch, so the clock bit survives, nothing matches, and the call falls through to ENOSYS — which glibc treats as fatal and aborts on.

I found this while trying to write a threaded stress test against master for an unrelated PR, and it turned out to be blocking any threaded test at all.

The change

Three parts, all in kernel/futex.c:

  • FUTEX_CLOCK_REALTIME joins FUTEX_CMD_MASK_. Like FUTEX_PRIVATE_FLAG it is a modifier on the command, not part of it.
  • FUTEX_WAIT_BITSET/FUTEX_WAKE_BITSET carry a mask. A waiter records the mask it waited with and only wakes if it overlaps the waker's. Plain FUTEX_WAIT/FUTEX_WAKE pass FUTEX_BITSET_MATCH_ANY, so their behaviour is unchanged. A zero mask is EINVAL, as on Linux.
  • Timeout base differs. FUTEX_WAIT's timeout is relative; FUTEX_WAIT_BITSET's is absolute, against CLOCK_REALTIME when the flag is set and CLOCK_MONOTONIC otherwise. It is converted before being handed to wait_for, and an already-expired deadline becomes a zero timeout rather than a negative one.

Testing

Same static i386 binary on real Linux (x86 Ubuntu) and on iSH:

before after real Linux
pthread_barrier_wait aborts, 3/3 runs completes completes
pthread_cond_timedwait that times out aborts ETIMEDOUT after 0.304s ETIMEDOUT after 0.303s
pthread_cond_timedwait that gets signalled aborts wakes on the signal, not the deadline same
plain pthread_cond_wait works works works

The last row is the regression check on the untouched FUTEX_WAIT path, and the timing rows check the absolute→relative conversion in both directions — a sign error there would show up as an immediate return or a hang rather than a wrong answer.

Also re-ran the pty, SIGURG, fstatat and threaded COW-fault tests I had to hand, plus an Alpine shell smoke with background jobs: unchanged.

Not included

The other unimplemented ops (FUTEX_CMP_REQUEUE, FUTEX_WAKE_OP, the PI family) are left alone. They are separate work and nothing I ran needed them; this is deliberately scoped to the one that makes glibc abort.

🤖 Generated with Claude Code

glibc's pthread_cond_timedwait and pthread_barrier_wait issue
FUTEX_WAIT_BITSET with FUTEX_CLOCK_REALTIME (op 265). sys_futex only
stripped FUTEX_PRIVATE_FLAG before its switch, so the clock bit stayed
on, nothing matched, and the call returned ENOSYS. glibc turns that into

    The futex facility returned an unexpected error code.

and aborts, which takes out any program using a barrier or an absolute
condition-variable timeout.

Three parts:

  * FUTEX_CLOCK_REALTIME joins FUTEX_PRIVATE_FLAG in FUTEX_CMD_MASK_. It
    is a modifier on the command, not part of it.
  * FUTEX_WAIT_BITSET/FUTEX_WAKE_BITSET carry a mask. A waiter records
    the mask it waited with and only wakes if it overlaps the waker's.
    Plain FUTEX_WAIT/FUTEX_WAKE pass FUTEX_BITSET_MATCH_ANY, so their
    behaviour is unchanged. A zero mask is EINVAL, as on Linux.
  * FUTEX_WAIT's timeout is relative but FUTEX_WAIT_BITSET's is absolute,
    against CLOCK_REALTIME when the flag is set and CLOCK_MONOTONIC
    otherwise, so it is converted before being handed to wait_for. An
    already-expired deadline becomes a zero timeout rather than a
    negative one.

Verified against real Linux with the same static i386 binary:

                                        before          after / Linux
  pthread_barrier_wait                  aborts (3/3)    completes
  pthread_cond_timedwait, times out     aborts          ETIMEDOUT at
                                                        0.304s (Linux
                                                        0.303s)
  pthread_cond_timedwait, signalled     aborts          wakes on the
                                                        signal, not the
                                                        deadline
  plain pthread_cond_wait               works           works

The last row is the regression check on the untouched FUTEX_WAIT path.
Also re-ran the pty, sigurg, fstatat and threaded COW tests I had to hand
plus an Alpine shell smoke: unchanged.
@saagarjha

Copy link
Copy Markdown
Member

This not a review but just FYI that we primary aim for Alpine compatibility. Nothing wrong with sending in glibc support and we are happy to merge it once it's been looked at but we will probably do this with reduced priority

@emkey1

emkey1 commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

That is a fair call, and I checked rather than assumed — musl does not need any of this.

From musl's sources:

file op used
src/thread/__timedwait.c:52 FUTEX_WAIT
src/thread/pthread_barrier_wait.c FUTEX_WAIT
src/thread/pthread_cond_timedwait.c FUTEX_REQUEUE

and __timedwait.c:42-45 converts the absolute deadline to a relative timeout in userspace before the call:

if (__clock_gettime(clk, &to)) return EINVAL;
to.tv_sec = at->tv_sec - to.tv_sec;
if ((to.tv_nsec = at->tv_nsec - to.tv_nsec) < 0) { ... }

So musl uses only FUTEX_WAIT/FUTEX_WAKE/FUTEX_REQUEUE — exactly the three already implemented. Your prioritization is right: this is glibc-only, and an Alpine guest will never hit it.

It is still a real wall for glibc guests, which is how I ran into it — a glibc binary using a barrier or an absolute condvar timeout aborts outright rather than degrading. Reduced priority is completely reasonable; leaving it here in case it is useful later.

(One incidental thing from reading that code, no action implied: musl prefers SYS_futex_time64 when available and falls back on -ENOSYS, so 32-bit time64 musl handles a missing futex_time64 gracefully.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants