fix: deadlock under -O3 from EINTR-unsafe semaphore usage and lock leaks

[donkikos]

Summary

vtrunkd built with the default -O3 deadlocks after some runtime; the only recovery is kill + restart (see also #63, #141, #198). Root cause: EINTR-unsafe semaphore usage combined with signal handlers installed without SA_RESTART, plus two secondary bugs that leak/over-post shm_sem and corrupt the shared resend buffer.

Root cause

vtrunkd processes signal each other (SIGUSR1) at runtime, and all handlers are installed without SA_RESTART. A signal landing inside sem_wait() makes it return -1/EINTR. All ~90 sem_wait call sites ignore the return value and proceed as if the lock was acquired, then do the paired sem_post() later. A single EINTR hit therefore raises the semaphore count above 1 — mutual exclusion on the shared memory is permanently broken, and the resulting shm corruption ends in the observed deadlock.

Why -O3 specifically: optimization shortens the critical sections, so processes spend a much larger fraction of time blocked inside sem_wait, making a signal hit vastly more likely. The bug exists at -O0 too; it just rarely fires.

Verification

Standalone two-process repro (shared sem_t in shm, SIGUSR1 bomber, handlers without SA_RESTART, canary-guarded critical section), built with -O3:

mode=BUGGY  eintr_hits=1 violations=334968 final_sem_value=2 -> MUTUAL EXCLUSION BROKEN
mode=FIXED  eintr_hits=0 violations=0      final_sem_value=1 -> OK

One EINTR was enough to produce 334k mutual-exclusion violations and leave the semaphore at 2. The EINTR-retry wrapper (identical to the one this PR adds to vtun.h) eliminates it completely.

Changes

• vtun.h — EINTR-retrying sem_wait wrapper applied via macro to all call sites (primary fix; one reviewable change instead of ~90 edits).
• linkfd.c get_resend_frame() — = → == in two conditions that were assigning resend_frames_buf[j].seq_num = *seq_num under resend_buf_sem, corrupting the shared resend buffer (present since the initial import).
• tunnel.c fd_server() — the usecount==0 break exited the loop holding shm_sem, while the fdserver_term path reached the same final sem_post without holding it (over-post → count above 1). Now tracks held state and acquires before teardown so the final post is always balanced.
• server.c — removed sem_init() called on a sem_t* alias of the sem_open()'d semaphore: a compile error on modern gcc (-Wincompatible-pointer-types as error) and wrong regardless — reinitializing a process-shared semaphore possibly in use.

Notes

• Builds clean with -O3 on gcc 13/14. Note: gcc ≥ 10 needs -fcommon due to pre-existing tentative definitions shared across TUs (unrelated to this fix, not touched here).
• 4 files changed, +29/−4.

[donkikos]

Added one commit on top: fix: bound the fd_server teardown wait on shm_sem.

The first commit made the SIGTERM teardown path in run_fd_server() acquire shm_sem before killing the linkers and zeroing the shared memory. That is the right intent (it stops an attacher's usecount++ from being wiped by the memset, and it avoids the over-post), but if the holder died inside its critical section the named semaphore is never posted again, so the fd_server hung forever in exactly the situation an operator sends SIGTERM to recover from. The old code, by accident, always completed the teardown.

The new commit adds sem_wait_bounded() next to the EINTR wrapper in vtun.h, waits at most 5 s in the teardown path, logs and proceeds on timeout (the final sem_post then restores the count to 1 on the assumption the holder is dead), and replaces the shm_sem_held flag with a goto so there is a single acquisition site.

Separate PRs for the independent findings from review: stale semaphore reset on server start, fd_server failure handling, and removal of the dead sem_wait_tw().