pipe2/dup3 O_CLOEXEC

[jakepetroules]

Summary

• pipe() + a separate fcntl(F_SETFD, FD_CLOEXEC) (and the equivalent for dup/dup2) is not atomic: a fork() racing on another thread between the two calls can inherit the not-yet-marked descriptor into an unrelated child this library spawns concurrently.
• Add _subprocess_pipe_cloexec, _subprocess_dup3_cloexec, and _subprocess_dup_cloexec to _SubprocessCShims. On Linux/FreeBSD/OpenBSD these call pipe2/dup3 directly (declared unconditionally there). On Darwin, which does not yet declare pipe2/dup3, they check for <Availability.h>'s __MAC_27_0 (only defined once the SDK knows about a macOS release new enough to vend them) and use __builtin_available to call the real syscalls once that SDK ships, falling back to the old non-atomic sequence otherwise -- the same technique swift-system's own CSystem shim uses, and the same fix applied to swift-build in pipe2/dup3 O_CLOEXEC swift-build#1520.
• Route every internal pipe/dup creation through these new primitives via FileDescriptor.cloexecPipe()/FileDescriptor.safeDuplicate(): the self-pipe used to report exec() failures back to the parent, the Linux/BSD/kqueue self-pipes used for shutdown/signal notification, and the pipes/duplicated descriptors used for a subprocess's own stdio.
• _subprocess_dup3_cloexec itself has no caller yet -- included for parity with _subprocess_pipe_cloexec so a future one doesn't have to reimplement the SDK-availability handling.

Two places that need the descriptors to not be close-on-exec

Making the stdio pipes close-on-exec exposed two spots where a descriptor is deliberately meant to survive into the child. Both are handled explicitly rather than by reverting to non-atomic creation, so the parent's descriptors stay close-on-exec for their whole lifetime and the fork race stays closed.

• Darwin pre-fork path (createSession/uid/gid/supplementary groups). That path spawns with POSIX_SPAWN_SETEXEC, which makes posix_spawn() exec the calling process instead of creating a new one. In that mode the exec-time close-on-exec sweep runs before the file actions are applied, so any close-on-exec descriptor named by an adddup2/addclose action is already gone and the whole spawn fails with EBADF. This is what broke testSubprocessPlatformOptionsCreateSession on macOS CI. Reproduced standalone: it happens for adddup2 sources and addclose targets alike, and with or without POSIX_SPAWN_CLOEXEC_DEFAULT. Fixed by passing the file-action descriptor list down to _subprocess_spawn and clearing FD_CLOEXEC on them in the forked child, which is a private copy about to exec. Nothing leaks into the exec'd image: POSIX_SPAWN_CLOEXEC_DEFAULT closes everything the file actions don't install, and the parent-side ends are closed outright by addclose. The ordinary (non-SETEXEC) posix_spawn() path is unaffected.
• Linux/BSD fork/exec path. Binding the child's stdio with dup2 is still correct -- dup2 clears close-on-exec on the descriptor it creates regardless of the source -- except when fd == target, where dup2 is a documented no-op that leaves the flag untouched. That can happen when the parent had that standard descriptor closed and the pipe got allocated over it, and with close-on-exec pipes it would leave the child's stream closed after exec. _subprocess_bind_standard_fd now clears the flag explicitly in that case. (posix_spawn_file_actions_adddup2 already specifies the equal-descriptor case as clearing FD_CLOEXEC, so the Darwin path needs nothing here.)

Test plan

• swift test on macOS 27: 171 tests in 7 suites pass, including testSubprocessPlatformOptionsCreateSession, which fails with EBADF without the pre-fork fix above and passes with it. Verified the same test passes on an unmodified main checkout, confirming the failure was introduced here and is now resolved.
• Standalone C programs to isolate the POSIX_SPAWN_SETEXEC behavior and to check _subprocess_bind_standard_fd's equal-descriptor handling (that path is Linux/BSD-only and isn't compiled on macOS).
• _SubprocessCShims compiles cleanly with -Wall -Wextra.
• swift format lint --configuration .swift-format --strict -r Sources/Subprocess Sources/_SubprocessCShims

[iCharlesHu]

@jakepetroules macOS failure seems related.

[sunshowers]

Hi! Came across this. Does this mean macOS 27 (finally?) has atomic CLOEXEC for pipes?