Return EINVAL from F_DUPFD instead of asserting

[emkey1]

fcntl(fd, F_DUPFD, -1) aborts ish.

arg is a dword_t, so the guest's -1 arrives as 0xffffffff and is handed straight to f_install_start, where it narrows back to a negative fd_t and trips assert(start >= 0). Any guest program can take the emulator down with a single syscall:

Assertion failed: (start >= 0), function f_install_start, file fd.c, line 159.

Linux returns EINVAL when the starting fd is negative or at/above RLIMIT_NOFILE, so this checks both before installing. F_DUPFD_CLOEXEC takes the same argument and needed the same check.

Testing

Repro program run under ish -f alpine, and the same binary run on real Linux (x86 Ubuntu) to confirm the expected behavior:

int r = fcntl(0, F_DUPFD, -1);
printf("fcntl(0, F_DUPFD, -1) -> %d (%s)\n", r, r < 0 ? strerror(errno) : "no error");
r = fcntl(0, F_DUPFD, 1 << 30);
printf("fcntl(0, F_DUPFD, 1<<30) -> %d (%s)\n", r, r < 0 ? strerror(errno) : "no error");

	result
Linux	both -1 (Invalid argument)
master	SIGABRT on the first call
this branch	both -1 (Invalid argument)

Also ran a shell smoke test over the Alpine rootfs (directory walks, /proc reads, fork/exec churn, redirects, pipes) with no change in behavior.

🤖 Generated with Claude Code