Large RLIMIT_NOFILE causes application hang

[markovejnovic]

Hey folks! Firstly, thank you for developing this neat piece of software.

We're working towards fuzzing https://github.com/oven-sh/bun and I'm currently at the stage of getting JSC fuzzed. So far, I've managed to fuzz JSC on my Linux 6.16.12-200.fc42.x86_64 x86_64 unknown machine.

I've, however, attempted to stick Fuzzilli in a docker container and unfortunately that caused Fuzzilli to misbehave. It seems to get stuck. I ran strace to see what's happening and it seems that Fuzzilli is going through and closing a bunch of FDs. Looking at the signature of when it locks up, I think the issue lies in Sources/libreprl/libreprl-posix.c:reprl_spawn_child.

That function contains this loop:

int tablesize = getdtablesize();
for (int i = 3; i < tablesize; i++) {
    if (i == REPRL_CHILD_CTRL_IN || i == REPRL_CHILD_CTRL_OUT || i == REPRL_CHILD_DATA_IN || i == REPRL_CHILD_DATA_OUT) {
        continue;
    }
    close(i);
}

The reason I don't think this is an issue on the host machine is because RLIMIT_NOFILE is set to 1024 on the host machine, and a significantly larger 1073741816 in the docker image which is running under root.

strace.log

Stop-Gap

Two options:

• Run as non-root (duh)
• Control RLIMIT_NOFILE inside the docker container

Fix?

One improvement I could implement is to use close_range(2). I'll see if I can hack up a PR.

[markovejnovic]

Implementing a fix here #541