build: fix ffi dependency compilation #5437

Job	Run time
Build slim tarball	2m 23s
aarch64-darwin: with shared libraries	2h 23m 27s
x86_64-darwin: with shared libraries	2h 24m 47s
aarch64-linux: with shared libraries	1h 20m 47s
x86_64-linux: with shared libraries	1h 56m 15s
	8h 7m 39s

Provide feedback