Implement worker auto-scaler (stability and performance boost)

[AJenbo]

This implements worker auto scaling. Rather then simply setting a fixed number of 8 workers this looks at the available CPU threads and RAM to determin the best number of active workers.

This has two consequences:

• System low on free memory won't crash by default
• Large systems won't be performance caped by a lower number of workers

On a co-workers memory strained MacBook maximumNumberOfProcesses normally has to be set to 4 or PHPStan will crash. With this solution PHPStan now limits it self to the available resources with out needing manual tweaking.

On my workstation with 32 available threads it results in a 30% performance lift by better utilizing the available threads instead of being caped to 8 cores.

For CI where there can be a discrepancy between reported cores and available cores this implementation looks at the cgroup and CFS to determin effective limits avoiding the issues that the fixed 8 default was trying to be a happy medium for.

The estimated worker memory usage is based on usage on 25 real projects (application based on WordPress, PHPBB, Laravel, micro frameworks, OpenCart, Phpactor, phpstan-src etc).

I would say it's fairly well tested, but it's probably also advised to do further testing on diverse systems, for one thing I don't have access to Windows or macOS.

[AJenbo]

I have now verified that it also works as expected on macOS and Docker.

[ondrejmirtes]

The 30 % improvement would of course be very welcome, but the more I thought about this, the less sense the infinite scaling makes to me.

Please describe the actual algorithm of how the scaling works. Give some examples of different project sizes / number of CPU cores / available memory and other variables going into the formula and what the result looks like.

I've set the limit to 8 recently because in my measurements, after that the returns were diminishing. It didn't seem worth it to consume twice the CPU for 10 % of speedup for example.

Maybe we can make a chart of how the performance scales? X axis number of processes, Y axis time spent analysing a project.

I also recently took advantage of the improvements in cpu-core-counter, 89f1382, hoping it'd solve a problem for some users where their PC would be unusable during PHPStan run. So I definitely don' want to completely exhaust a machine when running PHPStan.

Are you somehow estimating how much memory would each worker use? That doesn't seem possible to me.

Also - in some CI environments the CPU is shared, so although 32 cores are available, we shouldn't use all of them because the machine should also be available for other processes / adjacent runners.

[ondrejmirtes]

Maybe we can get the 90 % of your improvement just by increasing the default max from 8 to 16? Just an idea...

[AJenbo]

The 30 % improvement would of course be very welcome, but the more I thought about this, the less sense the infinite scaling makes to me.

Right, there of cause is no infinit scale to extract here :) I only have 16 cores and 32 threads to test with, the efficiency of my cores drops once i go past 16, but from my tests there is still a lot of performance to extract by improving the distribution here.

The 30% is specific to my main system, but I did see improviserets on a wide range of system configurations and no slow downs across the 25 real projects i measured.

I can try and see if I can get someone to run a test on a 128 core machine later so we can see if there should be an upper limit to how many cores it tries to scale to.

One thing to note is that I do have two followups planned which aims to improve how well PHPStan scales across multiple cores, which would improve this further. If there was no cost to horizontal scaling and things where perfectly balanced it should have been a roughly 300% improvement so there is some further potential here.

I've set the limit to 8 recently because in my measurements, after that the returns were diminishing. It didn't seem worth it to consume twice the CPU for 10 % of speedup for example.

Core scaling is a diminishing returns game since there are overheads, but without it it's going to be hard to pull 10% out of a hat.

Also - in some CI environments the CPU is shared, so although 32 cores are available, we shouldn't use all of them because the machine should also be available for other processes / adjacent runners.

That's exactly why I implemented reading of cgroup and CFS limits, these systems should actually see an improvement (I did test this) from the change since it now avoids managing 8 processes on systems that only allow for 2-4 concurrent threads per tenant.

Maybe we can get the 90 % of your improvement just by increasing the default max from 8 to 16? Just an idea...

This can hurt performance where there are fewer available threads, in fact on one resource constrained system where I tested this PR it went from 5m20s to 4m40s from the better balance (mainly by not over committing memory).

Bumping it to 16 would cause more low memory systems to start thrashing and even crash, and low core systems would be wasting time managing processes without any performance (at least true with shared systems with the current solution). I think 8 is a good fallback value, but it's not a good one-fits-all and 16 is worse.

I'll get back to you with graphs and the other things you asked for.

[AJenbo]

Oh one good news already with the pcntl_fork enablement you did the thread overhead is already reduced noticeably (13% improvement).

1-4 threads looks to be effectively liner scaling and after that it starts to taper off with a 2.2x overhead at 32 threads (something that can be improved by future work).