Potential performance improvements

[piotr-oles]

Hi! 👋 I'm an author of ForkTsCheckerWebpackPlugin. I recently removed support for EsLint in the plugin to focus on the TypeScript part. The eslint-webpack-plugin is a suggested replacement, but some users experience performance degradation when switching to this plugin (TypeStrong/fork-ts-checker-webpack-plugin#601) (for example from ~24s to ~40s).

Modification Proposal

Use compiler.modifiedFiles to get a list of files to lint and compiler.removedFiles to get a list of files to remove errors. This way you don't have to wait for compilation.hooks.finishModules to start the linting process.

Expected Behavior / Situation

The eslint-webpack-plugin has similar performance to fork-ts-checker-webpack-plugin@^6.0.0

Actual Behavior / Situation

The eslint-webpack-plugin is often significantly slower.

Please paste the results of npx webpack-cli info here, and mention other relevant information

I believe it's not related :)

[alexander-akait]

/cc @ricardogobbosouza What do you think?

[jsg2021]

What is the goal? Lint only modified files? or lint all files in the graph? We already have a config to only lint modified files.

Though, I like the new api hooks, I didn't know of them when worked on this last.

[piotr-oles]

The idea is to lint all files on the initial run and then lint only changed files on next compilations.

Maybe it would be also worth to provide "files" option to the plug-in to specify files to lint (so we can start linting early also on the initial run). We could then filter-out errors coming from files outside of the dependency graph and lint files that were missing (which hopefully will be 0). We can easily have files outside of the dependency graph - for example if you have typescript file that contains types only, this file will not become part of the dependency graph.

[jsg2021]

We made effort to ignore files out of the graph on purpose.

If memory serves, we toss a job at a worker pool for every file discovered. It doesn't wait for the finished callback to lint, only to report. We only re-lint modified files.

The performance difference you're seeing may be overhead of the worker pool?

[piotr-oles]

I don't think so, it's ~20-second difference. @eamodio, did you experience this difference on the initial compilation, or in the next compilations?

[eamodio]

Yeah, initial compilation was about a 20s difference, but even on change it seemed slower too.

[ricardogobbosouza]

The idea of ​​using graph files is exactly to be faster, even more that we use workers

Could you share a repository that can prove this slowness?

[jsg2021]

Machine details would help too. I could believe this plugin is a little biased to very-wide CPUs. (ie: being less ideal for low-core counts)

[eamodio]

I just pushed a branch here: https://github.com/gitkraken/vscode-gitlens/tree/feature/eslint-webpack -- that switches from using eslint from fork-ts-checker-webpack to eslint-webpack-plugin.

Using that same branch, just 1 commit back (e.g. using fork-ts-checker-webpack to run eslint), and turning off eslint caching, it would take ~23-24s to run a yarn run watch, whereas after the upgrade to eslint-webpack-plugin to takes ~39-40s.

I'm running on Windows 11 (10.0.22000 Build 22000) on a desktop PC with

• Processor: AMD Ryzen 9 3900X 12-Core Processor, 3801 Mhz, 12 Core(s), 24 Logical Processor(s)
• 32GB DDR4
• 1TB Sabrent Rocket 4.0 NVMe SSD - Gen4 PCIe x4

I've also tried to enable threading -- using threads: true, threads: 1, or threads: 2 and in all cases webpack never returns.

Hope that helps!

[jsg2021]

threads: number will limit the number of threads (0 obviously disabling). threads: boolean will enable/disable. The default thread limit is cpus - 1. If it's not returning, that sounds like a bug. Without threads enabled it will be much slower.

[Kaizer69]

I confirm @eamodio problem.
Using threads : true webpack never returns.

[JUST-Limbo]

I just pushed a branch here: https://github.com/gitkraken/vscode-gitlens/tree/feature/eslint-webpack -- that switches from using eslint from fork-ts-checker-webpack to eslint-webpack-plugin.

Using that same branch, just 1 commit back (e.g. using fork-ts-checker-webpack to run eslint), and turning off eslint caching, it would take ~23-24s to run a yarn run watch, whereas after the upgrade to eslint-webpack-plugin to takes ~39-40s.

I'm running on Windows 11 (10.0.22000 Build 22000) on a desktop PC with

• Processor: AMD Ryzen 9 3900X 12-Core Processor, 3801 Mhz, 12 Core(s), 24 Logical Processor(s)
• 32GB DDR4
• 1TB Sabrent Rocket 4.0 NVMe SSD - Gen4 PCIe x4

I've also tried to enable threading -- using threads: true, threads: 1, or threads: 2 and in all cases webpack never returns.

Hope that helps!

Did you mean when you are using threads: true , there are no eslint errors/warnings webpack returns in the vscode terminal after build complete?

I recently ran into the above problem: when using threads: true ,there are no eslint errors/warnings in the vscode terminal after build complete.
But,when it set to false,the expected errors/warnings come back.

[eamodio]

Correct, when using threads: true it never returns at all for me.