Improve benchmarks by prymitive · Pull Request #1858 · cloudflare/pint

prymitive · 2026-06-02T13:46:15Z

No description provided.

github-actions

This pull request was validated by pint.

✔️ No problems found

Stats

Stat	Value
Version	v0.84.0-70-g4e1e33a8
Number of rules parsed	9
Number of rules checked	9
Number of problems found	0
Number of offline checks	0
Number of online checks	0
Checks duration	0

Problems

No problems reported

codecov-commenter · 2026-06-03T11:02:25Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (4d57749) to head (4e1e33a).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##              main     #1858   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          113       113           
  Lines        14649     14648    -1     
=========================================
- Hits         14649     14648    -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2026-06-03T11:12:33Z

Benchmark diff:

goos: linux
goarch: amd64
pkg: github.com/cloudflare/pint/cmd/pint
cpu: AMD EPYC 9V74 80-Core Processor                
                               │   old.txt    │                 new.txt                  │
                               │    sec/op    │     sec/op      vs base                  │
GlobFinder-4                     19.18m ±  7%   19.83m ±    2%        ~ (p=0.073 n=10+6)
GitFinder-4                      398.5m ±  1%   412.3m ±    3%   +3.45% (p=0.001 n=10+6)
CheckRules-4                      9.681 ±  1%    9.665 ±    1%        ~ (p=0.492 n=10+6)
Check/promql/fragile-4           59.67µ ±  1%   59.65µ ±    1%        ~ (p=0.562 n=10+6)
Check/promql/vector_matching-4   86.73m ±  2%   88.16m ±    5%        ~ (p=0.118 n=10+6)
Check/rule/duplicate-4           5.580m ±  0%   5.627m ±    0%   +0.85% (p=0.001 n=10+6)
Check/promql/counter-4           2.127m ±  2%   2.178m ±    8%   +2.40% (p=0.022 n=10+6)
Check/alerts/absent-4            124.3µ ±  3%   123.4µ ±    9%        ~ (p=0.792 n=10+6)
Check/rule/reject-4              103.5µ ±  1%   103.3µ ±    5%        ~ (p=0.428 n=10+6)
Check/query/cost-4               9.595m ± 10%   9.830m ±    8%        ~ (p=0.958 n=10+6)
Check/promql/syntax-4            8.630µ ±  1%   8.687µ ±    0%   +0.66% (p=0.039 n=10+6)
Check/alerts/for-4               96.54µ ±  1%   98.43µ ±    2%   +1.96% (p=0.003 n=10+6)
Check/alerts/comparison-4        32.06µ ±  0%   32.08µ ±    2%        ~ (p=0.492 n=10+6)
Check/rule/dependency-4          5.331m ±  1%   5.403m ±    1%   +1.36% (p=0.000 n=10+6)
Check/promql/impossible-4        120.6µ ±  1%   125.3µ ±    8%   +3.92% (p=0.000 n=10+6)
Check/promql/rate-4              3.104m ±  1%   3.145m ±    2%        ~ (p=0.118 n=10+6)
Check/promql/series-4            406.9m ±  1%   415.8m ± 2117%   +2.19% (p=0.007 n=10+6)
Check/promql/offset-4            2.234m ±  1%   2.321m ±    3%   +3.91% (p=0.000 n=10+6)
Check/alerts/template-4          106.5m ±  2%   105.6m ±    3%        ~ (p=0.093 n=10+6)
Check/promql/regexp-4            1.515m ±  2%   1.524m ±    1%        ~ (p=0.492 n=10+6)
Check/group/interval-4           2.730µ ±  0%   2.734µ ±    0%        ~ (p=0.441 n=10+6)
Check/promql/range_query-4       2.282m ±  2%   2.303m ±    1%        ~ (p=0.492 n=10+6)
Check/labels/conflict-4          1.488m ±  1%   1.425m ±    2%   -4.22% (p=0.000 n=10+6)
Check/alerts/external_labels-4   25.51m ±  2%   25.72m ±    3%        ~ (p=0.713 n=10+6)
Check/alerts/count-4             9.973m ±  7%   9.611m ±    7%        ~ (p=0.313 n=10+6)
Check/alerts/annotation-4        304.1µ ±  2%   343.2µ ±    2%  +12.86% (p=0.000 n=10+6)
Check/promql/features-4          121.0µ ±  1%   123.7µ ±    1%   +2.27% (p=0.007 n=10+6)
Check/rule/label-4               600.6µ ±  1%   600.9µ ±    2%        ~ (p=0.875 n=10+6)
RuleIsIdentical-4                7.077m ±  1%   7.100m ±    1%        ~ (p=0.428 n=10+6)
GetChecksForEntry-4              120.6m ±  1%   116.0m ±    1%   -3.80% (p=0.000 n=10+6)
geomean                          2.379m         2.405m           +1.09%

                               │    old.txt     │                  new.txt                   │
                               │      B/op      │      B/op        vs base                   │
GlobFinder-4                     7.107Mi ± 0%     7.107Mi ±    0%       ~ (p=0.958 n=10+6)
GitFinder-4                      15.74Mi ± 1%     15.74Mi ±    0%       ~ (p=0.368 n=10+6)
CheckRules-4                     880.9Mi ± 0%     879.7Mi ±    1%       ~ (p=0.118 n=10+6)
Check/promql/fragile-4           9.203Ki ± 0%     9.203Ki ±    0%       ~ (p=0.750 n=10+6)
Check/promql/vector_matching-4   36.35Mi ± 0%     36.36Mi ±    1%  +0.04% (p=0.022 n=10+6)
Check/rule/duplicate-4             0.000 ± 0%       0.000 ±    0%       ~ (p=1.000 n=10+6) ¹
Check/promql/counter-4           251.2Ki ± 0%     251.2Ki ±    3%  +0.00% (p=0.002 n=10+6)
Check/alerts/absent-4            160.7Ki ± 0%     160.7Ki ±    0%       ~ (p=0.750 n=10+6)
Check/rule/reject-4                0.000 ± 0%       0.000 ±    0%       ~ (p=1.000 n=10+6) ¹
Check/query/cost-4               292.8Ki ± 0%     292.8Ki ±   16%       ~ (p=0.484 n=10+6)
Check/promql/syntax-4              0.000 ± 0%       0.000 ±    0%       ~ (p=1.000 n=10+6) ¹
Check/alerts/for-4               80.86Ki ± 0%     80.86Ki ±    0%  +0.00% (p=0.000 n=10+6)
Check/alerts/comparison-4          352.0 ± 0%       352.0 ±    0%       ~ (p=1.000 n=10+6) ¹
Check/rule/dependency-4          4.876Mi ± 0%     4.876Mi ±    0%       ~ (p=0.750 n=10+6)
Check/promql/impossible-4        80.86Ki ± 0%     80.86Ki ±    0%       ~ (p=1.000 n=10+6) ¹
Check/promql/rate-4              369.7Ki ± 0%     369.7Ki ±   10%       ~ (p=0.750 n=10+6)
Check/promql/series-4            57.65Mi ± 0%     57.65Mi ± 1161%       ~ (p=0.300 n=10+6)
Check/promql/offset-4            320.3Ki ± 0%     320.4Ki ±    0%  +0.00% (p=0.010 n=10+6)
Check/alerts/template-4          64.46Mi ± 0%     64.47Mi ±    0%  +0.02% (p=0.003 n=10+6)
Check/promql/regexp-4            913.4Ki ± 0%     913.6Ki ±    0%  +0.01% (p=0.003 n=10+6)
Check/group/interval-4             0.000 ± 0%       0.000 ±    0%       ~ (p=1.000 n=10+6) ¹
Check/promql/range_query-4       320.9Ki ± 0%     320.9Ki ±    0%  +0.00% (p=0.018 n=10+6)
Check/labels/conflict-4          159.3Ki ± 0%     159.3Ki ±    0%       ~ (p=1.250 n=10+6)
Check/alerts/external_labels-4   10.56Mi ± 0%     10.56Mi ±    0%       ~ (p=0.209 n=10+6)
Check/alerts/count-4             858.1Ki ± 4%     853.3Ki ±    5%       ~ (p=0.635 n=10+6)
Check/alerts/annotation-4          0.000 ± 0%       0.000 ±    0%       ~ (p=1.000 n=10+6) ¹
Check/promql/features-4          79.28Ki ± 0%     79.28Ki ±    0%       ~ (p=1.000 n=10+6) ¹
Check/rule/label-4               396.0Ki ± 0%     396.0Ki ±    0%  +0.00% (p=0.000 n=10+6)
RuleIsIdentical-4                  0.000 ± 0%       0.000 ±    0%       ~ (p=1.000 n=10+6) ¹
GetChecksForEntry-4              57.83Mi ± 0%     57.60Mi ±    0%  -0.40% (p=0.000 n=10+6)
geomean                                       ²                    -0.03%                  ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                               │    old.txt    │                 new.txt                  │
                               │   allocs/op   │   allocs/op    vs base                   │
GlobFinder-4                     110.6k ± 0%     110.6k ±   0%       ~ (p=1.022 n=10+6)
GitFinder-4                      142.9k ± 0%     142.9k ±   0%       ~ (p=0.811 n=10+6)
CheckRules-4                     9.418M ± 0%     9.401M ±   0%  -0.18% (p=0.000 n=10+6)
Check/promql/fragile-4            149.0 ± 0%      149.0 ±   0%       ~ (p=0.750 n=10+6)
Check/promql/vector_matching-4   430.4k ± 0%     430.4k ±   1%  +0.00% (p=0.010 n=10+6)
Check/rule/duplicate-4            0.000 ± 0%      0.000 ±   0%       ~ (p=1.000 n=10+6) ¹
Check/promql/counter-4           12.88k ± 0%     12.88k ±   1%       ~ (p=0.750 n=10+6)
Check/alerts/absent-4             615.0 ± 0%      615.0 ±   0%       ~ (p=1.000 n=10+6) ¹
Check/rule/reject-4               0.000 ± 0%      0.000 ±   0%       ~ (p=1.000 n=10+6) ¹
Check/query/cost-4               4.400k ± 0%     4.400k ±  13%       ~ (p=0.750 n=10+6)
Check/promql/syntax-4             0.000 ± 0%      0.000 ±   0%       ~ (p=1.000 n=10+6) ¹
Check/alerts/for-4               1.350k ± 0%     1.350k ±   0%       ~ (p=1.000 n=10+6) ¹
Check/alerts/comparison-4         4.000 ± 0%      4.000 ±   0%       ~ (p=1.000 n=10+6) ¹
Check/rule/dependency-4          3.857k ± 0%     3.857k ±   0%       ~ (p=1.000 n=10+6) ¹
Check/promql/impossible-4         246.0 ± 0%      246.0 ±   0%       ~ (p=1.000 n=10+6) ¹
Check/promql/rate-4              5.016k ± 0%     5.016k ±   6%       ~ (p=0.750 n=10+6)
Check/promql/series-4            829.9k ± 0%     829.9k ± 831%       ~ (p=0.263 n=10+6)
Check/promql/offset-4            7.394k ± 0%     7.394k ±   0%       ~ (p=1.000 n=10+6) ¹
Check/alerts/template-4          593.2k ± 0%     593.2k ±   0%  +0.00% (p=0.000 n=10+6)
Check/promql/regexp-4            10.21k ± 0%     10.21k ±   0%       ~ (p=1.000 n=10+6) ¹
Check/group/interval-4            0.000 ± 0%      0.000 ±   0%       ~ (p=1.000 n=10+6) ¹
Check/promql/range_query-4       7.406k ± 0%     7.406k ±   0%       ~ (p=1.000 n=10+6) ¹
Check/labels/conflict-4          3.306k ± 0%     3.306k ±   0%       ~ (p=1.000 n=10+6) ¹
Check/alerts/external_labels-4   182.2k ± 0%     182.2k ±   0%       ~ (p=0.266 n=10+6)
Check/alerts/count-4             19.34k ± 3%     19.27k ±   3%       ~ (p=0.444 n=10+6)
Check/alerts/annotation-4         0.000 ± 0%      0.000 ±   0%       ~ (p=1.000 n=10+6) ¹
Check/promql/features-4           231.0 ± 0%      231.0 ±   0%       ~ (p=1.000 n=10+6) ¹
Check/rule/label-4               6.612k ± 0%     6.612k ±   0%       ~ (p=1.000 n=10+6) ¹
RuleIsIdentical-4                 0.000 ± 0%      0.000 ±   0%       ~ (p=1.000 n=10+6) ¹
GetChecksForEntry-4              963.7k ± 0%     951.6k ±   0%  -1.26% (n=10+6)
geomean                                      ²                  -0.06%                  ²
¹ all samples are equal
² summaries must be >0 to compute geomean

pkg: github.com/cloudflare/pint/internal/diags
                             │   old.txt   │               new.txt                │
                             │   sec/op    │   sec/op     vs base                 │
InjectDiagnosticsLongExpr-4    6.037µ ± 1%   6.038µ ± 9%       ~ (p=0.958 n=10+6)
InjectDiagnosticsShortExpr-4   1.018µ ± 5%   1.001µ ± 1%  -1.63% (p=0.027 n=10+6)
geomean                        2.479µ        2.459µ       -0.81%

                             │   old.txt    │                 new.txt                 │
                             │     B/op     │     B/op      vs base                   │
InjectDiagnosticsLongExpr-4    3.578Ki ± 0%   3.578Ki ± 0%       ~ (p=1.000 n=10+6) ¹
InjectDiagnosticsShortExpr-4     304.0 ± 0%     304.0 ± 0%       ~ (p=1.000 n=10+6) ¹
geomean                        1.031Ki        1.031Ki       +0.00%
¹ all samples are equal

                             │  old.txt   │                new.txt                │
                             │ allocs/op  │ allocs/op   vs base                   │
InjectDiagnosticsLongExpr-4    39.00 ± 0%   39.00 ± 0%       ~ (p=1.000 n=10+6) ¹
InjectDiagnosticsShortExpr-4   13.00 ± 0%   13.00 ± 0%       ~ (p=1.000 n=10+6) ¹
geomean                        22.52        22.52       +0.00%
¹ all samples are equal

pkg: github.com/cloudflare/pint/internal/parser
        │   old.txt   │               new.txt                │
        │   sec/op    │   sec/op     vs base                 │
Parse-4   16.24m ± 0%   16.91m ± 7%  +4.14% (p=0.005 n=10+6)

        │   old.txt    │             new.txt              │
        │     B/op     │     B/op      vs base            │
Parse-4   6.387Mi ± 0%   6.387Mi ± 0%  ~ (p=0.936 n=10+6)

        │   old.txt   │              new.txt              │
        │  allocs/op  │  allocs/op   vs base              │
Parse-4   102.8k ± 0%   102.8k ± 0%  ~ (p=1.000 n=10+6) ¹
¹ all samples are equal

pkg: github.com/cloudflare/pint/internal/parser/source
               │   old.txt   │               new.txt                │
               │   sec/op    │   sec/op     vs base                 │
LabelsSource-4   7.194m ± 0%   7.334m ± 8%  +1.95% (p=0.022 n=10+6)

               │   old.txt    │             new.txt              │
               │     B/op     │     B/op      vs base            │
LabelsSource-4   6.086Mi ± 0%   6.085Mi ± 0%  ~ (p=0.792 n=10+6)

               │   old.txt   │             new.txt             │
               │  allocs/op  │  allocs/op   vs base            │
LabelsSource-4   48.40k ± 0%   48.40k ± 0%  ~ (p=0.300 n=10+6)

pkg: github.com/cloudflare/pint/internal/promapi
                    │   old.txt   │             new.txt             │
                    │   sec/op    │   sec/op     vs base            │
QueryCacheOnlySet-4   109.8n ± 0%   109.6n ± 1%  ~ (p=0.407 n=10+6)

                    │  old.txt   │             new.txt              │
                    │    B/op    │    B/op     vs base              │
QueryCacheOnlySet-4   64.00 ± 0%   64.00 ± 0%  ~ (p=1.000 n=10+6) ¹
¹ all samples are equal

                    │  old.txt   │             new.txt              │
                    │ allocs/op  │ allocs/op   vs base              │
QueryCacheOnlySet-4   1.000 ± 0%   1.000 ± 0%  ~ (p=1.000 n=10+6) ¹
¹ all samples are equal

github-actions Bot reviewed Jun 2, 2026

View reviewed changes

prymitive force-pushed the opts branch from 6e0b26f to bc076ac Compare June 2, 2026 14:02

github-actions Bot deleted a comment from codecov-commenter Jun 2, 2026

prymitive added 2 commits June 3, 2026 11:56

Speed up parsed rule

4394fd5

Reduce benchmark rounds

4e1e33a

prymitive force-pushed the opts branch from 0a074c9 to 4e1e33a Compare June 3, 2026 10:57

github-actions Bot deleted a comment from codecov-commenter Jun 3, 2026

prymitive merged commit 8b6dd75 into main Jun 3, 2026
20 checks passed

prymitive deleted the opts branch June 3, 2026 11:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve benchmarks#1858

Improve benchmarks#1858
prymitive merged 2 commits into
mainfrom
opts

prymitive commented Jun 2, 2026

Uh oh!

github-actions Bot left a comment •

edited

Loading

Uh oh!

codecov-commenter commented Jun 3, 2026

Uh oh!

github-actions Bot commented Jun 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

prymitive commented Jun 2, 2026

Uh oh!

github-actions Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

This pull request was validated by pint.

Uh oh!

codecov-commenter commented Jun 3, 2026

Codecov Report

Uh oh!

github-actions Bot commented Jun 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot left a comment •

edited

Loading