Skip to content

fix: derive benchmark Throughput and Mean time per paragraph from ns/op - #57

Merged
nicobistolfi merged 1 commit into
mainfrom
vigilante/issue-55-benchmark-pr-comment-always-shows-n-a-for-throughput-and-mean-time-per-paragraph
Aug 11, 2026
Merged

fix: derive benchmark Throughput and Mean time per paragraph from ns/op#57
nicobistolfi merged 1 commit into
mainfrom
vigilante/issue-55-benchmark-pr-comment-always-shows-n-a-for-throughput-and-mean-time-per-paragraph

Conversation

@nicobistolfi

Copy link
Copy Markdown
Contributor

Summary

  • .github/workflows/benchmark-profile.yml's Comment PR with results and Generate summary steps matched literal Throughput: and Mean time per paragraph: text that plain go test -bench -benchmem output never contains, so both fields always fell back to N/A (only ns/op matched and showed real data).
  • Both steps now derive Throughput (1e9 / ns_per_op paragraphs/second) and a human-readable Mean time per paragraph directly from the ns/op value that was already being parsed successfully, falling back to N/A only when ns/op itself can't be parsed (avoiding NaN/Infinity).
  • No Go code, benchmark functions, or profiling/artifact steps were changed — this is a CI reporting fix scoped entirely to the workflow file.

Closes #55

Test plan

  • actionlint on the modified workflow file — no findings.
  • Validated the workflow YAML parses correctly.
  • Extracted the JS derivation logic (github-script step) into a standalone Node.js snippet and ran it against sample go test -bench output — confirmed real Throughput/Mean-time values are produced and are mathematically consistent with ns/op, and confirmed fallback to N/A (no NaN/Infinity) when ns/op can't be parsed.
  • Extracted the bash derivation logic (Generate summary step) and ran it against the same sample output with awk — confirmed matching real values.
  • go build ./... and go vet ./... — unaffected, pass.
  • Verify on an actual PR run of benchmark-profile.yml that the posted comment and $GITHUB_STEP_SUMMARY show real, non-N/A values (requires this PR itself to trigger the workflow).

The Comment PR with results and Generate summary steps in
benchmark-profile.yml matched literal "Throughput:" and "Mean time per
paragraph:" text that go test -bench never prints, so both fields always
fell back to N/A. Derive them instead from the ns/op value that is
already parsed successfully, falling back to N/A only when ns/op itself
can't be parsed.

Closes #55
@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d3d965af-3590-4d4b-abef-eed3acb76bfd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@sonarqubecloud

Copy link
Copy Markdown

@codecov

codecov Bot commented Aug 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions

Copy link
Copy Markdown

📊 Benchmark & Profile Results

Performance Summary

  • Throughput: 2316.14 paragraphs/second
  • Mean time per paragraph: 431.75 µs
  • Nanoseconds per operation: 431753 ns/op

🔍 Interactive Profile Visualizations

View the interactive profile graphs directly in your browser:

📈 View CPU Profile →

  • Download the artifact benchmark-profiles-47072f5ec6cafeba5bc316bdaa90b5637bd83654
  • Open cpu_profile.html in your browser

💾 View Memory Profile →

  • Download the artifact benchmark-profiles-47072f5ec6cafeba5bc316bdaa90b5637bd83654
  • Open memory_profile.html in your browser

CPU Profile (Top 20)

File: deidentify.test
Build ID: f1b10fe871506f484c4d7c44a8c8012b97f0043d
Type: cpu
Time: 2026-08-11 17:47:16 UTC
Duration: 78.20s, Total samples = 192.06s (245.59%)
Showing nodes accounting for 185.11s, 96.38% of 192.06s total
Dropped 266 nodes (cum <= 0.96s)
Showing top 20 nodes out of 50
      flat  flat%   sum%        cum   cum%
    48.92s 25.47% 25.47%     57.82s 30.11%  regexp.(*machine).add
    30.39s 15.82% 41.29%     65.77s 34.24%  regexp.(*Regexp).tryBacktrack
       28s 14.58% 55.87%     72.82s 37.92%  regexp.(*machine).step
    22.55s 11.74% 67.61%     40.91s 21.30%  regexp/syntax.(*Inst).MatchRunePos
    18.27s  9.51% 77.13%     18.36s  9.56%  unicode.SimpleFold
     9.54s  4.97% 82.09%      9.55s  4.97%  regexp.(*bitState).shouldVisit (inline)
     4.90s  2.55% 84.65%      5.2...

Memory Profile (Top 20)

File: deidentify.test
Build ID: f1b10fe871506f484c4d7c44a8c8012b97f0043d
Type: alloc_space
Time: 2026-08-11 17:49:05 UTC
Showing nodes accounting for 1089.36MB, 99.13% of 1098.86MB total
Dropped 52 nodes (cum <= 5.49MB)
Showing top 20 nodes out of 36
      flat  flat%   sum%        cum   cum%
  347.09MB 31.59% 31.59%   791.27MB 72.01%  regexp.(*Regexp).replaceAll
  304.09MB 27.67% 59.26%  1095.36MB 99.68%  regexp.(*Regexp).ReplaceAllStringFunc
  244.29MB 22.23% 81.49%   244.29MB 22.23%  regexp.(*Regexp).get
  147.04MB 13.38% 94.87%   175.40MB 15.96%  regexp.(*Regexp).ReplaceAllStringFunc.func1
   27.50MB  2.50% 97.37%    27.50MB  2.50%  regexp.(*machine).alloc (inline)
   14.85MB  1.35% 98.73%    14.85MB  1.35%  regexp.(*bitState).reset
    2.50MB  0.23% 98.95%    28.51MB  2.59%  regexp.(*...

📥 All Artifacts Available

Download the complete profiling package from the workflow artifacts:

  • 🌐 HTML Reports: cpu_profile.html, memory_profile.html (interactive visualizations)
  • 📊 Graphs: CPU & Memory SVG/PNG files
  • 📝 Text Reports: Full profiling data in text format
  • 🎯 Focused Analysis: Deidentify-specific function profiling
  • 📋 Raw Profiles: .prof files for custom analysis
Full Benchmark Output
goos: linux
goarch: amd64
pkg: github.com/aliengiraffe/deidentify
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkParagraphDeidentification-4           	   83028	    431753 ns/op	   10595 B/op	      48 allocs/op
BenchmarkParagraphDeidentificationParallel-4   	  182320	    197085 ns/op	    8691 B/op	      47 allocs/op
PASS
ok  	github.com/aliengiraffe/deidentify	78.222s
goos: linux
goarch: amd64
pkg: github.com/aliengiraffe/deidentify
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkParagraphDeidentification-4           	   27666	    434824 ns/op
BenchmarkParagraphDeidentificationParallel-4   	   60501	    203020 ns/op
PASS
ok  	github.com/aliengiraffe/deidentify	30.747s
goos: linux
goarch: amd64
pkg: github.com/aliengiraffe/deidentify
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkParagraphDeidentificationParallel-4   	   58497	    201363 ns/op
PASS
ok  	github.com/aliengiraffe/deidentify	13.894s


Generated by Benchmark & Profile workflow run #31519210775 • Updated: Tue, 11 Aug 2026 17:49:34 GMT

@aotarola aotarola left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💚

@nicobistolfi
nicobistolfi merged commit edb936e into main Aug 11, 2026
9 checks passed
@nicobistolfi
nicobistolfi deleted the vigilante/issue-55-benchmark-pr-comment-always-shows-n-a-for-throughput-and-mean-time-per-paragraph branch August 11, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Benchmark PR comment always shows N/A for Throughput and Mean time per paragraph

2 participants