Skip to content

feat(bench): add public mixed workload preset - #7548

Merged
shekhirin merged 3 commits into
mainfrom
centaur/add-public-mix-preset-1788877953
Sep 8, 2026
Merged

shekhirin merged 3 commits into
mainfrom
centaur/add-public-mix-preset-1788877953

Conversation

@decofe

@decofe decofe commented Sep 8, 2026

Copy link
Copy Markdown
Member

Adds public-mix.yml with plain TIP-20 transfers, memo transfers, mints, and native MPP opens in relative submitted-transaction shares of 25:40:1:15. Renames the shared ABI to tip20.abi.json, adds transferWithMemo, and updates its preset references.

MPP uses an ordinary single-step open sequence supported by txgen main. A TODO links txgen #200 for adding signed settlement later. The multi-region runner separately needs public-mix added to its preset allowlist.

Validation: checked the open-only sequence against txgen main's sequence schema, removal of settlement/save fields, preserved transaction shares, ABI arguments, references, and the linked TODO. Nushell resolution for both presets and git diff --check passed. The earlier 60-second benchmark passed with settlement enabled; this open-only revision has not been benchmarked again.

Prompted by: @shekhirin

@decofe
decofe force-pushed the centaur/add-public-mix-preset-1788877953 branch 2 times, most recently from 6dfa336 to 53d6897 Compare September 8, 2026 14:42
Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
@decofe
decofe force-pushed the centaur/add-public-mix-preset-1788877953 branch from 53d6897 to 09a0efb Compare September 8, 2026 14:43
Use the existing preset from Tempo #7546 so public-mix resolves on this branch.

Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
@tempo-github-sts

tempo-github-sts Bot commented Sep 8, 2026

Copy link
Copy Markdown

cc @decofe

⚪ Benchmark complete: No Difference View job

⚪ Bench Comparison: No Difference

Refs: 07761a7 vs centaur/add-public-mix-preset-1788877953
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=public-mix duration=60 bloat=1 token-count=4 tps=1000 accounts=100 max-concurrent-requests=100 baseline=07761a78a4ac00988533aa8acbcb6667786b625d feature=centaur/add-public-mix-preset-1788877953 baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=1 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1000000000000 txgen-ref=57a891b99bc8a07b44bc0a67b2d98000c256e00c
  • Bloat: 1000 MiB
  • Token count: 4
  • Preset: public-mix
  • Target TPS: 1000
  • Duration: 60s
  • Run pairs: 1
  • Baseline blocks: 117
  • Feature blocks: 117

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 993 992 -0.10% ⚪
Gas Throughput [Mgas/s] 78.9 78.8 -0.13% ⚪
Block Time Mean [ms] 505.4 505.8 +0.08% ⚪
Block Time P50 [ms] 506.0 507.0 +0.20% ⚪
Block Time P90 [ms] 509.0 509.0 +0.00% ⚪
Block Time P99 [ms] 512.0 511.0 -0.20% ⚪

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 82.4 82.4 -0.03% ⚪
P50 [ms] 489.2 489.2 +0.00% ⚪
P90 [ms] 489.8 489.9 +0.02% ⚪
P99 [ms] 490.4 490.2 -0.04% ⚪
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 0.3 0.3 +0.00%
Finish P90 [ms] 0.3 0.3 +0.00%
Finish P99 [ms] 0.4 0.4 +0.00%
Pool Fetch P50 [ms] 0.0 0.0 0.00%
Pool Fetch P90 [ms] 0.1 0.1 +0.00%
Pool Fetch P99 [ms] 0.2 0.2 +0.00%
Stop Reason — Build Budget 115 115 +0.00%
Serialized Block Size P50 [KiB] 155.0 155.0 -0.01%
Serialized Block Size P90 [KiB] 158.1 158.2 +0.08%
Serialized Block Size P99 [KiB] 162.4 162.1 -0.17%
Serialized Block Size / Tx P50 [B/tx] 313.5 313.9 +0.13%
Serialized Block Size / Tx P90 [B/tx] 320.2 319.5 -0.22%
Serialized Block Size / Tx P99 [B/tx] 327.9 325.4 -0.76%
Fill Idle P50 [ms] 454.0 454.0 +0.00%
Fill Idle P90 [ms] 456.0 456.0 +0.00%
Fill Idle P99 [ms] 456.0 456.0 +0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 4522.8 4487.3 -0.78% ⚪
P50 [ms] 8.8 9.0 +2.27% ⚪
P90 [ms] 9.8 9.8 +0.00% ⚪
P99 [ms] 11.0 11.1 +0.91% ⚪

Observability

Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
@shekhirin

Copy link
Copy Markdown
Member

derek bench preset=public-mix run-pairs=1 run-side=feature duration=60

@tempo-github-sts

tempo-github-sts Bot commented Sep 8, 2026

Copy link
Copy Markdown

cc @shekhirin

⚪ Benchmark complete: No Difference View job

⚪ Feature Bench: No Difference

Refs: 07761a7 vs centaur/add-public-mix-preset-1788877953
Criteria: Feature-only run; baseline columns are intentionally empty.

Configuration

  • Derek command: derek bench mode=e2e preset=public-mix duration=60 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=500 baseline=07761a78a4ac00988533aa8acbcb6667786b625d feature=centaur/add-public-mix-preset-1788877953 baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=1 run-side=feature otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1000000000000
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: public-mix
  • Target TPS: 50000
  • Duration: 60s
  • Run pairs: 1
  • Run side: feature
  • Baseline blocks: 0
  • Feature blocks: 110

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 0 13510 +0.00% ⚪
Gas Throughput [Mgas/s] 0.0 1386.4 +0.00% ⚪
Block Time Mean [ms] 0.0 531.3 +0.00% ⚪
Block Time P50 [ms] 0.0 530.0 +0.00% ⚪
Block Time P90 [ms] 0.0 588.0 +0.00% ⚪
Block Time P99 [ms] 0.0 625.0 +0.00% ⚪

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 0.0 2538.6 +0.00% ⚪
P50 [ms] 0.0 301.0 +0.00% ⚪
P90 [ms] 0.0 337.9 +0.00% ⚪
P99 [ms] 0.0 361.9 +0.00% ⚪
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 0.0 5.0 n/a
Finish P90 [ms] 0.0 11.8 n/a
Finish P99 [ms] 0.0 17.6 n/a
Pool Fetch P50 [ms] 0.0 9.4 n/a
Pool Fetch P90 [ms] 0.0 14.6 n/a
Pool Fetch P99 [ms] 0.0 19.4 n/a
Invalid Tx Skips 0 265 n/a
Stop Reason — Build Budget 0 109 n/a
Serialized Block Size P50 [KiB] 0.0 2034.9 n/a
Serialized Block Size P90 [KiB] 0.0 2564.2 n/a
Serialized Block Size P99 [KiB] 0.0 2810.2 n/a
Serialized Block Size / Tx P50 [B/tx] 0.0 289.5 n/a
Serialized Block Size / Tx P90 [B/tx] 0.0 290.2 n/a
Serialized Block Size / Tx P99 [B/tx] 0.0 290.6 n/a
Fill Idle P50 [ms] 0.0 125.0 n/a
Fill Idle P90 [ms] 0.0 201.0 n/a
Fill Idle P99 [ms] 0.0 236.0 n/a

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 0.0 4951.0 +0.00% ⚪
P50 [ms] 0.0 145.5 +0.00% ⚪
P90 [ms] 0.0 199.1 +0.00% ⚪
P99 [ms] 0.0 244.4 +0.00% ⚪

Observability

@shekhirin
shekhirin marked this pull request as ready for review September 8, 2026 15:46

@mattsse mattsse left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@tempo-voight-kampff

Copy link
Copy Markdown

Hi @mattsse — your review approval was detected by Voight-Kampff, but another unexpired approval request is already waiting for your account. Your review did not trigger an approval prompt and is not counted toward this PR.

Wait for the current prompt to be approved, rejected, or expire, then re-submit the review to send a fresh approval request.

@shekhirin
shekhirin enabled auto-merge September 8, 2026 16:43
decofe added a commit that referenced this pull request Sep 9, 2026
Adds `public-mix.yml` with plain TIP-20 transfers, memo transfers,
mints, and native MPP opens in relative submitted-transaction shares of
25:40:1:15. Renames the shared ABI to `tip20.abi.json`, adds
`transferWithMemo`, and updates its preset references.

MPP uses an ordinary single-step open sequence supported by txgen main.
A TODO links [txgen #200](tempoxyz/txgen#200)
for adding signed settlement later. The multi-region runner separately
needs `public-mix` added to its preset allowlist.

Validation: checked the open-only sequence against txgen main's sequence
schema, removal of settlement/save fields, preserved transaction shares,
ABI arguments, references, and the linked TODO. Nushell resolution for
both presets and `git diff --check` passed. The [earlier 60-second
benchmark](https://github.com/tempoxyz/tempo/actions/runs/34240689460)
passed with settlement enabled; this open-only revision has not been
benchmarked again.

Prompted by: @shekhirin

---------

Co-authored-by: Alexey Shekhirin <5773434+shekhirin@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants