Skip to content

future-util: opts#31061

Open
StephanDollberg wants to merge 3 commits into
devfrom
stephan/future-util
Open

future-util: opts#31061
StephanDollberg wants to merge 3 commits into
devfrom
stephan/future-util

Conversation

@StephanDollberg

Copy link
Copy Markdown
Member

Remove task/scheduler-roundtrips and allocs from future-utils.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v26.1.x
  • v25.3.x
  • v25.2.x

Release Notes

  • none

Copilot AI review requested due to automatic review settings July 9, 2026 20:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes ssx future utility helpers to reduce overhead (scheduler round-trips / allocations) and adds a dedicated rpbench benchmark to measure ignore_shutdown_exceptions behavior across common success/failure paths.

Changes:

  • Reworks ssx::ignore_shutdown_exceptions to fast-path already-ready successful futures and use a shared shutdown-exception filter.
  • Reimplements ssx::spawn_with_gate_then to avoid try_with_gate and filter shutdown exceptions in a then_wrapped continuation.
  • Adds future_util_rpbench benchmark to exercise ignore_shutdown_exceptions with ready-success, shutdown-failure, and non-shutdown-failure inputs.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
src/v/ssx/future-util.h Optimizes shutdown-exception filtering and gate-spawn helper to reduce overhead on common paths.
src/v/ssx/tests/future_util_bench.cc Adds perf benchmarks focused on ignore_shutdown_exceptions fast path and exception filtering.
src/v/ssx/tests/BUILD Registers a new rpbench target for the future-util benchmark.

Comment thread src/v/ssx/future-util.h
Comment thread src/v/ssx/future-util.h

@dotnwat dotnwat left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how much goose value does this add?

Cover already-resolved futures for success, gate closure, abort, and
non-shutdown propagation. Keep producers out of line so callers cannot
specialize for a known future state.

Release/Bazel baseline (inst / tasks / allocs):
  ready             289 / 0 / 1
  gate closed    22.87K / 0 / 3
  abort          22.56K / 0 / 3
  non-shutdown   24.44K / 0 / 4
Return already-successful futures directly. Handle failures with one
continuation that classifies once. Shutdown errors are swallowed; other
errors are returned in a failed future. This removes the coroutine frame
and a second unwind.

Release/Bazel deltas vs b0d4b296:
  produce 1 KiB:
    inst 56.13K -> 55.06K (-1.07K, -1.9%)
    tasks 34 -> 34 (0, 0.0%); allocs 94 -> 90 (-4, -4.3%)
  raft quorum ack, 16 B:
    inst 159.82K -> 157.84K (-1.98K, -1.2%)
    tasks 105 -> 105 (0, 0.0%); allocs 265 -> 254 (-11, -4.2%)
  ready:
    inst 289 -> 85 (-204, -70.5%)
    tasks 0 -> 0 (0, 0.0%); allocs 1 -> 0 (-1, -100.0%)
  gate closed:
    inst 22.87K -> 15.50K (-7.37K, -32.2%)
    tasks 0 -> 0 (0, 0.0%); allocs 3 -> 2 (-1, -33.3%)
  abort requested:
    inst 22.56K -> 15.19K (-7.37K, -32.7%)
    tasks 0 -> 0 (0, 0.0%); allocs 3 -> 2 (-1, -33.3%)
  non-shutdown:
    inst 24.44K -> 17.30K (-7.14K, -29.2%)
    tasks 0 -> 0 (0, 0.0%); allocs 4 -> 3 (-1, -25.0%)
Combine gate release and exception filtering in one then_wrapped
continuation. Return immediately when the gate is closed, and reuse the
shared shutdown filter on failure. Each successful call saves one task
and one allocation.

Release/Bazel deltas vs c23ee68:
  produce 1 KiB:
    inst 55.45K -> 55.30K (-150, -0.3%, noise)
    tasks 34 -> 33 (-1, -2.9%); allocs 90 -> 89 (-1, -1.1%)
  raft quorum ack, 16 B:
    inst 157.82K -> 157.10K (-720, -0.5%, noise)
    tasks 105 -> 100 (-5, -4.8%); allocs 253 -> 249 (-4, -1.6%)
  future_util_rpbench:
    all four cases unchanged in inst, tasks, and allocs
@vbotbuildovich

Copy link
Copy Markdown
Collaborator

CI test results

test results on build#86940
test_status test_class test_method test_arguments test_kind job_url passed reason test_history
FLAKY(PASS) ShadowLinkConsumeGroupsMirroringTest test_continuous_group_sync {"source_cluster_spec": {"cluster_type": "redpanda"}, "storage_mode": "cloud", "with_failures": true} integration https://buildkite.com/redpanda/redpanda/builds/86940#019f48ed-2df1-4048-88f5-5f0e75a85bc6 10/11 Test PASSES after retries.No significant increase in flaky rate(baseline=0.0000, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1000, p1=0.3487, trust_threshold=0.5000) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowLinkConsumeGroupsMirroringTest&test_method=test_continuous_group_sync

@StephanDollberg

Copy link
Copy Markdown
Member Author

/microbench

@vbotbuildovich

Copy link
Copy Markdown
Collaborator

Performance change detected in https://buildkite.com/redpanda/redpanda/builds/86950#019f4af0-35d2-41e5-8ac5-1b869c3d5e26:

Performance changes detected in 60 tests
cloud_storage_rpbench.cstore_bench.cs_iteration_recompute_end_test_1000: inst -> +0.05pct
record_multiplexer_rpbench.record_multiplexer_bench_fixture.avro_linear_1_field_large: inst -> -0.02pct
record_multiplexer_rpbench.record_multiplexer_bench_fixture.avro_linear_1_field_small: inst -> -0.02pct
record_multiplexer_rpbench.record_multiplexer_bench_fixture.avro_linear_1_field_small_zstd: inst -> -0.02pct
record_multiplexer_rpbench.record_multiplexer_bench_fixture.json_linear_1_field_large: inst -> -0.01pct
record_multiplexer_rpbench.record_multiplexer_bench_fixture.json_linear_1_field_large_zstd: inst -> -0.01pct
record_multiplexer_rpbench.record_multiplexer_bench_fixture.json_linear_1_field_small: inst -> -0.02pct
record_multiplexer_rpbench.record_multiplexer_bench_fixture.json_linear_1_field_small_zstd: inst -> -0.01pct
record_multiplexer_rpbench.record_multiplexer_bench_fixture.protobuf_linear_1_field_large: inst -> -0.02pct
record_multiplexer_rpbench.record_multiplexer_bench_fixture.protobuf_linear_1_field_small: inst -> -0.02pct
record_multiplexer_rpbench.record_multiplexer_bench_fixture.protobuf_linear_1_field_small_zstd: inst -> -0.02pct
kafka_produce_partition_rpbench.produce_partition_fixture.1_KiB_dispatched: allocs -> -4.78pct
kafka_produce_partition_rpbench.produce_partition_fixture.1_KiB_dispatched: inst -> -1.89pct
kafka_produce_partition_rpbench.produce_partition_fixture.1_KiB_submitted: allocs -> -4.79pct
kafka_produce_partition_rpbench.produce_partition_fixture.1_KiB_submitted: inst -> -2.43pct
kafka_produce_partition_rpbench.produce_partition_fixture.1_dispatched: allocs -> -4.84pct
kafka_produce_partition_rpbench.produce_partition_fixture.1_dispatched: inst -> -2.42pct
kafka_produce_partition_rpbench.produce_partition_fixture.1_submitted: allocs -> -4.93pct
kafka_produce_partition_rpbench.produce_partition_fixture.1_submitted: inst -> -2.30pct
kafka_produce_partition_rpbench.produce_partition_fixture.4_KiB_dispatched: allocs -> -4.62pct
kafka_produce_partition_rpbench.produce_partition_fixture.4_KiB_dispatched: inst -> -2.17pct
kafka_produce_partition_rpbench.produce_partition_fixture.4_KiB_submitted: allocs -> -4.79pct
kafka_produce_partition_rpbench.produce_partition_fixture.4_KiB_submitted: inst -> -1.97pct
kafka_produce_partition_rpbench.produce_partition_fixture.8_KiB_dispatched: allocs -> -4.44pct
kafka_produce_partition_rpbench.produce_partition_fixture.8_KiB_dispatched: inst -> -1.45pct
kafka_produce_partition_rpbench.produce_partition_fixture.8_KiB_submitted: allocs -> -4.46pct
kafka_produce_partition_rpbench.produce_partition_fixture.8_KiB_submitted: inst -> -1.57pct
raft_replicate_rpbench.raft_benchmark_fixture.quorum_ack_enqueue_16B: allocs -> -4.73pct
raft_replicate_rpbench.raft_benchmark_fixture.quorum_ack_enqueue_16B: inst -> -2.11pct
raft_replicate_rpbench.raft_benchmark_fixture.quorum_ack_enqueue_16KiB: allocs -> -1.92pct
segment_appender_rpbench.appender_fixture.128_KiB_writes_flush: allocs -> -0.63pct
segment_appender_rpbench.appender_fixture.128_KiB_writes_flush: inst -> -0.12pct
segment_appender_rpbench.appender_fixture.1_byte_writes_flush: allocs -> -12.49pct
segment_appender_rpbench.appender_fixture.1_byte_writes_flush: inst -> -4.19pct
segment_appender_rpbench.appender_fixture.1_byte_writes_no_flush: allocs -> -12.49pct
segment_appender_rpbench.appender_fixture.1_byte_writes_no_flush: inst -> -4.23pct
segment_appender_rpbench.appender_fixture.4096_byte_writes_flush: allocs -> -6.64pct
segment_appender_rpbench.appender_fixture.4096_byte_writes_flush: inst -> -1.70pct
segment_appender_rpbench.appender_fixture.97_byte_writes_flush: allocs -> -11.82pct
segment_appender_rpbench.appender_fixture.97_byte_writes_flush: inst -> -3.87pct
read_fanout_rpbench.read_fanout_bench.baseline: allocs -> -11.04pct
read_fanout_rpbench.read_fanout_bench.baseline: inst -> -1.32pct
read_fanout_rpbench.read_fanout_bench.baseline_noop: allocs -> -16.53pct
read_fanout_rpbench.read_fanout_bench.baseline_noop: inst -> -6.00pct
read_fanout_rpbench.read_fanout_bench.bypass: allocs -> -18.05pct
read_fanout_rpbench.read_fanout_bench.bypass: inst -> -2.05pct
read_fanout_rpbench.read_fanout_bench.concurrent: allocs -> -6.10pct
read_fanout_rpbench.read_fanout_bench.concurrent: inst -> -1.22pct
read_fanout_rpbench.read_fanout_bench.serialized: allocs -> -0.24pct
read_fanout_rpbench.read_fanout_bench.serialized_noop: allocs -> -24.75pct
read_fanout_rpbench.read_fanout_bench.serialized_noop: inst -> -8.75pct
read_fanout_rpbench.read_fanout_bench.vectorized_noop: allocs -> -7.58pct
read_fanout_rpbench.read_fanout_bench.vectorized_noop: inst -> -2.89pct
read_merge_rpbench.read_merge_bench.baseline: allocs -> -6.67pct
read_merge_rpbench.read_merge_bench.baseline: inst -> -2.15pct
read_merge_rpbench.read_merge_bench.concurrent_same_object: allocs -> -10.42pct
read_merge_rpbench.read_merge_bench.concurrent_same_object: inst -> -0.75pct
read_merge_rpbench.read_merge_bench.concurrent_unique_objects: allocs -> -8.01pct
read_merge_rpbench.read_merge_bench.serial_unique_objects: allocs -> -7.14pct
read_merge_rpbench.read_merge_bench.serial_unique_objects: inst -> -2.09pct

See https://redpandadata.atlassian.net/wiki/x/LQAqLg for docs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants