[mikehorgan-dell]

Summary

Some storage systems reach full speed only when several buckets are active at once. spt run --bucket-count N (engine storage.targetBucket.count, 1 to 1000, default 1) spreads the objects a run creates over N buckets named <bucket>-0 to <bucket>-<N-1>.

• Placement: each new object goes to member floorMod(floorDiv(id, max(P, 1)), N), from its object id and the prefix-shard count P. Placement happens at the path layer, like --prefix-shards, so every bucket still gets every shard directory. With the default random naming every worker writes to every bucket at once. The spread is statistical, not equal at every instant.
• Identity: an existing object keeps the bucket its items file or manifest records. Reads, deletes and cleanup never derive a bucket again. The integrity manifest takes each result's bucket from its own operation path.
• Setup: the phase that writes the run's objects checks every member and creates the missing ones, as target-bucket provisioning already does for one bucket. Later phases only check.
  • A mixed step creates members only when its PUT weight is positive.
  • A mixed run from --read-items-file has no seed phase, so it creates the members in its own setup.
  • With more than one bucket, the base is never used as a member.
• Names: the index is not padded, so ps-3 is the same bucket at any count. Every generated name is checked against the AWS bucket naming rules before anything is sent.
• Drivers: s3, s3-aws and s3-rdma. Each s3 and s3-rdma worker checks each member once, with a HEAD on its first operation to that bucket.

With one bucket nothing changes: the same paths, requests and artifacts.

Changes

• Engine:
  • new TargetBucketSet (count, member paths, name rules, validation) and TargetBucketStep (what setup needs to know about mixed and pipeline steps);
  • TargetBuckets and TargetBucketProvisioning provision a step's members; the new hook LoadStepClientBase.targetBucketStep() is overridden by the mixed and pipeline clients;
  • OperationsBuilderImpl and DataOperationsBuilderImpl choose each new item's member path through nextOutputPath(item); LoadGeneratorBuilder.targetBucketCount() wires the count into the linear, pipeline, weighted and mixed steps;
  • IntegrityOperationManifestOutput records each result's own bucket;
  • schema and shipped defaults: storage.targetBucket.count: 1;
  • with a count above 1, the engine rejects these at configuration time, before any request:
    • storage.auth.file;
    • LIST, and a driver without bucket control;
    • copy steps: a CREATE with an input path or items file;
    • a mixed step with an input path;
    • a pipeline CREATE sub-step after the first;
    • expressions after the bucket in item.output.path.
• CLI:
  • --bucket-count writes the count into the generated defaults.yaml and restores it after engine overrides; an --engine-override that sets the count is rejected;
  • seeded delete with a count above 1 requires --delete-batch-size=1; read --items-file, delete --items-file, delete --delete-existing, read-verify, list, tables and mock reject a count above 1;
  • the summary shows Bucket Count: N (<first> ... <last>); spt replay keeps an archived run's count;
  • cli/tools/testmultibucket.sh [BUCKETS] (default 16) runs a short demo over a set.
• Docs: a "Bucket sets" section in cli/docs/TARGET_BUCKETS.md, SPT_SYNTAX.md and the s3 driver README, and the CHANGELOG under Unreleased › Added.

Compatibility

• Additive: one engine setting, default 1, and one CLI flag. With the default, configurations, generated scenarios, object paths, requests and result artifacts are unchanged.
• New default method LoadGeneratorBuilder.targetBucketCount() and new protected hook LoadStepClientBase.targetBucketStep(). Existing constructors are kept.

Performance: hot path, Class B

Placement adds a little work per operation in the operation builders:

• with one bucket, one field read and a predictable null branch;
• with N > 1, one offset() call, one floorMod and an array load, plus a floorDiv with prefix shards.

There is no allocation, lock, atomic or hand-off. The tolerance was declared before measuring: N = 1 within ±3% of main, and N = 16 at most 10 ns per operation over N = 1.

TargetBucketBuilderBench:

• 12 forks in ABBA order against main 11f7edc9, pinned to two cores;
• 5 s warm-up and 10 × 1 s rounds per case;
• median of forks in ns per operation; single mode keeps every operation reachable.

case	main	N = 1	Δ	N = 16, P = 0	N = 16, P = 64
ops / batch	20.55	20.99	+2.1%	21.90	23.06
ops / single	20.12	20.12	−0.05%	23.77	28.26 (+8.1)
data / batch	54.72	55.10	+0.7%	56.39	58.88
data / single	57.66	57.47	−0.3%	60.73	65.04 (+7.6)

Other benchmarks:

• ManifestWriterBench: with one bucket, 330 ns per result on the branch against 344 ns on main, with identical allocation (496.1 B per result).
• FirstUseColdStartBench: with 1000 members, the first-use HEADs overlap operations already in flight. At 1 ms per HEAD, three times the lab median, they add no measurable time. At 5 ms they add 0.85 s, against a 10 s budget.

Notes

• Some targets give every bucket its own resources. SeaweedFS refused writes into new buckets after about 18 buckets on the test lab. The docs tell users to check bucket capacity.
• Found during qualification but independent of this change:
  • spt run delete --label <name> exits 1 at the summary;
  • a distributed mixed run with --delete-items-file appears to load the whole file on every worker.