Skip to content

Add QBS versus generated C benchmark - #66

Merged
ip332 merged 1 commit into
mainfrom
benchmarks/qbs-vs-generated-c
Sep 16, 2026
Merged

ip332 merged 1 commit into
mainfrom
benchmarks/qbs-vs-generated-c

Conversation

@ip332

@ip332 ip332 commented Sep 16, 2026

Copy link
Copy Markdown
Owner

Summary

Add a same-schema benchmark comparing generated C with QBS-driven generic C. Both paths use benchmarks/schemas/workload.brd; QBS is generated at build time through quarry-schema-compiler --emit-qbs, and one initialized logical workload feeds both adapters. Correctness gates run before timing.

Correctness

  • Deterministic QBS generation: PASS
  • QBS image: 881 bytes
  • BRF payload: 252 bytes
  • Generated-C/generic-C BRF byte equality: PASS
  • Generated checksum: 711237
  • Generic checksum: 711237
  • Public get_value() -> ARRAY -> record_array_get() path: PASS

Performance baseline

Metric Generated C QBS-driven C
Encode 459.7 ns/op 1,472.7 ns/op
Encode throughput 2.18M ops/s 679K ops/s
Post-validation access 94.3 ns/op 265.0 ns/op
Access throughput 10.6M ops/s 3.77M ops/s

QBS parse: 355 ns/parse (2.82M parses/s). Ratios are approximately 3.2x for encode and 2.8x for post-validation access. These are optimized local measurements, not universal performance claims.

Schema-artifact footprint

Artifact Size
QBS image 881 bytes
Generated headers 5,060 bytes
Generated C source 41,195 bytes
Root generated object 9,392 bytes
Imported generated object 4,072 bytes
Combined generated objects 13,464 bytes

The 15.3x object/QBS ratio compares schema-specific artifacts only; it is not a complete application-footprint ratio. Both approaches require runtime/support code.

Generated object details: root .text 8,013 bytes/read-only 376 bytes; imported .text 3,485 bytes/read-only 108 bytes. Source/header counts are build-artifact measurements, not executable footprint.

Memory and integration

Configured parser/validation workspace is 13,056 bytes; exact consumed minimum is not exposed by the public API. Shared BRF buffers are treated consistently. The QBS comparison is integrated into the common benchmark runner, and generated object sections are printed through llvm-size when available.

Validation

Optimized benchmark build/run, common runner, strict C99 benchmark build, BRF equality, checksum equality, complete CTest (52/52), and git diff --check all pass.

Compatibility

Benchmark-only change. No production compiler/runtime, API/ABI, BRF, QBS, Schema IR, or compatibility epoch changes.

Deferred work

Embedded footprint, exact incremental generic-runtime attribution, multi-schema scaling, and production performance optimization are deferred. This PR establishes the pre-optimization baseline.

@ip332
ip332 merged commit 9abe679 into main Sep 16, 2026
5 checks passed
@ip332
ip332 deleted the benchmarks/qbs-vs-generated-c branch September 16, 2026 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant