feat(inmemory): add Rayforce grouped and parted adapter

[singaraiona]

Summary

• add Rayforce as an optional in-memory NYSE TAQ query engine
• import the full trade and quote schemas into a native Rayforce splayed database
• implement all 84 benchmark queries
• benchmark both grouped (time) and parted (sym,time) layouts
• support query filtering, persisted correctness output, standard merged PSV results, and the current solution-selection contract

Rayforce dependency

This adapter requires the fixes merged in RayforceDB/rayforce#326. Build Rayforce from clean release objects:

cd /path/to/rayforce
make clean
make release

Set RAYFORCE_BIN when the Rayforce and benchmark repositories are not sibling directories:

export RAYFORCE_BIN=/absolute/path/to/rayforce/rayforce

Data preparation

Rayforce needs the full-schema database. Generate the KDB database first because the Rayforce converter exports its source data from the sibling kdb directory:

DATAFORMAT=kdb ./generateDB.sh \
  "${NYSEBENCHMARKDIR}/${SIZE}/psv" \
  "${NYSEBENCHMARKDIR}/${SIZE}/kdb" \
  "${DATADATE}"

DATAFORMAT=rayforce RAYFORCE_BIN=/absolute/path/to/rayforce/rayforce \
  ./generateDB.sh \
  "${NYSEBENCHMARKDIR}/${SIZE}/psv" \
  "${NYSEBENCHMARKDIR}/${SIZE}/rayforce" \
  "${DATADATE}"

Run both Rayforce layouts alone with:

./benchmarks/inmemory/queryEngines.sh \
  --db-dir "${NYSEBENCHMARKDIR}/${SIZE}" \
  --param-dir "./artifacts/parameters/${SIZE}" \
  --datadate "${DATADATE}" \
  --threads "0 4 16 64" \
  --engines rayforce \
  --solutions ALL

Validation

• the 84 Rayforce query rows align one-for-one with the current seven-column query metadata contract
• grouped and parted adapter output was verified against the KDB-X reference across the full 84-query suite before the latest upstream merge
• the conflict-resolution merge passed Bash syntax checks, Python compilation, whitespace checks on PR-owned files, and query/metadata structure checks
• the full smoke test was not rerun during conflict resolution because the proprietary q executable was unavailable in that environment

Historical 53-query small-dataset snapshot (retained only as a regression/performance reference):

• grouped, 0-secondary-thread run: KDB-X 3.584 s, Rayforce 2.880 s (1.245x total speedup; 27/53 query wins)
• parted, 0-secondary-thread run: KDB-X 2.931 s, Rayforce 2.490 s (1.177x total speedup; 33/53 query wins)

These timing figures are not final published benchmark numbers.

[ryanhamilton]

👍 🥇

[ryanhamilton]

Does this repo intend to allow outside submissions for other database?
Could we agree a set of rules applied to all submissions (kx and others) equally?