matchX Engine

A deterministic, durable, multi-symbol limit-order-book engine in Go for studying and prototyping exchange infrastructure. Integer-only prices and quantities, strict price-time priority, centralized sequencing, mmap persistence, and bounded ingress make behavior repeatable under load and after restart.

This is production-oriented infrastructure, not a certified venue. Real-money deployment still requires venue-specific compliance, independent security/correctness review, disaster-recovery drills, and operational ownership.

Verified deployment

The complete delivery path was exercised against a live Google Cloud environment in July 2026:

• Terraform provisioned a stateful Compute Engine primary in europe-west6, a dedicated persistent WAL disk, VPC and restricted firewall rules, Secret Manager secrets, monitoring dashboards, alerts, service accounts, and GitHub Workload Identity Federation.
• Terraform state is held in a versioned GCS backend. A post-deployment plan returned No changes with detailed exit code 0.
• GitHub Actions authenticated to Google Cloud without a service-account key, built immutable engine and replication images, published SBOM/provenance build records, copied the deployment helpers through IAP, and completed the deployment workflow successfully.
• The deployed matchx-engine and matchx-replica containers both returned HTTP 204 from their live health endpoints.
• Runtime Secret Manager access is scoped to individual secrets, and the deployer can impersonate only the runtime identity required for deployment.

The verified environment is intentionally a cost-conscious single-primary portfolio deployment. Terraform supports an optional warm standby, but automatic failover, leader election, and multi-region consensus are not claimed.

Expert operations track

The repository now includes a verifiable operations track around the matching kernel:

• reproducible kernel benchmarks and an end-to-end percentile load generator;
• CodeQL, dependency, container and scheduled resilience workflows;
• keyless GitHub-to-GCP delivery through Workload Identity Federation;
• Terraform for a stateful Compute Engine primary, durable WAL disk, optional warm standby, Secret Manager, private replication and monitoring;
• immutable engine and replication-sidecar images;
• forced-termination recovery, smoke tests, incident response and cost teardown runbooks.

Start with the GCP architecture, then follow the deployment guide. Performance evidence and portfolio capture are documented in docs/performance.md.

Delivery architecture

GitHub Actions
  ├─ test, vet, race and security verification
  ├─ keyless OIDC authentication
  ▼
Google Workload Identity Federation
  ▼
Artifact Registry
  ├─ immutable matchX engine image
  └─ immutable replication image
  ▼                 IAP-only administration
Compute Engine primary ─────────────────────────┐
  ├─ matchx-engine :8080                        │
  ├─ matchx-replica :9090                       │
  ├─ Secret Manager runtime credentials         │
  └─ dedicated persistent mmap WAL disk         │
                                                ▼
                                  Cloud Monitoring and alerts

The VM has no public application firewall rule. Administrative deployment uses an Identity-Aware Proxy SSH tunnel; replication traffic is restricted to the private matchX subnet. Containers use read-only root filesystems and bounded temporary filesystems while the WAL is mounted separately on durable storage.

Capabilities

Matching and sequencing

• Limit and market orders; GTC, IOC, and atomic FOK
• Maker-price execution with FIFO at each price
• Cancel-aggressor and cancel-maker self-trade prevention
• Dedicated multi-producer ingress sequencer assigning one global sequence
• Stable FNV-1a symbol routing across single-writer shards
• Preallocated intrusive order nodes and pooled command/reply envelopes
• Per-account quantity, notional, and open-order limits
• Global, symbol, and account kill switches

Persistence and availability

• 4 MiB memory-mapped WAL segments with fixed 96-byte records
• CRC-32 validation, monotonic sequence checks, atomic manifests, and fail-closed recovery
• Automatic segment rotation and configurable periodic snapshots
• Snapshot + WAL recovery preserving FIFO state
• Authenticated incremental WAL streaming and sequence-checked warm standby
• Windows and Unix mmap implementations with durable flushes

Gateways and operations

• Authenticated REST order entry using timestamped HMAC-SHA256 and nonce replay protection
• Fixed-width 112-byte binary order-entry protocol
• FIX-compatible NewOrderSingle (35=D) parser and TCP session surface
• Bearer-protected administrative kill switches
• Prometheus text metrics, health endpoint, Docker image, and CI
• Deterministic randomized conformance, concurrency, recovery, replication, corruption, and chaos tests

Start

go run ./cmd/matchx \
  -addr :8080 \
  -binary-addr :9001 \
  -data ./data \
  -symbols BTC-USD,ETH-USD \
  -shards 2 \
  -hmac-secret "$MATCHX_HMAC_SECRET" \
  -admin-token "$MATCHX_ADMIN_TOKEN"

For local evaluation, omit -hmac-secret; production deployments should always set it. Prices are integer ticks and quantities are integer lots.

REST API

Method	Path	Purpose
POST	/v1/orders	Submit an order
DELETE	/v1/books/{symbol}/orders/{id}	Cancel an order
GET	/v1/books/{symbol}?depth=20	Read depth
POST	/v1/admin/kill/{scope}/{id}?enabled=true	Toggle a kill switch
GET	/metrics	Prometheus metrics
GET	/healthz	Liveness
GET	/readyz	Readiness

Example payload:

{
  "id": 1001,
  "account_id": 42,
  "symbol": "BTC-USD",
  "side": "buy",
  "type": "limit",
  "time_in_force": "gtc",
  "self_trade_prevention": "cancel_aggressor",
  "price": 6150000,
  "quantity": 10
}

Authenticated mutations require:

• X-MatchX-Timestamp: current Unix seconds
• X-MatchX-Nonce: unique value within the replay window
• X-MatchX-Signature: hex HMAC-SHA256 over timestamp + "\n" + nonce + "\n" + rawBody

Admin calls require Authorization: Bearer <admin-token>. See docs/protocols.md for binary and FIX layouts.

Architecture

HTTP / binary / FIX producers
             │
             ▼
   pooled ingress envelopes
             │
             ▼
 global deterministic sequencer
       │ stable symbol hash
       ├──────────────┬──────────────┐
       ▼              ▼              ▼
 shard 0          shard 1          shard N
 single writer    single writer    single writer
 books + risk     books + risk     books + risk
       │              │              │
       ▼              ▼              ▼
 mmap WAL + periodic snapshot + replication stream

A shard owns every mutation to its books. No book lock participates in matching. The sequencer establishes global command order; each shard preserves the subsequence routed to it. Commands are persisted and synchronously flushed before mutation. Snapshots contain FIFO-ordered resting orders; replay skips records already represented by the snapshot and rejects sequence regression, corruption, or truncated segments.

Verification

go test ./...
go vet -unsafeptr=false ./...
go test -race ./...                 # requires CGO
GOOS=linux GOARCH=amd64 go build ./...
go test -bench=. -benchmem ./internal/book

The suite includes deterministic property tests across generated order streams, simultaneous producers, multi-symbol restart equivalence, STP conformance, authentication replay rejection, standby replication, checksum bit flips, and WAL truncation. CI runs formatting, vet, race tests, and benchmarks on Linux.

Benchmarks isolate the in-memory book unless explicitly labeled otherwise. WAL fsync, network, gateway parsing, and replication materially change latency. Always publish hardware, OS, Go version, workload mix, durability mode, and latency percentiles with throughput claims.

Infrastructure verification

cd infra/terraform
terraform fmt -check -recursive
terraform validate
terraform plan -detailed-exitcode

A detailed exit code of 0 confirms that the managed cloud resources match the configuration. The deployment workflow is manually dispatched against the protected gcp-production GitHub environment so infrastructure changes do not silently deploy from ordinary commits.

Runtime verification

On the provisioned host:

docker ps
curl -i http://127.0.0.1:8080/healthz
curl -i http://127.0.0.1:9090/healthz

The expected response from both services is HTTP/1.1 204 No Content.

Durability and failure model

WAL data is written into preallocated mmap segments. Sync flushes the view and file before the command mutates its book. The atomic manifest records active segment, byte offset, and last sequence. Segments rotate automatically. A periodic snapshot is written to a temporary file and atomically renamed. Recovery loads the latest snapshot, replays newer records, validates CRC and monotonic sequences, and fails closed on damage.

The warm standby pulls complete WAL records from a bearer-authenticated primary endpoint and rejects non-monotonic input. Promotion, quorum consensus, automatic leader election, fencing, and cross-region orchestration are deliberately operator responsibilities; this is deterministic primary/standby replication, not a distributed consensus system.

Operational cautions

• The binary protocol is intentionally small and versioned, but still needs TLS at the load balancer or listener wrapper.
• The FIX surface implements the order-entry subset, not full FIX session sequencing, resend, or certification behavior.
• Slow market-data fanout, fee models, margin, settlement, surveillance, and regulatory reporting are outside the matching kernel.
• The response envelope is pooled after completion; trade result slices and external protocol parsing may still allocate.
• Symbols longer than 32 bytes cannot be represented in the current WAL record.
• The demonstrated GCP topology is a single-zone portfolio environment, not a claim of exchange-grade geographic redundancy.
• A stopped Compute Engine VM still retains billable persistent storage.

Completed roadmap

• Multi-producer ingress sequencer and pooled response ownership
• mmap segmented WAL, snapshots, rotation, and sequence-aware recovery
• Multi-symbol deterministic shard coordinator
• Binary/FIX surfaces, authentication, limits, STP, and kill switches
• Warm standby replication, conformance/property tests, metrics, and chaos tests

License

MIT