[Performance Gap] Establish large-schema SLOs, workload benchmarks, and a measured Rust boundary

[seonghobae]

Buyer-visible problem

pg-erd-cloud has focused performance improvements—background introspection, indexed queue claims, bounded parsing, memoized search, O(N) column indexing work, and proposed relationship-aware layout—but it has no published capacity envelope for a real enterprise schema. Buyers cannot tell whether 100, 1,000, or 10,000 tables will remain usable, how much memory a snapshot/export consumes, whether one large tenant starves others, or where a Rust implementation would provide meaningful leverage.

Rewriting Python by assertion would add risk without evidence. A complete product first needs a reproducible workload model and SLOs; only measured CPU/security hotspots should cross a narrow Rust/WASM/service boundary.

Product contract

Create a versioned Performance & Capacity Profile covering capture, persistence, diff, export, API, queue, and browser rendering.

Workload profiles

At minimum define and generate deterministic, anonymized profiles:

Profile	Schemas	Tables/views	Columns	FK edges	Indexes	Snapshots/project
small	5	100	2,000	200	300	20
medium	25	1,000	25,000	3,000	5,000	100
large	100	10,000	250,000	30,000	50,000	500

Add skewed cases: one 5,000-column relation, dense FK clusters, disconnected components, deep dependency chains, long quoted/multilingual identifiers, large comments, partition hierarchies, extension-owned index methods, and high snapshot/job concurrency.

Measured paths

• PostgreSQL/MySQL/Snowflake/Databricks introspection where supported;
• normalization and canonical snapshot hashing;
• snapshot JSON encode/decode and database persistence;
• schema diff and migration-plan compilation;
• DDL/DBML/Mermaid/Prisma/spec export;
• API list/detail/pagination/search;
• queue claim, retry, lease, cleanup, and fairness;
• frontend initial load, graph virtualization, search, selection, layout, zoom, export, and saved-view restore;
• Storybook/browser interaction under 200% zoom and constrained memory.

SLO/SLI output

Record p50/p95/p99 latency, throughput, peak RSS, allocations, database I/O, query count, lock wait, queue lag, browser heap, long tasks, frame time, artifact size, and cancellation time. Separate customer-facing SLOs from benchmark targets; do not claim an SLA until production evidence exists.

The first release candidate should define targets such as:

• no N+1 query growth with object count;
• bounded API response size and cursor pagination;
• cancellation produces a terminal or explicitly unknown state within a documented bound;
• large-profile backend operations never run synchronously in an HTTP request;
• UI remains operable through virtualization/progressive disclosure rather than rendering every node and column at once;
• one hot project/tenant cannot indefinitely starve smaller jobs;
• every benchmark publishes exact hardware/software/container/commit metadata.

Final numerical thresholds must be set from baseline runs, not invented in this issue.

Rust decision gate

After profiling, create an ADR for each hotspot considered for Rust. A Rust boundary is justified only when all of the following are true:

1. the path is production CPU/security critical and measured;
2. algorithm and wire/data contract are stable;
3. Python/TypeScript reference behavior and golden fixtures exist;
4. a bounded FFI/WASM/service interface avoids per-row crossing and excess context switching;
5. parity, fuzzing, memory safety, cancellation, packaging, rollback, and observability are proven;
6. the change improves a stated SLO or security property materially.

Candidate hotspots include canonical snapshot normalization/hashing, schema diff graph algorithms, identifier allocation, DBML/DDL parsing/rendering, and large relationship-aware layout. GPU work is not justified for these paths without evidence; browser graph layout may use worker/WASM before any GPU path.

Required tests and artifacts

• deterministic synthetic generator with no real customer/person/organization names;
• captured anonymized real-world shape profiles without production data values;
• benchmark regression thresholds resistant to noisy hosted runners;
• load/soak, cancellation, restart, and resource-pressure tests;
• PostgreSQL query plans and index/partition evidence;
• frontend browser traces and exact-value reports;
• flamegraphs/profiles stored as build artifacts, not committed binaries;
• CPU reference ↔ Rust parity and property/fuzz tests for any migrated kernel;
• production statement/branch coverage 100% and public API docstrings 100%.

Acceptance evidence

• docs/PERFORMANCE.md, capacity/runbook, OpenTelemetry dashboards, and buyer-facing limits;
• benchmark workflow separated from ordinary PR correctness gates but required for release candidates;
• exact-current-head benchmark manifest and reproducibility receipt;
• ADR deciding which paths remain Python/TypeScript and which, if any, move to Rust;
• docs/product-technical-gap-baseline.md, architecture, PRD/TRD, and CHANGELOG updated;
• qualifying current-head review/check evidence.

References — APA 7th

PostgreSQL Global Development Group. (2026). PostgreSQL 18 documentation: Table partitioning. https://www.postgresql.org/docs/18/ddl-partitioning.html

Gansner, E. R., Koutsofios, E., North, S. C., & Vo, K.-P. (1993). A technique for drawing directed graphs. IEEE Transactions on Software Engineering, 19(3), 214–230. https://doi.org/10.1109/32.221135

National Institute of Standards and Technology. (2022). Secure software development framework (SSDF) version 1.1 (NIST Special Publication 800-218). https://doi.org/10.6028/NIST.SP.800-218