docs: refresh README to reflect built system + Mantle on-chain - #55
Open
loficoded wants to merge 98 commits into
Open
docs: refresh README to reflect built system + Mantle on-chain#55loficoded wants to merge 98 commits into
loficoded wants to merge 98 commits into
Conversation
Foundation for Vector (Next.js App Router + TS strict, Bun toolchain):
- Single source of truth: lib/config/constants.ts — every scoring/routing/
timing/signal/policy/capital/chain constant, zod-validated at load and deeply
frozen (mutation throws; invalid values crash startup).
- Server-only env: zod schema + pure parser (env.schema.ts) with redacted,
value-free error messages; eager server-only entry (env.ts). Secrets never
reach the client bundle.
- Neon client: server-only pool singleton; checkDb() is a total function that
collapses every failure (refused/TLS/disconnect/timeout) to 'down'.
- /api/health: real SELECT 1, returns { ok, db, config_loaded, commit }
(200 up / 503 down). Pure formatter in lib/health.ts.
- SWR provider polling at the single ui_poll_ms cadence (no sockets).
- Tests (~10% happy / ~90% edge): unit, fuzz (seeded), integration (gated on
DATABASE_URL), e2e single-source invariant. 54 pass.
- Docs: config.md, env.md, ADR 0001; .env.example; README.
Verified: tsc --noEmit, eslint, prettier --check, bun test, next build — all clean.
P0.1 — App Skeleton & Seeded Config
What
- Full §7.1 schema as SQL DDL (lib/db/migrations/0001_*): agents, rounds,
intents, policy_events, executions, outcomes, scores, capital_allocations,
attestations, kill_switch — uuid PKs, timestamptz, numeric-only money/scores.
- Enum domains named separately (agents.status vs executions.status; etc.).
intents.action {open,close,modify,transfer}; capital_allocations.trigger
widened to {settle,attestation,crash,operator} per §6.2.
- Invariants in SQL: kill_switch singleton (id=1 + CHECK); attestations unique
(agent_id, round_id); intents.target_address only on transfer; outcomes.
execution_id nullable (seeded arc); value bounded to int128, value_decimals
to uint8; feedback_hash/tx_hash hex-format checks; weights/CaR/fees ranges.
All FKs ON DELETE RESTRICT (no dangling rows). Read-path indexes for P1.5.
- Migration runner (lib/db/migrate.ts): paired up/down SQL, schema_migrations
ledger, per-migration transaction (atomic), pg_advisory_lock (serializes
concurrent runners), idempotent. No ORM (see ADR 0002).
- Typed repository layer (lib/db/repos/*): parameterized insert/select per
table over an injected Queryable; zod row validation; numeric as string to
preserve precision. Parameter binding only — no SQL string concatenation.
- Smoke seed (one idempotent row per table) + data reset; CLI scripts
db:migrate / db:rollback / db:seed / db:reset.
- docs/data-model.md (tables, enum domains, truth map §7.2, mermaid ER,
migration runbook) + ADR 0002 (tooling rationale).
Tests
- unit: SQL builder + identifier guard (injection), migration plan/apply
(BEGIN/COMMIT/ROLLBACK), repo mapping/param-binding/enum/zod-reject.
- fuzz: assertIdent accepts iff safe pattern; buildInsert never inlines values.
- integration (real Neon, throwaway schema): every table+index present, happy
FK joins + leaderboard, singleton, unique attestation, FK violation, bad
enum, NOT NULL, target-only-on-transfer, numeric/int128/uint8/negative
guards, RESTRICT on delete-with-children, reset + re-seed, repo round-trip.
- e2e (real Neon): idempotent re-apply, full down→up integrity, atomic
rollback on mid-migration failure, two concurrent migrators serialize.
Verification
- tsc --noEmit, eslint, prettier: clean.
- next build: ok (DATABASE_URL set, unchanged P0.1 behavior).
- 105 tests pass / 0 fail across unit+fuzz+integration+e2e.
- `test` script now runs suites as separate processes: bun's mock.module is
process-global, so mock-based unit tests must not share a process with the
real-DB suites.
P0.2 — Neon data model & migrations (§7)
Implement Vector's single trust boundary: a typed, signed Intent and an
ordered validation pipeline that is independent of agents and the referee.
Only a structurally valid, authentic Intent crosses B1, so a prompt-injected
agent cannot bypass the gate — free-form model output is never executed.
What
- lib/intent/schema.ts: zod discriminated union on `action`
(open/modify/close/transfer), .strict(); JSON Schema export; Intent/
UnsignedIntent types derived via z.infer/z.input (single source of truth,
no drift under exactOptionalPropertyTypes).
- lib/intent/canonical.ts: numeric-as-string normalization (1 == 1.0 == "1"),
ISO-8601 ttl + nonce normalization, deterministic key-sorted serialization,
intent_hash = keccak256(canonical payload). Precision cap rejects absurd
literals without panic.
- lib/intent/sign.ts + verify.ts: EIP-191 personal_sign over the canonical
payload via viem; recovery/verification never throw on malformed input —
a failed auth is a deterministic reject. ERC-1271 left as the single seam
in verify.ts (out of scope for EOA seed agents).
- lib/intent/validate.ts: first-failing ordered checks
schema → signature → nonce → ttl → bounds → target_address, returning a
typed {ok,stage,code}. Nonce single-admission via an atomic reserve guard;
ttl skew/horizon opt-in. Policy (whitelist, caps, drain detection) is the
referee's job (P1.1), not this boundary.
- docs/intent-contract.md + docs/examples/signed-intent.json: normative spec
and a pinned, byte-stable conformance vector.
Reuse-first: crypto via viem (keccak256/EIP-191), JSON Schema via
zod-to-json-schema — no hand-rolled primitives.
Tests
- Happy ~10% / edge ~90%; unit + seeded fuzz (deterministic PRNG) + e2e +
integration (full path → real Neon `intents`, isolated schema).
- Golden/regression vectors pin payload, hash, and signature.
- lib/intent coverage: 100% functions / 100% lines.
Verification
- tsc, eslint, prettier clean; `bun run test` (unit+fuzz+integration+e2e)
green incl. DATABASE_URL; `next build` green.
Security audit of the P0.3 boundary found one real, exploitable issue in normalizeDecimal: the MAX_DECIMAL_DIGITS cap counted only significant digits, not the positional expansion from a scientific-notation exponent. A tiny literal such as `size: "1e999999999"` therefore slipped past the cap and was materialized into a multi-gigabyte string via `'0'.repeat(...)`, hanging or OOM-ing the process. This fires at the schema stage (a) of validateIntent — on attacker-controlled numeric fields (size/leverage/max_slippage/tp/sl) and *before* any signature verification — so it is an unauthenticated amplification DoS (~12 input bytes → gigabytes). Reproduced: "1e8000000" expanded to an 8 MB string; larger exponents hung the process. Root cause / fix: bound the full positional span (leading integer + trailing fractional places) by the same MAX_DECIMAL_DIGITS cap and reject in O(1) before any allocation. This is the minimal extension of the existing precision guard — no new dependency, consistent with the module's canonical-string approach. The cap sits at the same boundary as the digit cap (1e79 is the largest accepted power of ten; 1e80 is rejected); all legitimate financial magnitudes pass unchanged. Tests - unit: exponent bombs (±) throw `/maximum precision/` in <100ms each; 1e79 passes, 1e80 rejected (boundary). - e2e: an exponent bomb routed through validateIntent fails at stage `schema` in O(1) (no allocation), proving the gate cannot be hung pre-auth. Verification: tsc / eslint / prettier clean; full `bun run test` (unit+fuzz+integration+e2e, with DATABASE_URL) green; `next build` green; lib/intent coverage remains 100% functions / 100% lines. Audit notes (reviewed, no change needed): - signature binds agent_id + nonce (mutation breaks recovery); cross-agent replay is rejected at the signature stage. - ECDSA s-malleability is not exploitable: dedup keys on intent_hash / (agent_id,nonce), neither of which includes the signature. - nonce TOCTOU between read and durable reserve remains the caller's responsibility (DB unique index / createNonceGuard.reserve), as documented.
…nonce anti-aliasing Security/correctness fixes from the P0.3 audit, root-cause and minimal. No change to the accepted happy-path wire shape; golden vectors unchanged. What - canonical.ts `normalizeTimestamp`: reject lenient/ambiguous date strings. String input must now be a strict ISO-8601 instant with an explicit timezone (`Z` or `±HH:MM`/`±HHMM`); `number` is epoch-ms. Previously any string went to `new Date(...)`, so a timezone-less datetime was parsed in the *host's local zone* — making the canonical payload host-dependent (signature verification fails across hosts with different TZ, and ttl meaning shifts ±offset), and garbage like "2031" / "Jan 1 2030" / "01/02/2030" was silently accepted as a valid expiry. Determinism is the whole point of the canonical payload. - validate.ts `inUnitInterval`: compare on the canonical decimal string instead of `Number(d)`. A float comparison rounds e.g. "1.0000000000000001" down to 1 and admits a max_slippage strictly > 1; the gate must honour the exact bytes it signed/hashed (numerics are canonical strings end-to-end). - canonical.ts `normalizeNonce`: reject numeric nonces beyond Number.MAX_SAFE_INTEGER. Past 2^53 a JSON number has already lost precision, so two distinct nonces (2^53 vs 2^53+1) alias to one canonical token in the anti-replay key and wrongly reject a legitimate Intent as a replay. Use a string nonce for large/opaque values. Why these only - Domain separation (cross-instance/chain signed-Intent replay) and the published JSON-Schema↔parser fidelity gap are real but change the signed wire format / are architecture decisions — escalated separately, not auto-fixed. - maxTtlHorizon/isNonceUsed fail-open defaults are by-design opt-in (referee / durable store own them); left as-is. Tests - canonical: normalizeTimestamp rejects tz-less, bare year, locale dates, digit-string, out-of-range fields; accepts Z and numeric offset. normalizeNonce rejects unsafe-integer numbers, keeps string nonces of the same magnitude. - validate: max_slippage "1.0000000000000001" rejected; 0 / 1 / "0.5" accepted. Verification - bunx tsc --noEmit, eslint, prettier --check: clean. - bun test (unit+fuzz 147, integration+e2e 33+1skip on real Neon): green. - next build: ok.
…urity audit Root-cause, minimal fixes from the P0.2 audit. The SQL-injection posture (parameterized binds + assertIdent) and the SQL↔zod schema mapping were already sound and are unchanged. What - repos/_shared.ts `num()`: reject a JS `number` that is not an exactly representable safe integer. A non-integer (`0.1 + 0.2` → "0.30000000000000004") or an integer past 2^53 (an int128 `attestations.value` or a bigint block number passed as a number) has already lost precision before `num()` runs; coercing it via `.toString()` silently persisted a corrupted money/score/on-chain value into a `numeric` column, violating the project's "numeric is exact, never through a float" invariant. Safe integers (e.g. `score_r: 50`) and exact strings/bigints still pass; bad inputs now throw so the caller supplies an exact string. - migrate.ts `assertSessionConnection` (new, run at the start of `migrate()`): fail closed when the connection does not preserve session state across statements — i.e. a transaction-pooled endpoint (Neon `-pooler` / PgBouncer transaction mode). The runner relies on two session-scoped guarantees, the `pg_advisory_lock` migration mutex and `SET search_path`; on a pooled endpoint both silently no-op, so migrations would not be serialized and DDL could land in `public` instead of the target schema. Detected generically by setting a session GUC and reading it back on a separate statement. - migrate.ts `loadMigrations`: throw on a duplicate version+direction instead of letting readdir order silently pick a winner — completes the documented "a malformed set throws before any SQL runs" invariant (only missing-half was caught before). - migrate.ts error handling: a failing ROLLBACK no longer masks the original migration error (the real root cause), and a failing advisory-unlock in `finally` no longer turns an already-committed migration into a thrown error. - seed.ts `resetData`: refuse the destructive TRUNCATE-all when current_schema is `public`. The helper is exported from `lib/` with no guard; this blocks a `public`/production-bound connection being passed in by mistake. Legitimate callers run inside a dedicated non-public schema via search_path. Why these only - Schema CHECK gaps flagged by the audit (tp/sl ≥ 0, delta ∈ [-1,1]) are intentionally omitted per the data-model spec (referee owns intent validation) — left as-is. assertIdent reserved-word quoting / 63-byte bound and the health-probe in-flight-connection retention are defense-in-depth with no current exploit; not changed here. Tests - num: rejects non-integer / >2^53 / NaN / Infinity; accepts safe int, string, bigint (incl. int128-scale). - loadMigrations: throws on a duplicate up for a version. - applyMigration: a failing ROLLBACK still surfaces the original error. - assertSessionConnection: resolves on a persistent session, throws when state is dropped between statements. - resetData: refuses on `public`, truncates inside a non-public schema. Verification - bunx tsc --noEmit, eslint, prettier --check: clean. - bun test (unit+fuzz 93; integration+e2e 24+1skip on real Neon): green. - next build: ok.
What
- checkDb (lib/db/client.ts): run the `SELECT 1` probe on a dedicated pooled
client that is always released, and bound the query server-side with
`statement_timeout` (set via parameterized set_config). The wall-clock
Promise.race is kept only as a backstop on the HTTP response time.
Why
- The probe's timeout was a *racing promise*, not cancellation. When the timer
won, checkDb returned 'down' but the losing `getPool().query('SELECT 1')` kept
running on its acquired connection with no driver-level bound (the Pool has no
connectionTimeoutMillis / statement_timeout). Under a slow or hung backend
each in-flight probe pinned one of the pool's connections for as long as the
underlying op took — far beyond the 2s the caller was promised. /api/health is
unauthenticated and force-dynamic, so a burst during a DB blip (or a flood)
could pin every connection in the shared process-wide pool and amplify a
transient slowdown into a full outage. The advertised "bounded by 2s" was
illusory for resource holding.
- Now statement_timeout cancels the query server-side and the `finally` releases
the client promptly, so 'down' is reported AND the connection is freed. The
result stays a total function (every failure → 'down', never throws).
Scope
- Only the health probe is touched. The shared getPool() config is left as-is to
avoid changing connect semantics for the repo/migration pools; bounding a hung
*connect* (vs query) via Pool connectionTimeoutMillis is noted as an optional
follow-up. Other audit findings (missing baseline HTTP security headers; the
health page rendering the fetcher error instead of the db:down payload;
deepFreeze robustness on frozen/cyclic graphs; explorerTxUrl URL-encoding) are
defense-in-depth / non-security / latent-until-later-stage and were filtered,
not fixed.
Tests
- Updated tests/unit/health.route.test.ts: the Neon fake now models the
connect → client.query → release shape the probe uses (behavioral assertions
— 200/up, 503/down on reject, down on slow — unchanged).
Verification
- tsc --noEmit, eslint, prettier --check: clean.
- bun test per suite (the project's test command runs them as separate
processes): unit 63, fuzz 18, integration 17 (real Neon, incl. checkDb
up/concurrent/timeout), e2e 8 — all green.
- next build: ok.
… migration safety
…, validation & hardening
Implement the referee (architecture §6.3): the single path from a validated, signed Intent to the rail. A pure, deterministic evaluate() runs a fixed, ordered rule set and the first failing rule decides ALLOW/CLIP/REJECT/HALT; runReferee() re-validates via P0.3 and writes one policy_event per decision via the P0.2 repository. Rules (in order): kill switch (HALT) → market whitelist (REJECT/hard) → fresh-wallet transfer block (REJECT/hard) → per-trade size cap (CLIP/soft) → spend cap (CLIP or REJECT/soft) → leverage cap (CLIP/soft) → drawdown breaker (HALT). Caps use strict '>'; the drawdown breaker trips on reaching dd_breaker. Critical invariant: no transfer to a non-whitelisted address is ever ALLOWed or CLIPped (the drain block), covered by unit, fuzz, and e2e tests. Reuse, no reimplementation: - P0.3 lib/intent/validate.ts for pre-validation - P0.2 lib/db/repos/policy-events.ts for the audit write - P0.1 CONFIG.policy + fresh_wallet_criteria for caps/whitelist - exact decimal comparison via a new pure compareDecimal() in canonical.ts (no floats), reusing normalizeDecimal Tests: per-rule isolation + boundaries, ordering, severity mapping (100% line/ func coverage of lib/referee/*), fuzz invariants (domain closure, hard only for whitelist/transfer, HALT only for kill/drawdown, monotone CLIP, idempotency), integration (fake + real-Neon gated), and hard e2e (mass parallel drains, kill-switch race, all-rule conflict, adversarial addresses, extreme magnitudes). Docs: docs/referee.md.
The Neon pool in lib/db/client.ts is a process singleton (getPool: pool ??= ...).
When the suite runs with DATABASE_URL set, the gated db.integration test primes
that singleton with the real driver and then calls getPool().end() without
clearing the cached reference. The next file, health.route.test, relies on
mock.module('@neondatabase/serverless') to keep checkDb hermetic, but getPool()
hands back the stale (ended/real) pool instead of one built from the mock, so the
probe fails and the route returns 503 -> the "returns 200" test fails. It passed
in isolation and pairwise, which masked the cross-file leak; it reproduces
deterministically on main once DATABASE_URL is set.
Fix at the root: add a test-only resetPool() that drops the cached singleton, and
call it in the health test's beforeAll (so checkDb rebuilds a pool from the mocked
driver) and afterAll (so this file's mock pool never leaks to later files). No
production path uses resetPool; connection lifecycle stays the creator's job.
Verified: full suite with real Neon 192 pass / 0 fail (was 1 fail), deterministic
across repeated runs; without DATABASE_URL 171 pass / 29 skip / 0 fail; typecheck,
lint, format:check clean.
…ominate soft clips)
Security audit (P1.1) found that the single ordered "first-fires-decides" rule
list let an attacker pre-empt a terminal decision with an earlier soft CLIP:
oversizing a trade tripped size_cap (CLIP) before leverage_cap, the spend_cap
REJECT, and the drawdown_breaker HALT — so an over-leveraged / over-budget /
drawdown-breached trade could execute clipped instead of being rejected/halted.
Fix: evaluate in two phases.
- BLOCKING_RULES (HALT/REJECT) run first; first fire decides. drawdown_breaker
and the zero-budget spend reject move ahead of the soft caps so a terminal
decision can never be skipped.
- CLIPPING_RULES run only if nothing blocked and now *accumulate*: every
breached cap is clamped in one CLIP (size -> min(max_trade_size, remaining),
leverage -> max_leverage), so clipping one field can't let another through.
- spend_cap split into spendCapRejectRule (blocking) + spendCapClipRule
(clipping); both still report rule_fired='spend_cap'. A lone clip is reported
verbatim; multiple join rule ids with '+' and record each in detail.clips[].
Also: runReferee now fails closed — an unexpected error during validate/evaluate
records a terminal internal_error REJECT policy_event (err.name only, no message)
before re-throwing, preserving the one-event-per-decision audit invariant.
Tests: rewrote the ordering tests that encoded the bug; added regressions
(drawdown HALT and zero-budget REJECT beat an over-size clip; size+leverage both
clamped; size clamped to the smaller of cap/budget). Strengthened the fuzz
invariant so ANY clip result satisfies ALL caps, not just the rule that fired.
docs/referee.md: two-phase decision matrix + caller-contract / non-guarantees.
Adds a test-only resetPool() to lib/db/client.ts so a test that primes the process-wide pool (with the real driver, or after getPool().end()) can't leave a stale pool that later tests in the same process reuse and that defeats their driver mocks. Resolves the cross-test flake seen in the full suite with a live DB. (PR #7)
…clip-ordering bypass fix P1.1 referee — a pure two-phase evaluate() over a typed Intent: blocking rules (kill switch, market whitelist, fresh-wallet transfer block, drawdown breaker, zero-budget reject) decide first; clipping rules (size, over-budget spend, leverage) run only if nothing blocked and accumulate, so the post-clip Intent satisfies every cap at once. runReferee re-validates via the P0.3 boundary and records exactly one policy_event, failing closed (terminal internal_error REJECT) on unexpected errors. Includes the security hardening from the audit: terminal decisions (HALT/REJECT) now dominate soft CLIPs, so an attacker can no longer oversize `size` to trip an early size_cap clip and pre-empt a later leverage/spend reject or a drawdown halt. (PR #6)
…cale guard Pre-P1.2 hardening of audit findings, on a dedicated branch. #1 nonce-replay (durable): migration 0002 adds UNIQUE(agent_id, nonce) on intents so a replayed Intent insert fails atomically in the DB, independent of process-local state (P0.3 had only an in-memory guard). New repo primitives insertIntentReserving (INSERT ... ON CONFLICT DO NOTHING; null = replay) and isNonceUsed (durable read); buildInsert gains an onConflictDoNothing option and insertOneOrNull backs it. NULL nonces stay exempt (NULLs distinct). #3 statement_timeout leak: checkDb now scopes the probe timeout to a transaction (set_config(..., is_local=true) inside BEGIN/COMMIT) so it is discarded on commit and never leaks onto the pooled connection. #4 pool error handler: getPool attaches an idle-client error listener so a Neon idle-connection drop no longer crashes the process; logs only err.name. #5b numeric scale guard: the validator rejects a numeric field with finer fractional scale than its numeric(p,s) column can store (silent-rounding / integrity guard). Magnitude is intentionally NOT bounded here — the firewall clips over-large size/leverage (§6.5). schema: .max() length bounds on agent_id/market/target_address (pre-signature DoS guard). Tests: +ON CONFLICT/insertOneOrNull, +scale-guard & magnitude-pass cases, +schema length bounds, +checkDb transaction-shape and pool-error-swallow regressions, +intents anti-replay integration on live Neon. Full suite (live Neon) 271 pass / 1 skip / 0 fail; typecheck/lint/format clean.
…+ intent scale guard Pre-P1.2 hardening of audit findings (#1 durable UNIQUE(agent_id,nonce) anti-replay, #3 txn-scoped probe statement_timeout, #4 pool idle-error handler, #5b numeric fractional-scale validator guard, schema .max() length bounds). Reviewed by a 3-agent swarm (security/correctness, integrity/no-loss, independent re-validation): all PASS, no blockers. Full suite on live Neon: 271 pass / 1 skip / 0 fail; typecheck/lint/format clean.
Pure score() (architecture.txt §6.1): RoC -> bounded tanh perf -> capital risk-weight (anti-Sybil) -> policy/drawdown penalties -> clamp raw_r -> EWMA over history -> floor-crash on halt/drain to crash_cap. clean_r derived from hard==0; drain_r from referee rule #3. Inputs validated (RangeError on NaN/inf/negative/fractional). Deterministic double math, outputs quantized to column scale. Persistence: deriveScoreInputs(outcomes, policy_events) + recordScore (insert scores row + atomic agents.score_current/status update; gates on crash or score<s_min, never touches operator-halted). updateAgentScore is the only writer of score_current. Added getLatestScoreByAgent, listPolicyEventsByAgentRound. Scale reconciliation: penalties read as 0-100 points (matches CONFIG.scoring constants and the §6.1 'hard = dominant penalty, not crash' distinction); documented in docs/scoring.md. NEEDS OWNER CONFIRMATION. Tests: unit (steps/bounds/invariants/invalid-input/monotonicity), golden table, 5000-draw deterministic fuzz, record (FakeDb), integration (real Neon, isolated schema), e2e (catastrophe/recovery, alpha boundaries, reproducibility). lib/scoring/* 100% line+func coverage. Full suite 339 pass/1 skip/0 fail on live Neon; typecheck/lint/format clean.
Audit-driven fixes (real risks only; false positives/non-issues filtered):
- deriveScoreInputs: skip meta events (internal_error/pre_validation/allow) so
infra faults no longer penalize an agent's reputation; dedup violations per
intent_id (worst severity) so re-evaluations can't double-count.
- Drain floor-crash keyed on a shared FRESH_WALLET_TRANSFER_BLOCK_RULE const
exported from the referee rule (was a duplicated literal); regression test
pins the coupling and that a drain crashes.
- recordScore: idempotent ON CONFLICT DO NOTHING insert + converge the agent
gate from the persisted score on replay, so a crash that failed to gate is
healed on retry instead of left fail-open.
- getLatestScoreByAgent: chain EWMA prior by rounds.index (not created_at),
fixing out-of-order/same-tick nondeterminism.
- listOutcomesByAgentRound: deterministic (created_at, id) tiebreaker.
- scores.components_json: enforce {perf,w,policy,dd} contract at persistence
(strict zod); fix seed missing the w key.
- docs: concurrency precondition for P1.4 caller, tanh cross-engine note, and
open owner-decision security notes.
Full suite WITH live Neon: 343 pass / 1 skip / 0 fail; typecheck/lint/format clean.
…omplements router >=)
feat(scoring): P1.2 scoring engine (AgentScore in [0,100])
Pure, deterministic capital allocator (architecture.txt 6.2): softmax merit target with eligibility gate, hysteresis, max-step, cooldown, plus immediate crash/HALT gate-out. Pool conserved exactly (sum amount == pool_size) via integer largest-remainder apportionment, zero drift across rounds. Amounts and weights are fixed-scale decimal strings over BigInt (never floats). - lib/router: types, fixed-point arithmetic, route(), persistence layer - docs/capital-router.md; link from docs/config.md - unit (route + fixed-point), golden, fuzz, e2e, integration tests
…6.2) Routes a fixed pool by reputation: softmax merit target, four anti-oscillation mechanisms (eligibility gate, hysteresis, max-step, cooldown), forced gate-out on crash/HALT, round-0 bootstrap, and exact integer conservation (Σ amount == pool). Pure deterministic core (lib/router/route.ts) + thin persistence (record.ts).
capital_allocations was the only per-(agent,round) ledger without UNIQUE(agent_id,round_id) (scores and attestations both have it), so a settlement retry or a concurrent pass could append a second full row set and silently double the round's Σ amount — corrupting both the conservation audit and the prev-state the next pass routes from. - Add the UNIQUE(agent_id,round_id) constraint in migration 0003 and drop the now-redundant non-unique idx_capital_alloc_agent_round (the constraint carries its own backing index). Add CHECK(delta in [-1,1]) as defense-in-depth. - insertCapitalAllocation now inserts ON CONFLICT DO NOTHING and returns CapitalAllocationRow | null, mirroring scores.insertScore. - recordRoute turns a null (conflict) into idempotency: it stops trusting the pass's partial inserts and returns the authoritative persisted ledger, so a retry of an already-recorded round is a no-op. Document the one-transaction-per-round atomicity contract. - Regression: integration test asserts a re-run records no duplicate rows and keeps Σ amount == pool exactly.
…tent recordRoute Durability hardening for the capital ledger: a UNIQUE(agent_id,round_id) constraint (migration 0003) + ON CONFLICT DO NOTHING make recordRoute idempotent under settlement retries, so a re-run cannot double the round's Σ amount.
The prior allocation a pass routes from comes off the ledger; a corrupted or
adversarial value must fail deterministically, not skew the move policy.
- parseUnits silently read no-digit strings ('', '.', '-', '+', whitespace) as 0;
it now throws RangeError on a magnitude with no digits (signed parse and
truncation-beyond-scale are unchanged).
- toUnits rejects a magnitude past Number.MAX_SAFE_INTEGER (the 1e21 toFixed
cliff) up front, instead of failing cryptically deep in BigInt.
- route() validates prev amount/weight are non-negative, removing the
parseUnits(accepts neg) / formatUnits(throws on neg) asymmetry and preventing
a negative row from skewing prevSum/prevW or forcing a false cold start; the
prev weight is now carried as bigint units end to end.
- isMaterial parses the canonical fixed-point strings via parseUnits, not a lossy
Number.parseFloat — one parser end to end.
- docs: soften 'cannot oscillate' to per-step monotonicity (hysteresis/cooldown
bound and damp, they do not eliminate, adversarial self-oscillation); note
Math.exp is the only non-bit-identical step across runtimes; note Σ
target_weight need not equal 1 exactly (amount is the source of truth).
Regression: unit tests cover the parseUnits no-digit reject, the toUnits
MAX_SAFE_INTEGER reject, and route() rejecting a negative prev amount/weight.
…s + non-negative prev Input-validation hardening for the router: parseUnits/toUnits reject malformed and out-of-range magnitudes, route() rejects a negative prev amount/weight, and isMaterial uses the one canonical parser — a corrupted ledger row fails deterministically instead of skewing the move policy.
Drive a frozen scripted arc through the real referee -> scoring -> router pipeline on a seeded execution rail, producing a byte-reproducible end-to-end demo (signal -> decide -> intent -> referee -> execution -> outcome -> score -> [attestation seam] -> capital re-route). - lib/replay/: scheduler, compose, attack, rail (+fallback), control latch, setup (idempotent), orchestrator (runArc), barrel. - lib/agents/seed/: deterministic seed roster + strategies. - seed/: frozen DEMO_ARC dataset (pure fn of version/rounds/timing). - migration 0004: add 'seed' to execution_rail enum (+ guarded down). - docs/demo-spine.md: determinism contract + seams. - tests: unit/fuzz/e2e green locally; integration on real Neon (asserts conserved pool, leader crash via drain rule #3, reroute to runner-up, idempotency). Injected transfer blocked REJECT/hard (rule #3) -> leader crashes to crash_cap -> capital reroutes to seed-2.
Optimistic on-chain mirror written atomically inside settleRound; giveFeedback + reconcile run post-commit so latency never blocks the settle arc. Idempotent via UNIQUE(agent_id,round_id)+ON CONFLICT, tx_hash-IS-NULL submit latch, forward-only reconcile guard. Off-chain detail stored verbatim and served byte-identical so feedback_hash always re-verifies.
Add X-Content-Type-Options: nosniff to the verbatim-bytes endpoint and cover it with route tests (was uncovered).
PUBLIC_BASE_URL reused the rpcUrl validator (also accepts ws(s)), contradicting its contract and its only consumer buildFeedbackUri. Dedicated http(s)-only validator + regression tests.
Add lib/rail/byreal/* - a safe CLI shell-out that settles already-allowed Intents on the Byreal/Hyperliquid testnet venue and maps fills/PnL onto executions/outcomes. Stage-2 credibility layer. Determinism boundary (sec.3): real Byreal outcomes are an opt-in side-channel (RunArcOptions.credibilityRail) written as executions/outcomes(rail='byreal'); shown alongside the demo but never feed scoring. Scoring reads only seeded outcomes via listSeedOutcomesByAgentRound. Default-off => arc byte-identical (golden test unchanged). - argv-only spawn (no shell), minimal child env (credential sole-custody), timeout + output cap, ALLOW/CLIP-only, silent seed fallback, idempotency by intent_hash; mainnet refused unless explicitly opted in - VERIFY V3 resolved; [CORE] open/close/modify TP-SL + read PnL/positions; limit orders [ROADMAP] - unit + fuzz + e2e + DB-gated integration; docs/byreal-rail.md 68 non-DB byreal tests green; typecheck + lint clean; no regressions.
…y, observability Security-audit follow-ups on the P2.1 Byreal rail. Each is a real risk; false positives (shell injection, credential leak, prototype pollution, ReDoS, the mainnet construction guard, determinism/scoring isolation) were verified safe and left unchanged. - cli.ts: the SIGKILL escalation timer was cleared by finish() in the same synchronous turn it was set, so a CLI that ignores SIGTERM would never be force-killed (zombie/fd leak). Clear the timer only on `close`; unref it. - cli.ts: forward BYREAL_PERPS_NETWORK to the child as defense-in-depth so the subprocess is pinned to the validated network (load-bearing guard stays the construction-time mainnet refusal). - parse.ts: clamp a negative fee (venue rebate) to '0' — outcomes.fees is CHECK (fees >= 0), so a rebate would abort the insert and silently drop a real fill. Canonicalize '-0' and render JS-number fees as fixed-point (never exponent). Report a no-fill/no-resting order as 'sent', not 'filled'. - command.ts: require strictly-positive size/tp/sl, so no numeric can ever land in a positional argv slot looking like a flag (defense-in-depth). - orchestrator.ts: settleCredibility still degrades silently (arc must not stall) but now emits an operator signal (name only) so a failing rail or DB write is observable instead of vanishing. Regression tests added for each; typecheck + lint clean; 690 unit tests pass.
…a real DB Running the full suite against a live Neon Postgres + the public Mantle Sepolia RPC (the DB/RPC-gated tiers had never been exercised without creds) surfaced three pre-existing test bugs. All three are test-side; the production code paths they cover are validated and unchanged. - byreal.integration: the determinism-boundary assertion queried `scores.value`, but the column is `score_r` (migration 0001). The query threw 42703 every run, so the load-bearing claim — enabling the live rail leaves scores byte-identical — had never actually executed. Fixed to `score_r`; it now runs and passes. - read-api.e2e: the MockPool returned rows without the microsecond `cursor_t` alias the keyset design selects (CURSOR_KEY_SQL), so `paginate` minted an undefined cursor and the route 500'd. The real repo+route are correct and the integration suite covers the same pagination against real Postgres. Mock now mirrors the contract (cursor_t derived from created_at). - referee.e2e: an astronomically large size (>20 integer digits) is now rejected by the storability bound (numeric(38,18)) before it can reach the size_cap clip — intended behavior the test predated. Split into two cases: a large but storable size CLIPs to the cap; an unstorable size REJECTs at the gate. Verification: unit 690, fuzz 58, integration 46, e2e 45, on-chain read 9 — all pass, 0 fail, lint + typecheck clean.
The suite injected a fake giveFeedback/receipt client because a funded testnet wallet was out of band, so the real on-chain WRITE path (P1.8 task 3) had never executed. This adds a gated e2e that drives the production clients end to end against the live registries: register a fresh agent (operator key) -> assertCanAttest against the live Identity Registry -> giveFeedback (attestor key) -> wait for the receipt -> read the feedback back and assert the on-chain bytes (value 73, valueDecimals 0, tag1, tag2, isRevoked) match what was written. Gated on a funded operator + a *distinct* funded attestor key (+ RPC), so CI without testnet funds skips cleanly and stays green. A read-after-write barrier (waitFor) tolerates the public RPC load-balancing reads to replicas a block or two behind head; without it an immediate ownerOf after register can miss the mint. This is a test-only concern — production registers agents at seed time, long before any per-round attestation, and assertCanAttest fails safe (typed throw, no tx) if a read ever did lag. Verified: passes twice (non-flaky) against live Mantle Sepolia (chain 5003), typecheck + lint clean, and skips when keys are absent.
Add lib/rail/byreal/* - a safe CLI shell-out that settles already-allowed Intents on the Byreal/Hyperliquid testnet venue and maps fills/PnL onto executions/outcomes. Stage-2 credibility layer. Determinism boundary (sec.3): real Byreal outcomes are an opt-in side-channel (RunArcOptions.credibilityRail) written as executions/outcomes(rail='byreal'); shown alongside the demo but never feed scoring. Scoring reads only seeded outcomes via listSeedOutcomesByAgentRound. Default-off => arc byte-identical (golden test unchanged). - argv-only spawn (no shell), minimal child env (credential sole-custody), timeout + output cap, ALLOW/CLIP-only, silent seed fallback, idempotency by intent_hash; mainnet refused unless explicitly opted in - VERIFY V3 resolved; [CORE] open/close/modify TP-SL + read PnL/positions; limit orders [ROADMAP] - unit + fuzz + e2e + DB-gated integration; docs/byreal-rail.md 68 non-DB byreal tests green; typecheck + lint clean; no regressions.
…y, observability Security-audit follow-ups on the P2.1 Byreal rail. Each is a real risk; false positives (shell injection, credential leak, prototype pollution, ReDoS, the mainnet construction guard, determinism/scoring isolation) were verified safe and left unchanged. - cli.ts: the SIGKILL escalation timer was cleared by finish() in the same synchronous turn it was set, so a CLI that ignores SIGTERM would never be force-killed (zombie/fd leak). Clear the timer only on `close`; unref it. - cli.ts: forward BYREAL_PERPS_NETWORK to the child as defense-in-depth so the subprocess is pinned to the validated network (load-bearing guard stays the construction-time mainnet refusal). - parse.ts: clamp a negative fee (venue rebate) to '0' — outcomes.fees is CHECK (fees >= 0), so a rebate would abort the insert and silently drop a real fill. Canonicalize '-0' and render JS-number fees as fixed-point (never exponent). Report a no-fill/no-resting order as 'sent', not 'filled'. - command.ts: require strictly-positive size/tp/sl, so no numeric can ever land in a positional argv slot looking like a flag (defense-in-depth). - orchestrator.ts: settleCredibility still degrades silently (arc must not stall) but now emits an operator signal (name only) so a failing rail or DB write is observable instead of vanishing. Regression tests added for each; typecheck + lint clean; 690 unit tests pass.
…ores schema Split from a mixed real-DB test-fix commit: the byreal integration test asserted on a non-existent `scores.value` column (should be `score_r`), so the determinism-boundary assertion had never actually executed against Postgres.
The suite injected a fake giveFeedback/receipt client because a funded testnet wallet was out of band, so the real on-chain WRITE path (P1.8 task 3) had never executed. This adds a gated e2e that drives the production clients end to end against the live registries: register a fresh agent (operator key) -> assertCanAttest against the live Identity Registry -> giveFeedback (attestor key) -> wait for the receipt -> read the feedback back and assert the on-chain bytes (value 73, valueDecimals 0, tag1, tag2, isRevoked) match what was written. Gated on a funded operator + a *distinct* funded attestor key (+ RPC), so CI without testnet funds skips cleanly and stays green. A read-after-write barrier (waitFor) tolerates the public RPC load-balancing reads to replicas a block or two behind head; without it an immediate ownerOf after register can miss the mint. This is a test-only concern — production registers agents at seed time, long before any per-round attestation, and assertCanAttest fails safe (typed throw, no tx) if a read ever did lag. Verified: passes twice (non-flaky) against live Mantle Sepolia (chain 5003), typecheck + lint clean, and skips when keys are absent.
Split from a mixed real-DB test-fix commit (the byreal-specific hunk lives with the rail feature). Both are test-side only and fail on main today: - read-api.e2e: the MockPool lacked the microsecond `cursor_t` keyset alias, so the keyset-pagination path 500'd instead of exercising the cursor. - referee.e2e: split the size-magnitude case into a storable-over-cap value (clips to the size cap) and an unstorable 26-digit value (rejected at the numeric(38,18) storability bound), which the single combined case conflated.
… + env docs Three issues surfaced by an automated review swarm over the attestation/chain write path (0 critical; these are the actionable correctness/ops items): 1. submit.ts `toInt128`: the stored `value` column (numeric(39,0)) is wider than the registry's `int128 value` argument, but the parse only checked the integer grammar — an out-of-range value (corrupt/hand-edited row, or any non-scorer writer) would reach the writer and fail as a cryptic ABI-encode throw after precondition checks. Range-check it at the chain-write boundary so it is a deterministic typed error before any gas is spent. Reuses INT128_MIN/MAX from encode.ts. Regression test added (fails closed, zero writes). 2. client.ts self-feedback guard was dead code: `ATTESTOR_PRIVATE_KEY` is documented as "distinctness enforced at the client (assertDistinctSigners)", but assertDistinctSigners was never called, so a shared operator/attestor key would slip through to a guaranteed on-chain revert. Wire it into getAttestorAccount — the single entry point to attestor signing — so feedback writes fail closed on the misconfig. (The happy path is exercised by the live write-path e2e; the negative path is covered by the pure assertDistinctSignerKeys unit tests. A dedicated wiring test was intentionally omitted: ENV is a single eager process-global, so an in-suite negative test is order-flaky and a module mock would contaminate sibling files — not worth the fragility.) 3. .env.example was missing every P2.1 / write-path env var (BYREAL_PERPS_*, PUBLIC_BASE_URL) — a deployment footgun. Documented with their semantics. Verified: typecheck + lint clean; unit+fuzz 749 pass / 0 fail; integration + e2e green against the throwaway-schema DB and public RPC.
Live perps execution rail behind the deterministic seed: default-OFF, fails closed (no creds => byte-identical seed scores), no-shell CLI invocation, hardened subprocess/parse/observability, and a determinism-boundary integration test verified against real Postgres.
Test-side only (both failed on main): keyset-pagination MockPool cursor alias and the referee size-magnitude clip-vs-reject split.
…ardening Gated live write-path e2e (register -> assertCanAttest -> giveFeedback -> receipt -> read-back) against Mantle Sepolia, plus three audit fixes: int128 range-check at the chain-write boundary, wiring the previously-dead self-feedback guard into attestor signing, and documenting the new env vars.
P2.1: Byreal Perps CLI rail adapter
Implement a read-only Nansen Smart Money `netflows` signal injected into the leader's `context.signals.nansen`, behind a TTL cache + slow poller. - lib/signals/nansen: pure injectable client (typed errors, defensive zod parse, bounded body/rows, AbortController timeout), caching/slow-polling provider (doubly-gated refresh, in-flight dedup, credit budget, fail-open), server-only key loader (returns null when NANSEN_API_KEY unset), barrel. - lib/replay: nansenSignalsFor() leader-only injection helper; runArc gains an opt-in `nansen` provider, polled fire-and-forget per tick (never awaited). - Invariants: tick never blocks on the network; signal is read-only and cannot reach execution; default-off keeps the arc byte-identical. - Tests: unit (client/provider/inject), fuzz (client + provider), e2e (byte-identical signed arc with/without signal), integration (DB arc invariance under a flapping provider + NANSEN_API_KEY-gated live call). - docs/nansen-signal.md.
…dently `RunArcResult.finalAllocations` is the *set* of per-agent end-state allocations. The persisted read (`listAllocationsByRound`) orders by `created_at`, which is not a total order — rows inserted in one routing pass can tie, so the row order is DB-arbitrary and may differ run-to-run. The invariance assertion therefore must compare order-independently (same spirit as the adjacent `crashedAgentIds.sort()`), otherwise it flakes against real Neon while the actual allocations are identical.
A read-only, fail-open swarm audit surfaced a handful of genuinely reachable issues (the rest were false positives). Fix only the real ones, smallest change: client.ts - Timeout now spans the whole round-trip (connect AND body read). Previously the abort timer was cleared once headers arrived, so a slow-drip body could hang the fetch forever and pin the provider's in-flight dedup, silently disabling all future refreshes. One try now covers fetch+parse; typed errors pass through, any abort maps to NansenTimeoutError. - redirect: 'error' — a single-endpoint credentialed call must never follow a 3xx that could replay the `apiKey` header to another origin. - Cap raw rows *scanned* (MAX_ROWS_SCANNED) independently of output, so a 2 MiB array of unusable rows can't run hundreds of thousands of zod parses and stall the event loop. - Response size bound measured in UTF-8 bytes (Buffer.byteLength), not UTF-16 code units, so the 2 MiB cap holds for multi-byte payloads. - Retry-After parsed via a clamped helper (reject non-finite/negative, cap at 5 min) so a hostile header can't inject a negative/multi-year delay. provider.ts - Isolate the caller-supplied logger behind a `log()` wrapper. A throwing observability sink could previously reject the detached fetch (process-killing unhandled rejection) or throw synchronously into the tick via the budget path. Logging is now best-effort and never load-bearing; the fetch proceeds and fail-open holds regardless of sink health. constants.schema.ts - Nansen endpoint is now https-only (httpsUrl): the API key rides in a header, so a cleartext override must fail at config load. Tests: +6 regressions (redirect, slow-drip body timeout, scan cap, Retry-After clamp, throwing-logger fault isolation x2). Full unit 723 pass; nansen fuzz/e2e/integration green; lint/typecheck/prettier clean.
feat(signals): P2.2 Nansen smart-money signal (read-only, fail-open)
harden(signals): P2.2 Nansen security audit fixes
Pre-existing formatting drift in the byreal rail modules and their
e2e/integration/unit tests left `bun run format:check` red on main.
Pure formatting (no logic change); brings the documented format gate green.
- lib/rail/byreal/{adapter,index,parse}.ts
- tests/{e2e,integration,unit}/byreal.* + attestation.live-write.e2e
`listAllocationsByRound` and `listPolicyEventsByAgentRound` ordered only by `created_at`, leaving rows that share a timestamp in an engine-defined order — a latent nondeterminism in the determinism-critical pipeline. Add the `id ASC` tiebreaker, matching the established pattern in intents/outcomes repos. Regression test (real Neon, isolated schema) inserts two sibling rows with an identical created_at, larger id first; it fails on the tiebreak-less query and passes with the fix. - lib/db/repos/capital-allocations.ts: ORDER BY created_at ASC, id ASC - lib/db/repos/policy-events.ts: ORDER BY created_at ASC, id ASC - tests/integration/repos-order.integration.test.ts (new)
fix(db): total order on per-round reads
ENV is validated eagerly when API route modules are collected at build time, so a build without DATABASE_URL fails fast. Document it in the Scripts section so a cold checkout knows to set the variable before building.
docs: note next build requires DATABASE_URL
The header still described "Stage P0.1 — foundation only (scoring, referee and on-chain writes land in later stages)", which materially understated the project: referee, scoring, router, ERC-8004 integration, attestation, the Byreal rail and the Nansen signal are all implemented and tested. - Replace the stale P0.1 note with the real pipeline diagram and a one-paragraph description of each engine (referee / AgentScore / capital router). - Add an "On Mantle (on-chain)" section: canonical ERC-8004 registry addresses on Mantle Sepolia, the msg.sender authorization note, and the Byreal/Nansen side-channel determinism boundary. - Add a "Demo — the 90-second arc" section pointing at runArc(db, DEMO_ARC). - Expand the Docs index to surface demo-spine / referee / scoring / router / erc8004 / byreal / nansen. Docs-only; no code or behavior change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Refresh the README so it reflects the built system instead of the stale "Stage P0.1 — foundation only" note.
The old header said scoring, referee and on-chain writes "land in later stages" — but they are all implemented and tested. A judge opening the repo would materially undervalue the project.
Changes (docs-only, no code/behavior change)
msg.senderauthorization note, and the Byreal/Nansen side-channel determinism boundary.runArc(db, DEMO_ARC).Why now
Hackathon submission readiness (Mantle: The Turing Test 2026, track Agentic Wallets & Economy). The README is the repo's 60-second first impression for judges.