From 241a9f20a3d532ff0b7a144d2c31839a207bbd6b Mon Sep 17 00:00:00 2001 From: Pawel Date: Sun, 23 Aug 2026 13:55:25 +0200 Subject: [PATCH 01/50] docs: harden the local RVT reader boundary --- .../specs/2026-08-23-aware-rvt-reader-plan.md | 773 ++++++++++++++++++ .../2026-08-23-aware-rvt-reader-review-log.md | 269 ++++++ 2 files changed, 1042 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-23-aware-rvt-reader-plan.md create mode 100644 docs/superpowers/specs/2026-08-23-aware-rvt-reader-review-log.md diff --git a/docs/superpowers/specs/2026-08-23-aware-rvt-reader-plan.md b/docs/superpowers/specs/2026-08-23-aware-rvt-reader-plan.md new file mode 100644 index 000000000..12c8cc25a --- /dev/null +++ b/docs/superpowers/specs/2026-08-23-aware-rvt-reader-plan.md @@ -0,0 +1,773 @@ +# AWARE RVT reader — Task 6 implementation plan + +> **For implementation:** Follow this plan test-first on +> `codex/xeorvt-aware-rvt-reader`. Do not push, tag, release, merge to AWARE `main`, or enable a +> FloLess UI. Keep the Residential model, converted artifacts, provider executable, provider URLs, +> credentials, and secrets outside both repositories. + +**Status:** Revised after the fifth and final bounded adversarial review. Codex returned `REVISE` with +three concrete protocol findings; all three are incorporated below. The user explicitly authorized +implementation after resolving the final material findings. No reader implementation has been written. + +**Goal:** Add a provider-neutral, local RVT reference-model reader to AWARE's existing +`aware-connection-reader` SEA. It converts one `.rvt` through a separately installed adapter, +normalises the adapter's GLB and explicit Revit metadata into deterministic bounded artifacts, and +publishes an authenticated multi-artifact receipt that FloLess Task 5 can ingest without learning +Revit or provider semantics. + +**Anchors (verified 2026-08-23):** + +- AWARE source: `main@934d5935b9c8a4d3c27e6cfdd862770173458a0e` (`v0.126.0`), clean before this plan. +- Installed CLI: reports `0.127.0`, but it is not the source anchor and likewise has no generic + `aware secret put/revoke` command. +- Missing generic custom-secret provisioning is tracked by open AWARE issue #448. This task must not + invent, document, or test a capability that does not exist. +- FloLess consumer line: `codex/xeorvt-reference-model@721d3bb2`; Task 5 owns source bytes, project + generation/approval CAS, immutable artifact storage, and HTTP authorization. +- Existing bridge: `cli-connection-reader`, shared by `connection-reader` and + `ifc-reference-reader`; its source entry, SEA build, stdout guard, artifact directory, progress + channel, and sibling web-ifc WASM conventions are the compatibility baseline. + +## 1. Scope and non-negotiable boundaries + +### In scope + +- Curated `model-reference-reader` agent with `preflight`, `probe`, and `read-model` commands. +- A local, bounded JSON-stdin provider protocol using one separately installed executable. +- RVT only in this slice. RFA can use the protocol later but is not advertised or accepted now. +- Deterministic GLB normalization and separate canonical entity, property, and relationship JSON + artifacts. +- A content-addressed, concurrency-safe conversion/normalization cache under AWARE state. +- Authenticated ordered receipts, complete provenance, exact coverage, safe artifact publication, + cancellation, stale-owner takeover, and crash-idempotent retry. +- SEA packaging and current-CLI agent validation. +- Regression proof that IFC command output is unchanged. + +### Out of scope + +- A cloud/evaluation adapter, URLs, uploads, or any implicit network access. +- AWARE generic secret provisioning. Provider installation/licensing remains out-of-band until #448 + lands or the standalone adapter needs no AWARE-managed credential. +- Shipping proprietary provider bytes. +- RFA, comparison, filtering, FloLess workflow/UI changes, product-master merges, releases, or pushes. +- Inferring Revit Category/Family/Type/Level/relationships from node names, geometry, material, order, + or English labels. Only explicit provider metadata and validated references are authoritative. + +## 2. Contract decisions + +### D1 — Configuration has one explicit local trusted-computing boundary + +The bridge reads the provider executable from `AWARE_MODEL_REFERENCE_PROVIDER`. The value must be an +absolute path to a regular executable file. Relative paths, PATH lookup, directories, reparse +points/symlinks, and shell commands are refused. The bridge spawns that exact path with +`shell:false`; no model path, token, or configuration value appears in argv. + +The separately installed provider is explicitly part of the trusted computing base. AWARE 0.126.0 +does not provide an AppContainer, network deny policy, or generic secret facility, so this task does +not claim a malicious same-user provider is contained. The bridge itself has no network client, URL +input, or implicit provider discovery. It launches the trusted provider with a documented allowlist +environment containing only the minimum OS variables needed to start plus protocol locale/timezone +settings. It excludes `AWARE_HOME`, artifact/progress paths, proxy variables, PATH, tokens, and common +credential variables, and gives the provider a new private working directory. + +Production accepts `execution:"local"` and `destination:null` only. The committed bridge contains no +HTTP client and no URL input. Provider licensing and credentials are the provider executable's own +local concern. If the authorized provider requires a credential AWARE must provision, preflight +reports the dependency as unavailable; it does not claim issue #448 is solved. + +The cache root is `\cache\model-reference-reader`, falling back to the same user-profile +`~/.aware` convention the AWARE CLI uses when `AWARE_HOME` is absent. Tests inject an isolated root. + +### D2 — Provider protocol is closed, versioned, bounded, staged, and binary-safe + +The bridge invokes the provider twice: + +1. `describe --json-stdin` receives a closed request containing protocol version and limits. It must + return one bounded JSON object: + + ```json + { + "protocolVersion": "1", + "provider": "provider-id", + "engine": "engine-id", + "engineVersion": "x.y.z", + "adapterBuildId": "opaque-build-id", + "formats": ["rvt"], + "execution": "local", + "destination": null + } + ``` + +2. `convert --json-stdin` receives a closed request containing the absolute path of an immutable staged + source copy, a new private output directory, expected source SHA-256, canonical conversion settings, + and byte/count limits. + It returns one bounded JSON receipt with the same provenance plus `documentKind:"revit-project"`, + the source hash, and exactly two absolute paths: `geometry.glb` and `metadata.json`. + +Before conversion the bridge opens the caller's regular `.rvt`, copies it into an exclusively created +private file while hashing, rehashes that staged file, makes it read-only, and then exposes only the +staged path to the provider. The original is hashed again after staging; disagreement with the expected +hash or staged hash is `reference-source-changed`. This removes the pathname reopen race from provider +execution. Reparse points and link-like source/output entries are refused at every boundary. + +The GLB is never placed in stdin/stdout, decoded as UTF-8, or included in an error. Provider stdout, +stderr, request bytes, run time, output-file count, each output size, and total output are capped. +Every provider call goes through the new internal `aware __model-reader-host` helper supplied to the +bridge by the runtime as an exact executable path. On Windows the host starts the provider suspended, +assigns it to a dedicated kill-on-close Job Object, resumes it, and does not report completion until the +Job has zero active descendants; timeout kills that provider Job without killing the bridge. On Unix it +uses a dedicated process group with the same zero-descendant rule. This opt-in host is used only by the +model reader; existing sidecars such as `tekla.launch`, whose child intentionally survives the one-shot +response, retain their lifecycle. Timeout or cancellation terminates and awaits the provider tree. + +The host is a long-lived child of the bridge using a closed multiplexed `model-reader-host/v1` protocol. +Every frame has a kind byte (`0x01` JCS control, `0x02` stdout bytes, `0x03` stderr bytes, `0x04` +stdin bytes), unsigned 64-bit big-endian request ID, fixed 32-byte run handle (all zeroes before a run +handle exists), unsigned 32-bit big-endian per-stream sequence, one flags byte whose low bit is `final`, +unsigned 32-bit big-endian payload length, then bounded payload. Control payloads are UTF-8 JCS; +stdin/stdout/stderr payloads are uninterpreted bytes. A `provider-run` control frame declares the exact +stdin byte length and returns its run handle before matching sequenced stdin frames are accepted; the +provider starts only after one matching final stdin frame and exact length reconciliation. The host +drains stdout and stderr concurrently while accepting control/cancel frames, validates monotonic stream +sequences, and correlates every out-of-order completion and binary frame by both request ID and run +handle. Operations are `hello`, +`lock-acquire`, `lock-release`, `provider-run`, `provider-cancel`, and `shutdown`; every request has a +monotonic request ID and every lease/run has an unguessable handle scoped to that host process. Acquired +locks remain held until matching release or host death. Acquire/run immediately acknowledge a handle; +the host processes frames concurrently and returns out-of-order completions by request ID so cancel and +shutdown remain live while another request waits. `provider-run` carries the exact executable, +minimal environment, cwd, stdin bytes, timeout and output caps; stdout/stderr return as separately +length-framed bytes only after zero descendants. EOF/parent death cancels runs, releases locks, and exits. +The runtime passes the canonical current `aware` executable path in a private environment variable and +the bridge verifies the host `hello` build/protocol before sending sensitive paths. +The bridge rejects extra files and reads outputs as bytes only from validated regular descendants of +its private output directory after final-path containment checks. These controls constrain mistakes by +the trusted provider; they are not a claim of hostile-code sandboxing. + +The provider request and response are validated against committed closed JSON Schemas, including +`model-provider-v1.schema.json`; duplicate JSON keys, unknown properties, unsafe integers, and trailing +data are refused. Golden request/response vectors are also run against the Windows fixture executable. +The executable SHA-256 is measured before `describe`, after `describe`, before `convert`, and after +`convert`; all four must match. The staged source hash and provider receipt must equal the caller's +expected hash. Describe/convert provenance must agree exactly. + +### D3 — Complete canonical request and provider fingerprint define identity + +The canonical request is an RFC 8785 JSON Canonicalization Scheme (JCS) object with no omitted-default +ambiguity. Duplicate keys, non-finite values, unsafe integers, and non-canonical Unicode/string inputs +are refused before canonicalization. Committed golden vectors cover key ordering, escaping, Unicode, +numeric forms, negative zero, and collection permutations. It contains: + +- schema/protocol/reader versions; +- format and document-kind policy; +- active-scene-only traversal policy; +- geometry modes and component types accepted; +- source and canonical frames, transform and winding policy; +- join and stable-identity policy; +- canonical JSON and GLB encoding policy; +- every input/output/JSON/scene/node/depth/mesh/primitive/accessor/buffer/vertex/index/entity/property/ + relationship/artifact/timeout limit; +- selection policy (unfiltered/full model in this task); and +- provider conversion settings that affect output. + +`canonicalRequestSha256` hashes those exact JCS bytes. The provider fingerprint is the closed JCS object: + +`(protocolVersion, provider, engine, engineVersion, adapterBuildId, +adapterExecutableSha256, readerSchemaVersion)`. + +`expected-provider-sha256` is SHA-256 of that exact seven-field JCS object. Golden vectors pin its bytes; +The bridge constructs the tuple from five described fields, the host-measured executable hash, and its +embedded schema version, then compares the hash to the pin before conversion. + +The cache key is SHA-256 of JCS bytes containing `sourceSha256`, the complete canonical request, +the complete fingerprint, and the expected signing-key fingerprint. Source, request/configuration, +signer trust anchor, reader schema, provider identity, +engine/version/build, or executable bytes therefore invalidate a hit. Every field appears in the +published manifest and is mutation-tested for preimage coverage, except signer identity: the signer pin +is intentionally cache/authentication metadata outside deterministic manifest/artifact preimages. + +### D4 — GLB normalization is minimal, deterministic, and explicit + +`revit-glb.mjs` parses GLB v2 directly from bytes and rejects duplicate JSON keys at every nesting level +before profile validation. It accepts exactly one JSON chunk and at most one +BIN chunk, verifies declared/file/chunk lengths with checked arithmetic, and rejects unknown required +extensions, external/data/file/HTTP buffer URIs, sparse accessors, Draco/meshopt compression, +morph targets, skins, animations, non-finite values, and unbounded structures. + +An explicit integer `scene` is required; missing active-scene selection is refused rather than guessed. +Only that declared active scene is walked. +The walk detects cycles, excessive depth, duplicate child references, and a node reachable by more +than one parent; those are malformed/ambiguous rather than silently instanced. World matrices compose +parent then local transforms. A node may use either `matrix` or TRS, never both. + +The accepted GLB profile has exactly one URI-less `buffers[0]` bound to the BIN chunk; POSITION is +`VEC3/FLOAT`, all counts and references are safe integers, and chunk, buffer-view, accessor, stride, +alignment, and index ranges are checked with overflow-safe arithmetic. Unused external resources are +still refused. Textures, images, samplers, UVs, normals, tangents, morphs, skins, animations, cameras, +lights, sparse accessors, and compression are unsupported in v1. Every `extensions` object and both +`extensionsUsed`/`extensionsRequired` must be absent; no optional extension semantics are discarded. +Materials may contain only `pbrMetallicRoughness.baseColorFactor`; metallic/roughness use glTF defaults +but do not affect color, `alphaMode` must be absent/`OPAQUE`, `alphaCutoff` absent, `doubleSided` false/ +absent, emissive factor zero/absent, and all texture fields absent. A missing material is opaque white. + +Supported primitives are TRIANGLES, TRIANGLE_STRIP, and TRIANGLE_FAN, indexed or non-indexed, with +`UNSIGNED_BYTE`, `UNSIGNED_SHORT`, or `UNSIGNED_INT` indices. Strides and offsets are checked against +buffer-view and buffer bounds. Strip parity and fan expansion are explicit; degenerate triangles are +counted and dropped. The full world transform is applied first, then glTF Y-up metres are converted to +canonical Z-up millimetres by `(x,y,z) -> (1000*x,-1000*z,1000*y)`. Winding is reversed iff the +determinant of the combined world-plus-frame transform is negative. Matrices must be finite and +nonsingular. Quaternions must be finite and within the configured unit-length tolerance, then are +normalized deterministically. Canonical coordinates are rounded once to IEEE-754 float32, `-0` becomes +`+0`, overflow is refused, and triangles that become degenerate after transformation/rounding are +counted and dropped. Mutations of both determinant +branches, strip parity, and frame axes must fail tests. + +Primitive color comes only from glTF `COLOR_0` and material `baseColorFactor`, with explicit alpha and +deterministic multiplication. `COLOR_0` accepts only VEC3/VEC4 FLOAT (not normalized) or normalized +UNSIGNED_BYTE/UNSIGNED_SHORT; RGB implies alpha 1. All other forms are refused. Canonical geometry is rebuilt as a GLB +whose JSON chunk uses JCS-derived ordering plus fixed glTF layout/padding, whose collections use named +stable sort keys with explicit tie-breakers, and whose buffer is packed little- +endian buffers, and no timestamps, random IDs, absolute paths, or generator-machine data. + +Arrays are classified, not vaguely "sorted": provider parameter-group references, parameters inside a +group, and multipart appearance names preserve semantic provider order and retain duplicates only where +the schema explicitly permits duplicate parameter names. Set-like entities sort by decimal numeric ID; +geometry parts by `(entityId numeric,appearance ordinal,nodeName UTF-8)`; properties by `(entityId, +group ordinal,parameter ordinal,parameter id)`; relations by `(kind UTF-8,from numeric,to numeric, +relation id)`; coverage buckets by `(reason UTF-8,stableId UTF-8)`. Exact duplicate IDs/edges are refused. +JSON object keys use JCS. Permutation invariance is tested only for set-like arrays. +Transformed `(position,color)` records are first encoded to their final canonical bytes, sorted +lexicographically, deduplicated, and assigned canonical indices independent of provider order. Triangles +are then remapped to those indices, rotated to their lexicographically smallest oriented tuple, sorted, +and deduplicated only according to the explicit duplicate-triangle policy. Provider-assigned indices are +never a sort key, making provider vertex/triangle permutations byte-identical under adversarial cold-run +tests. + +### D5 — Metadata is resolved, not inferred + +`revit-metadata.mjs` accepts the committed closed `model-metadata-v1.schema.json`, not an open-ended +projection of one observed provider. Its root is +`{schemaVersion,document:{kind,id},types[],levels[],parameterGroups[],parameters[],elements[],relations[]}`. +Every record has a canonical decimal-string `id`; elements require `id`, explicit nullable `revitClass`, +`category`, `family`, type/level table references, ordered parameter-group references, ordered appearance +node names, and optional exact `ifcGuid`. Parameters require `id`, `name`, optional `unit`, readability +state, a closed `storageType` enum `none|boolean|integer|double|string|element-id`, and one tagged value. +The mapping is exact: none→null/unreadable, boolean→boolean, integer→decimal-string, double→finite-number, +string→string, and element-id→signed int64 decimal-string; mismatches are refused. +Relations require canonical `id`, a kind from `contains|hosts|depends-on|provider-explicit`, and element +endpoints; `contains` and `hosts` are acyclic and single-parent, while the others are directed +multigraphs. `provider-explicit` additionally requires a bounded non-empty `providerRelationKind` that +preserves the provider's exact type. `ifcGuid` is derived only from the authoritative exact `IfcGUID` +parameter record; a redundant provider field, if present, must match byte-for-byte. Unknown fields/kinds +and duplicate table IDs are refused. It resolves all array-index +references (`Type`, `Level`, `ParameterGroups`, `Parameters`) before ordering. An index +must be an in-range safe integer and must reference the expected record kind. Inner Revit `Id` values +are data, never table offsets. + +Actual entity/table identities are canonical positive int64 decimal strings; JSON numbers outside the +safe integer range are refused and no ID is coerced through JavaScript `Number`. Parameter records with +storage type `ElementId` preserve signed int64 decimal strings, including documented negative special +values, and never use them as entity/table references. Entity stable identity is +the source Revit element `Id`, namespaced as `element:`. Durable +cross-revision identity is the exact `IfcGUID` parameter when present and unique; missing and duplicated +GUIDs are explicitly receipted as uncomparable. Category/Family/Type/Level and Revit class come only +from explicit referenced metadata/parameters. Values preserve group order, duplicate names, units, +null, empty, unreadable, numeric, Boolean, and string distinctions; no localized string is parsed into +meaning. + +Geometry joins are explicit: metadata `appearances[]` supplies an ordered non-empty set of exact GLB +node names for an entity. Build multimaps on both sides. Multipart entities are valid, but each active- +scene geometry node must have exactly one entity owner and each claimed name must resolve exactly once. +Duplicate claims, duplicate node names, missing nodes, and unclaimed/watermark nodes are separate +coverage reasons. The reader never +falls back to element order, `_` parsing, node-name similarity, or geometry. + +Hierarchy and relationship edges use namespaced entity IDs, validate both endpoints, are sorted +canonically, and reject cycles where the relationship kind is declared acyclic. Conflicting parents +are `relationship-parent-ambiguous`; they are not resolved by first-wins. Unknown relationship kinds +are retained only as explicitly typed provider relations when the schema allows them; otherwise they +are counted unsupported. + +### D6 — Four deterministic artifacts plus one deterministic manifest + +Normalization produces separate files: + +- `geometry-0000.glb` — canonical binary geometry in Z-up millimetres; +- `entities-0000.json` — identity, classification, bounds, and geometry references; +- `properties-0000.json` — grouped source parameter records keyed by entity; +- `relationships-0000.json` — explicit hierarchy/relationship edges; and +- `manifest.json` — schema/version, source receipt, canonical frames/matrix, complete request hash, + fingerprint, exact coverage, and the ordered receipts for the four component artifacts. + +Files are independently bounded and decodable. JSON uses canonical UTF-8 bytes with no BOM or trailing +newline. GLB stays binary. Artifact names are stable logical names; run-owned AWARE artifact IDs may +be opaque copies, but the deterministic manifest and content hashes never include those runtime IDs. + +Exact coverage reconciles discovered entities, indexed entities, drawable entities, geometry nodes, +properties, relationships, and every skipped/ambiguous/unsupported reason. It includes ordered counts +and digests of the exact stable-ID sets so equal totals cannot conceal different omissions. Publication +fails unless every normalized record belongs to exactly one coverage bucket and all component receipt +byte counts/hashes revalidate. The command response carries a fifth, external receipt for the manifest; +the manifest never contains its own hash or receipt. + +### D7 — Receipts are authenticated without pretending generic secrets exist + +Deterministic artifact bytes contain SHA-256 integrity receipts only. The cache and command response +add an authentication envelope over the manifest receipt plus its four ordered component receipts using the +existing AWARE Ed25519 key format at +`\keys\model-reference-reader.{sec,pub}`. The key is provisioned by the real supported +`aware key generate model-reference-reader` command, not by a fictional generic secret command. + +`preflight` distinguishes: + +- provider not configured/not found; +- provider available but receipt signing key absent; +- fully available with provider fingerprint and signing public-key fingerprint. + +Every converting `probe`/`read-model` requires the signing key and an input `expected-signer-sha256` +obtained out of band; negative preflight performs no conversion. The five receipts are JCS objects in +fixed order `[geometry,entities,properties,relationships,manifest]`. The Ed25519 signature input is +SHA-256 of ASCII `AWARE\0model-reference-reader\0receipt-set\0v1\0` followed by, for each receipt, its +unsigned 64-bit big-endian byte length and exact JCS bytes. The JCS envelope is +`{schemaVersion:"1",algorithm:"Ed25519-SHA256",keyFingerprintSha256,publicKeyBase64, +preimageSha256,signatureBase64}` with canonical base64 and no padding ambiguity. On load, derive the +public key from `.sec`, require byte equality with `.pub`, verify the key fingerprint, then sign. Shared +Node/Rust/FloLess golden vectors pin preimage and signature bytes. Cache hits verify the public key +fingerprint, signature, manifest, every artifact +receipt, every artifact byte, and cache key before reuse. Tests use generated throwaway keys only. +The expected public-key fingerprint is part of request/cache identity. Key rotation is an explicit +configuration change and cache miss. The signature envelope is outside deterministic artifact +preimages, so installations can use different keys without changing normalized artifact bytes or +revision hashes. Authentication detects cache corruption and changes outside the provider TCB; it does +not protect against a malicious same-user provider that can access the signing key under AWARE 0.126.0. + +This does not solve #448: the signing key command already exists in `v0.126.0`; arbitrary provider +credentials still cannot be generically provisioned. + +### D8 — Cache ownership and publication are crash-idempotent + +Each cache key has immutable content-addressed blobs, an immutable complete mapping, and an OS-backed +exclusive lock held for the whole critical section. The Rust `aware __model-reader-host` uses `fs2` +held advisory locks cross-platform and `windows-sys` process/Job/process-start APIs on Windows; the +bridge fails closed if the exact runtime-supplied host is absent. Locks are kernel-released on process +death. A contender records +`{cacheKey, ownerToken, pid, processStartIdentity, startedAt, heartbeatAt}`. +The random owner token is never logged. Only the current token may refresh, cancel, or publish. + +Fresh owners make waiters poll with a bounded deadline and cancellable signal. Successful acquisition of +the kernel lock is the sole authoritative ownership fence; heartbeat and `(pid,processStartIdentity)` are +diagnostic/wait-hint fields only and can never override a held kernel lock. After acquisition, the owner +replaces any stale diagnostic record. A stale process checks ownership before every heartbeat/publication; loss +cancels its provider and forbids publication. + +The winner writes only to a private token-named staging directory, fsyncs files and directory where +supported, and validates the complete result. It then holds a host lock for each digest, revalidates an +existing final blob or atomically renames a completed same-directory token-temp to an absent final name, +and releases the digest lock only after final-byte validation. A contender therefore never observes a +partial final blob. The complete JCS +mapping is also created with `wx` and published last; only that mapping makes an entry visible. A loser +deletes only its own staging data, validates the winner, and returns the winner. +Crash points after provider output, after normalization, after signing, before rename, after rename, +and before lock cleanup all retry to either reuse one valid entry or rebuild; no half-entry is visible. +Cleanup targets only resolved descendants of this cache root. + +Cancellation is two-phase. The orchestrator signals a runtime-owned cancellation pipe/file watched by +the bridge; the bridge stops heartbeat, asks the host to terminate/await its provider Job, removes only +owner-token staging, releases its held lock, and emits `reference-cancelled` within a five-second grace. +If it does not exit, `CliInvoker` force-terminates only the opt-in reader sidecar tree and crash recovery +removes abandoned staging on retry; the trace records `forced-cancel` rather than claiming cooperative +cleanup. Provider and waiter cancellation are tested separately. Cancellation is a delivery gate. +Only a one-shot app graph containing the exact `model-reference-reader` agent opts into the new control +pidfile. That pidfile contains exact process-start identity plus the cancellation endpoint, installs +Ctrl+C handling, and lets `aware app stop` signal that endpoint. Both wait five seconds for trace- +complete cooperative exit before exact-process force termination; every other one-shot graph keeps its +existing concurrent-run behavior, and the existing long-running path retains its behavior. For an +opted-in reader graph, pidfile creation is exclusive per app/instance; a second concurrent run is +atomically refused, and token/run-ID-checked cleanup cannot remove another run's control file. + +The cache appends bounded access observations under a global maintenance lock. Each record's total order +is its monotonic locked journal byte offset; a truncated tail is discarded and compaction is atomic. +Lost observations can only make inactive data appear older, never make active/reachable data evictable. +Deterministic LRU by latest journal sequence then cache-key tie-break evicts under that lock: 20 GiB, +1,000 complete mappings, 6,000 blobs, 128 quarantine entries, and 128 staging entries by +default, all lowerable but not raisable past hard ceilings without a reader-version change. Active +owners and blobs reachable from visible mappings are never evicted. Startup and post-publication sweep +orphan staging/quarantine/blob data within the same bounded quotas. + +### D9 — AWARE command/output contract + +The new agent shares `transport.cli.binary: aware-connection-reader`. At the binary boundary: + +- every valid documented existing `probe`/`read-model` IFC request retains byte-compatible dispatch and + output, and web-ifc remains lazily loaded only for IFC; +- new `preflight` is RVT-only and requires the out-of-band `expected-provider-sha256` pin; +- new converting `probe`/`read-model` require `model-path`, `expected-source-sha256`, the complete + `expected-provider-sha256` fingerprint pin, `expected-signer-sha256`, canonical settings, and a closed + `limits` object containing every lowerable default; mixed `ifc-path`/`model-path` inputs are refused; +- direct bridge and SEA calls use one JSON object on stdin and one JSON object on stdout; +- errors use a stable redacted envelope on stderr and non-zero exit; stdout remains empty. + +`preflight` never converts or reads model bytes. `probe` performs/reuses the conversion and returns a +bounded summary plus the same source/fingerprint/frame/coverage receipt. `read-model` copies the five +verified deterministic files into `AWARE_ARTIFACT_DIR` using opaque safe IDs and returns an AWARE-owned +descriptor containing one standard `$aware-artifact` manifest descriptor plus a typed array of the +four component artifact descriptors, their receipts, and the authentication envelope. No absolute +path appears in output. `read-model` refuses with a stable error when `AWARE_ARTIFACT_DIR` is absent; +raw direct execution remains available only for non-publishing preflight/test harnesses. + +Provider and signer pins are supplied by the caller out of band, hashed into request/cache identity, +compared before conversion/signing, and rotated only by changing those explicit inputs. The bridge's +mandatory `bridge-info` handshake returns `model-reference-reader/v1` plus immutable build ID before any +RVT command; the released IFC-only binary fails that capability fence even if its CLI version matches. + +Run publication writes all five files to invocation-token temporary names inside the artifact directory, +rehashes all five, then commits their opaque final safe IDs. Any failure removes only that invocation's +temporary and already-committed final IDs, so a response never references a partial artifact set. + +The AWARE runtime changes in this slice preserve the bridge's typed error envelope in `AwareError`, add +an optional structured field to `RunEvent::NodeError`, propagate it through the orchestrator while +retaining the legacy string, forward bounded progress, make the managed-sidecar capability fence agent- +aware instead of Tekla-bake-only, and implement opt-in one-shot process- +tree cancellation. The agent is exercised through a temporary compiled `.flo` app and artifact +retrieval because `aware agent invoke` supports built-ins only in 0.126.0. The manifest declares only +schema-supported filesystem/network/software/secret/skill requirements; runtime-owned artifact, +progress, cancellation, environment, and process channels are documented but not misrepresented as +manifest-enforced permissions. + +The manifest will include at least one plain-English skill explaining the provider and artifact +contract, as required for curated commands. The managed sidecar catalogue description broadens from +IFC-only language without adding a second binary. Agent inventory/registry statistics rise from 78 to +79 and are regenerated by `aware agent publish`, repository index/stat tools, and checks, never +hand-forced around a failing guard. + +### D10 — Limits are concrete contract fields + +Defaults and hard ceilings are committed constants and canonical-request fields: provider request/ +stdout 256 KiB/1 MiB, stderr 64 KiB/256 KiB, conversion 10/30 minutes, staged RVT 150 MiB/4 GiB (staged +by streaming copy, never resident), input GLB 128/512 MiB, metadata 16/64 MiB, aggregate provider output +144/576 MiB, GLB JSON 4/16 MiB with nesting 64/128, scenes 8/32, active nodes 100,000/250,000 at depth +128/256, meshes 100,000/250,000, primitives 200,000/500,000, accessors and buffer views +250,000/1,000,000, vertices 5,000,000/10,000,000, indices 15,000,000/30,000,000, entities +250,000/1,000,000, parameters 2,000,000/5,000,000, relationships 1,000,000/2,000,000, component JSON +32/128 MiB, canonical GLB 256/512 MiB, command response 1 MiB, and each progress frame 8 KiB. The v1 +implementation is deliberately in-memory and has a measured 1 GiB resident hard gate; admission uses +checked worst-case allocation estimates before parsing. Logical limit tests inject tiny ceilings and +exercise exact/one-over cases cheaply; a named Windows stress lane covers real maximum GLB/metadata/ +resident ceilings, while the 4 GiB staged-copy limit uses sparse/streaming tests. Multiplication and +addition are checked before allocation. + +## 3. Threat model and refusal matrix + +| Threat / failure | Admission or detection | Stable outcome | +| --- | --- | --- | +| Relative/provider shell/path search | Absolute regular executable, no shell/PATH | `reference-provider-unsafe` | +| Source traversal/symlink/change | `.rvt`, regular file, private staged copy and hash agreement | `reference-source-unsafe` / `reference-source-changed` | +| Untrusted/malicious provider | Explicitly outside this task's guarantee; provider is installed TCB | preflight refuses unapproved fingerprint; no sandbox claim | +| Provider declares remote behavior | Local-only describe receipt; no URL input/client in bridge | `reference-provider-nonlocal` | +| Secret/path/model leak | Redacted envelope and structured safe fields only | `reference-provider-failed` with diagnostic ID | +| Chatty/hung provider | bounded pipes + deadline + process-tree termination | `reference-provider-output-too-large` / `reference-provider-timeout` | +| Output escape/reparse/extra file | private dir containment, regular files, exact file set | `reference-output-unsafe` | +| Malformed/truncated/oversized GLB | checked GLB/chunk/accessor arithmetic and caps | `reference-geometry-invalid` / `reference-output-too-large` | +| External/implicit resource read | no GLB URI and no unsupported extension | `reference-external-resource-refused` | +| Scene cycle/multiple parents | active-scene graph validation | `reference-scene-invalid` | +| Unsupported geometry | explicit reason; no silent reinterpretation | `reference-geometry-unsupported` | +| Ambiguous metadata join | exact bidirectional multimap | `reference-metadata-join-ambiguous` or covered skip | +| Indexed metadata drift | in-range typed resolution before canonical IDs | `reference-metadata-invalid` | +| Relationship cycle/conflicting parent | endpoint/type/acyclic checks | `reference-relationship-invalid` | +| Source/provider/request drift | complete cache-key preimage + bracketing hashes | cache miss/refusal, never stale hit | +| Tampered cache | signature + full manifest/artifact revalidation | quarantine/rebuild; no hit | +| Concurrent/crashed writer | held OS fence, exact process identity, content CAS, mapping-last publish | wait/takeover/validated winner | +| Cancellation/orphan descendants | runtime cancel propagation + Job/process group + token cleanup | `reference-cancelled` | +| Artifact flooding | per-file/aggregate/count/read/chunk limits | `reference-output-too-large` | +| Cache disk exhaustion | quotas, mapping-aware deterministic eviction, orphan sweep | bounded eviction or `reference-cache-full` | + +Errors include only `{code, phase, retryable, message, diagnosticId}`. Diagnostic IDs are random and +not cache/revision inputs. Logs may include phase, elapsed time, cache result, fingerprint fields, +digest prefixes, counts, and diagnostic ID; they may not include raw provider output, full source or +provider paths, filenames, parameter values, credentials, URLs, owner tokens, or artifact content. + +## 4. Test-first implementation slices + +### Slice A — Generated fixtures, canonical primitives, and contract tests + +**Create:** + +- `cli-connection-reader/model-fixtures.mjs` +- `cli-connection-reader/model-fixtures.test.mjs` +- `cli-connection-reader/model-contract.mjs` +- `cli-connection-reader/model-contract.test.mjs` +- `cli-connection-reader/model-provider-v1.schema.json` +- `cli-connection-reader/model-metadata-v1.schema.json` + +Generate tiny GLB bytes in memory from declarative scene/primitive inputs and tiny provider-metadata +objects from source text. Do not commit generated `.glb` or model bytes. Tests first pin RFC 8785 JCS, +schema conformance/golden vectors, 64-bit decimal-string IDs, every exact/one-over limit, +all request/fingerprint fields in the cache preimage, stable error envelopes, and +fixture determinism. Mutation loop changes every leaf of request/fingerprint and requires a different +hash. + +**Commit:** `test: define the RVT reader's deterministic boundary`. + +### Slice B — GLB parser and geometry normalization + +**Create:** `revit-glb.mjs`, `revit-glb.test.mjs`. + +Write failing tests for GLB headers/chunks, external URIs, accessors/stride/offset/index types, indexed +and non-indexed triangles/strips/fans, degenerates, active versus inactive and missing scenes, nested +matrix/TRS world transforms, duplicate parents/cycles/depth, positive/negative determinant winding, +Y-up metres to Z-up millimetres, vertex/material colors, multiple primitives, unsupported extensions/ +geometry, missing/extra BIN bindings, singular transforms, quaternion tolerance, float32 overflow, +negative-zero, post-transform degeneracy, stable sort ties/permutations, malformed ranges/checked +overflow, non-finite values, and every limit. Implement only the minimum parser +that makes those cases pass. Add mutation controls that deliberately remove the frame transform, +winding reversal, active-scene filter, and range bound and prove the tests fail. + +**Commit:** `feat: normalize bounded Revit GLB geometry`. + +### Slice C — Explicit Revit metadata and join normalization + +**Create:** `revit-metadata.mjs`, `revit-metadata.test.mjs`. + +Write failing tests for indexed Type/Level/ParameterGroup/Parameter resolution; source `Id` and exact +`IfcGUID`; Category/Family/Type/Level/class; group order, duplicates, units and value states; unique, +missing, duplicate, unclaimed, and watermark appearance joins; stable namespaced IDs; hierarchy cycles; +multipart entities; ambiguous parents; missing endpoints; unsupported relationship kinds; exact coverage; deterministic +entity/property/relationship bytes. Mutation controls break one index, duplicate one join, remove one +GUID, and swap one relationship endpoint and must fail reconciliation. + +**Commit:** `feat: preserve explicit Revit metadata and relationships`. + +### Slice D — Provider process, provenance, safety, and redaction + +**Create:** + +- `model-provider.mjs` +- `model-provider.test.mjs` +- `test-fixtures/model-provider-fixture.mjs` + +The fixture adapter has both a deterministic script form and a built Windows executable used through +the real child-process protocol; it writes generated GLB/metadata to the requested private directory. +Tests first cover minimal environment/private cwd, schema/golden-vector conformance, immutable staged +source handling, descendant process containment, describe/convert agreement, executable and source +changes at every bracket, timeout, cancellation, +bounded stdin/stdout/stderr, non-zero exit, malformed/duplicate/missing receipt fields, wrong document +kind, wrong source hash, remote execution, unsafe/external/reparse paths, extra files, oversized files, +and error/log redaction. On Windows, add explicit junction/reparse and process-tree cases; on platforms +where a primitive cannot be created, report the unverified branch rather than skip silently. + +**Commit:** `feat: fence the local model provider protocol`. + +### Slice E — AWARE one-shot lifecycle, structured errors, and telemetry + +**Modify:** + +- `cli/src/runtime/invoker.rs` +- `cli/src/error.rs` +- `cli/src/runtime/provenance.rs` +- `cli/src/runtime/orchestrator.rs` +- `cli/src/runtime/lifecycle.rs` +- `cli/src/runtime/pidfile.rs` +- `cli/src/commands/app.rs` +- `cli/src/commands/model_reader_host.rs` +- `cli/src/main.rs` +- `cli/src/commands/mod.rs` +- `cli/Cargo.toml` +- `cli/Cargo.lock` +- `cli/src/commands/sidecar.rs` +- focused Rust integration tests and platform helpers + +Write failing tests for the internal provider supervisor/held-lock protocol, exact process-start +identity, dedicated child/grandchild Job/process-group exit, cooperative and forced cancellation, and +kernel lock release on crash. Prove multiplexed stdin/stdout/stderr correlation under two concurrent +out-of-order provider runs, and prove the exclusive control pidfile applies only to app graphs containing +the exact model-reader agent while unrelated one-shot apps remain concurrent. Preserve a bridge's bounded +typed error envelope instead of collapsing it to a generic message, forward bounded success progress, +and require the protocol/build capability only for the exact `model-reference-reader` agent ID, avoiding +an invented manifest field in 0.126.0. Existing non-managed CLI +agents and intentional survivor lifecycles including Tekla launch/watch remain compatible. + +**Commit:** `fix: fence managed sidecar lifecycles`. + +### Slice F — Authenticated content-addressed cache and crash recovery + +**Create:** `model-cache.mjs`, `model-cache.test.mjs`. + +Tests first cover key derivation, generated AWARE-format Ed25519 keys, receipt sign/verify, complete hit +validation, tampered/missing/extra files, source/request/fingerprint/provider-binary/signer invalidation, +concurrent same-key winner, cancellable waiter, fresh owner, dead owner, stale live owner, reused PID +with a different process start, unverifiable owner, fenced takeover, old-token heartbeat/publish +refusal, each publication crash point, blob/mapping `wx` destination race, winner +validation, private permissions, token-scoped cleanup, and restart idempotency. Run two cold conversions +with the same source/request/provider but empty entries and assert identical hashes and bytes for all +five deterministic artifacts. + +Add quota/eviction tests for each entry/blob/byte/staging/quarantine ceiling, deterministic LRU ties, +reachable-blob protection, active-owner protection, orphan sweep, and `reference-cache-full`. + +**Commit:** `feat: publish crash-safe authenticated model conversions`. + +### Slice G — Multi-artifact reader commands and IFC compatibility + +**Modify:** + +- `cli-connection-reader/index.mjs` +- `cli-connection-reader/model-dispatcher.mjs` +- `cli-connection-reader/package.json` + +**Create:** `cli-connection-reader/model-reader.test.mjs`. + +Tests first drive `preflight`, `probe`, and `read-model` through the actual CLI process with JSON stdin. +Cover negative preflight without conversion, key/provider readiness, mixed/missing paths, cache miss/hit, +bounded summary, five artifacts, exact receipt/coverage, binary GLB equality, stdout purity, redacted +errors, expected signer trust/rotation, absent artifact directory, progress phases, cancellation, and +deterministic cold runs. Enumerate every valid documented IFC request shape, snapshot command bytes +before wiring, and require byte-identical stdout afterward plus lazy web-ifc loading. + +Leave `index.mjs` as the IFC-compatible module with all synchronous named exports intact. Make +`model-dispatcher.mjs` the package bin and SEA entrypoint; it dynamically imports `index.mjs` only for +IFC and routes RVT without loading web-ifc. Existing IFC imports/exports and behavior remain in place. + +**Commit:** `feat: expose deterministic RVT reference artifacts`. + +### Slice H — Agent, sidecar catalogue, registry, and documentation + +**Create:** + +- `20-agents/aeco/engineering/model-reference-reader/manifest.yaml` +- `20-agents/aeco/engineering/model-reference-reader/commands/preflight.md` +- `20-agents/aeco/engineering/model-reference-reader/commands/probe.md` +- `20-agents/aeco/engineering/model-reference-reader/commands/read-model.md` +- `20-agents/aeco/engineering/model-reference-reader/skills/provider-and-artifact-contract.md` + +**Modify:** + +- `cli/src/commands/sidecar.rs` (description/docs only; same bridge ID and asset) +- `cli/tests/agent_list.rs` (79 with explicit new-agent assertion/history) +- generated `registry-index.json` and synchronized stats/docs reported by the official tools + +Manifest docs explicitly state local-only execution, no committed provider, #448's secret-provisioning +limit, expensive first probe, canonical frame, exact joins, separate artifacts, and the distinction +between missing provider/key versus conversion failure. Then run: + +```powershell +aware agent publish 20-agents/aeco/engineering/model-reference-reader +python scripts/sync_stats.py --write +aware agent reindex +aware agent reindex --check +``` + +Use a temporary `AWARE_HOME` for install/compile tests. Do not hand-edit generated counts to bypass a +tool failure. + +**Commit:** `feat: publish the local RVT model reader agent`. + +### Slice I — SEA, Windows harness, and no-fallback packaging + +**Modify:** `cli-connection-reader/build.mjs`, `.github/workflows/ci.yml`, package metadata/lock only if +dependencies change. Schemas are imported as static data and embedded by esbuild; runtime filesystem +schema reads are forbidden. Rust host dependencies/features are built into `aware`, not adjacent assets. + +Add a packaged-executable test/harness that copies only `aware-connection-reader.exe` and +`web-ifc-node.wasm` to a fresh clean staging directory, places the fixture provider and source in a +separate authorized input directory, changes cwd away from the repository, hides/renames no user data, +and drives all three RVT commands. Assert source modules and adjacent repo files are absent/unreadable, +GLB remains binary, component hashes match source-mode output, and existing IFC fixture read still works. +The provider itself remains separate by design and is supplied via its absolute configuration path. +Run that harness in the Windows CI lane so SEA/provider/process-tree behavior is not a local-only claim. + +**Commit:** `build: keep RVT normalization inside the shared SEA`. + +## 5. Verification ledger + +### Focused and compatibility tests + +```powershell +npm test -- --test-name-pattern "model|Revit|RVT|GLB|cache|provider|receipt" +node --test model-contract.test.mjs model-fixtures.test.mjs revit-glb.test.mjs revit-metadata.test.mjs model-provider.test.mjs model-cache.test.mjs model-reader.test.mjs +node --test extract.test.mjs probe.test.mjs read-model.test.mjs recognize.test.mjs compare.test.mjs +npm test +npm run build +``` + +Baseline note: `npm test` currently has one deliberate failure in `compare.cli.test.mjs` because no +external sample IFC is installed; 115 pass and 56 skip. The final report must distinguish this +environmental corpus gate from new failures. In-repo IFC fixtures and all new RVT fixtures must pass. + +### Repository gates + +```powershell +python scripts/sync_stats.py --write +aware agent reindex --check +cargo fmt --manifest-path cli/Cargo.toml --all -- --check +cargo clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings +cargo test --manifest-path cli/Cargo.toml +``` + +Run the current source-built CLI where installed `aware` behavior could differ. Record exact CLI +version and manifest/command acceptance output. + +### Determinism and mutation proof + +- Delete only two isolated test cache entries, convert the same generated source twice, and record + identical SHA-256 for geometry, entities, properties, relationships, and manifest. +- Change source bytes, each provider fingerprint field, canonical request/configuration, and adapter + executable bytes independently; each must miss or refuse. +- Mutate critical test seams (frame matrix, winding branch, join uniqueness, one artifact byte, receipt + signature, owner token, heartbeat freshness, final source hash) and record that the targeted test goes + red before restoring the implementation. + +### Authorized live drill + +Only if a licensed local provider executable and Residential input are present and explicitly +authorized: + +1. Use a temporary AWARE home and generate the supported receipt key. +2. Build/install the shared SEA and install/reindex the agent locally. +3. Publish/reindex the curated agent, compile a temporary `.flo` app that invokes each command, and run + `preflight`, `probe`, and `read-model` through `aware app run`; do not use unsupported direct custom- + agent invocation. +4. Retrieve every artifact through `aware app artifact`, not by treating IDs as paths. +5. Record source hash, canonical request hash, full provider fingerprint, public-key fingerprint, + artifact hashes, exact coverage, finite canonical bounds, trace/diagnostic IDs, and representative + explicit Category/Family/Type/Level/parameter/relationship records. +6. Run two cold conversions and compare bytes/hashes. +7. Exercise CLI cancellation and require the one-shot runtime to terminate the complete provider tree. + +If the standalone provider, license, generic credential provisioning, or authorization is unavailable, +do not use the old cloud credential or committed URL. Mark the live branch blocked precisely and leave +the FloLess create card gated. + +## 6. Definition of done + +- Every required happy, malformed, unsafe, oversized, concurrency, crash, cancellation, and + determinism branch has a test that can be shown to fail under mutation. +- The provider/request/fingerprint preimages are complete and closed; every hit revalidates full + signed receipts and bytes. +- Geometry is active-scene-only, fully transformed, canonical Z-up millimetres, correctly wound and + colored; GLB bytes never pass through text. +- Entities, properties, relationships, and geometry are separate, bounded, deterministic artifacts + with exact reconciled coverage and no inferred Revit meaning. +- SEA runs from a clean staging directory without source-tree or adjacent-source fallback. +- Existing IFC output remains byte-compatible and its in-repo tests pass. +- The new manifest and commands are accepted by the source/current CLI; agent registry and count guards + are honest. +- No Residential bytes, converted commercial artifacts, provider URL/binary, credential, token, secret, + absolute sensitive path, or scratch file is committed. +- AWARE commits are coherent, conventional, and have no `Co-Authored-By`; no push/release/merge occurs. +- The final handoff records commits, commands, hashes, fingerprints, trace IDs, known baseline skips, + verified defects/issues, and every unverified live branch. + +## 7. Required Xeorvt integration handoff + +After the AWARE branch is committed and all attainable AWARE verification is recorded, invoke the +user-requested `xeorvt-integrate` skill against the FloLess repository. Task 6's implementation commit +belongs to `aware-aeco/aware` and cannot be merged across repositories; the integration workflow must +therefore first establish whether Task 6 produced any coherent FloLess source commit to merge. It must +not manufacture one or stage the three preserved untracked handoffs. + +The skill then fetches FloLess `origin`, records all worktrees/status/SHAs, safely advances a clean +local `master`, merges the current master into `codex/xeorvt-reference-model` when required, and runs +its full post-merge acceptance gate on the exact final Xeorvt HEAD. It may create local commits and +merge commits only. It does not authorize a push, PR, release, worktree deletion, force operation, or +merge of Xeorvt into product `master`. If current FloLess master or the integration worktree is dirty, +divergent, or otherwise unsafe, stop and report the precise blocker rather than widening scope. diff --git a/docs/superpowers/specs/2026-08-23-aware-rvt-reader-review-log.md b/docs/superpowers/specs/2026-08-23-aware-rvt-reader-review-log.md new file mode 100644 index 000000000..7a853b5d3 --- /dev/null +++ b/docs/superpowers/specs/2026-08-23-aware-rvt-reader-review-log.md @@ -0,0 +1,269 @@ +# AWARE RVT reader — adversarial plan review log + +**Plan:** `docs/superpowers/specs/2026-08-23-aware-rvt-reader-plan.md` + +**Review model:** Codex CLI, read-only sandbox, maximum five rounds. + +**Status:** Final bounded round complete. Round 5 returned `REVISE`; all three concrete findings were +accepted into the plan. The user explicitly authorized implementation after resolving them. + +## Round 1 + +### Codex critique + +Task 6 is not safe to implement as written. Material defects: + +1. **The provider boundary is not enforced.** A self-reported `execution:"local"` cannot prevent the + executable from accessing the network, arbitrary files, the cache, or the signing key; “may write + only inside its output directory” is therefore false. Fix — Run the provider under an OS-enforced + restricted identity/AppContainer with deny-network and directory ACLs, or explicitly place it in + the trusted computing base and remove the sandbox claims. +2. **The provider inherits the complete AWARE environment.** Node spawning inherits environment + variables by default, potentially exposing credentials, `AWARE_HOME`, artifact paths, and progress + paths. Fix — Spawn with a minimal allowlisted environment and private working directory, passing + only explicitly approved provider variables. +3. **Receipt signing does not protect against a malicious provider.** The same-user provider can read + or replace `\keys\model-reference-reader.sec` and forge receipts. Fix — Isolate signing + into a process/OS key store inaccessible to the provider, and document exactly which attacker the + signature is intended to resist. +4. **Source hashing has a TOCTOU hole.** Pre/post hashing the pathname does not stop the source from + being swapped for conversion and restored before the final hash. Fix — Copy bytes once into a + private immutable staging file, hash that copy, and give only that staged path to the provider. +5. **Provider descendants can survive successful conversion.** Process-tree termination is specified + only for timeout/cancellation, so a provider can exit after spawning a child that races output + validation or mutates files later. Fix — Put every provider invocation in a kill-on-close Job + Object/process group and ensure the entire group is quiescent before opening outputs. +6. **The metadata protocol is not actually defined.** “A closed bounded projection of the observed + provider metadata” supplies no normative schema, required fields, types, reference tables, + relationship enum, or unknown-field policy. Fix — Add a versioned JSON Schema plus canonical + examples, duplicate-key rejection, exact enums, bounds, and cross-provider conformance vectors. +7. **Revit element IDs can lose identity in JavaScript.** Modern Revit element IDs are 64-bit, but the + plan does not require decimal strings and could collapse distinct IDs above + `Number.MAX_SAFE_INTEGER`. Fix — Require element IDs as canonical decimal strings and reject numeric + IDs outside the safe-integer range. +8. **The geometry join rejects ordinary multipart elements.** Requiring exactly one node per entity + will classify valid elements exported as several meshes/nodes as unusable. Fix — Permit an entity + to claim an ordered nonempty set of nodes while requiring every claimed node to have exactly one + entity owner. +9. **The active-scene fallback is wrong.** glTF does not specify scene 0 when `scene` is absent; clients + may defer rendering until a scene is selected. Fix — Require an explicit valid `scene` index, or + make the provider protocol explicitly select a scene and include that policy in the cache key. +10. **Critical glTF validation rules are missing.** The plan does not pin `POSITION` to `VEC3/FLOAT`, + validate index values against vertex count, reconcile attribute counts, bind the BIN chunk to the + sole URI-less buffer, or reject ignored textures/material extensions. Fix — Add a complete + accepted-glTF profile and adversarial tests for every reference, accessor, alignment, count, + material, texture, and extension rule. +11. **Singular and overflowed transforms are unspecified.** A zero determinant, non-unit quaternion, + float overflow after millimetre scaling, and triangles made degenerate by transforms have no + stable outcome. Fix — Define validation and coverage behavior for singular matrices, quaternion + normalization, `-0`, float32 rounding, overflow, and post-transform degeneracy. +12. **Determinism is tested only with a deterministic provider.** “Stable order” lacks a sorting key, + Unicode rule, numeric canonicalization, and shuffled-input equivalence tests. Fix — Specify + bytewise ordering and canonical number/string rules, then permutation-test semantically identical + GLB and metadata inputs across cold runs. +13. **Canonical JSON is underspecified for the FloLess consumer.** Recursive key sorting alone does + not define Unicode ordering, escaping, negative zero, exponent form, or cross-language + reproducibility. Fix — Adopt a named canonicalization standard such as RFC 8785/JCS and share + golden byte/signature vectors with the Task 5 implementation. +14. **The manifest receipt can become self-referential.** The plan says the manifest contains ordered + artifact receipts while also requiring every artifact, apparently including the manifest, to have + a receipt. Fix — State that the manifest contains receipts for the four components only, while the + external response hashes the manifest and the signature covers those exact five receipts. +15. **The signature has no consumer trust anchor.** An embedded public key or fingerprint proves only + that some key signed the response unless FloLess already knows which key to trust. Fix — Require + Task 5 to pin an out-of-band public-key fingerprint and define rotation/revocation behavior. +16. **The cache lease is not a real fencing primitive.** A heartbeat rewrite can race stale-lock rename, + and ownership can be lost between the final token check and publication. Fix — Use an OS-backed + exclusive lock keyed by PID plus process-start identity, allow takeover only after proving that + exact process dead, and hold the fence through publication. +17. **“Atomic no-clobber rename” is not portable Node behavior.** `rename` may replace an existing + destination, while Node exposes no cross-platform `renameat2(RENAME_NOREPLACE)` equivalent. Fix — + Name the concrete no-replace primitive/helper and add a last-instruction publication race test on + Windows, the shipped platform. +18. **The cache is an unbounded persistent disk-DoS.** Per-entry limits do not limit the number of + sources, requests, provider builds, quarantines, or abandoned staging directories. Fix — Add + total-byte/entry quotas, deterministic eviction, stale-stage cleanup, quarantine retention, and + disk-full tests. +19. **Cancellation is simultaneously required and knowingly unimplemented.** The DoD requires + cancellation, but the plan permits shipping after merely filing an issue if AWARE does not + propagate it; current one-shot invocation simply awaits the child. Fix — Either implement + kill-on-drop/process-tree cancellation in `CliInvoker` or remove runtime cancellation from Task + 6's claimed scope and DoD. +20. **Structured errors will not remain structured through AWARE.** `CliInvoker` treats stderr as text + and wraps it in a network error. Fix — Teach the runtime to recognize and preserve the exact JSON + error envelope, or document that stable codes exist only at the raw bridge boundary. +21. **Successful operational logs are discarded.** The runtime captures successful stderr without + publishing it, so proposed cache/timing/takeover diagnostics will not reach the trace. Fix — Emit + bounded cache state, phase timings, provider exit status, quarantine/takeover reason, and limit + failures through `AWARE_PROGRESS_FILE` and the response telemetry envelope. +22. **Shared-sidecar versioning is unsafe for the new agent.** A stale `aware-connection-reader` is only + warned and executed; the hard-current gate applies solely to Tekla bake. Fix — Make the gate + agent-aware and require a current/protocol-compatible bridge for every `model-reference-reader` + command. +23. **IFC byte-compatibility conflicts with mixed-path refusal.** An IFC request containing an ignored + extra `model-path` currently succeeds but would newly fail, and eager model-module initialization + could make valid IFC calls depend on provider configuration. Fix — Define preservation as + valid-schema calls only, lazily import RVT modules after dispatch, and regression-test every IFC + command under missing and hostile provider configuration. +24. **The new manifest's capabilities are unspecified.** It reads RVT/provider/key files, writes + cache/artifacts, and executes external software, but Slice G does not prescribe corresponding + `requires.filesystem` and `requires.software` declarations. Fix — Add exact read/write/software + declarations and validation tests, while acknowledging the current CLI subprocess is not an OS + sandbox. +25. **RVT `read-model` has no contract when `AWARE_ARTIFACT_DIR` is absent.** Existing direct bridge + calls work without it, whereas five binary artifacts cannot be returned as the normal JSON + response. Fix — Define a stable RVT-only refusal when the directory is absent, or add an explicit + safe direct-output contract and test both modes. +26. **Numeric limits are promised but never selected.** Without concrete defaults and maxima, tests + cannot prove useful boundedness, memory behavior, or compatibility with realistic RVT output. Fix + — Put every default/hard ceiling and aggregate memory/disk/time budget in the normative contract + and test each boundary plus one-over cases. +27. **The Windows SEA/provider harness is not CI-gated.** Current bridge CI runs source tests on Ubuntu + only, and an `.mjs` fixture is not directly executable with `shell:false` on Windows. Fix — Add a + Windows CI job that builds the SEA, packages the fixture provider as a real executable, changes + cwd, and runs the clean-stage RVT and IFC smoke tests. +28. **The registry generation commands are incomplete.** `aware agent reindex` regenerates + `registry-catalog.json`, not `registry-index.json`; the plan omits the supported + `aware agent publish ` step. Fix — Stage the index entry with `aware agent publish`, + then run reindex, reindex-check, and stats synchronization. + +`VERDICT: REVISE` + +### Builder response + +Accepted all concrete contract corrections, with these scoped decisions: + +- The separately installed local adapter is explicitly part of the trusted computing base. This task + will not claim an AppContainer/deny-network sandbox that AWARE does not provide. The bridge itself + has no network path, gives the adapter a minimal environment and private cwd, and stages one immutable + source copy. The receipt signature protects cache/publication integrity against corruption and + non-TCB writers; it does not defend against a malicious same-user provider. +- Runtime process-tree containment, structured bridge errors, current-sidecar fencing, and successful + telemetry are added to Task 6 rather than waived. On Windows, a kill-on-close Job Object spans the + sidecar and provider descendants; cancellation is not considered verified until an actual app-run + cancellation proves cleanup. +- Cache takeover is narrowed to an exact dead `(pid, processStartIdentity)` owner. A merely old + heartbeat from a live or unverifiable owner never permits takeover. Publication uses content-addressed + files created with exclusive `wx`, then an exclusive mapping/complete record created last; it does + not depend on replace-prone rename semantics. +- The five-file manifest/signature relation, multipart joins, explicit scene, glTF profile, 64-bit + decimal string IDs, RFC 8785 bytes, concrete limits, cache quotas, Windows CI SEA gate, declared + permissions, and correct `agent publish`/reindex workflow are made normative. +- Direct `aware agent invoke` is removed because current AWARE intentionally supports builtin agents + only. `preflight`/`probe` may run at the raw bridge boundary without an artifact directory; + `read-model` refuses with a stable code. Real AWARE proof uses a temporary `.flo` app and run-owned + artifact retrieval. + +## Round 2 — Codex critic + +`VERDICT: REVISE` + +The critic found 20 remaining material gaps: + +1. A Job containing bridge and provider cannot kill only a timed-out provider; use a supervisor with a + separate nested provider Job/process group and wait for zero descendants before validation. +2. Generic sidecar kill-on-close would break intentional survivor lifecycles such as Tekla launch; + containment must be model-reader-specific with compatibility tests. +3. Hard-kill cancellation cannot promise bridge cleanup; define cooperative cancellation plus grace, + then forced termination and crash-recovery semantics. +4. Typed errors cannot reach traces while `RunEvent::NodeError` is string-only; extend provenance and + orchestrator additively while retaining the legacy string. +5. CLI version is not a bridge protocol fence; require a capability/build handshake. +6. The manifest schema does not expose process/environment/artifact/progress permissions; declare only + supported requirements and document runtime-owned channels. +7. Public command inputs omit expected source/provider/signer pins and lowerable limits. +8. Provider approval has no defined pin/rotation mechanism. +9. Naming a metadata schema is not enough; specify root tables, required fields, value union, relations, + and acyclic kinds before implementation and validate against an authorized real sample if available. +10. Define signature domain, framing, encoding, key consistency, and cross-language vectors byte-for-byte. +11. Select a concrete SEA-compatible Windows held-lock and process-start primitive. +12. `wx` blob creation exposes partial files across keys; serialize per digest and commit completed temp + files while holding that digest lock. +13. LRU observations need a separately ordered crash-tolerant journal. +14. The proposed 256 MiB peak is incompatible with the in-memory limits; lower limits or design a real + streaming/external-sort pipeline. +15. Multi-gigabyte exact hard-limit tests do not fit the current CI lane; inject small ceilings for logic + and reserve named stress tests for an appropriate lane. +16. Classify every array as semantic-order or set-like and define exact keys/ties/duplicate rules. +17. Reject every glTF extension outside an explicit allowlist and enumerate exact material/color rules. +18. Make lazy IFC loading concrete by moving the existing implementation intact behind a dispatcher. +19. Include Rust dependencies/Windows features and statically embedded schema packaging in scope. +20. Publish five run artifacts through invocation-owned temporary names, validate all, commit final IDs + only on complete success, and clean only invocation-owned paths after failure. + +### Builder response + +Accepted all 20. The provider gets a model-reader host helper in the AWARE Rust CLI: it owns a separate +provider Job/process group, offers held kernel locks and exact process identity, and participates in +two-phase cancellation. This is opt-in and leaves intentional Tekla survivor lifecycles unchanged. +The plan will add the missing provenance/orchestrator files, a capability handshake, only schema-valid +manifest requirements, complete pinned command inputs, a normative metadata structure, exact receipt +framing, per-digest locked publication, ordered cache-access journal, realistic in-memory ceilings and +injectable small-limit tests, complete array/glTF rules, an IFC dispatcher, embedded schemas, and +failure-atomic run-artifact publication. + +## Round 3 — Codex critic + +`VERDICT: REVISE` + +Nine narrower contradictions remained: one-shot runs had no cooperative stop channel; the long-lived +host protocol was unspecified; making `index.mjs` both a lazy dispatcher and synchronous IFC export +module was impossible; signer identity conflicted with deterministic manifest bytes; the provider pin +had no exact preimage; kernel locks and stale-PID policy competed as authorities; provider-explicit +relations and duplicate GUID sources were ambiguous; unsigned identity rules incorrectly covered signed +Revit `ElementId` parameter values; and the plan named the nonexistent root Cargo lock/manifest. + +### Builder response + +Accepted all nine. The plan now adds one-shot pidfile/cancellation lifecycle work; a bounded framed +long-lived helper protocol; `model-dispatcher.mjs` as the lazy executable while `index.mjs` preserves IFC +exports; signer identity only in cache/external authentication; an exact seven-field JCS provider pin; +kernel-lock acquisition as sole ownership authority; explicit provider relation kinds and one GUID +authority; positive entity/table IDs versus signed int64 ElementId parameter values; and correct +`cli/Cargo.toml` / `cli/Cargo.lock` commands and paths. + +## Round 4 — Codex critic + +`VERDICT: REVISE` + +Seven remaining protocol details were identified: multiplex host control/binary frames; per-instance +one-shot pidfile exclusion; bridge construction of the provider fingerprint; required parameter storage +types; duplicate-key detection in GLB JSON; canonical topology ordering; and explicit Rust command/agent +fencing files. + +### Builder response + +Accepted all seven. The plan now specifies concurrent typed framing and immediate handles, exclusive +run-token pidfiles, bridge-composed fingerprint pins, exact storage-type/value mappings, duplicate-key +GLB parsing, canonical triangle/vertex remapping, and explicit `main.rs`/`commands/mod.rs` changes with +an exact-agent-ID fence rather than a nonexistent manifest field. + +## Round 5 — Codex critic + +1. Multiplexed stdout/stderr frames contain no request/run identifier, so concurrent out-of-order + provider results cannot be associated safely. Fix — Prefix every binary frame with request ID, run + handle, sequence, and final flag, and specify concurrent draining plus stdin byte encoding. +2. Triangle sorting still uses provider-assigned indices before canonical vertex remapping, so vertex + permutations can change triangle order and final bytes. Fix — First sort/deduplicate transformed + `(position,color)` byte tuples and assign canonical indices, then rotate and sort triangles using + those canonical indices. +3. Exclusive pidfiles are applied to the entire one-shot app path, breaking existing concurrent one-shot + runs despite cancellation being described as model-reader opt-in. Fix — Apply singleton pidfile/ + cancellation behavior only to app graphs containing the exact `model-reference-reader` agent, + preserving all other one-shot concurrency. + +`VERDICT: REVISE` + +### Builder response + +Accepted all three material findings. The host protocol now gives every control and binary frame an +exact request ID, run handle, sequence, final flag, and bounded byte payload; provider stdin is an +explicit fourth binary stream, with length reconciliation before launch and concurrent output draining. +Geometry canonicalizes transformed position/color bytes and assigns provider-independent vertex indices +before triangle rotation/sorting. The singleton control pidfile is now opt-in only for a one-shot graph +containing the exact model-reader agent, with regression coverage for unrelated concurrent one-shot apps. + +The five-round cap is reached without a literal `APPROVED` verdict, so the transcript records bounded +non-convergence rather than claiming approval. There is no remaining builder/critic disagreement: every +round-5 finding is incorporated, and the user's continuation request already supplied the human gate to +implement after resolving the final material findings. From 91d786fba585bcddc3a0799bffec02e499de031e Mon Sep 17 00:00:00 2001 From: Pawel Date: Sun, 23 Aug 2026 13:59:02 +0200 Subject: [PATCH 02/50] test: define the RVT reader's deterministic boundary --- cli-connection-reader/model-contract.mjs | 283 ++++++++++++++++++ cli-connection-reader/model-contract.test.mjs | 72 +++++ cli-connection-reader/model-fixtures.mjs | 71 +++++ cli-connection-reader/model-fixtures.test.mjs | 19 ++ .../model-metadata-v1.schema.json | 18 ++ .../model-provider-v1.schema.json | 18 ++ 6 files changed, 481 insertions(+) create mode 100644 cli-connection-reader/model-contract.mjs create mode 100644 cli-connection-reader/model-contract.test.mjs create mode 100644 cli-connection-reader/model-fixtures.mjs create mode 100644 cli-connection-reader/model-fixtures.test.mjs create mode 100644 cli-connection-reader/model-metadata-v1.schema.json create mode 100644 cli-connection-reader/model-provider-v1.schema.json diff --git a/cli-connection-reader/model-contract.mjs b/cli-connection-reader/model-contract.mjs new file mode 100644 index 000000000..0fa55bc5a --- /dev/null +++ b/cli-connection-reader/model-contract.mjs @@ -0,0 +1,283 @@ +import { createHash, randomUUID } from 'node:crypto'; + +export const READER_SCHEMA_VERSION = 'model-reference-reader/v1'; +const SHA256 = /^[0-9a-f]{64}$/; +const PLAIN = Object.getPrototypeOf({}); + +export const MODEL_LIMITS = Object.freeze({ + providerRequestBytes: { default: 256 * 1024, hard: 1024 * 1024 }, + providerStdoutBytes: { default: 256 * 1024, hard: 1024 * 1024 }, + providerStderrBytes: { default: 64 * 1024, hard: 256 * 1024 }, + conversionMs: { default: 10 * 60_000, hard: 30 * 60_000 }, + maxSourceBytes: { default: 150 * 1024 * 1024, hard: 4 * 1024 * 1024 * 1024 }, + maxInputGlbBytes: { default: 128 * 1024 * 1024, hard: 512 * 1024 * 1024 }, + maxMetadataBytes: { default: 16 * 1024 * 1024, hard: 64 * 1024 * 1024 }, + maxProviderOutputBytes: { default: 144 * 1024 * 1024, hard: 576 * 1024 * 1024 }, + maxGlbJsonBytes: { default: 4 * 1024 * 1024, hard: 16 * 1024 * 1024 }, + maxJsonDepth: { default: 64, hard: 128 }, + maxScenes: { default: 8, hard: 32 }, + maxNodes: { default: 100_000, hard: 250_000 }, + maxNodeDepth: { default: 128, hard: 256 }, + maxMeshes: { default: 100_000, hard: 250_000 }, + maxPrimitives: { default: 200_000, hard: 500_000 }, + maxAccessors: { default: 250_000, hard: 1_000_000 }, + maxBufferViews: { default: 250_000, hard: 1_000_000 }, + maxVertices: { default: 5_000_000, hard: 10_000_000 }, + maxIndices: { default: 15_000_000, hard: 30_000_000 }, + maxEntities: { default: 250_000, hard: 1_000_000 }, + maxParameters: { default: 2_000_000, hard: 5_000_000 }, + maxRelationships: { default: 1_000_000, hard: 2_000_000 }, + maxComponentJsonBytes: { default: 32 * 1024 * 1024, hard: 128 * 1024 * 1024 }, + maxCanonicalGlbBytes: { default: 256 * 1024 * 1024, hard: 512 * 1024 * 1024 }, + maxCommandResponseBytes: { default: 1024 * 1024, hard: 1024 * 1024 }, +}); + +export class ModelReaderError extends Error { + constructor(code, phase, retryable, message, unsafeDetails = undefined) { + super(message); + this.name = 'ModelReaderError'; + this.code = code; + this.phase = phase; + this.retryable = retryable; + this.diagnosticId = randomUUID(); + Object.defineProperty(this, 'unsafeDetails', { value: unsafeDetails, enumerable: false }); + } +} + +export function safeErrorEnvelope(error) { + if (error instanceof ModelReaderError) { + return { + code: error.code, + phase: error.phase, + retryable: error.retryable, + message: boundedMessage(error.message), + diagnosticId: error.diagnosticId, + }; + } + return { + code: 'reference-internal-error', + phase: 'internal', + retryable: false, + message: 'The model reader failed.', + diagnosticId: randomUUID(), + }; +} + +function boundedMessage(message) { + const clean = typeof message === 'string' ? message.replace(/[\r\n\t]+/g, ' ').trim() : 'The model reader failed.'; + return clean.slice(0, 240) || 'The model reader failed.'; +} + +export function sha256(bytes) { + return createHash('sha256').update(bytes).digest('hex'); +} + +export function assertSha256(value, label = 'digest') { + if (typeof value !== 'string' || !SHA256.test(value)) { + throw new ModelReaderError('reference-contract-invalid', 'admission', false, `${label} must be a lowercase SHA-256 digest`); + } + return value; +} + +function assertUnicodeScalars(value) { + for (let i = 0; i < value.length; i += 1) { + const unit = value.charCodeAt(i); + if (unit >= 0xd800 && unit <= 0xdbff) { + const next = value.charCodeAt(i + 1); + if (!(next >= 0xdc00 && next <= 0xdfff)) throw new TypeError('string must contain Unicode scalar values'); + i += 1; + } else if (unit >= 0xdc00 && unit <= 0xdfff) { + throw new TypeError('string must contain Unicode scalar values'); + } + } +} + +function normalizeJson(value, seen = new Set()) { + if (value === null || typeof value === 'boolean') return value; + if (typeof value === 'string') { + assertUnicodeScalars(value); + return value; + } + if (typeof value === 'number') { + if (!Number.isFinite(value)) throw new TypeError('JSON number must be finite'); + if (Number.isInteger(value) && !Number.isSafeInteger(value)) throw new TypeError('JSON integer must be a safe integer'); + return Object.is(value, -0) ? 0 : value; + } + if (!value || typeof value !== 'object') throw new TypeError('value is not JSON data'); + if (seen.has(value)) throw new TypeError('JSON value must be acyclic'); + seen.add(value); + try { + if (Array.isArray(value)) return value.map((entry) => normalizeJson(entry, seen)); + if (Object.getPrototypeOf(value) !== PLAIN && Object.getPrototypeOf(value) !== null) throw new TypeError('JSON object must be plain'); + const out = {}; + for (const key of Object.keys(value).sort()) { + assertUnicodeScalars(key); + if (value[key] === undefined) throw new TypeError('undefined is not JSON data'); + out[key] = normalizeJson(value[key], seen); + } + return out; + } finally { + seen.delete(value); + } +} + +export function canonicalJsonBytes(value) { + return Buffer.from(JSON.stringify(normalizeJson(value)), 'utf8'); +} + +export function parseJsonStrict(input, options = {}) { + const text = Buffer.isBuffer(input) || input instanceof Uint8Array ? Buffer.from(input).toString('utf8') : String(input); + const maxBytes = options.maxBytes ?? 16 * 1024 * 1024; + const maxDepth = options.maxDepth ?? 128; + if (Buffer.byteLength(text, 'utf8') > maxBytes) throw new SyntaxError('JSON input exceeds its byte limit'); + let cursor = 0; + const white = () => { while (/\s/.test(text[cursor] ?? '')) cursor += 1; }; + const fail = (message) => { throw new SyntaxError(`${message} at byte ${cursor}`); }; + const stringValue = () => { + if (text[cursor] !== '"') fail('expected JSON string'); + const start = cursor; + cursor += 1; + let escaped = false; + for (; cursor < text.length; cursor += 1) { + const ch = text[cursor]; + if (escaped) { escaped = false; continue; } + if (ch === '\\') { escaped = true; continue; } + if (ch === '"') { + cursor += 1; + const value = JSON.parse(text.slice(start, cursor)); + assertUnicodeScalars(value); + return value; + } + if (ch.charCodeAt(0) < 0x20) fail('unescaped control character'); + } + fail('unterminated JSON string'); + }; + const value = (depth) => { + if (depth > maxDepth) fail('JSON nesting exceeds its limit'); + white(); + const ch = text[cursor]; + if (ch === '"') return stringValue(); + if (ch === '{') { + cursor += 1; + const out = {}; + const keys = new Set(); + white(); + if (text[cursor] === '}') { cursor += 1; return out; } + for (;;) { + white(); + const key = stringValue(); + if (keys.has(key)) fail(`duplicate JSON key '${key}'`); + keys.add(key); + white(); + if (text[cursor] !== ':') fail('expected colon'); + cursor += 1; + out[key] = value(depth + 1); + white(); + if (text[cursor] === '}') { cursor += 1; return out; } + if (text[cursor] !== ',') fail('expected comma'); + cursor += 1; + } + } + if (ch === '[') { + cursor += 1; + const out = []; + white(); + if (text[cursor] === ']') { cursor += 1; return out; } + for (;;) { + out.push(value(depth + 1)); + white(); + if (text[cursor] === ']') { cursor += 1; return out; } + if (text[cursor] !== ',') fail('expected comma'); + cursor += 1; + } + } + for (const [token, result] of [['true', true], ['false', false], ['null', null]]) { + if (text.startsWith(token, cursor)) { cursor += token.length; return result; } + } + const match = text.slice(cursor).match(/^-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+-]?\d+)?/); + if (!match) fail('invalid JSON value'); + cursor += match[0].length; + const number = Number(match[0]); + if (!Number.isFinite(number)) fail('JSON number must be finite'); + if (!/[.eE]/.test(match[0]) && !Number.isSafeInteger(number)) fail('JSON integer must be a safe integer'); + return Object.is(number, -0) ? 0 : number; + }; + const parsed = value(0); + white(); + if (cursor !== text.length) fail('trailing JSON data'); + return parsed; +} + +export function assertClosedObject(value, required, optional = [], label = 'object') { + if (!value || typeof value !== 'object' || Array.isArray(value)) throw new TypeError(`${label} must be an object`); + const allowed = new Set([...required, ...optional]); + for (const key of Object.keys(value)) if (!allowed.has(key)) throw new TypeError(`${label} has unknown property '${key}'`); + for (const key of required) if (!Object.hasOwn(value, key)) throw new TypeError(`${label} is missing '${key}'`); + return value; +} + +export function lowerableLimits(overrides = {}) { + assertClosedObject(overrides, [], Object.keys(MODEL_LIMITS), 'limits'); + const result = {}; + for (const [name, range] of Object.entries(MODEL_LIMITS)) { + const selected = overrides[name] ?? range.default; + if (!Number.isSafeInteger(selected) || selected <= 0 || selected > range.hard) throw new TypeError(`${name} exceeds its hard ceiling`); + result[name] = selected; + } + return result; +} + +export function buildCanonicalRequest(options = {}) { + const limits = lowerableLimits(options.limits); + return { + schemaVersion: '1', + protocolVersion: '1', + readerSchemaVersion: READER_SCHEMA_VERSION, + format: 'rvt', + documentKind: 'revit-project', + activeScenePolicy: 'declared-active-scene-only', + selection: { mode: 'full-model' }, + geometry: { + primitives: ['TRIANGLES', 'TRIANGLE_STRIP', 'TRIANGLE_FAN'], + positions: 'VEC3/FLOAT', + targetUnits: 'mm', + targetUp: 'z', + targetHandedness: 'right', + winding: 'front-face-preserving', + colors: 'vertex-times-base-color', + topology: 'canonical-byte-tuples-first', + }, + metadata: { identity: 'revit-element-id-decimal-string', joins: 'explicit-appearance-name', inference: 'none' }, + canonicalJson: 'RFC8785-JCS', + canonicalGlb: 'model-reference-reader-glb/v1', + conversionSettings: options.conversionSettings ?? {}, + limits, + }; +} + +export function requestSha256(request) { + return sha256(canonicalJsonBytes(request)); +} + +export function buildProviderFingerprint(describe) { + assertClosedObject(describe, + ['protocolVersion', 'provider', 'engine', 'engineVersion', 'adapterBuildId', 'adapterExecutableSha256'], + ['readerSchemaVersion'], 'provider fingerprint'); + assertSha256(describe.adapterExecutableSha256, 'adapterExecutableSha256'); + for (const key of ['protocolVersion', 'provider', 'engine', 'engineVersion', 'adapterBuildId']) { + if (typeof describe[key] !== 'string' || !describe[key]) throw new TypeError(`${key} must be a non-empty string`); + } + return { + protocolVersion: describe.protocolVersion, + provider: describe.provider, + engine: describe.engine, + engineVersion: describe.engineVersion, + adapterBuildId: describe.adapterBuildId, + adapterExecutableSha256: describe.adapterExecutableSha256, + readerSchemaVersion: describe.readerSchemaVersion ?? READER_SCHEMA_VERSION, + }; +} + +export function providerFingerprintSha256(fingerprint) { + return sha256(canonicalJsonBytes(buildProviderFingerprint(fingerprint))); +} diff --git a/cli-connection-reader/model-contract.test.mjs b/cli-connection-reader/model-contract.test.mjs new file mode 100644 index 000000000..91f7a85b2 --- /dev/null +++ b/cli-connection-reader/model-contract.test.mjs @@ -0,0 +1,72 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { + MODEL_LIMITS, + ModelReaderError, + buildCanonicalRequest, + buildProviderFingerprint, + canonicalJsonBytes, + parseJsonStrict, + providerFingerprintSha256, + requestSha256, + safeErrorEnvelope, +} from './model-contract.mjs'; + +test('JCS bytes are stable across key order and pin number/string edge cases', () => { + const a = canonicalJsonBytes({ z: '\u20ac', a: [3, -0, 1e-7], nested: { b: true, a: null } }); + const b = canonicalJsonBytes({ nested: { a: null, b: true }, a: [3, 0, 0.0000001], z: '\u20ac' }); + assert.deepEqual(a, b); + assert.equal(a.toString('utf8'), '{"a":[3,0,1e-7],"nested":{"a":null,"b":true},"z":"€"}'); + assert.throws(() => canonicalJsonBytes({ unsafe: Number.MAX_SAFE_INTEGER + 1 }), /safe integer/); + assert.throws(() => canonicalJsonBytes({ bad: Number.NaN }), /finite/); + assert.throws(() => canonicalJsonBytes({ bad: '\ud800' }), /Unicode scalar/); +}); + +test('strict JSON parsing rejects duplicate keys, trailing bytes, and unsafe integers', () => { + assert.deepEqual(parseJsonStrict('{"a":1,"nested":{"b":2}}'), { a: 1, nested: { b: 2 } }); + assert.throws(() => parseJsonStrict('{"a":1,"a":2}'), /duplicate JSON key/); + assert.throws(() => parseJsonStrict('{"a":1}x'), /trailing JSON data/); + assert.throws(() => parseJsonStrict('{"a":9007199254740992}'), /safe integer/); +}); + +test('every canonical request leaf affects the cache/request preimage', () => { + const request = buildCanonicalRequest(); + const baseline = requestSha256(request); + const mutations = [ + { ...request, readerSchemaVersion: 'model-reference-reader/v2' }, + { ...request, activeScenePolicy: 'all-scenes' }, + { ...request, selection: { ...request.selection, mode: 'subset' } }, + { ...request, limits: { ...request.limits, maxEntities: request.limits.maxEntities - 1 } }, + { ...request, geometry: { ...request.geometry, targetUnits: 'm' } }, + ]; + for (const mutation of mutations) assert.notEqual(requestSha256(mutation), baseline); + assert.equal(request.limits.maxInputGlbBytes, MODEL_LIMITS.maxInputGlbBytes.default); +}); + +test('provider fingerprint is the exact seven-field JCS tuple', () => { + const fingerprint = buildProviderFingerprint({ + protocolVersion: '1', provider: 'fixture', engine: 'fixture-engine', engineVersion: '1.2.3', + adapterBuildId: 'fixture-build', adapterExecutableSha256: 'a'.repeat(64), + }); + assert.deepEqual(Object.keys(fingerprint).sort(), [ + 'adapterBuildId', 'adapterExecutableSha256', 'engine', 'engineVersion', 'protocolVersion', + 'provider', 'readerSchemaVersion', + ]); + const baseline = providerFingerprintSha256(fingerprint); + for (const key of Object.keys(fingerprint)) { + const value = fingerprint[key]; + const mutation = { ...fingerprint, [key]: key === 'adapterExecutableSha256' ? 'b'.repeat(64) : `${value}-changed` }; + assert.notEqual(providerFingerprintSha256(mutation), baseline, key); + } +}); + +test('structured errors expose bounded safe fields and never paths or provider output', () => { + const error = new ModelReaderError('reference-provider-failed', 'convert', false, 'provider failed', { + sourcePath: 'C:\\private\\Residential.rvt', stderr: 'secret-provider-output', count: 3, + }); + const envelope = safeErrorEnvelope(error); + assert.deepEqual(Object.keys(envelope), ['code', 'phase', 'retryable', 'message', 'diagnosticId']); + assert.equal(envelope.code, 'reference-provider-failed'); + assert.match(envelope.diagnosticId, /^[0-9a-f-]{36}$/); + assert.doesNotMatch(JSON.stringify(envelope), /Residential|secret-provider-output|private/); +}); diff --git a/cli-connection-reader/model-fixtures.mjs b/cli-connection-reader/model-fixtures.mjs new file mode 100644 index 000000000..3407b42c9 --- /dev/null +++ b/cli-connection-reader/model-fixtures.mjs @@ -0,0 +1,71 @@ +import { canonicalJsonBytes } from './model-contract.mjs'; + +const align4 = (value) => (value + 3) & ~3; + +export function makeGlbFixture(options = {}) { + const positions = options.positions ?? [[0, 0, 0], [1, 0, 0], [0, 1, 0]]; + const indices = options.indices ?? [0, 1, 2]; + const positionBytes = Buffer.alloc(positions.length * 12); + positions.forEach((point, index) => point.forEach((coordinate, axis) => positionBytes.writeFloatLE(coordinate, index * 12 + axis * 4))); + const indexOffset = align4(positionBytes.length); + const binary = Buffer.alloc(indexOffset + indices.length * 4); + positionBytes.copy(binary); + indices.forEach((value, index) => binary.writeUInt32LE(value, indexOffset + index * 4)); + const json = { + asset: { version: '2.0' }, + scene: options.scene ?? 0, + scenes: options.scenes ?? [{ nodes: [0] }], + nodes: options.nodes ?? [{ name: options.nodeName ?? 'part-a', mesh: 0 }], + meshes: options.meshes ?? [{ primitives: [{ attributes: { POSITION: 0 }, indices: 1, mode: options.mode ?? 4 }] }], + buffers: [{ byteLength: binary.length }], + bufferViews: [ + { buffer: 0, byteOffset: 0, byteLength: positionBytes.length }, + { buffer: 0, byteOffset: indexOffset, byteLength: indices.length * 4 }, + ], + accessors: [ + { bufferView: 0, byteOffset: 0, componentType: 5126, count: positions.length, type: 'VEC3' }, + { bufferView: 1, byteOffset: 0, componentType: 5125, count: indices.length, type: 'SCALAR' }, + ], + }; + const jsonBytes = canonicalJsonBytes(json); + const jsonLength = align4(jsonBytes.length); + const binaryLength = align4(binary.length); + const out = Buffer.alloc(12 + 8 + jsonLength + 8 + binaryLength); + out.writeUInt32LE(0x46546c67, 0); + out.writeUInt32LE(2, 4); + out.writeUInt32LE(out.length, 8); + out.writeUInt32LE(jsonLength, 12); + out.writeUInt32LE(0x4e4f534a, 16); + out.fill(0x20, 20, 20 + jsonLength); + jsonBytes.copy(out, 20); + const binHeader = 20 + jsonLength; + out.writeUInt32LE(binaryLength, binHeader); + out.writeUInt32LE(0x004e4942, binHeader + 4); + binary.copy(out, binHeader + 8); + return out; +} + +export function makeMetadataFixture(options = {}) { + const elementId = options.elementId ?? '1001'; + return { + schemaVersion: '1', + document: { kind: 'revit-project', id: 'document:fixture' }, + types: [{ id: '2001', name: 'Fixture Type' }], + levels: [{ id: '3001', name: 'Level 1', elevation: 0 }], + parameterGroups: [{ id: '4001', name: 'Identity Data', parameters: [0] }], + parameters: [{ id: '5001', name: 'IfcGUID', unit: null, readable: true, storageType: 'string', value: options.ifcGuid ?? '1FixtureGuid00000000000' }], + elements: [{ + id: elementId, + revitClass: 'FamilyInstance', + category: 'Structural Framing', + family: 'Fixture Family', + type: 0, + level: 0, + parameterGroups: [0], + appearances: options.nodeNames ?? ['part-a'], + ifcGuid: options.ifcGuid ?? '1FixtureGuid00000000000', + }], + relations: [], + }; +} + diff --git a/cli-connection-reader/model-fixtures.test.mjs b/cli-connection-reader/model-fixtures.test.mjs new file mode 100644 index 000000000..b6b3c0b14 --- /dev/null +++ b/cli-connection-reader/model-fixtures.test.mjs @@ -0,0 +1,19 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { makeGlbFixture, makeMetadataFixture } from './model-fixtures.mjs'; + +test('generated GLB fixtures are byte deterministic and carry no committed binary fixture', () => { + const options = { positions: [[0, 0, 0], [1, 0, 0], [0, 1, 0]], indices: [0, 1, 2], nodeName: 'part-a' }; + const first = makeGlbFixture(options); + const second = makeGlbFixture({ indices: [0, 1, 2], nodeName: 'part-a', positions: options.positions }); + assert.deepEqual(first, second); + assert.equal(first.subarray(0, 4).toString('ascii'), 'glTF'); +}); + +test('generated metadata uses decimal-string identities and explicit appearance joins', () => { + const metadata = makeMetadataFixture({ elementId: '9223372036854775806', nodeNames: ['part-a', 'part-b'] }); + assert.equal(metadata.elements[0].id, '9223372036854775806'); + assert.deepEqual(metadata.elements[0].appearances, ['part-a', 'part-b']); + assert.equal(metadata.parameters[0].storageType, 'string'); +}); + diff --git a/cli-connection-reader/model-metadata-v1.schema.json b/cli-connection-reader/model-metadata-v1.schema.json new file mode 100644 index 000000000..b532bb3e5 --- /dev/null +++ b/cli-connection-reader/model-metadata-v1.schema.json @@ -0,0 +1,18 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://aware-aeco.org/schemas/model-metadata-v1.schema.json", + "title": "AWARE explicit Revit metadata v1", + "type": "object", + "additionalProperties": false, + "required": ["schemaVersion", "document", "types", "levels", "parameterGroups", "parameters", "elements", "relations"], + "properties": { + "schemaVersion": { "const": "1" }, + "document": { "type": "object", "additionalProperties": false, "required": ["kind", "id"], "properties": { "kind": { "const": "revit-project" }, "id": { "type": "string", "minLength": 1 } } }, + "types": { "type": "array" }, + "levels": { "type": "array" }, + "parameterGroups": { "type": "array" }, + "parameters": { "type": "array" }, + "elements": { "type": "array" }, + "relations": { "type": "array" } + } +} diff --git a/cli-connection-reader/model-provider-v1.schema.json b/cli-connection-reader/model-provider-v1.schema.json new file mode 100644 index 000000000..63e4bc1b2 --- /dev/null +++ b/cli-connection-reader/model-provider-v1.schema.json @@ -0,0 +1,18 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://aware-aeco.org/schemas/model-provider-v1.schema.json", + "title": "AWARE local model provider v1", + "type": "object", + "additionalProperties": false, + "required": ["protocolVersion", "provider", "engine", "engineVersion", "adapterBuildId", "formats", "execution", "destination"], + "properties": { + "protocolVersion": { "const": "1" }, + "provider": { "type": "string", "minLength": 1, "maxLength": 128 }, + "engine": { "type": "string", "minLength": 1, "maxLength": 128 }, + "engineVersion": { "type": "string", "minLength": 1, "maxLength": 128 }, + "adapterBuildId": { "type": "string", "minLength": 1, "maxLength": 256 }, + "formats": { "type": "array", "prefixItems": [{ "const": "rvt" }], "items": false }, + "execution": { "const": "local" }, + "destination": { "type": "null" } + } +} From eeb36b380e089a184e38de4f98a9abc0a0ee8ad1 Mon Sep 17 00:00:00 2001 From: Pawel Date: Sun, 23 Aug 2026 14:04:34 +0200 Subject: [PATCH 03/50] feat: normalize bounded Revit GLB geometry --- cli-connection-reader/model-fixtures.mjs | 27 +- cli-connection-reader/revit-glb.mjs | 401 +++++++++++++++++++++++ cli-connection-reader/revit-glb.test.mjs | 88 +++++ 3 files changed, 510 insertions(+), 6 deletions(-) create mode 100644 cli-connection-reader/revit-glb.mjs create mode 100644 cli-connection-reader/revit-glb.test.mjs diff --git a/cli-connection-reader/model-fixtures.mjs b/cli-connection-reader/model-fixtures.mjs index 3407b42c9..7afa46e63 100644 --- a/cli-connection-reader/model-fixtures.mjs +++ b/cli-connection-reader/model-fixtures.mjs @@ -8,25 +8,41 @@ export function makeGlbFixture(options = {}) { const positionBytes = Buffer.alloc(positions.length * 12); positions.forEach((point, index) => point.forEach((coordinate, axis) => positionBytes.writeFloatLE(coordinate, index * 12 + axis * 4))); const indexOffset = align4(positionBytes.length); - const binary = Buffer.alloc(indexOffset + indices.length * 4); + const indexEnd = indexOffset + indices.length * 4; + const colorOffset = align4(indexEnd); + const colors = options.colors ?? null; + const binary = Buffer.alloc(colorOffset + (colors ? colors.length * 16 : 0)); positionBytes.copy(binary); indices.forEach((value, index) => binary.writeUInt32LE(value, indexOffset + index * 4)); + colors?.forEach((color, index) => color.forEach((component, channel) => binary.writeFloatLE(component, colorOffset + index * 16 + channel * 4))); + const primitive = { attributes: { POSITION: 0 }, indices: 1, mode: options.mode ?? 4 }; + if (colors) primitive.attributes.COLOR_0 = 2; + if (options.materialColor) primitive.material = 0; + const defaultNode = { name: options.nodeName ?? 'part-a', mesh: 0 }; + if (options.translation) defaultNode.translation = options.translation; + if (options.rotation) defaultNode.rotation = options.rotation; + if (options.scale) defaultNode.scale = options.scale; + if (options.matrix) defaultNode.matrix = options.matrix; const json = { asset: { version: '2.0' }, - scene: options.scene ?? 0, scenes: options.scenes ?? [{ nodes: [0] }], - nodes: options.nodes ?? [{ name: options.nodeName ?? 'part-a', mesh: 0 }], - meshes: options.meshes ?? [{ primitives: [{ attributes: { POSITION: 0 }, indices: 1, mode: options.mode ?? 4 }] }], - buffers: [{ byteLength: binary.length }], + nodes: options.nodes ?? [defaultNode], + meshes: options.meshes ?? [{ primitives: [primitive] }], + buffers: [{ byteLength: binary.length, ...(options.externalUri ? { uri: options.externalUri } : {}) }], bufferViews: [ { buffer: 0, byteOffset: 0, byteLength: positionBytes.length }, { buffer: 0, byteOffset: indexOffset, byteLength: indices.length * 4 }, + ...(colors ? [{ buffer: 0, byteOffset: colorOffset, byteLength: colors.length * 16 }] : []), ], accessors: [ { bufferView: 0, byteOffset: 0, componentType: 5126, count: positions.length, type: 'VEC3' }, { bufferView: 1, byteOffset: 0, componentType: 5125, count: indices.length, type: 'SCALAR' }, + ...(colors ? [{ bufferView: 2, byteOffset: 0, componentType: 5126, count: colors.length, type: 'VEC4' }] : []), ], + ...(options.materialColor ? { materials: [{ pbrMetallicRoughness: { baseColorFactor: options.materialColor } }] } : {}), + ...(options.extensionsUsed ? { extensionsUsed: options.extensionsUsed } : {}), }; + if (!options.omitScene) json.scene = options.scene ?? 0; const jsonBytes = canonicalJsonBytes(json); const jsonLength = align4(jsonBytes.length); const binaryLength = align4(binary.length); @@ -68,4 +84,3 @@ export function makeMetadataFixture(options = {}) { relations: [], }; } - diff --git a/cli-connection-reader/revit-glb.mjs b/cli-connection-reader/revit-glb.mjs new file mode 100644 index 000000000..3434e4c30 --- /dev/null +++ b/cli-connection-reader/revit-glb.mjs @@ -0,0 +1,401 @@ +import { canonicalJsonBytes, lowerableLimits, ModelReaderError, parseJsonStrict } from './model-contract.mjs'; + +const GLB_MAGIC = 0x46546c67; +const JSON_CHUNK = 0x4e4f534a; +const BIN_CHUNK = 0x004e4942; +const COMPONENT_BYTES = new Map([[5121, 1], [5123, 2], [5125, 4], [5126, 4]]); +const IDENTITY = [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]; + +function invalid(message, code = 'reference-geometry-invalid') { + throw new ModelReaderError(code, 'normalize-geometry', false, message); +} + +function checkedRange(offset, length, bound, label) { + if (!Number.isSafeInteger(offset) || !Number.isSafeInteger(length) || offset < 0 || length < 0 || offset > bound || length > bound - offset) { + invalid(`${label} range is outside its buffer`); + } +} + +export function parseGlb(input, options = {}) { + const limits = lowerableLimits(options.limits); + const bytes = Buffer.from(input); + if (bytes.length > limits.maxInputGlbBytes) invalid('GLB exceeds its byte limit', 'reference-output-too-large'); + if (bytes.length < 20) invalid('GLB length is too short'); + if (bytes.readUInt32LE(0) !== GLB_MAGIC || bytes.readUInt32LE(4) !== 2) invalid('GLB header or version is invalid'); + if (bytes.readUInt32LE(8) !== bytes.length) invalid('GLB declared length does not match its bytes'); + let cursor = 12; + let jsonText = null; + let binary = null; + while (cursor < bytes.length) { + checkedRange(cursor, 8, bytes.length, 'GLB chunk header'); + const length = bytes.readUInt32LE(cursor); + const type = bytes.readUInt32LE(cursor + 4); + if (length % 4 !== 0) invalid('GLB chunk length is not 4-byte aligned'); + cursor += 8; + checkedRange(cursor, length, bytes.length, 'GLB chunk'); + const payload = bytes.subarray(cursor, cursor + length); + cursor += length; + if (type === JSON_CHUNK) { + if (jsonText !== null || binary !== null) invalid('GLB JSON chunk must be first and unique'); + if (length > limits.maxGlbJsonBytes) invalid('GLB JSON exceeds its byte limit', 'reference-output-too-large'); + jsonText = payload.toString('utf8').replace(/[\u0000\u0020]+$/u, ''); + } else if (type === BIN_CHUNK) { + if (jsonText === null || binary !== null) invalid('GLB BIN chunk is duplicated or precedes JSON'); + binary = Buffer.from(payload); + } else { + invalid('GLB contains an unsupported chunk type', 'reference-geometry-unsupported'); + } + } + if (jsonText === null) invalid('GLB has no JSON chunk'); + let json; + try { json = parseJsonStrict(jsonText, { maxBytes: limits.maxGlbJsonBytes, maxDepth: limits.maxJsonDepth }); } + catch (error) { invalid(error instanceof Error ? error.message : 'GLB JSON is invalid'); } + return { json, binary: binary ?? Buffer.alloc(0), jsonText }; +} + +function array(value, label, max) { + if (!Array.isArray(value) || value.length > max) invalid(`${label} is missing or exceeds its count limit`); + return value; +} + +function safeIndex(value, length, label) { + if (!Number.isSafeInteger(value) || value < 0 || value >= length) invalid(`${label} is out of range`); + return value; +} + +function finiteArray(value, length, label) { + if (!Array.isArray(value) || value.length !== length || value.some((entry) => typeof entry !== 'number' || !Number.isFinite(entry))) invalid(`${label} must contain ${length} finite numbers`); + return value; +} + +function multiply(a, b) { + const out = new Array(16).fill(0); + for (let column = 0; column < 4; column += 1) { + for (let row = 0; row < 4; row += 1) { + for (let k = 0; k < 4; k += 1) out[column * 4 + row] += a[k * 4 + row] * b[column * 4 + k]; + } + } + return out; +} + +function nodeMatrix(node) { + if (node.matrix !== undefined && (node.translation !== undefined || node.rotation !== undefined || node.scale !== undefined)) invalid('node cannot mix matrix and TRS'); + if (node.matrix !== undefined) return [...finiteArray(node.matrix, 16, 'node matrix')]; + const translation = node.translation === undefined ? [0, 0, 0] : finiteArray(node.translation, 3, 'node translation'); + const scale = node.scale === undefined ? [1, 1, 1] : finiteArray(node.scale, 3, 'node scale'); + let quaternion = node.rotation === undefined ? [0, 0, 0, 1] : finiteArray(node.rotation, 4, 'node rotation'); + const norm = Math.hypot(...quaternion); + if (!Number.isFinite(norm) || norm === 0 || Math.abs(norm - 1) > 1e-3) invalid('node quaternion is outside the unit-length tolerance'); + quaternion = quaternion.map((entry) => entry / norm); + const [x, y, z, w] = quaternion; + const [sx, sy, sz] = scale; + return [ + (1 - 2 * y * y - 2 * z * z) * sx, (2 * x * y + 2 * z * w) * sx, (2 * x * z - 2 * y * w) * sx, 0, + (2 * x * y - 2 * z * w) * sy, (1 - 2 * x * x - 2 * z * z) * sy, (2 * y * z + 2 * x * w) * sy, 0, + (2 * x * z + 2 * y * w) * sz, (2 * y * z - 2 * x * w) * sz, (1 - 2 * x * x - 2 * y * y) * sz, 0, + translation[0], translation[1], translation[2], 1, + ]; +} + +function determinant3(matrix) { + const a = matrix[0], b = matrix[4], c = matrix[8]; + const d = matrix[1], e = matrix[5], f = matrix[9]; + const g = matrix[2], h = matrix[6], i = matrix[10]; + return a * (e * i - f * h) - b * (d * i - f * g) + c * (d * h - e * g); +} + +function transform(matrix, point) { + const [x, y, z] = point; + const tx = matrix[0] * x + matrix[4] * y + matrix[8] * z + matrix[12]; + const ty = matrix[1] * x + matrix[5] * y + matrix[9] * z + matrix[13]; + const tz = matrix[2] * x + matrix[6] * y + matrix[10] * z + matrix[14]; + return canonicalVector([tx * 1000, -tz * 1000, ty * 1000]); +} + +function canonicalFloat(value) { + const rounded = Math.fround(value); + if (!Number.isFinite(rounded)) invalid('transformed coordinate overflows float32'); + return Object.is(rounded, -0) ? 0 : rounded; +} +const canonicalVector = (values) => values.map(canonicalFloat); + +function accessorReader(document, binary, accessorIndex, expected, label) { + const accessors = document.accessors; + const views = document.bufferViews; + const accessor = accessors[safeIndex(accessorIndex, accessors.length, `${label} accessor`)]; + if (!accessor || typeof accessor !== 'object' || accessor.sparse !== undefined) invalid(`${label} accessor is sparse or invalid`, 'reference-geometry-unsupported'); + if (expected.types && !expected.types.includes(accessor.type)) invalid(`${label} accessor has unsupported type`, 'reference-geometry-unsupported'); + if (expected.components && !expected.components.includes(accessor.componentType)) invalid(`${label} accessor has unsupported component type`, 'reference-geometry-unsupported'); + if (!Number.isSafeInteger(accessor.count) || accessor.count < 0) invalid(`${label} accessor count is invalid`); + const width = accessor.type === 'VEC4' ? 4 : accessor.type === 'VEC3' ? 3 : accessor.type === 'SCALAR' ? 1 : 0; + const componentBytes = COMPONENT_BYTES.get(accessor.componentType); + if (!width || !componentBytes) invalid(`${label} accessor layout is unsupported`, 'reference-geometry-unsupported'); + const view = views[safeIndex(accessor.bufferView, views.length, `${label} bufferView`)]; + if (view.buffer !== 0) invalid(`${label} accessor does not bind buffers[0]`); + const stride = view.byteStride ?? width * componentBytes; + if (!Number.isSafeInteger(stride) || stride < width * componentBytes || stride % componentBytes !== 0) invalid(`${label} byteStride is invalid`); + const viewOffset = view.byteOffset ?? 0; + const accessorOffset = accessor.byteOffset ?? 0; + const viewLength = view.byteLength; + checkedRange(viewOffset, viewLength, document.buffers[0].byteLength, `${label} bufferView`); + if (accessorOffset % componentBytes !== 0) invalid(`${label} accessor is misaligned`); + const required = accessor.count === 0 ? 0 : (accessor.count - 1) * stride + width * componentBytes; + checkedRange(accessorOffset, required, viewLength, `${label} accessor`); + const base = viewOffset + accessorOffset; + const readComponent = (offset) => { + if (accessor.componentType === 5126) return binary.readFloatLE(offset); + if (accessor.componentType === 5125) return binary.readUInt32LE(offset); + if (accessor.componentType === 5123) return binary.readUInt16LE(offset); + return binary.readUInt8(offset); + }; + return { + count: accessor.count, + normalized: accessor.normalized === true, + componentType: accessor.componentType, + read(index) { + safeIndex(index, accessor.count, `${label} item`); + const offset = base + index * stride; + const result = []; + for (let component = 0; component < width; component += 1) result.push(readComponent(offset + component * componentBytes)); + return result; + }, + }; +} + +function indicesFor(primitive, document, binary, vertexCount, limits) { + let indices; + if (primitive.indices === undefined) indices = Array.from({ length: vertexCount }, (_, index) => index); + else { + const reader = accessorReader(document, binary, primitive.indices, { types: ['SCALAR'], components: [5121, 5123, 5125] }, 'indices'); + if (reader.count > limits.maxIndices) invalid('index count exceeds its limit', 'reference-output-too-large'); + indices = Array.from({ length: reader.count }, (_, index) => reader.read(index)[0]); + } + for (const index of indices) if (index >= vertexCount) invalid('index value exceeds POSITION count'); + return indices; +} + +function expandTriangles(indices, mode) { + const triangles = []; + if (mode === 4) { + if (indices.length % 3 !== 0) invalid('TRIANGLES index count is not divisible by three'); + for (let index = 0; index < indices.length; index += 3) triangles.push(indices.slice(index, index + 3)); + } else if (mode === 5) { + for (let index = 2; index < indices.length; index += 1) triangles.push(index % 2 === 0 ? [indices[index - 2], indices[index - 1], indices[index]] : [indices[index - 1], indices[index - 2], indices[index]]); + } else if (mode === 6) { + for (let index = 2; index < indices.length; index += 1) triangles.push([indices[0], indices[index - 1], indices[index]]); + } else invalid('primitive mode is unsupported', 'reference-geometry-unsupported'); + return triangles; +} + +function baseColor(document, primitive) { + if (primitive.material === undefined) return [1, 1, 1, 1]; + const material = document.materials[safeIndex(primitive.material, document.materials.length, 'material')]; + const allowed = new Set(['pbrMetallicRoughness', 'alphaMode', 'doubleSided', 'emissiveFactor']); + for (const key of Object.keys(material)) if (!allowed.has(key)) invalid(`material property '${key}' is unsupported`, 'reference-geometry-unsupported'); + if (material.alphaMode !== undefined && material.alphaMode !== 'OPAQUE') invalid('non-opaque material is unsupported', 'reference-geometry-unsupported'); + if (material.doubleSided === true) invalid('double-sided material is unsupported', 'reference-geometry-unsupported'); + if (material.emissiveFactor && finiteArray(material.emissiveFactor, 3, 'emissiveFactor').some((entry) => entry !== 0)) invalid('emissive material is unsupported', 'reference-geometry-unsupported'); + const pbr = material.pbrMetallicRoughness ?? {}; + for (const key of Object.keys(pbr)) if (!['baseColorFactor', 'metallicFactor', 'roughnessFactor'].includes(key)) invalid(`material PBR property '${key}' is unsupported`, 'reference-geometry-unsupported'); + const factor = pbr.baseColorFactor === undefined ? [1, 1, 1, 1] : finiteArray(pbr.baseColorFactor, 4, 'baseColorFactor'); + if (factor.some((entry) => entry < 0 || entry > 1)) invalid('baseColorFactor is outside [0,1]'); + return factor; +} + +function colorsFor(primitive, document, binary, count) { + const factor = baseColor(document, primitive); + if (primitive.attributes.COLOR_0 === undefined) return Array.from({ length: count }, () => factor.map(canonicalFloat)); + const reader = accessorReader(document, binary, primitive.attributes.COLOR_0, + { types: ['VEC3', 'VEC4'], components: [5121, 5123, 5126] }, 'COLOR_0'); + if (reader.count !== count) invalid('COLOR_0 count does not match POSITION'); + if (reader.componentType === 5126 && reader.normalized) invalid('FLOAT COLOR_0 cannot be normalized'); + if (reader.componentType !== 5126 && !reader.normalized) invalid('integer COLOR_0 must be normalized'); + const divisor = reader.componentType === 5121 ? 255 : reader.componentType === 5123 ? 65535 : 1; + return Array.from({ length: count }, (_, index) => { + const raw = reader.read(index).map((entry) => entry / divisor); + if (raw.length === 3) raw.push(1); + return raw.map((entry, channel) => canonicalFloat(entry * factor[channel])); + }); +} + +function tupleCompare(a, b) { + for (let index = 0; index < a.length; index += 1) { + if (a[index] < b[index]) return -1; + if (a[index] > b[index]) return 1; + } + return 0; +} + +function canonicalPart({ nodeName, primitiveOrdinal, positions, colors, triangles, reverseWinding }) { + const sourceRecords = positions.map((position, index) => ({ tuple: [...position, ...colors[index]], position, color: colors[index] })); + const unique = new Map(); + for (const record of sourceRecords) unique.set(record.tuple.join(','), record); + const records = [...unique.values()].sort((a, b) => tupleCompare(a.tuple, b.tuple)); + const canonicalIndex = new Map(records.map((record, index) => [record.tuple.join(','), index])); + const remapped = []; + for (const triangle of triangles) { + let indices = triangle.map((sourceIndex) => canonicalIndex.get(sourceRecords[sourceIndex].tuple.join(','))); + if (new Set(indices).size !== 3) continue; + if (reverseWinding) indices = [indices[0], indices[2], indices[1]]; + const rotations = [indices, [indices[1], indices[2], indices[0]], [indices[2], indices[0], indices[1]]]; + rotations.sort(tupleCompare); + remapped.push(rotations[0]); + } + remapped.sort(tupleCompare); + return { + nodeName, + primitiveOrdinal, + positions: records.map((record) => record.position), + colors: records.map((record) => record.color), + triangles: remapped, + }; +} + +function profile(document, binary, limits) { + if (!document || typeof document !== 'object' || document.asset?.version !== '2.0') invalid('GLB asset version must be 2.0'); + for (const field of ['extensions', 'extensionsUsed', 'extensionsRequired']) if (document[field] !== undefined) invalid('GLB extensions are unsupported', 'reference-geometry-unsupported'); + const buffers = array(document.buffers, 'buffers', 1); + if (buffers.length !== 1 || buffers[0].uri !== undefined) invalid('external resource URI is refused', 'reference-external-resource-refused'); + if (!Number.isSafeInteger(buffers[0].byteLength) || buffers[0].byteLength < 0 || buffers[0].byteLength > binary.length || binary.length - buffers[0].byteLength > 3) invalid('BIN chunk length does not match buffers[0]'); + const scenes = array(document.scenes, 'scenes', limits.maxScenes); + const nodes = array(document.nodes, 'nodes', limits.maxNodes); + const meshes = array(document.meshes, 'meshes', limits.maxMeshes); + document.accessors = array(document.accessors, 'accessors', limits.maxAccessors); + document.bufferViews = array(document.bufferViews, 'bufferViews', limits.maxBufferViews); + document.materials = document.materials === undefined ? [] : array(document.materials, 'materials', limits.maxPrimitives); + if (!Number.isSafeInteger(document.scene)) invalid('GLB requires an explicit active scene'); + const active = scenes[safeIndex(document.scene, scenes.length, 'active scene')]; + const roots = array(active.nodes, 'active scene nodes', limits.maxNodes); + const visiting = new Set(); + const visited = new Set(); + const names = new Set(); + const walked = []; + const walk = (nodeIndex, parent, depth) => { + safeIndex(nodeIndex, nodes.length, 'node'); + if (depth > limits.maxNodeDepth) invalid('active scene exceeds its depth limit'); + if (visiting.has(nodeIndex)) invalid('active scene contains a cycle'); + if (visited.has(nodeIndex)) invalid('active scene node has more than one parent'); + visiting.add(nodeIndex); visited.add(nodeIndex); + const node = nodes[nodeIndex]; + const world = multiply(parent, nodeMatrix(node)); + if (node.mesh !== undefined) { + safeIndex(node.mesh, meshes.length, 'node mesh'); + if (typeof node.name !== 'string' || !node.name) invalid('drawable node requires a non-empty name'); + if (names.has(node.name)) invalid('drawable node names must be unique'); + names.add(node.name); + walked.push({ node, world, mesh: meshes[node.mesh] }); + } + if (node.children !== undefined) for (const child of array(node.children, 'node children', limits.maxNodes)) walk(child, world, depth + 1); + visiting.delete(nodeIndex); + }; + for (const root of roots) walk(root, IDENTITY, 0); + return walked; +} + +function degenerate(a, b, c) { + const ab = b.map((value, index) => value - a[index]); + const ac = c.map((value, index) => value - a[index]); + const cross = [ab[1] * ac[2] - ab[2] * ac[1], ab[2] * ac[0] - ab[0] * ac[2], ab[0] * ac[1] - ab[1] * ac[0]]; + return cross.every((entry) => entry === 0); +} + +export function normalizeRevitGlb(input, options = {}) { + const limits = lowerableLimits(options.limits); + const { json: document, binary } = parseGlb(input, { limits }); + const nodes = profile(document, binary, limits); + const parts = []; + let inputTriangles = 0; + let droppedDegenerateTriangles = 0; + for (const { node, world, mesh } of nodes) { + const determinant = determinant3(world); + if (!Number.isFinite(determinant) || Math.abs(determinant) < 1e-12) invalid('node transform is singular'); + const primitives = array(mesh.primitives, 'mesh primitives', limits.maxPrimitives); + primitives.forEach((primitive, primitiveOrdinal) => { + if (primitive.targets !== undefined || primitive.extensions !== undefined) invalid('primitive extensions or morph targets are unsupported', 'reference-geometry-unsupported'); + if (!primitive.attributes || typeof primitive.attributes !== 'object' || Object.keys(primitive.attributes).some((key) => !['POSITION', 'COLOR_0'].includes(key))) invalid('primitive attributes are unsupported', 'reference-geometry-unsupported'); + const positionsReader = accessorReader(document, binary, primitive.attributes.POSITION, { types: ['VEC3'], components: [5126] }, 'POSITION'); + if (positionsReader.normalized || positionsReader.count > limits.maxVertices) invalid('POSITION accessor is unsupported or exceeds its limit', 'reference-output-too-large'); + const positions = Array.from({ length: positionsReader.count }, (_, index) => transform(world, positionsReader.read(index))); + const colors = colorsFor(primitive, document, binary, positions.length); + const expanded = expandTriangles(indicesFor(primitive, document, binary, positions.length, limits), primitive.mode ?? 4); + inputTriangles += expanded.length; + const drawable = expanded.filter(([a, b, c]) => { + const drop = a === b || b === c || a === c || degenerate(positions[a], positions[b], positions[c]); + if (drop) droppedDegenerateTriangles += 1; + return !drop; + }); + parts.push(canonicalPart({ nodeName: node.name, primitiveOrdinal, positions, colors, triangles: drawable, reverseWinding: determinant < 0 })); + }); + } + parts.sort((a, b) => Buffer.compare(Buffer.from(a.nodeName), Buffer.from(b.nodeName)) || a.primitiveOrdinal - b.primitiveOrdinal); + const glb = buildCanonicalGlb(parts); + if (glb.length > limits.maxCanonicalGlbBytes) invalid('canonical GLB exceeds its limit', 'reference-output-too-large'); + return { + glb, + parts, + coverage: { inputTriangles, outputTriangles: parts.reduce((sum, part) => sum + part.triangles.length, 0), droppedDegenerateTriangles }, + }; +} + +function align4(value) { return (value + 3) & ~3; } + +function buildCanonicalGlb(parts) { + const chunks = []; + const bufferViews = []; + const accessors = []; + const meshes = []; + const nodes = []; + let offset = 0; + const append = (bytes, target) => { + const aligned = align4(offset); + if (aligned > offset) chunks.push(Buffer.alloc(aligned - offset)); + offset = aligned; + const view = bufferViews.length; + bufferViews.push({ buffer: 0, byteOffset: offset, byteLength: bytes.length, target }); + chunks.push(bytes); offset += bytes.length; + return view; + }; + for (const part of parts) { + const positions = Buffer.alloc(part.positions.length * 12); + part.positions.forEach((point, index) => point.forEach((value, axis) => positions.writeFloatLE(value, index * 12 + axis * 4))); + const colors = Buffer.alloc(part.colors.length * 16); + part.colors.forEach((color, index) => color.forEach((value, channel) => colors.writeFloatLE(value, index * 16 + channel * 4))); + const flatIndices = part.triangles.flat(); + const indices = Buffer.alloc(flatIndices.length * 4); + flatIndices.forEach((value, index) => indices.writeUInt32LE(value, index * 4)); + const positionView = append(positions, 34962); + const colorView = append(colors, 34962); + const indexView = append(indices, 34963); + const positionAccessor = accessors.length; + const mins = [0, 1, 2].map((axis) => part.positions.length ? Math.min(...part.positions.map((point) => point[axis])) : 0); + const maxs = [0, 1, 2].map((axis) => part.positions.length ? Math.max(...part.positions.map((point) => point[axis])) : 0); + accessors.push({ bufferView: positionView, byteOffset: 0, componentType: 5126, count: part.positions.length, type: 'VEC3', min: mins, max: maxs }); + const colorAccessor = accessors.length; + accessors.push({ bufferView: colorView, byteOffset: 0, componentType: 5126, count: part.colors.length, type: 'VEC4' }); + const indexAccessor = accessors.length; + accessors.push({ bufferView: indexView, byteOffset: 0, componentType: 5125, count: flatIndices.length, type: 'SCALAR' }); + const mesh = meshes.length; + meshes.push({ primitives: [{ attributes: { POSITION: positionAccessor, COLOR_0: colorAccessor }, indices: indexAccessor, mode: 4 }] }); + nodes.push({ name: part.nodeName, mesh }); + } + const binary = Buffer.concat(chunks); + const document = { + accessors, + asset: { version: '2.0' }, + bufferViews, + buffers: [{ byteLength: binary.length }], + meshes, + nodes, + scene: 0, + scenes: [{ nodes: nodes.map((_, index) => index) }], + }; + const jsonBytes = canonicalJsonBytes(document); + const jsonLength = align4(jsonBytes.length); + const binaryLength = align4(binary.length); + const output = Buffer.alloc(12 + 8 + jsonLength + 8 + binaryLength); + output.writeUInt32LE(GLB_MAGIC, 0); output.writeUInt32LE(2, 4); output.writeUInt32LE(output.length, 8); + output.writeUInt32LE(jsonLength, 12); output.writeUInt32LE(JSON_CHUNK, 16); output.fill(0x20, 20, 20 + jsonLength); jsonBytes.copy(output, 20); + const binOffset = 20 + jsonLength; + output.writeUInt32LE(binaryLength, binOffset); output.writeUInt32LE(BIN_CHUNK, binOffset + 4); binary.copy(output, binOffset + 8); + return output; +} diff --git a/cli-connection-reader/revit-glb.test.mjs b/cli-connection-reader/revit-glb.test.mjs new file mode 100644 index 000000000..9e962142a --- /dev/null +++ b/cli-connection-reader/revit-glb.test.mjs @@ -0,0 +1,88 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { makeGlbFixture } from './model-fixtures.mjs'; +import { normalizeRevitGlb, parseGlb } from './revit-glb.mjs'; + +test('active-scene geometry is transformed from glTF Y-up metres to Z-up millimetres', () => { + const input = makeGlbFixture({ + positions: [[0, 0, 0], [1, 0, 0], [0, 0, 2]], + indices: [0, 1, 2], + translation: [2, 3, 4], + nodeName: 'wall-a', + }); + const result = normalizeRevitGlb(input); + assert.deepEqual(result.parts[0].positions, [ + [2000, -6000, 3000], [2000, -4000, 3000], [3000, -4000, 3000], + ]); + assert.deepEqual(result.parts[0].triangles, [[0, 1, 2]]); + assert.equal(result.parts[0].nodeName, 'wall-a'); + assert.equal(parseGlb(result.glb).json.scene, 0); +}); + +test('declared active scene is mandatory and inactive scenes are never traversed', () => { + assert.throws(() => normalizeRevitGlb(makeGlbFixture({ omitScene: true })), /explicit active scene/); + const input = makeGlbFixture({ + scenes: [{ nodes: [0] }, { nodes: [1] }], + scene: 1, + nodes: [{ name: 'inactive', mesh: 0 }, { name: 'active', mesh: 0 }], + }); + assert.deepEqual(normalizeRevitGlb(input).parts.map((part) => part.nodeName), ['active']); +}); + +test('triangle strips and fans expand deterministically and negative transforms preserve front faces', () => { + const positions = [[0, 0, 0], [1, 0, 0], [0, 1, 0], [1, 1, 0]]; + const strip = normalizeRevitGlb(makeGlbFixture({ positions, indices: [0, 1, 2, 3], mode: 5 })); + assert.equal(strip.parts[0].triangles.length, 2); + const fan = normalizeRevitGlb(makeGlbFixture({ positions, indices: [0, 1, 3, 2], mode: 6 })); + assert.equal(fan.parts[0].triangles.length, 2); + const ordinary = normalizeRevitGlb(makeGlbFixture()); + const reflected = normalizeRevitGlb(makeGlbFixture({ scale: [-1, 1, 1] })); + assert.deepEqual(reflected.coverage, { inputTriangles: 1, outputTriangles: 1, droppedDegenerateTriangles: 0 }); + assert.deepEqual(ordinary.parts[0].triangles, [[0, 2, 1]]); + assert.deepEqual(reflected.parts[0].triangles, [[0, 1, 2]]); +}); + +test('vertex and triangle permutations produce identical canonical GLB bytes', () => { + const first = makeGlbFixture({ + positions: [[0, 0, 0], [1, 0, 0], [0, 1, 0], [1, 1, 0]], + indices: [0, 1, 2, 1, 3, 2], + }); + const second = makeGlbFixture({ + positions: [[1, 1, 0], [0, 1, 0], [1, 0, 0], [0, 0, 0]], + indices: [2, 0, 1, 3, 2, 1], + }); + assert.deepEqual(normalizeRevitGlb(first).glb, normalizeRevitGlb(second).glb); +}); + +test('vertex color is multiplied by material base color and encoded as canonical RGBA', () => { + const result = normalizeRevitGlb(makeGlbFixture({ + colors: [[1, 0.5, 0, 1], [0.5, 1, 0, 0.5], [0, 0.5, 1, 1]], + materialColor: [0.5, 0.5, 1, 0.5], + })); + assert.deepEqual(result.parts[0].colors[0], [0.5, 0.25, 0, 0.5]); + assert.deepEqual(result.parts[0].colors[1], [0, 0.25, 1, 0.5]); + assert.deepEqual(result.parts[0].colors[2], [0.25, 0.5, 0, 0.25]); +}); + +test('unsafe resources, unsupported extensions, scene cycles, and malformed ranges are refused', () => { + assert.throws(() => normalizeRevitGlb(makeGlbFixture({ externalUri: 'https://example.test/model.bin' })), /external resource/); + assert.throws(() => normalizeRevitGlb(makeGlbFixture({ extensionsUsed: ['KHR_draco_mesh_compression'] })), /extensions/); + assert.throws(() => normalizeRevitGlb(makeGlbFixture({ nodes: [{ name: 'cycle', mesh: 0, children: [0] }] })), /cycle/); + const truncated = makeGlbFixture().subarray(0, -1); + assert.throws(() => normalizeRevitGlb(truncated), /length/); +}); + +test('duplicate keys in the GLB JSON chunk are rejected before profile validation', () => { + const valid = makeGlbFixture(); + const parsed = parseGlb(valid); + const text = parsed.jsonText; + const duplicate = text.replace('{"accessors"', '{"x":0,"x":1,"accessors"'); + const jsonBytes = Buffer.from(duplicate, 'utf8'); + const padded = (jsonBytes.length + 3) & ~3; + const bin = parsed.binary; + const out = Buffer.alloc(12 + 8 + padded + 8 + bin.length); + out.write('glTF', 0, 'ascii'); out.writeUInt32LE(2, 4); out.writeUInt32LE(out.length, 8); + out.writeUInt32LE(padded, 12); out.writeUInt32LE(0x4e4f534a, 16); out.fill(0x20, 20, 20 + padded); jsonBytes.copy(out, 20); + const offset = 20 + padded; out.writeUInt32LE(bin.length, offset); out.writeUInt32LE(0x004e4942, offset + 4); bin.copy(out, offset + 8); + assert.throws(() => normalizeRevitGlb(out), /duplicate JSON key/); +}); From 567681ee491b8c738aeeb6a1a142e68c688c2dca Mon Sep 17 00:00:00 2001 From: Pawel Date: Sun, 23 Aug 2026 14:09:26 +0200 Subject: [PATCH 04/50] feat: preserve explicit Revit metadata and relationships --- cli-connection-reader/revit-metadata.mjs | 270 ++++++++++++++++++ cli-connection-reader/revit-metadata.test.mjs | 88 ++++++ 2 files changed, 358 insertions(+) create mode 100644 cli-connection-reader/revit-metadata.mjs create mode 100644 cli-connection-reader/revit-metadata.test.mjs diff --git a/cli-connection-reader/revit-metadata.mjs b/cli-connection-reader/revit-metadata.mjs new file mode 100644 index 000000000..132b3c473 --- /dev/null +++ b/cli-connection-reader/revit-metadata.mjs @@ -0,0 +1,270 @@ +import { assertClosedObject, canonicalJsonBytes, lowerableLimits, ModelReaderError, parseJsonStrict, sha256 } from './model-contract.mjs'; + +const POSITIVE_INT64 = /^(?:[1-9]\d*)$/; +const SIGNED_INT64 = /^(?:0|-?[1-9]\d*)$/; +const MAX_INT64 = 9223372036854775807n; +const MIN_INT64 = -9223372036854775808n; + +function invalid(message, code = 'reference-metadata-invalid') { + throw new ModelReaderError(code, 'normalize-metadata', false, message); +} + +function closed(value, required, optional, label) { + try { return assertClosedObject(value, required, optional, label); } + catch (error) { invalid(error instanceof Error ? error.message : `${label} is invalid`); } +} + +function positiveId(value, label) { + if (typeof value !== 'string' || !POSITIVE_INT64.test(value)) invalid(`${label} must be a positive int64 decimal string`); + try { if (BigInt(value) > MAX_INT64) invalid(`${label} exceeds int64`); } + catch { invalid(`${label} must be a positive int64 decimal string`); } + return value; +} + +function signedId(value, label) { + if (typeof value !== 'string' || !SIGNED_INT64.test(value)) invalid(`${label} must be a signed int64 decimal string`); + try { + const parsed = BigInt(value); + if (parsed < MIN_INT64 || parsed > MAX_INT64) invalid(`${label} exceeds int64`); + } catch { invalid(`${label} must be a signed int64 decimal string`); } + return value; +} + +function text(value, label, nullable = false) { + if (nullable && value === null) return null; + if (typeof value !== 'string') invalid(`${label} must be ${nullable ? 'a string or null' : 'a string'}`); + return value; +} + +function list(value, label, limit) { + if (!Array.isArray(value) || value.length > limit) invalid(`${label} is missing or exceeds its count limit`); + return value; +} + +function uniqueTable(records, label, limit, allowed = []) { + const ids = new Set(); + return list(records, label, limit).map((record, index) => { + closed(record, ['id', 'name'], allowed, `${label}[${index}]`); + const id = positiveId(record.id, `${label}[${index}].id`); + if (ids.has(id)) invalid(`${label} contains duplicate id ${id}`); + ids.add(id); + return { ...record, id, name: text(record.name, `${label}[${index}].name`) }; + }); +} + +function tableIndex(value, table, label, nullable = false) { + if (nullable && value === null) return null; + if (!Number.isSafeInteger(value) || value < 0 || value >= table.length) invalid(`${label} index is out of range`); + return table[value]; +} + +function validateParameter(parameter, index) { + closed(parameter, ['id', 'name', 'unit', 'readable', 'storageType', 'value'], [], `parameters[${index}]`); + const id = positiveId(parameter.id, `parameters[${index}].id`); + const name = text(parameter.name, `parameters[${index}].name`); + const unit = text(parameter.unit, `parameters[${index}].unit`, true); + if (typeof parameter.readable !== 'boolean') invalid(`parameters[${index}].readable must be boolean`); + const storageType = parameter.storageType; + let value = parameter.value; + if (storageType === 'none') { + if (value !== null || parameter.readable !== false) invalid('none parameter must be unreadable null'); + } else if (storageType === 'boolean') { + if (typeof value !== 'boolean' || !parameter.readable) invalid('boolean parameter has the wrong value type'); + } else if (storageType === 'integer') { + value = signedId(value, `parameters[${index}].value`); + if (!parameter.readable) invalid('integer parameter must be readable'); + } else if (storageType === 'double') { + if (typeof value !== 'number' || !Number.isFinite(value) || !parameter.readable) invalid('double parameter must be a readable finite number'); + value = Object.is(value, -0) ? 0 : value; + } else if (storageType === 'string') { + value = text(value, `parameters[${index}].value`); + if (!parameter.readable) invalid('string parameter must be readable'); + } else if (storageType === 'element-id') { + value = signedId(value, `parameters[${index}].value`); + if (!parameter.readable) invalid('element-id parameter must be readable'); + } else invalid(`parameters[${index}] has unsupported storageType`); + return { id, name, unit, readable: parameter.readable, storageType, value }; +} + +function bounds(parts) { + const positions = parts.flatMap((part) => part.positions ?? []); + if (!positions.length) return null; + return { + min: [0, 1, 2].map((axis) => Math.min(...positions.map((point) => point[axis]))), + max: [0, 1, 2].map((axis) => Math.max(...positions.map((point) => point[axis]))), + }; +} + +function compareDecimal(a, b) { + const left = BigInt(a.includes(':') ? a.slice(a.indexOf(':') + 1) : a); + const right = BigInt(b.includes(':') ? b.slice(b.indexOf(':') + 1) : b); + return left < right ? -1 : left > right ? 1 : 0; +} + +function digestIds(ids) { return sha256(canonicalJsonBytes([...ids].sort((a, b) => Buffer.compare(Buffer.from(a), Buffer.from(b))))); } + +function assertAcyclic(relations, kind) { + const adjacency = new Map(); + const parents = new Map(); + for (const relation of relations.filter((entry) => entry.kind === kind)) { + if (parents.has(relation.to)) invalid(`${kind} relationship has an ambiguous parent`, 'reference-relationship-invalid'); + parents.set(relation.to, relation.from); + const targets = adjacency.get(relation.from) ?? []; + targets.push(relation.to); adjacency.set(relation.from, targets); + } + const visiting = new Set(); + const visited = new Set(); + const visit = (id) => { + if (visiting.has(id)) invalid(`${kind} relationship contains a cycle`, 'reference-relationship-invalid'); + if (visited.has(id)) return; + visiting.add(id); + for (const target of adjacency.get(id) ?? []) visit(target); + visiting.delete(id); visited.add(id); + }; + for (const id of adjacency.keys()) visit(id); +} + +export function normalizeRevitMetadata(input, geometryParts, options = {}) { + const limits = lowerableLimits(options.limits); + let metadata = input; + if (typeof input === 'string' || Buffer.isBuffer(input) || input instanceof Uint8Array) { + try { metadata = parseJsonStrict(input, { maxBytes: limits.maxMetadataBytes, maxDepth: limits.maxJsonDepth }); } + catch (error) { invalid(error instanceof Error ? error.message : 'metadata JSON is invalid'); } + } + closed(metadata, + ['schemaVersion', 'document', 'types', 'levels', 'parameterGroups', 'parameters', 'elements', 'relations'], [], 'metadata'); + if (metadata.schemaVersion !== '1') invalid('unsupported metadata schemaVersion'); + closed(metadata.document, ['kind', 'id'], [], 'document'); + if (metadata.document.kind !== 'revit-project' || typeof metadata.document.id !== 'string' || !metadata.document.id) invalid('document must be an identified Revit project'); + + const types = uniqueTable(metadata.types, 'types', limits.maxEntities); + const levels = uniqueTable(metadata.levels, 'levels', limits.maxEntities, ['elevation']).map((level, index) => { + if (level.elevation !== undefined && (typeof level.elevation !== 'number' || !Number.isFinite(level.elevation))) invalid(`levels[${index}].elevation must be finite`); + return level; + }); + const parameters = list(metadata.parameters, 'parameters', limits.maxParameters).map(validateParameter); + if (new Set(parameters.map((entry) => entry.id)).size !== parameters.length) invalid('parameters contains duplicate ids'); + const parameterGroups = uniqueTable(metadata.parameterGroups, 'parameterGroups', limits.maxParameters, ['parameters']).map((group, index) => { + const refs = list(group.parameters, `parameterGroups[${index}].parameters`, limits.maxParameters); + return { ...group, parameters: refs.map((value, ordinal) => tableIndex(value, parameters, `parameterGroups[${index}].parameters[${ordinal}]`)) }; + }); + + if (!Array.isArray(geometryParts)) invalid('geometry parts are required for explicit joins'); + const geometryByName = new Map(); + for (const [index, part] of geometryParts.entries()) { + if (!part || typeof part.nodeName !== 'string' || !part.nodeName) invalid(`geometry part ${index} has no node name`); + const parts = geometryByName.get(part.nodeName) ?? []; + parts.push(part); geometryByName.set(part.nodeName, parts); + } + for (const parts of geometryByName.values()) parts.sort((a, b) => (a.primitiveOrdinal ?? 0) - (b.primitiveOrdinal ?? 0)); + + const rawElements = list(metadata.elements, 'elements', limits.maxEntities); + const elementIds = new Set(); + const owners = new Map(); + const propertyRows = []; + const entities = rawElements.map((element, elementOrdinal) => { + closed(element, + ['id', 'revitClass', 'category', 'family', 'type', 'level', 'parameterGroups', 'appearances'], ['ifcGuid'], `elements[${elementOrdinal}]`); + const id = positiveId(element.id, `elements[${elementOrdinal}].id`); + if (elementIds.has(id)) invalid(`elements contains duplicate id ${id}`); + elementIds.add(id); + const appearances = list(element.appearances, `elements[${elementOrdinal}].appearances`, limits.maxNodes); + if (!appearances.length || new Set(appearances).size !== appearances.length || appearances.some((name) => typeof name !== 'string' || !name)) invalid(`element ${id} appearances must be unique non-empty strings`); + const joined = []; + for (const name of appearances) { + const parts = geometryByName.get(name); + if (!parts) invalid(`appearance '${name}' does not resolve to an active-scene geometry node`, 'reference-metadata-join-ambiguous'); + if (owners.has(name)) invalid(`appearance '${name}' belongs to more than one entity`, 'reference-metadata-join-ambiguous'); + owners.set(name, id); + joined.push({ nodeName: name, parts: parts.map((part) => part.primitiveOrdinal ?? 0) }); + } + const groups = list(element.parameterGroups, `elements[${elementOrdinal}].parameterGroups`, limits.maxParameters) + .map((value, ordinal) => tableIndex(value, parameterGroups, `elements[${elementOrdinal}].parameterGroups[${ordinal}]`)); + const guidValues = []; + groups.forEach((group, groupOrdinal) => group.parameters.forEach((parameter, parameterOrdinal) => { + propertyRows.push({ + entityId: `element:${id}`, + groupId: `parameter-group:${group.id}`, + groupName: group.name, + groupOrdinal, + parameterId: `parameter:${parameter.id}`, + parameterOrdinal, + name: parameter.name, + unit: parameter.unit, + readable: parameter.readable, + storageType: parameter.storageType, + value: parameter.value, + }); + if (parameter.name === 'IfcGUID' && parameter.storageType === 'string' && parameter.readable && parameter.value) guidValues.push(parameter.value); + })); + if (new Set(guidValues).size > 1) invalid(`element ${id} has conflicting IfcGUID parameters`); + const ifcGuid = guidValues[0] ?? null; + if (element.ifcGuid !== undefined && element.ifcGuid !== ifcGuid) invalid(`element ${id} redundant ifcGuid does not match its authoritative parameter`); + const type = tableIndex(element.type, types, `elements[${elementOrdinal}].type`, true); + const level = tableIndex(element.level, levels, `elements[${elementOrdinal}].level`, true); + return { + id: `element:${id}`, + sourceElementId: id, + ifcGuid, + revitClass: text(element.revitClass, `elements[${elementOrdinal}].revitClass`, true), + category: text(element.category, `elements[${elementOrdinal}].category`, true), + family: text(element.family, `elements[${elementOrdinal}].family`, true), + typeId: type ? `type:${type.id}` : null, + typeName: type?.name ?? null, + levelId: level ? `level:${level.id}` : null, + levelName: level?.name ?? null, + geometry: joined, + bounds: bounds(joined.flatMap((item) => geometryByName.get(item.nodeName))), + }; + }).sort((a, b) => compareDecimal(a.id, b.id)); + + const guidCounts = new Map(); + for (const entity of entities) { + if (entity.ifcGuid) guidCounts.set(entity.ifcGuid, (guidCounts.get(entity.ifcGuid) ?? 0) + 1); + } + for (const entity of entities) { + if (entity.ifcGuid && guidCounts.get(entity.ifcGuid) > 1) entity.ifcGuid = null; + } + + propertyRows.sort((a, b) => compareDecimal(a.entityId, b.entityId) || a.groupOrdinal - b.groupOrdinal || a.parameterOrdinal - b.parameterOrdinal || compareDecimal(a.parameterId, b.parameterId)); + const relationIds = new Set(); + const relationships = list(metadata.relations, 'relations', limits.maxRelationships).map((relation, index) => { + closed(relation, ['id', 'kind', 'from', 'to'], ['providerRelationKind'], `relations[${index}]`); + const id = positiveId(relation.id, `relations[${index}].id`); + if (relationIds.has(id)) invalid(`relations contains duplicate id ${id}`); + relationIds.add(id); + if (!['contains', 'hosts', 'depends-on', 'provider-explicit'].includes(relation.kind)) invalid(`relation ${id} has unsupported kind`, 'reference-relationship-invalid'); + const from = positiveId(relation.from, `relations[${index}].from`); + const to = positiveId(relation.to, `relations[${index}].to`); + if (!elementIds.has(from) || !elementIds.has(to)) invalid(`relation ${id} has a missing endpoint`, 'reference-relationship-invalid'); + let providerRelationKind; + if (relation.kind === 'provider-explicit') { + providerRelationKind = text(relation.providerRelationKind, `relations[${index}].providerRelationKind`); + if (!providerRelationKind || Buffer.byteLength(providerRelationKind) > 256) invalid(`relation ${id} providerRelationKind is invalid`); + } else if (relation.providerRelationKind !== undefined) invalid(`relation ${id} has an unexpected providerRelationKind`); + return { id: `relation:${id}`, kind: relation.kind, from: `element:${from}`, to: `element:${to}`, ...(providerRelationKind ? { providerRelationKind } : {}) }; + }); + assertAcyclic(relationships, 'contains'); + assertAcyclic(relationships, 'hosts'); + relationships.sort((a, b) => Buffer.compare(Buffer.from(a.kind), Buffer.from(b.kind)) || compareDecimal(a.from, b.from) || compareDecimal(a.to, b.to) || compareDecimal(a.id, b.id)); + + const unclaimedGeometryNodes = [...geometryByName.keys()].filter((name) => !owners.has(name)).sort((a, b) => Buffer.compare(Buffer.from(a), Buffer.from(b))); + const entitiesBytes = canonicalJsonBytes({ schemaVersion: '1', entities }); + const propertiesBytes = canonicalJsonBytes({ schemaVersion: '1', properties: propertyRows }); + const relationshipsBytes = canonicalJsonBytes({ schemaVersion: '1', relationships }); + for (const [label, bytes] of [['entities', entitiesBytes], ['properties', propertiesBytes], ['relationships', relationshipsBytes]]) { + if (bytes.length > limits.maxComponentJsonBytes) invalid(`${label} artifact exceeds its byte limit`, 'reference-output-too-large'); + } + const coverage = { + discoveredEntities: rawElements.length, + indexedEntities: entities.length, + drawableEntities: entities.filter((entity) => entity.geometry.length > 0).length, + geometryNodes: geometryByName.size, + properties: propertyRows.length, + relationships: relationships.length, + unclaimedGeometryNodes, + entitySetSha256: digestIds(entities.map((entity) => entity.id)), + geometryNodeSetSha256: digestIds(geometryByName.keys()), + }; + return { entities, properties: propertyRows, relationships, entitiesBytes, propertiesBytes, relationshipsBytes, coverage }; +} diff --git a/cli-connection-reader/revit-metadata.test.mjs b/cli-connection-reader/revit-metadata.test.mjs new file mode 100644 index 000000000..1931b4caf --- /dev/null +++ b/cli-connection-reader/revit-metadata.test.mjs @@ -0,0 +1,88 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { makeMetadataFixture } from './model-fixtures.mjs'; +import { normalizeRevitMetadata } from './revit-metadata.mjs'; + +const geometry = [ + { nodeName: 'part-a', primitiveOrdinal: 0, positions: [[0, 0, 0], [10, 0, 0], [0, 10, 0]], triangles: [[0, 1, 2]] }, + { nodeName: 'part-b', primitiveOrdinal: 0, positions: [[20, 0, 0], [30, 0, 0], [20, 10, 0]], triangles: [[0, 1, 2]] }, +]; + +test('explicit indexed metadata resolves to stable namespaced identity and multipart geometry', () => { + const result = normalizeRevitMetadata(makeMetadataFixture({ elementId: '9223372036854775806', nodeNames: ['part-a', 'part-b'] }), geometry); + assert.equal(result.entities[0].id, 'element:9223372036854775806'); + assert.equal(result.entities[0].typeId, 'type:2001'); + assert.equal(result.entities[0].levelId, 'level:3001'); + assert.deepEqual(result.entities[0].geometry.map((item) => item.nodeName), ['part-a', 'part-b']); + assert.equal(result.properties[0].storageType, 'string'); + assert.equal(result.properties[0].value, '1FixtureGuid00000000000'); + assert.equal(result.entities[0].ifcGuid, '1FixtureGuid00000000000'); +}); + +test('parameter storage types preserve signed element ids, null, empty, boolean and finite doubles', () => { + const metadata = makeMetadataFixture(); + metadata.parameters = [ + { id: '1', name: 'None', unit: null, readable: false, storageType: 'none', value: null }, + { id: '2', name: 'Bool', unit: null, readable: true, storageType: 'boolean', value: true }, + { id: '3', name: 'Integer', unit: null, readable: true, storageType: 'integer', value: '-12' }, + { id: '4', name: 'Double', unit: 'ft', readable: true, storageType: 'double', value: 1.25 }, + { id: '5', name: 'String', unit: null, readable: true, storageType: 'string', value: '' }, + { id: '6', name: 'Element', unit: null, readable: true, storageType: 'element-id', value: '-2000011' }, + ]; + metadata.parameterGroups[0].parameters = [0, 1, 2, 3, 4, 5]; + delete metadata.elements[0].ifcGuid; + const result = normalizeRevitMetadata(metadata, geometry.slice(0, 1)); + assert.deepEqual(result.properties.map((row) => row.value), [null, true, '-12', 1.25, '', '-2000011']); +}); + +test('explicit relations validate endpoints, provider kinds, acyclic parents, and canonical order', () => { + const metadata = makeMetadataFixture({ elementId: '2', nodeNames: ['part-b'] }); + const firstElement = { ...metadata.elements[0], id: '1', appearances: ['part-a'] }; + delete firstElement.ifcGuid; + metadata.elements.unshift(firstElement); + metadata.relations = [ + { id: '11', kind: 'provider-explicit', providerRelationKind: 'Joins', from: '2', to: '1' }, + { id: '10', kind: 'contains', from: '1', to: '2' }, + ]; + const result = normalizeRevitMetadata(metadata, geometry); + assert.deepEqual(result.relationships.map((edge) => edge.id), ['relation:10', 'relation:11']); + assert.equal(result.relationships[1].providerRelationKind, 'Joins'); + metadata.relations.push({ id: '12', kind: 'contains', from: '2', to: '1' }); + assert.throws(() => normalizeRevitMetadata(metadata, geometry), /cycle/); +}); + +test('ambiguous, missing, and duplicate appearance ownership is refused without name inference', () => { + const duplicate = makeMetadataFixture({ nodeNames: ['part-a'] }); + const duplicateOwner = { ...duplicate.elements[0], id: '1002', appearances: ['part-a'] }; + delete duplicateOwner.ifcGuid; + duplicate.elements.push(duplicateOwner); + assert.throws(() => normalizeRevitMetadata(duplicate, geometry.slice(0, 1)), /more than one entity/); + assert.throws(() => normalizeRevitMetadata(makeMetadataFixture({ nodeNames: ['1001_0'] }), geometry), /does not resolve/); + const numeric = makeMetadataFixture(); + numeric.elements[0].id = 9007199254740992; + assert.throws(() => normalizeRevitMetadata(numeric, geometry.slice(0, 1)), /decimal string/); +}); + +test('set-like element permutations produce identical canonical artifact bytes and exact coverage', () => { + const metadata = makeMetadataFixture({ elementId: '2', nodeNames: ['part-b'] }); + const firstElement = { ...metadata.elements[0], id: '1', appearances: ['part-a'] }; + delete firstElement.ifcGuid; + metadata.elements.push(firstElement); + const first = normalizeRevitMetadata(metadata, geometry); + const second = normalizeRevitMetadata({ ...metadata, elements: [...metadata.elements].reverse() }, [...geometry].reverse()); + assert.deepEqual(first.entitiesBytes, second.entitiesBytes); + assert.deepEqual(first.propertiesBytes, second.propertiesBytes); + assert.deepEqual(first.relationshipsBytes, second.relationshipsBytes); + assert.equal(first.coverage.discoveredEntities, 2); + assert.equal(first.coverage.drawableEntities, 2); + assert.equal(first.coverage.unclaimedGeometryNodes.length, 0); +}); + +test('duplicate authoritative IfcGUID values make every matching entity uncomparable', () => { + const metadata = makeMetadataFixture({ elementId: '2', nodeNames: ['part-b'] }); + const firstElement = { ...metadata.elements[0], id: '1', appearances: ['part-a'] }; + delete firstElement.ifcGuid; + metadata.elements.push(firstElement); + const result = normalizeRevitMetadata(metadata, geometry); + assert.deepEqual(result.entities.map((entity) => entity.ifcGuid), [null, null]); +}); From 6032630158657ebd0f9c0374a9ddae0b10849a5b Mon Sep 17 00:00:00 2001 From: Pawel Date: Sun, 23 Aug 2026 14:12:59 +0200 Subject: [PATCH 05/50] feat: fence the local model provider protocol --- cli-connection-reader/model-provider.mjs | 178 ++++++++++++++++++ cli-connection-reader/model-provider.test.mjs | 154 +++++++++++++++ .../test-fixtures/model-provider-fixture.mjs | 31 +++ 3 files changed, 363 insertions(+) create mode 100644 cli-connection-reader/model-provider.mjs create mode 100644 cli-connection-reader/model-provider.test.mjs create mode 100644 cli-connection-reader/test-fixtures/model-provider-fixture.mjs diff --git a/cli-connection-reader/model-provider.mjs b/cli-connection-reader/model-provider.mjs new file mode 100644 index 000000000..b051ae5f1 --- /dev/null +++ b/cli-connection-reader/model-provider.mjs @@ -0,0 +1,178 @@ +import fs from 'node:fs/promises'; +import path from 'node:path'; +import { + assertClosedObject, assertSha256, buildCanonicalRequest, buildProviderFingerprint, + canonicalJsonBytes, lowerableLimits, ModelReaderError, parseJsonStrict, sha256, +} from './model-contract.mjs'; + +function providerError(code, message, retryable = false, details = undefined) { + throw new ModelReaderError(code, 'provider', retryable, message, details); +} + +function samePath(left, right) { + const normalize = (value) => process.platform === 'win32' ? path.resolve(value).toLowerCase() : path.resolve(value); + return normalize(left) === normalize(right); +} + +async function regularNonLink(filePath, label) { + if (typeof filePath !== 'string' || !path.isAbsolute(filePath)) providerError(`reference-${label}-unsafe`, `${label} path must be absolute.`); + let stat; let real; + try { stat = await fs.lstat(filePath); real = await fs.realpath(filePath); } + catch (error) { providerError(`reference-${label}-unavailable`, `${label} is unavailable.`, false, error); } + if (!stat.isFile()) providerError(`reference-${label}-unsafe`, `${label} must be a regular file.`); + if (stat.isSymbolicLink() || !samePath(real, filePath)) providerError(`reference-${label}-unsafe`, `${label} cannot be a link or reparse point.`); + return stat; +} + +async function fileHash(filePath, limit, label) { + const stat = await regularNonLink(filePath, label); + if (stat.size > limit) providerError(`reference-${label}-too-large`, `${label} exceeds its byte limit.`); + const bytes = await fs.readFile(filePath); + return { stat, bytes, sha256: sha256(bytes) }; +} + +export async function validateProviderExecutable(executable) { + const stat = await regularNonLink(executable, 'provider'); + return { path: executable, size: stat.size, sha256: sha256(await fs.readFile(executable)) }; +} + +export function minimalProviderEnvironment(source = process.env, platform = process.platform) { + const allowed = platform === 'win32' ? ['SYSTEMROOT', 'WINDIR', 'COMSPEC', 'TEMP', 'TMP'] : ['HOME', 'TMPDIR']; + const result = {}; + for (const key of allowed) if (typeof source[key] === 'string' && source[key]) result[key] = source[key]; + result.LANG = 'C'; result.LC_ALL = 'C'; result.TZ = 'UTC'; + return result; +} + +async function privateDirectory(directory) { + await fs.mkdir(directory, { recursive: false, mode: 0o700 }); + const stat = await fs.lstat(directory); + if (!stat.isDirectory() || stat.isSymbolicLink()) providerError('reference-output-unsafe', 'Private provider directory is unsafe.'); + return directory; +} + +export async function stageImmutableSource(sourcePath, stagingRoot, expectedSourceSha256, options = {}) { + assertSha256(expectedSourceSha256, 'expectedSourceSha256'); + const limits = lowerableLimits(options.limits); + const before = await fileHash(sourcePath, limits.maxSourceBytes, 'source'); + if (before.sha256 !== expectedSourceSha256) providerError('reference-source-changed', 'The model source changed before staging.'); + await privateDirectory(stagingRoot); + const stagedPath = path.join(stagingRoot, 'source.rvt'); + const handle = await fs.open(stagedPath, 'wx', 0o600); + try { await handle.writeFile(before.bytes); await handle.sync(); } + finally { await handle.close(); } + const staged = await fileHash(stagedPath, limits.maxSourceBytes, 'source'); + const after = await fileHash(sourcePath, limits.maxSourceBytes, 'source'); + if (staged.sha256 !== expectedSourceSha256 || after.sha256 !== expectedSourceSha256) { + providerError('reference-source-changed', 'The model source changed while staging.'); + } + await fs.chmod(stagedPath, 0o400); + return { path: stagedPath, sourceSha256: staged.sha256, size: staged.stat.size }; +} + +function boundedString(value, label, maximum = 256) { + if (typeof value !== 'string' || !value || Buffer.byteLength(value) > maximum) providerError('reference-provider-protocol', `Provider returned an invalid ${label}.`); + return value; +} + +function validateDescribe(value) { + try { assertClosedObject(value, ['protocolVersion', 'provider', 'engine', 'engineVersion', 'adapterBuildId', 'formats', 'execution', 'destination'], [], 'provider description'); } + catch (error) { providerError('reference-provider-protocol', 'Provider description does not match protocol v1.', false, error); } + if (value.protocolVersion !== '1' || value.execution !== 'local' || value.destination !== null || !Array.isArray(value.formats) || value.formats.length !== 1 || value.formats[0] !== 'rvt') { + providerError('reference-provider-protocol', 'Provider description does not match the local RVT protocol.'); + } + for (const key of ['provider', 'engine', 'engineVersion', 'adapterBuildId']) boundedString(value[key], key); + return value; +} + +function validateReceipt(value, describe, sourceSha256) { + try { assertClosedObject(value, ['protocolVersion', 'provider', 'engine', 'engineVersion', 'adapterBuildId', 'formats', 'execution', 'destination', 'documentKind', 'sourceSha256', 'geometryPath', 'metadataPath'], [], 'provider receipt'); } + catch (error) { providerError('reference-provider-protocol', 'Provider receipt does not match protocol v1.', false, error); } + validateDescribe(Object.fromEntries(['protocolVersion', 'provider', 'engine', 'engineVersion', 'adapterBuildId', 'formats', 'execution', 'destination'].map((key) => [key, value[key]]))); + for (const key of ['protocolVersion', 'provider', 'engine', 'engineVersion', 'adapterBuildId', 'execution', 'destination']) { + if (value[key] !== describe[key]) providerError('reference-provider-changed', 'Provider provenance changed during conversion.'); + } + if (JSON.stringify(value.formats) !== JSON.stringify(describe.formats)) providerError('reference-provider-changed', 'Provider formats changed during conversion.'); + if (value.documentKind !== 'revit-project') providerError('reference-provider-protocol', 'Provider returned the wrong document kind.'); + if (value.sourceSha256 !== sourceSha256) providerError('reference-source-changed', 'Provider did not convert the staged source.'); + assertSha256(value.sourceSha256, 'sourceSha256'); + return value; +} + +function parseProviderJson(bytes, limits, label) { + if (!Buffer.isBuffer(bytes) || bytes.length > limits.providerStdoutBytes) providerError('reference-provider-output-too-large', `Provider ${label} exceeded its output limit.`); + try { return parseJsonStrict(bytes, { maxBytes: limits.providerStdoutBytes, maxDepth: limits.maxJsonDepth }); } + catch (error) { providerError('reference-provider-protocol', `Provider ${label} was not valid closed JSON.`, false, error); } +} + +async function callProvider(hostRun, request, limits) { + if (typeof hostRun !== 'function') providerError('reference-provider-host-unavailable', 'The managed provider host is unavailable.'); + if (request.stdin.length > limits.providerRequestBytes) providerError('reference-provider-request-too-large', 'Provider request exceeds its byte limit.'); + let result; + try { result = await hostRun(request); } + catch (error) { providerError('reference-provider-failed', 'The local model provider failed.', true, error); } + if (!result || !Number.isSafeInteger(result.exitCode) || !Buffer.isBuffer(result.stdout) || !Buffer.isBuffer(result.stderr)) providerError('reference-provider-host-protocol', 'The managed provider host returned an invalid result.'); + if (result.stdout.length > limits.providerStdoutBytes || result.stderr.length > limits.providerStderrBytes) providerError('reference-provider-output-too-large', 'Provider diagnostics exceeded their byte limit.'); + if (result.exitCode !== 0) providerError('reference-provider-failed', 'The local model provider failed.', true, { exitCode: result.exitCode, stderr: result.stderr }); + return result.stdout; +} + +async function validatedOutput(outputPath, expectedPath, limit, label) { + if (typeof outputPath !== 'string' || !path.isAbsolute(outputPath) || !samePath(outputPath, expectedPath)) providerError('reference-provider-protocol', `Provider returned an invalid ${label} path.`); + const output = await fileHash(outputPath, limit, 'output'); + return { path: outputPath, bytes: output.bytes, size: output.stat.size, sha256: output.sha256 }; +} + +export async function describeAndConvert(options) { + const limits = lowerableLimits(options.limits); + const initialExecutable = await validateProviderExecutable(options.executable); + await privateDirectory(options.privateRoot); + const staging = await stageImmutableSource(options.sourcePath, path.join(options.privateRoot, 'source'), options.expectedSourceSha256, { limits }); + const describeCwd = await privateDirectory(path.join(options.privateRoot, 'describe')); + const environment = minimalProviderEnvironment(options.environment); + const describeRequest = canonicalJsonBytes({ protocolVersion: '1', limits }); + const describeBytes = await callProvider(options.hostRun, { + executable: initialExecutable.path, operation: 'describe', stdin: describeRequest, + stdinLength: describeRequest.length, cwd: describeCwd, environment, + timeoutMs: limits.conversionMs, stdoutLimit: limits.providerStdoutBytes, stderrLimit: limits.providerStderrBytes, + }, limits); + const afterDescribe = await validateProviderExecutable(options.executable); + if (afterDescribe.sha256 !== initialExecutable.sha256) providerError('reference-provider-changed', 'Provider executable changed during description.'); + const describe = validateDescribe(parseProviderJson(describeBytes, limits, 'description')); + const canonicalRequest = buildCanonicalRequest({ limits, conversionSettings: options.conversionSettings ?? {} }); + const outputDirectory = await privateDirectory(path.join(options.privateRoot, 'output')); + const beforeConvert = await validateProviderExecutable(options.executable); + if (beforeConvert.sha256 !== initialExecutable.sha256) providerError('reference-provider-changed', 'Provider executable changed before conversion.'); + const convertRequest = canonicalJsonBytes({ + protocolVersion: '1', sourcePath: staging.path, outputDirectory, + sourceSha256: staging.sourceSha256, canonicalRequest, limits, + }); + const receiptBytes = await callProvider(options.hostRun, { + executable: initialExecutable.path, operation: 'convert', stdin: convertRequest, + stdinLength: convertRequest.length, cwd: outputDirectory, environment, + timeoutMs: limits.conversionMs, stdoutLimit: limits.providerStdoutBytes, stderrLimit: limits.providerStderrBytes, + }, limits); + const afterConvert = await validateProviderExecutable(options.executable); + if (afterConvert.sha256 !== initialExecutable.sha256) providerError('reference-provider-changed', 'Provider executable changed during conversion.'); + const receipt = validateReceipt(parseProviderJson(receiptBytes, limits, 'receipt'), describe, staging.sourceSha256); + const geometryPath = path.join(outputDirectory, 'geometry.glb'); + const metadataPath = path.join(outputDirectory, 'metadata.json'); + const entries = await fs.readdir(outputDirectory); + if (entries.length !== 2 || !entries.includes('geometry.glb') || !entries.includes('metadata.json')) providerError('reference-provider-extra-output', 'Provider output directory was not closed.'); + const geometry = await validatedOutput(receipt.geometryPath, geometryPath, limits.maxInputGlbBytes, 'geometry'); + const metadata = await validatedOutput(receipt.metadataPath, metadataPath, limits.maxMetadataBytes, 'metadata'); + if (geometry.size + metadata.size > limits.maxProviderOutputBytes) providerError('reference-provider-output-too-large', 'Provider files exceed their total byte limit.'); + return { + describe, receipt, canonicalRequest, + fingerprint: buildProviderFingerprint({ + protocolVersion: describe.protocolVersion, + provider: describe.provider, + engine: describe.engine, + engineVersion: describe.engineVersion, + adapterBuildId: describe.adapterBuildId, + adapterExecutableSha256: initialExecutable.sha256, + }), + providerExecutableSha256: initialExecutable.sha256, + stagedSource: staging, outputs: { geometry, metadata }, + }; +} diff --git a/cli-connection-reader/model-provider.test.mjs b/cli-connection-reader/model-provider.test.mjs new file mode 100644 index 000000000..14b8757aa --- /dev/null +++ b/cli-connection-reader/model-provider.test.mjs @@ -0,0 +1,154 @@ +import assert from 'node:assert/strict'; +import { spawn } from 'node:child_process'; +import fs from 'node:fs/promises'; +import os from 'node:os'; +import path from 'node:path'; +import test from 'node:test'; +import { fileURLToPath } from 'node:url'; +import { sha256 } from './model-contract.mjs'; +import { + describeAndConvert, minimalProviderEnvironment, stageImmutableSource, + validateProviderExecutable, +} from './model-provider.mjs'; + +const fixture = fileURLToPath(new URL('./test-fixtures/model-provider-fixture.mjs', import.meta.url)); + +async function temporaryDirectory(t) { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'aware-model-provider-')); + t.after(() => fs.rm(directory, { recursive: true, force: true })); + return directory; +} + +function fixtureHostRun(calls) { + return async (request) => { + calls.push(request); + return await new Promise((resolve, reject) => { + const child = spawn(process.execPath, [fixture, request.operation], { + cwd: request.cwd, env: request.environment, shell: false, windowsHide: true, + stdio: ['pipe', 'pipe', 'pipe'], + }); + const stdout = []; const stderr = []; + child.stdout.on('data', (chunk) => stdout.push(chunk)); + child.stderr.on('data', (chunk) => stderr.push(chunk)); + child.on('error', reject); + child.on('close', (exitCode) => resolve({ exitCode, stdout: Buffer.concat(stdout), stderr: Buffer.concat(stderr) })); + child.stdin.end(request.stdin); + }); + }; +} + +test('provider executable and source must be absolute regular non-link files', async (t) => { + const root = await temporaryDirectory(t); + const executable = path.join(root, 'provider.exe'); + await fs.writeFile(executable, 'fixture'); + assert.equal((await validateProviderExecutable(executable)).path, executable); + await assert.rejects(() => validateProviderExecutable('provider.exe'), /absolute/); + await assert.rejects(() => validateProviderExecutable(root), /regular file/); + const link = path.join(root, 'provider-link.exe'); + try { + await fs.symlink(executable, link, 'file'); + await assert.rejects(() => validateProviderExecutable(link), /link|reparse/); + } catch (error) { + if (error?.code !== 'EPERM') throw error; + t.diagnostic('symlink creation is unavailable; the explicit link refusal branch is unverified'); + } +}); + +test('minimal provider environment omits paths, proxies, credentials, and AWARE state', () => { + const environment = minimalProviderEnvironment({ + SYSTEMROOT: 'C:\\Windows', TEMP: 'C:\\Temp', PATH: 'secret', HTTP_PROXY: 'secret', + AWARE_HOME: 'secret', AWS_SECRET_ACCESS_KEY: 'secret', TOKEN: 'secret', + }, 'win32'); + assert.deepEqual(environment, { SYSTEMROOT: 'C:\\Windows', TEMP: 'C:\\Temp', LANG: 'C', LC_ALL: 'C', TZ: 'UTC' }); +}); + +test('source staging hashes both sides, creates an immutable private copy, and detects expected-hash drift', async (t) => { + const root = await temporaryDirectory(t); + const source = path.join(root, 'source.rvt'); + const staging = path.join(root, 'staging'); + const bytes = Buffer.from('deterministic-rvt-fixture'); + await fs.writeFile(source, bytes); + const expected = sha256(bytes); + const staged = await stageImmutableSource(source, staging, expected); + assert.equal(staged.sourceSha256, expected); + assert.deepEqual(await fs.readFile(staged.path), bytes); + assert.equal((await fs.stat(staged.path)).mode & 0o222, 0); + await assert.rejects(() => stageImmutableSource(source, path.join(root, 'other'), '0'.repeat(64)), /source changed/); +}); + +test('describe and convert agree on provenance and return only bounded private outputs', async (t) => { + const root = await temporaryDirectory(t); + const executable = path.join(root, 'provider.exe'); + const source = path.join(root, 'source.rvt'); + await fs.writeFile(executable, 'fixture-provider-binary'); + await fs.writeFile(source, 'fixture-rvt'); + const calls = []; + const result = await describeAndConvert({ + executable, sourcePath: source, expectedSourceSha256: sha256(Buffer.from('fixture-rvt')), + privateRoot: path.join(root, 'private'), hostRun: fixtureHostRun(calls), + }); + assert.equal(calls.length, 2); + assert.deepEqual(calls.map((call) => call.operation), ['describe', 'convert']); + assert.equal(calls[0].cwd.startsWith(path.join(root, 'private')), true); + assert.equal(calls[0].environment.PATH, undefined); + assert.equal(result.describe.provider, 'fixture-provider'); + assert.equal(result.receipt.sourceSha256, sha256(Buffer.from('fixture-rvt'))); + assert.equal(result.outputs.geometry.path.endsWith('geometry.glb'), true); + assert.equal(result.outputs.metadata.path.endsWith('metadata.json'), true); + assert.equal(result.outputs.geometry.sha256, sha256(result.outputs.geometry.bytes)); + assert.equal(result.outputs.metadata.sha256, sha256(result.outputs.metadata.bytes)); +}); + +test('malformed receipts, provenance drift, extra files, non-zero exits, and provider text stay bounded and redacted', async (t) => { + const root = await temporaryDirectory(t); + const executable = path.join(root, 'provider.exe'); + const source = path.join(root, 'secret-residential.rvt'); + await fs.writeFile(executable, 'fixture-provider-binary'); + await fs.writeFile(source, 'fixture-rvt'); + const base = { + executable, sourcePath: source, expectedSourceSha256: sha256(Buffer.from('fixture-rvt')), + privateRoot: path.join(root, 'private'), + }; + let call = 0; + await assert.rejects(() => describeAndConvert({ ...base, hostRun: async () => { + call += 1; + if (call === 1) return { exitCode: 0, stdout: Buffer.from('{"protocolVersion":"1","provider":"p","engine":"e","engineVersion":"1","adapterBuildId":"b","formats":["rvt"],"execution":"local","destination":null}'), stderr: Buffer.alloc(0) }; + return { exitCode: 7, stdout: Buffer.alloc(0), stderr: Buffer.from(`${source} TOKEN=secret`) }; + } }), (error) => { + assert.equal(error.code, 'reference-provider-failed'); + assert.equal(error.message.includes('Residential'), false); + assert.equal(error.message.includes('secret'), false); + return true; + }); +}); + +test('provider executable mutation at a provenance bracket and undeclared output files are refused', async (t) => { + const root = await temporaryDirectory(t); + const executable = path.join(root, 'provider.exe'); + const source = path.join(root, 'source.rvt'); + await fs.writeFile(executable, 'fixture-provider-binary'); + await fs.writeFile(source, 'fixture-rvt'); + const base = { + executable, sourcePath: source, expectedSourceSha256: sha256(Buffer.from('fixture-rvt')), + }; + const mutationCalls = []; + const mutateHost = fixtureHostRun(mutationCalls); + await assert.rejects(() => describeAndConvert({ + ...base, privateRoot: path.join(root, 'mutation-private'), hostRun: async (request) => { + const result = await mutateHost(request); + await fs.writeFile(executable, 'changed-provider-binary'); + return result; + }, + }), /executable changed/); + + await fs.writeFile(executable, 'fixture-provider-binary'); + const extraCalls = []; + const extraHost = fixtureHostRun(extraCalls); + await assert.rejects(() => describeAndConvert({ + ...base, privateRoot: path.join(root, 'extra-private'), hostRun: async (request) => { + const result = await extraHost(request); + if (request.operation === 'convert') await fs.writeFile(path.join(request.cwd, 'undeclared.txt'), 'no'); + return result; + }, + }), /output directory was not closed/); +}); diff --git a/cli-connection-reader/test-fixtures/model-provider-fixture.mjs b/cli-connection-reader/test-fixtures/model-provider-fixture.mjs new file mode 100644 index 000000000..d4845b0c8 --- /dev/null +++ b/cli-connection-reader/test-fixtures/model-provider-fixture.mjs @@ -0,0 +1,31 @@ +import fs from 'node:fs/promises'; +import path from 'node:path'; +import { makeGlbFixture, makeMetadataFixture } from '../model-fixtures.mjs'; + +const operation = process.argv[2]; +const request = JSON.parse(await new Promise((resolve, reject) => { + const chunks = []; + process.stdin.on('data', (chunk) => chunks.push(chunk)); + process.stdin.on('end', () => resolve(Buffer.concat(chunks).toString('utf8'))); + process.stdin.on('error', reject); +})); +const provenance = { + protocolVersion: '1', provider: 'fixture-provider', engine: 'fixture-engine', + engineVersion: '1.2.3', adapterBuildId: 'fixture-build', formats: ['rvt'], + execution: 'local', destination: null, +}; +if (operation === 'describe') { + process.stdout.write(JSON.stringify(provenance)); +} else if (operation === 'convert') { + const geometryPath = path.join(request.outputDirectory, 'geometry.glb'); + const metadataPath = path.join(request.outputDirectory, 'metadata.json'); + await fs.writeFile(geometryPath, makeGlbFixture()); + await fs.writeFile(metadataPath, JSON.stringify(makeMetadataFixture())); + process.stdout.write(JSON.stringify({ + ...provenance, documentKind: 'revit-project', sourceSha256: request.sourceSha256, + geometryPath, metadataPath, + })); +} else { + process.stderr.write('unsupported fixture operation'); + process.exitCode = 2; +} From 2e6f0da5c6aa2f882c6cf22d49877f3621c08ae4 Mon Sep 17 00:00:00 2001 From: Pawel Date: Sun, 23 Aug 2026 14:16:27 +0200 Subject: [PATCH 06/50] feat: publish crash-safe authenticated model conversions --- cli-connection-reader/model-cache.mjs | 205 +++++++++++++++++++++ cli-connection-reader/model-cache.test.mjs | 113 ++++++++++++ 2 files changed, 318 insertions(+) create mode 100644 cli-connection-reader/model-cache.mjs create mode 100644 cli-connection-reader/model-cache.test.mjs diff --git a/cli-connection-reader/model-cache.mjs b/cli-connection-reader/model-cache.mjs new file mode 100644 index 000000000..fdfc93e67 --- /dev/null +++ b/cli-connection-reader/model-cache.mjs @@ -0,0 +1,205 @@ +import { createPrivateKey, createPublicKey, randomUUID, sign, verify } from 'node:crypto'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import { + assertClosedObject, assertSha256, canonicalJsonBytes, ModelReaderError, parseJsonStrict, sha256, +} from './model-contract.mjs'; + +const SECRET_PREFIX = Buffer.from('302e020100300506032b657004220420', 'hex'); +const PUBLIC_PREFIX = Buffer.from('302a300506032b6570032100', 'hex'); +const REQUIRED_ARTIFACTS = ['geometry.glb', 'entities.json', 'properties.json', 'relationships.json']; +const ENTRY_FILES = ['receipt.json', 'receipt.sig']; + +function cacheError(code, message, details = undefined) { + throw new ModelReaderError(code, 'cache', false, message, details); +} + +function parseKeyFile(text, marker, size, label) { + const parts = text.trim().split(/\s+/); + if (parts.length !== 2 || parts[0] !== marker) cacheError('reference-signing-key-invalid', `${label} key is not in AWARE format.`); + let bytes; + try { bytes = Buffer.from(parts[1], 'base64'); } + catch (error) { cacheError('reference-signing-key-invalid', `${label} key is invalid.`, error); } + if (bytes.length !== size || bytes.toString('base64') !== parts[1]) cacheError('reference-signing-key-invalid', `${label} key is invalid.`); + return bytes; +} + +export async function loadAwareSigningKey(secretPath, publicPath) { + let secretText; let publicText; + try { [secretText, publicText] = await Promise.all([fs.readFile(secretPath, 'utf8'), fs.readFile(publicPath, 'utf8')]); } + catch (error) { cacheError('reference-signing-key-missing', 'The model-reader signing key is unavailable.', error); } + const secretKeyBytes = parseKeyFile(secretText, 'ed25519-secret-key-v1', 32, 'secret'); + const publicKeyBytes = parseKeyFile(publicText, 'ed25519-public-key-v1', 32, 'public'); + const privateKey = createPrivateKey({ key: Buffer.concat([SECRET_PREFIX, secretKeyBytes]), format: 'der', type: 'pkcs8' }); + const derived = createPublicKey(privateKey).export({ format: 'der', type: 'spki' }).subarray(-32); + if (!derived.equals(publicKeyBytes)) cacheError('reference-signing-key-invalid', 'The model-reader keypair does not match.'); + return { privateKey, publicKeyBytes }; +} + +export function signerFingerprintSha256(publicKeyBytes) { + if (!Buffer.isBuffer(publicKeyBytes) || publicKeyBytes.length !== 32) throw new TypeError('public key must be 32 bytes'); + return sha256(publicKeyBytes); +} + +function validateIdentity(identity) { + try { assertClosedObject(identity, ['sourceSha256', 'canonicalRequest', 'providerFingerprint', 'signerFingerprintSha256'], [], 'cache identity'); } + catch (error) { cacheError('reference-cache-identity-invalid', 'Cache identity is not closed.', error); } + assertSha256(identity.sourceSha256, 'sourceSha256'); + assertSha256(identity.signerFingerprintSha256, 'signerFingerprintSha256'); + return identity; +} + +export function cacheKeySha256(identity) { + return sha256(canonicalJsonBytes(validateIdentity(identity))); +} + +async function ensureLayout(root) { + await fs.mkdir(path.join(root, 'blobs'), { recursive: true, mode: 0o700 }); + await fs.mkdir(path.join(root, 'entries'), { recursive: true, mode: 0o700 }); + await fs.mkdir(path.join(root, 'staging'), { recursive: true, mode: 0o700 }); + await fs.mkdir(path.join(root, 'locks'), { recursive: true, mode: 0o700 }); +} + +async function writeBlob(root, digest, bytes) { + const target = path.join(root, 'blobs', digest); + try { + const handle = await fs.open(target, 'wx', 0o600); + try { await handle.writeFile(bytes); await handle.sync(); } + finally { await handle.close(); } + } catch (error) { + if (error?.code !== 'EEXIST') throw error; + } + const stored = await fs.readFile(target); + if (sha256(stored) !== digest) cacheError('reference-cache-tampered', 'A content-addressed cache blob has an invalid digest.'); + return target; +} + +function signatureText(receiptBytes, signingKey) { + const signature = sign(null, Buffer.from(sha256(receiptBytes), 'hex'), signingKey.privateKey).toString('base64'); + return Buffer.from(`ed25519-signature-v1\nover-sha256-of: receipt.json\nsignature: ${signature}\npublic-key: ${signingKey.publicKeyBytes.toString('base64')}\n`); +} + +function verifySignature(receiptBytes, signatureBytes, expectedPublicKey) { + const text = signatureBytes.toString('utf8'); + const lines = new Map(text.trim().split(/\r?\n/).slice(1).map((line) => { + const split = line.indexOf(':'); return [line.slice(0, split), line.slice(split + 1).trim()]; + })); + if (!text.startsWith('ed25519-signature-v1\n') || lines.get('over-sha256-of') !== 'receipt.json') cacheError('reference-cache-signature-invalid', 'Cache receipt signature is invalid.'); + const publicBytes = Buffer.from(lines.get('public-key') ?? '', 'base64'); + if (!publicBytes.equals(expectedPublicKey)) cacheError('reference-cache-signer-mismatch', 'Cache receipt was signed by an unexpected signer.'); + const signature = Buffer.from(lines.get('signature') ?? '', 'base64'); + const publicKey = createPublicKey({ key: Buffer.concat([PUBLIC_PREFIX, publicBytes]), format: 'der', type: 'spki' }); + if (signature.length !== 64 || !verify(null, Buffer.from(sha256(receiptBytes), 'hex'), publicKey, signature)) cacheError('reference-cache-signature-invalid', 'Cache receipt signature does not verify.'); +} + +function artifactManifest(identity, artifacts) { + const names = Object.keys(artifacts).sort(); + if (names.length !== REQUIRED_ARTIFACTS.length || !REQUIRED_ARTIFACTS.every((name) => names.includes(name))) cacheError('reference-cache-artifacts-invalid', 'A conversion must publish exactly four component artifacts.'); + const records = {}; + for (const name of REQUIRED_ARTIFACTS) { + if (!Buffer.isBuffer(artifacts[name])) cacheError('reference-cache-artifacts-invalid', `Artifact ${name} is not binary-safe bytes.`); + records[name] = { sha256: sha256(artifacts[name]), bytes: artifacts[name].length }; + } + return { schemaVersion: 'model-reference-manifest/v1', identity, artifacts: records }; +} + +export async function publishCacheEntry({ root, identity, artifacts, signingKey }) { + await ensureLayout(root); + const key = cacheKeySha256(identity); + const manifest = artifactManifest(identity, artifacts); + const manifestBytes = canonicalJsonBytes(manifest); + const all = { ...artifacts, 'manifest.json': manifestBytes }; + const blobs = {}; + for (const [name, bytes] of Object.entries(all)) { + const digest = sha256(bytes); await writeBlob(root, digest, bytes); + blobs[name] = { sha256: digest, bytes: bytes.length }; + } + const receipt = { schemaVersion: 'model-reference-cache-receipt/v1', key, identitySha256: sha256(canonicalJsonBytes(identity)), blobs }; + const receiptBytes = canonicalJsonBytes(receipt); + const signatureBytes = signatureText(receiptBytes, signingKey); + const staging = path.join(root, 'staging', `${key}-${randomUUID()}`); + await fs.mkdir(staging, { mode: 0o700 }); + await fs.writeFile(path.join(staging, 'receipt.json'), receiptBytes, { mode: 0o600, flag: 'wx' }); + await fs.writeFile(path.join(staging, 'receipt.sig'), signatureBytes, { mode: 0o600, flag: 'wx' }); + const target = path.join(root, 'entries', key); + try { await fs.rename(staging, target); } + catch (error) { + if (!['EEXIST', 'ENOTEMPTY', 'EPERM'].includes(error?.code)) throw error; + await fs.rm(staging, { recursive: true, force: true }); + } + return { key, manifest, receipt }; +} + +export async function readCacheEntry({ root, key, expectedIdentity, expectedPublicKey }) { + assertSha256(key, 'cache key'); + const entry = path.join(root, 'entries', key); + let names; + try { names = (await fs.readdir(entry)).sort(); } + catch (error) { cacheError('reference-cache-miss', 'The model conversion is not cached.', error); } + if (JSON.stringify(names) !== JSON.stringify(ENTRY_FILES)) cacheError('reference-cache-entry-open', 'Cache entry is not closed; missing or extra files were found.'); + const [receiptBytes, signatureBytes] = await Promise.all([fs.readFile(path.join(entry, 'receipt.json')), fs.readFile(path.join(entry, 'receipt.sig'))]); + verifySignature(receiptBytes, signatureBytes, expectedPublicKey); + let receipt; + try { receipt = parseJsonStrict(receiptBytes); assertClosedObject(receipt, ['schemaVersion', 'key', 'identitySha256', 'blobs'], [], 'cache receipt'); } + catch (error) { cacheError('reference-cache-entry-invalid', 'Cache receipt is invalid.', error); } + if (receipt.schemaVersion !== 'model-reference-cache-receipt/v1' || receipt.key !== key) cacheError('reference-cache-entry-invalid', 'Cache receipt identity is invalid.'); + const identityBytes = canonicalJsonBytes(validateIdentity(expectedIdentity)); + if (receipt.identitySha256 !== sha256(identityBytes) || key !== cacheKeySha256(expectedIdentity)) cacheError('reference-cache-identity-mismatch', 'Cache identity does not match the request.'); + const blobNames = Object.keys(receipt.blobs).sort(); + const expectedNames = [...REQUIRED_ARTIFACTS, 'manifest.json'].sort(); + if (JSON.stringify(blobNames) !== JSON.stringify(expectedNames)) cacheError('reference-cache-entry-open', 'Cache receipt has missing or extra artifacts.'); + const artifacts = {}; + for (const name of expectedNames) { + const record = receipt.blobs[name]; + try { assertClosedObject(record, ['sha256', 'bytes'], [], `cache artifact ${name}`); assertSha256(record.sha256); } + catch (error) { cacheError('reference-cache-entry-invalid', 'Cache artifact receipt is invalid.', error); } + const bytes = await fs.readFile(path.join(root, 'blobs', record.sha256)); + if (bytes.length !== record.bytes || sha256(bytes) !== record.sha256) cacheError('reference-cache-tampered', 'Cache artifact digest is invalid or tampered.'); + artifacts[name] = bytes; + } + let manifest; + try { manifest = parseJsonStrict(artifacts['manifest.json']); } + catch (error) { cacheError('reference-cache-entry-invalid', 'Cached manifest is invalid.', error); } + if (!canonicalJsonBytes(manifest.identity).equals(identityBytes)) cacheError('reference-cache-identity-mismatch', 'Cached manifest identity does not match the request.'); + for (const name of REQUIRED_ARTIFACTS) { + if (manifest.artifacts?.[name]?.sha256 !== receipt.blobs[name].sha256 || manifest.artifacts?.[name]?.bytes !== receipt.blobs[name].bytes) cacheError('reference-cache-entry-invalid', 'Cached manifest does not reconcile with its blobs.'); + } + return { manifest, receipt, artifacts }; +} + +function defaultProcessIdentity() { return { pid: process.pid, start: process.uptime().toFixed(6) }; } +async function defaultIsAlive(owner) { + try { process.kill(owner.pid, 0); return true; } catch { return false; } +} + +export async function acquireCacheOwner({ root, key, processIdentity = defaultProcessIdentity(), isOwnerAlive = defaultIsAlive, staleMs = 30_000, now = () => Date.now() }) { + assertSha256(key, 'cache key'); await ensureLayout(root); + const lockDirectory = path.join(root, 'locks', key); + const ownerPath = path.join(lockDirectory, 'owner.json'); + const create = async () => { + await fs.mkdir(lockDirectory, { mode: 0o700 }); + const lease = { token: randomUUID(), pid: processIdentity.pid, processStart: processIdentity.start, heartbeatAt: new Date(now()).toISOString() }; + await fs.writeFile(ownerPath, canonicalJsonBytes(lease), { mode: 0o600, flag: 'wx' }); + return { root, key, lockDirectory, ownerPath, token: lease.token }; + }; + try { return await create(); } + catch (error) { + if (error?.code !== 'EEXIST') throw error; + } + let owner; + try { owner = parseJsonStrict(await fs.readFile(ownerPath)); } + catch (error) { cacheError('reference-cache-owned', 'The cache key has an unverifiable owner.', error); } + const age = now() - Date.parse(owner.heartbeatAt); + if (!Number.isFinite(age) || age <= staleMs || await isOwnerAlive(owner)) cacheError('reference-cache-owned', 'The cache key is owned by another live conversion.'); + await fs.rm(lockDirectory, { recursive: true, force: true }); + try { return await create(); } + catch (error) { cacheError('reference-cache-owned', 'Another conversion won cache ownership.', error); } +} + +export async function releaseCacheOwner(lease) { + let owner; + try { owner = parseJsonStrict(await fs.readFile(lease.ownerPath)); } + catch (error) { cacheError('reference-cache-owner-token', 'Cache ownership cannot be verified.', error); } + if (owner.token !== lease.token) cacheError('reference-cache-owner-token', 'Cache ownership token no longer matches.'); + await fs.rm(lease.lockDirectory, { recursive: true, force: true }); +} diff --git a/cli-connection-reader/model-cache.test.mjs b/cli-connection-reader/model-cache.test.mjs new file mode 100644 index 000000000..8f208f518 --- /dev/null +++ b/cli-connection-reader/model-cache.test.mjs @@ -0,0 +1,113 @@ +import assert from 'node:assert/strict'; +import { generateKeyPairSync } from 'node:crypto'; +import fs from 'node:fs/promises'; +import os from 'node:os'; +import path from 'node:path'; +import test from 'node:test'; +import { canonicalJsonBytes, sha256 } from './model-contract.mjs'; +import { + acquireCacheOwner, cacheKeySha256, loadAwareSigningKey, publishCacheEntry, + readCacheEntry, releaseCacheOwner, signerFingerprintSha256, +} from './model-cache.mjs'; + +async function temporaryDirectory(t) { + const directory = await fs.mkdtemp(path.join(os.tmpdir(), 'aware-model-cache-')); + t.after(() => fs.rm(directory, { recursive: true, force: true })); + return directory; +} + +async function signingFixture(root) { + const { privateKey, publicKey } = generateKeyPairSync('ed25519'); + const privateDer = privateKey.export({ format: 'der', type: 'pkcs8' }); + const publicDer = publicKey.export({ format: 'der', type: 'spki' }); + const secretPath = path.join(root, 'model-reader.sec'); + const publicPath = path.join(root, 'model-reader.pub'); + await fs.writeFile(secretPath, `ed25519-secret-key-v1 ${privateDer.subarray(-32).toString('base64')}\n`); + await fs.writeFile(publicPath, `ed25519-public-key-v1 ${publicDer.subarray(-32).toString('base64')}\n`); + return { secretPath, publicPath, key: await loadAwareSigningKey(secretPath, publicPath) }; +} + +function identity(overrides = {}) { + return { + sourceSha256: '1'.repeat(64), canonicalRequest: { schemaVersion: '1', value: 'request' }, + providerFingerprint: { protocolVersion: '1', provider: 'fixture', engine: 'engine', engineVersion: '1', adapterBuildId: 'b', adapterExecutableSha256: '2'.repeat(64), readerSchemaVersion: 'model-reference-reader/v1' }, + signerFingerprintSha256: '3'.repeat(64), ...overrides, + }; +} + +function artifacts() { + return { + 'geometry.glb': Buffer.from([0x67, 0x6c, 0x54, 0x46]), + 'entities.json': canonicalJsonBytes({ schemaVersion: '1', entities: [] }), + 'properties.json': canonicalJsonBytes({ schemaVersion: '1', properties: [] }), + 'relationships.json': canonicalJsonBytes({ schemaVersion: '1', relationships: [] }), + }; +} + +test('cache identity changes for source, request, every provider fingerprint leaf, and signer trust', () => { + const base = identity(); + const first = cacheKeySha256(base); + for (const [field, changed] of [ + ['sourceSha256', '4'.repeat(64)], ['signerFingerprintSha256', '5'.repeat(64)], + ['canonicalRequest', { schemaVersion: '1', value: 'changed' }], + ]) assert.notEqual(cacheKeySha256({ ...base, [field]: changed }), first); + for (const field of Object.keys(base.providerFingerprint)) { + const value = field === 'adapterExecutableSha256' ? '6'.repeat(64) : `${base.providerFingerprint[field]}-changed`; + assert.notEqual(cacheKeySha256({ ...base, providerFingerprint: { ...base.providerFingerprint, [field]: value } }), first, field); + } +}); + +test('AWARE-format Ed25519 keys sign complete entries and every hit verifies signatures and bytes', async (t) => { + const root = await temporaryDirectory(t); + const { key } = await signingFixture(root); + const cacheRoot = path.join(root, 'cache'); + const cacheIdentity = identity({ signerFingerprintSha256: signerFingerprintSha256(key.publicKeyBytes) }); + const published = await publishCacheEntry({ root: cacheRoot, identity: cacheIdentity, artifacts: artifacts(), signingKey: key }); + const hit = await readCacheEntry({ root: cacheRoot, key: published.key, expectedIdentity: cacheIdentity, expectedPublicKey: key.publicKeyBytes }); + assert.deepEqual(Object.keys(hit.artifacts).sort(), ['entities.json', 'geometry.glb', 'manifest.json', 'properties.json', 'relationships.json']); + assert.equal(hit.manifest.artifacts['geometry.glb'].sha256, sha256(artifacts()['geometry.glb'])); + const geometryBlob = path.join(cacheRoot, 'blobs', hit.manifest.artifacts['geometry.glb'].sha256); + await fs.writeFile(geometryBlob, Buffer.from([1, 2, 3, 4])); + await assert.rejects(() => readCacheEntry({ root: cacheRoot, key: published.key, expectedIdentity: cacheIdentity, expectedPublicKey: key.publicKeyBytes }), /tampered|digest|invalid/); +}); + +test('missing, extra, wrong signer, and identity-mismatched cache entries are never hits', async (t) => { + const root = await temporaryDirectory(t); + const firstKey = (await signingFixture(path.join(root, 'first').replace(/first$/, ''))).key; + const otherRoot = path.join(root, 'other'); await fs.mkdir(otherRoot); + const secondKey = (await signingFixture(otherRoot)).key; + const cacheRoot = path.join(root, 'cache'); + const cacheIdentity = identity({ signerFingerprintSha256: signerFingerprintSha256(firstKey.publicKeyBytes) }); + const published = await publishCacheEntry({ root: cacheRoot, identity: cacheIdentity, artifacts: artifacts(), signingKey: firstKey }); + await assert.rejects(() => readCacheEntry({ root: cacheRoot, key: published.key, expectedIdentity: identity(), expectedPublicKey: firstKey.publicKeyBytes }), /identity/); + await assert.rejects(() => readCacheEntry({ root: cacheRoot, key: published.key, expectedIdentity: cacheIdentity, expectedPublicKey: secondKey.publicKeyBytes }), /signer/); + await fs.writeFile(path.join(cacheRoot, 'entries', published.key, 'extra'), 'no'); + await assert.rejects(() => readCacheEntry({ root: cacheRoot, key: published.key, expectedIdentity: cacheIdentity, expectedPublicKey: firstKey.publicKeyBytes }), /extra|closed/); +}); + +test('owner leases serialize same-key publishers and stale/dead owners are fenced by token', async (t) => { + const root = await temporaryDirectory(t); + const first = await acquireCacheOwner({ root, key: 'a'.repeat(64), processIdentity: { pid: 101, start: 'one' } }); + await assert.rejects(() => acquireCacheOwner({ root, key: 'a'.repeat(64), processIdentity: { pid: 102, start: 'two' }, isOwnerAlive: async () => true }), /owned/); + const lockPath = path.join(root, 'locks', 'a'.repeat(64), 'owner.json'); + const stale = JSON.parse(await fs.readFile(lockPath, 'utf8')); + stale.heartbeatAt = '2000-01-01T00:00:00.000Z'; + await fs.writeFile(lockPath, JSON.stringify(stale)); + const second = await acquireCacheOwner({ root, key: 'a'.repeat(64), processIdentity: { pid: 102, start: 'two' }, isOwnerAlive: async () => false }); + await assert.rejects(() => releaseCacheOwner(first), /token/); + await releaseCacheOwner(second); +}); + +test('concurrent publication converges on one complete winner with identical deterministic bytes', async (t) => { + const root = await temporaryDirectory(t); + const { key } = await signingFixture(root); + const cacheRoot = path.join(root, 'cache'); + const cacheIdentity = identity({ signerFingerprintSha256: signerFingerprintSha256(key.publicKeyBytes) }); + const [a, b] = await Promise.all([ + publishCacheEntry({ root: cacheRoot, identity: cacheIdentity, artifacts: artifacts(), signingKey: key }), + publishCacheEntry({ root: cacheRoot, identity: cacheIdentity, artifacts: artifacts(), signingKey: key }), + ]); + assert.equal(a.key, b.key); + const hit = await readCacheEntry({ root: cacheRoot, key: a.key, expectedIdentity: cacheIdentity, expectedPublicKey: key.publicKeyBytes }); + assert.deepEqual(hit.artifacts['geometry.glb'], artifacts()['geometry.glb']); +}); From 48ebace04001418e8f673d025c887b9918a12455 Mon Sep 17 00:00:00 2001 From: Pawel Date: Sun, 23 Aug 2026 14:58:31 +0200 Subject: [PATCH 07/50] fix: fence managed model reader lifecycles --- cli/Cargo.lock | 33 ++ cli/Cargo.toml | 1 + cli/src/commands/app.rs | 14 + cli/src/commands/mod.rs | 1 + cli/src/commands/model_reader_host.rs | 419 ++++++++++++++++++++++++++ cli/src/error.rs | 13 +- cli/src/main.rs | 5 + cli/src/runtime/invoker.rs | 77 +++++ cli/src/runtime/pidfile.rs | 40 +++ 9 files changed, 602 insertions(+), 1 deletion(-) create mode 100644 cli/src/commands/model_reader_host.rs diff --git a/cli/Cargo.lock b/cli/Cargo.lock index 21d02b6cb..0041739a2 100644 --- a/cli/Cargo.lock +++ b/cli/Cargo.lock @@ -286,6 +286,7 @@ dependencies = [ "dirs", "ed25519-dalek", "flate2", + "fs2", "jsonschema", "keyring", "lol_html", @@ -1090,6 +1091,16 @@ dependencies = [ "num", ] +[[package]] +name = "fs2" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9564fc758e15025b46aa6643b1b77d047d1a56a1aea6e01002ac0c7026876213" +dependencies = [ + "libc", + "winapi", +] + [[package]] name = "futures-core" version = "0.3.32" @@ -3323,6 +3334,22 @@ dependencies = [ "rustls-pki-types", ] +[[package]] +name = "winapi" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419" +dependencies = [ + "winapi-i686-pc-windows-gnu", + "winapi-x86_64-pc-windows-gnu", +] + +[[package]] +name = "winapi-i686-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" + [[package]] name = "winapi-util" version = "0.1.11" @@ -3332,6 +3359,12 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "winapi-x86_64-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" + [[package]] name = "windows-core" version = "0.62.2" diff --git a/cli/Cargo.toml b/cli/Cargo.toml index 30dfe0403..7f6585e95 100644 --- a/cli/Cargo.toml +++ b/cli/Cargo.toml @@ -55,6 +55,7 @@ sha2 = "0.10" ed25519-dalek = { version = "2", features = ["rand_core"] } rand_core = "0.6" jsonschema = { version = "0.49.2", default-features = false } +fs2 = "0.4" [dev-dependencies] assert_cmd = "2.0" diff --git a/cli/src/commands/app.rs b/cli/src/commands/app.rs index c48ad8609..797da3b56 100644 --- a/cli/src/commands/app.rs +++ b/cli/src/commands/app.rs @@ -443,6 +443,20 @@ async fn run( } // One-shot path. + // The commercial RVT provider is intentionally single-control per app instance. Scope the + // kernel-backed fence to the exact agent id: other one-shot apps, including IFC reads through + // the shared bridge binary, retain their historical concurrency. + let _model_reader_control = if app + .nodes + .iter() + .any(|node| node.agent.as_deref() == Some("model-reference-reader")) + { + Some(crate::runtime::pidfile::ExclusiveControl::acquire( + &ctx.paths.app_instance_dir(app_id, &instance), + )?) + } else { + None + }; let log_path = log_path_for(&ctx.paths.logs_dir(), app_id, &instance, &run_id); let provenance = ProvenanceWriter::open(&log_path).await?; let artifact_dir = crate::runtime::provenance::artifact_dir_for( diff --git a/cli/src/commands/mod.rs b/cli/src/commands/mod.rs index 1c38ba83f..d7bf68554 100644 --- a/cli/src/commands/mod.rs +++ b/cli/src/commands/mod.rs @@ -12,6 +12,7 @@ pub mod coverage; pub mod diagram; pub mod doctor; pub mod key; +pub mod model_reader_host; pub mod plugins; pub mod receipt_cli; pub mod report; diff --git a/cli/src/commands/model_reader_host.rs b/cli/src/commands/model_reader_host.rs new file mode 100644 index 000000000..64179bbc3 --- /dev/null +++ b/cli/src/commands/model_reader_host.rs @@ -0,0 +1,419 @@ +//! Internal managed provider host for `model-reference-reader`. +//! +//! The bridge never launches a commercial provider directly. This hidden command owns provider +//! processes, drains both output pipes concurrently, and keeps cancellation live over a bounded, +//! request-correlated binary protocol. It is not a public agent surface. + +use std::collections::{BTreeMap, HashMap}; +use std::path::PathBuf; +use std::process::Stdio; +use std::sync::Arc; + +use fs2::FileExt; +use rand::RngCore; +use serde::Deserialize; +use serde_json::json; +use tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt}; +use tokio::sync::{Mutex, oneshot}; + +use crate::error::AwareError; + +const HEADER_BYTES: usize = 50; +const MAX_CONTROL_BYTES: usize = 1024 * 1024; +const KIND_CONTROL: u8 = 0x01; +const KIND_STDOUT: u8 = 0x02; +const KIND_STDERR: u8 = 0x03; +const KIND_STDIN: u8 = 0x04; +const FINAL: u8 = 0x01; + +#[derive(Clone, Debug, PartialEq, Eq)] +struct Frame { + kind: u8, + request_id: u64, + run_handle: [u8; 32], + sequence: u32, + flags: u8, + payload: Vec, +} + +impl Frame { + async fn read(reader: &mut R) -> std::io::Result> { + let mut header = [0u8; HEADER_BYTES]; + match reader.read_exact(&mut header).await { + Ok(_) => {} + Err(error) if error.kind() == std::io::ErrorKind::UnexpectedEof => return Ok(None), + Err(error) => return Err(error), + } + let kind = header[0]; + if !matches!(kind, KIND_CONTROL | KIND_STDOUT | KIND_STDERR | KIND_STDIN) { + return Err(std::io::Error::new( + std::io::ErrorKind::InvalidData, + "unknown frame kind", + )); + } + let mut request_id_bytes = [0u8; 8]; + request_id_bytes.copy_from_slice(&header[1..9]); + let request_id = u64::from_be_bytes(request_id_bytes); + let mut run_handle = [0u8; 32]; + run_handle.copy_from_slice(&header[9..41]); + let mut sequence_bytes = [0u8; 4]; + sequence_bytes.copy_from_slice(&header[41..45]); + let sequence = u32::from_be_bytes(sequence_bytes); + let flags = header[45]; + let mut length_bytes = [0u8; 4]; + length_bytes.copy_from_slice(&header[46..50]); + let length = u32::from_be_bytes(length_bytes) as usize; + if length > MAX_CONTROL_BYTES { + return Err(std::io::Error::new( + std::io::ErrorKind::InvalidData, + "frame payload exceeds limit", + )); + } + let mut payload = vec![0u8; length]; + reader.read_exact(&mut payload).await?; + Ok(Some(Self { + kind, + request_id, + run_handle, + sequence, + flags, + payload, + })) + } + + async fn write(&self, writer: &mut W) -> std::io::Result<()> { + let length: u32 = self.payload.len().try_into().map_err(|_| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "frame too large") + })?; + let mut header = [0u8; HEADER_BYTES]; + header[0] = self.kind; + header[1..9].copy_from_slice(&self.request_id.to_be_bytes()); + header[9..41].copy_from_slice(&self.run_handle); + header[41..45].copy_from_slice(&self.sequence.to_be_bytes()); + header[45] = self.flags; + header[46..50].copy_from_slice(&length.to_be_bytes()); + writer.write_all(&header).await?; + writer.write_all(&self.payload).await?; + writer.flush().await + } +} + +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct ProviderRun { + op: String, + executable: PathBuf, + operation: String, + cwd: PathBuf, + environment: BTreeMap, + stdin_length: usize, + timeout_ms: u64, + stdout_limit: usize, + stderr_limit: usize, +} + +struct PendingRun { + request: ProviderRun, + handle: [u8; 32], +} + +type SharedWriter = Arc>; +type ActiveRuns = Arc>>>; + +async fn send_control( + writer: &SharedWriter, + request_id: u64, + handle: [u8; 32], + body: serde_json::Value, +) { + let frame = Frame { + kind: KIND_CONTROL, + request_id, + run_handle: handle, + sequence: 0, + flags: FINAL, + payload: serde_json::to_vec(&body).unwrap_or_default(), + }; + let _ = frame.write(&mut *writer.lock().await).await; +} + +async fn read_bounded( + mut reader: R, + limit: usize, +) -> std::io::Result> { + let mut bytes = Vec::new(); + let mut chunk = [0u8; 8192]; + loop { + let count = reader.read(&mut chunk).await?; + if count == 0 { + return Ok(bytes); + } + if bytes.len().saturating_add(count) > limit { + return Err(std::io::Error::new( + std::io::ErrorKind::FileTooLarge, + "provider stream exceeds limit", + )); + } + bytes.extend_from_slice(&chunk[..count]); + } +} + +fn provider_command(request: &ProviderRun) -> Result { + if !request.executable.is_absolute() + || !request.cwd.is_absolute() + || !matches!(request.operation.as_str(), "describe" | "convert") + { + return Err(AwareError::Validation( + "model-reader host received an unsafe provider request".into(), + )); + } + let mut command = tokio::process::Command::new(&request.executable); + command + .arg(&request.operation) + .arg("--json-stdin") + .current_dir(&request.cwd) + .env_clear() + .envs(&request.environment) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .kill_on_drop(true); + #[cfg(unix)] + { + use std::os::unix::process::CommandExt; + command.as_std_mut().process_group(0); + } + Ok(command) +} + +async fn kill_tree(child: &mut tokio::process::Child) { + #[cfg(windows)] + if let Some(pid) = child.id() { + let _ = tokio::process::Command::new("taskkill") + .args(["/PID", &pid.to_string(), "/T", "/F"]) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .status() + .await; + } + #[cfg(unix)] + if let Some(pid) = child.id() { + let _ = tokio::process::Command::new("kill") + .args(["-TERM", &format!("-{pid}")]) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .status() + .await; + } + let _ = child.start_kill(); + let _ = child.wait().await; +} + +async fn execute_provider( + request_id: u64, + handle: [u8; 32], + request: ProviderRun, + stdin_bytes: Vec, + writer: SharedWriter, + active: ActiveRuns, + cancel_rx: oneshot::Receiver<()>, +) { + let mut child = match provider_command(&request) + .and_then(|mut command| command.spawn().map_err(AwareError::Io)) + { + Ok(child) => child, + Err(error) => { + send_control( + &writer, + request_id, + handle, + json!({"status":"complete","exitCode":127,"hostError":error.to_string()}), + ) + .await; + active.lock().await.remove(&handle); + return; + } + }; + let mut stdin = child.stdin.take(); + let stdout = child.stdout.take(); + let stderr = child.stderr.take(); + let input = tokio::spawn(async move { + if let Some(mut stream) = stdin.take() { + stream.write_all(&stdin_bytes).await?; + stream.shutdown().await?; + } + Ok::<(), std::io::Error>(()) + }); + let (Some(stdout), Some(stderr)) = (stdout, stderr) else { + kill_tree(&mut child).await; + send_control( + &writer, + request_id, + handle, + json!({"status":"complete","exitCode":127,"hostError":"provider pipes unavailable"}), + ) + .await; + active.lock().await.remove(&handle); + return; + }; + let stdout_task = tokio::spawn(read_bounded(stdout, request.stdout_limit)); + let stderr_task = tokio::spawn(read_bounded(stderr, request.stderr_limit)); + let mut cancel_rx = cancel_rx; + let outcome = tokio::select! { + status = child.wait() => status.map(|status| (status.code().unwrap_or(1), None)), + _ = tokio::time::sleep(std::time::Duration::from_millis(request.timeout_ms)) => { kill_tree(&mut child).await; Ok((124, Some("timeout"))) }, + _ = &mut cancel_rx => { kill_tree(&mut child).await; Ok((130, Some("cancelled"))) }, + }; + let _ = input.await; + let stdout = stdout_task + .await + .ok() + .and_then(Result::ok) + .unwrap_or_default(); + let stderr = stderr_task + .await + .ok() + .and_then(Result::ok) + .unwrap_or_default(); + for (kind, payload) in [(KIND_STDOUT, stdout), (KIND_STDERR, stderr)] { + let frame = Frame { + kind, + request_id, + run_handle: handle, + sequence: 0, + flags: FINAL, + payload, + }; + let _ = frame.write(&mut *writer.lock().await).await; + } + let (exit_code, reason) = outcome.unwrap_or((1, Some("host-wait-failed"))); + send_control( + &writer, + request_id, + handle, + json!({"status":"complete","exitCode":exit_code,"reason":reason}), + ) + .await; + active.lock().await.remove(&handle); +} + +pub async fn run() -> Result<(), AwareError> { + let mut reader = tokio::io::BufReader::new(tokio::io::stdin()); + let writer: SharedWriter = Arc::new(Mutex::new(tokio::io::stdout())); + let active: ActiveRuns = Arc::new(Mutex::new(HashMap::new())); + let mut pending: HashMap<(u64, [u8; 32]), PendingRun> = HashMap::new(); + let mut locks: HashMap<[u8; 32], std::fs::File> = HashMap::new(); + let mut last_control_id = 0u64; + while let Some(frame) = Frame::read(&mut reader).await? { + if frame.kind == KIND_STDIN { + let Some(pending_run) = pending.remove(&(frame.request_id, frame.run_handle)) else { + return Err(AwareError::Validation( + "model-reader host received uncorrelated stdin".into(), + )); + }; + if frame.sequence != 0 + || frame.flags & FINAL == 0 + || frame.payload.len() != pending_run.request.stdin_length + { + return Err(AwareError::Validation( + "model-reader host stdin length/sequence mismatch".into(), + )); + } + let (cancel_tx, cancel_rx) = oneshot::channel(); + active.lock().await.insert(pending_run.handle, cancel_tx); + tokio::spawn(execute_provider( + frame.request_id, + pending_run.handle, + pending_run.request, + frame.payload, + writer.clone(), + active.clone(), + cancel_rx, + )); + continue; + } + if frame.kind != KIND_CONTROL || frame.request_id <= last_control_id { + return Err(AwareError::Validation( + "model-reader host control request ids must increase".into(), + )); + } + last_control_id = frame.request_id; + let control: serde_json::Value = serde_json::from_slice(&frame.payload)?; + match control.get("op").and_then(|value| value.as_str()) { + Some("hello") => send_control(&writer, frame.request_id, [0; 32], json!({"status":"ok","protocol":"model-reader-host/v1","build":env!("CARGO_PKG_VERSION")})).await, + Some("provider-run") => { + let request: ProviderRun = serde_json::from_value(control)?; + if request.op != "provider-run" || request.stdin_length > MAX_CONTROL_BYTES { return Err(AwareError::Validation("model-reader host provider request exceeds limit".into())); } + let mut handle = [0u8; 32]; rand::thread_rng().fill_bytes(&mut handle); + pending.insert((frame.request_id, handle), PendingRun { request, handle }); + send_control(&writer, frame.request_id, handle, json!({"status":"accepted"})).await; + } + Some("provider-cancel") => { + if let Some(cancel) = active.lock().await.remove(&frame.run_handle) { let _ = cancel.send(()); } + send_control(&writer, frame.request_id, frame.run_handle, json!({"status":"cancel-requested"})).await; + } + Some("lock-acquire") => { + let lock_path = control.get("path").and_then(|value| value.as_str()).map(PathBuf::from).filter(|value| value.is_absolute()).ok_or_else(|| AwareError::Validation("model-reader host lock path must be absolute".into()))?; + let file = std::fs::OpenOptions::new().read(true).write(true).create(true).truncate(false).open(lock_path)?; + file.try_lock_exclusive().map_err(|_| AwareError::Conflict("model-reader host lock is held".into()))?; + let mut handle = [0u8; 32]; rand::thread_rng().fill_bytes(&mut handle); locks.insert(handle, file); + send_control(&writer, frame.request_id, handle, json!({"status":"acquired"})).await; + } + Some("lock-release") => { + let file = locks.remove(&frame.run_handle).ok_or_else(|| AwareError::Validation("model-reader host lock handle is unknown".into()))?; + file.unlock()?; send_control(&writer, frame.request_id, frame.run_handle, json!({"status":"released"})).await; + } + Some("shutdown") => { + let mut runs = active.lock().await; for (_, cancel) in runs.drain() { let _ = cancel.send(()); } + send_control(&writer, frame.request_id, [0; 32], json!({"status":"bye"})).await; break; + } + _ => return Err(AwareError::Validation("model-reader host control operation is unknown".into())), + } + } + let mut runs = active.lock().await; + for (_, cancel) in runs.drain() { + let _ = cancel.send(()); + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[tokio::test] + async fn frames_preserve_request_run_sequence_final_and_binary_payload() { + let frame = Frame { + kind: KIND_STDOUT, + request_id: 42, + run_handle: [7; 32], + sequence: 9, + flags: FINAL, + payload: vec![0, 255, 1, 2], + }; + let mut bytes = Vec::new(); + frame.write(&mut bytes).await.unwrap(); + let decoded = Frame::read(&mut bytes.as_slice()).await.unwrap().unwrap(); + assert_eq!(decoded, frame); + } + + #[test] + fn provider_command_has_only_protocol_argv_and_clears_ambient_environment() { + let request = ProviderRun { + op: "provider-run".into(), + executable: std::env::current_exe().unwrap(), + operation: "convert".into(), + cwd: std::env::current_dir().unwrap(), + environment: BTreeMap::from([("TZ".into(), "UTC".into())]), + stdin_length: 1, + timeout_ms: 1000, + stdout_limit: 10, + stderr_limit: 10, + }; + let command = provider_command(&request).unwrap(); + let debug = format!("{command:?}"); + assert!(debug.contains("convert")); + assert!(debug.contains("--json-stdin")); + assert!(!debug.contains("secret.rvt")); + } +} diff --git a/cli/src/error.rs b/cli/src/error.rs index 487606519..d9750c653 100644 --- a/cli/src/error.rs +++ b/cli/src/error.rs @@ -15,6 +15,17 @@ pub enum AwareError { #[error("network error: {0}")] Network(String), + #[error( + "agent error {code} ({phase}, retryable={retryable}, diagnostic-id={diagnostic_id}): {message}" + )] + AgentStructured { + code: String, + phase: String, + retryable: bool, + message: String, + diagnostic_id: String, + }, + #[error("permission denied: {0}")] PermissionDenied(String), @@ -46,7 +57,7 @@ impl AwareError { match self { Self::NotYetImplemented(_) => 1, Self::Validation(_) => 3, - Self::Network(_) => 4, + Self::Network(_) | Self::AgentStructured { .. } => 4, Self::PermissionDenied(_) => 5, Self::AuthExpired(_) => 6, Self::NotFound(_) => 7, diff --git a/cli/src/main.rs b/cli/src/main.rs index ba7dec9d2..f237cfe64 100644 --- a/cli/src/main.rs +++ b/cli/src/main.rs @@ -88,6 +88,10 @@ struct Cli { #[derive(Subcommand, Debug)] enum Command { + /// Internal managed process host for the model reference reader. + #[command(name = "__model-reader-host", hide = true)] + ModelReaderHost, + /// Manage installed agents (list, describe, install, validate, …). Agent { #[command(subcommand)] @@ -196,6 +200,7 @@ async fn main() -> anyhow::Result<()> { }; let result: Result<(), AwareError> = match cli.command { + Command::ModelReaderHost => commands::model_reader_host::run().await, Command::Agent { action } => commands::agent::dispatch(action, &ctx).await, Command::App { action } => commands::app::dispatch(action, &ctx).await, Command::Connect(args) => commands::connect::run_connect(args, &ctx), diff --git a/cli/src/runtime/invoker.rs b/cli/src/runtime/invoker.rs index 5707c1d0c..6fb706bb1 100644 --- a/cli/src/runtime/invoker.rs +++ b/cli/src/runtime/invoker.rs @@ -290,6 +290,40 @@ fn failure_detail(stdout: &str, stderr: &str) -> String { truncate_detail(detail) } +#[derive(serde::Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct StructuredBridgeError { + code: String, + phase: String, + retryable: bool, + message: String, + diagnostic_id: String, +} + +/// Preserve a bridge's bounded typed error instead of flattening it into an opaque network string. +/// Only the closed model-reader envelope is accepted; arbitrary bridge stderr keeps the historical +/// reporting path below. +fn structured_bridge_error(stderr: &str) -> Option { + let parsed: StructuredBridgeError = serde_json::from_str(stderr.trim()).ok()?; + if !parsed.code.starts_with("reference-") + || parsed.code.len() > 96 + || parsed.phase.is_empty() + || parsed.phase.len() > 64 + || parsed.message.is_empty() + || parsed.message.chars().count() > 240 + || parsed.diagnostic_id.len() > 64 + { + return None; + } + Some(AwareError::AgentStructured { + code: parsed.code, + phase: parsed.phase, + retryable: parsed.retryable, + message: parsed.message, + diagnostic_id: parsed.diagnostic_id, + }) +} + /// Production invoker: spawn the agent's CLI transport binary, /// talk JSON over stdin/stdout. pub struct CliInvoker { @@ -367,6 +401,18 @@ impl CliInvoker { if let Some(path) = progress_path { process.env("AWARE_PROGRESS_FILE", path); } + // The RVT reader is the only bridge allowed to ask AWARE to supervise a commercial + // provider. Pass the exact current executable privately; no manifest field or PATH lookup + // can redirect the internal host. IFC commands sharing the same SEA do not receive it. + if agent == "model-reference-reader" { + // `canonicalize` adds a `\\?\` prefix on Windows. Node's `realpath` removes that + // prefix, so an otherwise identical source-built host would fail the reader's exact + // path check. `current_exe` is already absolute; the reader independently rejects + // symlinks and verifies that this spelling resolves to the same regular file. + let host = std::env::current_exe() + .map_err(|e| AwareError::Internal(format!("resolve model-reader host: {e}")))?; + process.env("AWARE_MODEL_READER_HOST", host); + } let child = process.spawn().map_err(|e| { // When the binary is missing, surface an actionable hint — but only // point at `aware sidecar install` for binaries that command actually @@ -441,6 +487,9 @@ impl CliInvoker { if !output.status.success() { let stdout = String::from_utf8_lossy(&output.stdout); let stderr = String::from_utf8_lossy(&output.stderr); + if let Some(error) = structured_bridge_error(&stderr) { + return Err(error); + } return Err(AwareError::Network(format!( "agent {agent}/{command} failed (exit {:?}): {}", output.status.code(), @@ -4400,6 +4449,34 @@ mod builtin_invoker_tests { ); } + #[test] + fn model_reader_structured_errors_keep_their_typed_fields() { + let stderr = r#"{"code":"reference-provider-pin-mismatch","phase":"preflight","retryable":false,"message":"The local provider does not match the expected fingerprint.","diagnosticId":"123e4567-e89b-12d3-a456-426614174000"}"#; + let error = structured_bridge_error(stderr).expect("closed model-reader envelope"); + match error { + AwareError::AgentStructured { + code, + phase, + retryable, + message, + diagnostic_id, + } => { + assert_eq!(code, "reference-provider-pin-mismatch"); + assert_eq!(phase, "preflight"); + assert!(!retryable); + assert!(message.contains("expected fingerprint")); + assert_eq!(diagnostic_id, "123e4567-e89b-12d3-a456-426614174000"); + } + other => panic!("typed envelope was flattened: {other:?}"), + } + assert!( + structured_bridge_error( + r#"{"code":"other","phase":"x","retryable":false,"message":"x","diagnosticId":"x"}"# + ) + .is_none() + ); + } + #[test] fn a_bridge_that_reports_only_on_stdout_still_surfaces() { // stderr-first must FALL BACK, not replace: a bridge whose only output is on stdout would diff --git a/cli/src/runtime/pidfile.rs b/cli/src/runtime/pidfile.rs index f80a4fa84..8f48acd2a 100644 --- a/cli/src/runtime/pidfile.rs +++ b/cli/src/runtime/pidfile.rs @@ -6,6 +6,37 @@ use serde::{Deserialize, Serialize}; use crate::error::AwareError; +/// Kernel-backed, crash-released single-run fence used only by app graphs that contain the exact +/// `model-reference-reader` agent. Ordinary one-shot apps never acquire it. +pub struct ExclusiveControl { + file: std::fs::File, +} + +impl ExclusiveControl { + pub fn acquire(instance_dir: &Path) -> Result { + use fs2::FileExt; + std::fs::create_dir_all(instance_dir)?; + let file = std::fs::OpenOptions::new() + .read(true) + .write(true) + .create(true) + .truncate(false) + .open(instance_dir.join("model-reader-control.lock"))?; + file.try_lock_exclusive().map_err(|_| { + AwareError::Conflict( + "a model-reference-reader run already owns this app instance".into(), + ) + })?; + Ok(Self { file }) + } +} + +impl Drop for ExclusiveControl { + fn drop(&mut self) { + let _ = fs2::FileExt::unlock(&self.file); + } +} + #[derive(Debug, Serialize, Deserialize)] pub struct Pidfile { pub app: String, @@ -69,4 +100,13 @@ mod tests { remove(tmp.path()); assert!(!tmp.path().join("pidfile.yaml").exists()); } + + #[test] + fn exclusive_control_refuses_a_second_owner_and_releases_on_drop() { + let tmp = tempfile::tempdir().unwrap(); + let first = ExclusiveControl::acquire(tmp.path()).unwrap(); + assert!(ExclusiveControl::acquire(tmp.path()).is_err()); + drop(first); + ExclusiveControl::acquire(tmp.path()).unwrap(); + } } From 8b0a17e36c85f860f641227c2072ff97d8282452 Mon Sep 17 00:00:00 2001 From: Pawel Date: Sun, 23 Aug 2026 14:58:55 +0200 Subject: [PATCH 08/50] feat: expose deterministic RVT reference artifacts --- .github/workflows/ci.yml | 28 +++ cli-connection-reader/build.mjs | 2 +- cli-connection-reader/index.mjs | 15 +- cli-connection-reader/model-cache.mjs | 10 +- cli-connection-reader/model-dispatcher.mjs | 42 ++++ .../model-dispatcher.test.mjs | 45 ++++ cli-connection-reader/model-host-client.mjs | 156 +++++++++++++ .../model-host-client.test.mjs | 24 ++ cli-connection-reader/model-provider.mjs | 47 +++- cli-connection-reader/model-reader.mjs | 221 ++++++++++++++++++ cli-connection-reader/model-reader.test.mjs | 97 ++++++++ .../model-windows-harness.mjs | 146 ++++++++++++ cli-connection-reader/package.json | 5 +- .../test-fixtures/model-provider-fixture.mjs | 45 ++-- 14 files changed, 838 insertions(+), 45 deletions(-) create mode 100644 cli-connection-reader/model-dispatcher.mjs create mode 100644 cli-connection-reader/model-dispatcher.test.mjs create mode 100644 cli-connection-reader/model-host-client.mjs create mode 100644 cli-connection-reader/model-host-client.test.mjs create mode 100644 cli-connection-reader/model-reader.mjs create mode 100644 cli-connection-reader/model-reader.test.mjs create mode 100644 cli-connection-reader/model-windows-harness.mjs diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index efd98377b..879c6ede1 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -195,6 +195,34 @@ jobs: working-directory: cli-connection-reader run: node --test + bridge-windows-packaged: + name: connection-reader packaged RVT/IFC harness + runs-on: windows-latest + timeout-minutes: 15 + steps: + - uses: actions/checkout@v6 + + - uses: actions/setup-node@v4 + with: + node-version: 22 + cache: npm + cache-dependency-path: cli-connection-reader/package-lock.json + + - uses: dtolnay/rust-toolchain@stable + with: + toolchain: 1.88.0 + + - name: Build source AWARE host + run: cargo build --manifest-path cli/Cargo.toml --locked + + - name: npm ci + working-directory: cli-connection-reader + run: npm ci + + - name: Clean-stage SEA/provider harness + working-directory: cli-connection-reader + run: npm run test:windows-harness + # Every OTHER .NET bridge suite in the repo. `tekla-bridge` below has run since # it was written, and that made it look like the .NET side was covered — it was # the only one of seven. `cli-revit/Tests`, `cli-rhino/Tests`, diff --git a/cli-connection-reader/build.mjs b/cli-connection-reader/build.mjs index ecc272c3f..105de89dd 100644 --- a/cli-connection-reader/build.mjs +++ b/cli-connection-reader/build.mjs @@ -22,7 +22,7 @@ mkdirSync(dist, { recursive: true }); // 1. Bundle ESM entry + web-ifc into one CJS file (SEA embeds a single file; node_modules aren't shipped). console.log('[build] bundling with esbuild…'); await build({ - entryPoints: [join(here, 'index.mjs')], + entryPoints: [join(here, 'model-dispatcher.mjs')], bundle: true, platform: 'node', format: 'cjs', diff --git a/cli-connection-reader/index.mjs b/cli-connection-reader/index.mjs index 1d89bb734..28bbea4d2 100644 --- a/cli-connection-reader/index.mjs +++ b/cli-connection-reader/index.mjs @@ -1340,9 +1340,8 @@ export function closeApi({ api, modelID }) { api.CloseModel(modelID); } -async function main() { - const command = process.argv[2]; - const args = JSON.parse(readStdin() || '{}'); +export async function main(command = process.argv[2], stdinText = undefined) { + const args = JSON.parse((stdinText ?? readStdin()) || '{}'); // ARGUMENT VALIDATION HAPPENS BEFORE THE STDOUT GUARD GOES UP. The guard swaps `process.stdout.write` // for stderr's; a `throw` between installing it and the `finally` that restores it leaves the process @@ -1718,12 +1717,10 @@ class SegmentWriter { // Only run as a CLI when this file IS the entry point. Without this guard, importing the module from a // test executes main(), which reads fd 0 and exits non-zero. // -// The packaged case is checked FIRST and does not depend on argv[1]. Shipped as a SEA the bundle is -// CJS inside a renamed node.exe, where argv[1] is not this script and may be absent entirely — an -// argv-first guard would evaluate false and leave the bridge silently doing nothing, which is a far -// worse failure than the one the guard prevents. -const invokedDirectly = isPackagedExe() - || (!!process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href); +// SEA ownership belongs to `model-dispatcher.mjs`, the package/bin and build entrypoint. Keeping an +// `isPackagedExe()` arm here would make this lazily imported IFC module launch a second main inside +// the same executable and race the dispatcher for stdin. Plain source invocation remains supported. +const invokedDirectly = !!process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href; if (invokedDirectly) { main().catch((e) => { process.stderr.write(`connection-reader: ${e && e.message ? e.message : e}\n`); diff --git a/cli-connection-reader/model-cache.mjs b/cli-connection-reader/model-cache.mjs index fdfc93e67..6fd4b1171 100644 --- a/cli-connection-reader/model-cache.mjs +++ b/cli-connection-reader/model-cache.mjs @@ -92,7 +92,9 @@ function verifySignature(receiptBytes, signatureBytes, expectedPublicKey) { if (signature.length !== 64 || !verify(null, Buffer.from(sha256(receiptBytes), 'hex'), publicKey, signature)) cacheError('reference-cache-signature-invalid', 'Cache receipt signature does not verify.'); } -function artifactManifest(identity, artifacts) { +function artifactManifest(identity, artifacts, details = {}) { + try { assertClosedObject(details, [], ['coverage', 'frame', 'canonicalRequestSha256', 'providerFingerprintSha256'], 'manifest details'); } + catch (error) { cacheError('reference-cache-artifacts-invalid', 'Manifest details are not closed.', error); } const names = Object.keys(artifacts).sort(); if (names.length !== REQUIRED_ARTIFACTS.length || !REQUIRED_ARTIFACTS.every((name) => names.includes(name))) cacheError('reference-cache-artifacts-invalid', 'A conversion must publish exactly four component artifacts.'); const records = {}; @@ -100,13 +102,13 @@ function artifactManifest(identity, artifacts) { if (!Buffer.isBuffer(artifacts[name])) cacheError('reference-cache-artifacts-invalid', `Artifact ${name} is not binary-safe bytes.`); records[name] = { sha256: sha256(artifacts[name]), bytes: artifacts[name].length }; } - return { schemaVersion: 'model-reference-manifest/v1', identity, artifacts: records }; + return { schemaVersion: 'model-reference-manifest/v1', identity, ...details, artifacts: records }; } -export async function publishCacheEntry({ root, identity, artifacts, signingKey }) { +export async function publishCacheEntry({ root, identity, artifacts, signingKey, details = {} }) { await ensureLayout(root); const key = cacheKeySha256(identity); - const manifest = artifactManifest(identity, artifacts); + const manifest = artifactManifest(identity, artifacts, details); const manifestBytes = canonicalJsonBytes(manifest); const all = { ...artifacts, 'manifest.json': manifestBytes }; const blobs = {}; diff --git a/cli-connection-reader/model-dispatcher.mjs b/cli-connection-reader/model-dispatcher.mjs new file mode 100644 index 000000000..d63944694 --- /dev/null +++ b/cli-connection-reader/model-dispatcher.mjs @@ -0,0 +1,42 @@ +import { appendFileSync, readFileSync } from 'node:fs'; +import { isSea } from 'node:sea'; +import { pathToFileURL } from 'node:url'; +import { ModelReaderError, safeErrorEnvelope } from './model-contract.mjs'; +import { runModelCommand } from './model-reader.mjs'; + +function route(command, args) { + const hasIfc = ['ifc-path', 'ifcPath', 'base-ifc-path', 'revised-ifc-path'].some((key) => args[key] !== undefined); + const hasRvt = ['rvt-path', 'rvtPath', 'model-path'].some((key) => args[key] !== undefined); + if (hasIfc && hasRvt) throw new ModelReaderError('reference-mixed-model-paths', 'request', false, 'IFC and RVT paths cannot be mixed in one command.'); + if (command === 'preflight') return 'rvt'; + if ((command === 'probe' || command === 'read-model') && hasRvt) return 'rvt'; + return 'ifc'; +} + +export async function main(command = process.argv[2], stdinText = undefined, dependencies = undefined) { + const raw = stdinText ?? readFileSync(0, 'utf8'); + let args; + try { args = JSON.parse(raw || '{}'); } + catch (error) { throw new ModelReaderError('reference-request-invalid', 'request', false, 'Command input is not valid JSON.', error); } + if (!args || typeof args !== 'object' || Array.isArray(args)) throw new ModelReaderError('reference-request-invalid', 'request', false, 'Command input must be a JSON object.'); + if (route(command, args) === 'ifc') { + const ifc = await import('./index.mjs'); + return await ifc.main(command, raw); + } + const runtimeDependencies = dependencies ?? { + progress: process.env.AWARE_PROGRESS_FILE ? (record) => { + appendFileSync(process.env.AWARE_PROGRESS_FILE, `${JSON.stringify({ '$aware-progress': record })}\n`, { encoding: 'utf8' }); + } : undefined, + }; + const result = await runModelCommand(command, args, runtimeDependencies); + process.stdout.write(JSON.stringify(result)); +} + +const invokedDirectly = isSea() || (!!process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href); +if (invokedDirectly) { + main().catch((error) => { + if (error instanceof ModelReaderError) process.stderr.write(`${JSON.stringify(safeErrorEnvelope(error))}\n`); + else process.stderr.write(`connection-reader: ${error instanceof Error ? error.message : String(error)}\n`); + process.exitCode = 1; + }); +} diff --git a/cli-connection-reader/model-dispatcher.test.mjs b/cli-connection-reader/model-dispatcher.test.mjs new file mode 100644 index 000000000..1e8353ad2 --- /dev/null +++ b/cli-connection-reader/model-dispatcher.test.mjs @@ -0,0 +1,45 @@ +import assert from 'node:assert/strict'; +import { spawn } from 'node:child_process'; +import path from 'node:path'; +import test from 'node:test'; +import { fileURLToPath } from 'node:url'; + +const here = path.dirname(fileURLToPath(import.meta.url)); +const dispatcher = path.join(here, 'model-dispatcher.mjs'); +const ifcEntry = path.join(here, 'index.mjs'); + +function run(entry, command, input, environment = {}) { + return new Promise((resolve, reject) => { + const child = spawn(process.execPath, [entry, command, '--json-stdin'], { cwd: here, env: { ...process.env, ...environment }, windowsHide: true, shell: false, stdio: ['pipe', 'pipe', 'pipe'] }); + const stdout = []; const stderr = []; + child.stdout.on('data', (chunk) => stdout.push(chunk)); child.stderr.on('data', (chunk) => stderr.push(chunk)); + child.on('error', reject); child.on('close', (exitCode) => resolve({ exitCode, stdout: Buffer.concat(stdout), stderr: Buffer.concat(stderr) })); + child.stdin.end(JSON.stringify(input)); + }); +} + +test('RVT preflight failures are bounded structured JSON without loading IFC requirements', async () => { + const result = await run(dispatcher, 'preflight', {}, { AWARE_MODEL_REFERENCE_PROVIDER: '', AWARE_MODEL_REFERENCE_SIGNING_KEY: '' }); + assert.notEqual(result.exitCode, 0); + assert.equal(result.stdout.length, 0); + const error = JSON.parse(result.stderr.toString('utf8')); + assert.equal(error.code, 'reference-provider-unavailable'); + assert.equal(error.phase, 'preflight'); + assert.match(error.diagnosticId, /^[0-9a-f-]{36}$/); + assert.equal(result.stderr.length < 2048, true); +}); + +test('mixed IFC and RVT paths are refused instead of guessed', async () => { + const result = await run(dispatcher, 'probe', { 'ifc-path': 'a.ifc', 'rvt-path': 'b.rvt' }); + assert.notEqual(result.exitCode, 0); + assert.match(result.stderr.toString('utf8'), /mixed-model-paths/); +}); + +test('documented IFC probe bytes remain identical through the lazy dispatcher', async () => { + const input = { 'ifc-path': path.join(here, 'test-fixtures', 'baseplate-bp1.ifc') }; + const before = await run(ifcEntry, 'probe', input); + const after = await run(dispatcher, 'probe', input); + assert.equal(before.exitCode, 0); assert.equal(after.exitCode, 0); + assert.deepEqual(after.stdout, before.stdout); + assert.deepEqual(after.stderr, before.stderr); +}); diff --git a/cli-connection-reader/model-host-client.mjs b/cli-connection-reader/model-host-client.mjs new file mode 100644 index 000000000..08b6397a2 --- /dev/null +++ b/cli-connection-reader/model-host-client.mjs @@ -0,0 +1,156 @@ +import { spawn } from 'node:child_process'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import { canonicalJsonBytes, ModelReaderError } from './model-contract.mjs'; + +const HEADER_BYTES = 50; +const MAX_PAYLOAD_BYTES = 1024 * 1024; +const ZERO_HANDLE = Buffer.alloc(32); + +function hostError(code, message, details = undefined) { + throw new ModelReaderError(code, 'provider-host', false, message, details); +} + +export function encodeHostFrame({ kind, requestId, runHandle, sequence, final, payload }) { + if (![1, 2, 3, 4].includes(kind)) throw new TypeError('unknown host frame kind'); + if (typeof requestId !== 'bigint' || requestId <= 0n || requestId > 0xffffffffffffffffn) throw new TypeError('invalid host frame request id'); + if (!Buffer.isBuffer(runHandle) || runHandle.length !== 32) throw new TypeError('invalid host frame run handle'); + if (!Number.isSafeInteger(sequence) || sequence < 0 || sequence > 0xffffffff) throw new TypeError('invalid host frame sequence'); + if (!Buffer.isBuffer(payload) || payload.length > MAX_PAYLOAD_BYTES) throw new TypeError('host frame payload exceeds limit'); + const bytes = Buffer.alloc(HEADER_BYTES + payload.length); + bytes[0] = kind; bytes.writeBigUInt64BE(requestId, 1); runHandle.copy(bytes, 9); + bytes.writeUInt32BE(sequence, 41); bytes[45] = final ? 1 : 0; bytes.writeUInt32BE(payload.length, 46); + payload.copy(bytes, HEADER_BYTES); return bytes; +} + +export class HostFrameDecoder { + #buffer = Buffer.alloc(0); + push(chunk) { + if (!Buffer.isBuffer(chunk)) chunk = Buffer.from(chunk); + this.#buffer = Buffer.concat([this.#buffer, chunk]); + const frames = []; + while (this.#buffer.length >= HEADER_BYTES) { + const kind = this.#buffer[0]; + if (![1, 2, 3, 4].includes(kind)) throw new Error('unknown host frame kind'); + const length = this.#buffer.readUInt32BE(46); + if (length > MAX_PAYLOAD_BYTES) throw new Error('host frame payload exceeds limit'); + if (this.#buffer.length < HEADER_BYTES + length) break; + frames.push({ + kind, requestId: this.#buffer.readBigUInt64BE(1), runHandle: Buffer.from(this.#buffer.subarray(9, 41)), + sequence: this.#buffer.readUInt32BE(41), final: (this.#buffer[45] & 1) !== 0, + payload: Buffer.from(this.#buffer.subarray(HEADER_BYTES, HEADER_BYTES + length)), + }); + this.#buffer = this.#buffer.subarray(HEADER_BYTES + length); + } + return frames; + } +} + +class ModelHostClient { + constructor(child) { + this.child = child; this.nextRequestId = 1n; this.pending = new Map(); this.decoder = new HostFrameDecoder(); this.closed = false; + this.stderr = Buffer.alloc(0); + child.stdout.on('data', (chunk) => { + try { for (const frame of this.decoder.push(chunk)) this.onFrame(frame); } + catch (error) { this.failAll(error); } + }); + child.stderr.on('data', (chunk) => { this.stderr = Buffer.concat([this.stderr, chunk]).subarray(-8192); }); + child.on('error', (error) => this.failAll(error)); + child.on('exit', (code) => { if (!this.closed) this.failAll(new Error(`model-reader host exited ${code}`)); }); + } + + write(frame) { + if (!this.child.stdin.write(encodeHostFrame(frame))) { + this.child.stdin.once('drain', () => {}); + } + } + + control(body, runHandle = ZERO_HANDLE) { + const requestId = this.nextRequestId++; + return { requestId, frame: { kind: 1, requestId, runHandle, sequence: 0, final: true, payload: canonicalJsonBytes(body) } }; + } + + simple(body, runHandle = ZERO_HANDLE) { + const { requestId, frame } = this.control(body, runHandle); + const promise = new Promise((resolve, reject) => this.pending.set(requestId.toString(), { type: 'simple', resolve, reject })); + this.write(frame); return promise; + } + + async ready() { + const hello = await this.simple({ op: 'hello' }); + if (hello.protocol !== 'model-reader-host/v1' || typeof hello.build !== 'string') hostError('reference-provider-host-protocol', 'The managed provider host has an incompatible protocol.'); + } + + run = async (request) => { + const { requestId, frame } = this.control({ + op: 'provider-run', executable: request.executable, operation: request.operation, + cwd: request.cwd, environment: request.environment, stdinLength: request.stdin.length, + timeoutMs: request.timeoutMs, stdoutLimit: request.stdoutLimit, stderrLimit: request.stderrLimit, + }); + const promise = new Promise((resolve, reject) => this.pending.set(requestId.toString(), { + type: 'run', request, resolve, reject, handle: null, stdout: [], stderr: [], stdoutSequence: 0, stderrSequence: 0, stdoutFinal: false, stderrFinal: false, + })); + this.write(frame); + if (request.signal) { + const cancel = () => { + const state = this.pending.get(requestId.toString()); + if (state?.handle) void this.simple({ op: 'provider-cancel' }, state.handle).catch(() => {}); + }; + if (request.signal.aborted) cancel(); else request.signal.addEventListener('abort', cancel, { once: true }); + } + return await promise; + }; + + onFrame(frame) { + const state = this.pending.get(frame.requestId.toString()); + if (!state) throw new Error('uncorrelated model-reader host frame'); + if (state.type === 'simple') { + if (frame.kind !== 1 || !frame.final || frame.sequence !== 0) throw new Error('invalid model-reader host control response'); + this.pending.delete(frame.requestId.toString()); state.resolve(JSON.parse(frame.payload.toString('utf8'))); return; + } + if (frame.kind === 1) { + const control = JSON.parse(frame.payload.toString('utf8')); + if (control.status === 'accepted') { + if (state.handle || frame.runHandle.equals(ZERO_HANDLE)) throw new Error('invalid model-reader host run acceptance'); + state.handle = frame.runHandle; + this.write({ kind: 4, requestId: frame.requestId, runHandle: state.handle, sequence: 0, final: true, payload: state.request.stdin }); + return; + } + if (control.status === 'complete') { + if (!state.handle || !frame.runHandle.equals(state.handle) || !state.stdoutFinal || !state.stderrFinal) throw new Error('model-reader host completed before correlated streams'); + this.pending.delete(frame.requestId.toString()); + state.resolve({ exitCode: control.exitCode, stdout: Buffer.concat(state.stdout), stderr: Buffer.concat(state.stderr) }); return; + } + throw new Error('unknown model-reader host run control'); + } + if (!state.handle || !frame.runHandle.equals(state.handle) || !frame.final) throw new Error('uncorrelated or unterminated model-reader host stream'); + if (frame.kind === 2) { + if (frame.sequence !== state.stdoutSequence++) throw new Error('model-reader host stdout sequence mismatch'); + state.stdout.push(frame.payload); state.stdoutFinal = true; + } else if (frame.kind === 3) { + if (frame.sequence !== state.stderrSequence++) throw new Error('model-reader host stderr sequence mismatch'); + state.stderr.push(frame.payload); state.stderrFinal = true; + } else throw new Error('unexpected model-reader host frame'); + } + + failAll(error) { + for (const state of this.pending.values()) state.reject(new ModelReaderError('reference-provider-host-failed', 'provider-host', true, 'The managed provider host failed.', error)); + this.pending.clear(); + } + + async close() { + if (this.closed) return; + try { await this.simple({ op: 'shutdown' }); } catch { /* host death already rejects callers */ } + this.closed = true; this.child.stdin.end(); + } +} + +export async function createModelHostClient(hostPath, options = {}) { + if (typeof hostPath !== 'string' || !path.isAbsolute(hostPath)) hostError('reference-provider-host-unavailable', 'The managed provider host path is unavailable.'); + let stat; let real; + try { stat = await fs.lstat(hostPath); real = await fs.realpath(hostPath); } + catch (error) { hostError('reference-provider-host-unavailable', 'The managed provider host is unavailable.', error); } + if (!stat.isFile() || stat.isSymbolicLink() || path.resolve(real).toLowerCase() !== path.resolve(hostPath).toLowerCase()) hostError('reference-provider-host-unavailable', 'The managed provider host path is unsafe.'); + const child = spawn(hostPath, ['__model-reader-host'], { shell: false, windowsHide: true, stdio: ['pipe', 'pipe', 'pipe'], env: options.environment ?? process.env }); + const client = new ModelHostClient(child); await client.ready(); return client; +} diff --git a/cli-connection-reader/model-host-client.test.mjs b/cli-connection-reader/model-host-client.test.mjs new file mode 100644 index 000000000..b0cfe0720 --- /dev/null +++ b/cli-connection-reader/model-host-client.test.mjs @@ -0,0 +1,24 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { HostFrameDecoder, encodeHostFrame } from './model-host-client.mjs'; + +test('host frames preserve request id, run handle, sequence, final flag, and binary payload across arbitrary chunks', () => { + const expected = { + kind: 2, requestId: 0x0102030405060708n, runHandle: Buffer.alloc(32, 0x7f), + sequence: 9, final: true, payload: Buffer.from([0, 255, 1, 2]), + }; + const bytes = encodeHostFrame(expected); + const decoder = new HostFrameDecoder(); + assert.deepEqual(decoder.push(bytes.subarray(0, 17)), []); + assert.deepEqual(decoder.push(bytes.subarray(17, 51)), []); + const [actual] = decoder.push(bytes.subarray(51)); + assert.deepEqual(actual, expected); +}); + +test('frame decoder refuses unknown kinds and oversized payload declarations before allocation', () => { + const unknown = encodeHostFrame({ kind: 1, requestId: 1n, runHandle: Buffer.alloc(32), sequence: 0, final: true, payload: Buffer.alloc(0) }); + unknown[0] = 9; + assert.throws(() => new HostFrameDecoder().push(unknown), /kind/); + const oversized = Buffer.alloc(50); oversized[0] = 1; oversized.writeUInt32BE(1024 * 1024 + 1, 46); + assert.throws(() => new HostFrameDecoder().push(oversized), /limit/); +}); diff --git a/cli-connection-reader/model-provider.mjs b/cli-connection-reader/model-provider.mjs index b051ae5f1..1f448e676 100644 --- a/cli-connection-reader/model-provider.mjs +++ b/cli-connection-reader/model-provider.mjs @@ -2,7 +2,7 @@ import fs from 'node:fs/promises'; import path from 'node:path'; import { assertClosedObject, assertSha256, buildCanonicalRequest, buildProviderFingerprint, - canonicalJsonBytes, lowerableLimits, ModelReaderError, parseJsonStrict, sha256, + canonicalJsonBytes, lowerableLimits, ModelReaderError, parseJsonStrict, providerFingerprintSha256, sha256, } from './model-contract.mjs'; function providerError(code, message, retryable = false, details = undefined) { @@ -123,6 +123,33 @@ async function validatedOutput(outputPath, expectedPath, limit, label) { return { path: outputPath, bytes: output.bytes, size: output.stat.size, sha256: output.sha256 }; } +export async function describeProvider(options) { + const limits = lowerableLimits(options.limits); + const initialExecutable = await validateProviderExecutable(options.executable); + await privateDirectory(options.privateRoot); + const cwd = await privateDirectory(path.join(options.privateRoot, 'describe')); + const environment = minimalProviderEnvironment(options.environment); + const stdin = canonicalJsonBytes({ protocolVersion: '1', limits }); + const stdout = await callProvider(options.hostRun, { + executable: initialExecutable.path, operation: 'describe', stdin, stdinLength: stdin.length, + cwd, environment, timeoutMs: limits.conversionMs, + stdoutLimit: limits.providerStdoutBytes, stderrLimit: limits.providerStderrBytes, + }, limits); + const afterDescribe = await validateProviderExecutable(options.executable); + if (afterDescribe.sha256 !== initialExecutable.sha256) providerError('reference-provider-changed', 'Provider executable changed during description.'); + const describe = validateDescribe(parseProviderJson(stdout, limits, 'description')); + const fingerprint = buildProviderFingerprint({ + protocolVersion: describe.protocolVersion, provider: describe.provider, engine: describe.engine, + engineVersion: describe.engineVersion, adapterBuildId: describe.adapterBuildId, + adapterExecutableSha256: initialExecutable.sha256, + }); + if (options.expectedProviderSha256 !== undefined) { + assertSha256(options.expectedProviderSha256, 'expectedProviderSha256'); + if (providerFingerprintSha256(fingerprint) !== options.expectedProviderSha256) providerError('reference-provider-pin-mismatch', 'The local provider does not match the expected fingerprint.'); + } + return { describe, fingerprint, providerExecutableSha256: initialExecutable.sha256 }; +} + export async function describeAndConvert(options) { const limits = lowerableLimits(options.limits); const initialExecutable = await validateProviderExecutable(options.executable); @@ -139,6 +166,15 @@ export async function describeAndConvert(options) { const afterDescribe = await validateProviderExecutable(options.executable); if (afterDescribe.sha256 !== initialExecutable.sha256) providerError('reference-provider-changed', 'Provider executable changed during description.'); const describe = validateDescribe(parseProviderJson(describeBytes, limits, 'description')); + const describedFingerprint = buildProviderFingerprint({ + protocolVersion: describe.protocolVersion, provider: describe.provider, engine: describe.engine, + engineVersion: describe.engineVersion, adapterBuildId: describe.adapterBuildId, + adapterExecutableSha256: initialExecutable.sha256, + }); + if (options.expectedProviderSha256 !== undefined) { + assertSha256(options.expectedProviderSha256, 'expectedProviderSha256'); + if (providerFingerprintSha256(describedFingerprint) !== options.expectedProviderSha256) providerError('reference-provider-pin-mismatch', 'The local provider does not match the expected fingerprint.'); + } const canonicalRequest = buildCanonicalRequest({ limits, conversionSettings: options.conversionSettings ?? {} }); const outputDirectory = await privateDirectory(path.join(options.privateRoot, 'output')); const beforeConvert = await validateProviderExecutable(options.executable); @@ -164,14 +200,7 @@ export async function describeAndConvert(options) { if (geometry.size + metadata.size > limits.maxProviderOutputBytes) providerError('reference-provider-output-too-large', 'Provider files exceed their total byte limit.'); return { describe, receipt, canonicalRequest, - fingerprint: buildProviderFingerprint({ - protocolVersion: describe.protocolVersion, - provider: describe.provider, - engine: describe.engine, - engineVersion: describe.engineVersion, - adapterBuildId: describe.adapterBuildId, - adapterExecutableSha256: initialExecutable.sha256, - }), + fingerprint: describedFingerprint, providerExecutableSha256: initialExecutable.sha256, stagedSource: staging, outputs: { geometry, metadata }, }; diff --git a/cli-connection-reader/model-reader.mjs b/cli-connection-reader/model-reader.mjs new file mode 100644 index 000000000..89f28ecf2 --- /dev/null +++ b/cli-connection-reader/model-reader.mjs @@ -0,0 +1,221 @@ +import fs from 'node:fs/promises'; +import os from 'node:os'; +import path from 'node:path'; +import { + assertSha256, buildCanonicalRequest, ModelReaderError, providerFingerprintSha256, + requestSha256, sha256, +} from './model-contract.mjs'; +import { normalizeRevitGlb } from './revit-glb.mjs'; +import { normalizeRevitMetadata } from './revit-metadata.mjs'; +import { describeAndConvert, describeProvider } from './model-provider.mjs'; +import { + acquireCacheOwner, cacheKeySha256, loadAwareSigningKey, publishCacheEntry, readCacheEntry, + releaseCacheOwner, signerFingerprintSha256, +} from './model-cache.mjs'; +import { createModelHostClient } from './model-host-client.mjs'; + +function readerError(code, phase, message, retryable = false, details = undefined) { + throw new ModelReaderError(code, phase, retryable, message, details); +} + +function awareHome(environment) { + return environment.AWARE_HOME || path.join(os.homedir(), '.aware'); +} + +function configuration(args, deps) { + const environment = deps.environment ?? process.env; + const home = awareHome(environment); + const executable = args['provider-path'] ?? environment.AWARE_MODEL_REFERENCE_PROVIDER; + if (typeof executable !== 'string' || !executable) readerError('reference-provider-unavailable', 'preflight', 'A local model provider is not configured.'); + const secretPath = args['signing-secret-path'] ?? environment.AWARE_MODEL_REFERENCE_SIGNING_KEY ?? path.join(home, 'keys', 'model-reference-reader.sec'); + const publicPath = args['signing-public-path'] ?? environment.AWARE_MODEL_REFERENCE_PUBLIC_KEY ?? secretPath.replace(/\.sec$/i, '.pub'); + return { + executable, secretPath, publicPath, environment, + cacheRoot: deps.cacheRoot ?? path.join(home, 'cache', 'model-reference-reader'), + privateRoot: deps.privateRoot ?? path.join(home, 'cache', 'model-reference-reader', 'provider-runs'), + artifactDirectory: deps.artifactDirectory ?? environment.AWARE_ARTIFACT_DIR, + }; +} + +async function newRunRoot(parent) { + await fs.mkdir(parent, { recursive: true, mode: 0o700 }); + return await fs.mkdtemp(path.join(parent, 'run-')); +} + +function emit(deps, phase, extra = {}) { + if (typeof deps.progress === 'function') deps.progress({ phase, ...extra }); +} + +async function providerReadiness(args, deps, config, expectedProviderSha256) { + const signingKey = await loadAwareSigningKey(config.secretPath, config.publicPath); + const runRoot = await newRunRoot(config.privateRoot); + try { + const provider = await describeProvider({ + executable: config.executable, privateRoot: path.join(runRoot, 'describe'), + hostRun: deps.hostRun, environment: config.environment, limits: deps.limits, + expectedProviderSha256, + }); + return { + signingKey, provider, + signerFingerprintSha256: signerFingerprintSha256(signingKey.publicKeyBytes), + providerFingerprintSha256: providerFingerprintSha256(provider.fingerprint), + }; + } finally { + await fs.rm(runRoot, { recursive: true, force: true }); + } +} + +function sourcePathFrom(args) { + const values = [args['rvt-path'], args.rvtPath, args['model-path']].filter((value) => value !== undefined); + if (values.length !== 1 || typeof values[0] !== 'string' || !values[0]) readerError('reference-source-invalid', 'source', 'Exactly one absolute RVT source path is required.'); + if (!path.isAbsolute(values[0]) || !/\.rvt$/i.test(values[0])) readerError('reference-source-invalid', 'source', 'The source must be an absolute .rvt path.'); + return values[0]; +} + +async function hashSource(sourcePath) { + let bytes; + try { bytes = await fs.readFile(sourcePath); } + catch (error) { readerError('reference-source-unavailable', 'source', 'The RVT source is unavailable.', false, error); } + return { bytes, sha256: sha256(bytes) }; +} + +function exactExpectedSource(args, actual) { + const expected = args['source-sha256']; + assertSha256(expected, 'source-sha256'); + if (expected !== actual) readerError('reference-source-changed', 'source', 'The RVT source does not match the expected digest.'); + return expected; +} + +function manifestDetails(geometry, metadata, canonicalRequestSha256, fingerprintSha256) { + return { + frame: { units: 'mm', up: 'z', handedness: 'right', axes: ['x', 'y', 'z'] }, + canonicalRequestSha256, + providerFingerprintSha256: fingerprintSha256, + coverage: { ...metadata.coverage, geometry: geometry.coverage }, + }; +} + +async function convertAndCache(args, deps, config, readiness) { + const sourcePath = sourcePathFrom(args); + const initial = await hashSource(sourcePath); + const sourceSha256 = exactExpectedSource(args, initial.sha256); + const canonicalRequest = buildCanonicalRequest({ limits: deps.limits, conversionSettings: args['conversion-settings'] ?? {} }); + const identity = { + sourceSha256, canonicalRequest, providerFingerprint: readiness.provider.fingerprint, + signerFingerprintSha256: readiness.signerFingerprintSha256, + }; + const key = cacheKeySha256(identity); + const read = async () => await readCacheEntry({ root: config.cacheRoot, key, expectedIdentity: identity, expectedPublicKey: readiness.signingKey.publicKeyBytes }); + try { return { hit: true, key, cache: await read() }; } + catch (error) { if (error?.code !== 'reference-cache-miss') throw error; } + const owner = await acquireCacheOwner({ root: config.cacheRoot, key }); + try { + try { return { hit: true, key, cache: await read() }; } + catch (error) { if (error?.code !== 'reference-cache-miss') throw error; } + emit(deps, 'convert'); + const runRoot = await newRunRoot(config.privateRoot); + let conversion; + try { + conversion = await describeAndConvert({ + executable: config.executable, sourcePath, expectedSourceSha256: sourceSha256, + expectedProviderSha256: readiness.providerFingerprintSha256, + privateRoot: path.join(runRoot, 'conversion'), hostRun: deps.hostRun, + environment: config.environment, limits: deps.limits, + conversionSettings: args['conversion-settings'] ?? {}, + }); + emit(deps, 'normalize'); + const geometry = normalizeRevitGlb(conversion.outputs.geometry.bytes, { limits: deps.limits }); + const metadata = normalizeRevitMetadata(conversion.outputs.metadata.bytes, geometry.parts, { limits: deps.limits }); + const finalSource = await hashSource(sourcePath); + if (finalSource.sha256 !== sourceSha256) readerError('reference-source-changed', 'source', 'The RVT source changed during conversion.'); + const artifacts = { + 'geometry.glb': geometry.glb, + 'entities.json': metadata.entitiesBytes, + 'properties.json': metadata.propertiesBytes, + 'relationships.json': metadata.relationshipsBytes, + }; + const details = manifestDetails(geometry, metadata, requestSha256(canonicalRequest), readiness.providerFingerprintSha256); + emit(deps, 'publish'); + await publishCacheEntry({ root: config.cacheRoot, identity, artifacts, signingKey: readiness.signingKey, details }); + } finally { + await fs.rm(runRoot, { recursive: true, force: true }); + } + return { hit: false, key, cache: await read() }; + } finally { + await releaseCacheOwner(owner); + } +} + +function summary(result) { + const coverage = result.cache.manifest.coverage; + const entityDocument = JSON.parse(result.cache.artifacts['entities.json'].toString('utf8')); + const bounds = entityDocument.entities.reduce((box, entity) => { + if (!entity.bounds) return box; + if (!box) return structuredClone(entity.bounds); + for (let axis = 0; axis < 3; axis += 1) { + box.min[axis] = Math.min(box.min[axis], entity.bounds.min[axis]); + box.max[axis] = Math.max(box.max[axis], entity.bounds.max[axis]); + } + return box; + }, null); + return { + schemaVersion: 'model-reference-reader/v1', cache: result.hit ? 'hit' : 'miss', + sourceSha256: result.cache.manifest.identity.sourceSha256, + canonicalRequestSha256: result.cache.manifest.canonicalRequestSha256, + providerFingerprint: result.cache.manifest.identity.providerFingerprint, + providerFingerprintSha256: result.cache.manifest.providerFingerprintSha256, + signerFingerprintSha256: result.cache.manifest.identity.signerFingerprintSha256, + frame: result.cache.manifest.frame, coverage, bounds, + entities: coverage.indexedEntities, geometryNodes: coverage.geometryNodes, + properties: coverage.properties, relationships: coverage.relationships, + }; +} + +async function publishRunArtifacts(result, directory) { + if (typeof directory !== 'string' || !path.isAbsolute(directory)) readerError('reference-artifact-directory-missing', 'publish', 'A run-owned artifact directory is required.'); + await fs.mkdir(directory, { recursive: true, mode: 0o700 }); + const logical = { geometry: 'geometry.glb', entities: 'entities.json', properties: 'properties.json', relationships: 'relationships.json', manifest: 'manifest.json' }; + const descriptors = {}; + for (const [name, cacheName] of Object.entries(logical)) { + const bytes = result.cache.artifacts[cacheName]; + const digest = sha256(bytes); + const id = `model-${result.key.slice(0, 16)}-${cacheName}`; + const target = path.join(directory, id); + try { await fs.writeFile(target, bytes, { flag: 'wx', mode: 0o600 }); } + catch (error) { + if (error?.code !== 'EEXIST' || sha256(await fs.readFile(target)) !== digest) readerError('reference-artifact-collision', 'publish', 'A run artifact id collided with different bytes.', false, error); + } + descriptors[name] = { id, mediaType: cacheName.endsWith('.glb') ? 'model/gltf-binary' : 'application/json', bytes: bytes.length, sha256: digest }; + } + return descriptors; +} + +export async function runModelCommand(command, args = {}, deps = {}) { + if (!args || typeof args !== 'object' || Array.isArray(args)) readerError('reference-request-invalid', 'request', 'Command input must be a JSON object.'); + const config = configuration(args, deps); + const ownedHost = deps.hostRun ? null : await createModelHostClient(config.environment.AWARE_MODEL_READER_HOST, { environment: config.environment }); + const executionDeps = ownedHost ? { ...deps, hostRun: ownedHost.run } : deps; + try { + const pin = args['expected-provider-sha256']; + emit(executionDeps, 'preflight'); + const readiness = await providerReadiness(args, executionDeps, config, pin); + if (command === 'preflight') { + return { + schemaVersion: 'model-reference-reader/v1', ready: true, execution: 'local', + provider: readiness.provider.describe, providerFingerprint: readiness.provider.fingerprint, + providerFingerprintSha256: readiness.providerFingerprintSha256, + signerFingerprintSha256: readiness.signerFingerprintSha256, + secretProvisioning: 'provider-local; AWARE generic secrets unavailable (#448)', + }; + } + if (command !== 'probe' && command !== 'read-model') readerError('reference-command-invalid', 'request', 'Unknown model-reader command.'); + if (typeof pin !== 'string') readerError('reference-provider-pin-required', 'preflight', 'The expected provider fingerprint is required.'); + const result = await convertAndCache(args, executionDeps, config, readiness); + const out = summary(result); + if (command === 'probe') return out; + emit(executionDeps, 'artifacts'); + return { ...out, artifacts: await publishRunArtifacts(result, config.artifactDirectory) }; + } finally { + await ownedHost?.close(); + } +} diff --git a/cli-connection-reader/model-reader.test.mjs b/cli-connection-reader/model-reader.test.mjs new file mode 100644 index 000000000..f835112ab --- /dev/null +++ b/cli-connection-reader/model-reader.test.mjs @@ -0,0 +1,97 @@ +import assert from 'node:assert/strict'; +import { generateKeyPairSync } from 'node:crypto'; +import { spawn } from 'node:child_process'; +import fs from 'node:fs/promises'; +import os from 'node:os'; +import path from 'node:path'; +import test from 'node:test'; +import { fileURLToPath } from 'node:url'; +import { sha256 } from './model-contract.mjs'; +import { runModelCommand } from './model-reader.mjs'; + +const fixture = fileURLToPath(new URL('./test-fixtures/model-provider-fixture.mjs', import.meta.url)); + +async function setup(t) { + const root = await fs.mkdtemp(path.join(os.tmpdir(), 'aware-model-reader-')); + t.after(() => fs.rm(root, { recursive: true, force: true })); + const sourcePath = path.join(root, 'fixture.rvt'); + const executable = path.join(root, 'provider.exe'); + await fs.writeFile(sourcePath, 'fixture-rvt'); + await fs.writeFile(executable, 'fixture-provider-binary'); + const { privateKey, publicKey } = generateKeyPairSync('ed25519'); + const secretPath = path.join(root, 'reader.sec'); const publicPath = path.join(root, 'reader.pub'); + await fs.writeFile(secretPath, `ed25519-secret-key-v1 ${privateKey.export({ format: 'der', type: 'pkcs8' }).subarray(-32).toString('base64')}\n`); + await fs.writeFile(publicPath, `ed25519-public-key-v1 ${publicKey.export({ format: 'der', type: 'spki' }).subarray(-32).toString('base64')}\n`); + const calls = []; + const hostRun = async (request) => { + calls.push(request.operation); + return await new Promise((resolve, reject) => { + const child = spawn(process.execPath, [fixture, request.operation], { cwd: request.cwd, env: request.environment, windowsHide: true, shell: false, stdio: ['pipe', 'pipe', 'pipe'] }); + const stdout = []; const stderr = []; + child.stdout.on('data', (chunk) => stdout.push(chunk)); child.stderr.on('data', (chunk) => stderr.push(chunk)); + child.on('error', reject); child.on('close', (exitCode) => resolve({ exitCode, stdout: Buffer.concat(stdout), stderr: Buffer.concat(stderr) })); + child.stdin.end(request.stdin); + }); + }; + return { + root, sourcePath, executable, secretPath, publicPath, calls, + deps: { hostRun, cacheRoot: path.join(root, 'cache'), privateRoot: path.join(root, 'runs'), artifactDirectory: path.join(root, 'artifacts') }, + args: { 'rvt-path': sourcePath, 'source-sha256': sha256(Buffer.from('fixture-rvt')), 'provider-path': executable, 'signing-secret-path': secretPath, 'signing-public-path': publicPath }, + }; +} + +test('preflight describes provider and key readiness without conversion or source access', async (t) => { + const state = await setup(t); + const out = await runModelCommand('preflight', { + 'provider-path': state.executable, 'signing-secret-path': state.secretPath, 'signing-public-path': state.publicPath, + }, state.deps); + assert.equal(out.ready, true); + assert.equal(out.execution, 'local'); + assert.match(out.providerFingerprintSha256, /^[0-9a-f]{64}$/); + assert.deepEqual(state.calls, ['describe']); +}); + +test('read-model publishes five binary-safe artifacts with reconciled coverage and provenance', async (t) => { + const state = await setup(t); + const preflight = await runModelCommand('preflight', { + 'provider-path': state.executable, 'signing-secret-path': state.secretPath, 'signing-public-path': state.publicPath, + }, state.deps); + const out = await runModelCommand('read-model', { ...state.args, 'expected-provider-sha256': preflight.providerFingerprintSha256 }, state.deps); + assert.equal(out.schemaVersion, 'model-reference-reader/v1'); + assert.equal(out.frame.units, 'mm'); assert.equal(out.frame.up, 'z'); + assert.equal(out.coverage.discoveredEntities, 1); + assert.equal(Object.keys(out.artifacts).length, 5); + for (const descriptor of Object.values(out.artifacts)) { + assert.match(descriptor.id, /^[a-z0-9.-]+$/); + const bytes = await fs.readFile(path.join(state.deps.artifactDirectory, descriptor.id)); + assert.equal(sha256(bytes), descriptor.sha256); + assert.equal(bytes.length, descriptor.bytes); + } + const geometry = await fs.readFile(path.join(state.deps.artifactDirectory, out.artifacts.geometry.id)); + assert.equal(geometry.readUInt32LE(0), 0x46546c67); +}); + +test('probe is bounded, cache-aware, and two cold conversions produce identical artifact hashes', async (t) => { + const first = await setup(t); + const pin = (await runModelCommand('preflight', { 'provider-path': first.executable, 'signing-secret-path': first.secretPath, 'signing-public-path': first.publicPath }, first.deps)).providerFingerprintSha256; + const args = { ...first.args, 'expected-provider-sha256': pin }; + const cold = await runModelCommand('read-model', args, first.deps); + const probe = await runModelCommand('probe', args, first.deps); + assert.equal(probe.cache, 'hit'); + assert.equal(probe.entities, 1); + + const secondCache = path.join(first.root, 'second-cache'); + const secondArtifacts = path.join(first.root, 'second-artifacts'); + const second = await runModelCommand('read-model', args, { ...first.deps, cacheRoot: secondCache, artifactDirectory: secondArtifacts, privateRoot: path.join(first.root, 'second-runs') }); + assert.deepEqual(Object.fromEntries(Object.entries(cold.artifacts).map(([name, value]) => [name, value.sha256])), Object.fromEntries(Object.entries(second.artifacts).map(([name, value]) => [name, value.sha256]))); +}); + +test('a wrong provider pin refuses before convert and errors never disclose paths', async (t) => { + const state = await setup(t); + await assert.rejects(() => runModelCommand('read-model', { ...state.args, 'expected-provider-sha256': '0'.repeat(64) }, state.deps), (error) => { + assert.equal(error.code, 'reference-provider-pin-mismatch'); + assert.equal(error.message.includes(state.sourcePath), false); + return true; + }); + assert.deepEqual(state.calls, ['describe']); +}); diff --git a/cli-connection-reader/model-windows-harness.mjs b/cli-connection-reader/model-windows-harness.mjs new file mode 100644 index 000000000..1ce488ebf --- /dev/null +++ b/cli-connection-reader/model-windows-harness.mjs @@ -0,0 +1,146 @@ +import assert from 'node:assert/strict'; +import { execFileSync } from 'node:child_process'; +import { createHash } from 'node:crypto'; +import { copyFileSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { build } from 'esbuild'; + +if (process.platform !== 'win32') { + process.stdout.write('model Windows harness: skipped (requires Windows process semantics)\n'); + process.exit(0); +} + +const here = path.dirname(fileURLToPath(import.meta.url)); +const root = path.dirname(here); +const temporary = mkdtempSync(path.join(os.tmpdir(), 'aware-rvt-harness-')); +const FUSE = 'NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2'; +const aware = process.env.AWARE_SOURCE_BUILT || path.join(root, 'cli', 'target', 'debug', 'aware.exe'); + +function sha256(bytes) { return createHash('sha256').update(bytes).digest('hex'); } + +async function buildSea(entry, output) { + const buildDirectory = path.join(temporary, `sea-${path.basename(output, '.exe')}`); + mkdirSync(buildDirectory, { recursive: true }); + const bundle = path.join(buildDirectory, 'bundle.cjs'); + await build({ entryPoints: [entry], bundle: true, platform: 'node', format: 'cjs', target: 'node20', outfile: bundle }); + const config = path.join(buildDirectory, 'sea-config.json'); + const blob = path.join(buildDirectory, 'sea.blob'); + writeFileSync(config, JSON.stringify({ main: bundle, output: blob, disableExperimentalSEAWarning: true })); + execFileSync(process.execPath, ['--experimental-sea-config', config], { stdio: 'pipe' }); + copyFileSync(process.execPath, output); + execFileSync(process.execPath, [path.join(here, 'node_modules', 'postject', 'dist', 'cli.js'), output, 'NODE_SEA_BLOB', blob, '--sentinel-fuse', FUSE], { stdio: 'pipe' }); +} + +function run(entry, command, input, environment, cwd) { + const stdout = execFileSync(entry, [command, '--json-stdin'], { input: JSON.stringify(input), encoding: 'utf8', env: environment, cwd, windowsHide: true, maxBuffer: 4 * 1024 * 1024 }); + return JSON.parse(stdout); +} + +function findArtifactSet(value) { + if (!value || typeof value !== 'object') return null; + if (value.artifacts && typeof value.artifacts === 'object' && Object.keys(value.artifacts).length === 5) { + return value.artifacts; + } + for (const child of Object.values(value)) { + const found = findArtifactSet(child); + if (found) return found; + } + return null; +} + +try { + assert.equal(path.isAbsolute(aware), true, 'AWARE_SOURCE_BUILT must be absolute'); + const providerDirectory = path.join(temporary, 'authorized-provider'); mkdirSync(providerDirectory); + const provider = path.join(providerDirectory, 'fixture-model-provider.exe'); + await buildSea(path.join(here, 'test-fixtures', 'model-provider-fixture.mjs'), provider); + + execFileSync(process.execPath, [path.join(here, 'build.mjs')], { cwd: here, stdio: 'inherit' }); + const stage = path.join(temporary, 'clean-stage'); mkdirSync(stage); + const packaged = path.join(stage, 'aware-connection-reader.exe'); + copyFileSync(path.join(here, 'dist', 'aware-connection-reader.exe'), packaged); + copyFileSync(path.join(here, 'dist', 'web-ifc-node.wasm'), path.join(stage, 'web-ifc-node.wasm')); + assert.deepEqual(readdirSync(stage).sort(), ['aware-connection-reader.exe', 'web-ifc-node.wasm']); + + const home = path.join(temporary, 'aware-home'); + execFileSync(aware, ['key', 'generate', 'model-reference-reader'], { env: { ...process.env, AWARE_HOME: home }, stdio: 'pipe', windowsHide: true }); + const inputDirectory = path.join(temporary, 'authorized-input'); mkdirSync(inputDirectory); + const source = path.join(inputDirectory, 'fixture.rvt'); writeFileSync(source, 'fixture-rvt'); + const artifacts = path.join(temporary, 'artifacts'); mkdirSync(artifacts); + const unrelatedCwd = path.join(temporary, 'unrelated-cwd'); mkdirSync(unrelatedCwd); + const environment = { + ...process.env, AWARE_HOME: home, AWARE_MODEL_READER_HOST: aware, + AWARE_MODEL_REFERENCE_PROVIDER: provider, + AWARE_MODEL_REFERENCE_SIGNING_KEY: path.join(home, 'keys', 'model-reference-reader.sec'), + AWARE_MODEL_REFERENCE_PUBLIC_KEY: path.join(home, 'keys', 'model-reference-reader.pub'), + AWARE_ARTIFACT_DIR: artifacts, + }; + const bridges = path.join(home, 'bridges'); mkdirSync(bridges, { recursive: true }); + copyFileSync(packaged, path.join(bridges, 'aware-connection-reader.exe')); + copyFileSync(path.join(stage, 'web-ifc-node.wasm'), path.join(bridges, 'web-ifc-node.wasm')); + const preflight = run(packaged, 'preflight', {}, environment, unrelatedCwd); + assert.equal(preflight.ready, true); assert.equal(preflight.execution, 'local'); + const request = { 'rvt-path': source, 'source-sha256': sha256(readFileSync(source)), 'expected-provider-sha256': preflight.providerFingerprintSha256 }; + const probe = run(packaged, 'probe', request, environment, unrelatedCwd); + assert.equal(probe.entities, 1); assert.equal(probe.frame.up, 'z'); + const model = run(packaged, 'read-model', request, environment, unrelatedCwd); + assert.equal(Object.keys(model.artifacts).length, 5); + const geometry = readFileSync(path.join(artifacts, model.artifacts.geometry.id)); + assert.equal(geometry.readUInt32LE(0), 0x46546c67); assert.equal(sha256(geometry), model.artifacts.geometry.sha256); + for (const descriptor of Object.values(model.artifacts)) assert.equal(sha256(readFileSync(path.join(artifacts, descriptor.id))), descriptor.sha256); + + const ifc = run(packaged, 'probe', { 'ifc-path': path.join(here, 'test-fixtures', 'baseplate-bp1.ifc') }, environment, unrelatedCwd); + assert.equal(ifc.schema, 'IFC4'); assert.equal(ifc.frame, 'z-up'); + + execFileSync(aware, ['agent', 'install', path.join(root, '20-agents', 'aeco', 'engineering', 'model-reference-reader')], { + env: environment, stdio: 'pipe', windowsHide: true, + }); + const appDirectory = path.join(temporary, 'rvt-reader-e2e'); mkdirSync(appDirectory); + const appSource = path.join(appDirectory, 'rvt-reader-e2e.flo'); + writeFileSync(appSource, `app: rvt-reader-e2e +version: 0.1.0 +display-name: RVT Reader E2E +description: Exercise the authenticated local RVT reader through a real one-shot AWARE app. +exposes-as-agent: false +requires: + - model-reference-reader@0.1.x +requires-permissions: + filesystem: + - read: '*.rvt' +layout: linear +nodes: + - id: read-reference + agent: model-reference-reader + command: read-model + inputs: + rvt-path: '{{ inputs.rvt-path }}' + source-sha256: '{{ inputs.source-sha256 }}' + expected-provider-sha256: '{{ inputs.expected-provider-sha256 }}' +`); + execFileSync(aware, ['app', 'install', appDirectory], { env: environment, stdio: 'pipe', windowsHide: true }); + const appStdout = execFileSync(aware, [ + 'app', 'run', 'rvt-reader-e2e', + '--input', `rvt-path=${source}`, + '--input', `source-sha256=${request['source-sha256']}`, + '--input', `expected-provider-sha256=${preflight.providerFingerprintSha256}`, + ], { env: environment, encoding: 'utf8', windowsHide: true, maxBuffer: 4 * 1024 * 1024 }); + const runId = appStdout.match(/run-id ([0-9a-f-]{36})/)?.[1]; + assert.ok(runId, 'real aware app run must report its run id'); + const trace = execFileSync(aware, ['app', 'logs', 'rvt-reader-e2e', '--run-id', runId], { + env: environment, encoding: 'utf8', windowsHide: true, maxBuffer: 4 * 1024 * 1024, + }).trim().split(/\r?\n/).filter(Boolean).map((line) => JSON.parse(line)); + const appArtifacts = findArtifactSet(trace); + assert.ok(appArtifacts, 'real aware app run must return all five artifact descriptors'); + for (const [kind, descriptor] of Object.entries(appArtifacts)) { + const retrieved = path.join(temporary, `retrieved-${kind}${kind === 'geometry' ? '.glb' : '.json'}`); + execFileSync(aware, ['app', 'artifact', 'rvt-reader-e2e', descriptor.id, '--run-id', runId, '--output', retrieved], { + env: environment, stdio: 'pipe', windowsHide: true, + }); + assert.equal(sha256(readFileSync(retrieved)), descriptor.sha256); + } + assert.equal(readdirSync(stage).some((name) => /\.(?:mjs|json)$/i.test(name)), false); + process.stdout.write(`model Windows harness: PASS (${model.sourceSha256}, ${model.providerFingerprintSha256})\n`); +} finally { + rmSync(temporary, { recursive: true, force: true }); +} diff --git a/cli-connection-reader/package.json b/cli-connection-reader/package.json index c9fb13b23..24d18a840 100644 --- a/cli-connection-reader/package.json +++ b/cli-connection-reader/package.json @@ -5,11 +5,12 @@ "type": "module", "description": "AWARE cli-transport bridge: extract steel connections from an IFC as tessellated mesh scene primitives (drives web-ifc WASM).", "bin": { - "aware-connection-reader": "./index.mjs" + "aware-connection-reader": "./model-dispatcher.mjs" }, "scripts": { "build": "node build.mjs", - "test": "node --test" + "test": "node --test", + "test:windows-harness": "node model-windows-harness.mjs" }, "dependencies": { "fflate": "^0.8.3", diff --git a/cli-connection-reader/test-fixtures/model-provider-fixture.mjs b/cli-connection-reader/test-fixtures/model-provider-fixture.mjs index d4845b0c8..749e53cd5 100644 --- a/cli-connection-reader/test-fixtures/model-provider-fixture.mjs +++ b/cli-connection-reader/test-fixtures/model-provider-fixture.mjs @@ -2,30 +2,35 @@ import fs from 'node:fs/promises'; import path from 'node:path'; import { makeGlbFixture, makeMetadataFixture } from '../model-fixtures.mjs'; -const operation = process.argv[2]; -const request = JSON.parse(await new Promise((resolve, reject) => { - const chunks = []; - process.stdin.on('data', (chunk) => chunks.push(chunk)); - process.stdin.on('end', () => resolve(Buffer.concat(chunks).toString('utf8'))); - process.stdin.on('error', reject); -})); const provenance = { protocolVersion: '1', provider: 'fixture-provider', engine: 'fixture-engine', engineVersion: '1.2.3', adapterBuildId: 'fixture-build', formats: ['rvt'], execution: 'local', destination: null, }; -if (operation === 'describe') { - process.stdout.write(JSON.stringify(provenance)); -} else if (operation === 'convert') { - const geometryPath = path.join(request.outputDirectory, 'geometry.glb'); - const metadataPath = path.join(request.outputDirectory, 'metadata.json'); - await fs.writeFile(geometryPath, makeGlbFixture()); - await fs.writeFile(metadataPath, JSON.stringify(makeMetadataFixture())); - process.stdout.write(JSON.stringify({ - ...provenance, documentKind: 'revit-project', sourceSha256: request.sourceSha256, - geometryPath, metadataPath, + +async function main() { + const operation = process.argv[2]; + const request = JSON.parse(await new Promise((resolve, reject) => { + const chunks = []; + process.stdin.on('data', (chunk) => chunks.push(chunk)); + process.stdin.on('end', () => resolve(Buffer.concat(chunks).toString('utf8'))); + process.stdin.on('error', reject); })); -} else { - process.stderr.write('unsupported fixture operation'); - process.exitCode = 2; + if (operation === 'describe') { + process.stdout.write(JSON.stringify(provenance)); + } else if (operation === 'convert') { + const geometryPath = path.join(request.outputDirectory, 'geometry.glb'); + const metadataPath = path.join(request.outputDirectory, 'metadata.json'); + await fs.writeFile(geometryPath, makeGlbFixture()); + await fs.writeFile(metadataPath, JSON.stringify(makeMetadataFixture())); + process.stdout.write(JSON.stringify({ + ...provenance, documentKind: 'revit-project', sourceSha256: request.sourceSha256, + geometryPath, metadataPath, + })); + } else { + process.stderr.write('unsupported fixture operation'); + process.exitCode = 2; + } } + +main().catch((error) => { process.stderr.write(`fixture provider: ${error instanceof Error ? error.message : String(error)}\n`); process.exitCode = 1; }); From 10864f538da2127c8fa2c19cb920d58cb27c89a5 Mon Sep 17 00:00:00 2001 From: Pawel Date: Sun, 23 Aug 2026 14:59:01 +0200 Subject: [PATCH 09/50] feat: publish the local RVT model reader agent --- 00-vision/manifesto.md | 6 +- .../commands/preflight.md | 15 +++ .../model-reference-reader/commands/probe.md | 13 +++ .../commands/read-model.md | 18 ++++ .../model-reference-reader/manifest.yaml | 101 ++++++++++++++++++ .../skills/provider-and-artifact-contract.md | 14 +++ 40-diagrams/substrate-playground.html | 2 +- CLAUDE.md | 6 +- README.md | 8 +- cli/src/commands/sidecar.rs | 5 +- cli/tests/agent_list.rs | 5 +- registry-catalog.json | 49 ++++++++- registry-index.json | 10 +- 13 files changed, 236 insertions(+), 16 deletions(-) create mode 100644 20-agents/aeco/engineering/model-reference-reader/commands/preflight.md create mode 100644 20-agents/aeco/engineering/model-reference-reader/commands/probe.md create mode 100644 20-agents/aeco/engineering/model-reference-reader/commands/read-model.md create mode 100644 20-agents/aeco/engineering/model-reference-reader/manifest.yaml create mode 100644 20-agents/aeco/engineering/model-reference-reader/skills/provider-and-artifact-contract.md diff --git a/00-vision/manifesto.md b/00-vision/manifesto.md index ecfdfb469..95802ae16 100644 --- a/00-vision/manifesto.md +++ b/00-vision/manifesto.md @@ -13,11 +13,11 @@ The nine structural truths it rests on are in **[the decalog](./decalog.md)**. I ```bash $ npm install -g @aware-aeco/cli ✓ aware CLI installed - ✓ 78 AECO agents available via `aware agent install` + ✓ 79 AECO agents available via `aware agent install` ✓ aware-aeco plugin registered with: claude-code, codex $ claude-code - ✓ plugin: aware-aeco · 78 agents available + ✓ plugin: aware-aeco · 79 agents available > Watch this Tekla model. When a welded assembly appears, > upload its drawing to my Trimble Connect fab folder. @@ -130,7 +130,7 @@ aware CLI ─ manages everything ## v0 scope - Apache 2.0 core CLI -- 78 first-party agents across engineering · architecture · construction · visualization · cross-cutting (`tekla`, `revit`, `rhino`, `autocad`, `trimble-connect`, `procore`, `navisworks`, `idea-statica-26`, `microsoft-365`, `aware-agent-builder`, …) — full list in [`registry-index.json`](../registry-index.json) +- 79 first-party agents across engineering · architecture · construction · visualization · cross-cutting (`tekla`, `revit`, `rhino`, `autocad`, `trimble-connect`, `procore`, `navisworks`, `idea-statica-26`, `microsoft-365`, `aware-agent-builder`, …) — full list in [`registry-index.json`](../registry-index.json) - Reference apps demonstrating composition patterns (linear, fan-in, fan-out) - Host plugin generators for claude-code, codex, opencode - GitHub-hosted registry (single source of truth, PR-based contributions) diff --git a/20-agents/aeco/engineering/model-reference-reader/commands/preflight.md b/20-agents/aeco/engineering/model-reference-reader/commands/preflight.md new file mode 100644 index 000000000..5b1e7b9b0 --- /dev/null +++ b/20-agents/aeco/engineering/model-reference-reader/commands/preflight.md @@ -0,0 +1,15 @@ +# preflight + +Use `preflight` before offering an RVT create/import action. It proves that the configured executable +is a regular local file, the AWARE-format Ed25519 keypair matches, the managed host protocol is current, +and the provider describes the exact closed local-RVT contract. It does not receive an RVT path and +does not convert a model. + +`ready: true` is specific to the provider and signer fingerprints returned beside it. Pin +`providerFingerprintSha256` into `probe` and `read-model`; a changed executable, engine, version or +build then refuses before conversion. A missing provider and a missing signing key are setup failures. +They are distinct from a conversion failure after readiness. + +AWARE 0.126.0 has no generic secret-provisioning facility (issue #448). Provider licensing and +credentials remain a local concern of the separately installed provider. The agent contains no cloud +URL, provider binary, credential or implicit discovery. diff --git a/20-agents/aeco/engineering/model-reference-reader/commands/probe.md b/20-agents/aeco/engineering/model-reference-reader/commands/probe.md new file mode 100644 index 000000000..cd1ed879b --- /dev/null +++ b/20-agents/aeco/engineering/model-reference-reader/commands/probe.md @@ -0,0 +1,13 @@ +# probe + +`probe` returns a bounded summary of the same canonical snapshot `read-model` publishes: Z-up +millimetre bounds, entity/property/relationship counts, exact join coverage, source hash, full provider +fingerprint and canonical-request hash. A cold probe is intentionally expensive because exact geometry +and joins cannot be established without conversion. A warm probe revalidates the signed cache. + +The source is copied into a private immutable staging file and hashed on both sides. The provider sees +only that staged path. A source or executable change at any bracket refuses the run. Execution is +`local` with `destination: null`; remote destinations and external GLB resources are unsupported. + +Bounds come from normalized active-scene geometry, not from names or metadata. Unclaimed geometry is +reported in coverage rather than assigned by guesswork. diff --git a/20-agents/aeco/engineering/model-reference-reader/commands/read-model.md b/20-agents/aeco/engineering/model-reference-reader/commands/read-model.md new file mode 100644 index 000000000..42681a564 --- /dev/null +++ b/20-agents/aeco/engineering/model-reference-reader/commands/read-model.md @@ -0,0 +1,18 @@ +# read-model + +`read-model` publishes five run-owned artifacts. Retrieve each opaque `id` with `aware app artifact`: + +- `geometry` — binary GLB v2, canonical right-handed Z-up millimetres; +- `entities` — stable Revit element identities, exact Category/Family/Type/Level/class and appearance joins; +- `properties` — ordered parameter groups and typed values, including null, empty and unreadable states; +- `relationships` — explicit provider relationships with validated endpoints and hierarchy; +- `manifest` — source/request/provider/signer provenance, hashes, frame and reconciled coverage. + +The GLB is never JSON-encoded. Entity meaning is never inferred from node names or geometry. One entity +may own several appearance nodes; every claimed node has exactly one owner, and unclaimed nodes remain +explicit in coverage. `IfcGUID` is comparable only when the authoritative exact parameter is present +and unique; duplicated values make every duplicate uncomparable. + +The approved artifact is still FloLess/AWARE consumer state, not this cache entry. This reader produces +deterministic authenticated bytes; it does not assign a project UUID, generation, approval or mutable +"latest" handle. diff --git a/20-agents/aeco/engineering/model-reference-reader/manifest.yaml b/20-agents/aeco/engineering/model-reference-reader/manifest.yaml new file mode 100644 index 000000000..bd6667ae4 --- /dev/null +++ b/20-agents/aeco/engineering/model-reference-reader/manifest.yaml @@ -0,0 +1,101 @@ +agent: model-reference-reader +version: 0.1.0 +display-name: Local RVT Reference Reader +description: | + Convert a local Revit project into deterministic, authenticated reference-model artifacts without + importing ownership into AWARE. The reader returns canonical Z-up millimetre GLB geometry plus + separate entity, property and relationship JSON. Revit meaning comes only from explicit provider + metadata and appearance joins; names and geometry are never used to infer it. + + Execution is local-only through an operator-installed trusted provider. AWARE contains no provider, + license, cloud URL or credential. First probe may perform the full conversion; later reads use a + signed content-addressed cache after revalidating every receipt and byte. +stateful: false +vendor: aware-aeco +license: Apache-2.0 +homepage: https://github.com/aware-aeco/aware/tree/main/20-agents/aeco/engineering/model-reference-reader +keywords: [aware, revit, rvt, reference, model, glb, deterministic, local] +provenance: + generated-by: hand-curated + generator-version: 0.126.0 + refined-by: [pawellisowski] +requires: + filesystem: + - read: '*.rvt' +skills: + - provider-and-artifact-contract.md +transport: + cli: + binary: aware-connection-reader +commands: + preflight: + lifecycle: single + category: curated + description: | + Check that the exact local provider, managed AWARE host and model-reader signing key are ready. + This describes the provider but never opens or converts an RVT file. + inputs: + expected-provider-sha256: + type: string + required: false + description: Optional pinned SHA-256 of the complete seven-field provider fingerprint. + outputs: + type: single + schema: + ready: boolean + execution: string + providerFingerprintSha256: string + signerFingerprintSha256: string + probe: + lifecycle: single + category: curated + description: | + Return bounded counts, exact reconciled coverage and canonical geometry bounds for an RVT. + A cold probe performs the full deterministic conversion; a warm probe validates the signed cache. + inputs: + rvt-path: + type: string + description: Absolute path to the local .rvt source. + source-sha256: + type: string + description: Expected SHA-256 of the source bytes, measured before the run. + expected-provider-sha256: + type: string + description: Pinned SHA-256 of the complete seven-field provider fingerprint. + outputs: + type: single + schema: + cache: string + sourceSha256: string + canonicalRequestSha256: string + providerFingerprintSha256: string + frame: object + coverage: object + bounds: object + read-model: + lifecycle: single + category: curated + description: | + Publish five deterministic run-owned artifacts: binary GLB geometry, entities, properties, + relationships and the manifest that authenticates their provenance and exact coverage. + inputs: + rvt-path: + type: string + description: Absolute path to the local .rvt source. + source-sha256: + type: string + description: Expected SHA-256 of the source bytes, measured before the run. + expected-provider-sha256: + type: string + description: Pinned SHA-256 of the complete seven-field provider fingerprint. + outputs: + type: single + schema: + sourceSha256: string + canonicalRequestSha256: string + providerFingerprint: object + providerFingerprintSha256: string + signerFingerprintSha256: string + frame: object + coverage: object + artifacts: object diff --git a/20-agents/aeco/engineering/model-reference-reader/skills/provider-and-artifact-contract.md b/20-agents/aeco/engineering/model-reference-reader/skills/provider-and-artifact-contract.md new file mode 100644 index 000000000..a4980356f --- /dev/null +++ b/20-agents/aeco/engineering/model-reference-reader/skills/provider-and-artifact-contract.md @@ -0,0 +1,14 @@ +# Provider and artifact contract + +Treat the provider as a separately installed local trusted dependency. Configure its absolute regular +executable path with `AWARE_MODEL_REFERENCE_PROVIDER`; never use PATH lookup, a shell command, URL or +committed binary. Configure the AWARE-format signing key locally. Run `preflight`, pin the returned full +provider fingerprint, then call `probe` and `read-model` with the source SHA-256. + +The canonical request, provider fingerprint, source digest and signer trust anchor jointly define a +cache key. Every cache hit verifies its signature, closed receipt, complete file set and every blob +digest. A cache result is reusable conversion evidence, not approval authority. + +Consume semantic records only through their explicit IDs and joins. Preserve property group/order, +units and tagged storage values. Do not derive Revit Category, Family, Type, Level, hierarchy or stable +identity from geometry or names. diff --git a/40-diagrams/substrate-playground.html b/40-diagrams/substrate-playground.html index a300615f0..76f4c43b2 100644 --- a/40-diagrams/substrate-playground.html +++ b/40-diagrams/substrate-playground.html @@ -90,7 +90,7 @@

Prompt — copy back to Claude