From 20e2007c09928a58db840f6728a421b5426e4967 Mon Sep 17 00:00:00 2001 From: Chris Thoburn Date: Sat, 26 Sep 2026 08:21:29 +0000 Subject: [PATCH 1/2] Add RFC: WarpDrive: Virtualized Resources --- text/0000-virtualized-resources.md | 296 +++++++++++++++++++++++++++++ 1 file changed, 296 insertions(+) create mode 100644 text/0000-virtualized-resources.md diff --git a/text/0000-virtualized-resources.md b/text/0000-virtualized-resources.md new file mode 100644 index 0000000000..eb60c577c6 --- /dev/null +++ b/text/0000-virtualized-resources.md @@ -0,0 +1,296 @@ +--- +stage: proposed +start-date: 2026-09-26T00:00:00.000Z +release-date: +release-versions: +teams: + - data +prs: + accepted: +project-link: +suite: +--- + +# WarpDrive: Virtualized Resources + +## Summary + +A new category of resource schema, `kind: 'virtual'`, describes a resource that owns no data of +its own. It holds [pointers](/rfcs/0006-pointer-and-reference-fields.md) to other resources and +exposes their fields through a new field kind, `reflection`. When the cache receives a virtual +resource, each reflected value is written into the cache entry of the resource it belongs to; +when the app reads or edits a reflection, it reads or edits that same entry. A virtual resource +is to the frontend what a materialized view is to a database: a recombination of existing data +that never diverges from its sources. + +## Motivation + +Apps usually model resources one-to-one with the tables behind their API: + +```ts +interface User { + name: string; + address: Address; +} + +interface Address { + street: string; + city: string; + country: string; +} +``` + +APIs do not always follow suit. An endpoint may return a `user-street` built from both: + +```ts +interface UserStreet { + name: string; // from User + street: string; // from Address +} +``` + +Modeled as an ordinary resource, `user-street` is a second, independent copy of `name` and +`street`. When a user edits the street on a `user-street` form, the change is saved and shown +there but not on any `address` rendered elsewhere, and the reverse. The user sees fields they +have seen before, with values that disagree, and correctly concludes the app is broken. Keeping +the copies in sync today means hand-written glue in a handler or in the app. + +The same shape appears without any unusual API: + +- **Tables** want rows and columns flattened from several resources. +- **Forms** edit a few fields from each of several resources and ignore the rest. +- **Domain abstractions** present a concept that spans storage boundaries as a single thing. + +In each case the resources are distinct up to the moment the UI uses them. A virtual resource +codifies that moment in the schema, so WarpDrive can keep one copy of each value and route every +read and write to it. + +Virtual resources differ from projections or partials, which present a subset of a *single* +resource's fields under that resource's identity. A virtual resource assembles fields from +*multiple* resources and has an identity of its own. + +## Detailed design + +### Terminology + +- **Virtual resource**: a resource whose schema has `kind: 'virtual'`. Its cache entry stores + only its identity and its pointers. +- **Source**: a resource a virtual resource points at. +- **Reflection**: a field on a virtual resource that aliases a field on a source. + +### Schema + +```ts +store.schema.registerResource({ + type: 'user-street', + kind: 'virtual', + identity: { kind: '@id', name: 'id' }, + fields: [ + { kind: 'pointer', name: 'theUser', type: 'user' }, + { kind: 'pointer', name: 'theAddress', type: 'address' }, + { kind: 'reflection', name: 'name', source: 'theUser', sourceKey: 'name' }, + { kind: 'reflection', name: 'street', source: 'theAddress', sourceKey: 'street' }, + ], +}); +``` + +A virtual schema is registered like any other resource schema and shares the same type +namespace. It may contain only: + +- an identity field; +- `pointer` and `reference` fields (RFC 6), which name the sources; +- `reflection` fields; +- `derived` and `@local` fields, which already store nothing in the cache. + +Any other field kind is rejected by DEBUG-time schema validation, as is `legacy: true`. Sources +may be PolarisMode or LegacyMode resources. + +### The `reflection` field + +```ts +interface ReflectionField { + kind: 'reflection'; + name: string; + /** The name of a pointer, reference, or to-one reflection on this schema. */ + source: string; + /** The name of the field on the source's schema. */ + sourceKey: string; +} +``` + +A reflection takes on the kind, type, transform, and options of the field it reflects: a +reflection of a `field` with a `date` transform reads as a `Date`, and a reflection of a +relationship reads as that relationship. It cannot reflect an identity field; the source's +identity is available through the pointer. + +`source` may name another reflection when that reflection resolves to a to-one relationship. +This lets a payload identify only the root source: + +```ts +fields: [ + { kind: 'pointer', name: 'theUser', type: 'user' }, + { kind: 'reflection', name: 'theAddress', source: 'theUser', sourceKey: 'address' }, + { kind: 'reflection', name: 'name', source: 'theUser', sourceKey: 'name' }, + { kind: 'reflection', name: 'street', source: 'theAddress', sourceKey: 'street' }, +]; +``` + +Cycles are rejected by schema validation. + +### Receiving a virtual resource + +The payload is an ordinary resource. Reflected values appear under the reflection's name, and +pointers appear as relationships: + +```json +{ + "data": { + "type": "user-street", + "id": "1", + "attributes": { "name": "Chris", "street": "1 Main St" }, + "relationships": { + "theUser": { "data": { "type": "user", "id": "1" } }, + "theAddress": { "data": { "type": "address", "id": "7" } } + } + } +} +``` + +The cache stores `user-street:1` with only its pointers, then upserts `name` into `user:1` and +`street` into `address:7`, exactly as if those values had arrived on those resources. The usual +upsert rules apply: the newest value wins, and fields the payload omits are left alone. If a +source is not yet in the cache, the write creates a partial entry for it, which satisfies the +pointer's presence check. + +Pointers resolve before reflections are routed, so a reflection sourced through another +reflection routes to whichever resource that reflection resolves to after the payload is applied. +A reflected value whose source resolves to `null` has nowhere to go and fails a DEBUG assertion; +in production it is dropped. + +Deriving the pointers from the payload is the job of the request handler or serializer, as with +any other relationship. + +### Reading + +Reading a reflection reads the source's field, so its value and reactivity are the source's: +when `address:7`'s `street` changes from any request, mutation, or other view, every +`user-street` pointing at it updates. When the source is `null`, a reflection reads as +`undefined`. An unloaded pointer target asserts on read as described in RFC 6; an unloaded +reference target makes its reflections read as `undefined`. + +`cache.peek(key)` for a virtual resource returns the assembled resource, with reflected values +under `attributes` or `relationships` as appropriate, so serialization needs no special case. + +### Editing and saving + +An editable virtual record writes each reflection to the source's local state. The change is +visible immediately on every editable view of that source, and `hasChangedAttrs`, +`changedAttrs`, and `rollbackAttrs` on the virtual resource operate on the union of its +reflected fields across its sources. Changes the source has in fields the virtual resource does +not reflect are unaffected. + +Saving a virtual resource is an ordinary request built by the app. Committing it +(`willCommit`/`didCommit`/`commitWasRejected`) moves only its reflected fields into and out of +the in-flight state on each source. A response containing the virtual resource is received as +above; validation errors keyed by a reflection's name are reported on the virtual resource and +on the matching source field. + +A virtual resource can be created client-side from its pointers, for instance to back a form over +records that are already loaded: + +```ts +const form = store.createRecord('user-street', { theUser: user, theAddress: address }); +form.street = '2 Main St'; // a local change on `address` +``` + +Values for reflections passed at creation are written to the sources as local changes. + +### Unloading and deletion + +Unloading a virtual resource removes only its own entry; sources are untouched. Unloading or +deleting a source follows the pointer and reference semantics of RFC 6. Deleting a virtual +resource is not meaningful on its own and asserts; delete the sources instead. + +### Schema DSL + +`@warp-drive/schema-dsl` gains a `@Virtual` class decorator and a `@reflection` field decorator. +The generated type of a reflection is the type of the field it reflects. + +```ts +import { Virtual, pointer, reflection } from '@warp-drive/schema-dsl'; + +@Virtual +export class UserStreet { + @pointer({ type: 'user' }) + declare theUser: User; + + @pointer({ type: 'address' }) + declare theAddress: Address; + + @reflection({ source: 'theUser', sourceKey: 'name' }) + declare name: User['name']; + + @reflection({ source: 'theAddress', sourceKey: 'street' }) + declare street: Address['street']; +} +``` + +### Dependencies and scope + +This RFC depends on the `pointer` and `reference` field kinds from RFC 6. Out of scope: + +- form validation. Virtual resources are a natural home for validation that spans resources, + since they know which fields a form cares about and how they relate, but that belongs in a + separate RFC; +- deriving pointers declaratively from payload attributes such as `userId`; +- nested reflection paths in a single field (`sourceKey: 'address.street'`). + +## How we teach this + +The manual's schema section gains a page, *Virtual Resources*, introduced as "materialized views +for your frontend". It opens with the `user-street` example and the bug it prevents, then shows +the two uses most apps will reach for first: a table row and a form. The page emphasizes that a +virtual resource holds no data, so everything true of a source (freshness, dirtiness, unloading) +is true of its reflections. The resource-schema agent skill lists `kind: 'virtual'` and +`reflection`. API docs ship with the implementation. + +## Drawbacks + +- **Indirection.** A value shown on a virtual record lives elsewhere, which can surprise someone + debugging it. DevTools should show a reflection's source. +- **Writes fan out.** Receiving one virtual resource mutates several entries, and editing one can + make several resources dirty. +- **Partial entries.** Routing values into sources that were not otherwise loaded creates entries + holding only the reflected fields. +- **Field-scoped commits** are new: today a commit applies to a whole entry. +- **Third-party caches** must implement routing, assembly, and field-scoped commits to support + virtual schemas. + +## Alternatives + +- **A handler that splits the response** into its source resources and returns them. It keeps + the cache unchanged but loses the virtual shape: the app must reassemble it for every table row + or form, and saves must be split by hand. +- **`derived` fields over pointers.** They cover reading, but not receiving reflected data from + the API or writing through to the source. +- **Extending `alias` to point across resources.** `alias` renames a field within one entry; + giving it a second meaning muddles both. +- **Projections or partials.** They present a subset of one resource and cannot combine several. +- **Doing nothing.** Apps keep writing sync glue, or live with copies that disagree. + +Prior art: database materialized and updatable views, which recombine tables while keeping one +source of truth; normalized GraphQL caches, which store fragments from any query shape under the +identity each field belongs to. + +## Unresolved questions + +- **`sourceKey` naming.** Elsewhere `sourceKey` names a key in the cache; here it names a field on + the source schema. A different name, such as `field`, may avoid the overlap. +- **Own stored fields.** Whether a virtual resource may hold fields that belong to no source, such + as a value the server computes for the view. +- **Partial source entries.** Whether an entry created only by reflection should count as loaded + for pointers and cache policies, or be marked partial. +- **Identity.** Whether a virtual resource without a server id should derive its identity from its + pointers so that the same combination always yields the same record. +- **Commit granularity.** Whether field-scoped commits are feasible for every cache, or whether + committing a virtual resource should commit its sources whole. From 3f751b7e65ff48e32aae42dbd415071649b7ef5e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=F0=9F=A4=96=20WarpDrive=20RFC=20Sync?= <332362369+warpdrive-bot@users.noreply.github.com> Date: Sat, 26 Sep 2026 08:21:33 +0000 Subject: [PATCH 2/2] RFC #1243: fill prs.accepted and rename file with PR number --- ...virtualized-resources.md => 1243-virtualized-resources.md} | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename text/{0000-virtualized-resources.md => 1243-virtualized-resources.md} (99%) diff --git a/text/0000-virtualized-resources.md b/text/1243-virtualized-resources.md similarity index 99% rename from text/0000-virtualized-resources.md rename to text/1243-virtualized-resources.md index eb60c577c6..94819543a5 100644 --- a/text/0000-virtualized-resources.md +++ b/text/1243-virtualized-resources.md @@ -6,12 +6,12 @@ release-versions: teams: - data prs: - accepted: + accepted: https://github.com/emberjs/rfcs/pull/1243 project-link: suite: --- -# WarpDrive: Virtualized Resources +# Virtualized Resources ## Summary