Skip to content

fix(compilers/openapi): keep keys the source model does not name - #356

Merged
OmarAlJarrah merged 4 commits into
mainfrom
fix/openapi-unknown-key-census
Aug 11, 2026
Merged

fix(compilers/openapi): keep keys the source model does not name#356
OmarAlJarrah merged 4 commits into
mainfrom
fix/openapi-unknown-key-census

Conversation

@OmarAlJarrah

@OmarAlJarrah OmarAlJarrah commented Aug 9, 2026

Copy link
Copy Markdown
Member

Summary

A key the parsed OpenAPI model has no field for reached neither an IR field, an
Unmodeled entry, nor a diagnostic. Twelve object types were probed and all
twelve dropped it in silence, so two documents differing only in such a key
compiled to the same IR — a losslessness failure (invariant 2), and the reason a
misspelt operationId survives review.

The parser already takes the census: every core model records the keys it did not
name. Nothing read it. This adds one shared reader over that census and calls it
at each object's lowering, keying entries the way the rest of the compiler does —
unscoped on an object that lowers to a node with an Unmodeled map of its own,
and scoped by the source path from the carrier down for the objects that ride on
one (openapi:info/contact/slack, openapi:tags/0/color; tags by index, not
name, since two tags spelled alike would otherwise leave one entry).

The two vocabularies are graded apart, because the specifications differ:

  • Inside a JSON Schema, an unrecognized keyword is legal — the specification
    requires implementations to ignore what they do not recognize, and allows such a
    keyword to carry meaning for other tooling. Kept, and announced at info
    under openapi/unknown-schema-keyword: a recorded decision, not a fault.
  • Inside an OpenAPI object, extensions must be prefixed x-, so a key that is
    neither defined nor prefixed is invalid input. Kept all the same, and announced
    at warning under openapi/unknown-object-key.

Warning rather than error for the reason ReservedHeaderName is one: the document
still lowers, and harness.Check stops at the first error diagnostic, which would
hide every later finding in the file and leave any fixture carrying a stray key
unable to reach the invariant checks.

Both carry ReasonOutOfScope. That is a property of the construct — no IR node is
coming for either — while which of the two a key is says something about the
document, and the diagnostic channel is where that belongs.

Three further points:

  • The census is the complement of what the compiler read, not of what the model
    names.
    It runs last at each position and leaves alone any key already
    recorded, so $vocabulary and dependentRequired keep the reasons their own
    readers give them. Keywords no reader leaves a trace of are named in
    annotation.DecidedKeywords: $comment and $dynamicAnchor are deliberate
    drops recorded in the 2020-12 vocabulary table, and an expanded $dynamicRef
    must not also be preserved.
  • Bounded, at MaxUnknownKeys per object, with the remainder reported under
    openapi/unknown-key-budget rather than dropped in silence. The bound applies to
    what the census contributes, after the keys another reader already kept are
    filtered out, so a slot is never spent on an entry the document carries either
    way. The census is also sorted on a copy before use: the library builds it from a
    parallel walk, so its order is neither source order nor stable.
  • How a key is spelled decides neither its entry nor its carrier. A key written
    as an alias is reported by the parser under the name it resolves to while the
    mapping holds an alias node whose own value is the anchor, so a raw search found
    nothing; and a key holding a / spelled the scope of the object that path names,
    making a root info/contact/slack the same entry as the contact object's own
    slack, with the second site to reach the carrier dropping its key in silence.
    The first is looked up by the resolved name, the second escaped as one segment
    per ids.Scope. testdata/openapi/unknown_key_spellings.yaml is their fixture.
  • An entry another construct already holds is announced, under
    openapi/unknown-key-entry-taken, rather than skipped. A parameter and its
    schema are two objects whose entries share one unscoped map on ir.Parameter,
    as are a header and its schema on the ir.Property it lowers to, so a key both
    write is one entry between them. Whether an entry is "already recorded" is now
    decided by its provenance rather than its presence: a reader recording at the
    pointer the census would use has said it better, while an entry pointing
    elsewhere is a collision. Keeping both sides needs a namespace fix(compilers/openapi): read x-* at every object that carries it #345, fix(compilers/openapi): preserve keywords no lowering reads #348 and
    the validation-only reader already publish keys in, so it is filed as openapi: a parameter and its schema share one Unmodeled namespace #396
    rather than settled here. testdata/openapi/unknown_key_carrier_clash.yaml
    witnesses it.
  • A key the raw mapping does not present is announced, under
    openapi/unknown-key-unreachable, rather than passed over. The one class that
    reaches it is a key merged in through a <<: the parser reads a mapping through
    its merge keys and the raw readers here do not, which needs the merge-expanded
    view (internal/nodeview) this package cannot reach today. Filed as openapi: raw-node readers miss keys merged in through a << #395
    every raw-node reader in annotation shares the gap, not only this census.
  • The corpus turned out to contain one instance already. A flow mapping in
    allof-oneof-cooccurrence.yaml split description: named by position, not by target into a description and a stray not by target key; the description is
    now quoted so the fixture says what it meant.

Boundary with the neighbouring work

This is keys the source model does not name at all. Keys the model does name
which the elected lowering never reads are a different gap with a different fix
(#268, #283), and the census cannot see them by construction. Vendor x-*
extensions are out of scope here and covered by #345.

Kept where the carrier exists

Two positions have no carrier to key onto, and lose the key rather than reporting
it. Both are already refused: a request body declaring no content lowers to no
ir.Payload and draws a validation-required-field error, and a
securitySchemes entry naming no mechanism is refused outright (#294). A document
that compiles clean cannot reach either, so neither is a silent loss in a
document anyone can build on — but the census does not reach them, and that is
recorded at both sites in the code.

Not swept, and why

Path items are excluded: the library folds an unrecognized key into the path
item's embedded operations map and reports a type mismatch instead, so there is no
census to read. paths, responses and callbacks are maps where every key is a
valid entry — each embeds a sequenced map — so "unknown" is undefined for them.
The Components Object is not one of them and is swept: it is a fixed-field
struct, the library does take its census, and a key it does not define reached
neither an entry nor a diagnostic. The remaining objects — example, link, encoding, the OAuth flows, discriminator, xml, and
externalDocs below the document root — lower to no node that holds an
Unmodeled map today; the carriers for them are what #345 is adding, so their
census belongs on top of that rather than duplicating its scaffolding. Filed as
#357.

Test plan

  • testdata/openapi/unknown_keys.yaml writes one undeclared key at each of the
    eighteen positions, valued with the object it was written on. Living under
    testdata/ is what runs it through all six oracles — including the two-order
    diff and the round-trip — rather than through assertions alone.
  • TestUnknownKeys_KeptAtEveryObject finds each entry by walking the document's
    value graph and matching on key and value, so an entry written to the wrong
    carrier fails rather than passing because the assertion looked only where it
    expected. Every row was red before the change; reverting the lowering with the
    test kept turns all eighteen red again.
  • TestUnknownKeys_SchemaAndObjectAreGradedApart pins the info/warning split and
    the provenance of both.
  • TestUnknownKeys_WellFormedDocumentRecordsNothing is the control: a document
    writing only what the model names keeps nothing and reports nothing.
  • Unit tests in annotation cover the sort, the bound, the already-recorded skip,
    the decided-keyword skip, an unconvertible value, and the shapes the reader must
    survive rather than panic on.
  • Deleting a key from the fixture reddens the suite, and the full gate passes.

Closes #297

…ey-census

# Conflicts:
#	compilers/openapi/internal/auth/auth.go
#	compilers/openapi/internal/operation/content.go
#	compilers/openapi/internal/operation/operations.go
#	compilers/openapi/meta.go
Three ways an undeclared key still reached the IR in no form at all, each
leaving two documents that differ compiling to the same document.

The Components Object took no census. It was excluded as one of the maps
whose every key is a valid entry, which `paths`, `responses` and a callback
are — each embeds a sequenced map — while Components is a fixed-field struct
beside them, so a key it does not define is as undeclared as one anywhere
else. It is a census site now, keyed under "components" on the carrier #345
already gives its extensions.

A key written as an alias was reported by the parser under the name it
resolves to, while the mapping still holds an alias node whose own value is
the anchor; searching it raw found nothing, and a key with no node kept
nothing and said nothing. RawChildNode now compares the resolved name, which
is the name every caller asks by.

A key holding a "/" spelled the scope of the object that path names, so a
root key "info/contact/slack" was the same entry as the contact object's own
"slack" and the second site to reach the carrier dropped its key in silence.
The key is escaped as one segment, per the rule ids.Scope already records for
the scopes a document chooses the segments of.

Beside them: a key the raw mapping does not present at all is announced under
openapi/unknown-key-unreachable instead of passed over. The one class that
reaches it is a key merged in through a `<<`, which needs the merge-expanded
view this package cannot reach today (#395); the bound now applies to what the
census contributes rather than to keys another reader already kept, and
tagUnknownSites skips a nil entry the way tagExtensions does.
Re-reviewing the branch as it will be merged turned up two more ways a key
reached the IR in no form at all, both silent.

A parameter and its schema are two objects at two pointers whose Unmodeled
entries share one unscoped map, and so are a header and its schema. A key
both of them write is one entry between them, and the census skipped the
loser through the branch meant for a keyword another reader had already said
better. Sameness is now the entry's provenance rather than its presence: a
reader recording at the pointer the census would use has nothing added to it
and nothing said, while an entry held for a construct written elsewhere is
announced under openapi/unknown-key-entry-taken, naming the holder. Keeping
both needs a namespace three merged mechanisms already publish keys in, so
that is #396 rather than a side effect of this census.

RawChildNode returned the first pair spelling a key where the parser reads
the last, which no document could reach until a key could be spelled two
ways: an explicit pair and an aliased one are one key to the parser and two
nodes here, so the reader described a mapping by a value nothing else in the
compiler uses.
@OmarAlJarrah
OmarAlJarrah merged commit 5414c14 into main Aug 11, 2026
1 check passed
@OmarAlJarrah
OmarAlJarrah deleted the fix/openapi-unknown-key-census branch August 11, 2026 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

openapi: a key the source model does not name is dropped with no trace

1 participant