Skip to content

feat: generate descriptor documents for the order types - #57

Merged
mfw78 merged 5 commits into
developfrom
feat/order-descriptors
Aug 2, 2026
Merged

feat: generate descriptor documents for the order types#57
mfw78 merged 5 commits into
developfrom
feat/order-descriptors

Conversation

@mfw78

@mfw78 mfw78 commented Aug 1, 2026

Copy link
Copy Markdown

Descriptors for all five order types, the generator that derives them, and a Solidity test that checks them against the contracts they describe.

Where they live, and why

descriptors/, at the repository root, mirroring deployments/. That directory already establishes the pattern: machine-readable artifacts consumed by off-chain services, with a README carrying the schema and the rules.

Keeping them out of src/types/ is not just tidiness. forge build, forge fmt and the AST walk this generator itself performs all treat src/ as Solidity, so a JSON tree there is noise to each of them.

descriptors/
  README.md
  overlays/<Handler>.json    # hand-written: name, description, display, links, labels
  <Handler>.json             # generated, canonical bytes

The overlay is the only hand-written part. Everything a consumer decodes with is derived from compiler output, so it cannot drift from the contract without the build noticing.

The spec was wrong about where errors comes from

docs/discovery.md §1.4 said errors is derived "from the handler ABI (every reason error is a declared error)". It is not there. Every reason error in this codebase is declared at file scope, where solc omits it from the contract ABI. The TWAP ABI carries eight errors and not one of them is a reason code; they are the framework wrappers (OrderNotValid, PollTry*) plus the descriptor errors.

So errors is derived from the AST instead, identified as the names referenced as X.selector across the handler's transitive import closure. The closure matters: TWAP's own file references only three selectors, and the ten validation errors (InvalidSpan, InvalidNumParts, …) are raised inside the TWAPOrder library it calls. §1.4 is corrected in this PR.

staticInput.components also comes from the AST, which §1.4 already specified correctly, because the struct never crosses an external ABI boundary.

A wrong derivation that surfaced a real bug

The obvious way to derive offchainInput.required is "does the handler declare PollNeedsOffchainInput". That is unsound, and it produced required: false for all five handlers.

GoodAfterTime decodes its buy amount from offchainInput (abi.decode(offchainInput, (uint256))) and never declares that error. It is the one handler where required is genuinely true. So the field is instead derived from whether generateOrder actually reads its offchainInput parameter, which is exactly right for all five: the other four leave the parameter unnamed.

That also means GoodAfterTime currently mis-signals at runtime. Polled with empty offchainInput it does not return NEEDS_INPUT; the decode reverts and is mapped to TRY_NEXT_BLOCK, telling a monitoring service to retry next block forever for a condition that no amount of waiting resolves. Filed separately as #58; not fixed here, since this PR does not change handler behaviour.

Testing without a JS runtime

Two layers, neither needing node_modules.

dev/gen-descriptors.py --check          # committed bytes match what the pipeline emits
forge test --match-contract DescriptorDoc

The generator is standard library only, plus cast sig so selector hashing is foundry's own rather than a reimplementation. --check runs in CI and catches hand-edits and staleness; verified to exit 1 on a one-character edit.

The Solidity test asserts on-chain facts, which is why it is Solidity:

  • every errors key equals bytes4(keccak256("<name>()")), so a key and its name cannot disagree. This is the failure that matters most, since consumers key on the selector and display the name;
  • selectors are distinct, because a duplicate silently drops an entry;
  • component count matches the ABI-encoded struct width;
  • a provoked verdict's reasonCode is present in the descriptor: the divergence check §1.3 asks consumers to run, applied to ourselves.

Each assertion was mutation-checked against a deliberately corrupted document:

flip one selector key            SelectorMatchesDeclaredName:        FAIL
drop a staticInput component     ComponentCountMatchesEncodedWidth:  FAIL
remove the observed error        ObservedReasonCodeIsDocumented:     FAIL
restored                                                             6 passed

parseJsonKeys is declared as a local interface: the forge binary supports it but the vendored forge-std predates it, and one cheatcode does not justify bumping the submodule. fs_permissions is scoped read-only to ./descriptors.

Second commit: the document carries no handler identity

§1.3 required handler.{chainId,address} to match the contract the descriptor was resolved from. On the deterministic deployment path that is unsatisfiable.

OrderDescriptor takes the digest as a constructor argument and stores it immutable, so it is part of the initcode. deploy_ProdStack uses new X{salt: ...}, and CREATE2 derives the address from keccak256(0xff ++ deployer ++ salt ++ keccak256(initCode)). So address depends on digest depends on document bytes depends on address. No fixed point exists, and a salt search does not help, since the salt does not cancel the digest's contribution. (deploy_OrderTypes uses plain CREATE, where the address is a function of deployer and nonce, so the cycle is specific to the path intended for production.)

Only the address closes the loop; chain id is chosen up front. But chain id earns nothing either: every field in the document is chain- and deployment-independent, so requiring it would force one publication per chain for byte-identical content and defeat content addressing. Both are removed.

Binding comes from resolution, which is where it already came from. A consumer reads descriptorCommitment() from a specific contract and verifies fetched bytes against that digest, so a document is that contract's descriptor exactly when it hashes to what the contract returns. An identity field restates what the commitment proves.

This makes the generated documents complete rather than partial: one digest per handler version, publishable before a chain or address is chosen, valid for every deployment. The first commit's README described the missing field as a gap awaiting deployment; that was wrong, and this corrects it.

The merkle payload document in §3.3 keeps its chainId and per-order handler addresses: that document enumerates orders for an owner on a chain, so identity is load-bearing there and creates no cycle.

No descriptor bytes change, since the generator never emitted the field.

There is still no published descriptor-v1 JSON Schema to validate against; §1.3 expects one separately.

Third commit: the descriptor-v1 JSON Schema

§1.3 referred to a schema as "published separately", so a consumer had no way to reject a malformed document short of reimplementing the rules from prose. It now lives at descriptors/schema/descriptor-v1.json, and CI validates every document against it plus the schema against the draft 2020-12 metaschema.

It also makes the second commit's decision enforceable rather than documented: a not clause rejects any document carrying handler, so the CREATE2 cycle cannot be reintroduced by accident.

Checked against deliberately malformed documents rather than assumed to work:

unmodified TWAP                 PASS
+ handler identity              FAIL
uppercase selector key          FAIL
non-selector key                FAIL
missing display                 FAIL
non-canonical component type    FAIL
wrong version                   FAIL
extensions carrying free data   PASS   (deliberately open)

§1.3 also called for a content-addressed $id. That has the same self-reference problem as handler.address: a digest over the schema cannot be embedded in the schema it describes, since the value would depend on bytes that depend on the value. The $id is a stable identifier; publishing at a content-addressed location is fine so long as the address is carried by whatever references the schema, never inside it.

The schema sits under schema/ rather than beside the documents so descriptors/*.json keeps meaning "documents". Validating that glob would otherwise include the schema, which does not describe itself.

Validator: any draft 2020-12 implementation. CI uses check-jsonschema, a single pipx install, which also checks the schema against the metaschema. Locally, nix-shell -p check-jsonschema. No JS toolchain, and nothing added to the generator, which stays standard library only.

Fourth commit: tersen the prose

The markdown across this branch argued its case rather than stating it. Rationale belongs here, in the pull request, not in a document someone consults for facts.

Cut from descriptors/README.md, docs/discovery.md §1.3 and the Caller Identity section of docs/architecture.md: rhetorical framing ("two consequences follow, both intended", "the upshot is", "which is the accepted cost"), restatements of points already made, and asides about what a reader would be wrong to conclude. Facts, tables, commands and rules are unchanged. Net 30 lines removed.

It also fixed two errors in that prose, both introduced by the third commit:

  • the README referred to "the CREATE2 cycle described below" from a section above it;
  • the README still claimed no descriptor-v1 schema existed and that §1.3 expected one separately, which the third commit had already made false.

Fifth commit: reason errors must be nullary

All 22 reason errors are nullary today, and the protocol cannot carry anything else.

A handler raises one by passing X.selector to a framework wrapper whose payload is bytes4:

require(data.sellAmount > 0, IConditionalOrder.OrderNotValid(ZeroAmount.selector));

The reason error is a tag; it is never constructed. Arguments would be discarded at the raise site and could not be recovered from poll, getManifestPage, or even tryGenerateOrder, since that returns the wrapper's revert data and the reason error was never thrown.

Nothing prevented declaring one, though. The generator would have computed the selector over the full signature and emitted it, and the Solidity test would then have failed comparing it against keccak256("<name>()"), reporting a selector mismatch rather than the real problem.

The generator now rejects it explicitly. Verified against an injected StrikeNotReached(uint256), which exits 1 with:

StopLoss: reason error StrikeNotReached declares parameters (uint256);
reason errors must be nullary, since only the selector is propagated

Structured data travels in the wrapper instead, as waitUntil does for the timed verdicts. Recorded in docs/discovery.md §1.3.

Verification

172 tests pass (166 plus 6 new), forge build and forge fmt --check clean, --check reports all five current, and all five validate against the schema.

Rebased onto develop after #56 merged. The descriptor documents are byte-identical across that change (--check passes against a rebuilt AST), which is the expected result: removing sender from the generation surface touches neither the staticInput struct nor any reason error.

Adds descriptors for all five handlers, the generator that derives them, and a
Solidity test that checks them against the contracts they describe.

Descriptors live in `descriptors/`, mirroring `deployments/`: a root-level
directory of machine-readable artifacts consumed by off-chain services, with a
README carrying the schema and rules. Keeping them out of `src/` is not only
tidiness; `forge build`, `forge fmt` and the AST walk all treat `src/` as
Solidity, and a JSON tree there is noise to each of them.

Only the author overlay is hand-written: name, description, display template,
links, error labels. Everything a consumer decodes with is derived from
compiler output, so it cannot drift from the contract silently.

Two facts had to come from the AST rather than the ABI, and the second
contradicts what the spec said:

- `staticInput.components`, because the struct never crosses an external ABI
  boundary. This was already specified.
- `errors`, because every reason error here is declared at file scope, where
  solc omits it from the contract ABI. `docs/discovery.md` 1.4 said these come
  from the handler ABI; the TWAP ABI carries eight errors, none of which is a
  reason code. Reason errors are instead identified as those referenced as
  `X.selector` across the handler's transitive import closure, which is what
  picks up the validation errors raised inside `TWAPOrder`. Section 1.4 is
  corrected.

`offchainInput.required` is derived from whether `generateOrder` reads its
`offchainInput` parameter. Deriving it from `PollNeedsOffchainInput`, the
obvious signal, is unsound: `GoodAfterTime` decodes its buy amount from
`offchainInput` yet never declares that error, so the obvious derivation
reported `required: false` for the one handler where it is true.

Testing is Solidity plus a regeneration check, no JS runtime. The test asserts
each key equals `bytes4(keccak256("<name>()"))`, that selectors are distinct,
that the component count matches the ABI-encoded struct width, and that a
provoked verdict's `reasonCode` is documented, which is the divergence check
1.3 asks consumers to run. Each assertion was checked against a deliberately
corrupted document and fails when the descriptor is wrong.

`parseJsonKeys` is declared locally: the `forge` binary supports it but the
vendored `forge-std` predates it, and one cheatcode does not justify bumping
the submodule.

These documents are not publishable as they stand. `handler.{chainId,address}`
is stamped at deployment, which is what makes the digest per-deployment, and
there are no deployments.
@mfw78
mfw78 force-pushed the feat/order-descriptors branch from 4a29db9 to b0f62e1 Compare August 1, 2026 04:55
mfw78 added 3 commits August 1, 2026 06:27
The document carried `handler.{chainId,address}`, and 1.3 required it to match
the contract the descriptor was resolved from. That cannot be satisfied on the
deterministic deployment path.

`OrderDescriptor` takes the digest as a constructor argument and stores it
immutable, so it is part of the initcode. `deploy_ProdStack` deploys with
`new X{salt: ...}`, and CREATE2 derives the address from
`keccak256(0xff ++ deployer ++ salt ++ keccak256(initCode))`. The address
therefore depends on the digest, the digest on the document bytes, and the
bytes on the address. There is no fixed point to find, and a salt search does
not help because the salt does not cancel the digest's contribution.
`deploy_OrderTypes` uses plain CREATE, where the address is a function of
deployer and nonce, so the cycle is specific to the path intended for
production.

Only the address closes the loop; chain id is chosen before deployment. But
chain id earns nothing either: every field in the document is chain- and
deployment-independent, so requiring it would force one publication per chain
for byte-identical content and defeat content addressing. Both are removed.

Binding comes from resolution, which is where it already came from. A consumer
reads `descriptorCommitment()` from a specific contract and verifies the
fetched bytes against that digest, so a document is that contract's descriptor
exactly when it hashes to what the contract returns. An identity field inside
the document restates what the commitment proves.

The result is that the generated documents are complete rather than partial:
one digest per handler version, publishable before a chain or an address is
chosen, valid for every deployment. The previous commit described their missing
`handler` field as a gap waiting on a deployment, which was wrong.

The merkle payload document in 3.3 keeps its `chainId` and per-order `handler`
addresses. That document enumerates orders for an owner on a chain, so the
identity is load-bearing there and creates no cycle.

No descriptor bytes change: the generator never emitted the field.
1.3 referred to a descriptor-v1 schema as "published separately", which meant a
consumer had no way to reject a malformed document short of reimplementing the
rules from prose. The schema now lives at
`descriptors/schema/descriptor-v1.json` and CI validates every document against
it, and the schema against the draft 2020-12 metaschema.

It also makes the previous commit's decision enforceable rather than merely
documented. A document carrying `handler` is rejected by a `not` clause, so the
CREATE2 cycle cannot be reintroduced by accident: the digest is a constructor
argument and therefore part of the initcode that fixes the address, so a
document naming its own address could never be committed to.

Checked against deliberately malformed documents rather than assumed to work:

  unmodified TWAP                 PASS
  + handler identity              FAIL
  uppercase selector key          FAIL
  non-selector key                FAIL
  missing display                 FAIL
  non-canonical component type    FAIL
  wrong version                   FAIL
  extensions carrying free data   PASS   (deliberately open)

1.3 also described the schema as having a content-addressed `$id`. That has the
same self-reference problem: a digest over the schema cannot be embedded in the
schema it describes, since the value would depend on bytes that depend on the
value. The `$id` is a stable identifier, and publishing at a content-addressed
location is fine so long as the address is carried by whatever references the
schema rather than by the schema itself.

The schema lives under `schema/` rather than beside the documents so that
`descriptors/*.json` continues to mean "documents"; validating that glob against
the schema would otherwise include the schema itself, which does not describe
itself.

`extensions` is left open, since the point of the field is to carry what the
schema does not anticipate.
The markdown added across this branch argued its case rather than stating it.
Rationale belongs in the pull request, not in a document a reader consults for
facts.

Cut from `descriptors/README.md`, `docs/discovery.md` 1.3, and the Caller
Identity section of `docs/architecture.md`: rhetorical framing ("two
consequences follow, both intended", "the upshot is", "which is the accepted
cost"), restatements of a point already made, and asides about what a reader
would be wrong to conclude. The facts, tables, commands and rules are unchanged.

Two errors in the same prose, both introduced by the previous commit:

- The README referred to "the CREATE2 cycle described below" from a section
  above it.
- The README still said no descriptor-v1 schema existed and that 1.3 expected
  one separately, which the previous commit had already made false.

No descriptor bytes change.
@mfw78
mfw78 force-pushed the feat/order-descriptors branch from 5e8ad89 to a3eb98a Compare August 2, 2026 00:10
All 22 reason errors are nullary today, and the protocol cannot carry anything
else. A handler raises one by passing `X.selector` to a framework wrapper whose
payload is `bytes4`, so the error is a tag and is never constructed. Arguments
would be discarded at the raise site and could not be recovered from `poll`,
`getManifestPage` or `tryGenerateOrder`, since the last returns the wrapper's
revert data rather than the reason error's.

Nothing prevented declaring one. The generator would have computed the selector
over the full signature and emitted it, and the Solidity test would have failed
comparing it against `keccak256("<name>()")`, reporting a selector mismatch
rather than the actual problem.

The generator now rejects it by name and parameter list. Verified against an
injected `StrikeNotReached(uint256)`, which exits 1 with:

    StopLoss: reason error StrikeNotReached declares parameters (uint256);
    reason errors must be nullary, since only the selector is propagated

Structured data travels in the wrapper instead, as `waitUntil` does for the
timed verdicts. Recorded in `docs/discovery.md` 1.3.
@mfw78
mfw78 merged commit 0bd10a3 into develop Aug 2, 2026
1 check passed
@mfw78
mfw78 deleted the feat/order-descriptors branch August 2, 2026 11:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant