Validate Factory files against the server only (REMOTE-2868) - #15219
Merged
Conversation
Co-Authored-By: Warp <agent@warp.dev>
The bundled schemas and validator were the primary path, so they were also the drift. They become the offline floor: the validator reads the tree's schemaVersion, confirms the server publishes it, and submits the tree to the validation endpoint, falling back only when that cannot be done and always saying which path ran. Also corrects the bundled automation schema, which rejected the authoring aliases the apply path rewrites, and the corpus case that encoded the same mistake. Part of REMOTE-2868. Co-Authored-By: Warp <agent@warp.dev>
This was referenced Aug 17, 2026
Merged
advait-m
marked this pull request as ready for review
August 17, 2026 06:45
advait-m
approved these changes
Aug 17, 2026
The skill shipped a copy of the Factory file format so authoring worked offline. That copy goes stale inside a Warp release, and a stale copy does not degrade gracefully: it rejected the Linear and Slack trigger aliases (teams, projects, states, issues, channels, users, itemUsers) on a tree taken from warp-server's own testdata/valid, producing eleven false rejections. An agent clearing those diagnostics deletes working configuration. Reporting that a tree was not checked is recoverable. Reporting the wrong answer is not. So the six schemas, the restricted YAML reader, the JSON Schema evaluator, and the duplicated semantic checks are gone: 1,624 lines of validator become 374. The client now selects resource files by path, refuses symlinks, uploads bytes, and relays the server's verdict, so it can no longer disagree with the parser about what a document means. Exit 2 is new and distinct: the tree was not checked. It is neither a pass nor a failure, and the skill tells the agent to say exactly that. The corpus drops the 101 offline cases, whose subjects now live beside the parser in warp-server, and covers what the client still owns: which files it uploads, verbatim relay of the server's verdict, deferred resolutions, every route to a missing verdict, symlink refusal, and size caps. A guard fails the build if bundled schemas or a YAML parser reappear. Co-Authored-By: Warp <agent@warp.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The client half of REMOTE-2868. warp-server now owns the Factory file format and validates a tree with the same parser the apply path uses (warp-server#15209). The bundled
factory-filesskill stops carrying a copy of that format entirely.It also repoints REMOTE-2727's spec at the superseding one.
Why the copy had to go, rather than become an offline floor
The plan was to keep the bundled schemas as a fallback for when the server is unreachable. That was wrong, and the evidence is concrete: the schemas merged in warp#15039 reject the Linear and Slack trigger aliases —
teams,projects,states,issues,channels,users,itemUsers— that the apply path rewrites and accepts. Run against a tree taken from warp-server's owntestdata/valid, they produce eleven false rejections on valid input.A stale copy does not degrade gracefully. It reports a valid field as unknown, and an agent trying to reach a clean run resolves that by deleting working configuration. Reporting that a tree was not checked is recoverable and honest. Reporting the wrong answer is neither.
So there is no local fallback, and no local schema to go stale.
Changes
The validator only relays
scripts/validate_factory_files.pygoes from 1,624 lines to 374. Deleted: the six JSON Schema documents, the restricted YAML reader, the JSON Schema evaluator, and the duplicated semantic checks (cron grammar, alias rules, runner shapes, Scorer outcomes, filter conflicts) — all of which live beside the parser in warp-server.What remains: select resource files by path, refuse symlinks, upload the bytes, print what comes back. It never parses YAML, so it cannot disagree with the parser about what a document means. It does not even read
schemaVersion; the endpoint reads that itself and reports an unrecognised one as a diagnostic.--server-rootorWARP_SERVER_ROOTselects a local, staging, or self-hosted root; the default ishttps://app.warp.dev. No credential is required —WARP_API_KEYis forwarded when the environment already has one, as an Oz sandbox does.Three outcomes, not two
012Exit
2is neither a pass nor a failure, andSKILL.mdtells the agent to say exactly that rather than substituting its own reading of the schema. With--json, an unreached verdict carriesvalidated: falseand novalidkey at all, so there is nothing to misread.Documentation
SKILL.mdandreferences/validation.mddrop the offline path and the exact-versus-permissive schema guidance, and gain a section explaining why a local copy must not come back.references/scorers.mdstops describing forward-compatibility rules that were a property of the bundled schemas.Tests
The corpus drops its 101 offline cases — their subjects now live beside the parser — and covers what the client still owns, against a stub server: which files it uploads, verbatim relay of the server's verdict, deferred resolutions, every route to a missing verdict, symlink refusal (in-tree and escaping), size caps, and JSON output shape.
Two guards fail the build if the copy comes back: one in Python and one in Rust, both rejecting
*.schema.jsonunder the skill or a YAML parser in the validator.Verification
End to end, the real validator against the real warp-server handler over HTTP, using warp-server's fixtures:
testdata/valid— the previously-broken triage tree — passes unauthenticated, with all ten Linear/Slack aliases reported as deferred rather than rejected.testdata/invalid/aliases— eleven real server diagnostics with source positions, exit1.2, no verdict claimed.Also: corpus 10/10,
cargo test -p warp --lib ai::skills::bundled5/5,./script/formatclean,git diff --checkclean.Blast radius
No production Rust. The only
.rsfile in this PR isapp/src/ai/skills/bundled_tests.rs. Everything else is the bundled skill tree, its test script, and specs.Sequencing
Merge warp-server#15209 first — the client calls its endpoint.
The two plugin mirrors (claude-code-warp#81, codex-warp#3) are re-pinned to this commit and remain drafts. No plugin release is cut and
MINIMUM_PLATFORM_PLUGIN_VERSIONis untouched: no published plugin ships this skill yet, so there is no stale version to force anyone off.