Skip to content

Set webhook binding settings from job contract attach - #56

Open
rawwerks wants to merge 3 commits into
feat/program-aware-quotefrom
feat/job-contract-settings
Open

rawwerks wants to merge 3 commits into
feat/program-aware-quotefrom
feat/job-contract-settings

Conversation

@rawwerks

@rawwerks rawwerks commented Oct 7, 2026 •

Copy link
Copy Markdown

I'm an AI agent (Claude Code) opening this PR on behalf of Ray (@rawwerks). He asked for this change and approved opening the PR, but he did not write the text below.

Stacked on #53, which is stacked on #51. Retarget to main after they merge.

Why

Once a GitHub App's repository access is configured (on GitHub), everything the web app can do to a job should be possible from the CLI. #51 lets job create set a webhook's model, reasoning effort, repository, branch and commit output. But an existing webhook job couldn't be changed or inspected from the CLI:

  • job contract attach sent only program_ref and refused --model.
  • Re-attaching a bound program reset its settings to defaults.
  • job contract list showed none of a binding's settings.

The web app changes these by re-binding the same program with new settings: POST /triggers/{id}/contracts, including files when a new revision is deployed from the editor.

What changes

job contract attach JOB_ID OWNER/SLUG@REV gains these options for webhook jobs. Other job types refuse them after the job is read.

Option Sends
--model, --reasoning-effort model, reasoning_effort
--repo OWNER/NAME[@BRANCH] repository_url + repository_branch (the repository runs read as context)
--commit-output OWNER/NAME output (type: commit); it must be that repository, and output.repository is the repository's URL
--input KEY=VALUE, --inputs-file FILE inputs, merged over the saved inputs
--environment ENV environment
--file [NAME=]PATH files (UTF-8, base64); replaces the stored file set, as an editor re-deploy does. At most 20 files, 5 MiB each, 10 MiB in total
--clear-repo, --clear-commit-output, --clear-input KEY, --clear-files clears the named setting
--allow-reset accepts resetting stored files and environment that the service can't report
--replace OWNER/SLUG@REV replace_program_ref, to move the binding to another revision

Re-attaching and changing a bound program. The job is always read first.

  • Another job type, or a plain re-attach of a bound webhook program, sends just program_ref. The service treats re-attaching a bound program as a no-op.
  • Changing a bound webhook binding sends only the options given:
    • When the service reports each binding's environment, the CLI re-binds the same reference (replace_program_ref equal to program_ref) with just the changed fields, using null for clears. The service keeps the rest, including stored files.
    • A service that doesn't report them would reset stored files and environment on any re-bind. So there the CLI refuses unless the user gives --file/--clear-files and --environment, or accepts the reset with --allow-reset. A plain re-attach of a bound program is refused there too.
  • --repo naming the saved repository keeps its branch.
  • A different --repo while the saved commit output goes elsewhere is refused until --commit-output or --clear-commit-output is also given.
  • --commit-output takes OWNER/NAME; the service picks the branch.
  • --file names must be what the service stores: 1 to 200 characters of [A-Za-z0-9._-], with no leading dot and no ...

--replace onto another revision is a full replace on the service. The CLI sends the replaced binding's model, effort, repository, inputs and environment. Stored files can't be re-sent (only their metadata is listed), so when the old binding has any, --file, --clear-files or --allow-reset is required. --replace onto a program that's already bound is refused locally (the service refuses it too); change that binding in place instead.

The attach plan (--preview or CONFIRMATION_REQUIRED) quotes the hold for the binding as it will run: GET /run/quote with the program, job_type and the effective model, effort, environment and repository. The quote is advisory.

job contract list reports for each binding:

  • effective_model, rev_id, bound_at and is_platform_default;
  • a run_configuration holding reasoning_effort, inputs (cost-named inputs in input_entries), context_repositories, output, environment and stored_files ({name, size, sha256}; content is never shown).

Human output adds a settings: line that names inputs and files, not their values.

Also:

  • An interrupt during a plan's advisory quote (run submit, program draft / program save, contract attach) now stops the command in both ports instead of being swallowed.
  • run quote's help is reworded briefly.
  • Size budget, raised deliberately: the budget for cli service operations goes from 98,304 to 106,496 bytes (96 to 104 KiB) to fit the new job options. That's a policy knob; say if you'd rather trim help text instead.
  • The trigger-contracts response schema now names program_ref; it said programRef.

Parity notes:

  • The web app can't remove a single stored file either. It re-sends the whole set, which is what --file does.
  • Settings take effect for deliveries admitted after the change. A live webhook can be edited on a service that allows live edits.
  • --input sends the full resulting input set, so a concurrent edit of the same binding's inputs can be overwritten.

Evidence

Local, on macOS with Python 3.10.21, Rust from rust-toolchain.toml and Bun 1.4.2. These gates pass:

  • shared-contracts, service-help, public-surface-files and service-operations-corpus
  • differential-conformance and service-coverage (0 routes unaccounted)
  • rust-format and rust-clippy
  • service-operations-rust: 1162 cases, 0 failures (1170 on the integration branch)
  • bun-typecheck
  • service-operations-bun: 1162 cases, 0 failures (1170 on the integration branch)

Rust (prose-runner-core lib, service_jobs) and Bun (service-jobs, service-runs) unit tests also pass. The full rust-tests gate is load-flaky on this machine in unrelated process-timing tests; GitHub CI is authoritative for it.

Corpus: 48 new attach and list cases. Every attach exchange pins its exact request: method, path, query, Authorization presence, and expectedBody including the null clears. The cases cover:

  • the server-merge, older-service and unbound paths;
  • clears and files, --replace, and every local refusal;
  • the plan quote, including GitHub job types (repositories=1);
  • the read-back fields.

Every expected output was accepted only where the independently written Rust and Bun ports produced identical bytes.

Review: two independent reviews read the diff. The first, before this was opened, found:

  • a settings-loss bug in cross-revision --replace;
  • a commit-output URL difference between the ports;
  • interrupt handling, error-text and check-order differences between the ports.

All of them are fixed and pinned by cases. The second, after it was opened, found two blockers, both fixed in the second commit:

  • re-attaching to a non-webhook job sent replace_program_ref;
  • an older service could silently reset stored files and environment.

Its re-review confirmed those fixes and found no new blockers. The third commit closes its remaining points:

  • shared cases for the --file count limit, non-UTF-8 content, the 409 for an already-bound --replace target, and a live-binding refusal passing through;
  • a bare program_ref when nothing changes;
  • --allow-reset named in the webhook-only refusal;
  • one check order in both ports.

The 5 MiB and 10 MiB file limits are covered by unit tests in both ports rather than shared cases, which would need multi-megabyte fixtures.

Staging gate

Passed on a staging deployment that carries the matching service changes. The gate ran at the integration commit after this PR's second commit. The third commit only adds cases, the bare re-attach when nothing changes and refusal texts, none of which are on the gate's path.

Sequence:

  1. Created two test-mode GitHub webhook jobs (issue fix, PR review) with no program or repository.
  2. Attached the official examples with --repo …@main --reasoning-effort …. The preview quoted the hold, and the binding runs on the job-default model.
  3. Read the bindings back with contract list.
  4. Added --commit-output to the bound issue-fix binding. Only the output was sent, and the read-back shows the repository and effort kept, so the service-side merge works.
  5. Switched both jobs to live, then added GitHub webhooks on the canary. The ping returned 200.
  6. Issue: one live run, which fixed it, posted its report as a comment and pushed its commit branch.
  7. Pull request: one live run, which reviewed the full diff at the head checkout and posted the review as a comment.
  8. Cleaned up: the webhooks were removed, both jobs deleted with the CLI, and the issue and pull request closed.

Redactions: the staging host, the inbound endpoint capability URLs, the webhook secret, job and run IDs, the canary repository name and local paths are redacted. Nothing else is edited.

Transcript
# prose-cli integration/pending-prs @ 58ef6dc, dev-endpoint build (cargo build --release --locked -p prose-cli --features dev-endpoint), 2026-10-07T17:28:48Z

$ prose --version
prose 0.1.0 (rust)
[exit 0]

$ prose cli auth status --json
{"interaction":"cli.auth_status","operation":"auth.status","problem":null,"result":{"authenticated":true,"credentialSource":"store"},"schema":"openprose.service-operation/1"}
[exit 0]

$ prose --output json cli job create --spec-file - --yes   # stdin: {"type": "webhook", "name": "cli-parity-gate issue-fix", "delivery_mode": "test", "receiver": {"profile": "github", "events": ["issues.opened"], "authors": "trusted"}, "receiver_secret": "<redacted>", "reply": {"type": "github-comment"}}
{"interaction":"job.deploy","operation":"job.create","problem":null,"result":{"endpoint":"/webhooks/inbound/<redacted>","endpoint_url":"<staging>/webhooks/inbound/<redacted>","job":{"context_repository_branch":null,"context_repository_full_name":null,"created_at":1791394131538,"created_at_iso":"2026-10-07T17:28:51.538Z","id":"<issue-job>","interval_seconds":null,"mode":null,"name":"cli-parity-gate issue-fix","repository_branch":null,"repository_full_name":null,"repository_id":null,"type":"webhook","url":null},"status":{"active":true,"configured":true,"counts":{"awaiting_billing":0,"cancelled":0,"completed":0,"failed":0,"queued":0,"running":0},"delivery_mode":"test","last_error":null,"last_event_at":null,"last_event_at_iso":null,"last_run_id":null,"receiver_secret_configured":true,"reply_secret_configured":false,"secret_rotated_at":1791394132906,"secret_rotated_at_iso":"2026-10-07T17:28:52.906Z"}},"schema":"openprose.service-operation/1"}
[exit 0]

$ prose --output json cli job create --spec-file - --yes   # stdin: {"type": "webhook", "name": "cli-parity-gate pr-review", "delivery_mode": "test", "receiver": {"profile": "github", "events": ["pull_request.opened", "pull_request.reopened", "pull_request.ready_for_review", "pull_request.synchronize"], "authors": "trusted"}, "receiver_secret": "<redacted>", "reply": {"type": "github-comment"}}
{"interaction":"job.deploy","operation":"job.create","problem":null,"result":{"endpoint":"/webhooks/inbound/<redacted>","endpoint_url":"<staging>/webhooks/inbound/<redacted>","job":{"context_repository_branch":null,"context_repository_full_name":null,"created_at":1791394133477,"created_at_iso":"2026-10-07T17:28:53.477Z","id":"<pr-job>","interval_seconds":null,"mode":null,"name":"cli-parity-gate pr-review","repository_branch":null,"repository_full_name":null,"repository_id":null,"type":"webhook","url":null},"status":{"active":true,"configured":true,"counts":{"awaiting_billing":0,"cancelled":0,"completed":0,"failed":0,"queued":0,"running":0},"delivery_mode":"test","last_error":null,"last_event_at":null,"last_event_at_iso":null,"last_run_id":null,"receiver_secret_configured":true,"reply_secret_configured":false,"secret_rotated_at":1791394134283,"secret_rotated_at_iso":"2026-10-07T17:28:54.283Z"}},"schema":"openprose.service-operation/1"}
[exit 0]

$ prose --output json cli program show openprose/github-issue-fix
(program show result trimmed: openprose/github-issue-fix@7be66464e1de35fc, a public example program)
[exit 0]

$ prose cli job contract attach <issue-job> openprose/github-issue-fix@7be66464e1de35fc --repo OWNER/canary@main --reasoning-effort high --preview
Preview: Attach a pinned program to a job.
Summary: program openprose/github-issue-fix@7be66464e1de35fc; reasoning effort high
Effect: attaches a program to the job
Hold: $0.11 (set aside from the wallet while the run is live; not its price)
Nothing was changed.
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

$ prose cli job contract attach <issue-job> openprose/github-issue-fix@7be66464e1de35fc --repo OWNER/canary@main --reasoning-effort high --yes
Attached openprose/github-issue-fix@7be66464e1de35fc to job <issue-job>.
  settings: reasoning effort high; repository https://github.com/OWNER/canary@main
List the job's contracts with `cli job contract list <issue-job>`.
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

$ prose cli job contract attach <pr-job> openprose/github-pr-review@b5fe86a550313fd8 --repo OWNER/canary@main --reasoning-effort medium --yes
Attached openprose/github-pr-review@b5fe86a550313fd8 to job <pr-job>.
  settings: reasoning effort medium; repository https://github.com/OWNER/canary@main
List the job's contracts with `cli job contract list <pr-job>`.
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

$ prose cli job contract list <issue-job>
openprose/github-issue-fix@7be66464e1de35fc enabled=true model=gpt-6-luna
  settings: reasoning effort high; repository https://github.com/OWNER/canary@main
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

$ prose cli job contract list <pr-job>
openprose/github-pr-review@b5fe86a550313fd8 enabled=true model=gpt-6-luna
  settings: reasoning effort medium; repository https://github.com/OWNER/canary@main
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

$ prose cli job contract attach <issue-job> openprose/github-issue-fix@7be66464e1de35fc --commit-output OWNER/canary --yes
Attached openprose/github-issue-fix@7be66464e1de35fc to job <issue-job>.
  settings: commit output https://github.com/OWNER/canary
List the job's contracts with `cli job contract list <issue-job>`.
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

$ prose --output json cli job contract list <issue-job>
{"interaction":"job.contracts","operation":"job.contract.list","problem":null,"result":{"contracts":[{"bound_at":1791394151181,"bound_at_iso":"2026-10-07T17:29:11.181Z","effective_model":"gpt-6-luna","enabled":true,"is_platform_default":false,"model":"gpt-6-luna","program_ref":"openprose/github-issue-fix@7be66464e1de35fc","program_slug":"github-issue-fix","rev_id":"7be66464e1de35fc","run_configuration":{"context_repositories":[{"branch":"main","url":"https://github.com/OWNER/canary"}],"output":{"repository":"https://github.com/OWNER/canary","type":"commit"},"reasoning_effort":"high"}}],"max_contracts":5},"schema":"openprose.service-operation/1"}
[exit 0]

$ prose cli job update <issue-job> --spec-file /tmp/thr-ukar/live.json --yes
Job <issue-job> (webhook)
  name: cli-parity-gate issue-fix
  created: 2026-10-07T17:28:51.538Z
  active: true
  delivery mode: live
  secret rotated: 2026-10-07T17:28:52.906Z
  runs: none yet
Next: prose cli job deliveries <issue-job>
Next: prose cli job contract list <issue-job>
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

$ prose cli job update <pr-job> --spec-file /tmp/thr-ukar/live.json --yes
Job <pr-job> (webhook)
  name: cli-parity-gate pr-review
  created: 2026-10-07T17:28:53.477Z
  active: true
  delivery mode: live
  secret rotated: 2026-10-07T17:28:54.283Z
  runs: none yet
Next: prose cli job deliveries <pr-job>
Next: prose cli job contract list <pr-job>
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

$ prose cli job deliveries <issue-job>
2 2026-10-07T17:29:55.383Z admitted test=false runs=[{"program_ref":"openprose/github-issue-fix@7be66464e1de35fc","run_id":null,"status":"claimed"}]
1 2026-10-07T17:29:42.486Z handshake test=false runs=[]
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

$ prose cli job deliveries <pr-job>
2 2026-10-07T17:30:05.248Z admitted test=false runs=[{"program_ref":"openprose/github-pr-review@b5fe86a550313fd8","run_id":null,"status":"claimed"}]
1 2026-10-07T17:29:42.587Z handshake test=false runs=[]
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

$ prose cli job deliveries <issue-job>
2 2026-10-07T17:29:55.383Z admitted test=false runs=[{"program_ref":"openprose/github-issue-fix@7be66464e1de35fc","run_id":"<run-issue>","status":"completed"}]
1 2026-10-07T17:29:42.486Z handshake test=false runs=[]
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

$ prose cli job deliveries <pr-job>
2 2026-10-07T17:30:05.248Z admitted test=false runs=[{"program_ref":"openprose/github-pr-review@b5fe86a550313fd8","run_id":"<run-pr>","status":"completed"}]
1 2026-10-07T17:29:42.587Z handshake test=false runs=[]
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

$ prose cli job show <issue-job>
Job <issue-job> (webhook)
  name: cli-parity-gate issue-fix
  created: 2026-10-07T17:28:51.538Z
  active: true
  delivery mode: live
  last event: 2026-10-07T17:29:55.383Z
  secret rotated: 2026-10-07T17:28:52.906Z
  last run: <run-issue>
  runs: completed 1
Next: prose cli job deliveries <issue-job>
Next: prose cli job contract list <issue-job>
Next: prose cli run show <run-issue>
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

# GitHub side (canary repository)
$ gh issue create  -> issue #1 'CLI parity gate: add a one-line note to the README'
$ gh pr create     -> pull request #2 'CLI parity gate: README pointer'
issue #1 comments: [{"first_lines":"**OpenProse automated run** `<run-issue>` /  / # Issue fix report /  / - **Issue:** #1 — CLI parity gate: add a one-line note to the README / - **Repository:** `OWNER/canary`","user":"<staging GitHub App>"}]
pull request #2 comments: [{"first_lines":"**OpenProse automated run** `<run-pr>` /  / # PR #2 — CLI parity gate: README pointer / ","user":"<staging GitHub App>"}]
branches: ["cli-parity-gate-pr","main","<fix-branch>"]

$ prose cli job delete <issue-job> --yes
Deleted job <issue-job>.
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

$ prose cli job delete <pr-job> --yes
Deleted job <pr-job>.
[stderr] OpenProse (custom endpoint <staging>)
[exit 0]

# cleanup: GitHub webhooks deleted; jobs deleted above; canary issue #1 and pull request #2 closed; the run's fix branch kept as evidence.

🤖 Generated with Claude Code

rawwerks and others added 3 commits October 7, 2026 12:59
job contract attach gains --model, --reasoning-effort, --repo,
--commit-output, --input, --inputs-file, --environment, --file,
--clear-repo, --clear-commit-output, --clear-input, --clear-files and
--replace for webhook jobs; other job types refuse them after the job is
read. Re-attaching a bound program changes only the options given: on a
service that reports each binding's environment it re-binds the same
reference with just the changed fields (null clears); on an older
service it merges the saved settings and warns that stored files and
environment may be dropped. --replace onto another revision is a full
replace that carries the saved settings and environment (stored files
must be given again with --file); --replace onto a program that is
already bound is refused. The attach plan quotes the hold for the
binding as it will run, with the job's type.

job contract list reports each binding's saved settings, effective
model, revision and bind time. Interrupts during a plan's advisory quote
now stop the command. run quote's help is reworded briefly to keep the
published manifest within its size budget, and the contracts response
schema names program_ref.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Address a second review of job contract attach. The job is always read
first: another job type, and a plain re-attach of a bound webhook
program, send only program_ref. On a service that does not report stored
files and environment, re-binding a bound webhook program is refused
unless the files and environment are given or --allow-reset accepts the
reset; --replace needs --file, --clear-files or --allow-reset when the
old binding has stored files. The stderr notes and the live hint are
gone in favour of these refusals.

--commit-output takes OWNER/NAME (the service picks the branch), a new
--repo no longer drops a saved commit output silently, --repo naming the
saved repository keeps its branch, and --file names follow the names the
service stores. The published manifest size budget is raised to 106,496
bytes for the new options.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add shared cases for the --file count limit, non-UTF-8 file content, the
service's 409 for a --replace target that is already bound, and a
service refusal to change a live binding. A bound program on a merging
service now gets a bare program_ref when no setting changes (only
--allow-reset or a same-ref --replace), the webhook-only refusal names
--allow-reset, both ports check --commit-output against --repo before
reading files, and the --allow-reset help mentions --replace.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant