Skip to content

Feat/selection and outcomes - #56

Closed
sykosyber wants to merge 6 commits into
mainfrom
feat/selection-and-outcomes
Closed

sykosyber wants to merge 6 commits into
mainfrom
feat/selection-and-outcomes

Conversation

@sykosyber

Copy link
Copy Markdown
Member

Problem and result

Closes #

Who needs this change, what failed before, and what can they do afterward?

Evidence

  • Checks run and result:
  • User-visible proof (when applicable):
  • Security, data migration, and rollback impact:

Review

  • One issue owns this scope; unrelated changes are separate.

  • Tests exercise the changed behavior, including failure paths.

  • No credentials, personal applications, or private research are included.

  • Agent findings are fixed or answered with evidence.

  • A maintainer other than the author will approve.

  • Public capabilities, integration limits and lead-engineer credit match docs/public-copy.json; run pnpm public-copy:check and follow docs/PUBLIC-COPY.md when these change.

sykosyber and others added 6 commits September 5, 2026 17:21
An autonomous writer needs somewhere to get its facts and somewhere its
output can be checked without a human reading every letter. This adds both,
and keeps them apart: the fact ledger is objective and gated on human
verification, the style card is subjective and learned from corrections.
A style mistake is recoverable; an unsupported claim sent under the
applicant's name is not.

Facts are proposed by local resume extraction and are unusable until
verified, with an optional expiry for anything that goes stale. Drafts are
logged through a new WebMCP tool at any volume, but every sentence asserting
something checkable must cite a verified fact, and every number must appear
in a cited fact. A draft failing either rule is refused at the API and never
written, so an invented achievement cannot reach a review queue at all.

Review is batched and ordered by risk rather than convenience: an unseen
role cluster reviews at the first draft, then style drift, then low agent
confidence, then a full batch. Sessions group repeated openings so one
decision covers several drafts and turn corrections into style rules, since
a session that changes no rule leaves the next batch identical.

Autonomy is then earned per cluster. Enough near-unchanged reviews graduate
a cluster into staging its own drafts; a correction or expired fact demotes
it. Staging is not acceptance — status and accepted_draft are untouched, so
accepting an exact draft stays a human action, and agents still cannot
submit anything.

Every draft records the profile version that produced it, without which a
bad draft cannot be attributed to a bad agent rather than a stale profile.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The review screen is built to resist becoming a batched approval queue.
Reading twelve letters in sequence is the same work as before, so it leads
with repeated openings grouped into one decision, flags near-identical
drafts, and rolls up the facts leaned on most. Corrections are generalised
into proposed rules and confirmed as "always, or just here?", because most
people cannot describe their own voice but recognise a violation instantly.
Closing with an empty rule basket says plainly that the next batch will
repeat the same habits.

The profile screen keeps extraction and verification visibly separate: a
pasted resume proposes candidates and writes nothing, and each line stays
unusable until verified individually.

Docs state what agents may and may not do, and that citation enforcement
checks a claim against a verified fact rather than checking that the
surrounding wording is true.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Drafting was never the largest lever. Which jobs you apply to is: reply odds
vary by orders of magnitude with fit, while letter quality varies perhaps
twofold. This adds the selection half.

The objective is deliberately not the sum over applications. You accept one
job, so the quantity that matters is the expected value of the best single
offer — an order statistic, computed exactly rather than approximated by
tier heuristics. That one change produces a spread of odds by itself: once a
likely offer is held, further similar roles add almost nothing and a long
shot starts winning the comparison. Selection is greedy on gain per minute
under a stated attention budget, so no reach/match/floor ratio is hardcoded,
and every row carries the reason it earned its place.

Preferences supply the utility term and cannot be asked for directly. People
cannot state trade-off weights in the abstract but choose between two
concrete jobs instantly, so they are elicited by forced choice over real
postings and fitted by pairwise logistic regression. Domain fit is computed
from the person's own verified claims rather than learned as one weight per
role family, because a dozen comparisons cannot determine a dozen weights.
With no answers recorded the model reports no opinion instead of inventing
one.

Outcomes close the loop the state machine previously left open at Live loop.
Terminal states are local records of what happened and are deliberately not
valid import statuses, which leaves the existing import matrix untouched and
means rediscovery can never reopen an ended application — asserted against
the real upsert SQL as well as the TypeScript, since the two must agree.

A submission requires a receipt. A false "submitted" is doubly corrupting:
deduplication would stop you applying to a job you never applied to, and the
response-rate posterior would count a send that never happened. Imported
submissions are kept and shown but arrive unreceipted and are excluded from
the rates. Rates are reported as intervals with their evidence count, and
are descriptive only.

The read plane pulls public Greenhouse and Lever boards onto the existing
jobKey identity. It reads only, needs no credentials and no account, and
every posting arrives Held.

No write-plane adapters are included. Nothing here submits an application;
recording a submission is the applicant reporting what they already did.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The preferences screen shows two real jobs side by side and asks which you
would rather have, because that is a question people can answer accurately
and instantly. It shows the fitted weights as shares of one whole so the
implied trade-offs are visible and arguable, and says plainly that it has no
opinion until told something.

The plan screen presents the week rather than a queue: what to apply to,
what it cost of the budget, and why each row earned its place. It reports
what the same budget would buy under a plain value ranking, so the
portfolio's advantage is shown rather than asserted, and it says when the
value model is still too thin to lean on.

The track screen carries both halves of what happens after sending. Reply
rates are labelled descriptive and shown with their evidence count, live
applications are flagged when they carry no receipt, and each one lists the
exact claims it commits you to defending — which is the citation list, not a
document anyone had to write.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A digit made a sentence a claim regardless of whose it was, so "I read your
2024 post" and "I admire your 30-person team" both demanded a citation. Every
ordinary compliment about an employer tripped the gate, and a gate that
fights normal prose gets routed around rather than satisfied.

A number governed by a second-person possessive belongs to the employer. The
exemption is narrow on purpose: it applies only when no achievement verb of
the applicant's own appears in the sentence, so "I shipped 3 releases for
your 2024 launch" stays a claim and still needs its citation. A wrongly
blocked sentence is visible and recoverable; a wrongly admitted one is not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The citation gate, draft log, batch review and cluster graduation were
reachable through exactly one browser API that had never been run, while the
only drafting path that worked — the claude-draft packet flow — took facts
typed by hand and bypassed all of it. The product's headline safety property
was unenforced on the only route anyone could use.

Seven commands over routes that already exist: login, brief, log, draft,
plan, status, outcome. No decisions live in the CLI. What counts as a claim,
which jobs make the plan and when a cluster graduates stay in lib/, where
every transport reaches the same answer, and the client is shared so a later
MCP server cannot drift into different rules.

Exit codes are the contract. A domain refusal (3) is kept permanently
distinct from a server failure (4), because an agent that confuses them will
retry a refused draft until some phrasing slips past the gate — converting a
safety check into an obstacle to route around. A refusal prints the offending
sentence and writes nothing, asserted directly against the stored row count.

The boundary is that the CLI may do anything except exercise taste or
authorise an irreversible act: it cannot accept a draft, verify a fact, close
a review with rules, or answer a preference pair, and terminal outcomes need
explicit confirmation. Adding a transport must never add a capability.

Local only. Production identity comes from a trusted gateway the CLI cannot
present, so RELAY_URL is refused unless it is localhost.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sykosyber
sykosyber requested a review from sdcarlson as a code owner September 6, 2026 05:11

@sdcarlson sdcarlson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes on exact2b2918b44fa1e20551b538c5177ba07937f3394e. This branch predates the merged #49/#54 fixes. A read-only merge-tree against current main b8eedf2 reports extensive conflicts, including outcome/preferences APIs, four private pages, profile/utility and tests. The genuinely new top commit is the CLI work (8files,887additions); do not re-land the old profile/outcome implementation. Reconcile a focused CLI delta onto current main, preserving its version checks, server-derived preference vectors, conservative claim validation, session-expiry guards, required CI, and historical evidence. A direct tree comparison shows absent main changes, not proof Git would delete unchanged files; the actual merge conflicts are the integration blocker.

Two concrete new-client integration findings to fix during that reconciliation:

  1. integrations/cli.mjs outcome command posts id/kind/receipt without job.version. Current main /api/outcomes requires the observed version and returns409 when absent. Fetch the owned job/version and send it, preserving stale-write rejection; test the CLI against current main's API contract.
  2. integrations/client.mjs parses JSON before classifying status. A plaintext401 becomes EXIT.server(4), not EXIT.auth(2), so the advertised unattended-agent contract directs retry instead of sign-in. Handle401 before body parsing, preserving a useful auth error; cover plaintext and JSON401.

The PR description is still an empty template. Rewrite it around the new CLI behavior, exact reconciled validation, limitations and ownership before final review. No live records or source edits were used by this review.

@sykosyber sykosyber closed this Sep 6, 2026
sdcarlson pushed a commit that referenced this pull request Sep 6, 2026
<!-- CURSOR_AGENT_PR_BODY_BEGIN -->
## Problem and result

#56 reopened `feat/selection-and-outcomes` against current `main`. That
branch still contains the already-landed #26/#46/#49 application
(profile, plan, track, migrations) plus two newer Claude commits. GitHub
therefore reports conflicts on every hot file and does not give a clean
“new work” signal.

Unique work after the old #46 head (`0d3a006`):

1. **Relay CLI, phase 1** (`login`, `brief`, `log`, `draft`, `plan`,
`status`, `outcome`) — this PR.
2. **Employer-number citation exemption** — **not** included. Current
`main` already requires a citation for every digit, including `I read
your 2024 post on storage engines.` Seth asked that exemption deleted on
#49; restoring it would reopen the mixed-subject hole.

After this lands, an agent on `pnpm dev` can log drafts through the same
citation gate as the browser API. Exit 3 means fix the input and write
nothing. Exit 4 means retry. The CLI still cannot accept a draft, verify
a fact, close a review, answer a preference pair, or submit an
application. Terminal outcomes need `--yes`. Outcomes send the current
job `version` so stale records 409 instead of writing.

`draft --out` writes only after the citation gate and a successful log.
An existing file is left untouched unless `--force` is passed. HTTP 401
is classified as auth (exit 2) before the body is parsed. Non-JSON error
bodies use `exitFor(status)`: plaintext 400/403/404/409/413 are exit 3.
A malformed 200 stays exit 4.

Do not merge #56. Close it as a conflicted duplicate once this
replacement is the review target.

## Evidence

- On `origin/main` `1b6845b` (#55). Bounded origin drain and #49/#54
behavior are unchanged.
- Seth REQUEST_CHANGES on `6bcdcfd`: native `Response('Forbidden', {
status: 403 })` threw code 4. Fixed on `71ef3ca` by applying `exitFor`
when error-body parse fails.
- Local on `71ef3ca`: `pnpm lint` clean; `pnpm typecheck` clean; `pnpm
test` **243/243**; `pnpm public-copy:check` current. CLI unit tests
17/17 including native plaintext 400/403/404/409/413 and malformed 200.
- Hosted CI on `71ef3ca`: all 12 checks green, including **CI /
Integration (cli)** and required **CI / CI**. Run:
https://github.com/SyberLabs/relay/actions/runs/34015612819
- Security, data migration, and rollback impact: no schema change.
Local-only (`RELAY_URL` must be localhost). Session file is ignored
`private-data/.session` mode 600. Production identity is not supported.

## Review

- [x] One issue owns this scope; unrelated changes are separate. This is
the CLI transport only.
- [x] Tests exercise the changed behavior, including failure paths.
- [x] No credentials, personal applications, or private research are
included.
- [x] Agent findings from `2400761` and `6bcdcfd` are fixed. CodeQL
threads are left for Seth to resolve with his stated rationale; Extra
High does not dismiss them as a security-audit or production-auth
approval.
- [ ] A maintainer other than the author will approve the **latest
push**. Extra High cannot self-approve.

- [x] Public capabilities, integration limits and lead-engineer credit
match `docs/public-copy.json`; run `pnpm public-copy:check` and follow
`docs/PUBLIC-COPY.md` when these change.

Do not restore unused UI-kit deps from #56. Do not wait on hosting #3.
#22 stays a separate Wrangler flake.

<!-- CURSOR_AGENT_PR_BODY_END -->

<div><a
href="https://cursor.com/agents/bc-3877d527-b5e4-46d7-8c5f-573418309ece?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-web-light.png"><img
alt="Open in Web" width="114" height="28"
src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a>&nbsp;<a
href="https://cursor.com/background-agent?bcId=bc-3877d527-b5e4-46d7-8c5f-573418309ece&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img
alt="Open in Cursor" width="131" height="28"
src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a>&nbsp;</div>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: sykosyber <sykosyber@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants