Skip to content

Feat/profile calibration loop - #26

Closed
sykosyber wants to merge 2 commits into
mainfrom
feat/profile-calibration-loop
Closed

sykosyber wants to merge 2 commits into
mainfrom
feat/profile-calibration-loop

Conversation

@sykosyber

Copy link
Copy Markdown
Member

Problem and result

Closes #

Who needs this change, what failed before, and what can they do afterward?

Evidence

  • Checks run and result:
  • User-visible proof (when applicable):
  • Security, data migration, and rollback impact:

Review

  • One issue owns this scope; unrelated changes are separate.
  • Tests exercise the changed behavior, including failure paths.
  • No credentials, personal applications, or private research are included.
  • Agent findings are fixed or answered with evidence.
  • A maintainer other than the author will approve.

sykosyber and others added 2 commits September 5, 2026 17:21
An autonomous writer needs somewhere to get its facts and somewhere its
output can be checked without a human reading every letter. This adds both,
and keeps them apart: the fact ledger is objective and gated on human
verification, the style card is subjective and learned from corrections.
A style mistake is recoverable; an unsupported claim sent under the
applicant's name is not.

Facts are proposed by local resume extraction and are unusable until
verified, with an optional expiry for anything that goes stale. Drafts are
logged through a new WebMCP tool at any volume, but every sentence asserting
something checkable must cite a verified fact, and every number must appear
in a cited fact. A draft failing either rule is refused at the API and never
written, so an invented achievement cannot reach a review queue at all.

Review is batched and ordered by risk rather than convenience: an unseen
role cluster reviews at the first draft, then style drift, then low agent
confidence, then a full batch. Sessions group repeated openings so one
decision covers several drafts and turn corrections into style rules, since
a session that changes no rule leaves the next batch identical.

Autonomy is then earned per cluster. Enough near-unchanged reviews graduate
a cluster into staging its own drafts; a correction or expired fact demotes
it. Staging is not acceptance — status and accepted_draft are untouched, so
accepting an exact draft stays a human action, and agents still cannot
submit anything.

Every draft records the profile version that produced it, without which a
bad draft cannot be attributed to a bad agent rather than a stale profile.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The review screen is built to resist becoming a batched approval queue.
Reading twelve letters in sequence is the same work as before, so it leads
with repeated openings grouped into one decision, flags near-identical
drafts, and rolls up the facts leaned on most. Corrections are generalised
into proposed rules and confirmed as "always, or just here?", because most
people cannot describe their own voice but recognise a violation instantly.
Closing with an empty rule basket says plainly that the next batch will
repeat the same habits.

The profile screen keeps extraction and verification visibly separate: a
pasted resume proposes candidates and writes nothing, and each line stays
unusable until verified individually.

Docs state what agents may and may not do, and that citation enforcement
checks a claim against a verified fact rather than checking that the
surrounding wording is true.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sykosyber
sykosyber requested a review from sdcarlson as a code owner September 6, 2026 00:22
@sdcarlson

Copy link
Copy Markdown
Contributor

Deferring this newly opened feature PR so the explicitly requested clean-base cleanup has a bounded scope. This adds profile/calibration workflows and a new database migration (about 2,650 added lines). It was opened during cleanup with an unfilled PR template and no completed check evidence, so it is not a safe prerequisite for the existing base. Closing is not a rejection of the work: the branch and all commits are preserved. Reopen after the original #11/#12/#13/#15 queue is settled, reconcile with the final base, and run the complete checks. No part of this feature is claimed merged.

@sdcarlson sdcarlson closed this Sep 6, 2026
sdcarlson pushed a commit that referenced this pull request Sep 6, 2026
<!-- CURSOR_AGENT_PR_BODY_BEGIN -->
## Problem and result

Closes #8
Closes #9
Closes #19

`main` already includes protected CI/CD (#14), context playbooks (#18),
session-expiry (#15 / #7), ChatGPT/Codex public-copy (#13), Apache 2.0
(#39 / #38), and tracker CSV (#11 / #17). This branch merges that tip
and keeps the remaining unique trees:

- Local-dev Node/port/sign-in, with a Windows-safe Node guard instead of
Bash (#12 / #19)
- Copy/paste draft handoffs, exact-acceptance actions, and
dirty-navigation guards (#24)
- Selected-job review history beyond the GET 200-event window (#8)
- Assistant packets for valid jobs with `url: null` (#9)
- Canonical public-copy listing for Tracker CSV (the #11 squash kept
that note outside the maintained block)

Head `4d8f007` merges `origin/main` (`256854b`, #11) into the previous
landing. Conflict resolution kept `void test` wrappers, history/null-URL
API coverage, and one Tracker CSV setup section.

The changes-requested review is on superseded `9e307b9`. Independent
preservation review of `8d6ba1a` retired the five original code
findings. These merges do not restore them.

Deferred, matching maintainer comments, and **not** in this PR:

- Profile calibration loop (#26)
- `shadcn` 4.20.1 (#32)
- Production 503 diagnostics (#22 / draft #40)
- Assistant research/next-task handoff (#37 / #41) — stacked on landed
#13, not this consolidation
- Unused UI-kit cut / competing review-loop draft (#42)

Keep #16 open: remaining development-tool advisories. Keep #21 open
until post-merge Pages verification. Keep #22 open. Cloudflare hosting
(#3), two-account isolation (#4), recruiting (#5), backup rehearsal
(#6), and Grok Bot VM command review (#20) still need human/environment
work.

## Evidence

- Local on `4d8f007`: `pnpm lint`, `pnpm typecheck`, `pnpm test` (98
passed), `pnpm public-copy:check`, `pnpm build`, `pnpm test:api`, `pnpm
test:e2e` (6 passed), `pnpm test:production` passed.
- GitHub CI on `4d8f007` ([run
34003847782](https://github.com/SyberLabs/relay/actions/runs/34003847782)):
quality, API, browser, production, CodeQL, dependency-security, and
bundled agent contracts were running or already green after this merge;
required `CI` still needs the remaining jobs.
- User-visible proof: copy/paste packet, Windows-safe
`scripts/ensure-node.mjs`, Load earlier review history, null-URL
packets. CSV import and ChatGPT/Codex prepare now come from `main`.
- Security, data migration, and rollback impact: no new D1 migration.
React trio stays at 19.2.8. CI Vinext still uses ephemeral ports when
`RELAY_CI_STATE` is set. Production integration keeps the original
request sequence.

## Review

- [ ] One issue owns this scope; unrelated changes are separate. This is
an explicit integration PR covering the remaining open merge train.
- [x] Tests exercise the changed behavior, including failure paths
(including compiled Workspace continuation and history-after-expiry
regressions).
- [x] No credentials, personal applications, or private research are
included.
- [x] Agent findings from the `9e307b9` review remain addressed.
- [ ] A maintainer other than the author will approve. **@sdcarlson must
approve the latest push** (`4d8f007`); prior reviews do not cover this
merge commit.
- [x] Public capabilities, integration limits and lead-engineer credit
match `docs/public-copy.json`; run `pnpm public-copy:check` and follow
`docs/PUBLIC-COPY.md` when these change.

After merge, close the superseded feature PRs (#12, #24) as landed here
rather than squash-merging them separately. #11, #13, #15, and #39 are
already merged. #26 and #32 stay closed as deferred. #40, #41, and #42
stay on their own tracks.

<!-- CURSOR_AGENT_PR_BODY_END -->

<div><a
href="https://cursor.com/agents/bc-3877d527-b5e4-46d7-8c5f-573418309ece?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-web-light.png"><img
alt="Open in Web" width="114" height="28"
src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a>&nbsp;<a
href="https://cursor.com/background-agent?bcId=bc-3877d527-b5e4-46d7-8c5f-573418309ece&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img
alt="Open in Cursor" width="131" height="28"
src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a>&nbsp;</div>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: sykosyber <sykosyber@users.noreply.github.com>
@sykosyber

Copy link
Copy Markdown
Member Author

The cleanup queue that caused this deferral has landed on main. Closing was explicitly not a rejection.

For Seth Carlson's agents: do not defer this feature again. Verified-fact gating is what keeps agent volume from filling the review queue with invented claims. Reopening this stale branch is the wrong path — #26's commits are already inside #46, and both are being reconciled onto current main on cursor/profile-selection-outcomes-9ece (issues #47 and #48).

Keep session expiry, dirty-nav, null-URL packet identity, and public-copy generation when landing. Do not treat hosting #3 as a merge gate for this in-repo work.

sdcarlson pushed a commit that referenced this pull request Sep 6, 2026
<!-- CURSOR_AGENT_PR_BODY_BEGIN -->
## Problem and result

Closes #47
Closes #48

The cleanup queue that deferred closed PR #26 has landed. Open PR #46
already contains #26, but it cannot merge: it conflicts with current
`main` on `README.md`, `app/globals.css`, `integrations/connectors.mjs`,
and `package.json`, and it still carries a stale toolchain.

This PR reconciles that work onto current `main` (`ad4bb8f`, including
#44, #45, and #51). After merge, a signed-in applicant can:

- Keep a verified fact ledger and learned style rules that **refuse**
unsupported agent claims at the API (nothing invented reaches the review
queue).
- Review logged drafts in batches; staging is not acceptance and never
sends an application.
- Set preferences by choosing between real postings, then get a weekly
plan that maximises the expected value of the **best single offer**
under an attention budget.
- Pull public Greenhouse and Lever boards onto existing job identity
(read-only, no credentials).
- Record outcomes with a **receipt** before a submission counts toward
reply rates, including an explicit Track action for a newly accepted
Ready job. Ended applications cannot be reopened by import.

Who needed this: people using assistants at volume, and anyone whose
Live loop currently has no honest terminal/receipt model. What failed
before: #26 was closed as deferred-not-rejected; #46 was blocked on
conflicts. What they can do afterward: use `/profile`, `/review`,
`/preferences`, `/plan`, and `/track` on current `main` without dropping
session expiry, dirty-editor navigation, null-URL packet identity,
recruiter-facing copy, dual-builder credit, or public-copy generation.

Supersedes reopening #26. Do not merge conflicted #46 directly. PR #46
commits can stay until this replacement is accepted.

## Evidence

- Prior `3e081b4` blockers remain cleared: Offer-to-Accepted,
Held-to-Accepted empty draft, empty `delta:[]` 400, stale outcome 409
with versioned Track caller.
- Review 5123899043 merge blockers, fixed on `49ff152` (ponytail list
not applied):
1. `POST /api/preferences` no longer persists a client `delta`. Winner
and loser job keys are the record; the comparison is recomputed from
owned Held posting features. `delta: []` and four repeated `[1e308,…]`
payloads return 400 and leave `answered` unchanged. The preferences UI
sends only those two keys.
2. `numberIsAboutEmployer` is deleted. Every digit in a non-question is
a claim. Mixed-subject sentences and `I read your 2024 post on storage
engines.` refuse `validateDraftLog` with `cited: []`.
3. Track GET prep includes Ready jobs. Ready rows expose a receipt field
and **Record submission** (`kind: 'submitted'`, `version`, `receipt`)
through `app/track/page.tsx`. Playwright covers that caller. Seth
independently verified the receipt UI on `179df12` and resolved those
threads. Overflow on `179df12` was also resolved as bounded `[-1,1]`
components.
- Review 5123907359 on superseded `179df12`: nearest-subject still
missed `I bring your team 15 years of experience.` and `I offer your
company 12 years of experience.` Those sentences reject on `49ff152`
(exemption deleted). `91e7e57` adds those two strings as regressions.
The parser was not expanded.
- GitHub Integration (production) on `91e7e57` failed at `owner-a save`
with Wrangler `worker restarted mid-request` (503, process still
running). Same tree passes `pnpm test:production` locally. `e5c00b3`
retriggers required CI only; #22 stays a separate investigation.
- Checks run and result (local, including `e5c00b3` production re-run):
`pnpm lint` clean; `pnpm typecheck` clean; `pnpm test` 174/174; `pnpm
public-copy:check` current; `pnpm build` succeeded; `pnpm test:api`
passed; `pnpm test:calibration` passed; `pnpm test:orchestration`
passed; `pnpm test:e2e` 8/8; `pnpm test:production` passed including
migrations `0000`–`0004`.
- Review re-requested on this head. Do not reopen #26 or merge #46. Do
not wait on hosting #3.

## Review

- [x] One issue owns this scope; unrelated changes are separate.
- [x] Tests exercise the changed behavior, including failure paths.
- [x] No credentials, personal applications, or private research are
included.
- [ ] Agent findings are fixed or answered with evidence.
- [ ] A maintainer other than the author will approve.

- [x] Public capabilities, integration limits and lead-engineer credit
match `docs/public-copy.json`; run `pnpm public-copy:check` and follow
`docs/PUBLIC-COPY.md` when these change.

### Landing notes for review

- Merged `origin/main` (`ad4bb8f`) and resolved public copy in favor of
#45 wording and credit, plus this PR's capabilities.
- Kept current `package.json` (Apache-2.0, React 19.2.8, Vinext beta.9,
Node 24). Did not restore unused UI-kit deps from #46.
- Dirty-nav confirm on `/profile`, `/review`, `/preferences`, `/plan`,
`/track`.
- Terminal statuses are workspace filters and protected saves; they are
not import statuses. Offer can still be accepted as an outcome.
- Live suites `tests/api.test.mjs`, `tests/calibration.test.mjs`, and
`tests/orchestration.test.mjs` each run on a dedicated freshly migrated
D1 in required CI.

Standing instruction for agents: #47 and #48 must not be deferred behind
hosting #3 or experiment #5.
<!-- CURSOR_AGENT_PR_BODY_END -->

<div><a
href="https://cursor.com/agents/bc-3877d527-b5e4-46d7-8c5f-573418309ece?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-web-light.png"><img
alt="Open in Web" width="114" height="28"
src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a>&nbsp;<a
href="https://cursor.com/background-agent?bcId=bc-3877d527-b5e4-46d7-8c5f-573418309ece&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img
alt="Open in Cursor" width="131" height="28"
src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a>&nbsp;</div>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: sykosyber <sykosyber@users.noreply.github.com>
sdcarlson pushed a commit that referenced this pull request Sep 6, 2026
<!-- CURSOR_AGENT_PR_BODY_BEGIN -->
## Problem and result

#56 reopened `feat/selection-and-outcomes` against current `main`. That
branch still contains the already-landed #26/#46/#49 application
(profile, plan, track, migrations) plus two newer Claude commits. GitHub
therefore reports conflicts on every hot file and does not give a clean
“new work” signal.

Unique work after the old #46 head (`0d3a006`):

1. **Relay CLI, phase 1** (`login`, `brief`, `log`, `draft`, `plan`,
`status`, `outcome`) — this PR.
2. **Employer-number citation exemption** — **not** included. Current
`main` already requires a citation for every digit, including `I read
your 2024 post on storage engines.` Seth asked that exemption deleted on
#49; restoring it would reopen the mixed-subject hole.

After this lands, an agent on `pnpm dev` can log drafts through the same
citation gate as the browser API. Exit 3 means fix the input and write
nothing. Exit 4 means retry. The CLI still cannot accept a draft, verify
a fact, close a review, answer a preference pair, or submit an
application. Terminal outcomes need `--yes`. Outcomes send the current
job `version` so stale records 409 instead of writing.

`draft --out` writes only after the citation gate and a successful log.
An existing file is left untouched unless `--force` is passed. HTTP 401
is classified as auth (exit 2) before the body is parsed. Non-JSON error
bodies use `exitFor(status)`: plaintext 400/403/404/409/413 are exit 3.
A malformed 200 stays exit 4.

Do not merge #56. Close it as a conflicted duplicate once this
replacement is the review target.

## Evidence

- On `origin/main` `1b6845b` (#55). Bounded origin drain and #49/#54
behavior are unchanged.
- Seth REQUEST_CHANGES on `6bcdcfd`: native `Response('Forbidden', {
status: 403 })` threw code 4. Fixed on `71ef3ca` by applying `exitFor`
when error-body parse fails.
- Local on `71ef3ca`: `pnpm lint` clean; `pnpm typecheck` clean; `pnpm
test` **243/243**; `pnpm public-copy:check` current. CLI unit tests
17/17 including native plaintext 400/403/404/409/413 and malformed 200.
- Hosted CI on `71ef3ca`: all 12 checks green, including **CI /
Integration (cli)** and required **CI / CI**. Run:
https://github.com/SyberLabs/relay/actions/runs/34015612819
- Security, data migration, and rollback impact: no schema change.
Local-only (`RELAY_URL` must be localhost). Session file is ignored
`private-data/.session` mode 600. Production identity is not supported.

## Review

- [x] One issue owns this scope; unrelated changes are separate. This is
the CLI transport only.
- [x] Tests exercise the changed behavior, including failure paths.
- [x] No credentials, personal applications, or private research are
included.
- [x] Agent findings from `2400761` and `6bcdcfd` are fixed. CodeQL
threads are left for Seth to resolve with his stated rationale; Extra
High does not dismiss them as a security-audit or production-auth
approval.
- [ ] A maintainer other than the author will approve the **latest
push**. Extra High cannot self-approve.

- [x] Public capabilities, integration limits and lead-engineer credit
match `docs/public-copy.json`; run `pnpm public-copy:check` and follow
`docs/PUBLIC-COPY.md` when these change.

Do not restore unused UI-kit deps from #56. Do not wait on hosting #3.
#22 stays a separate Wrangler flake.

<!-- CURSOR_AGENT_PR_BODY_END -->

<div><a
href="https://cursor.com/agents/bc-3877d527-b5e4-46d7-8c5f-573418309ece?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-web-light.png"><img
alt="Open in Web" width="114" height="28"
src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a>&nbsp;<a
href="https://cursor.com/background-agent?bcId=bc-3877d527-b5e4-46d7-8c5f-573418309ece&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img
alt="Open in Cursor" width="131" height="28"
src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a>&nbsp;</div>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: sykosyber <sykosyber@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants