fix(publication): bound historical lineage backfill

[snowzlmbot]

First submitted: July 23, 2026, 1:05 PM (US Eastern Time) / 17:05 UTC.
Last body edit: July 23, 2026, 1:34 PM (US Eastern Time) / 17:34 UTC.
Latest head commit: July 23, 2026, 1:20 PM (US Eastern Time) / 17:20 UTC.

Related: #800
Follow-up to #807

What Problem This Solves

Resolves a publication-queue problem where duplicate protocol-v2 result lineages that predate #807 can remain pending indefinitely unless another equivalent artifact happens to arrive. Those rows add preparation and state-writer pressure without representing a new review decision.

The public snapshot at 2026-07-23T17:00:40Z showed 2,072 pending result publications, zero active publishers despite 50 available publication slots, an oldest pending age of 202,680 seconds, and a 60-minute net drain rate of -29/hour (41 arrivals, 12 resolutions). This does not imply that every pending row is a duplicate; it shows why historical duplicates need a deterministic dry-run and bounded cleanup path instead of relying only on future ingress.

Why This Change Was Made

#807 now enforces one effective lineage for new protocol-v2 deliveries. This follow-up extends the existing publication reconcile owner path so it can proactively identify and remove only proven historical siblings while preserving every active ownership fence and retry budget.

The maintenance workflow is intentionally one pass per manual dispatch. It runs a dry-run by default and can apply at most 100 removals, so operators can observe 15-minute and 60-minute flow, contention, and dead-letter safety before another pass.

User Impact

Operators can preview the historical cleanup set, apply one bounded pass, and see exactly how many stale revisions and duplicate lineages were removed, how many remain, how many active owners were protected, and the oldest eligible/remaining age. Review decisions, comments, apply work, batch size, and publication concurrency are unchanged.

Breaking Changes

No runtime API, queue schema, storage format, dependency, configuration, or workflow-dispatch input is removed.

The deprecated passes input remains accepted for compatibility. Values above one are explicitly logged and clamped to one observed pass per invocation. This intentionally narrows prior multi-pass behavior while keeping existing dispatch payloads valid; maintainer acceptance of that operational policy remains the merge gate.

Implementation

• Extend /internal/exact-review/publications/reconcile to classify both stale review revisions and same-lineage historical duplicates.
• Define a lineage with the same immutable tuple used by fix(publication): coalesce result lineages #807: target item, lease revision, and claim generation.
• Consider only pending and parked rows for removal.
• Preserve dispatching, leased, and active-batch-owned rows. If one owns a lineage, it remains authoritative and only unowned pending siblings are eligible.
• Without an active owner, keep the oldest queue slot and its attempts/failure budget, refresh that slot to the newest known producer artifact, and remove redundant siblings.
• Select the oldest eligible rows first and cap every request at 100 mutations.
• Return stale/lineage counts, retained-lineage refreshes, protected ownership, and oldest eligible/remaining ages.
• Keep the repair client compatible with a rolling deployment where the older dashboard response does not yet expose the new fields.
• Keep the deprecated passes input valid during migration, log values above one, and clamp every invocation to one observed pass.

Code-Path Analysis

• Entry: .github/workflows/exact-review-queue-maintenance.yml
• CLI: src/repair/exact-review-queue-maintenance.ts
• Authenticated client: src/repair/exact-review-batch-queue-client.ts
• Dashboard proxy: dashboard/worker.ts
• Queue owner and mutation boundary: ExactReviewQueue.reconcilePublicationCandidates() in dashboard/exact-review-queue.ts
• Active batch ownership: dashboard/exact-review-publication-batches.ts
• Sibling invariant: fix(publication): coalesce result lineages #807's enqueue-time lineage coalescing in dashboard/exact-review-queue.ts
• Focused tests: test/exact-review-publication-batches.test.ts and test/repair/exact-review-queue-maintenance-workflow.test.ts

Mainline Comparison

• Official default branch: openclaw/clawsweeper:main
• Current base: 8cdbe5755e22c2f869930f31b2e5d439e5149e49
• The base includes fix(publication): coalesce result lineages #807 (205b92b9b6dc1cea735ef610fd6a39e98d18a5d1), fix(review): supersede stale exact-review runs #749, and fix(validation): classify identity-capture starvation under the runtime budget #817.
• fix(publication): coalesce result lineages #807 handles new ingress and same-lineage redelivery; it does not make a standalone bounded pass over historical rows.
• No open PR with a publication-lineage reconcile/backfill title or scope was found before submission.

Branch and Base Provenance

• Base: openclaw/clawsweeper:main@8cdbe5755e22c2f869930f31b2e5d439e5149e49
• Head: snowzlmbot/clawsweeper:fix/publication-lineage-reconcile@bd2b85d5fafab888c9b64f8f31e2b61e6ac80c86
• Evidence branch: snowzlmbot/clawsweeper:evidence/pr-publication-lineage-reconcile@3708ce99ce51f64f0b6185a39d80abfaf6aba2f2
• The fork default branch remains a mirror-only branch and contains none of this contribution's implementation commits.

Dependency and Contract Verification

No dependency or lockfile changes. The existing signed internal queue endpoint, protocol-v2 lineage fields, durable batch ownership store, queue metrics, and state schema remain the authoritative contracts. Older dashboard responses remain accepted by the updated repair client during rollout.

Evidence

Current-head focused proof:

• Bounded historical lineage reconcile proof — successful on evidence head 3708ce99ce51f64f0b6185a39d80abfaf6aba2f2, which contains code head bd2b85d5fafab888c9b64f8f31e2b61e6ac80c86 unchanged plus evidence-only files.
• The run passed build:all, the historical three-sibling dry-run/two-pass convergence regression, active batch ownership fencing, compatibility-preserving one-pass workflow checks, an authenticated worker-route dry-run, formatting, and focused lint.
• Upstream CI run — successful on code head bd2b85d5fafab888c9b64f8f31e2b61e6ac80c86; pnpm check, workflow semantics, sparse repair build smoke, the Windows launcher, and crawl-remote integration all passed.
• Upstream CodeQL run — both Actions and JavaScript/TypeScript analysis passed on the same PR head.
• Public dashboard snapshot — captured from the public status surface; it shows the publication backlog and idle publication capacity without exposing private data.
• Live dashboard and JSON status remain the post-deploy observation sources.

Local preflight was limited to non-building static checks:

• git diff --check: passed.
• Changed-file oxfmt --check: passed.
• Changed-TypeScript oxlint --deny-warnings: 0 warnings, 0 errors.

Sanitized Proof Summary

The focused historical fixture begins with three pending rows for one immutable lineage. Dry-run reports two eligible duplicates and changes nothing. A one-item apply removes one sibling, preserves the oldest slot's seven attempts and failure history, refreshes it to the newest producer artifact, reports one remaining duplicate, and records the remaining age. A second bounded pass removes the last sibling. A separate fixture leases the retained row to an active batch and proves reconcile removes only the unowned sibling while fetch/completion ownership remains valid.

The evidence workflow also sends an HMAC-signed apply=false request through the real dashboard worker route into the real queue owner. Its redacted transcript is:

{"http_status":200,"apply":false,"scanned":2,"eligible":1,"changed":0,"eligible_remaining":1,"lineage_duplicate_eligible":1,"protected_batch_items":0,"protected_lineage_items":0,"oldest_eligible_age_seconds":60,"pending_before":2,"pending_after":2}

This proves authorization, response telemetry, and no mutation on the production code path without exposing credentials or queue identities.

Media Evidence

The linked screenshot is sufficient for the public backlog state. No video is needed: this change has no interactive UI or timing-dependent user flow, and the mutation behavior is deterministically covered by the linked current-head Actions run.

Review Context Addressed

• The first exact-head review identified removal of the passes input as a compatibility blocker. The input is now retained; larger values are accepted, logged as deprecated, and clamped to one pass. Existing dispatch payloads remain valid.
• The proof guidance requested an authenticated dry-run transcript. Evidence run 30029247413 now exercises the HMAC-protected worker route and real queue owner with apply=false, records eligible/protected/remaining fields, and proves pending state is unchanged.
• No production apply or queue mutation was performed. A maintainer-authorized live dry-run remains the final operational rollout gate before any apply dispatch.

Risk

• Incorrect equivalence could remove distinct work. The cleanup is restricted to complete protocol-v2 lineage tuples and does not cross review revisions or lanes.
• A cleanup could interfere with a publisher. Active batch membership, dispatching, and leased states are excluded and revalidated at mutation time.
• Selecting a fresher artifact could reset retry safety. The retained queue slot, attempts, first-failure time, and failure budget are preserved; only producer provenance is refreshed using fix(publication): coalesce result lineages #807's established ordering.
• Large historical state could create a long mutation. The endpoint scans the existing bounded Durable Object state but mutates at most 100 oldest eligible rows per request, and the workflow allows only one pass per manual dispatch.
• Operators could run another pass before observing health. Documentation explicitly requires a dry-run and stable positive net drain with unchanged contention/dead-letter safety before the next pass.

Rollback

Revert bd2b85d5fafab888c9b64f8f31e2b61e6ac80c86 and cf3f406d4ba9881e8f10e94df2170100069ecca8. No schema, dependency, or configuration migration is required. The previous stale-revision reconcile response remains compatible with the repair client. Already applied cleanup cannot recreate removed queue keys automatically, but removal is limited to redundant pending/parked siblings; no active owner or review decision is deleted.

Docs Updated

• docs/state-publication-batching-plan.md documents dry-run, the one-pass/100-item boundary, ownership protections, retained-slot behavior, telemetry, and the positive-drain gate.

Labels Considered

bug and performance are relevant. No upstream labels were applied by the contributor.

Related Work

• fix(publication): coalesce duplicate exact-result lineages before batch claim #800 — publication lineage invariant and bounded historical reconciliation requirement
• fix(publication): coalesce result lineages #807 — maintainer-owned clean implementation for new-ingress lineage coalescing
• fix(review): supersede stale exact-review runs #749 — stale active exact-review supersession
• Serialize duplicate publication items within batches #772 — one durable target item per publication batch

Not Tested / Limitations

• No production queue apply, dead-letter replay, concurrency increase, workflow pause, or live state mutation was performed from this branch.
• The authenticated dry-run proof uses the exact worker/auth/queue production code path with a bounded fixture; a live production dry-run remains a maintainer-authorized pre-apply gate.
• The 2,072-row public backlog is not assumed to consist entirely of duplicates; the live dry-run result after deployment is the authoritative eligible count.
• Full repository CI and CodeQL passed on the submitted head; the linked focused evidence run is the deterministic backfill/ownership/authentication proof.
• Post-deploy operation remains manual: live dry-run first, one bounded apply, then health observation before any later pass.

[clawsweeper]

ClawSweeper status: review started.

I am starting a fresh review of this pull request: fix(publication): bound historical lineage backfill This is item 1/1 in the current shard. Shard 0/1.

This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.

[snowzlmbot]

@clawsweeper review-pr

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.