fix(publication): keep current review results moving during historical cleanup

[snowzlmbot]

First submitted: July 24, 2026, 3:14 AM (US Eastern Time) / 07:14 UTC.
Last body edit: July 24, 2026, 5:41 AM (US Eastern Time) / 09:41 UTC.
Latest head commit: July 24, 2026, 4:14 AM (US Eastern Time) / 08:14 UTC.

Closes #826

What Problem This Solves

Fixes an issue where newly completed exact reviews could remain unpublished behind a large historical publication backlog when the oldest queued rows were later proven stale or superseded.

Production telemetry showed the queue shrinking without current results becoming visible: at 2026-07-24T06:20:22Z, publication pending was 802, the previous 60 minutes resolved 340 rows, published remained 0, and all 340 resolutions were superseded. The oldest publication was about 33 hours old and publication health remained critical.

Why This Change Was Made

Publication batch admission was strict createdAt FIFO. Increasing preparation concurrency made historical cleanup faster, but did not guarantee bounded service for a current authoritative result. This change adds a small, reversible fresh-authority reservation inside each existing batch while preserving historical FIFO and every existing mutation fence.

User Impact

Maintainers should see current exact-review conclusions reach their marker-backed comments with bounded service even while historical cleanup continues. Historical work still receives at least six of eight production batch positions and can use all positions when the fresh lane is empty.

Summary

• reserve at most two members of an eight-member batch for recent protocol-v2 publications whose source revision still matches the queue head;
• keep the remaining members oldest-first and return unused reserved capacity to historical work;
• revalidate the source revision after batch ownership is claimed and supersede drifted members before artifact preparation or mutation;
• expose fresh/historical ready counts and a feature-flag rollback surface;
• add deterministic coverage for bounded service, historical progress, disabled-flag FIFO, and post-claim source drift.

Breaking Changes

None. Queue request/response contracts remain additive and existing batch leases remain valid.

Implementation

dashboard/exact-review-queue.ts classifies a candidate as fresh only when all of these are true:

1. it is a ready pending publication;
2. it is within the configured recency window;
3. it carries a protocol-v2 immutable source revision;
4. that revision is still at least the queue's recorded head for the durable target.

Age alone never establishes authority. Admission moves only the configured bounded fresh reserve ahead of historical candidates, preserves FIFO within both groups, retains one durable target per batch, and keeps owner-homogeneous token selection. Batch fetch checks the source head again after ownership; drift is terminalized as superseded before publication work begins.

Production configuration enables a two-item reserve with a 15-minute recency bound. Setting EXACT_REVIEW_PUBLICATION_FRESH_LANE_ENABLED=0 restores strict historical FIFO without cancelling active batches.

Code-Path Analysis

• Runtime entry and admission owner: dashboard/exact-review-queue.ts (alarm, scheduleNext, exactReviewPublicationBatchDeparture, claimPublicationBatch, exactReviewQueueAdmittedItems).
• Durable ownership/fencing: dashboard/exact-review-publication-batches.ts.
• Worker caller: .github/workflows/exact-review-batch-publish.yml.
• Configuration: dashboard/wrangler.toml.
• Adjacent regression coverage: test/exact-review-publication-batches.test.ts and test/state-writer-workflow.test.ts.
• Operational contract: docs/state-publication-batching-plan.md.

The existing active-batch exclusion, publication head table, immutable queue revision, claim generation, lease owner, one-durable-item-per-batch rule, owner-scoped token, artifact validation, live GitHub re-fetch, and single state-writer boundary are unchanged.

Mainline Comparison

• Official repository/default branch: openclaw/clawsweeper:main.
• Base SHA: 822608f7ec5496791036f97d2d0c20a0a1656648.
• Base commit time: 2026-07-24T05:33:34Z.
• Re-verified at 2026-07-24T09:22Z; the official main head was unchanged.
• Recent adjacent changes reviewed: lineage coalescing (fix(publication): coalesce result lineages #807), delivered-verdict recovery (fix(repair): stop the review pipeline from stranding delivered verdicts #816), bounded reconciliation (fix(publication): bound historical lineage backfill #818), tupleless stale-member cleanup (fix(publication): stale tupleless batch items stop retrying #820), isolated bootstrap (fix(publication): stabilize batch clone bootstrap #821), parallel preparation (fix(publication): prepare batches concurrently #822), and eight concurrent owners (improve(publication): raise concurrent batch owners to eight #824).

Those changes improve cleanup, recovery, or capacity; none adds bounded fairness between current authority and historical cleanup.

Branch / Base Provenance

• Base: openclaw/clawsweeper:main@822608f7ec5496791036f97d2d0c20a0a1656648.
• Head: snowzlmbot/clawsweeper:fix/fresh-authority-admission@20b7f32282a81aedc3a478be3b92ae6bc816fa7e.
• The fork default branch was not used for development and contains no commits from this change.
• Runtime evidence is isolated on snowzlmbot/clawsweeper:evidence/issue-826-pr-827-runtime; it is not part of the PR head.

Dependency / Contract Verification

No dependency or lockfile changes. The implementation uses existing queue state, publication tuple, head-revision, batch ownership, and telemetry contracts.

Labels Considered

bug, performance, and operations. No contributor label permissions are assumed.

Validation

• git diff --check passed on exact head 20b7f32282a81aedc3a478be3b92ae6bc816fa7e.
• CI run 30078386507 passed pnpm check and the repository smoke jobs on that head.
• CodeQL run 30078386542 passed both Actions and JavaScript/TypeScript analysis on that head.
• Added deterministic tests for bounded fresh admission with historical progress.
• Added a rollback test proving the disabled flag restores strict FIFO.
• Added a post-claim source-head drift test proving the captured member is superseded before preparation.
• Added configuration-contract assertions for the two-of-eight reservation and 15-minute bound.
• Controlled runtime proof 30082176987 passed against the exact PR head in an isolated Wrangler/Workerd Durable Object runtime.

Controlled Runtime Proof

The evidence workflow checks out the exact PR head by immutable SHA, starts the real Worker and SQLite-backed Durable Object under Wrangler/Workerd, and drives the signed queue HTTP endpoints. It uses ephemeral local storage and performs no production queue or external GitHub mutations.

To keep the bounded proof under two minutes, only the recency window is shortened from the production value of 900 seconds to the implementation's minimum of 60 seconds. The batch size and reservation remain the proposed production values: eight members with two fresh slots. The normal exact-head CI separately asserts the committed 900-second configuration.

Observed results:

• first batch: fresh items 1090, 1091; historical items 1001 through 1006;
• second batch: remaining fresh item 1092 and historical items 1007, 1008;
• a claimed source revision that advanced before fetch was terminalized as superseded and returned zero preparation items;
• the newer source revision remained claimable and completed in the controlled runtime;
• final controlled state: pending 0, published 12, superseded 1;
• external GitHub mutations 0; production queue mutations 0.

Evidence

• Live queue dashboard — 2026-07-24T06:20:22Z: publication pending 802; 60-minute published 0; superseded 340; oldest age 119,469 seconds; health critical.
• Historical batch run 30070345350 — successful workflow, but final output reported materialized=0 and superseded=8, demonstrating that successful FIFO batches can spend all bounded slots on historical cleanup.
• Exact-head CI 30078386507 and CodeQL 30078386542 passed.
• Controlled runtime proof 30082176987 exercised bounded fresh service, continued historical progress, and post-claim source-drift fencing on exact head 20b7f32282a81aedc3a478be3b92ae6bc816fa7e.
• Issue #826 records the independent fairness gap and non-goals.

Evidence Source Map

• Production reproduction: public queue status endpoint and run 30070345350.
• Source-level reproduction: strict publication ordering in exactReviewQueueAdmittedItems and exactReviewPublicationBatchDeparture on the recorded base SHA.
• Regression proof: upstream CI and CodeQL runs on exact PR head.
• Runtime proof: isolated GitHub-hosted Wrangler/Workerd Durable Object run 30082176987 on exact PR head.

Sanitized Logs

2026-07-24T06:20:22Z publication.pending=802
last_60_minutes: resolved=340 published=0 superseded=340
oldest_pending_age_seconds=119469 health=critical
batch 30070345350: materialized=0 superseded=8
controlled runtime 30082176987: first_batch fresh=2 historical=6
controlled runtime 30082176987: second_batch fresh=1 historical=2
controlled runtime 30082176987: source_drift=superseded preparation_items=0
controlled runtime 30082176987: final pending=0 published=12 superseded=1

Media Evidence

Not applicable to the scheduler behavior itself. The public status endpoint, exact-head CI, controlled runtime transcript, and artifact report are the authoritative evidence. No UI rendering behavior changes.

Risk

• A fresh candidate can still become stale after admission; the added fetch-time head check and existing live mutation fences fail closed.
• Historical drain receives fewer positions only when current authoritative work exists, and the reserve is capped at two of eight.
• Selecting a fresh owner's batch can produce a partial owner-homogeneous batch; the existing concurrency cap, wait bound, and single-writer coordinator remain unchanged.
• Telemetry can distinguish actual published outcomes from superseded cleanup; queue-depth reduction alone remains insufficient for rollout success.
• The two-of-eight reservation and 15-minute production window remain an explicit maintainer policy decision; this PR stays Draft until that direction is confirmed.

Rollback

Set EXACT_REVIEW_PUBLICATION_FRESH_LANE_ENABLED=0. This is admission-only: active leases complete under their existing immutable fences, no stored queue data is deleted or migrated, and subsequent admission returns to strict historical FIFO.

Docs Updated

docs/state-publication-batching-plan.md documents authority classification, lane borrowing, preserved safety boundaries, telemetry, success criteria, and rollback.

Related Issue / PR

• Closes perf(publication): prevent historical FIFO cleanup from delaying current exact-review results #826.
• Related architecture and recovery work: RFC: single-writer state materialization via the queue DO (retire git-lease concurrency) #738, fix(publication): coalesce result lineages #807, fix(repair): stop the review pipeline from stranding delivered verdicts #816, fix(publication): bound historical lineage backfill #818, fix(publication): stale tupleless batch items stop retrying #820, fix(publication): stabilize batch clone bootstrap #821, fix(publication): prepare batches concurrently #822, improve(publication): raise concurrent batch owners to eight #824.

Not Tested / Limitations

• The change has not been deployed to production.
• The controlled runtime shortens the recency window to 60 seconds; exact-head CI verifies that the committed production value remains 900 seconds.
• Production acceptance still requires maintainer confirmation of the scheduling policy and post-deployment observation of current-result latency, published > 0, continued historical drain, and unchanged retry/dead-letter safety.

[snowzlmbot]

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

• State: Waiting
• Detail: Another exact-head review is already active. The durable queue will retry after its lease expires.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/30083840314
• Updated: 2026-07-24T09:48:15.856Z

[clawsweeper]

Codex review: needs maintainer review before merge. Reviewed July 24, 2026, 5:45 AM ET / 09:45 UTC.

ClawSweeper review

What this changes

The PR reserves up to two positions in each eight-item exact-review publication batch for recent protocol-v2 results that still match the queue head, while leaving the remaining positions in historical FIFO order and rechecking source drift before preparation.

Merge readiness

⚠️ Ready for maintainer review - 3 items remain

This is a focused, evidence-backed scheduler change with credible controlled-runtime proof, but it intentionally introduces a new production admission policy. Keep this PR open and draft until a maintainer confirms whether reserving two of eight batch positions for recent authoritative results, with a 15-minute recency bound, is the desired operational policy.

Priority: P2
Reviewed head: 20b7f32282a81aedc3a478be3b92ae6bc816fa7e
Owner decision: Required. See Decision needed.

Review scores

Measure	Result	What it means
Overall readiness	🐚 platinum hermit (4/6)	The patch is focused and well-covered with strong real-runtime proof; the remaining merge gate is maintainer approval of the production scheduling policy rather than an identified implementation defect.
Proof confidence	🦞 diamond lobster (5/6) ✨ media proof bonus	Sufficient (linked_artifact): The PR supplies exact-head controlled-runtime evidence from a real Wrangler/Workerd Durable Object setup, showing the improved admission result, continued historical progress, and post-claim drift fencing; no visual proof is relevant for this scheduler change.
Patch quality	🐚 platinum hermit (4/6)	No actionable review findings were identified.

Verification

Check	Result	Evidence
Real behavior	Verified	Sufficient (linked_artifact): The PR supplies exact-head controlled-runtime evidence from a real Wrangler/Workerd Durable Object setup, showing the improved admission result, continued historical progress, and post-claim drift fencing; no visual proof is relevant for this scheduler change.
Evidence reviewed	5 items	Current admission surface: The PR targets the Durable Object queue admission path, where current main constructs publication stats and batch ownership exclusions before scheduling publication work; the proposed fresh-lane classification is confined to that existing scheduler boundary. Bounded policy and rollback: The PR configures an enabled two-item reserve in an eight-item batch with a 900-second recency bound, and documents that disabling the flag restores strict historical FIFO without cancelling active leases. Regression coverage: The branch adds deterministic coverage for reserved fresh admission with historical progress, disabled-flag FIFO behavior, and post-claim source-head drift terminalization.
Findings	None	None.
Security	None	None.

How this fits together

Exact-review results enter a Durable Object publication queue before batch workers prepare artifacts and publish marker-backed review outcomes to GitHub and the state repository. The changed admission scheduler chooses which ready publications are claimed next while the existing ownership, revision, artifact, and mutation fences protect delivery.

flowchart LR
  A[Completed exact reviews] --> B[Durable publication queue]
  B --> C[Admission scheduler]
  C --> D[Recent authoritative reserve]
  C --> E[Historical FIFO selection]
  D --> F[Claimed publication batch]
  E --> F
  F --> G[Revalidate source revision]
  G --> H[Prepare and publish results]

Loading

Decision needed

Question	Recommendation
Should production publication batches reserve two of eight positions for authoritative results created within 15 minutes, or should maintainers choose a different fairness allocation before this scheduler policy ships?	Keep the proposed policy: Approve the two-of-eight, 15-minute reserve and merge with a post-deployment observation gate for current-result latency and historical drain.

Why: The patch preserves existing safety fences and provides a rollback flag, but the chosen ratio and time window intentionally trade some historical FIFO latency for bounded current-result service; that is an operational product decision rather than a mechanical correctness question.

Before merge

• Resolve merge risk (P1) - Enabling the new default deliberately changes the service order for an existing backlog: historical rows retain six reserved batch positions, but their completion latency can increase whenever current authoritative work exists.
• Resolve merge risk (P1) - The two-of-eight allocation and 15-minute recency bound are operational policy choices that controlled fixtures validate mechanically but cannot establish as the correct production balance without maintainer acceptance and post-deployment telemetry.

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Patch surface	5 files affected; 421 added, 16 removed	The change is concentrated in queue admission, deployment configuration, documentation, and focused scheduler/workflow tests.
Admission allocation	2 reserved of 8 batch positions	This is the concrete production policy tradeoff maintainers must approve before enabling the new default.

Merge-risk options

Maintainer options:

1. Confirm the production fairness policy (recommended)
   Approve the two-of-eight reserve and 15-minute bound only with a defined rollout observation window covering published outcomes, current-result latency, historical drain, retries, and dead-letter safety.
2. Keep the current FIFO policy pending direction
   Leave this PR draft if maintainers do not yet want existing historical publication order to yield capacity to newer authoritative results.

Technical review

Best possible solution:

Agree on the desired fairness policy, land one explicit and reversible configuration, then observe current-result latency, actual published outcomes, historical drain, retry behavior, and dead-letter safety in production before further tuning.

Do we have a high-confidence way to reproduce the issue?

Yes, at source level: current admission is historical FIFO and the supplied production telemetry plus controlled exact-head Worker run describe the starvation scenario and the proposed bounded-service result. A production deployment has not yet validated the chosen operating point.

Is this the best way to solve the issue?

Unclear; the admission design is narrow, reversible, and supported by deterministic and controlled-runtime evidence, but the specific two-of-eight and 15-minute policy requires maintainer acceptance before it can be called the best production solution.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 822608f7ec54.

Labels

Label changes:

• add P2: This is a bounded publication-scheduling improvement with real operational impact but no demonstrated immediate outage or user-facing regression caused by the patch.
• add merge-risk: 🚨 compatibility: The enabled default changes existing batch ordering and can increase historical-row latency during sustained fresh authoritative traffic.
• add proof: sufficient: Contributor real behavior proof is sufficient. The PR supplies exact-head controlled-runtime evidence from a real Wrangler/Workerd Durable Object setup, showing the improved admission result, continued historical progress, and post-claim drift fencing; no visual proof is relevant for this scheduler change.
• add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.

Label justifications:

• P2: This is a bounded publication-scheduling improvement with real operational impact but no demonstrated immediate outage or user-facing regression caused by the patch.
• merge-risk: 🚨 compatibility: The enabled default changes existing batch ordering and can increase historical-row latency during sustained fresh authoritative traffic.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
• proof: sufficient: Contributor real behavior proof is sufficient. The PR supplies exact-head controlled-runtime evidence from a real Wrangler/Workerd Durable Object setup, showing the improved admission result, continued historical progress, and post-claim drift fencing; no visual proof is relevant for this scheduler change.

Evidence

What I checked:

• Current admission surface: The PR targets the Durable Object queue admission path, where current main constructs publication stats and batch ownership exclusions before scheduling publication work; the proposed fresh-lane classification is confined to that existing scheduler boundary. (dashboard/exact-review-queue.ts:1825, 822608f7ec54)
• Bounded policy and rollback: The PR configures an enabled two-item reserve in an eight-item batch with a 900-second recency bound, and documents that disabling the flag restores strict historical FIFO without cancelling active leases. (dashboard/wrangler.toml:64, 20b7f32282a8)
• Regression coverage: The branch adds deterministic coverage for reserved fresh admission with historical progress, disabled-flag FIFO behavior, and post-claim source-head drift terminalization. (test/exact-review-publication-batches.test.ts:1918, 20b7f32282a8)
• Real runtime proof: The PR body links a controlled Wrangler/Workerd Durable Object run on the exact head that reports a first batch of two fresh plus six historical rows, continued progress in both lanes, and a drifted claimed revision terminalized before preparation. (20b7f32282a8)
• Adjacent feature history: Publication lineage coalescing was merged as commit 205b92b, while the later eight-owner capacity rollout merged as commit 8001184; the proposed reservation addresses admission fairness rather than either prior concern. (dashboard/exact-review-queue.ts:1825, 8001184d2223)

Likely related people:

• steipete: Authored the open single-writer publication architecture RFC and merged publication lineage coalescing, both of which define the queue and delivery boundaries this policy changes. (role: adjacent architecture contributor; confidence: high; commits: 205b92b9b6dc; files: dashboard/exact-review-queue.ts, docs/state-publication-batching-plan.md)
• vincentkoc: Merged the recent increase to eight concurrent publication owners, making the proposed two-of-eight admission allocation directly adjacent to their capacity work. (role: recent publication-capacity contributor; confidence: high; commits: 8001184d2223; files: dashboard/wrangler.toml, test/state-writer-workflow.test.ts)
• yetval: Merged delivery-recovery work in the publication pipeline, providing relevant context on preserving the existing guarded path while changing upstream admission. (role: recent delivery-path contributor; confidence: medium; commits: cf912d55a3c7; files: docs/state-publication-batching-plan.md)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

• Obtain explicit maintainer confirmation of the two-of-eight and 15-minute production policy.
• After deployment, record current-result latency, published-versus-superseded outcomes, historical drain, retries, and dead-letter safety before tuning the reserve.

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

[Takhoffman]

@clawsweeper automerge

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

[Takhoffman]

Maintainer policy approval recorded: enable the fresh-authority lane by default with exactly 2 reserved positions in each 8-member publication batch and a 15-minute freshness window. Preserve 6 historical FIFO positions and the documented immediate rollback via EXACT_REVIEW_PUBLICATION_FRESH_LANE_ENABLED=0. This product-decision gate is approved for landing and rollout.