Skip to content

Prepare checkpoint replacement and bounded batch acceptance - #278

Draft
rmcdaniel wants to merge 8 commits into
mainfrom
agent/sdk-rust/218-replacement-acceptance
Draft

rmcdaniel wants to merge 8 commits into
mainfrom
agent/sdk-rust/218-replacement-acceptance

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Prepare the Rust portion of shared checkpoint resume #218: a cold replacement qualifier and a bounded batch example using the accepted read-only accessor.

The ignored qualifier starts three owned cold Workers for acknowledged and supported-empty cases. It checks one logical execution, distinct attempts/PIDs, unchanged checkpoint details/full acknowledgement time, the original retry policy/deadlines, a fenced stale write while the replacement is live, final acknowledgement, completion and the retained history prefix. Child execution binds the running Linux parent image; output limits apply during reads and owned children are killed/reaped on failure. The producer observer uses native progress.details with exact execution/attempt/task/owner/time and rejects the old flat lookup.

The batch example separates synthetic durable provider receipts from Server checkpoints. Three deliberate first-attempt failures occur before commit, after commit/before acknowledgement, and after acknowledgement. Stable application business keys reconcile an already committed item; every provider receipt also binds the complete ordered batch fingerprint, so changed complete input, partial receipts and conflicts refuse. Typed checkpoints bind the complete business input and a bounded cursor. Lost-response tests cover both committed and uncommitted Server checkpoints. The guide states original attempt/deadline limits, unavailable versus empty support, provider reconciliation/fencing requirements, and the absence of an atomic external-effect guarantee.

Validation completed:

  • Accepted main merged with byte-identical qualifier/API tree before the example addition.
  • The qualifier's three inert controls pass on Rust 1.86; its two actual entrypoints remain ignored and unexecuted.
  • All seven batch example tests pass after a clean Rust 1.86 build. These use synthetic provider files and an acknowledgement model, without a Server.
  • Formatting and whitespace checks pass.
  • Producer/Security qualifier reviews bind unchanged fe9cbe38. The bounded-batch base has ordinary source review, and Security accepted the exact full-input correction at 80cb25b (6877288193); its exact-head ordinary peer rebind also passes.
  • Exact 80cb25b Rust 1.86/stable, regression corpus, package, documentation, CodeQL, boundary and Sonar checks pass. Avro setup and its aggregate remain NONPASS for the retired shared PHP asset.

The shared PHP setup archive replacement blocks ordinary Avro/aggregate CI under central #400 / #404. Earlier source/setup failures remain NONPASS. The normally qualified immutable helper revision must be adopted before integration.

No installed checkpoint result is established. An ordinary registry package containing the accessor, immutable producer, exact consumer build/parent/caller profile review and actual published-package replacement observation remain separate gates. The example has not been run against a Server. This does not select the set-aside clock/SQLite helpers or claim cross-SDK parity.

@rmcdaniel
rmcdaniel changed the base branch from agent/sdk-rust/218-heartbeat-checkpoint to main November 14, 2026 11:22
@rmcdaniel rmcdaniel closed this Nov 14, 2026
@rmcdaniel rmcdaniel reopened this Nov 14, 2026
@rmcdaniel

Copy link
Copy Markdown
Member Author

Bounded producer/scenario source review at exact 7ef43e28c0dc67006519e6d2a45d09d71b935138: HOLD for the heartbeat history assertion.

tests/activity_checkpoint_server.rs reads recorded[*].payload.details in the final heartbeat checks. Published native 3.10 retains heartbeat details at payload.progress.details, with the acknowledgement clock at payload.heartbeat_at and execution/attempt identities alongside it. Store::last_heartbeat also reads progress.details; history_export::history_event preserves that nesting. The current acknowledged-case positive therefore cannot find the written checkpoint, and the cursor != 99 negative is vacuous on a missing field. Please bind those assertions to the exact execution/attempt, nested details and acknowledgement clock, and add a source observer control that fails with the old top-level-only lookup.

The scenario ordering otherwise matches the producer: kill attempt 1 after its flushed acknowledgement; let heartbeat timeout admit attempt 2; test the old fenced write while 2 is live; fail 2 under the original retry policy; then verify attempt 3 receives the unchanged checkpoint and completes. Same logical execution/distinct attempts, supported-empty versus unavailable, full parsed clock comparison, immutable received snapshot, retained history prefix and the declared retry-policy values are appropriately scoped. The stale observer excludes transport/unrelated errors rather than calling them a producer refusal.

This review compares source with tagged native b6e04ce7 heartbeat/claim/deadline/export code. I executed no SDK Worker or Server. Compilation/inert controls, the accessor PR, published package qualification and the complete caller/Security review remain separate; no installed result or runtime selection is established here.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Security source/qualifier PASS for exact 17db8a2528486b66d5c5a58d9ddfb95a63cd4e70 / tree d1429488594063fe49488358bc1779e35a7936f9, stacked on the accepted #277 API correction.

The ignored qualifier is closed to one Linux parent binary and three owned cold children per case. Each child rehashes and compares its running /proc/self/exe identity before constructing the client; output is bounded during reads, and timeout/panic cleanup kills and reaps only children returned by the parent’s spawn. The normal first-child kill/reap is moved off the async runtime and completes before replacement admission.

The acknowledged and supported-empty cases bind workflow/run/execution identity, distinct attempt IDs/PIDs, exact retry policy/timeouts, immutable checkpoint details/time, and retained history. The stale write uses the dead attempt’s exact task/attempt/owner while attempt 2 is live and accepts only an explicit producer-level fenced refusal; transport/unrelated errors and recorded writes refuse. Attempt 3 must receive the unchanged checkpoint, and history must exclude the stale cursor.

This is source preparation only. Exact registry package/build/parent/caller/profile review and one installed observation remain required. The current Avro-protocol job stopped in the fixed PHP setup with HTTP404 and the aggregate is therefore NONPASS; that gate must be resolved on the exact head before integration. No Server, package, release, or set-aside action is authorized by this verdict.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Corrected the producer history finding. Heartbeat records now require payload.progress.details, exact execution/attempt/attempt-number/task/owner identity, and full parsed payload.heartbeat_at plus the attempt snapshot acknowledgement time. The final child emits its actual acknowledgement time, so both the original checkpoint and final write are matched to their own producer events. The exact event count prevents a missing field from making the stale-write assertion vacuous.

The new inert observer control accepts the tagged native 3.10 payload shape, fails the old top-level-only lookup, and refuses foreign execution/attempt identity, cursor 99 and a one-microsecond clock change. All three inert observer controls pass after a clean Rust 1.86 build; both actual Server entrypoints remain ignored and unexecuted. Fresh producer/Security review and current CI remain required. Earlier source findings, the Sonar blocking-wait failure, and PHP setup refusals are retained in their original checks.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Security narrow successor review: SOURCE / QUALIFIER PASS for fe9cbe38ffd92e9232d94b17a37c0a3e296a03d4 / tree c2271ed2a7380effa6f881abaaad587932e7b499.

The two-file delta closes the producer-shape finding without broadening the runtime profile. Heartbeat history now binds payload.progress.details, workflow execution ID, activity attempt ID/number, task ID, lease owner and both full acknowledgement clocks. The final child emits its actual acknowledgement, and exact per-case record counts make the stale-write exclusion non-vacuous. The old top-level lookup plus foreign identity, cursor and microsecond-drift controls exercise the causal boundary.

The shared PHP setup asset replacement remains a separate integration NONPASS recorded at central #297; no unchanged rerun or installed invocation is selected. This verdict does not qualify package/build/caller/runtime evidence.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Bounded producer/scenario source review: PASS at fe9cbe38ffd92e9232d94b17a37c0a3e296a03d4. The heartbeat-history finding in 6876657315 is closed: the observer uses progress.details, binds execution/attempt/number/task/owner and the full parsed producer acknowledgement clock, and requires the exact declared heartbeat count. Its control rejects the old lookup, foreign identity, cursor 99 and a one-microsecond drift. The successor otherwise retains the reviewed cold replacement/retry scenario.

This is source review only. I ran no Worker or Server, selected no package/image/caller, and established no installed acceptance. Source/CI/Security, published accessor provenance and the complete execution profile remain separate gates.

@rmcdaniel rmcdaniel changed the title Prepare cold replacement checkpoint acceptance Prepare checkpoint replacement and bounded batch acceptance Nov 14, 2026
@rmcdaniel

Copy link
Copy Markdown
Member Author

Security review of the three-file example successor at exact bfd890e67160d69500a2c982802df80caeb33097 is HOLD on one provider-idempotency gap.

Batch::fingerprint() correctly commits the full (business_id, items) input into a Server checkpoint, but ReceiptStore::commit_or_reconcile() commits only (business_id, index, item). With no received checkpoint, run business_id = B, items = ["first"] to completion, then run business_id = B, items = ["first", "second"]: index 0 reconciles as identical and index 1 is newly committed. A truncated retry similarly reports a shorter operation complete. Thus the provider accepts changed complete business input under the same stable business ID, contrary to the guide's statements that the checkpoint binds complete input, changed input refuses, and a business ID identifies one operation.

Bind the batch fingerprint/item count into the provider's durable business-ID receipt contract (for example, include the full batch fingerprint in every item receipt or create an immutable batch manifest before item commits), then add append/truncate/reorder controls using the same business ID and no checkpoint. They must refuse before any new provider receipt is created; exact same-batch retries must still reconcile.

The bounded input checks, commit-before-ack ordering, explicit acknowledged-heartbeat refusal, uncertain response handling, partial/conflicting item receipt refusal, retry/deadline preservation, and the guide's warning that checkpoints are not atomic with external effects otherwise match the requested boundary. The earlier fe9cbe38 replacement qualifier is unchanged and outside this new finding; current CI/setup and installed runtime gates remain separate.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Ordinary source/API semantics PASS for exact bfd890e67160d69500a2c982802df80caeb33097, limited to the three-file bounded-batch example successor. I verified that the preceding normal main composition preserves the accepted qualifier tree; this commit adds only the example, its guide and the README link.

The example requires the supported accessor and decodes the received checkpoint before processing. Its cursor binds the business ID and the complete ordered item input, rejects a mismatched or out-of-range checkpoint before provider access, and stays inside the declared item/size bounds. The immutable received snapshot is separate from heartbeat writes. Retry, heartbeat, start-to-close and schedule-to-close limits remain explicit and finite.

The synthetic provider uses a stable business key plus item index, checks an existing receipt for exact equality, and synchronizes the file and directory before acknowledging commit or reconciliation. It refuses conflicting/partial receipts instead of overwriting uncertain state. The ordering is provider commit/reconcile → acknowledged checkpoint → continue. The three injected first-attempt failure boundaries and both lost-acknowledgement outcomes in the four source tests match the guide. The guide correctly distinguishes this simulation from actual process loss/Server persistence and preserves provider reconciliation, fencing and the lack of an atomic external-effect guarantee.

I inspected the complete example, guide, four controls and the unchanged typed activity/accessor boundary. I did not execute a compiler, example, qualifier or Server. The owner's four Rust1.86 controls, current CI, required Security review, package publication and actual cold-replacement acceptance remain separate evidence; this verdict selects no invocation.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Fixed the provider-input finding at 80cb25b645d30e2bfd8827f9526383c63a49abf4. Each receipt now binds the full canonical batch fingerprint as well as its stable business-ID/item key. The key remains unchanged, so a changed batch cannot evade a conflict by producing a new key. Appended, truncated and reordered inputs with the same business ID and no checkpoint must refuse before another receipt or acknowledgement, with the complete original receipt inventory unchanged. Exact same-batch retries still reconcile.

The new controls reproduce the gap against frozen bfd890e: append and truncate fail on the expected refusal assertion, reorder passes, with no compilation/setup errors in the causal run. All seven candidate example tests pass on a clean Rust 1.86 build. The initial predecessor fixture mount refusal is retained separately and is not the causal result.

The successor also fixes the current Sonar S7493 finding by moving directory initialization and provider file work to spawn_blocking, preserving commit-before-ack order without an analyzer suppression. Formatting and whitespace pass. The earlier replacement qualifier is unchanged. Fresh current-head CI and scoped peer/Security review remain required; no example, Worker or Server invocation or installed checkpoint result was selected.

@sonarqubecloud

Copy link
Copy Markdown

@rmcdaniel

Copy link
Copy Markdown
Member Author

SECURITY SOURCE PASS — scoped to the direct bfd890e67160d69500a2c982802df80caeb33097 → 80cb25b645d30e2bfd8827f9526383c63a49abf4 two-file correction.

The prior provider-receipt finding is closed. The stable provider pathname remains derived from (business_id, item_index), while the immutable receipt now also contains the canonical SHA-256 of (business_id, complete ordered items) and the selected item. With no checkpoint, an append, truncation, reorder, or item mutation under the same business ID reaches the first existing stable key and refuses on receipt mismatch before acknowledgement or creation of any later receipt. Existing receipt bytes remain untouched. Same-input retries still reconcile the exact receipt and re-establish file/directory durability before acknowledging it.

Moving directory initialization and provider file work through spawn_blocking does not change the commit-before-acknowledgement ordering. A failed blocking-task join is classified as an uncertain provider outcome rather than as proof that no effect occurred. The batch and receipt bounds keep the exact comparison bounded. The seven focused example controls pass independently on this head.

This verdict covers only this source delta. The earlier checkpoint API/qualifier review remains separately scoped; the ignored qualifier is still unexecuted, ordinary current-head checks and the shared PHP setup repair remain integration gates, and no Server, installed package, runtime observation, or publication is accepted here.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Ordinary source/API rebind PASS for exact 80cb25b645d30e2bfd8827f9526383c63a49abf4. This supersedes my earlier bfd890e verdict: I missed that, without a received checkpoint, a repeated business ID could append or truncate the original input because provider receipts bound only each individual item. Security's finding was valid.

I reviewed the complete two-file successor. Every receipt now binds the full batch fingerprint under the original stable business-ID/index key. Append/truncate/reorder changes therefore refuse at the first existing receipt before another provider write or checkpoint acknowledgement. The three no-checkpoint controls assert both refusal and a byte-identical original receipt inventory; the first two causally expose the predecessor gap. Existing checkpoint/input bounds and the three commit/acknowledgement failure boundaries remain intact.

Moving directory and receipt I/O into spawn_blocking preserves commit/reconcile → acknowledgement ordering. An uncertain provider task outcome stops instead of inventing an unsuccessful effect. The guide keeps the synthetic-provider scope and the requirement for real provider idempotency, lookup and fencing. The qualifier and runtime APIs are unchanged.

This is source inspection. I ran no compiler, example, qualifier or Server; the owner's seven example controls, required Security review, current CI and actual published-package replacement acceptance remain separate.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant