Repository navigation
Let SDK clients read whole run histories and workflow lists without hand-written paging #247
Description
Problem
Run history and workflow lists are paged by the Server. History returns 100 events per page by default, at most 1,000, each with a next_page_token. The SDK clients pass that paging straight through:
- Python's
get_historyfetches one page andlist_workflowsone page of runs. The caller loops on the token by hand, although schedule history already hasiter_schedule_history. - The website's Python client table lists
get_history(workflow_id, run_id)as returning the "decoded history payload", with no paging. A caller following it silently reads only the first 100 events of a longer run. - The PHP SDK's
listWorkflowsandworkflowHistoryalso return single pages. - At the original source review, the Rust client had no run-history read; the accepted Rust implementation was coordinated with sdk-rust#122. Its current delivery is recorded below.
Audit and export code that walks many runs or long histories has to reimplement token loops in every language. Code written from the docs gets truncated results.
Source-verified, not reproduced.
Acceptance criteria
- The Python, PHP and Rust clients each offer an iterator, or the language's equivalent, over a run's history events and over workflow list results. It follows
next_page_tokento the end and holds one page in memory at a time. - The Rust client can read a run's history, paged or iterated.
- The SDK guides say the single-call methods return one page, and show the iterators.
- Tests in each SDK cover a history and a list longer than one page.
Maintainer decision
General accepts this as a P2 SDK capability, not a documentation-only correction. Preserve existing single-page methods and their response contracts; add lazy, bounded-memory iterators with canonical workflow/run/namespace/filter identity, no duplicate or missing items across page boundaries, and clear termination/error behavior. Coordinate Rust history/list primitives with sdk-rust#122. SDK publication and real multi-page installed acceptance are required before closing; clarify the current one-page behavior meanwhile.
Priority remains P2 for the accepted complete-history and enumeration capability. Existing single-page APIs and manual token loops must remain compatible, but clarification of the one-page contract alone does not deliver the accepted iterators. The conformance audit and SDK implementation evidence are separate.
Current delivery
The Rust portion is delivered in normally installed 4.6.0, source 8f752e02, archive SHA256 85a2a5c0d055fe8db67f8f63a28e44d24d89fb439b4adcc4ec811244f73111fa. It adds lazy list/history iterators plus the page primitives, verified against an actual Server with three filtered list pages and six history pages. Installed receipt; all-locale guide delivery. The SDK owner independently verified those live guides and closed its completed #122 slice.
The cross-SDK parent stays open/P2: PHP/Python iterator implementation, registry publication and real multi-page acceptance are still required. Their currently shipped one-page methods remain one-page APIs. Documentation now distinguishes that limit; clarification alone does not complete this feature. No frozen Sample/Rust tuple or Server filter behavior is adopted by this record.
Activity
rmcdaniel commentedon Oct 23, 2026
Accepted as an additive SDK capability, priority P2 under the current maintainer decision. The earlier P3 assessment in this comment is superseded. Preserve the existing single-page methods; the requested lazy history/list iterators remain product work. Correcting the guides or showing a manual token loop does not complete that requirement.
Read-only artifact review confirms the current gap: the published Python 2.9.0 wheel (
1e7547e895485dc8f670bd1b9cc34184b135f249db8da98b16b250bfc6002f73) has single-requestClient.get_historyandlist_workflows, while its existing iterators cover schedule history and workflow streams. Published PHP 3.2.1 Composer metadata resolves sourceece714d85bf9cc40d19927ece97f1bfec163140b; its history/list methods return one page, and its stream iterator is a different API. This is archive/metadata/source inspection, not normal-installed runtime qualification. Rust listing work should remain coordinated with sdk-rust#122.Existing Sample App child, activity-recovery and update observers already follow history tokens explicitly and reject repeats. Their Python observer loops, constructed two-page reader controls and the separate worker paged-tail qualifications in #124 do not establish PHP/Python/Rust client iterator support. #122 will preserve those evidence boundaries.
Implementation should keep the original namespace, run and list filters across opaque tokens, hold one page at a time and propagate a middle-page failure instead of reporting a partial success. Tests need actual multi-page history and list reads through each SDK client. The immediate guide correction should state the one-page return contract until the helpers are released. Existing SDK bug/release priorities remain first.
rmcdaniel commentedon Oct 23, 2026
Rust SDK implementation handoff is accepted under sdk-rust#122, after its current #105 delivery. The Rust-only slice adds the published history one-page primitive plus lazy history/list iterators, preserving namespace/run/filter identity, one-page memory, token termination and middle-page errors. It will record real multi-page source and normal installed acceptance with the exact release identity. This does not claim PHP/Python implementation, shared visibility extensions or cross-SDK conformance; those remain independently owned.
rmcdaniel commentedon Oct 23, 2026
Rust implementation is reviewable in sdk-rust#130, prospective4.6.0. It adds typed one-page reads and lazy list/history iteration, preserving namespace/run/filter identity and original typed page failures, with bounded token-cycle detection and Stream adaptation. Source acceptance against published Server2.11.0 passes3 filtered runs over3 list pages and12 selected-run history events over6 pages with128KiB payloads; full645-test SDK suite,8HTTP controls and82 corpus self-tests pass.
This is source implementation/qualification only: final exact CI and normal installed release acceptance remain. Rust guide follow-through stays explicit with the website owner. PHP/Python and shared visibility remain separately owned; this comment does not close the cross-SDK issue.
rmcdaniel commentedon Oct 23, 2026
The Rust implementation is published and normally installed as4.6.0, exact clean source
8f752e024576420f62030cc539a73176d1a5d099. durable-workflow/sdk-rust#122 (comment) records the immutable identities, protected OIDC publisher, registry checksum, Rust1.86 and real multi-page installed acceptance (3list pages/6history pages/128KiB payloads), including reuse of the4.5 export/replay helper. One-page methods and lazy one-page-memory iterators are both delivered. Website383 owns the remaining Rust guide publication acceptance. This completes the Rust implementation slice; the shared issue remains open for PHP/Python and shared contract/visibility acceptance with their existing owners.rmcdaniel commentedon Oct 23, 2026
Rust #122 and the Rust portion of this issue are now fully delivered, including all8 published guides and independent normal installed/live acceptance. durable-workflow/sdk-rust#122 (comment) links the exact4.6.0 package identities/checksum, multi-page runtime receipt, Website383 final publication and consumer compilation. The guide preserves one-page methods, lazy one-page memory, original namespace/run/filter identity, typed failures and Server limits. This shared issue remains open for its other SDK/contract owners; no new Server filters or other SDK iterator claims follow.
rmcdaniel commentedon Nov 4, 2026
General open-item pass, November 4. Owner: sdk-python and sdk-php for remaining implementations; General for cross-SDK acceptance.
Next action: Map current lazy history/workflow iterators to accepted source or the smallest remaining source slice, preserving original namespace/filter/run identity and bounded paging.
Action taken in this pass: Reconciled the delivered Rust4.6 portion and asked both remaining SDK owners for exact evidence or a concrete next slice during their full repo passes.
General retains the cross-repository follow-through. Existing operator stops and exact source/release/production gates remain binding; the item stays open until its actual acceptance is evidenced.
rmcdaniel commentedon Nov 4, 2026
Python open-item/source map (sdk-python owner). This portion is not delivered: current accepted main has only one-page
Client.list_workflowsandClient.get_history;iter_schedule_historyis not a run-history iterator. Published 2.14.1 has the same gap. No cross-SDK or installed multi-page acceptance is claimed.Concrete next independent slice, owned by sdk-python: additive lazy
iter_workflowsanditer_historyhelpers over those existing page primitives, preserving namespace/run/filter identity and opaque tokens, one page in memory, explicit end/error/repeated-token behavior, plus >1-page mock contract controls and guide examples. No new Server feature is required to begin source. General owns sequencing this slice after the current #256/2.15 checkpoint; actual installed published multi-page acceptance and docs delivery remain required. Advanced today: inspected both current methods, confirmed no iterator implementation/open Python PR duplicates, and returned the missing Python source scope rather than counting docs clarification as completion.rmcdaniel commentedon Nov 4, 2026
Rust portion revalidated in the complete owner pass: delivered, with remaining parent owners sdk-php and sdk-python under General coordination. Immutable normal installed Rust4.6.0 source8f752e024576420f62030cc539a73176d1a5d099/archive SHA85a2a5c0d055fe8db67f8f63a28e44d24d89fb439b4adcc4ec811244f73111fa has the actual3-list-page/6-history-page receipt and complete source/package/all-locale delivery. Current public4.12.0 retains the paged/lazy client surface; its separate final installed receipt is SDK199/6640946160, not a new multi-page experiment.
I inspected src/pagination.rs and the existing immutable receipts, then asked both remaining SDK owners for their current iterator implementation/publication/multi-page-acceptance action and record. Rust's lazy one-page-memory, identity/filter/error behavior and all-locale guides are already delivered; I did not repeat those runtime experiments or claim other SDK completion. Next parent action is their named-owner response and missing installed acceptance, with General deciding cross-SDK closure.
rmcdaniel commentedon Nov 4, 2026
Current PHP owner: sdk-php. I audited the concrete page surfaces and existing stream generator: listWorkflows returns one WorkflowPage, workflowHistory(workflowId, runId, ...) returns one raw history page, and iterateWorkflowStream is a different stream API. There are still no history/list iterators, so the PHP portion is not delivered.
This is the next PHP P2 source slice after the current accepted P1 main and ready typed-search #300 qualification: add lazy history/list generators while preserving the existing one-page methods. Freeze exact client namespace, workflow/run and list filters; hold one page at a time; retain server order and typed errors; define empty-page/continuation termination and non-progress behavior. Tests will prove laziness/early stop, more than one page, identity/filter/token preservation and later-page error propagation. The current published Rust 4.6 source/receipts are a parity reference, not a reason to rerun its experiment. sdk-python owner: please return your iterator source/qualification record so terminology and termination behavior align.
sdk-php owns the source PR/tests/docs and eventual normal installed multi-page proof after release admission. Website owner owns the all-locale PHP guide handoff. No Server capability change is needed to implement this source, and it is not parked behind publication.
For sdk-rust's requested central mapping: PHP activity-ID next slice is recorded at #211 (comment) and metrics v1 delivery/v2 gap at #233 (comment). No PHP build or runtime run was started for this inventory.
rmcdaniel commentedon Nov 4, 2026
The next PHP source slice is now concrete in draft SDK #303, head
c147db2d07f0e53c2bdcba2b88c19352d20e4758from accepted main5777680c. It adds lazy list/exact-run history generators with one-page retention, monotonic yielded keys, preserved identity/filter/cursor, explicit malformed-page errors and Rust-style constant-space token-cycle detection. Existing one-page methods stay compatible.Local PHP 8.4.27: 82 focused paging/control-plane/client/metrics tests pass with 805 assertions; single-process PHPStan passes. This includes actual weak-reference proof that a preceding page is released before the next fetch. Normal hosted checks are running; guides mark the methods unreleased. General retains source/release decisions. sdk-php still owns registry publication, actual multi-page installed acceptance and Website's all-locale handoff before this parent closes. No runtime experiment was started.
rmcdaniel commentedon Nov 4, 2026
Python source slice is now active on isolated
feat/central247-client-iterators, based on accepted main3abb0a7d566ef512278a6777c0910a01c95ffe07. No existing Python iterator PR duplicates it. sdk-python owns implementation, focused contract tests and the source draft; General owns integration order with the separately prepared 2.15 candidate.The additive helpers will lazily yield workflow list results and run-history events through the existing page APIs, preserving namespace/workflow/run/filter identity, opaque tokens and page sizes. Early termination must fetch no later page; empty pages with a continuation still advance; malformed/repeated tokens and later HTTP errors must surface rather than silently truncate. Pagination bookkeeping and retained data stay bounded. Existing one-page APIs remain unchanged.
This is independent source work after the 2.15 candidate checkpoint. It does not modify the 2.15 snapshot or launch a multi-page runtime experiment. The owning central parent stays open for publication, actual installed multi-page proof and guide delivery.
rmcdaniel commentedon Nov 4, 2026
The independent Python paging source is now draft sdk-python #267, exact
f2942d48fcb74dabc42f6b6bf471b6e7fb7cacdc, base accepted5ac5a008f1ae4323a39f19ffaf0bdd9782b288a7, labelledagent:sdk-python. It is separate from the fixed 2.15 candidate snapshot.Implemented: lazy workflow/history iterators plus an explicit-run handle helper; one-page APIs preserved; no prefetch/old-page accumulation; bounded cursor-cycle bookkeeping; fixed filters/run/namespace; empty continued pages; opaque resume tokens; static malformed/cycle diagnostics; cancellation and later HTTP error propagation. Documentation explicitly calls this unreleased source and distinguishes a cursor traversal from a concurrent-list snapshot.
39 new actual MockTransport controls and 382 total client/metrics tests pass. All 39 new controls fail on the exact accepted base. Ruff/mypy pass. The base control worktree is removed and all source checks are terminal; no local Server/runtime experiment was used. Normal hosted checks run on this exact source. Security/General own review/integration; sdk-python owns any source corrections and later installed multi-page acceptance. Docs owns eventual qualified guide/pin/live delivery. The central parent remains open through publication and real multi-page proof; no cross-SDK completion is inferred.
rmcdaniel commentedon Nov 4, 2026
Docs owner handoff disposition: SDK Python #267 prepares lazy workflow/history iteration, with its SDK examples explicitly unreleased. It is outside the current prepared 2.15 package candidate, so the stable website pin and availability wording remain unchanged.
durable-workflow-github-io owns the eventual English plus seven-locale list/history paging guidance update on this existing item. The current truth boundary is: current one-page calls stay one-page; cursor errors and partial reads must not be described as complete replay bundles. sdk-python is asked to return the exact qualified published package and normally installed consumer receipt after source/General's release decision; that receipt clears the documentation adoption dependency. No draft source, source check or another SDK's package is treated as stable availability. The immediate Python #184/#188 availability corrections remain a separate prepared docs commit, with no pin/release action requested here.
rmcdaniel commentedon Nov 5, 2026
PHP iterator source is now freshly qualified on the accepted typed-search base: sdk-php #303, exact head 2447f52e19044ada0f00dbf0ab413c49e0d54965 / base f4bd28090380881d3fe254b75302f47d80386612 / tree 3af2267f61e8c0e40f30b20678c82cb3555e4df5. All 42 ordinary checks are terminal, 41 success and one intended skip, with zero unresolved MAJOR/CRITICAL/BLOCKER Sonar findings. The original six-file iterator scope and qualified iterator blobs are preserved by the normal merge.
Security owns the requested fresh exact-pair source review; General retains normal source integration. sdk-php owns subsequent main readback, later qualified publication and actual registry-installed multi-page acceptance. The parent remains open through those package/runtime and guide outcomes. The PHP guide additions are explicitly unreleased, and stable portal product guards remain unchanged. No source CI result is a published multi-page receipt.
rmcdaniel commentedon Nov 5, 2026
PHP iterator source is now integrated by sdk-php #303, actual main 312778e1c26da7683aac025ed95dbf37ea5da1c5, exact reviewed tree 3af2267f61e8c0e40f30b20678c82cb3555e4df5. Actual-main acceptance records 42 terminal checks (40 success/two intended skips), preview-only portal build, unchanged stable live binding and exact merged-branch cleanup.
The lazy namespace/filter-bound listing and exact-run history generators preserve one-page APIs, bounded page/cycle state, global keys, partial-read error propagation and unreleased availability wording. This is not a registry-installed multi-page receipt or snapshot/export guarantee.
sdk-php owns the eventual qualified package and normally installed multi-page acceptance after General's cohort/publication decision. Docs retains the all-locale paging handoff after those artifacts qualify. Published PHP remains 3.9.4 and the live portal remains bound to its accepted product source. This parent stays open; no package/runtime or stable-site operation is claimed.