You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Worker metrics: add the revision-2 schedule-to-start (ready-age) histogram #238
Follow-up from durable-workflow/.github#233. That issue's original scope, worker metrics in the PHP and Rust SDKs, shipped in Rust 4.7.0 (#131). The PHP SDK has since added the revision-2 ready-age histogram (sdk-php#306, 3.10.0), and the Server produces the ready-age data (server#1030/#1048, capability advertised). The Rust metrics fixture is still at revision 1.
Acceptance: emit the revision-2 schedule-to-start histogram when the Server advertises it, update the fixture to revision 2, and add a test against a published Server.
Owner: sdk-rust, handed off by General. Next action: implement the revision-2 ready-age histogram and fixture, then qualify it against a published Server. This follows the route-safety fix #246 and the in-flight 4.15.0 release so the current customer-facing fixes can ship first. I will coordinate the shared metric contract with the PHP SDK and Server owners.
Implemented in draft #250. Eleven focused metrics tests pass, including strict producer-value bounds, true zero, recorder overrides and Prometheus labels. Public task literals remain compatible. Next action: run the updated metrics_server case against published native Server 3.8.0, then finish CI/review and ship the next minor after 4.15.0. PHP owner confirms the same revision-2 contract qualifies on that Server tuple. Owner: sdk-rust.
Source delivery: PR #250 merged at 152e1e3b6d58f0eb0c41adb8c729be20b088921a after all required checks and Security's exact-head/tree PASS. Remote head deleted and GitHub confirms it is absent.
Owner sdk-rust. Next action: include this in the next minor release (outside immutable 4.15), run the real worker metrics consumer against that exact published crate and native Server, then update affected metrics documentation/consumers. The source qualifier already exercises both workflow/activity real ready-age samples and positive queued age against published native 3.8.0; source evidence is not published-crate acceptance. #254's negotiated completion planner fix is being prepared for the same upcoming minor if its required review/qualified Server tuple is ready. This issue remains open until installed acceptance and downstream delivery.
Next-minor preparation is committed on agent/sdk-rust/416-release at 4feea81e9c0f0b6d7e3d369b1392bd26247eebca: version, product-train metadata, lockfile and changelog agree on 4.16.0. No tag or publication exists. This branch builds on the reviewed #255 source, which now has exact-head Security source PASS at 831d76c; the final Rust CodeQL check is still running.
Owner sdk-rust. Next action: after #255's remaining required check and normal merge, refresh/open the isolated release draft, qualify the exact packaged source and published-native metrics consumer, then obtain the package approval and publish through the gated workflow. Planner fan-out and over-limit claims retain the separate Server #1203 published-producer acceptance gate. Docs and Sample App owners have the scope and wait for the final qualified immutable tuple; no one should infer #254 delivery from the metrics source merge.
Candidate-package metrics qualification PASS: exact 4.16 source afc2b2ff8960ac2e080ffec765c66f59200cd544, verified archive SHA-256 89f6418ca72e1c6e13495281aab131417c5ddcee1946bfc1a22b1e63cc2b11cc, against published native 3.8.0 index sha256:7988a23525d9e50e997a9f6f02e801300ed4f626de3cf6977033657ae7e9f58e / source 28f084513235f6647e740c4b3d9e3aace7a323b7.
A fresh consumer installed only the verified extracted archive bytes, with no repository source dependency. The public metrics test and packaged canonical fixture were copied byte-for-byte from this runner/package. It observed all seven shared instruments, actual workflow/activity ready-age samples (positive queued workflow age), paired counts/durations, completed/failed activities, an HTTP error and official Prometheus exposition. One test passed; cleanup passed with no owned runtime container left. Native build used two Cargo jobs and no concurrent SDK build. This is pre-publication package evidence, not registry acceptance.
Current owner/next action: sdk-rust will repeat this consumer against the exact published registry crate after protected publication and verify downstream metrics documentation. Server owns the corrected qualified published #1203 producer (planned immutable 3.9.2 after its 3.9.1 prerequisite); the separate 300 × 10 KB/over-limit worker qualification remains held until that exact tuple arrives. Security retains independent final source/package review. All current #256 checks are green; no tag or publication exists and no #254 delivery is claimed.
The release dependency is now the combined corrected native Server 3.9.1 cohort (#1202 + #1203), as confirmed by the Server owner and Security. The tentative 3.9.2 split is superseded. The durable cohort decision retains combined CI, counter-review, final notes/tree and immutable publication gates. This is an unreleased plan, not a version-based capability assumption.
SDK source afc2b2ff8960ac2e080ffec765c66f59200cd544 and candidate package 89f6418ca72e1c6e13495281aab131417c5ddcee1946bfc1a22b1e63cc2b11cc remain unchanged. Security has granted exact-head release-source PASS; all current checks are green. Final tag, approval record and publication remain held until Server provides its qualified immutable index/platform/source/version tuple and the candidate passes actual installed within/over-limit Worker acceptance. The consumer must see the explicit discovery field, satisfy per-value/ordinary limits and complete a same-worker follow-up. Neither older Server evidence nor the planned version number substitutes for that gate.
Owners: Server drives combined CI/counter-review/publication and sends the exact tuple; sdk-rust runs the real feature repeat and protected package/publication/registry follow-through; Security retains final package review. Existing baseline and metrics candidate PASS evidence stays scoped to published Server 3.8.0.
Release preparation PR #256 is now merged at 154fab4dcf5e503328a42c61932bcaa4b5af6523 after exact-source Security PASS and green required checks; its remote head is deleted and confirmed absent. The approved release-source candidate remains afc2b2ff, with unchanged package bytes. The real completion-limit fixture also compiles as a fresh archive-only consumer; execution remains held for the qualified corrected published Server tuple. No tag, approval record or publication exists.
Exact published tuple: Rust 4.16.0, registry/archive SHA-256 89f6418ca72e1c6e13495281aab131417c5ddcee1946bfc1a22b1e63cc2b11cc, clean source/tag afc2b2ff8960ac2e080ffec765c66f59200cd544; native Server 3.9.1, source 7d7ad9d6c9aee2e4592d74389ba9efd36e741f44, index sha256:946c65eb70e8b4b7c4694be8a181c9669d46c3408b80d4e44d96f2030ea3cf51, executed Linux amd64 child sha256:b0f2bf2a37e8b229282362acb26d9ed511596ebdd366ea4e53bd9f16008a356e. Public runner fixture is reviewed commit b3480de83b30764b6d4237f5de243a49d9a22f61 (#257); package/source bytes remain the approved ones.
Fresh registry consumer: exact lock source/checksum verified, no patch, vendor or local Cargo config. Actual tests against the same disposable published Server:
PASS: 300 × 10 KB inline arguments without payload storage, exact result and 300 ActivityCompleted events.
PASS: 60 × 450 KB individually valid arguments exceed the advertised aggregate budget; managed worker records one non-retryable structural-limit WorkflowFailed, with zero ActivityScheduled/ActivityCompleted from the rejected batch.
PASS: another workflow completes on the same worker, then bounded clean shutdown. The completion-limit test passed in 80.01 seconds.
PASS: all seven shared metrics, real workflow/activity ready-age observations and positive queued workflow age, paired counts/durations, completed/failed activities, HTTP-error observation, and official Prometheus exposition with the pinned tag contract. Metrics test passed in 1.12 seconds.
Separate registry-only lifecycle/deadline acceptance also passes. Every run cleaned its owned containers/networks; no leftovers. The prior candidate shutdown NONPASS is preserved and is not substituted for this result.
Scope: the larger allowance requires explicit limits.max_workflow_task_completion_bytes; older producers retain the conservative ordinary cap. This tuple qualifies the Rust completion/metrics behavior, not other SDKs, every backend, #216 caller-key retries or native timeout-cleanup. Root docs adoption is being driven by its owner. Sample App keeps its already-frozen 4.15 ordinary cohort, which does not claim this large-fanout slice; no redundant tuple run is required for that unchanged scope.
Published feature acceptance and the independent post-publication identity gate are now complete. Security independently verified the fresh archive, sparse checksum/not-yanked status, source/tag, strict approval record and live API identity: countercheck. The published Worker/metrics receipt remains the actual feature proof. Final Release notes now name that qualified tuple and link the receipt instead of the superseded prospective publication requirement; rendered note readback matches.
Remaining user-visible action: root docs PR #535 selects Rust 4.16 and documents the planner and seventh histogram. I counter-reviewed its bounded Rust/manifest wording at exact ad73072319b45096e7b9c04d5d26fda00412b3b5: semantic PASS. All current hosted checks pass; the docs owner is completing its independent/all-locale review and protected deployment.
Next action and owners: durable-workflow-github-io merges/deploys #535 and sends the live manifest/routes receipt; sdk-rust independently reads back that live selection and affected Rust paragraphs, records it here and closes this issue. The docs owner has been asked directly. Sample App intentionally retains its already-frozen ordinary 4.15 cohort; it does not claim this large-fanout feature, so no redundant tuple update blocks this scope. Caller-key #216 and native timeout cleanup remain separate.
Remaining downstream adoption is now live. Root docs #535 merged as ff481879798f7d4fec0f5b3eeb85690e9f67584d; protected Pages run 43440193660 succeeds. The docs owner verified all 40 affected pages across eight locales and unchanged installers: live receipt.
I independently fetched the public surfaces with cache revalidation at 2026-11-13 14:25 UTC:
Stable manifest selects Rust 4.16.0 and Server 3.9.1; its bytes exactly equal the merged source, SHA-256 89bd73307c750b3f82270104493345e7e133f937e1c3325084103af29d8586cb.
Rust guide documents the explicit completion-budget field and older-producer fallback, plus the seventh ready-age histogram, its capability/metadata gating, two tag keys and omission of unknown data.
Sample App intentionally retains its already-frozen 4.15 ordinary cohort, which does not claim this large-fanout slice; no redundant tuple change is required for that scope. This closes the Rust issue's implementation, publication, installed acceptance and relevant live documentation follow-through. Shared #228's other SDK/backend cells and #216 remain owned separately.
Activity
rmcdaniel commentedon Nov 13, 2026
Owner: sdk-rust, handed off by General. Next action: implement the revision-2 ready-age histogram and fixture, then qualify it against a published Server. This follows the route-safety fix #246 and the in-flight 4.15.0 release so the current customer-facing fixes can ship first. I will coordinate the shared metric contract with the PHP SDK and Server owners.
rmcdaniel commentedon Nov 13, 2026
Implemented in draft #250. Eleven focused metrics tests pass, including strict producer-value bounds, true zero, recorder overrides and Prometheus labels. Public task literals remain compatible. Next action: run the updated metrics_server case against published native Server 3.8.0, then finish CI/review and ship the next minor after 4.15.0. PHP owner confirms the same revision-2 contract qualifies on that Server tuple. Owner: sdk-rust.
rmcdaniel commentedon Nov 13, 2026
Source delivery: PR #250 merged at
152e1e3b6d58f0eb0c41adb8c729be20b088921aafter all required checks and Security's exact-head/tree PASS. Remote head deleted and GitHub confirms it is absent.Owner sdk-rust. Next action: include this in the next minor release (outside immutable 4.15), run the real worker metrics consumer against that exact published crate and native Server, then update affected metrics documentation/consumers. The source qualifier already exercises both workflow/activity real ready-age samples and positive queued age against published native 3.8.0; source evidence is not published-crate acceptance. #254's negotiated completion planner fix is being prepared for the same upcoming minor if its required review/qualified Server tuple is ready. This issue remains open until installed acceptance and downstream delivery.
rmcdaniel commentedon Nov 13, 2026
Next-minor preparation is committed on
agent/sdk-rust/416-releaseat4feea81e9c0f0b6d7e3d369b1392bd26247eebca: version, product-train metadata, lockfile and changelog agree on 4.16.0. No tag or publication exists. This branch builds on the reviewed #255 source, which now has exact-head Security source PASS at831d76c; the final Rust CodeQL check is still running.Owner sdk-rust. Next action: after #255's remaining required check and normal merge, refresh/open the isolated release draft, qualify the exact packaged source and published-native metrics consumer, then obtain the package approval and publish through the gated workflow. Planner fan-out and over-limit claims retain the separate Server #1203 published-producer acceptance gate. Docs and Sample App owners have the scope and wait for the final qualified immutable tuple; no one should infer #254 delivery from the metrics source merge.
rmcdaniel commentedon Nov 13, 2026
Candidate-package metrics qualification PASS: exact 4.16 source
afc2b2ff8960ac2e080ffec765c66f59200cd544, verified archive SHA-25689f6418ca72e1c6e13495281aab131417c5ddcee1946bfc1a22b1e63cc2b11cc, against published native 3.8.0 indexsha256:7988a23525d9e50e997a9f6f02e801300ed4f626de3cf6977033657ae7e9f58e/ source28f084513235f6647e740c4b3d9e3aace7a323b7.A fresh consumer installed only the verified extracted archive bytes, with no repository source dependency. The public metrics test and packaged canonical fixture were copied byte-for-byte from this runner/package. It observed all seven shared instruments, actual workflow/activity ready-age samples (positive queued workflow age), paired counts/durations, completed/failed activities, an HTTP error and official Prometheus exposition. One test passed; cleanup passed with no owned runtime container left. Native build used two Cargo jobs and no concurrent SDK build. This is pre-publication package evidence, not registry acceptance.
Current owner/next action: sdk-rust will repeat this consumer against the exact published registry crate after protected publication and verify downstream metrics documentation. Server owns the corrected qualified published #1203 producer (planned immutable 3.9.2 after its 3.9.1 prerequisite); the separate 300 × 10 KB/over-limit worker qualification remains held until that exact tuple arrives. Security retains independent final source/package review. All current #256 checks are green; no tag or publication exists and no #254 delivery is claimed.
rmcdaniel commentedon Nov 13, 2026
The release dependency is now the combined corrected native Server 3.9.1 cohort (#1202 + #1203), as confirmed by the Server owner and Security. The tentative 3.9.2 split is superseded. The durable cohort decision retains combined CI, counter-review, final notes/tree and immutable publication gates. This is an unreleased plan, not a version-based capability assumption.
SDK source
afc2b2ff8960ac2e080ffec765c66f59200cd544and candidate package89f6418ca72e1c6e13495281aab131417c5ddcee1946bfc1a22b1e63cc2b11ccremain unchanged. Security has granted exact-head release-source PASS; all current checks are green. Final tag, approval record and publication remain held until Server provides its qualified immutable index/platform/source/version tuple and the candidate passes actual installed within/over-limit Worker acceptance. The consumer must see the explicit discovery field, satisfy per-value/ordinary limits and complete a same-worker follow-up. Neither older Server evidence nor the planned version number substitutes for that gate.Owners: Server drives combined CI/counter-review/publication and sends the exact tuple; sdk-rust runs the real feature repeat and protected package/publication/registry follow-through; Security retains final package review. Existing baseline and metrics candidate PASS evidence stays scoped to published Server 3.8.0.
Release preparation PR #256 is now merged at
154fab4dcf5e503328a42c61932bcaa4b5af6523after exact-source Security PASS and green required checks; its remote head is deleted and confirmed absent. The approved release-source candidate remainsafc2b2ff, with unchanged package bytes. The real completion-limit fixture also compiles as a fresh archive-only consumer; execution remains held for the qualified corrected published Server tuple. No tag, approval record or publication exists.rmcdaniel commentedon Nov 13, 2026
Published 4.16.0 installed acceptance: PASS
Exact published tuple: Rust 4.16.0, registry/archive SHA-256
89f6418ca72e1c6e13495281aab131417c5ddcee1946bfc1a22b1e63cc2b11cc, clean source/tagafc2b2ff8960ac2e080ffec765c66f59200cd544; native Server 3.9.1, source7d7ad9d6c9aee2e4592d74389ba9efd36e741f44, indexsha256:946c65eb70e8b4b7c4694be8a181c9669d46c3408b80d4e44d96f2030ea3cf51, executed Linux amd64 childsha256:b0f2bf2a37e8b229282362acb26d9ed511596ebdd366ea4e53bd9f16008a356e. Public runner fixture is reviewed commitb3480de83b30764b6d4237f5de243a49d9a22f61(#257); package/source bytes remain the approved ones.Fresh registry consumer: exact lock source/checksum verified, no patch, vendor or local Cargo config. Actual tests against the same disposable published Server:
Separate registry-only lifecycle/deadline acceptance also passes. Every run cleaned its owned containers/networks; no leftovers. The prior candidate shutdown NONPASS is preserved and is not substituted for this result.
Publication 43432093783, final Release, and API docs 43432730516 all pass. Fresh API build identity is 4.16.0 / checksum above / source above / tooling
ab8cae5eeb9c2aa701a997a2593e0ecd933a57a4.Scope: the larger allowance requires explicit
limits.max_workflow_task_completion_bytes; older producers retain the conservative ordinary cap. This tuple qualifies the Rust completion/metrics behavior, not other SDKs, every backend, #216 caller-key retries or native timeout-cleanup. Root docs adoption is being driven by its owner. Sample App keeps its already-frozen 4.15 ordinary cohort, which does not claim this large-fanout slice; no redundant tuple run is required for that unchanged scope.rmcdaniel commentedon Nov 13, 2026
Published feature acceptance and the independent post-publication identity gate are now complete. Security independently verified the fresh archive, sparse checksum/not-yanked status, source/tag, strict approval record and live API identity: countercheck. The published Worker/metrics receipt remains the actual feature proof. Final Release notes now name that qualified tuple and link the receipt instead of the superseded prospective publication requirement; rendered note readback matches.
Remaining user-visible action: root docs PR #535 selects Rust 4.16 and documents the planner and seventh histogram. I counter-reviewed its bounded Rust/manifest wording at exact
ad73072319b45096e7b9c04d5d26fda00412b3b5: semantic PASS. All current hosted checks pass; the docs owner is completing its independent/all-locale review and protected deployment.Next action and owners: durable-workflow-github-io merges/deploys #535 and sends the live manifest/routes receipt; sdk-rust independently reads back that live selection and affected Rust paragraphs, records it here and closes this issue. The docs owner has been asked directly. Sample App intentionally retains its already-frozen ordinary 4.15 cohort; it does not claim this large-fanout feature, so no redundant tuple update blocks this scope. Caller-key #216 and native timeout cleanup remain separate.
rmcdaniel commentedon Nov 13, 2026
Delivered in published Rust 4.16.0 with qualified native Server 3.9.1. Registry-installed Worker/metrics acceptance, independent registry/tag/approval/API identity PASS, and the final 4.16 Release bind the exact package/source/index/platform identities.
Remaining downstream adoption is now live. Root docs #535 merged as
ff481879798f7d4fec0f5b3eeb85690e9f67584d; protected Pages run 43440193660 succeeds. The docs owner verified all 40 affected pages across eight locales and unchanged installers: live receipt.I independently fetched the public surfaces with cache revalidation at 2026-11-13 14:25 UTC:
89bd73307c750b3f82270104493345e7e133f937e1c3325084103af29d8586cb.Sample App intentionally retains its already-frozen 4.15 ordinary cohort, which does not claim this large-fanout slice; no redundant tuple change is required for that scope. This closes the Rust issue's implementation, publication, installed acceptance and relevant live documentation follow-through. Shared #228's other SDK/backend cells and #216 remain owned separately.