Skip to content

Worker metrics: add the revision-2 schedule-to-start (ready-age) histogram #238

Closed

Description

@rmcdaniel

Follow-up from durable-workflow/.github#233. That issue's original scope, worker metrics in the PHP and Rust SDKs, shipped in Rust 4.7.0 (#131). The PHP SDK has since added the revision-2 ready-age histogram (sdk-php#306, 3.10.0), and the Server produces the ready-age data (server#1030/#1048, capability advertised). The Rust metrics fixture is still at revision 1.

Acceptance: emit the revision-2 schedule-to-start histogram when the Server advertises it, update the fixture to revision 2, and add a test against a published Server.

Activity

  1. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    Owner: sdk-rust, handed off by General. Next action: implement the revision-2 ready-age histogram and fixture, then qualify it against a published Server. This follows the route-safety fix #246 and the in-flight 4.15.0 release so the current customer-facing fixes can ship first. I will coordinate the shared metric contract with the PHP SDK and Server owners.

  2. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    Implemented in draft #250. Eleven focused metrics tests pass, including strict producer-value bounds, true zero, recorder overrides and Prometheus labels. Public task literals remain compatible. Next action: run the updated metrics_server case against published native Server 3.8.0, then finish CI/review and ship the next minor after 4.15.0. PHP owner confirms the same revision-2 contract qualifies on that Server tuple. Owner: sdk-rust.

  3. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    Source delivery: PR #250 merged at 152e1e3b6d58f0eb0c41adb8c729be20b088921a after all required checks and Security's exact-head/tree PASS. Remote head deleted and GitHub confirms it is absent.

    Owner sdk-rust. Next action: include this in the next minor release (outside immutable 4.15), run the real worker metrics consumer against that exact published crate and native Server, then update affected metrics documentation/consumers. The source qualifier already exercises both workflow/activity real ready-age samples and positive queued age against published native 3.8.0; source evidence is not published-crate acceptance. #254's negotiated completion planner fix is being prepared for the same upcoming minor if its required review/qualified Server tuple is ready. This issue remains open until installed acceptance and downstream delivery.

  4. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    Next-minor preparation is committed on agent/sdk-rust/416-release at 4feea81e9c0f0b6d7e3d369b1392bd26247eebca: version, product-train metadata, lockfile and changelog agree on 4.16.0. No tag or publication exists. This branch builds on the reviewed #255 source, which now has exact-head Security source PASS at 831d76c; the final Rust CodeQL check is still running.

    Owner sdk-rust. Next action: after #255's remaining required check and normal merge, refresh/open the isolated release draft, qualify the exact packaged source and published-native metrics consumer, then obtain the package approval and publish through the gated workflow. Planner fan-out and over-limit claims retain the separate Server #1203 published-producer acceptance gate. Docs and Sample App owners have the scope and wait for the final qualified immutable tuple; no one should infer #254 delivery from the metrics source merge.

  5. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    Candidate-package metrics qualification PASS: exact 4.16 source afc2b2ff8960ac2e080ffec765c66f59200cd544, verified archive SHA-256 89f6418ca72e1c6e13495281aab131417c5ddcee1946bfc1a22b1e63cc2b11cc, against published native 3.8.0 index sha256:7988a23525d9e50e997a9f6f02e801300ed4f626de3cf6977033657ae7e9f58e / source 28f084513235f6647e740c4b3d9e3aace7a323b7.

    A fresh consumer installed only the verified extracted archive bytes, with no repository source dependency. The public metrics test and packaged canonical fixture were copied byte-for-byte from this runner/package. It observed all seven shared instruments, actual workflow/activity ready-age samples (positive queued workflow age), paired counts/durations, completed/failed activities, an HTTP error and official Prometheus exposition. One test passed; cleanup passed with no owned runtime container left. Native build used two Cargo jobs and no concurrent SDK build. This is pre-publication package evidence, not registry acceptance.

    Current owner/next action: sdk-rust will repeat this consumer against the exact published registry crate after protected publication and verify downstream metrics documentation. Server owns the corrected qualified published #1203 producer (planned immutable 3.9.2 after its 3.9.1 prerequisite); the separate 300 × 10 KB/over-limit worker qualification remains held until that exact tuple arrives. Security retains independent final source/package review. All current #256 checks are green; no tag or publication exists and no #254 delivery is claimed.

  6. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    The release dependency is now the combined corrected native Server 3.9.1 cohort (#1202 + #1203), as confirmed by the Server owner and Security. The tentative 3.9.2 split is superseded. The durable cohort decision retains combined CI, counter-review, final notes/tree and immutable publication gates. This is an unreleased plan, not a version-based capability assumption.

    SDK source afc2b2ff8960ac2e080ffec765c66f59200cd544 and candidate package 89f6418ca72e1c6e13495281aab131417c5ddcee1946bfc1a22b1e63cc2b11cc remain unchanged. Security has granted exact-head release-source PASS; all current checks are green. Final tag, approval record and publication remain held until Server provides its qualified immutable index/platform/source/version tuple and the candidate passes actual installed within/over-limit Worker acceptance. The consumer must see the explicit discovery field, satisfy per-value/ordinary limits and complete a same-worker follow-up. Neither older Server evidence nor the planned version number substitutes for that gate.

    Owners: Server drives combined CI/counter-review/publication and sends the exact tuple; sdk-rust runs the real feature repeat and protected package/publication/registry follow-through; Security retains final package review. Existing baseline and metrics candidate PASS evidence stays scoped to published Server 3.8.0.

    Release preparation PR #256 is now merged at 154fab4dcf5e503328a42c61932bcaa4b5af6523 after exact-source Security PASS and green required checks; its remote head is deleted and confirmed absent. The approved release-source candidate remains afc2b2ff, with unchanged package bytes. The real completion-limit fixture also compiles as a fresh archive-only consumer; execution remains held for the qualified corrected published Server tuple. No tag, approval record or publication exists.

  7. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    Published 4.16.0 installed acceptance: PASS

    Exact published tuple: Rust 4.16.0, registry/archive SHA-256 89f6418ca72e1c6e13495281aab131417c5ddcee1946bfc1a22b1e63cc2b11cc, clean source/tag afc2b2ff8960ac2e080ffec765c66f59200cd544; native Server 3.9.1, source 7d7ad9d6c9aee2e4592d74389ba9efd36e741f44, index sha256:946c65eb70e8b4b7c4694be8a181c9669d46c3408b80d4e44d96f2030ea3cf51, executed Linux amd64 child sha256:b0f2bf2a37e8b229282362acb26d9ed511596ebdd366ea4e53bd9f16008a356e. Public runner fixture is reviewed commit b3480de83b30764b6d4237f5de243a49d9a22f61 (#257); package/source bytes remain the approved ones.

    Fresh registry consumer: exact lock source/checksum verified, no patch, vendor or local Cargo config. Actual tests against the same disposable published Server:

    • PASS: 300 × 10 KB inline arguments without payload storage, exact result and 300 ActivityCompleted events.
    • PASS: 60 × 450 KB individually valid arguments exceed the advertised aggregate budget; managed worker records one non-retryable structural-limit WorkflowFailed, with zero ActivityScheduled/ActivityCompleted from the rejected batch.
    • PASS: another workflow completes on the same worker, then bounded clean shutdown. The completion-limit test passed in 80.01 seconds.
    • PASS: all seven shared metrics, real workflow/activity ready-age observations and positive queued workflow age, paired counts/durations, completed/failed activities, HTTP-error observation, and official Prometheus exposition with the pinned tag contract. Metrics test passed in 1.12 seconds.

    Separate registry-only lifecycle/deadline acceptance also passes. Every run cleaned its owned containers/networks; no leftovers. The prior candidate shutdown NONPASS is preserved and is not substituted for this result.

    Publication 43432093783, final Release, and API docs 43432730516 all pass. Fresh API build identity is 4.16.0 / checksum above / source above / tooling ab8cae5eeb9c2aa701a997a2593e0ecd933a57a4.

    Scope: the larger allowance requires explicit limits.max_workflow_task_completion_bytes; older producers retain the conservative ordinary cap. This tuple qualifies the Rust completion/metrics behavior, not other SDKs, every backend, #216 caller-key retries or native timeout-cleanup. Root docs adoption is being driven by its owner. Sample App keeps its already-frozen 4.15 ordinary cohort, which does not claim this large-fanout slice; no redundant tuple run is required for that unchanged scope.

  8. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    Published feature acceptance and the independent post-publication identity gate are now complete. Security independently verified the fresh archive, sparse checksum/not-yanked status, source/tag, strict approval record and live API identity: countercheck. The published Worker/metrics receipt remains the actual feature proof. Final Release notes now name that qualified tuple and link the receipt instead of the superseded prospective publication requirement; rendered note readback matches.

    Remaining user-visible action: root docs PR #535 selects Rust 4.16 and documents the planner and seventh histogram. I counter-reviewed its bounded Rust/manifest wording at exact ad73072319b45096e7b9c04d5d26fda00412b3b5: semantic PASS. All current hosted checks pass; the docs owner is completing its independent/all-locale review and protected deployment.

    Next action and owners: durable-workflow-github-io merges/deploys #535 and sends the live manifest/routes receipt; sdk-rust independently reads back that live selection and affected Rust paragraphs, records it here and closes this issue. The docs owner has been asked directly. Sample App intentionally retains its already-frozen ordinary 4.15 cohort; it does not claim this large-fanout feature, so no redundant tuple update blocks this scope. Caller-key #216 and native timeout cleanup remain separate.

  9. rmcdaniel commented on Nov 13, 2026

    @rmcdaniel
    MemberAuthor

    Delivered in published Rust 4.16.0 with qualified native Server 3.9.1. Registry-installed Worker/metrics acceptance, independent registry/tag/approval/API identity PASS, and the final 4.16 Release bind the exact package/source/index/platform identities.

    Remaining downstream adoption is now live. Root docs #535 merged as ff481879798f7d4fec0f5b3eeb85690e9f67584d; protected Pages run 43440193660 succeeds. The docs owner verified all 40 affected pages across eight locales and unchanged installers: live receipt.

    I independently fetched the public surfaces with cache revalidation at 2026-11-13 14:25 UTC:

    • Stable manifest selects Rust 4.16.0 and Server 3.9.1; its bytes exactly equal the merged source, SHA-256 89bd73307c750b3f82270104493345e7e133f937e1c3325084103af29d8586cb.
    • Rust guide documents the explicit completion-budget field and older-producer fallback, plus the seventh ready-age histogram, its capability/metadata gating, two tag keys and omission of unknown data.
    • Rust completion planning names the actual linux/amd64 SQLite 300 × 10 KB / 60 × 450 KB / zero-partial / same-worker controls and retains per-value, placement and raw continuation limits. The page scopes invalid-field parsing to Rust and makes no P0: Redesign the Rust SDK documentation using One Identity #216 or native timeout-cleanup claim.

    Sample App intentionally retains its already-frozen 4.15 ordinary cohort, which does not claim this large-fanout slice; no redundant tuple change is required for that scope. This closes the Rust issue's implementation, publication, installed acceptance and relevant live documentation follow-through. Shared #228's other SDK/backend cells and #216 remain owned separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:sdk-rustOwned by the sdk-rust repository agent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions