Skip to content

Skip the workflow-task renewal before replay for a fresh poll grant #327

Closed

Description

@rmcdaniel

External performance report, topic 2b. The reported timings and proposed fixes have not been independently reproduced. The supplied patches are reference drafts for review and current-source qualification.

Problem

The Worker renews (heartbeats) every workflow task before it replays it.
That is 3,600 locked transitions across the reported three-repetition series, one per workflow task, even
though the grant was made moments earlier.

Proposal

Skip the renewal before replay when the poll answer meets all of these:

  • it states the Server clock (server_time);
  • it arrived under a second ago;
  • it has ten or more seconds of lease left.

Required safety properties to prove:

  • the completion is fenced whatever the worker sends first;
  • replay still renews on its normal schedule.

The rule reads the Server's clock from the answer, not the worker's.

Reported benchmark evidence

Reporter-supplied cumulative measurements, not an isolated-effect or independently verified performance claim: 200 sequentially started workflows, five sequential 20ms activities or five local activities each, six PHP worker processes, mean of three repetitions, one machine, fresh SQLite/MySQL8.0/PostgreSQL16 databases with default durability. "Five" and "local" below mean last-close wall time in seconds; S/M/P means SQLite/MySQL/PostgreSQL. Preserve regressions and noise in the reported results.

five local
slowest backend 40.52 → 31.49 9.44 → 7.09
S / M / P 22.31 / 31.49 / 21.08 —

Acceptance

  • No renewal is sent before replaying a fresh grant.
  • A grant that is old, or close to its expiry, or an answer without
    server_time (an older Server) is renewed as before.
  • tests/WorkerWorkflowTaskLeaseRetryTest.php covers each case.

Dependencies

Related existing work: #137.

Draft patches and validation boundary

Current main was independently read as afa5a95e3116cd69c731c821f00e8223d42f7ff6. The supplied draft series targets afa5a95e3116cd69c731c821f00e8223d42f7ff6; patches are cumulative reference material, not a current-main application or test result. Review and rebase on maintained source, preserve all existing tests and published compatibility, then establish causal controls and actual conformance before shipping. No patch was applied or executed during intake.

The following sanitized unified diffs identify proposed reference material:

  • 0002-t02b-Skip-the-workflow-task-renewal-before-replay-for-a-f.patch — 6,196 bytes, SHA-256 56caab1e3c97ddd2fe3b5d7b9daf385bfba719d8d4005b632e556312c975c7fd.

Next action: the owning maintainer reviews the candidate against current source and the acceptance criteria above. Preserve durability, lease/fencing, idempotency, fairness and whole-completion/history budgets. New producer capabilities need versioned schemas and real producer/consumer compatibility proof; historical/frozen evidence is not rewritten.

Maintainer source checkpoint (2026-11-06)

Source is normally merged and qualified in PHP SDK #334, main 588d680548851dcc998cbeb7e16626bc601dc473. Final main/docs/cleanup receipt binds all42 terminal main checks,45 focused controls,8,266 PHP8.4 tests, actual worker-corpus73→74 base-fail/candidate-pass and the source-bound preview artifact. Stable live docs are unchanged. The producer is Server#1070 at 866a4067d052ae70f22060fd656b6154d0debc36.

The optimization uses metadata before payload hydration, monotonic age and exact task/attempt/owner/expiry identity. It charges the full poll request and loading time; malformed or missing clocks, stale/short grants, changed fences and shutdown retain renewal. Completion fencing and local-activity/payload supervision remain covered.

Next owner/action: General qualifies publication of the coherent SDK release and verifies the actual published package against a supported published Server artifact containing Server#1070, plus older-Server renewal fallback. This issue stays open until that published behavior is verified. Source CI and reported benchmark timings do not establish installed or performance acceptance.

Activity

  1. added
    kind:featureA public product capability or experience is requested
    priority:P0Immediate release or data-integrity risk
    repo:sdk-phpOwned by the PHP SDK repository
    on Nov 6, 2026
  2. rmcdaniel commented on Nov 6, 2026

    @rmcdaniel
    MemberAuthor

    Sanitized reference diff 0002-t02b-Skip-the-workflow-task-renewal-before-replay-for-a-f.patch, part 1/1.

    This is an unverified, cumulative draft from the external report. Mail author/date/commit-message metadata was removed; the unified-diff hunks were retained. No patch was applied or executed, and no build/test or compatibility result is implied. The known gaps and acceptance criteria in this issue remain mandatory.

    Complete sanitized patch: 6,196 bytes, SHA-256 56caab1e3c97ddd2fe3b5d7b9daf385bfba719d8d4005b632e556312c975c7fd. This part: 6,196 bytes, SHA-256 56caab1e3c97ddd2fe3b5d7b9daf385bfba719d8d4005b632e556312c975c7fd. Parts concatenate in order; a part boundary is between complete file diffs.

    diff --git a/src/Worker.php b/src/Worker.php
    index 971ca54..b2e7312 100644
    --- a/src/Worker.php
    +++ b/src/Worker.php
    @@ -16,6 +16,7 @@ use DurableWorkflow\Exception\SagaCompensationFailed;
     use DurableWorkflow\Exception\ServerException;
     use DurableWorkflow\Exception\TransportException;
     use DurableWorkflow\Worker\ActivityContext;
    +use DurableWorkflow\Worker\ActivityLeaseAuthority;
     use DurableWorkflow\Worker\ActivityLeaseProcess;
     use DurableWorkflow\Worker\ActivityLeaseMaintenanceException;
     use DurableWorkflow\Worker\ActivityExecutionFailure;
    @@ -116,6 +117,8 @@ final class Worker
         /** @var (\Closure(): void)|null */
         private ?\Closure $unbindActivityLease = null;
         private ?CancellationRequest $claimCancellation = null;
    +    /** @var array{task_id: string, received: float, server_time: mixed}|null The grant the last workflow poll delivered. */
    +    private ?array $workflowGrant = null;
         private ?string $claimDeliveredCancellationId = null;
         private bool $preparedLocalActivityGroupsSupported = false;
         /** @var list<string> */
    @@ -803,6 +806,11 @@ final class Worker
             if ($this->stopForTerminalPoll($workflowPoll)) {
                 return false;
             }
    +        $this->workflowGrant = is_array($workflowPoll['task'] ?? null) ? [
    +            'task_id' => (string) ($workflowPoll['task']['task_id'] ?? ''),
    +            'received' => $this->now(),
    +            'server_time' => $workflowPoll['server_time'] ?? null,
    +        ] : null;
             $this->rememberWorkflowMemoCapability($workflowPoll);
             $capabilities = $workflowPoll['server_capabilities'] ?? null;
             $supportedCommands = is_array($capabilities) ? ($capabilities['supported_workflow_task_commands'] ?? null) : null;
    @@ -1428,7 +1436,7 @@ final class Worker
                     $this->observeClaimCancellation($task['cancellation_request']);
                 }
                 $history = $this->completeHistory($task, $leaseOwner, $attempt);
    -            if (!$this->renewWorkflowTaskLease($taskId, $leaseOwner, $attempt)) {
    +            if (!$this->freshWorkflowGrant($task) && !$this->renewWorkflowTaskLease($taskId, $leaseOwner, $attempt)) {
                     return;
                 }
                 if ($this->client->observabilityContextAvailable()) {
    @@ -1559,6 +1567,29 @@ final class Worker
                 && ($details['workflow_task_attempt'] ?? null) === (int) ($task['workflow_task_attempt'] ?? 1);
         }
     
    +    /**
    +     * Whether the poll delivered this task's grant under a second ago with ten
    +     * or more seconds left on the Server's clock. Such a grant needs no renewal
    +     * before replay: the completion is fenced, and replay renews on schedule.
    +     *
    +     * @param array<string, mixed> $task
    +     */
    +    private function freshWorkflowGrant(array $task): bool
    +    {
    +        $grant = $this->workflowGrant;
    +        $this->workflowGrant = null;
    +        if ($grant === null || $this->shutdownRequested || $grant['task_id'] !== (string) ($task['task_id'] ?? '')
    +            || $this->now() - $grant['received'] >= 1) {
    +            return false;
    +        }
    +        try {
    +            return ActivityLeaseAuthority::timestamp($task['lease_expires_at'] ?? null)
    +                - ActivityLeaseAuthority::timestamp($grant['server_time']) >= 10;
    +        } catch (\UnexpectedValueException) {
    +            return false;
    +        }
    +    }
    +
         private function renewWorkflowTaskLease(string $taskId, string $leaseOwner, int $taskAttempt): bool
         {
             $retryAttempt = 0;
    diff --git a/tests/WorkerWorkflowTaskLeaseRetryTest.php b/tests/WorkerWorkflowTaskLeaseRetryTest.php
    index 382dc19..8ef5cf7 100644
    --- a/tests/WorkerWorkflowTaskLeaseRetryTest.php
    +++ b/tests/WorkerWorkflowTaskLeaseRetryTest.php
    @@ -84,6 +84,60 @@ final class WorkerWorkflowTaskLeaseRetryTest extends TestCase
             self::assertEqualsWithDelta(1.0, $now, 0.000_01);
         }
     
    +    #[DataProvider('pollGrants')]
    +    public function testOnlyAFreshPollGrantSkipsTheRenewalBeforeReplay(string $leaseExpiresAt, int $renewals): void
    +    {
    +        $heartbeats = 0;
    +        $completions = 0;
    +        $transport = new FakeTransport(handler: static function (
    +            string $method,
    +            string $uri,
    +            array $headers,
    +            ?array $body,
    +        ) use ($leaseExpiresAt, &$heartbeats, &$completions): ?array {
    +            if (str_ends_with($uri, '/api/worker/workflow-tasks/poll')) {
    +                $poll = self::leasedWorkflowTask();
    +                $poll['task']['lease_expires_at'] = $leaseExpiresAt;
    +                $poll['server_time'] = '2026-07-18T00:39:00.000000Z';
    +
    +                return $poll;
    +            }
    +
    +            if (str_ends_with($uri, '/api/worker/workflow-tasks/workflow-task-1/heartbeat')) {
    +                ++$heartbeats;
    +
    +                return self::renewedLease();
    +            }
    +
    +            if (str_ends_with($uri, '/api/worker/workflow-tasks/workflow-task-1/complete')) {
    +                ++$completions;
    +
    +                return ['completed' => true];
    +            }
    +
    +            if (str_ends_with($uri, '/api/worker/activity-tasks/poll')) {
    +                return ['task' => null, 'poll_status' => 'stopped', 'reason' => 'worker_stopped'];
    +            }
    +
    +            self::fail("Unexpected worker request: {$method} {$uri}");
    +        });
    +        $worker = new Worker(new Client('https://server.example', transport: $transport), 'orders', workerId: 'worker-1');
    +        $worker->registerWorkflow('orders.workflow', static fn (WorkflowContext $context): string => 'completed');
    +
    +        self::assertTrue($worker->tick(0));
    +
    +        self::assertSame($renewals, $heartbeats);
    +        self::assertSame(1, $completions);
    +    }
    +
    +    /** @return iterable<string, array{string, int}> */
    +    public static function pollGrants(): iterable
    +    {
    +        yield 'a minute left' => ['2026-07-18T00:40:00.000000Z', 0];
    +        yield 'five seconds left' => ['2026-07-18T00:39:05.000000Z', 1];
    +        yield 'expiry without fractional digits' => ['2026-07-18T00:40:00Z', 1];
    +    }
    +
         public function testBackendLossRetriesTheSameWorkflowTaskHeartbeatFence(): void
         {
             $now = 0.0;
  3. rmcdaniel commented on Nov 6, 2026

    @rmcdaniel
    MemberAuthor

    The PHP repository agent has resumed ownership. I assessed a narrow P0 backport against exact published 3.9.4 source 11085f0c624674a4b70d3a6fb3e83eaff3869fcb, without bringing the combined 3.10 cohort along. The concrete candidate is a4c7a4a, with the complete stable-base delta.

    The 12-file delta changes only Worker helper/grant internals, their tests, the official worker fixture and additive corpus guards. All existing stable guards are preserved. composer.json, Client, Replayer, framework support, dependency constraints, capability declarations, protocol versions, docs and workflows are byte-identical to 3.9.4. The three helper/grant classes are byte-identical to accepted main. The only cherry-pick conflict was the corpus guard tail; unrelated 3.10 clock guards were excluded because their classes are absent on this base.

    Focused PHP 8.4.27 validation passes 65 tests / 349 assertions, covering helper reuse/death/forks/cleanup, TLS/auth, fencing, fresh versus missing/malformed/stale clocks, payload/history delay and renewal fallback. Full stable-source suite, static analysis and immutable-base consumer controls are in progress. This is a source assessment, not published-package or Server conformance.

    Next action / owners: sdk-php completes stable-source qualification and the eventual installed consumer acceptance. General owns the independent patch-cut disposition. Server #1070's additive workflow-poll clock is accepted source (866a4067) but is absent from the latest published native 3.0.5 release. The optimization therefore still needs an exact published Server tuple with that response clock for positive installed proof; missing clocks deliberately retain renewal. A safe patch can contain the guarded consumer without claiming the absent published producer or closing this issue. No 3.10/Workflow2.21/native3.2 requirement is silently backported, and no version/tag/dispatch/publication is selected.

  4. rmcdaniel commented on Nov 6, 2026

    @rmcdaniel
    MemberAuthor

    The independent stable-base source assessment is complete, and General selected the maintenance target in 326/6694918746. Draft #335 now targets 3.9.x, created at exact published 3.9.4 commit 11085f0c. Candidate f250aaec adds only CI/Sonar branch routing to the previously linked 12-file backport. All source checks, documentation/manual publisher behavior and the combined 3.10 cohort remain intact.

    Completed PHP 8.4.27 evidence: 65 focused tests / 349 assertions, the full stable-source suite of 3,790 tests / 19,875 assertions / 56 optional skips, PHPStan and Composer validation. The unchanged-base helper consumer fails with three helper PIDs instead of one. The official fresh-grant worker fixture fails against unchanged 3.9.4 because of the extra initial renewal and passes against the candidate. The unchanged corpus policy passes with 61 to 62 replay fixtures and a verified immutable-base worker fail/candidate pass. All stable guards are retained. Public-boundary, whitespace and diff secret/private-data review pass.

    Local source containers are removed and their exact prefix is absent. Two initial source-harness faults are retained as environment failures: the first container UID could not traverse the read-only dependency mount, and the first temporary PHP bridge omitted stdin forwarding. Corrected unprivileged UID/stdin controls passed without changing SDK assertions or guards. These are not product failures or installed evidence.

    Next action / owner: sdk-php owns normal exact-head #335 checks/review, maintenance integration, actual-branch qualification and cleanup. Published-package/Server acceptance remains open. No tag, package version, installed tuple, workflow dispatch or publication has been selected. #327's positive optimization still requires a published Server with the accepted #1070 clock; the old-Server renewal fallback remains covered at source level.

  5. rmcdaniel commented on Nov 6, 2026

    @rmcdaniel
    MemberAuthor

    Correction to the scope of the observed green PR335 Sonar result after inspecting the scanner and provider metadata:

    • Final PR scanner log requests PR335 into 3.9.x, computes Git merge base 11085f0c624674a4b70d3a6fb3e83eaff3869fcb, records revision b389182db2b1878a78306a9935fb88a8fe7da6c5, and receives a PASSED response.
    • The provider's current api/project_pull_requests/list metadata for key335 binds the same commit and target 3.9.x, but its analysis base is main, not 3.9.x. The component metadata reports the matching 2026-11-06T20:39:40+0000 analysis. PR336 separately shows both base and target as main.
    • Therefore the green check/zero findings are preserved as an observed provider result with a main analysis baseline. I withdraw the earlier interpretation that it independently qualifies the maintenance change against 3.9.x. Matching the head/merge tree proves source identity, not the missing analysis-baseline authority.

    Actual merge 9f1487e9 still has exact reviewed tree 7438c819 and the expected ordered parents. Ordinary maintenance CI and Public Boundary passed; merged non-main Sonar remains NONPASS/403. The official Free-plan PR/main restriction and this actual metadata agree that maintenance Sonar acceptance is still missing.

    I immediately returned this precise base/target distinction to General, who owns the account/configuration disposition. No branch/project/provider reroute, main-label/account/plan/credential change, private-project change, fee, retry, release or gate waiver is selected. #326/#327 remain open through real qualification and published delivery.

  6. rmcdaniel commented on Nov 6, 2026

    @rmcdaniel
    MemberAuthor

    The release handoff is now concrete: draft #337 proposes SDK 3.9.5 at ba94a9c, based on accepted maintenance source 9f1487e9. It changes only the changelog, Composer product-train and quickstart release identities, following the existing #289 process. No production, dependency, support or protocol changes are added. Main already has the equivalent helper/grant code through #333/#334.

    General owns the account/configuration and actual release disposition. The #335 maintenance Sonar NONPASS/403 and PR provider base=main/target=3.9.x distinction remain release-readiness gaps; normal source CI does not clear them. This draft is not a tag or publication.

    Next action: finish normal candidate checks, obtain the applicable release disposition through the existing safeguards, then publish the exact immutable patch through the normal process. sdk-php owns verifying the GitHub release and exact Packagist source/dist/package files, linking the release here and closing the delivered issue. Published SDK is still 3.9.4, so this issue remains open.

  7. rmcdaniel commented on Nov 7, 2026

    @rmcdaniel
    MemberAuthor

    The operator-selected maintenance release order is moving: #339 is normally merged as063e5af6 and release #337 is refreshed to exactdbeae40, with all runtime/support/dependency bytes equal corrected maintenance source. Fresh normal source checks are running; maintenance Sonar is explicitly not required under the operator decision, with its old wrong-base/403 evidence retained and nonblocking #342 filed. General owns actual3.9.5 execution; sdk-php owns exact registry/package/install verification. Supported producer clock/fallback evidence remains separately attributed; no published Server optimization or speedup is inferred from the source release.

  8. rmcdaniel commented on Nov 7, 2026

    @rmcdaniel
    MemberAuthor

    SDK3.9.5 is now published and normal Packagist installation is verified: exact installed evidence. Registry/lock/dist and all205 production files bind57562b53; the actual installed worker advertises3.9.5. The fresh-grant fixture has zero initial renewal, and the maintained fresh/stale/short/missing-clock controls pass among18 tests/87 assertions. Containers are removed. These are installed-SDK fixture results, not actual published-Server qualification or a speedup claim. The source/package delivery step is complete. Remaining owner/action: General reconciles the exact supported published Server containing accepted#1070 and actual older-Server fallback under the earlier acceptance boundary, with Server retaining producer ownership; sdk-php owns the resulting consumer verification. This issue stays open for that explicit remaining producer/tuple proof rather than silently replacing it with mocks.

  9. rmcdaniel commented on Nov 7, 2026

    @rmcdaniel
    MemberAuthor

    I checked whether already-published PHP Server2.11.8 could supply the positive clock producer without waiting for the native #1070 publication. It does not provide that field on the inspected ordinary workflow-poll source path: WorkerController::pollWorkflowTasks builds only task/poll_status, and WorkerProtocol::json adds protocol_version/server_capabilities, without server_time. The exact published commit isfa3fc2f2f6b8935ea4a6eea47f6a9221e2adafff (treec25fe048c0387519a44dcc1efac898f2aa062fa3; initial commit/tree label corrected); controller blobdf7bfff68cd98c52e7a1c60bc71174c399ba0127 and protocol blob1c40943935662888340149b1d7432aa5a0da169f bind this static read. The poller, poll-request store and response-progress middleware likewise contain no server_time field. This is a source audit, not a real PHP Server run.

    Therefore I have not relabeled that old producer as positive clock qualification. It remains a possible older-producer fallback subject only if the runtime owner selects its actual published tuple. Latest native release is still3.0.5; accepted #1070 clock source866a4067 was merged later and has no returned published-artifact handoff here.

    The published SDK3.9.5 manifest retains major Server release baselines2→2.5.0 and3→3.0.0; the legacy manual smoke workflow's exact flat2.5.0 check is narrower than those declarations and was not dispatched. General/Server own the exact supported published clock-producing artifact and bounded real-runtime selection, then sdk-php owns the installed consumer proof. The actual3.9.5 package/fixture verification is complete, but it does not fill this remaining producer gap.

  10. rmcdaniel commented on Nov 7, 2026

    @rmcdaniel
    MemberAuthor

    General settles the acceptance boundary under the later explicit operator release order337/6699384833: close this SDK issue as the delivered conditional SDK implementation in3.9.5, after the actual public release and normal-registry installed-package proof337/6699764564. This supersedes my older checkpoint's placement of the cross-repository runtime gate on this SDK issue; it does not claim that gate passed.

    The real installed package has the fresh-clock optimization, conservative poll-transit accounting, unchanged completion fencing and stale/short/malformed/missing-clock fallback. Both official installed-worker fixtures and18 focused tests/87 assertions passed with the parent/helper sources bound to the installed package. No reported benchmark or simulated producer is presented as a real published Server.

    The remaining product/runtime qualification is explicitly retained in reopened Server#1053, the existing owning producer issue, with General responsible for the publication/tuple decision and Server for the finite Source assessment. It requires a supported published Native Server containing accepted PR1070 and an actual older-Server fallback scenario using released SDK3.9.5. The full cross-product outcome remains unfinished there; native3.0.5 and PHP2.11.8 are not positive clock producers. No unsupported tuple, PHP Server release, performance acceptance or new build/runtime operation is selected by this disposition.

    SDK owner may now close#327 with the3.9.5 release link and this exact scope, as requested by the operator. General continues to own and drive the remaining producer outcome.

  11. rmcdaniel commented on Nov 7, 2026

    @rmcdaniel
    MemberAuthor

    Delivered as the conditional SDK implementation in 3.9.5, exact57562b53/tree6a58907, under the explicit scope disposition. Normal-registry installed proof binds source/dist/installed identities, all205 production files and both release manifests, actual3.9.5 worker identity, both official-worker fixtures and18 tests/87 assertions. Fresh grants avoid the initial renewal; original poll transit, completion fences and stale/short/malformed/missing-clock fallback are preserved. Equivalent main source is already accepted through #334. The remaining real supported published Native clock-producer and older-Server qualification is explicitly unfinished in reopened Server#1053, owned by General/Server with sdk-php consumer follow-through. This closure does not call that runtime gate or reported benchmark PASS, select a Server release/tuple, or represent fixture transports as a real published Server. Both bounded verification containers are removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:sdk-phpWork done by the sdk-php repo agentkind:featureA public product capability or experience is requestedpriority:P0Immediate release or data-integrity riskrepo:sdk-phpOwned by the PHP SDK repository

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions