Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/native-image-parity.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
run: |
set -euo pipefail
bash scripts/ci/pull-images.sh \
"$(jq -r '.server.images["linux/amd64"] | sub("^durableworkflow/server@"; "ghcr.io/durable-workflow/server@")' tests/Fixtures/ServerParityBaselines/php-2.11.7.json)" \
"$(jq -r '.server.images["linux/amd64"] | sub("^durableworkflow/server@"; "ghcr.io/durable-workflow/server@")' tests/Fixtures/ServerParityBaselines/php-2.11.8.json)" \
composer:2.10.3@sha256:da624c2c705d54e2f5a9bb77a5dc6c7d091711433ebefd7035672e84c2010fee

- name: Set up Docker Buildx
Expand Down
58 changes: 29 additions & 29 deletions .github/workflows/server-parity.yml

Large diffs are not rendered by default.

54 changes: 54 additions & 0 deletions docs/rust-server-port-checkpoints/2026-11-01-reference-2118.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# November 1, the parity reference moves to published 2.11.8

- **Before.** The Shared Server fixtures compared native and embedded
against published Server 2.11.7 (Core 2.15.7).
- **Reference.** Published Server 2.11.8, Core 2.15.12, as General
published it (Server #789 receipt 6558026433, #765 receipt 6558025643):
- source `fa3fc2f2f6b8935ea4a6eea47f6a9221e2adafff`;
- index `sha256:3848d5d5cd1b23bd2881e3fdecea64cc44b72e6fb812e577c2db6d0872d065dd`;
- linux/amd64 `sha256:980fe0a052089ced34ec3fcd2be2ea4e77436861d8fb537fb6654cb652a53e07`;
- linux/arm64 `sha256:cea163f8fcedf3a17302acf0141b073a85fbdb395ebbcd59e5af6f5fa9ad75e1`.

`tests/Fixtures/ServerParityBaselines/php-2.11.8.json` records it. The
SDK, CLI, Waterline and sample-app pins are unchanged. The takeover
cohorts keep their own source images.
- **What changed on 2.11.8, and the answers recorded.**
- **Post-commit payload deletion counts** (#911). Every retention pass
and namespace delete answer carries `external_payload_cleanup`, and the
payload cleanup status and pass carry `retention_cleanup`
(`pending`, `failed`, `blocked`). Native added them only while a
deletion was pending (#914); it now always does. `namespace-delete`,
`payload-cleanup`, `retention-external-payloads` and
`namespace-delete-payloads` pin the counts.
- **A child resolved under a leased parent task** (workflow#962, Core
2.15.10). `child-open-parent-task` gains a leased case: the parent's
resume task is claimed, the child completes, and the lease completes
with a timer and without the result. 2.11.8 then offers one follow-up
task that replays the child's result, as native does. Native left that
moment's run summary unprojected (a resolved child beside the timer);
it now describes it as 2.11.8 does: `timer_scheduled`, waiting for the
timer, no repair attention.
- **History import** (Core 2.15.8 to 2.15.12). The `workflow-import`
probe's run id conflict bundle renamed only `workflow.run_id`; 2.11.8
checks each command's and signal's `resolved_run_id` first, so the
probe now renames every `*run_id` naming the original run, and the case
still answers `target.run_id_conflict`. The cross-namespace attach now
answers `target.instance_namespace_conflict` before writing, and a run
imported into another namespace answers as a conflict; native matches
both, and the fixture's published-defect allowances for 2.11.1 to
2.11.7 are removed.
- **Embedded probes.** Core 2.15.12 resolves a run through its
instance's namespace and reserves a caller-named instance without
model events, so the probes' `creating` hooks left that instance's
namespace empty and every embedded recording failed to find its run.
The embedded probes now configure `workflows.v2.namespace`. A host
application without the hooks is unaffected.
- **Local checks before the pull request** (SQLite, this host):
- the four cleanup fixtures, `child-open-parent-task` and
`workflow-import` recorded against a fresh 2.11.8 harness pass, and
native built from this branch passes `child-open-parent-task` and
`workflow-import`;
- without the namespace setting every embedded recording fails on 2.11.8
and passes on 2.11.7; with it, embedded recordings pass on 2.11.8;
- the native SQLite execution suite and the node unit suite (see the pull
request for counts and the full corpus run).
78 changes: 51 additions & 27 deletions docs/rust-server-port.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,11 @@ Updated October 27, 2026.
- The parity reference moves to published 2.11.7 (#863), with native's
v1 projection history bytes in Core's canonical unit (#770)
([checkpoint](rust-server-port-checkpoints/2026-10-29-reference-2117.md)).
- The parity reference moves to published 2.11.8 (Core 2.15.12): native
always reports post-commit payload deletion counts, and a child's
resolution under a leased parent task now gives PHP a follow-up task
(workflow#962)
([checkpoint](rust-server-port-checkpoints/2026-11-01-reference-2118.md)).
- [#814](https://github.com/durable-workflow/server/pull/814): the shared
fixtures accept published PHP's typed `429 long_poll_capacity_exhausted`
poll refusal, and retry a locked Composer install after a transient
Expand Down Expand Up @@ -468,28 +473,45 @@ contract fixture before its correction.
and a signal against published 2.11.7
([checkpoint](rust-server-port-checkpoints/2026-11-01-key-order.md)).
- **Child completion while the parent has an open workflow task**
(November 1; ready task closed, leased task decided;
[checkpoint](rust-server-port-checkpoints/2026-11-01-child-open-parent-task.md)).
Published PHP 2.11.7 records the child resolution and, while the parent
has a ready or leased workflow task, creates none and leaves that task's
payload as it is (`dispatchParentResumeTasks`).
- **Ready task.** Native now does the same: the ready task replays the
parent with the resolution in its history. An update task does not
replay the workflow, so beside one the resolution still gets its own
resume task. The `child-open-parent-task` fixture records it against
published 2.11.7.
- **Leased task.** On 2.11.7 a lease whose worker read history before the
resolution completes without it, and the parent is left with no task;
the repair pass does not recover it
([workflow#962](https://github.com/durable-workflow/workflow/issues/962)).
Core 2.15.10, not yet in a published Server release, creates one child
resume task when such a lease completes. Native gives the resolution
its own resume task at once, as activity and timer resolutions do on
both servers. Compare again when the parity reference moves to a Server
release with Core 2.15.10.
(November 1, closed;
[checkpoint](rust-server-port-checkpoints/2026-11-01-child-open-parent-task.md),
[reference move](rust-server-port-checkpoints/2026-11-01-reference-2118.md)).
Published PHP records the child resolution and, while the parent has a
ready or leased workflow task, creates none and leaves that task's payload
as it is (`dispatchParentResumeTasks`).
- **Ready task.** Native does the same: the ready task replays the parent
with the resolution in its history. An update task does not replay the
workflow, so beside one the resolution still gets its own resume task.
- **Leased task.** A lease whose worker read history before the
resolution completes without it. Published 2.11.7 then left the parent
with no task ([workflow#962](https://github.com/durable-workflow/workflow/issues/962));
2.11.8 (Core 2.15.10 and later) gives it one follow-up task that
replays the child's result. Native gives the resolution its own resume
task at once, as activity and timer resolutions do, so after the lease
both servers offer exactly one task that replays the result. While the
lease's new timer is open beside that task, native now describes the
run as 2.11.8 does (`timer_scheduled`, waiting for the timer) instead of
leaving it unprojected. The `child-open-parent-task` fixture records
both cases against published 2.11.8.
- A signal creates no resume while the parent waits only on a child, as
on PHP: `PendingMessageTask::signalCanAdvanceRun` advances only signal
and condition waits.
- **History import fences of Core 2.15.8 to 2.15.12** (November 1,
narrowed with the reference move to 2.11.8;
[checkpoint](rust-server-port-checkpoints/2026-11-01-reference-2118.md)).
Published 2.11.8 checks a bundle's internal references before its target:
each command's and signal's `resolved_run_id` must name the imported run,
messages must bind to the bundle's commands, and lineage must stay within
verified endpoints (`EmbeddedV2ImportReferences`,
`EmbeddedV2ImportMessageReferences`, `EmbeddedV2ImportLineage`). It also
refuses an instance of another namespace before writing
(`target.instance_namespace_conflict`), and a run already imported into
another namespace as a run id conflict. Native now gives both target
answers as 2.11.8 does, and the `workflow-import` fixture records them;
the published-defect allowances for 2.11.1 to 2.11.7 are gone. Native's
own reference fence (#765) runs after the run id check, and its rules and
rule names have not yet been compared with Core's for bundles whose
references are inconsistent; no fixture case sends such a bundle.
- **Update delivery into a ready task** (November 1, decided: native
keeps its behavior). Published PHP merges an accepted update into the
run's ready workflow task (`ExternalWorkflowUpdateAdmission::wakeWorkflowTask`).
Expand Down Expand Up @@ -544,14 +566,16 @@ contract fixture before its correction.
the prune records the deletion in `dw_external_payload_deletions` (schema
version 8), so a prune that fails later keeps the run, its registrations
and every object. A provider failure after the commit leaves the deletion
pending, retried by the next pass. While one is pending the retention pass
reports `external_payload_cleanup`, the payload cleanup routes report
`retention_cleanup` (the pass answering 207), and namespace delete reports
`external_payload_cleanup` with 207; 2.11.8 adds those counts to clean
answers too, which native will do once CI's parity reference moves from
2.11.7, whose clean answers lack them. An online-backup hold still fails the prune before it commits. The
`retention-external-payloads` fixture records it against published 2.11.7
and passes unchanged against 2.11.8.
pending, retried by the next pass. As on 2.11.8, every answer carries the
counts (`pending`, `failed`, `blocked`): the retention pass and namespace
delete as `external_payload_cleanup`, the payload cleanup routes as
`retention_cleanup`. Namespace delete answers 207 while a deletion is
pending, and the cleanup pass also when it blocked a reference. An
online-backup hold still fails the prune before it commits. The
`retention-external-payloads`, `namespace-delete-payloads`,
`namespace-delete` and `payload-cleanup` fixtures pin the counts against
published 2.11.8
([checkpoint](rust-server-port-checkpoints/2026-11-01-reference-2118.md)).
A run holding a reference native cannot settle (PHP's stored format from a
workflow import, an envelope, or one on an inbound or durable stream) is
still skipped with `external_payload_cleanup_unavailable`. Since November 1,
Expand Down
10 changes: 10 additions & 0 deletions rust/src/runtime/child_summary.rs
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,16 @@ where
{
return Ok(true);
}
// A resolved child beside an open timer, while a workflow task that
// will apply it carries the run (a lease that completed without the
// child's result, workflow#962): published 2.11.8 describes the
// timer and reports no problem (`child-open-parent-task`, leased).
if summary.wait.as_ref().is_some_and(|wait| wait.kind == "timer")
&& summary.task_type.as_deref() == Some("workflow")
&& rows.iter().all(|row| DB::string(row, "status").is_ok_and(|status| status != "open"))
{
return Ok(true);
}
if summary
.wait
.as_ref()
Expand Down
1 change: 1 addition & 0 deletions rust/src/runtime/execution.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1800,6 +1800,7 @@ impl Runtime {
delegate!(show_service_call(names: [&str; 3], id: &str) -> Result<Value>);
delegate!(cancel_service_call(names: [&str; 3], id: &str, reason: Option<String>) -> Result<(StatusCode, Value)>);
delegate!(import_existing_run(run_id: &str) -> Result<Option<super::workflow_import::ExistingRun>>);
delegate!(import_instance_namespace(instance_id: &str) -> Result<Option<String>>);
delegate!(import_write(plan: &super::workflow_import::Plan) -> std::result::Result<super::workflow_import::Rows, String>);
delegate!(waterline_existing(workflow_id: &str, run_id: &str) -> Result<super::waterline_import::Existing>);
delegate!(projected_current_run(workflow_id: &str) -> Result<Option<String>>);
Expand Down
16 changes: 8 additions & 8 deletions rust/src/runtime/namespace_cleanup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,9 @@
//! `RuntimeExternalPayloadRegistry::deleteForNamespace` release them
//! (`delete_namespace_payloads`). Their provider deletions are recorded in the
//! delete's transaction and run only after it commits (`payload_deletions`),
//! so a failed delete keeps every object. While a deletion is still pending
//! the answer carries `external_payload_cleanup` and is 207.
//! so a failed delete keeps every object. The answer always carries their
//! `external_payload_cleanup` counts, as 2.11.8's does, and is 207 while a
//! deletion is still pending.
//! A namespace holding a reference native cannot settle (PHP's stored
//! format, an envelope, or one on a stream) refuses before changing anything,
//! with PHP's answer for unavailable external payload storage.
Expand Down Expand Up @@ -278,13 +279,12 @@ store_methods! {
Self::query("DELETE FROM workflow_namespaces WHERE name=$1").bind(&name).execute(&mut *tx).await?;
tx.commit().await?;
// The delete has committed; only now may provider objects go. A
// deletion left pending answers 207 with its counts.
// deletion left pending answers 207.
self.settle_payload_deletions(&staged).await;
let mut body = json!({"name": name, "status": "deleted", "deleted": deleted});
if let Some(cleanup) = self.reported_payload_deletions(&name).await? {
body["external_payload_cleanup"] = cleanup;
}
Ok(body)
Ok(json!({
"name": name, "status": "deleted", "deleted": deleted,
"external_payload_cleanup": self.payload_deletion_status(&name).await?,
}))
}

/// The namespace's external payloads, inside its delete's transaction:
Expand Down
4 changes: 1 addition & 3 deletions rust/src/runtime/operator_metrics.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1256,9 +1256,7 @@ where
let failed = report["failed"].as_i64().is_some_and(|failed| failed > 0);
let mut fields = policy.fields(self.namespace.as_ref());
fields.extend(report);
if let Some(cleanup) = self.reported_payload_deletions(self.namespace.as_ref()).await? {
fields.insert("external_payload_cleanup".into(), cleanup);
}
fields.insert("external_payload_cleanup".into(), self.payload_deletion_status(self.namespace.as_ref()).await?);
Ok((Value::Object(fields), failed))
}

Expand Down
19 changes: 8 additions & 11 deletions rust/src/runtime/payload_cleanup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,11 +60,11 @@ struct Report {
store_methods! {
/// `externalPayloadCleanupStatus`: the namespace's cleanup metrics.
pub(crate) async fn payload_cleanup_status(&self, config: &super::operator_metrics::MetricsConfig) -> Result<Value> {
let mut status = json!({"namespace": self.namespace.as_ref(), "cleanup": self.external_payload_cleanup_metrics(config).await?});
if let Some(cleanup) = self.reported_payload_deletions(self.namespace.as_ref()).await? {
status["retention_cleanup"] = cleanup;
}
Ok(status)
Ok(json!({
"namespace": self.namespace.as_ref(),
"cleanup": self.external_payload_cleanup_metrics(config).await?,
"retention_cleanup": self.payload_deletion_status(self.namespace.as_ref()).await?,
}))
}

/// The pass's report; the caller answers 207 when it blocked anything or
Expand Down Expand Up @@ -113,18 +113,15 @@ store_methods! {
.bind(DB::bind_time(cutoff))
.fetch_one(&self.pool)
.await?;
let mut body = json!({
Ok(json!({
"processed": report.processed, "deleted_references": report.deleted_references,
"deleted_backing_objects": report.deleted_backing_objects,
"shared_objects_preserved": report.shared_objects_preserved, "skipped": report.skipped,
"blocked": report.blocked, "storage_driver_failures": report.storage_driver_failures,
"backlog": backlog, "scan_limit": limit,
"scan_pressure": backlog > 0 && candidates.len() as i64 >= limit,
});
if let Some(cleanup) = self.reported_payload_deletions(self.namespace.as_ref()).await? {
body["retention_cleanup"] = cleanup;
}
Ok(body)
"retention_cleanup": self.payload_deletion_status(self.namespace.as_ref()).await?,
}))
}

/// `RuntimePayloadCompletionUploads::cleanup`: an expired budget whose
Expand Down
8 changes: 0 additions & 8 deletions rust/src/runtime/payload_deletions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -221,12 +221,4 @@ store_methods! {
Ok(json!({"pending": pending, "failed": failed, "blocked": blocked}))
}

/// The status where a response reports it: 2.11.8 always adds the counts,
/// but CI's parity reference is still 2.11.7, whose clean answers have no
/// such member. Until the reference moves to 2.11.8 native adds them only
/// when a deletion is pending; a clean answer stays 2.11.7's.
pub(super) async fn reported_payload_deletions(&self, namespace: &str) -> Result<Option<Value>> {
let status = self.payload_deletion_status(namespace).await?;
Ok((status["pending"].as_i64().unwrap_or(0) > 0).then_some(status))
}
}
21 changes: 21 additions & 0 deletions rust/src/runtime/workflow_import.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1054,6 +1054,15 @@ impl Runtime {
return Ok(answer("rejected", Some(prepared.eligibility.with_error("target.run_id_conflict",
"A workflow run with this run_id already exists and was not imported from the same bundle.")), &empty, None));
}
// Core 2.15.12 (Server 2.11.8): an instance of another namespace is
// refused before a dry run or a write, not left to the write's failure.
let instance_id = prepared.summary.instance_id.clone().unwrap_or_default();
if let Some(namespace) = self.import_instance_namespace(&instance_id).await?
&& namespace != prepared.summary.namespace
{
return Ok(answer("rejected", Some(prepared.eligibility.with_error("target.instance_namespace_conflict",
"A workflow instance with this instance_id already exists outside the target namespace.")), &empty, None));
}
let root = Php::Array(prepared.bundle.clone());
// Native's reference fence (Server #765), after PHP's eligibility and
// run checks so their answers stay PHP's, and before a dry run or a
Expand Down Expand Up @@ -1133,6 +1142,18 @@ store_methods! {
}))
}

/// The namespace of an existing workflow instance, if there is one.
pub(crate) async fn import_instance_namespace(&self, instance_id: &str) -> Result<Option<String>> {
let Some(row) = Self::query("SELECT namespace FROM workflow_instances WHERE id=$1")
.bind(instance_id)
.fetch_optional(&self.pool)
.await?
else {
return Ok(None);
};
Ok(Some(DB::optional_string(&row, "namespace")?.unwrap_or_default()))
}

pub(crate) fn namespace_name(&self) -> String {
self.namespace.as_ref().to_owned()
}
Expand Down
4 changes: 3 additions & 1 deletion rust/tests/execution/external_payloads/cleanup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,9 @@ async fn status(router: &Router) -> Value {
fn report(values: [i64; 7], backlog: i64, limit: i64, pressure: bool) -> Value {
json!({"processed": values[0], "deleted_references": values[1], "deleted_backing_objects": values[2],
"shared_objects_preserved": values[3], "skipped": values[4], "blocked": values[5],
"storage_driver_failures": values[6], "backlog": backlog, "scan_limit": limit, "scan_pressure": pressure})
"storage_driver_failures": values[6], "backlog": backlog, "scan_limit": limit, "scan_pressure": pressure,
// 2.11.8 always reports committed retention deletions still pending.
"retention_cleanup": {"pending": 0, "failed": 0, "blocked": 0}})
}

#[tokio::test]
Expand Down
Loading
Loading