Shared Server fixtures: fetch the verified parity CLI once instead of in every corpus job #1103

Activity

1. rmcdaniel

   added

   agent:server

   on Nov 8, 2026
2. rmcdaniel commented on Nov 8, 2026

   

   rmcdaniel

   on Nov 8, 2026

   Last edited by rmcdaniel

   MemberAuthor

   More actions

   Source proposal for #1103 (not pushed; local commit only)

   Identity.

   • Local branch ci/parity-cli-once: commit f997b91c41683478e8d1b5d702fa5e37d03ae601, tree 3a0e883419f2496e0817cd43dac26c385c1a1468.
   • Sole parent: M 2a15179f29801eec9f2a2d4e6a9c5f93233f104e, current main.
   • One file: .github/workflows/server-parity.yml, +38/−3. Delta 6,158 B / 0074e69ec4b49d1132b6b770c79ca51277fa2675475e07c13d52cb063523190c.

   Architecture choice.

   • published-sdk-adapters is the shared-input job. mysql-execution, postgres-storage and execution-and-codec all need it and require result == 'success', so the CLI is always present when any corpus part runs.
   • The CLI is not added to that job's existing published-sdk-patch-adapters artifact. That directory carries an exhaustive SHA256SUMS that consumers check as adapter-integrity.txt (published-sdk-patch-adapters.sh:29–30, published-sdk-patch-profile.sh:12). The CLI would either change that adapter evidence or sit there unlisted.
   • Instead, the same job uploads a small dedicated artifact, parity-cli-2.2.0.

   Change.

   1. Producer (published-sdk-adapters, right after the image pulls):

      • one curl --fail --location --retry 5 --retry-all-errors --retry-delay 5 --retry-max-time 180 of the same URL;
      • sha256sum -c against visibility.cli.phar_sha256 in the committed visibility-current-runs.json;
      • upload parity-cli-2.2.0 with retention-days: 1 and if-no-files-found: error.

   2. Each corpus job:

      • Fetch this run's verified parity CLI (download-artifact@3e5f45b2…, digest-mismatch: error), placed beside the existing Fetch this run's … adapter downloads;
      • the step that used to curl now runs cp parity-cli/dw.phar parity-consumers/dw.phar, then the unchanged sha256sum -c > parity-evidence/cli-artifact.txt line.

      DW_PARITY_CLI_PHAR and everything after it are byte-identical.

   What is preserved.

   • The exact PHAR identity, verified twice: by the producer, and by each consumer into its evidence file.
   • No new trust root, and the curl/TLS defaults are unchanged. The digest is committed in the repository, which is Confine the remaining CI, conformance and perf downloads to checked CA stores (GHSA-6q48-6c7w-9gw2) #677's option (b).
   • Every corpus job, part and fixture, and every required check.
   • External requests drop from up to 7 per run to 1, and that one has a 180 s retry budget. Correction (09:56Z): one external download invocation per run (up to six HTTP attempts within a 180 s retry budget) replaces the separate per-job invocations. This is not one HTTP request, and no wall-time improvement is claimed.

   Static checks (nothing executed).

   • The YAML parses.
   • ServerParityPartsTest: the new consumer step has no part condition and is named Fetch …, so it matches EVERY_PART, like the existing adapter fetches.
   • ServerParityAdapterCacheTest: still 5 -adapters.sh build steps, with restore before them and save after.
   • check-docker-workflow-isolation.py: the new commands take only literal arguments.
   • native-image-corpus.sh (a different workflow, already one fetch per run) is untouched.

   Integration without touching RUNNER_PATHS or the product bindings

   • Not through the runner branch. server-parity.yml is not in RUNNER_PATHS, so putting it on feat/server-native32-qualification-runner would fail qualification_runner_frozen_source. The runner stays at E4 6754daa8 and Qualify the runner against the published Native 3.3.0 index #1101 stays open and unchanged.

   • Its own PR to main, branch ci/parity-cli-once from M, with its own source train and merge group. That merge group is the first real test of the single fetch.

   • After it merges, main becomes M + fix. E4 keeps its sole parent M:

     • compare M...E4 is still ahead 1, subject file only;
     • a later fresh E4 entry produces merge [main+fix, E4], so E4 becomes an ancestor of main and qualification_runner_not_on_main passes.

     No rebase of E4 and no change to the subject are needed.

   • Product bindings unchanged. The 3.3.0 tag and image index D stay bound to M (release.source_commit = M). Main moving past M is already allowed by the E4 carry.

   • Order: CI-fix PR → its queue → main custody → then a fresh E4 entry, if selected → E4 custody → Reduce Native poll and completion database work #1099. Each step needs its own selection.

   Full delta (6,158 B / 0074e69e)

   diff --git a/.github/workflows/server-parity.yml b/.github/workflows/server-parity.yml
   index 64c530f9..fe30e344 100644
   --- a/.github/workflows/server-parity.yml
   +++ b/.github/workflows/server-parity.yml
   @@ -272,6 +272,23 @@ jobs:
                mirror.gcr.io/library/rust@sha256:300ec56abce8cc9448ddea2172747d048ed902a3090e6b57babb2bf19f754081 \
                mirror.gcr.io/library/python@sha256:bf44cdfcb76cd3b41e879bc058fc37ec5872002ccfde7fcb765e218cde0cd79c \
                "$(baseline php-2.11.8)"
   +      # Every corpus job replays CLI 2.2.0. Fetch it once for the whole run,
   +      # bound to its committed digest, instead of once per corpus job; each
   +      # consumer verifies the same digest again before use.
   +      - name: Fetch the parity CLI once, bound to its committed digest
   +        run: |
   +          set -euo pipefail
   +          mkdir -p parity-cli
   +          curl --fail --location --retry 5 --retry-all-errors --retry-delay 5 --retry-max-time 180 \
   +            --output parity-cli/dw.phar https://github.com/durable-workflow/cli/releases/download/2.2.0/dw.phar
   +          printf '%s  %s\n' "$(jq -r '.visibility.cli.phar_sha256' tests/Fixtures/ServerParity/visibility-current-runs.json)" parity-cli/dw.phar | sha256sum -c
   +      - name: Share the verified parity CLI with the corpus jobs
   +        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
   +        with:
   +          name: parity-cli-2.2.0
   +          path: parity-cli
   +          retention-days: 1
   +          if-no-files-found: error
          # The adapters' pinned, --locked dependencies build the same way every
          # run. Restore their public Cargo registry data and build tree, keyed
          # by the toolchain image, the runner and every tracked adapter input.
   @@ -582,6 +599,12 @@ jobs:
              name: corrected-marker-adapters
              path: corrected-marker-adapters
              digest-mismatch: error
   +      - name: Fetch this run's verified parity CLI
   +        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
   +        with:
   +          name: parity-cli-2.2.0
   +          path: parity-cli
   +          digest-mismatch: error
          - name: Create TLS fixture certificates
            run: |
              set -euo pipefail
   @@ -826,7 +849,7 @@ jobs:
              image=$(jq -r '.server.images["linux/amd64"] | sub("^durableworkflow/server@"; "ghcr.io/durable-workflow/server@")' tests/Fixtures/ServerParityBaselines/php-2.11.8.json)
              export DW_PARITY_RUNNER_REVISION=$(git rev-parse HEAD)
              mkdir -p parity-consumers
   -          curl --fail --location --retry 3 --output parity-consumers/dw.phar https://github.com/durable-workflow/cli/releases/download/2.2.0/dw.phar
   +          cp parity-cli/dw.phar parity-consumers/dw.phar
              printf '%s  %s\n' "$(jq -r '.visibility.cli.phar_sha256' tests/Fixtures/ServerParity/visibility-current-runs.json)" parity-consumers/dw.phar | sha256sum -c > parity-evidence/cli-artifact.txt
              export DW_PARITY_CLI_PHAR=/source/parity-consumers/dw.phar
              docker run --rm --user=1000:1000 --entrypoint node --network "$RESOURCE_SCOPE" \
   @@ -1360,6 +1383,12 @@ jobs:
              name: corrected-marker-adapters
              path: corrected-marker-adapters
              digest-mismatch: error
   +      - name: Fetch this run's verified parity CLI
   +        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
   +        with:
   +          name: parity-cli-2.2.0
   +          path: parity-cli
   +          digest-mismatch: error
          # Exact dependency keys include the actual compiler pin, Cargo configuration
          # and vendored crates. Server outputs are discarded before rebuilding.
          - name: Restore native Cargo dependencies from trusted main
   @@ -1582,7 +1611,7 @@ jobs:
              image=$(jq -r '.server.images["linux/amd64"] | sub("^durableworkflow/server@"; "ghcr.io/durable-workflow/server@")' tests/Fixtures/ServerParityBaselines/php-2.11.8.json)
              export DW_PARITY_RUNNER_REVISION=$(git rev-parse HEAD)
              mkdir -p parity-consumers
   -          curl --fail --location --retry 3 --output parity-consumers/dw.phar https://github.com/durable-workflow/cli/releases/download/2.2.0/dw.phar
   +          cp parity-cli/dw.phar parity-consumers/dw.phar
              printf '%s  %s\n' "$(jq -r '.visibility.cli.phar_sha256' tests/Fixtures/ServerParity/visibility-current-runs.json)" parity-consumers/dw.phar | sha256sum -c > parity-evidence/cli-artifact.txt
              export DW_PARITY_CLI_PHAR=/source/parity-consumers/dw.phar
              docker run --rm --user=1000:1000 --entrypoint node --network "$RESOURCE_SCOPE" \
   @@ -2144,6 +2173,12 @@ jobs:
              name: corrected-marker-adapters
              path: corrected-marker-adapters
              digest-mismatch: error
   +      - name: Fetch this run's verified parity CLI
   +        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
   +        with:
   +          name: parity-cli-2.2.0
   +          path: parity-cli
   +          digest-mismatch: error
          # Exact dependency keys include the actual compiler pin, Cargo configuration
          # and vendored crates. Server outputs are discarded before rebuilding.
          - name: Restore native Cargo dependencies from trusted main
   @@ -2410,7 +2445,7 @@ jobs:
              image=$(jq -r '.server.images["linux/amd64"] | sub("^durableworkflow/server@"; "ghcr.io/durable-workflow/server@")' tests/Fixtures/ServerParityBaselines/php-2.11.8.json)
              export DW_PARITY_RUNNER_REVISION=$(git rev-parse HEAD)
              mkdir -p parity-consumers
   -          curl --fail --location --retry 3 --output parity-consumers/dw.phar https://github.com/durable-workflow/cli/releases/download/2.2.0/dw.phar
   +          cp parity-cli/dw.phar parity-consumers/dw.phar
              printf '%s  %s\n' "$(jq -r '.visibility.cli.phar_sha256' tests/Fixtures/ServerParity/visibility-current-runs.json)" parity-consumers/dw.phar | sha256sum -c > parity-evidence/cli-artifact.txt
              export DW_PARITY_CLI_PHAR=/source/parity-consumers/dw.phar
              docker run --rm --user=1000:1000 --entrypoint node --network "$RESOURCE_SCOPE" \
3. rmcdaniel commented on Nov 8, 2026

   

   rmcdaniel

   on Nov 8, 2026

   MemberAuthor

   More actions

   SOURCE PASS. One draft PR / normal source train is selected for the CLI-download CI fix in #1103, owned by Server. Both E4 merge-queue NONPASS results remain attributed; no E4 re-entry is selected.

   Subject: f997b91c41683478e8d1b5d702fa5e37d03ae601, sole parent/current main M 2a15179f29801eec9f2a2d4e6a9c5f93233f104e, tree 3a0e883419f2496e0817cd43dac26c385c1a1468. One file, .github/workflows/server-parity.yml, +38/−3; binary delta 6,158 B / SHA-256 0074e69ec4b49d1132b6b770c79ca51277fa2675475e07c13d52cb063523190c.

   I read the complete proposal/diff, rebuilt the exact tree independently, and ran the relevant ServerParityPartsTest and ServerParityAdapterCacheTest: all 11 tests passed, no skip. The producer downloads once and checks the committed PHAR digest before uploading a dedicated run-scoped artifact. Every consumer downloads it with digest-mismatch set to error, copies it into the existing path, and retains the unchanged PHAR checksum check and evidence. Its required producer dependency, all parts/fixtures and all required checks remain present. No adapter inventory or product source changes.

   Use precise wording in the issue/PR: one external download invocation, with up to six HTTP attempts, replaces the separate per-job invocations. The retry budget is 180 seconds; do not describe that as exactly one HTTP request or a measured wall-time improvement. Preserve the producer and consumer log lines so the fresh train can establish actual request use and timing.

   Before push, re-read this raw decision; require exact head/tree/delta, clean worktree, actual main = base = merge-base = M, new remote branch absent, no duplicate PR and a clean secret scan. Push this head once with a must-be-absent branch lease and open an agent:server draft PR. Any identity drift refuses without a push. Let its one ordinary source train reach terminal; no cancellation, rerun or successor push is selected.

   Keep E4 6754daa8 and #1101 unchanged. This workflow file is outside RUNNER_PATHS, so it must integrate through its own PR and later normal queue decision. After that custody, E4 can merge without rebasing; its comparison against release M stays subject-only. Product 3.3.0 tag/image remain M/D. #1099 stays deferred.

   No ready/queue/merge, release, dispatch, local harness, daemon or image effect is selected here. The local stop otherwise remains, and the already-selected synthetic SQL cell's terminal result is a separate NONPASS to preserve and diagnose.
4. rmcdaniel

   added 2 commits that reference this issue on Nov 8, 2026

   Restrict the parity CLI fetch and its redirects to HTTPS

   ...

   2c22dc9

   Fall back to the release-asset API URL when the CLI download fails

   ...

   f2b0269
5. rmcdaniel

   closed this as completedin #1104on Nov 8, 2026