diff --git a/.github/openclaw/qualify-windows-arm64.ps1 b/.github/openclaw/qualify-windows-arm64.ps1 new file mode 100644 index 000000000000..aef074b11c02 --- /dev/null +++ b/.github/openclaw/qualify-windows-arm64.ps1 @@ -0,0 +1,40 @@ +param( + [Parameter(Mandatory)][string]$ArtifactDirectory, + [Parameter(Mandatory)][string]$OutputDirectory, + [Parameter(Mandatory)][string]$Source +) +$ErrorActionPreference = 'Stop' +if ([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture -ne 'Arm64') { + throw 'Qualification requires native Windows ARM64.' +} +$label = 'bun-webkit-windows-arm64' +$archive = Join-Path $ArtifactDirectory "$label.tar.gz" +$manifest = Get-Content -Raw (Join-Path $ArtifactDirectory "$label.manifest.json") | ConvertFrom-Json +$digest = (Get-FileHash -Algorithm SHA256 -LiteralPath $archive).Hash.ToLowerInvariant() +if ($manifest.source -cne $Source -or $manifest.lane.label -cne $label -or + $manifest.files."$label.tar.gz".sha256 -cne $digest -or + $manifest.files."$label.tar.gz".size -ne (Get-Item -LiteralPath $archive).Length) { + throw 'Windows ARM64 artifact provenance mismatch.' +} +New-Item -ItemType Directory -Path $OutputDirectory | Out-Null +& tar -xzf $archive -C $OutputDirectory +if ($LASTEXITCODE -ne 0) { throw 'Archive extraction failed.' } +$jsc = Join-Path $OutputDirectory 'bun-webkit/bin/jsc.exe' +$ffi = Join-Path $OutputDirectory 'bun-webkit/bin/testFFI.exe' +foreach ($exe in @($jsc, $ffi)) { + $bytes = [IO.File]::ReadAllBytes($exe) + $pe = [BitConverter]::ToInt32($bytes, 0x3c) + if ([BitConverter]::ToUInt16($bytes, $pe + 4) -ne 0xaa64) { throw "$exe is not an ARM64 PE." } +} +$probe = @' +function f(n) { let s = 0; for (let i = 0; i < n; ++i) s = (s + i * 7) % 1000003; return s; } +for (let i = 0; i < 1000; ++i) { if (f(100000) !== 545003) throw new Error("JIT result"); } +if (!(numberOfDFGCompiles(f) > 0 && numberOfDFGCompiles(f) < 1000000)) throw new Error("DFG unavailable"); +print("windows-arm64-jsc-ok"); +'@ +$result = & $jsc -e $probe +if ($LASTEXITCODE -ne 0 -or $result -cne 'windows-arm64-jsc-ok') { throw 'Native JSC startup/JIT probe failed.' } +& $ffi +if ($LASTEXITCODE -ne 0) { throw 'Native testFFI failed.' } +@{ passed = $true; source = $Source; lane = $label; sha256 = $digest; architecture = 'arm64'; startup = $true; dfg = $true; ffi = $true } | + ConvertTo-Json | Set-Content -Encoding utf8 (Join-Path $OutputDirectory 'gate.json') diff --git a/.github/openclaw/release-notes.md b/.github/openclaw/release-notes.md index 1fb8b75ce88c..6cafb51159de 100644 --- a/.github/openclaw/release-notes.md +++ b/.github/openclaw/release-notes.md @@ -1,9 +1,11 @@ -This release adds engine PR #8's repeated stack-coordinate cache to the qualified OpenClaw engine on upstream `5718a6ec579b98362ea7276a426deedcc6281ef5`. +This release adds the non-LTO Windows ARM64 artifact to the qualified OpenClaw engine on upstream `5718a6ec579b98362ea7276a426deedcc6281ef5`, with native Windows 11 ARM64 startup, DFG JIT and FFI gates bound to the archive and source. -- Cache requested provider-relative stack line and column conversions under the existing line-table lock, preserving document offsets, builtin overrides, and persistent bytecode format; thanks @steipete. Regressions cover repeated reads, independent providers, UTF-16 columns, and compressed line-table boundaries across interpreter/JIT tiers and cache modes. +- Build Windows ARM64 with the unchanged upstream cross-compilation recipe and require both JSC and testFFI to be native ARM64 PE files. The previously released engine changes remain included: +- Cache requested provider-relative stack line and column conversions under the existing line-table lock, preserving document offsets, builtin overrides, and persistent bytecode format; thanks @steipete. Regressions cover repeated reads, independent providers, UTF-16 columns, and compressed line-table boundaries across interpreter/JIT tiers and cache modes. + - Release free allocator memory from idle compiler threads and long Atomics waits; thanks @Jarred-Sumner. - Preserve Segmenter surrogate boundaries, opt-in Proxy global prototypes, module-loader AsyncLocalStorage context, and per-VM worker heap/stack budgets; thanks @steipete and @robobun. - Give require(esm) a separate live namespace with its own `__esModule` marker while preserving unmarked import namespaces; thanks @robobun. @@ -12,7 +14,7 @@ The previously released engine changes remain included: - Synchronize the accounting-related parallel-helper assertion and visited-memory counters. - Preserve syntax-selected stack positions for calls, constructors, property reads and async continuations, including live/captured stacks and cache replay; thanks @robobun for the upstream source-position work. -Qualification includes all nine artifact builds, Linux JSC module/promise/namespace/Segmenter and allocation-sampling regressions, the W113 paired Bun selection, a build of the exact prepared upstream-synced Bun runtime with its required namespace API adapter and candidate manifest with startup and unchanged upstream memory-release tests, and a separate build with feature adapters for the complete fork selection and patch regressions. Syntax-selected stack positions are checked across execution tiers, live/captured stacks, and cache replay. Other targets have build/provenance proof; native runtime qualification is Linux x64. +Qualification includes all ten artifact builds, Linux JSC module/promise/namespace/Segmenter and allocation-sampling regressions, the W113 paired Bun selection, a build of the exact prepared upstream-synced Bun runtime with its required namespace API adapter and candidate manifest with startup and unchanged upstream memory-release tests, and a separate build with feature adapters for the complete fork selection and patch regressions. Syntax-selected stack positions are checked across execution tiers, live/captured stacks, and cache replay. Other targets have build/provenance proof; native runtime qualification covers Linux x64 and Windows ARM64 JSC startup, DFG JIT and FFI probes. Bun consumer qualification is separate. The new engine requires Bun's updated mimalloc idle hook and matching embedding adapters. A manifest-only update of the original sync tree does not compile: its old namespace marker access must be replaced by the namespace adapter (Bun #106), paired atomically with this engine. Consumers must rebuild with the manifest's exact headers and libraries; an older executable is not interchangeable. diff --git a/.github/openclaw/release.py b/.github/openclaw/release.py index a9e94ee9729c..aba2f3b73ffa 100644 --- a/.github/openclaw/release.py +++ b/.github/openclaw/release.py @@ -50,9 +50,16 @@ def assemble(inputs,out,source): shutil.copyfile(f,out/name) artifacts[name]={'url':f'https://github.com/{REPO}/releases/download/{tag}/{name}','sha256':sha(f),'size':f.stat().st_size} lanes.append(manifest) + windows=json.loads((inputs/'windows-arm64-qualification/gate.json').read_text()) + if (windows.get('passed') is not True or windows.get('source')!=source + or windows.get('lane')!='bun-webkit-windows-arm64' or windows.get('architecture')!='arm64' + or windows.get('sha256')!=artifacts['bun-webkit-windows-arm64.tar.gz']['sha256'] + or any(windows.get(k) is not True for k in ('startup','dfg','ffi'))): + raise ValueError('missing or mismatched native Windows ARM64 qualification') + gate['windows_arm64']=windows # Retain low-level provenance and test logs as a separate archive. execute('tar','-czf',str(out/'provenance.tar.gz'),'-C',str(inputs),*[ - item for lane in config['lanes'] for item in [lane['label']+'/provenance',lane['label']+'/'+lane['label']+'.manifest.json']], 'qualification') + item for lane in config['lanes'] for item in [lane['label']+'/provenance',lane['label']+'/'+lane['label']+'.manifest.json']], 'qualification', 'windows-arm64-qualification') license_text='OpenClaw WebKit source and licenses\n\n'+f'Source: https://github.com/{REPO}/tree/{source}\nSource archive: https://github.com/{REPO}/archive/{source}.tar.gz\n\n' for p in ['Source/JavaScriptCore/COPYING.LIB','Source/WTF/wtf/COPYING.LIB']: file=ROOT/p @@ -92,7 +99,7 @@ def authorize(source,run_id): jobs=api('-H','Cache-Control: max-age=0',f'repos/{REPO}/actions/runs/{run_id}/jobs?filter=latest&per_page=100') required=[j for j in jobs['jobs'] if j['name']!='publish'] expected_builds={'build ('+l['label']+')' for l in json.loads((ROOT/'.github/openclaw/variants.json').read_text())['lanes']} - if ({j['name'] for j in required}!=expected_builds|{'preflight','test','assemble'} + if ({j['name'] for j in required}!=expected_builds|{'preflight','test','windows-arm64','assemble'} or any(j['conclusion']!='success' or j['run_attempt']!=run['run_attempt'] for j in required)): raise ValueError('all exact build, test and assembly jobs must pass') pending=api('-H','Cache-Control: max-age=0',f'repos/{REPO}/actions/runs/{run_id}/pending_deployments') @@ -121,7 +128,7 @@ def publish(directory,source): raise ValueError('publishing requires reviewed source on an explicitly permitted owned branch') # Tag creation fails atomically if the name exists. Neither existing drafts nor releases are resumed. api('-X','POST',f'repos/{REPO}/git/refs','-f','ref=refs/tags/'+tag,'-f','sha='+source) - body=f'OpenClaw-built WebKit at `{source}`. All nine lanes and Linux qualification passed.\n\n'+(ROOT/'.github/openclaw/release-notes.md').read_text() + body=f'OpenClaw-built WebKit at `{source}`. All {len(manifest["artifacts"])} lanes, Linux qualification and native Windows ARM64 engine probes passed.\n\n'+(ROOT/'.github/openclaw/release-notes.md').read_text() draft=api('-X','POST',f'repos/{REPO}/releases','-f','tag_name='+tag,'-f','name='+tag,'-f','body='+body,'-F','draft=true') release_id=draft['id'] execute('gh','release','upload',tag,'-R',REPO,*[str(p) for p in sorted(directory.iterdir())]) diff --git a/.github/openclaw/test_release.py b/.github/openclaw/test_release.py index 828db29d91b2..81422454b491 100644 --- a/.github/openclaw/test_release.py +++ b/.github/openclaw/test_release.py @@ -22,9 +22,23 @@ def setUp(self): label=lane['label'];d=self.inputs/label;d.mkdir();(d/'provenance').mkdir() f=d/(label+'.tar.gz');f.write_bytes(label.encode()) (d/(label+'.manifest.json')).write_text(json.dumps({'source':SOURCE,'lane':{'label':label},'files':{f.name:{'sha256':release.sha(f),'size':f.stat().st_size}}})) + w=self.inputs/'windows-arm64-qualification';w.mkdir() + archive=self.inputs/'bun-webkit-windows-arm64/bun-webkit-windows-arm64.tar.gz' + (w/'gate.json').write_text(json.dumps({'passed':True,'source':SOURCE,'lane':'bun-webkit-windows-arm64','architecture':'arm64','sha256':release.sha(archive),'startup':True,'dfg':True,'ffi':True})) + def test_windows_gate_rejects_wrong_source_digest_or_failed_probe(self): + p=self.inputs/'windows-arm64-qualification/gate.json';original=json.loads(p.read_text()) + for key,value in [('source','b'*40),('sha256','0'*64),('architecture','x64'),('startup',False),('dfg',False),('ffi',False)]: + with self.subTest(key=key): + p.write_text(json.dumps({**original,key:value})) + with self.assertRaisesRegex(ValueError,'native Windows ARM64'):self.assemble() + import shutil + shutil.rmtree(self.out) + def test_windows_gate_is_required(self): + (self.inputs/'windows-arm64-qualification/gate.json').unlink() + with self.assertRaises(FileNotFoundError):self.assemble() def assemble(self):release.assemble(self.inputs,self.out,SOURCE) def test_complete_set_roundtrip(self): - self.assemble();self.assertEqual(len(release.verify(self.out,SOURCE)['artifacts']),9) + self.assemble();self.assertEqual(len(release.verify(self.out,SOURCE)['artifacts']),10) def test_changed_lane_bytes_rejected(self): next(self.inputs.glob('*/*.tar.gz')).write_bytes(b'tampered') with self.assertRaises(ValueError):self.assemble() @@ -77,7 +91,7 @@ def test_authorization_requires_every_exact_green_job(self): def test_authorization_accepts_ref_qualified_path(self): run={'head_sha':SOURCE,'head_branch':'openclaw/main','event':'workflow_dispatch','path':'.github/workflows/openclaw-artifacts.yml@openclaw/main','run_attempt':1} config=json.loads((release.ROOT/'.github/openclaw/variants.json').read_text()) - names=['preflight','test','assemble']+['build ('+l['label']+')' for l in config['lanes']] + names=['preflight','test','windows-arm64','assemble']+['build ('+l['label']+')' for l in config['lanes']] jobs={'jobs':[{'name':n,'conclusion':'success','run_attempt':1} for n in names]} pending=[{'environment':{'name':'openclaw-artifacts','id':99},'current_user_can_approve':True}] with patch.object(release,'api',side_effect=[{'enabled':True},run,jobs,pending,{}]) as api,patch.object(release,'execute') as execute: @@ -88,7 +102,7 @@ def test_authorization_accepts_ref_qualified_path(self): def test_authorization_accepts_protected_batch_branch(self): run={'head_sha':SOURCE,'head_branch':'openclaw/batch-1','event':'workflow_dispatch','path':'.github/workflows/openclaw-artifacts.yml@openclaw/batch-1','run_attempt':1} config=json.loads((release.ROOT/'.github/openclaw/variants.json').read_text()) - names=['preflight','test','assemble']+['build ('+l['label']+')' for l in config['lanes']] + names=['preflight','test','windows-arm64','assemble']+['build ('+l['label']+')' for l in config['lanes']] jobs={'jobs':[{'name':n,'conclusion':'success','run_attempt':1} for n in names]} pending=[{'environment':{'name':'openclaw-artifacts','id':99},'current_user_can_approve':True}] with patch.object(release,'api',side_effect=[{'enabled':True},run,jobs,pending,{}]),patch.object(release,'execute') as execute: @@ -133,7 +147,7 @@ def test_wrong_engine_gate_source_rejected(self): def test_authorization_accepts_rebased_release_branch(self): run={'head_sha':SOURCE,'head_branch':'openclaw/release-5718a6ec','event':'workflow_dispatch','path':'.github/workflows/openclaw-artifacts.yml','run_attempt':1} config=json.loads((release.ROOT/'.github/openclaw/variants.json').read_text()) - names=['preflight','test','assemble']+['build ('+l['label']+')' for l in config['lanes']] + names=['preflight','test','windows-arm64','assemble']+['build ('+l['label']+')' for l in config['lanes']] jobs={'jobs':[{'name':n,'conclusion':'success','run_attempt':1} for n in names]} pending=[{'environment':{'name':'openclaw-artifacts','id':99},'current_user_can_approve':True}] with patch.object(release,'api',side_effect=[{'enabled':True},run,jobs,pending,{}]),patch.object(release,'execute') as execute: diff --git a/.github/openclaw/variants.json b/.github/openclaw/variants.json index 9965abafd972..b3a515840ca8 100644 --- a/.github/openclaw/variants.json +++ b/.github/openclaw/variants.json @@ -38,9 +38,13 @@ { "label": "bun-webkit-windows-amd64-lto", "consumer": "openclaw-release.yml optional windows-x64" + }, + { + "label": "bun-webkit-windows-arm64", + "consumer": "openclaw-release.yml windows-arm64: release without LTO" } ], - "excluded": "Windows arm64, Android, FreeBSD, debug and ASAN are not selected by the fork's active OpenClaw workflows. There is no separate x64 baseline WebKit archive.", + "excluded": "Android, FreeBSD, debug and ASAN are not selected by the fork's active OpenClaw workflows. Windows arm64 uses upstream's non-LTO lane because LLVM CodeView cannot encode its LTO register tuples. There is no separate x64 baseline WebKit archive.", "upstream_sync": { "base_commit": "d2d2a26ef973cdd97b37953f5dba58052acad74f", "upstream_commit": "1878660bb47a6a87ebe518cea0581ef4bf9b71f8", diff --git a/.github/workflows/openclaw-artifacts.yml b/.github/workflows/openclaw-artifacts.yml index f566c8a84bf5..9ac93a721a4d 100644 --- a/.github/workflows/openclaw-artifacts.yml +++ b/.github/workflows/openclaw-artifacts.yml @@ -156,8 +156,30 @@ jobs: ${{ runner.temp }}/qualification/sync/runtime-cache/*.json if-no-files-found: error retention-days: 7 + windows-arm64: + needs: build + runs-on: windows-11-arm + timeout-minutes: 15 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 + with: + persist-credentials: false + sparse-checkout: .github + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c + with: + name: bun-webkit-windows-arm64 + path: ${{ runner.temp }}/windows-artifact + - name: Qualify native ARM64 engine + shell: pwsh + run: ./.github/openclaw/qualify-windows-arm64.ps1 -ArtifactDirectory "$env:RUNNER_TEMP/windows-artifact" -OutputDirectory "$env:RUNNER_TEMP/windows-proof" -Source $env:GITHUB_SHA + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a + with: + name: windows-arm64-qualification + path: ${{ runner.temp }}/windows-proof/gate.json + if-no-files-found: error + retention-days: 7 assemble: - needs: [build, test] + needs: [build, test, windows-arm64] runs-on: ubuntu-24.04 timeout-minutes: 30 steps: @@ -185,7 +207,7 @@ jobs: retention-days: 7 publish: if: inputs.publish == true && (github.ref == 'refs/heads/openclaw/main' || github.ref == 'refs/heads/openclaw/batch-1' || github.ref == 'refs/heads/openclaw/release-5718a6ec') - needs: [build, test, assemble] + needs: [build, test, windows-arm64, assemble] environment: openclaw-artifacts permissions: contents: write diff --git a/.github/workflows/openclaw-checks.yml b/.github/workflows/openclaw-checks.yml index fa01c1c169cc..8b02b858f46b 100644 --- a/.github/workflows/openclaw-checks.yml +++ b/.github/workflows/openclaw-checks.yml @@ -9,6 +9,60 @@ concurrency: group: openclaw-engine-checks-${{ github.ref }} cancel-in-progress: true jobs: + windows-arm64-build: + if: github.repository == 'openclaw/WebKit' + runs-on: blacksmith-32vcpu-ubuntu-2404 + timeout-minutes: 150 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 + with: + fetch-depth: 64 + persist-credentials: false + sparse-checkout-cone-mode: false + sparse-checkout: | + /* + !/LayoutTests/ + !/JSTests/ + !/PerformanceTests/ + !/Websites/ + !/ManualTests/ + !/WebDriverTests/ + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 + with: + node-version: '26.3.0' + - name: Build unchanged upstream Windows ARM64 lane + env: + BUILDX_METADATA_PROVENANCE: max + run: python3 .github/openclaw/artifacts.py build --lane bun-webkit-windows-arm64 --output "$RUNNER_TEMP/windows-artifact" + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a + with: + name: bun-webkit-windows-arm64 + path: ${{ runner.temp }}/windows-artifact + compression-level: 0 + if-no-files-found: error + retention-days: 7 + windows-arm64: + needs: windows-arm64-build + runs-on: windows-11-arm + timeout-minutes: 15 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 + with: + persist-credentials: false + sparse-checkout: .github + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c + with: + name: bun-webkit-windows-arm64 + path: ${{ runner.temp }}/windows-artifact + - name: Qualify native ARM64 engine + shell: pwsh + run: ./.github/openclaw/qualify-windows-arm64.ps1 -ArtifactDirectory "$env:RUNNER_TEMP/windows-artifact" -OutputDirectory "$env:RUNNER_TEMP/windows-proof" -Source $env:GITHUB_SHA + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a + with: + name: windows-arm64-qualification + path: ${{ runner.temp }}/windows-proof/gate.json + if-no-files-found: error + retention-days: 7 linux-native: if: github.repository == 'openclaw/WebKit' runs-on: blacksmith-32vcpu-ubuntu-2404 diff --git a/OPENCLAW.md b/OPENCLAW.md index dec1e15a9e55..6384a6d9cd2e 100644 --- a/OPENCLAW.md +++ b/OPENCLAW.md @@ -21,8 +21,9 @@ Bun adapters and a joint rebuild because engine class layouts change. commit and each active workflow-to-artifact mapping. The release workflow uses LTO for Linux glibc x64/arm64, macOS x64/arm64, and its optional Linux musl x64/arm64 and Windows x64 targets. Fork CI adds non-LTO Linux x64 and macOS arm64. -That is nine artifacts. Windows arm64, debug, ASAN, Android and FreeBSD are not -selected by these OpenClaw workflows. x64 has one Nehalem baseline. +Windows ARM64 adds the upstream non-LTO release lane, for ten artifacts. LLVM's +CodeView register-tuple limitation prevents ARM64 LTO (oven-sh/bun#31345). +Debug, ASAN, Android and FreeBSD are not selected by these OpenClaw workflows. x64 has one Nehalem baseline. ## Add a patch and qualify @@ -46,7 +47,10 @@ they do not update or publish the Bun fork. The job runs the original 44-file fork selector output plus its two dependency checks, requires identical all-pass results, and checks Segmenter, four Proxy regressions, both 14-variant ALS hook modes and at least 14 worker resource-limit cases. Other platforms are built; -this gate does not claim native runtime qualification on every architecture. +this gate does not claim native runtime qualification on every architecture. A +Windows 11 ARM64 job verifies archive provenance and native ARM64 PE identity, +then runs JSC startup, DFG JIT and testFFI probes. Assembly and publication require +its source- and archive-bound receipt. ## Publish and verify @@ -65,7 +69,7 @@ concurrent GC. This workflow provides the pre-merge check; it cannot publish. For final publication, dispatch the reviewed full SHA with `publish: true`. This builds every lane and runs all qualification gates before waiting at the protected `openclaw-artifacts` environment. Only the publication job has repository write -permission. It assembles exactly the nine archives, `manifest.json`, +permission. It assembles exactly the ten archives, `manifest.json`, `SHA256SUMS`, `provenance.tar.gz`, and `LICENSE-SOURCES.txt`. Repository immutable releases must be enabled. Once the publishing dispatch is @@ -146,7 +150,7 @@ baseline. Engine and Bun headers and libraries must be rebuilt together. ## Unreleased -No changes yet. +- Add the upstream non-LTO Windows ARM64 archive and require native Windows 11 ARM64 startup, JIT and FFI qualification before publication. ## Repeated stack coordinates (2026-10-05)