diff --git a/AGENTS.md b/AGENTS.md index 6c288035f..fab99a3cf 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -29,6 +29,20 @@ Notes: - Prefer isolated worktrees for PR validation. Use `git-wt` for worktree workflows; `wt.exe` may resolve to WorkTrunk instead of Windows Terminal, so use the full Windows Terminal path when explicitly launching Terminal. - Do not claim completion without reporting validation results. +## Targeted Validation Paths + +Run the required validation above for every code change, then add the targeted path that matches the touched subsystem. + +### MXC / `system.run` / Windows node command execution + +When changing MXC sandboxing, `system.run`, exec approvals, Windows node command execution, gateway setup/connect E2E behavior, or files under `src\OpenClaw.Shared\Mxc`, run: + +```powershell +.\scripts\validate-mxc-e2e.ps1 +``` + +The script sets `OPENCLAW_RUN_E2E` and `OPENCLAW_RUN_MXC_E2E` itself, then runs the real WSL Gateway -> Windows node -> `system.run` MXC E2E proofs. It fails if the MXC proof skips. Use `-AllowSkip` only to document that the current host is not MXC-capable; do not report an `-AllowSkip` run as merge validation for MXC-related work. + ## Architecture Context for New Agents Start with these docs before changing connection, pairing, node, MCP, or tray UX behavior: diff --git a/docs/TEST_COVERAGE.md b/docs/TEST_COVERAGE.md index 67889f6cf..bbf8027fa 100644 --- a/docs/TEST_COVERAGE.md +++ b/docs/TEST_COVERAGE.md @@ -59,6 +59,19 @@ The method inventory is a source scan of `[Fact]` and `[Theory]` attributes. Use - **OpenClaw.WinNode.Cli.Tests** covers the standalone Windows node CLI contract. - **OpenClawTray.OnboardingV2.Tests** and **OpenClawTray.FunctionalUI.Tests** cover newer UI surfaces outside the main tray test project. +## Formal validation paths + +Use the smallest lane that proves the changed subsystem, but always include the +required closeout lane for code changes. + +| Lane | Entry point | Required when | +|---|---|---| +| Required closeout | `.\build.ps1`, Shared tests, Tray tests | Every code change and every agent closeout | +| GitHub-hosted PR/main CI | `.github\workflows\ci.yml` | Every pull request and push to `main`; runs normal E2E shards but skips MXC proofs on hosted runners | +| Local E2E | `OPENCLAW_RUN_E2E=1` with `OpenClaw.E2ETests` | Gateway setup/connect, recovery, or pairing changes that need real WSL Gateway coverage | +| Local MXC E2E | `.\scripts\validate-mxc-e2e.ps1` | MXC sandboxing, `system.run`, exec approvals, Windows node command execution, gateway setup/connect changes that affect MXC | +| Product WSL setup validation | `.\scripts\validate-wsl-gateway.ps1` | Tray onboarding/setup-engine changes that must prove the product WSL install path | + ## Running tests ```powershell @@ -76,6 +89,10 @@ dotnet test $env:OPENCLAW_RUN_E2E = "1" dotnet test .\tests\OpenClaw.E2ETests\OpenClaw.E2ETests.csproj -r win-x64 +# Formal MXC validation path. This sets the required integration/E2E env vars +# itself and fails when MXC proofs skip unless -AllowSkip is explicitly supplied. +.\scripts\validate-mxc-e2e.ps1 + # Single project dotnet test .\tests\OpenClaw.Connection.Tests\OpenClaw.Connection.Tests.csproj diff --git a/docs/WINDOWS_NODE_TESTING.md b/docs/WINDOWS_NODE_TESTING.md index d5fb27179..0d2038e53 100644 --- a/docs/WINDOWS_NODE_TESTING.md +++ b/docs/WINDOWS_NODE_TESTING.md @@ -122,17 +122,17 @@ When the node connects, it advertises these capabilities: ### Local sandbox validation - Sandbox integration tests are intended for local Windows development machines and may skip when the required local sandbox prerequisites are unavailable. - Build the tray app before running local sandbox validation so the required sandbox helper binaries are present in the app output. +- For MXC-related merge validation, prefer the formal script below because it sets the required gates and fails if MXC is skipped. ```powershell - .\build.ps1 - $env:OPENCLAW_RUN_INTEGRATION='1' - dotnet test .\tests\OpenClaw.Shared.Tests\OpenClaw.Shared.Tests.csproj --filter "FullyQualifiedName~Mxc" + .\scripts\validate-mxc-e2e.ps1 ``` ### Full Gateway `system.run` MXC runtime proof - The focused E2E below provisions a fresh WSL Gateway, starts an isolated tray instance, sets a local exec approval rule through MCP, invokes `system.run` through the real Gateway `node.invoke` path, and verifies tray MXC diagnostics show contained `mxc-direct-appc` execution for both allowed execution and denied writes to the tray data directory. - Run it when validating the Gateway/Windows node runtime path, not just direct MCP or shared library behavior. - GitHub-hosted Actions runners do not provide a working MXC/AppContainer runtime. The regular cloud E2E matrix should report these MXC proofs as skipped while still running the rest of setup-connect. Run the proof on a local MXC-enabled Windows machine. Only set `OPENCLAW_RUN_MXC_E2E=1` in GitHub Actions when using an MXC-enabled self-hosted runner. +- Use `.\scripts\validate-mxc-e2e.ps1` for normal local validation. It sets `OPENCLAW_RUN_E2E` and `OPENCLAW_RUN_MXC_E2E`, runs the real Gateway MXC proofs, and fails if the MXC proof skips. `-AllowSkip` is only for documenting a non-MXC host, not for merge validation of MXC-related work. - When reproducing this manually against an existing Gateway, make sure `gateway.nodes.allowCommands` includes `system.run`, `system.run.prepare`, and `system.which`, then approve any `pending-reapproval` request with `openclaw nodes approve `. The node can advertise `system.run` while the Gateway still blocks it until both gates are updated. ```powershell diff --git a/scripts/validate-mxc-e2e.ps1 b/scripts/validate-mxc-e2e.ps1 new file mode 100644 index 000000000..cdd2b8be4 --- /dev/null +++ b/scripts/validate-mxc-e2e.ps1 @@ -0,0 +1,213 @@ +<# +.SYNOPSIS + Runs the formal local MXC validation path. + +.DESCRIPTION + Builds the required Windows projects, enables the E2E/MXC gates, and runs + the real WSL Gateway -> Windows node -> system.run MXC E2E proofs. + + This script is intentionally stricter than the regular GitHub-hosted E2E + shard: MXC skips fail by default so MXC-related work cannot accidentally + claim end-to-end validation on a host that did not exercise MXC. + +.PARAMETER NoBuild + Skip build steps and run against existing outputs. + +.PARAMETER AllowSkip + Return success when MXC tests are reported but skipped. Use only for + discovery/documentation of a non-MXC host; do not use as merge validation + for MXC, system.run, exec approval, Windows node command execution, or + gateway setup/connect changes. +#> +[CmdletBinding()] +param( + [switch]$NoBuild, + + [switch]$AllowSkip, + + [ValidateSet("Debug", "Release")] + [string]$Configuration = "Debug", + + [ValidateSet("win-x64", "win-arm64")] + [string]$RuntimeIdentifier, + + [string]$ResultsDirectory +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = "Stop" + +$repoRoot = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot "..")).Path +Set-Location $repoRoot + +if ([string]::IsNullOrWhiteSpace($RuntimeIdentifier)) { + $RuntimeIdentifier = switch ([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture) { + ([System.Runtime.InteropServices.Architecture]::Arm64) { "win-arm64"; break } + default { "win-x64" } + } +} + +if ([string]::IsNullOrWhiteSpace($ResultsDirectory)) { + $ResultsDirectory = Join-Path $repoRoot "TestResults\MxcE2E" +} + +New-Item -ItemType Directory -Force -Path $ResultsDirectory | Out-Null + +function Invoke-Checked { + param( + [Parameter(Mandatory = $true)][string]$Name, + [Parameter(Mandatory = $true)][scriptblock]$Command + ) + + Write-Host "" + Write-Host "=== $Name ===" -ForegroundColor Cyan + & $Command + if ($LASTEXITCODE -ne 0) { + throw "$Name failed with exit code $LASTEXITCODE." + } +} + +function Read-Trx { + param([Parameter(Mandatory = $true)][string]$Path) + + if (-not (Test-Path -LiteralPath $Path)) { + throw "Expected TRX file was not created: $Path" + } + + [xml](Get-Content -LiteralPath $Path -Raw) +} + +function Get-TrxUnitTestResults { + param([Parameter(Mandatory = $true)][xml]$Trx) + + @($Trx.SelectNodes("//*[local-name()='UnitTestResult']")) +} + +function Get-TrxResultText { + param([Parameter(Mandatory = $true)][System.Xml.XmlElement]$Result) + + $parts = New-Object System.Collections.Generic.List[string] + foreach ($node in @($Result.SelectNodes(".//*[local-name()='Message' or local-name()='StdOut' or local-name()='StdErr']"))) { + if ($node -and -not [string]::IsNullOrWhiteSpace($node.InnerText)) { + $parts.Add($node.InnerText) + } + } + + [string]::Join("`n", $parts) +} + +function Assert-GatewayMxcProofsPassed { + param([Parameter(Mandatory = $true)][xml]$Trx) + + $expectedProofs = @( + "RealGateway_SystemRun_ExecutesThroughWindowsNodeMxcSandbox", + "RealGateway_SystemRun_BlocksWritesToTrayDataDirectoryInMxcSandbox" + ) + $results = Get-TrxUnitTestResults -Trx $Trx + $errors = New-Object System.Collections.Generic.List[string] + + foreach ($proof in $expectedProofs) { + $result = @($results | Where-Object { $_.GetAttribute("testName") -like "*$proof*" }) | Select-Object -First 1 + if ($null -eq $result) { + $errors.Add("Gateway MXC proof was not reported in TRX: $proof") + continue + } + + $outcome = $result.GetAttribute("outcome") + if ($outcome -eq "Passed") { + Write-Host "Gateway MXC proof passed: $proof" -ForegroundColor Green + continue + } + + if ($outcome -eq "NotExecuted" -or $outcome -eq "Skipped") { + $text = Get-TrxResultText -Result $result + $skipMessage = if ([string]::IsNullOrWhiteSpace($text)) { "no skip reason in TRX" } else { $text.Trim() } + $message = "Gateway MXC proof skipped: $proof ($skipMessage)" + if ($AllowSkip) { + Write-Warning $message + } else { + $errors.Add("$message. Run on an MXC-enabled Windows machine or pass -AllowSkip only when documenting a blocked host.") + } + continue + } + + $errors.Add("Gateway MXC proof '$proof' had unexpected outcome '$outcome'.") + } + + if ($errors.Count -gt 0) { + throw [string]::Join("`n", $errors) + } +} + +function Set-ProcessEnv { + param( + [Parameter(Mandatory = $true)][string]$Name, + [string]$Value + ) + + [Environment]::SetEnvironmentVariable($Name, $Value, "Process") +} + +$trackedEnvVars = @( + "OPENCLAW_REPO_ROOT", + "OPENCLAW_RUN_E2E", + "OPENCLAW_RUN_MXC_E2E" +) +$previousEnv = @{} +foreach ($name in $trackedEnvVars) { + $previousEnv[$name] = [Environment]::GetEnvironmentVariable($name, "Process") +} + +try { + Set-ProcessEnv -Name "OPENCLAW_REPO_ROOT" -Value $repoRoot + Set-ProcessEnv -Name "OPENCLAW_RUN_E2E" -Value "1" + Set-ProcessEnv -Name "OPENCLAW_RUN_MXC_E2E" -Value "1" + + Write-Host "OpenClaw MXC validation" + Write-Host " Repo: $repoRoot" + Write-Host " Configuration: $Configuration" + Write-Host " RuntimeIdentifier: $RuntimeIdentifier" + Write-Host " Results: $ResultsDirectory" + if ($AllowSkip) { + Write-Warning "-AllowSkip is enabled. This run may document a non-MXC host, but it is not sufficient merge validation for MXC-related work." + } + + if (-not $NoBuild) { + Invoke-Checked -Name "Build repository" -Command { + $powerShellExe = (Get-Process -Id $PID).Path + & $powerShellExe -NoProfile -File (Join-Path $repoRoot "build.ps1") -Configuration $Configuration + } + + Invoke-Checked -Name "Build tray app for $RuntimeIdentifier" -Command { + & dotnet build ".\src\OpenClaw.Tray.WinUI\OpenClaw.Tray.WinUI.csproj" -c $Configuration -r $RuntimeIdentifier + } + + Invoke-Checked -Name "Build E2E tests for $RuntimeIdentifier" -Command { + & dotnet build ".\tests\OpenClaw.E2ETests\OpenClaw.E2ETests.csproj" -c $Configuration -r $RuntimeIdentifier + } + } + + $e2eTrx = Join-Path $ResultsDirectory "OpenClaw.E2ETests.Mxc.trx" + $e2eConsoleLog = Join-Path $ResultsDirectory "OpenClaw.E2ETests.Mxc.console.log" + Invoke-Checked -Name "Run Gateway MXC E2E proofs" -Command { + & dotnet test ".\tests\OpenClaw.E2ETests\OpenClaw.E2ETests.csproj" ` + --no-build ` + --no-restore ` + -c $Configuration ` + -r $RuntimeIdentifier ` + --verbosity normal ` + --results-directory $ResultsDirectory ` + --logger "trx;LogFileName=OpenClaw.E2ETests.Mxc.trx" ` + --logger "console;verbosity=detailed" ` + --filter "FullyQualifiedName~OpenClaw.E2ETests.Setup.MxcSetupAndConnectTests" ` + 2>&1 | Tee-Object -FilePath $e2eConsoleLog + } + Assert-GatewayMxcProofsPassed -Trx (Read-Trx -Path $e2eTrx) + + Write-Host "" + Write-Host "MXC validation completed successfully." -ForegroundColor Green +} finally { + foreach ($name in $trackedEnvVars) { + [Environment]::SetEnvironmentVariable($name, $previousEnv[$name], "Process") + } +}