diff --git a/DESIGN.md b/DESIGN.md index 47def131..7b70db54 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -12,16 +12,12 @@ backend owns a structured layer (a browser DOM, a native UIA/AX tree) the resolution ladder's TOP rung re-finds the recorded target as an ELEMENT and acts on it deterministically (`StructuralActionBackend`, see Resolution ladder). Structure is preferred where present; the visual ladder is the fallback floor -for pixel-only substrates (RDP/Citrix/canvas). Backend evidence varies and is -stated honestly: the **shipped, end-to-end-exercised** backend is -Playwright-driven (headless-capable, CI-friendly, permission-free) and is the -only path proven against a real third-party app. Beyond it, a `WindowsBackend` -(UI Automation over the WindowsAgentArena server) is **proven structurally on a -local Windows-on-ARM VM** (record → compile → replay, DB-judged), and a FreeRDP -`RDPBackend` plus a Citrix/remote-display pixel-only backend exist as -**spikes, not validated integrations** — their live behavior is unmeasured to -the degree disclosed in `docs/backends/RDP.md` and `docs/desktop/CITRIX_PIXEL.md`. -They are adapters onto the one protocol, not rewrites. +for pixel-only substrates (RDP/Citrix/canvas). Playwright, Windows UIA, macOS +AX, Linux AT-SPI, RDP, and Citrix/pixel-window adapters implement the same +backend contract. Their evidence is bound to exact fixtures, counted tasks, +environments, or deployment qualifications in `docs/PRODUCT_STATUS.md` and +`docs/VERIFICATION.md`. A backend name alone does not show that an application +or environment is qualified. ## Core contracts (additive-only; do not change without updating this doc) @@ -371,8 +367,10 @@ referral → New Encounter → click "Triage" → click Note field → type note `Recorder` so frames/events are captured (before frame, act, wait settle, after frame). -`PlaywrightBackend(page)` implements `Backend` (chromium, fixed viewport -1280x800, deviceScaleFactor=1). `Recorder(backend, out_dir)` wraps a backend +`PlaywrightBackend(page)` implements `Backend` for Chromium. The MockMed +fixture starts at 1280x800 with `deviceScaleFactor=1`. An attached recording can +start a new exact geometry epoch after a stable viewport or device-scale +change. `Recorder(backend, out_dir)` wraps a backend with the same action methods plus `type_text(text, param=None)` and `finish() -> recording dir`. diff --git a/README.md b/README.md index 91630595..bd5f9116 100644 --- a/README.md +++ b/README.md @@ -12,13 +12,13 @@ [Discussions](https://github.com/OpenAdaptAI/openadapt-flow/discussions) · [Contributing](CONTRIBUTING.md) -**openadapt-flow is the OpenAdapt engine: a governed demonstration compiler.** -Record a task once, compile it to a deterministic program, and replay that -program deterministically with zero model calls on the healthy path. Instead of -silently doing the wrong thing when an interface drifts, it re-resolves from the -evidence the demonstration retained, or it **halts** for a human or an AI, gated -by an identity check and independent effect verification. It runs entirely on -your machine; nothing egresses unless you opt in. +**openadapt-flow is the demonstration compiler and governed runtime behind +OpenAdapt.** It compiles a demonstrated GUI workflow into a deterministic, +locally executable program. Healthy runs make no model calls. When an interface +drifts, Flow re-resolves from retained evidence. A person or configured model +can propose a repair. Identity, effect, and policy checks still apply, and Flow +halts when verification fails. It runs on your machine and doesn't send data +anywhere unless you opt in. It targets repeated workflows across every interface an operator touches: browser pages, native Windows / macOS / Linux desktops, and remote-display @@ -48,14 +48,14 @@ rounds against the wrong-target check. ## Try it -The canonical first run uses the [OpenAdapt](https://github.com/OpenAdaptAI/openadapt) -launcher, which handles Python versions, virtual environments, and shell quoting -for you: +The optional product check uses the +[OpenAdapt](https://github.com/OpenAdaptAI/openadapt) launcher, which handles +Python versions, virtual environments, and shell quoting for you: ```bash curl -fsSL https://raw.githubusercontent.com/OpenAdaptAI/openadapt-flow/main/scripts/install.sh | sh -openadapt quickstart # the whole loop, VERIFIED +openadapt quickstart # optional product check, VERIFIED ``` Prefer plain pip? Two commands (quote the brackets; on Windows `cmd.exe` use @@ -64,7 +64,7 @@ double quotes: `pip install "openadapt[browser]"`): ```bash pip install 'openadapt[browser]' -openadapt quickstart # the whole loop, VERIFIED +openadapt quickstart # optional product check, VERIFIED ``` **Requirements:** Python 3.10–3.12 (3.13+ is not yet supported; the installer @@ -76,9 +76,7 @@ name: ```bash pip install 'openadapt-flow[browser]' -openadapt-flow tutorial # same loop as `openadapt quickstart` - -openadapt-flow tutorial --break-it # then watch it catch a lie +openadapt-flow tutorial # optional product check ``` `tutorial` records a demonstration against the bundled MockMed fixture, mines its @@ -87,10 +85,8 @@ policy, and verifies the write by reading the system of record out of band — a path the app never calls, so the screen cannot influence it. It ends `VERIFIED` with zero model calls. -`--break-it` reruns the **same certified bundle** against a backend that lies: -the server rejects the write *after* the app paints its success banner, so every -on-screen check passes while nothing lands. The independent read refutes the -mined contract and the engine **HALTS** at the consequential step. +Your first real workflow starts with a small read-only task and test data: +[record it in your own app](#record-your-own-app-on-any-substrate). Full walkthrough, including `--guided` and the hand-driven `demo-record` / `compile` / `lint` / `certify` / `replay` stages: @@ -167,6 +163,9 @@ Artifacts: [baseline run report](docs/showcase/baseline-run/REPORT.md) and ## Record your own app, on any substrate +Start with one small read-only task and test data. Write down the result you +expect. Record the task, compile it, then inspect and lint the bundle: + Six substrates run on the same `Backend` protocol and the same governed runtime, selected with `--backend web | windows | macos | linux | rdp | citrix` on `record`, `replay`, and `run`. The browser is one surface among six, not a @@ -177,9 +176,26 @@ compiled bundle is bound to the exact surface it was recorded on. ```bash openadapt-flow record --backend web --url https://your.app --out rec openadapt-flow compile rec --out bundle --name my-task -openadapt-flow replay bundle --backend web --url https://your.app +openadapt-flow visualize bundle -o graph.html +openadapt-flow lint bundle ``` +Confirm that the bundle contains only the read-only task you selected. If lint +reports a state-changing, unknown, consequential, or irreversible action, stop +and [qualify it](https://openadapt.ai/qualify) before Flow first actuates it. + +Then replay the read-only workflow with the browser visible: + +```bash +openadapt-flow replay bundle --backend web --url https://your.app --headed \ + --run-dir first-run +``` + +Review `first-run/REPORT.md`. Confirm that the recorded steps and the final +result match what you expected before you expand the task. + +Qualify the exact app and environment before unattended use. + - Install matrix, exact commands for all six substrates, the counted evidence behind each, and the two remote execution modes: [backends and surface support](docs/SURFACES.md). diff --git a/claims.yaml b/claims.yaml index 5446119b..b93a3ee3 100644 --- a/claims.yaml +++ b/claims.yaml @@ -47,8 +47,9 @@ claims: # ------------------------------------------------------------------ web - id: web-supported claim: >- - Web (browser) workflows are supported today: record a GUI workflow once, - then replay it deterministically and locally. + The Playwright browser path records a GUI workflow and replays it + deterministically and locally. Required CI exercises the recorder, + compiler, and replay contract. surfaces: [README.md, website, docs] tier: supported evidence: @@ -199,17 +200,18 @@ claims: reflected evidence from Python at the settled boundary instead. caveats: - >- - "Supported" is scoped to the reference headless-browser backend in this - registry. Desktop and remote-display workflows use the separately scoped - acceptance and code-qualified claims below. + `supported` names the registry's required-CI evidence tier. It is not a + product state or a general application claim. Production requires active + release admissions for all seven product targets. A Production run also + needs an active workflow admission for the exact sealed bundle. - >- The full record->compile->replay browser suite runs in the required e2e-browser PR gate and repeats in the weekly compatibility matrix. - >- Existing-session attachment is Chromium-only and loopback-only. It - requires a dedicated browser process started with remote debugging. - It does not claim support for the Capture Chrome extension prototype - or direct extension replay. + requires a dedicated browser process with remote debugging. The + Playwright-native browser path owns this claim. Capture extension code + is outside its evidence scope. # -------------------------------------------------- deterministic $0 replay - id: deterministic-zero-model-replay diff --git a/docs/LIMITS.md b/docs/LIMITS.md index afcbc3d5..10f2b084 100644 --- a/docs/LIMITS.md +++ b/docs/LIMITS.md @@ -314,12 +314,12 @@ halt does not prove earlier actions were harmless. ## Interaction and environment limits -The recorded-surface evidence is strongest when the target remains inside one -captured browser surface and the demonstration exposes observable outcomes. +Qualification is strongest when the demonstration retains exact observations +and exposes independently verifiable outcomes. | Condition | Current boundary | | --- | --- | -| Zoom, DPI, font, layout, or viewport changes | Structural evidence can survive some reflow and visual rungs can survive some movement, but support is workflow-specific. Large rescale or reflow can halt. | +| Zoom, DPI, font, layout, viewport, or monitor changes | Browser attach recording accepts a stable viewport or device-scale change between actions, starts a new exact geometry epoch, and refuses an action that overlaps the transition. Native offline conversion can normalize a stable source-window move, resize, monitor change, or scale change into one fixed output viewport when every timeline row carries valid mapping metadata. It refuses a malformed timeline or a changed output viewport. RDP and Citrix live backends can rebaseline geometry between actions and refuse a change during the exact-frame lease. Other rescale or reflow behavior remains workflow-specific. | | Native select menus, file choosers, permission prompts, and secure desktops | OS or browser chrome may not appear in the captured surface and may be unrecordable or undrivable. Prefer an application-level or keyboard/API path. | | New tabs and windows | Opening a tab can be observed structurally where the backend supports it; interaction inside additional windows and multi-window coordination are not a general supported path. | | Drag and drop or gesture-heavy controls | Not a general supported primitive. Use a structured/API alternative or validate a purpose-built workflow. | diff --git a/docs/PRODUCT_STATUS.md b/docs/PRODUCT_STATUS.md index f39139f0..84703f71 100644 --- a/docs/PRODUCT_STATUS.md +++ b/docs/PRODUCT_STATUS.md @@ -47,16 +47,16 @@ product lifecycle state. | AI-assisted repair | **Required CI contracts; deployment evidence required** | Local and remote VLM contracts, egress gates, refusal behavior, and retention boundaries are tested. | It is off by default. A model cannot authorize an action or prove identity or effect. The exact endpoint and task require workflow qualification. | | Human teaching (`teach`) | **Required CI contracts; field evidence required** | Halt-to-correction-to-guarded-promotion and regression refusal run in default CI. | Evidence is controlled and synthetic. Broad authoring UX and field recovery time require deployment evidence. | | Windows UIA replay | **Counted task acceptance** | Candidate `20260717-candidate-56759c8-v2` completed 3/3 exact WinForms trials with independently confirmed SQLite effects and 12 native UIA delivery receipts. Stale and ambiguous targets each refused 3/3; silent incorrect successes, over-halts, and model calls were zero. See [`benchmark/windows_uia/results.json`](../benchmark/windows_uia/results.json). | Acceptance covers the in-tree WinForms workflow and exact Windows VM. Each third-party application is qualified against its own controls, versions, identity rules, and effect oracle. | -| Desktop recording (`windows` / `macos` / `linux` / `rdp` / `citrix`) | **Required CI plus substrate acceptance** | `openadapt-capture` conversion, compile, and replay orchestration run in CI for every desktop selector, and the native substrate qualifications below prove the corresponding actuation paths. | Offline pixel capture cannot reconstruct structural accessibility evidence. Workflows that require UIA, AX, or AT-SPI identity use a live structural observer or are re-armed against the qualified application before release. Regulated profiles require declared secret handling and fail-closed privacy configuration. | -| Native macOS desktop actuation | **Scoped acceptance** | Candidate `b1b61a5` completed 3/3 TextEdit replace-and-save trials with exact file-byte effects and refused a two-window ambiguous selector without changing either file. See the [accepted evidence adjudication](../benchmark/macos_native/textedit_counted_3plus1_b1b61a5_20260717.adjudication.json). | Acceptance covers TextEdit on one macOS 15.7.3 Apple Silicon host and active user session. Customer applications require workflow-specific qualification. | +| Desktop recording (`windows` / `macos` / `linux` / `rdp` / `citrix`) | **Required CI plus substrate acceptance** | `openadapt-capture` is the canonical native screen, mouse, keyboard, timing, window-scope, and media-capture component. Capture conversion, compile, and replay orchestration run in CI for every desktop selector, and the native substrate qualifications below prove the corresponding actuation paths. | Offline pixel capture cannot reconstruct structural accessibility evidence. A workflow that requires UIA, AX, or AT-SPI identity must retain a live structural observation or receive that evidence during qualification. Regulated profiles require declared secret handling and fail-closed privacy configuration. | +| Native macOS desktop actuation | **Counted task acceptance** | Candidate `b1b61a5` completed 3/3 TextEdit replace-and-save trials with exact file-byte effects and refused a two-window ambiguous selector without changing either file. See the [accepted evidence adjudication](../benchmark/macos_native/textedit_counted_3plus1_b1b61a5_20260717.adjudication.json). | Acceptance covers TextEdit on one macOS 15.7.3 Apple Silicon host and active user session. Customer applications require workflow-specific qualification. | | Native macOS AX structured identity | **Counted task acceptance plus required CI** | The macOS backend implements the same structured-layer contract as the browser DOM, Windows UIA, and Linux AT-SPI backends: it records a stable AX locator, re-finds the UNIQUE element at replay, refuses ambiguous / truncated / scope-escaping enumeration instead of guessing, and returns structured text under a point. Headless unit CI covers record/locate/refuse; a live-AX TextEdit run produced real evidence ([AX identity adjudication](../benchmark/macos_native/ax_identity_20260720.adjudication.json)); the record→compile→replay conformance test asserts zero model calls on healthy replay. See [`tests/test_macos_structural.py`](../tests/test_macos_structural.py) and the [capability matrix](../tests/test_backend_capability_matrix.py). | The backend uses gated point-bound physical click after structural resolution rather than claiming AXPress everywhere. AX exposure varies by application; controls without durable AX identity use the visual ladder. | | Native Linux desktop actuation | **Counted task acceptance plus required CI** | The required `linux-atspi-x11` job runs a real GTK3 application against AT-SPI inside an isolated Xvfb/session-D-Bus environment: 3 clean exact-file-effect trials, 3 ambiguous-target refusals, and 3 stale-target refusals. Unit CI covers the remaining window, traversal, capture, physical-input, and portal boundaries. | Acceptance is bounded to the in-tree GTK3 workflow and CI image. Each application and environment retains its own qualification. The built-in driver uses X11; Wayland requires a live operator-approved XDG portal session and refuses without one. | | RDP | **Counted task acceptance plus required CI** | Candidate `82a658a` completed 3/3 real-network Aardwolf RDP trials into Windows 11, with a guest-tools file oracle, zero failures, zero silent incorrect successes, zero over-halts, and zero model calls. The public multi-window FreeRDP campaign adds a bounded 27-trial contract with independent SQLite, CSV, and Maildir oracles. The backend also rebaselines a changed framebuffer between actions, refuses a change during the exact-frame lease, refuses unsupported horizontal scroll before delivery, and classifies transport failures as uncertain delivery. See the [accepted batch](../benchmark/rdp/ACCEPTED_BATCH_82A658A.md) and [campaign contract](../benchmark/rdp_multiapp/README.md). | The accepted batch covers the tested 1280×800 transport/input task. The multi-window fixture uses synthetic applications. Target applications, identity/effect rules, session policies, and display conditions are qualified per deployment. A composite multi-monitor session remains deployment-qualified evidence, not part of the accepted 1280×800 batch. | | Citrix / pixel-only remote display | **Required CI plus counted no-DOM stand-in** | `--backend citrix` binds an exact Citrix Workspace window, readiness marker, pixel-only ladder, governed run, durable resume, and report; required CI covers those orchestration and refusal contracts. The window driver recalculates capture scale after a resize or cross-monitor move and refuses DPI or geometry drift during input. The public real-ICA preflight adds distinct authority keys, executable and oracle attestations, a signed display and monitor-topology observation, explicit reliability metrics, one-use campaign state, crash recovery, and uncertain-dispatch handling. Separately, the retained no-DOM driver qualification passed 3 healthy effect-confirmed trials and 3 drift safe-halts with zero model calls, silent incorrect successes, or false completion, and records `code_readiness_accepted=true`. | The counted stand-in and preflight do not claim live ICA/HDX acceptance. A live result remains bound to the exact Workspace/server/application/display matrix, customer-approved executable, and independent effect oracle. Deployment-specific recipes, data, and thresholds stay outside the public repository. | -| Identity verification | **Required CI; exact workflow binding required** | Wrong-entity refusal and adversarial corpora run in CI. | Unarmed clicks have no identity check. Real compiled bundles currently arm only a subset of clicks. | +| Identity verification | **Required CI; exact workflow binding required** | Wrong-entity refusal and adversarial corpora run in CI. | An action without an identity contract has no entity check. Workflow admission must bind the exact armed actions and identity authority. | | System-of-record effect verification | **Required CI; exact verifier binding required** | REST, FHIR, SQL, file, and document verifier contracts catch fault classes that screen-only verification misses. A deployment with multiple reviewed read boundaries selects and preflights the strongest evidence tier before input, retains that binding through durable resume, and never downgrades after an action. | Effects are not generally inferred; both authored effects and a configured verifier are required. A selected verifier that becomes unavailable halts or enters reconciliation. | -| Lint and certification policies | **Required CI** | The CLI reports coverage gaps and refuses bundles that violate a selected policy. | Certification is opt-in; `replay` remains the permissive tutorial path. Use fail-closed `run` for a deployment. | -| Durable pause, approval, and resume | **Required CI; authenticated operator route required** | Checkpoint, bundle-version binding, approval, stale-pause, and resume semantics are tested. | Operator identity is recorded, not integrated with an enterprise IdP; field operation is unmeasured. | +| Lint and certification policies | **Required CI** | The CLI reports coverage gaps and refuses bundles that violate a selected policy. | `replay` remains the permissive tutorial path. Governed deployment uses fail-closed `run` plus the required release and workflow admissions. | +| Durable pause, approval, and resume | **Required CI; authenticated operator route required** | Checkpoint, bundle-version binding, approval, stale-pause, and resume semantics are tested. | The engine records an asserted operator identity. Desktop, Cloud, or a customer-local identity route must authenticate that principal. | | Typed business decisions | **Required CI; authenticated operator route required** | A typed qualification API adds or updates a finite decision node without manual manifest edits and invalidates stale certification. The graph runtime pauses at the certified choice, validates a supplied principal and role, retains a signed durable receipt, restores it after a crash, revalidates the live application, and permits only the certified successor branch. | The engine does not authenticate a user. Desktop, Cloud, or a customer-local identity route must supply an authenticated principal. A decision never replaces entity identity or effect verification. | | Reviewed judgment cases | **Required CI contracts; reviewed workflow evidence required** | Qualification binds typed facts, local evidence hashes, reviewer provenance, and the exact decision contract to reviewed examples or counterfactuals. It preserves permanent human authority, requires reciprocal contrasts for an automatic-rule candidate, and refuses certification when a case still needs evidence. | The case layer does not synthesize executable policy from one or more examples. A reviewed automatic rule must be authored and qualified through the normal program path. | | Qualified remote decision tasks | **Required CI contracts; negotiated peer schema required** | An explicitly negotiated V2 task binds optional reviewed entity wording to the exact qualification, bundle, step, policy, and pause. V1 stays byte-compatible, and an unavailable or unrecognized class renders as the signed neutral `record` or `item` fallback. | V2 requires `openadapt-types` 0.10.x and a consumer that negotiates the schema. Actual entity identifiers and live revalidation stay inside the customer-controlled runner. | diff --git a/docs/TUTORIAL.md b/docs/TUTORIAL.md index 1b02d841..c3ac5190 100644 --- a/docs/TUTORIAL.md +++ b/docs/TUTORIAL.md @@ -1,13 +1,11 @@ -# The bundled tutorial, end to end +# The optional bundled product check `openadapt-flow tutorial` (which `openadapt quickstart` delegates to) is the -complete free path against the bundled MockMed application, a synthetic -practice-management fixture served through its real transactional backend. +optional installation and product check. It runs against MockMed, a bundled +synthetic practice-management fixture with a real transactional backend. ```bash -openadapt-flow tutorial # the whole loop, VERIFIED -openadapt-flow tutorial --break-it # then watch it catch a lie -openadapt-flow tutorial --guided # perform the demo yourself +openadapt-flow tutorial # optional product check, VERIFIED ``` ## What `tutorial` does @@ -20,16 +18,10 @@ record out of band — a path the application itself never calls, so the screen cannot influence it. It ends `VERIFIED` with zero model calls, and writes a shareable `receipt.png` / `receipt.json` beside the run. -## What `--break-it` does - -`--break-it` reruns the **same certified bundle** against a backend that lies: -the server rejects the write *after* the application has painted its success -banner, so every on-screen check passes while nothing lands. The independent -read of the system of record refutes the mined `record_written` contract and the -engine **HALTS** at the consequential step instead of believing the screen. The -caught fault's evidence is a clearly-labeled local `run-broken/REPORT.md`; no -shareable receipt is emitted for it, because only `VERIFIED` runs may use the -success rail. +For your first real workflow, use a small read-only task and test data. Record +it, inspect and lint the compiled bundle, then continue to supervised replay +only if it contains the read-only task you selected. The exact commands are in +[Record your own app](../README.md#record-your-own-app-on-any-substrate). ## What `--guided` does @@ -37,10 +29,16 @@ For a live walkthrough, perform the demonstration yourself and then watch the compiled replay at a visible pace. The recording browser closes after OpenAdapt observes the saved record through the separate read-only interface. OpenAdapt then compiles, certifies, and replays what you demonstrated. If you prefer a -fully automatic presentation, use -`openadapt-flow tutorial --headed --presentation-delay 1`. The delay applies -only to this bundled tutorial. The ordinary `tutorial`, `replay`, and `run` -paths keep their normal execution speed. +fully automatic presentation, use: + +```bash +openadapt-flow tutorial --guided +openadapt-flow tutorial --headed --presentation-delay 1 +``` + +The delay in the second command applies only to this bundled tutorial. The +ordinary `tutorial`, `replay`, and `run` paths keep their normal execution +speed. The receipt the tutorial emits is generated from a closed allow-list — outcomes, counts, digests, and validated package versions — so it can carry no screenshot, diff --git a/docs/VERIFICATION.md b/docs/VERIFICATION.md index 3bbe0e0f..ef20aaaa 100644 --- a/docs/VERIFICATION.md +++ b/docs/VERIFICATION.md @@ -20,7 +20,7 @@ ### `web-supported` — supported — bound to required CI pass evidence -> Web (browser) workflows are supported today: record a GUI workflow once, then replay it deterministically and locally. +> The Playwright browser path records a GUI workflow and replays it deterministically and locally. Required CI exercises the recorder, compiler, and replay contract. - Surfaces: README.md, website, docs - Strongest evidence strength: **supported** (tier is `supported`) @@ -50,9 +50,9 @@ **Caveats (honest limits):** -- "Supported" is scoped to the reference headless-browser backend in this registry. Desktop and remote-display workflows use the separately scoped acceptance and code-qualified claims below. +- `supported` names the registry's required-CI evidence tier. It is not a product state or a general application claim. Production requires active release admissions for all seven product targets. A Production run also needs an active workflow admission for the exact sealed bundle. - The full record->compile->replay browser suite runs in the required e2e-browser PR gate and repeats in the weekly compatibility matrix. -- Existing-session attachment is Chromium-only and loopback-only. It requires a dedicated browser process started with remote debugging. It does not claim support for the Capture Chrome extension prototype or direct extension replay. +- Existing-session attachment is Chromium-only and loopback-only. It requires a dedicated browser process with remote debugging. The Playwright-native browser path owns this claim. Capture extension code is outside its evidence scope. ### `deterministic-zero-model-replay` — supported — bound to required CI pass evidence diff --git a/docs/deployment/ON_PREM_VLM.md b/docs/deployment/ON_PREM_VLM.md index 49fc5d99..b129de83 100644 --- a/docs/deployment/ON_PREM_VLM.md +++ b/docs/deployment/ON_PREM_VLM.md @@ -4,10 +4,11 @@ A single GPU appliance serves the identity-veto, grounding, and state-verificati VLM tiers to a fleet of **GPU-less** automation runners over the LAN. The runtime stays GPU-free and **patient data never leaves the building**. -> Status: appliance + fail-safe clients. Runtime wiring into the identity ladder -> and the resolution ladder's grounder slot lands separately (after the -> identity-ladder PR #33 merges). This document is the contract those integrations -> target. +> Current integration: deployment replay, resume, and attended execution use +> all three appliance tiers. When an operator configures the appliance and +> enables model grounding, the runtime adds the remote grounder, veto-only +> identity tier, and drift-oracle state verifier. The default run stays local +> and model-free. ## Topology @@ -159,20 +160,22 @@ state = RemoteStateVerifier(client) # yes / no / uncertain ## Integration -Wired into the `replay` CLI. An appliance is **opt-in** — set three env vars on -the runner and the grounding rung and identity veto tier come online; leave them -unset (the default) and the run stays fully local and model-free. +Flow uses the shared deployment constructor for replay, resume, and attended +execution. An appliance is **opt-in**: set three env vars on the runner and +enable model grounding. Leave the URL unset or keep model grounding disabled, +and the run stays fully local and model-free. ```bash export OPENADAPT_FLOW_VLM_URL="https://gpu-box.lan:8077" # unset => dormant export OPENADAPT_FLOW_VLM_TOKEN="$(cat /etc/openadapt/vlm_token)" export OPENADAPT_FLOW_VLM_TIMEOUT=2.0 # optional, seconds -openadapt-flow replay bundle +openadapt-flow replay bundle --allow-model-grounding ``` `appliance_from_env()` (`runtime/remote_vlm.py`) reads these and returns a -`RemoteAppliance` (or `None`); the CLI passes its handles into -`Replayer(grounder=..., identity_vlm=...)`. +`RemoteAppliance` (or `None`). The shared deployment constructor adds its +grounder as the model fallback and passes its identity and state-verifier +handles into the `Replayer`. - **Grounder slot:** `RemoteGrounder` satisfies the `Grounder` protocol (`runtime/grounder.py`) and drops into the resolution ladder's grounder slot @@ -184,10 +187,11 @@ openadapt-flow replay bundle `VERIFY → "same"` (fail-to-veto), and `MISMATCH`/`ABSTAIN` (the latter the default on any uncertainty or appliance outage) → `"different"` (halt). The tier can only veto; a down appliance means more halts, never a wrong click. -- **Drift-oracle postcondition** (`RemoteStateVerifier`): *not yet wired.* It - needs a postcondition-failure hook in the replayer (call the verifier only - when a deterministic postcondition false-fails under render drift; `"uncertain"` - keeps it a halt). Tracked as a follow-up. +- **Drift-oracle postcondition** (`RemoteStateVerifier`): after deterministic + checks and one settle retry fail, the replayer asks the verifier only about + render-drift-sensitive `text_present` and `region_stable` conditions. Only a + confident `yes` rescues the condition. A `no`, `uncertain`, error, or appliance + outage keeps the failure. Every call and rescue is recorded in the run result. ## PHI data-flow boundary diff --git a/docs/desktop/PHASE1.md b/docs/desktop/PHASE1.md index cf1f59fa..8b1f62da 100644 --- a/docs/desktop/PHASE1.md +++ b/docs/desktop/PHASE1.md @@ -153,13 +153,16 @@ is authoritative, not the stored ratio. space, so the adapter applies **no** scaling (rescaling would double-scale every click), screens each mouse action against the recorded bounds-timeline window events (out-of-window input refuses conversion — it targeted a -different window), refuses sessions where the target window was resized -(capture video and Flow recordings currently use one fixed viewport), verifies -extracted frames have that exact viewport, and stamps the output `meta.json` -with `window_capture` provenance. The `record --backend rdp|citrix` -orchestration adds `backend_hints` (`rdp_window` / `rdp_window_title`) naming -the recorded target window for remote replay. Native Windows and macOS window -recordings do not receive remote hints. +different window), validates resize-normalization metadata, verifies that +extracted frames use one fixed output viewport, and stamps the output `meta.json` +with `window_capture` provenance. Capture can normalize a stable source-window +move, resize, monitor change, or scale change into that output viewport when +each timeline row carries valid `source_viewport`, `content_rect`, and +`fit_scale` metadata. Flow refuses a malformed timeline or a changed output +viewport. The `record --backend rdp|citrix` orchestration adds +`backend_hints` (`rdp_window` / `rdp_window_title`) naming the recorded target +window for remote replay. Native Windows and macOS window recordings do not +receive remote hints. **Frame selection.** For an event at wall-clock `T`: *before* = last video frame at/before `T`; *after* = frame at `T + settle_s` (default 1.0 s), diff --git a/docs/desktop/RECORDING.md b/docs/desktop/RECORDING.md index a05c3caf..a714a5a3 100644 --- a/docs/desktop/RECORDING.md +++ b/docs/desktop/RECORDING.md @@ -82,6 +82,31 @@ can describe only the remote-client window or canvas, not controls inside the remote session. Those remote recordings instead use the external black-box visual, relational, identity, and fresh-frame contracts. +### Native source geometry + +A current window-scoped Capture session stores every frame with one exact +window-geometry row and source ordinal. Each action names the earlier pair that +supplied its coordinates. The pair binds the process start identity, display +topology, bounds, scale, fixed viewport, normalization rectangle, and geometry +generation. Recorder shutdown also retains one frame after input has stopped. + +The adapter verifies the artifact manifest and completion terminal for every +sealed v2 window capture. This includes external RDP and Citrix recordings. It +reads a private immutable snapshot, selects the action's exact bound frame, and +then selects the first retained frame after the action. It doesn't substitute a +nearby timestamp. + +For `windows`, `macos`, and `linux`, the adapter also emits `source_geometry`. +That object binds the source identities to the exact PNG in the Flow recording. +The compiler checks its PNG digest, requires one complete source sequence, and +carries the closed object on each compiled step. + +This contract does not promote local native geometry into an external +RDP/Citrix workflow. The CLI leaves `source_surface` unset for `rdp` and +`citrix`, and the compiler refuses a native source binding on `rdp`, `citrix`, +or `web`. Those surfaces keep their existing pixel, OCR, identity-region, and +fresh-frame contracts. + ### Secret handling The browser recorder blacks out a secret field's pixels using the field's DOM diff --git a/docs/validation/VALIDATION.md b/docs/validation/VALIDATION.md index 948d01ad..cfaf305d 100644 --- a/docs/validation/VALIDATION.md +++ b/docs/validation/VALIDATION.md @@ -688,6 +688,24 @@ Reported with equal honesty: postcondition system catching real cross-patient drift when an identity-bearing assertion happens to survive compilation. +## Advanced rejected-write verification + +The advanced tutorial flag runs the normal verified tutorial, then reuses its +exact certified bundle against the sample backend's `optimistic` fault: + +```bash +openadapt-flow tutorial --simulate-rejected-write +``` + +The sample UI paints its success banner, but the server rejects the write and +the system of record remains empty. The independent effect verifier must refute +`record_written`. The fault run halts and classifies the transaction as +`RECONCILIATION_REQUIRED`. It makes no blind retry or replay dispatch and writes +local evidence to +`run-rejected-write/REPORT.md`. It emits no success receipt. This synthetic +fixture tests the effect-verification boundary; it doesn't qualify a customer +workflow. + ## Reproduce ```bash diff --git a/docs/verification.json b/docs/verification.json index 63803a82..b4903604 100644 --- a/docs/verification.json +++ b/docs/verification.json @@ -8,7 +8,7 @@ "claims": [ { "id": "web-supported", - "claim": "Web (browser) workflows are supported today: record a GUI workflow once, then replay it deterministically and locally.", + "claim": "The Playwright browser path records a GUI workflow and replays it deterministically and locally. Required CI exercises the recorder, compiler, and replay contract.", "tier": "supported", "reproducibility": null, "surfaces": [ @@ -18,9 +18,9 @@ ], "strongest_evidence": "supported", "caveats": [ - "\"Supported\" is scoped to the reference headless-browser backend in this registry. Desktop and remote-display workflows use the separately scoped acceptance and code-qualified claims below.", + "`supported` names the registry's required-CI evidence tier. It is not a product state or a general application claim. Production requires active release admissions for all seven product targets. A Production run also needs an active workflow admission for the exact sealed bundle.", "The full record->compile->replay browser suite runs in the required e2e-browser PR gate and repeats in the weekly compatibility matrix.", - "Existing-session attachment is Chromium-only and loopback-only. It requires a dedicated browser process started with remote debugging. It does not claim support for the Capture Chrome extension prototype or direct extension replay." + "Existing-session attachment is Chromium-only and loopback-only. It requires a dedicated browser process with remote debugging. The Playwright-native browser path owns this claim. Capture extension code is outside its evidence scope." ], "evidence": [ { diff --git a/openadapt_flow/__init__.py b/openadapt_flow/__init__.py index 5b7c47ca..5a782114 100644 --- a/openadapt_flow/__init__.py +++ b/openadapt_flow/__init__.py @@ -1,6 +1,6 @@ """openadapt-flow: record once, compile, replay deterministically, heal on drift.""" -__version__ = "1.33.0" +__version__ = "1.34.0" from openadapt_flow.ir import ( # noqa: F401 ActionKind, diff --git a/openadapt_flow/__main__.py b/openadapt_flow/__main__.py index b603836e..2d1b1ddb 100644 --- a/openadapt_flow/__main__.py +++ b/openadapt_flow/__main__.py @@ -65,13 +65,23 @@ import sys from contextlib import contextmanager from pathlib import Path -from typing import TYPE_CHECKING, Any, Iterator, Literal, Optional, Sequence, cast +from typing import ( + TYPE_CHECKING, + Any, + Iterator, + Literal, + Mapping, + Optional, + Sequence, + cast, +) from urllib.parse import urlsplit from uuid import UUID if TYPE_CHECKING: # pragma: no cover from openadapt_flow.backend import Backend from openadapt_flow.ir import ExecutionTargetKind, RunReport + from openadapt_flow.runtime.authorization import RuntimeParamScalar from openadapt_flow.tutorial import BreakItResult _VIEWPORT = {"width": 1280, "height": 800} @@ -154,7 +164,7 @@ def _reject_unbound_record_target_flags(args: argparse.Namespace, backend: str) "web": set(), "windows": set(), "macos": {"macos_app", "macos_window_title"}, - "linux": set(), + "linux": {"linux_app", "linux_window_title"}, "rdp": {"rdp_window", "rdp_window_title", "rdp_readiness_text"}, "citrix": {"rdp_window", "rdp_window_title", "rdp_readiness_text"}, } @@ -167,15 +177,10 @@ def _reject_unbound_record_target_flags(args: argparse.Namespace, backend: str) "Scope the local recording with --window/--window-title, then " "pass --agent-url to replay or run" ) - elif backend == "linux" and attr in { - "linux_app", - "linux_window_title", - "linux_allow_physical_input", - }: + elif backend == "linux" and attr == "linux_allow_physical_input": reason = ( - "the current Capture component has no Linux window-scoping " - "primitive. Record the local Linux desktop without this flag, " - "then pass it to replay or run" + "this flag permits replay-time physical input. It doesn't " + "change which window Capture records, so pass it to replay or run" ) elif backend == "rdp" and attr == "rdp_host": reason = ( @@ -221,13 +226,6 @@ def _resolve_record_capture_window( owner = getattr(args, "window", None) title = getattr(args, "window_title", None) - if backend == "linux" and (owner is not None or title is not None): - raise SystemExit( - "record --backend linux: --window/--window-title cannot be applied " - "because the current Capture component has no Linux " - "window-scoping primitive. Record the local Linux desktop without " - "these flags. Nothing was recorded." - ) if backend == "macos": owner = _merge_record_window_selector( owner, @@ -243,6 +241,21 @@ def _resolve_record_capture_window( target_flag="--macos-window-title", backend=backend, ) + elif backend == "linux": + owner = _merge_record_window_selector( + owner, + getattr(args, "linux_app", None), + generic_flag="--window", + target_flag="--linux-app", + backend=backend, + ) + title = _merge_record_window_selector( + title, + getattr(args, "linux_window_title", None), + generic_flag="--window-title", + target_flag="--linux-window-title", + backend=backend, + ) elif backend in ("rdp", "citrix"): # The capture selector is a local owner/title substring. The replay # selector can be an exact process identity (for example ``wfica32``), @@ -258,7 +271,7 @@ def _resolve_record_capture_window( def _replay_params( pairs: Sequence[str] | None, params_file: str | None = None, -) -> dict[str, str]: +) -> dict[str, "RuntimeParamScalar"]: """Load replay bindings without requiring sensitive values in argv. ``--params-file`` is intended for managed runners: the file can be staged @@ -267,7 +280,9 @@ def _replay_params( """ import json - params: dict[str, str] = {} + from openadapt_flow.runtime.authorization import is_runtime_param_scalar + + params: dict[str, RuntimeParamScalar] = {} if params_file: path = Path(params_file) try: @@ -281,11 +296,9 @@ def _replay_params( for key, value in raw.items(): if not isinstance(key, str) or not key: raise SystemExit("--params-file keys must be non-empty strings") - if not isinstance(value, (str, int, float, bool)) or isinstance( - value, (dict, list) - ): + if not is_runtime_param_scalar(value): raise SystemExit(f"--params-file value for {key!r} must be a scalar") - params[key] = str(value) + params[key] = value params.update(_parse_params(pairs)) return params @@ -400,7 +413,10 @@ def _deployment_sections(args: argparse.Namespace): return cfg, effects, actuation -def _deployment_runtime(args: argparse.Namespace, params: dict[str, str] | None = None): +def _deployment_runtime( + args: argparse.Namespace, + params: Mapping[str, "RuntimeParamScalar"] | None = None, +): """Resolve the deployment wiring for a replay/run from ``--config`` + flags. Returns ``(cfg, effect_verifier, api_actuator, durable, allow_egress)``. @@ -416,10 +432,14 @@ def _deployment_runtime(args: argparse.Namespace, params: dict[str, str] | None ignores it. """ from openadapt_flow.deployment import build_api_actuator, build_effect_verifier + from openadapt_flow.runtime.authorization import runtime_params_for_gui cfg, effects, actuation = _deployment_sections(args) try: - effect_verifier = build_effect_verifier(effects, params=params) + effect_verifier = build_effect_verifier( + effects, + params=runtime_params_for_gui(params or {}), + ) api_actuator = build_api_actuator(actuation) except ValueError as e: raise SystemExit(str(e)) @@ -1228,6 +1248,7 @@ def _cmd_record_desktop(args: argparse.Namespace, backend: str) -> int: params=params, identifier_region=identifier_region, window=window, + source_surface=(backend if backend in ("windows", "macos", "linux") else None), backend_kind=backend if backend in ("rdp", "citrix") else None, replay_window=getattr(args, "rdp_window", None), replay_window_title=getattr(args, "rdp_window_title", None), @@ -1307,6 +1328,15 @@ def _cmd_tutorial(args: argparse.Namespace) -> int: presentation_delay_s = args.presentation_delay if presentation_delay_s is None: presentation_delay_s = GUIDED_PRESENTATION_DELAY_S if args.guided else 0.0 + deprecated_break_it = bool(getattr(args, "deprecated_break_it", False)) + if deprecated_break_it: + print( + "warning: --break-it is deprecated; use --simulate-rejected-write.", + file=sys.stderr, + ) + simulate_rejected_write = bool( + getattr(args, "simulate_rejected_write", False) + ) try: result = run_tutorial( out, @@ -1316,7 +1346,8 @@ def _cmd_tutorial(args: argparse.Namespace) -> int: interactive_record=interactive_record, presentation_delay_s=presentation_delay_s, echo=print, - break_it=args.break_it, + simulate_rejected_write=simulate_rejected_write, + break_it=deprecated_break_it, ) except TutorialError as e: print(f"\nTutorial REFUSED: {e}") @@ -1346,12 +1377,8 @@ def _cmd_tutorial(args: argparse.Namespace) -> int: print(f" {result.receipt_paths['json']}") if result.break_it is not None: - _print_break_it_narrative(result.break_it) + _print_rejected_write_narrative(result.break_it) elif result.execution_outcome == "VERIFIED": - print( - "\nNext: rerun this same bundle against a backend that lies -- and " - "watch the engine halt:\n openadapt-flow tutorial --break-it" - ) print(f"\n{_next_steps_block()}") else: # Presentation-only epilogue for the non-VERIFIED endings: what @@ -1362,7 +1389,7 @@ def _cmd_tutorial(args: argparse.Namespace) -> int: return 0 -def _print_break_it_narrative(broken: "BreakItResult") -> None: +def _print_rejected_write_narrative(broken: "BreakItResult") -> None: """Tell the caught-fault story from the halted run's own evidence. Every fact printed here was read back from the broken run's report or the @@ -1375,7 +1402,10 @@ def _print_break_it_narrative(broken: "BreakItResult") -> None: ) if broken.screen_claim_text: claim += f'\n (observed on screen: "{broken.screen_claim_text}")' - print("\n--- break-it: the same certified bundle, against a backend that lies ---") + print( + "\n--- rejected-write verification: the same certified bundle, " + "against a backend that lies ---" + ) print(f"\n Injected fault: {broken.fault!r} -- the server rejects the write") print(" AFTER the app reports success") print(f" The screen claimed: {claim}") @@ -1386,6 +1416,10 @@ def _print_break_it_narrative(broken: "BreakItResult") -> None: " read of the system of record, which holds " f"{broken.system_of_record_records} record(s)" ) + print( + f" The backend saw: {broken.rejected_writes} rejected write attempt " + "(no retry)" + ) print( f" The engine did: {broken.execution_outcome} at the consequential " "step instead of claiming\n" @@ -1405,10 +1439,6 @@ def _print_break_it_narrative(broken: "BreakItResult") -> None: "\nNo shareable receipt for the halted run: only VERIFIED runs may use " "the\nsuccess rail. The halt itself is the demonstration." ) - print( - "\nNext: record your own workflow:\n" - " openadapt-flow record --backend web --url " - ) def _cmd_compile(args: argparse.Namespace) -> int: @@ -4670,8 +4700,8 @@ def _add_backend_flags(p: argparse.ArgumentParser) -> None: metavar="APP", help=( "Exact AT-SPI application name for --backend linux (e.g. gedit). " - "Replay/run only: the current Capture path records the local Linux " - "desktop and refuses this flag. Overrides backend.linux_app." + "During record this scopes Capture to the matching local X11 " + "window; during replay/run it overrides backend.linux_app." ), ) p.add_argument( @@ -4680,9 +4710,8 @@ def _add_backend_flags(p: argparse.ArgumentParser) -> None: metavar="TITLE", help=( "Exact top-level window title for --backend linux. Zero or " - "multiple matches are refused. Replay/run only: the current " - "Capture path records the local Linux desktop and refuses this " - "flag. Overrides backend.linux_window_title." + "multiple matches are refused. During record this scopes Capture; " + "during replay/run it overrides backend.linux_window_title." ), ) p.add_argument( @@ -5118,17 +5147,23 @@ def build_parser() -> argparse.ArgumentParser: help="Skip writing the local receipt (the run and its report are unchanged)", ) p.add_argument( - "--break-it", + "--simulate-rejected-write", action="store_true", - dest="break_it", help=( - "After the clean VERIFIED run, rerun the SAME certified bundle " - "against a backend that silently rejects the write AFTER the app " - "paints its success banner -- and watch the engine HALT instead of " - "believing the screen. The halted run's evidence lands in " - "/run-broken/REPORT.md" + "After the clean VERIFIED run, this advanced fixture reruns the " + "same certified bundle against a sample backend that reports " + "success in the UI but rejects the write. OpenAdapt must refuse " + "false success and classify the transaction as " + "RECONCILIATION_REQUIRED. Evidence lands in " + "/run-rejected-write/REPORT.md" ), ) + p.add_argument( + "--break-it", + action="store_true", + dest="deprecated_break_it", + help=argparse.SUPPRESS, + ) p.set_defaults(func=_cmd_tutorial) p = sub.add_parser("compile", help="Compile a recording into a workflow bundle") diff --git a/openadapt_flow/action_evidence.py b/openadapt_flow/action_evidence.py index 2ef604e1..97987687 100644 --- a/openadapt_flow/action_evidence.py +++ b/openadapt_flow/action_evidence.py @@ -13,6 +13,7 @@ from typing import Any, Optional from openadapt_flow.ir import ActionKind, Step +from openadapt_flow.runtime.authorization import RuntimeParamScalar, runtime_param_text AUTOMATED_GUI_ACTUATIONS = frozenset( {"uia", "dom", "guarded_coordinate", "guarded_keyboard", "remote_guarded"} @@ -177,7 +178,7 @@ def _delivery_receipt_error( step: Step, result: Any, *, - params: Mapping[str, str], + params: Mapping[str, RuntimeParamScalar], ) -> Optional[str]: receipt = result.delivery_receipt if receipt is None: @@ -226,7 +227,10 @@ def _delivery_receipt_error( return "non-drag delivery receipt contains a destination fingerprint" if step.action is ActionKind.SELECT_OPTION: - selected = params.get(step.param) if step.param is not None else step.text + selected_value = params.get(step.param) if step.param is not None else step.text + selected = ( + runtime_param_text(selected_value) if selected_value is not None else None + ) if selected is None or step.selection_commit_key is None: return "selection delivery receipt lacks its compiled input contract" if ( @@ -287,7 +291,7 @@ def action_evidence_error( step: Step, result: Any, *, - params: Mapping[str, str] | None = None, + params: Mapping[str, RuntimeParamScalar] | None = None, identity_required: bool = False, strict_production: bool = True, ) -> Optional[str]: diff --git a/openadapt_flow/adapters/capture.py b/openadapt_flow/adapters/capture.py index 42b37022..0e42d91c 100644 --- a/openadapt_flow/adapters/capture.py +++ b/openadapt_flow/adapters/capture.py @@ -18,15 +18,13 @@ # key_char, canonical_key_name, ...) captured media # external-FFmpeg video or supported frame store -The adapter is a thin bridge over openadapt-capture's **public API** — it does -*no* raw SQL and knows nothing about capture's schema. It calls -``CaptureSession.load(dir)`` and iterates ``.actions(include_moves=False)``, -which runs capture's own event-processing pipeline (raw mouse/keyboard streams --> merged clicks / drags / typed text) and exposes each merged action as a -public ``Action`` (``.type``, ``.timestamp``, ``.x/.y/.dx/.dy``, -``.button/.text/.keys``). Frames come from ``CaptureSession.get_frame_at`` (the -same tested frame-extraction path ``Action.screenshot`` uses), so the adapter -inherits capture's decoding and survives capture's future schema changes. +The adapter reads events and frames through openadapt-capture's public API. It +does one immutable, read-only config lookup before loading the database. That +lookup distinguishes a current v2 window capture, which must have a completion +seal, from a legacy capture that can use Capture's migration-compatible loader. +It then iterates ``.actions(include_moves=False)``. Capture's processing +pipeline turns the raw mouse and keyboard stream into clicks, drags, and typed +text exposed through the public ``Action`` model. Action mapping (capture ``Action.type`` -> flow event ``kind``): @@ -107,14 +105,15 @@ A window-scoped session that declares a coordinate space this adapter does not understand is refused loudly rather than converted with guessed scaling. -Frame selection: ``CaptureSession.get_frame_at`` abstracts captured media -(default external-FFmpeg video and any supported frame store). For an event at -wall-clock time ``T`` the *before* frame is ``get_frame_at(T)`` and the *after* frame is -``get_frame_at(T + settle_s)`` clamped to just before the next event — an -approximation of the live Recorder's perceptual-hash settle wait (see -docs/desktop/PHASE1.md). A per-action frame may be unavailable; a missing -*before* frame for a click is fatal (the compiler requires it), while a missing -*after* frame simply yields no postconditions for that step. +Frame selection has two contracts. Every sealed v2 window capture names the +exact before frame by source ordinal and keeps the first retained frame after +the action as its after frame. This includes v2 RDP and Citrix captures, without +promoting their local client-window geometry into native identity. The adapter +decodes both exact ordinal bindings and refuses a missing frame. Legacy captures +keep the existing ``get_frame_at`` behavior: sample the action time and +``T + settle_s``, clamped to the next event. A missing click before frame is +fatal. A missing legacy after frame leaves that step without a visual +postcondition. Scroll deltas: pynput reports wheel *notches* with positive ``dy`` = scroll up, while the flow recording stores *pixels* with positive ``dy`` = view down @@ -137,17 +136,35 @@ observations. UIA describes the local remote-client canvas, not controls inside the remote session, so those surfaces remain external black-box workflows using pixels, OCR, relational anchors, identity regions, and fresh-frame verification. + +Native Windows, macOS, and Linux source bindings are separate. When the CLI +converts a sealed v2 window capture for one of those surfaces, each output event +gets a closed ``source_geometry`` object. It binds the exact source session, +terminal, artifact manifest, action/frame ordinals, PNG digest, process-bound +window identity, display topology, and geometry epoch. The compiler accepts it +only when the recording surface matches. Browser, RDP, and Citrix compilation +refuses this native object. """ from __future__ import annotations +import hashlib import json import math +import sqlite3 +import stat import uuid from datetime import datetime, timezone from pathlib import Path from typing import TYPE_CHECKING, Any, Callable, Optional, Sequence, TypeGuard +from openadapt_flow.source_geometry import ( + NATIVE_SOURCE_GEOMETRY_SCHEMA_VERSION, + NativeSourceGeometry, + native_source_geometry_sha256, +) +from openadapt_flow.source_terminal import SourceCaptureTerminal + if TYPE_CHECKING: # pragma: no cover from openadapt_capture.capture import Action, CaptureSession from PIL.Image import Image @@ -248,6 +265,175 @@ def _require_capture() -> "type[CaptureSession]": return CaptureSession +def _declares_v2_window_capture(capture_dir: Path) -> bool: + """Inspect only the immutable read-only DB surface before legacy loading.""" + db_path = capture_dir / "recording.db" + details = db_path.lstat() + if not stat.S_ISREG(details.st_mode): + raise ValueError("capture recording.db must be a regular file") + database = sqlite3.connect( + f"{db_path.resolve().as_uri()}?mode=ro&immutable=1", + uri=True, + ) + try: + row = database.execute("SELECT config FROM recording").fetchone() + finally: + database.close() + if row is None: + return False + raw_config = row[0] + if raw_config is None: + return False + try: + config = json.loads(raw_config) if isinstance(raw_config, str) else raw_config + except json.JSONDecodeError as exc: + raise ValueError("capture recording config is malformed") from exc + return ( + isinstance(config, dict) + and isinstance(config.get("capture_window"), dict) + and config["capture_window"].get("schema_version") + == "openadapt.capture.window-scoped/v2" + ) + + +def _load_capture_session(CaptureSession, capture_dir: Path): + """Use the sealed snapshot path for current captures, legacy path otherwise.""" + terminal_path = capture_dir / "capture-terminal.json" + manifest_path = capture_dir / "capture-artifact-manifest.json" + if terminal_path.exists() or manifest_path.exists(): + return CaptureSession.load_verified(capture_dir) + if _declares_v2_window_capture(capture_dir): + raise ValueError( + "v2 window capture has no immutable terminal and artifact manifest" + ) + return CaptureSession.load(capture_dir) + + +def _source_terminal(session: "CaptureSession") -> SourceCaptureTerminal: + raw = getattr(session, "terminal", None) + if raw is None: + raise ValueError("native source geometry requires a verified capture terminal") + if hasattr(raw, "model_dump"): + raw = raw.model_dump(mode="json") + return SourceCaptureTerminal.model_validate(raw) + + +def _native_source_geometry( + *, + action: "Action", + session: "CaptureSession", + terminal: SourceCaptureTerminal, + source_surface: str, + frame_sha256: str, +) -> NativeSourceGeometry: + """Build one closed native action/frame/geometry binding.""" + action_ordinal = getattr(action, "source_ordinal", None) + frame_ordinal = getattr(action, "screenshot_source_ordinal", None) + window_ordinal = getattr(action, "window_event_source_ordinal", None) + generation = getattr(action, "window_geometry_generation", None) + if ( + action_ordinal is None + or frame_ordinal is None + or window_ordinal is None + or generation is None + or frame_ordinal != window_ordinal + ): + raise ValueError("native action has an incomplete source geometry binding") + window_events = { + event.source_ordinal: event for event in session.window_capture_events_v2() + } + window_event = window_events.get(window_ordinal) + if window_event is None or window_event.window_capture_v2 is None: + raise ValueError("native action names a missing v2 window geometry event") + state = window_event.window_capture_v2 + if state.geometry_generation != generation: + raise ValueError("native action generation differs from its window geometry") + payload = { + "schema_version": NATIVE_SOURCE_GEOMETRY_SCHEMA_VERSION, + "source_surface": source_surface, + "source_capture_session_sha256": terminal.source_capture_session_sha256, + "source_capture_terminal_sha256": terminal.terminal_sha256, + "source_artifact_manifest_sha256": terminal.artifact_manifest_sha256, + "source_action_ordinal": action_ordinal, + "source_frame_ordinal": frame_ordinal, + "frame_sha256": frame_sha256, + "window_id": state.window_id, + "owner": state.owner, + "pid": state.pid, + "process_start_time": state.process_start_time, + "coordinate_source": state.coordinate_source, + "geometry_generation": state.geometry_generation, + "geometry_epoch_sha256": state.geometry_epoch_sha256, + "display_topology_sha256": state.display_topology_sha256, + "bounds": state.bounds, + "scale_x": state.scale_x, + "scale_y": state.scale_y, + "viewport": state.viewport, + "source_viewport": state.source_viewport, + "content_rect": state.content_rect, + "fit_scale": state.fit_scale, + } + payload["binding_sha256"] = native_source_geometry_sha256(payload) + return NativeSourceGeometry.model_validate(payload) + + +def _exact_native_frames( + session: "CaptureSession", + action: "Action", +) -> tuple["Image", "Image"]: + """Return the bound before frame and first ordinal-later retained frame.""" + action_ordinal = getattr(action, "source_ordinal", None) + frame_ordinal = getattr(action, "screenshot_source_ordinal", None) + frame_timestamp = getattr(action, "screenshot_timestamp", None) + if action_ordinal is None or frame_ordinal is None or frame_timestamp is None: + raise ValueError("native action has no exact source-frame binding") + + frames = list(session.frames()) + if not frames: + raise ValueError("native capture has no retained source frames") + ordinals = [frame.source_ordinal for frame in frames] + if any(ordinal is None for ordinal in ordinals): + raise ValueError("native capture has a frame without a source ordinal") + parsed_ordinals = [int(ordinal) for ordinal in ordinals if ordinal is not None] + if parsed_ordinals != sorted(parsed_ordinals) or len(parsed_ordinals) != len( + set(parsed_ordinals) + ): + raise ValueError("native capture frame ordinals are not unique and ordered") + + bound = next( + ( + frame + for frame in frames + if frame.source_ordinal == frame_ordinal + and frame.timestamp == frame_timestamp + ), + None, + ) + if bound is None: + raise ValueError("native action names no exact retained source frame") + after = next( + (frame for frame in frames if int(frame.source_ordinal) > action_ordinal), + None, + ) + if after is None: + raise ValueError("native action has no ordinal-later retained after frame") + try: + return ( + session.get_exact_frame( + bound.timestamp, + source_ordinal=int(bound.source_ordinal), + ), + session.get_exact_frame( + after.timestamp, + source_ordinal=int(after.source_ordinal), + ), + ) + except LookupError as exc: + raise ValueError( + "native action frame binding cannot be decoded exactly" + ) from exc + + def _window_capture_meta(session: "CaptureSession") -> Optional[dict[str, Any]]: """Window-scoping metadata for a window-scoped session, else None. @@ -859,7 +1045,7 @@ def _flow_events( events: list[dict[str, Any]] = [] # A run of typed characters, buffered so the compiler sees one ``type`` # event per typed value (capture emits one key.type per key-release burst). - text_run: dict[str, Any] = {"chars": [], "ts": None} + text_run: dict[str, Any] = {"chars": [], "ts": None, "source_action": None} def flush_text() -> None: if not text_run["chars"]: @@ -869,12 +1055,14 @@ def flush_text() -> None: "kind": "type", "text": text, "_ts": text_run["ts"], + "_source_action": text_run["source_action"], } if text in value_to_param: line["param"] = value_to_param[text] events.append(line) text_run["chars"] = [] text_run["ts"] = None + text_run["source_action"] = None for action in actions: atype = action.type @@ -897,6 +1085,7 @@ def flush_text() -> None: "x": int(round((action.x or 0.0) * scale)), "y": int(round((action.y or 0.0) * scale)), "_ts": ts, + "_source_action": action, } structural = ( _capture_structural_locator(action) if include_structural else None @@ -913,6 +1102,7 @@ def flush_text() -> None: "dx": int(round((action.dx or 0.0) * SCROLL_PIXELS_PER_NOTCH)), "dy": int(round(-(action.dy or 0.0) * SCROLL_PIXELS_PER_NOTCH)), "_ts": ts, + "_source_action": action, } ) elif atype == "mouse.drag": @@ -934,6 +1124,7 @@ def flush_text() -> None: "end_x": int(round((start_x + float(action.dx)) * scale)), "end_y": int(round((start_y + float(action.dy)) * scale)), "_ts": ts, + "_source_action": action, } structural = ( _capture_structural_locator(action) if include_structural else None @@ -969,6 +1160,7 @@ def flush_text() -> None: "modifiers": modifiers, "key": trigger, "_ts": ts, + "_source_action": action, } ) elif atype == "mouse.move": @@ -1019,6 +1211,7 @@ def _convert_key_type( if not text_run["chars"]: text_run["ts"] = ts text_run["chars"].append(action.text) + text_run["source_action"] = action return # Empty text: a named special key press (Enter/Tab/...) or a bare modifier. @@ -1034,7 +1227,14 @@ def _convert_key_type( if mapped is None: raise ValueError(f"unmapped key {name!r} at t={ts:.3f}; extend _KEY_NAME_MAP") flush_text() - events.append({"kind": "key", "key": mapped, "_ts": ts}) + events.append( + { + "kind": "key", + "key": mapped, + "_ts": ts, + "_source_action": action, + } + ) def _write_png(path: Path, image: "Image") -> None: @@ -1049,6 +1249,7 @@ def convert_capture( params: Optional[dict[str, str]] = None, settle_s: float = 1.0, include_structural: bool = False, + source_surface: Optional[str] = None, ) -> Path: """Convert an openadapt-capture session into a flow recording directory. @@ -1115,6 +1316,8 @@ def convert_capture( CaptureSession = _require_capture() capture_dir = Path(capture_dir) out_dir = Path(out_recording_dir) + if source_surface not in (None, "windows", "macos", "linux"): + raise ValueError("source_surface must be windows, macos, linux, or null") params = dict(params or {}) value_to_param: dict[str, str] = {} @@ -1126,7 +1329,7 @@ def convert_capture( ) value_to_param[value] = name - session = CaptureSession.load(capture_dir) + session = _load_capture_session(CaptureSession, capture_dir) try: window_capture = _window_capture_meta(session) desktop_capture = _desktop_capture_meta(session) @@ -1164,6 +1367,25 @@ def convert_capture( else: scale = float(session.pixel_ratio or 1.0) actions = list(session.actions(include_moves=False)) + source_terminal: Optional[SourceCaptureTerminal] = None + exact_v2_frame_selection = ( + window_capture is not None + and window_capture.get("schema_version") + == "openadapt.capture.window-scoped/v2" + ) + native_geometry_enabled = source_surface is not None and exact_v2_frame_selection + if native_geometry_enabled: + platform_surface = { + "win32": "windows", + "darwin": "macos", + "linux": "linux", + }.get(str(session.platform).lower()) + if platform_surface != source_surface: + raise ValueError( + "native source surface differs from the sealed Capture platform " + f"({source_surface!r} != {session.platform!r})" + ) + source_terminal = _source_terminal(session) scoped_viewport: Optional[tuple[int, int]] = None scope_label: Optional[str] = None if window_capture is not None: @@ -1203,11 +1425,17 @@ def convert_capture( for i, event in enumerate(events): ts = float(event["_ts"]) - before_img = session.get_frame_at(ts, tolerance=FRAME_TOLERANCE_S) - t_after = ts + settle_s - if i + 1 < len(events): - t_after = min(t_after, float(events[i + 1]["_ts"])) - after_img = session.get_frame_at(t_after, tolerance=FRAME_TOLERANCE_S) + source_action = event.get("_source_action") + if exact_v2_frame_selection: + if source_action is None: + raise ValueError(f"v2 capture event {i} has no source action binding") + before_img, after_img = _exact_native_frames(session, source_action) + else: + before_img = session.get_frame_at(ts, tolerance=FRAME_TOLERANCE_S) + t_after = ts + settle_s + if i + 1 < len(events): + t_after = min(t_after, float(events[i + 1]["_ts"])) + after_img = session.get_frame_at(t_after, tolerance=FRAME_TOLERANCE_S) if scoped_viewport is not None: assert scope_label is not None @@ -1229,7 +1457,8 @@ def convert_capture( "anchored" ) if before_img is not None: - _write_png(frames_dir / f"{i:04d}_before.png", before_img) + before_path = frames_dir / f"{i:04d}_before.png" + _write_png(before_path, before_img) if viewport is None: viewport = [before_img.width, before_img.height] if after_img is not None: @@ -1241,7 +1470,24 @@ def convert_capture( used_params[event["param"]] = event["text"] line: dict[str, Any] = {"i": i} - line.update({k: v for k, v in event.items() if k != "_ts"}) + line.update({k: v for k, v in event.items() if not k.startswith("_")}) + if native_geometry_enabled: + if before_img is None: + raise ValueError( + f"native event {i} has no exact bound source frame" + ) + assert source_terminal is not None + assert source_action is not None + assert source_surface is not None + before_png = (frames_dir / f"{i:04d}_before.png").read_bytes() + source_geometry = _native_source_geometry( + action=source_action, + session=session, + terminal=source_terminal, + source_surface=source_surface, + frame_sha256=hashlib.sha256(before_png).hexdigest(), + ) + line["source_geometry"] = source_geometry.model_dump(mode="json") line["t"] = round(ts - started_at, 3) lines.append(json.dumps(line)) diff --git a/openadapt_flow/backend.py b/openadapt_flow/backend.py index aa22e74c..08d1c4b2 100644 --- a/openadapt_flow/backend.py +++ b/openadapt_flow/backend.py @@ -9,6 +9,13 @@ from __future__ import annotations +import hashlib +import json +import math +import re +import struct +import unicodedata +from dataclasses import dataclass from typing import TYPE_CHECKING, Any, Optional, Protocol, runtime_checkable if TYPE_CHECKING: # pragma: no cover @@ -20,6 +27,667 @@ ) +_PNG_SIGNATURE = b"\x89PNG\r\n\x1a\n" + + +def _sha256_json(value: object) -> str: + payload = json.dumps( + value, + sort_keys=True, + separators=(",", ":"), + ensure_ascii=True, + ).encode("ascii") + return hashlib.sha256(payload).hexdigest() + + +def _png_viewport(png: bytes) -> tuple[int, int]: + """Return the exact PNG dimensions without decoding mutable image state.""" + + if len(png) < 24 or not png.startswith(_PNG_SIGNATURE): + raise ValueError("frame observation requires valid PNG bytes") + width, height = struct.unpack(">II", png[16:24]) + if width <= 0 or height <= 0: + raise ValueError("frame observation viewport must be positive") + return int(width), int(height) + + +def frame_observation_identity(value: object) -> str: + """Return one canonical privacy-safe identity digest for frame metadata.""" + + return _sha256_json(value) + + +def _identity_text(value: str, *, name: str) -> str: + normalized = " ".join(unicodedata.normalize("NFKC", value).split()).casefold() + if not normalized: + raise ValueError(f"{name} must be non-empty") + return normalized + + +def _domain_separated_identity_digest(domain: bytes, payload: object) -> str: + canonical = json.dumps( + payload, + sort_keys=True, + separators=(",", ":"), + ensure_ascii=False, + allow_nan=False, + ).encode("utf-8") + return hashlib.sha256(domain + canonical).hexdigest() + + +def window_identity_sha256( + *, + window_id: str, + pid: int, + process_start_time: Optional[str], + owner: str, +) -> str: + """Canonical exact-window digest shared by every frame adapter.""" + + if not isinstance(window_id, str) or not window_id.strip(): + raise ValueError("window_id must be a non-empty string") + if isinstance(pid, bool) or not isinstance(pid, int) or pid < 0: + raise ValueError("pid must be a non-negative integer") + if process_start_time is not None and ( + not isinstance(process_start_time, str) or not process_start_time.strip() + ): + raise ValueError("process_start_time must be a non-empty string or null") + if not isinstance(owner, str) or not owner: + raise ValueError("owner must be a non-empty string") + payload = { + "schema": "openadapt.window-identity.v1", + "window_id": window_id.strip(), + "pid": pid, + "process_start_time": ( + process_start_time.strip() if process_start_time is not None else None + ), + # Match the producer contract exactly. Do not normalize or collapse + # owner text: those transforms can alias distinct qualified values. + "owner": owner.casefold(), + } + return _domain_separated_identity_digest( + b"openadapt.window-identity.v1\x00", + payload, + ) + + +def session_identity_sha256( + *, + authority: str, + session_id: str, + session_start_time: Optional[str], + principal_identity_sha256: Optional[str], +) -> str: + """Canonical exact-session digest shared by every frame adapter.""" + + if not isinstance(session_id, str) or not session_id.strip(): + raise ValueError("session_id must be a non-empty string") + if session_start_time is not None and ( + not isinstance(session_start_time, str) or not session_start_time.strip() + ): + raise ValueError("session_start_time must be a non-empty string or null") + if principal_identity_sha256 is not None and not re.fullmatch( + r"[0-9a-f]{64}", principal_identity_sha256 + ): + raise ValueError("principal_identity_sha256 must be lower-hex SHA-256") + payload = { + "schema": "openadapt.session-identity.v1", + "authority": _identity_text(authority, name="authority"), + "session_id": session_id.strip(), + "session_start_time": ( + session_start_time.strip() if session_start_time is not None else None + ), + "principal_identity_sha256": principal_identity_sha256, + } + return _domain_separated_identity_digest( + b"openadapt.session-identity.v1\x00", + payload, + ) + + +@dataclass(frozen=True, slots=True) +class DisplayGeometry: + """One stable display in global physical or logical screen coordinates.""" + + display_id: str + bounds: tuple[float, float, float, float] + scale: tuple[float, float] + + def __post_init__(self) -> None: + if not isinstance(self.display_id, str) or not self.display_id: + raise ValueError("display_id must be non-empty") + if len(self.bounds) != 4 or not all( + math.isfinite(value) for value in self.bounds + ): + raise ValueError("display bounds must contain four finite values") + if self.bounds[2] <= 0 or self.bounds[3] <= 0: + raise ValueError("display width and height must be positive") + if len(self.scale) != 2 or not all( + math.isfinite(value) and value > 0 for value in self.scale + ): + raise ValueError("display scale must contain two positive values") + + +def display_topology_sha256( + displays: tuple[DisplayGeometry, ...], + *, + coordinate_space: str, +) -> str: + """Digest one canonical, order-independent complete display topology.""" + + if not displays: + raise ValueError("display topology requires at least one display") + if not isinstance(coordinate_space, str) or not coordinate_space: + raise ValueError("display coordinate_space must be non-empty") + ids = [display.display_id for display in displays] + if len(ids) != len(set(ids)): + raise ValueError("display topology contains duplicate display identities") + return frame_observation_identity( + { + "schema": "openadapt.display-topology.v1", + "coordinate_space": coordinate_space, + "displays": [ + { + "display_id": display.display_id, + "bounds": list(display.bounds), + "scale": list(display.scale), + } + for display in sorted(displays, key=lambda item: item.display_id) + ], + } + ) + + +def select_display_for_bounds( + displays: tuple[DisplayGeometry, ...], + bounds: tuple[float, float, float, float], +) -> DisplayGeometry: + """Select the display with the largest exact overlap with target bounds. + + Equal overlap is ambiguous and refuses. A zero-overlap target also refuses. + Negative global origins are valid. + """ + + if not displays: + raise ValueError("display selection requires at least one display") + x, y, width, height = bounds + if width <= 0 or height <= 0 or not all(math.isfinite(v) for v in bounds): + raise ValueError("target display-selection bounds must be finite and positive") + + def overlap(display: DisplayGeometry) -> float: + dx, dy, dw, dh = display.bounds + return max(0.0, min(x + width, dx + dw) - max(x, dx)) * max( + 0.0, min(y + height, dy + dh) - max(y, dy) + ) + + ranked = sorted( + ((overlap(display), display.display_id, display) for display in displays), + key=lambda item: (-item[0], item[1]), + ) + if ranked[0][0] <= 0: + raise ValueError("target bounds do not intersect a known display") + if len(ranked) > 1 and ranked[0][0] == ranked[1][0]: + raise ValueError("target bounds overlap multiple displays equally") + return ranked[0][2] + + +def frame_geometry_epoch( + *, + viewport: tuple[int, int], + viewport_width: int, + viewport_height: int, + origin: tuple[float, float], + scale: Optional[tuple[float, float]], + device_pixel_ratio: Optional[float], + display_id: str, + display_bounds: tuple[float, float, float, float], + display_scale: tuple[float, float], + topology_sha256: str, + window_identity_sha256: str, + session_identity_sha256: str, + page_identity_sha256: Optional[str] = None, + top_level_frame_identity_sha256: Optional[str] = None, +) -> str: + """Digest every fact that can change a frame-to-input coordinate mapping.""" + + return _sha256_json( + { + "schema": "openadapt.frame-geometry.v1", + "viewport": list(viewport), + "viewport_width": viewport_width, + "viewport_height": viewport_height, + "origin": list(origin), + "scale": list(scale) if scale is not None else None, + "device_pixel_ratio": device_pixel_ratio, + "display_id": display_id, + "display_bounds": list(display_bounds), + "display_scale": list(display_scale), + "topology_sha256": topology_sha256, + "window_identity_sha256": window_identity_sha256, + "session_identity_sha256": session_identity_sha256, + "page_identity_sha256": page_identity_sha256, + "top_level_frame_identity_sha256": top_level_frame_identity_sha256, + } + ) + + +def frame_coordinate_mapping_epoch( + *, + viewport: tuple[int, int], + viewport_width: int, + viewport_height: int, + origin: tuple[float, float], + scale: Optional[tuple[float, float]], + device_pixel_ratio: Optional[float], + display_id: str, + display_bounds: tuple[float, float, float, float], + display_scale: tuple[float, float], + topology_sha256: str, + window_identity_sha256: str, + session_identity_sha256: str, +) -> str: + """Digest the surface facts that map frame pixels to live input space. + + Page and document identities are intentionally absent. A successful action + can navigate or replace the top-level document without changing the window, + viewport, display, or pixel mapping. Actuation still binds the stricter + ``geometry_epoch``. Read-only frame-region checks use this mapping epoch so + an expected navigation does not look like a resize. + """ + + return _sha256_json( + { + "schema": "openadapt.frame-coordinate-mapping.v1", + "viewport": list(viewport), + "viewport_width": viewport_width, + "viewport_height": viewport_height, + "origin": list(origin), + "scale": list(scale) if scale is not None else None, + "device_pixel_ratio": device_pixel_ratio, + "display_id": display_id, + "display_bounds": list(display_bounds), + "display_scale": list(display_scale), + "topology_sha256": topology_sha256, + "window_identity_sha256": window_identity_sha256, + "session_identity_sha256": session_identity_sha256, + } + ) + + +@dataclass(frozen=True, slots=True) +class FrameObservation: + """One immutable frame and the exact geometry/context that produced it. + + A resolver must use ``viewport`` from this object. An input lease must bind + this object, not a PNG followed by a later backend property read. The three + identity digests contain no window title, account name, or captured text. + """ + + png: bytes + viewport: tuple[int, int] + viewport_width: int + viewport_height: int + origin: tuple[float, float] + scale: Optional[tuple[float, float]] + device_pixel_ratio: Optional[float] + display_id: str + display_bounds: tuple[float, float, float, float] + display_scale: tuple[float, float] + topology_sha256: str + window_identity_sha256: str + session_identity_sha256: str + page_identity_sha256: Optional[str] + top_level_frame_identity_sha256: Optional[str] + geometry_epoch: str + + def __post_init__(self) -> None: + png = bytes(self.png) + object.__setattr__(self, "png", png) + if _png_viewport(png) != self.viewport: + raise ValueError( + "frame observation viewport does not match its exact PNG bytes" + ) + if self.viewport_width <= 0 or self.viewport_height <= 0: + raise ValueError("frame observation top-level viewport must be positive") + if len(self.origin) != 2 or not all(math.isfinite(v) for v in self.origin): + raise ValueError("frame observation origin must contain two finite values") + if self.scale is not None and ( + len(self.scale) != 2 + or not all(math.isfinite(v) and v > 0 for v in self.scale) + ): + raise ValueError("frame observation scale must be positive when present") + if self.device_pixel_ratio is not None and ( + not math.isfinite(self.device_pixel_ratio) or self.device_pixel_ratio <= 0 + ): + raise ValueError( + "frame observation device_pixel_ratio must be positive when present" + ) + if not isinstance(self.display_id, str) or not self.display_id: + raise ValueError("frame observation display_id must be non-empty") + if len(self.display_bounds) != 4 or not all( + math.isfinite(value) for value in self.display_bounds + ): + raise ValueError( + "frame observation display_bounds must contain four finite values" + ) + if self.display_bounds[2] <= 0 or self.display_bounds[3] <= 0: + raise ValueError("frame observation display bounds must be positive") + if len(self.display_scale) != 2 or not all( + math.isfinite(value) and value > 0 for value in self.display_scale + ): + raise ValueError("frame observation display_scale must be positive") + for name in ( + "topology_sha256", + "window_identity_sha256", + "session_identity_sha256", + "geometry_epoch", + ): + value = getattr(self, name) + if len(value) != 64 or any(ch not in "0123456789abcdef" for ch in value): + raise ValueError(f"frame observation {name} must be lower-hex SHA-256") + if (self.page_identity_sha256 is None) != ( + self.top_level_frame_identity_sha256 is None + ): + raise ValueError( + "browser page and top-level frame identities must be supplied together" + ) + for name in ("page_identity_sha256", "top_level_frame_identity_sha256"): + value = getattr(self, name) + if value is not None and not re.fullmatch(r"[0-9a-f]{64}", value): + raise ValueError(f"frame observation {name} must be lower-hex SHA-256") + expected_epoch = frame_geometry_epoch( + viewport=self.viewport, + viewport_width=self.viewport_width, + viewport_height=self.viewport_height, + origin=self.origin, + scale=self.scale, + device_pixel_ratio=self.device_pixel_ratio, + display_id=self.display_id, + display_bounds=self.display_bounds, + display_scale=self.display_scale, + topology_sha256=self.topology_sha256, + window_identity_sha256=self.window_identity_sha256, + session_identity_sha256=self.session_identity_sha256, + page_identity_sha256=self.page_identity_sha256, + top_level_frame_identity_sha256=self.top_level_frame_identity_sha256, + ) + if self.geometry_epoch != expected_epoch: + raise ValueError( + "frame observation geometry_epoch does not bind its geometry" + ) + + @property + def frame_sha256(self) -> str: + """Digest of the exact encoded PNG bytes used by this observation.""" + + return hashlib.sha256(self.png).hexdigest() + + @property + def coordinate_mapping_epoch(self) -> str: + """Digest the coordinate mapping without document-lifecycle identity.""" + + return frame_coordinate_mapping_epoch( + viewport=self.viewport, + viewport_width=self.viewport_width, + viewport_height=self.viewport_height, + origin=self.origin, + scale=self.scale, + device_pixel_ratio=self.device_pixel_ratio, + display_id=self.display_id, + display_bounds=self.display_bounds, + display_scale=self.display_scale, + topology_sha256=self.topology_sha256, + window_identity_sha256=self.window_identity_sha256, + session_identity_sha256=self.session_identity_sha256, + ) + + @classmethod + def create( + cls, + png: bytes, + *, + viewport_width: Optional[int] = None, + viewport_height: Optional[int] = None, + origin: tuple[float, float], + scale: Optional[tuple[float, float]], + device_pixel_ratio: Optional[float], + display_id: str, + display_bounds: tuple[float, float, float, float], + display_scale: tuple[float, float], + topology_sha256: str, + window_identity_sha256: str, + session_identity_sha256: str, + page_identity_sha256: Optional[str] = None, + top_level_frame_identity_sha256: Optional[str] = None, + ) -> "FrameObservation": + viewport = _png_viewport(png) + normalized_viewport_width = ( + viewport[0] if viewport_width is None else int(viewport_width) + ) + normalized_viewport_height = ( + viewport[1] if viewport_height is None else int(viewport_height) + ) + normalized_origin = (float(origin[0]), float(origin[1])) + normalized_scale = ( + (float(scale[0]), float(scale[1])) if scale is not None else None + ) + normalized_dpr = ( + float(device_pixel_ratio) if device_pixel_ratio is not None else None + ) + normalized_display_bounds = ( + float(display_bounds[0]), + float(display_bounds[1]), + float(display_bounds[2]), + float(display_bounds[3]), + ) + normalized_display_scale = ( + float(display_scale[0]), + float(display_scale[1]), + ) + return cls( + png=png, + viewport=viewport, + viewport_width=normalized_viewport_width, + viewport_height=normalized_viewport_height, + origin=normalized_origin, + scale=normalized_scale, + device_pixel_ratio=normalized_dpr, + display_id=display_id, + display_bounds=normalized_display_bounds, + display_scale=normalized_display_scale, + topology_sha256=topology_sha256, + window_identity_sha256=window_identity_sha256, + session_identity_sha256=session_identity_sha256, + page_identity_sha256=page_identity_sha256, + top_level_frame_identity_sha256=top_level_frame_identity_sha256, + geometry_epoch=frame_geometry_epoch( + viewport=viewport, + viewport_width=normalized_viewport_width, + viewport_height=normalized_viewport_height, + origin=normalized_origin, + scale=normalized_scale, + device_pixel_ratio=normalized_dpr, + display_id=display_id, + display_bounds=normalized_display_bounds, + display_scale=normalized_display_scale, + topology_sha256=topology_sha256, + window_identity_sha256=window_identity_sha256, + session_identity_sha256=session_identity_sha256, + page_identity_sha256=page_identity_sha256, + top_level_frame_identity_sha256=top_level_frame_identity_sha256, + ), + ) + + +FrameRegion = tuple[int, int, int, int] +NormalizedRegion = tuple[float, float, float, float] + + +def _validate_frame_region( + region: FrameRegion, + *, + viewport: tuple[int, int], + name: str, +) -> FrameRegion: + x, y, width, height = (int(value) for value in region) + if x < 0 or y < 0 or width <= 0 or height <= 0: + raise ValueError(f"{name} must be a positive in-frame region") + if x + width > viewport[0] or y + height > viewport[1]: + raise ValueError(f"{name} exceeds the source frame viewport") + return x, y, width, height + + +def _validate_identity_sha256(value: str, *, name: str) -> str: + if len(value) != 64 or any(ch not in "0123456789abcdef" for ch in value): + raise ValueError(f"{name} must be lower-hex SHA-256") + return value + + +@dataclass(frozen=True, slots=True) +class TargetRelativeRegionBinding: + """One evidence region relative to an exact resolved target or anchor.""" + + name: str + source_region: FrameRegion + anchor_identity_sha256: str + source_anchor_region: FrameRegion + normalized_offset: NormalizedRegion + + +@dataclass(frozen=True, slots=True) +class FrameTargetBinding: + """Exact-frame target geometry; viewport normalization is display-only. + + Runtime identity, effect, and REGION_STABLE checks may reuse a dependent + region after a geometry epoch change only by resolving its named anchor + identity again and applying ``normalized_offset`` to that fresh anchor. + ``presentation_normalized_region`` is never runtime evidence. + """ + + frame_sha256: str + geometry_epoch: str + source_viewport: tuple[int, int] + source_target_region: FrameRegion + target_identity_sha256: str + target_relative_regions: tuple[TargetRelativeRegionBinding, ...] + presentation_normalized_region: NormalizedRegion + + +def bind_target_region( + observation: FrameObservation, + target_region: FrameRegion, + *, + target_identity_sha256: str, + dependent_regions: tuple[tuple[str, FrameRegion], ...] = (), +) -> FrameTargetBinding: + """Bind target-relative evidence to one exact frame observation. + + This helper deliberately does not accept a viewport-relative evidence + region. A caller with a non-target region must name the exact anchor that + owns it and use :func:`bind_anchor_relative_region`. + """ + + target = _validate_frame_region( + target_region, + viewport=observation.viewport, + name="target_region", + ) + identity = _validate_identity_sha256( + target_identity_sha256, + name="target_identity_sha256", + ) + tx, ty, tw, th = target + relatives = tuple( + bind_anchor_relative_region( + observation, + name=name, + region=region, + anchor_identity_sha256=identity, + anchor_region=target, + ) + for name, region in dependent_regions + ) + vw, vh = observation.viewport + return FrameTargetBinding( + frame_sha256=observation.frame_sha256, + geometry_epoch=observation.geometry_epoch, + source_viewport=observation.viewport, + source_target_region=target, + target_identity_sha256=identity, + target_relative_regions=relatives, + presentation_normalized_region=(tx / vw, ty / vh, tw / vw, th / vh), + ) + + +def bind_anchor_relative_region( + observation: FrameObservation, + *, + name: str, + region: FrameRegion, + anchor_identity_sha256: str, + anchor_region: FrameRegion, +) -> TargetRelativeRegionBinding: + """Bind a named evidence region to one exact independently resolved anchor.""" + + if not name.strip(): + raise ValueError("target-relative region requires a non-empty name") + source = _validate_frame_region( + region, + viewport=observation.viewport, + name=f"dependent region {name!r}", + ) + anchor = _validate_frame_region( + anchor_region, + viewport=observation.viewport, + name=f"anchor region for {name!r}", + ) + identity = _validate_identity_sha256( + anchor_identity_sha256, + name=f"anchor identity for {name!r}", + ) + x, y, width, height = source + ax, ay, aw, ah = anchor + return TargetRelativeRegionBinding( + name=name.strip(), + source_region=source, + anchor_identity_sha256=identity, + source_anchor_region=anchor, + normalized_offset=( + (x - ax) / aw, + (y - ay) / ah, + width / aw, + height / ah, + ), + ) + + +@runtime_checkable +class FrameObservationBackend(Protocol): + """Backend that proves frame bytes and geometry in one read operation.""" + + def observe_frame(self) -> FrameObservation: + """Return one immutable frame observation.""" + ... + + +@runtime_checkable +class ActuationObservationBackend(Protocol): + """Backend that arms input against one exact frame observation.""" + + def acquire_actuation_observation(self) -> FrameObservation: + """Acquire and arm one fresh observation for the next input edge.""" + ... + + +@runtime_checkable +class FrameObservationLeaseBackend(Protocol): + """Input backend that consumes the descriptor resolved by the runtime.""" + + def bind_input_observation(self, observation: FrameObservation) -> None: + """Bind the next input lease to ``observation`` or refuse it.""" + ... + + class StructuralResolutionRefused(RuntimeError): """A structural backend found candidates but could not prove uniqueness. @@ -79,6 +747,10 @@ def __init__( changed_pixel_count: int, changed_bbox: Optional[tuple[int, int, int, int]], frame_size: tuple[int, int], + expected_geometry_epoch: Optional[str] = None, + observed_geometry_epoch: Optional[str] = None, + expected_observation: Optional[FrameObservation] = None, + observed_observation: Optional[FrameObservation] = None, ) -> None: super().__init__( "surface changed before input because frame content changed; " @@ -95,10 +767,61 @@ def __init__( raise ValueError("fresh-actuation bounding box must be positive") if x + width > frame_size[0] or y + height > frame_size[1]: raise ValueError("fresh-actuation bounding box exceeds the frame") + if (expected_geometry_epoch is None) != (observed_geometry_epoch is None): + raise ValueError( + "fresh-actuation geometry epochs must be supplied together" + ) + if (expected_observation is None) != (observed_observation is None): + raise ValueError("fresh-actuation observations must be supplied together") + if expected_observation is not None and observed_observation is not None: + if expected_geometry_epoch is None: + expected_geometry_epoch = expected_observation.geometry_epoch + observed_geometry_epoch = observed_observation.geometry_epoch + elif ( + expected_geometry_epoch != expected_observation.geometry_epoch + or observed_geometry_epoch != observed_observation.geometry_epoch + ): + raise ValueError( + "fresh-actuation observations do not match supplied epochs" + ) + for value in (expected_geometry_epoch, observed_geometry_epoch): + if value is not None and ( + len(value) != 64 or any(ch not in "0123456789abcdef" for ch in value) + ): + raise ValueError("fresh-actuation geometry epoch must be SHA-256") self.operation = operation self.changed_pixel_count = changed_pixel_count self.changed_bbox = changed_bbox self.frame_size = frame_size + self.expected_geometry_epoch = expected_geometry_epoch + self.observed_geometry_epoch = observed_geometry_epoch + self.expected_observation = expected_observation + self.observed_observation = observed_observation + + +class DisplayTopologyChanged(RuntimeError): + """The complete display topology changed outside an admitted transition. + + This is not a retryable stale-frame mismatch. Hot-plug, display removal, + arrangement changes, and DPI changes can change the meaning or availability + of every coordinate. The runtime must start a separately qualified session + transition before it can continue. + """ + + def __init__( + self, + *, + expected_observation: FrameObservation, + observed_observation: FrameObservation, + ) -> None: + if expected_observation.topology_sha256 == observed_observation.topology_sha256: + raise ValueError("display-topology exception requires a topology change") + super().__init__( + "display topology changed outside an admitted topology-transition " + "contract; invalidate this execution session" + ) + self.expected_observation = expected_observation + self.observed_observation = observed_observation @runtime_checkable diff --git a/openadapt_flow/backends/linux_backend.py b/openadapt_flow/backends/linux_backend.py index 7287d5c5..2ce6e86a 100644 --- a/openadapt_flow/backends/linux_backend.py +++ b/openadapt_flow/backends/linux_backend.py @@ -32,15 +32,27 @@ import re import secrets import sys +import threading import unicodedata from dataclasses import dataclass, field from datetime import datetime, timezone from pathlib import Path from typing import Any, Callable, Optional, Protocol, runtime_checkable -from PIL import Image, ImageGrab - -from openadapt_flow.backend import StructuralResolutionRefused +from PIL import Image, ImageChops, ImageGrab + +from openadapt_flow.backend import ( + DisplayGeometry, + DisplayTopologyChanged, + FrameObservation, + FreshActuationRequired, + StructuralResolutionRefused, + display_topology_sha256, + frame_observation_identity, + select_display_for_bounds, + session_identity_sha256, + window_identity_sha256, +) from openadapt_flow.ir import ( ActionDeliveryReceipt, StructuralHandle, @@ -107,6 +119,10 @@ def focus_window(self, window: LinuxWindow) -> bool: ... def capture_window(self, window: LinuxWindow) -> tuple[bytes, int, int]: ... + def display_topology(self) -> tuple[DisplayGeometry, ...]: + """Return every stable display in AT-SPI screen coordinates.""" + ... + def element_at_point( self, window: LinuxWindow, x: int, y: int ) -> Optional[LinuxElement]: ... @@ -208,6 +224,24 @@ def _linux_session_facts() -> Optional[tuple[str, int, int]]: return None +def _linux_process_start_time(pid: int) -> Optional[str]: + """Return the kernel process-start tick for exact PID reuse detection.""" + + if not sys.platform.startswith("linux") or pid <= 0: + return None + try: + # /proc//stat field 2 can contain spaces and parentheses. Split + # only after the final ')' so field 22 remains unambiguous. + stat = Path(f"/proc/{pid}/stat").read_text(encoding="ascii").strip() + suffix = stat.rsplit(")", 1)[1].strip().split() + start_ticks = int(suffix[19]) + if start_ticks < 0: + return None + return f"linux-proc-start-ticks:{start_ticks}" + except (OSError, UnicodeError, ValueError, IndexError): + return None + + def _clean_text(value: object) -> Optional[str]: text = " ".join(str(value or "").split()) return text or None @@ -285,6 +319,11 @@ def __init__( ] ] = None self._guarded_keyboard: Optional[tuple[LinuxWindow, str, Optional[str]]] = None + self._last_frame_observation: Optional[FrameObservation] = None + self._bound_input_observation: Optional[FrameObservation] = None + self._fresh_actuation_invalidated = False + self._backend_session_identity = secrets.token_hex(16) + self._input_lock = threading.RLock() self._assert_session_supported() def _assert_session_supported(self) -> None: @@ -352,6 +391,23 @@ def viewport(self) -> tuple[int, int]: return self._viewport def screenshot(self) -> bytes: + """Compatibility byte view of :meth:`observe_frame`.""" + + return self.observe_frame().png + + def observe_frame(self) -> FrameObservation: + """Capture one exact Linux window and its atomic coordinate context.""" + + with self._input_lock: + return self._observe_frame_locked() + + @property + def last_frame_observation(self) -> Optional[FrameObservation]: + """The descriptor created by the most recent completed frame capture.""" + + return self._last_frame_observation + + def _observe_frame_locked(self) -> FrameObservation: window = self._resolve_window() if ( self._require_active_window_for_capture @@ -387,7 +443,71 @@ def screenshot(self) -> bytes: ) self._captured_window = current self._viewport = (width, height) - return png + session_facts = _linux_session_facts() + if session_facts is None: + session = session_identity_sha256( + authority=f"linux:{self._client.session_type}", + session_id=self._backend_session_identity, + session_start_time=None, + principal_identity_sha256=None, + ) + else: + boot_id, audit_session_id, uid = session_facts + session = session_identity_sha256( + authority="linux-kernel-audit", + session_id=str(audit_session_id), + session_start_time=f"linux-boot-id:{boot_id}", + principal_identity_sha256=frame_observation_identity( + {"schema": "openadapt.linux-uid.v1", "uid": uid} + ), + ) + display_probe = getattr(self._client, "display_topology", None) + displays = ( + tuple(display_probe()) + if callable(display_probe) + else ( + DisplayGeometry( + display_id=f"test-window-display:{current.native_id}", + bounds=( + float(current.bounds[0]), + float(current.bounds[1]), + float(current.bounds[2]), + float(current.bounds[3]), + ), + scale=(1.0, 1.0), + ), + ) + ) + try: + display = select_display_for_bounds(displays, current.bounds) + topology = display_topology_sha256( + displays, + coordinate_space=f"linux-{self._client.session_type}-screen", + ) + except ValueError as exc: + raise LinuxBackendError( + "Linux display topology is unavailable or ambiguous" + ) from exc + window_identity = window_identity_sha256( + window_id=current.native_id, + pid=current.pid, + process_start_time=_linux_process_start_time(current.pid), + owner=current.app_name, + ) + observation = FrameObservation.create( + png, + origin=(float(current.bounds[0]), float(current.bounds[1])), + scale=(float(width / current.bounds[2]), float(height / current.bounds[3])), + device_pixel_ratio=float(width / current.bounds[2]), + display_id=display.display_id, + display_bounds=display.bounds, + display_scale=display.scale, + topology_sha256=topology, + window_identity_sha256=window_identity, + session_identity_sha256=session, + ) + self._last_frame_observation = observation + return observation # -- live execution-context identity ----------------------------------- @@ -671,27 +791,121 @@ def arm_guarded_coordinate(self, x: int, y: int) -> None: fingerprint, self.session_identity(), ) + self._bound_input_observation = self._last_frame_observation def cancel_guarded_coordinate(self) -> None: self._guarded_coordinate = None + def bind_input_observation(self, observation: FrameObservation) -> None: + """Bind the next guarded Linux input to one exact frame descriptor.""" + + with self._input_lock: + current = self._last_frame_observation + if self._fresh_actuation_invalidated: + raise LinuxBackendError( + "Linux frame lease is invalidated and requires fresh resolution" + ) + if ( + current is None + or current.frame_sha256 != observation.frame_sha256 + or current.geometry_epoch != observation.geometry_epoch + ): + raise StructuralResolutionRefused( + "Linux input observation does not match the resolved frame" + ) + self._bound_input_observation = observation + + def reset_fresh_actuation_state(self) -> None: + """Reset one proved zero-input invalidation without granting input.""" + + with self._input_lock: + if not self._fresh_actuation_invalidated: + raise LinuxBackendError( + "Linux fresh-actuation reset requires an invalidated lease" + ) + self._fresh_actuation_invalidated = False + self._bound_input_observation = None + + @staticmethod + def _frame_difference( + expected_png: bytes, + observed_png: bytes, + ) -> tuple[int, tuple[int, int, int, int]]: + expected = Image.open(io.BytesIO(expected_png)).convert("RGB") + observed = Image.open(io.BytesIO(observed_png)).convert("RGB") + if expected.size != observed.size: + width, height = observed.size + return width * height, (0, 0, width, height) + difference = ImageChops.difference(expected, observed) + raw_bbox = difference.getbbox() + if raw_bbox is None: + width, height = observed.size + return width * height, (0, 0, width, height) + x1, y1, x2, y2 = raw_bbox + channels = difference.split() + mask = channels[0] + for channel in channels[1:]: + mask = ImageChops.lighter(mask, channel) + changed_pixel_count = sum(mask.histogram()[1:]) + return changed_pixel_count, (x1, y1, x2 - x1, y2 - y1) + def _assert_guarded_frame( self, window: LinuxWindow, expected_frame_sha256: str, expected_session: Optional[str], ) -> None: - current = self._require_same_window(window, require_active=True) - png, width, height = self._client.capture_window(current) - if (width, height) != current.bounds[2:]: - raise LinuxBackendError( - "Linux target window dimensions changed before guarded input" - ) - if not hmac.compare_digest( - hashlib.sha256(png).hexdigest(), expected_frame_sha256 + bound = self._bound_input_observation + if bound is None or not hmac.compare_digest( + bound.frame_sha256, expected_frame_sha256 ): + raise StructuralResolutionRefused( + "Linux guarded input lacks its exact atomic frame descriptor" + ) + current = self._resolve_window() + if current.native_id != window.native_id or current.pid != window.pid: + raise StructuralResolutionRefused( + "Linux target window identity changed before guarded input" + ) + if not self._client.window_is_active(current): raise LinuxBackendError( - "Linux target frame changed after identity verification" + "the exact Linux target window is not active; refusing input" + ) + observation = self._observe_frame_locked() + if observation.topology_sha256 != bound.topology_sha256: + self._fresh_actuation_invalidated = True + self._bound_input_observation = None + raise DisplayTopologyChanged( + expected_observation=bound, + observed_observation=observation, + ) + if observation.geometry_epoch != bound.geometry_epoch: + self._fresh_actuation_invalidated = True + self._bound_input_observation = None + raise FreshActuationRequired( + operation="linux_guarded_input", + changed_pixel_count=(observation.viewport[0] * observation.viewport[1]), + changed_bbox=(0, 0, *observation.viewport), + frame_size=observation.viewport, + expected_geometry_epoch=bound.geometry_epoch, + observed_geometry_epoch=observation.geometry_epoch, + expected_observation=bound, + observed_observation=observation, + ) + if not hmac.compare_digest(observation.frame_sha256, expected_frame_sha256): + changed_pixel_count, changed_bbox = self._frame_difference( + bound.png, + observation.png, + ) + self._fresh_actuation_invalidated = True + self._bound_input_observation = None + raise FreshActuationRequired( + operation="linux_guarded_input", + changed_pixel_count=changed_pixel_count, + changed_bbox=changed_bbox, + frame_size=observation.viewport, + expected_observation=bound, + observed_observation=observation, ) if expected_session is not None and not hmac.compare_digest( self.session_identity() or "", expected_session @@ -699,6 +913,7 @@ def _assert_guarded_frame( raise LinuxBackendError( "Linux desktop session changed after identity verification" ) + self._bound_input_observation = None def act_guarded_coordinate( self, @@ -708,6 +923,24 @@ def act_guarded_coordinate( expected_frame_sha256: str, double: bool = False, button: str = "left", + ) -> ActionDeliveryReceipt: + with self._input_lock: + return self._act_guarded_coordinate_locked( + x, + y, + expected_frame_sha256=expected_frame_sha256, + double=double, + button=button, + ) + + def _act_guarded_coordinate_locked( + self, + x: int, + y: int, + *, + expected_frame_sha256: str, + double: bool, + button: str, ) -> ActionDeliveryReceipt: pending = self._guarded_coordinate self._guarded_coordinate = None @@ -755,7 +988,10 @@ def act_guarded_coordinate( ) def guarded_keyboard_frame(self) -> bytes: - return self.screenshot() + return self.guarded_keyboard_observation().png + + def guarded_keyboard_observation(self) -> FrameObservation: + return self.observe_frame() def arm_guarded_keyboard(self, x: int, y: int) -> None: """Bind the unique live focused AT-SPI element at the resolved point.""" @@ -777,6 +1013,7 @@ def arm_guarded_keyboard(self, x: int, y: int) -> None: _fingerprint(focused), self.session_identity(), ) + self._bound_input_observation = self._last_frame_observation def cancel_guarded_keyboard(self) -> None: self._guarded_keyboard = None @@ -811,15 +1048,16 @@ def type_text_guarded( *, expected_frame_sha256: str, ) -> ActionDeliveryReceipt: - focused, fingerprint = self._consume_guarded_keyboard(expected_frame_sha256) - self._require_qualification_input_guard() - if text and not self._client.replace_text(focused, text): - raise LinuxBackendError("Linux guarded text delivery was rejected") - return _receipt( - "guarded_atspi_type", - native=True, - target_fingerprint=fingerprint, - ) + with self._input_lock: + focused, fingerprint = self._consume_guarded_keyboard(expected_frame_sha256) + self._require_qualification_input_guard() + if text and not self._client.replace_text(focused, text): + raise LinuxBackendError("Linux guarded text delivery was rejected") + return _receipt( + "guarded_atspi_type", + native=True, + target_fingerprint=fingerprint, + ) def press_guarded( self, @@ -827,21 +1065,24 @@ def press_guarded( *, expected_frame_sha256: str, ) -> ActionDeliveryReceipt: - _, fingerprint = self._consume_guarded_keyboard(expected_frame_sha256) - if not self._allow_physical_input: - raise LinuxBackendError( - "guarded Linux KEY delivery requires the explicitly qualified " - "physical-input fallback" + with self._input_lock: + _, fingerprint = self._consume_guarded_keyboard(expected_frame_sha256) + if not self._allow_physical_input: + raise LinuxBackendError( + "guarded Linux KEY delivery requires the explicitly qualified " + "physical-input fallback" + ) + self._require_qualification_input_guard() + if not self._client.physical_press(key): + raise LinuxBackendError( + f"Linux guarded key delivery was rejected: {key!r}" + ) + self._clear_focused_element() + return _receipt( + "guarded_atspi_key", + native=False, + target_fingerprint=fingerprint, ) - self._require_qualification_input_guard() - if not self._client.physical_press(key): - raise LinuxBackendError(f"Linux guarded key delivery was rejected: {key!r}") - self._clear_focused_element() - return _receipt( - "guarded_atspi_key", - native=False, - target_fingerprint=fingerprint, - ) def _physical_element_click( self, candidate: LinuxElement, expected: str, *, double: bool @@ -1026,6 +1267,69 @@ def portal_session_ready(self) -> bool: # not a boolean environment toggle. This client has no such transport. return False + def display_topology(self) -> tuple[DisplayGeometry, ...]: + """Return stable GDK monitor identities in AT-SPI screen coordinates.""" + + try: + import gi + + gi.require_version("Gdk", "3.0") + from gi.repository import Gdk + + display = Gdk.Display.get_default() + if display is None: + raise LinuxBackendError("GDK returned no interactive display") + count = int(display.get_n_monitors()) + monitors: list[DisplayGeometry] = [] + for index in range(count): + monitor = display.get_monitor(index) + if monitor is None: + continue + geometry = monitor.get_geometry() + connector_probe = getattr(monitor, "get_connector", None) + connector = ( + str(connector_probe() or "") if callable(connector_probe) else "" + ) + identity_material = { + "schema": "openadapt.linux-display-identity.v1", + "connector": connector or None, + "manufacturer": str(monitor.get_manufacturer() or ""), + "model": str(monitor.get_model() or ""), + "width_mm": int(monitor.get_width_mm()), + "height_mm": int(monitor.get_height_mm()), + } + monitors.append( + DisplayGeometry( + display_id=( + f"gdk-connector:{connector}" + if connector + else "gdk-display:" + + frame_observation_identity(identity_material) + ), + bounds=( + float(geometry.x), + float(geometry.y), + float(geometry.width), + float(geometry.height), + ), + scale=( + float(monitor.get_scale_factor()), + float(monitor.get_scale_factor()), + ), + ) + ) + # This also rejects indistinguishable duplicate monitors. Such a + # topology cannot preserve stable display identity across reorders. + display_topology_sha256( + tuple(monitors), + coordinate_space=f"linux-{self._session_type}-screen", + ) + return tuple(monitors) + except LinuxBackendError: + raise + except Exception as exc: + raise LinuxBackendError("GDK display topology is unavailable") from exc + @staticmethod def _call(obj: Any, *names: str, default: Any = None) -> Any: for name in names: diff --git a/openadapt_flow/backends/playwright_backend.py b/openadapt_flow/backends/playwright_backend.py index 478cc1fd..32187c1c 100644 --- a/openadapt_flow/backends/playwright_backend.py +++ b/openadapt_flow/backends/playwright_backend.py @@ -1,15 +1,17 @@ """Playwright-driven reference backend (sync API, chromium, headless-capable). Implements the `openadapt_flow.backend.Backend` protocol against a Playwright -`Page`: full-viewport PNG screenshots, mouse clicks at pixel coordinates, -keyboard typing, and key/chord presses. Viewport is fixed at 1280x800 with -deviceScaleFactor=1 so CSS pixels equal screenshot pixels. +`Page`: atomic full-viewport PNG observations, DOM-guarded pointer and keyboard +input, live viewport/DPR transitions, and exact page/frame identity. Production +observation uses CSS-scale screenshots, so resolver pixels and browser input +coordinates stay in one space after a resize or monitor-scale change. """ from __future__ import annotations import hashlib import hmac +import io import math import re import uuid @@ -18,10 +20,21 @@ from typing import TYPE_CHECKING, Any, Callable, Literal, Optional from urllib.parse import urlsplit +from PIL import Image + if TYPE_CHECKING: # pragma: no cover from playwright.sync_api import Page -from openadapt_flow.backend import ActionDeliveryUncertain, StructuralResolutionRefused +from openadapt_flow.backend import ( + ActionDeliveryUncertain, + DisplayTopologyChanged, + FrameObservation, + FreshActuationRequired, + StructuralResolutionRefused, + frame_observation_identity, + session_identity_sha256, + window_identity_sha256, +) from openadapt_flow.ir import ( ActionDeliveryReceipt, StructuralHandle, @@ -34,6 +47,12 @@ VIEWPORT: tuple[int, int] = (1280, 800) _MASKED_SCREENSHOT_ATTEMPTS = 3 +_ATOMIC_OBSERVATION_ATTEMPTS = 3 + + +class BrowserObservationStabilityError(RuntimeError): + """The browser could not produce one stable atomic frame observation.""" + _MODIFIER_ALIASES = { "meta": "Meta", @@ -141,6 +160,23 @@ class _FramePoint: frame_path: tuple[str, ...] +@dataclass(frozen=True) +class _BrowserGeometry: + """One privacy-safe top-level browser geometry sample.""" + + viewport_width: int + viewport_height: int + device_pixel_ratio: float + display_id: str + display_bounds: tuple[float, float, float, float] + display_scale: tuple[float, float] + topology_sha256: str + page_identity_sha256: str + top_level_frame_identity_sha256: str + window_identity_sha256: str + session_identity_sha256: str + + # The descriptor stays inside the page-local guard store. It binds the exact # actionable node, its ancestry, and the enclosing record row while excluding # the target's own cell (the same identity boundary as ``structured_text_at``). @@ -746,12 +782,33 @@ def __init__( self._structural_state_reader = structural_state_reader self._screenshot_guard = screenshot_guard self._screenshot_frame_generation = 0 + self._top_level_frame_generation = 0 self._screenshot_frame_listener = self._handle_screenshot_frame_lifecycle + self._top_level_navigation_listener = self._handle_top_level_navigation self._screenshot_frame_tracking = False - if self._screenshot_mask_selectors: + event_listener = getattr(self.page, "on", None) + if callable(event_listener): for event in ("frameattached", "framedetached", "framenavigated"): - self.page.on(event, self._screenshot_frame_listener) + event_listener(event, self._screenshot_frame_listener) + event_listener("framenavigated", self._top_level_navigation_listener) self._screenshot_frame_tracking = True + identity_nonce = uuid.uuid4().hex + self._page_identity_sha256 = frame_observation_identity( + { + "schema": "openadapt.playwright-page-identity.v1", + "backend_nonce": identity_nonce, + "page_object": id(self.page), + } + ) + self._context_identity_sha256 = frame_observation_identity( + { + "schema": "openadapt.playwright-context-identity.v1", + "backend_nonce": identity_nonce, + "context_object": id(getattr(self.page, "context", self.page)), + } + ) + self._last_frame_observation: Optional[FrameObservation] = None + self._bound_input_observation: Optional[FrameObservation] = None # Opaque per-backend key keeps the WeakMap private from ordinary page # code. Python retains only token material keyed by the public # SHA-256 fingerprint; target/row text stays page-local and ephemeral. @@ -1730,6 +1787,9 @@ def act_structural( # qualification observer can cover context that the target page # cannot declare (for example a browser inside a managed remote # session), so the DOM guard alone is not sufficient. + self._consume_input_observation( + operation="dom_double_click" if double else "dom_click" + ) self._assert_qualification_environment_current() try: if double: @@ -1750,6 +1810,8 @@ def act_structural( ) from exc except ActionDeliveryUncertain: raise + except (FreshActuationRequired, DisplayTopologyChanged): + raise except StructuralResolutionRefused: raise except Exception as exc: @@ -1838,6 +1900,7 @@ def current_token_locator( ) from exc source = current_token_locator(source_locator, source_guard) destination = current_token_locator(destination_locator, destination_guard) + self._consume_input_observation(operation="guarded_dom_drag") self._assert_qualification_environment_current() try: source.drag_to(destination, timeout=1000) @@ -1850,6 +1913,8 @@ def current_token_locator( ) from exc except ActionDeliveryUncertain: raise + except (FreshActuationRequired, DisplayTopologyChanged): + raise except StructuralResolutionRefused: raise except Exception as exc: @@ -2046,6 +2111,7 @@ def act_guarded_coordinate( "visual target, frame, record, or context changed after the " "pre-dispatch actionability trial" ) + self._consume_input_observation(operation=operation) self._assert_qualification_environment_current() try: if double: @@ -2065,6 +2131,8 @@ def act_guarded_coordinate( ) from exc except ActionDeliveryUncertain: raise + except (FreshActuationRequired, DisplayTopologyChanged): + raise except StructuralResolutionRefused: raise except Exception as exc: @@ -2181,6 +2249,29 @@ def capture() -> bytes: # into a matching frame. return previous + def guarded_keyboard_observation(self) -> FrameObservation: + """Bind a caret-stable keyboard frame to exact browser geometry.""" + + for _attempt in range(_ATOMIC_OBSERVATION_ATTEMPTS): + generation = self._screenshot_frame_generation + before = self._read_browser_geometry() + png = self.guarded_keyboard_frame() + try: + self.page.evaluate("() => null") + except Exception as exc: + raise BrowserObservationStabilityError( + "the browser disconnected after keyboard-frame capture" + ) from exc + after = self._read_browser_geometry() + if generation == self._screenshot_frame_generation and before == after: + observation = self._observation_from_geometry(png, before) + self._last_frame_observation = observation + return observation + raise BrowserObservationStabilityError( + "the browser geometry or frame identity changed during every " + "caret-stable observation attempt" + ) + def _act_guarded_keyboard( self, *, @@ -2210,6 +2301,7 @@ def _act_guarded_keyboard( "focused keyboard target, frame, record, or context changed " "before delivery" ) + self._consume_input_observation(operation=operation) self._assert_qualification_environment_current() try: deliver(token_locator) @@ -2222,6 +2314,8 @@ def _act_guarded_keyboard( ) from exc except ActionDeliveryUncertain: raise + except (FreshActuationRequired, DisplayTopologyChanged): + raise except StructuralResolutionRefused: raise except Exception as exc: @@ -2272,6 +2366,16 @@ def _handle_screenshot_frame_lifecycle(self, _frame: Any = None) -> None: self._screenshot_frame_generation += 1 + def _handle_top_level_navigation(self, frame: Any) -> None: + """Give each new top-level document an exact frame identity.""" + + try: + if frame is self.page.main_frame: + self._top_level_frame_generation += 1 + except Exception: + # A disconnected page cannot produce another accepted observation. + self._top_level_frame_generation += 1 + def stop_screenshot_mask_tracking(self) -> None: """Remove recording-only frame listeners from an external page.""" @@ -2282,6 +2386,12 @@ def stop_screenshot_mask_tracking(self) -> None: self.page.remove_listener(event, self._screenshot_frame_listener) except Exception: pass + try: + self.page.remove_listener( + "framenavigated", self._top_level_navigation_listener + ) + except Exception: + pass self._screenshot_frame_tracking = False @staticmethod @@ -2290,7 +2400,7 @@ def _same_frames(left: tuple[Any, ...], right: tuple[Any, ...]) -> bool: before is after for before, after in zip(left, right) ) - def screenshot(self) -> bytes: + def _capture_screenshot_bytes(self) -> bytes: """Return a stable current full-viewport frame as PNG bytes.""" if self._screenshot_guard is not None: self._screenshot_guard() @@ -2333,6 +2443,253 @@ def screenshot(self) -> bytes: "screenshot attempt; recording was refused" ) + def _read_browser_geometry(self) -> _BrowserGeometry: + """Read one top-level viewport, DPR, display, page, and frame sample.""" + + try: + closed_probe = getattr(self.page, "is_closed", None) + if callable(closed_probe) and closed_probe(): + raise BrowserObservationStabilityError("the browser page is closed") + raw = self.page.evaluate( + """() => ({ + viewportWidth: window.innerWidth, + viewportHeight: window.innerHeight, + devicePixelRatio: window.devicePixelRatio || 1, + screenWidth: window.screen.width, + screenHeight: window.screen.height, + availLeft: Number.isFinite(window.screen.availLeft) + ? window.screen.availLeft : 0, + availTop: Number.isFinite(window.screen.availTop) + ? window.screen.availTop : 0, + availWidth: window.screen.availWidth || window.screen.width, + availHeight: window.screen.availHeight || window.screen.height, + colorDepth: window.screen.colorDepth || null, + pixelDepth: window.screen.pixelDepth || null, + })""" + ) + except BrowserObservationStabilityError: + raise + except Exception as exc: + raise BrowserObservationStabilityError( + "the browser geometry or page identity is unavailable" + ) from exc + try: + width = int(raw.get("viewportWidth", raw.get("width"))) + height = int(raw.get("viewportHeight", raw.get("height"))) + dpr = float(raw.get("devicePixelRatio", raw.get("dpr", 1.0))) + display_bounds = ( + float(raw.get("availLeft", 0.0)), + float(raw.get("availTop", 0.0)), + float(raw.get("availWidth", raw.get("screenWidth", width))), + float(raw.get("availHeight", raw.get("screenHeight", height))), + ) + except (AttributeError, KeyError, TypeError, ValueError) as exc: + raise BrowserObservationStabilityError( + "the browser returned invalid geometry" + ) from exc + if ( + width <= 0 + or height <= 0 + or not math.isfinite(dpr) + or dpr <= 0 + or not all(math.isfinite(value) for value in display_bounds) + or display_bounds[2] <= 0 + or display_bounds[3] <= 0 + ): + raise BrowserObservationStabilityError( + "the browser returned non-positive viewport or display geometry" + ) + display_identity_material = { + "schema": "openadapt.browser-display-identity.v1", + "screen_width": raw.get("screenWidth"), + "screen_height": raw.get("screenHeight"), + "available_bounds": list(display_bounds), + "device_pixel_ratio": dpr, + "color_depth": raw.get("colorDepth"), + "pixel_depth": raw.get("pixelDepth"), + } + display_id = "browser-display:" + frame_observation_identity( + display_identity_material + ) + top_frame_identity = frame_observation_identity( + { + "schema": "openadapt.playwright-top-level-frame-identity.v1", + "page_identity_sha256": self._page_identity_sha256, + "frame_object": id(getattr(self.page, "main_frame", self.page)), + "document_generation": self._top_level_frame_generation, + } + ) + window_identity = window_identity_sha256( + window_id=self._page_identity_sha256, + pid=0, + process_start_time=None, + owner="playwright-page", + ) + session_identity = session_identity_sha256( + authority="playwright-browser-context", + session_id=self._context_identity_sha256, + session_start_time=self._context_identity_sha256, + principal_identity_sha256=None, + ) + # Chromium does not expose a complete host monitor inventory without a + # separately granted multi-screen permission. Bind that limitation to + # this exact browser context. A selected-display or DPR change still + # opens a geometry epoch; it does not masquerade as a topology hot-plug. + topology = frame_observation_identity( + { + "schema": "openadapt.browser-topology-authority.v1", + "context_identity_sha256": self._context_identity_sha256, + "inventory": "not-exposed-by-page", + } + ) + return _BrowserGeometry( + viewport_width=width, + viewport_height=height, + device_pixel_ratio=dpr, + display_id=display_id, + display_bounds=display_bounds, + display_scale=(dpr, dpr), + topology_sha256=topology, + page_identity_sha256=self._page_identity_sha256, + top_level_frame_identity_sha256=top_frame_identity, + window_identity_sha256=window_identity, + session_identity_sha256=session_identity, + ) + + def observe_frame(self) -> FrameObservation: + """Capture one exact screenshot with stable browser geometry and identity.""" + + for _attempt in range(_ATOMIC_OBSERVATION_ATTEMPTS): + generation = self._screenshot_frame_generation + before = self._read_browser_geometry() + if generation != self._screenshot_frame_generation: + continue + png = self._capture_screenshot_bytes() + try: + self.page.evaluate("() => null") + except Exception as exc: + raise BrowserObservationStabilityError( + "the browser disconnected after screenshot capture" + ) from exc + after = self._read_browser_geometry() + if generation != self._screenshot_frame_generation or before != after: + continue + observation = self._observation_from_geometry(png, before) + self._last_frame_observation = observation + return observation + raise BrowserObservationStabilityError( + "the browser viewport, frame tree, or page identity did not stay " + "stable across an atomic screenshot" + ) + + @staticmethod + def _observation_from_geometry( + png: bytes, + geometry: _BrowserGeometry, + ) -> FrameObservation: + """Bind already-proven stable geometry to its exact PNG bytes.""" + + with Image.open(io.BytesIO(png)) as image: + png_width, png_height = image.size + scale = ( + png_width / geometry.viewport_width, + png_height / geometry.viewport_height, + ) + if not all( + math.isclose(value, 1.0, rel_tol=0.0, abs_tol=1e-9) for value in scale + ): + raise BrowserObservationStabilityError( + "browser screenshot pixels do not match CSS input coordinates; " + "use screenshot_scale='css' for atomic replay" + ) + return FrameObservation.create( + png, + viewport_width=geometry.viewport_width, + viewport_height=geometry.viewport_height, + origin=(0.0, 0.0), + scale=scale, + device_pixel_ratio=geometry.device_pixel_ratio, + display_id=geometry.display_id, + display_bounds=geometry.display_bounds, + display_scale=geometry.display_scale, + topology_sha256=geometry.topology_sha256, + window_identity_sha256=geometry.window_identity_sha256, + session_identity_sha256=geometry.session_identity_sha256, + page_identity_sha256=geometry.page_identity_sha256, + top_level_frame_identity_sha256=(geometry.top_level_frame_identity_sha256), + ) + + @property + def last_frame_observation(self) -> Optional[FrameObservation]: + """Return the most recent complete atomic browser observation.""" + + return self._last_frame_observation + + def screenshot(self) -> bytes: + """Compatibility raw screenshot path for recording and inspection.""" + + return self._capture_screenshot_bytes() + + def acquire_actuation_observation(self) -> FrameObservation: + """Acquire one stable browser frame for fresh target resolution.""" + + return self.observe_frame() + + def bind_input_observation(self, observation: FrameObservation) -> None: + """Bind the resolver's exact browser observation to the next input edge.""" + + if ( + observation.page_identity_sha256 != self._page_identity_sha256 + or observation.top_level_frame_identity_sha256 is None + ): + raise StructuralResolutionRefused( + "browser input observation belongs to another page or frame contract" + ) + self._bound_input_observation = observation + + def reset_fresh_actuation_state(self) -> None: + """Clear only zero-edge leases before one bounded fresh resolution.""" + + self._bound_input_observation = None + self.cancel_guarded_coordinate() + self.cancel_guarded_keyboard() + self.cancel_pending_structural_guards() + + def _consume_input_observation(self, *, operation: str) -> None: + """Recheck geometry/page identity and consume a zero-edge input lease.""" + + expected = self._bound_input_observation + if expected is None: + return + observed = self.observe_frame() + self._bound_input_observation = None + if expected.topology_sha256 != observed.topology_sha256: + raise DisplayTopologyChanged( + expected_observation=expected, + observed_observation=observed, + ) + if ( + expected.page_identity_sha256 != observed.page_identity_sha256 + or expected.top_level_frame_identity_sha256 + != observed.top_level_frame_identity_sha256 + or expected.window_identity_sha256 != observed.window_identity_sha256 + or expected.session_identity_sha256 != observed.session_identity_sha256 + ): + raise StructuralResolutionRefused( + "browser page or top-level frame identity changed before input" + ) + if expected.geometry_epoch != observed.geometry_epoch: + raise FreshActuationRequired( + operation=operation, + changed_pixel_count=observed.viewport[0] * observed.viewport[1], + changed_bbox=(0, 0, *observed.viewport), + frame_size=observed.viewport, + expected_geometry_epoch=expected.geometry_epoch, + observed_geometry_epoch=observed.geometry_epoch, + expected_observation=expected, + observed_observation=observed, + ) + def click(self, x: int, y: int, *, double: bool = False) -> None: """Click (or double-click) at pixel coordinates via the mouse.""" self._assert_qualification_environment_current() @@ -2410,6 +2767,7 @@ def drag_guarded( raise StructuralResolutionRefused( "visual drag source, record, or context changed before delivery" ) + self._consume_input_observation(operation="guarded_coordinate_drag") current_sha256 = hashlib.sha256(self.screenshot()).hexdigest() if not hmac.compare_digest(current_sha256, expected_frame_sha256): raise StructuralResolutionRefused( diff --git a/openadapt_flow/backends/rdp_backend.py b/openadapt_flow/backends/rdp_backend.py index 858a1350..d36d6f7c 100644 --- a/openadapt_flow/backends/rdp_backend.py +++ b/openadapt_flow/backends/rdp_backend.py @@ -65,8 +65,12 @@ from openadapt_flow.backend import ( ActionDeliveryUncertain, + FrameObservation, FreshActuationRequired, StructuralResolutionRefused, + frame_observation_identity, + session_identity_sha256, + window_identity_sha256, ) from openadapt_flow.ir import ActionDeliveryReceipt, Point from openadapt_flow.remote_frame_contract import RemoteFrameContract @@ -366,6 +370,9 @@ def __init__( self._qualification_environment: Optional[tuple[str, str, str, str]] = None self._qualification_input_guard: Optional[Callable[[], None]] = None self._actuation_frame_png: Optional[bytes] = None + self._last_frame_observation: Optional[FrameObservation] = None + self._actuation_observation: Optional[FrameObservation] = None + self._connection_identity = uuid.uuid4().hex self._actuation_lease_state = _LEASE_NONE # Keep capture/geometry validation and a complete input gesture in one # critical section. A concurrent screenshot may otherwise replace the @@ -413,11 +420,77 @@ def screenshot(self) -> bytes: else self._last_frame_digest ) self._last_session_identity = self._session_identity_from_frame(png) + self._last_frame_observation = self._observation_from_png(png) if self._actuation_lease_state == _LEASE_ARMED: self._invalidate_actuation_lease() return png + @property + def last_frame_observation(self) -> Optional[FrameObservation]: + """The descriptor created by the most recent completed frame capture.""" + + return self._last_frame_observation + + def _observation_from_png( + self, + png: bytes, + *, + viewport: Optional[tuple[int, int]] = None, + session_identity: Optional[str] = None, + ) -> FrameObservation: + """Bind one RDP framebuffer to its exact connection and geometry.""" + + viewport = self._viewport if viewport is None else viewport + if viewport is None: + raise RuntimeError("RDP frame observation has no framebuffer geometry") + principal = session_identity or self._last_session_identity + session = session_identity_sha256( + authority=f"rdp:{type(self._transport).__qualname__}", + session_id=self._connection_identity, + session_start_time=None, + principal_identity_sha256=principal, + ) + topology = frame_observation_identity( + { + "schema": "openadapt.rdp-topology.v1", + "transport": type(self._transport).__qualname__, + "connection": self._connection_identity, + "coordinate_space": "remote-virtual-framebuffer", + } + ) + window = window_identity_sha256( + window_id=f"rdp-framebuffer:{self._connection_identity}", + pid=0, + process_start_time=None, + owner=type(self._transport).__qualname__, + ) + return FrameObservation.create( + png, + origin=(0.0, 0.0), + scale=(1.0, 1.0), + device_pixel_ratio=1.0, + display_id=f"rdp-virtual-display:{self._connection_identity}", + display_bounds=(0.0, 0.0, float(viewport[0]), float(viewport[1])), + display_scale=(1.0, 1.0), + topology_sha256=topology, + window_identity_sha256=window, + session_identity_sha256=session, + ) + + def observe_frame(self) -> FrameObservation: + """Capture one atomic RDP framebuffer observation.""" + + with self._input_lock: + observation = self._observation_from_png(self.screenshot()) + self._last_frame_observation = observation + return observation + def acquire_actuation_frame(self) -> bytes: + """Compatibility byte view of :meth:`acquire_actuation_observation`.""" + + return self.acquire_actuation_observation().png + + def acquire_actuation_observation(self) -> FrameObservation: """Capture readiness and arm a one-shot exact-content input lease. A direct RDP transport is already the selected remote session; the @@ -426,7 +499,8 @@ def acquire_actuation_frame(self) -> bytes: when dimensions, readiness, or pixels changed. """ with self._input_lock: - png = self.screenshot() + observation = self.observe_frame() + png = observation.png if self._readiness_probe is not None and not self._readiness_probe(png): self._invalidate_actuation_lease() raise RuntimeError( @@ -453,8 +527,25 @@ def acquire_actuation_frame(self) -> bytes: "fresh actuation frame" ) self._actuation_frame_png = png + self._actuation_observation = observation self._actuation_lease_state = _LEASE_ARMED - return png + return observation + + def bind_input_observation(self, observation: FrameObservation) -> None: + """Require the runtime to consume the exact armed RDP descriptor.""" + + with self._input_lock: + armed = self._actuation_observation + if ( + self._actuation_lease_state != _LEASE_ARMED + or armed is None + or armed.frame_sha256 != observation.frame_sha256 + or armed.geometry_epoch != observation.geometry_epoch + ): + self._invalidate_actuation_lease() + raise StructuralResolutionRefused( + "RDP input observation does not match its exact fresh-frame lease" + ) def arm_remote_frame_contract( self, *, protected_regions: tuple[tuple[int, int, int, int], ...] @@ -476,6 +567,7 @@ def reset_fresh_actuation_state(self) -> None: ) self._actuation_lease_state = _LEASE_NONE self._actuation_frame_png = None + self._actuation_observation = None # -- Optional ExecutionContextIdentityBackend -------------------------- @@ -1122,7 +1214,28 @@ def _ensure_input_ready( current = (int(w), int(h)) if current[0] <= 0 or current[1] <= 0: raise RuntimeError(f"RDP framebuffer has invalid dimensions {current!r}") + current_img = self._to_image(frame, current[0], current[1]) + current_png = self._png_bytes(current_img) + exact_lease_ready = self._actuation_lease_state == _LEASE_ARMED if self._viewport != current: + armed = self._actuation_observation + if exact_lease_ready and armed is not None: + current_observation = self._observation_from_png( + current_png, + viewport=current, + session_identity=self._last_session_identity, + ) + self._invalidate_actuation_lease() + raise FreshActuationRequired( + operation=operation, + changed_pixel_count=current[0] * current[1], + changed_bbox=(0, 0, current[0], current[1]), + frame_size=current, + expected_geometry_epoch=armed.geometry_epoch, + observed_geometry_epoch=current_observation.geometry_epoch, + expected_observation=armed, + observed_observation=current_observation, + ) raise RuntimeError( f"RDP framebuffer changed from {self._viewport!r} to {current!r}; " "capture and re-resolve before sending input" @@ -1138,7 +1251,6 @@ def _ensure_input_ready( "RDP actuation lease was invalidated by another observation; " "refusing input and requiring a fresh lease" ) - exact_lease_ready = self._actuation_lease_state == _LEASE_ARMED if ( self._readiness_probe is not None or self._actuation_lease_state == _LEASE_ARMED @@ -1148,8 +1260,6 @@ def _ensure_input_ready( # Evaluate readiness on the current framebuffer, not merely the # resolver's leased image: a lock/disconnect or content change can # appear while the dimensions stay unchanged. - current_img = self._to_image(frame, current[0], current[1]) - current_png = self._png_bytes(current_img) if self._readiness_probe is not None and not self._readiness_probe( current_png ): @@ -1169,6 +1279,29 @@ def _ensure_input_ready( "capture; refusing input" ) if exact_lease_ready: + armed = self._actuation_observation + if armed is None: + self._invalidate_actuation_lease() + raise StructuralResolutionRefused( + "RDP actuation lease has no atomic frame observation" + ) + current_observation = self._observation_from_png( + current_png, + viewport=current, + session_identity=current_session_identity, + ) + if current_observation.geometry_epoch != armed.geometry_epoch: + self._invalidate_actuation_lease() + raise FreshActuationRequired( + operation=operation, + changed_pixel_count=current[0] * current[1], + changed_bbox=(0, 0, current[0], current[1]), + frame_size=current, + expected_geometry_epoch=armed.geometry_epoch, + observed_geometry_epoch=current_observation.geometry_epoch, + expected_observation=armed, + observed_observation=current_observation, + ) raw_digest = self._canonical_frame_digest(current_img) digest = ( self._remote_frame_contract.comparison_digest(current_png) @@ -1189,6 +1322,8 @@ def _ensure_input_ready( changed_pixel_count=changed_pixel_count, changed_bbox=changed_bbox, frame_size=current, + expected_observation=armed, + observed_observation=current_observation, ) # The complete qualification environment was checked against # this exact leased image in ``acquire_actuation_frame``. The @@ -1222,6 +1357,7 @@ def _ensure_input_ready( ) self._actuation_lease_state = _LEASE_NONE self._actuation_frame_png = None + self._actuation_observation = None self._assert_frame_fresh() def set_qualification_input_guard( @@ -1242,6 +1378,7 @@ def _invalidate_actuation_lease(self) -> None: if self._actuation_lease_state == _LEASE_ARMED: self._actuation_lease_state = _LEASE_INVALIDATED self._actuation_frame_png = None + self._actuation_observation = None def _fresh_identity_frame(self) -> Optional[bytes]: """Passively capture a fresh framebuffer without replacing a valid lease.""" diff --git a/openadapt_flow/backends/remote_display.py b/openadapt_flow/backends/remote_display.py index 4df320d7..b96f97bc 100644 --- a/openadapt_flow/backends/remote_display.py +++ b/openadapt_flow/backends/remote_display.py @@ -69,8 +69,16 @@ from openadapt_flow.backend import ( ActionDeliveryUncertain, + DisplayGeometry, + DisplayTopologyChanged, + FrameObservation, FreshActuationRequired, StructuralResolutionRefused, + display_topology_sha256, + frame_observation_identity, + select_display_for_bounds, + session_identity_sha256, + window_identity_sha256, ) from openadapt_flow.ir import ActionDeliveryReceipt from openadapt_flow.remote_frame_contract import RemoteFrameContract @@ -82,6 +90,11 @@ _LEASE_INVALIDATED = 2 _SESSION_DIGEST_HEX_LENGTH = 64 +_MACOS_CHROME_AX_TITLE_SUFFIXES = { + "Google Chrome": "Google Chrome", + "Google Chrome for Testing": "Google Chrome for Testing", +} + class RemoteDisplayError(RuntimeError): """A remote-display capture/inject operation failed (or is not permitted).""" @@ -120,6 +133,16 @@ class WindowInfo: on_screen: bool = True +def _macos_ax_title_matches(window: WindowInfo, ax_title: object) -> bool: + """Match one CG title to AX without widening beyond known Chrome suffixes.""" + + candidate = str(ax_title or "") + exact = candidate == window.title + suffix = _MACOS_CHROME_AX_TITLE_SUFFIXES.get(window.owner) + known_suffix = suffix is not None and candidate == f"{window.title} - {suffix}" + return exact or known_suffix + + # macOS US-layout virtual key codes for printable characters. A synthetic # Unicode keystroke (keycode 0 + ``CGEventKeyboardSetUnicodeString``) is NOT # forwarded into a Parallels/Citrix guest — the remote display forwards @@ -390,6 +413,10 @@ def capture(self, window_id: int) -> tuple[bytes, int, int]: """Capture window ``window_id``; return ``(png_bytes, px_w, px_h)``.""" ... + def display_topology(self) -> tuple[DisplayGeometry, ...]: + """Return every stable host display in the window coordinate space.""" + ... + def activate(self, pid: int) -> None: """Un-hide and bring the app owning ``pid`` frontmost (route keystrokes).""" ... @@ -603,6 +630,9 @@ def __init__( self._last_frame_digest: Optional[bytes] = None self._last_comparison_digest: Optional[bytes] = None self._actuation_frame_png: Optional[bytes] = None + self._last_frame_observation: Optional[FrameObservation] = None + self._actuation_observation: Optional[FrameObservation] = None + self._backend_session_identity = uuid.uuid4().hex self._last_session_identity: Optional[str] = None self._qualification_environment: Optional[tuple[str, str, str, str]] = None self._qualification_input_guard: Optional[Callable[[], None]] = None @@ -740,6 +770,7 @@ def screenshot(self) -> bytes: else self._last_frame_digest ) self._last_session_identity = self._session_identity_from_frame(png) + self._last_frame_observation = self._observation_from_state(png) # An ordinary observation is not permission to perform a # consequential remote action. Only acquire_actuation_frame arms # the one-shot content lease after focus/readiness are established. @@ -749,9 +780,109 @@ def screenshot(self) -> bytes: if self._actuation_lease_state == _LEASE_ARMED: self._actuation_lease_state = _LEASE_INVALIDATED self._actuation_frame_png = None + self._actuation_observation = None return png + @property + def last_frame_observation(self) -> Optional[FrameObservation]: + """The descriptor created by the most recent completed frame capture.""" + + return self._last_frame_observation + + def _observation_from_state( + self, + png: bytes, + *, + window: Optional[WindowInfo] = None, + viewport: Optional[tuple[int, int]] = None, + scale: Optional[tuple[float, float]] = None, + session_identity: Optional[str] = None, + ) -> FrameObservation: + """Build a descriptor only from facts captured under ``_input_lock``.""" + + win = self._frame_window if window is None else window + size = self._viewport if viewport is None else viewport + if win is None or size is None: + raise RemoteDisplayError("remote-display observation has no window state") + pixel_scale = (self._scale_x, self._scale_y) if scale is None else scale + principal = session_identity or self._last_session_identity + session = session_identity_sha256( + authority=f"remote-display:{type(self._client).__qualname__}", + session_id=self._backend_session_identity, + session_start_time=None, + principal_identity_sha256=principal, + ) + display_probe = getattr(self._client, "display_topology", None) + displays = ( + tuple(display_probe()) + if callable(display_probe) + else ( + DisplayGeometry( + display_id=f"test-window-display:{win.window_id}", + bounds=( + float(win.bounds[0]), + float(win.bounds[1]), + float(win.bounds[2]), + float(win.bounds[3]), + ), + scale=(float(pixel_scale[0]), float(pixel_scale[1])), + ), + ) + ) + try: + display = select_display_for_bounds(displays, win.bounds) + topology = display_topology_sha256( + displays, + coordinate_space="host-screen-points-top-left", + ) + except ValueError as exc: + raise RemoteDisplayError( + "remote-display monitor topology is unavailable or ambiguous" + ) from exc + topology = frame_observation_identity( + { + "schema": "openadapt.remote-display-frame-topology.v1", + "display_topology_sha256": topology, + "client": type(self._client).__qualname__, + } + ) + process_start_probe = getattr(self._client, "process_start_time", None) + process_start_time = ( + process_start_probe(win.pid) if callable(process_start_probe) else None + ) + window_identity = window_identity_sha256( + window_id=str(win.window_id), + pid=win.pid, + process_start_time=process_start_time, + owner=win.owner, + ) + return FrameObservation.create( + png, + origin=(float(win.bounds[0]), float(win.bounds[1])), + scale=(float(pixel_scale[0]), float(pixel_scale[1])), + device_pixel_ratio=float(pixel_scale[0]), + display_id=display.display_id, + display_bounds=display.bounds, + display_scale=display.scale, + topology_sha256=topology, + window_identity_sha256=window_identity, + session_identity_sha256=session, + ) + + def observe_frame(self) -> FrameObservation: + """Capture the exact window pixels and their host coordinate mapping.""" + + with self._input_lock: + observation = self._observation_from_state(self.screenshot()) + self._last_frame_observation = observation + return observation + def acquire_actuation_frame(self) -> bytes: + """Compatibility byte view of :meth:`acquire_actuation_observation`.""" + + return self.acquire_actuation_observation().png + + def acquire_actuation_observation(self) -> FrameObservation: """Acquire the exact client window and arm a one-shot content lease. The runtime re-resolves the target and record identity on the returned @@ -774,7 +905,8 @@ def acquire_actuation_frame(self) -> bytes: "app-frontmost, and keyboard-frontmost; refusing to acquire " "an actuation lease" ) - png = self.screenshot() + observation = self.observe_frame() + png = observation.png assert self._frame_window is not None lease = self._frame_window current = self._resolve_window(refresh=True) @@ -816,7 +948,24 @@ def acquire_actuation_frame(self) -> bytes: ) self._actuation_lease_state = _LEASE_ARMED self._actuation_frame_png = png - return png + self._actuation_observation = observation + return observation + + def bind_input_observation(self, observation: FrameObservation) -> None: + """Require the runtime to consume the exact armed window descriptor.""" + + with self._input_lock: + armed = self._actuation_observation + if ( + self._actuation_lease_state != _LEASE_ARMED + or armed is None + or armed.frame_sha256 != observation.frame_sha256 + or armed.geometry_epoch != observation.geometry_epoch + ): + self._invalidate_actuation_lease() + raise StructuralResolutionRefused( + "remote-display input observation does not match its exact lease" + ) def arm_remote_frame_contract( self, *, protected_regions: tuple[tuple[int, int, int, int], ...] @@ -827,8 +976,11 @@ def arm_remote_frame_contract( def reset_fresh_actuation_state(self) -> None: """Reset only a typed zero-input content invalidation. - This clears the stale prepared point but grants no actuation authority. - The runtime must prepare, acquire, and validate a new lease. + This clears the stale prepared point and passively observes the current + geometry, but grants no actuation authority. The passive observation + is required after a resize or cross-display move: pointer preparation + must map through the new window bounds before the runtime can acquire + and validate its replacement lease. """ with self._input_lock: @@ -839,7 +991,12 @@ def reset_fresh_actuation_state(self) -> None: ) self._actuation_lease_state = _LEASE_NONE self._actuation_frame_png = None + self._actuation_observation = None self._prepared_pointer_point = None + # ``screenshot`` refreshes only observation state. It cannot arm + # a consequential lease. A later prepare/acquire/bind sequence is + # still required before the next input edge. + self.screenshot() # -- Optional ExecutionContextIdentityBackend -------------------------- @@ -1423,14 +1580,18 @@ def _ensure_input_ready( self._assert_frame_fresh() assert self._frame_window is not None lease = self._frame_window + if current.window_id != lease.window_id or current.pid != lease.pid: + raise RemoteDisplayError( + "remote-display window identity changed since capture; capture " + "and re-resolve before input" + ) if ( - current.window_id != lease.window_id - or current.pid != lease.pid - or current.bounds != lease.bounds + current.bounds != lease.bounds + and self._actuation_lease_state != _LEASE_ARMED ): raise RemoteDisplayError( - "remote-display window identity or geometry changed since capture; " - "capture and re-resolve before input" + "remote-display window geometry changed since capture; capture " + "and re-resolve before input" ) assert self._viewport is not None if point is not None: @@ -1455,12 +1616,16 @@ def _ensure_input_ready( # lease. This detects a lock/disconnect or content change after the # runtime re-resolved the fresh actuation frame. png, px_w, px_h = self._client.capture(current.window_id) - if _png_size(png) != self._viewport or (px_w, px_h) != self._viewport: - self._actuation_lease_state = _LEASE_INVALIDATED + observed_size = _png_size(png) + if observed_size != (int(px_w), int(px_h)): + self._invalidate_actuation_lease() raise RemoteDisplayError( - "remote-display dimensions changed during readiness check; " - "capture and re-resolve before input" + "remote-display capture dimensions disagree before input" ) + observed_scale = ( + observed_size[0] / current.bounds[2], + observed_size[1] / current.bounds[3], + ) if self._readiness_probe is not None and not self._readiness_probe(png): self._actuation_lease_state = _LEASE_INVALIDATED raise RemoteDisplayError( @@ -1488,6 +1653,37 @@ def _ensure_input_ready( "target resolution; refusing input" ) if self._actuation_lease_state == _LEASE_ARMED: + armed = self._actuation_observation + if armed is None: + self._invalidate_actuation_lease() + raise StructuralResolutionRefused( + "remote-display lease has no atomic frame observation" + ) + current_observation = self._observation_from_state( + png, + window=current, + viewport=observed_size, + scale=observed_scale, + session_identity=current_session_identity, + ) + if current_observation.topology_sha256 != armed.topology_sha256: + self._invalidate_actuation_lease() + raise DisplayTopologyChanged( + expected_observation=armed, + observed_observation=current_observation, + ) + if current_observation.geometry_epoch != armed.geometry_epoch: + self._invalidate_actuation_lease() + raise _RemoteDisplayFreshActuationRequired( + operation=operation, + changed_pixel_count=observed_size[0] * observed_size[1], + changed_bbox=(0, 0, observed_size[0], observed_size[1]), + frame_size=observed_size, + expected_geometry_epoch=armed.geometry_epoch, + observed_geometry_epoch=current_observation.geometry_epoch, + expected_observation=armed, + observed_observation=current_observation, + ) # Consume once before the first input edge. A double click or # multi-character type is one gesture and must not invalidate # itself after its first state-changing edge. @@ -1512,14 +1708,55 @@ def _ensure_input_ready( changed_pixel_count=changed_pixel_count, changed_bbox=changed_bbox, frame_size=self._viewport, + expected_observation=armed, + observed_observation=current_observation, ) - if consume_actuation_lease: - self._actuation_lease_state = _LEASE_NONE - self._actuation_frame_png = None # Activation, window resolution, capture and readiness/OCR may all # block. Re-resolve the exact window/key identity and age again at the # last common point before input. post = self._resolve_window(refresh=True) + post_geometry_changed = ( + post.window_id == lease.window_id + and post.pid == lease.pid + and post.bounds != lease.bounds + ) + if post_geometry_changed and self._actuation_lease_state == _LEASE_ARMED: + armed = self._actuation_observation + assert armed is not None + post_png, post_width, post_height = self._client.capture(post.window_id) + post_size = _png_size(post_png) + if post_size != (int(post_width), int(post_height)): + self._invalidate_actuation_lease() + raise RemoteDisplayError( + "remote-display post-check capture dimensions disagree" + ) + post_observation = self._observation_from_state( + post_png, + window=post, + viewport=post_size, + scale=( + post_size[0] / post.bounds[2], + post_size[1] / post.bounds[3], + ), + session_identity=self._session_identity_from_frame(post_png), + ) + if post_observation.topology_sha256 != armed.topology_sha256: + self._invalidate_actuation_lease() + raise DisplayTopologyChanged( + expected_observation=armed, + observed_observation=post_observation, + ) + self._invalidate_actuation_lease() + raise _RemoteDisplayFreshActuationRequired( + operation=operation, + changed_pixel_count=self._viewport[0] * self._viewport[1], + changed_bbox=(0, 0, self._viewport[0], self._viewport[1]), + frame_size=self._viewport, + expected_geometry_epoch=armed.geometry_epoch, + observed_geometry_epoch=post_observation.geometry_epoch, + expected_observation=armed, + observed_observation=post_observation, + ) if ( not post.on_screen or not self._window_focus_matches(post) @@ -1534,6 +1771,10 @@ def _ensure_input_ready( if self._qualification_input_guard is not None: self._qualification_input_guard() self._assert_frame_fresh() + if consume_actuation_lease and self._actuation_lease_state == _LEASE_ARMED: + self._actuation_lease_state = _LEASE_NONE + self._actuation_frame_png = None + self._actuation_observation = None def set_qualification_input_guard( self, guard: Optional[Callable[[], None]] @@ -1553,6 +1794,7 @@ def _invalidate_actuation_lease(self) -> None: if self._actuation_lease_state == _LEASE_ARMED: self._actuation_lease_state = _LEASE_INVALIDATED self._actuation_frame_png = None + self._actuation_observation = None @staticmethod def _frame_difference( @@ -1729,6 +1971,63 @@ def input_trusted(self) -> bool: except Exception: # noqa: BLE001 - absence == untrusted return False + def process_start_time(self, pid: int) -> Optional[str]: + """Return the exact AppKit launch timestamp for one live process.""" + + try: + from AppKit import NSRunningApplication + + app = NSRunningApplication.runningApplicationWithProcessIdentifier_(pid) + launch_date = app.launchDate() if app is not None else None + if launch_date is None: + return None + return f"unix-seconds:{float(launch_date.timeIntervalSince1970()):.6f}" + except Exception: # noqa: BLE001 - unavailable is explicit in the digest + return None + + def display_topology(self) -> tuple[DisplayGeometry, ...]: + """Return stable CoreGraphics display IDs, bounds, and pixel scales.""" + + try: + import Quartz + + status, display_ids, count = Quartz.CGGetActiveDisplayList(64, None, None) + if int(status) != 0: + raise RemoteDisplayError( + f"CoreGraphics display enumeration failed ({int(status)})" + ) + displays: list[DisplayGeometry] = [] + for raw_display_id in tuple(display_ids or ())[: int(count)]: + display_id = int(raw_display_id) + rect = Quartz.CGDisplayBounds(display_id) + width = float(rect.size.width) + height = float(rect.size.height) + if width <= 0 or height <= 0: + continue + scale_x = float(Quartz.CGDisplayPixelsWide(display_id)) / width + scale_y = float(Quartz.CGDisplayPixelsHigh(display_id)) / height + displays.append( + DisplayGeometry( + display_id=f"cgdisplay:{display_id}", + bounds=( + float(rect.origin.x), + float(rect.origin.y), + width, + height, + ), + scale=(scale_x, scale_y), + ) + ) + if not displays: + raise RemoteDisplayError("CoreGraphics returned no active displays") + return tuple(displays) + except RemoteDisplayError: + raise + except Exception as exc: + raise RemoteDisplayError( + "CoreGraphics display topology is unavailable" + ) from exc + def resolve_key(self, token: str) -> Optional[tuple[int, bool]]: """macOS virtual key code for a named-key/character chord token.""" return resolve_mac_key(token) @@ -1907,10 +2206,11 @@ def raise_window(self, window: WindowInfo) -> bool: App activation alone does not select a document when macOS restores several windows into one process. Match the already-unique - CoreGraphics target by its exact AX title, require one AX candidate, - and perform ``AXRaise``. The native backend independently re-checks - the exact CoreGraphics window id afterward, so a stale title or failed - AX mapping remains a fail-closed refusal. + CoreGraphics target by its exact AX title, or by one explicitly allowed + Chrome application-name suffix, require one AX candidate, and perform + ``AXRaise``. The native backend independently re-checks the exact + CoreGraphics window id afterward, so a stale title or failed AX mapping + remains a fail-closed refusal. """ try: from ApplicationServices import ( @@ -1937,7 +2237,7 @@ def raise_window(self, window: WindowInfo) -> bool: title_error, title = AXUIElementCopyAttributeValue( candidate, kAXTitleAttribute, None ) - if title_error == 0 and str(title or "") == window.title: + if title_error == 0 and _macos_ax_title_matches(window, title): matches.append(candidate) if len(matches) != 1: return False @@ -2006,7 +2306,7 @@ def _focused_element_for_window(window: WindowInfo) -> Any: title_error, title = AXUIElementCopyAttributeValue( candidate, kAXTitleAttribute, None ) - if title_error == 0 and str(title or "") == window.title: + if title_error == 0 and _macos_ax_title_matches(window, title): matching_windows.append(candidate) if len(matching_windows) != 1: return None @@ -2039,7 +2339,7 @@ def _focused_element_for_window(window: WindowInfo) -> Any: top_title_error, top_title = AXUIElementCopyAttributeValue( top_level, kAXTitleAttribute, None ) - if top_title_error != 0 or str(top_title or "") != window.title: + if top_title_error != 0 or not _macos_ax_title_matches(window, top_title): return None return focused except Exception: # noqa: BLE001 - unknown AX focus fails closed @@ -2090,7 +2390,7 @@ def focused_element_token_at_point( kAXTitleAttribute, None, ) - if title_error != 0 or str(title or "") != window.title: + if title_error != 0 or not _macos_ax_title_matches(window, title): return None # A hit test may return an internal text child while AX focus is on @@ -2179,7 +2479,8 @@ def replace_selected_text(self, window: WindowInfo, text: str) -> bool: delivery. Accessibility selected-text replacement is layout independent and returns an explicit delivery result. This method requires a unique AX window title and proves that the focused element belongs to it before - writing; the caller separately verifies the exact topmost CG id. The + writing; only an explicitly allowed Chrome application-name suffix may + differ. The caller separately verifies the exact topmost CG id. The active-app PID remains mandatory only for global/physical input. """ try: @@ -2208,7 +2509,7 @@ def replace_selected_text(self, window: WindowInfo, text: str) -> bool: title_error, title = AXUIElementCopyAttributeValue( candidate, kAXTitleAttribute, None ) - if title_error == 0 and str(title or "") == window.title: + if title_error == 0 and _macos_ax_title_matches(window, title): matching_windows.append(candidate) if len(matching_windows) != 1: return False @@ -2241,7 +2542,7 @@ def replace_selected_text(self, window: WindowInfo, text: str) -> bool: top_title_error, top_title = AXUIElementCopyAttributeValue( top_level, kAXTitleAttribute, None ) - if top_title_error != 0 or str(top_title or "") != window.title: + if top_title_error != 0 or not _macos_ax_title_matches(window, top_title): return False if top_level != target: return False @@ -2281,7 +2582,7 @@ def exact_window_focused_main(self, window: WindowInfo) -> bool: title_error, title = AXUIElementCopyAttributeValue( candidate, kAXTitleAttribute, None ) - if title_error == 0 and str(title or "") == window.title: + if title_error == 0 and _macos_ax_title_matches(window, title): matches.append(candidate) if len(matches) != 1: return False diff --git a/openadapt_flow/backends/win_agent/server.py b/openadapt_flow/backends/win_agent/server.py index 4d0a4439..4e4a2fde 100644 --- a/openadapt_flow/backends/win_agent/server.py +++ b/openadapt_flow/backends/win_agent/server.py @@ -75,6 +75,7 @@ import json import os import secrets +import struct import subprocess import time import traceback @@ -95,7 +96,204 @@ CERTFILE_ENV_VAR = "OAFLOW_AGENT_CERTFILE" KEYFILE_ENV_VAR = "OAFLOW_AGENT_KEYFILE" -GrabFn = Callable[[], bytes] +_FRAME_GEOMETRY_HEADER = "X-OpenAdapt-Frame-Geometry" +_FRAME_BINDING_HEADER = "X-OpenAdapt-Frame-Binding-SHA256" + + +@dataclass(frozen=True) +class MonitorGeometry: + """One physical-pixel monitor rectangle in virtual-desktop coordinates.""" + + device: str + left: int + top: int + width: int + height: int + dpi_x: int + dpi_y: int + primary: bool + + def to_payload(self) -> dict[str, Any]: + return { + "device": self.device, + "left": self.left, + "top": self.top, + "width": self.width, + "height": self.height, + "dpi_x": self.dpi_x, + "dpi_y": self.dpi_y, + "primary": self.primary, + } + + +@dataclass(frozen=True) +class FrameGeometry: + """Exact physical-pixel coordinate space for one captured desktop frame.""" + + origin_x: int + origin_y: int + width: int + height: int + monitors: tuple[MonitorGeometry, ...] + + def to_payload(self) -> dict[str, Any]: + return { + "version": 1, + "coordinate_space": "physical_virtual_desktop", + "dpi_awareness": "per_monitor_v2", + "origin_x": self.origin_x, + "origin_y": self.origin_y, + "width": self.width, + "height": self.height, + "monitors": [monitor.to_payload() for monitor in self.monitors], + } + + def canonical_bytes(self) -> bytes: + return json.dumps( + self.to_payload(), sort_keys=True, separators=(",", ":") + ).encode("utf-8") + + def frame_to_virtual(self, x: int, y: int) -> tuple[int, int]: + if not (0 <= x < self.width and 0 <= y < self.height): + raise ValueError( + f"frame point {(x, y)!r} is outside {(self.width, self.height)!r}" + ) + return self.origin_x + x, self.origin_y + y + + def virtual_to_frame(self, x: int, y: int) -> tuple[int, int]: + frame_x = x - self.origin_x + frame_y = y - self.origin_y + if not (0 <= frame_x < self.width and 0 <= frame_y < self.height): + raise ValueError( + f"virtual point {(x, y)!r} is outside the captured desktop" + ) + return frame_x, frame_y + + @classmethod + def from_payload(cls, value: object) -> "FrameGeometry": + if not isinstance(value, dict): + raise ValueError("frame geometry must be an object") + required = { + "version", + "coordinate_space", + "dpi_awareness", + "origin_x", + "origin_y", + "width", + "height", + "monitors", + } + if set(value) != required: + raise ValueError("frame geometry has an invalid field set") + if value["version"] != 1: + raise ValueError("unsupported frame geometry version") + if value["coordinate_space"] != "physical_virtual_desktop": + raise ValueError("frame geometry is not physical virtual-desktop space") + if value["dpi_awareness"] != "per_monitor_v2": + raise ValueError("frame geometry is not Per-Monitor-v2 aware") + + def integer(name: str, *, positive: bool = False) -> int: + item = value[name] + if isinstance(item, bool) or not isinstance(item, int): + raise ValueError(f"frame geometry {name} must be an integer") + if abs(item) > 1_000_000 or (positive and item <= 0): + raise ValueError(f"frame geometry {name} is out of bounds") + return item + + origin_x = integer("origin_x") + origin_y = integer("origin_y") + width = integer("width", positive=True) + height = integer("height", positive=True) + raw_monitors = value["monitors"] + if not isinstance(raw_monitors, list) or not 1 <= len(raw_monitors) <= 32: + raise ValueError("frame geometry needs 1-32 monitors") + monitors: list[MonitorGeometry] = [] + monitor_keys = { + "device", + "left", + "top", + "width", + "height", + "dpi_x", + "dpi_y", + "primary", + } + for raw in raw_monitors: + if not isinstance(raw, dict) or set(raw) != monitor_keys: + raise ValueError("monitor geometry has an invalid field set") + device = raw["device"] + if not isinstance(device, str) or not 1 <= len(device) <= 128: + raise ValueError("monitor device must be a bounded string") + + def monitor_integer(name: str, *, positive: bool = False) -> int: + item = raw[name] + if isinstance(item, bool) or not isinstance(item, int): + raise ValueError(f"monitor {name} must be an integer") + if abs(item) > 1_000_000 or (positive and item <= 0): + raise ValueError(f"monitor {name} is out of bounds") + return item + + primary = raw["primary"] + if not isinstance(primary, bool): + raise ValueError("monitor primary must be boolean") + monitor = MonitorGeometry( + device=device, + left=monitor_integer("left"), + top=monitor_integer("top"), + width=monitor_integer("width", positive=True), + height=monitor_integer("height", positive=True), + dpi_x=monitor_integer("dpi_x", positive=True), + dpi_y=monitor_integer("dpi_y", positive=True), + primary=primary, + ) + if not (48 <= monitor.dpi_x <= 960 and 48 <= monitor.dpi_y <= 960): + raise ValueError("monitor DPI is outside the supported range") + if not ( + origin_x <= monitor.left + and origin_y <= monitor.top + and monitor.left + monitor.width <= origin_x + width + and monitor.top + monitor.height <= origin_y + height + ): + raise ValueError("monitor rectangle is outside the virtual desktop") + monitors.append(monitor) + if sum(monitor.primary for monitor in monitors) != 1: + raise ValueError("frame geometry must contain one primary monitor") + if len({monitor.device.casefold() for monitor in monitors}) != len(monitors): + raise ValueError("monitor device identifiers must be unique") + ordered = tuple( + sorted(monitors, key=lambda item: (item.left, item.top, item.device)) + ) + return cls(origin_x, origin_y, width, height, ordered) + + +@dataclass(frozen=True) +class CapturedDesktopFrame: + """A PNG and the exact virtual-desktop geometry captured with it.""" + + png: bytes + geometry: FrameGeometry + + +def encode_frame_geometry_header(geometry: FrameGeometry) -> str: + return base64.urlsafe_b64encode(geometry.canonical_bytes()).decode("ascii") + + +def decode_frame_geometry_header(value: str) -> FrameGeometry: + if not isinstance(value, str) or not value: + raise ValueError("missing frame geometry header") + try: + raw = base64.b64decode(value, altchars=b"-_", validate=True) + payload = json.loads(raw) + except Exception as exc: + raise ValueError("invalid frame geometry header") from exc + return FrameGeometry.from_payload(payload) + + +def frame_binding_sha256(png: bytes, geometry: FrameGeometry) -> str: + return hashlib.sha256(geometry.canonical_bytes() + b"\0" + png).hexdigest() + + +GrabFn = Callable[[], bytes | CapturedDesktopFrame] InputFn = Callable[[dict[str, Any]], dict[str, Any]] UiaFn = Callable[[str, dict[str, Any]], dict[str, Any]] ContextFn = Callable[[], dict[str, Any]] @@ -163,6 +361,292 @@ def _bounded_int(value: object, label: str, *, limit: int = 1_000_000) -> int: return value +def _png_size(png: bytes) -> tuple[int, int]: + if len(png) < 24 or not png.startswith(_PNG_SIGNATURE): + raise ValueError("not a PNG frame") + width, height = struct.unpack(">II", png[16:24]) + if width <= 0 or height <= 0: + raise ValueError("PNG frame has invalid dimensions") + return int(width), int(height) + + +def _synthetic_frame_geometry(png: bytes) -> FrameGeometry: + """Compatibility geometry for an injected byte-only test/legacy grabber.""" + + width, height = _png_size(png) + return FrameGeometry( + origin_x=0, + origin_y=0, + width=width, + height=height, + monitors=( + MonitorGeometry( + device="DISPLAY1", + left=0, + top=0, + width=width, + height=height, + dpi_x=96, + dpi_y=96, + primary=True, + ), + ), + ) + + +def _coerce_captured_frame(value: bytes | CapturedDesktopFrame) -> CapturedDesktopFrame: + frame = ( + CapturedDesktopFrame(value, _synthetic_frame_geometry(value)) + if isinstance(value, bytes) + else value + ) + if not isinstance(frame, CapturedDesktopFrame): + raise TypeError("desktop grabber returned an unsupported frame type") + size = _png_size(frame.png) + if size != (frame.geometry.width, frame.geometry.height): + raise ValueError("captured PNG dimensions do not match its frame geometry") + # Re-parse our own payload so a hand-built injected geometry cannot bypass + # the same exact validation used for HTTP input. + geometry = FrameGeometry.from_payload(frame.geometry.to_payload()) + return CapturedDesktopFrame(frame.png, geometry) + + +def _frame_geometry_payload(value: object) -> FrameGeometry: + try: + return FrameGeometry.from_payload(value) + except ValueError as exc: + raise AgentRequestError(400, "invalid_schema", str(exc)) from exc + + +def _require_per_monitor_v2() -> None: + """Make the current Windows agent thread use physical monitor pixels.""" + + if os.name != "nt": + raise RuntimeError("Per-Monitor-v2 desktop geometry requires Windows") + import ctypes # noqa: PLC0415 - Windows-only, lazy by design + + win_dll = getattr(ctypes, "WinDLL", None) + if win_dll is None: + raise RuntimeError("Windows DPI APIs are unavailable") + user32 = win_dll("user32", use_last_error=True) + setter = getattr(user32, "SetThreadDpiAwarenessContext", None) + getter = getattr(user32, "GetThreadDpiAwarenessContext", None) + equal = getattr(user32, "AreDpiAwarenessContextsEqual", None) + if setter is None or getter is None or equal is None: + raise RuntimeError("Windows Per-Monitor-v2 DPI APIs are unavailable") + context = ctypes.c_void_p(-4) # DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2 + setter.argtypes = [ctypes.c_void_p] + setter.restype = ctypes.c_void_p + getter.argtypes = [] + getter.restype = ctypes.c_void_p + equal.argtypes = [ctypes.c_void_p, ctypes.c_void_p] + equal.restype = ctypes.c_int + if not setter(context): + last_error = getattr(ctypes, "get_last_error", lambda: 0) + raise OSError(last_error(), "cannot enable Per-Monitor-v2 DPI") + if not equal(getter(), context): + raise RuntimeError("Windows thread did not enter Per-Monitor-v2 DPI mode") + + +def _current_frame_geometry() -> FrameGeometry: + """Read the exact Windows virtual desktop and per-monitor DPI topology.""" + + _require_per_monitor_v2() + import ctypes # noqa: PLC0415 - Windows-only, lazy by design + from ctypes import wintypes # noqa: PLC0415 + + win_dll = getattr(ctypes, "WinDLL", None) + if win_dll is None: + raise RuntimeError("Windows monitor APIs are unavailable") + user32 = win_dll("user32", use_last_error=True) + + class MonitorInfoExW(ctypes.Structure): + _fields_ = [ + ("cbSize", wintypes.DWORD), + ("rcMonitor", wintypes.RECT), + ("rcWork", wintypes.RECT), + ("dwFlags", wintypes.DWORD), + ("szDevice", wintypes.WCHAR * 32), + ] + + monitors: list[MonitorGeometry] = [] + callback_factory = getattr(ctypes, "WINFUNCTYPE", ctypes.CFUNCTYPE) + monitor_enum_proc = callback_factory( + wintypes.BOOL, + wintypes.HMONITOR, + wintypes.HDC, + ctypes.POINTER(wintypes.RECT), + wintypes.LPARAM, + ) + user32.GetMonitorInfoW.argtypes = [ + wintypes.HMONITOR, + ctypes.POINTER(MonitorInfoExW), + ] + user32.GetMonitorInfoW.restype = wintypes.BOOL + user32.EnumDisplayMonitors.argtypes = [ + wintypes.HDC, + ctypes.POINTER(wintypes.RECT), + monitor_enum_proc, + wintypes.LPARAM, + ] + user32.EnumDisplayMonitors.restype = wintypes.BOOL + shcore = None + try: + shcore = win_dll("shcore", use_last_error=True) + except OSError: + pass + + def callback( + monitor: Any, + _hdc: Any, + _rect: Any, + _data: Any, + ) -> bool: + info = MonitorInfoExW() + info.cbSize = ctypes.sizeof(info) + if not user32.GetMonitorInfoW(monitor, ctypes.byref(info)): + return False + dpi_x = wintypes.UINT(96) + dpi_y = wintypes.UINT(96) + if shcore is not None: + get_dpi = getattr(shcore, "GetDpiForMonitor", None) + if get_dpi is not None: + get_dpi.argtypes = [ + wintypes.HMONITOR, + ctypes.c_int, + ctypes.POINTER(wintypes.UINT), + ctypes.POINTER(wintypes.UINT), + ] + get_dpi.restype = ctypes.c_long + if get_dpi(monitor, 0, ctypes.byref(dpi_x), ctypes.byref(dpi_y)) != 0: + dpi_x.value = 96 + dpi_y.value = 96 + rect = info.rcMonitor + monitors.append( + MonitorGeometry( + device=str(info.szDevice), + left=int(rect.left), + top=int(rect.top), + width=int(rect.right - rect.left), + height=int(rect.bottom - rect.top), + dpi_x=int(dpi_x.value), + dpi_y=int(dpi_y.value), + primary=bool(info.dwFlags & 1), + ) + ) + return True + + callback_ref = monitor_enum_proc(callback) + if not user32.EnumDisplayMonitors(None, None, callback_ref, 0): + last_error = getattr(ctypes, "get_last_error", lambda: 0) + raise OSError(last_error(), "cannot enumerate Windows monitors") + if not monitors: + raise RuntimeError("Windows reported no active monitors") + get_metric = user32.GetSystemMetrics + get_metric.argtypes = [ctypes.c_int] + get_metric.restype = ctypes.c_int + geometry = FrameGeometry( + origin_x=int(get_metric(76)), # SM_XVIRTUALSCREEN + origin_y=int(get_metric(77)), # SM_YVIRTUALSCREEN + width=int(get_metric(78)), # SM_CXVIRTUALSCREEN + height=int(get_metric(79)), # SM_CYVIRTUALSCREEN + monitors=tuple(monitors), + ) + return FrameGeometry.from_payload(geometry.to_payload()) + + +def _normalize_virtual_point( + x: int, y: int, geometry: FrameGeometry +) -> tuple[int, int]: + """Normalize a physical virtual-desktop point for absolute SendInput.""" + + if not ( + geometry.origin_x <= x < geometry.origin_x + geometry.width + and geometry.origin_y <= y < geometry.origin_y + geometry.height + ): + raise ValueError("virtual input point is outside the captured desktop") + normalized_x = ( + 0 + if geometry.width == 1 + else round((x - geometry.origin_x) * 65535 / (geometry.width - 1)) + ) + normalized_y = ( + 0 + if geometry.height == 1 + else round((y - geometry.origin_y) * 65535 / (geometry.height - 1)) + ) + return normalized_x, normalized_y + + +def _send_virtual_pointer_sequence( + events: list[tuple[int, int, int]], geometry: FrameGeometry +) -> None: + """Send virtual-desktop-aware absolute pointer events through SendInput.""" + + if _current_frame_geometry() != geometry: + raise AgentRequestError( + 409, + "stale_geometry", + "virtual desktop geometry changed at the SendInput boundary", + ) + import ctypes # noqa: PLC0415 - Windows-only, lazy by design + from ctypes import wintypes # noqa: PLC0415 + + class MouseInput(ctypes.Structure): + _fields_ = [ + ("dx", wintypes.LONG), + ("dy", wintypes.LONG), + ("mouseData", wintypes.DWORD), + ("dwFlags", wintypes.DWORD), + ("time", wintypes.DWORD), + ("dwExtraInfo", ctypes.c_size_t), + ] + + class InputUnion(ctypes.Union): + _fields_ = [("mi", MouseInput)] + + class Input(ctypes.Structure): + _anonymous_ = ("union",) + _fields_ = [("type", wintypes.DWORD), ("union", InputUnion)] + + mouse_move = 0x0001 + mouse_absolute = 0x8000 + mouse_virtual_desktop = 0x4000 + base_flags = mouse_move | mouse_absolute | mouse_virtual_desktop + inputs: list[Input] = [] + for virtual_x, virtual_y, edge_flags in events: + normalized_x, normalized_y = _normalize_virtual_point( + virtual_x, virtual_y, geometry + ) + inputs.append( + Input( + type=0, # INPUT_MOUSE + mi=MouseInput( + dx=normalized_x, + dy=normalized_y, + mouseData=0, + dwFlags=base_flags | edge_flags, + time=0, + dwExtraInfo=0, + ), + ) + ) + if not inputs: + return + array_type = Input * len(inputs) + array = array_type(*inputs) + win_dll = getattr(ctypes, "WinDLL", None) + if win_dll is None: + raise RuntimeError("Windows SendInput is unavailable") + user32 = win_dll("user32", use_last_error=True) + user32.SendInput.argtypes = [wintypes.UINT, ctypes.POINTER(Input), ctypes.c_int] + user32.SendInput.restype = wintypes.UINT + sent = int(user32.SendInput(len(array), array, ctypes.sizeof(Input))) + if sent != len(array): + last_error = getattr(ctypes, "get_last_error", lambda: 0) + raise OSError(last_error(), f"SendInput delivered {sent}/{len(array)} events") + + def _delivery_receipt( operation: str, *, @@ -199,6 +683,7 @@ def _perform_input(payload: dict[str, Any]) -> dict[str, Any]: "keys", "horizontal_notches", "vertical_notches", + "frame_geometry", } ), label="input request", @@ -208,11 +693,12 @@ def _perform_input(payload: dict[str, Any]) -> dict[str, Any]: raise AgentRequestError(400, "unsupported_action", "unsupported input action") if action == "click": - expected = {"action", "x", "y", "double", "button"} - if set(data) - expected or not {"x", "y"}.issubset(data): + expected = {"action", "x", "y", "double", "button", "frame_geometry"} + if set(data) - expected or not {"x", "y", "frame_geometry"}.issubset(data): raise AgentRequestError(400, "invalid_schema", "invalid click fields") - x = _bounded_int(data["x"], "x") - y = _bounded_int(data["y"], "y") + frame_x = _bounded_int(data["x"], "x") + frame_y = _bounded_int(data["y"], "y") + geometry = _frame_geometry_payload(data["frame_geometry"]) double = data.get("double", False) if not isinstance(double, bool): raise AgentRequestError(400, "invalid_schema", "double must be boolean") @@ -225,13 +711,23 @@ def _perform_input(payload: dict[str, Any]) -> dict[str, Any]: raise AgentRequestError( 400, "invalid_schema", "double right click is unsupported" ) - import pyautogui # noqa: PLC0415 - Windows-only, lazy by design - - pyautogui.FAILSAFE = False + current_geometry = _current_frame_geometry() + if current_geometry != geometry: + raise AgentRequestError( + 409, + "stale_geometry", + "virtual desktop geometry changed before pointer input", + ) + try: + virtual_x, virtual_y = geometry.frame_to_virtual(frame_x, frame_y) + except ValueError as exc: + raise AgentRequestError(400, "invalid_schema", str(exc)) from exc + down = 0x0008 if button == "right" else 0x0002 + up = 0x0010 if button == "right" else 0x0004 + edges = [(virtual_x, virtual_y, down), (virtual_x, virtual_y, up)] if double: - pyautogui.doubleClick(x, y, button=button) - else: - pyautogui.click(x, y, button=button) + edges.extend([(virtual_x, virtual_y, down), (virtual_x, virtual_y, up)]) + _send_virtual_pointer_sequence(edges, geometry) return _delivery_receipt( "physical_double_click" if double @@ -240,24 +736,43 @@ def _perform_input(payload: dict[str, Any]) -> dict[str, Any]: ) if action == "drag": - if set(data) != {"action", "x", "y", "end_x", "end_y"}: + if set(data) != { + "action", + "x", + "y", + "end_x", + "end_y", + "frame_geometry", + }: raise AgentRequestError(400, "invalid_schema", "invalid drag fields") - x = _bounded_int(data["x"], "x") - y = _bounded_int(data["y"], "y") - end_x = _bounded_int(data["end_x"], "end_x") - end_y = _bounded_int(data["end_y"], "end_y") - import pyautogui # noqa: PLC0415 - Windows-only, lazy by design - - pyautogui.FAILSAFE = False - down_sent = False + frame_x = _bounded_int(data["x"], "x") + frame_y = _bounded_int(data["y"], "y") + frame_end_x = _bounded_int(data["end_x"], "end_x") + frame_end_y = _bounded_int(data["end_y"], "end_y") + geometry = _frame_geometry_payload(data["frame_geometry"]) + current_geometry = _current_frame_geometry() + if current_geometry != geometry: + raise AgentRequestError( + 409, + "stale_geometry", + "virtual desktop geometry changed before pointer input", + ) try: - pyautogui.moveTo(x, y) - pyautogui.mouseDown(button="left") - down_sent = True - pyautogui.moveTo(end_x, end_y, duration=0.2) - finally: - if down_sent: - pyautogui.mouseUp(button="left") + virtual_x, virtual_y = geometry.frame_to_virtual(frame_x, frame_y) + virtual_end_x, virtual_end_y = geometry.frame_to_virtual( + frame_end_x, frame_end_y + ) + except ValueError as exc: + raise AgentRequestError(400, "invalid_schema", str(exc)) from exc + _send_virtual_pointer_sequence( + [ + (virtual_x, virtual_y, 0), + (virtual_x, virtual_y, 0x0002), + (virtual_end_x, virtual_end_y, 0), + (virtual_end_x, virtual_end_y, 0x0004), + ], + geometry, + ) return _delivery_receipt("physical_drag", native=False) if action == "type_text": @@ -804,23 +1319,40 @@ def __post_init__(self) -> None: ) -def _grab_desktop_png() -> bytes: - """Capture the full virtual desktop as PNG bytes (mss + Pillow). +def _grab_desktop_png() -> CapturedDesktopFrame: + """Capture one frame with its exact physical virtual-desktop topology.""" - Imported lazily and only on the screenshot path so the module loads on any - OS. ``monitors[0]`` is the union of all monitors, so multi-monitor / DPI - layouts are captured whole with absolute coordinates. - """ import mss # noqa: PLC0415 - Windows-only, imported lazily by design from PIL import Image # noqa: PLC0415 + before = _current_frame_geometry() with mss.mss() as sct: mon = sct.monitors[0] + mss_geometry = ( + int(mon["left"]), + int(mon["top"]), + int(mon["width"]), + int(mon["height"]), + ) + expected_geometry = ( + before.origin_x, + before.origin_y, + before.width, + before.height, + ) + if mss_geometry != expected_geometry: + raise RuntimeError( + "mss virtual desktop does not match the Per-Monitor-v2 topology" + ) raw = sct.grab(mon) img = Image.frombytes("RGB", raw.size, raw.rgb) + after = _current_frame_geometry() + if after != before: + raise RuntimeError("monitor topology changed during desktop capture") buf = io.BytesIO() img.save(buf, format="PNG") - return buf.getvalue() + frame = CapturedDesktopFrame(buf.getvalue(), before) + return _coerce_captured_frame(frame) def _active_console_session() -> int: @@ -858,8 +1390,8 @@ def _bounded_application_identifier(name: str) -> Optional[str]: return identifier -def _foreground_application_identity() -> Optional[str]: - """Observe the foreground executable without reading its window title.""" +def _foreground_window_identity() -> Optional[dict[str, Any]]: + """Observe one PHI-free exact foreground-window/process identity.""" if os.name != "nt": return None @@ -892,6 +1424,14 @@ def _foreground_application_identity() -> Optional[str]: ctypes.POINTER(wintypes.DWORD), ] kernel32.QueryFullProcessImageNameW.restype = wintypes.BOOL + kernel32.GetProcessTimes.argtypes = [ + wintypes.HANDLE, + ctypes.POINTER(wintypes.FILETIME), + ctypes.POINTER(wintypes.FILETIME), + ctypes.POINTER(wintypes.FILETIME), + ctypes.POINTER(wintypes.FILETIME), + ] + kernel32.GetProcessTimes.restype = wintypes.BOOL kernel32.CloseHandle.argtypes = [wintypes.HANDLE] kernel32.CloseHandle.restype = wintypes.BOOL hwnd = user32.GetForegroundWindow() @@ -913,13 +1453,44 @@ def _foreground_application_identity() -> Optional[str]: process, 0, path, ctypes.byref(size) ): return None - return _bounded_application_identifier(path.value) + owner = _bounded_application_identifier(path.value) + if owner is None: + return None + created = wintypes.FILETIME() + exited = wintypes.FILETIME() + kernel = wintypes.FILETIME() + user = wintypes.FILETIME() + if not kernel32.GetProcessTimes( + process, + ctypes.byref(created), + ctypes.byref(exited), + ctypes.byref(kernel), + ctypes.byref(user), + ): + return None + creation_ticks = (int(created.dwHighDateTime) << 32) | int( + created.dwLowDateTime + ) + window_value = getattr(hwnd, "value", hwnd) + return { + "window_id": str(int(window_value)), + "pid": int(pid.value), + "process_start_time": str(creation_ticks), + "owner": owner, + } finally: kernel32.CloseHandle(process) except Exception: # noqa: BLE001 - unavailable means unverifiable return None +def _foreground_application_identity() -> Optional[str]: + """Observe the foreground executable without reading its window title.""" + + window = _foreground_window_identity() + return str(window["owner"]) if window is not None else None + + def _native_session_digest() -> Optional[str]: """Hash the live machine + interactive Windows logon-session identity.""" @@ -1025,11 +1596,13 @@ class TokenStatistics(ctypes.Structure): def _execution_context_identity() -> dict[str, Any]: """Return PHI-free identities observed from live Windows OS state.""" + window = _foreground_window_identity() return { "status": "ok", - "application": _foreground_application_identity(), + "application": window["owner"] if window is not None else None, "session": _native_session_digest(), "workflow_state": None, + "window": window, } @@ -1054,10 +1627,19 @@ def log_message(self, *args: object) -> None: # noqa: D401 - silence # -- helpers --------------------------------------------------------- - def _send(self, status: int, body: bytes, ctype: str) -> None: + def _send( + self, + status: int, + body: bytes, + ctype: str, + *, + headers: Optional[dict[str, str]] = None, + ) -> None: self.send_response(status) self.send_header("Content-Type", ctype) self.send_header("Content-Length", str(len(body))) + for name, value in (headers or {}).items(): + self.send_header(name, value) self.end_headers() if self.command != "HEAD": self.wfile.write(body) @@ -1079,6 +1661,150 @@ def _authorized(self) -> bool: def _reject_unauthorized(self) -> None: self._send_json(401, {"status": "error", "error": "unauthorized"}) + @staticmethod + def _capture_frame() -> CapturedDesktopFrame: + return _coerce_captured_frame(grab_fn()) + + @staticmethod + def _expected_geometry(value: object) -> FrameGeometry: + return _frame_geometry_payload(value) + + @staticmethod + def _require_geometry_match( + expected: FrameGeometry, current: FrameGeometry + ) -> None: + if expected != current: + raise AgentRequestError( + 409, + "stale_geometry", + "virtual desktop geometry changed after frame capture", + ) + + def _prepare_direct_input(self, data: dict[str, Any]) -> dict[str, Any]: + action = data.get("action") + if action not in {"click", "drag"}: + return data + expected_value = data.get("expected_frame_geometry") + expected_frame = data.get("expected_frame_sha256") + if expected_value is None or expected_frame is None: + raise AgentRequestError( + 400, + "invalid_schema", + "coordinate input requires an exact frame and geometry", + ) + if ( + not isinstance(expected_frame, str) + or len(expected_frame) != 64 + or any(char not in "0123456789abcdef" for char in expected_frame) + ): + raise AgentRequestError( + 400, + "invalid_schema", + "expected_frame_sha256 must be lowercase SHA-256", + ) + expected = self._expected_geometry(expected_value) + allowed = set(data) - { + "expected_frame_geometry", + "expected_frame_sha256", + } + prepared = {key: data[key] for key in allowed} + current = self._capture_frame() + if not hmac.compare_digest( + hashlib.sha256(current.png).hexdigest(), expected_frame + ): + raise AgentRequestError( + 409, "stale_frame", "desktop frame changed before pointer input" + ) + self._require_geometry_match(expected, current.geometry) + prepared["frame_geometry"] = current.geometry.to_payload() + return prepared + + def _prepare_uia( + self, operation: str, data: dict[str, Any] + ) -> tuple[dict[str, Any], Optional[FrameGeometry]]: + if operation == "act": + return data, None + geometry_value = data.get("frame_geometry") + if geometry_value is None: + raise AgentRequestError( + 400, + "invalid_schema", + "UIA frame operation requires frame_geometry", + ) + expected = self._expected_geometry(geometry_value) + current = self._capture_frame().geometry + self._require_geometry_match(expected, current) + prepared = { + key: value for key, value in data.items() if key != "frame_geometry" + } + if operation in {"locator-at", "text-at-point", "focused-at-point"}: + frame_x = _bounded_int(prepared.get("x"), "x") + frame_y = _bounded_int(prepared.get("y"), "y") + try: + virtual_x, virtual_y = current.frame_to_virtual(frame_x, frame_y) + except ValueError as exc: + raise AgentRequestError(400, "invalid_schema", str(exc)) from exc + prepared["x"] = virtual_x + prepared["y"] = virtual_y + return prepared, current + + @staticmethod + def _map_uia_result( + operation: str, + result: dict[str, Any], + geometry: Optional[FrameGeometry], + ) -> dict[str, Any]: + if operation != "find" or geometry is None: + return result + candidates = result.get("candidates") + if not isinstance(candidates, list): + return result + mapped_candidates: list[object] = [] + for candidate in candidates: + if not isinstance(candidate, dict): + mapped_candidates.append(candidate) + continue + mapped = dict(candidate) + point = candidate.get("point") + bounds = candidate.get("bounds") + try: + if ( + isinstance(point, list) + and len(point) == 2 + and all( + isinstance(item, int) and not isinstance(item, bool) + for item in point + ) + ): + frame_point = geometry.virtual_to_frame(point[0], point[1]) + mapped["point"] = [frame_point[0], frame_point[1]] + if ( + isinstance(bounds, list) + and len(bounds) == 4 + and all( + isinstance(item, int) and not isinstance(item, bool) + for item in bounds + ) + ): + left = bounds[0] - geometry.origin_x + top = bounds[1] - geometry.origin_y + right = bounds[2] - geometry.origin_x + bottom = bounds[3] - geometry.origin_y + if not ( + 0 <= left < right <= geometry.width + and 0 <= top < bottom <= geometry.height + ): + raise ValueError( + "UIA bounds are outside the captured frame" + ) + mapped["bounds"] = [left, top, right, bottom] + except ValueError as exc: + raise AgentRequestError( + 409, "stale_geometry", "UIA geometry is outside the frame" + ) from exc + mapped_candidates.append(mapped) + return {**result, "candidates": mapped_candidates} + # -- routes ---------------------------------------------------------- def do_GET(self) -> None: # noqa: N802 - stdlib naming @@ -1095,6 +1821,8 @@ def do_GET(self) -> None: # noqa: N802 - stdlib naming "context_identity_v1", "typed_input_v1", "guarded_input_v1", + "frame_geometry_v1", + "frame_observation_v1", "uia_v1", *(["legacy_exec"] if config.allow_legacy_exec else []), ], @@ -1106,7 +1834,7 @@ def do_GET(self) -> None: # noqa: N802 - stdlib naming self._reject_unauthorized() return try: - png = grab_fn() + frame = self._capture_frame() except Exception as e: # noqa: BLE001 - report, never crash loop self._send_json( 500, @@ -1117,12 +1845,19 @@ def do_GET(self) -> None: # noqa: N802 - stdlib naming }, ) return - if not png.startswith(_PNG_SIGNATURE): - self._send_json( - 500, {"status": "error", "error": "grabber did not return PNG"} - ) - return - self._send(200, png, "image/png") + self._send( + 200, + frame.png, + "image/png", + headers={ + _FRAME_GEOMETRY_HEADER: encode_frame_geometry_header( + frame.geometry + ), + _FRAME_BINDING_HEADER: frame_binding_sha256( + frame.png, frame.geometry + ), + }, + ) return self._send_json(404, {"status": "error", "error": "not found"}) @@ -1202,6 +1937,7 @@ def do_POST(self) -> None: # noqa: N802 - stdlib naming required=frozenset( { "expected_frame_sha256", + "expected_frame_geometry", "expected_context", "input", } @@ -1223,6 +1959,9 @@ def do_POST(self) -> None: # noqa: N802 - stdlib naming "invalid_schema", "expected_frame_sha256 must be lowercase SHA-256", ) + expected_geometry = self._expected_geometry( + guarded["expected_frame_geometry"] + ) expected_context = _exact_object( guarded["expected_context"], required=frozenset( @@ -1259,15 +1998,16 @@ def do_POST(self) -> None: # noqa: N802 - stdlib naming ), label="guarded input payload", ) - current_png = grab_fn() - if not current_png.startswith(_PNG_SIGNATURE): + try: + current_frame = self._capture_frame() + except Exception as exc: raise AgentRequestError( 503, "capture_unavailable", "guarded frame capture is unavailable", - ) + ) from exc if not hmac.compare_digest( - hashlib.sha256(current_png).hexdigest(), + hashlib.sha256(current_frame.png).hexdigest(), expected_frame, ): raise AgentRequestError( @@ -1275,6 +2015,9 @@ def do_POST(self) -> None: # noqa: N802 - stdlib naming "stale_frame", "desktop frame changed after identity verification", ) + self._require_geometry_match( + expected_geometry, current_frame.geometry + ) current_context = context_fn() for key, expected in expected_context.items(): if current_context.get(key) != expected: @@ -1292,11 +2035,20 @@ def do_POST(self) -> None: # noqa: N802 - stdlib naming required=frozenset({"x", "y"}), label="focus_point", ) + try: + virtual_focus = current_frame.geometry.frame_to_virtual( + _bounded_int(focus["x"], "focus_point.x"), + _bounded_int(focus["y"], "focus_point.y"), + ) + except ValueError as exc: + raise AgentRequestError( + 400, "invalid_schema", str(exc) + ) from exc focus_result = uia_fn( "focused-at-point", { - "x": _bounded_int(focus["x"], "focus_point.x"), - "y": _bounded_int(focus["y"], "focus_point.y"), + "x": virtual_focus[0], + "y": virtual_focus[1], }, ) if focus_result.get("focused") is not True: @@ -1312,15 +2064,24 @@ def do_POST(self) -> None: # noqa: N802 - stdlib naming "invalid_schema", "focus_point is only valid for keyboard input", ) - result = input_fn(input_payload) - else: - result = ( - input_fn(data) - if kind == "input" - else context_fn() - if kind == "context" - else uia_fn(operation, data) + result = input_fn( + { + **input_payload, + "frame_geometry": current_frame.geometry.to_payload(), + } + if action in {"click", "drag"} + else input_payload ) + else: + if kind == "input": + result = input_fn(self._prepare_direct_input(data)) + elif kind == "context": + result = context_fn() + else: + prepared, geometry = self._prepare_uia(operation, data) + result = self._map_uia_result( + operation, uia_fn(operation, prepared), geometry + ) except AgentRequestError as exc: self._send_json( exc.status, diff --git a/openadapt_flow/backends/windows_backend.py b/openadapt_flow/backends/windows_backend.py index 9d3aab0c..838fbc43 100644 --- a/openadapt_flow/backends/windows_backend.py +++ b/openadapt_flow/backends/windows_backend.py @@ -36,16 +36,34 @@ from __future__ import annotations import base64 +import hashlib +import hmac import re import struct +import threading import warnings from dataclasses import dataclass -from typing import Callable, Optional +from typing import Callable, NoReturn, Optional from urllib.parse import urlparse import requests -from openadapt_flow.backend import ActionDeliveryUncertain, StructuralResolutionRefused +from openadapt_flow.backend import ( + ActionDeliveryUncertain, + DisplayGeometry, + DisplayTopologyChanged, + FrameObservation, + FreshActuationRequired, + StructuralResolutionRefused, + display_topology_sha256, + session_identity_sha256, + window_identity_sha256, +) +from openadapt_flow.backends.win_agent.server import ( + FrameGeometry, + decode_frame_geometry_header, + frame_binding_sha256, +) from openadapt_flow.ir import ( ActionDeliveryReceipt, StructuralHandle, @@ -70,6 +88,8 @@ SCROLL_PIXELS_PER_NOTCH = 100 _PNG_SIGNATURE = b"\x89PNG\r\n\x1a\n" +_FRAME_GEOMETRY_HEADER = "X-OpenAdapt-Frame-Geometry" +_FRAME_BINDING_HEADER = "X-OpenAdapt-Frame-Binding-SHA256" _CONTEXT_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:/-]{0,127}$") _SESSION_ID_RE = re.compile(r"^[a-f0-9]{64}$") @@ -82,10 +102,27 @@ class _TypedResponse: payload: Optional[dict] +@dataclass(frozen=True) +class _FrameBinding: + """The geometry that arrived with one exact screenshot response.""" + + frame_sha256: str + geometry: FrameGeometry + png: bytes + + class _TypedRouteUnavailable(RuntimeError): """An explicitly legacy-enabled dev agent lacks the typed endpoint.""" +class _WindowsInputChanged(RuntimeError): + """The typed agent proved that no edge crossed on a stale input lease.""" + + def __init__(self, code: str, message: str) -> None: + super().__init__(message) + self.code = code + + # Playwright-style modifier names (as recorded / emitted by the replayer, # e.g. 'ControlOrMeta+a') -> pyautogui key names. On Windows the Meta/Command # key maps to the Windows key and ControlOrMeta resolves to Ctrl. @@ -229,11 +266,16 @@ def __init__( self._auth_token = auth_token or None self._pin_fingerprint = pin_fingerprint or None self._allow_legacy_exec = bool(allow_legacy_exec) + self._observation_lock = threading.RLock() + self._frame_binding: Optional[_FrameBinding] = None + self._last_frame_observation: Optional[FrameObservation] = None + self._actuation_observation: Optional[FrameObservation] = None + self._bound_input_observation: Optional[FrameObservation] = None self._guarded_coordinate: Optional[ - tuple[tuple[int, int], dict[str, Optional[str]]] + tuple[tuple[int, int], dict[str, Optional[str]], _FrameBinding] ] = None self._guarded_keyboard: Optional[ - tuple[tuple[int, int], dict[str, Optional[str]]] + tuple[tuple[int, int], dict[str, Optional[str]], _FrameBinding] ] = None scheme = urlparse(self.server_url).scheme.lower() @@ -302,11 +344,84 @@ def _request_kwargs(self) -> dict: @property def viewport(self) -> tuple[int, int]: - """(width, height) of the VM screen, derived from a screenshot.""" + """Current captured virtual-desktop size in physical frame pixels.""" if self._viewport is None: - self._viewport = _png_size(self.screenshot()) + self.screenshot() + assert self._viewport is not None return self._viewport + @staticmethod + def _legacy_frame_geometry(png: bytes) -> FrameGeometry: + width, height = _png_size(png) + return FrameGeometry.from_payload( + { + "version": 1, + "coordinate_space": "physical_virtual_desktop", + "dpi_awareness": "per_monitor_v2", + "origin_x": 0, + "origin_y": 0, + "width": width, + "height": height, + "monitors": [ + { + "device": "DISPLAY1", + "left": 0, + "top": 0, + "width": width, + "height": height, + "dpi_x": 96, + "dpi_y": 96, + "primary": True, + } + ], + } + ) + + def _bind_screenshot(self, response: requests.Response, png: bytes) -> None: + size = _png_size(png) + headers = getattr(response, "headers", {}) + encoded_geometry = headers.get(_FRAME_GEOMETRY_HEADER) + expected_binding = headers.get(_FRAME_BINDING_HEADER) + if encoded_geometry is None and expected_binding is None: + if not self._allow_legacy_exec: + raise RuntimeError( + "screenshot omitted its virtual-desktop frame geometry" + ) + geometry = self._legacy_frame_geometry(png) + else: + if encoded_geometry is None or expected_binding is None: + raise RuntimeError("screenshot frame binding is incomplete") + geometry = decode_frame_geometry_header(encoded_geometry) + if ( + not isinstance(expected_binding, str) + or len(expected_binding) != 64 + or not hmac.compare_digest( + expected_binding, frame_binding_sha256(png, geometry) + ) + ): + raise RuntimeError("screenshot frame/geometry binding is invalid") + if size != (geometry.width, geometry.height): + raise RuntimeError( + "screenshot dimensions do not match its virtual-desktop geometry" + ) + self._viewport = size + self._frame_binding = _FrameBinding( + frame_sha256=hashlib.sha256(png).hexdigest(), + geometry=geometry, + png=bytes(png), + ) + + def _require_frame_binding(self) -> _FrameBinding: + if self._frame_binding is None: + try: + self.screenshot() + except Exception as exc: + raise StructuralResolutionRefused( + "Windows input has no bound virtual-desktop frame geometry" + ) from exc + assert self._frame_binding is not None + return self._frame_binding + def screenshot(self) -> bytes: """Return the current frame as PNG bytes (with retries). @@ -330,7 +445,7 @@ def screenshot(self) -> bytes: f"screenshot HTTP {resp.status_code}: {resp.text[:200]}" ) png = resp.content - _png_size(png) # validates signature and header + self._bind_screenshot(resp, png) return png except Exception as e: # noqa: BLE001 - retried, then re-raised last_error = e @@ -340,6 +455,218 @@ def screenshot(self) -> bytes: f"screenshot failed after {self._screenshot_max_retries} attempts" ) from last_error + @staticmethod + def _display_geometries(geometry: FrameGeometry) -> tuple[DisplayGeometry, ...]: + return tuple( + DisplayGeometry( + display_id=monitor.device, + bounds=( + float(monitor.left), + float(monitor.top), + float(monitor.width), + float(monitor.height), + ), + scale=(monitor.dpi_x / 96.0, monitor.dpi_y / 96.0), + ) + for monitor in geometry.monitors + ) + + def _frame_identity_context(self) -> dict: + def legacy_context() -> dict: + return { + "application": "windows-legacy-agent", + "session": hashlib.sha256( + f"windows-legacy-session\0{self.server_url}".encode("utf-8") + ).hexdigest(), + "workflow_state": None, + "window": { + "window_id": "windows-virtual-desktop", + "pid": 0, + "process_start_time": None, + "owner": "windows-legacy-agent", + }, + } + + payload = self._execution_context() + if payload is None: + if not self._allow_legacy_exec: + raise StructuralResolutionRefused( + "Windows frame observation has no live execution identity" + ) + return legacy_context() + session = payload.get("session") + window = payload.get("window") + if not isinstance(session, str) or not _SESSION_ID_RE.fullmatch(session): + if self._allow_legacy_exec: + return legacy_context() + raise StructuralResolutionRefused( + "Windows frame observation has no exact logon-session identity" + ) + if not isinstance(window, dict) or set(window) != { + "window_id", + "pid", + "process_start_time", + "owner", + }: + if self._allow_legacy_exec: + return legacy_context() + raise StructuralResolutionRefused( + "Windows frame observation has no exact foreground-window identity" + ) + window_id = window.get("window_id") + pid = window.get("pid") + process_start_time = window.get("process_start_time") + owner = window.get("owner") + if ( + not isinstance(window_id, str) + or not window_id + or len(window_id) > 128 + or isinstance(pid, bool) + or not isinstance(pid, int) + or pid <= 0 + or not isinstance(process_start_time, str) + or not process_start_time + or len(process_start_time) > 128 + or not isinstance(owner, str) + or not _CONTEXT_ID_RE.fullmatch(owner) + or payload.get("application") != owner + ): + if self._allow_legacy_exec: + return legacy_context() + raise StructuralResolutionRefused( + "Windows foreground-window identity is invalid" + ) + return { + "application": owner, + "session": session, + "workflow_state": payload.get("workflow_state"), + "window": { + "window_id": window_id, + "pid": pid, + "process_start_time": process_start_time, + "owner": owner, + }, + } + + def _observation_from_binding( + self, + binding: _FrameBinding, + context: dict, + ) -> FrameObservation: + geometry = binding.geometry + displays = self._display_geometries(geometry) + window = context["window"] + return FrameObservation.create( + binding.png, + origin=(float(geometry.origin_x), float(geometry.origin_y)), + scale=(1.0, 1.0), + device_pixel_ratio=1.0, + display_id="windows-virtual-desktop", + display_bounds=( + float(geometry.origin_x), + float(geometry.origin_y), + float(geometry.width), + float(geometry.height), + ), + display_scale=(1.0, 1.0), + topology_sha256=display_topology_sha256( + displays, + coordinate_space=( + "windows-physical-virtual-desktop-per-monitor-v2" + ), + ), + window_identity_sha256=window_identity_sha256( + window_id=window["window_id"], + pid=window["pid"], + process_start_time=window["process_start_time"], + owner=window["owner"], + ), + session_identity_sha256=session_identity_sha256( + authority="windows-native-session-digest-v1", + session_id=context["session"], + session_start_time=None, + principal_identity_sha256=None, + ), + ) + + def observe_frame(self) -> FrameObservation: + """Capture one frame between two equal exact OS identity samples.""" + + with self._observation_lock: + last_error: Optional[BaseException] = None + for _attempt in range(self._screenshot_max_retries): + try: + before = self._frame_identity_context() + self.screenshot() + binding = self._require_frame_binding() + after = self._frame_identity_context() + if before != after: + last_error = StructuralResolutionRefused( + "Windows foreground window or session changed during " + "desktop capture" + ) + continue + observation = self._observation_from_binding(binding, before) + self._last_frame_observation = observation + return observation + except StructuralResolutionRefused as exc: + last_error = exc + raise StructuralResolutionRefused( + "Windows could not retain a stable atomic frame observation" + ) from last_error + + @property + def last_frame_observation(self) -> Optional[FrameObservation]: + return self._last_frame_observation + + def acquire_actuation_observation(self) -> FrameObservation: + observation = self.observe_frame() + self._actuation_observation = observation + self._bound_input_observation = None + return observation + + def bind_input_observation(self, observation: FrameObservation) -> None: + acquired = self._actuation_observation + binding = self._require_frame_binding() + if ( + acquired is None + or acquired.frame_sha256 != observation.frame_sha256 + or acquired.geometry_epoch != observation.geometry_epoch + or binding.frame_sha256 != observation.frame_sha256 + ): + self._bound_input_observation = None + raise StructuralResolutionRefused( + "Windows input observation does not match its exact frame lease" + ) + self._bound_input_observation = observation + + def reset_fresh_actuation_state(self) -> None: + self._actuation_observation = None + self._bound_input_observation = None + self.cancel_guarded_coordinate() + self.cancel_guarded_keyboard() + + def _raise_fresh_input_required( + self, + *, + expected: FrameObservation, + operation: str, + ) -> NoReturn: + observed = self.observe_frame() + if observed.topology_sha256 != expected.topology_sha256: + raise DisplayTopologyChanged( + expected_observation=expected, + observed_observation=observed, + ) + raise FreshActuationRequired( + operation=operation, + changed_pixel_count=observed.viewport[0] * observed.viewport[1], + changed_bbox=(0, 0, observed.viewport[0], observed.viewport[1]), + frame_size=observed.viewport, + expected_observation=expected, + observed_observation=observed, + ) + def _post_typed_read(self, path: str, payload: dict) -> _TypedResponse: """POST a typed observation without turning absence into an action. @@ -441,10 +768,19 @@ def _post_typed_action(self, path: str, payload: dict) -> dict: f"UIA actuation refused ({code}): {message or 'target changed'}" ) if ( - path == "/input/guarded" + path in {"/input", "/input/guarded"} and response.status_code == 409 - and code in {"stale_context", "stale_focus", "stale_frame"} + and code + in {"stale_context", "stale_focus", "stale_frame", "stale_geometry"} ): + if path == "/input/guarded" and code in { + "stale_frame", + "stale_geometry", + }: + raise _WindowsInputChanged( + code, + message or "the guarded Windows input lease changed", + ) raise StructuralResolutionRefused( "guarded Windows input refused " f"({code}): {message or 'identity binding changed'}" @@ -573,6 +909,23 @@ def workflow_state_identity(self) -> Optional[str]: return None + def _geometry_payload(self) -> dict: + return self._require_frame_binding().geometry.to_payload() + + def _frame_to_virtual_point(self, x: int, y: int) -> tuple[int, int]: + binding = self._require_frame_binding() + try: + return binding.geometry.frame_to_virtual(int(x), int(y)) + except ValueError as exc: + raise StructuralResolutionRefused(str(exc)) from exc + + def _virtual_to_frame_point(self, x: int, y: int) -> tuple[int, int]: + binding = self._require_frame_binding() + try: + return binding.geometry.virtual_to_frame(int(x), int(y)) + except ValueError as exc: + raise StructuralResolutionRefused(str(exc)) from exc + # -- structured-text identity (openadapt_flow.backend.IdentityBackend) -- def structured_text_at(self, x: int, y: int) -> Optional[str]: @@ -597,13 +950,21 @@ def structured_text_at(self, x: int, y: int) -> Optional[str]: unavailable, or when nothing is under the point (never raises) -- the identity ladder then falls back to the OCR tier. """ - typed = self._post_typed_read("/uia/text-at-point", {"x": int(x), "y": int(y)}) + try: + geometry = self._geometry_payload() + except Exception: + return None + typed = self._post_typed_read( + "/uia/text-at-point", + {"x": int(x), "y": int(y), "frame_geometry": geometry}, + ) if typed.available: value = typed.payload.get("text") if typed.payload is not None else None return str(value) if isinstance(value, str) and value else None if not self._allow_legacy_exec: return None + virtual_x, virtual_y = self._frame_to_virtual_point(x, y) snippet = ( "import json\n" "def _oaflow_structured_text_at(px, py):\n" @@ -664,7 +1025,7 @@ def structured_text_at(self, x: int, y: int) -> Optional[str]: " text = ' '.join(parts).split()\n" " return ' '.join(text) if text else None\n" "print('<>' + json.dumps(" - f"_oaflow_structured_text_at({int(x)}, {int(y)})) " + f"_oaflow_structured_text_at({virtual_x}, {virtual_y})) " "+ '<>')\n" ) body = self._execute_read(snippet) @@ -758,7 +1119,14 @@ def structural_locator_at(self, x: int, y: int) -> Optional[StructuralLocator]: unavailable, or the WAA server does not echo output (never raises) -- the step then relies on the visual anchor. """ - typed = self._post_typed_read("/uia/locator-at", {"x": int(x), "y": int(y)}) + try: + geometry = self._geometry_payload() + except Exception: + return None + typed = self._post_typed_read( + "/uia/locator-at", + {"x": int(x), "y": int(y), "frame_geometry": geometry}, + ) if typed.available: value = typed.payload.get("locator") if typed.payload is not None else None if not isinstance(value, dict): @@ -770,6 +1138,7 @@ def structural_locator_at(self, x: int, y: int) -> Optional[StructuralLocator]: if not self._allow_legacy_exec: return None + virtual_x, virtual_y = self._frame_to_virtual_point(x, y) snippet = ( "import json\n" "def _oaflow_locator_at(px, py):\n" @@ -824,7 +1193,7 @@ def structural_locator_at(self, x: int, y: int) -> Optional[StructuralLocator]: " return None\n" " return {'automation_id': aid, 'role': role, 'name': name}\n" "print('<>' + json.dumps(" - f"_oaflow_locator_at({int(x)}, {int(y)})) " + f"_oaflow_locator_at({virtual_x}, {virtual_y})) " "+ '<>')\n" ) value = self._read_structured_json(snippet) @@ -853,9 +1222,16 @@ def locate_structural( name = locator.name or "" if not aid and not (role and name): return None + try: + geometry = self._geometry_payload() + except Exception: + return None typed = self._post_typed_read( "/uia/find", - {"locator": locator.model_dump(mode="json", exclude_none=True)}, + { + "locator": locator.model_dump(mode="json", exclude_none=True), + "frame_geometry": geometry, + }, ) if typed.available: payload = typed.payload @@ -976,7 +1352,8 @@ def locate_structural( or not all(isinstance(v, (int, float)) for v in value) ): return None - return StructuralHandle(point=(int(value[0]), int(value[1]))) + point = self._virtual_to_frame_point(int(value[0]), int(value[1])) + return StructuralHandle(point=point) def act_structural( self, @@ -1027,12 +1404,13 @@ def arm_guarded_coordinate(self, x: int, y: int) -> None: self.cancel_guarded_coordinate() point = (int(x), int(y)) - width, height = self.viewport + binding = self._require_frame_binding() + width, height = binding.geometry.width, binding.geometry.height if not (0 <= point[0] < width and 0 <= point[1] < height): raise StructuralResolutionRefused( "Windows guarded coordinate is outside the captured viewport" ) - self._guarded_coordinate = (point, self._guard_context()) + self._guarded_coordinate = (point, self._guard_context(), binding) def cancel_guarded_coordinate(self) -> None: self._guarded_coordinate = None @@ -1045,12 +1423,13 @@ def arm_guarded_keyboard(self, x: int, y: int) -> None: self.cancel_guarded_keyboard() point = (int(x), int(y)) - width, height = self.viewport + binding = self._require_frame_binding() + width, height = binding.geometry.width, binding.geometry.height if not (0 <= point[0] < width and 0 <= point[1] < height): raise StructuralResolutionRefused( "Windows guarded keyboard point is outside the captured viewport" ) - self._guarded_keyboard = (point, self._guard_context()) + self._guarded_keyboard = (point, self._guard_context(), binding) def cancel_guarded_keyboard(self) -> None: self._guarded_keyboard = None @@ -1081,6 +1460,22 @@ def _guarded_receipt( ) return receipt + def _consume_bound_input_observation( + self, + binding: _FrameBinding, + ) -> FrameObservation: + observation = self._bound_input_observation + self._bound_input_observation = None + self._actuation_observation = None + if ( + observation is None + or observation.frame_sha256 != binding.frame_sha256 + ): + raise StructuralResolutionRefused( + "Windows guarded input has no bound atomic frame observation" + ) + return observation + def act_guarded_coordinate( self, x: int, @@ -1096,11 +1491,15 @@ def act_guarded_coordinate( raise StructuralResolutionRefused( "Windows coordinate actuation has no pre-identity binding" ) - point, context = pending + point, context, binding = pending if point != (int(x), int(y)): raise StructuralResolutionRefused( "Windows coordinate target changed after identity verification" ) + if not hmac.compare_digest(binding.frame_sha256, expected_frame_sha256): + raise StructuralResolutionRefused( + "Windows coordinate frame changed after identity verification" + ) if button not in {"left", "right"}: raise StructuralResolutionRefused( f"unsupported Windows pointer button {button!r}" @@ -1109,25 +1508,33 @@ def act_guarded_coordinate( raise StructuralResolutionRefused( "Windows guarded right-button double click is unsupported" ) + observation = self._consume_bound_input_observation(binding) operation = ( "physical_double_click" if double else ("physical_right_click" if button == "right" else "physical_click") ) - response = self._post_typed_action( - "/input/guarded", - { - "expected_frame_sha256": expected_frame_sha256, - "expected_context": context, - "input": { - "action": "click", - "x": point[0], - "y": point[1], - "double": bool(double), - "button": button, + try: + response = self._post_typed_action( + "/input/guarded", + { + "expected_frame_sha256": expected_frame_sha256, + "expected_frame_geometry": binding.geometry.to_payload(), + "expected_context": context, + "input": { + "action": "click", + "x": point[0], + "y": point[1], + "double": bool(double), + "button": button, + }, }, - }, - ) + ) + except _WindowsInputChanged: + self._raise_fresh_input_required( + expected=observation, + operation=operation, + ) return self._guarded_receipt(response, operation) def drag_guarded( @@ -1145,30 +1552,42 @@ def drag_guarded( raise StructuralResolutionRefused( "Windows drag has no pre-identity source binding" ) - point, context = pending + point, context, binding = pending if point != (int(x), int(y)): raise StructuralResolutionRefused( "Windows drag source changed after identity verification" ) - width, height = self.viewport + if not hmac.compare_digest(binding.frame_sha256, expected_frame_sha256): + raise StructuralResolutionRefused( + "Windows drag frame changed after identity verification" + ) + width, height = binding.geometry.width, binding.geometry.height if not (0 <= int(end_x) < width and 0 <= int(end_y) < height): raise StructuralResolutionRefused( "Windows drag destination is outside the captured viewport" ) - response = self._post_typed_action( - "/input/guarded", - { - "expected_frame_sha256": expected_frame_sha256, - "expected_context": context, - "input": { - "action": "drag", - "x": point[0], - "y": point[1], - "end_x": int(end_x), - "end_y": int(end_y), + observation = self._consume_bound_input_observation(binding) + try: + response = self._post_typed_action( + "/input/guarded", + { + "expected_frame_sha256": expected_frame_sha256, + "expected_frame_geometry": binding.geometry.to_payload(), + "expected_context": context, + "input": { + "action": "drag", + "x": point[0], + "y": point[1], + "end_x": int(end_x), + "end_y": int(end_y), + }, }, - }, - ) + ) + except _WindowsInputChanged: + self._raise_fresh_input_required( + expected=observation, + operation="physical_drag", + ) return self._guarded_receipt(response, "physical_drag") def type_text_guarded( @@ -1183,20 +1602,32 @@ def type_text_guarded( raise StructuralResolutionRefused( "Windows text actuation has no pre-identity focused binding" ) - point, context = pending - response = self._post_typed_action( - "/input/guarded", - { - "expected_frame_sha256": expected_frame_sha256, - "expected_context": context, - "focus_point": {"x": point[0], "y": point[1]}, - "input": { - "action": "type_text", - "text": text, - "interval_s": self._type_interval_s, + point, context, binding = pending + if not hmac.compare_digest(binding.frame_sha256, expected_frame_sha256): + raise StructuralResolutionRefused( + "Windows keyboard frame changed after identity verification" + ) + observation = self._consume_bound_input_observation(binding) + try: + response = self._post_typed_action( + "/input/guarded", + { + "expected_frame_sha256": expected_frame_sha256, + "expected_frame_geometry": binding.geometry.to_payload(), + "expected_context": context, + "focus_point": {"x": point[0], "y": point[1]}, + "input": { + "action": "type_text", + "text": text, + "interval_s": self._type_interval_s, + }, }, - }, - ) + ) + except _WindowsInputChanged: + self._raise_fresh_input_required( + expected=observation, + operation="physical_type_text", + ) return self._guarded_receipt(response, "physical_type_text") def press_guarded( @@ -1211,20 +1642,33 @@ def press_guarded( raise StructuralResolutionRefused( "Windows key actuation has no pre-identity focused binding" ) - point, context = pending - response = self._post_typed_action( - "/input/guarded", - { - "expected_frame_sha256": expected_frame_sha256, - "expected_context": context, - "focus_point": {"x": point[0], "y": point[1]}, - "input": {"action": "press", "keys": normalize_chord(key)}, - }, - ) + point, context, binding = pending + if not hmac.compare_digest(binding.frame_sha256, expected_frame_sha256): + raise StructuralResolutionRefused( + "Windows keyboard frame changed after identity verification" + ) + observation = self._consume_bound_input_observation(binding) + try: + response = self._post_typed_action( + "/input/guarded", + { + "expected_frame_sha256": expected_frame_sha256, + "expected_frame_geometry": binding.geometry.to_payload(), + "expected_context": context, + "focus_point": {"x": point[0], "y": point[1]}, + "input": {"action": "press", "keys": normalize_chord(key)}, + }, + ) + except _WindowsInputChanged: + self._raise_fresh_input_required( + expected=observation, + operation="physical_press", + ) return self._guarded_receipt(response, "physical_press") def click(self, x: int, y: int, *, double: bool = False) -> None: """Click (or double-click) through the bounded typed input contract.""" + binding = self._require_frame_binding() try: response = self._post_typed_action( "/input", @@ -1233,21 +1677,23 @@ def click(self, x: int, y: int, *, double: bool = False) -> None: "x": int(x), "y": int(y), "double": bool(double), + "expected_frame_sha256": binding.frame_sha256, + "expected_frame_geometry": binding.geometry.to_payload(), }, ) self._validate_physical_receipt( response, "physical_double_click" if double else "physical_click" ) return - except _TypedRouteUnavailable: - if not self._allow_legacy_exec: - raise - fn = "doubleClick" if double else "click" - self._execute(f"import pyautogui; pyautogui.{fn}({int(x)}, {int(y)})") + except _TypedRouteUnavailable as exc: + raise StructuralResolutionRefused( + "legacy Windows agents cannot safely map virtual-desktop coordinates" + ) from exc def right_click(self, x: int, y: int) -> None: """Right-click through the bounded typed input contract.""" + binding = self._require_frame_binding() try: response = self._post_typed_action( "/input", @@ -1257,20 +1703,21 @@ def right_click(self, x: int, y: int) -> None: "y": int(y), "double": False, "button": "right", + "expected_frame_sha256": binding.frame_sha256, + "expected_frame_geometry": binding.geometry.to_payload(), }, ) self._validate_physical_receipt(response, "physical_right_click") return - except _TypedRouteUnavailable: - if not self._allow_legacy_exec: - raise - self._execute( - f"import pyautogui; pyautogui.click({int(x)}, {int(y)}, button='right')" - ) + except _TypedRouteUnavailable as exc: + raise StructuralResolutionRefused( + "legacy Windows agents cannot safely map virtual-desktop coordinates" + ) from exc def drag(self, x: int, y: int, end_x: int, end_y: int) -> None: """Drag through the bounded typed input contract.""" + binding = self._require_frame_binding() try: response = self._post_typed_action( "/input", @@ -1280,20 +1727,16 @@ def drag(self, x: int, y: int, end_x: int, end_y: int) -> None: "y": int(y), "end_x": int(end_x), "end_y": int(end_y), + "expected_frame_sha256": binding.frame_sha256, + "expected_frame_geometry": binding.geometry.to_payload(), }, ) self._validate_physical_receipt(response, "physical_drag") return - except _TypedRouteUnavailable: - if not self._allow_legacy_exec: - raise - self._execute( - "import pyautogui; " - f"pyautogui.moveTo({int(x)}, {int(y)}); " - "pyautogui.mouseDown(button='left'); " - f"pyautogui.moveTo({int(end_x)}, {int(end_y)}, duration=0.2); " - "pyautogui.mouseUp(button='left')" - ) + except _TypedRouteUnavailable as exc: + raise StructuralResolutionRefused( + "legacy Windows agents cannot safely map virtual-desktop coordinates" + ) from exc def type_text(self, text: str) -> None: """Type text into the currently focused element. diff --git a/openadapt_flow/compiler/compile.py b/openadapt_flow/compiler/compile.py index 8a6059dd..439e2c0f 100644 --- a/openadapt_flow/compiler/compile.py +++ b/openadapt_flow/compiler/compile.py @@ -35,6 +35,7 @@ ExecutionMode, ExecutionTargetKind, Landmark, + NativeSourceGeometry, ParamKind, ParamSpec, Point, @@ -1774,6 +1775,11 @@ def cached_lines(i: int, suffix: str, png: bytes) -> list[OcrLine]: # in pass 2, once every click target's label is known — target labels # are mutable evidence (rename drift) and must not be asserted. pending: list[tuple[Step, Optional[bytes], Optional[bytes], dict]] = [] + has_native_source_geometry = any( + event.get("source_geometry") is not None for event in events + ) + source_sequence_identity: Optional[tuple[str, str, str, str]] = None + previous_source_action_ordinal = 0 for event in events: i = int(event["i"]) kind = event["kind"] @@ -1787,6 +1793,52 @@ def cached_lines(i: int, suffix: str, png: bytes) -> list[OcrLine]: ) before_png = _read_png(before_path) after_png = _read_png(after_path) + source_geometry_raw = event.get("source_geometry") + source_geometry: Optional[NativeSourceGeometry] = None + if has_native_source_geometry and source_geometry_raw is None: + raise ValueError( + "native source geometry must cover every executable recording event" + ) + if source_geometry_raw is not None: + if not isinstance(source_geometry_raw, dict): + raise ValueError(f"event {i} source_geometry must be a closed object") + source_geometry = NativeSourceGeometry.model_validate(source_geometry_raw) + if surface not in ("windows", "macos", "linux"): + raise ValueError( + "native source geometry is valid only for an in-session " + f"native surface, not {surface!r}" + ) + if source_geometry.source_surface != surface: + raise ValueError( + "native source geometry surface differs from the compiled " + "recording surface" + ) + if before_png is None or hashlib.sha256(before_png).hexdigest() != ( + source_geometry.frame_sha256 + ): + raise ValueError( + f"event {i} native source geometry names a different before frame" + ) + sequence_identity = ( + source_geometry.source_surface, + source_geometry.source_capture_session_sha256, + source_geometry.source_capture_terminal_sha256, + source_geometry.source_artifact_manifest_sha256, + ) + if source_sequence_identity is None: + source_sequence_identity = sequence_identity + elif sequence_identity != source_sequence_identity: + raise ValueError( + "native source geometry mixes capture or terminal identities" + ) + if ( + source_geometry.source_action_ordinal + <= previous_source_action_ordinal + ): + raise ValueError( + "native source action ordinals must be unique and strictly increasing" + ) + previous_source_action_ordinal = source_geometry.source_action_ordinal before_viewport = _validated_event_viewport( event, key="viewport_before", @@ -2064,6 +2116,7 @@ def cached_lines(i: int, suffix: str, png: bytes) -> list[OcrLine]: "right_click": ActionKind.RIGHT_CLICK, "drag": ActionKind.DRAG, }[kind], + source_geometry=source_geometry, anchor=anchor, drag_end_anchor=drag_end_anchor, identity_armed=identity_armed, @@ -2122,6 +2175,7 @@ def cached_lines(i: int, suffix: str, png: bytes) -> list[OcrLine]: id=step_id, intent=intent, action=ActionKind.TYPE, + source_geometry=source_geometry, text=text, param=param, secret=secret, @@ -2140,6 +2194,7 @@ def cached_lines(i: int, suffix: str, png: bytes) -> list[OcrLine]: id=step_id, intent=f"press {key}", action=ActionKind.KEY, + source_geometry=source_geometry, key=key, ), before_png, @@ -2156,6 +2211,7 @@ def cached_lines(i: int, suffix: str, png: bytes) -> list[OcrLine]: id=step_id, intent=f"press {'+'.join([*modifiers, key])}", action=ActionKind.HOTKEY, + source_geometry=source_geometry, key=key, modifiers=modifiers, ), @@ -2181,6 +2237,7 @@ def cached_lines(i: int, suffix: str, png: bytes) -> list[OcrLine]: id=step_id, intent=f"scroll by ({dx}, {dy})", action=ActionKind.SCROLL, + source_geometry=source_geometry, scroll_dx=dx, scroll_dy=dy, ), diff --git a/openadapt_flow/compiler/induction.py b/openadapt_flow/compiler/induction.py index 54a32a10..84904730 100644 --- a/openadapt_flow/compiler/induction.py +++ b/openadapt_flow/compiler/induction.py @@ -110,6 +110,7 @@ Transition, Workflow, ) +from openadapt_flow.runtime.authorization import runtime_param_text TraceInput = Union[Workflow, str, Path] @@ -717,7 +718,10 @@ def induce_program( program=program, subflows=subflows, param_specs=param_specs, - params={k: (v.example or "") for k, v in param_specs.items()}, + params={ + key: runtime_param_text(spec.example) if spec.example is not None else "" + for key, spec in param_specs.items() + }, data_sources=data_sources, ) result.program = program diff --git a/openadapt_flow/console/halt_detail.py b/openadapt_flow/console/halt_detail.py index d91adcfc..c171882e 100644 --- a/openadapt_flow/console/halt_detail.py +++ b/openadapt_flow/console/halt_detail.py @@ -53,6 +53,7 @@ from openadapt_flow.console import data from openadapt_flow.ir import ActionKind, Anchor, Rung, Step, Workflow from openadapt_flow.runtime import identity as _id +from openadapt_flow.runtime.authorization import runtime_params_for_gui from openadapt_flow.runtime.durable.checkpoint import CheckpointStore, PendingEscalation #: The resolution ladder in strongest-first order, taken from the engine's own @@ -444,9 +445,9 @@ def halt_detail( anchor = step.anchor if step is not None else None params: dict[str, str] = {} if report is not None and getattr(report, "params", None): - params = dict(report.params) + params = runtime_params_for_gui(report.params) elif pending is not None: - params = dict(pending.params) + params = runtime_params_for_gui(pending.params) role, label = _safe_target_label(step, params) resolved_rung = None diff --git a/openadapt_flow/deployment.py b/openadapt_flow/deployment.py index 09449741..7359e9bf 100644 --- a/openadapt_flow/deployment.py +++ b/openadapt_flow/deployment.py @@ -908,7 +908,7 @@ def build_replayer( def _resolve_config_exprs( section: str, exprs: Mapping[str, ValueExpr], - params: Optional[Mapping[str, str]], + params: Optional[Mapping[str, object]], ) -> dict[str, str]: """Resolve a config's ``ValueExpr`` mapping against the run's params. @@ -945,7 +945,7 @@ def _require_env(name: str, what: str) -> str: def build_effect_verifier( - cfg: EffectsConfig, params: Optional[Mapping[str, str]] = None + cfg: EffectsConfig, params: Optional[Mapping[str, object]] = None ) -> Optional[Any]: """Construct the configured ``EffectVerifier`` (or None for ``kind: none``). @@ -1073,7 +1073,7 @@ def sanitized(value: Any, *, key: str = "") -> Any: def _build_effect_verifier_unredacted( - cfg: EffectsConfig, params: Optional[Mapping[str, str]] = None + cfg: EffectsConfig, params: Optional[Mapping[str, object]] = None ) -> Optional[Any]: """The per-kind construction behind :func:`build_effect_verifier`.""" kind = (cfg.kind or "none").strip().lower() diff --git a/openadapt_flow/desktop_record.py b/openadapt_flow/desktop_record.py index 5eb117f8..fa459407 100644 --- a/openadapt_flow/desktop_record.py +++ b/openadapt_flow/desktop_record.py @@ -52,10 +52,11 @@ # Platforms where openadapt-capture's window-scoped capture is implemented # (openadapt_capture.window_capture.resolve_window / capture_window). Kept in # lock-step with that module: recording ONE window in its own pixel space needs -# a per-window capture primitive (macOS CGWindowListCreateImage / Windows -# Win32 + region grab). Elsewhere we refuse UP FRONT rather than start a -# full-screen capture that silently ignores the requested --window scope. -WINDOW_CAPTURE_PLATFORMS = ("darwin", "win32") +# a per-window capture primitive (macOS CGWindowListCreateImage, Windows +# Win32, or Linux X11/XComposite). Elsewhere we refuse UP FRONT rather than +# start a full-screen capture that silently ignores the requested --window +# scope. +WINDOW_CAPTURE_PLATFORMS = ("darwin", "win32", "linux") class _CaptureRecorder(Protocol): @@ -127,6 +128,7 @@ def record_desktop_capture( params: Optional[dict[str, str]] = None, identifier_region: Optional[tuple[int, int, int, int]] = None, window: Optional[dict[str, Optional[str]]] = None, + source_surface: Optional[str] = None, backend_kind: Optional[str] = None, replay_window: Optional[str] = None, replay_window_title: Optional[str] = None, @@ -169,6 +171,9 @@ def record_desktop_capture( surfaced into ``meta.json`` by the capture adapter. Refused up front on hosts where capture has no per-window primitive (see :data:`WINDOW_CAPTURE_PLATFORMS`). + source_surface: Native source identity (``windows``, ``macos``, or + ``linux``). Current v2 window captures use it to retain a sealed + per-action geometry binding. Remote surfaces leave it unset. backend_kind: Optional replay substrate identity (``rdp`` or ``citrix``) to seal into the compiled bundle's local execution hints. Other backends do not use the ``rdp_*`` target contract. @@ -206,6 +211,8 @@ def record_desktop_capture( raise ValueError( "backend_kind execution hints are supported only for rdp or citrix" ) + if source_surface not in (None, "windows", "macos", "linux"): + raise ValueError("source_surface must be windows, macos, linux, or null") out_dir = Path(out_dir) out_dir.mkdir(parents=True, exist_ok=True) @@ -221,13 +228,21 @@ def record_desktop_capture( if convert is None: from openadapt_flow.adapters.capture import convert_capture - convert = functools.partial( - convert_capture, - # A native Windows window remains a native UIA surface. Only an - # explicitly remote target suppresses the local client-window UIA - # observation, which cannot see controls inside RDP/Citrix. - include_structural=backend_kind not in ("rdp", "citrix"), - ) + # A native Windows window remains a native UIA surface. Only an + # explicitly remote target suppresses the local client-window UIA + # observation, which cannot see controls inside RDP/Citrix. + include_structural = backend_kind not in ("rdp", "citrix") + if source_surface is not None: + convert = functools.partial( + convert_capture, + include_structural=include_structural, + source_surface=source_surface, + ) + else: + convert = functools.partial( + convert_capture, + include_structural=include_structural, + ) if announce: scope_line = "" diff --git a/openadapt_flow/execution_profiles.py b/openadapt_flow/execution_profiles.py index eff5bc8c..4f344045 100644 --- a/openadapt_flow/execution_profiles.py +++ b/openadapt_flow/execution_profiles.py @@ -26,6 +26,11 @@ action_evidence_error, ) from openadapt_flow.decision_delivery import DecisionDeliveryTier +from openadapt_flow.runtime.authorization import ( + RuntimeParamScalar, + runtime_param_text, + runtime_params_for_gui, +) from openadapt_flow.verification import VerificationTier if TYPE_CHECKING: @@ -289,7 +294,7 @@ def _api_identity_evidence_is_exact( workflow: Workflow, step: Any, check: Any, - scoped_params: Mapping[str, str], + scoped_params: Mapping[str, RuntimeParamScalar], effects: list[Any], ) -> bool: """Return whether an API result matches its exact identity binding. @@ -303,6 +308,10 @@ def _api_identity_evidence_is_exact( binding = step.api_binding if binding is None or not binding.identity or check is None: return False + try: + text_params = runtime_params_for_gui(scoped_params) + except ValueError: + return False project = workflow.qualification policy = project.identity_policies.get(step.id) if project is not None else None if policy is not None: @@ -316,7 +325,7 @@ def _api_identity_evidence_is_exact( check=check, step=step, actuation_path="api", - runtime_params=scoped_params, + runtime_params=text_params, recorded_params=workflow.params, ) is not None @@ -353,7 +362,7 @@ def _api_identity_evidence_is_exact( return False for identity in binding.identity: - if not scoped_params.get(identity.param): + if identity.param not in text_params or text_params[identity.param] == "": return False effect_path = tuple(identity.effect_field.split(".")) if not any( @@ -413,7 +422,7 @@ def _program_action_trace( workflow: Workflow, visited_states: list[str], *, - runtime_params: Mapping[str, str] | None = None, + runtime_params: Mapping[str, str | bool | int | float] | None = None, runtime_worklists: Mapping[str, list[dict[str, str]]] | None = None, transition_evidence: list[Any] | None = None, exception_evidence: list[Any] | None = None, @@ -498,7 +507,7 @@ def _rows(relation: str) -> list[dict[str, str]] | None: return None if declared is None else list(declared.rows) def _reported_guard_value( - predicate: Any, current_params: Mapping[str, str] + predicate: Any, current_params: Mapping[str, str | bool | int | float] ) -> bool | None: """Recompute guards whose inputs are retained in the run report. @@ -511,9 +520,13 @@ def _reported_guard_value( kind = predicate.kind if kind is PredicateKind.PARAM_EQUALS: - return predicate.param is not None and str( - current_params.get(predicate.param) - ) == str(predicate.value) + return ( + predicate.param is not None + and predicate.value is not None + and predicate.param in current_params + and runtime_param_text(current_params[predicate.param]) + == predicate.value + ) if kind is PredicateKind.AND: values = [ _reported_guard_value(item, current_params) @@ -618,7 +631,7 @@ def _validated_evidence_target( graph_id: str, state: Any, scope: tuple[Any, ...], - current_params: Mapping[str, str], + current_params: Mapping[str, str | bool | int | float], ) -> str | None: nonlocal evaluator_contract_sha256 group = _matching_evidence_group( @@ -726,7 +739,7 @@ def _validated_evidence_target( recomputed_visual = evaluate_program_predicate( transition.guard, frame, - current_params, + runtime_params_for_gui(current_params), vision=transition_predicate_vision, viewport=item.observed_viewport, asset_loader=retained_assets.get, @@ -745,7 +758,7 @@ def _validated_attended_target( graph_id: str, state: Any, scope: tuple[Any, ...], - current_params: Mapping[str, str], + current_params: Mapping[str, str | bool | int | float], ) -> tuple[bool, str | None, str | None]: nonlocal attended_evidence_cursor, expected_evidence_decision_index evidence = attended_transition_evidence or [] @@ -869,7 +882,7 @@ def _validated_business_decision( graph_id: str, state: Any, scope: tuple[Any, ...], - current_params: dict[str, str], + current_params: dict[str, str | bool | int | float], ) -> str: nonlocal business_evidence_cursor, expected_evidence_decision_index evidence = business_decision_evidence or [] @@ -1013,7 +1026,7 @@ def _validated_exception_target( def _selected_transition_target( state: Any, - current_params: dict[str, str], + current_params: Mapping[str, str | bool | int | float], *, graph_id: str, scope: tuple[Any, ...], @@ -1064,7 +1077,7 @@ def _next_state( state: Any, graph: Any, occurrence_index: int | None, - current_params: dict[str, str], + current_params: Mapping[str, str | bool | int | float], *, graph_id: str, scope: tuple[Any, ...], @@ -1219,7 +1232,7 @@ def _consume_graph( scope: tuple[Any, ...], *, depth: int, - current_params: dict[str, str], + current_params: dict[str, str | bool | int | float], ) -> None: nonlocal cursor, halted_at_requested_action if depth > 64: @@ -1677,7 +1690,7 @@ def classify_execution_outcome( assert minimum is not None from openadapt_flow.ir import ActionKind - def _scoped_params(result: Any) -> dict[str, str] | None: + def _scoped_params(result: Any) -> dict[str, str | bool | int | float] | None: if workflow.program is None and result.program_scope: return None scoped = dict(report.params) @@ -1697,16 +1710,20 @@ def _scoped_params(result: Any) -> dict[str, str] | None: return scoped def _reported_parameter_predicate_value( - predicate: Any, current_params: Mapping[str, str] + predicate: Any, current_params: Mapping[str, str | bool | int | float] ) -> bool | None: """Recompute a guard only when all of its inputs are report-bound.""" from openadapt_flow.ir import PredicateKind if predicate.kind is PredicateKind.PARAM_EQUALS: - return predicate.param is not None and str( - current_params.get(predicate.param) - ) == str(predicate.value) + return ( + predicate.param is not None + and predicate.value is not None + and predicate.param in current_params + and runtime_param_text(current_params[predicate.param]) + == predicate.value + ) if predicate.kind is PredicateKind.AND: values = [ _reported_parameter_predicate_value(item, current_params) @@ -1802,7 +1819,7 @@ def _reported_parameter_predicate_value( check=result.identity, step=step, actuation_path=("api" if result.actuation == "api" else "gui"), - runtime_params=scoped_params, + runtime_params=runtime_params_for_gui(scoped_params), recorded_params=workflow.params, evidence_root=transition_evidence_root, recorded_asset_sha256=recorded_asset_sha256, @@ -1928,7 +1945,7 @@ def _reported_parameter_predicate_value( try: expected_hashes = Counter( effect.resolved_contract_hash( - scoped_params, + runtime_params_for_gui(scoped_params), opaque_param_sha256=opaque, ) for effect in effects @@ -1976,7 +1993,7 @@ def _reported_parameter_predicate_value( ) try: effect_hash = effect.resolved_contract_hash( - scoped_params, + runtime_params_for_gui(scoped_params), opaque_param_sha256=opaque, ) except ValueError: @@ -2163,7 +2180,7 @@ def build_outcome_envelope( envelope_requirements = () effect_requirements_valid = False - def _scoped_params(result: Any) -> dict[str, str] | None: + def _scoped_params(result: Any) -> dict[str, str | bool | int | float] | None: if workflow.program is None and result.program_scope: return None scoped = dict(report.params) @@ -2279,7 +2296,7 @@ def _scoped_params(result: Any) -> dict[str, str] | None: "qualified effect contract does not match" ) effect_hash = effect.resolved_contract_hash( - scoped_params, + runtime_params_for_gui(scoped_params), opaque_param_sha256=opaque, ) expected_hashes.append(effect_hash) diff --git a/openadapt_flow/interactive_recorder.py b/openadapt_flow/interactive_recorder.py index 559520db..d545218c 100644 --- a/openadapt_flow/interactive_recorder.py +++ b/openadapt_flow/interactive_recorder.py @@ -71,6 +71,7 @@ import ctypes import errno +import hashlib import io import ipaddress import json @@ -80,12 +81,14 @@ import sys import tempfile import uuid +from datetime import datetime, timezone from pathlib import Path from typing import Any, Callable, Optional from urllib.parse import urlsplit from PIL import Image +from openadapt_flow.backend import FrameObservation from openadapt_flow.backends.playwright_backend import PlaywrightBackend from openadapt_flow.recorder import Recorder @@ -2208,11 +2211,15 @@ def __init__( self.backend: Optional[PlaywrightBackend] = None self.recorder: Optional[Recorder] = None self._last_frame: bytes = b"" + self._last_frame_observation: Optional[FrameObservation] = None + self._observation_sequence = 0 self._last_structural: dict[str, Any] = {} self._attached_geometry: Optional[tuple[int, int, float]] = None self._initial_attached_viewport: Optional[tuple[int, int]] = None self._viewport_dirty = False self._viewport_history: list[dict[str, Any]] = [] + self._terminal_incomplete_reason: Optional[str] = None + self._incomplete_published = False # Source-time secret boundary state. Each document builds a fresh # closure, so a later document never saw the value an earlier one # received. Once a declared secret receives input, reflected text from @@ -2338,7 +2345,7 @@ def start(self) -> None: self.backend = PlaywrightBackend( self.page, - screenshot_scale="device" if self._owns_browser else "css", + screenshot_scale="css", screenshot_mask_selectors=_secret_screenshot_selectors( self._secret_fields, marker_attribute=self._secret_marker_attribute, @@ -2356,11 +2363,16 @@ def start(self) -> None: ) if self._owns_browser: self._last_frame = self.recorder._wait_settled() + assert self.backend.last_frame_observation is not None + self._retain_frame_observation( + self.backend.last_frame_observation, + boundary="initial", + ) self._last_structural = self._structural_state() else: self._rebaseline_attached_viewport() - except Exception: - self.abort() + except Exception as exc: + self.abort(reason=exc) raise def _register_existing_closed_shadow_boundaries(self) -> None: @@ -2555,8 +2567,8 @@ def run(self) -> Path: break except KeyboardInterrupt: print("\n[record] stopping…") - except Exception: - self.abort() + except Exception as exc: + self.abort(reason=exc) raise return self.finish() @@ -2566,8 +2578,8 @@ def run_script(self, script: Callable[[Any, Callable[[], None]], None]) -> Path: flush and finish.""" try: script(self.page, self.pump) - except Exception: - self.abort() + except Exception as exc: + self.abort(reason=exc) raise return self.finish() @@ -2609,6 +2621,9 @@ def finish(self) -> Path: # at the final path, and a crash in that window publishes a # surface-unbound recording. meta["surface"] = self._surface + meta["frame_observation_schema"] = "openadapt.browser-frame-observation.v1" + meta["observation_journal"] = "observation_journal.jsonl" + meta["observation_count"] = self._observation_sequence if self._structural_text_withheld: # The operator must be able to see that Flow dropped URL and # title evidence, and why. Silence here would read as evidence @@ -2643,12 +2658,12 @@ def finish(self) -> Path: if self._listener_error is not None: raise self._listener_error return self._promote_recording() - except Exception: - self.abort() + except Exception as exc: + self.abort(reason=exc) raise - def abort(self) -> None: - """Detach and remove only this session's unpublished temporary output.""" + def abort(self, *, reason: Optional[BaseException] = None) -> None: + """Detach and retain a clearly incomplete session when evidence exists.""" self.done = True self._pyq.clear() @@ -2658,7 +2673,104 @@ def abort(self) -> None: try: self._stop_browser_connection() finally: - self._discard_recording_dir() + if self._last_frame_observation is not None: + self._publish_incomplete_recording(reason=reason) + else: + self._discard_recording_dir() + + def _retain_frame_observation( + self, + observation: FrameObservation, + *, + boundary: str, + ) -> None: + """Append one exact local frame observation and its immutable PNG.""" + + if self._recording_dir is None: + raise BrowserAttachError("the recording journal is unavailable") + observations_dir = self._recording_dir / "observations" + observations_dir.mkdir(parents=True, exist_ok=True) + sequence = self._observation_sequence + frame_path = f"observations/{sequence:06d}.png" + (self._recording_dir / frame_path).write_bytes(observation.png) + record = { + "schema": "openadapt.browser-frame-observation.v1", + "sequence": sequence, + "boundary": boundary, + "event_count": self.recorder.event_count if self.recorder else 0, + "frame_path": frame_path, + "frame_sha256": observation.frame_sha256, + "viewport": list(observation.viewport), + "viewport_width": observation.viewport_width, + "viewport_height": observation.viewport_height, + "device_pixel_ratio": observation.device_pixel_ratio, + "origin": list(observation.origin), + "scale": list(observation.scale) if observation.scale else None, + "display_id": observation.display_id, + "display_bounds": list(observation.display_bounds), + "display_scale": list(observation.display_scale), + "topology_sha256": observation.topology_sha256, + "window_identity_sha256": observation.window_identity_sha256, + "session_identity_sha256": observation.session_identity_sha256, + "page_identity_sha256": observation.page_identity_sha256, + "top_level_frame_identity_sha256": ( + observation.top_level_frame_identity_sha256 + ), + "geometry_epoch": observation.geometry_epoch, + } + with (self._recording_dir / "observation_journal.jsonl").open("a") as file: + file.write(json.dumps(record, sort_keys=True, separators=(",", ":"))) + file.write("\n") + self._observation_sequence += 1 + self._last_frame_observation = observation + + def _publish_incomplete_recording( + self, + *, + reason: Optional[BaseException], + ) -> None: + """Publish evidence with an explicit terminal incomplete marker.""" + + if self._incomplete_published or self._recording_dir is None: + return + reason_code = self._terminal_incomplete_reason + if reason_code is None: + if reason is None: + reason_code = "operator_aborted" + elif isinstance(reason, BrowserAttachError): + reason_code = "invalid_event" + else: + reason_code = "browser_or_recording_disconnected" + meta_path = self._recording_dir / "meta.json" + if meta_path.exists(): + meta_path.unlink() + marker = { + "schema": "openadapt.browser-recording-incomplete.v1", + "complete": False, + "terminal_reason": reason_code, + "created_at": datetime.now(timezone.utc).isoformat(), + "session_id_sha256": hashlib.sha256( + self._session_id.encode("ascii") + ).hexdigest(), + "source": ( + "openadapt-flow-playwright" + if self._owns_browser + else "openadapt-flow-playwright-cdp" + ), + "event_count": self.recorder.event_count if self.recorder else 0, + "observation_count": self._observation_sequence, + "last_valid_observation_sequence": self._observation_sequence - 1, + } + (self._recording_dir / "incomplete.json").write_text( + json.dumps(marker, indent=2) + ) + try: + self._promote_recording() + except BrowserAttachError: + # A competing destination must never cause evidence deletion. The + # clearly marked partial directory remains at its reserved path. + return + self._incomplete_published = True def _prepare_recording_dir(self) -> None: """Reserve a fresh sibling directory without changing the final path.""" @@ -2682,7 +2794,7 @@ def _prepare_recording_dir(self) -> None: self._recording_dir = Path(temporary) def _promote_recording(self) -> Path: - """Atomically publish the complete recording at the requested path.""" + """Atomically publish a complete or explicitly incomplete session.""" assert self._recording_dir is not None try: @@ -2712,6 +2824,7 @@ def _handle_page_close(self, _page: Any = None) -> None: self.done = True if not self._owns_browser and self._listener_error is None: + self._terminal_incomplete_reason = "page_disconnected" self._listener_error = BrowserAttachError( "the selected browser tab closed before Flow could retain the " "final evidence; recording stopped without complete metadata" @@ -2991,6 +3104,12 @@ def _enqueue_browser_event( "scroll", "viewport", }: + self._terminal_incomplete_reason = "invalid_event" + self._listener_error = BrowserAttachError( + "the browser emitted an unsupported event kind; recording " + "stopped before accepting the event" + ) + self.done = True return self._track_secret_document( kind, event, raw_doc_id, holds_secret=doc_holds_secret @@ -3356,6 +3475,11 @@ def _pump(self) -> bool: self.page.wait_for_timeout(self._poll_ms) except Exception: self.done = True + self._terminal_incomplete_reason = "browser_disconnected" + if self._listener_error is None: + self._listener_error = BrowserAttachError( + "the browser disconnected before recording completion" + ) return False if not self._drain_event_queue(): # Distinct scroll gestures are separated by pauses; flush a @@ -3473,7 +3597,9 @@ def _accumulate_input(self, ev: dict[str, Any]) -> None: # keystroke and the next click, so this frame is the settled field — # not a screen the next click has already navigated to. assert self.backend is not None - pt["after_frame"] = self.backend.screenshot() + observation = self.backend.observe_frame() + pt["after_frame"] = observation.png + pt["after_observation"] = observation pt["structural_after"] = self._structural_state() def _flush_type(self) -> None: @@ -3530,7 +3656,11 @@ def _flush_type(self) -> None: after_png=after_png, structural_after=structural_after, ) - self._set_last(after_png, structural_after) + self._set_last( + after_png, + structural_after, + observation=pt.get("after_observation"), + ) def _accumulate_scroll(self, ev: dict[str, Any]) -> None: if self._pending_scroll is None: @@ -3544,7 +3674,9 @@ def _accumulate_scroll(self, ev: dict[str, Any]) -> None: ps["dy"] += int(ev.get("dy", 0)) # Post-scroll after-state, captured now (before any following action). assert self.backend is not None - ps["after_frame"] = self.backend.screenshot() + observation = self.backend.observe_frame() + ps["after_frame"] = observation.png + ps["after_observation"] = observation ps["structural_after"] = self._structural_state() def _flush_scroll(self) -> None: @@ -3562,7 +3694,11 @@ def _flush_scroll(self) -> None: after_png=after_png, structural_after=structural_after, ) - self._set_last(after_png, structural_after) + self._set_last( + after_png, + structural_after, + observation=ps.get("after_observation"), + ) def _record_pointer(self, ev: dict[str, Any]) -> None: assert self.recorder is not None @@ -3674,8 +3810,25 @@ def _rebaseline_attached_viewport(self) -> None: with Image.open(io.BytesIO(frame)) as image: frame_size = image.size if before == after and frame_size == after[:2]: + observation = self.backend.last_frame_observation + if ( + observation is None + or observation.png != frame + or observation.viewport_width != after[0] + or observation.viewport_height != after[1] + or observation.device_pixel_ratio != after[2] + ): + continue self._attached_geometry = after self._last_frame = frame + self._retain_frame_observation( + observation, + boundary=( + "initial" + if self._initial_attached_viewport is None + else "geometry_transition" + ), + ) self._last_structural = self._structural_state() self._viewport_dirty = False viewport = after[:2] @@ -3708,11 +3861,17 @@ def _advance(self) -> None: """After an IMMEDIATE step (click/key), the current settled frame becomes the next step's BEFORE frame.""" assert self.backend is not None - self._last_frame = self.backend.screenshot() + observation = self.backend.observe_frame() + self._last_frame = observation.png + self._retain_frame_observation(observation, boundary="event") self._last_structural = self._structural_state() def _set_last( - self, after_png: Optional[bytes], structural_after: Optional[dict] + self, + after_png: Optional[bytes], + structural_after: Optional[dict], + *, + observation: Optional[FrameObservation] = None, ) -> None: """After a DEFERRED/coalesced step (type/scroll), the next step's BEFORE frame is the after-state captured when the step actually @@ -3720,6 +3879,8 @@ def _set_last( already have moved on from.""" if after_png is not None: self._last_frame = after_png + if observation is not None: + self._retain_frame_observation(observation, boundary="event") else: self._advance() return diff --git a/openadapt_flow/ir.py b/openadapt_flow/ir.py index b54bf8d6..1cec1922 100644 --- a/openadapt_flow/ir.py +++ b/openadapt_flow/ir.py @@ -23,13 +23,14 @@ import hashlib import json +import math import os import re import secrets from datetime import datetime, timezone from enum import Enum from pathlib import Path, PurePosixPath -from typing import TYPE_CHECKING, Any, Final, Iterator, Literal, Optional +from typing import TYPE_CHECKING, Any, Final, Iterator, Literal, Optional, Union from pydantic import ( BaseModel, @@ -42,6 +43,7 @@ ) from openadapt_flow.qualification_faults import FaultMutationReceipt +from openadapt_flow.source_geometry import NativeSourceGeometry if TYPE_CHECKING: # Type-only import for the Step.effects forward reference. The RUNTIME @@ -637,6 +639,7 @@ class ParamKind(str, Enum): DATE = "date" ENUM = "enum" NUMBER = "number" + BOOLEAN = "boolean" ENTITY_REF = "entity_ref" @@ -651,7 +654,7 @@ class ParamSpec(BaseModel): name: str type: ParamKind = ParamKind.STRING - example: Optional[str] = Field( + example: Optional[Union[str, bool, int, float]] = Field( default=None, description="Recorded demo value; also the replay default when the " "caller supplies no value for this parameter.", @@ -915,6 +918,13 @@ class Step(BaseModel): id: str intent: str = Field(description="Human-readable purpose of the step") action: ActionKind + source_geometry: Optional[NativeSourceGeometry] = Field( + default=None, + description=( + "Verified source frame, process identity, display topology, and " + "native geometry epoch for this demonstrated action" + ), + ) anchor: Optional[Anchor] = None # None for pure keyboard/wait steps text: Optional[str] = None # literal text for TYPE param: Optional[str] = None # if set, TYPE text comes from params[param] @@ -1104,6 +1114,14 @@ def _validate_rich_action_contract(self) -> "Step": ), ) + @model_serializer(mode="wrap") + def _serialize_source_geometry_compatible(self, handler: Any) -> dict[str, Any]: + """Keep legacy Step JSON unchanged when no native source binding exists.""" + data: dict[str, Any] = handler(self) + if self.source_geometry is None: + data.pop("source_geometry", None) + return data + # -- Workflow-program IR, Phase 2 (RFC docs/design/WORKFLOW_PROGRAM_IR.md §2) -- # @@ -2371,6 +2389,54 @@ class FreshActuationEvent(BaseModel): changed_pixel_count: int = Field(ge=1) changed_bbox: Region frame_size: tuple[int, int] + expected_geometry_epoch: Optional[str] = Field( + default=None, + pattern=r"^[0-9a-f]{64}$", + exclude_if=lambda value: value is None, + ) + observed_geometry_epoch: Optional[str] = Field( + default=None, + pattern=r"^[0-9a-f]{64}$", + exclude_if=lambda value: value is None, + ) + expected_display_id: Optional[str] = Field( + default=None, + min_length=1, + max_length=256, + exclude_if=lambda value: value is None, + ) + observed_display_id: Optional[str] = Field( + default=None, + min_length=1, + max_length=256, + exclude_if=lambda value: value is None, + ) + expected_display_bounds: Optional[tuple[float, float, float, float]] = Field( + default=None, + exclude_if=lambda value: value is None, + ) + observed_display_bounds: Optional[tuple[float, float, float, float]] = Field( + default=None, + exclude_if=lambda value: value is None, + ) + expected_display_scale: Optional[tuple[float, float]] = Field( + default=None, + exclude_if=lambda value: value is None, + ) + observed_display_scale: Optional[tuple[float, float]] = Field( + default=None, + exclude_if=lambda value: value is None, + ) + expected_topology_sha256: Optional[str] = Field( + default=None, + pattern=r"^[0-9a-f]{64}$", + exclude_if=lambda value: value is None, + ) + observed_topology_sha256: Optional[str] = Field( + default=None, + pattern=r"^[0-9a-f]{64}$", + exclude_if=lambda value: value is None, + ) target_intersection: Optional[bool] = None identity_intersection: Optional[bool] = None retried: bool @@ -2385,6 +2451,42 @@ def _valid_geometry(self) -> "FreshActuationEvent": raise ValueError("fresh-actuation bounding box must be positive") if x + width > frame_width or y + height > frame_height: raise ValueError("fresh-actuation bounding box exceeds the frame") + if (self.expected_geometry_epoch is None) != ( + self.observed_geometry_epoch is None + ): + raise ValueError( + "fresh-actuation geometry epochs must be supplied together" + ) + descriptor_values = ( + self.expected_display_id, + self.observed_display_id, + self.expected_display_bounds, + self.observed_display_bounds, + self.expected_display_scale, + self.observed_display_scale, + self.expected_topology_sha256, + self.observed_topology_sha256, + ) + if any(value is not None for value in descriptor_values) and not all( + value is not None for value in descriptor_values + ): + raise ValueError( + "fresh-actuation old/new display descriptors must be complete" + ) + for bounds in (self.expected_display_bounds, self.observed_display_bounds): + if bounds is not None and ( + not all(math.isfinite(value) for value in bounds) + or bounds[2] <= 0 + or bounds[3] <= 0 + ): + raise ValueError( + "fresh-actuation display bounds must be finite and positive" + ) + for scale in (self.expected_display_scale, self.observed_display_scale): + if scale is not None and not all( + math.isfinite(value) and value > 0 for value in scale + ): + raise ValueError("fresh-actuation display scale must be positive") return self @@ -3634,7 +3736,7 @@ class RunReport(BaseModel): ) required_identity_step_ids: list[str] = Field(default_factory=list) approved_unverified_effect_step_ids: list[str] = Field(default_factory=list) - params: dict[str, str] = Field(default_factory=dict) + params: dict[str, Union[str, bool, int, float]] = Field(default_factory=dict) results: list[StepResult] = Field(default_factory=list) success: bool = False # Workflow-program IR, Phase 2: the outcome of the terminal state the graph diff --git a/openadapt_flow/learning/halt_loop.py b/openadapt_flow/learning/halt_loop.py index 72df1e5c..65997069 100644 --- a/openadapt_flow/learning/halt_loop.py +++ b/openadapt_flow/learning/halt_loop.py @@ -47,6 +47,7 @@ learn_from_traces, ) from openadapt_flow.learning.trace import ExecutionTrace, TraceStep +from openadapt_flow.runtime.authorization import runtime_params_for_gui from openadapt_flow.runtime.healing.governance import RegressionGate @@ -78,7 +79,7 @@ def execution_trace_from_halt( outcome="failure", steps=steps, facts=facts, - params=dict(params or report.params), + params=runtime_params_for_gui(params or report.params), failure_reason=halt.reason, ) diff --git a/openadapt_flow/learning/teach.py b/openadapt_flow/learning/teach.py index 66a74577..abe2d538 100644 --- a/openadapt_flow/learning/teach.py +++ b/openadapt_flow/learning/teach.py @@ -65,6 +65,7 @@ class the halt->learn loop was built for (splice a guarded, reversible dismiss from openadapt_flow.learning.loop import Inducer, LearnOutcome from openadapt_flow.learning.synth_stream import StructuralDiffInducer from openadapt_flow.learning.trace import ExecutionTrace, TraceStep +from openadapt_flow.runtime.authorization import runtime_params_for_gui class TeachError(Exception): @@ -245,7 +246,7 @@ def _correction_from_spec( resolution_steps=resolution_steps, tail_intents=tail, trace_id=f"{report.workflow_name}-correction", - params=spec.params or dict(report.params), + params=spec.params or runtime_params_for_gui(report.params), ) return correction, _baseline_success(program, report, tail) diff --git a/openadapt_flow/qualification.py b/openadapt_flow/qualification.py index 95e68266..ef5791f7 100644 --- a/openadapt_flow/qualification.py +++ b/openadapt_flow/qualification.py @@ -2556,7 +2556,11 @@ def _case_run_report_integrity_error( QualificationRefusalCode.CASE_ATTESTATION_INVALID, "case-input bytes do not match the signed case-result binding", ) - from openadapt_flow.runtime.authorization import parse_runtime_inputs_bytes + from openadapt_flow.runtime.authorization import ( + parse_runtime_inputs_bytes, + runtime_param_text, + runtime_params_for_gui, + ) try: case_params, case_worklists = parse_runtime_inputs_bytes( @@ -2569,7 +2573,9 @@ def _case_run_report_integrity_error( "case input is not a valid canonical governed-input artifact", ) - def scoped_case_params(item: Any) -> Optional[dict[str, str]]: + def scoped_case_params( + item: Any, + ) -> Optional[dict[str, str | bool | int | float]]: if workflow.program is None: return dict(case_params) if not item.program_scope else None if not item.program_scope or item.program_scope[0].graph_id != "__program__": @@ -3155,9 +3161,10 @@ def delivery_receipt_error(item: Any, step: "Step") -> Optional[str]: ) if selected_value is None or step.selection_commit_key is None: return "selection delivery receipt lacks its compiled input contract" + selected_text = runtime_param_text(selected_value) if ( receipt.selection_value_sha256 - != hashlib.sha256(selected_value.encode("utf-8")).hexdigest() + != hashlib.sha256(selected_text.encode("utf-8")).hexdigest() or receipt.selection_commit_key != step.selection_commit_key ): return "selection delivery receipt differs from the compiled input" @@ -3238,7 +3245,7 @@ def identity_evidence_error( check=item.identity, step=step, actuation_path=actuation_path, - runtime_params=scoped, + runtime_params=runtime_params_for_gui(scoped), recorded_params=workflow.params, evidence_root=run_evidence_root, recorded_asset_sha256=recorded_asset_sha256, @@ -3248,7 +3255,7 @@ def identity_evidence_error( check=item.identity, step=step, actuation_path=actuation_path, - runtime_params=scoped, + runtime_params=runtime_params_for_gui(scoped), recorded_params=workflow.params, evidence_root=run_evidence_root, recorded_asset_sha256=recorded_asset_sha256, @@ -3277,7 +3284,7 @@ def resolved_effect_contracts( ( index, effect.resolved_contract_hash( - scoped, + runtime_params_for_gui(scoped), opaque_param_sha256={"__run_id__": result.run_id_sha256 or ""}, ), effect_policies.get((step.id, actuation_path, index)), @@ -3913,7 +3920,7 @@ def result_has_sufficient_effect_evidence(item: Any) -> bool: ( index, effect.resolved_contract_hash( - resolved_params, + runtime_params_for_gui(resolved_params), opaque_param_sha256={ "__run_id__": result.run_id_sha256 or "" }, @@ -4006,7 +4013,11 @@ def result_has_sufficient_effect_evidence(item: Any) -> bool: check=item.identity, step=step, actuation_path=actuation_path, - runtime_params=scoped_case_params(item), + runtime_params=( + runtime_params_for_gui(scoped) + if (scoped := scoped_case_params(item)) is not None + else None + ), recorded_params=workflow.params, evidence_root=run_evidence_root, recorded_asset_sha256=( diff --git a/openadapt_flow/recorder.py b/openadapt_flow/recorder.py index eb093d32..412c435d 100644 --- a/openadapt_flow/recorder.py +++ b/openadapt_flow/recorder.py @@ -230,7 +230,7 @@ def event_count(self) -> int: def _record(self, event: dict[str, Any], act: Callable[[], None]) -> None: """Capture before frame, act, wait settle, capture after, log event.""" - before = self._backend.screenshot() + before = self._capture_frame() structural_before = self._structural_state() # Structured identity of the clicked target (DOM / a11y text), when # the backend exposes it: captured on the BEFORE frame, before the @@ -495,12 +495,12 @@ def _wait_settled(self) -> bytes: The last PNG frame captured (settled if achieved before timeout). """ deadline = time.monotonic() + self._settle_timeout_s - png = self._backend.screenshot() + png = self._capture_frame() prev = _phash(png) consecutive = 1 while consecutive < self._settle_stable_frames and time.monotonic() < deadline: time.sleep(self._settle_interval_s) - png = self._backend.screenshot() + png = self._capture_frame() cur = _phash(png) if cur == prev: consecutive += 1 @@ -508,3 +508,11 @@ def _wait_settled(self) -> bytes: consecutive = 1 prev = cur return png + + def _capture_frame(self) -> bytes: + """Prefer an atomic frame descriptor when the backend provides one.""" + + observer = getattr(self._backend, "observe_frame", None) + if callable(observer): + return observer().png + return self._backend.screenshot() diff --git a/openadapt_flow/runner/__init__.py b/openadapt_flow/runner/__init__.py index 013f6222..3d975663 100644 --- a/openadapt_flow/runner/__init__.py +++ b/openadapt_flow/runner/__init__.py @@ -1,15 +1,8 @@ -"""EXPERIMENTAL runner-client LIBRARY (verification, lease logic, evidence, -command mapping) for the hosted control-plane / local-execution-plane runner -protocol. NO daemon, NO network loop, NO CLI verb — deliberately. +"""Flow-owned hosted-runner verification and execution library. -Scope: the merged ``/api/runners/*`` control-plane surface in openadapt-cloud -(``src/lib/runners.ts``) is mock-gated (410 in live) and its transport-facing -half is scheduled to CHANGE before any customer daemon can ship (poll cadence -and hosting economics, lease renewal + sleep reclaim, control-verb channel, -mandatory params-by-reference for regulated orgs — see -``docs/design/RUNNER_CLIENT_LIBRARY.md`` for the verified findings and the -required revisions). This package therefore contains only the transport- -agnostic half that SURVIVES that revision: +Desktop owns the authenticated register, poll, and callback transport loop. +This package owns the transport-independent trust, admission, one-use, +governed-execution, and terminal-classification boundary: * :mod:`~openadapt_flow.runner.protocol` — strict typed models of the dispatch wire contract (contract drift is a refusal, not a best guess); @@ -31,7 +24,9 @@ evidence queue (a run that finishes offline reports late, never never); * :mod:`~openadapt_flow.runner.commands` — mapping of governed dispatch verbs onto the EXISTING CLI entry points (``run`` / ``resume``); unmappable verbs - refuse. + refuse; +* :mod:`~openadapt_flow.runner.hosted_adapter` — the strict Cloud lease wire, + protected local trust, managed child bridge, and no-replay result contract. """ from openadapt_flow.runner.commands import ( @@ -52,6 +47,23 @@ read_managed_dispatch_envelope, write_managed_dispatch_envelope, ) +from openadapt_flow.runner.hosted_adapter import ( + RUNNER_RENEWAL_HEADER, + CallbackRequest, + CallbackResponse, + DeliveryAuthority, + HostedDispatch, + HostedDispatchRefusal, + HostedRecoveryBinding, + HostedRunnerAdapter, + HostedRunnerTransport, + HostedRunResult, + PollRequest, + RegisterCapabilities, + RegisterRequest, + RegisterResponse, + registration_renewal_headers, +) from openadapt_flow.runner.lease import ( CompletionDisposition, LeaseError, @@ -78,8 +90,17 @@ __all__ = [ "CompletionDisposition", + "CallbackRequest", + "CallbackResponse", + "DeliveryAuthority", "DispatchParseError", "EvidenceOutbox", + "HostedDispatch", + "HostedDispatchRefusal", + "HostedRecoveryBinding", + "HostedRunResult", + "HostedRunnerAdapter", + "HostedRunnerTransport", "LeaseError", "ManagedDispatchEnvelope", "ManagedDispatchEnvelopeError", @@ -88,6 +109,11 @@ "LeasedDispatch", "Refusal", "RefusalCode", + "PollRequest", + "RUNNER_RENEWAL_HEADER", + "RegisterCapabilities", + "RegisterRequest", + "RegisterResponse", "RunnerConfig", "RunnerConfigError", "RunnerDispatchPayload", @@ -103,6 +129,7 @@ "map_control_verb", "read_managed_dispatch_envelope", "parse_dispatch", + "registration_renewal_headers", "server_reclaim_outcome", "verify_dispatch", "write_managed_dispatch_envelope", diff --git a/openadapt_flow/runner/commands.py b/openadapt_flow/runner/commands.py index a92d2a32..6feee772 100644 --- a/openadapt_flow/runner/commands.py +++ b/openadapt_flow/runner/commands.py @@ -3,9 +3,8 @@ The runner never grows a private execution path: a dispatched run is the same fail-closed ``openadapt-flow run`` admission gate + shared replayer the local CLI uses, in a child process (crash isolation; the design doc's "the agent -shells them"). This module only BUILDS argv — executing it belongs to the -future daemon, which is deliberately not in this library (see -``docs/design/RUNNER_CLIENT_LIBRARY.md``). +shells them"). This module only builds argv. The hosted adapter executes that +argv in the managed child process. Verb coverage, honestly stated: @@ -51,6 +50,7 @@ def build_run_argv( params_file: Optional[Path], *, managed_dispatch_file: Path, + qualification_authority_file: Optional[Path] = None, ) -> list[str]: """The exact governed CLI invocation for a verified ``run`` dispatch. @@ -81,6 +81,8 @@ def build_run_argv( ] if params_file is not None: argv += ["--params-file", str(params_file)] + if qualification_authority_file is not None: + argv += ["--qualification-authority-file", str(qualification_authority_file)] if verified.bundle.policy: argv += ["--policy", verified.bundle.policy] if ( diff --git a/openadapt_flow/runner/config.py b/openadapt_flow/runner/config.py index 30b72e48..5fcb26c9 100644 --- a/openadapt_flow/runner/config.py +++ b/openadapt_flow/runner/config.py @@ -3,7 +3,7 @@ Nothing writes this file programmatically. It names the deployment profiles a dispatch may reference and the exact sealed bundles (by content digest) this machine is willing to execute — a digest absent from this file is refused. -That is the no-remote-code-delivery hard line: the future runner daemon only +That is the no-remote-code-delivery hard line: the hosted adapter only ever executes bundles the operator ALREADY installed and listed here; the dispatch's ``bundle.url`` is never fetched. @@ -42,16 +42,27 @@ import os import re +import stat from dataclasses import dataclass, field from pathlib import Path -from typing import Any, Optional +from typing import Any, Literal, Optional from openadapt_flow.hosted import HostedError +from openadapt_flow.private_file import ( + PrivateFileAclError, + windows_descriptor_has_private_acl, +) _HEX64_RE = re.compile(r"^[a-f0-9]{64}$") +_UUID_RE = re.compile( + r"^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$" +) +_SAFE_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:@/+\-]{0,199}$") +LocalReleaseTarget = Literal["flow", "desktop", "capture"] -def _load_manifest_toml(path: Path) -> dict[str, Any]: + +def _load_manifest_toml(path: Path, *, protected: bool = False) -> dict[str, Any]: """Full-TOML parse (the manifest uses ``[[bundles]]`` array tables, which ``hosted._load_toml``'s 3.10 minimal fallback cannot represent). Uses stdlib ``tomllib`` on 3.11+ and the declared ``tomli`` dependency on 3.10. @@ -60,6 +71,81 @@ def _load_manifest_toml(path: Path) -> dict[str, Any]: import tomllib except ModuleNotFoundError: # pragma: no cover - Python 3.10 import tomli as tomllib # type: ignore[no-redef] + if protected: + flags = os.O_RDONLY + if hasattr(os, "O_NOFOLLOW"): + flags |= os.O_NOFOLLOW + try: + path_before = path.lstat() + if not stat.S_ISREG(path_before.st_mode) or stat.S_ISLNK( + path_before.st_mode + ): + raise RunnerConfigError( + "hosted runner manifest is not a private regular file" + ) + descriptor = os.open(path, flags) + except OSError as exc: + raise RunnerConfigError( + "hosted runner manifest could not be opened safely" + ) from exc + try: + before = os.fstat(descriptor) + try: + private = ( + windows_descriptor_has_private_acl(descriptor) + if os.name == "nt" + else ( + before.st_uid == os.geteuid() + and stat.S_IMODE(before.st_mode) == 0o600 + ) + ) + except PrivateFileAclError as exc: + raise RunnerConfigError( + "hosted runner manifest ACL could not be verified" + ) from exc + if ( + not stat.S_ISREG(before.st_mode) + or before.st_size > 1024 * 1024 + or not private + ): + raise RunnerConfigError( + "hosted runner manifest is not a private regular file" + ) + chunks: list[bytes] = [] + remaining = before.st_size + while remaining: + chunk = os.read(descriptor, min(remaining, 64 * 1024)) + if not chunk: + break + chunks.append(chunk) + remaining -= len(chunk) + raw = b"".join(chunks) + after = os.fstat(descriptor) + try: + path_after = path.lstat() + except OSError as exc: + raise RunnerConfigError( + "hosted runner manifest changed during its protected read" + ) from exc + if ( + len(raw) != before.st_size + or stat.S_ISLNK(path_after.st_mode) + or (path_before.st_dev, path_before.st_ino) + != (before.st_dev, before.st_ino) + or (path_after.st_dev, path_after.st_ino) + != (before.st_dev, before.st_ino) + or (before.st_dev, before.st_ino, before.st_size, before.st_mtime_ns) + != (after.st_dev, after.st_ino, after.st_size, after.st_mtime_ns) + ): + raise RunnerConfigError( + "hosted runner manifest changed during its protected read" + ) + finally: + os.close(descriptor) + try: + return tomllib.loads(raw.decode("utf-8")) + except (UnicodeDecodeError, tomllib.TOMLDecodeError) as exc: + raise RunnerConfigError("hosted runner manifest is not valid TOML") from exc try: with path.open("rb") as fh: return tomllib.load(fh) @@ -103,6 +189,9 @@ class TrustedBundle: allow_unverified_writes: bool = False #: Local escape hatch mirroring ``run --allow-unencrypted``. allow_unencrypted: bool = False + #: Exact archive/object digest named by the workflow admission. Hosted + #: execution requires this local pin; ordinary local execution does not. + artifact_sha256: Optional[str] = None @dataclass(frozen=True) @@ -112,6 +201,32 @@ class BusinessDecisionServiceConfig: key_file: Path +@dataclass(frozen=True) +class LocalRuntimeRelease: + """One independently installed target release used during enrollment.""" + + target: LocalReleaseTarget + admission_id: str + admission_sha256: str + release_version: str + release_artifact_sha256: str + + +@dataclass(frozen=True) +class AdmissionTrustFiles: + """Local signer and revocation state used to verify hosted admissions.""" + + signer_registry: Path + state: Path + + +@dataclass(frozen=True) +class WorkflowAdmissionTrustFiles(AdmissionTrustFiles): + """Local v2 expectation that is independent from the leased artifact.""" + + expected_bindings: Path + + @dataclass(frozen=True) class RunnerConfig: """Parsed trust manifest.""" @@ -121,9 +236,14 @@ class RunnerConfig: profiles: dict[str, Path] = field(default_factory=dict) bundles: dict[str, TrustedBundle] = field(default_factory=dict) #: Capability advertisement (deployment.yaml backend kinds this machine - #: can drive) for the future register/poll payloads. Advisory only. + #: can drive) for hosted registration. Advisory only. backends: tuple[str, ...] = ("web",) business_decisions: Optional[BusinessDecisionServiceConfig] = None + local_runtime_release: tuple[LocalRuntimeRelease, ...] = () + product_release_admission: Optional[AdmissionTrustFiles] = None + workflow_admission: Optional[WorkflowAdmissionTrustFiles] = None + params_ref_root: Optional[Path] = None + evidence_runner_private_key: Optional[Path] = None def _parse_param_patterns(raw: object, index: int) -> dict[str, str]: @@ -147,7 +267,9 @@ def _parse_param_patterns(raw: object, index: int) -> dict[str, str]: return patterns -def load_runner_config(path: Optional[Path] = None) -> RunnerConfig: +def load_runner_config( + path: Optional[Path] = None, *, protected: bool = False +) -> RunnerConfig: """Load and validate ``runner.toml``. Fail loudly on anything malformed.""" cfg_path = path or runner_config_path() if not cfg_path.is_file(): @@ -156,7 +278,7 @@ def load_runner_config(path: Optional[Path] = None) -> RunnerConfig: "list the deployment profiles and the exact sealed bundles (by " "content digest) this machine may execute." ) - data = _load_manifest_toml(cfg_path) + data = _load_manifest_toml(cfg_path, protected=protected) runner_tbl = data.get("runner") or {} if not isinstance(runner_tbl, dict): @@ -214,7 +336,18 @@ def load_runner_config(path: Optional[Path] = None) -> RunnerConfig: param_patterns=_parse_param_patterns(entry.get("param_patterns"), i), allow_unverified_writes=bool(entry.get("allow_unverified_writes", False)), allow_unencrypted=bool(entry.get("allow_unencrypted", False)), + artifact_sha256=( + str(entry["artifact_sha256"]) + if entry.get("artifact_sha256") is not None + else None + ), ) + if bundles[digest].artifact_sha256 is not None and not _HEX64_RE.fullmatch( + bundles[digest].artifact_sha256 or "" + ): + raise RunnerConfigError( + f"[[bundles]] entry {i} artifact_sha256 must be 64 lowercase hex" + ) decision_tbl = data.get("business_decisions") business_decisions = None @@ -237,6 +370,126 @@ def load_runner_config(path: Optional[Path] = None) -> RunnerConfig: ) business_decisions = BusinessDecisionServiceConfig(key_file=key_file) + local_release_tbl = data.get("local_runtime_release") or {} + if not isinstance(local_release_tbl, dict): + raise RunnerConfigError("[local_runtime_release] must be a table") + local_runtime_release: list[LocalRuntimeRelease] = [] + expected_release_targets: tuple[LocalReleaseTarget, ...] = ( + "flow", + "desktop", + "capture", + ) + for target in expected_release_targets: + entry = local_release_tbl.get(target) + if entry is None: + continue + if not isinstance(entry, dict) or set(entry) != { + "admission_id", + "admission_sha256", + "release_version", + "release_artifact_sha256", + }: + raise RunnerConfigError( + f"[local_runtime_release.{target}] has an invalid exact shape" + ) + admission_sha256 = str(entry["admission_sha256"]) + artifact_sha256 = str(entry["release_artifact_sha256"]) + admission_id = str(entry["admission_id"]) + release_version = str(entry["release_version"]) + if not _HEX64_RE.fullmatch(admission_sha256) or not _HEX64_RE.fullmatch( + artifact_sha256 + ): + raise RunnerConfigError( + f"[local_runtime_release.{target}] contains an invalid digest" + ) + if not _UUID_RE.fullmatch(admission_id) or not _SAFE_ID_RE.fullmatch( + release_version + ): + raise RunnerConfigError( + f"[local_runtime_release.{target}] contains an invalid identity" + ) + local_runtime_release.append( + LocalRuntimeRelease( + target=target, + admission_id=admission_id, + admission_sha256=admission_sha256, + release_version=release_version, + release_artifact_sha256=artifact_sha256, + ) + ) + unknown_release_targets = sorted( + set(local_release_tbl).difference(expected_release_targets) + ) + if unknown_release_targets: + raise RunnerConfigError( + "[local_runtime_release] contains unknown target(s): " + + ", ".join(unknown_release_targets) + ) + + def admission_trust_files(table_name: str) -> Optional[AdmissionTrustFiles]: + table = data.get(table_name) + if table is None: + return None + if not isinstance(table, dict) or set(table) != {"signer_registry", "state"}: + raise RunnerConfigError(f"[{table_name}] has an invalid exact shape") + registry = Path(str(table["signer_registry"])).expanduser() + state = Path(str(table["state"])).expanduser() + if not registry.is_file() or not state.is_file(): + raise RunnerConfigError( + f"[{table_name}] trust files must be existing regular files" + ) + return AdmissionTrustFiles(signer_registry=registry, state=state) + + product_release_admission = admission_trust_files("product_release_admission") + workflow_table = data.get("workflow_admission") + workflow_admission = None + if workflow_table is not None: + if not isinstance(workflow_table, dict) or set(workflow_table) != { + "signer_registry", + "state", + "expected_bindings", + }: + raise RunnerConfigError("[workflow_admission] has an invalid exact shape") + workflow_paths = { + key: Path(str(workflow_table[key])).expanduser() + for key in ("signer_registry", "state", "expected_bindings") + } + if any(not path.is_file() for path in workflow_paths.values()): + raise RunnerConfigError( + "[workflow_admission] trust files must be existing regular files" + ) + workflow_admission = WorkflowAdmissionTrustFiles( + signer_registry=workflow_paths["signer_registry"], + state=workflow_paths["state"], + expected_bindings=workflow_paths["expected_bindings"], + ) + + params_tbl = data.get("params") + params_ref_root = None + if params_tbl is not None: + if not isinstance(params_tbl, dict) or set(params_tbl) != {"protected_root"}: + raise RunnerConfigError("[params] has an invalid exact shape") + params_ref_root = Path(str(params_tbl["protected_root"])).expanduser() + if not params_ref_root.is_dir(): + raise RunnerConfigError( + "params.protected_root must be an existing directory" + ) + + evidence_tbl = data.get("evidence_runner") + evidence_runner_private_key = None + if evidence_tbl is not None: + if not isinstance(evidence_tbl, dict) or set(evidence_tbl) != { + "private_key_file" + }: + raise RunnerConfigError("[evidence_runner] has an invalid exact shape") + evidence_runner_private_key = Path( + str(evidence_tbl["private_key_file"]) + ).expanduser() + if not evidence_runner_private_key.is_file(): + raise RunnerConfigError( + "evidence_runner.private_key_file must be an existing file" + ) + return RunnerConfig( name=name, host=host, @@ -244,4 +497,9 @@ def load_runner_config(path: Optional[Path] = None) -> RunnerConfig: bundles=bundles, backends=tuple(str(b).strip() for b in backends_raw), business_decisions=business_decisions, + local_runtime_release=tuple(local_runtime_release), + product_release_admission=product_release_admission, + workflow_admission=workflow_admission, + params_ref_root=params_ref_root, + evidence_runner_private_key=evidence_runner_private_key, ) diff --git a/openadapt_flow/runner/hosted_adapter.py b/openadapt_flow/runner/hosted_adapter.py new file mode 100644 index 00000000..7fd50a0c --- /dev/null +++ b/openadapt_flow/runner/hosted_adapter.py @@ -0,0 +1,1809 @@ +"""Strict Flow-owned bridge between a hosted lease and governed execution. + +The Desktop host owns HTTP and credential storage. This module owns every +decision that can authorize or classify execution: admission verification, +local trust, input resolution, one-use reservation, managed child execution, +evidence projection, and terminal verification. +""" + +from __future__ import annotations + +import hashlib +import json +import os +import re +import stat +import subprocess +from base64 import b64decode, b64encode +from dataclasses import dataclass, field, replace +from datetime import datetime, timezone +from pathlib import Path, PurePosixPath +from typing import Any, Callable, Literal, Mapping, Protocol, Union +from urllib.parse import urlsplit + +from cryptography.hazmat.primitives import serialization +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey +from pydantic import BaseModel, ConfigDict, Field, model_validator + +from openadapt_flow.ir import RunReport, Workflow +from openadapt_flow.private_file import ( + PrivateFileAclError, + windows_descriptor_has_private_acl, +) +from openadapt_flow.production_qualification import ( + ProductionQualificationAuthority, + ProductionQualificationGuard, + _read_private_json, +) +from openadapt_flow.qualification_admission_v2 import ( + QualificationAdmissionEnvelope, + QualificationAdmissionExpected, + QualificationSignerRegistry, + canonical_json, + contract_sha256, + verify_qualification_admission, +) +from openadapt_flow.runner.commands import build_run_argv +from openadapt_flow.runner.config import RunnerConfig, load_runner_config +from openadapt_flow.runner.dispatch_envelope import write_managed_dispatch_envelope +from openadapt_flow.runner.evidence import failure_events, refusal_events, report_events +from openadapt_flow.runner.inputs import resolve_admitted_params +from openadapt_flow.runner.product_release import ( + ProductReleaseAdmissionArtifact, + ProductReleaseAdmissionPayload, + load_product_release_signer_trust, + verify_product_release_admission, +) +from openadapt_flow.runner.protocol import ( + DispatchParamsValues, + RunnerDispatchPayload, + validate_runtime_param_name, +) +from openadapt_flow.runner.protocol import ( + dispatch_binding_sha256 as governed_dispatch_binding_sha256, +) +from openadapt_flow.runner.verify import Refusal, RefusalCode, verify_dispatch +from openadapt_flow.runtime.authorization import RuntimeParamScalar +from openadapt_flow.runtime.durable.authority import ( + REMOTE_AUTHORITY_TOKEN_ENV, + REMOTE_AUTHORITY_URL_ENV, + REMOTE_DISPATCH_SESSION_ID_ENV, + DurableAuthority, +) +from openadapt_flow.runtime.durable.checkpoint import CheckpointStore +from openadapt_flow.terminal_verification_v2 import ( + ProductionTerminalVerificationContext, + ProductionTerminalVerificationEnvelope, + ProductionTerminalVerificationExpected, + build_production_terminal_verification, + evidence_runner_signer_sha256, + prepare_production_terminal_evidence, + verify_production_terminal_verification_from_report, +) +from openadapt_flow.transaction import ( + DuplicateActuation, + IdempotencyLedger, + TransactionOutcome, + classify_transaction_outcome, +) + +_UUID = r"^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$" +_HEX64 = r"^[a-f0-9]{64}$" +_IDEMPOTENCY = r"^[A-Za-z0-9][A-Za-z0-9._:-]{15,199}$" +_LEASE_TOKEN = r"^oal_[a-f0-9]{64}$" +_RUNNER_TOKEN = r"^oar_[a-f0-9]{64}$" +_SAFE_ID = r"^[A-Za-z0-9][A-Za-z0-9._:@/+\-]{0,199}$" +_UTC_SECONDS = re.compile(r"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$") +_MAX_ARTIFACT_BYTES = 2 * 1024 * 1024 + +# The current runner credential is never part of a request model. Desktop may +# project it into this header only for POST /api/runners/register on the exact +# protected runner origin. +RUNNER_RENEWAL_HEADER = "x-openadapt-runner-renewal-token" + + +def registration_renewal_headers(current_runner_token: str | None) -> dict[str, str]: + """Return the one register-only renewal header without retaining it.""" + + if current_runner_token is None or current_runner_token == "": + return {} + if re.fullmatch(_RUNNER_TOKEN, current_runner_token) is None: + raise ValueError("current runner renewal credential is invalid") + return {RUNNER_RENEWAL_HEADER: current_runner_token} + + +def _utc_seconds(value: str, *, label: str) -> datetime: + if _UTC_SECONDS.fullmatch(value) is None: + raise ValueError(f"{label} is not canonical UTC seconds") + try: + return datetime.fromisoformat(value[:-1] + "+00:00") + except ValueError as exc: + raise ValueError(f"{label} is not canonical UTC seconds") from exc + + +class _Closed(BaseModel): + model_config = ConfigDict(strict=True, extra="forbid", frozen=True) + + +class LocalRuntimeReleaseBinding(_Closed): + target: Literal["flow", "desktop", "capture"] + admission_id: str = Field(pattern=_UUID) + admission_sha256: str = Field(pattern=_HEX64) + release_version: str = Field(pattern=_SAFE_ID) + release_artifact_sha256: str = Field(pattern=_HEX64) + + +CapabilityKind = Literal[ + "web", "windows", "macos", "linux", "rdp", "citrix", "rdp_window" +] + + +class RegisterCapabilities(_Closed): + backends: tuple[CapabilityKind, ...] = Field(min_length=1, max_length=16) + attended: bool + effects_substrates: tuple[CapabilityKind, ...] = Field(min_length=1, max_length=16) + + @model_validator(mode="after") + def _closed_capabilities(self) -> "RegisterCapabilities": + for label, values in ( + ("backend", self.backends), + ("effect substrate", self.effects_substrates), + ): + if len(values) != len(set(values)): + raise ValueError(f"runner {label} capabilities are invalid") + return self + + +class RegisterRequest(_Closed): + schema_version: Literal["openadapt.hosted-runner-registration/v1"] = ( + "openadapt.hosted-runner-registration/v1" + ) + name: str = Field(min_length=1, max_length=80) + platform: Literal["windows", "macos", "linux"] + agent_version: str = Field(min_length=1, max_length=40) + engine_version: str = Field(min_length=1, max_length=40) + mode: Literal["attended", "service"] + capabilities: RegisterCapabilities + local_runtime_release: dict[ + Literal["flow", "desktop", "capture"], LocalRuntimeReleaseBinding + ] + + @model_validator(mode="after") + def _exact_local_targets(self) -> "RegisterRequest": + if set(self.local_runtime_release) != {"flow", "desktop", "capture"} or any( + key != item.target for key, item in self.local_runtime_release.items() + ): + raise ValueError( + "local runtime release targets must be flow, desktop, capture" + ) + return self + + +class RegisterResponse(_Closed): + schema_version: Literal["openadapt.hosted-runner-registration-result/v1"] + runner_id: str = Field(pattern=_UUID) + tenant_id: str = Field(pattern=_UUID) + runner_session_id: str = Field(pattern=_UUID) + runner_token: str = Field(pattern=_RUNNER_TOKEN, repr=False) + token_expires_at: str + + @model_validator(mode="after") + def _canonical_expiry(self) -> "RegisterResponse": + _utc_seconds(self.token_expires_at, label="runner token expiry") + return self + + +class PollRequest(_Closed): + schema_version: Literal["openadapt.hosted-runner-poll/v1"] = ( + "openadapt.hosted-runner-poll/v1" + ) + runner_session_id: str = Field(pattern=_UUID) + wait_seconds: int = Field(ge=0, le=25) + lease_seconds: int = Field(ge=1, le=900) + + +class AdmissionArtifactBytes(_Closed): + artifact_bytes_base64: str = Field(min_length=4, max_length=2_796_204) + artifact_sha256: str = Field(pattern=_HEX64) + + def decode(self) -> bytes: + try: + raw = b64decode(self.artifact_bytes_base64, validate=True) + except ValueError as exc: + raise ValueError("admission artifact is not canonical base64") from exc + if len(raw) > _MAX_ARTIFACT_BYTES: + raise ValueError("admission artifact exceeds the size limit") + if b64encode(raw).decode("ascii") != self.artifact_bytes_base64: + raise ValueError("admission artifact is not canonical base64") + if hashlib.sha256(raw).hexdigest() != self.artifact_sha256: + raise ValueError("admission artifact digest does not match its bytes") + return raw + + @model_validator(mode="after") + def _bytes_match_digest(self) -> "AdmissionArtifactBytes": + self.decode() + return self + + +class HostedDispatch(_Closed): + schema_version: Literal["openadapt.hosted-runner/v1"] + dispatch_id: str = Field(pattern=_UUID) + tenant_id: str = Field(pattern=_UUID) + runner_id: str = Field(pattern=_UUID) + runner_session_id: str = Field(pattern=_UUID) + dispatch_session_id: str = Field(pattern=_UUID) + run_id: str = Field(pattern=_UUID) + workflow_id: str = Field(pattern=_UUID) + workflow_version_id: str = Field(pattern=_UUID) + execution_authority_id: str = Field(pattern=_UUID) + execution_authority_sha256: str = Field(pattern=_HEX64) + execution_authority_signer_sha256: str = Field(pattern=_HEX64) + idempotency_key: str = Field(pattern=_IDEMPOTENCY) + lease_token: str = Field(pattern=_LEASE_TOKEN, repr=False) + lease_expires_at: str + product_release_admission: AdmissionArtifactBytes + workflow_admission: AdmissionArtifactBytes + managed_delivery_authority_url: str = Field(min_length=1, max_length=2048) + delivery_authority_token: str = Field(pattern=_HEX64, repr=False) + payload: RunnerDispatchPayload + + @model_validator(mode="after") + def _exact_run_binding(self) -> "HostedDispatch": + _utc_seconds(self.lease_expires_at, label="hosted lease expiry") + if ( + self.payload.run_id != self.run_id + or self.payload.workflow_id != self.workflow_id + ): + raise ValueError("hosted lease identity does not match its payload") + if self.payload.bundle.version_id != self.workflow_version_id: + raise ValueError("hosted lease workflow version does not match its bundle") + return self + + +class HostedRecoveryBinding(_Closed): + """Callback state without params or the delivery-authority credential. + + This projection remains credential-bearing because it retains the lease + token required for the exact terminal callback. + """ + + schema_version: Literal["openadapt.hosted-runner-recovery/v1"] = ( + "openadapt.hosted-runner-recovery/v1" + ) + dispatch_id: str = Field(pattern=_UUID) + runner_session_id: str = Field(pattern=_UUID) + dispatch_session_id: str = Field(pattern=_UUID) + run_id: str = Field(pattern=_UUID) + workflow_id: str = Field(pattern=_UUID) + idempotency_key: str = Field(pattern=_IDEMPOTENCY) + lease_token: str = Field(pattern=_LEASE_TOKEN, repr=False) + product_release_admission_sha256: str = Field(pattern=_HEX64) + workflow_admission_sha256: str = Field(pattern=_HEX64) + bundle_content_digest: str = Field(pattern=_HEX64) + authorization_id: str = Field(min_length=1, max_length=128) + + +class HostedTerminalEvent(_Closed): + schema_version: Literal["openadapt.hosted-runner-terminal/v1"] = ( + "openadapt.hosted-runner-terminal/v1" + ) + run_id: str = Field(pattern=_UUID) + outcome: Literal[ + "VERIFIED", + "HALTED_BEFORE_EFFECT", + "RECONCILIATION_REQUIRED", + "FAILED_PLATFORM", + "CANCELED", + "REJECTED_POLICY", + "COMPLETED_UNVERIFIED", + "ROLLED_BACK", + ] + report_sha256: str = Field(pattern=_HEX64) + started: bool + uncertain_delivery: bool + terminal_verification_artifact_bytes_base64: str | None = Field( + default=None, max_length=2_796_204 + ) + terminal_verification_artifact_sha256: str | None = Field( + default=None, pattern=_HEX64 + ) + + @model_validator(mode="after") + def _terminal_outcome_requires_exact_proof(self) -> "HostedTerminalEvent": + has_proof = self.terminal_verification_artifact_bytes_base64 is not None + if has_proof != (self.terminal_verification_artifact_sha256 is not None): + raise ValueError("terminal verification binding is incomplete") + if self.outcome in {"VERIFIED", "HALTED_BEFORE_EFFECT"} and not has_proof: + raise ValueError(f"{self.outcome} requires exact terminal verification") + if ( + self.outcome + not in { + "VERIFIED", + "HALTED_BEFORE_EFFECT", + "RECONCILIATION_REQUIRED", + } + and has_proof + ): + raise ValueError("terminal callback outcome cannot carry a v2 proof") + if has_proof: + assert self.terminal_verification_artifact_bytes_base64 is not None + assert self.terminal_verification_artifact_sha256 is not None + try: + raw = b64decode( + self.terminal_verification_artifact_bytes_base64, + validate=True, + ) + proof = ProductionTerminalVerificationEnvelope.model_validate_json(raw) + except (ValueError, TypeError) as exc: + raise ValueError("terminal verification artifact is invalid") from exc + if ( + len(raw) > _MAX_ARTIFACT_BYTES + or b64encode(raw).decode("ascii") + != self.terminal_verification_artifact_bytes_base64 + or canonical_json(proof) != raw + or proof.artifact_sha256() != self.terminal_verification_artifact_sha256 + ): + raise ValueError("terminal verification artifact binding is invalid") + if ( + proof.payload.run_id != self.run_id + or proof.payload.run_report_sha256 != self.report_sha256 + or proof.payload.run_report_object_sha256 != self.report_sha256 + ): + raise ValueError("terminal verification names a different run report") + if proof.payload.run_receipt.transaction_outcome != self.outcome: + raise ValueError("terminal verification names a different outcome") + return self + + +class HostedRunResult(_Closed): + kind: Literal["result"] = "result" + dispatch_id: str = Field(pattern=_UUID) + run_id: str = Field(pattern=_UUID) + outcome: TransactionOutcome + evidence_batch: tuple[dict[str, Any], ...] + terminal_verification: ProductionTerminalVerificationEnvelope | None = None + started: bool + uncertain_delivery: bool + report_sha256: str = Field(pattern=_HEX64) + + @model_validator(mode="after") + def _closed_terminal(self) -> "HostedRunResult": + if ( + self.outcome + in { + TransactionOutcome.VERIFIED, + TransactionOutcome.HALTED_BEFORE_EFFECT, + } + and self.terminal_verification is None + ): + raise ValueError("closed terminal outcome requires a signed v2 proof") + if self.terminal_verification is not None and self.outcome not in { + TransactionOutcome.VERIFIED, + TransactionOutcome.HALTED_BEFORE_EFFECT, + TransactionOutcome.RECONCILIATION_REQUIRED, + }: + raise ValueError("terminal outcome cannot carry a signed v2 proof") + if self.uncertain_delivery != ( + self.outcome is TransactionOutcome.RECONCILIATION_REQUIRED + ): + raise ValueError("uncertain delivery has an invalid terminal outcome") + if self.terminal_verification is not None and ( + self.terminal_verification.payload.run_id != self.run_id + or self.terminal_verification.payload.run_report_sha256 + != self.report_sha256 + or self.terminal_verification.payload.run_report_object_sha256 + != self.report_sha256 + or self.terminal_verification.payload.run_receipt.transaction_outcome + != self.outcome.value + ): + raise ValueError( + "terminal verification names a different run report or outcome" + ) + return self + + +class HostedDispatchRefusal(_Closed): + kind: Literal["refusal"] = "refusal" + dispatch_id: str | None = None + run_id: str | None = None + code: str = Field(min_length=1, max_length=64) + detail: str = Field(min_length=1, max_length=400) + evidence_batch: tuple[dict[str, Any], ...] = () + started: Literal[False] = False + uncertain_delivery: Literal[False] = False + outcome: Literal["REJECTED_POLICY"] = "REJECTED_POLICY" + report_sha256: str = Field(default="0" * 64, pattern=_HEX64) + + +class CallbackRequest(_Closed): + schema_version: Literal["openadapt.hosted-runner-callback/v1"] = ( + "openadapt.hosted-runner-callback/v1" + ) + dispatch_id: str = Field(pattern=_UUID) + runner_session_id: str = Field(pattern=_UUID) + idempotency_key: str = Field(pattern=_IDEMPOTENCY) + lease_token: str = Field(pattern=_LEASE_TOKEN, repr=False) + product_release_admission_sha256: str = Field(pattern=_HEX64) + workflow_admission_sha256: str = Field(pattern=_HEX64) + events: tuple[dict[str, Any], ...] = Field(min_length=1, max_length=10_001) + + +class CallbackResponse(_Closed): + schema_version: Literal["openadapt.hosted-runner-callback-result/v1"] + status: Literal["accepted", "duplicate"] + run_id: str = Field(pattern=_UUID) + outcome: TransactionOutcome + dispatch_state: Literal["closed"] + accepted_events: int = Field(ge=0, le=10_001) + + +class HostedRunnerTransport(Protocol): + """Desktop-owned HTTP surface. Credentials stay in its transport state.""" + + def register(self, request: RegisterRequest) -> RegisterResponse: ... + + def poll(self, request: PollRequest) -> HostedDispatch | None: ... + + def callback(self, run_id: str, request: CallbackRequest) -> CallbackResponse: ... + + +@dataclass(frozen=True) +class DeliveryAuthority: + """Run-scoped configuration for the existing per-input-edge authority path.""" + + url: str + token: str = field(repr=False) + + def __post_init__(self) -> None: + try: + parsed = urlsplit(self.url) + except ValueError as exc: + raise ValueError("managed delivery authority URL is invalid") from exc + if ( + parsed.scheme != "https" + or not parsed.hostname + or parsed.username is not None + or parsed.password is not None + or parsed.query + or parsed.fragment + or parsed.path != "/api/internal/managed-delivery-permit" + ): + raise ValueError( + "managed delivery authority URL is not a pinned HTTPS edge" + ) + if re.fullmatch(_HEX64, self.token) is None: + raise ValueError("managed delivery authority token is invalid") + + def child_environment(self) -> dict[str, str]: + return { + REMOTE_AUTHORITY_URL_ENV: self.url, + REMOTE_AUTHORITY_TOKEN_ENV: self.token, + } + + +@dataclass(frozen=True) +class ManagedExecution: + returncode: int + report_bytes: bytes | None + terminal_verification: ProductionTerminalVerificationEnvelope | None = None + + +ManagedRunner = Callable[[list[str], Path, Mapping[str, str]], ManagedExecution] + + +def _subprocess_runner( + argv: list[str], run_dir: Path, child_env: Mapping[str, str] +) -> ManagedExecution: + process = subprocess.run( # nosec - argv is built from verified local material + argv, + capture_output=True, + text=True, + env=dict(child_env), + ) + report_path = run_dir / "report.json" + report_bytes = report_path.read_bytes() if report_path.is_file() else None + return ManagedExecution(process.returncode, report_bytes) + + +class HostedRunnerAdapter: + def __init__( + self, + ledger_path: Path, + *, + runner: ManagedRunner = _subprocess_runner, + ) -> None: + self.ledger_path = Path(ledger_path) + self._ledger = IdempotencyLedger( + self.ledger_path, namespace="openadapt-hosted-runner/v1" + ) + self._runner = runner + self._release_state_path = self.ledger_path.with_suffix( + self.ledger_path.suffix + ".product-release.json" + ) + + @staticmethod + def _protected_runner_origin(config: RunnerConfig) -> str: + raw = config.host + if raw is None: + raise ValueError("hosted runner requires a protected runner host origin") + try: + parsed = urlsplit(raw) + port = parsed.port + except ValueError as exc: + raise ValueError("protected runner host origin is invalid") from exc + canonical = f"https://{parsed.netloc}" + if ( + parsed.scheme != "https" + or not parsed.hostname + or parsed.hostname != parsed.hostname.lower() + or parsed.netloc != parsed.netloc.lower() + or parsed.username is not None + or parsed.password is not None + or parsed.path + or parsed.query + or parsed.fragment + or port == 443 + or raw != canonical + ): + raise ValueError("protected runner host is not one canonical HTTPS origin") + return canonical + + def protected_runner_origin(self, runner_config: Path) -> str: + """Return the origin from one protected, strictly parsed runner config.""" + + return self._protected_runner_origin( + load_runner_config(runner_config, protected=True) + ) + + def registration_request( + self, + *, + runner_config: Path, + name: str, + platform: Literal["windows", "macos", "linux"], + agent_version: str, + engine_version: str, + mode: Literal["attended", "service"], + capabilities: RegisterCapabilities | Mapping[str, object], + ) -> RegisterRequest: + config = load_runner_config(runner_config, protected=True) + self._protected_runner_origin(config) + releases = config.local_runtime_release + if tuple(item.target for item in releases) != ("flow", "desktop", "capture"): + raise ValueError( + "hosted registration requires exact flow, desktop, and capture releases" + ) + if not isinstance(capabilities, RegisterCapabilities): + if not isinstance(capabilities, Mapping) or set(capabilities) != { + "backends", + "attended", + "effects_substrates", + }: + raise ValueError("runner capabilities have an invalid exact shape") + backends = capabilities["backends"] + effects = capabilities["effects_substrates"] + attended = capabilities["attended"] + if ( + not isinstance(backends, (list, tuple)) + or not isinstance(effects, (list, tuple)) + or type(attended) is not bool + ): + raise ValueError("runner capabilities have an invalid exact shape") + capabilities = RegisterCapabilities( + backends=tuple(backends), + attended=attended, + effects_substrates=tuple(effects), + ) + return RegisterRequest( + name=name, + platform=platform, + agent_version=agent_version, + engine_version=engine_version, + mode=mode, + capabilities=capabilities, + local_runtime_release={ + item.target: LocalRuntimeReleaseBinding(**item.__dict__) + for item in releases + }, + ) + + @staticmethod + def _load_json(path: Path) -> object: + try: + return _read_private_json(path) + except (OSError, ValueError) as exc: + raise ValueError(f"admission trust state {path} is invalid") from exc + + @staticmethod + def _read_private_bytes(path: Path, *, maximum_bytes: int, label: str) -> bytes: + """Read one owner-only regular file without following a final link.""" + + path = Path(path) + flags = os.O_RDONLY + if hasattr(os, "O_NOFOLLOW"): + flags |= os.O_NOFOLLOW + try: + path_before = path.lstat() + if not stat.S_ISREG(path_before.st_mode) or stat.S_ISLNK( + path_before.st_mode + ): + raise ValueError(f"{label} is not a private regular file") + descriptor = os.open(path, flags) + except OSError as exc: + raise ValueError(f"{label} could not be opened safely") from exc + try: + before = os.fstat(descriptor) + try: + private_permissions = ( + windows_descriptor_has_private_acl(descriptor) + if os.name == "nt" + else ( + before.st_uid == os.geteuid() + and stat.S_IMODE(before.st_mode) == 0o600 + ) + ) + except PrivateFileAclError as exc: + raise ValueError(f"{label} ACL could not be verified") from exc + if ( + not stat.S_ISREG(before.st_mode) + or before.st_size > maximum_bytes + or not private_permissions + ): + raise ValueError(f"{label} is not a private regular file") + chunks: list[bytes] = [] + remaining = min(before.st_size, maximum_bytes) + 1 + while remaining: + chunk = os.read(descriptor, min(remaining, 64 * 1024)) + if not chunk: + break + chunks.append(chunk) + remaining -= len(chunk) + raw = b"".join(chunks) + after = os.fstat(descriptor) + try: + path_after = path.lstat() + except OSError as exc: + raise ValueError(f"{label} changed during its protected read") from exc + if ( + len(raw) != before.st_size + or len(raw) > maximum_bytes + or stat.S_ISLNK(path_after.st_mode) + or (path_before.st_dev, path_before.st_ino) + != (before.st_dev, before.st_ino) + or (path_after.st_dev, path_after.st_ino) + != (before.st_dev, before.st_ino) + or (before.st_dev, before.st_ino, before.st_size, before.st_mtime_ns) + != (after.st_dev, after.st_ino, after.st_size, after.st_mtime_ns) + ): + raise ValueError(f"{label} changed during its protected read") + return raw + finally: + os.close(descriptor) + + def _load_evidence_private_key(self, config: RunnerConfig) -> Ed25519PrivateKey: + path = config.evidence_runner_private_key + if path is None: + raise ValueError("hosted runner has no evidence-runner private key") + raw = self._read_private_bytes( + path, maximum_bytes=4096, label="evidence-runner private key" + ) + try: + if len(raw) == 32: + key = Ed25519PrivateKey.from_private_bytes(raw) + else: + loaded = serialization.load_pem_private_key(raw, password=None) + if not isinstance(loaded, Ed25519PrivateKey): + raise ValueError("evidence-runner key is not Ed25519") + key = loaded + except (TypeError, ValueError) as exc: + raise ValueError("evidence-runner private key is invalid") from exc + return key + + def _accept_newest_product_sequence( + self, payload: ProductReleaseAdmissionPayload, artifact_sha256: str + ) -> None: + current: dict[str, object] | None = None + if self._release_state_path.exists(): + metadata = self._release_state_path.lstat() + if not stat.S_ISREG(metadata.st_mode) or ( + os.name != "nt" and stat.S_IMODE(metadata.st_mode) != 0o600 + ): + raise ValueError("product release sequence ledger is unsafe") + loaded = self._load_json(self._release_state_path) + if not isinstance(loaded, dict): + raise ValueError("product release sequence ledger is invalid") + current = loaded + if current is not None: + sequence = current.get("sequence") + digest = current.get("artifact_sha256") + if not isinstance(sequence, int) or not isinstance(digest, str): + raise ValueError("product release sequence ledger is invalid") + if payload.sequence < sequence: + raise ValueError("product release admission sequence is stale") + if payload.sequence == sequence and artifact_sha256 != digest: + raise ValueError("product release admission changed at one sequence") + if payload.sequence == sequence: + return + raw = json.dumps( + {"sequence": payload.sequence, "artifact_sha256": artifact_sha256}, + sort_keys=True, + separators=(",", ":"), + ).encode("utf-8") + self._release_state_path.parent.mkdir(parents=True, exist_ok=True) + tmp = self._release_state_path.with_suffix( + self._release_state_path.suffix + ".tmp" + ) + descriptor = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) + try: + os.write(descriptor, raw) + os.fsync(descriptor) + finally: + os.close(descriptor) + os.replace(tmp, self._release_state_path) + if os.name != "nt": + os.chmod(self._release_state_path, 0o600) + + def _verify_product_release( + self, dispatch: HostedDispatch, config: RunnerConfig + ) -> ProductReleaseAdmissionPayload: + trust_files = config.product_release_admission + if trust_files is None: + raise ValueError("hosted runner has no product release admission trust") + raw = dispatch.product_release_admission.decode() + artifact = ProductReleaseAdmissionArtifact.model_validate_json(raw) + if ( + artifact.artifact_sha256() + != dispatch.product_release_admission.artifact_sha256 + ): + raise ValueError("product release artifact canonical digest changed") + trust = load_product_release_signer_trust( + self._load_json(trust_files.signer_registry) + ) + state = self._load_json(trust_files.state) + if not isinstance(state, dict) or set(state) != { + "newest_sequence", + "revoked_set_ids", + }: + raise ValueError("product release authority state is invalid") + newest = state["newest_sequence"] + revoked = state["revoked_set_ids"] + if ( + not isinstance(newest, int) + or not isinstance(revoked, list) + or any(not isinstance(item, str) for item in revoked) + ): + raise ValueError("product release authority state is invalid") + payload = verify_product_release_admission( + artifact, + trusted_signers=trust, + newest_sequence=newest, + revoked_set_ids=frozenset(revoked), + ) + local = {item.target: item for item in config.local_runtime_release} + if set(local) != {"flow", "desktop", "capture"}: + raise ValueError("hosted runner local release inventory is incomplete") + admitted = {item.target: item for item in payload.targets} + for target, installed in local.items(): + item = admitted[target] + if ( + installed.admission_id, + installed.admission_sha256, + installed.release_version, + installed.release_artifact_sha256, + ) != ( + item.admission_id, + item.admission_sha256, + item.release_id, + item.release_artifact_sha256, + ): + raise ValueError(f"local {target} release is not exactly admitted") + self._accept_newest_product_sequence( + payload, dispatch.product_release_admission.artifact_sha256 + ) + return payload + + def _verify_workflow_admission( + self, + dispatch: HostedDispatch, + config: RunnerConfig, + *, + evidence_private_key: Ed25519PrivateKey, + ) -> tuple[ProductionQualificationAuthority, bytes]: + trust_files = config.workflow_admission + if trust_files is None: + raise ValueError("hosted runner has no workflow admission trust") + raw = dispatch.workflow_admission.decode() + envelope = QualificationAdmissionEnvelope.model_validate_json(raw) + canonical = json.dumps( + envelope.model_dump(mode="json"), + sort_keys=True, + separators=(",", ":"), + ensure_ascii=False, + ).encode("utf-8") + if canonical != raw or ( + envelope.artifact_sha256() != dispatch.workflow_admission.artifact_sha256 + ): + raise ValueError("workflow admission canonical digest changed") + authorization = dispatch.payload.authorization + local_fields = tuple( + name + for name in authorization.model_fields + if name.startswith("production_qualification_") + ) + ("qualification_admission", "qualification_admission_sha256") + if any(getattr(authorization, name) is not None for name in local_fields): + raise ValueError("dispatch authorization supplies runner-local authority") + state = self._load_json(trust_files.state) + if not isinstance(state, dict) or set(state) != {"revoked_admission_ids"}: + raise ValueError("workflow admission authority state is invalid") + revoked = state["revoked_admission_ids"] + if not isinstance(revoked, list) or any( + not isinstance(item, str) for item in revoked + ): + raise ValueError("workflow admission authority state is invalid") + registry_raw = self._load_json(trust_files.signer_registry) + registry = QualificationSignerRegistry.model_validate(registry_raw) + if registry.model_dump(mode="json") != registry_raw: + raise ValueError("workflow signer registry is not canonical") + expected_raw = self._load_json(trust_files.expected_bindings) + expected = QualificationAdmissionExpected.model_validate(expected_raw) + if expected.model_dump(mode="json") != expected_raw: + raise ValueError("workflow admission expected bindings are not canonical") + + trusted = config.bundles.get(dispatch.payload.bundle.content_digest) + if trusted is None or trusted.artifact_sha256 is None: + raise ValueError("hosted bundle lacks its local artifact digest pin") + workflow = Workflow.load(trusted.path) + manifest = workflow.manifest + project = workflow.qualification + if manifest is None or project is None: + raise ValueError("hosted workflow is not sealed and qualified") + template = manifest.provenance.governed_authorization_template + if template is None: + raise ValueError("hosted workflow lacks its governed template") + profile_path = config.profiles.get(dispatch.payload.deployment_profile_id) + if profile_path is None: + raise ValueError("hosted workflow profile is not locally configured") + deployment_bytes = self._read_private_bytes( + profile_path, + maximum_bytes=1024 * 1024, + label="hosted deployment profile", + ) + deployment_sha256 = hashlib.sha256(deployment_bytes).hexdigest() + effect_contract_sha256 = contract_sha256( + [ + item.model_dump(mode="json") + for item in template.qualified_effect_requirements + ] + ) + public_key = evidence_private_key.public_key().public_bytes( + encoding=serialization.Encoding.Raw, + format=serialization.PublicFormat.Raw, + ) + evidence_key_sha256 = evidence_runner_signer_sha256(public_key) + local_flow = next( + (item for item in config.local_runtime_release if item.target == "flow"), + None, + ) + if local_flow is None: + raise ValueError("hosted runner lacks its local Flow release binding") + local_expected = { + "tenant_id": dispatch.tenant_id, + "workflow_id": dispatch.workflow_id, + "workflow_version_id": dispatch.workflow_version_id, + "bundle_version_id": dispatch.workflow_version_id, + "bundle_artifact_sha256": trusted.artifact_sha256, + "bundle_content_digest": manifest.content_digest, + "environment_digest": project.environment.environment_digest, + "governed_authorization_template_sha256": template.template_sha256, + "environment_contract_sha256": ( + template.qualification_environment_contract_sha256 + ), + "input_policy_sha256": template.parameter_contract_sha256, + "action_policy_sha256": template.qualification_project_contract_sha256, + "identity_contract_sha256": template.identity_contract_sha256, + "effect_contract_sha256": effect_contract_sha256, + "evidence_runner_signer_sha256": evidence_key_sha256, + "deployment_manifest_sha256": deployment_sha256, + } + mismatches = sorted( + name + for name, value in local_expected.items() + if getattr(expected, name) != value + ) + runtime = expected.runtime_build_identity + if ( + runtime.flow_version != local_flow.release_version + or runtime.flow_wheel_sha256 != local_flow.release_artifact_sha256 + ): + mismatches.append("runtime_build_identity") + if mismatches: + raise ValueError( + "local workflow admission expectation differs from live state: " + + ", ".join(sorted(set(mismatches))) + ) + verify_qualification_admission( + envelope, + registry=registry, + expected=expected, + revoked_admission_ids=frozenset(revoked), + ) + return ( + ProductionQualificationAuthority( + qualification_admission=envelope, + qualification_admission_sha256=envelope.artifact_sha256(), + expected=expected, + qualification_signer_registry=registry, + qualification_signer_registry_sha256=registry.artifact_sha256(), + permit_trust_snapshot=None, + revoked_admission_ids=tuple(sorted(set(revoked))), + ), + deployment_bytes, + ) + + @staticmethod + def _write_private_json( + path: Path, value: BaseModel | Mapping[str, object] + ) -> Path: + payload: object = ( + value.model_dump(mode="json") if isinstance(value, BaseModel) else value + ) + raw = json.dumps( + payload, sort_keys=True, separators=(",", ":"), ensure_ascii=False + ).encode("utf-8") + flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL + if hasattr(os, "O_NOFOLLOW"): + flags |= os.O_NOFOLLOW + descriptor = os.open(path, flags, 0o600) + try: + os.write(descriptor, raw) + os.fsync(descriptor) + finally: + os.close(descriptor) + return path + + @staticmethod + def _write_private_bytes(path: Path, raw: bytes) -> Path: + flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL + if hasattr(os, "O_NOFOLLOW"): + flags |= os.O_NOFOLLOW + descriptor = os.open(path, flags, 0o600) + try: + offset = 0 + while offset < len(raw): + written = os.write(descriptor, raw[offset:]) + if written <= 0: + raise OSError("protected file write did not make progress") + offset += written + os.fsync(descriptor) + finally: + os.close(descriptor) + return path + + def _resolve_params( + self, + dispatch: HostedDispatch, + config: RunnerConfig, + ) -> dict[str, RuntimeParamScalar]: + trusted = config.bundles.get(dispatch.payload.bundle.content_digest) + if trusted is None: + raise ValueError("hosted bundle is not locally trusted") + workflow = Workflow.load(trusted.path) + if isinstance(dispatch.payload.params, DispatchParamsValues): + supplied = dict(dispatch.payload.params.values) + inline = True + else: + root = config.params_ref_root + if root is None: + raise ValueError("parameter reference has no protected local root") + ref = dispatch.payload.params.ref + parsed_url = urlsplit(ref) + relative = PurePosixPath(ref) + if ( + parsed_url.scheme + or parsed_url.netloc + or parsed_url.query + or parsed_url.fragment + or relative.is_absolute() + or not relative.parts + or any(part in {"", ".", ".."} for part in relative.parts) + or "\\" in ref + ): + raise ValueError("parameter reference is not a safe local path") + root_stat = root.lstat() + if ( + not stat.S_ISDIR(root_stat.st_mode) + or stat.S_ISLNK(root_stat.st_mode) + or ( + os.name != "nt" + and ( + root_stat.st_uid != os.geteuid() + or stat.S_IMODE(root_stat.st_mode) & 0o077 + ) + ) + ): + raise ValueError("parameter reference root is not protected") + current = root + for component in relative.parts[:-1]: + current /= component + component_stat = current.lstat() + if ( + not stat.S_ISDIR(component_stat.st_mode) + or stat.S_ISLNK(component_stat.st_mode) + or ( + os.name != "nt" + and ( + component_stat.st_uid != os.geteuid() + or stat.S_IMODE(component_stat.st_mode) & 0o077 + ) + ) + ): + raise ValueError("parameter reference traverses an unsafe path") + raw = self._read_private_bytes( + root.joinpath(*relative.parts), + maximum_bytes=256 * 1024, + label="parameter reference", + ) + try: + supplied = json.loads(raw.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError) as exc: + raise ValueError("parameter reference is not valid JSON") from exc + if not isinstance(supplied, dict): + raise ValueError("parameter reference has an invalid exact shape") + inline = False + for name in supplied: + if not isinstance(name, str): + raise ValueError("runtime parameter name is invalid") + validate_runtime_param_name(name) + return resolve_admitted_params(workflow, supplied, inline=inline) + + @staticmethod + def _write_params(path: Path, params: dict[str, RuntimeParamScalar]) -> Path | None: + if not params: + return None + flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL + if hasattr(os, "O_NOFOLLOW"): + flags |= os.O_NOFOLLOW + descriptor = os.open(path, flags, 0o600) + try: + raw = json.dumps( + params, + sort_keys=True, + separators=(",", ":"), + allow_nan=False, + ).encode() + os.write(descriptor, raw) + os.fsync(descriptor) + finally: + os.close(descriptor) + return path + + def _produce_terminal_verification( + self, + *, + dispatch: HostedDispatch, + report: RunReport, + run_dir: Path, + qualification: ProductionQualificationAuthority, + private_key: Ed25519PrivateKey, + verified_params: dict[str, RuntimeParamScalar], + dispatch_binding_sha256: str, + ) -> tuple[ProductionTerminalVerificationEnvelope, str]: + """Build, retain, reread, and verify one exact terminal-v2 proof.""" + + store = CheckpointStore(run_dir) + manifest = store.read_manifest() + if ( + manifest is None + or manifest.delivery_authority_kind != "cloud_runner" + or manifest.remote_delivery_run_id != dispatch.run_id + or manifest.managed_dispatch_binding_sha256 != dispatch_binding_sha256 + or manifest.params != verified_params + ): + raise ValueError("retained managed run manifest differs from the dispatch") + authorization = manifest.governed_authorization + admission = qualification.qualification_admission + evidence_identity = admission.payload.evidence_identity + expected_production_binding = { + "production_qualification_admission_id": admission.payload.admission_id, + "production_qualification_admission_sha256": ( + qualification.qualification_admission_sha256 + ), + "production_qualification_evidence_identity_sha256": ( + evidence_identity.artifact_sha256() + ), + "production_qualification_runtime_validation_id": ( + qualification.expected.runtime_validation_id + ), + "production_qualification_signer_registry_sha256": ( + qualification.qualification_signer_registry_sha256 + ), + "production_qualification_signer_registry_revision": ( + qualification.qualification_signer_registry.revision + ), + "production_qualification_signer_registry_expires_at": ( + qualification.qualification_signer_registry.expires_at + ), + "production_qualification_authority_sha256": ( + qualification.immutable_binding_sha256() + ), + } + if authorization is None or any( + getattr(authorization, field) != value + for field, value in expected_production_binding.items() + ): + raise ValueError("retained production authority differs from admission") + if ( + dispatch_binding_sha256 != dispatch.payload.dispatch_binding_sha256 + or governed_dispatch_binding_sha256(dispatch.run_id, authorization) + != dispatch.payload.dispatch_binding_sha256 + ): + raise ValueError("retained governed authorization differs from dispatch") + qualification_case_id_sha256 = ( + None + if authorization.qualification_case_id is None + else hashlib.sha256( + authorization.qualification_case_id.encode("utf-8") + ).hexdigest() + ) + authorized_effect_contracts = { + approval.step_id: list(approval.effect_contract_hashes) + for approval in authorization.unverified_write_approvals + } + if ( + report.params != verified_params + or report.governed_authorization_id != authorization.authorization_id + or report.governed_authorization_created_at != authorization.created_at + or report.governed_runtime_inputs_digest + != authorization.runtime_inputs_digest + or report.governed_policy_name != authorization.admitted_policy_name + or report.governed_policy_contract_sha256 + != authorization.admitted_policy_contract_sha256 + or report.governed_minimum_effect_tier != authorization.minimum_effect_tier + or report.governed_approval_source != authorization.approval_source + or report.execution_profile != authorization.execution_profile + or tuple(report.governed_qualified_effect_requirements) + != authorization.qualified_effect_requirements + or tuple(report.required_identity_step_ids) + != authorization.required_identity_step_ids + or report.approved_unverified_effect_step_ids + != [item.step_id for item in authorization.unverified_write_approvals] + or report.governed_authorized_effect_contracts + != authorized_effect_contracts + or report.governed_qualification_project_id + != authorization.qualification_project_id + or report.governed_qualification_project_revision + != authorization.qualification_project_revision + or report.governed_qualification_project_contract_sha256 + != authorization.qualification_project_contract_sha256 + or report.governed_qualification_campaign_id_sha256 + != authorization.qualification_campaign_id_sha256 + or report.governed_qualification_case_id_sha256 + != qualification_case_id_sha256 + or report.governed_qualification_case_input_sha256 + != authorization.qualification_case_input_sha256 + or report.governed_qualification_run_id_sha256 + != authorization.qualification_run_id_sha256 + or report.governed_qualification_case_kind + != authorization.qualification_case_kind + or report.governed_qualification_case_action_paths + != authorization.qualification_case_action_paths + or report.governed_qualification_fault_driver_id + != authorization.qualification_fault_driver_id + or report.governed_qualification_fault_driver_contract_sha256 + != authorization.qualification_fault_driver_contract_sha256 + or report.governed_qualification_fault_driver_key_id + != authorization.qualification_fault_driver_key_id + or report.governed_qualification_fault_step_id_sha256 + != authorization.qualification_fault_step_id_sha256 + ): + raise ValueError("run report differs from retained governed inputs") + + prepared = prepare_production_terminal_evidence(report) + chain = DurableAuthority(run_dir, store).production_delivery_permit_chain( + allow_empty=( + prepared.transaction_outcome is TransactionOutcome.HALTED_BEFORE_EFFECT + ) + ) + first = chain.entries[0] if chain.entries else None + expected = qualification.expected + runtime = expected.runtime_build_identity + flow_run_id_sha256 = hashlib.sha256(dispatch.run_id.encode("utf-8")).hexdigest() + runner_id_sha256 = hashlib.sha256( + dispatch.runner_id.encode("utf-8") + ).hexdigest() + runner_session_id_sha256 = hashlib.sha256( + dispatch.runner_session_id.encode("utf-8") + ).hexdigest() + if first is not None and ( + first.run_id != dispatch.run_id + or first.flow_run_id_sha256 != flow_run_id_sha256 + or first.execution_authority_id != dispatch.execution_authority_id + or first.execution_authority_sha256 != dispatch.execution_authority_sha256 + or first.authority_signer_sha256 + != dispatch.execution_authority_signer_sha256 + or first.admission_artifact_sha256 + != qualification.qualification_admission_sha256 + or first.evidence_identity_sha256 + != admission.payload.evidence_identity.artifact_sha256() + or first.environment_digest != expected.environment_digest + or first.qualification_signer_registry_sha256 + != qualification.qualification_signer_registry_sha256 + or first.qualification_signer_registry_revision + != qualification.qualification_signer_registry.revision + or first.authenticated_runner_id_sha256 != runner_id_sha256 + or first.authenticated_session_id_sha256 != runner_session_id_sha256 + ): + raise ValueError("retained delivery chain differs from admitted live state") + + now = datetime.now(timezone.utc).replace(microsecond=0) + now_text = now.isoformat().replace("+00:00", "Z") + context = ProductionTerminalVerificationContext( + run_id=dispatch.run_id, + tenant_id=dispatch.tenant_id, + workflow_id=dispatch.workflow_id, + workflow_version_id=dispatch.workflow_version_id, + bundle_version_id=dispatch.workflow_version_id, + bundle_artifact_sha256=expected.bundle_artifact_sha256, + environment_digest=expected.environment_digest, + environment_contract_sha256=expected.environment_contract_sha256, + runtime_environment_sha256=expected.runtime_environment_sha256, + identity_contract_sha256=expected.identity_contract_sha256, + effect_contract_sha256=expected.effect_contract_sha256, + runtime_validation_id=expected.runtime_validation_id, + runtime_substrate=runtime.substrate, + admission_id=admission.payload.admission_id, + admission_artifact_sha256=(qualification.qualification_admission_sha256), + admission_policy_sha256=evidence_identity.admission_policy_sha256, + evidence_identity_sha256=evidence_identity.artifact_sha256(), + admitted_runtime_build_sha256=runtime.artifact_sha256(), + evidence_runner_signer_sha256=expected.evidence_runner_signer_sha256, + qualification_signer_registry_sha256=( + qualification.qualification_signer_registry_sha256 + ), + qualification_signer_registry_revision=( + qualification.qualification_signer_registry.revision + ), + execution_authority_id=dispatch.execution_authority_id, + execution_authority_sha256=dispatch.execution_authority_sha256, + execution_authority_signer_sha256=( + dispatch.execution_authority_signer_sha256 + ), + permit_chain=chain, + run_report_object_version="sha256:" + prepared.report_sha256, + verified_at=now_text, + issued_at=now_text, + ) + built = build_production_terminal_verification( + report, + context=context, + private_key=private_key, + ) + if ( + built.report_bytes != prepared.report_bytes + or built.report_sha256 != prepared.report_sha256 + ): + raise ValueError("terminal report changed during proof production") + envelope_bytes = canonical_json(built.envelope) + payload = built.envelope.payload + final = chain.entries[-1] if chain.entries else None + live_expected = ProductionTerminalVerificationExpected( + run_id=dispatch.run_id, + flow_run_id_sha256=flow_run_id_sha256, + tenant_id=dispatch.tenant_id, + workflow_id=dispatch.workflow_id, + workflow_version_id=dispatch.workflow_version_id, + bundle_version_id=dispatch.workflow_version_id, + bundle_artifact_sha256=expected.bundle_artifact_sha256, + bundle_content_digest=expected.bundle_content_digest, + environment_digest=expected.environment_digest, + environment_contract_sha256=expected.environment_contract_sha256, + runtime_environment_sha256=expected.runtime_environment_sha256, + identity_contract_sha256=expected.identity_contract_sha256, + effect_contract_sha256=expected.effect_contract_sha256, + runtime_validation_id=expected.runtime_validation_id, + runtime_substrate=runtime.substrate, + admission_id=admission.payload.admission_id, + admission_artifact_sha256=(qualification.qualification_admission_sha256), + admission_policy_sha256=evidence_identity.admission_policy_sha256, + evidence_identity_sha256=evidence_identity.artifact_sha256(), + admitted_runtime_build_sha256=runtime.artifact_sha256(), + evidence_runner_signer_sha256=expected.evidence_runner_signer_sha256, + qualification_signer_registry_sha256=( + qualification.qualification_signer_registry_sha256 + ), + qualification_signer_registry_revision=( + qualification.qualification_signer_registry.revision + ), + execution_authority_id=dispatch.execution_authority_id, + execution_authority_sha256=dispatch.execution_authority_sha256, + execution_authority_signer_sha256=( + dispatch.execution_authority_signer_sha256 + ), + permit_chain_sha256=chain.permit_chain_sha256, + permit_count=len(chain.entries), + final_authority_sequence=( + final.authority_sequence if final is not None else 0 + ), + final_runtime_delivery_sequence=( + final.runtime_delivery_sequence if final is not None else 0 + ), + authenticated_runner_id_sha256=runner_id_sha256, + authenticated_session_id_sha256=runner_session_id_sha256, + acknowledged_one_use_claim_ids=tuple( + item.one_use_claim_id for item in chain.entries + ), + workflow_contract_sha256=payload.workflow_contract_sha256, + execution_outcome_sha256=payload.execution_outcome_sha256, + run_receipt_sha256=payload.run_receipt_sha256, + run_report_sha256=built.report_sha256, + run_report_object_version=context.run_report_object_version, + run_report_object_sha256=built.report_sha256, + evidence_manifests=payload.evidence_manifests, + ) + artifact_sha256 = verify_production_terminal_verification_from_report( + built.envelope, + report_bytes=built.report_bytes, + expected=live_expected, + now=now, + ) + if artifact_sha256 != hashlib.sha256(envelope_bytes).hexdigest(): + raise ValueError("terminal verification artifact digest changed") + + # Final-named evidence exists only after the complete in-memory proof + # passes. If storage or the required reread fails, remove only files + # created by this call so no failed terminalization leaves success + # artifacts behind. + report_path = run_dir / "production-terminal-report.json" + envelope_path = run_dir / "production-terminal-verification.json" + written: list[Path] = [] + try: + self._write_private_bytes(report_path, built.report_bytes) + written.append(report_path) + self._write_private_bytes(envelope_path, envelope_bytes) + written.append(envelope_path) + stored_report = self._read_private_bytes( + report_path, + maximum_bytes=_MAX_ARTIFACT_BYTES, + label="production terminal report", + ) + stored_envelope = self._read_private_bytes( + envelope_path, + maximum_bytes=_MAX_ARTIFACT_BYTES, + label="production terminal verification", + ) + if stored_report != built.report_bytes or stored_envelope != envelope_bytes: + raise ValueError("stored terminal evidence changed after write") + reread = ProductionTerminalVerificationEnvelope.model_validate_json( + stored_envelope + ) + if canonical_json(reread) != stored_envelope: + raise ValueError("stored terminal verification is not canonical") + reread_sha256 = verify_production_terminal_verification_from_report( + reread, + report_bytes=stored_report, + expected=live_expected, + now=now, + ) + if reread_sha256 != artifact_sha256: + raise ValueError("stored terminal verification digest changed") + except Exception: + for path in reversed(written): + try: + path.unlink() + except OSError: + pass + raise + return reread, built.report_sha256 + + @staticmethod + def _refusal( + dispatch: HostedDispatch | None, code: str, detail: str + ) -> HostedDispatchRefusal: + events: tuple[dict[str, Any], ...] = () + if dispatch is not None: + refusal = Refusal(RefusalCode.MALFORMED_DISPATCH, detail[:300]) + events = tuple( + refusal_events( + refusal, + run_id=dispatch.run_id, + workflow_id=dispatch.workflow_id, + bundle_digest=dispatch.payload.bundle.content_digest, + authorization_id=dispatch.payload.authorization.authorization_id, + ) + ) + return HostedDispatchRefusal( + dispatch_id=dispatch.dispatch_id if dispatch else None, + run_id=dispatch.run_id if dispatch else None, + code=code, + detail=detail[:400], + evidence_batch=events, + ) + + @staticmethod + def recovery_binding(dispatch: HostedDispatch) -> HostedRecoveryBinding: + """Project the exact credential-bearing state needed after a crash.""" + + return HostedRecoveryBinding( + dispatch_id=dispatch.dispatch_id, + runner_session_id=dispatch.runner_session_id, + dispatch_session_id=dispatch.dispatch_session_id, + run_id=dispatch.run_id, + workflow_id=dispatch.workflow_id, + idempotency_key=dispatch.idempotency_key, + lease_token=dispatch.lease_token, + product_release_admission_sha256=( + dispatch.product_release_admission.artifact_sha256 + ), + workflow_admission_sha256=dispatch.workflow_admission.artifact_sha256, + bundle_content_digest=dispatch.payload.bundle.content_digest, + authorization_id=dispatch.payload.authorization.authorization_id, + ) + + def reconciliation_required( + self, + binding: HostedRecoveryBinding | Mapping[str, object], + *, + code: str = "runner_result_lost", + ) -> HostedRunResult: + """Close a crash window without re-entering the execution path.""" + + parsed = HostedRecoveryBinding.model_validate(binding) + if ( + not code + or len(code) > 64 + or any( + character not in "abcdefghijklmnopqrstuvwxyz0123456789_" + for character in code + ) + ): + raise ValueError("reconciliation code is invalid") + return HostedRunResult( + dispatch_id=parsed.dispatch_id, + run_id=parsed.run_id, + outcome=TransactionOutcome.RECONCILIATION_REQUIRED, + evidence_batch=tuple( + failure_events( + run_id=parsed.run_id, + bundle_digest=parsed.bundle_content_digest, + authorization_id=parsed.authorization_id, + ) + ), + started=True, + uncertain_delivery=True, + report_sha256="0" * 64, + ) + + def execute( + self, + dispatch: HostedDispatch | Mapping[str, object], + *, + runner_config: Path, + run_dir: Path, + authority: DeliveryAuthority, + ) -> Union[HostedRunResult, HostedDispatchRefusal]: + parsed: HostedDispatch | None = None + try: + parsed = HostedDispatch.model_validate(dispatch) + expiry = _utc_seconds(parsed.lease_expires_at, label="hosted lease expiry") + if datetime.now(timezone.utc) >= expiry: + raise ValueError("hosted lease expired before execution") + if ( + authority.url != parsed.managed_delivery_authority_url + or authority.token != parsed.delivery_authority_token + ): + raise ValueError("delivery authority does not match the hosted lease") + config = load_runner_config(runner_config, protected=True) + configured_origin = self._protected_runner_origin(config) + authority_host = urlsplit(authority.url) + if ( + configured_origin + != f"{authority_host.scheme}://{authority_host.netloc}" + ): + raise ValueError( + "delivery authority origin differs from the protected runner host" + ) + self._verify_product_release(parsed, config) + evidence_private_key = self._load_evidence_private_key(config) + qualification, deployment_bytes = self._verify_workflow_admission( + parsed, + config, + evidence_private_key=evidence_private_key, + ) + params = self._resolve_params(parsed, config) + run_dir = Path(run_dir) + run_dir.mkdir(parents=True, exist_ok=False, mode=0o700) + if os.name != "nt": + run_dir.chmod(0o700) + run_dir_stat = run_dir.lstat() + if ( + not stat.S_ISDIR(run_dir_stat.st_mode) + or stat.S_ISLNK(run_dir_stat.st_mode) + or ( + os.name != "nt" + and ( + run_dir_stat.st_uid != os.geteuid() + or stat.S_IMODE(run_dir_stat.st_mode) != 0o700 + ) + ) + ): + raise ValueError("hosted run directory is not protected") + profile_path = self._write_private_bytes( + run_dir / "deployment.yaml", deployment_bytes + ) + staged_deployment_bytes = self._read_private_bytes( + profile_path, + maximum_bytes=1024 * 1024, + label="staged hosted deployment profile", + ) + if staged_deployment_bytes != deployment_bytes: + raise ValueError("staged hosted deployment profile changed") + staged_profiles = dict(config.profiles) + staged_profiles[parsed.payload.deployment_profile_id] = profile_path + staged_config = replace(config, profiles=staged_profiles) + verified = verify_dispatch( + parsed.payload, + staged_config, + resolved_params=params, + ) + if isinstance(verified, Refusal): + return self._refusal(parsed, verified.code.value, verified.reason()) + except Exception as exc: + return self._refusal( + parsed, + "hosted_admission_refused", + f"prestart_{type(exc).__name__}", + ) + + reservation_key = f"{parsed.tenant_id}:{parsed.idempotency_key}" + try: + self._ledger.reserve(reservation_key, run_id=parsed.run_id) + except DuplicateActuation: + return self.reconciliation_required( + self.recovery_binding(parsed), code="dispatch_already_consumed" + ) + + try: + params_file = self._write_params(run_dir / "params.json", params) + qualification_authority_file = self._write_private_json( + run_dir / "qualification-authority.json", qualification + ) + guard = ProductionQualificationGuard( + qualification_authority_file, + remote_permit_revalidation=True, + ) + production_binding = guard.authorization_binding(verified.workflow) + local_authorization = verified.payload.authorization.model_copy( + update=production_binding + ) + local_payload = verified.payload.model_copy( + update={"authorization": local_authorization} + ) + verified = replace(verified, payload=local_payload) + dispatch_file = write_managed_dispatch_envelope( + run_dir / "managed-dispatch.json", verified + ) + argv = build_run_argv( + verified, + run_dir, + params_file, + managed_dispatch_file=dispatch_file, + qualification_authority_file=qualification_authority_file, + ) + child_env = os.environ.copy() + child_env.pop(REMOTE_AUTHORITY_URL_ENV, None) + child_env.pop(REMOTE_AUTHORITY_TOKEN_ENV, None) + child_env.pop(REMOTE_DISPATCH_SESSION_ID_ENV, None) + child_env.update(authority.child_environment()) + child_env[REMOTE_DISPATCH_SESSION_ID_ENV] = parsed.dispatch_session_id + except Exception: # preparation failed before the managed child started + self._ledger.record_outcome( + reservation_key, + TransactionOutcome.FAILED_PLATFORM, + run_id=parsed.run_id, + ) + events = tuple( + failure_events( + run_id=parsed.run_id, + bundle_digest=parsed.payload.bundle.content_digest, + authorization_id=parsed.payload.authorization.authorization_id, + ) + ) + return HostedRunResult( + dispatch_id=parsed.dispatch_id, + run_id=parsed.run_id, + outcome=TransactionOutcome.FAILED_PLATFORM, + evidence_batch=events, + started=False, + uncertain_delivery=False, + report_sha256="0" * 64, + ) + + try: + # Once this call starts, the child can reach a real input edge. Any + # lost or malformed result is uncertain and can never be retried. + execution = self._runner(argv, run_dir, child_env) + except Exception: + outcome = TransactionOutcome.RECONCILIATION_REQUIRED + self._ledger.record_outcome(reservation_key, outcome, run_id=parsed.run_id) + return HostedRunResult( + dispatch_id=parsed.dispatch_id, + run_id=parsed.run_id, + outcome=outcome, + evidence_batch=tuple( + failure_events( + run_id=parsed.run_id, + bundle_digest=parsed.payload.bundle.content_digest, + authorization_id=parsed.payload.authorization.authorization_id, + ) + ), + started=True, + uncertain_delivery=True, + report_sha256="0" * 64, + ) + + if execution.report_bytes is None: + outcome = TransactionOutcome.RECONCILIATION_REQUIRED + self._ledger.record_outcome(reservation_key, outcome, run_id=parsed.run_id) + return HostedRunResult( + dispatch_id=parsed.dispatch_id, + run_id=parsed.run_id, + outcome=outcome, + evidence_batch=tuple( + failure_events( + run_id=parsed.run_id, + bundle_digest=parsed.payload.bundle.content_digest, + authorization_id=parsed.payload.authorization.authorization_id, + ) + ), + started=True, + uncertain_delivery=True, + report_sha256="0" * 64, + ) + report: RunReport | None = None + report_digest = hashlib.sha256(execution.report_bytes).hexdigest() + try: + report = RunReport.model_validate_json(execution.report_bytes) + outcome = classify_transaction_outcome(report) + proof: ProductionTerminalVerificationEnvelope | None = None + if execution.terminal_verification is not None: + raise ValueError("managed child supplied an untrusted terminal proof") + if outcome in { + TransactionOutcome.VERIFIED, + TransactionOutcome.HALTED_BEFORE_EFFECT, + TransactionOutcome.RECONCILIATION_REQUIRED, + }: + if outcome is TransactionOutcome.VERIFIED and execution.returncode != 0: + raise ValueError("managed child exited unsuccessfully") + terminal_config = load_runner_config(runner_config, protected=True) + if self._protected_runner_origin(terminal_config) != configured_origin: + raise ValueError("protected runner origin changed during execution") + self._verify_product_release(parsed, terminal_config) + terminal_key = self._load_evidence_private_key(terminal_config) + terminal_qualification, terminal_deployment = ( + self._verify_workflow_admission( + parsed, + terminal_config, + evidence_private_key=terminal_key, + ) + ) + if ( + terminal_qualification != qualification + or terminal_deployment != deployment_bytes + ): + raise ValueError("production admission changed during execution") + proof, report_digest = self._produce_terminal_verification( + dispatch=parsed, + report=report, + run_dir=run_dir, + qualification=qualification, + private_key=terminal_key, + verified_params=params, + dispatch_binding_sha256=verified.payload.dispatch_binding_sha256, + ) + if proof.payload.run_receipt.transaction_outcome != outcome.value: + raise ValueError("terminal proof outcome differs from the report") + except Exception: # noqa: BLE001 - post-delivery terminalization fails closed + outcome = TransactionOutcome.RECONCILIATION_REQUIRED + proof = None + if ( + outcome + in { + TransactionOutcome.VERIFIED, + TransactionOutcome.HALTED_BEFORE_EFFECT, + } + and proof is None + ): + outcome = TransactionOutcome.RECONCILIATION_REQUIRED + self._ledger.record_outcome(reservation_key, outcome, run_id=parsed.run_id) + if report is None: + events = tuple( + failure_events( + run_id=parsed.run_id, + bundle_digest=parsed.payload.bundle.content_digest, + authorization_id=parsed.payload.authorization.authorization_id, + ) + ) + else: + events = tuple( + report_events( + report, + run_id=parsed.run_id, + workflow_id=parsed.workflow_id, + bundle_digest=parsed.payload.bundle.content_digest, + authorization_id=parsed.payload.authorization.authorization_id, + consequential_steps=verified.consequential_steps, + effect_covered_consequential_steps=( + verified.effect_covered_consequential_steps + ), + ) + ) + uncertain = outcome is TransactionOutcome.RECONCILIATION_REQUIRED + return HostedRunResult( + dispatch_id=parsed.dispatch_id, + run_id=parsed.run_id, + outcome=outcome, + evidence_batch=events, + terminal_verification=proof, + started=True, + uncertain_delivery=uncertain, + report_sha256=report_digest, + ) + + def callback_request( + self, + dispatch: HostedDispatch | HostedRecoveryBinding | Mapping[str, object], + result: HostedRunResult | HostedDispatchRefusal, + ) -> CallbackRequest: + if isinstance(dispatch, HostedDispatch): + binding: HostedDispatch | HostedRecoveryBinding = dispatch + elif isinstance(dispatch, HostedRecoveryBinding): + binding = dispatch + else: + schema = dispatch.get("schema_version") + if schema == "openadapt.hosted-runner-recovery/v1": + binding = HostedRecoveryBinding.model_validate(dispatch) + else: + binding = HostedDispatch.model_validate(dispatch) + if result.dispatch_id != binding.dispatch_id or result.run_id != binding.run_id: + raise ValueError("hosted result does not bind the callback lease") + events = list(result.evidence_batch) + proof_base64 = None + proof_digest = None + if isinstance(result, HostedRunResult): + if result.terminal_verification is not None: + proof_bytes = canonical_json(result.terminal_verification) + proof_digest = result.terminal_verification.artifact_sha256() + if hashlib.sha256(proof_bytes).hexdigest() != proof_digest: + raise ValueError("terminal proof digest differs from exact bytes") + proof_base64 = b64encode(proof_bytes).decode("ascii") + terminal = HostedTerminalEvent( + run_id=binding.run_id, + outcome=( + result.outcome.value + if isinstance(result.outcome, TransactionOutcome) + else result.outcome + ), + report_sha256=result.report_sha256, + started=result.started, + uncertain_delivery=result.uncertain_delivery, + terminal_verification_artifact_bytes_base64=proof_base64, + terminal_verification_artifact_sha256=proof_digest, + ) + events.append(terminal.model_dump(mode="json")) + return CallbackRequest( + dispatch_id=binding.dispatch_id, + runner_session_id=binding.runner_session_id, + idempotency_key=binding.idempotency_key, + lease_token=binding.lease_token, + product_release_admission_sha256=( + binding.product_release_admission.artifact_sha256 + if isinstance(binding, HostedDispatch) + else binding.product_release_admission_sha256 + ), + workflow_admission_sha256=( + binding.workflow_admission.artifact_sha256 + if isinstance(binding, HostedDispatch) + else binding.workflow_admission_sha256 + ), + events=tuple(events), + ) diff --git a/openadapt_flow/runner/inputs.py b/openadapt_flow/runner/inputs.py new file mode 100644 index 00000000..48d1ebf1 --- /dev/null +++ b/openadapt_flow/runner/inputs.py @@ -0,0 +1,107 @@ +"""Fail-closed resolution of hosted values from the sealed input schema.""" + +from __future__ import annotations + +import math +from datetime import date + +from openadapt_flow.ir import ParamKind, ParamSpec, Workflow +from openadapt_flow.runtime.authorization import ( + RuntimeParamScalar, + effective_runtime_params, + is_runtime_param_scalar, +) + + +class AdmittedInputError(ValueError): + """Hosted inputs do not fit the exact schema sealed into the workflow.""" + + +def _validate_value(spec: ParamSpec, value: RuntimeParamScalar) -> None: + if spec.type is ParamKind.ENUM: + if not isinstance(value, str) or not spec.choices or value not in spec.choices: + raise AdmittedInputError( + f"parameter {spec.name!r} is outside its admitted enum" + ) + elif spec.type is ParamKind.DATE: + if not isinstance(value, str): + raise AdmittedInputError( + f"parameter {spec.name!r} is not an ISO date string" + ) + try: + parsed = date.fromisoformat(value) + except ValueError as exc: + raise AdmittedInputError( + f"parameter {spec.name!r} is not an ISO date" + ) from exc + if parsed.isoformat() != value: + raise AdmittedInputError( + f"parameter {spec.name!r} is not a canonical ISO date" + ) + elif spec.type is ParamKind.NUMBER: + if type(value) not in {int, float}: + raise AdmittedInputError(f"parameter {spec.name!r} is not a number") + number = float(value) + if not math.isfinite(number): + raise AdmittedInputError(f"parameter {spec.name!r} must be a finite number") + elif spec.type is ParamKind.BOOLEAN: + if type(value) is not bool: + raise AdmittedInputError(f"parameter {spec.name!r} is not a Boolean") + elif not isinstance(value, str): + raise AdmittedInputError(f"parameter {spec.name!r} is not a string") + + +def resolve_admitted_params( + workflow: Workflow, + supplied: dict[str, RuntimeParamScalar], + *, + inline: bool, +) -> dict[str, RuntimeParamScalar]: + """Resolve exact hosted params without inventing or widening the schema. + + Hosted execution requires the sealed typed schema. Inline input can never + carry a declared secret. Values obtained through a customer-local reference + resolver may carry a secret, but they still pass the same exact name and + type checks before the authorization digest is recomputed. + """ + + if not workflow.param_specs: + raise AdmittedInputError( + "hosted execution requires a sealed typed parameter schema" + ) + if any(name != spec.name for name, spec in workflow.param_specs.items()): + raise AdmittedInputError("the sealed parameter schema has a name mismatch") + + admitted_names = set(workflow.param_specs) + unknown = sorted(set(supplied).difference(admitted_names)) + if unknown: + raise AdmittedInputError( + "hosted input contains parameter(s) outside the admitted schema: " + + ", ".join(unknown) + ) + if inline: + inline_secrets = sorted(set(supplied).intersection(workflow.secret_params)) + if inline_secrets: + raise AdmittedInputError( + "inline hosted input contains declared secret parameter(s): " + + ", ".join(inline_secrets) + ) + + resolved = effective_runtime_params(workflow, supplied) + unknown_defaults = sorted(set(resolved).difference(admitted_names)) + if unknown_defaults: + raise AdmittedInputError( + "workflow defaults fall outside the admitted parameter schema" + ) + for name, spec in sorted(workflow.param_specs.items()): + value = resolved.get(name) + if value is None: + if spec.required: + raise AdmittedInputError(f"required parameter {name!r} is missing") + continue + if not is_runtime_param_scalar(value): + raise AdmittedInputError( + f"parameter {name!r} is not a finite JSON scalar value" + ) + _validate_value(spec, value) + return resolved diff --git a/openadapt_flow/runner/product_release.py b/openadapt_flow/runner/product_release.py new file mode 100644 index 00000000..22db9849 --- /dev/null +++ b/openadapt_flow/runner/product_release.py @@ -0,0 +1,267 @@ +"""Verification of the signed seven-target Product release admission.""" + +from __future__ import annotations + +import hashlib +import json +import re +from base64 import b64decode, urlsafe_b64decode, urlsafe_b64encode +from datetime import datetime, timezone +from typing import Literal, Mapping + +from cryptography.exceptions import InvalidSignature +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey +from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator + +DOMAIN = b"openadapt.product-release-admission-payload.v1\0" +TARGETS = ("agent", "capture", "cloud", "desktop", "docs", "flow", "openadapt") +_HEX64 = r"^[a-f0-9]{64}$" +_UUID = r"^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$" +_KEY_ID = r"^release-admission-ed25519-[a-f0-9]{16}$" +_SAFE_ID = r"^[A-Za-z0-9][A-Za-z0-9._:@/+\-]{0,199}$" +_UTC = re.compile(r"^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$") + + +class ProductReleaseAdmissionError(ValueError): + """The aggregate admission is invalid or inactive.""" + + +class _Closed(BaseModel): + model_config = ConfigDict(strict=True, extra="forbid", frozen=True) + + +def _canonical_json(value: object) -> bytes: + if isinstance(value, BaseModel): + value = value.model_dump(mode="json") + return json.dumps( + value, sort_keys=True, separators=(",", ":"), ensure_ascii=False + ).encode("utf-8") + + +def _utc(value: str, *, field: str) -> datetime: + if _UTC.fullmatch(value) is None: + raise ValueError(f"{field} is not canonical UTC seconds") + try: + parsed = datetime.fromisoformat(value[:-1] + "+00:00") + except ValueError as exc: + raise ValueError(f"{field} is not UTC seconds") from exc + if parsed.tzinfo is None or parsed.microsecond != 0: + raise ValueError(f"{field} is not UTC seconds") + return parsed.astimezone(timezone.utc) + + +class ProductReleaseTarget(_Closed): + target: Literal["agent", "capture", "cloud", "desktop", "docs", "flow", "openadapt"] + admission_id: str = Field(pattern=_UUID) + admission_sha256: str = Field(pattern=_HEX64) + release_id: str = Field(pattern=_SAFE_ID) + release_artifact_sha256: str = Field(pattern=_HEX64) + admission_issued_at: str + admission_expires_at: str + revoked_at: str | None + artifact_authority_sha256: str = Field(pattern=_HEX64) + artifact_authority_state: Literal["active", "revoked", "expired", "unavailable"] + artifact_authority_checked_at: str + artifact_authority_expires_at: str + + @model_validator(mode="after") + def _chronology(self) -> "ProductReleaseTarget": + issued = _utc(self.admission_issued_at, field="target admission_issued_at") + expires = _utc(self.admission_expires_at, field="target admission_expires_at") + checked = _utc( + self.artifact_authority_checked_at, + field="target artifact_authority_checked_at", + ) + authority_expires = _utc( + self.artifact_authority_expires_at, + field="target artifact_authority_expires_at", + ) + if issued >= expires or checked >= authority_expires: + raise ValueError("product release target chronology is invalid") + if self.revoked_at is not None: + _utc(self.revoked_at, field="target revoked_at") + return self + + +class ProductReleaseAdmissionPayload(_Closed): + schema_version: Literal["openadapt.product-release-admission-payload/v1"] + set_id: str = Field(pattern=_UUID) + sequence: int = Field(gt=0, le=9_007_199_254_740_991) + policy_sha256: str = Field(pattern=_HEX64) + issued_at: str + expires_at: str + targets: tuple[ProductReleaseTarget, ...] = Field(min_length=7, max_length=7) + + @model_validator(mode="after") + def _closed_set(self) -> "ProductReleaseAdmissionPayload": + issued = _utc(self.issued_at, field="product admission issued_at") + expires = _utc(self.expires_at, field="product admission expires_at") + if issued >= expires: + raise ValueError("product release admission chronology is invalid") + if tuple(item.target for item in self.targets) != TARGETS: + raise ValueError( + "product release admission targets are not exact and ordered" + ) + return self + + def canonical_bytes(self) -> bytes: + return _canonical_json(self) + + def payload_sha256_value(self) -> str: + return hashlib.sha256(DOMAIN + self.canonical_bytes()).hexdigest() + + +class ProductReleaseSigner(_Closed): + algorithm: Literal["ed25519"] + key_id: str = Field(pattern=_KEY_ID) + public_key: str + + @field_validator("public_key") + @classmethod + def _key(cls, value: str) -> str: + try: + raw = b64decode(value, validate=True) + except ValueError as exc: + raise ValueError("product release signer key is invalid") from exc + if len(raw) != 32: + raise ValueError("product release signer key is invalid") + return value + + @model_validator(mode="after") + def _key_id_matches(self) -> "ProductReleaseSigner": + raw = b64decode(self.public_key, validate=True) + expected = "release-admission-ed25519-" + hashlib.sha256(raw).hexdigest()[:16] + if self.key_id != expected: + raise ValueError("product release signer key id is invalid") + return self + + +class ProductReleaseAdmissionArtifact(_Closed): + schema_version: Literal["openadapt.product-release-admission-artifact/v1"] + payload: ProductReleaseAdmissionPayload + payload_sha256: str = Field(pattern=_HEX64) + signer: ProductReleaseSigner + signature: str = Field(min_length=86, max_length=86) + + @model_validator(mode="after") + def _self_consistent(self) -> "ProductReleaseAdmissionArtifact": + if self.payload_sha256 != self.payload.payload_sha256_value(): + raise ValueError("product release admission payload digest is invalid") + try: + signature = urlsafe_b64decode(self.signature + "==") + except ValueError as exc: + raise ValueError("product release admission signature is invalid") from exc + if ( + len(signature) != 64 + or urlsafe_b64encode(signature).decode("ascii").rstrip("=") + != self.signature + ): + raise ValueError("product release admission signature is invalid") + try: + Ed25519PublicKey.from_public_bytes( + b64decode(self.signer.public_key, validate=True) + ).verify(signature, DOMAIN + self.payload.canonical_bytes()) + except (InvalidSignature, ValueError) as exc: + raise ValueError("product release admission signature is invalid") from exc + return self + + def artifact_sha256(self) -> str: + return hashlib.sha256(_canonical_json(self)).hexdigest() + + +class ProductReleaseSignerTrust(_Closed): + public_key: str + status: Literal["active", "revoked"] + revoked_at: str | None + + @model_validator(mode="after") + def _state(self) -> "ProductReleaseSignerTrust": + if self.status == "active" and self.revoked_at is not None: + raise ValueError("active product release signer has a revocation time") + if self.status == "revoked" and self.revoked_at is None: + raise ValueError("revoked product release signer lacks a revocation time") + if self.revoked_at is not None: + _utc(self.revoked_at, field="product release signer revoked_at") + return self + + +def verify_product_release_admission( + artifact: ProductReleaseAdmissionArtifact, + *, + trusted_signers: Mapping[str, ProductReleaseSignerTrust], + newest_sequence: int, + revoked_set_ids: set[str] | frozenset[str] = frozenset(), + now: datetime | None = None, +) -> ProductReleaseAdmissionPayload: + """Verify signature, authority state, time, revocation, and newest sequence.""" + + try: + artifact = ProductReleaseAdmissionArtifact.model_validate_json( + _canonical_json(artifact) + ) + except ValueError as exc: + raise ProductReleaseAdmissionError(str(exc)) from exc + trust = trusted_signers.get(artifact.signer.key_id) + if trust is None or trust.public_key != artifact.signer.public_key: + raise ProductReleaseAdmissionError("product release signer is not trusted") + if trust.status != "active": + raise ProductReleaseAdmissionError("product release signer is revoked") + payload = artifact.payload + if payload.set_id in revoked_set_ids: + raise ProductReleaseAdmissionError("product release admission is revoked") + if payload.sequence != newest_sequence: + raise ProductReleaseAdmissionError("product release admission is superseded") + current = (now or datetime.now(timezone.utc)).astimezone(timezone.utc) + if ( + not _utc(payload.issued_at, field="issued_at") + <= current + < _utc(payload.expires_at, field="expires_at") + ): + raise ProductReleaseAdmissionError("product release admission is not active") + for target in payload.targets: + if target.revoked_at is not None: + raise ProductReleaseAdmissionError( + f"product release target {target.target} is revoked" + ) + if target.artifact_authority_state != "active": + raise ProductReleaseAdmissionError( + f"product release target {target.target} authority is not active" + ) + if ( + not _utc(target.admission_issued_at, field="target issued_at") + <= current + < _utc(target.admission_expires_at, field="target expires_at") + ): + raise ProductReleaseAdmissionError( + f"product release target {target.target} admission is not active" + ) + if ( + not _utc(target.artifact_authority_checked_at, field="authority checked_at") + <= current + < _utc(target.artifact_authority_expires_at, field="authority expires_at") + ): + raise ProductReleaseAdmissionError( + f"product release target {target.target} authority is stale" + ) + return payload + + +def load_product_release_signer_trust( + raw: object, +) -> dict[str, ProductReleaseSignerTrust]: + if not isinstance(raw, dict) or not raw: + raise ProductReleaseAdmissionError( + "product release signer trust is unavailable" + ) + try: + parsed = { + str(key): ProductReleaseSignerTrust.model_validate(value) + for key, value in raw.items() + } + except ValueError as exc: + raise ProductReleaseAdmissionError( + "product release signer trust is invalid" + ) from exc + if any(re.fullmatch(_KEY_ID, key) is None for key in parsed): + raise ProductReleaseAdmissionError("product release signer key id is invalid") + return parsed diff --git a/openadapt_flow/runner/protocol.py b/openadapt_flow/runner/protocol.py index f469b603..018c223d 100644 --- a/openadapt_flow/runner/protocol.py +++ b/openadapt_flow/runner/protocol.py @@ -17,11 +17,17 @@ import hashlib import json +import re from typing import Union from pydantic import BaseModel, ConfigDict, Field, ValidationError, model_validator -from openadapt_flow.runtime.authorization import GovernedRunAuthorization +from openadapt_flow.runtime.authorization import ( + GovernedRunAuthorization, + RuntimeParamScalar, + is_runtime_param_scalar, + normalize_runtime_param_scalar, +) #: The only job kind the v1 client executes. JOB_KIND_GOVERNED_RUN = "governed_run" @@ -34,6 +40,21 @@ #: Cloud long-poll ceiling (runners.ts POLL_MAX_WAIT_S). POLL_MAX_WAIT_S = 25 +# This is the exact closed grammar used by Cloud's production runtime schema. +# Keeping hosted parameter keys in ASCII also makes Python and JavaScript key +# ordering byte-identical for the authorization digest. +RUNTIME_PARAM_NAME_PATTERN = r"^[A-Za-z_][A-Za-z0-9_]{0,127}$" +_RUNTIME_PARAM_NAME = re.compile(RUNTIME_PARAM_NAME_PATTERN) + + +def validate_runtime_param_name(name: str) -> str: + """Refuse a hosted parameter name outside the shared Cloud grammar.""" + + if _RUNTIME_PARAM_NAME.fullmatch(name) is None: + raise ValueError("runtime parameter name is invalid") + return name + + #: Runtime-local v2 authority bindings are deliberately OUTSIDE the dispatch #: binding digest. The digest grammar is a cross-repo contract: Cloud and Flow #: must hash byte-identical payloads, so fields Cloud does not emit may not @@ -43,6 +64,8 @@ #: v2 permit carries its own admission and authority digests. DISPATCH_BINDING_LOCAL_FIELDS = frozenset( { + "qualification_admission", + "qualification_admission_sha256", "production_qualification_admission_id", "production_qualification_admission_sha256", "production_qualification_evidence_identity_sha256", @@ -108,13 +131,30 @@ class DispatchParamsValues(BaseModel): model_config = ConfigDict(frozen=True, extra="forbid") - values: dict[str, str] + values: dict[str, RuntimeParamScalar] = Field(max_length=100) + + @model_validator(mode="after") + def _finite_json_scalars(self) -> "DispatchParamsValues": + normalized: dict[str, RuntimeParamScalar] = {} + for key, value in self.values.items(): + validate_runtime_param_name(key) + if not is_runtime_param_scalar(value): + raise ValueError( + "runtime parameters must be finite JSON-safe scalar values" + ) + normalized[key] = normalize_runtime_param_scalar(value) + object.__setattr__(self, "values", normalized) + return self class DispatchParamsRef(BaseModel): - """Regulated-lane params-by-reference. Parsed, but refused in v1: the - local reference resolver does not exist yet, and guessing values would - break the runtime-inputs digest binding.""" + """Regulated-lane reference to protected customer-local parameters. + + The hosted adapter treats ``ref`` only as a relative path under the + operator-configured protected root. It refuses URLs, traversal, and links, + then checks ``expected_digest`` after typed local resolution. Resolved + values stay local and never enter the callback. + """ model_config = ConfigDict(frozen=True, extra="forbid") @@ -133,7 +173,7 @@ class RunnerDispatchPayload(BaseModel): # exact UUID across runner restart and reassignment. run_id: str = Field( pattern=( - "^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$" + "^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$" ) ) workflow_id: str diff --git a/openadapt_flow/runner/verify.py b/openadapt_flow/runner/verify.py index 6e644387..996b2d96 100644 --- a/openadapt_flow/runner/verify.py +++ b/openadapt_flow/runner/verify.py @@ -49,6 +49,10 @@ DispatchParamsValues, RunnerDispatchPayload, ) +from openadapt_flow.runtime.authorization import ( + RuntimeParamScalar, + runtime_param_text, +) if TYPE_CHECKING: # pragma: no cover - typing only from pathlib import Path @@ -119,7 +123,7 @@ class VerifiedDispatch: payload: RunnerDispatchPayload bundle: TrustedBundle profile_path: "Path" - params: dict[str, str] + params: dict[str, RuntimeParamScalar] workflow: "Workflow" #: Whole-workflow coverage counts for the terminal run_summary (computed #: from the sealed bundle, not from the cloud's claims). @@ -143,6 +147,7 @@ def verify_dispatch( *, now: Optional[datetime] = None, active_workflow_ids: Optional[set[str]] = None, + resolved_params: Optional[dict[str, RuntimeParamScalar]] = None, ) -> VerifiedDispatch | Refusal: """Independently verify ``payload`` against local trust. Never executes. @@ -194,18 +199,47 @@ def verify_dispatch( "this runner never downloads bundles", ) - if not isinstance(payload.params, DispatchParamsValues): - return Refusal( - RefusalCode.PARAMS_REF_UNSUPPORTED, - "params-by-reference (regulated lane) has no local resolver in v1", - ) - if trusted.params_ref_required: + from openadapt_flow.ir import Workflow + + try: + workflow = Workflow.load(trusted.path) + except Exception as exc: # noqa: BLE001 - crypto/integrity/shape: refuse return Refusal( - RefusalCode.PARAMS_VALUES_REFUSED, - "this bundle requires params-by-reference; inline params.values " - "dispatches are refused on this machine", + RefusalCode.BUNDLE_LOAD_FAILED, + f"trusted bundle failed to load: {type(exc).__name__}", ) - params = dict(payload.params.values) + + if isinstance(payload.params, DispatchParamsValues): + if trusted.params_ref_required: + return Refusal( + RefusalCode.PARAMS_VALUES_REFUSED, + "this bundle requires params-by-reference; inline params.values " + "dispatches are refused on this machine", + ) + params = dict(payload.params.values) + if resolved_params is not None and resolved_params != params: + return Refusal( + RefusalCode.RUNTIME_INPUTS_MISMATCH, + "locally resolved params differ from inline dispatch values", + ) + else: + if resolved_params is None: + return Refusal( + RefusalCode.PARAMS_REF_UNSUPPORTED, + "params-by-reference requires an explicit local resolver", + ) + params = dict(resolved_params) + from openadapt_flow.runtime.authorization import runtime_inputs_digest + + resolved_digest = runtime_inputs_digest(workflow, params, None) + if ( + resolved_digest != payload.params.expected_digest + or resolved_digest != payload.authorization.runtime_inputs_digest + ): + return Refusal( + RefusalCode.RUNTIME_INPUTS_MISMATCH, + "locally resolved parameter values do not match the admitted digest", + ) if trusted.param_patterns: for key in sorted(params): @@ -215,7 +249,7 @@ def verify_dispatch( RefusalCode.PARAM_DOMAIN_REFUSED, f"param {key!r} has no operator-pinned domain pattern", ) - if re.fullmatch(pattern, params[key]) is None: + if re.fullmatch(pattern, runtime_param_text(params[key])) is None: return Refusal( RefusalCode.PARAM_DOMAIN_REFUSED, f"param {key!r} does not match its pinned domain pattern", @@ -246,16 +280,6 @@ def verify_dispatch( "contract requires screenshots_may_leave_box=false", ) - from openadapt_flow.ir import Workflow - - try: - workflow = Workflow.load(trusted.path) - except Exception as exc: # noqa: BLE001 - crypto/integrity/shape: refuse - return Refusal( - RefusalCode.BUNDLE_LOAD_FAILED, - f"trusted bundle failed to load: {type(exc).__name__}", - ) - fit_refusal = payload.authorization.validate_workflow(workflow) if fit_refusal is not None: return Refusal(RefusalCode.AUTHORIZATION_MISMATCH, fit_refusal) diff --git a/openadapt_flow/runtime/authorization.py b/openadapt_flow/runtime/authorization.py index 5e814c85..a59e8e82 100644 --- a/openadapt_flow/runtime/authorization.py +++ b/openadapt_flow/runtime/authorization.py @@ -10,11 +10,12 @@ import hashlib import json +import math import re import threading from datetime import datetime, timezone from pathlib import Path -from typing import Literal +from typing import Literal, Mapping, TypeAlias, Union, cast from uuid import uuid4 from pydantic import BaseModel, ConfigDict, Field, model_validator @@ -30,22 +31,167 @@ _CONSUMED_LOCK = threading.Lock() _QUALIFICATION_ID_RE = r"^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$" +RuntimeParamScalar: TypeAlias = Union[str, bool, int, float] +_JS_SAFE_INTEGER = 9_007_199_254_740_991 -def effective_runtime_params( - workflow: Workflow, supplied: dict[str, str] | None + +def is_runtime_param_scalar(value: object) -> bool: + """Return whether ``value`` is one exact supported finite JSON scalar.""" + + if type(value) in {str, bool}: + return True + if type(value) is int: + if -_JS_SAFE_INTEGER <= value <= _JS_SAFE_INTEGER: + return True + # JSON has one Number type. Admit a large integer spelling only when + # the JavaScript renderer of its finite IEEE-754 value returns that + # same spelling. The normalizer converts it to that Number before any + # retained artifact. + try: + number = float(value) + except OverflowError: + return False + return math.isfinite(number) and _javascript_number_text(number) == str(value) + return type(value) is float and math.isfinite(value) + + +def normalize_runtime_param_scalar(value: object) -> RuntimeParamScalar: + """Return one exact finite JSON scalar in its cross-language representation.""" + + if not is_runtime_param_scalar(value): + raise ValueError("runtime parameter is not a finite interoperable JSON scalar") + if type(value) is int and not -_JS_SAFE_INTEGER <= value <= _JS_SAFE_INTEGER: + return float(value) + return cast(RuntimeParamScalar, value) + + +def _javascript_number_text(value: int | float) -> str: + """Return the finite number text used by JavaScript ``JSON.stringify``. + + The hosted service uses ``JSON.stringify`` for primitive values in its + sorted-key canonical JSON. Python and JavaScript choose different exponent + formatting thresholds, so ordinary ``json.dumps`` is not interoperable. + An out-of-safe-range Python integer reaches this function only when its + spelling is the canonical JavaScript rendering of the normalized Number. + """ + + if type(value) is int: + if not is_runtime_param_scalar(value): + raise ValueError("integer parameter exceeds the JSON safe-integer range") + if not -_JS_SAFE_INTEGER <= value <= _JS_SAFE_INTEGER: + return _javascript_number_text(float(value)) + return str(value) + if type(value) is not float or not math.isfinite(value): + raise ValueError("number parameter must be finite") + if value == 0: + return "0" + + negative = value < 0 + source = repr(abs(value)).lower() + mantissa, separator, exponent_text = source.partition("e") + exponent = int(exponent_text) if separator else 0 + integer, dot, fraction = mantissa.partition(".") + digits = integer + (fraction if dot else "") + decimal_exponent = exponent - len(fraction) + while len(digits) > 1 and digits.endswith("0"): + digits = digits[:-1] + decimal_exponent += 1 + decimal_point = len(digits) + decimal_exponent + + absolute = abs(value) + if 1e-6 <= absolute < 1e21: + if decimal_point <= 0: + rendered = "0." + ("0" * -decimal_point) + digits + elif decimal_point >= len(digits): + rendered = digits + ("0" * (decimal_point - len(digits))) + else: + rendered = digits[:decimal_point] + "." + digits[decimal_point:] + else: + scientific_exponent = decimal_point - 1 + rendered = digits[0] + if len(digits) > 1: + rendered += "." + digits[1:] + rendered += "e" + ("+" if scientific_exponent >= 0 else "") + rendered += str(scientific_exponent) + return ("-" if negative else "") + rendered + + +def _hosted_canonical_json(value: object) -> str: + """Match the hosted sorted-key ``canonicalJson`` implementation exactly.""" + + if value is None: + return "null" + if type(value) is bool: + return "true" if value else "false" + if type(value) in {int, float}: + return _javascript_number_text(cast(Union[int, float], value)) + if isinstance(value, str): + return json.dumps(value, ensure_ascii=False, separators=(",", ":")) + if isinstance(value, list): + return "[" + ",".join(_hosted_canonical_json(item) for item in value) + "]" + if isinstance(value, dict): + if any(not isinstance(key, str) for key in value): + raise ValueError("canonical JSON object keys must be strings") + # JavaScript Array.sort compares UTF-16 code units. Python's default + # string order compares Unicode code points, which differs for astral + # characters versus BMP characters at U+D800 and above. + ordered_keys = sorted(value, key=lambda key: key.encode("utf-16-be")) + members = ( + json.dumps(key, ensure_ascii=False) + + ":" + + _hosted_canonical_json(value[key]) + for key in ordered_keys + ) + return "{" + ",".join(members) + "}" + raise ValueError(f"unsupported canonical JSON value: {type(value).__name__}") + + +def runtime_param_text(value: RuntimeParamScalar) -> str: + """Render one admitted scalar only at the final GUI text boundary.""" + + value = normalize_runtime_param_scalar(value) + if isinstance(value, str): + return value + if type(value) is bool: + return "true" if value else "false" + return _javascript_number_text(value) + + +def runtime_params_for_gui( + params: Mapping[str, RuntimeParamScalar], ) -> dict[str, str]: + """Convert the already-authorized typed parameter set to GUI text.""" + + return {name: runtime_param_text(value) for name, value in params.items()} + + +def effective_runtime_params( + workflow: Workflow, supplied: Mapping[str, RuntimeParamScalar] | None +) -> dict[str, RuntimeParamScalar]: """Resolve defaults exactly as :meth:`Replayer.run` does.""" - merged = dict(workflow.params) + merged: dict[str, RuntimeParamScalar] = dict(workflow.params) for name, spec in workflow.param_specs.items(): if spec.example is not None: merged.setdefault(name, spec.example) merged.update(supplied or {}) - return merged + try: + return { + name: normalize_runtime_param_scalar(value) + for name, value in merged.items() + } + except ValueError as exc: + invalid = [ + name for name, value in merged.items() if not is_runtime_param_scalar(value) + ] + raise ValueError( + "runtime parameters must be strings, Booleans, or finite JSON-safe " + "numbers: " + ", ".join(sorted(invalid)) + ) from exc def runtime_inputs_bytes( workflow: Workflow, - params: dict[str, str] | None, + params: Mapping[str, RuntimeParamScalar] | None, worklists: dict[str, list[dict[str, str]]] | None, *, interstitials: list[Interstitial] | None = None, @@ -64,9 +210,7 @@ def runtime_inputs_bytes( payload["interstitials"] = [ interstitial.model_dump(mode="json") for interstitial in interstitials ] - canonical = json.dumps( - payload, sort_keys=True, separators=(",", ":"), ensure_ascii=False - ) + canonical = _hosted_canonical_json(payload) return canonical.encode("utf-8") @@ -74,7 +218,7 @@ def parse_runtime_inputs_bytes( value: bytes, *, workflow: Workflow, -) -> tuple[dict[str, str], dict[str, list[dict[str, str]]]]: +) -> tuple[dict[str, RuntimeParamScalar], dict[str, list[dict[str, str]]]]: """Parse bytes that this workflow's runtime serializer can emit exactly.""" try: @@ -88,7 +232,7 @@ def parse_runtime_inputs_bytes( params = payload.get("params") worklists = payload.get("worklists") if not isinstance(params, dict) or any( - not isinstance(key, str) or not isinstance(item, str) + not isinstance(key, str) or not is_runtime_param_scalar(item) for key, item in params.items() ): raise ValueError("runtime-input artifact has invalid parameters") @@ -135,12 +279,12 @@ def parse_runtime_inputs_bytes( ) if canonical != value: raise ValueError("runtime-input artifact is not in canonical form") - return dict(params), canonical_worklists + return effective_runtime_params(workflow, dict(params)), canonical_worklists def runtime_inputs_digest( workflow: Workflow, - params: dict[str, str] | None, + params: Mapping[str, RuntimeParamScalar] | None, worklists: dict[str, list[dict[str, str]]] | None, *, interstitials: list[Interstitial] | None = None, @@ -692,7 +836,7 @@ def validate_execution( workflow: Workflow, *, bundle_dir: Path | str, - params: dict[str, str] | None, + params: Mapping[str, RuntimeParamScalar] | None, worklists: dict[str, list[dict[str, str]]] | None, interstitials: list[Interstitial] | None = None, continuation: bool = False, @@ -713,7 +857,7 @@ def validate_execution_snapshot( workflow: Workflow, *, bundle_dir: Path | str, - params: dict[str, str] | None, + params: Mapping[str, RuntimeParamScalar] | None, worklists: dict[str, list[dict[str, str]]] | None, interstitials: list[Interstitial] | None = None, continuation: bool = False, diff --git a/openadapt_flow/runtime/durable/attended.py b/openadapt_flow/runtime/durable/attended.py index 75fb208b..0afde4dc 100644 --- a/openadapt_flow/runtime/durable/attended.py +++ b/openadapt_flow/runtime/durable/attended.py @@ -49,6 +49,7 @@ effects_for_actuation, project_step_safety, ) +from openadapt_flow.runtime.authorization import RuntimeParamScalar from openadapt_flow.runtime.durable.approval import ( ApprovalRecord, ApprovalRequired, @@ -2749,7 +2750,7 @@ def _validated_attended_result( *, identity: Optional[IdentityCheck], skipped: bool, - params: dict[str, str], + params: dict[str, RuntimeParamScalar], manifest: Any, ) -> StepResult: """Return the exact evidence shape emitted by an attended completion.""" @@ -2813,7 +2814,7 @@ def checkpoint_human_completed_step( capability: AttendedPauseCapability, approval: ApprovalRecord, result: StepResult, - params: dict[str, str], + params: dict[str, RuntimeParamScalar], key: Optional[str] = None, ) -> RunCheckpoint: """Advance a linear resume point after outcome verification, without acting.""" @@ -3344,7 +3345,7 @@ def _program_context( store: CheckpointStore, workflow: Workflow, capability: AttendedPauseCapability, - ) -> tuple[PendingEscalation, State, dict[str, str]]: + ) -> tuple[PendingEscalation, State, dict[str, RuntimeParamScalar]]: pending = store.read_pending() state = _program_pause_state(workflow, pending) if pending is not None else None if ( @@ -3370,7 +3371,7 @@ def _resume_program( approval: ApprovalRecord, pending: PendingEscalation, state: State, - params: dict[str, str], + params: dict[str, RuntimeParamScalar], result: StepResult, skipped: bool, target_state_id: Optional[str], @@ -3753,7 +3754,12 @@ def _continue_run_locked( approval: ApprovalRecord, ) -> AttendedExecutionResult: program_context: Optional[ - tuple[PendingEscalation, State, dict[str, str], Optional[str]] + tuple[ + PendingEscalation, + State, + dict[str, RuntimeParamScalar], + Optional[str], + ] ] = None try: store, manifest, workflow = self._load(run_dir, capability) @@ -3907,7 +3913,12 @@ def _reconcile_run_locked( ) reconciliation_delivery_state = capability.delivery_state program_context: Optional[ - tuple[PendingEscalation, State, dict[str, str], Optional[str]] + tuple[ + PendingEscalation, + State, + dict[str, RuntimeParamScalar], + Optional[str], + ] ] = None try: store, manifest, workflow = self._load(run_dir, capability) @@ -4087,7 +4098,12 @@ def _skip_run_locked( approval: ApprovalRecord, ) -> AttendedExecutionResult: program_context: Optional[ - tuple[PendingEscalation, State, dict[str, str], Optional[str]] + tuple[ + PendingEscalation, + State, + dict[str, RuntimeParamScalar], + Optional[str], + ] ] = None try: store, manifest, workflow = self._load(run_dir, capability) diff --git a/openadapt_flow/runtime/durable/authority.py b/openadapt_flow/runtime/durable/authority.py index 591853c8..08113081 100644 --- a/openadapt_flow/runtime/durable/authority.py +++ b/openadapt_flow/runtime/durable/authority.py @@ -52,9 +52,10 @@ AUTHORITY_DB_ENV = "OPENADAPT_DURABLE_AUTHORITY_DB" REMOTE_AUTHORITY_URL_ENV = "OPENADAPT_DURABLE_AUTHORITY_URL" -# Reuse the enrolled runner credential. The operator configures one trust -# relationship with the control plane, not a second delivery-only secret. +# The managed parent injects a run-scoped delivery-authority credential. Keep +# the established environment name for child-runtime compatibility. REMOTE_AUTHORITY_TOKEN_ENV = "OPENADAPT_RUNNER_TOKEN" +REMOTE_DISPATCH_SESSION_ID_ENV = "OPENADAPT_DURABLE_DISPATCH_SESSION_ID" # This observer exists only for the closed synthetic Execute acceptance run. # A Modal launcher owns the fixed, pre-opened non-blocking pipe at descriptor # three. A bundle, CLI invocation, or remote caller cannot select a path, @@ -70,9 +71,8 @@ JOURNAL_GENESIS_DIGEST = "sha256:" + hashlib.sha256(b"").hexdigest() JOURNAL_MAC_DOMAIN = b"openadapt-attended-journal-v1\0" _REMOTE_UUID_RE = re.compile( - r"[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-" - r"[89ab][0-9a-f]{3}-[0-9a-f]{12}", - re.IGNORECASE, + r"[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-" + r"[89ab][0-9a-f]{3}-[0-9a-f]{12}" ) _REMOTE_AUTHORITY_ID_RE = _REMOTE_UUID_RE _REMOTE_TOKEN_RE = re.compile(r"[a-f0-9]{32}") @@ -2006,6 +2006,7 @@ def _require_remote_delivery_permit( ) url = os.getenv(REMOTE_AUTHORITY_URL_ENV, "") token = os.getenv(REMOTE_AUTHORITY_TOKEN_ENV, "") + expected_dispatch_session_id = os.getenv(REMOTE_DISPATCH_SESSION_ID_ENV, "") if not url or not token: raise DurableAuthorityBusy( "production delivery requires configured remote authority credentials" @@ -2121,6 +2122,10 @@ def _require_remote_delivery_permit( and _REMOTE_UUID_RE.fullmatch(response["permit_id"]) and isinstance(response["dispatch_session_id"], str) and _REMOTE_UUID_RE.fullmatch(response["dispatch_session_id"]) + and ( + not expected_dispatch_session_id + or response["dispatch_session_id"] == expected_dispatch_session_id + ) and isinstance(response["one_use_claim_id"], str) and _REMOTE_UUID_RE.fullmatch(response["one_use_claim_id"]) and isinstance(response["permit_artifact_sha256"], str) @@ -2413,7 +2418,9 @@ def acknowledge_remote_delivery( self._emit_synthetic_delivery_marker(marker_payload) return entry - def production_delivery_permit_chain(self) -> ProductionDeliveryPermitChain: + def production_delivery_permit_chain( + self, *, allow_empty: bool = False + ) -> ProductionDeliveryPermitChain: """Rebuild the exact acknowledged chain from protected retained bytes.""" with self._transaction() as connection: @@ -2428,6 +2435,8 @@ def production_delivery_permit_chain(self) -> ProductionDeliveryPermitChain: (self.path_key,), ).fetchall() if not rows: + if allow_empty: + return ProductionDeliveryPermitChain.build(()) raise DurableAuthorityBusy( "the production delivery permit chain is unavailable" ) diff --git a/openadapt_flow/runtime/durable/business_decision.py b/openadapt_flow/runtime/durable/business_decision.py index 3d52b81c..858264fe 100644 --- a/openadapt_flow/runtime/durable/business_decision.py +++ b/openadapt_flow/runtime/durable/business_decision.py @@ -27,6 +27,7 @@ ProgramExecutionScopeFrame, Workflow, ) +from openadapt_flow.runtime.authorization import RuntimeParamScalar from openadapt_flow.runtime.durable.approval import ( ApprovalRecord, ApprovalRequired, @@ -806,7 +807,7 @@ def issue( graph_id: str, state_id: str, frames: list[GraphFrame], - params: dict[str, str], + params: dict[str, RuntimeParamScalar], spec: BusinessDecisionSpec, governed_runtime_inputs_digest: str | None, now: datetime | None = None, @@ -933,7 +934,7 @@ def _validate_live_binding( graph_id: str, state_id: str, frames: list[GraphFrame], - params: dict[str, str], + params: dict[str, RuntimeParamScalar], spec: BusinessDecisionSpec, governed_runtime_inputs_digest: str | None, ) -> None: @@ -1183,7 +1184,7 @@ def consume( graph_id: str, state_id: str, frames: list[GraphFrame], - params: dict[str, str], + params: dict[str, RuntimeParamScalar], spec: BusinessDecisionSpec, governed_runtime_inputs_digest: str | None, now: datetime | None = None, diff --git a/openadapt_flow/runtime/durable/checkpoint.py b/openadapt_flow/runtime/durable/checkpoint.py index 6072cf9c..aa69f853 100644 --- a/openadapt_flow/runtime/durable/checkpoint.py +++ b/openadapt_flow/runtime/durable/checkpoint.py @@ -62,7 +62,10 @@ ProgramTransitionEvidence, Resolution, ) -from openadapt_flow.runtime.authorization import GovernedRunAuthorization +from openadapt_flow.runtime.authorization import ( + GovernedRunAuthorization, + RuntimeParamScalar, +) from openadapt_flow.runtime.durable.approval import ApprovalRecord from openadapt_flow.runtime.durable.program_checkpoint import ( GraphFrame, @@ -115,7 +118,7 @@ class RunManifest(BaseModel): bundle_dir: str #: The run's fully-resolved parameter bindings (defaults + caller #: overrides), so a resume re-binds identically. - params: dict[str, str] = Field(default_factory=dict) + params: dict[str, RuntimeParamScalar] = Field(default_factory=dict) #: Stable at-most-once reservation for the complete logical durable run. #: A resumed leg reuses it only after it proves ownership in the ledger. idempotency_key: Optional[str] = None @@ -206,7 +209,7 @@ class RunCheckpoint(BaseModel): #: point at an arbitrary successor state. next_step_index: int #: The run's parameter bindings at checkpoint time (resume re-binds these). - params: dict[str, str] = Field(default_factory=dict) + params: dict[str, RuntimeParamScalar] = Field(default_factory=dict) #: Verification evidence carried for the audit trail / operator. effect_verified: Optional[bool] = None effect_approved_unverified: bool = False @@ -295,7 +298,7 @@ class PendingEscalation(BaseModel): resume_from_index: int = 0 resume_from_step_id: Optional[str] = None #: The run's parameter bindings, so an approved resume re-binds identically. - params: dict[str, str] = Field(default_factory=dict) + params: dict[str, RuntimeParamScalar] = Field(default_factory=dict) #: ``rejected`` is TERMINAL: an operator answered the attended pause with #: ``reject``, asserting this run must not proceed. The file is retained #: rather than cleared so the audit trail keeps WHY the run stopped and diff --git a/openadapt_flow/runtime/durable/controller.py b/openadapt_flow/runtime/durable/controller.py index e1994a84..1b5c8ccd 100644 --- a/openadapt_flow/runtime/durable/controller.py +++ b/openadapt_flow/runtime/durable/controller.py @@ -43,7 +43,10 @@ StepResult, Workflow, ) -from openadapt_flow.runtime.authorization import GovernedRunAuthorization +from openadapt_flow.runtime.authorization import ( + GovernedRunAuthorization, + RuntimeParamScalar, +) from openadapt_flow.runtime.durable.approval import StateDiverged from openadapt_flow.runtime.durable.checkpoint import ( CheckpointStore, @@ -262,7 +265,7 @@ def __init__( run_id: str, workflow_name: str, bundle_dir: Path | str, - params: dict[str, str], + params: dict[str, RuntimeParamScalar], worklists: dict[str, list[dict[str, str]]], idempotency_key: Optional[str] = None, save_healed_to: Optional[Path | str] = None, @@ -452,7 +455,7 @@ def record( step_index: int, step: Step, result: StepResult, - params: dict[str, str], + params: dict[str, RuntimeParamScalar], *, workflow: Optional[Workflow] = None, transition_observation: Optional["TransitionObservation"] = None, @@ -617,7 +620,7 @@ def record_program_halt( state_id: str, intent: str, result: StepResult, - params: dict[str, str], + params: dict[str, RuntimeParamScalar], workflow: Optional[Workflow] = None, transition_observation: Optional["TransitionObservation"] = None, program_frames: Optional[list[GraphFrame]] = None, diff --git a/openadapt_flow/runtime/durable/program_checkpoint.py b/openadapt_flow/runtime/durable/program_checkpoint.py index 40b91bb3..f9bd5111 100644 --- a/openadapt_flow/runtime/durable/program_checkpoint.py +++ b/openadapt_flow/runtime/durable/program_checkpoint.py @@ -20,6 +20,7 @@ import hashlib import json +from collections.abc import Mapping from datetime import datetime, timezone from pathlib import Path from typing import Any, Literal, Optional @@ -40,6 +41,7 @@ ProgramTransitionEvidence, Resolution, ) +from openadapt_flow.runtime.authorization import RuntimeParamScalar #: The synthetic ``graph_id`` of the top-level ``Workflow.program`` graph (every #: OTHER graph is a named entry in ``Workflow.subflows`` -- including a loop @@ -142,7 +144,7 @@ class GraphFrame(BaseModel): state_id: str #: The parameter bindings in scope for this graph frame (a loop body frame's #: scope is the parent's params merged with the current row). - params: dict[str, str] = Field(default_factory=dict) + params: dict[str, RuntimeParamScalar] = Field(default_factory=dict) #: Present iff this frame is a loop-body iteration -- the loop's cursor. loop: Optional[LoopCursor] = None @@ -156,7 +158,7 @@ def control_frames_hash(frames: list[GraphFrame]) -> str: return "sha256:" + hashlib.sha256(canonical).hexdigest() -def bound_params_sha256(params: dict[str, str]) -> str: +def bound_params_sha256(params: Mapping[str, RuntimeParamScalar]) -> str: """Return a PHI-free digest of one exact attended parameter scope.""" canonical = json.dumps( @@ -227,7 +229,7 @@ class ProgramCheckpoint(BaseModel): #: :class:`GraphFrame`). ``frames[-1]`` is the leaf (the verified state). frames: list[GraphFrame] = Field(min_length=1) #: The parameter bindings in scope at the leaf (resume re-binds these). - bound_params: dict[str, str] = Field(default_factory=dict) + bound_params: dict[str, RuntimeParamScalar] = Field(default_factory=dict) #: Contract hashes (``Effect.contract_hash``) of the effects CONFIRMED AT #: THIS state -- appended to the run's completed-effect ledger. Union across #: all checkpoints = every already-performed consequential write, so a resume diff --git a/openadapt_flow/runtime/durable/resume.py b/openadapt_flow/runtime/durable/resume.py index aaaa1cd9..35ee8bb2 100644 --- a/openadapt_flow/runtime/durable/resume.py +++ b/openadapt_flow/runtime/durable/resume.py @@ -30,6 +30,7 @@ from __future__ import annotations +from collections.abc import Mapping from datetime import datetime from pathlib import Path from types import SimpleNamespace @@ -37,6 +38,10 @@ from openadapt_flow.ir import ExecutionTargetKind, RunReport, Step, Workflow from openadapt_flow.policy import effects_for_actuation +from openadapt_flow.runtime.authorization import ( + RuntimeParamScalar, + runtime_params_for_gui, +) from openadapt_flow.runtime.durable.approval import ( ApprovalRecord, ApprovalRequired, @@ -62,13 +67,13 @@ def _resolved_step_effects( step: Step, *, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], run_id: str, actuation: Optional[str], ) -> list[Effect]: """Resolve the exact path-specific effects declared by one retained step.""" - namespace = {**params, "__run_id__": run_id} + namespace = {**runtime_params_for_gui(params), "__run_id__": run_id} return [ effect.resolve(namespace) for effect in effects_for_actuation(step, actuation) ] @@ -77,7 +82,7 @@ def _resolved_step_effects( def _validate_retained_step_proof( *, step: Step, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], run_id: str, skipped: bool, actuation: Optional[str], @@ -1103,7 +1108,7 @@ def _resume_program( checkpoint: Optional[ProgramCheckpoint], checkpoints: list[ProgramCheckpoint], bundle_dir: Path, - params: dict[str, str], + params: dict[str, RuntimeParamScalar], worklists: dict[str, list[dict[str, str]]], save_healed_to: Optional[Path | str], live_bundle_version: str, diff --git a/openadapt_flow/runtime/effects/adapter.py b/openadapt_flow/runtime/effects/adapter.py index 559e7de1..212824dd 100644 --- a/openadapt_flow/runtime/effects/adapter.py +++ b/openadapt_flow/runtime/effects/adapter.py @@ -118,7 +118,7 @@ #: run parameters for ``ValueExpr({param: ...})`` binding. A factory must fail #: LOUD (raise ``ValueError``) on missing config or secrets -- never construct #: a silently broken verifier. -VerifierFactory = Callable[[Any, Optional[Mapping[str, str]]], Any] +VerifierFactory = Callable[[Any, Optional[Mapping[str, object]]], Any] class ConnectionProbe(BaseModel): diff --git a/openadapt_flow/runtime/effects/effect.py b/openadapt_flow/runtime/effects/effect.py index 87bd6abe..908bca6f 100644 --- a/openadapt_flow/runtime/effects/effect.py +++ b/openadapt_flow/runtime/effects/effect.py @@ -93,7 +93,7 @@ def _exactly_one_source(self) -> "ValueExpr": raise ValueError("effect parameter name must not be empty") return self - def resolve(self, params: Mapping[str, str]) -> Optional[str]: + def resolve(self, params: Mapping[str, object]) -> Optional[str]: """Resolve to a concrete string against ``params``. A ``param`` reference reads ``params[param]`` (``None`` when the run did @@ -102,10 +102,16 @@ def resolve(self, params: Mapping[str, str]) -> Optional[str]: record). A pure literal returns its literal unchanged. """ if self.param is not None: - return params.get(self.param) + if self.param not in params: + return None + # Keep the authorized JSON scalar typed until this exact + # string-only effect-verifier boundary. + from openadapt_flow.runtime.authorization import runtime_param_text + + return runtime_param_text(params[self.param]) # type: ignore[arg-type] return self.literal - def resolved(self, params: Mapping[str, str]) -> "ValueExpr": + def resolved(self, params: Mapping[str, object]) -> "ValueExpr": """Return a pure-literal copy of this expression bound to ``params``.""" return ValueExpr(literal=self.resolve(params)) @@ -482,7 +488,7 @@ def requires_baseline(self) -> bool: # -- run-time parameter binding (P0-3) ----------------------------------- def resolve( self, - params: Mapping[str, str], + params: Mapping[str, object], *, opaque_param_sha256: Mapping[str, str] | None = None, ) -> "Effect": @@ -519,7 +525,7 @@ def resolve( def resolved_contract_hash( self, - params: Mapping[str, str], + params: Mapping[str, object], *, opaque_param_sha256: Mapping[str, str] | None = None, ) -> str: @@ -538,14 +544,14 @@ def resolved_contract_hash( char not in "0123456789abcdef" for char in digest ): raise ValueError(f"opaque parameter {name!r} has an invalid digest") - if ( - name in params - and hashlib.sha256(str(params[name]).encode("utf-8")).hexdigest() - != digest - ): - raise ValueError( - f"opaque parameter {name!r} digest does not match its value" - ) + if name in params: + resolved_value = ValueExpr(param=name).resolve(params) + if resolved_value is None or ( + hashlib.sha256(resolved_value.encode("utf-8")).hexdigest() != digest + ): + raise ValueError( + f"opaque parameter {name!r} digest does not match its value" + ) missing = self.referenced_params().difference(params).difference(opaque) if missing: raise ValueError("effect contract references an unavailable parameter") @@ -558,7 +564,7 @@ def value(expr: ValueExpr | None) -> object: "opaque_param": expr.param, "sha256": opaque[expr.param], } - return str(expr.resolved(params)) + return expr.resolve(params) payload: dict[str, object] = { "kind": self.kind.value, diff --git a/openadapt_flow/runtime/program_predicates.py b/openadapt_flow/runtime/program_predicates.py index 6707e97c..8ba9452f 100644 --- a/openadapt_flow/runtime/program_predicates.py +++ b/openadapt_flow/runtime/program_predicates.py @@ -269,7 +269,7 @@ def predicate_uses_frame(predicate: Predicate | None) -> bool: def evaluate_program_predicate( predicate: Predicate, frame_png: bytes, - params: Mapping[str, str], + params: Mapping[str, object], *, vision: Any, viewport: tuple[int, int] | None, @@ -308,9 +308,11 @@ def evaluate_program_predicate( if kind is PredicateKind.TEXT_ABSENT: return not (predicate.text and vision.text_present(frame_png, predicate.text)) if kind is PredicateKind.PARAM_EQUALS: - return predicate.param is not None and str(params.get(predicate.param)) == str( - predicate.value - ) + if predicate.param is None or predicate.param not in params: + return False + from openadapt_flow.runtime.authorization import runtime_param_text + + return runtime_param_text(params[predicate.param]) == predicate.value # type: ignore[arg-type] if kind is PredicateKind.AND: return all( evaluate_program_predicate( diff --git a/openadapt_flow/runtime/replayer.py b/openadapt_flow/runtime/replayer.py index 98bcc887..d0bb3aaf 100644 --- a/openadapt_flow/runtime/replayer.py +++ b/openadapt_flow/runtime/replayer.py @@ -41,20 +41,32 @@ import math import os import re -import struct import time import uuid from copy import deepcopy from datetime import date, datetime, timezone from pathlib import Path -from typing import TYPE_CHECKING, Any, Callable, Literal, Optional, TypeVar, cast +from typing import ( + TYPE_CHECKING, + Any, + Callable, + Literal, + Mapping, + Optional, + TypeVar, + cast, +) from urllib.parse import urlsplit from openadapt_flow.backend import ( ActionDeliveryUncertain, + ActuationObservationBackend, Backend, BrowserPresentationGeometryBackend, FocusedElementActuationLeaseBackend, + FrameObservation, + FrameObservationBackend, + FrameObservationLeaseBackend, FreshActuationReacquisitionBackend, FreshActuationRequired, GuardedCoordinateActionBackend, @@ -68,6 +80,9 @@ RichPointerActionBackend, SelectOptionBackend, StructuralResolutionRefused, + frame_observation_identity, + session_identity_sha256, + window_identity_sha256, ) from openadapt_flow.bundle_validation import compute_parameter_schema_digest from openadapt_flow.identity_signals import ( @@ -136,7 +151,10 @@ from openadapt_flow.runtime import identity as identity_mod from openadapt_flow.runtime.authorization import ( GovernedRunAuthorization, + RuntimeParamScalar, runtime_inputs_digest, + runtime_param_text, + runtime_params_for_gui, ) from openadapt_flow.runtime.durable.approval import StateDiverged from openadapt_flow.runtime.durable.program_checkpoint import ( @@ -207,24 +225,6 @@ PC_TEMPLATE_SEARCH_PAD = 80 -def _frame_viewport(frame_png: bytes) -> tuple[int, int]: - """Return the viewport of THIS frame, read from the frame itself. - - ``self.backend.viewport`` is a live read. Pairing it with an - already-captured frame lets a resize or move between capture and use - reinterpret that frame under geometry it never had, which can place a - coordinate where the operator never demonstrated one. A PNG carries its own - dimensions in the IHDR chunk, so the frame can always answer for itself. - """ - - if len(frame_png) < 24 or not frame_png.startswith(b"\x89PNG\r\n\x1a\n"): - raise ValueError("frame viewport requires valid PNG bytes") - width, height = struct.unpack(">II", frame_png[16:24]) - if width <= 0 or height <= 0: - raise ValueError("frame viewport must be positive") - return int(width), int(height) - - PC_TEMPLATE_THRESHOLD = 0.9 # REGION_STABLE asserts recorded structure, not palette. The ordinary # grayscale template matcher remains the first (stricter) check; this edge-map @@ -335,7 +335,7 @@ def __init__(self, outcome: str, reason: str, *, safety: bool = False) -> None: # Empty means the halt is not eligible for a generic attended # Continue/Skip transition. self.program_frames: list[GraphFrame] = [] - self.program_params: dict[str, str] = {} + self.program_params: dict[str, RuntimeParamScalar] = {} self.program_history_hash: str = "" @@ -619,10 +619,12 @@ def __init__( self._governed_asset_hashes: dict[str, str] = {} self._governed_plaintext_assets = False self._governed_asset_mutation: Optional[str] = None - self._governed_base_params: Optional[dict[str, str]] = None + self._governed_base_params: Optional[dict[str, RuntimeParamScalar]] = None self._active_runtime_worklists: Optional[dict[str, list[dict[str, str]]]] = None self._active_delivery_resolution: Optional[Resolution] = None self._active_delivery_region: Optional[Region] = None + self._active_frame_observation: Optional[FrameObservation] = None + self._last_frame_observation: Optional[FrameObservation] = None self._current_graph_id: Optional[str] = None self._execution_workflow_source: Optional[Workflow] = None self._execution_workflow_snapshot: Optional[Workflow] = None @@ -746,6 +748,99 @@ def __init__( QualificationEnvironmentObservation ] = None + def _retain_frame_observation( + self, + observation: FrameObservation, + *, + active: bool = False, + ) -> FrameObservation: + self._last_frame_observation = observation + if active: + self._active_frame_observation = observation + return observation + + def _legacy_single_surface_observation(self, png: bytes) -> FrameObservation: + """Bind an old Backend to the exact PNG-local coordinate surface. + + The legacy Backend contract defines clicks in screenshot pixel space. + It can therefore prove PNG dimensions, frame-local origin, and its + process-local surface identity without a later ``viewport`` read. It + cannot claim external window/session continuity; production native and + remote adapters implement ``observe_frame`` with stronger identities. + """ + + viewport = exact_png_size(png) + backend_type = ( + f"{type(self.backend).__module__}.{type(self.backend).__qualname__}" + ) + backend_identity = { + "schema": "openadapt.legacy-frame-surface.v1", + "backend_type": backend_type, + "backend_instance": id(self.backend), + } + window_identity = window_identity_sha256( + window_id=f"legacy:{id(self.backend)}", + pid=0, + process_start_time=None, + owner=backend_type, + ) + session_identity = session_identity_sha256( + authority=backend_type, + session_id=str(id(self.backend)), + session_start_time=None, + principal_identity_sha256=None, + ) + topology = frame_observation_identity( + { + **backend_identity, + "viewport": list(viewport), + "coordinate_space": "png-local", + } + ) + return FrameObservation.create( + png, + origin=(0.0, 0.0), + scale=None, + device_pixel_ratio=None, + display_id="legacy-png-surface", + display_bounds=(0.0, 0.0, float(viewport[0]), float(viewport[1])), + display_scale=(1.0, 1.0), + topology_sha256=topology, + window_identity_sha256=window_identity, + session_identity_sha256=session_identity, + ) + + def _capture_frame_observation( + self, + *, + active: bool = False, + ) -> FrameObservation: + if isinstance(self.backend, FrameObservationBackend): + observation = self.backend.observe_frame() + else: + observation = self._legacy_single_surface_observation( + self.backend.screenshot() + ) + return self._retain_frame_observation(observation, active=active) + + def _observation_for_frame(self, frame_png: bytes) -> FrameObservation: + for observation in ( + self._active_frame_observation, + self._last_frame_observation, + getattr(self.backend, "last_frame_observation", None), + ): + if ( + isinstance(observation, FrameObservation) + and observation.png is frame_png + ): + return observation + return self._legacy_single_surface_observation(frame_png) + + def _viewport_for_frame(self, frame_png: bytes) -> tuple[int, int]: + """Return geometry bound to ``frame_png``, never a later backend read.""" + + return self._observation_for_frame(frame_png).viewport + @staticmethod def _canonical_resume_value(value: Any) -> Any: """Return one deterministic JSON value for a resume admission field.""" @@ -772,7 +867,7 @@ def _durable_resume_payload( run_dir: Path, bundle_dir: Path, run_id: str, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], worklists: dict[str, list[dict[str, str]]], resume_from: Optional[int], resume_program: Optional[ProgramCheckpoint], @@ -869,7 +964,7 @@ def _admit_durable_resume( run_dir: Path, bundle_dir: Path, run_id: Optional[str], - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], worklists: dict[str, list[dict[str, str]]], resume_from: Optional[int], resume_program: Optional[ProgramCheckpoint], @@ -934,7 +1029,7 @@ def _consume_durable_resume_admission( run_dir: Path, bundle_dir: Path, run_id: str, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], worklists: dict[str, list[dict[str, str]]], resume_from: Optional[int], resume_program: Optional[ProgramCheckpoint], @@ -1004,7 +1099,7 @@ def run( self, workflow: Workflow, *, - params: Optional[dict[str, str]] = None, + params: Optional[dict[str, RuntimeParamScalar]] = None, worklists: Optional[dict[str, list[dict[str, str]]]] = None, bundle_dir: Path, run_dir: Path, @@ -1155,7 +1250,7 @@ def run( # with caller-supplied values overriding both. A v0 bundle (empty # ``param_specs``) collapses to exactly the old ``{**workflow.params, # **caller}`` merge. - merged: dict[str, str] = {**workflow.params} + merged: dict[str, RuntimeParamScalar] = {**workflow.params} for pname, spec in workflow.param_specs.items(): if spec.example is not None: merged.setdefault(pname, spec.example) @@ -1564,7 +1659,11 @@ def acknowledge_delivery(self_nonlocal) -> None: missing = sorted( pname for pname, spec in workflow.param_specs.items() - if spec.required and not params.get(pname) + if spec.required + and ( + pname not in params + or (isinstance(params[pname], str) and params[pname] == "") + ) ) if missing: report.results.append( @@ -2223,7 +2322,7 @@ def _observe_screen_texts(self) -> list[str]: for text presence; returns [] when OCR is unavailable (pixel-only substrate / a vision stub without ocr).""" try: - frame = self.backend.screenshot() + frame = self._capture_frame_observation().png except Exception: return [] ocr = getattr(self.vision, "ocr", None) @@ -2316,7 +2415,7 @@ def _interpret_program( self, workflow: Workflow, *, - params: dict[str, str], + params: dict[str, RuntimeParamScalar], worklists: dict[str, list[dict[str, str]]], bundle_dir: Path, run_dir: Path, @@ -2355,7 +2454,7 @@ def _interpret_program( # Where the interpreter currently is (for a durable pause record). self._current_state_id: str = "" self._current_intent: str = "" - self._current_params: dict[str, str] = dict(params) + self._current_params: dict[str, RuntimeParamScalar] = dict(params) if durable_run is not None: self._bundle_version = _bundle_version(bundle_dir) if self._durable_resume_mode == "program": @@ -2553,7 +2652,7 @@ def _walk_graph( *, graph_id: str, workflow: Workflow, - params: dict[str, str], + params: dict[str, RuntimeParamScalar], worklists: dict[str, list[dict[str, str]]], bundle_dir: Path, run_dir: Path, @@ -2612,7 +2711,7 @@ def _run_states_from( frame: dict, *, workflow: Workflow, - params: dict[str, str], + params: dict[str, RuntimeParamScalar], worklists: dict[str, list[dict[str, str]]], bundle_dir: Path, run_dir: Path, @@ -2653,7 +2752,7 @@ def _run_states_from( state.decision.question if state.decision is not None else state.id ) ) - self._current_params = params + self._current_params = dict(params) if state.kind is StateKind.TERMINAL: self._raise_on_governed_asset_mutation() @@ -2689,7 +2788,7 @@ def _exec_state( graph: ProgramGraph, *, workflow: Workflow, - params: dict[str, str], + params: dict[str, RuntimeParamScalar], worklists: dict[str, list[dict[str, str]]], bundle_dir: Path, run_dir: Path, @@ -2801,7 +2900,7 @@ def _exec_action_state( graph: ProgramGraph, *, workflow: Workflow, - params: dict[str, str], + params: dict[str, RuntimeParamScalar], bundle_dir: Path, run_dir: Path, report: RunReport, @@ -2948,7 +3047,7 @@ def _exec_loop_state( state: State, *, workflow: Workflow, - params: dict[str, str], + params: dict[str, RuntimeParamScalar], worklists: dict[str, list[dict[str, str]]], bundle_dir: Path, run_dir: Path, @@ -3079,7 +3178,7 @@ def _exec_business_decision_state( state: State, *, workflow: Workflow, - params: dict[str, str], + params: dict[str, RuntimeParamScalar], bundle_dir: Path, report: RunReport, run_dir: Path, @@ -3233,7 +3332,7 @@ def _select_transition( state: State, *, workflow: Workflow, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], bundle_dir: Path, report: Optional[RunReport] = None, run_dir: Optional[Path] = None, @@ -3370,10 +3469,7 @@ def _retain_program_transition_evidence( ) frame_sha256 = hashlib.sha256(frame).hexdigest() frame_ref = f"private/program-transitions/{frame_sha256}.png" - viewport = ( - int(self.backend.viewport[0]), - int(self.backend.viewport[1]), - ) + viewport = self._viewport_for_frame(frame) try: retained_frame_size = exact_png_size(frame) except Exception as exc: @@ -3587,14 +3683,14 @@ def _program_scope_from_frames( def _params_with_business_decisions( self, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], scope: list[ProgramExecutionScopeFrame], evidence: list[BusinessDecisionEvidence], *, run_dir: Path, workflow: Workflow, governed_runtime_inputs_digest: str | None, - ) -> dict[str, str]: + ) -> dict[str, RuntimeParamScalar]: """Reapply authenticated decisions made in this exact frame scope.""" if not evidence: @@ -3621,7 +3717,10 @@ def _params_with_business_decisions( return resolved def _skip_completed_effect_state( - self, state: State, params: dict[str, str], report: RunReport + self, + state: State, + params: Mapping[str, RuntimeParamScalar], + report: RunReport, ) -> bool: """Idempotency guard: skip an action state whose declared effects were ALL already CONFIRMED (in the completed-effect ledger). @@ -3818,7 +3917,7 @@ def _record_program_checkpoint( self, state: State, result: StepResult, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], report: RunReport, ) -> None: """Persist a verified-state interpreter checkpoint (Tier-3, program mode). @@ -4363,7 +4462,7 @@ def revalidate_attended_program_completion( *, graph_id: str, state_id: str, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], bundle_dir: Path, run_dir: Path, run_id: str, @@ -4439,7 +4538,7 @@ def select_attended_program_transition( *, graph_id: str, state_id: str, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], bundle_dir: Path, ) -> Optional[str]: """Select and prove one exact successor for an attended action state.""" @@ -4509,7 +4608,7 @@ def revalidate_attended_completion( workflow: Workflow, *, step_index: int, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], bundle_dir: Path, run_dir: Path, run_id: str, @@ -5463,7 +5562,7 @@ def _apply_screen_fault_mutation( return before_png if mutation.replace_effect_verifier: raise RuntimeError("qualification_screen_fault_replaced_verifier") - after_png = self.backend.screenshot() + after_png = self._capture_frame_observation().png self._validate_fault_mutation( context, mutation.receipt, @@ -5516,7 +5615,7 @@ def _run_step( *, workflow: Workflow, step_index: int, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], bundle_dir: Path, run_dir: Path, new_crops: dict[str, bytes], @@ -5673,10 +5772,14 @@ def _run_step( # capture only a raw diagnostic frame here so it is not preceded by an # unobservable proceed-anyway settle attempt. before_png = ( - self.backend.screenshot() + self._capture_frame_observation().png if self.require_settled else self.vision.wait_settled(self.backend) ) + self._retain_frame_observation( + self._observation_for_frame(before_png), + active=True, + ) result.before_png = self._save_step_png( run_dir, evidence_step_id, "before", before_png ) @@ -6146,7 +6249,6 @@ def _run_step( and self._step_needs_consequential_revalidation( step, workflow ) - and isinstance(self.backend, RemoteActuationBackend) and reacquisition_backend is not None ) if can_retry: @@ -6384,7 +6486,7 @@ def _run_step( selection_error: Optional[str] = None if step.action is ActionKind.SELECT_OPTION: selection_text = ( - params[step.param] + runtime_param_text(params[step.param]) if step.param is not None else step.text or "" ) @@ -6691,7 +6793,7 @@ def _api_request_pointer_field_path(pointer: str) -> tuple[str, ...]: def _api_identity_refusal( self, step: Step, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], workflow: Workflow, effects: list["Effect"], result: StepResult, @@ -6730,7 +6832,8 @@ def _api_identity_refusal( f"({step.intent}): semantic signal {identity.key!r} is not " "part of the qualified identity policy" ) - value = params.get(identity.param) + raw_value = params.get(identity.param) + value = runtime_param_text(raw_value) if raw_value is not None else None if value is None or value == "": return ( f"API identity verification HALTED step '{step.id}' " @@ -6803,7 +6906,7 @@ def _api_identity_refusal( def _try_api_tier( self, step: Step, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], result: StepResult, *, workflow: Workflow, @@ -7009,7 +7112,10 @@ def _try_api_tier( result.delivery_attempted = True try: self._require_qualification_environment_current() - outcome = self.api_actuator.actuate(binding, params) + outcome = self.api_actuator.actuate( + binding, + runtime_params_for_gui(params), + ) except Exception as exc: # noqa: BLE001 - external actuator boundary # The actuator contract is no-throw, but a deployment adapter can # still violate it. The delivery boundary was crossed immediately @@ -7353,7 +7459,7 @@ def _set_api_unavailable_refusal( # -- system-of-record effect verification ----------------------------------- def _resolve_effects( - self, effects: list["Effect"], params: dict[str, str] + self, effects: list["Effect"], params: Mapping[str, RuntimeParamScalar] ) -> list["Effect"]: """Bind each effect's ``ValueExpr`` contract to THIS run's params (P0-3). @@ -7364,7 +7470,10 @@ def _resolve_effects( idempotency key can be bound per-run. A pure-literal (v1) effect is returned value-identical -- ``resolve`` is a no-op for it. """ - namespace = {**params, "__run_id__": self._run_id} + namespace = { + **{name: runtime_param_text(value) for name, value in params.items()}, + "__run_id__": self._run_id, + } run_id_sha256 = hashlib.sha256(self._run_id.encode("utf-8")).hexdigest() return [ effect.resolve( @@ -7831,7 +7940,7 @@ def _identity_gate_error( step: Step, resolution: Resolution, frame_png: bytes, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], workflow: Workflow, bundle_dir: Path, result: StepResult, @@ -8412,7 +8521,7 @@ def _revalidate_consequential_actuation( resolution: Optional[Resolution], matched_region: Optional[Region], before_png: bytes, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], workflow: Workflow, bundle_dir: Path, result: StepResult, @@ -8529,21 +8638,34 @@ def _revalidate_consequential_actuation( f"step '{step.id}' ({step.intent}): {detail}", ) try: - fresh_png = ( - self.backend.acquire_actuation_frame() - if isinstance(self.backend, RemoteActuationBackend) - else ( - cast( - GuardedKeyboardActionBackend, self.backend - ).guarded_keyboard_frame() - if ( - guarded_keyboard_backend - or guarded_type_pointer_backend - or guarded_click_pointer_backend - ) - else self.backend.screenshot() + if isinstance(self.backend, ActuationObservationBackend): + fresh_observation = self.backend.acquire_actuation_observation() + elif isinstance(self.backend, RemoteActuationBackend): + fresh_observation = self._legacy_single_surface_observation( + self.backend.acquire_actuation_frame() ) - ) + elif ( + guarded_keyboard_backend + or guarded_type_pointer_backend + or guarded_click_pointer_backend + ): + guarded_observer = getattr( + self.backend, + "guarded_keyboard_observation", + None, + ) + if callable(guarded_observer): + fresh_observation = guarded_observer() + else: + fresh_observation = self._legacy_single_surface_observation( + cast( + GuardedKeyboardActionBackend, self.backend + ).guarded_keyboard_frame() + ) + else: + fresh_observation = self._capture_frame_observation() + self._retain_frame_observation(fresh_observation, active=True) + fresh_png = fresh_observation.png except Exception as exc: # noqa: BLE001 - backend boundary must halt if self.governed_authorization is not None: result.safety_halt = True @@ -8740,6 +8862,19 @@ def _revalidate_consequential_actuation( f"overlaps protected evidence: {type(exc).__name__}" ), ) + if error is None and isinstance(self.backend, FrameObservationLeaseBackend): + try: + self.backend.bind_input_observation(fresh_observation) + except Exception as exc: # noqa: BLE001 - lease boundary must halt + self._cancel_guarded_coordinate() + self._cancel_guarded_keyboard() + return ( + fresh_resolution, + fresh_region, + fresh_png, + "Actuation preflight HALTED because the exact frame " + f"observation lease was refused ({type(exc).__name__})", + ) # Retain the exact observation that authorizes the next input edge. # Composite TYPE/SELECT_OPTION and retry paths can re-resolve inside # ``_act`` after the outer scope captured its initial geometry. A typed @@ -8811,7 +8946,7 @@ def _resolve_step( self.grounder if allow_grounder else None, step.intent, template_png=template_png, - viewport=_frame_viewport(screen_png), + viewport=self._viewport_for_frame(screen_png), structural=structural, allow_target_ocr=allow_target_ocr, ) @@ -8989,12 +9124,41 @@ def _fresh_actuation_event( if identity_regions else None ) + expected_observation = exc.expected_observation + observed_observation = exc.observed_observation + expected_display_id = None + observed_display_id = None + expected_display_bounds = None + observed_display_bounds = None + expected_display_scale = None + observed_display_scale = None + expected_topology_sha256 = None + observed_topology_sha256 = None + if expected_observation is not None and observed_observation is not None: + expected_display_id = expected_observation.display_id + observed_display_id = observed_observation.display_id + expected_display_bounds = expected_observation.display_bounds + observed_display_bounds = observed_observation.display_bounds + expected_display_scale = expected_observation.display_scale + observed_display_scale = observed_observation.display_scale + expected_topology_sha256 = expected_observation.topology_sha256 + observed_topology_sha256 = observed_observation.topology_sha256 return FreshActuationEvent( attempt=attempt, operation=exc.operation, changed_pixel_count=exc.changed_pixel_count, changed_bbox=exc.changed_bbox, frame_size=exc.frame_size, + expected_geometry_epoch=exc.expected_geometry_epoch, + observed_geometry_epoch=exc.observed_geometry_epoch, + expected_display_id=expected_display_id, + observed_display_id=observed_display_id, + expected_display_bounds=expected_display_bounds, + observed_display_bounds=observed_display_bounds, + expected_display_scale=expected_display_scale, + observed_display_scale=observed_display_scale, + expected_topology_sha256=expected_topology_sha256, + observed_topology_sha256=observed_topology_sha256, target_intersection=target_intersection, identity_intersection=identity_intersection, retried=retried, @@ -9003,9 +9167,9 @@ def _fresh_actuation_event( def _active_program_frame_refusal( self, workflow: Workflow, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], step: Step, - base_params: dict[str, str], + base_params: Mapping[str, RuntimeParamScalar], ) -> Optional[str]: """Bind the live interpreter path to the sealed program before input.""" @@ -9124,7 +9288,7 @@ def _active_program_frame_refusal( def _fresh_actuation_authorization_refusal( self, workflow: Workflow, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], step: Step, ) -> Optional[str]: """Recheck exact governed authority and inputs before fresh input.""" @@ -9357,7 +9521,7 @@ def _qualification_campaign_refusal(self, workflow: Workflow) -> Optional[str]: def _delivery_authorization_refusal( self, workflow: Workflow, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], step: Step, result: StepResult, ) -> Optional[str]: @@ -9421,7 +9585,7 @@ def _act( self, step: Step, resolution: Optional[Resolution], - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], *, workflow: Workflow, step_index: int, @@ -9909,7 +10073,7 @@ def _act( f"Step '{step.id}' ({step.intent}) requires parameter " f"'{step.param}' but it was not provided" ) - text = params[step.param] + text = runtime_param_text(params[step.param]) elif step.text is not None: text = step.text else: @@ -10124,7 +10288,7 @@ def _act( else None ) else: - before_png = self.backend.screenshot() + before_png = self._capture_frame_observation(active=True).png elif self._prev_was_click(workflow, step_index, graph_ctx): field_point = self._last_click_point field_region = self._last_click_region @@ -10698,7 +10862,7 @@ def _handle_interstitials( step: Step, before_png: bytes, bundle_dir: Path, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], result: StepResult, audit_events: list[InterstitialActionResult], workflow: Optional[Workflow], @@ -10843,7 +11007,7 @@ def _handle_interstitials( None, # NEVER ground a dismissal: stay model-free it.name, template_png=template_png, - viewport=_frame_viewport(before_png), + viewport=self._viewport_for_frame(before_png), structural=structural, ) if res is None: @@ -11024,7 +11188,7 @@ def _apply_step_gates( step: Step, before_png: bytes, bundle_dir: Path, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], result: StepResult, workflow: Optional[Workflow] = None, ) -> tuple[bool, Optional[str], bytes]: @@ -11156,7 +11320,7 @@ def _predicate_holds( pred: Predicate, frame_png: bytes, bundle_dir: Path, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], *, workflow: Optional[Workflow] = None, ) -> bool: @@ -11171,9 +11335,9 @@ def _predicate_holds( return evaluate_program_predicate( pred, frame_png, - params, + runtime_params_for_gui(params), vision=self.vision, - viewport=_frame_viewport(frame_png), + viewport=self._viewport_for_frame(frame_png), asset_loader=lambda rel: self._asset_bytes( bundle_dir, rel, @@ -11241,7 +11405,7 @@ def _compare_qualified_signal_text( signal: Any, anchor: Anchor, live: Optional[str], - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], workflow: Workflow, ) -> Literal["verified", "conflict", "unverifiable"]: """Compare one live source without returning or logging identity values.""" @@ -11280,7 +11444,7 @@ def _compare_qualified_signal_text( ) if set(parameter_names) != set(signal.params): return "unverifiable" - live_values = {**workflow.params, **params} + live_values = {**workflow.params, **runtime_params_for_gui(params)} live_form, used = parameterize_identity_text( live, live_values, @@ -11314,7 +11478,7 @@ def _compare_qualified_signal_text( match=signal.match.value, normalizers=signal.normalizers, live=live, - params=params, + params=runtime_params_for_gui(params), param_examples=workflow.params, parameter_names=signal.params, extract_pattern=extract_pattern, @@ -11427,7 +11591,7 @@ def _verify_signal_quorum( step: Step, resolution: Resolution, before_png: bytes, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], workflow: Workflow, bundle_dir: Optional[Path], policy: Any, @@ -11540,7 +11704,10 @@ def _verify_signal_quorum( # pixels than the demonstration. Exact/explicitly normalized # OCR may verify it only when the expected value does not # occupy the known glyph-confusable identifier class. - live_values = {**workflow.params, **params} + live_values = { + **workflow.params, + **runtime_params_for_gui(params), + } vulnerable = any( identity_mod.identity_rests_on_confusable_identifier( live_values.get(name) @@ -11592,7 +11759,7 @@ def _compare_direct_signal_text( recorded: Optional[str], live: Optional[str], *, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], workflow: Workflow, ) -> tuple[ Literal["verified", "conflict", "unverifiable"], @@ -11611,7 +11778,7 @@ def _compare_direct_signal_text( return "unverifiable", signal.params live, used = parameterize_identity_text( live, - {**workflow.params, **params}, + {**workflow.params, **runtime_params_for_gui(params)}, names=parameter_names, minimum_chars=identity_mod.MIN_PARAM_CHARS, case_sensitive=not ( @@ -11635,7 +11802,7 @@ def _verify_identity( step: Step, resolution: Resolution, before_png: bytes, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], workflow: Workflow, bundle_dir: Optional[Path] = None, ) -> IdentityCheck: @@ -11689,7 +11856,7 @@ def _verify_identity( step=step, resolution=resolution, before_png=before_png, - params=params, + params=runtime_params_for_gui(params), workflow=workflow, bundle_dir=bundle_dir, policy=identity_policy, @@ -11717,7 +11884,7 @@ def structured_tier() -> Optional[IdentityCheck]: return itmpl.verify_structured_template( tmpl, live, - params=params, + params=runtime_params_for_gui(params), param_examples=workflow.params, ) return identity_mod.verify_structured_identity(recorded, live) @@ -11823,7 +11990,7 @@ def _verify_identity_ocr( step: Step, resolution: Resolution, before_png: bytes, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], workflow: Workflow, ) -> IdentityCheck: """OCR name+DOB-primary identity tier (the pixel-substrate fallback). @@ -11863,7 +12030,9 @@ def _verify_identity_ocr( rh, ) band = identity_mod.band_region( - resolution.point, anchor.region[3], self.backend.viewport + resolution.point, + anchor.region[3], + self._viewport_for_frame(before_png), ) # The recorded band was extracted EXCLUDING the target's own crop # (labels are mutable evidence the ladder heals through) and @@ -11923,7 +12092,7 @@ def attempt( return itmpl.verify_template_identity( anchor.identity_template, observed, - params=params, + params=runtime_params_for_gui(params), param_examples=workflow.params, ) # This branch means no identity_template, so the constructor-time @@ -11933,7 +12102,7 @@ def attempt( return identity_mod.verify_target_identity( anchor.context_text, observed, - params=params, + params=runtime_params_for_gui(params), param_examples=workflow.params, ) @@ -11986,9 +12155,16 @@ def _field_region( self, field_point: Optional[Point], structural_region: Optional[Region] = None, + *, + frame_png: Optional[bytes] = None, ) -> Optional[Region]: """Region to observe for typed input, or None for the whole frame.""" - vw, vh = self.backend.viewport + if frame_png is None: + observation = self._last_frame_observation + if observation is None: + observation = self._capture_frame_observation() + frame_png = observation.png + vw, vh = self._viewport_for_frame(frame_png) if structural_region is not None: sx, sy, sw, sh = structural_region x = max(0, sx - FIELD_REGION_PAD) @@ -12082,7 +12258,11 @@ def _typed_input_landed( retyping is only safe when nothing changed). """ after_png = self.vision.wait_settled(self.backend) - region = self._field_region(field_point, field_region) + region = self._field_region( + field_point, + field_region, + frame_png=baseline_png, + ) if baseline_field_value is not None: expected = baseline_field_value + text after_at_point = ( @@ -12156,7 +12336,7 @@ def _verify_typed_input( result: StepResult, *, workflow: Workflow, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], bundle_dir: Path, run_dir: Path, resolution: Optional[Resolution], @@ -12317,7 +12497,7 @@ def _verify_typed_input( result.input_verified = False return retry_error else: - retry_baseline = self.backend.screenshot() + retry_baseline = self._capture_frame_observation().png if ( needs_revalidation and field_point is not None @@ -12546,7 +12726,7 @@ def _implicit_scroll_target_ready( *, workflow: Workflow, bundle_dir: Path, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], ) -> bool: """Resolve an implicit scroll target and honor its armed identity. @@ -12574,7 +12754,7 @@ def _implicit_scroll_target_ready( None, # scroll readiness must remain deterministic and model-free step.intent, template_png=template_png, - viewport=_frame_viewport(frame_png), + viewport=self._viewport_for_frame(frame_png), structural=structural, ) except OcrResolutionRefused: @@ -12616,7 +12796,7 @@ def _act_scroll( bundle_dir: Path, run_dir: Path, before_png: bytes, - params: dict[str, str], + params: Mapping[str, RuntimeParamScalar], result: StepResult, graph_ctx: Optional["_GraphStepContext"] = None, ) -> Optional[str]: @@ -12795,12 +12975,12 @@ def _wait_for_scroll_transition_and_settle(self, baseline_png: bytes) -> bytes: """ deadline = time.monotonic() + self.settle_readiness_timeout_s - frame = self.backend.screenshot() + frame = self._capture_frame_observation().png while not self.vision.pixels_changed(baseline_png, frame): if time.monotonic() >= deadline: return frame time.sleep(self.poll_interval_s) - frame = self.backend.screenshot() + frame = self._capture_frame_observation().png return self.vision.wait_settled(self.backend) def _prepare_remote_scroll_input( @@ -12821,7 +13001,15 @@ def _prepare_remote_scroll_input( if not isinstance(self.backend, RemoteActuationBackend): return None try: - self.backend.acquire_actuation_frame() + if isinstance(self.backend, ActuationObservationBackend): + observation = self.backend.acquire_actuation_observation() + else: + observation = self._legacy_single_surface_observation( + self.backend.acquire_actuation_frame() + ) + self._retain_frame_observation(observation, active=True) + if isinstance(self.backend, FrameObservationLeaseBackend): + self.backend.bind_input_observation(observation) except Exception as exc: # noqa: BLE001 - backend boundary must halt if self.governed_authorization is not None: result.safety_halt = True @@ -13016,7 +13204,7 @@ def _poll_postconditions( if time.monotonic() >= deadline: return False, frame_png time.sleep(self.poll_interval_s) - frame_png = self.backend.screenshot() + frame_png = self._capture_frame_observation().png return True, frame_png def _postcondition_passes( @@ -13057,6 +13245,19 @@ def _postcondition_passes( if kind == "region_stable": if pc.region is None or pc.phash is None: return True + source_observation = self._active_frame_observation + current_observation = self._observation_for_frame(frame_png) + if ( + source_observation is not None + and current_observation.coordinate_mapping_epoch + != source_observation.coordinate_mapping_epoch + ): + # A raw region has no authority after reflow. A future IR field + # can carry a named target/anchor binding and map the region + # from a newly resolved identity. Until that exact binding is + # present, refuse this postcondition rather than applying old + # viewport coordinates to a new geometry epoch. + return False region = tuple(pc.region) # Template check first: real apps re-layout by a few pixels # between runs (auto-scrolling panes, variable banner heights), @@ -13065,7 +13266,9 @@ def _postcondition_passes( template_png = self._postcondition_template(pc, bundle_dir) if template_png is not None: search = pad_region( - region, PC_TEMPLATE_SEARCH_PAD, _frame_viewport(frame_png) + region, + PC_TEMPLATE_SEARCH_PAD, + self._viewport_for_frame(frame_png), ) match = self.vision.find_template( frame_png, diff --git a/openadapt_flow/source_geometry.py b/openadapt_flow/source_geometry.py new file mode 100644 index 00000000..d33998cc --- /dev/null +++ b/openadapt_flow/source_geometry.py @@ -0,0 +1,123 @@ +"""Closed per-action native source-geometry evidence.""" + +from __future__ import annotations + +import hashlib +import json +import math +from typing import Literal + +from pydantic import BaseModel, ConfigDict, Field, model_validator + +NATIVE_SOURCE_GEOMETRY_SCHEMA_VERSION = "openadapt.flow.native-source-geometry/v1" +_BINDING_DOMAIN = b"openadapt.flow.native-source-geometry.v1\0" +NATIVE_SOURCE_GEOMETRY_BINDING_FIELDS = ( + "schema_version", + "source_surface", + "source_capture_session_sha256", + "source_capture_terminal_sha256", + "source_artifact_manifest_sha256", + "source_action_ordinal", + "source_frame_ordinal", + "frame_sha256", + "window_id", + "owner", + "pid", + "process_start_time", + "coordinate_source", + "geometry_generation", + "geometry_epoch_sha256", + "display_topology_sha256", + "bounds", + "scale_x", + "scale_y", + "viewport", + "source_viewport", + "content_rect", + "fit_scale", +) + + +def native_source_geometry_sha256(payload: dict) -> str: + """Hash the exact, closed native source binding field list.""" + closed = { + field: payload.get(field) for field in NATIVE_SOURCE_GEOMETRY_BINDING_FIELDS + } + raw = json.dumps( + closed, + sort_keys=True, + separators=(",", ":"), + ensure_ascii=False, + allow_nan=False, + ).encode("utf-8") + return hashlib.sha256(_BINDING_DOMAIN + raw).hexdigest() + + +class NativeSourceGeometry(BaseModel): + """One native action bound to one verified source frame and geometry epoch.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + schema_version: Literal["openadapt.flow.native-source-geometry/v1"] + source_surface: Literal["windows", "macos", "linux"] + source_capture_session_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + source_capture_terminal_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + source_artifact_manifest_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + source_action_ordinal: int = Field(ge=1) + source_frame_ordinal: int = Field(ge=1) + frame_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + window_id: str = Field(min_length=1) + owner: str = Field(min_length=1) + pid: int = Field(gt=0) + process_start_time: float = Field(gt=0) + coordinate_source: str = Field(min_length=1) + geometry_generation: int = Field(ge=1) + geometry_epoch_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + display_topology_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + bounds: tuple[float, float, float, float] + scale_x: float = Field(gt=0) + scale_y: float = Field(gt=0) + viewport: tuple[int, int] + source_viewport: tuple[int, int] + content_rect: tuple[int, int, int, int] + fit_scale: float = Field(gt=0) + binding_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + + @model_validator(mode="after") + def _closed_binding(self) -> "NativeSourceGeometry": + if self.source_action_ordinal <= self.source_frame_ordinal: + raise ValueError("a native action must follow its bound source frame") + numeric = ( + self.process_start_time, + *self.bounds, + self.scale_x, + self.scale_y, + self.fit_scale, + ) + if not all(math.isfinite(float(value)) for value in numeric): + raise ValueError("native source geometry must be finite") + if self.bounds[2] <= 0 or self.bounds[3] <= 0: + raise ValueError("native source bounds must be positive") + if any(value <= 0 for value in (*self.viewport, *self.source_viewport)): + raise ValueError("native source viewports must be positive") + left, top, width, height = self.content_rect + if ( + left < 0 + or top < 0 + or width <= 0 + or height <= 0 + or left + width > self.viewport[0] + or top + height > self.viewport[1] + ): + raise ValueError("native source content rectangle is outside its viewport") + expected_scale_x = width / self.bounds[2] + expected_scale_y = height / self.bounds[3] + if not math.isclose(self.scale_x, expected_scale_x) or not math.isclose( + self.scale_y, + expected_scale_y, + ): + raise ValueError("native source scales differ from its content geometry") + payload = self.model_dump(mode="json", exclude={"binding_sha256"}) + if self.binding_sha256 != native_source_geometry_sha256(payload): + raise ValueError("native source geometry binding digest is invalid") + return self diff --git a/openadapt_flow/source_terminal.py b/openadapt_flow/source_terminal.py new file mode 100644 index 00000000..7ec5d9ec --- /dev/null +++ b/openadapt_flow/source_terminal.py @@ -0,0 +1,61 @@ +"""Strict consumer model for an immutable openadapt-capture terminal.""" + +from __future__ import annotations + +import hashlib +import json +from typing import Literal, Optional + +from pydantic import BaseModel, ConfigDict, Field, model_validator + +_TERMINAL_DOMAIN = b"openadapt.capture-terminal.v2\0" + + +def _canonical_json_bytes(payload: object) -> bytes: + return json.dumps( + payload, + sort_keys=True, + separators=(",", ":"), + ensure_ascii=False, + allow_nan=False, + ).encode("utf-8") + + +class SourceCaptureEventCounts(BaseModel): + """Committed source event counts at recorder completion.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + action: int = Field(ge=0) + screen: int = Field(ge=0) + window: int = Field(ge=0) + browser: int = Field(ge=0) + video: int = Field(ge=0) + + +class SourceCaptureTerminal(BaseModel): + """The exact Capture v2 terminal accepted by Flow.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + schema_version: Literal["openadapt.capture-terminal/v2"] + state: Literal["COMPLETE"] + reason_code: Literal["normal_stop"] + source_capture_session_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + started_at: str = Field(min_length=20) + ended_at: str = Field(min_length=20) + event_counts: SourceCaptureEventCounts + last_source_ordinal: Optional[int] = Field(default=None, ge=1) + artifact_manifest_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + artifact_manifest_size_bytes: int = Field(gt=0) + terminal_sha256: str = Field(pattern=r"^[0-9a-f]{64}$") + + @model_validator(mode="after") + def _valid_digest(self) -> "SourceCaptureTerminal": + payload = self.model_dump(mode="json", exclude={"terminal_sha256"}) + expected = hashlib.sha256( + _TERMINAL_DOMAIN + _canonical_json_bytes(payload) + ).hexdigest() + if self.terminal_sha256 != expected: + raise ValueError("source capture terminal digest is invalid") + return self diff --git a/openadapt_flow/terminal_verification_v2.py b/openadapt_flow/terminal_verification_v2.py index 925df798..93a13750 100644 --- a/openadapt_flow/terminal_verification_v2.py +++ b/openadapt_flow/terminal_verification_v2.py @@ -1,9 +1,9 @@ """Production terminal-verification v2 contracts and producer. -Flow builds one success-only artifact after it revalidates a complete VERIFIED -report, run receipt, signed permit chain, and class-separated evidence. The -acceptor must reconstruct every value from immutable storage before it admits -the result as Production success. +Flow builds one signed terminal artifact after it revalidates a complete +VERIFIED, HALTED_BEFORE_EFFECT, or RECONCILIATION_REQUIRED report. The +acceptor reconstructs every value from immutable storage. Only a VERIFIED +artifact can authorize a Production success or a billable result. """ from __future__ import annotations @@ -32,13 +32,31 @@ model_validator, ) -from openadapt_flow.ir import PostconditionContractEvidence, RunReport +from openadapt_flow.ir import ( + EffectVerificationEvidence, + PostconditionContractEvidence, + RunReport, +) from openadapt_flow.qualification_admission_v2 import ( MAX_PERMIT_SNAPSHOT_AGE, ClosedSignedModel, canonical_json, ) -from openadapt_flow.receipt import ReceiptError, RunReceipt, build_receipt +from openadapt_flow.receipt import ( + ReceiptError, + RunReceipt, + _hour_utc, + _over_halt_count, + _receipt_builder_version, + build_receipt, +) +from openadapt_flow.transaction import ( + TransactionOutcome, + _attempt_state, + _effect_absence_proven, + _is_consequential_result, + classify_transaction_outcome, +) SCHEMA: Final[Literal["openadapt.production-terminal-verification/v2"]] = ( "openadapt.production-terminal-verification/v2" @@ -104,6 +122,7 @@ class PreparedProductionTerminalEvidence: execution_outcome: "ProductionExecutionOutcome" run_receipt: "ProductionRunReceipt" run_receipt_sha256: str + transaction_outcome: TransactionOutcome report: RunReport = dataclass_field(repr=False) @@ -130,10 +149,10 @@ def evidence_runner_signer_sha256(public_key: bytes) -> str: class TerminalContractCounts(ClosedSignedModel): - authorization: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - identity: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - postcondition: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - effect: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) + authorization: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + identity: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + postcondition: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + effect: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) class ProductionExecutionOutcome(ClosedSignedModel): @@ -142,11 +161,16 @@ class ProductionExecutionOutcome(ClosedSignedModel): version: Literal["openadapt.execution-outcome/v1"] = ( "openadapt.execution-outcome/v1" ) - outcome: Literal["VERIFIED"] = "VERIFIED" + outcome: Literal[ + "VERIFIED", + "HALTED", + "FAILED", + "ROLLED_BACK", + ] = "VERIFIED" profile: Literal["standard", "regulated"] - production_eligible: Literal[True] = True + production_eligible: StrictBool = True qualification_evidence_only: Literal[False] = False - execution_completed: Literal[True] = True + execution_completed: StrictBool = True required_contracts: TerminalContractCounts passed_contracts: TerminalContractCounts workflow_contract_sha256: str = Field(pattern=_SHA256_RE) @@ -164,27 +188,33 @@ class ProductionExecutionOutcome(ClosedSignedModel): "model", ], ..., - ] = Field(min_length=4, max_length=7) + ] = Field(min_length=1, max_length=7) model_calls: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) external_network_calls: Literal["none", "observed"] - compensation_actions: Literal[0] = 0 + compensation_actions: StrictInt = Field(default=0, ge=0, le=JS_MAX_SAFE_INTEGER) @model_validator(mode="after") - def _complete_verified_outcome(self) -> "ProductionExecutionOutcome": - if self.passed_contracts != self.required_contracts: - raise ValueError( - "terminal VERIFIED outcome lacks complete contract coverage" - ) + def _complete_terminal_outcome(self) -> "ProductionExecutionOutcome": + required = self.required_contracts.model_dump(mode="python") + passed = self.passed_contracts.model_dump(mode="python") + if any(passed[key] > required[key] for key in required): + raise ValueError("terminal passed contract counts exceed required counts") if self.required_contracts.authorization != 1: - raise ValueError("terminal VERIFIED outcome requires one authorization") + raise ValueError("terminal outcome requires one authorization") + if self.passed_contracts.authorization != 1: + raise ValueError("terminal outcome requires one passed authorization") if len(self.postcondition_evidence) != self.required_contracts.postcondition: raise ValueError("terminal postcondition evidence cardinality is invalid") + if ( + sum(item.verdict == "passed" for item in self.postcondition_evidence) + != self.passed_contracts.postcondition + ): + raise ValueError("terminal postcondition evidence count is invalid") if any( - item.verdict != "passed" - or item.workflow_contract_sha256 != self.workflow_contract_sha256 + item.workflow_contract_sha256 != self.workflow_contract_sha256 for item in self.postcondition_evidence ): - raise ValueError("terminal postcondition evidence is not fully verified") + raise ValueError("terminal postcondition evidence binding is invalid") keys = tuple( (item.result_index, item.contract_kind, item.contract_index) for item in self.postcondition_evidence @@ -202,20 +232,47 @@ def _complete_verified_outcome(self) -> "ProductionExecutionOutcome": self.evidence_classes ) != len(set(self.evidence_classes)): raise ValueError("terminal evidence classes must be unique and ordered") - required_classes = {"authorization", "identity", "postcondition"} - if not required_classes.issubset(self.evidence_classes): - raise ValueError( - "terminal VERIFIED outcome lacks required evidence classes" - ) + if "authorization" not in self.evidence_classes: + raise ValueError("terminal outcome lacks authorization evidence") + if (self.passed_contracts.identity > 0) != ( + "identity" in self.evidence_classes + ): + raise ValueError("terminal identity evidence class is invalid") + if (self.passed_contracts.postcondition > 0) != ( + "postcondition" in self.evidence_classes + ): + raise ValueError("terminal postcondition evidence class is invalid") effect_classes = { item for item in self.evidence_classes if item.startswith("effect_tier_") } - if len(effect_classes) != 1: - raise ValueError("terminal VERIFIED outcome requires one effect tier") + if (self.passed_contracts.effect > 0) != bool(effect_classes) or len( + effect_classes + ) > 1: + raise ValueError("terminal effect evidence classes are invalid") if (self.model_calls > 0) != ("model" in self.evidence_classes): raise ValueError("terminal model evidence does not match its call count") if self.model_calls > 0 and self.external_network_calls != "observed": raise ValueError("terminal model calls require observed network evidence") + if (self.compensation_actions > 0) != (self.outcome == "ROLLED_BACK"): + raise ValueError("terminal compensation evidence is invalid") + if self.outcome == "VERIFIED": + if self.passed_contracts != self.required_contracts: + raise ValueError( + "terminal VERIFIED outcome lacks complete contract coverage" + ) + if not self.production_eligible or not self.execution_completed: + raise ValueError("terminal VERIFIED outcome is not production complete") + required_classes = {"authorization", "identity", "postcondition"} + if not required_classes.issubset(self.evidence_classes): + raise ValueError( + "terminal VERIFIED outcome lacks required evidence classes" + ) + if len(effect_classes) != 1: + raise ValueError("terminal VERIFIED outcome requires one effect tier") + elif self.production_eligible: + raise ValueError("only a VERIFIED terminal outcome is production eligible") + elif self.outcome == "HALTED" and self.execution_completed: + raise ValueError("terminal HALTED outcome cannot claim completed execution") return self def artifact_sha256(self) -> str: @@ -225,20 +282,25 @@ def artifact_sha256(self) -> str: class ProductionRunReceipt(ClosedSignedModel): - """Cross-language integer projection of every current RunReceipt field.""" + """Cross-language integer projection of one terminal report.""" schema_version: Literal["openadapt.production-run-receipt/v1"] = ( "openadapt.production-run-receipt/v1" ) - source_schema_version: Literal["openadapt.run-receipt/v2"] = ( - "openadapt.run-receipt/v2" - ) - outcome: Literal["VERIFIED"] - transaction_outcome: Literal["VERIFIED"] + source_schema_version: Literal[ + "openadapt.run-receipt/v2", + "openadapt.run-report/v1", + ] + outcome: Literal["VERIFIED", "HALTED", "FAILED", "ROLLED_BACK"] + transaction_outcome: Literal[ + "VERIFIED", + "HALTED_BEFORE_EFFECT", + "RECONCILIATION_REQUIRED", + ] profile: Literal["standard", "regulated"] - production_eligible: Literal[True] + production_eligible: StrictBool steps_total: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - steps_ok: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) + steps_ok: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) heals: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) model_calls: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) est_cost_microusd: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) @@ -266,8 +328,9 @@ class ProductionRunReceipt(ClosedSignedModel): "model", ], ..., - ] = Field(min_length=4, max_length=7) + ] = Field(min_length=1, max_length=7) effect_tier_reached: Literal[ + "none", "independent_system", "independent_session", "persisted_state_reacquisition", @@ -275,14 +338,14 @@ class ProductionRunReceipt(ClosedSignedModel): authorization_required: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) authorization_confirmed: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) identity_required: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - identity_confirmed: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) + identity_confirmed: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) postconditions_required: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - postconditions_confirmed: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - effects_required: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - effects_confirmed: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - identity_armed: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - identity_applicable: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - over_halt_count: Literal[0] + postconditions_confirmed: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + effects_required: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + effects_confirmed: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + identity_armed: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + identity_applicable: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + over_halt_count: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) substrate: Literal["web", "windows", "macos", "linux", "rdp", "citrix"] provenance: Literal["production"] receipt_builder_version: str = Field( @@ -310,14 +373,14 @@ def _closed_receipt(self) -> "ProductionRunReceipt": ), ("effect", self.effects_required, self.effects_confirmed), ): - if required != confirmed: - raise ValueError(f"production receipt lacks complete {label} coverage") + if confirmed > required: + raise ValueError(f"terminal receipt exceeds required {label} coverage") if self.authorization_required != 1: - raise ValueError("production receipt requires one authorization") - if self.steps_ok != self.steps_total: - raise ValueError("production receipt requires all steps to succeed") - if self.identity_armed != self.identity_applicable: - raise ValueError("production receipt requires complete identity arming") + raise ValueError("terminal receipt requires one authorization") + if self.authorization_confirmed != 1: + raise ValueError("terminal receipt requires one passed authorization") + if self.steps_ok > self.steps_total: + raise ValueError("terminal receipt successful step count is invalid") if self.evidence_classes != tuple(sorted(self.evidence_classes)) or len( self.evidence_classes ) != len(set(self.evidence_classes)): @@ -327,6 +390,67 @@ def _closed_receipt(self) -> "ProductionRunReceipt": for value in self.rung_histogram.values() ): raise ValueError("production receipt rung counts are invalid") + effect_classes = { + item for item in self.evidence_classes if item.startswith("effect_tier_") + } + expected_effect_tier = { + "effect_tier_1": "independent_system", + "effect_tier_2": "independent_session", + "effect_tier_3": "persisted_state_reacquisition", + } + if self.effects_confirmed == 0: + if effect_classes or self.effect_tier_reached != "none": + raise ValueError("terminal receipt effect evidence is inconsistent") + elif ( + len(effect_classes) != 1 + or self.effect_tier_reached + != (expected_effect_tier[next(iter(effect_classes))]) + ): + raise ValueError("terminal receipt effect tier is inconsistent") + if self.transaction_outcome == "VERIFIED": + if ( + self.outcome != "VERIFIED" + or not self.production_eligible + or self.source_schema_version != "openadapt.run-receipt/v2" + or self.steps_ok != self.steps_total + or self.identity_armed != self.identity_applicable + or self.over_halt_count != 0 + or self.effect_tier_reached == "none" + ): + raise ValueError("VERIFIED terminal receipt is incomplete") + for label, required, confirmed in ( + ( + "authorization", + self.authorization_required, + self.authorization_confirmed, + ), + ("identity", self.identity_required, self.identity_confirmed), + ( + "postcondition", + self.postconditions_required, + self.postconditions_confirmed, + ), + ("effect", self.effects_required, self.effects_confirmed), + ): + if required != confirmed: + raise ValueError( + f"VERIFIED terminal receipt lacks complete {label} coverage" + ) + elif self.production_eligible: + raise ValueError( + "non-VERIFIED terminal receipt cannot be production eligible" + ) + elif self.source_schema_version != "openadapt.run-report/v1": + raise ValueError( + "non-VERIFIED terminal receipt must derive from its report" + ) + elif self.transaction_outcome == "HALTED_BEFORE_EFFECT": + if ( + self.outcome != "HALTED" + or self.effects_confirmed != 0 + or self.effect_tier_reached != "none" + ): + raise ValueError("HALTED terminal receipt does not prove no effect") return self @@ -347,6 +471,7 @@ def project_production_run_receipt(receipt: RunReceipt) -> ProductionRunReceipt: # instead of a static cast; a value outside the production subset raises. return ProductionRunReceipt.model_validate( { + "source_schema_version": "openadapt.run-receipt/v2", "outcome": receipt.outcome, "transaction_outcome": receipt.transaction_outcome, "profile": receipt.profile, @@ -383,10 +508,128 @@ def project_production_run_receipt(receipt: RunReceipt) -> ProductionRunReceipt: ) +def project_production_terminal_run_receipt( + report: RunReport, + *, + transaction_outcome: TransactionOutcome, +) -> ProductionRunReceipt: + """Project a non-success terminal report without creating a success receipt.""" + + if transaction_outcome not in { + TransactionOutcome.HALTED_BEFORE_EFFECT, + TransactionOutcome.RECONCILIATION_REQUIRED, + }: + raise ProductionTerminalVerificationError( + "non-success terminal receipt has an invalid transaction outcome" + ) + envelope = report.outcome_envelope + if ( + envelope is None + or report.execution_outcome not in {"HALTED", "FAILED", "ROLLED_BACK"} + or report.execution_profile not in {"standard", "regulated"} + or report.production_eligible + or report.execution_completed is True + and report.execution_outcome == "HALTED" + or report.transaction_outcome != transaction_outcome.value + or report.transaction_billable is not False + or report.bundle_content_digest is None + ): + raise ProductionTerminalVerificationError( + "terminal report lacks its closed non-success transaction contract" + ) + if not report.results: + raise ProductionTerminalVerificationError( + "terminal report contains no retained step result" + ) + report_bytes = canonical_json(report.model_dump(mode="json")) + report_sha256 = hashlib.sha256(report_bytes).hexdigest() + try: + micros = Decimal(str(round(float(report.est_model_cost_usd), 6))) * Decimal( + 1_000_000 + ) + except (InvalidOperation, TypeError, ValueError) as exc: + raise ProductionTerminalVerificationError( + "terminal report cost is not an exact integer microusd value" + ) from exc + if micros != micros.to_integral_value(): + raise ProductionTerminalVerificationError( + "terminal report cost is not an exact integer microusd value" + ) + required = envelope.required_contracts + passed = envelope.passed_contracts + confirmed_tiers = [ + int(item.verification_tier) + for result in report.results + for item in result.effect_evidence + if item.final_verdict == "confirmed" + and item.observed_effect == "present" + and item.verification_tier is not None + ] + if int(passed.effect) > 0 and len(confirmed_tiers) != int(passed.effect): + raise ProductionTerminalVerificationError( + "terminal report lacks exact confirmed effect tiers" + ) + effect_tier_reached = ( + { + 1: "independent_system", + 2: "independent_session", + 3: "persisted_state_reacquisition", + }.get(max(confirmed_tiers)) + if confirmed_tiers + else "none" + ) + if effect_tier_reached is None: + raise ProductionTerminalVerificationError( + "terminal report effect tier is outside the production range" + ) + try: + return ProductionRunReceipt.model_validate( + { + "source_schema_version": "openadapt.run-report/v1", + "outcome": report.execution_outcome, + "transaction_outcome": transaction_outcome.value, + "profile": report.execution_profile, + "production_eligible": False, + "steps_total": len(report.results), + "steps_ok": sum(1 for result in report.results if result.ok), + "heals": int(report.heal_count), + "model_calls": int(report.model_calls), + "est_cost_microusd": int(micros), + "duration_ms": int(round(float(report.total_ms))), + "rung_histogram": dict(report.rung_counts), + "evidence_classes": tuple(sorted(envelope.evidence_classes)), + "effect_tier_reached": effect_tier_reached, + "authorization_required": int(required.authorization), + "authorization_confirmed": int(passed.authorization), + "identity_required": int(required.identity), + "identity_confirmed": int(passed.identity), + "postconditions_required": int(required.postcondition), + "postconditions_confirmed": int(passed.postcondition), + "effects_required": int(required.effect), + "effects_confirmed": int(passed.effect), + "identity_armed": int(report.identity_armed_steps), + "identity_applicable": int(report.identity_applicable_steps), + "over_halt_count": _over_halt_count(report), + "substrate": report.execution_target_kind, + "provenance": "production", + "receipt_builder_version": _receipt_builder_version(), + "external_network_calls": envelope.external_network_calls, + "bundle_digest": report.bundle_content_digest, + "source_receipt_digest": report_sha256, + "source_receipt_sha256": report_sha256, + "generated_at": _hour_utc(report.started_at), + } + ) + except (TypeError, ValueError) as exc: + raise ProductionTerminalVerificationError( + "terminal report cannot produce a closed non-success receipt" + ) from exc + + def prepare_production_terminal_evidence( report: RunReport, ) -> PreparedProductionTerminalEvidence: - """Revalidate the exact report and build the only terminal-safe projections. + """Revalidate the exact report and build its terminal-safe projections. The returned report bytes are the bytes that the runner must store as the immutable report object. The production proof later binds the storage @@ -404,12 +647,42 @@ def prepare_production_terminal_evidence( envelope is None or validated.run_id_sha256 is None or validated.bundle_content_digest is None + or validated.workflow_contract_sha256 is None ): raise ProductionTerminalVerificationError( "terminal report lacks its run or bundle binding" ) + transaction_outcome = classify_transaction_outcome(validated) + if transaction_outcome not in { + TransactionOutcome.VERIFIED, + TransactionOutcome.HALTED_BEFORE_EFFECT, + TransactionOutcome.RECONCILIATION_REQUIRED, + }: + raise ProductionTerminalVerificationError( + "terminal report does not have an admitted terminal transaction outcome" + ) + if ( + validated.transaction_outcome != transaction_outcome.value + or validated.transaction_billable is not transaction_outcome.is_billable + ): + raise ProductionTerminalVerificationError( + "terminal report transaction fields differ from its retained evidence" + ) + if transaction_outcome is TransactionOutcome.HALTED_BEFORE_EFFECT and any( + _is_consequential_result(result) and not _effect_absence_proven(result) + for result in validated.results + ): + raise ProductionTerminalVerificationError( + "terminal HALTED report lacks exact effect-absence evidence" + ) try: - receipt = project_production_run_receipt(build_receipt(validated)) + if transaction_outcome is TransactionOutcome.VERIFIED: + receipt = project_production_run_receipt(build_receipt(validated)) + else: + receipt = project_production_terminal_run_receipt( + validated, + transaction_outcome=transaction_outcome, + ) outcome = ProductionExecutionOutcome.model_validate( { "version": envelope.version, @@ -435,7 +708,7 @@ def prepare_production_terminal_evidence( ) except (ReceiptError, ValueError) as exc: raise ProductionTerminalVerificationError( - "terminal report is not a complete production VERIFIED outcome" + "terminal report is not a complete production terminal outcome" ) from exc report_bytes = canonical_json(validated.model_dump(mode="json")) report_sha256 = hashlib.sha256(report_bytes).hexdigest() @@ -448,6 +721,7 @@ def prepare_production_terminal_evidence( execution_outcome=outcome, run_receipt=receipt, run_receipt_sha256=receipt_sha256, + transaction_outcome=transaction_outcome, report=validated, ) @@ -538,7 +812,7 @@ class ProductionIdentitySignal(ClosedSignedModel): class ProductionIdentityResult(ClosedSignedModel): result_index: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) - status: Literal["verified"] + status: Literal["verified", "mismatch", "abstain", "unreadable"] mode: Literal["context", "param", "structured", "pixel", "vlm", "signal_quorum"] signals: tuple[ProductionIdentitySignal, ...] = Field(max_length=32) @@ -559,17 +833,17 @@ class ProductionIdentityEvidenceManifest(ClosedSignedModel): ) identity_contract_sha256: str = Field(pattern=_SHA256_RE) workflow_contract_sha256: str = Field(pattern=_SHA256_RE) - required: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - confirmed: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - results: tuple[ProductionIdentityResult, ...] = Field( - min_length=1, max_length=10_000 - ) + required: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + confirmed: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + results: tuple[ProductionIdentityResult, ...] = Field(max_length=10_000) manifest_sha256: str = Field(pattern=_SHA256_RE) @model_validator(mode="after") def _digest(self) -> "ProductionIdentityEvidenceManifest": - if self.required != self.confirmed or len(self.results) != self.required: - raise ValueError("production identity evidence coverage is incomplete") + if self.confirmed > self.required or len(self.results) > self.required: + raise ValueError("production identity evidence coverage is invalid") + if sum(item.status == "verified" for item in self.results) != self.confirmed: + raise ValueError("production identity confirmed count is invalid") indices = tuple(item.result_index for item in self.results) if indices != tuple(sorted(indices)) or len(indices) != len(set(indices)): raise ValueError("production identity evidence results are invalid") @@ -586,23 +860,22 @@ class ProductionPostconditionEvidenceManifest(ClosedSignedModel): "openadapt.production-postcondition-evidence/v1" ) workflow_contract_sha256: str = Field(pattern=_SHA256_RE) - required: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - confirmed: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - records: tuple[PostconditionContractEvidence, ...] = Field( - min_length=1, max_length=10_000 - ) + required: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + confirmed: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + records: tuple[PostconditionContractEvidence, ...] = Field(max_length=10_000) manifest_sha256: str = Field(pattern=_SHA256_RE) @model_validator(mode="after") def _digest(self) -> "ProductionPostconditionEvidenceManifest": - if self.required != self.confirmed or len(self.records) != self.required: - raise ValueError("production postcondition evidence coverage is incomplete") + if self.confirmed > self.required or len(self.records) != self.required: + raise ValueError("production postcondition evidence coverage is invalid") if any( - item.verdict != "passed" - or item.workflow_contract_sha256 != self.workflow_contract_sha256 + item.workflow_contract_sha256 != self.workflow_contract_sha256 for item in self.records ): raise ValueError("production postcondition evidence is invalid") + if sum(item.verdict == "passed" for item in self.records) != self.confirmed: + raise ValueError("production postcondition confirmed count is invalid") expected = _evidence_manifest_sha256( b"openadapt-production-postcondition-evidence-v1\0", self ) @@ -624,23 +897,98 @@ class ProductionEffectEvidence(ClosedSignedModel): reconciliation_actions: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) +class ProductionTerminalEffectState(ClosedSignedModel): + """Remote-safe state for one effect contract on a non-success terminal.""" + + record_kind: Literal["terminal_effect_state"] = "terminal_effect_state" + result_index: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + effect_contract_hash: str = Field(pattern=r"^sha256:[a-f0-9]{64}$") + attempt_state: Literal[ + "not_actuated", + "delivered", + "actuated_api", + "delivery_uncertain", + ] + observed_effect: Literal["present", "absent", "conflicting", "unknown"] + effect_verified: StrictBool + verification_performed: StrictBool + verifier_identity: str | None = Field( + default=None, pattern=r"^sha256:[a-f0-9]{64}$" + ) + verification_tier: StrictInt | None = Field(default=None, ge=1, le=3) + final_verdict: Literal["confirmed", "refuted", "indeterminate"] | None + resolved_delivery_uncertainty: StrictBool + absence_basis: Literal["not_actuated", "verifier_refuted", "none"] + reconciliation_completed: StrictBool + reconciliation_actions: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + + @model_validator(mode="after") + def _closed_terminal_effect_state(self) -> "ProductionTerminalEffectState": + if self.verification_performed != (self.final_verdict is not None): + raise ValueError("terminal effect verification fields are inconsistent") + if self.verification_performed != (self.verification_tier is not None): + raise ValueError("terminal effect verification tier is inconsistent") + if self.verification_performed != (self.verifier_identity is not None): + raise ValueError("terminal effect verifier identity is inconsistent") + if self.effect_verified != ( + self.final_verdict == "confirmed" and self.observed_effect == "present" + ): + raise ValueError("terminal effect verified state is inconsistent") + if self.absence_basis == "not_actuated": + if ( + self.attempt_state != "not_actuated" + or self.observed_effect != "absent" + or self.verification_performed + or self.resolved_delivery_uncertainty + ): + raise ValueError("terminal non-actuation evidence is invalid") + elif self.absence_basis == "verifier_refuted": + if ( + self.attempt_state == "not_actuated" + or self.observed_effect != "absent" + or self.final_verdict != "refuted" + or not self.verification_performed + ): + raise ValueError("terminal verifier absence evidence is invalid") + elif self.attempt_state == "not_actuated" or ( + self.final_verdict == "refuted" and self.observed_effect == "absent" + ): + raise ValueError("terminal absence evidence must name its basis") + if self.reconciliation_completed != (self.reconciliation_actions > 0): + raise ValueError("terminal effect reconciliation fields are inconsistent") + return self + + class ProductionEffectEvidenceManifest(ClosedSignedModel): schema_version: Literal["openadapt.production-effect-evidence/v1"] = ( "openadapt.production-effect-evidence/v1" ) effect_contract_sha256: str = Field(pattern=_SHA256_RE) workflow_contract_sha256: str = Field(pattern=_SHA256_RE) - required: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - confirmed: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) - records: tuple[ProductionEffectEvidence, ...] = Field( - min_length=1, max_length=10_000 + required: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + confirmed: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) + records: tuple[ProductionEffectEvidence | ProductionTerminalEffectState, ...] = ( + Field(max_length=10_000) ) manifest_sha256: str = Field(pattern=_SHA256_RE) @model_validator(mode="after") def _digest(self) -> "ProductionEffectEvidenceManifest": - if self.required != self.confirmed or len(self.records) != self.required: - raise ValueError("production effect evidence coverage is incomplete") + if self.confirmed > self.required or len(self.records) != self.required: + raise ValueError("production effect evidence coverage is invalid") + if all(isinstance(item, ProductionEffectEvidence) for item in self.records): + if self.required != self.confirmed: + raise ValueError("production effect success coverage is incomplete") + elif ( + sum( + isinstance(item, ProductionTerminalEffectState) + and item.final_verdict == "confirmed" + and item.observed_effect == "present" + for item in self.records + ) + != self.confirmed + ): + raise ValueError("production effect confirmed count is invalid") keys = tuple( (item.result_index, item.effect_contract_hash) for item in self.records ) @@ -1196,9 +1544,7 @@ class ProductionDeliveryPermitChain(ClosedSignedModel): schema_version: Literal["openadapt.production-delivery-permit-chain/v2"] = ( PERMIT_CHAIN_SCHEMA ) - entries: tuple[ProductionDeliveryPermit, ...] = Field( - min_length=1, max_length=10_000 - ) + entries: tuple[ProductionDeliveryPermit, ...] = Field(max_length=10_000) permit_chain_sha256: str = Field(pattern=_SHA256_RE) @model_validator(mode="after") @@ -1208,6 +1554,19 @@ def _closed_chain(self) -> "ProductionDeliveryPermitChain": # in-memory mutation cannot enter a trusted chain with a stale digest. for item in self.entries: ProductionDeliveryPermit.model_validate(item.model_dump(mode="json")) + expected = hashlib.sha256( + PERMIT_CHAIN_DOMAIN + + canonical_json( + { + "schema_version": self.schema_version, + "entries": [item.model_dump(mode="json") for item in self.entries], + } + ) + ).hexdigest() + if self.permit_chain_sha256 != expected: + raise ValueError("delivery permit chain digest is invalid") + if not self.entries: + return self authority = self.entries[0].execution_authority_id authority_digest = self.entries[0].execution_authority_sha256 admission_digest = self.entries[0].admission_artifact_sha256 @@ -1281,17 +1640,6 @@ def _closed_chain(self) -> "ProductionDeliveryPermitChain": for previous, current in zip(event_times, event_times[1:]) ): raise ValueError("delivery permit chronology is invalid") - expected = hashlib.sha256( - PERMIT_CHAIN_DOMAIN - + canonical_json( - { - "schema_version": self.schema_version, - "entries": [item.model_dump(mode="json") for item in self.entries], - } - ) - ).hexdigest() - if self.permit_chain_sha256 != expected: - raise ValueError("delivery permit chain digest is invalid") return self @classmethod @@ -1410,25 +1758,109 @@ def build_production_evidence_manifests( confirmed=outcome.passed_contracts.postcondition, records=outcome.postcondition_evidence, ) - effect_records: list[ProductionEffectEvidence] = [] + effect_records: list[ProductionEffectEvidence | ProductionTerminalEffectState] = [] for result_index, result in enumerate(report.results): if result.skipped or result.exception_handled: continue - for item in result.effect_evidence: + if prepared.transaction_outcome is TransactionOutcome.VERIFIED: + for effect_item in result.effect_evidence: + effect_records.append( + ProductionEffectEvidence.model_validate( + { + "result_index": result_index, + "effect_contract_hash": effect_item.effect_contract_hash, + "verifier_identity": effect_item.verifier_identity, + "verification_tier": effect_item.verification_tier, + "final_verdict": effect_item.final_verdict, + "observed_effect": effect_item.observed_effect, + "reconciliation_completed": ( + effect_item.reconciliation_completed + ), + "reconciliation_actions": ( + effect_item.reconciliation_actions + ), + } + ) + ) + continue + if not _is_consequential_result(result): + continue + evidence_by_hash: dict[str, list[EffectVerificationEvidence]] = {} + for effect_item in result.effect_evidence: + evidence_by_hash.setdefault(effect_item.effect_contract_hash, []).append( + effect_item + ) + effect_hashes = tuple(sorted(set(result.effect_contract_hashes))) + if len(effect_hashes) != len(result.effect_contract_hashes): + raise ProductionTerminalVerificationError( + "terminal effect contract hashes are duplicated" + ) + attempt_state = _attempt_state(result) + uncertainty_resolved = bool( + result.delivery_uncertainty is not None + and result.delivery_uncertainty.resolved_by_contract + ) + for effect_hash in effect_hashes: + matches = evidence_by_hash.pop(effect_hash, []) + if len(matches) > 1: + raise ProductionTerminalVerificationError( + "terminal effect evidence is duplicated" + ) + terminal_item: EffectVerificationEvidence | None = ( + matches[0] if matches else None + ) + if terminal_item is None: + observed_effect = ( + "absent" if attempt_state == "not_actuated" else "unknown" + ) + final_verdict = None + verification_tier = None + verifier_identity = None + reconciliation_completed = False + reconciliation_actions = 0 + absence_basis = ( + "not_actuated" if attempt_state == "not_actuated" else "none" + ) + else: + observed_effect = terminal_item.observed_effect + final_verdict = terminal_item.final_verdict + verification_tier = terminal_item.verification_tier + verifier_identity = terminal_item.verifier_identity + reconciliation_completed = terminal_item.reconciliation_completed + reconciliation_actions = terminal_item.reconciliation_actions + absence_basis = ( + "verifier_refuted" + if terminal_item.final_verdict == "refuted" + and terminal_item.observed_effect == "absent" + else "none" + ) effect_records.append( - ProductionEffectEvidence.model_validate( + ProductionTerminalEffectState.model_validate( { "result_index": result_index, - "effect_contract_hash": item.effect_contract_hash, - "verifier_identity": item.verifier_identity, - "verification_tier": item.verification_tier, - "final_verdict": item.final_verdict, - "observed_effect": item.observed_effect, - "reconciliation_completed": item.reconciliation_completed, - "reconciliation_actions": item.reconciliation_actions, + "effect_contract_hash": effect_hash, + "attempt_state": attempt_state, + "observed_effect": observed_effect, + "effect_verified": bool( + terminal_item is not None + and terminal_item.final_verdict == "confirmed" + and terminal_item.observed_effect == "present" + ), + "verification_performed": terminal_item is not None, + "verifier_identity": verifier_identity, + "verification_tier": verification_tier, + "final_verdict": final_verdict, + "resolved_delivery_uncertainty": uncertainty_resolved, + "absence_basis": absence_basis, + "reconciliation_completed": reconciliation_completed, + "reconciliation_actions": reconciliation_actions, } ) ) + if evidence_by_hash: + raise ProductionTerminalVerificationError( + "terminal effect evidence lacks a declared contract" + ) effect_records.sort(key=lambda item: (item.result_index, item.effect_contract_hash)) effect = build_evidence_manifest( ProductionEffectEvidenceManifest, @@ -1482,7 +1914,7 @@ class ProductionTerminalVerificationPayload(ClosedSignedModel): execution_authority_sha256: str = Field(pattern=_SHA256_RE) execution_authority_signer_sha256: str = Field(pattern=_SHA256_RE) permit_chain: ProductionDeliveryPermitChain - permit_count: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) + permit_count: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) final_authority_sequence: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) final_runtime_delivery_sequence: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) workflow_contract_sha256: str = Field(pattern=_SHA256_RE) @@ -1498,7 +1930,7 @@ class ProductionTerminalVerificationPayload(ClosedSignedModel): issued_at: str @model_validator(mode="after") - def _closed_terminal_success(self) -> "ProductionTerminalVerificationPayload": + def _closed_terminal_outcome(self) -> "ProductionTerminalVerificationPayload": if hashlib.sha256(self.run_id.encode("utf-8")).hexdigest() != ( self.flow_run_id_sha256 ): @@ -1508,49 +1940,63 @@ def _closed_terminal_success(self) -> "ProductionTerminalVerificationPayload": if self.workflow_version_id != self.bundle_version_id: raise ValueError("terminal workflow and bundle versions must match") chain = self.permit_chain - final = chain.entries[-1] - if ( - self.execution_authority_id, - self.execution_authority_sha256, - self.execution_authority_signer_sha256, - self.admission_artifact_sha256, - ) != ( - final.execution_authority_id, - final.execution_authority_sha256, - final.authority_signer_sha256, - final.admission_artifact_sha256, - ): - raise ValueError("terminal authority does not match its permit chain") - if ( - self.run_id != final.run_id - or self.flow_run_id_sha256 != final.flow_run_id_sha256 - ): - raise ValueError("terminal run identity does not match its permit chain") - if ( - self.evidence_identity_sha256, - self.environment_digest, - self.qualification_signer_registry_sha256, - self.qualification_signer_registry_revision, - ) != ( - final.evidence_identity_sha256, - final.environment_digest, - final.qualification_signer_registry_sha256, - final.qualification_signer_registry_revision, - ): - raise ValueError("terminal qualification state does not match its permits") - if ( - self.permit_count, - self.final_authority_sequence, - self.final_runtime_delivery_sequence, - ) != ( - len(chain.entries), - final.authority_sequence, - final.runtime_delivery_sequence, - ): - raise ValueError("terminal permit counts do not match the permit chain") receipt = self.run_receipt outcome = self.execution_outcome manifests = self.evidence_manifests + if chain.entries: + final = chain.entries[-1] + if ( + self.execution_authority_id, + self.execution_authority_sha256, + self.execution_authority_signer_sha256, + self.admission_artifact_sha256, + ) != ( + final.execution_authority_id, + final.execution_authority_sha256, + final.authority_signer_sha256, + final.admission_artifact_sha256, + ): + raise ValueError("terminal authority does not match its permit chain") + if ( + self.run_id != final.run_id + or self.flow_run_id_sha256 != final.flow_run_id_sha256 + ): + raise ValueError( + "terminal run identity does not match its permit chain" + ) + if ( + self.evidence_identity_sha256, + self.environment_digest, + self.qualification_signer_registry_sha256, + self.qualification_signer_registry_revision, + ) != ( + final.evidence_identity_sha256, + final.environment_digest, + final.qualification_signer_registry_sha256, + final.qualification_signer_registry_revision, + ): + raise ValueError( + "terminal qualification state does not match its permits" + ) + if ( + self.permit_count, + self.final_authority_sequence, + self.final_runtime_delivery_sequence, + ) != ( + len(chain.entries), + final.authority_sequence, + final.runtime_delivery_sequence, + ): + raise ValueError("terminal permit counts do not match the permit chain") + elif ( + receipt.transaction_outcome != "HALTED_BEFORE_EFFECT" + or self.permit_count != 0 + or self.final_authority_sequence != 0 + or self.final_runtime_delivery_sequence != 0 + ): + raise ValueError( + "only a HALTED_BEFORE_EFFECT terminal may use an empty permit chain" + ) if ( self.workflow_contract_sha256 != outcome.workflow_contract_sha256 or self.execution_outcome_sha256 != outcome.artifact_sha256() @@ -1622,17 +2068,56 @@ def _closed_terminal_success(self) -> "ProductionTerminalVerificationPayload": raise ValueError( "terminal run receipt does not match its execution outcome" ) + transaction_outcome = receipt.transaction_outcome + if transaction_outcome == "VERIFIED": + if ( + outcome.outcome != "VERIFIED" + or not outcome.production_eligible + or not outcome.execution_completed + or not chain.entries + ): + raise ValueError("terminal VERIFIED proof is incomplete") + elif transaction_outcome == "HALTED_BEFORE_EFFECT": + if ( + outcome.outcome != "HALTED" + or outcome.production_eligible + or outcome.execution_completed + or receipt.production_eligible + or any( + not isinstance(item, ProductionTerminalEffectState) + or item.absence_basis not in {"not_actuated", "verifier_refuted"} + for item in manifests.effect.records + ) + ): + raise ValueError("terminal HALTED proof lacks exact effect absence") + elif ( + transaction_outcome != "RECONCILIATION_REQUIRED" + or outcome.outcome == "VERIFIED" + or outcome.production_eligible + or receipt.production_eligible + or not chain.entries + ): + raise ValueError("terminal reconciliation proof is invalid") + if transaction_outcome != "VERIFIED" and ( + receipt.source_receipt_digest != self.run_report_sha256 + or receipt.source_receipt_sha256 != self.run_report_sha256 + ): + raise ValueError("terminal non-success receipt does not bind its report") verified = _parse_utc(self.verified_at, field="terminal verified_at") issued = _parse_utc(self.issued_at, field="terminal issued_at") if not verified <= issued <= verified + timedelta(minutes=5): raise ValueError("terminal proof issue time is invalid") - final_delivered = _parse_utc(final.delivered_at, field="permit delivered_at") - registry_expires = _parse_utc( - final.qualification_signer_registry_expires_at, - field="permit registry expires_at", - ) - if not final_delivered <= verified < registry_expires: - raise ValueError("terminal verification is outside permit chronology") + if chain.entries: + final = chain.entries[-1] + final_delivered = _parse_utc( + final.delivered_at, field="permit delivered_at" + ) + registry_expires = _parse_utc( + final.qualification_signer_registry_expires_at, + field="permit registry expires_at", + ) + if not final_delivered <= verified < registry_expires: + raise ValueError("terminal verification is outside permit chronology") if self.run_report_object_sha256 != self.run_report_sha256: raise ValueError( "terminal report object must contain the exact revalidated report bytes" @@ -1734,14 +2219,12 @@ class ProductionTerminalVerificationExpected(ClosedSignedModel): execution_authority_sha256: str = Field(pattern=_SHA256_RE) execution_authority_signer_sha256: str = Field(pattern=_SHA256_RE) permit_chain_sha256: str = Field(pattern=_SHA256_RE) - permit_count: StrictInt = Field(ge=1, le=JS_MAX_SAFE_INTEGER) + permit_count: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) final_authority_sequence: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) final_runtime_delivery_sequence: StrictInt = Field(ge=0, le=JS_MAX_SAFE_INTEGER) authenticated_runner_id_sha256: str = Field(pattern=_SHA256_RE) authenticated_session_id_sha256: str = Field(pattern=_SHA256_RE) - acknowledged_one_use_claim_ids: tuple[str, ...] = Field( - min_length=1, max_length=10_000 - ) + acknowledged_one_use_claim_ids: tuple[str, ...] = Field(max_length=10_000) workflow_contract_sha256: str = Field(pattern=_SHA256_RE) execution_outcome_sha256: str = Field(pattern=_SHA256_RE) run_receipt_sha256: str = Field(pattern=_SHA256_RE) @@ -1885,7 +2368,7 @@ def build_production_terminal_verification( execution_authority_sha256=context.execution_authority_sha256, permit_chain=context.permit_chain, ) - final = context.permit_chain.entries[-1] + final = context.permit_chain.entries[-1] if context.permit_chain.entries else None payload = ProductionTerminalVerificationPayload( run_id=context.run_id, flow_run_id_sha256=prepared.flow_run_id_sha256, @@ -1919,8 +2402,10 @@ def build_production_terminal_verification( execution_authority_signer_sha256=(context.execution_authority_signer_sha256), permit_chain=context.permit_chain, permit_count=len(context.permit_chain.entries), - final_authority_sequence=final.authority_sequence, - final_runtime_delivery_sequence=final.runtime_delivery_sequence, + final_authority_sequence=(final.authority_sequence if final is not None else 0), + final_runtime_delivery_sequence=( + final.runtime_delivery_sequence if final is not None else 0 + ), workflow_contract_sha256=(prepared.execution_outcome.workflow_contract_sha256), execution_outcome=prepared.execution_outcome, execution_outcome_sha256=prepared.execution_outcome.artifact_sha256(), diff --git a/openadapt_flow/tutorial.py b/openadapt_flow/tutorial.py index 5ce08dd2..2251e4fa 100644 --- a/openadapt_flow/tutorial.py +++ b/openadapt_flow/tutorial.py @@ -77,14 +77,15 @@ #: key so the mined contract can assert at-most-once honestly. TUTORIAL_ENTRY_QUERY = "?fault=ok&idempotency=demo#tasks" -#: The fault mode ``--break-it`` injects. ``optimistic`` is the sharpest -#: demonstration of the product's claim: the backend REJECTS the write AFTER +#: The fault mode ``--simulate-rejected-write`` injects. ``optimistic`` is the +#: sharpest demonstration of the product's claim. +#: The backend REJECTS the write AFTER #: the application has already painted its success banner, so every on-screen #: check passes while nothing landed in the system of record. Only an #: independent read of that system can catch it -- which is the point. TUTORIAL_BREAK_FAULT = "optimistic" -#: The entry query for the ``--break-it`` rerun. Identical to the clean +#: The entry query for the ``--simulate-rejected-write`` rerun. Identical to the #: query except for the fault mode; the bundle, the policy, the gate, and the #: verifier are all unchanged. TUTORIAL_BREAK_ENTRY_QUERY = f"?fault={TUTORIAL_BREAK_FAULT}&idempotency=demo#tasks" @@ -96,7 +97,7 @@ class TutorialError(RuntimeError): @dataclass class BreakItResult: - """What the ``--break-it`` rerun proved, for the CLI's narrative. + """What the rejected-write simulation proved for the CLI narrative. Every field is read from the halted run's own report or from the fault server's ground-truth store -- nothing here is scripted output. @@ -121,6 +122,9 @@ class BreakItResult: #: Rows in the independent system of record after the run (0: the write #: the screen claimed was never persisted). system_of_record_records: int + #: Exact rejected POST attempts observed by the fault backend. One proves + #: that the fixture dispatched once and did not retry blindly. + rejected_writes: int @dataclass @@ -144,25 +148,39 @@ class TutorialResult: # billable. This local-only tutorial never reports usage to Cloud. reported_to_metering: bool = False receipt_paths: dict[str, Path] = field(default_factory=dict) - #: Present only when the tutorial ran with ``break_it=True``: the same - #: certified bundle, rerun against a backend that lies, and the engine's - #: halt that caught it. + #: Present only for the advanced rejected-write simulation: the same + #: certified bundle, rerun against a backend that lies, with the retained + #: halt evidence that caught the false success. break_it: Optional[BreakItResult] = None def _next_steps_block() -> str: """The closing block the CLI prints after a plain VERIFIED tutorial run. - Only the success rail earns it: a halt or a ``--break-it`` rerun ends on - its own narrative instead. The links match the flagship OpenAdapt - README so every surface points at the same three destinations. + Only the primary success rail earns it. A halt or an advanced verification + simulation ends on its own evidence instead. """ return ( - "That run made no model call and sent nothing off this computer.\n" - " What it proves https://openadapt.ai/execute\n" - " Run it on your work https://openadapt.ai/qualify\n" - " Community https://discord.gg/yF527cQbDG" + "Next: automate one small read-only task with test data in your own " + "app. Write down the result you expect.\n" + " Record openadapt-flow record --backend web " + "--url https://your-app.example --out recording\n" + " Compile openadapt-flow compile recording --out bundle " + "--name my-task\n" + " Inspect openadapt-flow visualize bundle -o graph.html\n" + " Lint openadapt-flow lint bundle\n" + "Confirm that the bundle contains only the read-only task you " + "selected. If lint reports a state-changing, unknown, consequential, " + "or irreversible action, stop and qualify its identity, effect, and " + "policy evidence before Flow first actuates it:\n" + " https://openadapt.ai/qualify\n" + " Replay openadapt-flow replay bundle --backend web " + "--url https://your-app.example --headed --run-dir first-run\n" + " Review first-run/REPORT.md\n" + "Confirm that the recorded steps and final result match what you " + "expected before you expand the task.\n" + "Qualify the exact app and environment before unattended use." ) @@ -434,8 +452,9 @@ def run_tutorial_workflow( ) -> Any: """Admit and execute the tutorial under the ``standard`` profile. - ``entry_query`` defaults to the clean :data:`TUTORIAL_ENTRY_QUERY`; the - ``--break-it`` rerun passes :data:`TUTORIAL_BREAK_ENTRY_QUERY` instead. + ``entry_query`` defaults to the clean :data:`TUTORIAL_ENTRY_QUERY`. + The rejected-write simulation passes :data:`TUTORIAL_BREAK_ENTRY_QUERY` + instead. Nothing else differs between the two runs: same bundle, same policy, same gate, same verifier. """ @@ -520,6 +539,7 @@ def run_tutorial( interactive_record: bool = False, presentation_delay_s: float = 0.0, echo: Optional[Callable[[str], None]] = None, + simulate_rejected_write: bool = False, break_it: bool = False, ) -> TutorialResult: """Run the complete free path and return its evidence. @@ -527,14 +547,16 @@ def run_tutorial( Stages: serve -> record -> compile -> certify -> run (standard profile, independent effect verification) -> receipt. - With ``break_it=True`` the SAME certified bundle is then rerun against a - backend that injects the :data:`TUTORIAL_BREAK_FAULT` fault -- the server - rejects the write after the application has already painted its success - banner -- and the engine is expected to HALT rather than believe the - screen. The rerun's evidence lands in ``/run-broken`` and on - :attr:`TutorialResult.break_it`. If the engine does NOT halt, this - function raises: an uncaught injected fault is a product failure, never a - tutorial variant. + With ``simulate_rejected_write=True`` the same certified bundle is then + rerun against a backend that injects the + :data:`TUTORIAL_BREAK_FAULT` fault. The server rejects the write after the + application has already painted its success banner, and the engine must + halt rather than believe the screen. The rerun's evidence lands in + ``/run-rejected-write`` and on :attr:`TutorialResult.break_it`. + + ``break_it=True`` preserves the deprecated Python API and its historical + ``/run-broken`` artifact path. Both modes enforce the same exact + rejected-write evidence contract. Raises: TutorialError: a stage produced insufficient evidence. Nothing here @@ -650,11 +672,14 @@ def run_tutorial( receipt = _build_tutorial_receipt(report) result.receipt_paths = write_receipt(receipt, run_dir) - if break_it: + if simulate_rejected_write or break_it: + fault_run_dir = root / ( + "run-rejected-write" if simulate_rejected_write else "run-broken" + ) result.break_it = _run_break_it( workflow=workflow, bundle_dir=bundle_dir, - run_dir=root / "run-broken", + run_dir=fault_run_dir, headed=headed, say=say, ) @@ -682,10 +707,12 @@ def _run_break_it( from openadapt_flow.report import render_run_report say("") - say("[break-it] Rerun the SAME certified bundle, but this time the backend") - say(f"[break-it] lies: fault mode {TUTORIAL_BREAK_FAULT!r} rejects the write") - say("[break-it] AFTER the app has painted its success banner.") - base_url, _db, stop = serve() + say("[rejected-write] Rerun the same certified bundle against the fault.") + say( + f"[rejected-write] Fault mode {TUTORIAL_BREAK_FAULT!r} rejects the " + "write after the app reports success." + ) + base_url, fault_db, stop = serve() try: report = run_tutorial_workflow( base_url=base_url, @@ -696,6 +723,7 @@ def _run_break_it( entry_query=TUTORIAL_BREAK_ENTRY_QUERY, ) record_count = len(_records(base_url)) + rejected_writes = int(fault_db.snapshot()["rejected_writes"]) finally: stop() report_path = render_run_report(run_dir) @@ -714,8 +742,7 @@ def _run_break_it( ) refuted = sum( 1 - for result in report.results - for evidence in result.effect_evidence + for evidence in (step_result.effect_evidence if step_result is not None else []) if evidence.final_verdict == "refuted" ) claim_text: Optional[str] = None @@ -725,8 +752,46 @@ def _run_break_it( None, ) envelope = report.outcome_envelope + effects_required = ( + int(envelope.required_contracts.effect) if envelope is not None else 0 + ) + screen_claimed_success = bool(step_result and step_result.postconditions_ok) + halt_reason = report.halt.reason if report.halt is not None else "" + evidence_failures: list[str] = [] + if report.transaction_outcome != "RECONCILIATION_REQUIRED": + evidence_failures.append( + "transaction outcome was not RECONCILIATION_REQUIRED" + ) + if report.transaction_billable is not False: + evidence_failures.append("transaction was not explicitly non-billable") + if step_result is None: + evidence_failures.append("the consequential step has no result") + elif not screen_claimed_success: + evidence_failures.append("the consequential screen postcondition did not pass") + if effects_required < 1: + evidence_failures.append("the run required no effect contract") + if refuted < 1: + evidence_failures.append("the consequential effect was not refuted") + if record_count != 0: + evidence_failures.append("the system of record was not empty") + if rejected_writes != 1: + evidence_failures.append( + f"the backend observed {rejected_writes} rejected writes instead of one" + ) + lowered_halt_reason = halt_reason.lower() + if ( + "record_written" not in lowered_halt_reason + or "refut" not in lowered_halt_reason + ): + evidence_failures.append("the halt reason did not name the refuted record write") + if evidence_failures: + raise TutorialError( + "the rejected-write verification FAILED its evidence contract: " + + "; ".join(evidence_failures) + ) + say( - f"[break-it] {report.execution_outcome}: the system of record holds " + f"[rejected-write] {report.execution_outcome}: the system of record holds " f"{record_count} record(s); the screen said otherwise." ) return BreakItResult( @@ -736,12 +801,11 @@ def _run_break_it( execution_outcome=str(report.execution_outcome), transaction_outcome=report.transaction_outcome, transaction_billable=report.transaction_billable, - screen_claimed_success=bool(step_result and step_result.postconditions_ok), + screen_claimed_success=screen_claimed_success, screen_claim_text=claim_text, - effects_required=( - int(envelope.required_contracts.effect) if envelope is not None else 0 - ), + effects_required=effects_required, effects_refuted=refuted, - halt_reason=(report.halt.reason if report.halt is not None else ""), + halt_reason=halt_reason, system_of_record_records=record_count, + rejected_writes=rejected_writes, ) diff --git a/openadapt_flow/visualize/builder.py b/openadapt_flow/visualize/builder.py index 5608d968..be45cdff 100644 --- a/openadapt_flow/visualize/builder.py +++ b/openadapt_flow/visualize/builder.py @@ -14,6 +14,7 @@ import re from typing import TYPE_CHECKING, Callable, Optional +from openadapt_flow.runtime.authorization import runtime_param_text from openadapt_flow.visualize.spec import ( BundleMeta, EdgeKind, @@ -293,7 +294,11 @@ def _bundle_meta( type=spec.type.value, required=spec.required, secret=name in (workflow.secret_params or []), - example=spec.example, + example=( + runtime_param_text(spec.example) + if spec.example is not None + else None + ), choices=list(spec.choices), ) ) diff --git a/public-artifacts.json b/public-artifacts.json index ab6c81d8..d54b0fbc 100644 --- a/public-artifacts.json +++ b/public-artifacts.json @@ -617,7 +617,7 @@ }, { "path": "claims.yaml", - "sha256": "6a20ce8ec0c242e7177f1ca5af174e3072a25e84cbb4d307da7729b240397687" + "sha256": "a27f060053d113d0c4ba63e54936e76771968cd55f24e033bb778619b57c7f46" }, { "path": "deploy/on-prem/docker-compose.yml", @@ -1825,7 +1825,7 @@ }, { "path": "docs/verification.json", - "sha256": "3a99ee1287e452279b36cf3529da4fae3c55399dfaa1bd29b056bb4d86819c04" + "sha256": "fb7ab966277345460887f3e06eaa10cc1aee68649bea26ea308635874db9cc8c" }, { "path": "openadapt_flow/console/static/console.css", diff --git a/pyproject.toml b/pyproject.toml index f1e5871e..faa60490 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "openadapt-flow" -version = "1.33.0" +version = "1.34.0" description = "Compile demonstrated GUI workflows into deterministic local replay with governed repair and refusal" readme = "README.md" license = { text = "MIT" } diff --git a/tests/e2e/test_tutorial_break_it_e2e.py b/tests/e2e/test_tutorial_break_it_e2e.py index f2d72611..333053ce 100644 --- a/tests/e2e/test_tutorial_break_it_e2e.py +++ b/tests/e2e/test_tutorial_break_it_e2e.py @@ -1,4 +1,4 @@ -"""``tutorial --break-it``, end to end: the engine catches the fault it names. +"""Rejected-write tutorial simulation, end to end. The clean free path is pinned by :mod:`test_free_path_e2e` (golden task). This module covers the OTHER half of the tutorial's demonstration: the SAME @@ -16,7 +16,8 @@ * the lie was real: the consequential step's on-screen postconditions all PASSED on the broken run -- the halt did not come from the screen; * the caught fault leaves a clearly-labeled LOCAL report - (``run-broken/REPORT.md`` leads with ``HALTED``) and NO shareable receipt, + (``run-rejected-write/REPORT.md`` leads with ``HALTED``) and NO shareable + receipt, and ``report-run`` still refuses the halted run -- the success rail was not weakened to make the fault showable. """ @@ -33,10 +34,13 @@ @pytest.fixture(scope="module") -def break_it_path(tmp_path_factory: pytest.TempPathFactory) -> TutorialResult: - """Run the tutorial once with ``break_it=True`` for the whole module.""" +def rejected_write_path(tmp_path_factory: pytest.TempPathFactory) -> TutorialResult: + """Run the advanced simulation once for the whole module.""" - return run_tutorial(tmp_path_factory.mktemp("break-it"), break_it=True) + return run_tutorial( + tmp_path_factory.mktemp("rejected-write"), + simulate_rejected_write=True, + ) def _report(run_dir: Path) -> RunReport: @@ -45,34 +49,37 @@ def _report(run_dir: Path) -> RunReport: ) -def test_the_clean_half_still_verifies(break_it_path: TutorialResult) -> None: - """--break-it prepends, never replaces, the clean VERIFIED run.""" +def test_the_clean_half_still_verifies(rejected_write_path: TutorialResult) -> None: + """The simulation follows and never replaces the clean VERIFIED run.""" - assert break_it_path.execution_outcome == "VERIFIED" - assert break_it_path.receipt_paths, "the clean half stopped emitting its receipt" - assert break_it_path.receipt_paths["json"].is_file() + assert rejected_write_path.execution_outcome == "VERIFIED" + assert rejected_write_path.receipt_paths, ( + "the clean half stopped emitting its receipt" + ) + assert rejected_write_path.receipt_paths["json"].is_file() def test_the_engine_halts_on_the_injected_fault( - break_it_path: TutorialResult, + rejected_write_path: TutorialResult, ) -> None: """The aha itself: same bundle, lying backend, HALT -- not success.""" - broken = break_it_path.break_it + broken = rejected_write_path.break_it assert broken is not None assert broken.fault == "optimistic" assert broken.execution_outcome == "HALTED" assert broken.transaction_outcome == "RECONCILIATION_REQUIRED" - assert broken.transaction_billable is not True + assert broken.transaction_billable is False # Nothing landed: the write the screen claimed does not exist. assert broken.system_of_record_records == 0 + assert broken.rejected_writes == 1 assert broken.effects_refuted >= 1 # The engine's own explanation names the refuted contract, not a guess. assert "record_written" in broken.halt_reason assert "refuted" in broken.halt_reason -def test_the_screen_really_did_lie(break_it_path: TutorialResult) -> None: +def test_the_screen_really_did_lie(rejected_write_path: TutorialResult) -> None: """The halt came from the independent verifier, not from the screen. If the on-screen postconditions had failed, the run would have halted for @@ -80,7 +87,7 @@ def test_the_screen_really_did_lie(break_it_path: TutorialResult) -> None: verification. The whole point is that the screen PASSED. """ - broken = break_it_path.break_it + broken = rejected_write_path.break_it assert broken is not None assert broken.screen_claimed_success is True @@ -98,11 +105,11 @@ def test_the_screen_really_did_lie(break_it_path: TutorialResult) -> None: def test_the_caught_fault_is_showable_but_not_a_success( - break_it_path: TutorialResult, tmp_path: Path + rejected_write_path: TutorialResult, tmp_path: Path ) -> None: """A clearly-labeled LOCAL report exists; the success rail does not bend.""" - broken = break_it_path.break_it + broken = rejected_write_path.break_it assert broken is not None # The local evidence: REPORT.md leads with the honest outcome. @@ -122,13 +129,13 @@ def test_the_caught_fault_is_showable_but_not_a_success( def test_the_broken_run_is_separate_from_the_clean_evidence( - break_it_path: TutorialResult, + rejected_write_path: TutorialResult, ) -> None: - """The halt lands in run-broken/ and never contaminates the clean run.""" + """The halt stays separate from the clean run evidence.""" - broken = break_it_path.break_it + broken = rejected_write_path.break_it assert broken is not None - assert broken.run_dir != break_it_path.run_dir - assert broken.run_dir.name == "run-broken" - clean = _report(break_it_path.run_dir) + assert broken.run_dir != rejected_write_path.run_dir + assert broken.run_dir.name == "run-rejected-write" + clean = _report(rejected_write_path.run_dir) assert clean.execution_outcome == "VERIFIED" diff --git a/tests/fixtures/terminal_verification_v2_terminal_vectors.json b/tests/fixtures/terminal_verification_v2_terminal_vectors.json new file mode 100644 index 00000000..5764f76e --- /dev/null +++ b/tests/fixtures/terminal_verification_v2_terminal_vectors.json @@ -0,0 +1 @@ +{"key_id":"evidence-runner-ed25519-56475aa75463474c","private_key_base64":"AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8=","public_key_base64":"A6EHv/POEL4dcN0Y50vAmWfk1jCbpQ1fHdyGZBJVMbg=","schema_version":"openadapt.production-terminal-cross-language-vectors/v2","signature_domain_base64":"b3BlbmFkYXB0LXByb2R1Y3Rpb24tdGVybWluYWwtdmVyaWZpY2F0aW9uLXYyAA==","signer_sha256":"56475aa75463474c0285df5dbf2bcab73da651358839e9b77481b2eab107708c","vectors":[{"callback":{"artifact_bytes_source":"envelope_canonical_base64","outcome":"HALTED_BEFORE_EFFECT","report_sha256":"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb","run_id":"00000000-0000-4000-8000-000000000001","schema_version":"openadapt.hosted-runner-terminal/v1","started":true,"uncertain_delivery":false},"effect_state":{"absence_basis":"not_actuated","attempt_state":"not_actuated","effect_contract_hash":"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","effect_verified":false,"final_verdict":null,"observed_effect":"absent","reconciliation_actions":0,"reconciliation_completed":false,"record_kind":"terminal_effect_state","resolved_delivery_uncertainty":false,"result_index":0,"verification_performed":false,"verification_tier":null,"verifier_identity":null},"envelope_canonical_base64":"eyJwYXlsb2FkIjp7ImFkbWlzc2lvbl9hcnRpZmFjdF9zaGEyNTYiOiJkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkIiwiYWRtaXNzaW9uX2lkIjoiMDAwMDAwMDAtMDAwMC00MDAwLTgwMDAtMDAwMDAwMDAwMDA2IiwiYWRtaXNzaW9uX3BvbGljeV9zaGEyNTYiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIiwiYWRtaXR0ZWRfcnVudGltZV9idWlsZF9zaGEyNTYiOiJjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjIiwiYnVuZGxlX2FydGlmYWN0X3NoYTI1NiI6ImJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmIiLCJidW5kbGVfY29udGVudF9kaWdlc3QiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIiwiYnVuZGxlX3ZlcnNpb25faWQiOiIwMDAwMDAwMC0wMDAwLTQwMDAtODAwMC0wMDAwMDAwMDAwMDQiLCJlZmZlY3RfY29udHJhY3Rfc2hhMjU2IjoiZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZSIsImVudmlyb25tZW50X2NvbnRyYWN0X3NoYTI1NiI6ImJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmIiLCJlbnZpcm9ubWVudF9kaWdlc3QiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIiwiZXZpZGVuY2VfaWRlbnRpdHlfc2hhMjU2IjoiZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZSIsImV2aWRlbmNlX21hbmlmZXN0cyI6eyJhdXRob3JpemF0aW9uIjp7ImFkbWlzc2lvbl9hcnRpZmFjdF9zaGEyNTYiOiJkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkIiwiYWRtaXNzaW9uX2lkIjoiMDAwMDAwMDAtMDAwMC00MDAwLTgwMDAtMDAwMDAwMDAwMDA2IiwiZXhlY3V0aW9uX2F1dGhvcml0eV9pZCI6IjAwMDAwMDAwLTAwMDAtNDAwMC04MDAwLTAwMDAwMDAwMDAwOCIsImV4ZWN1dGlvbl9hdXRob3JpdHlfc2hhMjU2IjoiYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYSIsImdvdmVybmVkX2F1dGhvcml6YXRpb25faWRfc2hhMjU2IjoiYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYSIsIm1hbmlmZXN0X3NoYTI1NiI6IjMyY2QzNjEzYmVlOWJjNjkwNjkzYTM5NWQ5OTMzNTQ3ZjllZWI5ZjZmYWE1ZGJkYmIxNGExMDYyOThiZTQ2NWMiLCJwZXJtaXRfY2hhaW5fc2hhMjU2IjoiODJkYWI1NDI4ZTI1MmQxOGE1OWIwMTc3NzM5ZTAwY2FhZGI3NjJkY2Y1OWY0MDM2ZGVhNGQ3N2MzYzMyMDNjYiIsInNjaGVtYV92ZXJzaW9uIjoib3BlbmFkYXB0LnByb2R1Y3Rpb24tYXV0aG9yaXphdGlvbi1ldmlkZW5jZS92MSJ9LCJlZmZlY3QiOnsiY29uZmlybWVkIjowLCJlZmZlY3RfY29udHJhY3Rfc2hhMjU2IjoiZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZSIsIm1hbmlmZXN0X3NoYTI1NiI6ImViYjU4MzYxNzhlZGZhYzYyOWJkOWJjYTkwZWNkMDE5NTkwOTQ5NjZhMGRkNjNiYWVhN2E2MmY2ZTNiMDRmZjgiLCJyZWNvcmRzIjpbeyJhYnNlbmNlX2Jhc2lzIjoibm90X2FjdHVhdGVkIiwiYXR0ZW1wdF9zdGF0ZSI6Im5vdF9hY3R1YXRlZCIsImVmZmVjdF9jb250cmFjdF9oYXNoIjoic2hhMjU2OmFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEiLCJlZmZlY3RfdmVyaWZpZWQiOmZhbHNlLCJmaW5hbF92ZXJkaWN0IjpudWxsLCJvYnNlcnZlZF9lZmZlY3QiOiJhYnNlbnQiLCJyZWNvbmNpbGlhdGlvbl9hY3Rpb25zIjowLCJyZWNvbmNpbGlhdGlvbl9jb21wbGV0ZWQiOmZhbHNlLCJyZWNvcmRfa2luZCI6InRlcm1pbmFsX2VmZmVjdF9zdGF0ZSIsInJlc29sdmVkX2RlbGl2ZXJ5X3VuY2VydGFpbnR5IjpmYWxzZSwicmVzdWx0X2luZGV4IjowLCJ2ZXJpZmljYXRpb25fcGVyZm9ybWVkIjpmYWxzZSwidmVyaWZpY2F0aW9uX3RpZXIiOm51bGwsInZlcmlmaWVyX2lkZW50aXR5IjpudWxsfV0sInJlcXVpcmVkIjoxLCJzY2hlbWFfdmVyc2lvbiI6Im9wZW5hZGFwdC5wcm9kdWN0aW9uLWVmZmVjdC1ldmlkZW5jZS92MSIsIndvcmtmbG93X2NvbnRyYWN0X3NoYTI1NiI6ImFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEifSwiaWRlbnRpdHkiOnsiY29uZmlybWVkIjoxLCJpZGVudGl0eV9jb250cmFjdF9zaGEyNTYiOiJkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkIiwibWFuaWZlc3Rfc2hhMjU2IjoiMTM1Yzg2M2VkYzdmYWIxOGNhMjliNDkyOTNmYzI1YTFjOWJkNGRlMGU5ZWQyOGQ0Y2VjNmU4ZDBhNjc5MTIyOSIsInJlcXVpcmVkIjoxLCJyZXN1bHRzIjpbeyJtb2RlIjoic3RydWN0dXJlZCIsInJlc3VsdF9pbmRleCI6MCwic2lnbmFscyI6W10sInN0YXR1cyI6InZlcmlmaWVkIn1dLCJzY2hlbWFfdmVyc2lvbiI6Im9wZW5hZGFwdC5wcm9kdWN0aW9uLWlkZW50aXR5LWV2aWRlbmNlL3YxIiwid29ya2Zsb3dfY29udHJhY3Rfc2hhMjU2IjoiYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYSJ9LCJwb2xpY3kiOnsiYWRtaXNzaW9uX3BvbGljeV9zaGEyNTYiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIiwiZWZmZWN0X2NvbnRyYWN0X3NoYTI1NiI6ImVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWUiLCJlbnZpcm9ubWVudF9jb250cmFjdF9zaGEyNTYiOiJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiIiwiZW52aXJvbm1lbnRfZGlnZXN0IjoiYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYSIsImdvdmVybmVkX3BvbGljeV9jb250cmFjdF9zaGEyNTYiOiJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiIiwiZ292ZXJuZWRfcnVudGltZV9pbnB1dHNfZGlnZXN0IjoiY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjYyIsImlkZW50aXR5X2NvbnRyYWN0X3NoYTI1NiI6ImRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGQiLCJtYW5pZmVzdF9zaGEyNTYiOiI0Zjk4OGQwMTViY2Q3Y2Q3ZTA1YmEwNjYzZDg5ZjBhOTY4ZjBkOGFjZTk5YjA5MGViYWJlZTUyYTFjNTUwNDIxIiwibWluaW11bV9lZmZlY3RfdGllciI6MSwicnVudGltZV9lbnZpcm9ubWVudF9zaGEyNTYiOiJjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjIiwic2NoZW1hX3ZlcnNpb24iOiJvcGVuYWRhcHQucHJvZHVjdGlvbi1wb2xpY3ktZXZpZGVuY2UvdjEifSwicG9zdGNvbmRpdGlvbiI6eyJjb25maXJtZWQiOjAsIm1hbmlmZXN0X3NoYTI1NiI6IjNlYTRjNjRhMzBiMGNmMTdiY2EwNGYzMjYwZTBhNDcxOGIyN2Y1MWUxZmRjMmJiODhhMTgyNjMwYTM0ZGQ1ZmYiLCJyZWNvcmRzIjpbeyJhY3Rpb25fa2luZCI6ImNsaWNrIiwiYWN0dWF0aW9uX3BhdGgiOiJndWkiLCJjb250cmFjdF9pbmRleCI6MCwiY29udHJhY3Rfa2luZCI6ImV4cGxpY2l0X3ByZWRpY2F0ZSIsImNvbnRyYWN0X3NoYTI1NiI6IjBhZGUwYjUyMGJjMDA3ODkxY2FjY2EwMDE3MTU3ODQ0MmEyYTc0Y2VmMTVmZjY0ZGRlNDVkZjA3YWZiNjRhZWIiLCJyZXN1bHRfaW5kZXgiOjAsInN0ZXBfY29udHJhY3Rfc2hhMjU2IjoiMWE1OTUzNmE0ODhiMWM0NmFhY2YwYmEzNzkzMGI4NDA2MmMxNDI3NTVmYTkzOTE1MmVmNWUyOGM0MDk0NzFlZCIsInN0ZXBfaW5kZXgiOjAsInZlcmRpY3QiOiJ1bnZlcmlmaWFibGUiLCJ3b3JrZmxvd19jb250cmFjdF9zaGEyNTYiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIn1dLCJyZXF1aXJlZCI6MSwic2NoZW1hX3ZlcnNpb24iOiJvcGVuYWRhcHQucHJvZHVjdGlvbi1wb3N0Y29uZGl0aW9uLWV2aWRlbmNlL3YxIiwid29ya2Zsb3dfY29udHJhY3Rfc2hhMjU2IjoiYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYSJ9fSwiZXZpZGVuY2VfcnVubmVyX3NpZ25lcl9zaGEyNTYiOiI1NjQ3NWFhNzU0NjM0NzRjMDI4NWRmNWRiZjJiY2FiNzNkYTY1MTM1ODgzOWU5Yjc3NDgxYjJlYWIxMDc3MDhjIiwiZXhlY3V0aW9uX2F1dGhvcml0eV9pZCI6IjAwMDAwMDAwLTAwMDAtNDAwMC04MDAwLTAwMDAwMDAwMDAwOCIsImV4ZWN1dGlvbl9hdXRob3JpdHlfc2hhMjU2IjoiYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYSIsImV4ZWN1dGlvbl9hdXRob3JpdHlfc2lnbmVyX3NoYTI1NiI6ImNjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2MiLCJleGVjdXRpb25fb3V0Y29tZSI6eyJjb21wZW5zYXRpb25fYWN0aW9ucyI6MCwiZXZpZGVuY2VfY2xhc3NlcyI6WyJhdXRob3JpemF0aW9uIiwiaWRlbnRpdHkiXSwiZXhlY3V0aW9uX2NvbXBsZXRlZCI6ZmFsc2UsImV4dGVybmFsX25ldHdvcmtfY2FsbHMiOiJub25lIiwibW9kZWxfY2FsbHMiOjAsIm91dGNvbWUiOiJIQUxURUQiLCJwYXNzZWRfY29udHJhY3RzIjp7ImF1dGhvcml6YXRpb24iOjEsImVmZmVjdCI6MCwiaWRlbnRpdHkiOjEsInBvc3Rjb25kaXRpb24iOjB9LCJwb3N0Y29uZGl0aW9uX2V2aWRlbmNlIjpbeyJhY3Rpb25fa2luZCI6ImNsaWNrIiwiYWN0dWF0aW9uX3BhdGgiOiJndWkiLCJjb250cmFjdF9pbmRleCI6MCwiY29udHJhY3Rfa2luZCI6ImV4cGxpY2l0X3ByZWRpY2F0ZSIsImNvbnRyYWN0X3NoYTI1NiI6IjBhZGUwYjUyMGJjMDA3ODkxY2FjY2EwMDE3MTU3ODQ0MmEyYTc0Y2VmMTVmZjY0ZGRlNDVkZjA3YWZiNjRhZWIiLCJyZXN1bHRfaW5kZXgiOjAsInN0ZXBfY29udHJhY3Rfc2hhMjU2IjoiMWE1OTUzNmE0ODhiMWM0NmFhY2YwYmEzNzkzMGI4NDA2MmMxNDI3NTVmYTkzOTE1MmVmNWUyOGM0MDk0NzFlZCIsInN0ZXBfaW5kZXgiOjAsInZlcmRpY3QiOiJ1bnZlcmlmaWFibGUiLCJ3b3JrZmxvd19jb250cmFjdF9zaGEyNTYiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIn1dLCJwcm9kdWN0aW9uX2VsaWdpYmxlIjpmYWxzZSwicHJvZmlsZSI6InN0YW5kYXJkIiwicXVhbGlmaWNhdGlvbl9ldmlkZW5jZV9vbmx5IjpmYWxzZSwicmVxdWlyZWRfY29udHJhY3RzIjp7ImF1dGhvcml6YXRpb24iOjEsImVmZmVjdCI6MSwiaWRlbnRpdHkiOjEsInBvc3Rjb25kaXRpb24iOjF9LCJ2ZXJzaW9uIjoib3BlbmFkYXB0LmV4ZWN1dGlvbi1vdXRjb21lL3YxIiwid29ya2Zsb3dfY29udHJhY3Rfc2hhMjU2IjoiYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYSJ9LCJleGVjdXRpb25fb3V0Y29tZV9zaGEyNTYiOiJmZTM4OWJhYjBlYjQ0NDdjZWMxOTk4Mjc0ZjcyN2NhNTcwMDQ5ZmVmZDU1N2RiNzk5MDQ3Njk5ZjJmNzhmMTQyIiwiZXhlY3V0aW9uX3B1cnBvc2UiOiJwcm9kdWN0aW9uIiwiZmluYWxfYXV0aG9yaXR5X3NlcXVlbmNlIjowLCJmaW5hbF9ydW50aW1lX2RlbGl2ZXJ5X3NlcXVlbmNlIjowLCJmbG93X3J1bl9pZF9zaGEyNTYiOiIxMWU1OTRmNDgxOTU4YzEwZTMwMTVkMGJmMDQ0N2EyMmYwNjhhOGE2NDdmNDc1ZGYxNWNlMmM3YWI0YjhmM2YxIiwiaWRlbnRpdHlfY29udHJhY3Rfc2hhMjU2IjoiZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZCIsImlzc3VlZF9hdCI6IjIwMjYtMDgtMThUMTI6MDA6MDNaIiwicGVybWl0X2NoYWluIjp7ImVudHJpZXMiOltdLCJwZXJtaXRfY2hhaW5fc2hhMjU2IjoiODJkYWI1NDI4ZTI1MmQxOGE1OWIwMTc3NzM5ZTAwY2FhZGI3NjJkY2Y1OWY0MDM2ZGVhNGQ3N2MzYzMyMDNjYiIsInNjaGVtYV92ZXJzaW9uIjoib3BlbmFkYXB0LnByb2R1Y3Rpb24tZGVsaXZlcnktcGVybWl0LWNoYWluL3YyIn0sInBlcm1pdF9jb3VudCI6MCwicXVhbGlmaWNhdGlvbl9zaWduZXJfcmVnaXN0cnlfcmV2aXNpb24iOjcsInF1YWxpZmljYXRpb25fc2lnbmVyX3JlZ2lzdHJ5X3NoYTI1NiI6ImVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWUiLCJydW5faWQiOiIwMDAwMDAwMC0wMDAwLTQwMDAtODAwMC0wMDAwMDAwMDAwMDEiLCJydW5fcmVjZWlwdCI6eyJhdXRob3JpemF0aW9uX2NvbmZpcm1lZCI6MSwiYXV0aG9yaXphdGlvbl9yZXF1aXJlZCI6MSwiYnVuZGxlX2RpZ2VzdCI6ImFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEiLCJkdXJhdGlvbl9tcyI6MTAwLCJlZmZlY3RfdGllcl9yZWFjaGVkIjoibm9uZSIsImVmZmVjdHNfY29uZmlybWVkIjowLCJlZmZlY3RzX3JlcXVpcmVkIjoxLCJlc3RfY29zdF9taWNyb3VzZCI6MCwiZXZpZGVuY2VfY2xhc3NlcyI6WyJhdXRob3JpemF0aW9uIiwiaWRlbnRpdHkiXSwiZXh0ZXJuYWxfbmV0d29ya19jYWxscyI6Im5vbmUiLCJnZW5lcmF0ZWRfYXQiOiIyMDI2LTA4LTE4VDEyOjAwOjAwWiIsImhlYWxzIjowLCJpZGVudGl0eV9hcHBsaWNhYmxlIjoxLCJpZGVudGl0eV9hcm1lZCI6MSwiaWRlbnRpdHlfY29uZmlybWVkIjoxLCJpZGVudGl0eV9yZXF1aXJlZCI6MSwibW9kZWxfY2FsbHMiOjAsIm91dGNvbWUiOiJIQUxURUQiLCJvdmVyX2hhbHRfY291bnQiOjAsInBvc3Rjb25kaXRpb25zX2NvbmZpcm1lZCI6MCwicG9zdGNvbmRpdGlvbnNfcmVxdWlyZWQiOjEsInByb2R1Y3Rpb25fZWxpZ2libGUiOmZhbHNlLCJwcm9maWxlIjoic3RhbmRhcmQiLCJwcm92ZW5hbmNlIjoicHJvZHVjdGlvbiIsInJlY2VpcHRfYnVpbGRlcl92ZXJzaW9uIjoiMS4yLjMiLCJydW5nX2hpc3RvZ3JhbSI6e30sInNjaGVtYV92ZXJzaW9uIjoib3BlbmFkYXB0LnByb2R1Y3Rpb24tcnVuLXJlY2VpcHQvdjEiLCJzb3VyY2VfcmVjZWlwdF9kaWdlc3QiOiJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiIiwic291cmNlX3JlY2VpcHRfc2hhMjU2IjoiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYiIsInNvdXJjZV9zY2hlbWFfdmVyc2lvbiI6Im9wZW5hZGFwdC5ydW4tcmVwb3J0L3YxIiwic3RlcHNfb2siOjAsInN0ZXBzX3RvdGFsIjoxLCJzdWJzdHJhdGUiOiJ3ZWIiLCJ0cmFuc2FjdGlvbl9vdXRjb21lIjoiSEFMVEVEX0JFRk9SRV9FRkZFQ1QifSwicnVuX3JlY2VpcHRfc2hhMjU2IjoiYWI1MGEwMDZhODg4YjNmOTBlMWU3MWVmMjA5MjFhZjcxN2IyZjQxYTM2NzZlNjA1ZmJkYjgyMGNiNmI1M2NjNCIsInJ1bl9yZXBvcnRfb2JqZWN0X3NoYTI1NiI6ImJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmIiLCJydW5fcmVwb3J0X29iamVjdF92ZXJzaW9uIjoidmVyc2lvbjpoYWx0ZWQ6MSIsInJ1bl9yZXBvcnRfc2hhMjU2IjoiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYiIsInJ1bnRpbWVfZW52aXJvbm1lbnRfc2hhMjU2IjoiY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjYyIsInJ1bnRpbWVfc3Vic3RyYXRlIjoid2ViIiwicnVudGltZV92YWxpZGF0aW9uX2lkIjoiMDAwMDAwMDAtMDAwMC00MDAwLTgwMDAtMDAwMDAwMDAwMDA1Iiwic2NoZW1hX3ZlcnNpb24iOiJvcGVuYWRhcHQucHJvZHVjdGlvbi10ZXJtaW5hbC12ZXJpZmljYXRpb24vdjIiLCJ0ZW5hbnRfaWQiOiIwMDAwMDAwMC0wMDAwLTQwMDAtODAwMC0wMDAwMDAwMDAwMDIiLCJ0ZXJtaW5hbF9zZXF1ZW5jZSI6MSwidmVyaWZpZWRfYXQiOiIyMDI2LTA4LTE4VDEyOjAwOjAyWiIsIndvcmtmbG93X2NvbnRyYWN0X3NoYTI1NiI6ImFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEiLCJ3b3JrZmxvd19pZCI6IjAwMDAwMDAwLTAwMDAtNDAwMC04MDAwLTAwMDAwMDAwMDAwMyIsIndvcmtmbG93X3ZlcnNpb25faWQiOiIwMDAwMDAwMC0wMDAwLTQwMDAtODAwMC0wMDAwMDAwMDAwMDQifSwic2lnbmF0dXJlIjoiNUptaHdlQXN0OGZ0eUwtbjd4a1JLNTE0cEJCbkU1TzlobUZ3RjRDbmtKOEtpVHBKeExFQWs2ZHNzTUhfTkVSM2QzbVFOeENBQkNHckRkY29tSkE5Q1EiLCJzaWduZXIiOnsiYWxnb3JpdGhtIjoiZWQyNTUxOSIsImtleV9pZCI6ImV2aWRlbmNlLXJ1bm5lci1lZDI1NTE5LTU2NDc1YWE3NTQ2MzQ3NGMiLCJwdWJsaWNfa2V5IjoiQTZFSHYvUE9FTDRkY04wWTUwdkFtV2ZrMWpDYnBRMWZIZHlHWkJKVk1iZz0ifX0=","name":"halted-before-effect-zero-permit","payload_canonical_sha256":"26f4073246b4998650636bd3f1cef10835c7f51d91ccfa1d0854bc67015e6792","signature":"5JmhweAst8ftyL-n7xkRK514pBBnE5O9hmFwF4CnkJ8KiTpJxLEAk6dssMH_NER3d3mQNxCABCGrDdcomJA9CQ","terminal_verification_artifact_sha256":"43f6fc7809d789aa42785b7f76ce2dd751b2c3251c8fc74ec1f7bfe3a70b6577"},{"callback":{"artifact_bytes_source":"envelope_canonical_base64","outcome":"RECONCILIATION_REQUIRED","report_sha256":"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb","run_id":"00000000-0000-4000-8000-000000000001","schema_version":"openadapt.hosted-runner-terminal/v1","started":true,"uncertain_delivery":true},"effect_state":{"absence_basis":"none","attempt_state":"delivery_uncertain","effect_contract_hash":"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","effect_verified":false,"final_verdict":null,"observed_effect":"unknown","reconciliation_actions":0,"reconciliation_completed":false,"record_kind":"terminal_effect_state","resolved_delivery_uncertainty":false,"result_index":0,"verification_performed":false,"verification_tier":null,"verifier_identity":null},"envelope_canonical_base64":"eyJwYXlsb2FkIjp7ImFkbWlzc2lvbl9hcnRpZmFjdF9zaGEyNTYiOiJkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkIiwiYWRtaXNzaW9uX2lkIjoiMDAwMDAwMDAtMDAwMC00MDAwLTgwMDAtMDAwMDAwMDAwMDA2IiwiYWRtaXNzaW9uX3BvbGljeV9zaGEyNTYiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIiwiYWRtaXR0ZWRfcnVudGltZV9idWlsZF9zaGEyNTYiOiJjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjIiwiYnVuZGxlX2FydGlmYWN0X3NoYTI1NiI6ImJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmIiLCJidW5kbGVfY29udGVudF9kaWdlc3QiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIiwiYnVuZGxlX3ZlcnNpb25faWQiOiIwMDAwMDAwMC0wMDAwLTQwMDAtODAwMC0wMDAwMDAwMDAwMDQiLCJlZmZlY3RfY29udHJhY3Rfc2hhMjU2IjoiZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZSIsImVudmlyb25tZW50X2NvbnRyYWN0X3NoYTI1NiI6ImJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmIiLCJlbnZpcm9ubWVudF9kaWdlc3QiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIiwiZXZpZGVuY2VfaWRlbnRpdHlfc2hhMjU2IjoiZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZSIsImV2aWRlbmNlX21hbmlmZXN0cyI6eyJhdXRob3JpemF0aW9uIjp7ImFkbWlzc2lvbl9hcnRpZmFjdF9zaGEyNTYiOiJkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkIiwiYWRtaXNzaW9uX2lkIjoiMDAwMDAwMDAtMDAwMC00MDAwLTgwMDAtMDAwMDAwMDAwMDA2IiwiZXhlY3V0aW9uX2F1dGhvcml0eV9pZCI6IjAwMDAwMDAwLTAwMDAtNDAwMC04MDAwLTAwMDAwMDAwMDAwOCIsImV4ZWN1dGlvbl9hdXRob3JpdHlfc2hhMjU2IjoiYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYSIsImdvdmVybmVkX2F1dGhvcml6YXRpb25faWRfc2hhMjU2IjoiYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYSIsIm1hbmlmZXN0X3NoYTI1NiI6ImYwY2ExOWE2YjNlYWJlNTk0ZGJjOTA2ODljMmY3NGNiZTVlODJkNGQ5NDdkYjI5YTVkMDdiMTY0MWU4OTYxY2YiLCJwZXJtaXRfY2hhaW5fc2hhMjU2IjoiNTA0OGZlOTQ1ODgyZDI4MDI2MDA4ZDdlOTEzN2FkMDE2ZmRkMDFhZWFmNTg4NDgxZmI1YmQwZjI0MmE1MDA0MyIsInNjaGVtYV92ZXJzaW9uIjoib3BlbmFkYXB0LnByb2R1Y3Rpb24tYXV0aG9yaXphdGlvbi1ldmlkZW5jZS92MSJ9LCJlZmZlY3QiOnsiY29uZmlybWVkIjowLCJlZmZlY3RfY29udHJhY3Rfc2hhMjU2IjoiZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZSIsIm1hbmlmZXN0X3NoYTI1NiI6ImFmMTUwOWNjMmU5ZmExZjViOGI4MTVlN2IxZjYwNDYxOWIyYTM1ZWMwZGU1ZTBkMzRiYmZhNjdjYzkxZGJiYzciLCJyZWNvcmRzIjpbeyJhYnNlbmNlX2Jhc2lzIjoibm9uZSIsImF0dGVtcHRfc3RhdGUiOiJkZWxpdmVyeV91bmNlcnRhaW4iLCJlZmZlY3RfY29udHJhY3RfaGFzaCI6InNoYTI1NjphYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIiwiZWZmZWN0X3ZlcmlmaWVkIjpmYWxzZSwiZmluYWxfdmVyZGljdCI6bnVsbCwib2JzZXJ2ZWRfZWZmZWN0IjoidW5rbm93biIsInJlY29uY2lsaWF0aW9uX2FjdGlvbnMiOjAsInJlY29uY2lsaWF0aW9uX2NvbXBsZXRlZCI6ZmFsc2UsInJlY29yZF9raW5kIjoidGVybWluYWxfZWZmZWN0X3N0YXRlIiwicmVzb2x2ZWRfZGVsaXZlcnlfdW5jZXJ0YWludHkiOmZhbHNlLCJyZXN1bHRfaW5kZXgiOjAsInZlcmlmaWNhdGlvbl9wZXJmb3JtZWQiOmZhbHNlLCJ2ZXJpZmljYXRpb25fdGllciI6bnVsbCwidmVyaWZpZXJfaWRlbnRpdHkiOm51bGx9XSwicmVxdWlyZWQiOjEsInNjaGVtYV92ZXJzaW9uIjoib3BlbmFkYXB0LnByb2R1Y3Rpb24tZWZmZWN0LWV2aWRlbmNlL3YxIiwid29ya2Zsb3dfY29udHJhY3Rfc2hhMjU2IjoiYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYSJ9LCJpZGVudGl0eSI6eyJjb25maXJtZWQiOjEsImlkZW50aXR5X2NvbnRyYWN0X3NoYTI1NiI6ImRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGQiLCJtYW5pZmVzdF9zaGEyNTYiOiIxMzVjODYzZWRjN2ZhYjE4Y2EyOWI0OTI5M2ZjMjVhMWM5YmQ0ZGUwZTllZDI4ZDRjZWM2ZThkMGE2NzkxMjI5IiwicmVxdWlyZWQiOjEsInJlc3VsdHMiOlt7Im1vZGUiOiJzdHJ1Y3R1cmVkIiwicmVzdWx0X2luZGV4IjowLCJzaWduYWxzIjpbXSwic3RhdHVzIjoidmVyaWZpZWQifV0sInNjaGVtYV92ZXJzaW9uIjoib3BlbmFkYXB0LnByb2R1Y3Rpb24taWRlbnRpdHktZXZpZGVuY2UvdjEiLCJ3b3JrZmxvd19jb250cmFjdF9zaGEyNTYiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIn0sInBvbGljeSI6eyJhZG1pc3Npb25fcG9saWN5X3NoYTI1NiI6ImFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEiLCJlZmZlY3RfY29udHJhY3Rfc2hhMjU2IjoiZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZSIsImVudmlyb25tZW50X2NvbnRyYWN0X3NoYTI1NiI6ImJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmIiLCJlbnZpcm9ubWVudF9kaWdlc3QiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIiwiZ292ZXJuZWRfcG9saWN5X2NvbnRyYWN0X3NoYTI1NiI6ImJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmIiLCJnb3Zlcm5lZF9ydW50aW1lX2lucHV0c19kaWdlc3QiOiJjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjIiwiaWRlbnRpdHlfY29udHJhY3Rfc2hhMjU2IjoiZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZCIsIm1hbmlmZXN0X3NoYTI1NiI6IjRmOTg4ZDAxNWJjZDdjZDdlMDViYTA2NjNkODlmMGE5NjhmMGQ4YWNlOTliMDkwZWJhYmVlNTJhMWM1NTA0MjEiLCJtaW5pbXVtX2VmZmVjdF90aWVyIjoxLCJydW50aW1lX2Vudmlyb25tZW50X3NoYTI1NiI6ImNjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2MiLCJzY2hlbWFfdmVyc2lvbiI6Im9wZW5hZGFwdC5wcm9kdWN0aW9uLXBvbGljeS1ldmlkZW5jZS92MSJ9LCJwb3N0Y29uZGl0aW9uIjp7ImNvbmZpcm1lZCI6MCwibWFuaWZlc3Rfc2hhMjU2IjoiM2VhNGM2NGEzMGIwY2YxN2JjYTA0ZjMyNjBlMGE0NzE4YjI3ZjUxZTFmZGMyYmI4OGExODI2MzBhMzRkZDVmZiIsInJlY29yZHMiOlt7ImFjdGlvbl9raW5kIjoiY2xpY2siLCJhY3R1YXRpb25fcGF0aCI6Imd1aSIsImNvbnRyYWN0X2luZGV4IjowLCJjb250cmFjdF9raW5kIjoiZXhwbGljaXRfcHJlZGljYXRlIiwiY29udHJhY3Rfc2hhMjU2IjoiMGFkZTBiNTIwYmMwMDc4OTFjYWNjYTAwMTcxNTc4NDQyYTJhNzRjZWYxNWZmNjRkZGU0NWRmMDdhZmI2NGFlYiIsInJlc3VsdF9pbmRleCI6MCwic3RlcF9jb250cmFjdF9zaGEyNTYiOiIxYTU5NTM2YTQ4OGIxYzQ2YWFjZjBiYTM3OTMwYjg0MDYyYzE0Mjc1NWZhOTM5MTUyZWY1ZTI4YzQwOTQ3MWVkIiwic3RlcF9pbmRleCI6MCwidmVyZGljdCI6InVudmVyaWZpYWJsZSIsIndvcmtmbG93X2NvbnRyYWN0X3NoYTI1NiI6ImFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEifV0sInJlcXVpcmVkIjoxLCJzY2hlbWFfdmVyc2lvbiI6Im9wZW5hZGFwdC5wcm9kdWN0aW9uLXBvc3Rjb25kaXRpb24tZXZpZGVuY2UvdjEiLCJ3b3JrZmxvd19jb250cmFjdF9zaGEyNTYiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIn19LCJldmlkZW5jZV9ydW5uZXJfc2lnbmVyX3NoYTI1NiI6IjU2NDc1YWE3NTQ2MzQ3NGMwMjg1ZGY1ZGJmMmJjYWI3M2RhNjUxMzU4ODM5ZTliNzc0ODFiMmVhYjEwNzcwOGMiLCJleGVjdXRpb25fYXV0aG9yaXR5X2lkIjoiMDAwMDAwMDAtMDAwMC00MDAwLTgwMDAtMDAwMDAwMDAwMDA4IiwiZXhlY3V0aW9uX2F1dGhvcml0eV9zaGEyNTYiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIiwiZXhlY3V0aW9uX2F1dGhvcml0eV9zaWduZXJfc2hhMjU2IjoiMjRmNmVkNmFjYmZlMTAwOWMwMzBkN2NhNTY3YzMzY2E0ODMwOTExNDk4MjM2YjU1NjFhNmM4MmFiZWM1ZGUyOCIsImV4ZWN1dGlvbl9vdXRjb21lIjp7ImNvbXBlbnNhdGlvbl9hY3Rpb25zIjowLCJldmlkZW5jZV9jbGFzc2VzIjpbImF1dGhvcml6YXRpb24iLCJpZGVudGl0eSJdLCJleGVjdXRpb25fY29tcGxldGVkIjpmYWxzZSwiZXh0ZXJuYWxfbmV0d29ya19jYWxscyI6Im5vbmUiLCJtb2RlbF9jYWxscyI6MCwib3V0Y29tZSI6IkhBTFRFRCIsInBhc3NlZF9jb250cmFjdHMiOnsiYXV0aG9yaXphdGlvbiI6MSwiZWZmZWN0IjowLCJpZGVudGl0eSI6MSwicG9zdGNvbmRpdGlvbiI6MH0sInBvc3Rjb25kaXRpb25fZXZpZGVuY2UiOlt7ImFjdGlvbl9raW5kIjoiY2xpY2siLCJhY3R1YXRpb25fcGF0aCI6Imd1aSIsImNvbnRyYWN0X2luZGV4IjowLCJjb250cmFjdF9raW5kIjoiZXhwbGljaXRfcHJlZGljYXRlIiwiY29udHJhY3Rfc2hhMjU2IjoiMGFkZTBiNTIwYmMwMDc4OTFjYWNjYTAwMTcxNTc4NDQyYTJhNzRjZWYxNWZmNjRkZGU0NWRmMDdhZmI2NGFlYiIsInJlc3VsdF9pbmRleCI6MCwic3RlcF9jb250cmFjdF9zaGEyNTYiOiIxYTU5NTM2YTQ4OGIxYzQ2YWFjZjBiYTM3OTMwYjg0MDYyYzE0Mjc1NWZhOTM5MTUyZWY1ZTI4YzQwOTQ3MWVkIiwic3RlcF9pbmRleCI6MCwidmVyZGljdCI6InVudmVyaWZpYWJsZSIsIndvcmtmbG93X2NvbnRyYWN0X3NoYTI1NiI6ImFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEifV0sInByb2R1Y3Rpb25fZWxpZ2libGUiOmZhbHNlLCJwcm9maWxlIjoic3RhbmRhcmQiLCJxdWFsaWZpY2F0aW9uX2V2aWRlbmNlX29ubHkiOmZhbHNlLCJyZXF1aXJlZF9jb250cmFjdHMiOnsiYXV0aG9yaXphdGlvbiI6MSwiZWZmZWN0IjoxLCJpZGVudGl0eSI6MSwicG9zdGNvbmRpdGlvbiI6MX0sInZlcnNpb24iOiJvcGVuYWRhcHQuZXhlY3V0aW9uLW91dGNvbWUvdjEiLCJ3b3JrZmxvd19jb250cmFjdF9zaGEyNTYiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIn0sImV4ZWN1dGlvbl9vdXRjb21lX3NoYTI1NiI6ImZlMzg5YmFiMGViNDQ0N2NlYzE5OTgyNzRmNzI3Y2E1NzAwNDlmZWZkNTU3ZGI3OTkwNDc2OTlmMmY3OGYxNDIiLCJleGVjdXRpb25fcHVycG9zZSI6InByb2R1Y3Rpb24iLCJmaW5hbF9hdXRob3JpdHlfc2VxdWVuY2UiOjAsImZpbmFsX3J1bnRpbWVfZGVsaXZlcnlfc2VxdWVuY2UiOjksImZsb3dfcnVuX2lkX3NoYTI1NiI6IjExZTU5NGY0ODE5NThjMTBlMzAxNWQwYmYwNDQ3YTIyZjA2OGE4YTY0N2Y0NzVkZjE1Y2UyYzdhYjRiOGYzZjEiLCJpZGVudGl0eV9jb250cmFjdF9zaGEyNTYiOiJkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkIiwiaXNzdWVkX2F0IjoiMjAyNi0wOC0xOFQxMjowMDowM1oiLCJwZXJtaXRfY2hhaW4iOnsiZW50cmllcyI6W3siZGVsaXZlcnlfcmVjZWlwdF9hcnRpZmFjdCI6eyJwYXlsb2FkIjp7ImF1dGhlbnRpY2F0ZWRfcnVubmVyX2lkX3NoYTI1NiI6ImJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmIiLCJhdXRoZW50aWNhdGVkX3Nlc3Npb25faWRfc2hhMjU2IjoiY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjYyIsImRlbGl2ZXJlZF9hdCI6IjIwMjYtMDgtMThUMTI6MDA6MDFaIiwiZXhlY3V0aW9uX2F1dGhvcml0eV9pZCI6IjAwMDAwMDAwLTAwMDAtNDAwMC04MDAwLTAwMDAwMDAwMDAwOCIsIm9uZV91c2VfY2xhaW1faWQiOiIwMDAwMDAwMC0wMDAwLTQwMDAtODAwMC0wMDAwMDAwMDAwMTAiLCJwZXJtaXRfYXJ0aWZhY3Rfc2hhMjU2IjoiOTcyYTgyYWY0NTZkYTU0MmJhYzc4NDYwMTg4OTY2ODg5MWQ4YjdjY2QyNDI5MDQ4MTZiMzI5MmUxNTdlNTI4OSIsInBlcm1pdF9pZCI6InBlcm1pdDoxIiwicnVudGltZV9kZWxpdmVyeV9zZXF1ZW5jZSI6OSwic2NoZW1hX3ZlcnNpb24iOiJvcGVuYWRhcHQucHJvZHVjdGlvbi1kZWxpdmVyeS1yZWNlaXB0LXBheWxvYWQvdjIifSwicGF5bG9hZF9zaGEyNTYiOiI4YThmNmQ4ODc1MGNiYjZiM2JkNjYyMDkzYWQyOWUyYmMyMGM2YWI2OTkxZDcwZTU4ODY5MWM4YjNhYjY2MmI5Iiwic2NoZW1hX3ZlcnNpb24iOiJvcGVuYWRhcHQucHJvZHVjdGlvbi1kZWxpdmVyeS1yZWNlaXB0LWFydGlmYWN0L3YyIiwic2lnbmF0dXJlIjoiQU9LcUNnUzFzMFhhLWl5RTk2Vi15bFB6UWN2MnV2T3N1eWdIenZ1MlJ5VjdGTHNIRl9MVmZxeXg0dTJwdW1sRWwwWWs1WXU3MEpWVWtzWnBhVXBERHciLCJzaWduZXIiOnsiYWxnb3JpdGhtIjoiZWQyNTUxOSIsImtleV9pZCI6ImRlbGl2ZXJ5LWF1dGhvcml0eS1lZDI1NTE5LTI0ZjZlZDZhY2JmZTEwMDkiLCJwdWJsaWNfa2V5IjoiS2F5NjRVRzh5dkN5TGhxVTAwMEx4elllVW0wTC9oTElsNVM4a3lLV2JkYz0ifX0sImRlbGl2ZXJ5X3JlY2VpcHRfYXJ0aWZhY3Rfc2hhMjU2IjoiYzAwYTFiYTcwMTVmM2NiZTA1YTc2YWVhYmZmNjE5ZDc5M2U3Mzc1YTMxMGUxOTVmY2JiZDJkMjk0OGYyYjIxNCIsInBlcm1pdF9hcnRpZmFjdCI6eyJwYXlsb2FkIjp7ImFjdGlvbl9yZXF1ZXN0X3NoYTI1NiI6ImRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGQiLCJhZG1pc3Npb25fYXJ0aWZhY3Rfc2hhMjU2IjoiZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZCIsImF1dGhvcml0eV9zZXF1ZW5jZSI6MCwiZW52aXJvbm1lbnRfZGlnZXN0IjoiYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYSIsImV2aWRlbmNlX2lkZW50aXR5X3NoYTI1NiI6ImVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWUiLCJleGVjdXRpb25fYXV0aG9yaXR5X2lkIjoiMDAwMDAwMDAtMDAwMC00MDAwLTgwMDAtMDAwMDAwMDAwMDA4IiwiZXhlY3V0aW9uX2F1dGhvcml0eV9zaGEyNTYiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIiwiZmxvd19ydW5faWRfc2hhMjU2IjoiMTFlNTk0ZjQ4MTk1OGMxMGUzMDE1ZDBiZjA0NDdhMjJmMDY4YThhNjQ3ZjQ3NWRmMTVjZTJjN2FiNGI4ZjNmMSIsImlucHV0X2VkZ2Vfc2VxdWVuY2UiOjEsImlzc3VlZF9hdCI6IjIwMjYtMDgtMThUMTI6MDA6MDBaIiwicGVybWl0X2lkIjoicGVybWl0OjEiLCJxdWFsaWZpY2F0aW9uX3NpZ25lcl9yZWdpc3RyeV9jaGVja2VkX2F0IjoiMjAyNi0wOC0xOFQxMTo1OTozMFoiLCJxdWFsaWZpY2F0aW9uX3NpZ25lcl9yZWdpc3RyeV9leHBpcmVzX2F0IjoiMjAyNi0wOC0yMFQxMTowMDowMFoiLCJxdWFsaWZpY2F0aW9uX3NpZ25lcl9yZWdpc3RyeV9yZXZpc2lvbiI6NywicXVhbGlmaWNhdGlvbl9zaWduZXJfcmVnaXN0cnlfc2hhMjU2IjoiZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZSIsInJ1bl9pZCI6IjAwMDAwMDAwLTAwMDAtNDAwMC04MDAwLTAwMDAwMDAwMDAwMSIsInJ1bl9yZXF1ZXN0X3NoYTI1NiI6ImNjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2MiLCJzY2hlbWFfdmVyc2lvbiI6Im9wZW5hZGFwdC5wcm9kdWN0aW9uLWRlbGl2ZXJ5LXBlcm1pdC1wYXlsb2FkL3YyIn0sInBheWxvYWRfc2hhMjU2IjoiYzg3ZjU3NTFhYTliMjc5Y2NlMjNhYTE3OGFmZjk1NmUxOWY2OTBmY2QyZjlmNzBlMGQ5OTk3ODJhNjkwZDZiOCIsInNjaGVtYV92ZXJzaW9uIjoib3BlbmFkYXB0LnByb2R1Y3Rpb24tZGVsaXZlcnktcGVybWl0LWFydGlmYWN0L3YyIiwic2lnbmF0dXJlIjoiMEJ5SGdxRHhRWnJuZ0xJRi02dFpPcTlsWGFtY0FRQzBkb1VUbmVLTlBlUy1TeW9PYlF5T2tBdVBGSzVQWWNOek9iWnhnc2dBTGRXYUlfc2FVQ3BsQmciLCJzaWduZXIiOnsiYWxnb3JpdGhtIjoiZWQyNTUxOSIsImtleV9pZCI6ImRlbGl2ZXJ5LWF1dGhvcml0eS1lZDI1NTE5LTI0ZjZlZDZhY2JmZTEwMDkiLCJwdWJsaWNfa2V5IjoiS2F5NjRVRzh5dkN5TGhxVTAwMEx4elllVW0wTC9oTElsNVM4a3lLV2JkYz0ifX0sInBlcm1pdF9hcnRpZmFjdF9zaGEyNTYiOiI5NzJhODJhZjQ1NmRhNTQyYmFjNzg0NjAxODg5NjY4ODkxZDhiN2NjZDI0MjkwNDgxNmIzMjkyZTE1N2U1Mjg5Iiwic2NoZW1hX3ZlcnNpb24iOiJvcGVuYWRhcHQucHJvZHVjdGlvbi1kZWxpdmVyeS1wZXJtaXQtY2hhaW4tZW50cnkvdjIifV0sInBlcm1pdF9jaGFpbl9zaGEyNTYiOiI1MDQ4ZmU5NDU4ODJkMjgwMjYwMDhkN2U5MTM3YWQwMTZmZGQwMWFlYWY1ODg0ODFmYjViZDBmMjQyYTUwMDQzIiwic2NoZW1hX3ZlcnNpb24iOiJvcGVuYWRhcHQucHJvZHVjdGlvbi1kZWxpdmVyeS1wZXJtaXQtY2hhaW4vdjIifSwicGVybWl0X2NvdW50IjoxLCJxdWFsaWZpY2F0aW9uX3NpZ25lcl9yZWdpc3RyeV9yZXZpc2lvbiI6NywicXVhbGlmaWNhdGlvbl9zaWduZXJfcmVnaXN0cnlfc2hhMjU2IjoiZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZSIsInJ1bl9pZCI6IjAwMDAwMDAwLTAwMDAtNDAwMC04MDAwLTAwMDAwMDAwMDAwMSIsInJ1bl9yZWNlaXB0Ijp7ImF1dGhvcml6YXRpb25fY29uZmlybWVkIjoxLCJhdXRob3JpemF0aW9uX3JlcXVpcmVkIjoxLCJidW5kbGVfZGlnZXN0IjoiYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYSIsImR1cmF0aW9uX21zIjoxMDAsImVmZmVjdF90aWVyX3JlYWNoZWQiOiJub25lIiwiZWZmZWN0c19jb25maXJtZWQiOjAsImVmZmVjdHNfcmVxdWlyZWQiOjEsImVzdF9jb3N0X21pY3JvdXNkIjowLCJldmlkZW5jZV9jbGFzc2VzIjpbImF1dGhvcml6YXRpb24iLCJpZGVudGl0eSJdLCJleHRlcm5hbF9uZXR3b3JrX2NhbGxzIjoibm9uZSIsImdlbmVyYXRlZF9hdCI6IjIwMjYtMDgtMThUMTI6MDA6MDBaIiwiaGVhbHMiOjAsImlkZW50aXR5X2FwcGxpY2FibGUiOjEsImlkZW50aXR5X2FybWVkIjoxLCJpZGVudGl0eV9jb25maXJtZWQiOjEsImlkZW50aXR5X3JlcXVpcmVkIjoxLCJtb2RlbF9jYWxscyI6MCwib3V0Y29tZSI6IkhBTFRFRCIsIm92ZXJfaGFsdF9jb3VudCI6MCwicG9zdGNvbmRpdGlvbnNfY29uZmlybWVkIjowLCJwb3N0Y29uZGl0aW9uc19yZXF1aXJlZCI6MSwicHJvZHVjdGlvbl9lbGlnaWJsZSI6ZmFsc2UsInByb2ZpbGUiOiJzdGFuZGFyZCIsInByb3ZlbmFuY2UiOiJwcm9kdWN0aW9uIiwicmVjZWlwdF9idWlsZGVyX3ZlcnNpb24iOiIxLjIuMyIsInJ1bmdfaGlzdG9ncmFtIjp7fSwic2NoZW1hX3ZlcnNpb24iOiJvcGVuYWRhcHQucHJvZHVjdGlvbi1ydW4tcmVjZWlwdC92MSIsInNvdXJjZV9yZWNlaXB0X2RpZ2VzdCI6ImJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmIiLCJzb3VyY2VfcmVjZWlwdF9zaGEyNTYiOiJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiIiwic291cmNlX3NjaGVtYV92ZXJzaW9uIjoib3BlbmFkYXB0LnJ1bi1yZXBvcnQvdjEiLCJzdGVwc19vayI6MCwic3RlcHNfdG90YWwiOjEsInN1YnN0cmF0ZSI6IndlYiIsInRyYW5zYWN0aW9uX291dGNvbWUiOiJSRUNPTkNJTElBVElPTl9SRVFVSVJFRCJ9LCJydW5fcmVjZWlwdF9zaGEyNTYiOiIxOGViMmM0ZjQxNTNkYWIwNTNiZjAyN2MyNmY2OGFmYTA2MTMxN2UzYzJlN2Q1YTYwZWU1OGMzZWU2YjNkN2M2IiwicnVuX3JlcG9ydF9vYmplY3Rfc2hhMjU2IjoiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYiIsInJ1bl9yZXBvcnRfb2JqZWN0X3ZlcnNpb24iOiJ2ZXJzaW9uOnJlY29uY2lsaWF0aW9uOjEiLCJydW5fcmVwb3J0X3NoYTI1NiI6ImJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmJiYmIiLCJydW50aW1lX2Vudmlyb25tZW50X3NoYTI1NiI6ImNjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2MiLCJydW50aW1lX3N1YnN0cmF0ZSI6IndlYiIsInJ1bnRpbWVfdmFsaWRhdGlvbl9pZCI6IjAwMDAwMDAwLTAwMDAtNDAwMC04MDAwLTAwMDAwMDAwMDAwNSIsInNjaGVtYV92ZXJzaW9uIjoib3BlbmFkYXB0LnByb2R1Y3Rpb24tdGVybWluYWwtdmVyaWZpY2F0aW9uL3YyIiwidGVuYW50X2lkIjoiMDAwMDAwMDAtMDAwMC00MDAwLTgwMDAtMDAwMDAwMDAwMDAyIiwidGVybWluYWxfc2VxdWVuY2UiOjEsInZlcmlmaWVkX2F0IjoiMjAyNi0wOC0xOFQxMjowMDowMloiLCJ3b3JrZmxvd19jb250cmFjdF9zaGEyNTYiOiJhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhIiwid29ya2Zsb3dfaWQiOiIwMDAwMDAwMC0wMDAwLTQwMDAtODAwMC0wMDAwMDAwMDAwMDMiLCJ3b3JrZmxvd192ZXJzaW9uX2lkIjoiMDAwMDAwMDAtMDAwMC00MDAwLTgwMDAtMDAwMDAwMDAwMDA0In0sInNpZ25hdHVyZSI6Imd2ZHZuSlNPUzRWazg3M2k4eDBDMmE5MUE0Qm9WOFFvc05xdmMwaUVHMkJOOXpHOUlwamk5dXI1a0VpME5xdW5HVDdLVFZFczhfLUk2VG1TN21ERUJBIiwic2lnbmVyIjp7ImFsZ29yaXRobSI6ImVkMjU1MTkiLCJrZXlfaWQiOiJldmlkZW5jZS1ydW5uZXItZWQyNTUxOS01NjQ3NWFhNzU0NjM0NzRjIiwicHVibGljX2tleSI6IkE2RUh2L1BPRUw0ZGNOMFk1MHZBbVdmazFqQ2JwUTFmSGR5R1pCSlZNYmc9In19","name":"reconciliation-required-nonempty-permit","payload_canonical_sha256":"ef33d988e8895b431f8d06b134cac38fa0b6628b75c0fbe155e30ebb786bd8bf","signature":"gvdvnJSOS4Vk873i8x0C2a91A4BoV8QosNqvc0iEG2BN9zG9Ipji9ur5kEi0NqunGT7KTVEs8_-I6TmS7mDEBA","terminal_verification_artifact_sha256":"d4959d87ddb589221ddcddc6094bf42948c9357b51769239834815036fc36cad"}]} diff --git a/tests/test_backend_factory.py b/tests/test_backend_factory.py index 0ef92eb9..90ce21f5 100644 --- a/tests/test_backend_factory.py +++ b/tests/test_backend_factory.py @@ -462,6 +462,9 @@ def fake_record(out_dir, *, task_description, params, **kwargs): assert _cmd_record(args) == 0 assert captured["params"] == {} assert captured["window"] is None # no --window: full-screen capture + assert captured["source_surface"] == ( + kind if kind in ("windows", "macos", "linux") else None + ) assert captured["backend_kind"] == (kind if kind in ("rdp", "citrix") else None) diff --git a/tests/test_browser_attach.py b/tests/test_browser_attach.py index d23199b5..cce276e2 100644 --- a/tests/test_browser_attach.py +++ b/tests/test_browser_attach.py @@ -1782,7 +1782,8 @@ def expose_late_unbound_closed_secret(page, pump): cdp_endpoint=endpoint, script=expose_late_unbound_closed_secret, ) - assert not late_closed_output.exists() + assert not (late_closed_output / "meta.json").exists() + assert (late_closed_output / "incomplete.json").is_file() with sync_playwright() as late_cleanup_playwright: late_cleanup_browser = late_cleanup_playwright.chromium.connect_over_cdp( endpoint @@ -1927,7 +1928,8 @@ def replace_page_privacy_guard(page, pump): cdp_endpoint=endpoint, script=replace_page_privacy_guard, ) - assert not replaced_guard_output.exists() + assert not (replaced_guard_output / "meta.json").exists() + assert (replaced_guard_output / "incomplete.json").is_file() with sync_playwright() as guard_cleanup_playwright: guard_cleanup_browser = guard_cleanup_playwright.chromium.connect_over_cdp( endpoint @@ -2027,6 +2029,9 @@ def type_existing_closed_shadow_secret(page, pump): cdp_endpoint=endpoint, script=lambda _page, _pump: None, ) + assert not (refused_closed_output / "meta.json").exists() + # Startup refused before Flow could retain one valid atomic + # observation. There is no evidence journal to preserve. assert not refused_closed_output.exists() with sync_playwright() as refusal_cleanup_playwright: refusal_cleanup_browser = ( @@ -2296,7 +2301,11 @@ def attach_and_detach_during_every_capture(**kwargs): }""" ) frame_race_session.abort() - assert not (tmp_path / "recording-frame-race-probe").exists() + frame_race_recording = tmp_path / "recording-frame-race-probe" + assert not (frame_race_recording / "meta.json").exists() + assert json.loads((frame_race_recording / "incomplete.json").read_text())[ + "terminal_reason" + ] == "operator_aborted" assert process.poll() is None interleaved_recording = tmp_path / "recording-interleaved-action-refusal" @@ -2309,21 +2318,23 @@ def attach_and_detach_during_every_capture(**kwargs): assert interleaved_session.page is not None assert interleaved_session.backend is not None interleaved_session.page.click("#note") - original_backend_screenshot = interleaved_session.backend.screenshot + original_backend_capture = ( + interleaved_session.backend._capture_screenshot_bytes + ) interleaved_action_injected = False - def screenshot_after_second_action() -> bytes: + def capture_after_second_action() -> bytes: nonlocal interleaved_action_injected if not interleaved_action_injected: interleaved_action_injected = True interleaved_session.page.click("#save") interleaved_session.page.wait_for_timeout(0) - return original_backend_screenshot() + return original_backend_capture() monkeypatch.setattr( interleaved_session.backend, - "screenshot", - screenshot_after_second_action, + "_capture_screenshot_bytes", + capture_after_second_action, ) try: with pytest.raises(BrowserAttachError, match="more than one logical"): @@ -2663,7 +2674,8 @@ def leave_origin_and_return(page, pump): cdp_endpoint=endpoint, script=leave_origin_and_return, ) - assert not origin_bounce_recording.exists() + assert not (origin_bounce_recording / "meta.json").exists() + assert (origin_bounce_recording / "incomplete.json").is_file() assert process.poll() is None overlap_recording = tmp_path / "recording-resize-overlap-refusal" @@ -2732,6 +2744,27 @@ def resize_then_record(page, pump): assert meta["viewport_mode"] == "per-event" assert meta["viewport_history"][-1]["viewport"] == [900, 600] assert len(meta["viewport_history"]) >= 2 + assert meta["frame_observation_schema"] == ( + "openadapt.browser-frame-observation.v1" + ) + observations = [ + json.loads(line) + for line in (recording / "observation_journal.jsonl") + .read_text() + .splitlines() + ] + assert len(observations) == meta["observation_count"] + assert any( + observation["boundary"] == "geometry_transition" + for observation in observations + ) + assert len( + {observation["geometry_epoch"] for observation in observations} + ) >= 2 + assert all( + (recording / observation["frame_path"]).is_file() + for observation in observations + ) assert events assert {tuple(event["viewport_before"]) for event in events} == { (1000, 650), diff --git a/tests/test_cli_tutorial_break_it.py b/tests/test_cli_tutorial_break_it.py index 024f07e4..bbb228b5 100644 --- a/tests/test_cli_tutorial_break_it.py +++ b/tests/test_cli_tutorial_break_it.py @@ -1,14 +1,16 @@ -"""CLI wiring and fail-loud contract for ``tutorial --break-it``. +"""CLI wiring for the advanced rejected-write verification fixture. Browser-free: the tutorial's heavy loop is faked, and the real end-to-end behavior (record, run, halt) is owned by ``tests/e2e/test_tutorial_break_it_e2e.py``. What is proven here, cheaply and on every unit run: -* the flag reaches :func:`openadapt_flow.tutorial.run_tutorial` and its result +* ``--simulate-rejected-write`` reaches + :func:`openadapt_flow.tutorial.run_tutorial` and its result drives the printed narrative -- screen claim, refuted verifier read, HALTED outcome, evidence path, and the no-receipt rule -- from run evidence, not from a script; -* the plain tutorial ends with a pointer at ``--break-it`` as the next step; +* the plain tutorial points at a real first workflow, never this fixture; +* the deprecated ``--break-it`` alias stays hidden and warns on stderr; * :func:`openadapt_flow.tutorial._run_break_it` raises loudly when the engine does NOT halt on the injected fault, and extracts the narrative facts from a halted report when it does. @@ -63,10 +65,13 @@ def _verified_result( ) -def _broken_result(root: Path) -> BreakItResult: +def _broken_result( + root: Path, *, run_dir_name: str = "run-rejected-write" +) -> BreakItResult: + run_dir = root / run_dir_name return BreakItResult( - run_dir=root / "run-broken", - report_path=root / "run-broken" / "REPORT.md", + run_dir=run_dir, + report_path=run_dir / "REPORT.md", fault="optimistic", execution_outcome="HALTED", transaction_outcome="RECONCILIATION_REQUIRED", @@ -77,10 +82,11 @@ def _broken_result(root: Path) -> BreakItResult: effects_refuted=1, halt_reason="record_written refuted against the rest system of record", system_of_record_records=0, + rejected_writes=1, ) -def test_break_it_flag_reaches_run_tutorial_and_drives_the_narrative( +def test_simulate_rejected_write_reaches_tutorial_and_drives_the_narrative( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], @@ -92,31 +98,42 @@ def fake_run_tutorial(work_dir: Path, **kwargs: Any) -> TutorialResult: return _verified_result(Path(work_dir), break_it=_broken_result(Path(work_dir))) monkeypatch.setattr(tutorial_module, "run_tutorial", fake_run_tutorial) - assert main(["tutorial", "--break-it", "--out", str(tmp_path / "t")]) == 0 - assert seen["break_it"] is True + assert ( + main( + [ + "tutorial", + "--simulate-rejected-write", + "--out", + str(tmp_path / "t"), + ] + ) + == 0 + ) + assert seen["simulate_rejected_write"] is True + assert seen["break_it"] is False out = capsys.readouterr().out # The two runs are labeled so VERIFIED is never misread as the broken one. assert "clean run" in out - assert "break-it: the same certified bundle" in out + assert "rejected-write verification" in out # The narrative's three beats, from evidence fields. assert "every on-screen check passed" in out assert '"Encounter saved"' in out assert "1/2 declared effect(s) REFUTED" in out assert "0 record(s)" in out + assert "1 rejected write attempt (no retry)" in out assert "HALTED at the consequential step" in out assert "RECONCILIATION_REQUIRED" in out # The engine's own halt reason is quoted, not paraphrased. assert "record_written refuted" in out # Where the evidence lives, and what may NOT be claimed. - assert str(tmp_path / "t" / "run-broken" / "REPORT.md") in out + assert str(tmp_path / "t" / "run-rejected-write" / "REPORT.md") in out assert "NOT a success receipt" in out assert "No shareable receipt for the halted run" in out - # And what to do next. - assert "record your own workflow" in out + assert "openadapt-flow tutorial --break-it" not in out -def test_plain_tutorial_points_at_break_it_next( +def test_plain_tutorial_points_at_a_real_first_workflow( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], @@ -129,13 +146,67 @@ def fake_run_tutorial(work_dir: Path, **kwargs: Any) -> TutorialResult: monkeypatch.setattr(tutorial_module, "run_tutorial", fake_run_tutorial) assert main(["tutorial", "--out", str(tmp_path / "t")]) == 0 + assert seen["simulate_rejected_write"] is False assert seen["break_it"] is False out = capsys.readouterr().out assert "REPORT.md" in out assert "receipt.json" in out - assert "openadapt-flow tutorial --break-it" in out - assert "break-it: the same certified bundle" not in out + assert "one small read-only task with test data" in out + assert "openadapt-flow record --backend web" in out + assert "openadapt-flow compile recording" in out + assert "openadapt-flow visualize bundle" in out + assert "openadapt-flow lint bundle" in out + assert "openadapt-flow replay bundle" in out + assert "--run-dir first-run" in out + assert "first-run/REPORT.md" in out + assert "before Flow first actuates it" in out + assert out.index("openadapt-flow lint bundle") < out.index( + "before Flow first actuates it" + ) + assert out.index("before Flow first actuates it") < out.index( + "openadapt-flow replay bundle" + ) + assert "--simulate-rejected-write" not in out + assert "--break-it" not in out + + +def test_rejected_write_help_hides_the_deprecated_alias( + capsys: pytest.CaptureFixture[str], +) -> None: + with pytest.raises(SystemExit) as exc: + main(["tutorial", "--help"]) + assert exc.value.code == 0 + out = capsys.readouterr().out + assert "--simulate-rejected-write" in out + assert "--break-it" not in out + + +def test_deprecated_break_it_alias_warns_and_runs_the_same_fixture( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + capsys: pytest.CaptureFixture[str], +) -> None: + seen: dict[str, Any] = {} + + def fake_run_tutorial(work_dir: Path, **kwargs: Any) -> TutorialResult: + seen.update(kwargs) + return _verified_result( + Path(work_dir), + break_it=_broken_result(Path(work_dir), run_dir_name="run-broken"), + ) + + monkeypatch.setattr(tutorial_module, "run_tutorial", fake_run_tutorial) + assert main(["tutorial", "--break-it", "--out", str(tmp_path / "t")]) == 0 + assert seen["simulate_rejected_write"] is False + assert seen["break_it"] is True + + captured = capsys.readouterr() + assert "rejected-write verification" in captured.out + assert str(tmp_path / "t" / "run-broken" / "REPORT.md") in captured.out + assert captured.err == ( + "warning: --break-it is deprecated; use --simulate-rejected-write.\n" + ) # --------------------------------------------------------------------------- @@ -153,14 +224,26 @@ def _fake_workflow() -> Any: def _wire_break_it_fakes( - monkeypatch: pytest.MonkeyPatch, tmp_path: Path, report: Any + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, + report: Any, + *, + record_count: int = 0, + rejected_writes: int = 1, ) -> None: + fault_db = SimpleNamespace( + snapshot=lambda: {"rejected_writes": rejected_writes} + ) monkeypatch.setattr( fault_server_module, "serve", - lambda: ("http://127.0.0.1:9/", None, lambda: None), + lambda: ("http://127.0.0.1:9/", fault_db, lambda: None), + ) + monkeypatch.setattr( + tutorial_module, + "_records", + lambda base_url: [{} for _ in range(record_count)], ) - monkeypatch.setattr(tutorial_module, "_records", lambda base_url: []) monkeypatch.setattr( tutorial_module, "run_tutorial_workflow", lambda **kwargs: report ) @@ -182,21 +265,19 @@ def test_run_break_it_refuses_an_uncaught_fault( _run_break_it( workflow=_fake_workflow(), bundle_dir=tmp_path / "bundle", - run_dir=tmp_path / "run-broken", + run_dir=tmp_path / "run-rejected-write", headed=False, say=lambda message: None, ) -def test_run_break_it_extracts_the_narrative_from_the_halted_report( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: +def _halted_report() -> Any: save_result = SimpleNamespace( step_id="step_005", postconditions_ok=True, effect_evidence=[SimpleNamespace(final_verdict="refuted")], ) - report = SimpleNamespace( + return SimpleNamespace( execution_outcome="HALTED", transaction_outcome="RECONCILIATION_REQUIRED", transaction_billable=False, @@ -205,14 +286,22 @@ def test_run_break_it_extracts_the_narrative_from_the_halted_report( observed_texts=["MockMed", "Encountersaved-"], reason="record_written refuted -- nothing landed", ), - outcome_envelope=SimpleNamespace(required_contracts=SimpleNamespace(effect=2)), + outcome_envelope=SimpleNamespace( + required_contracts=SimpleNamespace(effect=2) + ), ) + + +def test_run_break_it_extracts_the_narrative_from_the_halted_report( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + report = _halted_report() _wire_break_it_fakes(monkeypatch, tmp_path, report) broken = _run_break_it( workflow=_fake_workflow(), bundle_dir=tmp_path / "bundle", - run_dir=tmp_path / "run-broken", + run_dir=tmp_path / "run-rejected-write", headed=False, say=lambda message: None, ) @@ -223,5 +312,76 @@ def test_run_break_it_extracts_the_narrative_from_the_halted_report( assert broken.effects_required == 2 assert broken.effects_refuted == 1 assert broken.system_of_record_records == 0 + assert broken.rejected_writes == 1 assert broken.halt_reason == "record_written refuted -- nothing landed" - assert broken.report_path == tmp_path / "run-broken" / "REPORT.md" + assert broken.report_path == tmp_path / "run-rejected-write" / "REPORT.md" + + +@pytest.mark.parametrize( + "defect", + [ + "wrong_transaction_outcome", + "billable", + "unknown_billing", + "missing_consequential_result", + "screen_postcondition_failed", + "effect_not_refuted", + "effect_not_required", + "unrelated_halt_reason", + "missing_halt", + "record_persisted", + "no_rejected_write", + "retried_rejected_write", + ], +) +def test_run_break_it_requires_the_complete_rejected_write_contract( + defect: str, + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + report = _halted_report() + record_count = 0 + rejected_writes = 1 + + if defect == "wrong_transaction_outcome": + report.transaction_outcome = "HALTED" + elif defect == "billable": + report.transaction_billable = True + elif defect == "unknown_billing": + report.transaction_billable = None + elif defect == "missing_consequential_result": + report.results = [] + elif defect == "screen_postcondition_failed": + report.results[0].postconditions_ok = False + elif defect == "effect_not_refuted": + report.results[0].effect_evidence[0].final_verdict = "confirmed" + elif defect == "effect_not_required": + report.outcome_envelope.required_contracts.effect = 0 + elif defect == "unrelated_halt_reason": + report.halt.reason = "the browser closed" + elif defect == "missing_halt": + report.halt = None + elif defect == "record_persisted": + record_count = 1 + elif defect == "no_rejected_write": + rejected_writes = 0 + elif defect == "retried_rejected_write": + rejected_writes = 2 + else: # pragma: no cover - the parametrization owns this value. + raise AssertionError(f"unknown defect: {defect}") + + _wire_break_it_fakes( + monkeypatch, + tmp_path, + report, + record_count=record_count, + rejected_writes=rejected_writes, + ) + with pytest.raises(TutorialError, match="FAILED its evidence contract"): + _run_break_it( + workflow=_fake_workflow(), + bundle_dir=tmp_path / "bundle", + run_dir=tmp_path / "run-rejected-write", + headed=False, + say=lambda message: None, + ) diff --git a/tests/test_cli_tutorial_next_steps.py b/tests/test_cli_tutorial_next_steps.py index d19da3b9..7b796eda 100644 --- a/tests/test_cli_tutorial_next_steps.py +++ b/tests/test_cli_tutorial_next_steps.py @@ -1,13 +1,13 @@ """The closing "next steps" block after a plain VERIFIED tutorial run. -Browser-free, mirroring ``tests/test_cli_tutorial_break_it.py``: the +Browser-free, mirroring the rejected-write CLI tests: the tutorial's heavy loop is faked and only the CLI wiring is proven here. -* :func:`openadapt_flow.tutorial._next_steps_block` carries the three - destinations the flagship README points at; +* :func:`openadapt_flow.tutorial._next_steps_block` carries the real + record, compile, inspect, lint, replay, and qualification path; * the plain VERIFIED tutorial prints the block after the receipt paths; -* a ``--break-it`` run and a non-VERIFIED run do NOT print it -- the block - belongs to the success rail only. +* an advanced rejected-write simulation and a non-VERIFIED run do not print + it because the block belongs to the primary success rail. """ from __future__ import annotations @@ -25,9 +25,7 @@ _next_steps_block, ) -EXECUTE_URL = "https://openadapt.ai/execute" QUALIFY_URL = "https://openadapt.ai/qualify" -DISCORD_URL = "https://discord.gg/yF527cQbDG" def _result( @@ -65,8 +63,8 @@ def _result( def _broken_result(root: Path) -> BreakItResult: return BreakItResult( - run_dir=root / "run-broken", - report_path=root / "run-broken" / "REPORT.md", + run_dir=root / "run-rejected-write", + report_path=root / "run-rejected-write" / "REPORT.md", fault="optimistic", execution_outcome="HALTED", transaction_outcome="RECONCILIATION_REQUIRED", @@ -77,6 +75,7 @@ def _broken_result(root: Path) -> BreakItResult: effects_refuted=1, halt_reason="record_written refuted against the rest system of record", system_of_record_records=0, + rejected_writes=1, ) @@ -87,12 +86,27 @@ def fake_run_tutorial(work_dir: Path, **kwargs: Any) -> TutorialResult: monkeypatch.setattr(tutorial_module, "run_tutorial", fake_run_tutorial) -def test_next_steps_block_carries_the_three_readme_urls() -> None: +def test_next_steps_block_carries_the_real_first_workflow() -> None: block = _next_steps_block() - assert EXECUTE_URL in block + assert "one small read-only task with test data" in block + assert "openadapt-flow record --backend web" in block + assert "openadapt-flow compile recording" in block + assert "openadapt-flow visualize bundle" in block + assert "openadapt-flow lint bundle" in block + assert "openadapt-flow replay bundle" in block + assert "--run-dir first-run" in block + assert "first-run/REPORT.md" in block assert QUALIFY_URL in block - assert DISCORD_URL in block - assert "no model call" in block + assert "identity, effect, and policy evidence" in block + assert "before Flow first actuates it" in block + assert block.index("openadapt-flow lint bundle") < block.index( + "before Flow first actuates it" + ) + assert block.index("before Flow first actuates it") < block.index( + "openadapt-flow replay bundle" + ) + assert "--simulate-rejected-write" not in block + assert "--break-it" not in block def test_verified_tutorial_prints_the_block_after_the_receipt( @@ -106,10 +120,10 @@ def test_verified_tutorial_prints_the_block_after_the_receipt( out = capsys.readouterr().out assert _next_steps_block() in out # After the receipt paths, at the very end of the run's story. - assert out.index("receipt.json") < out.index(EXECUTE_URL) + assert out.index("receipt.json") < out.index("openadapt-flow record") -def test_break_it_run_does_not_print_the_block( +def test_rejected_write_simulation_does_not_print_the_first_workflow_block( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], @@ -118,12 +132,21 @@ def test_break_it_run_does_not_print_the_block( monkeypatch, lambda root, kwargs: _result(root, break_it=_broken_result(root)), ) - assert main(["tutorial", "--break-it", "--out", str(tmp_path / "t")]) == 0 + assert ( + main( + [ + "tutorial", + "--simulate-rejected-write", + "--out", + str(tmp_path / "t"), + ] + ) + == 0 + ) out = capsys.readouterr().out - assert EXECUTE_URL not in out assert QUALIFY_URL not in out - assert DISCORD_URL not in out + assert "openadapt-flow record --backend web" not in out def test_unverified_tutorial_does_not_print_the_block( @@ -138,6 +161,5 @@ def test_unverified_tutorial_does_not_print_the_block( assert main(["tutorial", "--out", str(tmp_path / "t")]) == 1 out = capsys.readouterr().out - assert EXECUTE_URL not in out assert QUALIFY_URL not in out - assert DISCORD_URL not in out + assert "openadapt-flow record --backend web" not in out diff --git a/tests/test_desktop_record.py b/tests/test_desktop_record.py index ed9f7f5e..43679309 100644 --- a/tests/test_desktop_record.py +++ b/tests/test_desktop_record.py @@ -448,6 +448,46 @@ def fake_convert( assert observed["include_structural"] is expected +def test_orchestration_passes_exact_native_source_surface( + tmp_path: Path, + monkeypatch, +) -> None: + observed: dict[str, object] = {} + log: list = [] + + def fake_convert( + cap_dir, + out_dir, + *, + params=None, + include_structural=None, + source_surface=None, + ): + observed["include_structural"] = include_structural + observed["source_surface"] = source_surface + (Path(out_dir) / "meta.json").write_text( + json.dumps({"id": "x", "viewport": [800, 600], "params": {}}) + ) + return Path(out_dir) + + monkeypatch.setattr( + "openadapt_flow.adapters.capture.convert_capture", + fake_convert, + ) + record_desktop_capture( + tmp_path / "native", + source_surface="windows", + recorder_factory=_make(log), + stop=lambda: True, + announce=False, + ) + + assert observed == { + "include_structural": True, + "source_surface": "windows", + } + + def test_orchestration_stamps_exact_citrix_replay_binding(tmp_path: Path) -> None: log: list = [] @@ -531,20 +571,10 @@ def test_cli_record_web_requires_url(tmp_path: Path) -> None: ["--agent-url", "http://localhost:5001"], "local Capture session cannot bind to a WAA endpoint", ), - ( - "linux", - ["--linux-app", "gedit"], - "no Linux window-scoping primitive", - ), - ( - "linux", - ["--linux-window-title", "Untitled Document 1"], - "no Linux window-scoping primitive", - ), ( "linux", ["--linux-allow-physical-input"], - "no Linux window-scoping primitive", + "permits replay-time physical input", ), ( "rdp", @@ -623,6 +653,32 @@ def test_cli_record_macos_target_scopes_capture(tmp_path: Path, monkeypatch) -> assert captured["window"] == {"owner": "TextEdit", "title": "notes.txt"} +def test_cli_record_linux_target_scopes_capture(tmp_path: Path, monkeypatch) -> None: + """Linux target flags bind the exact local X11 Capture window.""" + captured: dict = {} + monkeypatch.setattr( + "openadapt_flow.desktop_record.record_desktop_capture", + _fake_desktop_record(captured), + ) + + rc = _run_cli( + [ + "record", + "--backend", + "linux", + "--linux-app", + "gedit", + "--linux-window-title", + "notes.txt", + "--out", + str(tmp_path / "rec"), + ] + ) + + assert rc == 0 + assert captured["window"] == {"owner": "gedit", "title": "notes.txt"} + + @pytest.mark.parametrize( ("backend", "generic_flags", "target_flags"), [ @@ -632,6 +688,12 @@ def test_cli_record_macos_target_scopes_capture(tmp_path: Path, monkeypatch) -> ["--window-title", "notes.txt"], ["--macos-window-title", "draft.txt"], ), + ("linux", ["--window", "Notes"], ["--linux-app", "gedit"]), + ( + "linux", + ["--window-title", "notes.txt"], + ["--linux-window-title", "draft.txt"], + ), ], ) def test_cli_record_refuses_conflicting_capture_targets_before_capture( @@ -834,7 +896,7 @@ def test_record_desktop_window_forwarded_to_factory( """`record_desktop_capture(window=...)` reaches the default factory closure.""" from openadapt_flow import desktop_record - # The platform guard only permits window capture on darwin/win32; force a + # The platform guard permits window capture on darwin/win32/linux; force a # supported platform so this forwarding test runs on the Linux CI host. monkeypatch.setattr(desktop_record.sys, "platform", "darwin") @@ -872,7 +934,7 @@ def test_record_desktop_window_unsupported_platform( """ from openadapt_flow import desktop_record - monkeypatch.setattr(desktop_record.sys, "platform", "linux") + monkeypatch.setattr(desktop_record.sys, "platform", "freebsd13") with pytest.raises(SystemExit, match="not supported on this host"): record_desktop_capture( tmp_path / "rec", diff --git a/tests/test_durable_authority_v13.py b/tests/test_durable_authority_v13.py index a9240dea..0db17584 100644 --- a/tests/test_durable_authority_v13.py +++ b/tests/test_durable_authority_v13.py @@ -38,6 +38,7 @@ AUTHORITY_DB_ENV, REMOTE_AUTHORITY_TOKEN_ENV, REMOTE_AUTHORITY_URL_ENV, + REMOTE_DISPATCH_SESSION_ID_ENV, SYNTHETIC_DELIVERY_MARKER_ENABLED_ENV, SYNTHETIC_DELIVERY_MARKER_RUN_ID_ENV, DurableAuthority, @@ -523,6 +524,8 @@ def test_v2_permit_remains_pending_until_signed_receipt_commits_edge( authority.before_initial_delivery(manifest) with pytest.raises(DurableAuthorityBusy, match="remains uncertain"): authority.production_delivery_permit_chain() + with pytest.raises(DurableAuthorityBusy, match="remains uncertain"): + authority.production_delivery_permit_chain(allow_empty=True) entry = authority.acknowledge_remote_delivery(manifest, permit) @@ -534,6 +537,69 @@ def test_v2_permit_remains_pending_until_signed_receipt_commits_edge( assert entry.runtime_delivery_sequence == 0 +def test_pre_actuation_terminal_can_retain_an_empty_delivery_chain( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + manifest, authority = _remote_initial_authority( + tmp_path, + monkeypatch, + _issued_permit_transport("unused", "4" * 64), + ) + + chain = authority.production_delivery_permit_chain(allow_empty=True) + + assert chain.entries == () + assert authority.validate(manifest).delivery_sequence == 0 + with pytest.raises(DurableAuthorityBusy, match="unavailable"): + authority.production_delivery_permit_chain() + + +def test_remote_permit_refuses_a_different_hosted_dispatch_session( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + transport = _issued_permit_transport("wrong-session", "4" * 64) + manifest, authority = _remote_initial_authority(tmp_path, monkeypatch, transport) + monkeypatch.setenv( + REMOTE_DISPATCH_SESSION_ID_ENV, + "30000000-0000-4000-8000-000000000002", + ) + + with pytest.raises(DurableAuthorityBusy, match="does not match request"): + authority.before_initial_delivery(manifest) + + assert authority.validate(manifest).delivery_sequence == 0 + + +def test_lost_delivery_acknowledgment_response_blocks_replay_dispatch( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + issued = _issued_permit_transport("lost-ack", "5" * 64) + permit_calls = 0 + acknowledgment_calls = 0 + + def transport(url: str, headers: dict[str, str], body: bytes) -> bytes: + nonlocal permit_calls, acknowledgment_calls + response = issued(url, headers, body) + if url.endswith("/managed-delivery-acknowledgment"): + acknowledgment_calls += 1 + raise TimeoutError("acknowledgment response lost after server commit") + permit_calls += 1 + return response + + manifest, authority = _remote_initial_authority(tmp_path, monkeypatch, transport) + permit = authority.before_initial_delivery(manifest) + assert permit is not None + + with pytest.raises(DurableAuthorityBusy, match="unavailable or refused"): + authority.acknowledge_remote_delivery(manifest, permit) + with pytest.raises(DurableAuthorityBusy, match="lacks an acknowledgment"): + authority.before_initial_delivery(manifest) + + assert permit_calls == 1 + assert acknowledgment_calls == 1 + assert authority.validate(manifest).delivery_sequence == 0 + + def test_receipt_digest_mismatch_keeps_delivery_uncertain_and_blocks_next_edge( tmp_path: Path, monkeypatch: pytest.MonkeyPatch ) -> None: diff --git a/tests/test_effect_kit_config.py b/tests/test_effect_kit_config.py index 221b1623..4a43a4bd 100644 --- a/tests/test_effect_kit_config.py +++ b/tests/test_effect_kit_config.py @@ -76,6 +76,25 @@ def test_path_params_bind_run_params_url_quoted(self, monkeypatch): assert "applicant=OpenAdapt%20Synthetic" in v.records_path assert v.headers == {"Authorization": "Bearer tok"} + def test_path_params_render_typed_scalars_with_json_text(self): + cfg = EffectsConfig( + kind="rest", + base_url="http://sor.local", + records_path="/x?enabled={enabled}&count={count}&ratio={ratio}", + path_params={ + "enabled": {"param": "enabled"}, + "count": {"param": "count"}, + "ratio": {"param": "ratio"}, + }, + ) + + verifier = build_effect_verifier( + cfg, + params={"enabled": False, "count": 0, "ratio": 1e-7}, + ) + + assert verifier.records_path == "/x?enabled=false&count=0&ratio=1e-7" + def test_auth_headers_sent_on_reads_only_when_configured(self): class _Session: def __init__(self): diff --git a/tests/test_execution_profiles.py b/tests/test_execution_profiles.py index 827f5e35..0a2da9cd 100644 --- a/tests/test_execution_profiles.py +++ b/tests/test_execution_profiles.py @@ -69,7 +69,9 @@ from openadapt_flow.runner.evidence import summary_status from openadapt_flow.runtime.authorization import ( GovernedRunAuthorization, + RuntimeParamScalar, runtime_inputs_digest, + runtime_param_text, ) from openadapt_flow.runtime.durable import ApprovalRequired, CheckpointStore, resume from openadapt_flow.runtime.effects import ( @@ -1523,21 +1525,26 @@ def _workflow(transitions: list[Transition]) -> Workflow: ), ) - def _report(workflow: Workflow, *, route: str) -> RunReport: + def _report( + workflow: Workflow, + *, + route: RuntimeParamScalar, + selected_state: str = "second", + ) -> RunReport: report = RunReport( workflow_name=workflow.name, started_at="2026-07-28T00:00:00Z", success=True, execution_completed=True, terminal_outcome="success", - visited_states=["pick", "second", "done"], + visited_states=["pick", selected_state, "done"], params={"route": route}, governed_authorization_id="authorization-1", governed_runtime_inputs_digest="b" * 64, results=[ StepResult( - step_id="second", - intent="second", + step_id=selected_state, + intent=selected_state, ok=True, starting_state_settled=True, delivery_attempted=True, @@ -1550,7 +1557,9 @@ def _report(workflow: Workflow, *, route: str) -> RunReport: _bind_report_to_workflow(report, workflow) pick = workflow.program.states["pick"] first_guard_matches = bool( - pick.transitions[0].guard is not None and route == "first" + pick.transitions[0].guard is not None + and pick.transitions[0].guard.value is not None + and runtime_param_text(route) == pick.transitions[0].guard.value ) report.program_transition_evidence = [ *_transition_evidence( @@ -1562,7 +1571,7 @@ def _report(workflow: Workflow, *, route: str) -> RunReport: ), *_transition_evidence( decision_index=1, - state=workflow.program.states["second"], + state=workflow.program.states[selected_state], verdicts=[True], target="done", inputs_digest=report.governed_runtime_inputs_digest or "", @@ -1611,6 +1620,34 @@ def _report(workflow: Workflow, *, route: str) -> RunReport: ) is ExecutionOutcome.VERIFIED ) + for value, expected_text in [ + (False, "false"), + (0, "0"), + (1e-7, "1e-7"), + (1e20, "100000000000000000000"), + (1e21, "1e+21"), + ]: + typed_guard = _workflow( + [ + Transition( + guard=Predicate( + kind=PredicateKind.PARAM_EQUALS, + param="route", + value=expected_text, + ), + target="first", + ), + Transition(target="second"), + ] + ) + assert ( + classify_execution_outcome( + _report(typed_guard, route=value, selected_state="first"), + typed_guard, + ExecutionProfile.STANDARD, + ) + is ExecutionOutcome.VERIFIED + ) exact = _report(ordered_guard, route="second") for update in ( {"graph_id": "forged-graph"}, diff --git a/tests/test_governed_authorization.py b/tests/test_governed_authorization.py index e6879534..bd500352 100644 --- a/tests/test_governed_authorization.py +++ b/tests/test_governed_authorization.py @@ -2,6 +2,7 @@ from __future__ import annotations +import hashlib import json import pytest @@ -28,6 +29,7 @@ from openadapt_flow.runtime.authorization import ( GovernedRunAuthorization, UnverifiedWriteApproval, + runtime_inputs_bytes, runtime_inputs_digest, ) from openadapt_flow.runtime.durable import ( @@ -119,6 +121,60 @@ def _authorization( ) +def test_runtime_inputs_match_hosted_javascript_scalar_vector() -> None: + workflow = Workflow(name="cross-language-scalars") + params = { + "whole": 7, + "float": 1.5, + "small": 1e-7, + "mid": 1e20, + "large": 1e21, + "bool_true": True, + "bool_false": False, + "string_int": "7", + "string_float": "1.5", + "string_bool": "true", + } + expected = ( + b'{"params":{"bool_false":false,"bool_true":true,"float":1.5,' + b'"large":1e+21,"mid":100000000000000000000,"small":1e-7,' + b'"string_bool":"true",' + b'"string_float":"1.5","string_int":"7","whole":7},' + b'"worklists":{}}' + ) + + actual = runtime_inputs_bytes(workflow, params, None) + + assert actual == expected + assert hashlib.sha256(actual).hexdigest() == ( + "1f9cf0ab74f3194d4759be349239e6f7bbeef13f1f4641bd9e4e55b018728293" + ) + + +def test_runtime_inputs_sort_object_keys_as_javascript_utf16() -> None: + workflow = Workflow(name="cross-language-unicode-order") + params = {"\ue000": 1, "\U0001f600": 2} + expected = '{"params":{"😀":2,"":1},"worklists":{}}'.encode() + + actual = runtime_inputs_bytes(workflow, params, None) + + assert actual == expected + assert hashlib.sha256(actual).hexdigest() == ( + "f70fa507dff3b9f0d54ab13325e5588a18b718f697d3e4a7877eadf06dcc7196" + ) + assert runtime_inputs_digest(workflow, params, None) != runtime_inputs_digest( + workflow, + {name: str(value) for name, value in params.items()}, + None, + ) + + +@pytest.mark.parametrize("value", [9_007_199_254_740_993, float("nan"), float("inf")]) +def test_runtime_inputs_refuse_non_interoperable_numbers(value: object) -> None: + with pytest.raises(ValueError, match="runtime parameters"): + runtime_inputs_bytes(Workflow(name="unsafe-number"), {"value": value}, None) + + def test_in_memory_semantic_mutation_halts_before_action(tmp_path): step = context_click_step("Jane Sample 1980-01-15 MRN 123") workflow, bundle = _seal(tmp_path, Workflow(name="semantic", steps=[step])) diff --git a/tests/test_hosted_runner_adapter.py b/tests/test_hosted_runner_adapter.py new file mode 100644 index 00000000..1debe264 --- /dev/null +++ b/tests/test_hosted_runner_adapter.py @@ -0,0 +1,1307 @@ +from __future__ import annotations + +import hashlib +import json +import os +from base64 import b64decode, b64encode, urlsafe_b64encode +from dataclasses import replace +from datetime import datetime, timezone +from pathlib import Path +from types import SimpleNamespace + +import pytest +from cryptography.hazmat.primitives import serialization +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey + +import openadapt_flow.runner.hosted_adapter as hosted +from openadapt_flow.__main__ import _replay_params +from openadapt_flow.ir import ParamKind, ParamSpec, Workflow +from openadapt_flow.runner.hosted_adapter import ( + RUNNER_RENEWAL_HEADER, + AdmissionArtifactBytes, + CallbackRequest, + DeliveryAuthority, + HostedDispatch, + HostedRunnerAdapter, + HostedRunResult, + HostedTerminalEvent, + ManagedExecution, + PollRequest, + RegisterCapabilities, + RegisterRequest, + registration_renewal_headers, +) +from openadapt_flow.runner.inputs import resolve_admitted_params +from openadapt_flow.runner.product_release import ( + DOMAIN, + TARGETS, + ProductReleaseAdmissionArtifact, + ProductReleaseAdmissionError, + ProductReleaseAdmissionPayload, + ProductReleaseSignerTrust, + verify_product_release_admission, +) +from openadapt_flow.runner.protocol import ( + DispatchParamsRef, + DispatchParamsValues, + RunnerDispatchPayload, + dispatch_binding_sha256, +) +from openadapt_flow.runner.verify import VerifiedDispatch +from openadapt_flow.runtime.authorization import ( + runtime_inputs_bytes, + runtime_param_text, +) +from openadapt_flow.runtime.durable.authority import REMOTE_DISPATCH_SESSION_ID_ENV +from openadapt_flow.terminal_verification_v2 import ( + ProductionDeliveryPermit, + ProductionDeliveryPermitChain, + ProductionDeliveryPermitPayload, + ProductionDeliveryReceiptPayload, + delivery_authority_signer_sha256, + evidence_runner_signer_sha256, + sign_production_delivery_permit, + sign_production_delivery_receipt, + sign_production_terminal_verification, +) +from openadapt_flow.transaction import TransactionOutcome +from tests.test_run_receipt import _report as _production_report +from tests.test_runner_client_lib import dispatch_payload +from tests.test_terminal_verification_v2 import ( + _halted_payload, + _payload, + _private_key, + _reconciliation_payload, +) + +pytest_plugins = ("tests.test_runner_client_lib",) + + +def _release_payload() -> dict[str, object]: + targets = [] + for index, target in enumerate(TARGETS, start=1): + targets.append( + { + "target": target, + "admission_id": f"00000000-0000-4000-8000-{index:012d}", + "admission_sha256": f"{index:x}" * 64, + "release_id": "1.2.3", + "release_artifact_sha256": f"{index + 7:x}" * 64, + "admission_issued_at": "2026-08-25T00:00:00Z", + "admission_expires_at": "2026-08-28T00:00:00Z", + "revoked_at": None, + "artifact_authority_sha256": f"{index + 8:x}" * 64, + "artifact_authority_state": "active", + "artifact_authority_checked_at": "2026-08-25T00:00:00Z", + "artifact_authority_expires_at": "2026-08-28T00:00:00Z", + } + ) + return { + "schema_version": "openadapt.product-release-admission-payload/v1", + "set_id": "00000000-0000-4000-8000-000000000099", + "sequence": 7, + "policy_sha256": "a" * 64, + "issued_at": "2026-08-25T00:00:00Z", + "expires_at": "2026-08-28T00:00:00Z", + "targets": tuple(targets), + } + + +def _release_artifact() -> tuple[ + ProductReleaseAdmissionArtifact, ProductReleaseSignerTrust +]: + private_key = Ed25519PrivateKey.from_private_bytes(bytes(range(1, 33))) + public_key = private_key.public_key().public_bytes( + serialization.Encoding.Raw, + serialization.PublicFormat.Raw, + ) + payload = ProductReleaseAdmissionPayload.model_validate(_release_payload()) + signature = private_key.sign(DOMAIN + payload.canonical_bytes()) + public_b64 = b64encode(public_key).decode("ascii") + artifact = ProductReleaseAdmissionArtifact.model_validate( + { + "schema_version": "openadapt.product-release-admission-artifact/v1", + "payload": payload, + "payload_sha256": payload.payload_sha256_value(), + "signer": { + "algorithm": "ed25519", + "key_id": ( + "release-admission-ed25519-" + + hashlib.sha256(public_key).hexdigest()[:16] + ), + "public_key": public_b64, + }, + "signature": urlsafe_b64encode(signature).decode("ascii").rstrip("="), + } + ) + return artifact, ProductReleaseSignerTrust( + public_key=public_b64, + status="active", + revoked_at=None, + ) + + +@pytest.mark.parametrize("sequence", [True, "7"]) +def test_product_release_sequence_refuses_scalar_coercion(sequence: object) -> None: + raw = _release_payload() + raw["sequence"] = sequence + with pytest.raises(ValueError): + ProductReleaseAdmissionPayload.model_validate(raw) + + +def test_product_release_refuses_noncanonical_utc() -> None: + raw = _release_payload() + raw["issued_at"] = "2026-08-25T00:00:00+00:00" + with pytest.raises(ValueError, match="canonical UTC"): + ProductReleaseAdmissionPayload.model_validate(raw) + + +def test_product_release_refuses_revoked_signer() -> None: + artifact, trust = _release_artifact() + revoked = trust.model_copy( + update={"status": "revoked", "revoked_at": "2026-08-25T00:00:00Z"} + ) + with pytest.raises(ProductReleaseAdmissionError, match="revoked"): + verify_product_release_admission( + artifact, + trusted_signers={artifact.signer.key_id: revoked}, + newest_sequence=7, + now=datetime(2026, 8, 26, tzinfo=timezone.utc), + ) + + +def _hosted_dispatch(workflow) -> HostedDispatch: + workflow_id = "33333333-3333-4333-8333-333333333333" + version_id = "44444444-4444-4444-8444-444444444444" + payload_raw = dispatch_payload( + workflow, + workflow_id=workflow_id, + bundle={ + "version_id": version_id, + "content_digest": workflow.manifest.content_digest, + "url": "https://invalid.example/never-fetched", + }, + ) + payload = RunnerDispatchPayload.model_validate(payload_raw) + artifact_raw = b"{}" + artifact = AdmissionArtifactBytes( + artifact_bytes_base64=b64encode(artifact_raw).decode("ascii"), + artifact_sha256=hashlib.sha256(artifact_raw).hexdigest(), + ) + authority_key = Ed25519PrivateKey.from_private_bytes(bytes(range(32, 64))) + authority_public_key = authority_key.public_key().public_bytes( + serialization.Encoding.Raw, + serialization.PublicFormat.Raw, + ) + return HostedDispatch( + schema_version="openadapt.hosted-runner/v1", + dispatch_id="11111111-1111-4111-8111-111111111111", + dispatch_session_id="12111111-1111-4111-8111-111111111111", + tenant_id="22222222-2222-4222-8222-222222222222", + runner_id="55555555-5555-4555-8555-555555555555", + runner_session_id="66666666-6666-4666-8666-666666666666", + run_id=payload.run_id, + workflow_id=workflow_id, + workflow_version_id=version_id, + execution_authority_id="00000000-0000-4000-8000-000000000008", + execution_authority_sha256="1" * 64, + execution_authority_signer_sha256=delivery_authority_signer_sha256( + authority_public_key + ), + idempotency_key="hosted-dispatch-0001", + lease_token="oal_" + "a" * 64, + lease_expires_at="2099-01-01T00:00:00Z", + product_release_admission=artifact, + workflow_admission=artifact, + managed_delivery_authority_url=( + "https://cloud.example/api/internal/managed-delivery-permit" + ), + delivery_authority_token="b" * 64, + payload=payload, + ) + + +def test_hosted_dispatch_accepts_lowercase_v8_run_id(sealed) -> None: + workflow, _ = sealed + dispatch = _hosted_dispatch(workflow) + run_id = "018f6c0a-4cce-8f47-8d71-c3d63bf1c001" + payload = dispatch.payload.model_copy( + update={ + "run_id": run_id, + "dispatch_binding_sha256": dispatch_binding_sha256( + run_id, dispatch.payload.authorization + ), + } + ) + + parsed = HostedDispatch.model_validate( + dispatch.model_dump(mode="python") | {"run_id": run_id, "payload": payload} + ) + + assert parsed.run_id == run_id + + +def test_registration_refuses_without_protected_runner_origin( + monkeypatch, tmp_path, config +) -> None: + adapter = HostedRunnerAdapter(tmp_path / "ledger.sqlite") + monkeypatch.setattr(hosted, "load_runner_config", lambda *_args, **_kwargs: config) + + with pytest.raises(ValueError, match="protected runner host"): + adapter.registration_request( + runner_config=tmp_path / "runner.toml", + name="runner", + platform="linux", + agent_version="1.0.0", + engine_version="1.33.0", + mode="service", + capabilities=RegisterCapabilities( + backends=("linux",), + attended=False, + effects_substrates=("linux",), + ), + ) + + +def test_protected_runner_origin_is_public_strict_accessor( + monkeypatch, tmp_path, config +) -> None: + adapter = HostedRunnerAdapter(tmp_path / "ledger.sqlite") + local_config = replace(config, host="https://cloud.example") + monkeypatch.setattr( + hosted, "load_runner_config", lambda *_args, **_kwargs: local_config + ) + + assert adapter.protected_runner_origin(tmp_path / "runner.toml") == ( + "https://cloud.example" + ) + + +def test_runner_renewal_token_has_one_register_only_header_boundary() -> None: + token = "oar_" + "a" * 64 + + assert registration_renewal_headers(None) == {} + assert registration_renewal_headers("") == {} + assert registration_renewal_headers(token) == { + RUNNER_RENEWAL_HEADER: token, + } + for invalid in ("runner-token", "oar_" + "A" * 64, "oar_" + "a" * 63): + with pytest.raises(ValueError, match="renewal credential"): + registration_renewal_headers(invalid) + + for request_type in (RegisterRequest, PollRequest, HostedDispatch, CallbackRequest): + assert "runner_token" not in request_type.model_fields + assert RUNNER_RENEWAL_HEADER not in request_type.model_fields + assert all("renewal" not in name for name in request_type.model_fields) + + +def test_dispatch_param_scalars_round_trip_without_coercion() -> None: + values = { + "text": "1.5", + "enabled": True, + "count": 7, + "ratio": 1.5, + "small": 1e-7, + "large": 1e21, + } + + parsed = DispatchParamsValues.model_validate({"values": values}) + + assert parsed.model_dump(mode="json")["values"] == values + assert type(parsed.values["enabled"]) is bool + assert type(parsed.values["count"]) is int + assert type(parsed.values["ratio"]) is float + + +@pytest.mark.parametrize( + "value", + [None, {}, [], float("nan"), float("inf"), 9_007_199_254_740_993], +) +def test_dispatch_param_scalars_refuse_invalid_values(value: object) -> None: + with pytest.raises(ValueError): + DispatchParamsValues.model_validate({"values": {"value": value}}) + + +@pytest.mark.parametrize( + "name", + ["\ue000", "\U0001f600", "has-dash", "a" * 129], +) +def test_dispatch_param_names_use_shared_ascii_grammar(name: str) -> None: + with pytest.raises(ValueError, match="parameter name"): + DispatchParamsValues.model_validate({"values": {name: "value"}}) + + +def test_private_params_file_preserves_scalar_types(tmp_path) -> None: + values = {"text": "false", "enabled": False, "count": 0, "ratio": 1.5} + + path = HostedRunnerAdapter._write_params(tmp_path / "params.json", values) + + assert path is not None + assert json.loads(path.read_bytes()) == values + if os.name != "nt": + assert path.stat().st_mode & 0o777 == 0o600 + + +def test_scalar_dispatch_to_gui_boundary_preserves_exact_types(tmp_path) -> None: + values = { + "string_int": "7", + "string_float": "1.5", + "string_bool": "true", + "bool_true": True, + "bool_false": False, + "whole": 7, + "float": 1.5, + "small": 1e-7, + "mid": 1e20, + "large": 1e21, + } + kinds = { + "string_int": ParamKind.STRING, + "string_float": ParamKind.STRING, + "string_bool": ParamKind.STRING, + "bool_true": ParamKind.BOOLEAN, + "bool_false": ParamKind.BOOLEAN, + "whole": ParamKind.NUMBER, + "float": ParamKind.NUMBER, + "small": ParamKind.NUMBER, + "mid": ParamKind.NUMBER, + "large": ParamKind.NUMBER, + } + workflow = Workflow( + name="scalar-path", + param_specs={ + name: ParamSpec(name=name, type=kind, required=True) + for name, kind in kinds.items() + }, + ) + + wire = DispatchParamsValues.model_validate({"values": values}) + admitted = resolve_admitted_params(workflow, dict(wire.values), inline=True) + expected = ( + b'{"params":{"bool_false":false,"bool_true":true,"float":1.5,' + b'"large":1e+21,"mid":100000000000000000000,"small":1e-7,' + b'"string_bool":"true","string_float":"1.5","string_int":"7",' + b'"whole":7},"worklists":{}}' + ) + params_path = HostedRunnerAdapter._write_params(tmp_path / "params.json", admitted) + assert params_path is not None + child_params = _replay_params(None, str(params_path)) + + assert runtime_inputs_bytes(workflow, admitted, None) == expected + assert child_params == values + assert {name: type(value) for name, value in child_params.items()} == { + name: type(value) for name, value in values.items() + } + assert { + name: runtime_param_text(value) for name, value in child_params.items() + } == { + "string_int": "7", + "string_float": "1.5", + "string_bool": "true", + "bool_true": "true", + "bool_false": "false", + "whole": "7", + "float": "1.5", + "small": "1e-7", + "mid": "100000000000000000000", + "large": "1e+21", + } + + +def _prepared_adapter(monkeypatch, tmp_path, config, workflow, runner): + config = replace(config, host="https://cloud.example") + adapter = HostedRunnerAdapter(tmp_path / "ledger.sqlite", runner=runner) + dispatch = _hosted_dispatch(workflow) + verified = VerifiedDispatch( + payload=dispatch.payload, + bundle=config.bundles[workflow.manifest.content_digest], + profile_path=config.profiles["default"], + params={"visit_date": "2026-07-01"}, + workflow=workflow, + consequential_steps=1, + effect_covered_consequential_steps=1, + ) + monkeypatch.setattr(hosted, "load_runner_config", lambda _, **__: config) + monkeypatch.setattr(adapter, "_verify_product_release", lambda *_: None) + monkeypatch.setattr(adapter, "_load_evidence_private_key", lambda *_: object()) + monkeypatch.setattr( + adapter, + "_verify_workflow_admission", + lambda *_, **__: ({}, b"runtime:\n durable: false\n"), + ) + monkeypatch.setattr(adapter, "_resolve_params", lambda *_: verified.params) + monkeypatch.setattr( + hosted, + "verify_dispatch", + lambda _payload, staged_config, **_kwargs: replace( + verified, + profile_path=staged_config.profiles[dispatch.payload.deployment_profile_id], + ), + ) + + class Guard: + def __init__(self, *_args, **_kwargs): + pass + + def authorization_binding(self, _workflow): + return {} + + monkeypatch.setattr(hosted, "ProductionQualificationGuard", Guard) + return adapter, dispatch + + +def _terminal_delivery_chain( + dispatch: HostedDispatch, + *, + admission_sha256: str, + evidence_identity_sha256: str, + environment_digest: str, + registry_sha256: str, +) -> ProductionDeliveryPermitChain: + authority_key = Ed25519PrivateKey.from_private_bytes(bytes(range(32, 64))) + permit_payload = ProductionDeliveryPermitPayload( + execution_authority_id="00000000-0000-4000-8000-000000000008", + execution_authority_sha256="1" * 64, + permit_id="permit:hosted:1", + run_id=dispatch.run_id, + flow_run_id_sha256=hashlib.sha256(dispatch.run_id.encode("utf-8")).hexdigest(), + run_request_sha256="3" * 64, + action_request_sha256="4" * 64, + admission_artifact_sha256=admission_sha256, + evidence_identity_sha256=evidence_identity_sha256, + environment_digest=environment_digest, + qualification_signer_registry_sha256=registry_sha256, + qualification_signer_registry_revision=7, + qualification_signer_registry_checked_at="2026-08-26T11:59:30Z", + qualification_signer_registry_expires_at="2026-08-28T12:00:00Z", + input_edge_sequence=1, + authority_sequence=0, + issued_at="2026-08-26T12:00:00Z", + ) + permit = sign_production_delivery_permit(permit_payload, authority_key) + receipt_payload = ProductionDeliveryReceiptPayload( + execution_authority_id=permit_payload.execution_authority_id, + permit_id=permit_payload.permit_id, + permit_artifact_sha256=permit.artifact_sha256(), + authenticated_runner_id_sha256=hashlib.sha256( + dispatch.runner_id.encode("utf-8") + ).hexdigest(), + authenticated_session_id_sha256=hashlib.sha256( + dispatch.runner_session_id.encode("utf-8") + ).hexdigest(), + one_use_claim_id="00000000-0000-4000-8000-000000000010", + runtime_delivery_sequence=9, + delivered_at="2026-08-26T12:00:01Z", + ) + receipt = sign_production_delivery_receipt(receipt_payload, authority_key) + return ProductionDeliveryPermitChain.build( + (ProductionDeliveryPermit.build(permit, receipt),) + ) + + +def test_outer_adapter_builds_stores_rereads_and_verifies_terminal_v2( + monkeypatch, tmp_path, sealed +) -> None: + workflow, _ = sealed + dispatch = _hosted_dispatch(workflow) + verified_params = dict(dispatch.payload.params.values) + report = _production_report( + run_id_sha256=hashlib.sha256(dispatch.run_id.encode("utf-8")).hexdigest(), + bundle_content_digest=dispatch.payload.bundle.content_digest, + params=verified_params, + ) + authorization = dispatch.payload.authorization.model_copy( + update={ + "admitted_policy_name": report.governed_policy_name + or dispatch.payload.authorization.admitted_policy_name, + "admitted_policy_contract_sha256": (report.governed_policy_contract_sha256), + "execution_profile": report.execution_profile, + "minimum_effect_tier": report.governed_minimum_effect_tier, + "qualified_effect_requirements": tuple( + report.governed_qualified_effect_requirements + ), + "required_identity_step_ids": tuple(report.required_identity_step_ids), + "approval_source": report.governed_approval_source, + } + ) + payload = dispatch.payload.model_copy( + update={ + "authorization": authorization, + "dispatch_binding_sha256": dispatch_binding_sha256( + dispatch.run_id, authorization + ), + } + ) + dispatch = dispatch.model_copy(update={"payload": payload}) + report = report.model_copy( + update={ + "governed_authorization_id": authorization.authorization_id, + "governed_authorization_created_at": authorization.created_at, + "governed_approval_source": authorization.approval_source, + "governed_policy_name": authorization.admitted_policy_name, + "governed_runtime_inputs_digest": authorization.runtime_inputs_digest, + } + ) + private_key = _private_key() + public_key = private_key.public_key().public_bytes( + serialization.Encoding.Raw, + serialization.PublicFormat.Raw, + ) + admission_sha256 = "5" * 64 + evidence_identity_sha256 = "6" * 64 + environment_digest = "7" * 64 + registry_sha256 = "8" * 64 + qualification_sha256 = "f" * 64 + evidence_identity = SimpleNamespace( + admission_policy_sha256="9" * 64, + artifact_sha256=lambda: evidence_identity_sha256, + ) + runtime = SimpleNamespace( + substrate="web", + artifact_sha256=lambda: "0" * 64, + ) + expected = SimpleNamespace( + bundle_artifact_sha256="b" * 64, + bundle_content_digest=dispatch.payload.bundle.content_digest, + environment_digest=environment_digest, + environment_contract_sha256="a" * 64, + runtime_environment_sha256="b" * 64, + identity_contract_sha256="c" * 64, + effect_contract_sha256="d" * 64, + runtime_validation_id="00000000-0000-4000-8000-000000000006", + runtime_build_identity=runtime, + evidence_runner_signer_sha256=evidence_runner_signer_sha256(public_key), + ) + admission = SimpleNamespace( + payload=SimpleNamespace( + admission_id="00000000-0000-4000-8000-000000000001", + evidence_identity=evidence_identity, + ) + ) + qualification = SimpleNamespace( + qualification_admission_sha256=admission_sha256, + qualification_admission=admission, + expected=expected, + qualification_signer_registry_sha256=registry_sha256, + qualification_signer_registry=SimpleNamespace( + revision=7, + expires_at="2026-08-28T12:00:00Z", + ), + immutable_binding_sha256=lambda: qualification_sha256, + ) + chain = _terminal_delivery_chain( + dispatch, + admission_sha256=admission_sha256, + evidence_identity_sha256=evidence_identity_sha256, + environment_digest=environment_digest, + registry_sha256=registry_sha256, + ) + binding = dispatch.payload.dispatch_binding_sha256 + local_authorization = authorization.model_copy( + update={ + "production_qualification_admission_id": (admission.payload.admission_id), + "production_qualification_admission_sha256": admission_sha256, + "production_qualification_evidence_identity_sha256": ( + evidence_identity_sha256 + ), + "production_qualification_runtime_validation_id": ( + expected.runtime_validation_id + ), + "production_qualification_signer_registry_sha256": registry_sha256, + "production_qualification_signer_registry_revision": 7, + "production_qualification_signer_registry_expires_at": ( + "2026-08-28T12:00:00Z" + ), + "production_qualification_authority_sha256": qualification_sha256, + } + ) + manifest = SimpleNamespace( + delivery_authority_kind="cloud_runner", + remote_delivery_run_id=dispatch.run_id, + managed_dispatch_binding_sha256=binding, + params=verified_params, + governed_authorization=local_authorization, + ) + + class Store: + def __init__(self, _run_dir): + pass + + def read_manifest(self): + return manifest + + class Authority: + def __init__(self, _run_dir, _store): + pass + + def production_delivery_permit_chain(self, **_kwargs): + return chain + + fixed_now = datetime(2026, 8, 26, 12, 0, 2, tzinfo=timezone.utc) + + class FixedDatetime(datetime): + @classmethod + def now(cls, tz=None): + return fixed_now if tz is not None else fixed_now.replace(tzinfo=None) + + monkeypatch.setattr(hosted, "CheckpointStore", Store) + monkeypatch.setattr(hosted, "DurableAuthority", Authority) + monkeypatch.setattr(hosted, "datetime", FixedDatetime) + run_dir = tmp_path / "run" + run_dir.mkdir(mode=0o700) + adapter = HostedRunnerAdapter(tmp_path / "ledger.sqlite") + + mismatched_dir = tmp_path / "mismatched-run" + mismatched_dir.mkdir(mode=0o700) + with pytest.raises(ValueError, match="run report differs"): + adapter._produce_terminal_verification( + dispatch=dispatch, + report=report.model_copy(update={"params": {"visit_date": "wrong"}}), + run_dir=mismatched_dir, + qualification=qualification, + private_key=private_key, + verified_params=verified_params, + dispatch_binding_sha256=binding, + ) + assert not (mismatched_dir / "production-terminal-report.json").exists() + assert not (mismatched_dir / "production-terminal-verification.json").exists() + + storage_failure_dir = tmp_path / "storage-failure-run" + storage_failure_dir.mkdir(mode=0o700) + original_read = adapter._read_private_bytes + + def fail_proof_reread(path, *, maximum_bytes, label): + if label == "production terminal verification": + raise OSError("simulated protected storage failure") + return original_read(path, maximum_bytes=maximum_bytes, label=label) + + monkeypatch.setattr(adapter, "_read_private_bytes", fail_proof_reread) + with pytest.raises(OSError, match="storage failure"): + adapter._produce_terminal_verification( + dispatch=dispatch, + report=report, + run_dir=storage_failure_dir, + qualification=qualification, + private_key=private_key, + verified_params=verified_params, + dispatch_binding_sha256=binding, + ) + assert not (storage_failure_dir / "production-terminal-report.json").exists() + assert not (storage_failure_dir / "production-terminal-verification.json").exists() + monkeypatch.setattr(adapter, "_read_private_bytes", original_read) + + proof, report_sha256 = adapter._produce_terminal_verification( + dispatch=dispatch, + report=report, + run_dir=run_dir, + qualification=qualification, + private_key=private_key, + verified_params=verified_params, + dispatch_binding_sha256=binding, + ) + + report_path = run_dir / "production-terminal-report.json" + proof_path = run_dir / "production-terminal-verification.json" + assert hashlib.sha256(report_path.read_bytes()).hexdigest() == report_sha256 + assert proof_path.read_bytes() == hosted.canonical_json(proof) + assert hashlib.sha256(proof_path.read_bytes()).hexdigest() == ( + proof.artifact_sha256() + ) + if os.name != "nt": + assert report_path.stat().st_mode & 0o777 == 0o600 + assert proof_path.stat().st_mode & 0o777 == 0o600 + assert "2026-07-01" in report_path.read_text(encoding="utf-8") + assert "2026-07-01" not in proof_path.read_text(encoding="utf-8") + + +def test_terminal_admission_is_revalidated_after_child_execution( + monkeypatch, tmp_path, config, sealed +) -> None: + workflow, _ = sealed + report = _production_report() + calls = 0 + + def runner(_argv, _run_dir, _child_env): + nonlocal calls + calls += 1 + return ManagedExecution( + returncode=0, + report_bytes=report.model_dump_json().encode(), + ) + + adapter, dispatch = _prepared_adapter( + monkeypatch, tmp_path, config, workflow, runner + ) + release_checks = 0 + + def verify_release(*_args): + nonlocal release_checks + release_checks += 1 + if release_checks == 2: + raise ValueError("release admission was revoked during execution") + + monkeypatch.setattr(adapter, "_verify_product_release", verify_release) + monkeypatch.setattr( + adapter, + "_produce_terminal_verification", + lambda **_kwargs: pytest.fail("revoked run must not produce terminal proof"), + ) + authority = DeliveryAuthority( + dispatch.managed_delivery_authority_url, + dispatch.delivery_authority_token, + ) + + first = adapter.execute( + dispatch, + runner_config=tmp_path / "runner.toml", + run_dir=tmp_path / "run", + authority=authority, + ) + + assert first.outcome is TransactionOutcome.RECONCILIATION_REQUIRED + assert first.started is True + assert first.terminal_verification is None + assert calls == 1 + assert release_checks == 2 + + +@pytest.mark.parametrize( + ("fault", "execution"), + [ + ( + "backend_response_lost", + RuntimeError("backend response lost after possible actuation"), + ), + ( + "delivery_acknowledgment_response_lost", + RuntimeError("delivery acknowledgment response lost after backend call"), + ), + ( + "receipt_unavailable", + ManagedExecution(returncode=1, report_bytes=None), + ), + ( + "malformed_terminal_report", + ManagedExecution(returncode=0, report_bytes=b"not-json"), + ), + ], + ids=( + "backend-response-lost", + "delivery-acknowledgment-response-lost", + "receipt-unavailable", + "malformed-terminal-report", + ), +) +def test_hosted_uncertain_delivery_fault_never_replays( + monkeypatch, tmp_path, config, sealed, fault, execution +) -> None: + workflow, _ = sealed + calls = 0 + seen_child_env = None + seen_argv = None + + def runner(argv, _run_dir, child_env): + nonlocal calls + nonlocal seen_child_env + nonlocal seen_argv + calls += 1 + seen_child_env = child_env + seen_argv = argv + if isinstance(execution, Exception): + raise execution + return execution + + adapter, dispatch = _prepared_adapter( + monkeypatch, tmp_path, config, workflow, runner + ) + authority = DeliveryAuthority( + dispatch.managed_delivery_authority_url, + dispatch.delivery_authority_token, + ) + first = adapter.execute( + dispatch, + runner_config=tmp_path / "runner.toml", + run_dir=tmp_path / "run", + authority=authority, + ) + second = adapter.execute( + dispatch, + runner_config=tmp_path / "runner.toml", + run_dir=tmp_path / "run-again", + authority=authority, + ) + + assert first.outcome is TransactionOutcome.RECONCILIATION_REQUIRED + assert first.started is True + assert first.uncertain_delivery is True + assert second.outcome is TransactionOutcome.RECONCILIATION_REQUIRED + assert second.started is True + assert second.uncertain_delivery is True + assert calls == 1 + assert seen_child_env[REMOTE_DISPATCH_SESSION_ID_ENV] == ( + dispatch.dispatch_session_id + ) + profile_path = Path(seen_argv[seen_argv.index("--config") + 1]) + assert profile_path == tmp_path / "run" / "deployment.yaml" + assert profile_path.read_bytes() == b"runtime:\n durable: false\n" + if os.name != "nt": + assert profile_path.stat().st_mode & 0o777 == 0o600 + assert fault in { + "backend_response_lost", + "delivery_acknowledgment_response_lost", + "receipt_unavailable", + "malformed_terminal_report", + } + + +def test_params_reference_resolves_only_from_protected_local_root( + monkeypatch, tmp_path, config, sealed +) -> None: + workflow, _ = sealed + workflow.param_specs = { + "visit_date": ParamSpec( + name="visit_date", + type=ParamKind.DATE, + required=True, + ) + } + dispatch = _hosted_dispatch(workflow) + expected_digest = dispatch.payload.authorization.runtime_inputs_digest + dispatch = dispatch.model_copy( + update={ + "payload": dispatch.payload.model_copy( + update={ + "params": DispatchParamsRef( + ref="records/run.json", + expected_digest=expected_digest, + ) + } + ) + } + ) + root = tmp_path / "params" + nested = root / "records" + nested.mkdir(parents=True, mode=0o700) + root.chmod(0o700) + nested.chmod(0o700) + ref_file = nested / "run.json" + ref_file.write_text(json.dumps({"visit_date": "2026-07-01"}), encoding="utf-8") + ref_file.chmod(0o600) + local_config = replace( + config, + host="https://cloud.example", + params_ref_root=root, + ) + monkeypatch.setattr(hosted.Workflow, "load", lambda *_: workflow) + + adapter = HostedRunnerAdapter(tmp_path / "resolver-ledger.sqlite") + assert adapter._resolve_params(dispatch, local_config) == { + "visit_date": "2026-07-01" + } + + traversing = dispatch.model_copy( + update={ + "payload": dispatch.payload.model_copy( + update={ + "params": DispatchParamsRef( + ref="../outside.json", + expected_digest=expected_digest, + ) + } + ) + } + ) + with pytest.raises(ValueError, match="safe local path"): + adapter._resolve_params(traversing, local_config) + + +@pytest.mark.skipif(os.name == "nt", reason="POSIX symlink contract") +def test_params_reference_refuses_symlink( + monkeypatch, tmp_path, config, sealed +) -> None: + workflow, _ = sealed + workflow.param_specs = { + "visit_date": ParamSpec(name="visit_date", type=ParamKind.DATE) + } + dispatch = _hosted_dispatch(workflow) + expected_digest = dispatch.payload.authorization.runtime_inputs_digest + root = tmp_path / "params" + root.mkdir(mode=0o700) + target = tmp_path / "target.json" + target.write_text(json.dumps({"visit_date": "2026-07-01"}), encoding="utf-8") + target.chmod(0o600) + (root / "run.json").symlink_to(target) + dispatch = dispatch.model_copy( + update={ + "payload": dispatch.payload.model_copy( + update={ + "params": DispatchParamsRef( + ref="run.json", + expected_digest=expected_digest, + ) + } + ) + } + ) + monkeypatch.setattr(hosted.Workflow, "load", lambda *_: workflow) + adapter = HostedRunnerAdapter(tmp_path / "resolver-ledger.sqlite") + + with pytest.raises(ValueError, match="private regular file"): + adapter._resolve_params(dispatch, replace(config, params_ref_root=root)) + + +def test_params_reference_digest_mismatch_refuses_before_managed_runner( + monkeypatch, tmp_path, config, sealed +) -> None: + workflow, _ = sealed + workflow.param_specs = { + "visit_date": ParamSpec( + name="visit_date", + type=ParamKind.DATE, + required=True, + ) + } + dispatch = _hosted_dispatch(workflow) + root = tmp_path / "params" + root.mkdir(mode=0o700) + ref_file = root / "run.json" + ref_file.write_text(json.dumps({"visit_date": "2026-07-01"}), encoding="utf-8") + ref_file.chmod(0o600) + dispatch = dispatch.model_copy( + update={ + "payload": dispatch.payload.model_copy( + update={ + "params": DispatchParamsRef( + ref="run.json", + expected_digest="f" * 64, + ) + } + ) + } + ) + local_config = replace( + config, + host="https://cloud.example", + params_ref_root=root, + ) + calls = 0 + + def runner(*_args): + nonlocal calls + calls += 1 + raise AssertionError("managed runner must not start") + + adapter = HostedRunnerAdapter(tmp_path / "ledger.sqlite", runner=runner) + monkeypatch.setattr( + hosted, "load_runner_config", lambda *_args, **_kwargs: local_config + ) + monkeypatch.setattr(adapter, "_verify_product_release", lambda *_: None) + monkeypatch.setattr(adapter, "_load_evidence_private_key", lambda *_: object()) + monkeypatch.setattr( + adapter, + "_verify_workflow_admission", + lambda *_, **__: ({}, b"runtime:\n durable: false\n"), + ) + monkeypatch.setattr(hosted.Workflow, "load", lambda *_: workflow) + authority = DeliveryAuthority( + dispatch.managed_delivery_authority_url, + dispatch.delivery_authority_token, + ) + + result = adapter.execute( + dispatch, + runner_config=tmp_path / "runner.toml", + run_dir=tmp_path / "run", + authority=authority, + ) + + assert result.outcome == "REJECTED_POLICY" + assert result.code == "runtime_inputs_mismatch" + assert result.started is False + assert result.uncertain_delivery is False + assert calls == 0 + + +@pytest.mark.parametrize("manifest_kind", ["missing", "malformed", "public", "symlink"]) +def test_untrusted_runner_manifest_refuses_before_managed_runner( + tmp_path, sealed, manifest_kind +) -> None: + if manifest_kind == "symlink" and os.name == "nt": + pytest.skip("POSIX symlink contract") + workflow, _ = sealed + dispatch = _hosted_dispatch(workflow) + manifest = tmp_path / "runner.toml" + if manifest_kind == "malformed": + manifest.write_text("[runner\n", encoding="utf-8") + manifest.chmod(0o600) + elif manifest_kind == "public": + manifest.write_text("[runner]\nname = 'runner'\n", encoding="utf-8") + manifest.chmod(0o644) + elif manifest_kind == "symlink": + target = tmp_path / "target.toml" + target.write_text("[runner]\nname = 'runner'\n", encoding="utf-8") + target.chmod(0o600) + manifest.symlink_to(target) + calls = 0 + + def runner(*_args): + nonlocal calls + calls += 1 + raise AssertionError("managed runner must not start") + + adapter = HostedRunnerAdapter(tmp_path / "ledger.sqlite", runner=runner) + authority = DeliveryAuthority( + dispatch.managed_delivery_authority_url, + dispatch.delivery_authority_token, + ) + + result = adapter.execute( + dispatch, + runner_config=manifest, + run_dir=tmp_path / "run", + authority=authority, + ) + + assert result.outcome == "REJECTED_POLICY" + assert result.code == "hosted_admission_refused" + assert result.detail.startswith("prestart_") + assert result.started is False + assert result.uncertain_delivery is False + assert calls == 0 + + +@pytest.mark.parametrize( + "runner_host", + [ + None, + "http://cloud.example", + "https://cloud.example/", + "https://different.example", + ], +) +def test_protected_runner_host_binds_delivery_authority_origin( + monkeypatch, tmp_path, config, sealed, runner_host +) -> None: + workflow, _ = sealed + dispatch = _hosted_dispatch(workflow) + calls = 0 + + def runner(*_args): + nonlocal calls + calls += 1 + raise AssertionError("managed runner must not start") + + adapter = HostedRunnerAdapter(tmp_path / "ledger.sqlite", runner=runner) + local_config = replace(config, host=runner_host) + monkeypatch.setattr( + hosted, "load_runner_config", lambda *_args, **_kwargs: local_config + ) + authority = DeliveryAuthority( + dispatch.managed_delivery_authority_url, + dispatch.delivery_authority_token, + ) + + result = adapter.execute( + dispatch, + runner_config=tmp_path / "runner.toml", + run_dir=tmp_path / "run", + authority=authority, + ) + + assert result.outcome == "REJECTED_POLICY" + assert result.started is False + assert result.detail == "prestart_ValueError" + assert calls == 0 + + +def test_protected_profile_mutation_refuses_before_managed_runner( + monkeypatch, tmp_path, config, sealed +) -> None: + workflow, _ = sealed + dispatch = _hosted_dispatch(workflow) + config = replace(config, host="https://cloud.example") + profile_path = config.profiles["default"] + profile_path.chmod(0o600) + calls = 0 + + def runner(*_args): + nonlocal calls + calls += 1 + raise AssertionError("managed runner must not start") + + adapter = HostedRunnerAdapter(tmp_path / "ledger.sqlite", runner=runner) + monkeypatch.setattr(hosted, "load_runner_config", lambda *_args, **_kwargs: config) + monkeypatch.setattr(adapter, "_verify_product_release", lambda *_: None) + monkeypatch.setattr(adapter, "_load_evidence_private_key", lambda *_: object()) + + def verify_profile(*_args, **_kwargs): + raw = adapter._read_private_bytes( + profile_path, + maximum_bytes=1024 * 1024, + label="hosted deployment profile", + ) + return {}, raw + + monkeypatch.setattr(adapter, "_verify_workflow_admission", verify_profile) + real_read = os.read + changed = False + + def mutating_read(descriptor, count): + nonlocal changed + chunk = real_read(descriptor, count) + if not changed: + changed = True + profile_path.write_bytes(chunk + b"# changed\n") + profile_path.chmod(0o600) + return chunk + + monkeypatch.setattr(hosted.os, "read", mutating_read) + authority = DeliveryAuthority( + dispatch.managed_delivery_authority_url, + dispatch.delivery_authority_token, + ) + + result = adapter.execute( + dispatch, + runner_config=tmp_path / "runner.toml", + run_dir=tmp_path / "run", + authority=authority, + ) + + assert result.outcome == "REJECTED_POLICY" + assert result.started is False + assert result.detail == "prestart_ValueError" + assert calls == 0 + + +def test_parsed_refusal_callback_contains_closed_terminal(tmp_path, sealed) -> None: + workflow, _ = sealed + dispatch = _hosted_dispatch(workflow) + adapter = HostedRunnerAdapter(tmp_path / "ledger.sqlite") + refusal = adapter._refusal(dispatch, "hosted_admission_refused", "refused") + + callback = adapter.callback_request(dispatch, refusal) + + terminal = callback.events[-1] + assert terminal["schema_version"] == "openadapt.hosted-runner-terminal/v1" + assert terminal["outcome"] == "REJECTED_POLICY" + assert terminal["started"] is False + assert terminal["uncertain_delivery"] is False + + +def test_recovery_callback_retains_exact_terminal_v2_envelope(tmp_path, sealed) -> None: + workflow, _ = sealed + dispatch = _hosted_dispatch(workflow) + adapter = HostedRunnerAdapter(tmp_path / "ledger.sqlite") + proof = sign_production_terminal_verification(_payload(), _private_key()) + binding = adapter.recovery_binding(dispatch).model_copy( + update={"run_id": proof.payload.run_id} + ) + result = HostedRunResult( + dispatch_id=dispatch.dispatch_id, + run_id=binding.run_id, + outcome=TransactionOutcome.VERIFIED, + evidence_batch=(), + terminal_verification=proof, + started=True, + uncertain_delivery=False, + report_sha256=proof.payload.run_report_sha256, + ) + + callback = adapter.callback_request(binding, result) + terminal = HostedTerminalEvent.model_validate(callback.events[-1]) + assert terminal.terminal_verification_artifact_bytes_base64 is not None + raw = b64decode(terminal.terminal_verification_artifact_bytes_base64, validate=True) + assert hashlib.sha256(raw).hexdigest() == ( + terminal.terminal_verification_artifact_sha256 + ) + decoded = json.loads(raw) + assert decoded["payload"]["schema_version"] == ( + "openadapt.production-terminal-verification/v2" + ) + assert "params" not in decoded["payload"] + assert "report" not in decoded["payload"] + assert callback.runner_session_id == dispatch.runner_session_id + assert callback.workflow_admission_sha256 == ( + dispatch.workflow_admission.artifact_sha256 + ) + + +@pytest.mark.parametrize( + ("outcome", "payload_factory", "uncertain_delivery"), + [ + ( + TransactionOutcome.HALTED_BEFORE_EFFECT, + _halted_payload, + False, + ), + ( + TransactionOutcome.RECONCILIATION_REQUIRED, + _reconciliation_payload, + True, + ), + ], +) +def test_recovery_callback_retains_signed_non_success_terminal_v2( + tmp_path, + sealed, + outcome, + payload_factory, + uncertain_delivery, +) -> None: + workflow, _ = sealed + dispatch = _hosted_dispatch(workflow) + adapter = HostedRunnerAdapter(tmp_path / "ledger.sqlite") + proof = sign_production_terminal_verification(payload_factory(), _private_key()) + binding = adapter.recovery_binding(dispatch).model_copy( + update={"run_id": proof.payload.run_id} + ) + result = HostedRunResult( + dispatch_id=dispatch.dispatch_id, + run_id=binding.run_id, + outcome=outcome, + evidence_batch=(), + terminal_verification=proof, + started=True, + uncertain_delivery=uncertain_delivery, + report_sha256=proof.payload.run_report_sha256, + ) + + callback = adapter.callback_request(binding, result) + terminal = HostedTerminalEvent.model_validate(callback.events[-1]) + + assert terminal.outcome == outcome.value + assert terminal.uncertain_delivery is uncertain_delivery + assert terminal.terminal_verification_artifact_bytes_base64 is not None + assert terminal.terminal_verification_artifact_sha256 == proof.artifact_sha256() + + +def test_safe_halt_callback_requires_signed_terminal_proof() -> None: + with pytest.raises(ValueError, match="requires exact terminal verification"): + HostedTerminalEvent( + run_id=_halted_payload().run_id, + outcome="HALTED_BEFORE_EFFECT", + report_sha256="b" * 64, + started=True, + uncertain_delivery=False, + ) + + +def test_callback_refuses_terminal_proof_for_a_different_run(tmp_path, sealed) -> None: + workflow, _ = sealed + dispatch = _hosted_dispatch(workflow) + adapter = HostedRunnerAdapter(tmp_path / "ledger.sqlite") + proof = sign_production_terminal_verification(_payload(), _private_key()) + result = HostedRunResult.model_construct( + dispatch_id=dispatch.dispatch_id, + run_id=dispatch.run_id, + outcome=TransactionOutcome.VERIFIED, + evidence_batch=(), + terminal_verification=proof, + started=True, + uncertain_delivery=False, + report_sha256=proof.payload.run_report_sha256, + ) + + with pytest.raises(ValueError, match="different run report"): + adapter.callback_request(adapter.recovery_binding(dispatch), result) diff --git a/tests/test_linux_backend.py b/tests/test_linux_backend.py index 0b451b32..7e3eade7 100644 --- a/tests/test_linux_backend.py +++ b/tests/test_linux_backend.py @@ -8,7 +8,9 @@ from openadapt_flow.backend import ( Backend, + DisplayGeometry, ExecutionContextIdentityBackend, + FreshActuationRequired, GuardedCoordinateActionBackend, GuardedKeyboardActionBackend, IdentityBackend, @@ -69,6 +71,7 @@ def __init__( candidates: list[LinuxElement] | None = None, active: bool = True, truncated: bool = False, + displays: tuple[DisplayGeometry, ...] | None = None, ) -> None: self._session_type = session_type self._portal_ready = portal_ready @@ -85,6 +88,13 @@ def __init__( self.native_succeeds = True self.replace_succeeds = True self.physical_succeeds = True + self.displays = displays or ( + DisplayGeometry( + "fake-primary", + (-10_000.0, -10_000.0, 20_000.0, 20_000.0), + (1.0, 1.0), + ), + ) @property def session_type(self) -> str: @@ -117,6 +127,9 @@ def capture_window(self, window): image.save(output, format="PNG") return output.getvalue(), image.width, image.height + def display_topology(self): + return self.displays + def element_at_point(self, window, x, y): self.calls.append(("element-at", window.native_id, x, y)) return self.text_at_point @@ -195,6 +208,92 @@ def test_guarded_keyboard_refuses_focus_change_before_delivery() -> None: assert not any(call[0] == "replace" for call in client.calls) +def test_atomic_observation_refuses_linux_resize_before_first_input_edge() -> None: + client = FakeLinuxClient(candidates=[TEXT_ELEMENT]) + target = backend(client, allow_physical_input=True) + observation = target.guarded_keyboard_observation() + target.arm_guarded_keyboard(300, 200) + target.bind_input_observation(observation) + resized = LinuxWindow( + TARGET_WINDOW.native_id, + TARGET_WINDOW.app_name, + TARGET_WINDOW.title, + TARGET_WINDOW.pid, + (100, 200, 800, 600), + ) + client.windows = [resized] + + with pytest.raises(FreshActuationRequired) as error: + target.type_text_guarded( + "must not land", + expected_frame_sha256=observation.frame_sha256, + ) + + assert error.value.expected_geometry_epoch == observation.geometry_epoch + assert error.value.observed_geometry_epoch != observation.geometry_epoch + assert not any(call[0] == "replace" for call in client.calls) + + +def test_atomic_observation_refuses_cross_monitor_scale_change() -> None: + displays = ( + DisplayGeometry("left", (-1000.0, 0.0, 1000.0, 800.0), (1.0, 1.0)), + DisplayGeometry("right", (0.0, 0.0, 1200.0, 900.0), (2.0, 2.0)), + ) + initial = LinuxWindow( + TARGET_WINDOW.native_id, + TARGET_WINDOW.app_name, + TARGET_WINDOW.title, + TARGET_WINDOW.pid, + (-800, 100, 640, 480), + ) + client = FakeLinuxClient( + windows=[initial], + candidates=[TEXT_ELEMENT], + displays=displays, + ) + client.focused = LinuxElement( + TEXT_ELEMENT.native_id, + TEXT_ELEMENT.accessible_id, + TEXT_ELEMENT.role, + TEXT_ELEMENT.name, + TEXT_ELEMENT.app_name, + TEXT_ELEMENT.window_title, + TEXT_ELEMENT.pid, + (-700, 200, 580, 380), + TEXT_ELEMENT.supported_operations, + ) + target = LinuxBackend( + app="gedit", + window_title="oa-trial.txt", + client=client, + allow_physical_input=True, + ) + observation = target.guarded_keyboard_observation() + target.arm_guarded_keyboard(300, 200) + target.bind_input_observation(observation) + moved = LinuxWindow( + initial.native_id, + initial.app_name, + initial.title, + initial.pid, + (100, 100, 640, 480), + ) + client.windows = [moved] + + with pytest.raises(FreshActuationRequired) as error: + target.type_text_guarded( + "must not land", + expected_frame_sha256=observation.frame_sha256, + ) + + assert error.value.expected_observation is observation + assert error.value.observed_observation is not None + assert error.value.expected_observation.display_id == "left" + assert error.value.observed_observation.display_id == "right" + assert error.value.observed_observation.display_scale == (2.0, 2.0) + assert not any(call[0] == "replace" for call in client.calls) + + def test_execution_context_identity_is_live_and_title_free( monkeypatch: pytest.MonkeyPatch, ) -> None: diff --git a/tests/test_macos_client.py b/tests/test_macos_client.py index 11d60614..2ba0fce5 100644 --- a/tests/test_macos_client.py +++ b/tests/test_macos_client.py @@ -14,8 +14,8 @@ from openadapt_flow.backends.remote_display import MacWindowClient, WindowInfo -def _window() -> WindowInfo: - return WindowInfo(41, "TextEdit", "oa-trial.txt", 9001, (0, 0, 400, 300)) +def _window(*, owner: str = "TextEdit", title: str = "oa-trial.txt") -> WindowInfo: + return WindowInfo(41, owner, title, 9001, (0, 0, 400, 300)) def test_find_windows_requires_exact_case_insensitive_identity(monkeypatch) -> None: @@ -240,7 +240,9 @@ def test_window_id_at_point_includes_nonzero_layer_overlay(monkeypatch) -> None: def _ax_module( *, focused_title: str = "oa-trial.txt", + target_title: str = "oa-trial.txt", duplicate: bool = False, + duplicate_title: str | None = None, focused_window_matches: bool = True, is_main: bool = True, selected_settable: bool = True, @@ -265,8 +267,8 @@ def _ax_module( other = object() values = { (app, "windows"): [target, duplicate_target] if duplicate else [target], - (target, "title"): "oa-trial.txt", - (duplicate_target, "title"): "oa-trial.txt", + (target, "title"): target_title, + (duplicate_target, "title"): duplicate_title or target_title, (app, "focused-window"): target if focused_window_matches else other, (target, "main"): is_main, (app, "focused-element"): focused, @@ -321,6 +323,105 @@ def test_raise_window_selects_exact_ax_document_and_requests_key_state( ] +@pytest.mark.parametrize( + ("owner", "suffix"), + [ + ("Google Chrome", "Google Chrome"), + ("Google Chrome for Testing", "Google Chrome for Testing"), + ], +) +def test_raise_window_accepts_only_corresponding_known_chrome_ax_suffix( + monkeypatch, owner: str, suffix: str +) -> None: + cg_title = "Example Order Management" + module, app, target, _focused, calls = _ax_module( + target_title=f"{cg_title} - {suffix}" + ) + monkeypatch.setitem(sys.modules, "ApplicationServices", module) + + assert MacWindowClient().raise_window(_window(owner=owner, title=cg_title)) is True + assert calls == [ + ("set", app, "focused-window", target), + ("set", target, "main", True), + ("set", target, "focused", True), + ("action", target, "raise"), + ] + + +@pytest.mark.parametrize( + ("owner", "cg_title", "ax_title", "expected"), + [ + ( + "Google Chrome", + "Example Order Management", + "Example Order Management", + True, + ), + ( + "Google Chrome", + "Example Order Management", + "Example Order Management - Google Chrome", + True, + ), + ( + "Google Chrome for Testing", + "Example Order Management", + "Example Order Management - Google Chrome for Testing", + True, + ), + ( + "Google Chrome", + "Example Order Management", + "Different Application", + False, + ), + ( + "Google Chrome", + "Example Order Management", + "Example - Google Chrome Order Management", + False, + ), + ( + "TextEdit", + "Example Order Management", + "Example Order Management - Google Chrome", + False, + ), + ], +) +def test_exact_window_focused_main_applies_only_known_chrome_suffixes( + monkeypatch, + owner: str, + cg_title: str, + ax_title: str, + expected: bool, +) -> None: + module, _app, _target, _focused, _calls = _ax_module(target_title=ax_title) + monkeypatch.setitem(sys.modules, "ApplicationServices", module) + + assert ( + MacWindowClient().exact_window_focused_main( + _window(owner=owner, title=cg_title) + ) + is expected + ) + + +def test_chrome_suffix_matching_preserves_ax_ambiguity(monkeypatch) -> None: + cg_title = "Example Order Management" + module, _app, _target, _focused, calls = _ax_module( + target_title=cg_title, + duplicate=True, + duplicate_title=f"{cg_title} - Google Chrome", + ) + monkeypatch.setitem(sys.modules, "ApplicationServices", module) + window = _window(owner="Google Chrome", title=cg_title) + + assert MacWindowClient().raise_window(window) is False + assert MacWindowClient().exact_window_focused_main(window) is False + assert calls == [] + + def test_raise_window_refuses_duplicate_exact_ax_titles(monkeypatch) -> None: module, _app, _target, _focused, calls = _ax_module(duplicate=True) monkeypatch.setitem(sys.modules, "ApplicationServices", module) diff --git a/tests/test_native_source_geometry.py b/tests/test_native_source_geometry.py new file mode 100644 index 00000000..80b9e9a9 --- /dev/null +++ b/tests/test_native_source_geometry.py @@ -0,0 +1,374 @@ +"""Sealed Capture v2 to Flow native source-geometry contract.""" + +from __future__ import annotations + +import hashlib +import io +import json +from pathlib import Path + +import pytest +from openadapt_capture.db import create_db, crud +from openadapt_capture.db.models import ActionEvent, Recording, Screenshot, WindowEvent +from openadapt_capture.events import window_geometry_epoch_sha256 +from openadapt_capture.terminal import seal_capture +from PIL import Image, ImageDraw + +from openadapt_flow.adapters.capture import convert_capture +from openadapt_flow.compiler.compile import compile_recording +from openadapt_flow.source_geometry import ( + NATIVE_SOURCE_GEOMETRY_BINDING_FIELDS, + NativeSourceGeometry, + native_source_geometry_sha256, +) + +T0 = 100_000.0 +FRAME_SIZE = (320, 200) +FRAME_ORDINAL = 1 +ACTION_ORDINAL = 3 +AFTER_FRAME_ORDINAL = 4 + + +def _png(color: tuple[int, int, int], label: str) -> bytes: + image = Image.new("RGB", FRAME_SIZE, color) + draw = ImageDraw.Draw(image) + draw.rectangle((100, 70, 220, 130), outline=(0, 0, 0), width=3) + draw.text((120, 92), label, fill=(0, 0, 0)) + output = io.BytesIO() + image.save(output, format="PNG") + return output.getvalue() + + +def _geometry_state() -> dict: + payload = { + "schema_version": "openadapt.capture.window-scoped/v2", + "window_capture": True, + "window_id": "42", + "owner": "FixtureApp", + "pid": 4242, + "process_start_time": 99_000.0, + "coordinate_source": "test-screen-points", + "geometry_generation": 1, + "display_topology_sha256": "a" * 64, + "bounds": [10.0, 20.0, 320.0, 200.0], + "scale": 1.0, + "scale_x": 1.0, + "scale_y": 1.0, + "viewport": list(FRAME_SIZE), + "source_viewport": list(FRAME_SIZE), + "content_rect": [0, 0, *FRAME_SIZE], + "fit_scale": 1.0, + "on_screen": True, + } + payload["geometry_epoch_sha256"] = window_geometry_epoch_sha256(payload) + return payload + + +def _capture_config(state: dict) -> dict: + return { + "capture_window": { + **state, + "target": {"owner": "FixtureApp", "title": None}, + "title": "Fixture Window", + "initial_bounds": state["bounds"], + "coordinate_space": "window_pixels", + } + } + + +def _make_capture(tmp_path: Path, *, sealed: bool = True) -> Path: + capture_dir = tmp_path / "capture" + capture_dir.mkdir() + before = _png((240, 240, 240), "Run") + after = _png((210, 240, 210), "Done") + state = _geometry_state() + + engine, session_factory = create_db(str(capture_dir / "recording.db")) + session = session_factory() + try: + recording = Recording( + timestamp=T0, + monitor_width=FRAME_SIZE[0], + monitor_height=FRAME_SIZE[1], + platform="win32", + task_description="native geometry fixture", + double_click_interval_seconds=0.5, + double_click_distance_pixels=5.0, + config=_capture_config(state), + ) + session.add(recording) + session.flush() + for timestamp, ordinal, png in ( + (T0 + 1.0, FRAME_ORDINAL, before), + (T0 + 1.3, AFTER_FRAME_ORDINAL, after), + ): + session.add( + Screenshot( + recording_id=recording.id, + recording_timestamp=T0, + timestamp=timestamp, + source_ordinal=ordinal, + png_data=png, + png_sha256=hashlib.sha256(png).hexdigest(), + ) + ) + session.add( + WindowEvent( + recording_id=recording.id, + timestamp=timestamp, + source_ordinal=ordinal, + title="Fixture Window", + left=10, + top=20, + width=320, + height=200, + window_id="42", + state=state, + ) + ) + for timestamp, ordinal, pressed in ( + (T0 + 1.1, 2, True), + (T0 + 1.2, ACTION_ORDINAL, False), + ): + session.add( + ActionEvent( + recording_id=recording.id, + name="click", + timestamp=timestamp, + source_ordinal=ordinal, + mouse_x=160.0, + mouse_y=100.0, + mouse_button_name="left", + mouse_pressed=pressed, + screenshot_timestamp=T0 + 1.0, + screenshot_source_ordinal=FRAME_ORDINAL, + after_screenshot_timestamp=T0 + 1.3, + after_screenshot_source_ordinal=AFTER_FRAME_ORDINAL, + window_event_timestamp=T0 + 1.0, + window_event_source_ordinal=FRAME_ORDINAL, + after_window_event_timestamp=T0 + 1.3, + after_window_event_source_ordinal=AFTER_FRAME_ORDINAL, + window_geometry_generation=1, + after_window_geometry_generation=1, + ) + ) + session.commit() + crud.post_process_events(session, recording) + finally: + session.close() + engine.dispose() + + if sealed: + seal_capture( + capture_dir, + session_id="fixture-session", + process_started_at=T0 - 1, + capture_started_at=T0, + capture_ended_at=T0 + 2, + event_counts={ + "action": 2, + "screen": 2, + "window": 2, + "browser": 0, + "video": 0, + }, + last_source_ordinal=AFTER_FRAME_ORDINAL, + ) + return capture_dir + + +def _stamp_surface(recording_dir: Path, surface: str) -> None: + path = recording_dir / "meta.json" + meta = json.loads(path.read_text()) + meta["surface"] = surface + path.write_text(json.dumps(meta, indent=2)) + + +def _append_second_event(recording_dir: Path) -> tuple[dict, dict]: + events_path = recording_dir / "events.jsonl" + first = json.loads(events_path.read_text().splitlines()[0]) + second = json.loads(json.dumps(first)) + second["i"] = 1 + frames = recording_dir / "frames" + for suffix in ("before", "after"): + (frames / f"0001_{suffix}.png").write_bytes( + (frames / f"0000_{suffix}.png").read_bytes() + ) + return first, second + + +def test_sealed_native_capture_compiles_exact_source_geometry(tmp_path: Path) -> None: + capture_dir = _make_capture(tmp_path) + recording_dir = tmp_path / "recording" + convert_capture(capture_dir, recording_dir, source_surface="windows") + + [event] = [ + json.loads(line) + for line in (recording_dir / "events.jsonl").read_text().splitlines() + ] + geometry = event["source_geometry"] + before_png = (recording_dir / "frames" / "0000_before.png").read_bytes() + with Image.open(recording_dir / "frames" / "0000_after.png") as after_image: + assert after_image.getpixel((0, 0)) == (210, 240, 210) + assert geometry["source_action_ordinal"] == ACTION_ORDINAL + assert geometry["source_frame_ordinal"] == FRAME_ORDINAL + assert geometry["frame_sha256"] == hashlib.sha256(before_png).hexdigest() + assert geometry["display_topology_sha256"] == "a" * 64 + + _stamp_surface(recording_dir, "windows") + workflow = compile_recording( + recording_dir, + tmp_path / "bundle", + name="native geometry", + ) + assert workflow.steps[0].source_geometry is not None + assert ( + workflow.steps[0].source_geometry.binding_sha256 == geometry["binding_sha256"] + ) + + +def test_remote_v2_conversion_uses_exact_ordinal_frames_without_native_geometry( + tmp_path: Path, +) -> None: + capture_dir = _make_capture(tmp_path) + recording_dir = tmp_path / "recording" + + convert_capture(capture_dir, recording_dir, source_surface=None) + + [event] = [ + json.loads(line) + for line in (recording_dir / "events.jsonl").read_text().splitlines() + ] + assert "source_geometry" not in event + with Image.open(recording_dir / "frames" / "0000_after.png") as after_image: + assert after_image.getpixel((0, 0)) == (210, 240, 210) + + +def test_native_conversion_rejects_surface_platform_mismatch(tmp_path: Path) -> None: + capture_dir = _make_capture(tmp_path) + + with pytest.raises(ValueError, match="differs from the sealed Capture platform"): + convert_capture( + capture_dir, + tmp_path / "recording", + source_surface="macos", + ) + + +def test_v2_capture_refuses_before_unsealed_database_can_be_opened( + tmp_path: Path, +) -> None: + capture_dir = _make_capture(tmp_path, sealed=False) + database = capture_dir / "recording.db" + before = ( + database.stat().st_mtime_ns, + hashlib.sha256(database.read_bytes()).hexdigest(), + ) + + with pytest.raises(ValueError, match="no immutable terminal"): + convert_capture(capture_dir, tmp_path / "recording", source_surface="windows") + + after = ( + database.stat().st_mtime_ns, + hashlib.sha256(database.read_bytes()).hexdigest(), + ) + assert after == before + + +def test_compiler_refuses_native_geometry_on_remote_surface(tmp_path: Path) -> None: + capture_dir = _make_capture(tmp_path) + recording_dir = tmp_path / "recording" + convert_capture(capture_dir, recording_dir, source_surface="windows") + _stamp_surface(recording_dir, "rdp") + + with pytest.raises(ValueError, match="only for an in-session native surface"): + compile_recording(recording_dir, tmp_path / "bundle", name="wrong surface") + + +def test_compiler_refuses_tampered_exact_before_frame(tmp_path: Path) -> None: + capture_dir = _make_capture(tmp_path) + recording_dir = tmp_path / "recording" + convert_capture(capture_dir, recording_dir, source_surface="windows") + _stamp_surface(recording_dir, "windows") + (recording_dir / "frames" / "0000_before.png").write_bytes( + _png((255, 200, 200), "Changed") + ) + + with pytest.raises(ValueError, match="different before frame"): + compile_recording(recording_dir, tmp_path / "bundle", name="tampered") + + +def test_compiler_requires_native_geometry_on_every_event(tmp_path: Path) -> None: + capture_dir = _make_capture(tmp_path) + recording_dir = tmp_path / "recording" + convert_capture(capture_dir, recording_dir, source_surface="windows") + _stamp_surface(recording_dir, "windows") + first, second = _append_second_event(recording_dir) + second.pop("source_geometry") + (recording_dir / "events.jsonl").write_text( + json.dumps(first) + "\n" + json.dumps(second) + "\n" + ) + + with pytest.raises(ValueError, match="cover every executable"): + compile_recording(recording_dir, tmp_path / "bundle", name="partial") + + +def test_compiler_rejects_mixed_native_capture_identities(tmp_path: Path) -> None: + capture_dir = _make_capture(tmp_path) + recording_dir = tmp_path / "recording" + convert_capture(capture_dir, recording_dir, source_surface="windows") + _stamp_surface(recording_dir, "windows") + first, second = _append_second_event(recording_dir) + geometry = second["source_geometry"] + geometry["source_capture_session_sha256"] = "f" * 64 + geometry["source_action_ordinal"] += 1 + geometry["binding_sha256"] = native_source_geometry_sha256(geometry) + (recording_dir / "events.jsonl").write_text( + json.dumps(first) + "\n" + json.dumps(second) + "\n" + ) + + with pytest.raises(ValueError, match="mixes capture or terminal"): + compile_recording(recording_dir, tmp_path / "bundle", name="mixed") + + +def test_compiler_rejects_duplicate_native_action_ordinals(tmp_path: Path) -> None: + capture_dir = _make_capture(tmp_path) + recording_dir = tmp_path / "recording" + convert_capture(capture_dir, recording_dir, source_surface="windows") + _stamp_surface(recording_dir, "windows") + first, second = _append_second_event(recording_dir) + (recording_dir / "events.jsonl").write_text( + json.dumps(first) + "\n" + json.dumps(second) + "\n" + ) + + with pytest.raises(ValueError, match="unique and strictly increasing"): + compile_recording(recording_dir, tmp_path / "bundle", name="duplicate") + + +def test_legacy_step_json_omits_absent_source_geometry() -> None: + from openadapt_flow.ir import ActionKind, Step + + dumped = Step( + id="step_000", intent="press Enter", action=ActionKind.KEY, key="Enter" + ) + assert "source_geometry" not in dumped.model_dump(mode="json") + assert json.dumps( + dumped.model_dump(mode="json"), + sort_keys=True, + separators=(",", ":"), + ) == ( + '{"action":"key","anchor":null,"api_binding":null,"drag_end_anchor":null,' + '"effects":[],"expect":[],"field_label":null,"guard":null,"id":"step_000",' + '"identifier_crop_missing_reason":null,"identity_armed":null,' + '"identity_unarmed_reason":null,"intent":"press Enter","key":"Enter",' + '"modifiers":[],"param":null,"risk":"reversible","risk_explanation":null,' + '"risk_review_required":false,"scroll_dx":null,"scroll_dy":null,"secret":false,' + '"selection_commit_key":null,"selection_region":null,"text":null,' + '"timeout_s":10.0,"wait_until":null}' + ) + + +def test_native_source_geometry_digest_covers_every_model_field() -> None: + assert set(NATIVE_SOURCE_GEOMETRY_BINDING_FIELDS) == ( + set(NativeSourceGeometry.model_fields) - {"binding_sha256"} + ) diff --git a/tests/test_program_ir_phase1.py b/tests/test_program_ir_phase1.py index 63d43377..7f19ac29 100644 --- a/tests/test_program_ir_phase1.py +++ b/tests/test_program_ir_phase1.py @@ -182,6 +182,29 @@ def test_missing_required_param_fails_fast_naming_it(bundle, run_dir): assert backend.actions == [] # nothing ran +@pytest.mark.parametrize( + ("kind", "value"), + [(ParamKind.BOOLEAN, False), (ParamKind.NUMBER, 0)], +) +def test_required_false_and_zero_are_present(kind, value, bundle, run_dir): + wf = Workflow( + name="wf", + param_specs={"required": ParamSpec(name="required", type=kind, required=True)}, + steps=[key_step()], + ) + backend = FakeBackend() + report = Replayer(backend, vision=FakeVision()).run( + wf, + params={"required": value}, + bundle_dir=bundle, + run_dir=run_dir, + ) + assert report.success is True + assert report.params["required"] == value + assert type(report.params["required"]) is type(value) + assert backend.actions == [("press", "Enter")] + + # -- wait_until: bounded readiness, fail-safe HALT on timeout ----------------- diff --git a/tests/test_rdp_backend.py b/tests/test_rdp_backend.py index 2a7ca366..55df61f8 100644 --- a/tests/test_rdp_backend.py +++ b/tests/test_rdp_backend.py @@ -193,6 +193,25 @@ def reset(self) -> None: self.wheel_events.clear() +def test_atomic_observation_refuses_resize_before_first_rdp_input_edge() -> None: + transport = FakeRDPTransport([Image.new("RGB", (400, 300), "white")]) + backend = FreeRDPBackend(transport) + observation = backend.acquire_actuation_observation() + backend.bind_input_observation(observation) + transport.screens = [Image.new("RGB", (500, 300), "white")] + + with pytest.raises(FreshActuationRequired) as error: + backend.click_guarded( + 100, + 100, + expected_frame_sha256=observation.frame_sha256, + ) + + assert error.value.expected_geometry_epoch == observation.geometry_epoch + assert error.value.observed_geometry_epoch != observation.geometry_epoch + assert transport.pointer_events == [] + + class TransportError(RuntimeError): """The kind of error a real RDP transport raises mid-operation (a timeout on the wire). A distinct type so tests assert on exactly it.""" diff --git a/tests/test_remote_display_backend.py b/tests/test_remote_display_backend.py index e9db24ba..dde2433c 100644 --- a/tests/test_remote_display_backend.py +++ b/tests/test_remote_display_backend.py @@ -22,6 +22,8 @@ from openadapt_flow.backend import ( ActionDeliveryUncertain, Backend, + DisplayGeometry, + DisplayTopologyChanged, ExecutionContextIdentityBackend, FreshActuationRequired, IdentityBackend, @@ -55,6 +57,7 @@ def __init__( px: tuple[int, int] = (3024, 1888), key_window_id: int | None = None, hit_window_id: int | None = None, + displays: tuple[DisplayGeometry, ...] | None = None, ) -> None: self.trusted = trusted self._frontmost = frontmost @@ -78,6 +81,13 @@ def __init__( self.frame_overrides: dict[tuple[int, int], tuple[int, int, int]] = {} self.png_kwargs = {} self.calls: list[tuple] = [] + self.displays = displays or ( + DisplayGeometry( + "fake-primary", + (-10_000.0, -10_000.0, 20_000.0, 20_000.0), + (2.0, 2.0), + ), + ) def input_trusted(self) -> bool: return self.trusted @@ -109,6 +119,9 @@ def capture(self, window_id): img.save(buf, format="PNG", **self.png_kwargs) return buf.getvalue(), self.px[0], self.px[1] + def display_topology(self): + return self.displays + def activate(self, pid): self.calls.append(("activate", pid)) @@ -153,6 +166,122 @@ def test_exposes_pixel_targeting_plus_context_identity_protocol() -> None: assert not hasattr(backend, "locate_structural") +def test_atomic_observation_refuses_window_move_before_first_input_edge() -> None: + initial = WindowInfo( + window_id=1, + owner="Parallels Desktop", + title="Windows 11", + pid=99, + bounds=(0.0, 0.0, 200.0, 150.0), + on_screen=True, + ) + backend, client = _backend(window=initial, px=(400, 300)) + backend.observe_frame() + backend.prepare_pointer_actuation(100, 100) + observation = backend.acquire_actuation_observation() + backend.bind_input_observation(observation) + moved = WindowInfo( + window_id=client.window.window_id, + owner=client.window.owner, + title=client.window.title, + pid=client.window.pid, + bounds=(250.0, 75.0, 200.0, 150.0), + on_screen=True, + ) + client.window = moved + client.windows = [moved] + + with pytest.raises(FreshActuationRequired) as error: + backend.click_guarded( + 100, + 100, + expected_frame_sha256=observation.frame_sha256, + ) + + assert error.value.expected_geometry_epoch == observation.geometry_epoch + assert error.value.observed_geometry_epoch != observation.geometry_epoch + assert not any(call[0] == "mouse" for call in client.calls) + + +def test_cross_monitor_move_retains_negative_origin_scale_and_display_ids() -> None: + displays = ( + DisplayGeometry("left", (-800.0, 0.0, 800.0, 600.0), (1.0, 1.0)), + DisplayGeometry("right", (0.0, 0.0, 1000.0, 800.0), (2.0, 2.0)), + ) + initial = WindowInfo( + window_id=1, + owner="Parallels Desktop", + title="Windows 11", + pid=99, + bounds=(-700.0, 40.0, 200.0, 150.0), + on_screen=True, + ) + backend, client = _backend(window=initial, px=(200, 150), displays=displays) + backend.observe_frame() + backend.prepare_pointer_actuation(100, 75) + observation = backend.acquire_actuation_observation() + backend.bind_input_observation(observation) + assert observation.origin[0] < 0 + assert observation.display_id == "left" + assert observation.display_scale == (1.0, 1.0) + + moved = WindowInfo( + window_id=1, + owner=initial.owner, + title=initial.title, + pid=initial.pid, + bounds=(100.0, 40.0, 200.0, 150.0), + on_screen=True, + ) + client.window = moved + client.windows = [moved] + client.px = (400, 300) + + with pytest.raises(FreshActuationRequired) as error: + backend.click_guarded( + 100, + 75, + expected_frame_sha256=observation.frame_sha256, + ) + + assert error.value.expected_observation is observation + assert error.value.observed_observation is not None + assert error.value.expected_observation.display_id == "left" + assert error.value.observed_observation.display_id == "right" + assert error.value.observed_observation.display_scale == (2.0, 2.0) + assert error.value.expected_observation.topology_sha256 == ( + error.value.observed_observation.topology_sha256 + ) + assert not any(call[0] == "mouse" for call in client.calls) + + +def test_hotplug_invalidates_session_instead_of_retrying_input() -> None: + original_displays = ( + DisplayGeometry("primary", (0.0, 0.0, 1200.0, 900.0), (2.0, 2.0)), + ) + backend, client = _backend(displays=original_displays) + backend.observe_frame() + backend.prepare_pointer_actuation(100, 100) + observation = backend.acquire_actuation_observation() + backend.bind_input_observation(observation) + client.displays = original_displays + ( + DisplayGeometry("hotplug", (-900.0, 0.0, 900.0, 700.0), (1.0, 1.0)), + ) + + with pytest.raises(DisplayTopologyChanged) as error: + backend.click_guarded( + 100, + 100, + expected_frame_sha256=observation.frame_sha256, + ) + + assert error.value.expected_observation is observation + assert error.value.observed_observation.topology_sha256 != ( + observation.topology_sha256 + ) + assert not any(call[0] == "mouse" for call in client.calls) + + def test_live_context_markers_preserve_an_unchanged_actuation_lease() -> None: sensitive_title = "Patient Jane Doe - Account 004271" client = FakeClient( @@ -825,6 +954,77 @@ def click(self, x, y, *, double=False): ] +def test_replayer_reacquires_after_resize_before_first_input_edge(tmp_path) -> None: + from tests.test_replayer import FakeVision, Match, click_step, make_png + + class ResizeBeforeFirstEdgeBackend(RemoteDisplayBackend): + def __init__(self, *, client): + super().__init__(client=client, settle_s=0.0) + self.click_attempts = 0 + self.first_refusal: FreshActuationRequired | None = None + + def click(self, x, y, *, double=False): + self.click_attempts += 1 + if self.click_attempts == 1: + resized = WindowInfo( + window_id=self._client.window.window_id, + owner=self._client.window.owner, + title=self._client.window.title, + pid=self._client.window.pid, + bounds=(0.0, 0.0, 400.0, 300.0), + on_screen=True, + ) + self._client.window = resized + self._client.windows = [resized] + self._client.px = (400, 300) + try: + return super().click(x, y, double=double) + except FreshActuationRequired as exc: + self.first_refusal = exc + raise + + from openadapt_flow.ir import Workflow + from openadapt_flow.runtime.replayer import Replayer + + initial = WindowInfo( + window_id=1, + owner="Parallels Desktop", + title="Windows 11", + pid=99, + bounds=(0.0, 0.0, 300.0, 200.0), + on_screen=True, + ) + client = FakeClient(window=initial, px=(300, 200)) + backend = ResizeBeforeFirstEdgeBackend(client=client) + vision = FakeVision() + vision.template_results = [ + Match(point=(110, 105), region=(100, 100, 50, 20), confidence=0.95) + for _ in range(3) + ] + bundle = tmp_path / "bundle" + (bundle / "templates").mkdir(parents=True) + (bundle / "templates" / "btn.png").write_bytes(make_png((50, 20))) + + report = Replayer(backend, vision=vision).run( + Workflow(name="resize-retry", steps=[click_step(risk="irreversible")]), + bundle_dir=bundle, + run_dir=tmp_path / "run", + ) + + assert report.success is True + assert backend.click_attempts == 2 + assert len(vision.template_calls) == 3 + assert len([call for call in client.calls if call[0] == "mouse"]) == 2 + assert backend.first_refusal is not None + assert backend.first_refusal.expected_observation is not None + assert backend.first_refusal.observed_observation is not None + assert backend.first_refusal.expected_observation.viewport == (300, 200) + assert backend.first_refusal.observed_observation.viewport == (400, 300) + assert [event.retried for event in report.results[0].fresh_actuation_events] == [ + True + ] + + def test_bound_actuation_accepts_same_pixels_with_different_png_encoding() -> None: client = FakeClient() client.png_kwargs = {"compress_level": 0} diff --git a/tests/test_replayer.py b/tests/test_replayer.py index 9a139360..544b07a3 100644 --- a/tests/test_replayer.py +++ b/tests/test_replayer.py @@ -12,7 +12,14 @@ import pytest from PIL import Image -from openadapt_flow.backend import ActionDeliveryUncertain, FreshActuationRequired +from openadapt_flow.backend import ( + ActionDeliveryUncertain, + FrameObservation, + FreshActuationRequired, + frame_observation_identity, + session_identity_sha256, + window_identity_sha256, +) from openadapt_flow.ir import ( ActionDeliveryReceipt, ActionKind, @@ -583,6 +590,141 @@ def run_dir(tmp_path): return tmp_path / "run" +def test_geometry_epoch_change_between_actions_forces_fresh_resolution( + bundle, + run_dir, +): + class ChangingAtomicBackend: + def __init__(self) -> None: + self.state = 0 + self.actions: list[tuple] = [] + self.observed_epochs: list[str] = [] + + @property + def viewport(self): + raise AssertionError("Replayer must not read viewport after capture") + + def observe_frame(self) -> FrameObservation: + size = (300, 200) if self.state == 0 else (600, 400) + observation = FrameObservation.create( + make_png(size), + origin=(0.0, 0.0), + scale=(1.0, 1.0), + device_pixel_ratio=1.0, + display_id="test-display", + display_bounds=(0.0, 0.0, float(size[0]), float(size[1])), + display_scale=(1.0, 1.0), + topology_sha256=frame_observation_identity( + {"schema": "test-topology.v1", "viewport": list(size)} + ), + window_identity_sha256=window_identity_sha256( + window_id="test-window", + pid=1, + process_start_time="test-start", + owner="Test Backend", + ), + session_identity_sha256=session_identity_sha256( + authority="test", + session_id="test-session", + session_start_time="test-start", + principal_identity_sha256=None, + ), + ) + self.observed_epochs.append(observation.geometry_epoch) + return observation + + def screenshot(self) -> bytes: + return self.observe_frame().png + + def click(self, x, y, *, double=False): + self.actions.append(("click", x, y, double)) + self.state = 1 + + def type_text(self, text): + self.actions.append(("type", text)) + + def press(self, key): + self.actions.append(("press", key)) + + def scroll(self, dx, dy): + self.actions.append(("scroll", dx, dy)) + + backend = ChangingAtomicBackend() + vision = FakeVision() + vision.template_results = [ + Match(point=(110, 105), region=(100, 100, 50, 20), confidence=0.95), + Match(point=(220, 210), region=(200, 200, 100, 40), confidence=0.95), + ] + vision.text_results = { + "Ready": Match(point=(10, 10), region=(5, 5, 20, 10), confidence=0.95) + } + expected = [ + Postcondition( + kind=PostconditionKind.TEXT_PRESENT, + text="Ready", + timeout_s=0.1, + ) + ] + workflow = Workflow( + name="geometry-change", + steps=[ + click_step("first", expect=expected), + click_step("second", expect=expected), + ], + ) + + report = Replayer(backend, vision=vision).run( + workflow, + bundle_dir=bundle, + run_dir=run_dir, + ) + assert report.success is True + assert backend.actions == [ + ("click", 110, 105, False), + ("click", 220, 210, False), + ] + assert len(vision.template_calls) == 2 + assert len(set(backend.observed_epochs)) == 2 + + +def test_coordinate_mapping_epoch_allows_navigation_but_detects_resize() -> None: + def observation(*, page: str, size: tuple[int, int]) -> FrameObservation: + return FrameObservation.create( + make_png(size), + origin=(0.0, 0.0), + scale=(1.0, 1.0), + device_pixel_ratio=1.0, + display_id="test-display", + display_bounds=(0.0, 0.0, float(size[0]), float(size[1])), + display_scale=(1.0, 1.0), + topology_sha256=frame_observation_identity({"schema": "test-topology.v1"}), + window_identity_sha256=window_identity_sha256( + window_id="test-window", + pid=1, + process_start_time="test-start", + owner="Test Backend", + ), + session_identity_sha256=session_identity_sha256( + authority="test", + session_id="test-session", + session_start_time="test-start", + principal_identity_sha256=None, + ), + page_identity_sha256=hashlib.sha256(page.encode()).hexdigest(), + top_level_frame_identity_sha256=hashlib.sha256( + f"frame:{page}".encode() + ).hexdigest(), + ) + + before = observation(page="before", size=(300, 200)) + navigated = observation(page="after", size=(300, 200)) + resized = observation(page="after", size=(600, 400)) + + assert before.geometry_epoch != navigated.geometry_epoch + assert before.coordinate_mapping_epoch == navigated.coordinate_mapping_epoch + assert before.coordinate_mapping_epoch != resized.coordinate_mapping_epoch + + def test_happy_path_click_then_param_type(bundle, run_dir): vision = FakeVision() vision.template_results = [ diff --git a/tests/test_run_gate.py b/tests/test_run_gate.py index 09f53c76..83be1a2e 100644 --- a/tests/test_run_gate.py +++ b/tests/test_run_gate.py @@ -1389,7 +1389,7 @@ def capture(args): expected = { "patient_id": secret_value, "count": "3", - "approved": "True", + "approved": True, } authorization = captured["authorization"] assert authorization.runtime_inputs_digest == runtime_inputs_digest( diff --git a/tests/test_runner_client_lib.py b/tests/test_runner_client_lib.py index 507100d4..482c3d48 100644 --- a/tests/test_runner_client_lib.py +++ b/tests/test_runner_client_lib.py @@ -333,7 +333,7 @@ def test_dispatch_binding_known_vector(self): dispatch_binding_sha256( "11111111-1111-4111-8111-111111111111", authorization ) - == "sha256:efd01f7c8c56a0df02200d684a5ab6104e47ec769090b2d76eb090624cdcc272" + == "sha256:367411c4ff350c05d6dad465db3dd1f57e8d47d620d3c0f16b70adec0857047e" ) def test_dispatch_binding_refuses_changed_run_or_authorization(self, sealed): @@ -492,6 +492,44 @@ def test_full_admit_returns_execution_snapshot(self, sealed, config): assert verdict.effect_covered_consequential_steps == 0 assert verdict.workflow.manifest is not None + def test_param_domains_use_canonical_scalar_text(self, tmp_path, sealed, profile): + workflow, bundle = sealed + params = { + "enabled": False, + "count": 0, + "small": 1e-7, + "fixed": 1e20, + "large": 1e21, + } + manifest = write_manifest( + tmp_path, + f""" +[runner] +name = "n" +[profiles] +default = "{profile}" +[[bundles]] +content_digest = "{workflow.manifest.content_digest}" +path = "{bundle}" +[bundles.param_patterns] +enabled = '^false$' +count = '^0$' +small = '^1e-7$' +fixed = '^100000000000000000000$' +large = '^1e[+]21$' +""", + ) + cfg = load_runner_config(manifest) + authorization = mint_authorization(workflow, params) + verdict = verified_or_refusal( + workflow, + cfg, + params={"values": params}, + authorization=authorization, + ) + assert not isinstance(verdict, Refusal) + assert verdict.params == params + class TestVerifyRefusals: def test_unknown_job_kind(self, sealed, config): diff --git a/tests/test_terminal_verification_v2.py b/tests/test_terminal_verification_v2.py index 66b65fd1..e96245df 100644 --- a/tests/test_terminal_verification_v2.py +++ b/tests/test_terminal_verification_v2.py @@ -12,14 +12,21 @@ from pydantic import ValidationError from openadapt_flow.ir import ( + ActionDeliveryUncertainty, ActionKind, + EffectVerificationEvidence, + ExecutionOutcomeEnvelope, + IdentityCheck, + OutcomeContractCounts, PostconditionContractEvidence, + RunReport, postcondition_contract_sha256, postcondition_step_contract_sha256, ) from openadapt_flow.qualification_admission_v2 import canonical_json from openadapt_flow.receipt import RunReceipt from openadapt_flow.terminal_verification_v2 import ( + SIGNATURE_DOMAIN, ProductionAuthorizationEvidenceManifest, ProductionDeliveryPermit, ProductionDeliveryPermitChain, @@ -35,11 +42,15 @@ ProductionPolicyEvidenceManifest, ProductionPostconditionEvidenceManifest, ProductionRunReceipt, + ProductionTerminalEffectState, + ProductionTerminalVerificationContext, + ProductionTerminalVerificationEnvelope, ProductionTerminalVerificationError, ProductionTerminalVerificationExpected, ProductionTerminalVerificationPayload, TerminalContractCounts, build_evidence_manifest, + build_production_terminal_verification, evidence_runner_signer_sha256, project_production_run_receipt, rebuild_production_delivery_permit_chain_from_artifacts, @@ -48,6 +59,7 @@ sign_production_terminal_verification, verify_production_terminal_verification, ) +from tests.test_run_receipt import _report as _production_report SHA_A = "a" * 64 SHA_B = "b" * 64 @@ -137,6 +149,220 @@ def _outcome() -> ProductionExecutionOutcome: ) +def _terminal_postcondition( + verdict: str, +) -> PostconditionContractEvidence: + return _postcondition().model_copy(update={"verdict": verdict}) + + +def _halted_outcome() -> ProductionExecutionOutcome: + return ProductionExecutionOutcome( + outcome="HALTED", + profile="standard", + production_eligible=False, + execution_completed=False, + required_contracts=TerminalContractCounts( + authorization=1, + identity=1, + postcondition=1, + effect=1, + ), + passed_contracts=TerminalContractCounts( + authorization=1, + identity=1, + postcondition=0, + effect=0, + ), + workflow_contract_sha256=SHA_A, + postcondition_evidence=(_terminal_postcondition("unverifiable"),), + evidence_classes=("authorization", "identity"), + model_calls=0, + external_network_calls="none", + ) + + +def _halted_receipt() -> ProductionRunReceipt: + return ProductionRunReceipt( + source_schema_version="openadapt.run-report/v1", + outcome="HALTED", + transaction_outcome="HALTED_BEFORE_EFFECT", + profile="standard", + production_eligible=False, + steps_total=1, + steps_ok=0, + heals=0, + model_calls=0, + est_cost_microusd=0, + duration_ms=100, + rung_histogram={}, + evidence_classes=("authorization", "identity"), + effect_tier_reached="none", + authorization_required=1, + authorization_confirmed=1, + identity_required=1, + identity_confirmed=1, + postconditions_required=1, + postconditions_confirmed=0, + effects_required=1, + effects_confirmed=0, + identity_armed=1, + identity_applicable=1, + over_halt_count=0, + substrate="web", + provenance="production", + receipt_builder_version="1.2.3", + external_network_calls="none", + bundle_digest=SHA_A, + source_receipt_digest=SHA_B, + source_receipt_sha256=SHA_B, + generated_at="2026-08-18T12:00:00Z", + ) + + +def _reconciliation_outcome() -> ProductionExecutionOutcome: + return ProductionExecutionOutcome( + outcome="HALTED", + profile="standard", + production_eligible=False, + execution_completed=False, + required_contracts=TerminalContractCounts( + authorization=1, + identity=1, + postcondition=1, + effect=1, + ), + passed_contracts=TerminalContractCounts( + authorization=1, + identity=1, + postcondition=0, + effect=0, + ), + workflow_contract_sha256=SHA_A, + postcondition_evidence=(_terminal_postcondition("unverifiable"),), + evidence_classes=("authorization", "identity"), + model_calls=0, + external_network_calls="none", + ) + + +def _reconciliation_receipt() -> ProductionRunReceipt: + return _halted_receipt().model_copy( + update={ + "transaction_outcome": "RECONCILIATION_REQUIRED", + "evidence_classes": ("authorization", "identity"), + "identity_confirmed": 1, + } + ) + + +def _actual_non_success_report( + transaction_outcome: str, +) -> RunReport: + base = _production_report() + assert base.outcome_envelope is not None + base_result = base.results[0] + postcondition = base.outcome_envelope.postcondition_evidence[0].model_copy( + update={"verdict": "unverifiable"} + ) + if transaction_outcome == "HALTED_BEFORE_EFFECT": + identity = IdentityCheck( + status="verified", + mode="structured", + coverage=1.0, + ) + result = base_result.model_copy( + update={ + "risk": "irreversible", + "ok": False, + "identity": identity, + "postconditions_ok": None, + "effect_verified": False, + "effect_evidence": [], + "delivery_attempted": False, + "safety_halt": True, + } + ) + passed = OutcomeContractCounts( + authorization=1, + identity=1, + postcondition=0, + effect=0, + ) + evidence_classes = ["authorization", "identity"] + else: + uncertainty = ActionDeliveryUncertainty( + operation="guarded_coordinate_click", + native=False, + observed_at="2026-07-27T15:34:57+00:00", + cause_type="ConnectionResetError", + verification_attempted=True, + postconditions_confirmed=False, + effects_confirmed=False, + resolved_by_contract=False, + ) + effect = EffectVerificationEvidence( + effect_contract_hash=base_result.effect_contract_hashes[0], + substrate="rest", + verifier_identity="sha256:" + SHA_B, + verification_tier=1, + initial_verdict="indeterminate", + final_verdict="indeterminate", + observed_effect="unknown", + ) + result = base_result.model_copy( + update={ + "risk": "irreversible", + "ok": False, + "postconditions_ok": False, + "effect_verified": False, + "effect_evidence": [effect], + "delivery_attempted": True, + "delivery_uncertainty": uncertainty, + "safety_halt": True, + } + ) + passed = OutcomeContractCounts( + authorization=1, + identity=1, + postcondition=0, + effect=0, + ) + evidence_classes = ["authorization", "identity"] + envelope = ExecutionOutcomeEnvelope( + outcome="HALTED", + profile="standard", + production_eligible=False, + execution_completed=False, + required_contracts=OutcomeContractCounts( + authorization=1, + identity=1, + postcondition=1, + effect=1, + ), + passed_contracts=passed, + workflow_contract_sha256=base.workflow_contract_sha256, + postcondition_evidence=[postcondition], + evidence_classes=evidence_classes, + model_calls=0, + external_network_calls="observed", + ) + return RunReport.model_validate( + base.model_dump(mode="json") + | { + "run_id_sha256": hashlib.sha256(IDS["run_id"].encode()).hexdigest(), + "execution_outcome": "HALTED", + "transaction_outcome": transaction_outcome, + "transaction_billable": False, + "transaction_platform_fault": False, + "production_eligible": False, + "execution_completed": False, + "outcome_envelope": envelope.model_dump(mode="json"), + "success": False, + "results": [result.model_dump(mode="json")], + } + ) + + def _receipt() -> ProductionRunReceipt: unsigned = { "schema_version": "openadapt.run-receipt/v2", @@ -319,6 +545,208 @@ def _manifests( ) +def _halted_manifests( + chain: ProductionDeliveryPermitChain, +) -> ProductionEvidenceManifests: + success = _manifests(_permit_chain()) + authorization = build_evidence_manifest( + ProductionAuthorizationEvidenceManifest, + governed_authorization_id_sha256=SHA_A, + admission_id=IDS["admission_id"], + admission_artifact_sha256=SHA_D, + execution_authority_id=IDS["execution_authority_id"], + execution_authority_sha256=SHA_A, + permit_chain_sha256=chain.permit_chain_sha256, + ) + identity = build_evidence_manifest( + ProductionIdentityEvidenceManifest, + identity_contract_sha256=SHA_D, + workflow_contract_sha256=SHA_A, + required=1, + confirmed=1, + results=( + ProductionIdentityResult( + result_index=0, + status="verified", + mode="structured", + signals=(), + ), + ), + ) + postcondition = build_evidence_manifest( + ProductionPostconditionEvidenceManifest, + workflow_contract_sha256=SHA_A, + required=1, + confirmed=0, + records=(_terminal_postcondition("unverifiable"),), + ) + effect = build_evidence_manifest( + ProductionEffectEvidenceManifest, + effect_contract_sha256=SHA_E, + workflow_contract_sha256=SHA_A, + required=1, + confirmed=0, + records=( + ProductionTerminalEffectState( + result_index=0, + effect_contract_hash="sha256:" + SHA_A, + attempt_state="not_actuated", + observed_effect="absent", + effect_verified=False, + verification_performed=False, + verifier_identity=None, + verification_tier=None, + final_verdict=None, + resolved_delivery_uncertainty=False, + absence_basis="not_actuated", + reconciliation_completed=False, + reconciliation_actions=0, + ), + ), + ) + return ProductionEvidenceManifests( + policy=success.policy, + authorization=authorization, + identity=identity, + postcondition=postcondition, + effect=effect, + ) + + +def _reconciliation_manifests( + chain: ProductionDeliveryPermitChain, +) -> ProductionEvidenceManifests: + halted = _halted_manifests(chain) + identity = build_evidence_manifest( + ProductionIdentityEvidenceManifest, + identity_contract_sha256=SHA_D, + workflow_contract_sha256=SHA_A, + required=1, + confirmed=1, + results=( + ProductionIdentityResult( + result_index=0, + status="verified", + mode="structured", + signals=(), + ), + ), + ) + effect = build_evidence_manifest( + ProductionEffectEvidenceManifest, + effect_contract_sha256=SHA_E, + workflow_contract_sha256=SHA_A, + required=1, + confirmed=0, + records=( + ProductionTerminalEffectState( + result_index=0, + effect_contract_hash="sha256:" + SHA_A, + attempt_state="delivery_uncertain", + observed_effect="unknown", + effect_verified=False, + verification_performed=False, + verifier_identity=None, + verification_tier=None, + final_verdict=None, + resolved_delivery_uncertainty=False, + absence_basis="none", + reconciliation_completed=False, + reconciliation_actions=0, + ), + ), + ) + return halted.model_copy(update={"identity": identity, "effect": effect}) + + +def _halted_payload() -> ProductionTerminalVerificationPayload: + receipt = _halted_receipt() + outcome = _halted_outcome() + chain = ProductionDeliveryPermitChain.build(()) + return ProductionTerminalVerificationPayload( + **IDS, + flow_run_id_sha256=hashlib.sha256(IDS["run_id"].encode("utf-8")).hexdigest(), + bundle_artifact_sha256=SHA_B, + bundle_content_digest=SHA_A, + environment_digest=SHA_A, + environment_contract_sha256=SHA_B, + runtime_environment_sha256=SHA_C, + identity_contract_sha256=SHA_D, + effect_contract_sha256=SHA_E, + runtime_substrate="web", + admission_artifact_sha256=SHA_D, + admission_policy_sha256=SHA_A, + evidence_identity_sha256=SHA_E, + admitted_runtime_build_sha256=SHA_C, + evidence_runner_signer_sha256=evidence_runner_signer_sha256(_public_key()), + qualification_signer_registry_sha256=SHA_E, + qualification_signer_registry_revision=7, + execution_authority_sha256=SHA_A, + execution_authority_signer_sha256=SHA_C, + permit_chain=chain, + permit_count=0, + final_authority_sequence=0, + final_runtime_delivery_sequence=0, + workflow_contract_sha256=SHA_A, + execution_outcome=outcome, + execution_outcome_sha256=outcome.artifact_sha256(), + run_receipt=receipt, + run_receipt_sha256=hashlib.sha256( + canonical_json(receipt.model_dump(mode="json")) + ).hexdigest(), + run_report_sha256=SHA_B, + run_report_object_version="version:halted:1", + run_report_object_sha256=SHA_B, + evidence_manifests=_halted_manifests(chain), + verified_at="2026-08-18T12:00:02Z", + issued_at="2026-08-18T12:00:03Z", + ) + + +def _reconciliation_payload() -> ProductionTerminalVerificationPayload: + chain = _permit_chain() + receipt = _reconciliation_receipt() + outcome = _reconciliation_outcome() + return ProductionTerminalVerificationPayload( + **IDS, + flow_run_id_sha256=hashlib.sha256(IDS["run_id"].encode("utf-8")).hexdigest(), + bundle_artifact_sha256=SHA_B, + bundle_content_digest=SHA_A, + environment_digest=SHA_A, + environment_contract_sha256=SHA_B, + runtime_environment_sha256=SHA_C, + identity_contract_sha256=SHA_D, + effect_contract_sha256=SHA_E, + runtime_substrate="web", + admission_artifact_sha256=SHA_D, + admission_policy_sha256=SHA_A, + evidence_identity_sha256=SHA_E, + admitted_runtime_build_sha256=SHA_C, + evidence_runner_signer_sha256=evidence_runner_signer_sha256(_public_key()), + qualification_signer_registry_sha256=SHA_E, + qualification_signer_registry_revision=7, + execution_authority_sha256=SHA_A, + execution_authority_signer_sha256=chain.entries[0].authority_signer_sha256, + permit_chain=chain, + permit_count=1, + final_authority_sequence=0, + final_runtime_delivery_sequence=9, + workflow_contract_sha256=SHA_A, + execution_outcome=outcome, + execution_outcome_sha256=outcome.artifact_sha256(), + run_receipt=receipt, + run_receipt_sha256=hashlib.sha256( + canonical_json(receipt.model_dump(mode="json")) + ).hexdigest(), + run_report_sha256=SHA_B, + run_report_object_version="version:reconciliation:1", + run_report_object_sha256=SHA_B, + evidence_manifests=_reconciliation_manifests(chain), + verified_at="2026-08-18T12:00:02Z", + issued_at="2026-08-18T12:00:03Z", + ) + + def _payload() -> ProductionTerminalVerificationPayload: receipt = _receipt() chain = _permit_chain() @@ -365,6 +793,16 @@ def _payload() -> ProductionTerminalVerificationPayload: def _expected( payload: ProductionTerminalVerificationPayload, ) -> ProductionTerminalVerificationExpected: + if payload.permit_chain.entries: + authenticated_runner_id_sha256 = payload.permit_chain.entries[ + 0 + ].authenticated_runner_id_sha256 + authenticated_session_id_sha256 = payload.permit_chain.entries[ + 0 + ].authenticated_session_id_sha256 + else: + authenticated_runner_id_sha256 = SHA_B + authenticated_session_id_sha256 = SHA_C return ProductionTerminalVerificationExpected( run_id=payload.run_id, flow_run_id_sha256=payload.flow_run_id_sha256, @@ -396,12 +834,8 @@ def _expected( permit_count=payload.permit_count, final_authority_sequence=payload.final_authority_sequence, final_runtime_delivery_sequence=payload.final_runtime_delivery_sequence, - authenticated_runner_id_sha256=( - payload.permit_chain.entries[0].authenticated_runner_id_sha256 - ), - authenticated_session_id_sha256=( - payload.permit_chain.entries[0].authenticated_session_id_sha256 - ), + authenticated_runner_id_sha256=authenticated_runner_id_sha256, + authenticated_session_id_sha256=authenticated_session_id_sha256, acknowledged_one_use_claim_ids=tuple( entry.one_use_claim_id for entry in payload.permit_chain.entries ), @@ -430,6 +864,173 @@ def test_terminal_v2_signs_and_verifies_exact_production_success() -> None: ) +def test_terminal_v2_signs_and_verifies_zero_permit_safe_halt() -> None: + payload = _halted_payload() + envelope = sign_production_terminal_verification(payload, _private_key()) + + digest = verify_production_terminal_verification( + envelope, + expected=_expected(payload), + now=NOW, + ) + + assert digest == envelope.artifact_sha256() + assert payload.execution_outcome.outcome == "HALTED" + assert payload.run_receipt.transaction_outcome == "HALTED_BEFORE_EFFECT" + assert payload.permit_count == 0 + assert payload.permit_chain.entries == () + assert all( + isinstance(record, ProductionTerminalEffectState) + and record.absence_basis in {"not_actuated", "verifier_refuted"} + for record in payload.evidence_manifests.effect.records + ) + + +def test_terminal_v2_refuses_zero_permit_verified_claim() -> None: + data = _halted_payload().model_dump(mode="json") + data["run_receipt"]["transaction_outcome"] = "VERIFIED" + + with pytest.raises(ValidationError): + ProductionTerminalVerificationPayload.model_validate(data) + + +def test_terminal_v2_refuses_safe_halt_without_effect_absence() -> None: + data = _halted_payload().model_dump(mode="json") + effect = data["evidence_manifests"]["effect"] + record = effect["records"][0] + record.update( + { + "attempt_state": "delivery_uncertain", + "observed_effect": "unknown", + "absence_basis": "none", + } + ) + unsigned = {key: value for key, value in effect.items() if key != "manifest_sha256"} + effect["manifest_sha256"] = hashlib.sha256( + b"openadapt-production-effect-evidence-v1\0" + canonical_json(unsigned) + ).hexdigest() + + with pytest.raises(ValidationError, match="effect absence"): + ProductionTerminalVerificationPayload.model_validate(data) + + +def test_terminal_effect_state_requires_exact_verifier_and_verdict_binding() -> None: + record = _halted_payload().evidence_manifests.effect.records[0] + assert isinstance(record, ProductionTerminalEffectState) + data = record.model_dump(mode="json") + + with pytest.raises(ValidationError, match="verified state"): + ProductionTerminalEffectState.model_validate(data | {"effect_verified": True}) + with pytest.raises(ValidationError, match="verifier identity"): + ProductionTerminalEffectState.model_validate( + data + | { + "attempt_state": "delivered", + "verification_performed": True, + "verification_tier": 1, + "final_verdict": "refuted", + "absence_basis": "verifier_refuted", + } + ) + + refuted = ProductionTerminalEffectState.model_validate( + data + | { + "attempt_state": "delivered", + "verification_performed": True, + "verifier_identity": "sha256:" + SHA_B, + "verification_tier": 1, + "final_verdict": "refuted", + "absence_basis": "verifier_refuted", + } + ) + assert refuted.effect_verified is False + + +def test_terminal_v2_signs_and_verifies_reconciliation_proof() -> None: + payload = _reconciliation_payload() + envelope = sign_production_terminal_verification(payload, _private_key()) + + digest = verify_production_terminal_verification( + envelope, + expected=_expected(payload), + now=NOW, + ) + + assert digest == envelope.artifact_sha256() + assert payload.run_receipt.transaction_outcome == "RECONCILIATION_REQUIRED" + assert payload.permit_count == 1 + record = payload.evidence_manifests.effect.records[0] + assert isinstance(record, ProductionTerminalEffectState) + assert record.attempt_state == "delivery_uncertain" + assert record.observed_effect == "unknown" + + +@pytest.mark.parametrize( + "transaction_outcome", + ["HALTED_BEFORE_EFFECT", "RECONCILIATION_REQUIRED"], +) +def test_terminal_v2_builds_non_success_proof_from_exact_run_report( + transaction_outcome: str, +) -> None: + report = _actual_non_success_report(transaction_outcome) + chain = ( + ProductionDeliveryPermitChain.build(()) + if transaction_outcome == "HALTED_BEFORE_EFFECT" + else _permit_chain() + ) + context = ProductionTerminalVerificationContext( + run_id=IDS["run_id"], + tenant_id=IDS["tenant_id"], + workflow_id=IDS["workflow_id"], + workflow_version_id=IDS["workflow_version_id"], + bundle_version_id=IDS["bundle_version_id"], + bundle_artifact_sha256=SHA_B, + environment_digest=SHA_A, + environment_contract_sha256=SHA_B, + runtime_environment_sha256=SHA_C, + identity_contract_sha256=SHA_D, + effect_contract_sha256=SHA_E, + runtime_validation_id=IDS["runtime_validation_id"], + runtime_substrate="web", + admission_id=IDS["admission_id"], + admission_artifact_sha256=SHA_D, + admission_policy_sha256=SHA_A, + evidence_identity_sha256=SHA_E, + admitted_runtime_build_sha256=SHA_C, + evidence_runner_signer_sha256=evidence_runner_signer_sha256(_public_key()), + qualification_signer_registry_sha256=SHA_E, + qualification_signer_registry_revision=7, + execution_authority_id=IDS["execution_authority_id"], + execution_authority_sha256=SHA_A, + execution_authority_signer_sha256=( + chain.entries[0].authority_signer_sha256 if chain.entries else SHA_C + ), + permit_chain=chain, + run_report_object_version="version:terminal-report:1", + verified_at="2026-08-18T12:00:02Z", + issued_at="2026-08-18T12:00:03Z", + ) + + built = build_production_terminal_verification( + report, + context=context, + private_key=_private_key(), + ) + payload = built.envelope.payload + + assert payload.run_receipt.transaction_outcome == transaction_outcome + assert payload.run_report_sha256 == hashlib.sha256(built.report_bytes).hexdigest() + assert ( + verify_production_terminal_verification( + built.envelope, + expected=_expected(payload), + now=NOW, + ) + == built.envelope.artifact_sha256() + ) + + @pytest.mark.parametrize( ("field", "value", "message"), [ @@ -697,6 +1298,53 @@ def test_delivery_cross_language_vector_is_exact() -> None: assert rebuilt == chain +def test_non_success_terminal_cross_language_vectors_are_exact() -> None: + fixture = json.loads( + Path("tests/fixtures/terminal_verification_v2_terminal_vectors.json").read_text( + encoding="utf-8" + ) + ) + assert b64decode(fixture["signature_domain_base64"], validate=True) == ( + SIGNATURE_DOMAIN + ) + key = Ed25519PrivateKey.from_private_bytes( + b64decode(fixture["private_key_base64"], validate=True) + ) + payloads = { + "halted-before-effect-zero-permit": _halted_payload(), + "reconciliation-required-nonempty-permit": _reconciliation_payload(), + } + + for vector in fixture["vectors"]: + raw = b64decode(vector["envelope_canonical_base64"], validate=True) + envelope = ProductionTerminalVerificationEnvelope.model_validate_json(raw) + payload = payloads[vector["name"]] + expected = sign_production_terminal_verification(payload, key) + + assert canonical_json(envelope) == raw + assert envelope == expected + assert ( + hashlib.sha256(payload.canonical_bytes()).hexdigest() + == (vector["payload_canonical_sha256"]) + ) + assert envelope.signature == vector["signature"] + assert ( + envelope.artifact_sha256() + == (vector["terminal_verification_artifact_sha256"]) + ) + assert ( + envelope.payload.evidence_manifests.effect.records[0].model_dump( + mode="json" + ) + == vector["effect_state"] + ) + callback = vector["callback"] + assert callback["run_id"] == payload.run_id + assert callback["outcome"] == payload.run_receipt.transaction_outcome + assert callback["report_sha256"] == payload.run_report_sha256 + assert callback["artifact_bytes_source"] == "envelope_canonical_base64" + + def test_delivery_artifact_rebuild_rejects_noncanonical_stored_bytes() -> None: entry = _permit_chain().entries[0] with pytest.raises( diff --git a/tests/test_tutorial_presentation.py b/tests/test_tutorial_presentation.py index 85da06d4..9da74300 100644 --- a/tests/test_tutorial_presentation.py +++ b/tests/test_tutorial_presentation.py @@ -57,7 +57,8 @@ def fake_run(work_dir: Path, **kwargs: Any) -> TutorialResult: presentation_delay=None, name=None, no_receipt=True, - break_it=False, + simulate_rejected_write=False, + deprecated_break_it=False, ) assert _cmd_tutorial(args) == 0 diff --git a/tests/test_win_agent_server.py b/tests/test_win_agent_server.py index 78d88cc2..c7b6e05e 100644 --- a/tests/test_win_agent_server.py +++ b/tests/test_win_agent_server.py @@ -18,9 +18,19 @@ import pytest import requests +from openadapt_flow.backend import ( + DisplayTopologyChanged, + FrameObservationBackend, + FreshActuationRequired, +) from openadapt_flow.backends.win_agent import AgentConfig, create_server +from openadapt_flow.backends.win_agent import server as win_agent_server from openadapt_flow.backends.win_agent.server import ( AgentRequestError, + CapturedDesktopFrame, + FrameGeometry, + MonitorGeometry, + _normalize_virtual_point, _perform_input, _perform_uia, ) @@ -28,12 +38,104 @@ _PNG_SIGNATURE = b"\x89PNG\r\n\x1a\n" +def _test_context( + *, + application: str = "accuro", + session: str = "a" * 64, + window_id: str = "4096", + pid: int = 321, + process_start_time: str = "133801632000000000", +) -> dict: + return { + "status": "ok", + "application": application, + "session": session, + "workflow_state": None, + "window": { + "window_id": window_id, + "pid": pid, + "process_start_time": process_start_time, + "owner": application, + }, + } + + def _fake_png() -> bytes: """Minimal valid-enough PNG: signature + IHDR with a 4x2 size.""" ihdr = struct.pack(">II", 4, 2) return _PNG_SIGNATURE + b"\x00\x00\x00\x0dIHDR" + ihdr + b"\x00" * 8 +def _fake_png_size(width: int, height: int) -> bytes: + ihdr = struct.pack(">II", width, height) + return _PNG_SIGNATURE + b"\x00\x00\x00\x0dIHDR" + ihdr + b"\x00" * 8 + + +def _geometry( + *, + origin_x: int = 0, + origin_y: int = 0, + width: int = 4, + height: int = 2, + dpi: int = 96, + device: str = "DISPLAY1", +) -> FrameGeometry: + return FrameGeometry( + origin_x=origin_x, + origin_y=origin_y, + width=width, + height=height, + monitors=( + MonitorGeometry( + device=device, + left=origin_x, + top=origin_y, + width=width, + height=height, + dpi_x=dpi, + dpi_y=dpi, + primary=True, + ), + ), + ) + + +def _negative_origin_geometry() -> FrameGeometry: + return FrameGeometry.from_payload( + { + "version": 1, + "coordinate_space": "physical_virtual_desktop", + "dpi_awareness": "per_monitor_v2", + "origin_x": -1920, + "origin_y": -200, + "width": 4480, + "height": 1640, + "monitors": [ + { + "device": "DISPLAY2", + "left": -1920, + "top": -200, + "width": 1920, + "height": 1080, + "dpi_x": 144, + "dpi_y": 144, + "primary": False, + }, + { + "device": "DISPLAY1", + "left": 0, + "top": 0, + "width": 2560, + "height": 1440, + "dpi_x": 192, + "dpi_y": 192, + "primary": True, + }, + ], + } + ) + + class RunningAgent: """A started agent server plus its base URL (context-managed).""" @@ -129,6 +231,7 @@ def uia_fn(operation, payload): AgentConfig(host="127.0.0.1", port=0), input_fn=input_fn, uia_fn=uia_fn, + context_fn=_test_context, ) yield a a.close() @@ -152,6 +255,7 @@ def test_default_agent_disables_arbitrary_exec_and_advertises_typed_contract( assert "context_identity_v1" in health["capabilities"] assert "typed_input_v1" in health["capabilities"] assert "uia_v1" in health["capabilities"] + assert "frame_observation_v1" in health["capabilities"] assert "legacy_exec" not in health["capabilities"] response = requests.post( f"{typed_agent.url}/execute_windows", @@ -175,6 +279,7 @@ def test_typed_input_and_uia_receipts_never_claim_outcome( "application", "session", "workflow_state", + "window", } assert "title" not in context.text.casefold() expected_echo = requests.post( @@ -186,7 +291,14 @@ def test_typed_input_and_uia_receipts_never_claim_outcome( delivered = requests.post( f"{typed_agent.url}/input", - json={"action": "click", "x": 1, "y": 2, "double": False}, + json={ + "action": "click", + "x": 1, + "y": 1, + "double": False, + "expected_frame_sha256": hashlib.sha256(_fake_png()).hexdigest(), + "expected_frame_geometry": _geometry().to_payload(), + }, timeout=5, ) assert delivered.status_code == 200 @@ -194,7 +306,10 @@ def test_typed_input_and_uia_receipts_never_claim_outcome( found = requests.post( f"{typed_agent.url}/uia/find", - json={"locator": {"automation_id": "duplicate"}}, + json={ + "locator": {"automation_id": "duplicate"}, + "frame_geometry": _geometry().to_payload(), + }, timeout=5, ).json() assert found["match"] == "ambiguous" @@ -217,17 +332,136 @@ def test_windows_backend_guarded_key_roundtrip(typed_agent: RunningAgent) -> Non from openadapt_flow.backends import WindowsBackend backend = WindowsBackend(typed_agent.url, viewport=(4, 2)) - frame = backend.guarded_keyboard_frame() + observation = backend.acquire_actuation_observation() backend.arm_guarded_keyboard(1, 1) + backend.bind_input_observation(observation) receipt = backend.press_guarded( "Enter", - expected_frame_sha256=hashlib.sha256(frame).hexdigest(), + expected_frame_sha256=observation.frame_sha256, ) assert receipt.operation == "physical_press" assert receipt.outcome_verified is False +def test_windows_atomic_observation_maps_negative_origin_monitor_topology() -> None: + from openadapt_flow.backends import WindowsBackend + + geometry = _negative_origin_geometry() + frame = CapturedDesktopFrame( + _fake_png_size(geometry.width, geometry.height), + geometry, + ) + agent = RunningAgent( + AgentConfig(host="127.0.0.1", port=0), + grab_fn=lambda: frame, + context_fn=_test_context, + ) + try: + backend = WindowsBackend(agent.url) + assert isinstance(backend, FrameObservationBackend) + + observation = backend.observe_frame() + + assert observation.viewport == (4480, 1640) + assert observation.origin == (-1920.0, -200.0) + assert observation.display_id == "windows-virtual-desktop" + assert observation.display_bounds == (-1920.0, -200.0, 4480.0, 1640.0) + assert observation.scale == (1.0, 1.0) + assert len(observation.topology_sha256) == 64 + assert len(observation.window_identity_sha256) == 64 + assert len(observation.session_identity_sha256) == 64 + finally: + agent.close() + + +def test_windows_guarded_content_change_raises_fresh_before_input() -> None: + from openadapt_flow.backends import WindowsBackend + + state = {"frame": CapturedDesktopFrame(_fake_png(), _geometry())} + delivered: list[dict] = [] + + def input_fn(payload): + delivered.append(payload) + return { + "status": "delivered", + "receipt_id": "must-not-deliver", + "operation": "physical_press", + "native": False, + "target_fingerprint": None, + "delivered_at": "2026-08-20T00:00:00+00:00", + "outcome_verified": False, + } + + agent = RunningAgent( + AgentConfig(host="127.0.0.1", port=0), + grab_fn=lambda: state["frame"], + input_fn=input_fn, + uia_fn=lambda operation, payload: {"status": "ok", "focused": True}, + context_fn=_test_context, + ) + try: + backend = WindowsBackend(agent.url) + expected = backend.acquire_actuation_observation() + backend.arm_guarded_keyboard(1, 1) + backend.bind_input_observation(expected) + state["frame"] = CapturedDesktopFrame( + _fake_png() + b"changed-after-identity", + _geometry(), + ) + + with pytest.raises(FreshActuationRequired) as error: + backend.press_guarded( + "Enter", + expected_frame_sha256=expected.frame_sha256, + ) + + assert error.value.expected_observation is expected + assert error.value.observed_observation is not None + assert error.value.observed_observation.geometry_epoch == ( + expected.geometry_epoch + ) + assert error.value.observed_observation.frame_sha256 != expected.frame_sha256 + assert delivered == [] + finally: + agent.close() + + +def test_windows_guarded_topology_change_is_not_a_blind_retry() -> None: + from openadapt_flow.backends import WindowsBackend + + original = CapturedDesktopFrame(_fake_png(), _geometry()) + state = {"frame": original} + delivered: list[dict] = [] + agent = RunningAgent( + AgentConfig(host="127.0.0.1", port=0), + grab_fn=lambda: state["frame"], + input_fn=lambda payload: delivered.append(payload) or {}, + uia_fn=lambda operation, payload: {"status": "ok", "focused": True}, + context_fn=_test_context, + ) + try: + backend = WindowsBackend(agent.url) + expected = backend.acquire_actuation_observation() + backend.arm_guarded_keyboard(1, 1) + backend.bind_input_observation(expected) + changed_geometry = _geometry(width=6, height=3) + state["frame"] = CapturedDesktopFrame( + _fake_png_size(6, 3), + changed_geometry, + ) + + with pytest.raises(DisplayTopologyChanged): + backend.press_guarded( + "Enter", + expected_frame_sha256=expected.frame_sha256, + ) + + assert delivered == [] + finally: + agent.close() + + @pytest.mark.parametrize("mutation", ["frame", "context", "focus"]) def test_guarded_input_refuses_post_identity_change(mutation: str) -> None: state = { @@ -283,6 +517,7 @@ def uia_fn(operation, payload): f"{agent.url}/input/guarded", json={ "expected_frame_sha256": expected_frame, + "expected_frame_geometry": _geometry().to_payload(), "expected_context": { "application": "accuro", "session": "a" * 64, @@ -300,6 +535,123 @@ def uia_fn(operation, payload): assert delivered == [] +def test_guarded_input_refuses_geometry_change_with_no_input() -> None: + old_geometry = _geometry(dpi=96) + new_geometry = _geometry(dpi=144) + state = { + "frame": CapturedDesktopFrame(_fake_png(), old_geometry), + } + delivered: list[dict] = [] + + def input_fn(payload): + delivered.append(payload) + return { + "status": "delivered", + "receipt_id": "must-not-deliver", + "operation": "physical_click", + "native": False, + "target_fingerprint": None, + "delivered_at": "2026-08-20T00:00:00+00:00", + "outcome_verified": False, + } + + agent = RunningAgent( + AgentConfig(host="127.0.0.1", port=0), + grab_fn=lambda: state["frame"], + input_fn=input_fn, + context_fn=lambda: { + "status": "ok", + "application": "accuro", + "session": "a" * 64, + "workflow_state": None, + }, + ) + try: + expected_frame = hashlib.sha256(state["frame"].png).hexdigest() + state["frame"] = CapturedDesktopFrame(_fake_png(), new_geometry) + response = requests.post( + f"{agent.url}/input/guarded", + json={ + "expected_frame_sha256": expected_frame, + "expected_frame_geometry": old_geometry.to_payload(), + "expected_context": { + "application": "accuro", + "session": "a" * 64, + "workflow_state": None, + }, + "input": {"action": "click", "x": 1, "y": 1}, + }, + timeout=5, + ) + finally: + agent.close() + + assert response.status_code == 409 + assert response.json()["code"] == "stale_geometry" + assert delivered == [] + + +@pytest.mark.parametrize( + ("current_frame", "expected_code"), + [ + (CapturedDesktopFrame(_fake_png() + b"changed", _geometry()), "stale_frame"), + (CapturedDesktopFrame(_fake_png(), _geometry(dpi=144)), "stale_geometry"), + ], +) +def test_direct_pointer_refuses_frame_or_geometry_mismatch_with_no_input( + current_frame: CapturedDesktopFrame, + expected_code: str, +) -> None: + captured = CapturedDesktopFrame(_fake_png(), _geometry()) + delivered: list[dict] = [] + agent = RunningAgent( + AgentConfig(host="127.0.0.1", port=0), + grab_fn=lambda: current_frame, + input_fn=lambda payload: delivered.append(payload) or {}, + ) + try: + response = requests.post( + f"{agent.url}/input", + json={ + "action": "click", + "x": 1, + "y": 1, + "expected_frame_sha256": hashlib.sha256(captured.png).hexdigest(), + "expected_frame_geometry": captured.geometry.to_payload(), + }, + timeout=5, + ) + finally: + agent.close() + + assert response.status_code == 409 + assert response.json()["code"] == expected_code + assert delivered == [] + + +def test_backend_refreshes_viewport_from_each_bound_frame() -> None: + state = { + "frame": CapturedDesktopFrame(_fake_png_size(4, 2), _geometry()), + } + agent = RunningAgent( + AgentConfig(host="127.0.0.1", port=0), + grab_fn=lambda: state["frame"], + ) + try: + from openadapt_flow.backends import WindowsBackend + + backend = WindowsBackend(agent.url) + backend.screenshot() + assert backend.viewport == (4, 2) + state["frame"] = CapturedDesktopFrame( + _fake_png_size(6, 3), _geometry(width=6, height=3) + ) + backend.screenshot() + assert backend.viewport == (6, 3) + finally: + agent.close() + + def test_invalid_input_schema_refuses_before_loading_pyautogui(monkeypatch) -> None: monkeypatch.setitem(sys.modules, "pyautogui", None) with pytest.raises(AgentRequestError) as caught: @@ -308,6 +660,106 @@ def test_invalid_input_schema_refuses_before_loading_pyautogui(monkeypatch) -> N assert caught.value.code == "invalid_schema" +def test_pointer_input_maps_negative_frame_origin_through_sendinput( + monkeypatch: pytest.MonkeyPatch, +) -> None: + geometry = _negative_origin_geometry() + sent: list[tuple[list[tuple[int, int, int]], FrameGeometry]] = [] + monkeypatch.setattr(win_agent_server, "_current_frame_geometry", lambda: geometry) + monkeypatch.setattr( + win_agent_server, + "_send_virtual_pointer_sequence", + lambda events, current: sent.append((events, current)), + ) + + receipt = _perform_input( + { + "action": "click", + "x": 100, + "y": 250, + "double": False, + "button": "left", + "frame_geometry": geometry.to_payload(), + } + ) + + assert receipt["operation"] == "physical_click" + assert sent == [ + ( + [(-1820, 50, 0x0002), (-1820, 50, 0x0004)], + geometry, + ) + ] + assert _normalize_virtual_point(-1820, 50, geometry) == ( + round(100 * 65535 / 4479), + round(250 * 65535 / 1639), + ) + + +def test_pointer_input_maps_a_secondary_monitor_drag_through_sendinput( + monkeypatch: pytest.MonkeyPatch, +) -> None: + geometry = _negative_origin_geometry() + sent: list[list[tuple[int, int, int]]] = [] + monkeypatch.setattr(win_agent_server, "_current_frame_geometry", lambda: geometry) + monkeypatch.setattr( + win_agent_server, + "_send_virtual_pointer_sequence", + lambda events, _geometry: sent.append(events), + ) + + _perform_input( + { + "action": "drag", + "x": 2000, + "y": 300, + "end_x": 4200, + "end_y": 1200, + "frame_geometry": geometry.to_payload(), + } + ) + + assert sent == [ + [ + (80, 100, 0), + (80, 100, 0x0002), + (2280, 1000, 0), + (2280, 1000, 0x0004), + ] + ] + + +def test_pointer_input_refuses_topology_or_dpi_change_before_sendinput( + monkeypatch: pytest.MonkeyPatch, +) -> None: + captured = _negative_origin_geometry() + changed_payload = captured.to_payload() + changed_payload["monitors"][0]["dpi_x"] = 192 + changed_payload["monitors"][0]["dpi_y"] = 192 + changed = FrameGeometry.from_payload(changed_payload) + sent: list[object] = [] + monkeypatch.setattr(win_agent_server, "_current_frame_geometry", lambda: changed) + monkeypatch.setattr( + win_agent_server, + "_send_virtual_pointer_sequence", + lambda *_args: sent.append(object()), + ) + + with pytest.raises(AgentRequestError) as caught: + _perform_input( + { + "action": "click", + "x": 100, + "y": 250, + "frame_geometry": captured.to_payload(), + } + ) + + assert caught.value.status == 409 + assert caught.value.code == "stale_geometry" + assert sent == [] + + class _FakeRect: def __init__(self, left: int, top: int, right: int, bottom: int) -> None: self.left = left diff --git a/tests/test_windows_context_identity_native.py b/tests/test_windows_context_identity_native.py index 6128a1d0..56361354 100644 --- a/tests/test_windows_context_identity_native.py +++ b/tests/test_windows_context_identity_native.py @@ -30,6 +30,7 @@ from openadapt_flow.backends.win_agent import AgentConfig, create_server from openadapt_flow.backends.win_agent.server import ( _foreground_application_identity, + _foreground_window_identity, _native_session_digest, ) @@ -75,6 +76,13 @@ def test_live_session_digest_roundtrips_through_typed_agent() -> None: assert "title" not in payload application = payload["application"] assert application is None or _APPLICATION_RE.fullmatch(application) + window = payload["window"] + assert (application is None) == (window is None) + if window is not None: + assert window["owner"] == application + assert int(window["window_id"]) > 0 + assert window["pid"] > 0 + assert int(window["process_start_time"]) > 0 backend = WindowsBackend(url, auth_token="context-probe") assert backend.session_identity() == direct @@ -137,6 +145,23 @@ def close_handle(handle: Any) -> int: calls.append(("close", int(handle))) return 1 + def get_process_times( + process: Any, + created_pointer: Any, + exited_pointer: Any, + kernel_pointer: Any, + user_pointer: Any, + ) -> int: + del exited_pointer, kernel_pointer, user_pointer + calls.append(("process_times", int(process))) + created = ctypes.cast( + created_pointer, + ctypes.POINTER(wintypes.FILETIME), + ).contents + created.dwLowDateTime = 123 + created.dwHighDateTime = 456 + return 1 + class FakeUser32: GetForegroundWindow = _FakeFunction(get_foreground_window) GetWindowThreadProcessId = _FakeFunction(get_window_thread_process_id) @@ -144,6 +169,7 @@ class FakeUser32: class FakeKernel32: OpenProcess = _FakeFunction(open_process) QueryFullProcessImageNameW = _FakeFunction(query_process_image) + GetProcessTimes = _FakeFunction(get_process_times) CloseHandle = _FakeFunction(close_handle) def fake_win_dll(name: str, *, use_last_error: bool) -> object: @@ -156,11 +182,24 @@ def fake_win_dll(name: str, *, use_last_error: bool) -> object: monkeypatch.setattr(ctypes, "WinDLL", fake_win_dll) + assert _foreground_window_identity() == { + "window_id": "101", + "pid": 4242, + "process_start_time": str((456 << 32) | 123), + "owner": "accuro.emr", + } assert _foreground_application_identity() == "accuro.emr" assert calls == [ ("foreground", None), ("window_pid", 101), ("open_process", (0x1000, False, 4242)), ("process_image", (202, 0)), + ("process_times", 202), + ("close", 202), + ("foreground", None), + ("window_pid", 101), + ("open_process", (0x1000, False, 4242)), + ("process_image", (202, 0)), + ("process_times", 202), ("close", 202), ] diff --git a/uv.lock b/uv.lock index 0bacf324..efcc28a9 100644 --- a/uv.lock +++ b/uv.lock @@ -2136,7 +2136,7 @@ wheels = [ [[package]] name = "openadapt-flow" -version = "1.33.0" +version = "1.34.0" source = { editable = "." } dependencies = [ { name = "cryptography" },