Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ All notable changes to OriginWeave are documented in this file. The format follo
- Cumulative interactive-first RAM, VRAM, batch, local-model, admission, pause, and compositor-pressure mitigation plans, including active-consumer reduction at exact hard limits.
- Real pinned-Chrome WebDriver evidence for the controlled Agent Task fixture: the CI lane uses an isolated profile, disables extensions, types and submits synthetic text, observes the same-document post-condition, and proves profile cleanup; this does not claim a shipped OriginWeave browser adapter.
- Active pinned-Chrome Agent Task evidence verifies browser-computed role/name for controlled input and submit targets before action; this remains test-harness semantic evidence and does not claim a product semantic observer or authority.
- Active pinned-Chrome Agent Task evidence records browser-process RSS, semantic-observation bytes, action latency, and task duration from bounded trusted adapter inputs; this remains test evidence and does not claim process-set attribution or product resource telemetry.
- Universally value-redacted network evidence with explicit path, metadata, and provenance bounds; ambiguous path rejection; validated source URLs; lowercase SHA-256 identifiers; and verification state.
- Rust 1.97.1 build contract, strict Clippy and rustdoc gates, and exact production function, line, region, and branch coverage enforcement.
- Hourly bounded OpenCode product-development workflow using `NVIDIA_NIM_API_KEY`, an unprivileged disposable workspace, loopback-only model broker, independently verified patches, and publication through a dedicated `OPENCODE_PR_TOKEN` that cannot review or merge.
Expand Down
4 changes: 2 additions & 2 deletions docs/DOCUMENTATION_FITNESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,9 +164,9 @@ Active #65 supplies a deterministic synthetic local web fixture with a labelled

### 3.20 Bounded browser process-set resource evidence

Active #51→#66 establishes two distinct layers: #51 owns single explicitly supplied Linux PID sampling and the bounded telemetry value boundary; #66 owns bounded duplicate-safe aggregation/sampling over an exact caller-owned PID set. #66's exact current contract rejects empty, zero-PID, duplicate, oversized and overflow states and fails closed if any member cannot be sampled.
Active #51→#66 establishes two distinct layers: #51 owns single explicitly supplied Linux PID sampling and the bounded telemetry value boundary; #66 owns bounded duplicate-safe aggregation/sampling over an exact caller-owned PID set. #66's exact current contract rejects empty, zero-PID, duplicate, oversized and overflow states and fails closed if any member cannot be sampled. Active PR #72 records browser-process RSS, semantic-observation bytes, action latency, and task duration for the controlled pinned-Chrome fixture from bounded trusted adapter inputs.

**Resolution:** aggregate resource measurement must not silently undercount a known caller-owned process set, but process membership remains an external attribution responsibility. The implementation does not discover Chromium PIDs, prove process ancestry/task ownership, walk cgroups, sample GPU/VRAM or create a durable telemetry store.
**Resolution:** aggregate resource measurement must not silently undercount a known caller-owned process set, but process membership remains an external attribution responsibility. PR #72 is bounded resource evidence for test repeatability; it does not discover Chromium PIDs, prove process ancestry/task ownership, walk cgroups, sample GPU/VRAM or create a durable telemetry store, and does not turn the fixture into a product resource adapter.

## 4. Durable product decisions captured by the canonical graph

Expand Down
6 changes: 6 additions & 0 deletions docs/TEST_STRATEGY.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,12 @@ controlled input and submit target before the real WebDriver action. CSS remains
a fixture-harness locator; this does not establish OriginWeave node authority,
semantic provenance or policy dispatch.

Active PR #72 additionally records bounded browser-process RSS,
semantic-observation bytes, action latency and task duration for the same
controlled fixture. These are test-harness resource evidence from trusted
adapter inputs; they do not establish Chromium process-set attribution,
GPU/VRAM telemetry or a product resource adapter.

### 3.5 Buyer acceptance

Versioned task packs measure repeatable product outcomes rather than one lucky agent run. The benchmark artifact records browser build, OriginWeave version, model/provider/reasoning configuration, seed where supported, policy profile, hardware profile and source fixtures.
Expand Down
15 changes: 12 additions & 3 deletions docs/traceability/action-postcondition-evidence.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,14 @@ PR #71 extends the pinned-Chrome fixture lane by reading WebDriver's browser-com

This is bounded browser-computed evidence for a synthetic test target, not the OriginWeave semantic observation adapter. CSS locators remain test-harness selectors, and the lane does not create OriginWeave node handles, source-channel provenance, policy authority, or permission to execute page-advertised actions.

### PR #72 — bounded Agent Task resource evidence

**Capability maturity:** `IMPLEMENTED_ON_ACTIVE_PR`

PR #72 records browser-process RSS, semantic-observation bytes, action latency, and total task duration while the pinned-Chrome fixture runs. The measurements are bounded, positive observations from the trusted ChromeDriver process identifier and the controlled semantic payload; they make the real fixture's resource and timing evidence inspectable without introducing a new telemetry subsystem.

This is resource evidence for the active test harness, not process-set attribution or a product resource adapter. It does not discover Chromium children, prove task ownership or ancestry, walk cgroups, sample GPU/VRAM, or export durable product telemetry.

## 4. Non-transitive success semantics

The intended first-slice chain is:
Expand All @@ -96,7 +104,7 @@ VerifiedActionOutcomeEvidence type existence -/> proof of real Chromium executio
controlled fixture success -/> proof of an OriginWeave product browser runtime
```

PR #64 now rejects a caller-supplied observation timestamp that predates caller-supplied dispatch time, but the type cannot independently prove the clock source, that a real browser actually dispatched the action, that the supplied provenance belongs to the claimed browser target/node, or that the observed state was caused by that action. PR #70 proves real Chromium execution against the controlled fixture and PR #71 adds browser-computed role/name evidence, but their test-harness CSS locators and direct WebDriver calls are not the OriginWeave adapter/runtime composition required under issue #28.
PR #64 now rejects a caller-supplied observation timestamp that predates caller-supplied dispatch time, but the type cannot independently prove the clock source, that a real browser actually dispatched the action, that the supplied provenance belongs to the claimed browser target/node, or that the observed state was caused by that action. PR #70 proves real Chromium execution against the controlled fixture, PR #71 adds browser-computed role/name evidence, and PR #72 adds bounded resource evidence, but their test-harness CSS locators, direct WebDriver calls, and fixture-scoped measurements are not the OriginWeave adapter/runtime composition required under issue #28.

## 5. Active prerequisite graph for issue #28

Expand All @@ -111,9 +119,10 @@ The first real Chromium vertical slice remains distributed across bounded active
- PR #64 — verified and caller-timestamp-ordered post-condition action-outcome evidence; and
- PR #65 — controlled hostile local Agent Task workflow fixture; and
- PR #70 — real WebDriver execution of that fixture on pinned Chrome, without claiming a product browser adapter; and
- PR #71 — browser-computed role/name evidence before controlled action, without claiming a product semantic observer.
- PR #71 — browser-computed role/name evidence before controlled action, without claiming a product semantic observer; and
- PR #72 — bounded browser-process RSS, semantic-observation byte, latency, and task-duration resource evidence, without claiming process-set attribution or a product resource adapter.

These active PRs are non-shipped evidence. PR #70/#71 prove bounded browser-level and semantic evidence, but the active set does not itself compose WebDriver BiDi/CDP transport, OriginWeave authority translation, trusted Chromium process attribution, policy-authorized real input dispatch, causal post-condition observation, or deterministic end-to-end teardown/recovery into one protected-main runtime.
These active PRs are non-shipped evidence. PR #70/#71/#72 prove bounded browser-level, semantic, and resource evidence, but the active set does not itself compose WebDriver BiDi/CDP transport, OriginWeave authority translation, trusted Chromium process attribution, policy-authorized real input dispatch, causal post-condition observation, or deterministic end-to-end teardown/recovery into one protected-main runtime.

## 6. Remaining issue #28 boundary

Expand Down
131 changes: 109 additions & 22 deletions scripts/ci/run_mv3_compatibility.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@
restart-persistence behavior. It also executes the controlled Agent Task fixture
with extensions disabled in a fresh profile, verifies browser-computed role/name
for the controlled action targets, performs real WebDriver input and click
operations, verifies the observable post-condition, and proves profile cleanup
without treating page content as instruction or authority.
operations, verifies the observable post-condition, proves the controlled action
preserves its loaded URL, and records bounded runtime resource evidence without
treating page content as instruction or authority.
"""

from __future__ import annotations
Expand Down Expand Up @@ -41,6 +42,8 @@
STARTUP_TIMEOUT_SECONDS = 20.0
FIXTURE_TIMEOUT_SECONDS = 20.0
MAX_WEBDRIVER_RESPONSE_BYTES = 1_048_576
MAX_PROC_STATUS_CHARACTERS = 65_536
MAX_U64 = (1 << 64) - 1
W3C_ELEMENT_KEY = "element-6066-11e4-a52e-4f735466cecf"
PATH_TOKEN_CHARACTERS = frozenset(string.ascii_letters + string.digits + "-_.")

Expand Down Expand Up @@ -202,6 +205,43 @@ def _get_element_semantics(
return role, label


def _parse_linux_proc_status_rss_bytes(status_text: str) -> int:
"""Parse exactly one positive Linux ``VmRSS`` kB field into bounded bytes."""

rss_values: list[int] = []
for line in status_text.splitlines():
if not line.startswith("VmRSS:"):
continue
fields = line.split()
if len(fields) != 3 or fields[0] != "VmRSS:" or fields[2] != "kB":
raise ValueError("malformed Linux VmRSS field")
raw_kibibytes = fields[1]
if not raw_kibibytes.isascii() or not raw_kibibytes.isdigit():
raise ValueError("malformed Linux VmRSS value")
kibibytes = int(raw_kibibytes, 10)
if kibibytes <= 0:
raise ValueError("Linux VmRSS must be positive")
if kibibytes > MAX_U64 // 1024:
raise OverflowError("Linux VmRSS exceeds u64 byte range")
rss_values.append(kibibytes * 1024)
if len(rss_values) != 1:
raise ValueError("Linux proc status must contain exactly one VmRSS field")
return rss_values[0]


def _sample_linux_process_rss_bytes(process_id: int) -> int:
"""Read one attributed Linux process RSS through a bounded ``/proc`` status file."""

if isinstance(process_id, bool) or not isinstance(process_id, int) or process_id <= 0:
raise ValueError("invalid Linux process identifier")
status_path = pathlib.Path("/proc") / str(process_id) / "status"
with status_path.open("r", encoding="utf-8", errors="strict") as status_file:
status_text = status_file.read(MAX_PROC_STATUS_CHARACTERS + 1)
if len(status_text) > MAX_PROC_STATUS_CHARACTERS:
raise RuntimeError("Linux proc status exceeded the bounded text limit")
return _parse_linux_proc_status_rss_bytes(status_text)


def _wait_for_extension_evidence(
driver_port: int,
session_id: str,
Expand Down Expand Up @@ -490,7 +530,7 @@ def _run_agent_task_browser_pass(
fixture_url: str,
profile_dir: str,
) -> dict[str, Any]:
"""Execute one synthetic Agent Task through real WebDriver input in pinned Chrome."""
"""Execute one synthetic Agent Task and measure bounded real-browser evidence."""

started = time.monotonic()
driver_port = _free_loopback_port()
Expand Down Expand Up @@ -535,15 +575,22 @@ def _run_agent_task_browser_pass(
capabilities = session.get("capabilities", {})
if not isinstance(raw_session_id, str):
raise RuntimeError("ChromeDriver did not return an Agent Task session id")
if not isinstance(capabilities, dict):
raise RuntimeError("ChromeDriver Agent Task capabilities are malformed")
session_id = _path_token(raw_session_id, "session identifier")
browser_version = (
capabilities.get("browserVersion") if isinstance(capabilities, dict) else None
)
browser_version = capabilities.get("browserVersion")
browser_process_id = capabilities.get("goog:processID")
if browser_version != PINNED_CHROME_VERSION:
raise RuntimeError(
f"unexpected Agent Task Chrome version: expected {PINNED_CHROME_VERSION}, "
f"got {browser_version!r}"
)
if (
isinstance(browser_process_id, bool)
or not isinstance(browser_process_id, int)
or browser_process_id <= 0
):
raise RuntimeError("ChromeDriver did not return a valid browser process id")

_json_request(
driver_port,
Expand All @@ -557,9 +604,8 @@ def _run_agent_task_browser_pass(
_webdriver_path(session_id, "/url"),
).get("value")
if initial_url != fixture_url:
raise RuntimeError(
f"Agent Task initial URL mismatch: expected {fixture_url!r}, got {initial_url!r}"
)
raise RuntimeError("Agent Task did not load the requested fixture URL")

input_element = _find_element(driver_port, session_id, "#task-text")
input_role, input_name = _get_element_semantics(
driver_port,
Expand All @@ -568,18 +614,6 @@ def _run_agent_task_browser_pass(
)
if input_role != "textbox" or input_name != "Task text":
raise RuntimeError("Agent Task input semantic evidence mismatch")
_json_request(
driver_port,
"POST",
_element_command_path(session_id, input_element, "/clear"),
{},
)
_json_request(
driver_port,
"POST",
_element_command_path(session_id, input_element, "/value"),
{"text": AGENT_TASK_INPUT_VALUE, "value": list(AGENT_TASK_INPUT_VALUE)},
)
submit_element = _find_element(
driver_port,
session_id,
Expand All @@ -592,12 +626,44 @@ def _run_agent_task_browser_pass(
)
if submit_role != "button" or submit_name != "Submit task":
raise RuntimeError("Agent Task submit semantic evidence mismatch")
semantic_observation = {
"input": {"role": input_role, "name": input_name},
"submit": {"role": submit_role, "name": submit_name},
}
semantic_observation_bytes = len(
json.dumps(
semantic_observation,
ensure_ascii=False,
separators=(",", ":"),
sort_keys=True,
).encode("utf-8")
)
if semantic_observation_bytes <= 0:
raise RuntimeError("Agent Task semantic observation was empty")

action_started = time.monotonic()
_json_request(
driver_port,
"POST",
_element_command_path(session_id, input_element, "/clear"),
{},
)
_json_request(
driver_port,
"POST",
_element_command_path(session_id, input_element, "/value"),
{"text": AGENT_TASK_INPUT_VALUE, "value": list(AGENT_TASK_INPUT_VALUE)},
)
_json_request(
driver_port,
"POST",
_element_command_path(session_id, submit_element, "/click"),
{},
)
action_latency_ms = round((time.monotonic() - action_started) * 1000, 3)
if action_latency_ms <= 0:
raise RuntimeError("Agent Task measured a non-positive action latency")

post_submit_url = _json_request(
driver_port,
"GET",
Expand All @@ -606,6 +672,7 @@ def _run_agent_task_browser_pass(
url_unchanged = post_submit_url == initial_url
if not url_unchanged:
raise RuntimeError("Agent Task URL changed during submission")

result_element = _find_element(driver_port, session_id, "#task-result")
state = _json_request(
driver_port,
Expand All @@ -620,6 +687,10 @@ def _run_agent_task_browser_pass(
_validate_agent_task_submitted_state(state)
if text != AGENT_TASK_INPUT_VALUE:
raise RuntimeError("Agent Task result did not match the synthetic typed value")
browser_process_rss_bytes = _sample_linux_process_rss_bytes(browser_process_id)
task_duration_ms = round((time.monotonic() - started) * 1000, 3)
if task_duration_ms <= 0:
raise RuntimeError("Agent Task measured a non-positive task duration")
return {
"browser_version": browser_version,
"post_condition": True,
Expand All @@ -628,7 +699,11 @@ def _run_agent_task_browser_pass(
"input_semantics_verified": True,
"submit_semantics_verified": True,
"extensions_disabled": True,
"duration_ms": round((time.monotonic() - started) * 1000),
"browser_process_rss_bytes": browser_process_rss_bytes,
"semantic_observation_bytes": semantic_observation_bytes,
"action_latency_ms": action_latency_ms,
"task_duration_ms": task_duration_ms,
"duration_ms": round(task_duration_ms),
}
finally:
try:
Expand Down Expand Up @@ -677,6 +752,10 @@ def _run_agent_task_trial(
"input_semantics_verified": result["input_semantics_verified"],
"submit_semantics_verified": result["submit_semantics_verified"],
"extensions_disabled": result["extensions_disabled"],
"browser_process_rss_bytes": result["browser_process_rss_bytes"],
"semantic_observation_bytes": result["semantic_observation_bytes"],
"action_latency_ms": result["action_latency_ms"],
"task_duration_ms": result["task_duration_ms"],
"profile_cleaned": profile_cleaned,
"duration_ms": round((time.monotonic() - trial_started) * 1000),
}
Expand Down Expand Up @@ -806,6 +885,14 @@ def main() -> int:
and trial.get("submit_semantics_verified") is True
and trial.get("extensions_disabled") is True
and trial.get("profile_cleaned") is True
and isinstance(trial.get("browser_process_rss_bytes"), int)
and trial["browser_process_rss_bytes"] > 0
and isinstance(trial.get("semantic_observation_bytes"), int)
and trial["semantic_observation_bytes"] > 0
and isinstance(trial.get("action_latency_ms"), (int, float))
and trial["action_latency_ms"] > 0
and isinstance(trial.get("task_duration_ms"), (int, float))
and trial["task_duration_ms"] >= trial["action_latency_ms"]
for trial in agent_task_trials
if trial.get("passed") is True
)
Expand Down
Loading
Loading