Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,13 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip install -e ".[dev]"
# agentplane-control-plane 0.2.0 isn't on PyPI yet — install straight from
# the repo so the [contract] tests (real control-plane, not just the fake)
# actually run in CI instead of silently skipping. tokenops has no embedded
# ledger to fall back to (tokenops#118), so these are load-bearing, not
# optional coverage.
python -m pip install "agentplane-control-plane @ git+https://github.com/theagentplane/control-plane@main"
python -m pip install -e ".[dev,contract]"

- name: Ruff check
run: python -m ruff check src tests examples
Expand Down
58 changes: 57 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,68 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `[contract]` optional-dependency group (`agentplane-control-plane>=0.2.0`). Kept out
of `[dev]` while the 0.2.0 line is unreleased; the plane-backed tests
`importorskip("control_plane")`, so `[dev]`-only CI stays green.
- `Ledger.close_run(run_id)` — drops the per-process `RunState`; called by
- `Ledger.close_run(run_id)` — drops the per-process `LocalRunState`; called by
`tokenops_run` on scope exit so a long-lived / shared-governor process does not
accumulate per-run window state (#115).
- `PolicyInstance.data_scope` (`local` | `global`, default `local`) — which tier a
policy's detector reads: Tier-1 `LocalRunState` cache, or the plane's authoritative
state via `precheck` (#118). Persisted (`policy_instances.data_scope`, additive
migration) and round-tripped through `governance_config_for`; not yet consumed by
`build_governor` — the Governor doesn't group detectors by scope until the
`LedgerBackend` rewire lands.
- `Ledger(backend=...)` — routes every write through `LedgerBackend.apply_events` and
every spend/inflight/halt read through `read_state` (`precheck`), alongside the
existing `store=`/in-memory modes (mutually exclusive with `store`) (#118).
`step`/`spent_add` batch together per crossing so the ack's `totals` cover the run
total in one round trip; `admit`/`complete`/`halt_mark`/`halt_clear` are separate
one-shot writes. `velocity`/`recent`/`window`/`step_count` stay Tier-1-only (never a
backend round trip) — those are inherently local, per-process reads. Not yet wired
into `build_governor`/`ControlPlaneClient`; `tests/test_ledger_backend_mode.py`
exercises it directly against `FakeLedgerBackend`, including two `Ledger` instances
sharing one backend (the cross-process case).
- `tokenops.control.dev_plane` — launches a real `agentplane-control-plane` on a real
localhost TCP port, in-process. Used by `tokenops.demo` (below) and by
`tests/conftest.py::live_plane_url` for tests that must exercise
`ControlPlaneClient.from_env()` itself (an in-process ASGI app isn't reachable that
way — `from_env()` builds its own plain `httpx.Client`).

### Changed

- **tokenops has no ledger of its own anymore (#118) — `ControlPlaneClient.from_env()`
requires `CONTROL_PLANE_URL`/`TOKENOPS_URL` and raises if neither is set.** There is
no more `TOKENOPS_EMBEDDED` env var and no code path left that silently falls back to
a local SQLite ledger; `build_governor`/`tokenops_run` construct `Ledger(backend=...)`
(an `HttpLedgerBackend`, i.e. real `precheck`/`events:batch` traffic to the plane) for
every live run. The `store=` constructor kwarg on `ControlPlaneClient`/`tokenops_run`
remains as an explicit, visible test-only escape hatch (dependency injection for unit
tests that don't want a running plane) — it is never reachable from `from_env()`, so
no environment misconfiguration can select it.
- `should_mount_run_registration()` always returns `False` now — registration is
always centralized on the plane; there's no embedded mode left for an agent to
self-host `POST /v1/runs` under.
- `tokenops.demo` (`python -m tokenops.demo`) launches a real control plane in-process
(`tokenops.control.dev_plane`) instead of using the now-removed embedded ledger, and
configures its budget/policy over the plane's own HTTP API (`PUT /v1/budgets` /
`PUT /v1/policies`) — the zero-setup promise holds, but needs
`agentplane-control-plane` importable (`pip install "agent-tokenops[contract]"` today,
or once released; otherwise point `CONTROL_PLANE_URL` at a plane you're already
running).
- CI now installs `agentplane-control-plane` straight from the control-plane repo
(`git+https://github.com/theagentplane/control-plane@main`) in addition to
`.[dev,contract]` — the `[contract]` tests and the `tests/examples/` e2e suite
(`tests/conftest.py::live_plane_url`) now actually run a real control plane in CI
instead of silently skipping; they're load-bearing coverage now, not optional.
- `tests/examples/test_bench_e2e.py` and `tests/examples/test_triad_e2e.py` now
configure their policies/budgets on a real, in-process control plane
(`live_plane_url`) over its own HTTP API (`PUT /v1/budgets` / `PUT /v1/policies`)
instead of a local `Store` under `TOKENOPS_EMBEDDED=1` — these are now the concrete
demonstration that governance policies configured on the plane reach a live,
multi-agent run and HALT/steer it (step_cap, cost_budget, output_runaway CANCEL+RETRY,
tool_output_cap deep swap), not just that the plumbing compiles.

- `Ledger`'s `RunState` renamed `LocalRunState` (#118, locked decision #9) — makes the
two-tier model explicit ahead of the `LedgerBackend` rewire: this is the per-process
Tier-1 cache, not the plane's authoritative `run_state`.
- `Ledger.record` no longer writes a zero-delta spend row for non-priced crossings
(tool calls, un-rolled-up delegates) — a free crossing is a *step*, not spend. The
cost ledger only moves on priced events (#118).
Expand Down
38 changes: 18 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,10 @@ add your own.

## 🚀 Quickstart

Requires Python 3.10+.
Requires Python 3.10+ and a running control plane — TokenOps has no ledger of its
own, so every agent (even a single process) governs against one. Zero-setup taste:
`python -m tokenops.demo` launches a throwaway plane for you automatically. For your
own agent, start one first (see [Quickdeploy](#-quickdeploy) below), then:

### 1. Put it in your agent

Expand All @@ -76,14 +79,16 @@ Anywhere else (Cursor, Copilot, ...), paste this:
<details>
<summary><b>Manual</b>, about ten lines</summary>

Wrap your model call once, then hand the wrapped version to your agent.
Wrap your model call once, then hand the wrapped version to your agent. Needs
`CONTROL_PLANE_URL` (or `TOKENOPS_URL`) pointing at a running control plane —
see [Quickdeploy](#-quickdeploy).

```python
from tokenops import ControlPlaneClient, tokenops_run
from tokenops.control import Halt, wrap_complete
from tokenops.providers import complete

client = ControlPlaneClient.from_env()
client = ControlPlaneClient.from_env() # raises if CONTROL_PLANE_URL isn't set

with tokenops_run(client=client, service="my-agent", intent="research",
provider="openai", model="gpt-4o") as bound:
Expand Down Expand Up @@ -131,9 +136,9 @@ run is out, even from another process.
## 🐳 Quickdeploy

> [!TIP]
> The control plane (`python -m tokenops.server`) shares one budget across
> processes and powers the dashboard. A single-process agent doesn't need it
> running at all.
> Every agent needs a control plane running — TokenOps has no ledger of its own.
> A single-process agent still needs one, just not the multi-process sharing this
> section is about. `python -m tokenops.server` powers the dashboard too.

One command, plane + dashboard:

Expand Down Expand Up @@ -184,8 +189,8 @@ export TOKENOPS_URL=http://localhost:7700
export TOKENOPS_DB=tokenops.db # plane and every agent read the same file
```

> `TOKENOPS_EMBEDDED=1` overrides `TOKENOPS_URL`. Leave it unset here, or each
> process silently falls back to its own local ledger and gets the full budget.
> tokenops has no ledger of its own — every process governs against the plane at
> `TOKENOPS_URL`. There is no local-fallback env var to accidentally leave unset.

PyPI name is `agent-tokenops`; the import is `tokenops`. Extras:
`pip install "agent-tokenops[examples]"` for the LangChain benches,
Expand Down Expand Up @@ -251,22 +256,15 @@ Chronicle records decision boundaries; TokenOps attaches as the cost/governance

| Variable | Purpose |
|---|---|
| `TOKENOPS_URL` | Remote plane base URL (e.g. `http://localhost:7700`) → HTTP `register_run` |
| `TOKENOPS_EMBEDDED` | Set to `1` to force in-process `Store` (tests / single-process) |
| `TOKENOPS_DB` | SQLite path shared by plane + agents |
| `TOKENOPS_URL` (or `CONTROL_PLANE_URL`) | The control plane's base URL (e.g. `http://localhost:8800`) — **required** |
| `TOKENOPS_CONFIG` | YAML for governance seed (core: `src/tokenops/config/default.yaml`) |

`TOKENOPS_URL` also accepts the aliases `CONTROL_PLANE_URL` and
tokenops has no ledger or run registry of its own — `ControlPlaneClient.from_env()`
raises if `TOKENOPS_URL`/`CONTROL_PLANE_URL` isn't set, rather than silently falling
back to anything local. Every process talks to the same plane, so they share one
budget by construction. `TOKENOPS_URL` also accepts the alias
`TOKENOPS_CONTROL_PLANE_URL`.

Production / multi-process: set `TOKENOPS_URL`; agents must **not** mount `/v1/runs`. Tests: `TOKENOPS_EMBEDDED=1` (or omit URL).

> **Precedence.** `ControlPlaneClient.from_env` takes the HTTP path only when a
> URL is set **and** `TOKENOPS_EMBEDDED` is not `1`. Setting both falls back to a
> local SQLite file with no warning, and every process then gets its own full
> budget. Check with
> `print("embedded" if client.embedded else client.url)`.

</details>

<details>
Expand Down
3 changes: 1 addition & 2 deletions docs/control-plane-deploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,9 @@ Agents in your app (or `examples/`) set `TOKENOPS_URL=http://tokenops:7700` so t

| Var | Meaning |
|-----|---------|
| `TOKENOPS_URL` | Plane base URL for `ControlPlaneClient` |
| `TOKENOPS_URL` | Plane base URL for `ControlPlaneClient` — **required**; tokenops has no local-ledger fallback |
| `TOKENOPS_DB` | Shared SQLite path |
| `TOKENOPS_CONFIG` | Governance seed YAML |
| `TOKENOPS_EMBEDDED=1` | Force in-process Store (tests) |

## Run compose (plane only)

Expand Down
17 changes: 10 additions & 7 deletions docs/guides/onboarding.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Chronicle records decision boundaries; TokenOps observes them for cost/governanc
| **Python 3.10+** | Required |
| **`pip install agent-tokenops`** | Import is still `tokenops`. Pulls Chronicle ≥0.2.0, FastAPI, httpx, provider clients, etc. |
| **Control plane + shared DB** *(multi-process)* | `TOKENOPS_URL` (e.g. `http://localhost:7700`) and `TOKENOPS_DB` shared by plane + agents. Seed governance once (`make db-reset`). |
| **Or embedded Store** *(single-process / tests)* | Omit `TOKENOPS_URL` or set `TOKENOPS_EMBEDDED=1`. |
| **Or a test-only local `Store`** *(no running plane needed)* | `tokenops_run(store=...)` / `ControlPlaneClient(store=...)` — never the default; tokenops has no env var that selects a local ledger. |
| **LLM API keys** | Only for real model calls — not required for TokenOps itself or offline tests. |
| **FastAPI** | Only if you use `instrument_app`. Non-FastAPI: pass kwargs / `RequestContext` to `tokenops_run`. |
| **Chronicle `@boundary`** | Tools *and* LLM: `kind="llm"` runs pre_call via `on_enter`; tools stay observe-only. LLM-only stacks can use bare `@boundary(..., kind="llm")` under `tokenops_run` instead of `wrap_complete`. |
Expand Down Expand Up @@ -135,16 +135,19 @@ The **agent** (via `instrument_app` / `tokenops_run` kwargs), not the UI. Client
- **LLM calls:** `@boundary(..., kind="llm")` under `tokenops_run` runs **pre_call** (via `on_enter`) and observe — enough for LLM-only stacks without `wrap_complete`.
- **Tools** (search, fetch, etc.): still need `@boundary` + the crossing hook so they appear on the ledger / are governed. Without that, TokenOps only sees LLM crossings you decorate (or put through `wrap_complete`).

### Embedded Store vs `TOKENOPS_URL`?
### Do I need a control plane running?

| Mode | When |
|------|------|
| `TOKENOPS_URL` set | Production / multi-process: register via HTTP; share `TOKENOPS_DB` with the plane. Agents must **not** mount `/v1/runs`. |
| `TOKENOPS_EMBEDDED=1` or no URL | Tests / single process: in-process `Store`. |
Yes, always — tokenops has no ledger or run registry of its own. `ControlPlaneClient.from_env()`
raises if `TOKENOPS_URL`/`CONTROL_PLANE_URL` isn't set. Run one locally (`control-plane serve`,
part of the `agentplane-control-plane` package, or its Docker image) and point every agent
process at it; they'll share one budget by construction. Tests that don't want a real plane
running use `tokenops_run(store=...)` / `ControlPlaneClient(store=...)` with a local `Store` —
an explicit, visible test double, not a production mode.

### Do I construct `Store(...)` in the agent?

Prefer `ControlPlaneClient.from_env()` and `tokenops_run`. Happy path does not require user-facing `Store(...)` construction.
No — `ControlPlaneClient.from_env()` and `tokenops_run()` are the only supported production
path. `Store(...)` is a test-only construct for exercising policy logic without a running plane.

### Tools usually cost $0 — why govern them?

Expand Down
12 changes: 0 additions & 12 deletions examples/a2a/client.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
from __future__ import annotations

import os

from examples.a2a import messages
from examples.a2a.messages import parse_findings, parse_steps, parse_token_usage, summarize_request
from examples.a2a.server import fetch_agent_card, fetch_agent_card_sync, post_task, post_task_sync
Expand Down Expand Up @@ -53,11 +51,6 @@ def submit_task_sync_with_meta(
Research registers the run when ``X-TokenOps-Run-Id`` is absent — clients should
not call ``/v1/runs`` themselves for the default Chat / bench flow.
"""
if (
not (os.environ.get("TOKENOPS_URL") or "").strip()
and os.environ.get("TOKENOPS_EMBEDDED") != "1"
):
os.environ.setdefault("TOKENOPS_EMBEDDED", "1")
payload = messages.task_request(
task=task, bench={"corpus_profile": corpus_profile}, intent=intent
)
Expand Down Expand Up @@ -86,11 +79,6 @@ async def submit_task(
intent: str = "",
user_dims: dict[str, str] | None = None,
) -> RunResult:
if (
not (os.environ.get("TOKENOPS_URL") or "").strip()
and os.environ.get("TOKENOPS_EMBEDDED") != "1"
):
os.environ.setdefault("TOKENOPS_EMBEDDED", "1")
payload = messages.task_request(
task=task, bench={"corpus_profile": corpus_profile}, intent=intent
)
Expand Down
7 changes: 0 additions & 7 deletions examples/brief/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@

from __future__ import annotations

import os

from examples.a2a.messages import parse_findings, parse_steps, parse_token_usage, task_request
from examples.a2a.server import post_task, post_task_sync
from examples.agents.types import Finding, RunResult, StepEvent, TokenUsage
Expand Down Expand Up @@ -49,11 +47,6 @@ def submit_brief_sync_with_meta(
governance_mode: GovernanceMode = GovernanceMode.ENFORCE,
) -> tuple[RunResult, dict[str, object]]:
"""POST the topic to Scout (entry). Scout registers the run when run_id is omitted."""
if (
not (os.environ.get("TOKENOPS_URL") or "").strip()
and os.environ.get("TOKENOPS_EMBEDDED") != "1"
):
os.environ.setdefault("TOKENOPS_EMBEDDED", "1")
payload = task_request(task=topic, bench={"corpus_profile": corpus_profile}, intent=intent)
if user_dims:
payload["user_dims"] = user_dims
Expand Down
7 changes: 0 additions & 7 deletions examples/triad/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@

from __future__ import annotations

import os

from examples.a2a.messages import parse_findings, parse_steps, parse_token_usage, task_request
from examples.a2a.server import post_task, post_task_sync
from examples.agents.types import Finding, RunResult, StepEvent, TokenUsage
Expand Down Expand Up @@ -67,11 +65,6 @@ def submit_goal_sync_with_meta(
The Planner registers the run on the control plane when ``X-TokenOps-Run-Id``
is absent — clients should not call ``/v1/runs`` themselves for the triad UI.
"""
if (
not (os.environ.get("TOKENOPS_URL") or "").strip()
and os.environ.get("TOKENOPS_EMBEDDED") != "1"
):
os.environ.setdefault("TOKENOPS_EMBEDDED", "1")
payload = task_request(task=goal, bench={"corpus_profile": corpus_profile}, intent=intent)
if user_dims:
payload["user_dims"] = user_dims
Expand Down
4 changes: 2 additions & 2 deletions src/tokenops/control/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@
wrap_complete,
wrap_stream,
)
from tokenops.control.ledger import Budget, Ledger, RunState, segment_key
from tokenops.control.ledger import Budget, Ledger, LocalRunState, segment_key
from tokenops.control.ledger_backend import (
AggregateState,
ApplyResult,
Expand Down Expand Up @@ -112,7 +112,7 @@
# ledger
"Budget",
"Ledger",
"RunState",
"LocalRunState",
"segment_key",
# ledger backend (remote-only)
"LedgerBackend",
Expand Down
Loading
Loading