0.2.0: ledger precheck + events:batch, run_state, data_scope (TokenOps remote-only) - #12
Merged
Merged
Conversation
First increment of the 0.2.0 remote-only work (#10). Adds the batched ledger write path the TokenOps SDK will use instead of per-op /v1/ledger/* calls: - events:batch route (scope: ingest), Durability header validated, max_batch / max_body enforced. - SqliteStore.apply_events(): applies spent_add (multi-target fan-out) / admit / complete / step / halt_mark / halt_clear in array order, one transaction, rolls back the whole batch on any bad event. - ledger_events(tenant_id, idempotency_key) dedupe table — replayed keys are counted in `deduped`, not re-applied; ack still returns current totals. - run_state(tenant_id, run_id) — step_count, bounded window ring (64), velocity; fed by `step` events; SqliteStore.get_run_state(). - docs/api-contract.md — the ratified TokenOps <-> control-plane wire contract. - tests/test_ledger_events.py — fan-out, ordering, idempotent replay, admit/complete, step accumulation, halt, rollback, 400s. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…s on full dedup - tests/conftest.py: make_client fixture closes SqliteStore/EnvelopeStore before removing the temp dir (open SQLite handles block unlink on Windows; the whole suite failed locally on Windows with WinError 32). - migrate test_health.py / test_envelopes.py / test_ledger_events.py onto it. - apply_events: collect spent_add targets into `touched` before the dedup check, so a fully-deduped batch still returns current totals in the ack (contract §6). Full suite: 17 passed. ruff clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Consolidated pre_call read — one request returns halt state plus the requested spent / inflight / window slices, replacing the 2-4 GETs the SDK does today. - SqliteStore.precheck(tenant_id, run_id, segment_keys, budgets, want) - route scope: read - want defaults to [spent, inflight, halt]; "window" pulls the run_state slice - unknown run -> zeros, not 404 (pre_call must not fail on a not-yet-seen run) tests/test_ledger_precheck.py: 6 tests. Full suite 22 passed, ruff clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…dditive Reframes 0.2.0 as additive-only on the wire and schema so a current tokenops (API mode, no dep on this package) keeps working against it. The breaking fold is deferred to control-plane 0.3.0, released after tokenops <next>. - _migrate() -> _apply_migrations(): forward-only, tracked in PRAGMA user_version (SqliteStore.SCHEMA_VERSION = 2). Existing ad-hoc ALTERs become the legacy (version < 1) branch. Placeholder for the v3 destructive fold. - _SCHEMA (CREATE IF NOT EXISTS) already adds ledger_events / run_state to any DB. - docs/api-contract.md: §8/§13/§15 + header — 0.2.0 additive, run-records write path deprecated (kept, ignores steps/cost_micros), run_registrations drop + create_run removal -> 0.3.0. - CHANGELOG. tests/test_migrations.py: fresh DB at current version; reopen idempotent; a legacy user_version=0 DB upgrades additively (new tables added, run_registrations NOT dropped, data intact). Full suite: 25 passed. ruff clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…TCH narrowing Remaining additive 0.2.0 items (#10). Nothing removed — 0.1-era clients keep working; the destructive fold stays scheduled for 0.3.0. - policy_instances.data_scope (local|global, default local): models, schema, _ensure_columns ALTER for existing DBs, serde, upsert, YAML seed (params.pop), and emitted in governance_config_for output. - POST /v1/runs returns registered_at; register_run returns the persisted row so the SDK binds without a follow-up GET .../registration. RunRegistration gains registered_at; _registration / serde read it. - PATCH /v1/run-records: client-sent steps / cost_micros are dropped + logged (derived from run_state / ledger_spent; 0.3.0 will reject). - GET|POST|DELETE /v1/ledger/runs/{run_id}/halt — run-scoped halt; the old /v1/ledger/halt/* routes stay as aliases. - GET /health advertises max_batch / max_body_bytes. - _migrate() -> _ensure_columns(): runs every open, fully guarded (fresh / 0.1 / intermediate DBs all converge). - version -> 0.2.0 (pyproject, settings). - docs: api-contract §8/§10/§13/§15, DESIGN.md TokenOps section, README compatibility + breaking-changes table, CHANGELOG 0.2.0. tests: test_data_scope.py (5), test_runs_and_halt.py (5), test_migrations.py extended (data_scope ALTER on a legacy DB). Full suite 34 passed, ruff clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Prerequisite for running tokenops locally against a real (non-mocked) plane: one command to bring it up in the background, one to check it, one to take it down -- no separate terminal tab to babysit. - control_plane/servicectl.py: start()/stop()/status(), singleton per state dir (platformdirs user-state dir, overridable via CONTROL_PLANE_STATE_DIR for tests). start() spawns `python -m control_plane.cli serve` detached (POSIX: start_new_session; Windows: DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP), redirects stdout/stderr to a log file, and polls /health before declaring success -- a process that never comes up is reaped, not left orphaned and untracked. stop() escalates terminate() -> kill() on a timeout. Both are idempotent. PID checks confirm the process is actually a `control_plane.cli serve` invocation (cmdline match) before trusting a stored PID -- PIDs get reused. Explicit, undocumented-as-graceful limitation: Windows has no SIGTERM, so stop() there is a hard stop, not a drain. - cli.py: `start` / `stop` / `status` subcommands. - New deps: psutil, platformdirs. - README quick start now leads with start/status/stop; serve documented as the foreground/scripting form. tests/test_servicectl.py -- integration tests that spawn a REAL subprocess and hit real HTTP (not mocked): full lifecycle (start -> /health -> status -> stop -> confirm process actually exited), idempotent start/stop, stale-state-file recovery, and a smoke pass over the actual TokenOps-facing routes (runs/governance/events:batch/precheck/halt) through the started instance. Manually verified the Windows detach/terminate path locally (this repo's CI is ubuntu-only, so it only exercises the POSIX branch). Full suite: 42 passed. ruff clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…stop) The repo's first container image. For a shared/hosted plane, not local dev -- kept clearly separate from the start/stop/status work in the previous commit. - Dockerfile: python:3.12-slim, builds from source (pip install .), non-root user, SQLite on a /data volume, built-in HEALTHCHECK against /health. Entrypoint is `control-plane serve` (foreground, PID 1) -- deliberately NOT `start`: Docker/Kubernetes is already the process supervisor (restart policy, health checks, stdout/stderr log collection), and a background/detached mode would exit PID 1 as soon as it spawned its child, taking the container down with it. - docker-compose.yml: build + run on :8800 with a named volume for the DB. - .dockerignore: excludes tests/docs/ui/caches/local db files from the build context. - README: new Docker section explaining the split from start/stop/status and why serve is the right entrypoint for a container. Verified without a Docker daemon (none available here): built the actual wheel (`python -m build`), confirmed hatchling packages web/ templates+static and default.yaml correctly, installed the wheel into a clean venv (what the image does), and ran the full start -> /health -> index page (proves Jinja2 templates + static assets resolve) -> stop cycle against it end to end. Full suite: 42 passed. ruff clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 1 of the TokenOps remote-only effort. Closes #10. Companion: theagentplane/tokenops#118.
Additive — nothing removed. A 0.1-era client keeps working against this. The
destructive fold (drop
run_registrations, removecreate_run, narrowPATCH) issplit out to #11 and ships as 0.3.0 after
tokenops <next>— not needed for thedecoupling.
What's here
POST /v1/ledger/events:batchspent_add(multi-target fan-out),admit/complete,step,halt_mark/halt_clear; array order, one transaction, idempotent peridempotency_key(ledger_eventstable); ack returnstotals+haltedPOST /v1/ledger/precheckspent/inflight/windowslicesrun_statetablestep_count/ bounded window ring (64) / velocity, fed bystepeventspolicy_instances.data_scopelocal|global(defaultlocal) — schema, migration ALTER, serde,PUT /v1/policies, YAML seed, emitted inGET /v1/governance/{agent}POST /v1/runsregistered_at— SDK binds without a follow-upGET .../registrationPATCH /v1/run-recordssteps/cost_microsare ignored + logged (derived server-side; 0.3.0 rejects)GET|POST|DELETE /v1/ledger/runs/{run_id}/halt/v1/ledger/halt/*kept as aliasesGET /healthmax_batch/max_body_bytesPRAGMA user_version_apply_migrations()(SCHEMA_VERSION = 2); opening a 0.1 DB adds the new tables/column, bumps the version, no data loss. v3 = the #11 folddocs/api-contract.mdmake_clientfixture closes SQLite before temp-dir cleanup — the suite was fully red on Windows before this (open handles →WinError 32);test_health.py/test_envelopes.pymigrated onto itTests
test_ledger_events.py(10),test_ledger_precheck.py(6),test_data_scope.py(5),test_runs_and_halt.py(5),test_migrations.py(3, incl. a legacyuser_version=0DB upgrading additively). Full suite: 34 passed.
ruff checkclean.Also in this PR:
control-plane start/stop/status(Tier 1 one-click run)Running the plane locally is now a prerequisite for TokenOps's remote-only work, so
this adds a managed background mode instead of requiring a dedicated terminal tab:
platformdirsuser-state dir); all three are idempotent.cmdline()match), not assumed — PIDs get reused by the OS.start_new_session(POSIX) /DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP(Windows).startnever lies: polls/health, kills and reports the log tail if the process never comes up.stopthere is a hard stop, not a graceful drain.psutil,platformdirs.tests/test_servicectl.py— real integration tests, no mocking of the OS layer: spawnsan actual subprocess, hits real HTTP. Full lifecycle (start →
/health→ status → stop→ confirm the process actually exited), idempotent start/stop, stale-state-file
recovery, and a smoke pass over the real TokenOps-facing routes (
/v1/runs,/v1/governance,/v1/ledger/events:batch,/v1/ledger/precheck,/v1/ledger/runs/{id}/halt) through the managed instance. The Windows detach/terminatepath was verified manually (this repo's CI is Ubuntu-only).
Full suite now: 42 passed.
ruff checkclean.Not in this PR
Idempotency-Keyheader on the legacy single-opspent/add+inflight/*writes —those are 0.3.0: destructive cleanup — fold run_registrations, remove create_run + legacy ledger wrappers #11-removal fodder and the SDK writes via
events:batch.🤖 Generated with Claude Code