You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
tokenops_run() calls governance_config_for(service) on every run start / agent hop unless an explicit governor= is passed (src/tokenops/control/run.py:224-226).
Embedded mode routes through governance_cache.py (process cache keyed (store_path, agent), invalidated on Store governance writes) — effectively free after the first call.
Remote / HTTP mode routes client.governance_config_for -> HttpStore.governance_config_for -> GET /v1/governance/{agent} with no cache. Every run start is a blocking round trip to the plane for config that only changes on Admin edits.
Under the HTTP hard-dependency (see #55, #60), and given TokenOps runs per turn, this adds a plane round trip to every turn purely for near-static config.
Desired behavior
Client-side governance-config cache that:
Loads once on application bootstrap (or lazily on first use) and is held in an in-memory store keyed by (base_url, agent).
Two-level TTL, both evaluated lazily on request arrival (when a run hits the agent) — not on a background timer:
Soft TTL: past it, the cached value is still served for this run, and a background refresh is kicked off to rebuild from a fresh GET /v1/governance/{agent}. No request blocks.
Hard TTL: past it, the cache entry is considered stale-unsafe — the next request blocks on a synchronous rebuild before building the Governor.
On a successful fetch (foreground or background), the entry is replaced and both TTL clocks reset.
Explicit invalidation still works (Admin edit -> clear_governance_config_cache, or a plane-push/webhook later).
Problem
tokenops_run()callsgovernance_config_for(service)on every run start / agent hop unless an explicitgovernor=is passed (src/tokenops/control/run.py:224-226).governance_cache.py(process cache keyed(store_path, agent), invalidated on Store governance writes) — effectively free after the first call.client.governance_config_for->HttpStore.governance_config_for->GET /v1/governance/{agent}with no cache. Every run start is a blocking round trip to the plane for config that only changes on Admin edits.Under the HTTP hard-dependency (see #55, #60), and given TokenOps runs per turn, this adds a plane round trip to every turn purely for near-static config.
Desired behavior
Client-side governance-config cache that:
(base_url, agent).GET /v1/governance/{agent}. No request blocks.clear_governance_config_cache, or a plane-push/webhook later).Notes / scope
governance_cache.pyso it wraps the HTTP path, not just the embedded Store path.TOKENOPS_GOVERNANCE_SOFT_TTL_S,TOKENOPS_GOVERNANCE_HARD_TTL_S) with sane defaults (soft ~60s, hard ~600s — tune).(base_url, agent)so concurrent runs don't stampede the plane.(store_path, agent)to(base_url, agent)in remote mode.build_governormutates its input dict).Acceptance sketch
GET /v1/governance/{agent}calls on the run hot path; refreshes happen in the background after soft TTL.Related: #55, #60, #109.