diff --git a/README.md b/README.md index f807598..0151975 100644 --- a/README.md +++ b/README.md @@ -291,7 +291,9 @@ it injects the `shared_state` tool and a state summary into the sub-agent's syst
Production concerns -- **Cost tracking** — `CostConfig` carries separate input/output/cache-read/cache-write rates; `session_cost_usd()` gives a running total, and `is_configured()` distinguishes "free" from "pricing unknown" +- **Cost tracking** — `CostConfig` carries separate input/output/cache-read/cache-write rates plus optional context tiers; `session_cost_usd()` gives a running total, `AgentEvent::AgentEnd` carries a `SessionStats` rollup, and `is_configured()` distinguishes "free" from "pricing unknown" +- **Loop detection** — a model calling one tool with identical arguments forever trips none of the turn/token/duration limits until the whole budget is spent. On by default: steers on the third consecutive repeat, stops on the next, and emits `AgentEvent::LoopDetected` either way +- **Retrievable tool output** — head-tail truncation discards the middle irrecoverably. Attach a `SharedState` and the full text is stashed, with the marker naming a key the model can fetch - **Telemetry** — `tracing` spans per loop / LLM stream / tool, recording tokens and cost. OpenTelemetry is bridged app-side via `tracing-opentelemetry`; the library carries no OTel dependency by design - **GASP** (`features = ["gasp"]`) — record runs into a [GASP](https://github.com/yologdev/gasp) agent repo; yoagent is a tested-conformant runtime, with the 7-check suite running in CI - **Serde throughout** — every core type is `Serialize` / `Deserialize` / `PartialEq`, so sessions persist and replay @@ -384,8 +386,6 @@ MSRV is **1.86**, enforced in CI. Raising it is a minor-version change. MIT — see [LICENSE](LICENSE). -Inspired by [pi-agent-core](https://github.com/badlogic/pi-mono/tree/main/packages/agent) (TypeScript). - [crates-shield]: https://img.shields.io/crates/v/yoagent?labelColor=black&style=flat-square&logo=rust&color=orange [crates-link]: https://crates.io/crates/yoagent diff --git a/docs/concepts/context-management.md b/docs/concepts/context-management.md index 326d847..1744cd4 100644 --- a/docs/concepts/context-management.md +++ b/docs/concepts/context-management.md @@ -144,6 +144,52 @@ Keeps the last `keep_recent` messages in full detail. Older assistant messages a Drops the smallest span of middle messages that reaches the target, keeping at least `keep_first` from the start and `keep_recent` from the end. A constant marker message stands in for what was removed; the count goes to the debug log rather than into the marker, so the text does not change from pass to pass. +## Retrievable tool output + +Head-tail truncation on the append path keeps a huge tool result from eating the +context, but the middle is gone irrecoverably — it survives only in the event +stream, which the *agent* cannot read. + +Attach a `SharedState` and the full text is stashed, with the marker naming +where it went: + +```rust +let agent = Agent::from_config(config) + .with_shared_state(SharedState::new()); +``` + +``` +[... 1847 lines truncated — full output: shared_state get "tool-out-tc_01abc-9f2a-b0" ...] +``` + +The `shared_state` tool is registered for the run, so the model can act on the +pointer. **Opt-in**: with no store attached, truncation behaves exactly as +before and the marker advertises no retrieval it cannot honour. + +Keys are block-qualified — a result carrying several text blocks gets one key +per block, suffixed by the block's position in the content vector, so +text/image/text yields `…-b0` and `…-b2`. The key combines the tool call id +with a hash of the output, because Gemini synthesizes call ids as a per-response +index that restarts every turn; id alone would let turn 1's frozen marker +resolve to turn 5's content. + +Two limits worth knowing: + +- **Lossy compaction drops the marker but not the stash entry.** Levels 2 and 3 + drop whole turns, taking the pointer with them, while the stored value lives + on and keeps consuming cap quota. +- **Stash entries are evictable; caller keys are not.** Both backends evict + oldest-first under their cap, but only `tool-out-*` entries — losing one + degrades a marker to an ordinary "key not found" the agent can act on, and the + head+tail is still in the transcript. Nothing regenerates an artifact you + stored yourself, so when only caller keys remain the write reports capacity + instead. + +`SubAgentTool::with_context_config` makes the same path reachable for +sub-agents; their stash is scoped, and scoped keys are excluded from the system +prompt summary so a second delegation does not see the first one's keys and cold +-start the prefix cache. + ## Prefix Cache Stability Providers cache request prefixes — automatically on DeepSeek, explicitly via `cache_control` on Anthropic. A cache hit needs the new request to share a byte-identical prefix with the last one, so **every rewrite of already-sent history costs full price for every token from the rewrite point onward**. @@ -189,15 +235,52 @@ In input-token spend that is −9.2% to −21.3% on DeepSeek and −15.2% to − Prevents runaway agents: ```rust +#[non_exhaustive] pub struct ExecutionLimits { pub max_turns: usize, // Default: 50 pub max_total_tokens: usize, // Default: 1,000,000 pub max_duration: Duration, // Default: 600s (10 min) + pub max_consecutive_identical_tool_calls: Option, // Default: Some(3) } ``` When a limit is reached, the agent stops with a message like `"[Agent stopped: Max turns reached (50/50)]"`. +## Loop detection + +The cheapest catastrophic failure is a model calling one tool with the same +arguments forever. The three limits above all fire eventually — but only after +the run has burned its entire turn, token and wall-clock budget to discover it +achieved nothing. + +`max_consecutive_identical_tool_calls` is on by default at `Some(3)`, with two +escalations that mirror the house pattern of steering before aborting: + +1. **First trip** injects a steering message and continues. A model repeating a + call is often retrying something transient, and aborting immediately would + regress that legitimate case. +2. **A later trip on the same signature** stops the run. + +Both emit `AgentEvent::LoopDetected { tool_name, repetitions, aborted }`, so a +UI can show the intervention and an audit can tell a loop abort from a +turn-limit stop. + +Signatures compare `serde_json::Value`, not serialized text — two calls +differing only in key order are the same call. Counting covers duplicates +*within* one batch as well as across turns, because `ToolExecutionStrategy` +defaults to `Parallel` and a model can emit the same call three times in a +single message. + +**Consecutive**, and that word is load-bearing: a different call resets the +streak, so an alternating `[a, b, a, b, …]` loop is *not* detected. That is a +deliberate trade — an agent working through a list calls one tool repeatedly and +legitimately, and a detector that fired on interleaved repeats would be worse +than none. + +```rust +ExecutionLimits::default().with_max_consecutive_identical_tool_calls(None) // off +``` + ## Disabling Context Management ```rust diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 553a70c..7eb06b1 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -83,13 +83,21 @@ let config = ContextConfig::from_context_window(200_000); Prevents runaway agents: ```rust +#[non_exhaustive] // build with Default::default() + with_* pub struct ExecutionLimits { pub max_turns: usize, // Default: 50 pub max_total_tokens: usize, // Default: 1,000,000 pub max_duration: Duration, // Default: 600s + pub max_consecutive_identical_tool_calls: Option, // Default: Some(3) } ``` +```rust +ExecutionLimits::default() + .with_max_turns(20) + .with_max_consecutive_identical_tool_calls(None) // disable loop detection +``` + ## ThinkingLevel ```rust @@ -111,14 +119,51 @@ compat flags enable it; the Google and Bedrock providers currently ignore Token pricing per million: ```rust +#[non_exhaustive] // build with new() + with_*, not a literal pub struct CostConfig { pub input_per_million: f64, pub output_per_million: f64, pub cache_read_per_million: f64, pub cache_write_per_million: f64, + pub context_tiers: Vec, // empty = one flat rate at every size } ``` +Cache rates are set with builders rather than positionally. Four same-typed +`f64` arguments in a row is a transposition hazard, and no vendor publishes them +in one order — Anthropic lists input / cache-write / cache-read / output, OpenAI +lists input / cached-input / output: + +```rust +CostConfig::new(5.0, 30.0) // input, output — output is always dearer + .with_cache_read(0.5) + .with_cache_write(6.25) +``` + +All-zero rates mean **pricing unknown**, not free. `is_configured()` reports +which, and `session_cost_usd()` returns `None` for an unpriced model rather than +$0. + +### Context tiers + +Some vendors charge more above a prompt-size threshold. `cost_usd` selects by +the request's **prompt** tokens (`input + cache_read + cache_write`), so a long +reply to a short prompt stays on the base rate: + +```rust +CostConfig::new(5.0, 30.0) + .with_context_tier(ContextTier::new(272_000, 10.0, 45.0).with_cache_read(1.0)) +``` + +Tiers are kept sorted, and `cost_usd` takes the last one the prompt clears, so a +multi-step schedule works. **No shipped preset sets one** — see +`ModelConfig::gpt_5_5`'s docs for why the one candidate stayed flat. + +One caveat if you add a tier: prompt size is derived as +`input + cache_read + cache_write`, which holds only where the provider +subtracts cached tokens out of `input`. `bedrock.rs` populates neither cache +field, so a heavily-cached prompt reads small there. + ## ModelConfig Presets yoagent provides first-class `ModelConfig::*` constructors for Anthropic, OpenAI, Google Gemini, xAI, Groq, DeepSeek, Mistral, MiniMax, Z.ai, Qwen, Ollama, and local OpenAI-compatible servers. diff --git a/src/agent_loop.rs b/src/agent_loop.rs index c435251..009707c 100644 --- a/src/agent_loop.rs +++ b/src/agent_loop.rs @@ -1,6 +1,6 @@ //! The core agent loop: prompt → LLM stream → tool execution → repeat. //! -//! This is the heart of yoagent. Inspired by pi-agent-core's agent-loop.ts: +//! This is the heart of yoagent: //! //! - `agent_loop()` starts with new prompt messages //! - `agent_loop_continue()` resumes from existing context