Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -291,7 +291,9 @@ it injects the `shared_state` tool and a state summary into the sub-agent's syst
<details>
<summary><b>Production concerns</b></summary>

- **Cost tracking** — `CostConfig` carries separate input/output/cache-read/cache-write rates; `session_cost_usd()` gives a running total, and `is_configured()` distinguishes "free" from "pricing unknown"
- **Cost tracking** — `CostConfig` carries separate input/output/cache-read/cache-write rates plus optional context tiers; `session_cost_usd()` gives a running total, `AgentEvent::AgentEnd` carries a `SessionStats` rollup, and `is_configured()` distinguishes "free" from "pricing unknown"
- **Loop detection** — a model calling one tool with identical arguments forever trips none of the turn/token/duration limits until the whole budget is spent. On by default: steers on the third consecutive repeat, stops on the next, and emits `AgentEvent::LoopDetected` either way
- **Retrievable tool output** — head-tail truncation discards the middle irrecoverably. Attach a `SharedState` and the full text is stashed, with the marker naming a key the model can fetch
- **Telemetry** — `tracing` spans per loop / LLM stream / tool, recording tokens and cost. OpenTelemetry is bridged app-side via `tracing-opentelemetry`; the library carries no OTel dependency by design
- **GASP** (`features = ["gasp"]`) — record runs into a [GASP](https://github.com/yologdev/gasp) agent repo; yoagent is a tested-conformant runtime, with the 7-check suite running in CI
- **Serde throughout** — every core type is `Serialize` / `Deserialize` / `PartialEq`, so sessions persist and replay
Expand Down Expand Up @@ -384,8 +386,6 @@ MSRV is **1.86**, enforced in CI. Raising it is a minor-version change.

MIT — see [LICENSE](LICENSE).

Inspired by [pi-agent-core](https://github.com/badlogic/pi-mono/tree/main/packages/agent) (TypeScript).

<!-- Badge link definitions -->
[crates-shield]: https://img.shields.io/crates/v/yoagent?labelColor=black&style=flat-square&logo=rust&color=orange
[crates-link]: https://crates.io/crates/yoagent
Expand Down
83 changes: 83 additions & 0 deletions docs/concepts/context-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,52 @@ Keeps the last `keep_recent` messages in full detail. Older assistant messages a

Drops the smallest span of middle messages that reaches the target, keeping at least `keep_first` from the start and `keep_recent` from the end. A constant marker message stands in for what was removed; the count goes to the debug log rather than into the marker, so the text does not change from pass to pass.

## Retrievable tool output

Head-tail truncation on the append path keeps a huge tool result from eating the
context, but the middle is gone irrecoverably — it survives only in the event
stream, which the *agent* cannot read.

Attach a `SharedState` and the full text is stashed, with the marker naming
where it went:

```rust
let agent = Agent::from_config(config)
.with_shared_state(SharedState::new());
```

```
[... 1847 lines truncated — full output: shared_state get "tool-out-tc_01abc-9f2a-b0" ...]
```

The `shared_state` tool is registered for the run, so the model can act on the
pointer. **Opt-in**: with no store attached, truncation behaves exactly as
before and the marker advertises no retrieval it cannot honour.

Keys are block-qualified — a result carrying several text blocks gets one key
per block, suffixed by the block's position in the content vector, so
text/image/text yields `…-b0` and `…-b2`. The key combines the tool call id
with a hash of the output, because Gemini synthesizes call ids as a per-response
index that restarts every turn; id alone would let turn 1's frozen marker
resolve to turn 5's content.

Two limits worth knowing:

- **Lossy compaction drops the marker but not the stash entry.** Levels 2 and 3
drop whole turns, taking the pointer with them, while the stored value lives
on and keeps consuming cap quota.
- **Stash entries are evictable; caller keys are not.** Both backends evict
oldest-first under their cap, but only `tool-out-*` entries — losing one
degrades a marker to an ordinary "key not found" the agent can act on, and the
head+tail is still in the transcript. Nothing regenerates an artifact you
stored yourself, so when only caller keys remain the write reports capacity
instead.

`SubAgentTool::with_context_config` makes the same path reachable for
sub-agents; their stash is scoped, and scoped keys are excluded from the system
prompt summary so a second delegation does not see the first one's keys and cold
-start the prefix cache.

## Prefix Cache Stability

Providers cache request prefixes — automatically on DeepSeek, explicitly via `cache_control` on Anthropic. A cache hit needs the new request to share a byte-identical prefix with the last one, so **every rewrite of already-sent history costs full price for every token from the rewrite point onward**.
Expand Down Expand Up @@ -189,15 +235,52 @@ In input-token spend that is −9.2% to −21.3% on DeepSeek and −15.2% to −
Prevents runaway agents:

```rust
#[non_exhaustive]
pub struct ExecutionLimits {
pub max_turns: usize, // Default: 50
pub max_total_tokens: usize, // Default: 1,000,000
pub max_duration: Duration, // Default: 600s (10 min)
pub max_consecutive_identical_tool_calls: Option<usize>, // Default: Some(3)
}
```

When a limit is reached, the agent stops with a message like `"[Agent stopped: Max turns reached (50/50)]"`.

## Loop detection

The cheapest catastrophic failure is a model calling one tool with the same
arguments forever. The three limits above all fire eventually — but only after
the run has burned its entire turn, token and wall-clock budget to discover it
achieved nothing.

`max_consecutive_identical_tool_calls` is on by default at `Some(3)`, with two
escalations that mirror the house pattern of steering before aborting:

1. **First trip** injects a steering message and continues. A model repeating a
call is often retrying something transient, and aborting immediately would
regress that legitimate case.
2. **A later trip on the same signature** stops the run.

Both emit `AgentEvent::LoopDetected { tool_name, repetitions, aborted }`, so a
UI can show the intervention and an audit can tell a loop abort from a
turn-limit stop.

Signatures compare `serde_json::Value`, not serialized text — two calls
differing only in key order are the same call. Counting covers duplicates
*within* one batch as well as across turns, because `ToolExecutionStrategy`
defaults to `Parallel` and a model can emit the same call three times in a
single message.

**Consecutive**, and that word is load-bearing: a different call resets the
streak, so an alternating `[a, b, a, b, …]` loop is *not* detected. That is a
deliberate trade — an agent working through a list calls one tool repeatedly and
legitimately, and a detector that fired on interleaved repeats would be worse
than none.

```rust
ExecutionLimits::default().with_max_consecutive_identical_tool_calls(None) // off
```

## Disabling Context Management

```rust
Expand Down
45 changes: 45 additions & 0 deletions docs/reference/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,13 +83,21 @@ let config = ContextConfig::from_context_window(200_000);
Prevents runaway agents:

```rust
#[non_exhaustive] // build with Default::default() + with_*
pub struct ExecutionLimits {
pub max_turns: usize, // Default: 50
pub max_total_tokens: usize, // Default: 1,000,000
pub max_duration: Duration, // Default: 600s
pub max_consecutive_identical_tool_calls: Option<usize>, // Default: Some(3)
}
```

```rust
ExecutionLimits::default()
.with_max_turns(20)
.with_max_consecutive_identical_tool_calls(None) // disable loop detection
```

## ThinkingLevel

```rust
Expand All @@ -111,14 +119,51 @@ compat flags enable it; the Google and Bedrock providers currently ignore
Token pricing per million:

```rust
#[non_exhaustive] // build with new() + with_*, not a literal
pub struct CostConfig {
pub input_per_million: f64,
pub output_per_million: f64,
pub cache_read_per_million: f64,
pub cache_write_per_million: f64,
pub context_tiers: Vec<ContextTier>, // empty = one flat rate at every size
}
```

Cache rates are set with builders rather than positionally. Four same-typed
`f64` arguments in a row is a transposition hazard, and no vendor publishes them
in one order — Anthropic lists input / cache-write / cache-read / output, OpenAI
lists input / cached-input / output:

```rust
CostConfig::new(5.0, 30.0) // input, output — output is always dearer
.with_cache_read(0.5)
.with_cache_write(6.25)
```

All-zero rates mean **pricing unknown**, not free. `is_configured()` reports
which, and `session_cost_usd()` returns `None` for an unpriced model rather than
$0.

### Context tiers

Some vendors charge more above a prompt-size threshold. `cost_usd` selects by
the request's **prompt** tokens (`input + cache_read + cache_write`), so a long
reply to a short prompt stays on the base rate:

```rust
CostConfig::new(5.0, 30.0)
.with_context_tier(ContextTier::new(272_000, 10.0, 45.0).with_cache_read(1.0))
```

Tiers are kept sorted, and `cost_usd` takes the last one the prompt clears, so a
multi-step schedule works. **No shipped preset sets one** — see
`ModelConfig::gpt_5_5`'s docs for why the one candidate stayed flat.

One caveat if you add a tier: prompt size is derived as
`input + cache_read + cache_write`, which holds only where the provider
subtracts cached tokens out of `input`. `bedrock.rs` populates neither cache
field, so a heavily-cached prompt reads small there.

## ModelConfig Presets

yoagent provides first-class `ModelConfig::*` constructors for Anthropic, OpenAI, Google Gemini, xAI, Groq, DeepSeek, Mistral, MiniMax, Z.ai, Qwen, Ollama, and local OpenAI-compatible servers.
Expand Down
2 changes: 1 addition & 1 deletion src/agent_loop.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
//! The core agent loop: prompt → LLM stream → tool execution → repeat.
//!
//! This is the heart of yoagent. Inspired by pi-agent-core's agent-loop.ts:
//! This is the heart of yoagent:
//!
//! - `agent_loop()` starts with new prompt messages
//! - `agent_loop_continue()` resumes from existing context
Expand Down
Loading