diff --git a/README.md b/README.md index 71194f4..bd0c2d4 100644 --- a/README.md +++ b/README.md @@ -86,7 +86,8 @@ const agent = createSmartAgent({ model, tools: [echo], useTodoList: true, - limits: { maxToolCalls: 5, maxToken: 8000 }, + limits: { maxToolCalls: 5 }, + summarization: { maxTokens: 8000 }, tracing: { enabled: true }, }); @@ -98,7 +99,7 @@ const result = await agent.invoke({ console.log(result.content); ``` -The smart wrapper injects a system prompt, manages TODO tooling, and runs summarization passes whenever `limits.maxToken` would be exceeded. +The smart wrapper injects a system prompt, manages TODO tooling, and runs summarization passes whenever `summarization.maxTokens` would be exceeded. ### Base agent (minimal loop) @@ -130,7 +131,7 @@ console.log(res.content); ## Key capabilities -- **Summarization pipeline** – automatic chunking keeps tool call history within `contextTokenLimit` / `summaryTokenLimit`, archiving originals so `get_tool_response` can fetch them later. +- **Summarization pipeline** – automatic compaction keeps history bounded via `summarization.maxTokens` and a bounded summarization prompt (`summaryPromptMaxTokens`), archiving originals so `get_tool_response` can fetch them later. - **Planning discipline** – when `useTodoList` is true the system prompt enforces a plan-first workflow and emits `plan` events as todos change. - **Structured output** – supply `outputSchema` and the framework adds a hidden `response` finalize tool; parsed JSON is returned as `result.output`. - **Usage normalization** – provider `usage` blobs are normalized into `{ prompt_tokens, completion_tokens, total_tokens }` with cached token tracking and totals grouped by model. @@ -186,7 +187,7 @@ OPENAI_API_KEY=... npx tsx basic/basic.ts The agent is a deterministic while-loop – no external graph runtime. Each turn flows through: 1. **resolver** – normalize state (messages, counters, limits). -2. **contextSummarize** (optional) – when token estimates exceed `limits.maxToken`, archive heavy tool outputs. +2. **contextSummarize** (optional) – when token estimates exceed `summarization.maxTokens`, archive heavy tool outputs. 3. **agent** – invoke the model (binding tools when supported). 4. **tools** – execute proposed tool calls with configurable parallelism. 5. **toolLimitFinalize** – if tool-call cap is hit, inject a system notice so the next assistant turn must answer directly. @@ -245,7 +246,7 @@ npm publish --access public ## Troubleshooting - **Missing tool calls** – ensure your model supports `bindTools`. If not, wrap with `withTools(model, tools)` to provide best-effort behavior. -- **Summaries too aggressive** – adjust `limits.maxToken`, `contextTokenLimit`, and `summaryTokenLimit`, or disable with `summarization: false`. +- **Summaries too aggressive** – adjust `summarization.maxTokens` / `summarization.summaryPromptMaxTokens`, or disable with `summarization: false`. - **Large tool responses** – return structured payloads and rely on `get_tool_response` for raw data instead of dumping megabytes inline. - **Usage missing** – some providers do not report usage; customize `usageConverter` to normalize proprietary shapes. diff --git a/docs/api/agent.md b/docs/api/agent.md index 69fbd07..c790daf 100644 --- a/docs/api/agent.md +++ b/docs/api/agent.md @@ -267,13 +267,15 @@ const customModel: ModelAdapter = { Monitor agent execution via events: ```typescript -type SmartAgentEvent = - | { type: "plan"; version: number; todoList: TodoItem[] } - | { type: "tool_execution"; tool: string; args: any; result: any } - | { type: "summarization"; summary: string; archivedCount: number } - | { type: "pause"; reason: string; metadata: any } - | { type: "resume"; stage: string } - | { type: "error"; error: Error }; +type SmartAgentEvent = + | { type: "plan"; source: string; operation?: string; version?: number } + | { type: "tool_call"; phase: "start" | "success" | "error" | "skipped"; name: string } + | { type: "summarization"; summary: string; messagesCompressed?: number } + | { type: "finalAnswer"; content: string } + | { type: "metadata"; modelName?: string; usage?: any } + | { type: "progress"; stage?: string; message?: string } + | { type: "stream"; text: string; isFinal?: boolean } + | { type: "cancelled"; stage?: string; reason?: string }; ``` ## See Also @@ -281,4 +283,4 @@ type SmartAgentEvent = - [Tools API](/api/tools) - Creating and using tools - [Nodes API](/api/nodes) - Understanding the execution graph - [Types API](/api/types) - Complete TypeScript definitions -- [State Management](/guide/state-management) - Working with agent state +- [State Management](/state-management) - Working with agent state diff --git a/docs/api/types.md b/docs/api/types.md index bd49c6f..da779f5 100644 --- a/docs/api/types.md +++ b/docs/api/types.md @@ -173,10 +173,7 @@ type TodoStatus = "not-started" | "in-progress" | "completed"; ```typescript interface AgentLimits { maxToolCalls?: number; // Default: 50 - maxParallelTools?: number; // Default: 5 - maxToken?: number; // Default: 10000 - contextTokenLimit?: number; // Default: 8000 - summaryTokenLimit?: number; // Default: 1000 + maxParallelTools?: number; // Maximum tools per turn } ``` @@ -207,26 +204,28 @@ Note: `SmartAgentTracingConfig` is an alias for `TracingConfig`. ```typescript type SmartAgentEvent = | PlanEvent - | ToolExecutionEvent + | ToolCallEvent | SummarizationEvent - | PauseEvent - | ResumeEvent - | ErrorEvent; + | FinalAnswerEvent + | MetadataEvent + | ProgressEvent + | StreamEvent + | CancelledEvent; interface PlanEvent { type: "plan"; - version: number; - todoList: TodoItem[]; - timestamp: number; + source: "manage_todo_list" | "system"; + operation?: "write" | "read"; + version?: number; } -interface ToolExecutionEvent { - type: "tool_execution"; - tool: string; - args: any; - result: any; - duration: number; - timestamp: number; +interface ToolCallEvent { + type: "tool_call"; + phase: "start" | "success" | "error" | "skipped"; + name: string; + args?: any; + result?: any; + durationMs?: number; } interface SummarizationEvent { @@ -255,24 +254,10 @@ interface SummarizationEvent { archivedCount?: number; } -interface PauseEvent { - type: "pause"; - reason: string; - metadata?: any; - timestamp: number; -} - -interface ResumeEvent { - type: "resume"; - stage: string; - timestamp: number; -} - -interface ErrorEvent { - type: "error"; - error: Error; - phase?: string; - timestamp: number; +interface CancelledEvent { + type: "cancelled"; + stage?: string; + reason?: string; } ``` @@ -285,25 +270,18 @@ interface AgentInvokeResult { content: string; // Final assistant message output?: any; // Parsed structured output state: SmartState; // Final state - usage?: UsageInfo; // Token usage - error?: Error; // Error if failed - paused?: boolean; // True if paused + metadata: { usage?: any }; // Metadata (including normalized usage) } ``` -### UsageInfo +### Usage ```typescript -interface UsageInfo { - input_tokens: number; - output_tokens: number; - total_tokens: number; - - // Provider-specific (optional) - cache_read_tokens?: number; - cache_creation_tokens?: number; - reasoning_tokens?: number; -} +// Usage is exposed under result.metadata.usage and mirrors normalized provider output. +type Usage = { + perRequest?: Array; + totals?: Record; +}; ``` ## State Management @@ -416,4 +394,4 @@ function isSystemMessage(msg: Message): msg is SystemMessage; - [Agent API](/api/agent) - Agent creation and configuration - [Tools API](/api/tools) - Tool development -- [State Management](/guide/state-management) - Working with state +- [State Management](/state-management) - Working with state diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 794a8be..e5bb0f2 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -62,7 +62,7 @@ Decisions about summarization or finalize insertion are factored into small "dec - Large tool outputs are stored in `toolHistory`. When compaction triggers, older heavy entries are summarized (rewritten) and moved to an archived list with a reversible reference (executionId). - A companion tool `get_tool_response` allows the model to request raw unsummarized data for a specific execution id when needed, mitigating lossiness. -- Targets: `contextTokenLimit` for working context size, `summaryTokenLimit` for each compressed block. Defaults are intentionally conservative. +- Targets: `summarization.maxTokens` for when compaction should run, and `summarization.summaryPromptMaxTokens` for bounding summarization prompt size. ## Planning Mode diff --git a/docs/core-concepts/README.md b/docs/core-concepts/README.md index b50bfe4..ff1f252 100644 --- a/docs/core-concepts/README.md +++ b/docs/core-concepts/README.md @@ -59,20 +59,22 @@ Provide `outputSchema` (Zod). The framework: ## 7. Limits -`AgentLimits` control throughput and summarization (also exported as `SmartAgentLimits` for backward compatibility): +`AgentLimits` control throughput (also exported as `SmartAgentLimits` for backward compatibility): - `maxToolCalls` – total tool executions allowed per invocation. - `maxParallelTools` – concurrent tool executions per agent turn. -- `maxToken` – token threshold before the next model call; exceeding it triggers `contextSummarize`. -- `contextTokenLimit` – target token budget for the live transcript. -- `summaryTokenLimit` – target size of each generated summary (per chunk). + +Summarization controls live under `SmartAgentOptions.summarization`: +- `summarization.maxTokens` – threshold before the next model call; exceeding it triggers `contextSummarize`. +- `summarization.summaryPromptMaxTokens` – cap for summarization prompt size. +- `summarization.promptTemplate` – optional custom prompt template. ## 8. Summarization lifecycle 1. Estimate token usage using `countApproxTokens` (~4 chars per token). -2. When over budget, chunk the transcript (keeping tool-call groups together). -3. Ask the model to summarize each chunk; iteratively merge partials. -4. Replace tool responses in `messages` with `SUMMARIZED executionId:'...'` markers. -5. Move originals to `toolHistoryArchived` so `get_tool_response` can fetch them later. -6. Emit a `summarization` event with the merged summary and archive count. +2. When over budget, build a bounded summarization prompt (`summaryPromptMaxTokens`). +3. Include previous summary context for iterative summarization. +4. Replace tool responses in `messages` with `SUMMARIZED` placeholders. +5. Append a synthetic `summarize_context` assistant/tool pair with the summary. +6. Emit a `summarization` event with token/cost metadata when available. ## 9. Pause & resume runs diff --git a/docs/faq/README.md b/docs/faq/README.md index 7385b43..1c3edf8 100644 --- a/docs/faq/README.md +++ b/docs/faq/README.md @@ -14,10 +14,10 @@ Your model may not support structured tool calls. Try: - Switching to a model that supports OpenAI-style tool calling (e.g. GPT-4o variants). ## When does summarization run? -When `limits.maxToken` would be exceeded before the next model call, the `contextSummarize` node compacts history. +When `summarization.maxTokens` would be exceeded before the next model call, the `contextSummarize` node compacts history. ## How do I disable summarization? -It is enabled by default. Pass `summarization: false` to `createSmartAgent({ ... })` to turn it off. When disabled, `limits.maxToken` will not trigger compaction. +It is enabled by default. Pass `summarization: false` to `createSmartAgent({ ... })` to turn it off. When disabled, token-threshold compaction is skipped. ## Can I use MCP tools? Yes. Most MCP clients expose LangChain-style tools. Pass them through `fromLangchainTools(...)` first, then include the returned objects in the `tools` array. They behave like any other SDK-native tool. diff --git a/docs/getting-started/README.md b/docs/getting-started/README.md index 19ce095..a67a786 100644 --- a/docs/getting-started/README.md +++ b/docs/getting-started/README.md @@ -66,7 +66,8 @@ const agent = createSmartAgent({ model, tools: [echo], useTodoList: true, - limits: { maxToolCalls: 5, maxToken: 6000 }, + limits: { maxToolCalls: 5 }, + summarization: { maxTokens: 6000 }, tracing: { enabled: true }, }); @@ -240,7 +241,7 @@ Proceed to: | Issue | Likely Cause | Fix | |-------|--------------|-----| | No tool calls emitted | Model lacks tool calling | Use OpenAI-compatible model or fake scenario | -| Summarization not triggering | `maxToken` not reached or disabled | Lower `maxToken` or remove `summarization:false` | +| Summarization not triggering | `summarization.maxTokens` not reached or summarization disabled | Lower `summarization.maxTokens` or remove `summarization:false` | | Parsed output missing | Schema mismatch / invalid JSON | Inspect `res.content`, adjust prompt, broaden schema | | Handoff ignored | Tool not included | Ensure `handoffs` array includes the target agent | | Trace file missing | `tracing.enabled` false | Enable tracing or ensure the process can write to `logs/` | diff --git a/docs/guide/architecture.md b/docs/guide/architecture.md index e8d7552..e494db0 100644 --- a/docs/guide/architecture.md +++ b/docs/guide/architecture.md @@ -57,7 +57,7 @@ Decisions about summarization or finalize insertion are factored into small "dec - Large tool outputs are stored in `toolHistory`. When compaction triggers, older heavy entries are summarized (rewritten) and moved to an archived list with a reversible reference (executionId). - A companion tool `get_tool_response` allows the model to request raw unsummarized data for a specific execution id when needed, mitigating lossiness. -- Targets: `contextTokenLimit` for working context size, `summaryTokenLimit` for each compressed block. Defaults are intentionally conservative. +- Targets: `summarization.maxTokens` for when compaction should run, and `summarization.summaryPromptMaxTokens` for bounding summarization prompt size. ## Planning Mode diff --git a/docs/guide/faq.md b/docs/guide/faq.md index 6d67e88..cec39b0 100644 --- a/docs/guide/faq.md +++ b/docs/guide/faq.md @@ -9,10 +9,10 @@ Your model may not support structured tool calls. Try: - Switching to a model that supports OpenAI-style tool calling (e.g. GPT-4o variants). ## When does summarization run? -When `limits.maxToken` would be exceeded before the next model call, the `contextSummarize` node compacts history. +When `summarization.maxTokens` would be exceeded before the next model call, the `contextSummarize` node compacts history. ## How do I disable summarization? -It is enabled by default. Pass `summarization: false` to `createSmartAgent({ ... })` to turn it off. When disabled, `limits.maxToken` will not trigger compaction. +It is enabled by default. Pass `summarization: false` to `createSmartAgent({ ... })` to turn it off. When disabled, token-threshold compaction is skipped. ## Can I use MCP tools? Yes. Most MCP clients expose LangChain-style tools. Pass them through `fromLangchainTools(...)` first, then include the returned objects in the `tools` array. They behave like any other SDK-native tool. diff --git a/docs/guide/getting-started.md b/docs/guide/getting-started.md index de3350f..a4522bc 100644 --- a/docs/guide/getting-started.md +++ b/docs/guide/getting-started.md @@ -61,7 +61,8 @@ const agent = createSmartAgent({ model, tools: [echo], useTodoList: true, - limits: { maxToolCalls: 5, maxToken: 6000 }, + limits: { maxToolCalls: 5 }, + summarization: { maxTokens: 6000 }, tracing: { enabled: true }, }); @@ -235,7 +236,7 @@ Proceed to: | Issue | Likely Cause | Fix | |-------|--------------|-----| | No tool calls emitted | Model lacks tool calling | Use OpenAI-compatible model or fake scenario | -| Summarization not triggering | `maxToken` not reached or disabled | Lower `maxToken` or remove `summarization:false` | +| Summarization not triggering | `summarization.maxTokens` not reached or summarization disabled | Lower `summarization.maxTokens` or remove `summarization:false` | | Parsed output missing | Schema mismatch / invalid JSON | Inspect `res.content`, adjust prompt, broaden schema | | Handoff ignored | Tool not included | Ensure `handoffs` array includes the target agent | | Trace file missing | `tracing.enabled` false | Enable tracing or ensure the process can write to `logs/` | diff --git a/docs/guide/limits-tokens.md b/docs/guide/limits-tokens.md index 6171c8b..e96c29a 100644 --- a/docs/guide/limits-tokens.md +++ b/docs/guide/limits-tokens.md @@ -5,9 +5,8 @@ - **`maxToolCalls`** – total tool executions allowed across the entire invocation. Once reached, additional tool calls are skipped and a finalize message is injected. - **`maxParallelTools`** – maximum concurrent tool executions per agent turn (default 1). Adjust to balance throughput vs. rate limits. -- **`maxToken`** – estimated token threshold for the *next* agent turn. Exceeding this triggers the summarization node before the model call. -- **`contextTokenLimit`** – desired size of the live transcript after summarization (used as a target, not a hard cap). -- **`summaryTokenLimit`** – target length for each generated summary chunk (defaults to a generous value if omitted). +- **`summarization.maxTokens`** – estimated token threshold for the *next* agent turn. Exceeding this triggers the summarization node before the model call. +- **`summarization.summaryPromptMaxTokens`** – upper bound for the summarization prompt size (keeps summarization calls within model context limits). ## Tool limit finalize @@ -23,19 +22,17 @@ On the next agent turn, the model sees the finalize notice and must produce a di Summarization is enabled by default for smart agents. It activates when: ``` -estimatedTokens(messages) > limits.maxToken +estimatedTokens(messages) > summarization.maxTokens ``` Steps: -1. Chunk the transcript while keeping tool call/response pairs together. -2. Summarize each chunk using the configured model. -3. Merge partial summaries iteratively to respect `summaryTokenLimit`. -4. Replace tool responses with `SUMMARIZED executionId:'...'` markers. -5. Move original tool outputs to `toolHistoryArchived`. -6. Add a synthetic assistant/tool pair labelled `context_summarize` containing the merged summary. -7. Emit a `summarization` event and reset `toolHistory` for future runs. +1. Build a bounded summarization prompt (uses `summaryPromptMaxTokens`). +2. Optionally include the previous summary (hierarchical summary chaining). +3. Replace tool response content with `SUMMARIZED` to reduce token load. +4. Append a synthetic assistant/tool pair labelled `summarize_context` containing the summary. +5. Store the latest summary in `state.summaries` for the next round. -Disable summarization entirely via `summarization: false`. When disabled, `maxToken` is ignored. +Disable summarization entirely via `summarization: false`. When disabled, threshold-based compaction is skipped. ## Token heuristics @@ -44,6 +41,6 @@ Disable summarization entirely via `summarization: false`. When disabled, `maxTo ## Tips - Return concise tool payloads to minimize summarization churn. Keep raw content accessible via IDs or `get_tool_response`. -- Increase `summaryTokenLimit` if summaries feel too lossy, but note that larger summaries consume more budget. +- Increase `summarization.maxTokens` if compaction is too frequent, or raise `summarization.summaryPromptMaxTokens` when summaries miss needed context. - For conversations with user-provided long context, consider pre-summarizing or chunking prior to passing into the agent. - Monitor `summarization` events to visualize how often compaction occurs and whether limits need tuning. diff --git a/docs/index.md b/docs/index.md index 909a0e5..0a97628 100644 --- a/docs/index.md +++ b/docs/index.md @@ -88,7 +88,8 @@ const agent = createSmartAgent({ model, tools: [echo], useTodoList: true, - limits: { maxToolCalls: 5, maxToken: 6000 }, + limits: { maxToolCalls: 5 }, + summarization: { maxTokens: 6000 }, tracing: { enabled: true }, });