Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,8 @@ const agent = createSmartAgent({
model,
tools: [echo],
useTodoList: true,
limits: { maxToolCalls: 5, maxToken: 8000 },
limits: { maxToolCalls: 5 },
summarization: { maxTokens: 8000 },
tracing: { enabled: true },
});

Expand All @@ -98,7 +99,7 @@ const result = await agent.invoke({
console.log(result.content);
```

The smart wrapper injects a system prompt, manages TODO tooling, and runs summarization passes whenever `limits.maxToken` would be exceeded.
The smart wrapper injects a system prompt, manages TODO tooling, and runs summarization passes whenever `summarization.maxTokens` would be exceeded.

### Base agent (minimal loop)

Expand Down Expand Up @@ -130,7 +131,7 @@ console.log(res.content);

## Key capabilities

- **Summarization pipeline** – automatic chunking keeps tool call history within `contextTokenLimit` / `summaryTokenLimit`, archiving originals so `get_tool_response` can fetch them later.
- **Summarization pipeline** – automatic compaction keeps history bounded via `summarization.maxTokens` and a bounded summarization prompt (`summaryPromptMaxTokens`), archiving originals so `get_tool_response` can fetch them later.
- **Planning discipline** – when `useTodoList` is true the system prompt enforces a plan-first workflow and emits `plan` events as todos change.
- **Structured output** – supply `outputSchema` and the framework adds a hidden `response` finalize tool; parsed JSON is returned as `result.output`.
- **Usage normalization** – provider `usage` blobs are normalized into `{ prompt_tokens, completion_tokens, total_tokens }` with cached token tracking and totals grouped by model.
Expand Down Expand Up @@ -186,7 +187,7 @@ OPENAI_API_KEY=... npx tsx basic/basic.ts
The agent is a deterministic while-loop – no external graph runtime. Each turn flows through:

1. **resolver** – normalize state (messages, counters, limits).
2. **contextSummarize** (optional) – when token estimates exceed `limits.maxToken`, archive heavy tool outputs.
2. **contextSummarize** (optional) – when token estimates exceed `summarization.maxTokens`, archive heavy tool outputs.
3. **agent** – invoke the model (binding tools when supported).
4. **tools** – execute proposed tool calls with configurable parallelism.
5. **toolLimitFinalize** – if tool-call cap is hit, inject a system notice so the next assistant turn must answer directly.
Expand Down Expand Up @@ -245,7 +246,7 @@ npm publish --access public
## Troubleshooting

- **Missing tool calls** – ensure your model supports `bindTools`. If not, wrap with `withTools(model, tools)` to provide best-effort behavior.
- **Summaries too aggressive** – adjust `limits.maxToken`, `contextTokenLimit`, and `summaryTokenLimit`, or disable with `summarization: false`.
- **Summaries too aggressive** – adjust `summarization.maxTokens` / `summarization.summaryPromptMaxTokens`, or disable with `summarization: false`.
- **Large tool responses** – return structured payloads and rely on `get_tool_response` for raw data instead of dumping megabytes inline.
- **Usage missing** – some providers do not report usage; customize `usageConverter` to normalize proprietary shapes.

Expand Down
18 changes: 10 additions & 8 deletions docs/api/agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -267,18 +267,20 @@ const customModel: ModelAdapter = {
Monitor agent execution via events:

```typescript
type SmartAgentEvent =
| { type: "plan"; version: number; todoList: TodoItem[] }
| { type: "tool_execution"; tool: string; args: any; result: any }
| { type: "summarization"; summary: string; archivedCount: number }
| { type: "pause"; reason: string; metadata: any }
| { type: "resume"; stage: string }
| { type: "error"; error: Error };
type SmartAgentEvent =
| { type: "plan"; source: string; operation?: string; version?: number }
| { type: "tool_call"; phase: "start" | "success" | "error" | "skipped"; name: string }
| { type: "summarization"; summary: string; messagesCompressed?: number }
| { type: "finalAnswer"; content: string }
| { type: "metadata"; modelName?: string; usage?: any }
| { type: "progress"; stage?: string; message?: string }
| { type: "stream"; text: string; isFinal?: boolean }
| { type: "cancelled"; stage?: string; reason?: string };
```

## See Also

- [Tools API](/api/tools) - Creating and using tools
- [Nodes API](/api/nodes) - Understanding the execution graph
- [Types API](/api/types) - Complete TypeScript definitions
- [State Management](/guide/state-management) - Working with agent state
- [State Management](/state-management) - Working with agent state
80 changes: 29 additions & 51 deletions docs/api/types.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,10 +173,7 @@ type TodoStatus = "not-started" | "in-progress" | "completed";
```typescript
interface AgentLimits {
maxToolCalls?: number; // Default: 50
maxParallelTools?: number; // Default: 5
maxToken?: number; // Default: 10000
contextTokenLimit?: number; // Default: 8000
summaryTokenLimit?: number; // Default: 1000
maxParallelTools?: number; // Maximum tools per turn
}
```

Expand Down Expand Up @@ -207,26 +204,28 @@ Note: `SmartAgentTracingConfig` is an alias for `TracingConfig`.
```typescript
type SmartAgentEvent =
| PlanEvent
| ToolExecutionEvent
| ToolCallEvent
| SummarizationEvent
| PauseEvent
| ResumeEvent
| ErrorEvent;
| FinalAnswerEvent
| MetadataEvent
| ProgressEvent
| StreamEvent
| CancelledEvent;

interface PlanEvent {
type: "plan";
version: number;
todoList: TodoItem[];
timestamp: number;
source: "manage_todo_list" | "system";
operation?: "write" | "read";
version?: number;
}

interface ToolExecutionEvent {
type: "tool_execution";
tool: string;
args: any;
result: any;
duration: number;
timestamp: number;
interface ToolCallEvent {
type: "tool_call";
phase: "start" | "success" | "error" | "skipped";
name: string;
args?: any;
result?: any;
durationMs?: number;
}

interface SummarizationEvent {
Expand Down Expand Up @@ -255,24 +254,10 @@ interface SummarizationEvent {
archivedCount?: number;
}

interface PauseEvent {
type: "pause";
reason: string;
metadata?: any;
timestamp: number;
}

interface ResumeEvent {
type: "resume";
stage: string;
timestamp: number;
}

interface ErrorEvent {
type: "error";
error: Error;
phase?: string;
timestamp: number;
interface CancelledEvent {
type: "cancelled";
stage?: string;
reason?: string;
}
```

Expand All @@ -285,25 +270,18 @@ interface AgentInvokeResult {
content: string; // Final assistant message
output?: any; // Parsed structured output
state: SmartState; // Final state
usage?: UsageInfo; // Token usage
error?: Error; // Error if failed
paused?: boolean; // True if paused
metadata: { usage?: any }; // Metadata (including normalized usage)
}
```

### UsageInfo
### Usage

```typescript
interface UsageInfo {
input_tokens: number;
output_tokens: number;
total_tokens: number;

// Provider-specific (optional)
cache_read_tokens?: number;
cache_creation_tokens?: number;
reasoning_tokens?: number;
}
// Usage is exposed under result.metadata.usage and mirrors normalized provider output.
type Usage = {
perRequest?: Array<any>;
totals?: Record<string, { input: number; output: number; total: number; cachedInput: number }>;
};
```

## State Management
Expand Down Expand Up @@ -416,4 +394,4 @@ function isSystemMessage(msg: Message): msg is SystemMessage;

- [Agent API](/api/agent) - Agent creation and configuration
- [Tools API](/api/tools) - Tool development
- [State Management](/guide/state-management) - Working with state
- [State Management](/state-management) - Working with state
2 changes: 1 addition & 1 deletion docs/architecture/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Decisions about summarization or finalize insertion are factored into small "dec

- Large tool outputs are stored in `toolHistory`. When compaction triggers, older heavy entries are summarized (rewritten) and moved to an archived list with a reversible reference (executionId).
- A companion tool `get_tool_response` allows the model to request raw unsummarized data for a specific execution id when needed, mitigating lossiness.
- Targets: `contextTokenLimit` for working context size, `summaryTokenLimit` for each compressed block. Defaults are intentionally conservative.
- Targets: `summarization.maxTokens` for when compaction should run, and `summarization.summaryPromptMaxTokens` for bounding summarization prompt size.

## Planning Mode

Expand Down
20 changes: 11 additions & 9 deletions docs/core-concepts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,20 +59,22 @@ Provide `outputSchema` (Zod). The framework:

## 7. Limits

`AgentLimits` control throughput and summarization (also exported as `SmartAgentLimits` for backward compatibility):
`AgentLimits` control throughput (also exported as `SmartAgentLimits` for backward compatibility):
- `maxToolCalls` – total tool executions allowed per invocation.
- `maxParallelTools` – concurrent tool executions per agent turn.
- `maxToken` – token threshold before the next model call; exceeding it triggers `contextSummarize`.
- `contextTokenLimit` – target token budget for the live transcript.
- `summaryTokenLimit` – target size of each generated summary (per chunk).

Summarization controls live under `SmartAgentOptions.summarization`:
- `summarization.maxTokens` – threshold before the next model call; exceeding it triggers `contextSummarize`.
- `summarization.summaryPromptMaxTokens` – cap for summarization prompt size.
- `summarization.promptTemplate` – optional custom prompt template.

## 8. Summarization lifecycle
1. Estimate token usage using `countApproxTokens` (~4 chars per token).
2. When over budget, chunk the transcript (keeping tool-call groups together).
3. Ask the model to summarize each chunk; iteratively merge partials.
4. Replace tool responses in `messages` with `SUMMARIZED executionId:'...'` markers.
5. Move originals to `toolHistoryArchived` so `get_tool_response` can fetch them later.
6. Emit a `summarization` event with the merged summary and archive count.
2. When over budget, build a bounded summarization prompt (`summaryPromptMaxTokens`).
3. Include previous summary context for iterative summarization.
4. Replace tool responses in `messages` with `SUMMARIZED` placeholders.
5. Append a synthetic `summarize_context` assistant/tool pair with the summary.
6. Emit a `summarization` event with token/cost metadata when available.

## 9. Pause & resume runs

Expand Down
4 changes: 2 additions & 2 deletions docs/faq/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ Your model may not support structured tool calls. Try:
- Switching to a model that supports OpenAI-style tool calling (e.g. GPT-4o variants).

## When does summarization run?
When `limits.maxToken` would be exceeded before the next model call, the `contextSummarize` node compacts history.
When `summarization.maxTokens` would be exceeded before the next model call, the `contextSummarize` node compacts history.

## How do I disable summarization?
It is enabled by default. Pass `summarization: false` to `createSmartAgent({ ... })` to turn it off. When disabled, `limits.maxToken` will not trigger compaction.
It is enabled by default. Pass `summarization: false` to `createSmartAgent({ ... })` to turn it off. When disabled, token-threshold compaction is skipped.

## Can I use MCP tools?
Yes. Most MCP clients expose LangChain-style tools. Pass them through `fromLangchainTools(...)` first, then include the returned objects in the `tools` array. They behave like any other SDK-native tool.
Expand Down
5 changes: 3 additions & 2 deletions docs/getting-started/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,8 @@ const agent = createSmartAgent({
model,
tools: [echo],
useTodoList: true,
limits: { maxToolCalls: 5, maxToken: 6000 },
limits: { maxToolCalls: 5 },
summarization: { maxTokens: 6000 },
tracing: { enabled: true },
});

Expand Down Expand Up @@ -240,7 +241,7 @@ Proceed to:
| Issue | Likely Cause | Fix |
|-------|--------------|-----|
| No tool calls emitted | Model lacks tool calling | Use OpenAI-compatible model or fake scenario |
| Summarization not triggering | `maxToken` not reached or disabled | Lower `maxToken` or remove `summarization:false` |
| Summarization not triggering | `summarization.maxTokens` not reached or summarization disabled | Lower `summarization.maxTokens` or remove `summarization:false` |
| Parsed output missing | Schema mismatch / invalid JSON | Inspect `res.content`, adjust prompt, broaden schema |
| Handoff ignored | Tool not included | Ensure `handoffs` array includes the target agent |
| Trace file missing | `tracing.enabled` false | Enable tracing or ensure the process can write to `logs/` |
Expand Down
2 changes: 1 addition & 1 deletion docs/guide/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Decisions about summarization or finalize insertion are factored into small "dec

- Large tool outputs are stored in `toolHistory`. When compaction triggers, older heavy entries are summarized (rewritten) and moved to an archived list with a reversible reference (executionId).
- A companion tool `get_tool_response` allows the model to request raw unsummarized data for a specific execution id when needed, mitigating lossiness.
- Targets: `contextTokenLimit` for working context size, `summaryTokenLimit` for each compressed block. Defaults are intentionally conservative.
- Targets: `summarization.maxTokens` for when compaction should run, and `summarization.summaryPromptMaxTokens` for bounding summarization prompt size.

## Planning Mode

Expand Down
4 changes: 2 additions & 2 deletions docs/guide/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ Your model may not support structured tool calls. Try:
- Switching to a model that supports OpenAI-style tool calling (e.g. GPT-4o variants).

## When does summarization run?
When `limits.maxToken` would be exceeded before the next model call, the `contextSummarize` node compacts history.
When `summarization.maxTokens` would be exceeded before the next model call, the `contextSummarize` node compacts history.

## How do I disable summarization?
It is enabled by default. Pass `summarization: false` to `createSmartAgent({ ... })` to turn it off. When disabled, `limits.maxToken` will not trigger compaction.
It is enabled by default. Pass `summarization: false` to `createSmartAgent({ ... })` to turn it off. When disabled, token-threshold compaction is skipped.

## Can I use MCP tools?
Yes. Most MCP clients expose LangChain-style tools. Pass them through `fromLangchainTools(...)` first, then include the returned objects in the `tools` array. They behave like any other SDK-native tool.
Expand Down
5 changes: 3 additions & 2 deletions docs/guide/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,8 @@ const agent = createSmartAgent({
model,
tools: [echo],
useTodoList: true,
limits: { maxToolCalls: 5, maxToken: 6000 },
limits: { maxToolCalls: 5 },
summarization: { maxTokens: 6000 },
tracing: { enabled: true },
});

Expand Down Expand Up @@ -235,7 +236,7 @@ Proceed to:
| Issue | Likely Cause | Fix |
|-------|--------------|-----|
| No tool calls emitted | Model lacks tool calling | Use OpenAI-compatible model or fake scenario |
| Summarization not triggering | `maxToken` not reached or disabled | Lower `maxToken` or remove `summarization:false` |
| Summarization not triggering | `summarization.maxTokens` not reached or summarization disabled | Lower `summarization.maxTokens` or remove `summarization:false` |
| Parsed output missing | Schema mismatch / invalid JSON | Inspect `res.content`, adjust prompt, broaden schema |
| Handoff ignored | Tool not included | Ensure `handoffs` array includes the target agent |
| Trace file missing | `tracing.enabled` false | Enable tracing or ensure the process can write to `logs/` |
Expand Down
23 changes: 10 additions & 13 deletions docs/guide/limits-tokens.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,8 @@

- **`maxToolCalls`** – total tool executions allowed across the entire invocation. Once reached, additional tool calls are skipped and a finalize message is injected.
- **`maxParallelTools`** – maximum concurrent tool executions per agent turn (default 1). Adjust to balance throughput vs. rate limits.
- **`maxToken`** – estimated token threshold for the *next* agent turn. Exceeding this triggers the summarization node before the model call.
- **`contextTokenLimit`** – desired size of the live transcript after summarization (used as a target, not a hard cap).
- **`summaryTokenLimit`** – target length for each generated summary chunk (defaults to a generous value if omitted).
- **`summarization.maxTokens`** – estimated token threshold for the *next* agent turn. Exceeding this triggers the summarization node before the model call.
- **`summarization.summaryPromptMaxTokens`** – upper bound for the summarization prompt size (keeps summarization calls within model context limits).

## Tool limit finalize

Expand All @@ -23,19 +22,17 @@ On the next agent turn, the model sees the finalize notice and must produce a di
Summarization is enabled by default for smart agents. It activates when:

```
estimatedTokens(messages) > limits.maxToken
estimatedTokens(messages) > summarization.maxTokens
```

Steps:
1. Chunk the transcript while keeping tool call/response pairs together.
2. Summarize each chunk using the configured model.
3. Merge partial summaries iteratively to respect `summaryTokenLimit`.
4. Replace tool responses with `SUMMARIZED executionId:'...'` markers.
5. Move original tool outputs to `toolHistoryArchived`.
6. Add a synthetic assistant/tool pair labelled `context_summarize` containing the merged summary.
7. Emit a `summarization` event and reset `toolHistory` for future runs.
1. Build a bounded summarization prompt (uses `summaryPromptMaxTokens`).
2. Optionally include the previous summary (hierarchical summary chaining).
3. Replace tool response content with `SUMMARIZED` to reduce token load.
4. Append a synthetic assistant/tool pair labelled `summarize_context` containing the summary.
5. Store the latest summary in `state.summaries` for the next round.

Disable summarization entirely via `summarization: false`. When disabled, `maxToken` is ignored.
Disable summarization entirely via `summarization: false`. When disabled, threshold-based compaction is skipped.

## Token heuristics

Expand All @@ -44,6 +41,6 @@ Disable summarization entirely via `summarization: false`. When disabled, `maxTo
## Tips

- Return concise tool payloads to minimize summarization churn. Keep raw content accessible via IDs or `get_tool_response`.
- Increase `summaryTokenLimit` if summaries feel too lossy, but note that larger summaries consume more budget.
- Increase `summarization.maxTokens` if compaction is too frequent, or raise `summarization.summaryPromptMaxTokens` when summaries miss needed context.
- For conversations with user-provided long context, consider pre-summarizing or chunking prior to passing into the agent.
- Monitor `summarization` events to visualize how often compaction occurs and whether limits need tuning.
3 changes: 2 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,8 @@ const agent = createSmartAgent({
model,
tools: [echo],
useTodoList: true,
limits: { maxToolCalls: 5, maxToken: 6000 },
limits: { maxToolCalls: 5 },
summarization: { maxTokens: 6000 },
tracing: { enabled: true },
});

Expand Down