Problem
The context window has limited space. The current /purge tool can only remove or compress content, which may permanently discard information that might be needed later in a long running session. This forces a tradeoff between freeing space and preserving potentially useful context.
Proposed solution
Add a new operation to /purge, for example offload or swap, that moves a message or a paired ToolUse/ToolResult block to disk and replaces it in the context window with a compact pointer or retrieval ID. This is analogous to OS virtual memory paging: the context window acts like RAM, while disk acts as backing store. We can later build a hierarchical memory architecture, where hot context stays in the window, warm context resides on disk, and cold context lives in longer term storage, so the agent can page content back in when needed.
Use case
During a long coding session, the model may read many files or receive large tool outputs. Some of these outputs are no longer needed immediately but could be useful later, for example a large file read that was superseded by later edits. With the current /purge, removing that message loses the content forever. With the new operation, the content is stored on disk and replaced by a small reference. If the model later needs the content, it can retrieve it using the reference, similar to how an OS pages memory back from disk.
Alternatives considered
The existing remove operation deletes content permanently. The replace operation can compress content but may lose important details. Another workaround is to manually save content to a file and reference that file, but this requires the model to manage external storage explicitly and is error prone. The proposed operation automates this within the tool.
Impact
This would be useful in almost every long running session, especially when the context window is nearly full and the model needs to decide what to keep. It would improve workflow by reducing information loss and allowing more efficient use of the context window. The model could maintain a larger effective working set without exceeding the token limit.
Additional context
The concept is inspired by virtual memory management in operating systems, where pages are swapped between RAM and disk. Similar ideas appear in some LLM agent frameworks that implement hierarchical or tiered memory.
Problem
The context window has limited space. The current
/purgetool can only remove or compress content, which may permanently discard information that might be needed later in a long running session. This forces a tradeoff between freeing space and preserving potentially useful context.Proposed solution
Add a new operation to
/purge, for exampleoffloadorswap, that moves a message or a paired ToolUse/ToolResult block to disk and replaces it in the context window with a compact pointer or retrieval ID. This is analogous to OS virtual memory paging: the context window acts like RAM, while disk acts as backing store. We can later build a hierarchical memory architecture, where hot context stays in the window, warm context resides on disk, and cold context lives in longer term storage, so the agent can page content back in when needed.Use case
During a long coding session, the model may read many files or receive large tool outputs. Some of these outputs are no longer needed immediately but could be useful later, for example a large file read that was superseded by later edits. With the current
/purge, removing that message loses the content forever. With the new operation, the content is stored on disk and replaced by a small reference. If the model later needs the content, it can retrieve it using the reference, similar to how an OS pages memory back from disk.Alternatives considered
The existing
removeoperation deletes content permanently. Thereplaceoperation can compress content but may lose important details. Another workaround is to manually save content to a file and reference that file, but this requires the model to manage external storage explicitly and is error prone. The proposed operation automates this within the tool.Impact
This would be useful in almost every long running session, especially when the context window is nearly full and the model needs to decide what to keep. It would improve workflow by reducing information loss and allowing more efficient use of the context window. The model could maintain a larger effective working set without exceeding the token limit.
Additional context
The concept is inspired by virtual memory management in operating systems, where pages are swapped between RAM and disk. Similar ideas appear in some LLM agent frameworks that implement hierarchical or tiered memory.