Put an AI agent in a verified improvement loop.
Set constraints and metrics; the agent proposes changes in a loop.
Only changes that pass checks get merged. Improvements compound across iterations.
works with Claude Code · Codex · OpenCode
How it works · Quick start · Examples · Configuration · Other features · FAQ
 |
 |
AutoHelix is a lightweight harness that lets you point an agent at a goal and leave it looping unattended. It provides the isolation, validation, scope, and budget control to make that safe, and it's general enough for any measurable goal, from optimizing code to training models.
Looping an agent can be far more effective than a single prompt, because later iterations fix and build on what came before, and because it gives you a knob for controlling how much effort (time or cost) to put toward the goal rather than letting the agent decide when it's done. AutoHelix makes this pattern easy to run and customize, so you can let the agent climb for as long as you want.
You define a goal, the constraints that must hold, and the metrics to track. The agent proposes changes and the harness decides if they land.
Core features:
- Isolation. Each iteration runs in a fresh git worktree. A rejected attempt's code is cleanly discarded.
- Scope enforcement. Only files marked
editablecan be changed; everything else is frozen. - Constraints gate changes. Tests, type-checks, or any command that must exit 0. Break a constraint and the iteration is discarded.
- Metrics track progress. Metrics are captured and shown to the next iteration. Optionally gate on them to reject regressions. Logs and artifacts can be saved too.
- Persistent memory. Each iteration is a fresh session, but the agent reads and writes notes that survive across all iterations, including rejected ones.
- Budget control. Cap the run by iterations, time, or cost.
Install from source:
git clone https://github.com/awslabs/AutoHelix.git
cd AutoHelix
python -m venv .venv && source .venv/bin/activate
pip install -e .Or pip install autohelix for just the CLI, without the bundled examples.
AutoHelix uses Claude Code by default (looks for claude on your PATH). Codex and
OpenCode are also supported — see agent setup.
cd my-project
autohelix init # initialize config and state
autohelix run # start the loopinit creates a commented autohelix.yaml to fill in — see the configuration section
below. Other commands are described in the CLI reference.
The examples/ directory includes concrete tasks ranging from
reviewer-driven refinement to workflow and algorithm optimization. To list
what's available:
python examples/setup_example.py --listCode optimization. AutoHelix can iteratively improve the performance of existing code while preserving user-defined correctness constraints. We tested this capability on two established benchmarks: AlgoTune for CPU algorithms and KernelBench for GPU kernels. In both evaluations, median speedup increased with each iteration, reaching approximately 8.5× across 154 AlgoTune tasks and 3.12× across 270 KernelBench tasks after five iterations.
 AlgoTune · CPU algorithms |
 KernelBench · GPU kernels |
Both runs used Claude Opus 4.8; KernelBench ran on an H200. The AlgoTune result
uses our own evaluation setup, which is not identical to its official harness.
See the examples/algotune and
examples/kernelbench setups, plus the
results/algotune and
results/kernelbench methodology and per-task results.
Model training. The same loop works when each iteration is a training run rather than a
code edit: the agent writes the training code and trains a model, and AutoHelix scores the
result with a frozen eval the agent can't modify. In
examples/posttrain
(inspired by PostTrainBench), the agent is asked to post-train
Qwen3-4B-Base on GSM8K. Given an open-ended goal (no method prescribed), the agent
worked its way up from supervised fine-tuning through rejection sampling to reinforcement
learning (GRPO) — reaching 93% accuracy over 8 iterations on one H200. See the
ML Experiments Guide for setup tips.
Advanced agent loops. AutoHelix can also support longer-running autonomous
work. The research example iterates through experiments,
evidence, and reports; task-queue works through a
persistent list of coding tasks; and
workflow-optimization improves a
model-powered workflow's prompts, skills, context, and orchestration against a
frozen evaluator.
In the wild. Beyond the bundled examples, we've used AutoHelix on real projects. In Hybrid Model Factory — an open-source toolkit for training hybrid architectures (Attention + SSMs) from our team — the agent autonomously developed a faster sequence-parallelism implementation. Over multiple runs (with some algorithmic hints in the goal), it arrived at a state-passing approach that propagates recurrent state between GPUs instead of redistributing the full sequence, reaching ~5× faster per layer and 1.6× faster end-to-end training at 128K sequence length on a 27B model.
A run is defined by a single autohelix.yaml. Every field is documented in
docs/config.md.
Project layout
my-project/
├── autohelix.yaml # your config
└── .autohelix/ # runtime state (gitignored)
├── prompt.md # the prompt the agent gets each iteration (editable)
├── history.jsonl # per-iteration results
├── notes/ # agent's persistent notes (the only agent-owned state)
├── observations/ # per-iteration captured metrics output and artifacts
├── reviews/ # reviewer feedback per iteration (if a reviewer is set)
├── hints.md # messages from `autohelix hint`
├── logs/ # per-iteration prompts and agent output
├── output/ # dashboard.html and reports
└── worktrees/ # temp iteration dirs (recreated each time)
Report
autohelix report runs a one-shot agent over the run's history, notes, and git log and
writes a summary to .autohelix/output/report.md. See CLI → report.
Reviewer
Beyond quantitative metrics, you can add a reviewer: block with a custom prompt to get qualitative feedback from an LLM. Each review is archived to .autohelix/reviews/iter-N.md,
and the latest one guides the next iteration. See Concepts → Reviewer.
Dashboard
After every iteration, AutoHelix regenerates a self-contained HTML dashboard at
.autohelix/output/dashboard.html. You can open it in a browser
to watch metrics, outcomes, and per-iteration changes as the run progresses. See
Concepts → Dashboard.
Time limits
Two time controls:
budget.time— caps total wall-clock for the whole run. Checked before each iteration starts; doesn't interrupt one in progress.budget.iteration_time— per-iteration deadline. A hook injects a countdown into the agent's context after each tool call, so it can wrap up and write notes before the deadline hits.
See Config → budget.
Docker sandbox
Optionally run AutoHelix inside Docker for filesystem isolation. The agent can't reach
your home directory, SSH keys, or other projects. Build the image with docker/build.sh
and run via docker/run.sh. See Docker sandbox.
Hints
autohelix hint "..." (run in a second terminal) drops a note the running agent picks up
at the start of its next iteration. Useful to steer a run without stopping it. See
CLI → hint.
Full customization
Beyond autohelix.yaml, you can edit the full per-iteration prompt template
and control how the agent organizes its persistent notes. See
Concepts → Prompt Template and
Concepts → Notes.
Parallel runs (alpha)
autohelix parallel executes multiple AutoHelix runs concurrently, each with its own config and
worktree, sharing notes at a configurable frequency so each agent can see what the others
have tried. See CLI → parallel.
How does AutoHelix differ from other "agent in a loop" tools?
Running AI agents iteratively is becoming a popular strategy — a kind of test-time scaling
for agents, where more iterations buy more quality. Tools like
autoresearch,
Ralph, and Claude Code's
/goal and /loop commands each implement variations of this pattern.
Compared to other tools, AutoHelix adds a minimal amount of structure to ensure that long-running autonomous loops don't go off track, so you can let your agent work while keeping control over the direction.
| AutoHelix | autoresearch | Ralph | /goal |
/loop |
|
|---|---|---|---|---|---|
| Goal-directed | ✓ any metric(s) | ✓ val loss | ✓ task list | ✓ boolean | ✗ |
| Agent-agnostic | ✓ | ✓ | ✓ | ✗ | ✗ |
| Validation gates | ✓ | ~ self-checks | ✗ | ✗ | ✗ |
| Isolation on failure | ✓ worktree | ~ self-resets | ✗ | ✗ | ✗ |
| Fresh context each iteration | ✓ | ✗ | ✓ | ✗ | ✗ |
| Memory across iterations | ✓ notes, metrics, artifacts | session | progress file | session | session |
| Scope enforcement | ✓ | ~ by instruction | ✗ | ✗ | ✗ |
| Budget control | ✓ iters, time, cost | ✗ | iters only | iters only | ✗ |
| Time-aware agent | ✓ | ✗ | ✗ | ✗ | ✗ |
| Parallel exploration | ✓ | ✗ | ✗ | ✗ | ✗ |
What about reward hacking?
The agent optimizes what you measure: if your benchmark has loopholes, the agent may find and exploit them, and more iterations apply more pressure to do so. Autonomous loops shift the burden from "write perfect code" to "write strong checks."
You can look out for gaming by reading autohelix report, the agent's notes, or a
reviewer's feedback. Then close the loophole with a new
constraint and re-run.
Is it safe to run unattended?
AutoHelix isolates the agent at two levels:
- Git worktree (default) — each iteration runs in a separate working copy. Rejected iterations are deleted entirely; accepted ones merge back cleanly.
- Docker sandbox (optional) — filesystem isolation: the agent can't access your home directory, SSH keys, or other projects. (Network is not isolated, since the agent needs it to reach the model API.)
Worktree isolation protects your codebase from bad iterations, but it is not a security sandbox — the agent still runs on your host and could in principle access the filesystem or network. For focused optimization tasks (the typical use case), the agent works toward a narrow goal, not exploring adversarially, so running without a sandbox is usually a reasonable tradeoff. For stronger guarantees, or on sensitive machines, use the Docker mode.
We'd love to hear how you're using AutoHelix — the goals you point it at, patterns you wish it supported, or rough edges you hit. Please open an issue for bugs, ideas, or use cases you'd like to share. Contributions are welcome.
@software{autohelix2026,
title = {AutoHelix: Verified Iteration for AI Agents},
author = {Trager, Matthew and Mansimov, Elman and Zhang, Yi and Xia, Wei and Soatto, Stefano},
year = {2026},
version = {0.1.1},
url = {https://github.com/awslabs/AutoHelix}
}AutoHelix is licensed under the Apache License 2.0.