Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ jobs:
# namespace and every sandboxed command dies with
# "loopback: Failed RTM_NEWADDR: Operation not permitted".
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 || true
bwrap --ro-bind / / --unshare-net --dev /dev --proc /proc -- /bin/sh -lc 'echo sandbox-ok'
bwrap --ro-bind / / --unshare-net --unshare-pid --dev /dev --proc /proc -- /bin/sh -lc 'echo sandbox-ok'
- run: npm ci --prefer-offline --no-audit --no-fund
# Coverage thresholds live in vitest.config.ts (ratcheted floor below the
# measured baseline — see that file's comment). Previously `npm test` had
Expand Down Expand Up @@ -146,7 +146,7 @@ jobs:
- name: Bubblewrap alone must not be assumed sufficient
run: |
set -uo pipefail
if bwrap --ro-bind / / --unshare-net --dev /dev --proc /proc -- /usr/bin/true 2>/dev/null; then
if bwrap --ro-bind / / --unshare-net --unshare-pid --dev /dev --proc /proc -- /usr/bin/true 2>/dev/null; then
echo "::warning::The sandbox now works without lifting the AppArmor restriction — the README's sysctl step may be obsolete on this image"
else
echo "Confirmed: installing bubblewrap alone does not give a working sandbox on this runner"
Expand All @@ -157,5 +157,5 @@ jobs:
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
# The same namespaces src/verify/runner.ts sets up. Probing a subset
# would pass on a host that denies one of the others.
bwrap --ro-bind / / --unshare-net --dev /dev --proc /proc -- /usr/bin/true
bwrap --ro-bind / / --unshare-net --unshare-pid --dev /dev --proc /proc -- /usr/bin/true
echo "OS verification sandbox is available on ubuntu-latest with bubblewrap + the AppArmor sysctl"
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ jobs:
if: steps.ver.outputs.exists == 'false'
run: |
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 || true
bwrap --ro-bind / / --unshare-net --dev /dev --proc /proc -- /bin/sh -lc 'echo sandbox-ok'
bwrap --ro-bind / / --unshare-net --unshare-pid --dev /dev --proc /proc -- /bin/sh -lc 'echo sandbox-ok'

- if: steps.ver.outputs.exists == 'false'
run: npm ci
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

## [Unreleased]

### Added

- **`advisor` role: a second, independent review of the same diff.** Ported from the harness agent patterns. A separate, independently-prompted model is asked one narrow question — *what concrete defects did the reviewer miss?* — and its only permitted effect is to make the gate **more** cautious: it may append concrete findings the reviewer did not report and raise severity, but the merged decision is the max rank of the two (`approve < revise < reject`), so an advisor `approve` can never soften a reviewer `revise`/`reject`, and a raised severity with no concrete finding is discarded. Any error, timeout, empty, or unparseable output fails **open** — `ran: false`, the reviewer's result untouched, no exit code changed. `review`, `review --max` (per area), and the `--fix` re-review loop all run it before dedupe, so its findings are deduped against history like any other. Disabled by default (a second paid call per review), configured under `autonomous.defaultRoles.advisor`; its model must come from a different family than the reviewer's or it is a second identical opinion.
- **Declarative per-role subagent settings: `tools` and `effort`.** Each role (`worker`/`reviewer`/`advisor`/…) may declare `tools.allow` / `tools.deny` and `effort`. An allow-list can only **narrow** the role's default tool set — it can never grant a tool the role would not otherwise have, so a misconfiguration cannot hand a read-only reviewer `bash`. `deny` is applied after `allow` (deny wins) and supports a trailing `*` (`scratch_*`). `effort` selects the native-loop reasoning level for the stage.

## 0.24.3 — 2026-09-28

### Added
Expand Down
29 changes: 29 additions & 0 deletions config.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,18 @@ autonomous:
# run ends on completion, the repeated-tool-call guard, or timeoutMs — a turn
# count is not a property of the task. Set a number only to cap cost hard.
# maxTurns: 0
#
# Declarative tool scoping per role. `allow` can only NARROW the role's
# default tools — it can never grant one the role would not otherwise have,
# so an allow-list naming `bash` on a read-only role stays withheld. `deny`
# is applied after `allow` (deny wins), and supports a trailing `*`
# (`scratch_*`). Use this to keep a stage's blast radius explicit instead
# of relying on the stage's defaults.
# tools:
# allow: [read_file, write_file, edit_file, bash]
# deny: [web_fetch, web_search]
# reasoning effort for this role's native-loop adapter (low|medium|high)
# effort: medium
reviewer:
enabled: true
model: gpt-5.6-sol # Correctness gate; light profile lowers this to Terra
Expand All @@ -192,6 +204,23 @@ autonomous:
# diff-scaled default (300s base); a slow model needs more — measured
# 2026-09-17: half the reviews died at 300s. maxTurns 0 = no ceiling.
timeoutMs: 600000
# Second, independently-prompted review of the SAME diff. Its only permitted
# effect is to add findings the reviewer missed and to raise severity — it
# can never soften the reviewer's verdict, and any raised severity without a
# concrete finding is discarded. DISABLED by default: it is a second paid
# call on every review.
#
# Its model must come from a DIFFERENT family than the reviewer's. On the
# reviewer's own model the advisor is a second identical opinion — the same
# weights re-deriving the same blind spots (the trap modelCompat.ts
# documents for `escalate`). The shipped reviewer default is
# deepseek/deepseek-v4-flash; z-ai/glm-5.2 is the family-independent
# alternative measured at 100% detect / 6% false-reject (6s avg) on the
# planted-defect fixtures where the reviewer scored 0% false-reject at 36s.
advisor:
enabled: false
model: z-ai/glm-5.2
timeoutMs: 45000 # 45s single-turn ceiling, as the guard arbiter uses
tester:
enabled: false
model: gpt-5.6-terra # Used only if deterministic verify cannot run
Expand Down
129 changes: 129 additions & 0 deletions src/adapters/agenticLoop.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -534,6 +534,135 @@ describe('runAgenticLoop tool exposure options', () => {
expect(toolNames).not.toContain('search_memory');
});

it('exposes only the allow-listed tools, and a deny entry removes what allow kept', async () => {
let toolNames: string[] = [];

await runAgenticLoop({
prompt: 'x',
cwd: process.cwd(),
model: 'test',
webTools: false,
memoryTools: false,
// `search_files` is allow-listed and then denied: allow can never add a
// tool back that deny removed, not even a member of `allow` itself.
toolAllow: ['read_file', 'search_files', 'write_file'],
toolDeny: ['search_files'],
maxTurns: 1,
callApi: async (_messages, tools) => {
toolNames = tools.map((tool) => tool.function.name);
return finalResp('done');
},
});

expect(toolNames).toEqual(['read_file', 'write_file']);
});

it('a deny wildcard withholds the whole scratch family', async () => {
let toolNames: string[] = [];

await runAgenticLoop({
prompt: 'x',
cwd: process.cwd(),
model: 'test',
webTools: false,
memoryTools: false,
scratchpadRunId: 'AGT-0000',
toolDeny: ['scratch_*'],
maxTurns: 1,
callApi: async (_messages, tools) => {
toolNames = tools.map((tool) => tool.function.name);
return finalResp('done');
},
});

expect(toolNames).not.toContain('scratch_write');
expect(toolNames).not.toContain('scratch_read');
expect(toolNames).toContain('read_file');
});

it('an allow-list cannot resurrect bash on a read-only run', async () => {
// The narrowing-only invariant: `allow` intersects with the composition, so
// naming a withheld tool does not expose it. A read-only run (the reviewer's
// shape) keeps bash hidden no matter what the role declared.
let toolNames: string[] = [];

await runAgenticLoop({
prompt: 'x',
cwd: process.cwd(),
model: 'test',
readOnly: true,
webTools: false,
memoryTools: false,
toolAllow: ['bash', 'read_file', 'write_file'],
maxTurns: 1,
callApi: async (_messages, tools) => {
toolNames = tools.map((tool) => tool.function.name);
return finalResp('done');
},
});

expect(toolNames).toEqual(['read_file']);
});

it('refuses a tool the role scope narrowed away, at dispatch as well as in the schema', async () => {
// The schema above and the dispatch set below are the same narrowed array,
// so a provider that emits a withheld name anyway is answered, not obeyed.
let turn = 0;
let deniedResult = '';

await runAgenticLoop({
prompt: 'x',
cwd: process.cwd(),
model: 'test',
webTools: false,
memoryTools: false,
toolAllow: ['read_file'],
maxTurns: 2,
callApi: async (messages, tools) => {
if (turn++ === 0) {
expect(tools.map((tool) => tool.function.name)).toEqual(['read_file']);
return toolCallResp('hidden-write', 'write_file', { path: 'out.txt', content: 'x' });
}
deniedResult = messages.at(-1)?.content ?? '';
return finalResp('done');
},
});

expect(deniedResult).toContain('TOOL_NOT_ALLOWED');
expect(deniedResult).toContain('write_file');
});

it('leaves MCP and coordination tools to their own flags, not the role list', async () => {
// RoleConfig.tools names built-ins (see its doc): a role narrowing to the
// file tools must not silently lose its board access, which the MCP and
// coordination flags govern.
let toolNames: string[] = [];

await runAgenticLoop({
prompt: 'x',
cwd: process.cwd(),
model: 'test',
webTools: false,
memoryTools: false,
toolAllow: ['read_file'],
maxTurns: 1,
mcpTools: [{
type: 'function',
function: { name: 'linear__get_issue', description: '', parameters: { type: 'object' } },
}],
coordinationContext: { repository: '/repo', taskId: 'supervisor', actor: 'orchestrator' },
callApi: async (_messages, tools) => {
toolNames = tools.map((tool) => tool.function.name);
return finalResp('done');
},
});

expect(toolNames).toContain('read_file');
expect(toolNames).toContain('linear__get_issue');
expect(toolNames).toContain('coordination_read');
expect(toolNames).not.toContain('write_file');
});

it('withholds the scratch tools when the run has no scratchpad', async () => {
let toolNames: string[] = [];
await runAgenticLoop({
Expand Down
61 changes: 54 additions & 7 deletions src/adapters/agenticLoop.ts
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,16 @@ export interface AgenticLoopOptions {
webTools?: boolean;
/** Expose search_memory (default true). Disabled for isolated/temp repo benchmarks. */
memoryTools?: boolean;
/**
* Declarative per-role tool scope (RoleConfig.tools), naming BUILT-IN tools.
* `toolAllow` keeps only the names it lists; `toolDeny` then removes names from
* what remains, a trailing `*` standing for a prefix (`scratch_*`). Both run over
* the built-ins left by the readOnly/shell/web rules, so they only ever NARROW —
* `allow` cannot resurrect `bash` on a readOnly run. MCP and coordination tools
* keep their own flags; the narrowed set is also the dispatch allow-list.
*/
toolAllow?: string[];
toolDeny?: string[];
/**
* Run whose scratchpad `scratch_write`/`scratch_read` address (AGT-4459).
* Absent means no scratchpad: the two tools are withheld from the model and
Expand Down Expand Up @@ -287,6 +297,33 @@ export interface AgenticLoopResult {

// ============ 에이전틱 루프 ============

/**
* Apply a role's declarative `tools.allow` / `tools.deny` (RoleConfig) to the
* built-in tools the readOnly / scratch / shell / web filters have already shaped.
*
* It runs LAST and only removes entries — an allow-list cannot resurrect a tool an
* earlier rule withheld (a `bash` in `allow` stays hidden on a readOnly run), and
* `deny` follows `allow`, so the two cannot contradict each other. A `deny` entry
* ending in `*` matches by prefix (`scratch_*`), how the scratch tools are
* addressed as a family.
*/
function applyRoleToolScope(
tools: ToolDefinition[],
toolAllow?: string[],
toolDeny?: string[],
): ToolDefinition[] {
let scoped = tools;
if (toolAllow && toolAllow.length > 0) {
const allowed = new Set(toolAllow);
scoped = scoped.filter((tool) => allowed.has(tool.function.name));
}
return toolDeny && toolDeny.length > 0
? scoped.filter((tool) => !toolDeny.some((denied) => denied.endsWith('*')
? tool.function.name.startsWith(denied.slice(0, -1))
: tool.function.name === denied))
: scoped;
}

/**
* 에이전틱 도구 루프 실행
*
Expand Down Expand Up @@ -359,6 +396,8 @@ async function runAgenticLoopInner(
bashTimeoutMs,
webTools = true,
memoryTools = true,
toolAllow,
toolDeny,
scratchpadRunId,
shellTools: requestedShellTools = true,
sandboxExecutorSessionFactory,
Expand Down Expand Up @@ -444,23 +483,31 @@ async function runAgenticLoopInner(
const visibleBaseTools = readOnly
? shellFilteredTools.filter((t) => !['write_file', 'edit_file', 'bash', 'remember'].includes(t.function.name))
: shellFilteredTools;
const tools = enableTools
const builtinTools = enableTools
? [
...visibleBaseTools,
...(filesystemTools && applyPatch && editFormat === 'json' && !readOnly ? [APPLY_PATCH_TOOL] : []),
// Not in readOnly: it spawns compiler subprocesses, matching bash's exclusion.
...(filesystemTools && diagnosticsTool && !readOnly && shellTools ? [DIAGNOSTICS_TOOL] : []),
// Both are withheld in readOnly. A read-only run exists because the
// material under inspection is untrusted, and a fetch is an outbound
// channel for anything the agent can read — the provider credential
// included. MCP servers are withheld for the mirror reason: OpenSwarm's
// own memory server exposes writes, so injected content could leave
// something behind for a later run. (INT-3189)
// Also withheld in readOnly: the material under inspection is untrusted,
// and a fetch is an outbound channel for anything the agent can read —
// the provider credential included. (INT-3189)
...(webTools && !readOnly ? WEB_TOOL_DEFINITIONS : []),
]
: [];
// MCP and coordination tools keep their own flags; the role list names built-ins.
// MCP is withheld in readOnly for the mirror reason: our memory server exposes
// writes, so injected content could leave something behind. (INT-3189)
const externalTools = enableTools
? [
...(readOnly ? [] : humanSurfaceFilteredMcp.tools),
...(readOnly || !coordinationContext ? [] : COORDINATION_TOOL_DEFINITIONS),
]
: [];
// The role's declared scope goes LAST, over the built-ins every rule above has
// already shaped, so it intersects with them instead of overriding one — see
// applyRoleToolScope. `allowedToolNames` below is built from the same result.
const tools = [...applyRoleToolScope(builtinTools, toolAllow, toolDeny), ...externalTools];
// The provider-visible schema is not an enforcement boundary. Carry the
// exact same set into dispatch so a hidden tool call cannot reach a globally
// registered MCP route (or another built-in withheld for this run).
Expand Down
2 changes: 2 additions & 0 deletions src/adapters/atlascloud.ts
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,8 @@ export class AtlasCloudCliAdapter implements CliAdapter {
webTools: options.webTools,
memoryTools: options.memoryTools,
shellTools: options.shellTools,
toolAllow: options.toolAllow,
toolDeny: options.toolDeny,
filesystemTools: options.filesystemTools,
diagnosticsTool: options.diagnosticsTool,
readOnly: options.readOnly,
Expand Down
Loading