Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,8 @@ bsk session stop <id>

Use `bsk --help` or `bsk <command> --help` for command options. Always stop your session when finished, including after a failed task; borrowed tabs are returned to their original window.

For a local session in the last-focused user window, start with `bsk session start --in-window --json`. It creates a session-owned tab; stopping closes that tab, not the user window. Existing user tabs still require an explicit borrow. Window dimensions and remote connections do not support this option.

</details>

If your agent sandbox removes background processes after each command, use the [sandbox setup guide](docs/sandboxed-agents.md). It explains how to keep the daemon in a persistent host environment and connect with shared `BSK_HOME` and `BSK_AUTO_START=0`.
Expand Down
2 changes: 2 additions & 0 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,8 @@ bsk session stop <id>

通过 `bsk --help` 或 `bsk <命令> --help` 查看参数。完成或失败后都应结束会话;借用的标签页会归还到原窗口。

本地会话可用 `bsk session start --in-window --json` 在最近聚焦的用户窗口中新建会话页签。停止时只关闭会话页签,不关闭用户窗口;已有用户页签仍需显式借用。此选项不支持窗口尺寸参数或远程连接。

</details>

如果 Agent 沙盒会在每条命令后回收后台进程,请使用[沙盒配置指南](docs/sandboxed-agents.md):在宿主环境保持 daemon 运行,Agent 通过共享的 `BSK_HOME` 和 `BSK_AUTO_START=0` 连接。
Expand Down
8 changes: 4 additions & 4 deletions apps/extension/PRIVACY.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Depending on the commands the user (via their AI agent) sends to the selected da

| Category | What is accessed | Why |
|---|---|---|
| **Web page content** | The DOM, accessibility tree, HTML, and screenshots of pages controlled in the "Agent Window," tabs borrowed according to the browser's confirmation setting, or pages selected for user-initiated Quick Actions. | Required to read pages, locate elements, verify results, and capture requested screenshots. |
| **Web page content** | The DOM, accessibility tree, HTML, and screenshots of pages controlled in a dedicated Agent Window or an opt-in local shared-window session, tabs borrowed according to the browser's confirmation setting, or pages selected for user-initiated Quick Actions. | Required to read pages, locate elements, verify results, and capture requested screenshots. |
| **User input simulated by the agent** | Mouse clicks, keystrokes, and form values that the AI agent dispatches through the Chrome DevTools Protocol (CDP). | Required to perform automation actions the user has asked the agent to do. |
| **Website debugging evidence** | For the selected tab while capture is active: request URLs, methods, headers, request and response bodies when available, status, timing, errors and related network metadata; console messages; agent actions and supported manual input, click, submission and navigation events; bounded visible page text, form-field values and page/performance context. Configured HTTP rules and replay outcomes are also recorded. | Used to inspect and reproduce website behavior, associate operations with requests and page changes, analyze performance, and review or export debugging history. |
| **Tab and window metadata** | Tab IDs, URLs, titles and window IDs, including user tabs listed to select a tab for borrowing. | Required to target automation commands at the correct tab/window. |
Expand All @@ -50,8 +50,8 @@ The Extension requests the following Chrome permissions. Each is used solely for
- **`activeTab`** — Allow temporary access to the active tab when the user invokes the Extension, for user-initiated Quick Actions.
- **`scripting`** — Inject the full-page screenshot helper into the selected page when it is missing, such as after an extension reload.
- **`webNavigation`** — Track page navigation and frames so captures, recordings, and human-help completion checks follow the correct document.
- **`tabs`** — Inspect, create, and close tabs in the Agent Window; query tab metadata.
- **`windows`** — Create and manage the dedicated Agent Window that isolates agent activity from the user's normal browsing.
- **`tabs`** — Inspect, create, and close session-owned tabs; query tab metadata. An opt-in local shared-window session creates tabs in a user window but does not thereby control other user tabs.
- **`windows`** — Create and manage dedicated Agent Windows, or identify the user window selected for an opt-in local shared-window session. Shared-session cleanup does not close that user window.
- **`alarms`** — Periodically wake the service worker to keep the selected connection alive and renew remote device authorization.
- **`idle`** — Detect when the device returns from idle/locked so the Extension can promptly re-establish the selected WebSocket connection after the machine wakes. No idle data is stored or transmitted.
- **`notifications`** — Show a system notification to obtain user approval before borrowing a user-owned tab when browser confirmation is enabled.
Expand Down Expand Up @@ -88,7 +88,7 @@ Users can at any time:
- Uninstall the Extension from `chrome://extensions`, which removes extension storage. Audit files on the daemon host and exported copies must be deleted separately.
- Stop website debugging from Quick Actions → Website debugging, or ask the agent to stop capture. Open its history page to review or export records and delete stopped records, including when no daemon is connected. Stop an active capture before deleting its record.
- Turn operation audit off in Quick Features to stop collecting new audit operations while retaining existing audit history. Previously recorded tasks still receive their final lifecycle status. This switch does not stop website debugging capture or delete its history.
- Close the Agent Window to stop all agent automation immediately.
- Stop a session with `bsk session stop SESSION_ID`. A dedicated session also ends when its Agent Window closes; a shared-window session ends when its last controlled tab closes. Stopping a shared session does not close the user window.
- Enable confirmation before borrowing and deny tab-borrow prompts to keep existing tabs off-limits.
- Disable the connection, choose Local connection, or stop the selected daemon to disconnect.
- Revoke a paired device from the server with `bsk daemon revoke DEVICE_ID`, or use the gateway operator’s revocation controls.
Expand Down
21 changes: 21 additions & 0 deletions apps/extension/src/debug/__tests__/manager.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,27 @@ afterEach(() => {
});

describe("task-scoped debug lifecycle", () => {
it("keeps a shared task listed when its host tab query fails", async () => {
const f = await fixture();
const sessions = new SessionManager({
sharedWindow: {
host: async () => ({ id: 200, type: "normal", incognito: false }) as chrome.windows.Window,
create: async () => 8,
get: async () => ({ id: 8, windowId: 200 }) as chrome.tabs.Tab,
remove: async () => {},
},
});
await sessions.start("shared", { inWindow: true, focused: false });
const debug = new DebugManager(sessions, f.cdp, {
get: f.tabs.get,
query: vi.fn(async () => {
throw new Error("host query denied");
}),
});
active.push(debug);
await expect(debug.tasks()).resolves.toMatchObject([{ session_id: "shared" }]);
});

it("reserves global capacity before concurrent startups yield", async () => {
const f = await fixture();
active.push(f.manager);
Expand Down
34 changes: 25 additions & 9 deletions apps/extension/src/debug/manager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,10 @@ import {
} from "@/browser-driver/chromium-cdp";
import {
isAgentControlledTab,
preferredSharedTab,
type SessionContext,
type SessionManager,
sessionWindowId,
} from "@/session-manager/manager";
import type { CdpRunner, ChromeTabsApi } from "@/tools/shared";
import type { RequestFrame } from "@/transport/types";
Expand Down Expand Up @@ -318,7 +320,7 @@ export class DebugManager {
if (this.sessions.get(sessionId) !== owner)
throw new Error("task ended during debug start");
const tab = await wait(this.tabs.get(tabId));
if (tab.windowId !== this.sessions.get(sessionId)?.agentWindowId)
if (tab.windowId !== sessionWindowId(owner))
throw new Error("debug tab must remain in its Agent Window");
if (!this.runs.has(id) || state.run.state !== "capturing")
throw new Error("capture stopped during debug start");
Expand Down Expand Up @@ -557,15 +559,24 @@ export class DebugManager {
if (!params?.session_id || !this.active(params.session_id)) return;
const context = this.sessions.get(params.session_id);
if (!context) return;
const tabId =
params.tab_id ??
(
let tabId = params.tab_id;
if (tabId === undefined && context.container.mode === "in_window") {
const tabs = await deadline(
this.tabs.query({ windowId: sessionWindowId(context) }),
OBSERVATION_TIMEOUT_MS,
signal,
).catch(() => undefined);
// If Chrome cannot list the host window, retain the prior debug behavior.
tabId = tabs ? preferredSharedTab(context, tabs)?.id : context.activeTabId;
} else if (tabId === undefined) {
tabId = (
await deadline(
this.tabs.query({ windowId: context.agentWindowId, active: true }),
this.tabs.query({ windowId: sessionWindowId(context), active: true }),
OBSERVATION_TIMEOUT_MS,
signal,
)
)[0]?.id;
}
if (tabId === undefined || !this.owned(params.session_id, tabId)) return;
const state = this.active(params.session_id, tabId);
if (!state) return;
Expand Down Expand Up @@ -841,8 +852,7 @@ export class DebugManager {
);
if (!this.owned(state.run.session_id, state.run.tab_id) || state.run.state !== "capturing")
return { at, state: "unavailable" };
if (tab.windowId !== this.sessions.get(state.run.session_id)?.agentWindowId)
return { at, state: "unavailable" };
if (tab.windowId !== sessionWindowId(state.owner)) return { at, state: "unavailable" };
const lines: string[] = [];
let chars = 0;
let truncated = false;
Expand Down Expand Up @@ -988,7 +998,13 @@ export class DebugManager {
this.sync();
return Promise.all(
this.sessions.list().map(async (context) => {
const tab = (await this.tabs.query({ windowId: context.agentWindowId, active: true }))[0];
const tab =
context.container.mode === "in_window"
? preferredSharedTab(
context,
await this.tabs.query({ windowId: sessionWindowId(context) }).catch(() => []),
)
: (await this.tabs.query({ windowId: sessionWindowId(context), active: true }))[0];
const latest = [...this.runs.values()]
.filter(({ run, released }) => !released && run.session_id === context.sessionId)
.at(-1);
Expand Down Expand Up @@ -1354,7 +1370,7 @@ export class DebugManager {
const tab = await this.tabs.get(state.run.tab_id);
if (
!this.owned(params.session_id, state.run.tab_id) ||
tab.windowId !== this.sessions.get(params.session_id)?.agentWindowId
tab.windowId !== sessionWindowId(state.owner)
)
throw new Error("debug tab must remain in its Agent Window");
if (signal?.aborted) throw new Error("debug action cancelled");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ async function fixture() {
tabs: {
sendMessage,
onDetached,
onAttached: event(),
onActivated: event(),
onUpdated: event(),
onCreated: event(),
Expand Down
59 changes: 30 additions & 29 deletions apps/extension/src/entrypoints/background.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ import { attachUiChannel } from "@/lib/ui-channel";
import { attachLongScreenshot } from "@/long-screenshot/background";
import { createDisconnectCleanup } from "@/session-manager/disconnect-cleanup";
import { attachSessionEventHandler } from "@/session-manager/event-handler";
import { isAgentControlledTab, SessionManager } from "@/session-manager/manager";
import { isAgentControlledTab, SessionManager, sessionWindowId } from "@/session-manager/manager";
import {
attachBorrowNotificationButtonHandler,
attachBorrowNotificationClickHandler,
Expand Down Expand Up @@ -89,8 +89,7 @@ export default defineBackground(() => {
onDocumentChanged: (tabId) => sessions.invalidateTabRefs(tabId),
shouldAutoAcceptDialog: async (tabId) => {
const tab = await chrome.tabs.get(tabId);
const session = sessions.findByWindowId(tab.windowId);
return session !== null && (!session.remote || isAgentControlledTab(session, tabId));
return sessions.canAutoAcceptDialog(tabId, tab.windowId);
},
});
const debug = new DebugManager(sessions, cdp, chrome.tabs, Date.now, new LocalDebugArchive());
Expand Down Expand Up @@ -141,25 +140,7 @@ export default defineBackground(() => {
if (controlModes.get(sessionId) === mode) return;
controlModes.set(sessionId, mode);
const ctx = sessions.get(sessionId);
if (ctx) void pushOverlayStateForWindow(ctx.agentWindowId);
}

function overlayStateForWindow(windowId?: number): OverlayAgentStateMessage {
const ctx = typeof windowId === "number" ? sessions.findByWindowId(windowId) : null;
if (!ctx) {
return {
type: OVERLAY_AGENT_STATE,
sessionId: null,
mode: "hidden",
...nextOverlayVersion(),
};
}
return {
type: OVERLAY_AGENT_STATE,
sessionId: ctx.sessionId,
mode: controlModes.get(ctx.sessionId) ?? "control",
...nextOverlayVersion(),
};
if (ctx) void pushOverlayStateForWindow(sessionWindowId(ctx));
}

/**
Expand All @@ -169,8 +150,14 @@ export default defineBackground(() => {
*/
function overlayStateForTab(tabId?: number, windowId?: number): OverlayAgentStateMessage {
if (typeof tabId === "number" && typeof windowId === "number") {
const ctx = sessions.findByWindowId(windowId);
if (ctx && isAgentControlledTab(ctx, tabId)) return overlayStateForWindow(windowId);
const ctx = sessions.findByTabId(tabId);
if (ctx && sessionWindowId(ctx) === windowId)
return {
type: OVERLAY_AGENT_STATE,
sessionId: ctx.sessionId,
mode: controlModes.get(ctx.sessionId) ?? "control",
...nextOverlayVersion(),
};
}
return {
type: OVERLAY_AGENT_STATE,
Expand Down Expand Up @@ -207,7 +194,7 @@ export default defineBackground(() => {
}

function pushAllAgentOverlayStates(): void {
const windowIds = new Set(sessions.list().map((ctx) => ctx.agentWindowId));
const windowIds = new Set(sessions.list().map((ctx) => sessionWindowId(ctx)));
for (const windowId of windowIds) {
void pushOverlayStateForWindow(windowId);
}
Expand All @@ -231,10 +218,12 @@ export default defineBackground(() => {
}

function pushOverlayStateForAgentWindow(windowId: number): void {
if (!sessions.findByWindowId(windowId)) return;
if (!sessions.sessionsInWindow(windowId).length) return;
void pushOverlayStateForWindow(windowId);
}
chrome.tabs.onActivated.addListener((activeInfo) => {
const ctx = sessions.findByTabId(activeInfo.tabId);
if (ctx?.container.mode === "in_window") ctx.activeTabId = activeInfo.tabId;
pushOverlayStateForAgentWindow(activeInfo.windowId);
});
chrome.tabs.onUpdated.addListener((_tabId, changeInfo, tab) => {
Expand All @@ -247,7 +236,7 @@ export default defineBackground(() => {
// state; it never infers or mutates ownership from event ordering.
chrome.tabs.onCreated.addListener((tab) => {
if (typeof tab.windowId !== "number" || typeof tab.id !== "number") return;
if (!sessions.findByWindowId(tab.windowId)) return;
if (!sessions.sessionsInWindow(tab.windowId).length) return;
void pushOverlayStateForTab(tab.id, tab.windowId);
});
chrome.tabs.onDetached.addListener((tabId) => {
Expand All @@ -264,6 +253,18 @@ export default defineBackground(() => {
sessions.forgetClosedTab(tabId, { isWindowClosing: removeInfo.isWindowClosing });
debug.sync();
});
chrome.tabs.onAttached.addListener((tabId, info) => {
const ctx = sessions.findByTabId(tabId);
if (!ctx || ctx.container.mode !== "in_window" || sessionWindowId(ctx) === info.newWindowId)
return;
ctx.agentCreatedTabs.delete(tabId);
ctx.borrowedTabs.delete(tabId);
ctx.observedTabs?.delete(tabId);
ctx.refStore.invalidateTab(tabId);
void cdp.releaseSessionTab(ctx.sessionId, tabId).catch(console.warn);
void pushOverlayStateForTab(tabId, info.newWindowId);
sessions.checkEmpty(ctx);
});
// Re-sync the storage.session flag on SW startup so a previous SW's
// stale `true` does not keep waking us on every page load until the
// first mutation (review M4/M5 round 3 m-R3-1).
Expand Down Expand Up @@ -343,6 +344,7 @@ export default defineBackground(() => {
// overlay — Agent Windows boot on about:blank, which has no
// content script, so they cannot surface an authorization decision.
isAgentWindowId: (windowId) => sessions.findByWindowId(windowId) !== null,
isTabAllowed: (tabId) => sessions.findByTabId(tabId) === null,
// Resolve i18n strings per-borrow so language switches take effect
// without re-creating the dispatcher.
notificationCopy: makeBorrowNotificationCopy(),
Expand Down Expand Up @@ -460,8 +462,7 @@ export default defineBackground(() => {
if (!msg || typeof msg !== "object" || !("kind" in msg)) return false;

if (msg.kind === OVERLAY_MSG_WHO_AM_I) {
const windowId = sender.tab?.windowId;
const ctx = typeof windowId === "number" ? sessions.findByWindowId(windowId) : null;
const ctx = typeof sender.tab?.id === "number" ? sessions.findByTabId(sender.tab.id) : null;
sendResponse({ sessionId: ctx?.sessionId ?? null });
return false;
}
Expand Down
12 changes: 6 additions & 6 deletions apps/extension/src/lib/__tests__/connection-controller.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,13 @@ function handshake(

describe("computeConnectedState (protocol-based compat)", () => {
it("returns connected when daemon protocol equals extension protocol", () => {
expect(computeConnectedState(handshake("1.3", "1.3"), MIN_COMPATIBLE_PROTOCOL)).toEqual({
expect(computeConnectedState(handshake("1.4", "1.3"), MIN_COMPATIBLE_PROTOCOL)).toEqual({
kind: "connected",
});
});

it("returns version_skew when daemon protocol minor is newer", () => {
expect(computeConnectedState(handshake("1.4", "1.3"))).toEqual({
expect(computeConnectedState(handshake("1.5", "1.3"))).toEqual({
kind: "version_skew",
});
});
Expand Down Expand Up @@ -66,7 +66,7 @@ describe("computeConnectedState (protocol-based compat)", () => {
const result = computeConnectedState({
server: "browser-skill-daemon",
version: "0.1.0",
protocol_version: "1.3",
protocol_version: "1.4",
min_compatible_peer: "0.1.0",
});
expect(result).toEqual({ kind: "connected" });
Expand Down Expand Up @@ -234,7 +234,7 @@ describe("ConnectionController connectionEnabled", () => {
onDisconnected,
});
const first = transport.send.mock.calls[0]?.[0] as { id: string };
transport.emitMessage({ id: first.id, result: handshake("1.3", "1.3") });
transport.emitMessage({ id: first.id, result: handshake("1.4", "1.3") });
await vi.waitFor(() => expect(controller.snapshot().state).toBe("connected"));

vi.mocked(getLabel).mockResolvedValueOnce("Work profile");
Expand Down Expand Up @@ -299,11 +299,11 @@ describe("ConnectionController connectionEnabled", () => {
const second = transport.send.mock.calls[1]?.[0] as { id: string };
expect(second.id).not.toBe(first.id);

transport.emitMessage({ id: first.id, result: handshake("1.3", "1.3") });
transport.emitMessage({ id: first.id, result: handshake("1.4", "1.3") });
await Promise.resolve();
expect(controller.snapshot().state).not.toBe("connected");

transport.emitMessage({ id: second.id, result: handshake("1.3", "1.3") });
transport.emitMessage({ id: second.id, result: handshake("1.4", "1.3") });
await vi.waitFor(() => expect(controller.snapshot().state).toBe("connected"));
});
});
2 changes: 1 addition & 1 deletion apps/extension/src/lib/__tests__/task-preview.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ afterEach(() => {
function fixture() {
const task = {
remote: true,
agentWindowId: 10,
container: { mode: "window" as const, agentWindowId: 10 },
refStore: { documentRevision: () => 1 },
agentCreatedTabs: new Set([5]),
borrowedTabs: new Map(),
Expand Down
Loading