From 483416c9aa073d093335f0f5bd0bb3fc193e1ec4 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Fri, 21 Aug 2026 18:54:09 +0000 Subject: [PATCH 1/8] Add Devin skills for testing SDKs and CLI visual output Co-Authored-By: mish@e2b.dev --- .agents/skills/testing-cli/SKILL.md | 47 +++++++++++++++++ .agents/skills/testing-js-sdk/SKILL.md | 50 ++++++++++++++++++ .agents/skills/testing-python-sdk/SKILL.md | 44 ++++++++++++++++ .../verifying-cli-visual-output/SKILL.md | 51 +++++++++++++++++++ 4 files changed, 192 insertions(+) create mode 100644 .agents/skills/testing-cli/SKILL.md create mode 100644 .agents/skills/testing-js-sdk/SKILL.md create mode 100644 .agents/skills/testing-python-sdk/SKILL.md create mode 100644 .agents/skills/verifying-cli-visual-output/SKILL.md diff --git a/.agents/skills/testing-cli/SKILL.md b/.agents/skills/testing-cli/SKILL.md new file mode 100644 index 0000000000..5905b75e8e --- /dev/null +++ b/.agents/skills/testing-cli/SKILL.md @@ -0,0 +1,47 @@ +--- +name: testing-cli +description: "Build, run, and test the E2B CLI (packages/cli) locally. Use when changing CLI commands, running its vitest suites, or executing the CLI against real sandboxes." +--- + +# Testing the CLI + +All commands run in `packages/cli`. Use Node 24 (`nvm use 24`) and pnpm. + +## Build and run locally + +```bash +pnpm build # tsc typecheck + tsdown bundle -> dist/index.js +node dist/index.js --help +node dist/index.js sandbox list +``` + +Auth: the CLI reads `E2B_API_KEY` from the environment first, then `~/.e2b/config.json` (or `.env.local` at the repo root). Never run `e2b auth login` in headless environments — export the key instead. + +Useful non-interactive patterns: + +```bash +node dist/index.js sandbox create base --detach # returns sandbox ID, no attached terminal +node dist/index.js sandbox exec -- bash -lc 'pwd' # `--` stops CLI flag parsing +node dist/index.js sandbox kill +``` + +## Automated tests + +```bash +pnpm run test # vitest; globalSetup runs `pnpm build` first +npx vitest run tests/utils/table.test.ts # single file +``` + +- Tests spawn the **built** CLI (`dist/index.js`) via helpers in `tests/setup.ts` (`runCli`, `runCliWithPipedStdin`) — rebuild happens automatically through globalSetup, but if you bypass vitest, run `pnpm build` yourself after editing `src/`. +- `tests/commands/**` cover command behavior; some hit the real API and need `E2B_API_KEY`. +- Unit tests import from `src/` directly (vitest aliases `e2b` to `../js-sdk/src`), so keep command logic in exported, testable functions (see `buildTableRows`/`sortSandboxes` in `src/commands/sandbox/list.ts`). + +For checking rendered output (tables, colors, alignment), use the `verifying-cli-visual-output` skill. + +## Before committing + +```bash +pnpm run lint && pnpm run typecheck +``` + +Public-surface changes need a changeset (`pnpm changeset` at repo root). diff --git a/.agents/skills/testing-js-sdk/SKILL.md b/.agents/skills/testing-js-sdk/SKILL.md new file mode 100644 index 0000000000..1dbaafd60e --- /dev/null +++ b/.agents/skills/testing-js-sdk/SKILL.md @@ -0,0 +1,50 @@ +--- +name: testing-js-sdk +description: "Run and write tests for the E2B JavaScript SDK (packages/js-sdk). Use when changing js-sdk code, debugging its vitest suites, or verifying SDK behavior against real sandboxes." +--- + +# Testing the JS SDK + +All commands run in `packages/js-sdk`. Use Node 24 (`nvm use 24`) and pnpm. + +## Test projects + +Tests are organized into vitest projects in `vitest.config.mts`: + +- `unit` — everything in `tests/**/*.test.ts` except runtimes, template, and connectionConfig. Many of these are **integration tests that create real sandboxes** and require `E2B_API_KEY`. +- `template` — template builder tests (`tests/template/**`), 180s timeout, require `E2B_API_KEY`. +- `connectionConfig` — offline config tests. +- `browser` — Playwright/chromium tests (`pnpm run playwright:install` first). + +## Running + +```bash +# everything (needs E2B_API_KEY) +pnpm run test + +# one file — fastest loop, preferred while iterating +npx vitest run tests/api/inflight.test.ts + +# one project +npx vitest run --project connectionConfig + +# alternate runtimes +pnpm run test:bun # bun: unit + connectionConfig + template +pnpm run test:deno # deno: same projects +``` + +`E2B_API_KEY` is read from the environment or from `.env` via dotenv (the repo also keeps defaults in `.env.local` at the root or `~/.e2b/config.json`). Purely offline unit tests (e.g. `tests/api/inflight.test.ts`, `tests/utils.test.ts`) run without a key. + +## Writing tests + +- Use the helpers in `tests/setup.ts`: `sandboxTest` / `templateTest` fixtures create and clean up sandboxes; `isDebug` gates behavior when `E2B_DEBUG` is set (local envd at debug port). +- Offline tests that need HTTP mock the API with `msw`; see `tests/api/` for patterns. Test isolation is required (`isolate: true`) because suites patch global fetch. +- SDK changes must be mirrored in the Python SDK (sync + async) with equivalent tests — see `testing-python-sdk`. + +## Before committing + +```bash +pnpm run lint && pnpm run typecheck +``` + +Public-surface changes need a changeset (`pnpm changeset` at repo root). diff --git a/.agents/skills/testing-python-sdk/SKILL.md b/.agents/skills/testing-python-sdk/SKILL.md new file mode 100644 index 0000000000..8918e984fa --- /dev/null +++ b/.agents/skills/testing-python-sdk/SKILL.md @@ -0,0 +1,44 @@ +--- +name: testing-python-sdk +description: "Run and write tests for the E2B Python SDK (packages/python-sdk). Use when changing python-sdk code (sync or async), running pytest suites, or mirroring JS SDK changes in Python." +--- + +# Testing the Python SDK + +All commands run in `packages/python-sdk`. Use uv for everything (`uv run ...`); never pip. + +## Layout + +- `tests/sync/` and `tests/async/` — integration tests against real sandboxes (need `E2B_API_KEY`). Sync and async variants must stay equivalent. +- `tests/test_*.py` (top level) — offline unit tests (transports, codecs, config parsing, etc.). +- `tests/conftest.py` — fixtures that create/clean up sandboxes; `pytest.ini` sets `asyncio_mode=auto`, a 30s per-test timeout, and `pythonpath = tests` for shared helpers like `envd_frame_server`. + +## Running + +```bash +# full suite, 4 workers (needs E2B_API_KEY in env) +pnpm run test # == uv run pytest -n 4 --verbose -x + +# single file / test — preferred while iterating +uv run pytest tests/test_paginator.py -v +uv run pytest tests/sync/sandbox_sync/test_create.py -v -k "metadata" + +# offline-only quick check (skip integration dirs) +uv run pytest tests --ignore=tests/sync --ignore=tests/async -q +``` + +The `skip_debug` marker skips a test when `E2B_DEBUG` is set (local envd). + +## Writing tests + +- Every behavior change must land in **both** sync and async implementations with matching tests in `tests/sync/` and `tests/async/` (and mirror the JS SDK — see `testing-js-sdk`). +- Async tests need no decorator (`asyncio_mode=auto`). +- Reuse conftest fixtures rather than creating sandboxes by hand. + +## Before committing + +```bash +pnpm run lint && pnpm run typecheck # ruff check/format + ty check +``` + +Public-surface changes need a changeset (`pnpm changeset` at repo root). diff --git a/.agents/skills/verifying-cli-visual-output/SKILL.md b/.agents/skills/verifying-cli-visual-output/SKILL.md new file mode 100644 index 0000000000..2f947fbd45 --- /dev/null +++ b/.agents/skills/verifying-cli-visual-output/SKILL.md @@ -0,0 +1,51 @@ +--- +name: verifying-cli-visual-output +description: "Verify the visual/terminal output of the E2B CLI: tables, colors, alignment, spinners, TTY vs piped behavior. Use when changing anything the CLI prints or when reviewing output formatting." +--- + +# Verifying CLI Visual Output + +The CLI's output style follows kubectl: borderless tables, uppercase headers, left-aligned columns with 3-space padding, no trailing whitespace (`packages/cli/src/utils/table.ts` → `renderTable`). Colors/bold come from `chalk` (`src/utils/format.ts`). + +## 1. Unit-test the rendering (preferred) + +Capture `console.log` lines and assert exact strings — see `tests/utils/table.test.ts`: + +```ts +const lines: string[] = [] +vi.spyOn(console, 'log').mockImplementation((l: string) => lines.push(l)) +renderTable(rows, columns) +expect(lines).toEqual(['SANDBOX ID NAME', 'sbx-1 alpha']) +``` + +Keep row-building logic in exported pure functions (e.g. `buildTableRows` in `src/commands/sandbox/list.ts`) so formatting is testable without the API. + +## 2. Eyeball the real output + +```bash +cd packages/cli && pnpm build +node dist/index.js sandbox list +``` + +Checks to make by eye: +- Headers uppercase; columns aligned even with wide cells (widths use `wcswidth`, so CJK/emoji count as 2 cells). +- No borders, no trailing whitespace (`node dist/index.js sandbox list | cat -A` — no spaces before `$`). +- Long values (metadata JSON) don't break alignment of preceding columns. +- Empty result sets print a sensible message, not a lone header or a crash. + +## 3. TTY vs piped behavior + +chalk auto-strips colors when stdout is not a TTY, so piped output must stay clean and parseable: + +```bash +node dist/index.js sandbox list | head # no ANSI escape codes expected +node dist/index.js sandbox list | grep -c $'\e' # should be 0 +FORCE_COLOR=1 node dist/index.js sandbox list # force colors while piping, to inspect them +script -qec "node dist/index.js sandbox list" /dev/null # run under a real PTY +``` + +Interactive commands (spinners, prompts via `inquirer`) need a PTY — use the `script` trick above or a tty-enabled shell; never leave them attached in CI-style runs (prefer `--detach` variants). + +## 4. Screenshot for PRs + +For user-facing output changes, run the command in a real terminal, take a screenshot, and embed it in the PR description — reviewers care about how it looks, not just the strings. From c62f5fe72281fbb9534d57550a787194a7d69f3b Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Fri, 21 Aug 2026 18:59:31 +0000 Subject: [PATCH 2/8] Fold in tips from prior testing-cli-tables skill Co-Authored-By: mish@e2b.dev --- .agents/skills/testing-cli/SKILL.md | 13 ++++++++++++- .agents/skills/verifying-cli-visual-output/SKILL.md | 10 +++++++++- 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/.agents/skills/testing-cli/SKILL.md b/.agents/skills/testing-cli/SKILL.md index 5905b75e8e..1ee4da9c58 100644 --- a/.agents/skills/testing-cli/SKILL.md +++ b/.agents/skills/testing-cli/SKILL.md @@ -10,7 +10,9 @@ All commands run in `packages/cli`. Use Node 24 (`nvm use 24`) and pnpm. ## Build and run locally ```bash +source ~/.nvm/nvm.sh && nvm use 24 pnpm build # tsc typecheck + tsdown bundle -> dist/index.js +# or from the repo root: pnpm --filter @e2b/cli build node dist/index.js --help node dist/index.js sandbox list ``` @@ -20,11 +22,17 @@ Auth: the CLI reads `E2B_API_KEY` from the environment first, then `~/.e2b/confi Useful non-interactive patterns: ```bash -node dist/index.js sandbox create base --detach # returns sandbox ID, no attached terminal +node dist/index.js sandbox create base -d # detach: prints sandbox ID, no attached terminal node dist/index.js sandbox exec -- bash -lc 'pwd' # `--` stops CLI flag parsing node dist/index.js sandbox kill ``` +The team account usually has other live sandboxes/snapshots, so a truly empty list state may be unreachable; simulate it with a non-matching filter: `node dist/index.js sandbox list --metadata nomatch=zzz` → "No sandboxes found". + +There is no CLI flag for custom sandbox metadata — create such sandboxes via the JS SDK instead (build it with `pnpm --filter e2b build`, then `require('/packages/js-sdk')` from a small node script). + +Validate JSON output mode: `node dist/index.js --format json | node -e 'JSON.parse(require("fs").readFileSync(0,"utf8"))'`. + ## Automated tests ```bash @@ -45,3 +53,6 @@ pnpm run lint && pnpm run typecheck ``` Public-surface changes need a changeset (`pnpm changeset` at repo root). + +## Devin Secrets Needed +- `E2B_API_KEY` diff --git a/.agents/skills/verifying-cli-visual-output/SKILL.md b/.agents/skills/verifying-cli-visual-output/SKILL.md index 2f947fbd45..576391cfcb 100644 --- a/.agents/skills/verifying-cli-visual-output/SKILL.md +++ b/.agents/skills/verifying-cli-visual-output/SKILL.md @@ -20,6 +20,8 @@ expect(lines).toEqual(['SANDBOX ID NAME', 'sbx-1 alpha']) Keep row-building logic in exported pure functions (e.g. `buildTableRows` in `src/commands/sandbox/list.ts`) so formatting is testable without the API. +To exercise the renderer directly (e.g. wide chars in a padded middle column), run a small script importing `packages/cli/src/utils/table.ts` with `npx tsx`. + ## 2. Eyeball the real output ```bash @@ -31,7 +33,8 @@ Checks to make by eye: - Headers uppercase; columns aligned even with wide cells (widths use `wcswidth`, so CJK/emoji count as 2 cells). - No borders, no trailing whitespace (`node dist/index.js sandbox list | cat -A` — no spaces before `$`). - Long values (metadata JSON) don't break alignment of preceding columns. -- Empty result sets print a sensible message, not a lone header or a crash. +- Empty result sets print a sensible message, not a lone header or a crash. The team account usually has live sandboxes, so simulate the empty state with a non-matching filter: `node dist/index.js sandbox list --metadata nomatch=zzz` → "No sandboxes found". +- `template list` and `snapshot list` outputs are long; pipe through `head` when capturing. ## 3. TTY vs piped behavior @@ -49,3 +52,8 @@ Interactive commands (spinners, prompts via `inquirer`) need a PTY — use the ` ## 4. Screenshot for PRs For user-facing output changes, run the command in a real terminal, take a screenshot, and embed it in the PR description — reviewers care about how it looks, not just the strings. + +GUI tips: in a 1024px-wide terminal, shrink the font (Konsole: Ctrl+minus x2) so wide tables don't wrap. Typing CJK/emoji via computer-use keyboard drops characters — write test scripts to a file with a file tool and run them from the terminal instead. + +## Devin Secrets Needed +- `E2B_API_KEY` From 9fdbb67f9e5be144c2e58670093be356c65da0e2 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Fri, 21 Aug 2026 19:02:26 +0000 Subject: [PATCH 3/8] Add CI flakiness and mock-suite insights from prior sessions Co-Authored-By: mish@e2b.dev --- .agents/skills/testing-js-sdk/SKILL.md | 7 ++++++- .agents/skills/testing-python-sdk/SKILL.md | 6 ++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/.agents/skills/testing-js-sdk/SKILL.md b/.agents/skills/testing-js-sdk/SKILL.md index 1dbaafd60e..9145a114c9 100644 --- a/.agents/skills/testing-js-sdk/SKILL.md +++ b/.agents/skills/testing-js-sdk/SKILL.md @@ -38,9 +38,14 @@ pnpm run test:deno # deno: same projects ## Writing tests - Use the helpers in `tests/setup.ts`: `sandboxTest` / `templateTest` fixtures create and clean up sandboxes; `isDebug` gates behavior when `E2B_DEBUG` is set (local envd at debug port). -- Offline tests that need HTTP mock the API with `msw`; see `tests/api/` for patterns. Test isolation is required (`isolate: true`) because suites patch global fetch. +- Offline tests that need HTTP mock the API with `msw`; see `tests/api/` for patterns, and `tests/volume/mockVolumeContent.ts` for stateful in-process mocks of a whole API surface. Test isolation is required (`isolate: true`) because suites patch global fetch. - SDK changes must be mirrored in the Python SDK (sync + async) with equivalent tests — see `testing-python-sdk`. +## CI notes + +- SDK integration jobs run against both **production and staging**; staging jobs are known to flake (template-build 500s, network-egress curl errors). Before assuming your change broke CI, check whether the same job fails on the base branch or rerun the job. +- New API parameters often work on staging before production — a production-only failure of a new-feature test usually means the API isn't deployed there yet, not a code bug. + ## Before committing ```bash diff --git a/.agents/skills/testing-python-sdk/SKILL.md b/.agents/skills/testing-python-sdk/SKILL.md index 8918e984fa..55ddd5aa2d 100644 --- a/.agents/skills/testing-python-sdk/SKILL.md +++ b/.agents/skills/testing-python-sdk/SKILL.md @@ -35,6 +35,12 @@ The `skip_debug` marker skips a test when `E2B_DEBUG` is set (local envd). - Async tests need no decorator (`asyncio_mode=auto`). - Reuse conftest fixtures rather than creating sandboxes by hand. +## CI notes + +- Integration jobs run against both **production and staging**; staging jobs are known to flake (template-build 500s, transient backend errors). Check the base branch or rerun before blaming your change. +- New API parameters often reach staging before production — a production-only failure of a new-feature test usually means the API isn't deployed there yet. +- `tests/shared/` holds offline suites shared between sync and async (request shaping, encoding, proxy config); prefer extending those for behavior that doesn't need a live sandbox. + ## Before committing ```bash From b0569170c9c8d72450b947b6525912fe5f48b45a Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Fri, 21 Aug 2026 19:08:31 +0000 Subject: [PATCH 4/8] Scope skills to the testing process only Co-Authored-By: mish@e2b.dev --- .agents/skills/testing-cli/SKILL.md | 11 ++--------- .agents/skills/testing-js-sdk/SKILL.md | 15 ++------------- .agents/skills/testing-python-sdk/SKILL.md | 18 ++---------------- .../verifying-cli-visual-output/SKILL.md | 15 ++------------- 4 files changed, 8 insertions(+), 51 deletions(-) diff --git a/.agents/skills/testing-cli/SKILL.md b/.agents/skills/testing-cli/SKILL.md index 1ee4da9c58..89f59319bf 100644 --- a/.agents/skills/testing-cli/SKILL.md +++ b/.agents/skills/testing-cli/SKILL.md @@ -1,6 +1,6 @@ --- name: testing-cli -description: "Build, run, and test the E2B CLI (packages/cli) locally. Use when changing CLI commands, running its vitest suites, or executing the CLI against real sandboxes." +description: "Build, run, and test the E2B CLI (packages/cli) locally. Use when running its vitest suites or executing the CLI against real sandboxes." --- # Testing the CLI @@ -42,17 +42,10 @@ npx vitest run tests/utils/table.test.ts # single file - Tests spawn the **built** CLI (`dist/index.js`) via helpers in `tests/setup.ts` (`runCli`, `runCliWithPipedStdin`) — rebuild happens automatically through globalSetup, but if you bypass vitest, run `pnpm build` yourself after editing `src/`. - `tests/commands/**` cover command behavior; some hit the real API and need `E2B_API_KEY`. -- Unit tests import from `src/` directly (vitest aliases `e2b` to `../js-sdk/src`), so keep command logic in exported, testable functions (see `buildTableRows`/`sortSandboxes` in `src/commands/sandbox/list.ts`). +- Unit tests import from `src/` directly (vitest aliases `e2b` to `../js-sdk/src`), so they run without building the SDK. For checking rendered output (tables, colors, alignment), use the `verifying-cli-visual-output` skill. -## Before committing - -```bash -pnpm run lint && pnpm run typecheck -``` - -Public-surface changes need a changeset (`pnpm changeset` at repo root). ## Devin Secrets Needed - `E2B_API_KEY` diff --git a/.agents/skills/testing-js-sdk/SKILL.md b/.agents/skills/testing-js-sdk/SKILL.md index 9145a114c9..5ee7b642f4 100644 --- a/.agents/skills/testing-js-sdk/SKILL.md +++ b/.agents/skills/testing-js-sdk/SKILL.md @@ -1,6 +1,6 @@ --- name: testing-js-sdk -description: "Run and write tests for the E2B JavaScript SDK (packages/js-sdk). Use when changing js-sdk code, debugging its vitest suites, or verifying SDK behavior against real sandboxes." +description: "Run tests for the E2B JavaScript SDK (packages/js-sdk). Use when debugging its vitest suites or verifying SDK behavior against real sandboxes." --- # Testing the JS SDK @@ -35,21 +35,10 @@ pnpm run test:deno # deno: same projects `E2B_API_KEY` is read from the environment or from `.env` via dotenv (the repo also keeps defaults in `.env.local` at the root or `~/.e2b/config.json`). Purely offline unit tests (e.g. `tests/api/inflight.test.ts`, `tests/utils.test.ts`) run without a key. -## Writing tests - -- Use the helpers in `tests/setup.ts`: `sandboxTest` / `templateTest` fixtures create and clean up sandboxes; `isDebug` gates behavior when `E2B_DEBUG` is set (local envd at debug port). -- Offline tests that need HTTP mock the API with `msw`; see `tests/api/` for patterns, and `tests/volume/mockVolumeContent.ts` for stateful in-process mocks of a whole API surface. Test isolation is required (`isolate: true`) because suites patch global fetch. -- SDK changes must be mirrored in the Python SDK (sync + async) with equivalent tests — see `testing-python-sdk`. +Fixtures in `tests/setup.ts` (`sandboxTest`/`templateTest`) create and clean up sandboxes automatically; `E2B_DEBUG` switches suites to a local envd. Fully offline suites (msw-mocked, e.g. `tests/api/`, `tests/volume/`) are the fastest signal when no key is available. ## CI notes - SDK integration jobs run against both **production and staging**; staging jobs are known to flake (template-build 500s, network-egress curl errors). Before assuming your change broke CI, check whether the same job fails on the base branch or rerun the job. - New API parameters often work on staging before production — a production-only failure of a new-feature test usually means the API isn't deployed there yet, not a code bug. -## Before committing - -```bash -pnpm run lint && pnpm run typecheck -``` - -Public-surface changes need a changeset (`pnpm changeset` at repo root). diff --git a/.agents/skills/testing-python-sdk/SKILL.md b/.agents/skills/testing-python-sdk/SKILL.md index 55ddd5aa2d..2667685d38 100644 --- a/.agents/skills/testing-python-sdk/SKILL.md +++ b/.agents/skills/testing-python-sdk/SKILL.md @@ -1,6 +1,6 @@ --- name: testing-python-sdk -description: "Run and write tests for the E2B Python SDK (packages/python-sdk). Use when changing python-sdk code (sync or async), running pytest suites, or mirroring JS SDK changes in Python." +description: "Run tests for the E2B Python SDK (packages/python-sdk). Use when running its pytest suites (sync or async) or verifying SDK behavior against real sandboxes." --- # Testing the Python SDK @@ -29,22 +29,8 @@ uv run pytest tests --ignore=tests/sync --ignore=tests/async -q The `skip_debug` marker skips a test when `E2B_DEBUG` is set (local envd). -## Writing tests - -- Every behavior change must land in **both** sync and async implementations with matching tests in `tests/sync/` and `tests/async/` (and mirror the JS SDK — see `testing-js-sdk`). -- Async tests need no decorator (`asyncio_mode=auto`). -- Reuse conftest fixtures rather than creating sandboxes by hand. - ## CI notes - Integration jobs run against both **production and staging**; staging jobs are known to flake (template-build 500s, transient backend errors). Check the base branch or rerun before blaming your change. - New API parameters often reach staging before production — a production-only failure of a new-feature test usually means the API isn't deployed there yet. -- `tests/shared/` holds offline suites shared between sync and async (request shaping, encoding, proxy config); prefer extending those for behavior that doesn't need a live sandbox. - -## Before committing - -```bash -pnpm run lint && pnpm run typecheck # ruff check/format + ty check -``` - -Public-surface changes need a changeset (`pnpm changeset` at repo root). +- `tests/shared/` holds offline suites shared between sync and async (request shaping, encoding, proxy config) — the fastest signal when no key is available. diff --git a/.agents/skills/verifying-cli-visual-output/SKILL.md b/.agents/skills/verifying-cli-visual-output/SKILL.md index 576391cfcb..3e271599bd 100644 --- a/.agents/skills/verifying-cli-visual-output/SKILL.md +++ b/.agents/skills/verifying-cli-visual-output/SKILL.md @@ -7,20 +7,9 @@ description: "Verify the visual/terminal output of the E2B CLI: tables, colors, The CLI's output style follows kubectl: borderless tables, uppercase headers, left-aligned columns with 3-space padding, no trailing whitespace (`packages/cli/src/utils/table.ts` → `renderTable`). Colors/bold come from `chalk` (`src/utils/format.ts`). -## 1. Unit-test the rendering (preferred) +## 1. Run the rendering tests -Capture `console.log` lines and assert exact strings — see `tests/utils/table.test.ts`: - -```ts -const lines: string[] = [] -vi.spyOn(console, 'log').mockImplementation((l: string) => lines.push(l)) -renderTable(rows, columns) -expect(lines).toEqual(['SANDBOX ID NAME', 'sbx-1 alpha']) -``` - -Keep row-building logic in exported pure functions (e.g. `buildTableRows` in `src/commands/sandbox/list.ts`) so formatting is testable without the API. - -To exercise the renderer directly (e.g. wide chars in a padded middle column), run a small script importing `packages/cli/src/utils/table.ts` with `npx tsx`. +Exact-string assertions for the renderer live in `tests/utils/table.test.ts` (`npx vitest run tests/utils/table.test.ts`). To exercise the renderer directly (e.g. wide chars in a padded middle column), run a small script importing `packages/cli/src/utils/table.ts` with `npx tsx`. ## 2. Eyeball the real output From 2ca2d8757e6916ec3244ccf662120a1412ce0df5 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Fri, 21 Aug 2026 19:09:57 +0000 Subject: [PATCH 5/8] Remove DEV.md Co-Authored-By: mish@e2b.dev --- DEV.md | 3 --- 1 file changed, 3 deletions(-) delete mode 100644 DEV.md diff --git a/DEV.md b/DEV.md deleted file mode 100644 index ef83fd7b27..0000000000 --- a/DEV.md +++ /dev/null @@ -1,3 +0,0 @@ -# Releasing e2b cli - -to create a changeset run `pnpm run changeset` From 97e895785559fa893a4090929e7b2ab4343ab4b4 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Fri, 21 Aug 2026 19:11:57 +0000 Subject: [PATCH 6/8] Restore unit-test rendering section in visual-output skill Co-Authored-By: mish@e2b.dev --- .../skills/verifying-cli-visual-output/SKILL.md | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/.agents/skills/verifying-cli-visual-output/SKILL.md b/.agents/skills/verifying-cli-visual-output/SKILL.md index 3e271599bd..5bf0a301ef 100644 --- a/.agents/skills/verifying-cli-visual-output/SKILL.md +++ b/.agents/skills/verifying-cli-visual-output/SKILL.md @@ -7,9 +7,20 @@ description: "Verify the visual/terminal output of the E2B CLI: tables, colors, The CLI's output style follows kubectl: borderless tables, uppercase headers, left-aligned columns with 3-space padding, no trailing whitespace (`packages/cli/src/utils/table.ts` → `renderTable`). Colors/bold come from `chalk` (`src/utils/format.ts`). -## 1. Run the rendering tests +## 1. Unit-test the rendering (preferred) -Exact-string assertions for the renderer live in `tests/utils/table.test.ts` (`npx vitest run tests/utils/table.test.ts`). To exercise the renderer directly (e.g. wide chars in a padded middle column), run a small script importing `packages/cli/src/utils/table.ts` with `npx tsx`. +Capture `console.log` lines and assert exact strings — see `tests/utils/table.test.ts` (`npx vitest run tests/utils/table.test.ts`): + +```ts +const lines: string[] = [] +vi.spyOn(console, 'log').mockImplementation((l: string) => lines.push(l)) +renderTable(rows, columns) +expect(lines).toEqual(['SANDBOX ID NAME', 'sbx-1 alpha']) +``` + +Keep row-building logic in exported pure functions (e.g. `buildTableRows` in `src/commands/sandbox/list.ts`) so formatting is testable without the API. + +To exercise the renderer directly (e.g. wide chars in a padded middle column), run a small script importing `packages/cli/src/utils/table.ts` with `npx tsx`. ## 2. Eyeball the real output From c7d74289ab4d74b2ffba77d0624c356ec21ce037 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Fri, 21 Aug 2026 19:17:01 +0000 Subject: [PATCH 7/8] Add skills for env setup, codegen, CI debugging, releasing, templates, and issue triage Co-Authored-By: mish@e2b.dev --- .agents/skills/building-templates/SKILL.md | 36 +++++++++++++++ .agents/skills/codegen-and-specs/SKILL.md | 39 ++++++++++++++++ .agents/skills/debugging-ci/SKILL.md | 29 ++++++++++++ .agents/skills/releasing/SKILL.md | 30 +++++++++++++ .../setting-up-testing-environment/SKILL.md | 41 +++++++++++++++++ .agents/skills/triaging-issues/SKILL.md | 45 +++++++++++++++++++ 6 files changed, 220 insertions(+) create mode 100644 .agents/skills/building-templates/SKILL.md create mode 100644 .agents/skills/codegen-and-specs/SKILL.md create mode 100644 .agents/skills/debugging-ci/SKILL.md create mode 100644 .agents/skills/releasing/SKILL.md create mode 100644 .agents/skills/setting-up-testing-environment/SKILL.md create mode 100644 .agents/skills/triaging-issues/SKILL.md diff --git a/.agents/skills/building-templates/SKILL.md b/.agents/skills/building-templates/SKILL.md new file mode 100644 index 0000000000..19190c5e37 --- /dev/null +++ b/.agents/skills/building-templates/SKILL.md @@ -0,0 +1,36 @@ +--- +name: building-templates +description: "Build E2B sandbox templates: the CLI Dockerfile flow, the Template SDK builder, and the repo's own prepared templates. Use when creating/rebuilding templates or debugging template builds." +--- + +# Building Templates + +Two ways to define a template: + +## 1. CLI Dockerfile flow + +`e2b template create ` reads `e2b.Dockerfile` (or `Dockerfile`) in the working directory (or `-p `) and builds it into a sandbox template. Useful flags: `-c/--cmd` (start command), `--ready-cmd` (must exit 0 before the template is considered ready — doubles as a build-time smoke test), `--cpu-count`, `--memory-mb` (even number, default 1024), `--no-cache`. Needs `E2B_API_KEY`. Related: `template list`, `template delete`, `template publish/unpublish`, and `template migrate` (converts a legacy `e2b.Dockerfile` + `e2b.toml` pair to the Template SDK format). + +## 2. Template SDK (js-sdk) + +Programmatic builder: `Template().fromPythonImage('3')...` then `Template.build(template, 'name:tag', {...})` (see `packages/js-sdk/src/template/`, examples in the `Template` class jsdoc). Live examples of usage — including file-context handling and `defaultBuildLogger` — are in `packages/js-sdk/tests/template/` (run via the `template` vitest project, 180s timeout, needs `E2B_API_KEY`). + +## Repo's prepared templates (`templates/`) + +- `templates/base` — the default public template; also pushed to DockerHub. +- `templates/httpbin` — private echo-server sidecar used by the JS network-transform tests; built with `--cmd 'go-httpbin ...'` and a curl `--ready-cmd`. + +They are rebuilt via the manual `Build and push prepared templates` workflow (`.github/workflows/templates.yml`, `workflow_dispatch` with a template picker). Locally the same commands work, e.g.: + +```bash +cd templates/base && e2b template create base --memory-mb 512 +``` + +## Debugging build failures + +- Build logs stream from the API; HTTP 500s during template builds are a known transient on staging — retry before digging. +- If the build hangs at "ready", the `--ready-cmd` never exited 0 — run it inside a sandbox of the base image to check. +- The build runs remotely; local Docker is not used for `e2b template create`. + +## Devin Secrets Needed +- `E2B_API_KEY` diff --git a/.agents/skills/codegen-and-specs/SKILL.md b/.agents/skills/codegen-and-specs/SKILL.md new file mode 100644 index 0000000000..ff4cd1508c --- /dev/null +++ b/.agents/skills/codegen-and-specs/SKILL.md @@ -0,0 +1,39 @@ +--- +name: codegen-and-specs +description: "Update API specs and regenerate SDK/CLI client code in the E2B repo. Use when bumping spec refs, adding new API surface, or when the generated-files CI check fails." +--- + +# Codegen and Specs + +## Ownership: never hand-edit `spec/` + +Files under `spec/` are synced from upstream repos and any manual edit will be overwritten: + +- `spec/openapi.yml`, envd specs → `e2b-dev/infra` +- `spec/volume-api.yml` → `e2b-dev/belt` + +Copybara (`copy.bara.sky`) mirrors them in; `spec/infra-ref` and `spec/belt-ref` pin the upstream commits used by codegen. `spec/README.md` is the authoritative doc. + +## Updating generated code + +```bash +# 1. bump the pin (or let copybara have updated the spec already) +echo > spec/infra-ref # or spec/belt-ref + +# 2. refetch pinned specs + regenerate everything in Docker +make codegen +``` + +`make codegen` builds `codegen.Dockerfile` and runs `make generate` inside it, so results don't depend on local tool versions. Fetch failures only warn and fall back to the tracked copies — check the output if you expected a spec change. Belt spec fetches need a token; public infra specs fetch anonymously. + +Fetch-only helpers: `pnpm fetch:api-spec`, `pnpm fetch:envd-spec`, `pnpm fetch:volume-spec` (override the pin with `E2B_INFRA_REF=main` / `E2B_BELT_REF=main`). + +## What gets generated + +Redocly first filters the OpenAPI specs by SDK tags (internal/`x-internal` ops are stripped), then clients are generated into `packages/js-sdk`, `packages/python-sdk`, and the CLI. Commit the generated diffs together with the spec/ref change. + +## CI + +`generated_files.yml` re-runs generation on PRs touching `spec/**`, codegen Dockerfiles, `copy.bara.sky`, redocly config, fetch scripts, packages, or lockfiles, and fails if the committed generated files are stale. If it fails, run `make codegen` locally and commit the diff — don't patch generated files by hand. + +Gotcha from past sessions: a new API parameter appearing in the staging spec may not be deployed to production yet — generated code can be ahead of the production API (see `debugging-ci`). diff --git a/.agents/skills/debugging-ci/SKILL.md b/.agents/skills/debugging-ci/SKILL.md new file mode 100644 index 0000000000..3fbac8c9b6 --- /dev/null +++ b/.agents/skills/debugging-ci/SKILL.md @@ -0,0 +1,29 @@ +--- +name: debugging-ci +description: "Understand and debug GitHub Actions failures on E2B pull requests: the workflow matrix, which checks are required, and known flaky jobs. Use whenever a PR check fails." +--- + +# Debugging CI + +## The matrix (`.github/workflows/sdk_tests.yml`) + +A paths filter decides which suites run (docs-only `.md` changes trigger nothing; `spec/**`, lockfiles, `.tool-versions`, or the workflow files trigger everything): + +- **Production** JS / Python / CLI test suites — these feed the required aggregate check **`SDK Tests Status`**, which also fails if change-detection itself fails or jobs are cancelled. +- **Staging** JS / Python / CLI suites — same reusable workflows against `E2B_DOMAIN_STAGING`; staging JS runs `node-only` (Bun/Deno/Cloudflare coverage only runs on production). +- `generated_files.yml` — fails when committed generated code is stale; fix with `make codegen` (see `codegen-and-specs`). + +The JS workflow's full production matrix covers Node (Ubuntu + Windows), Bun, Deno, Cloudflare, and a Cloudflare deploy leg. + +## Known flakes and skew (from past sessions) + +- **Staging jobs flake**: template-build HTTP 500s, network-egress curl failures, other transient backend errors. Before blaming your change, check whether the same job fails on the base branch, or rerun the job. +- **Staging-before-production skew**: new API features usually reach staging first. A *production-only* failure of a test exercising a brand-new API parameter typically means the API isn't deployed to production yet, not an SDK bug. +- **Cloudflare deploy** can hit propagation / read-after-write races on freshly created preview workers — treat an isolated intermittent failure there as advisory and rerun. + +## Debug loop + +1. Get the failing job and its logs (Devin: `git_pr_checks` → `git_ci_job_logs` with the job id). +2. Reproduce locally with the same command the job runs (the reusable workflows run `pnpm run test` / `uv run pytest` in the package dir; export `E2B_DOMAIN` to mimic staging). +3. If it's a live-sandbox test, remember it needs `E2B_API_KEY`; offline mock suites are the fastest local signal. +4. Never conclude a failure is preexisting/flaky without evidence — verify against the base branch or a rerun. diff --git a/.agents/skills/releasing/SKILL.md b/.agents/skills/releasing/SKILL.md new file mode 100644 index 0000000000..d7dfd0108c --- /dev/null +++ b/.agents/skills/releasing/SKILL.md @@ -0,0 +1,30 @@ +--- +name: releasing +description: "How versioning and releases work in the E2B repo: changesets, the manual release workflow, and release candidates. Use when preparing a release, deciding whether a PR needs a changeset, or debugging the publish pipeline." +--- + +# Releasing + +## Changesets + +Versions are driven by changesets in `.changeset/`. Generate one with `pnpm changeset` at the repo root **when a PR changes the public surface** of `packages/cli` (`@e2b/cli`), `packages/js-sdk` (`e2b`), or `packages/python-sdk` (`@e2b/python-sdk`). Internal scripts, devtools, tests, docs, and skills don't need one. The python package participates in changesets via its npm-shim name `@e2b/python-sdk`. + +## Production release (`.github/workflows/release.yml`) + +Manual `workflow_dispatch`, and it hard-fails on any ref other than `main` (a feature branch carrying changesets would otherwise publish real packages). Flow: + +1. **Preflight** — parses `.tool-versions` for pinned tooling, runs `is_release.sh` / `is_release_for_package.sh` to decide which of js-sdk / python-sdk / cli have pending changesets, and builds a Slack "itinerary" (advisory only — `continue-on-error`). +2. **Tests** — the full JS / Python / CLI suites run for each package being released. +3. **Publish** (`publish_packages.yml`) — versions via changesets, publishes to npm and PyPI (needs `E2B_API_KEY`, `PYPI_TOKEN`), and pushes the version-bump commit. + +Production and candidate runs on the same ref share a concurrency group, so they serialize. + +## Release candidates (`release-candidate.yml`) + +Manual dispatch from any branch with per-package booleans, a dist-tag (`rc`/`beta`/`snapshot`), an optional preid (defaults to the branch name), and an optional skip-tests flag. Publishes prerelease versions via `publish_candidates.yml` without touching main. Use this to let users try an unmerged fix. + +## Debugging a failed release + +- Failure Slack notifications fire from the workflow; check which stage failed (preflight vs tests vs publish). +- Test-stage failures are the normal SDK suites — see `debugging-ci`. +- "Nothing to release" means no pending changesets — check `pnpm changeset status`. diff --git a/.agents/skills/setting-up-testing-environment/SKILL.md b/.agents/skills/setting-up-testing-environment/SKILL.md new file mode 100644 index 0000000000..ce00f7f573 --- /dev/null +++ b/.agents/skills/setting-up-testing-environment/SKILL.md @@ -0,0 +1,41 @@ +--- +name: setting-up-testing-environment +description: "Initialize the E2B repo for local testing: toolchain versions, dependency install, SDK/CLI builds, and credentials. Use at the start of any session that will run or test code in this repo." +--- + +# Setting Up the Testing Environment + +## Toolchain + +`.tool-versions` at the repo root is the source of truth (CI parses it): Node 22.x, pnpm, Python 3.10, uv, deno. Locally, `source ~/.nvm/nvm.sh && nvm use 24` also works for all packages (the CLI needs Node ≥ 20). + +Use pnpm for Node and uv for Python — never npm/yarn/pip. + +## Install and build + +```bash +pnpm install # repo root; installs all workspaces +pnpm --filter e2b build # js-sdk -> packages/js-sdk/dist +pnpm --filter @e2b/cli build # cli -> packages/cli/dist/index.js +cd packages/python-sdk && uv sync # python env (uv run ... after this) +``` + +## Credentials + +- `E2B_API_KEY` is required for anything that touches real sandboxes (most js-sdk tests, python `tests/sync|async`, CLI integration tests, live CLI commands). It's read from the environment; defaults may also live in `.env.local` at the repo root or `~/.e2b/config.json`. +- `E2B_DOMAIN` switches the target environment (unset = production). +- `E2B_DEBUG` switches suites to a local envd; only set it when you're running envd locally. +- Never run `e2b auth login` in headless environments — export the key instead. + +## Smoke check + +```bash +cd packages/cli && node dist/index.js sandbox list # verifies key + build +cd packages/js-sdk && npx vitest run --project connectionConfig # offline, no key +cd packages/python-sdk && uv run pytest tests/test_connection_config.py -q +``` + +Then use the package-specific skills: `testing-js-sdk`, `testing-python-sdk`, `testing-cli`, `verifying-cli-visual-output`. + +## Devin Secrets Needed +- `E2B_API_KEY` diff --git a/.agents/skills/triaging-issues/SKILL.md b/.agents/skills/triaging-issues/SKILL.md new file mode 100644 index 0000000000..0b48314a15 --- /dev/null +++ b/.agents/skills/triaging-issues/SKILL.md @@ -0,0 +1,45 @@ +--- +name: triaging-issues +description: "Triage conventions for GitHub issues in e2b-dev/E2B: classification dimensions, label usage, spam handling, and the triage comment format. Use when classifying or labeling an issue." +--- + +# Triaging Issues + +New issues are normally auto-triaged by an automation; follow the same conventions when triaging manually so results stay consistent. + +## Classification dimensions + +- `type`: bug | feature | refactor | question | chore | security +- `priority`: critical | high | medium | low +- `complexity`: trivial | small | medium | large +- `affected_area`: best-guess component (e.g. "js-sdk/filesystem", "python-sdk/commands", "cli/templates", "CI pipeline", "unknown") +- `actionable`: whether there's enough information to act +- `needs_clarification`: only true when proceeding would risk building the wrong thing entirely; list the specific questions +- `summary`: one-sentence restatement (note assumptions here instead of blocking) + +Be decisive: classify with best judgment when intent is clear, even if details are thin. + +## Labels + +Apply matching **existing** repository labels (`gh label list` to see them — e.g. bug, feature, Improvement, sdk, cli, envd, Infrastructure, Build System, Code Interpreter) with `gh issue edit --add-label ...`. Never create new labels. + +## Spam + +Check the issue and each comment for spam/promotional content (off-topic self-promotion, link-farming, credit-solicitation, AI filler). Hide spam comments via the GraphQL `minimizeComment` mutation with `classifier: SPAM` (`gh api graphql`). If the issue itself is spam: `type: chore`, `priority: low`, note "spam" in the summary and that it should be closed. + +## Comment format + +Post the classification as an issue comment: + +```md +**Triage** + +- **type**: bug +- **priority**: high +- **complexity**: small +- **affected_area**: js-sdk/network transform +- **actionable**: yes +- **needs_clarification**: false +- **questions**: (only when needs_clarification is true, as a nested list) +- **summary**: One-sentence restatement of the work item. +``` From 3daee27a03dbd2d983c91bfa2b50b2145fa8b078 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Fri, 21 Aug 2026 19:17:34 +0000 Subject: [PATCH 8/8] Note streaming exception to sync/async test parity Co-Authored-By: mish@e2b.dev --- .agents/skills/testing-python-sdk/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.agents/skills/testing-python-sdk/SKILL.md b/.agents/skills/testing-python-sdk/SKILL.md index 2667685d38..e748e594c8 100644 --- a/.agents/skills/testing-python-sdk/SKILL.md +++ b/.agents/skills/testing-python-sdk/SKILL.md @@ -9,7 +9,7 @@ All commands run in `packages/python-sdk`. Use uv for everything (`uv run ...`); ## Layout -- `tests/sync/` and `tests/async/` — integration tests against real sandboxes (need `E2B_API_KEY`). Sync and async variants must stay equivalent. +- `tests/sync/` and `tests/async/` — integration tests against real sandboxes (need `E2B_API_KEY`). Sync and async variants must stay equivalent, except for streaming: sync has no background pump, so callbacks are passed to `wait()` and `watch_dir` is polled via `get_new_events()`. - `tests/test_*.py` (top level) — offline unit tests (transports, codecs, config parsing, etc.). - `tests/conftest.py` — fixtures that create/clean up sandboxes; `pytest.ini` sets `asyncio_mode=auto`, a 30s per-test timeout, and `pythonpath = tests` for shared helpers like `envd_frame_server`.