-
Notifications
You must be signed in to change notification settings - Fork 3
Add Sandbox API support #67
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mmilutinovic371
wants to merge
4
commits into
main
Choose a base branch
from
add-sandboxes-support
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
da29fb4
feat(sandboxes): add Sandbox API support
mmilutinovic371 c440f0b
fix(sandboxes): use /workspace instead of /work in remaining paths
mmilutinovic371 000a14f
fix(sandboxes): address review feedback (security, correctness, deps)
mmilutinovic371 058dfd1
fix(sandboxes): export SandboxState from the sandbox barrel
mmilutinovic371 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -14,6 +14,84 @@ Check out our docs [here.](https://deepinfra.github.io/deepinfra-node/) | |
| npm install deepinfra | ||
| ``` | ||
|
|
||
| ## Sandboxes | ||
|
|
||
| Create an isolated Linux microVM, run bash/python inside it, move files in and | ||
| out, and tear it down — in a few lines: | ||
|
|
||
| ```typescript | ||
| import { Sandbox } from "deepinfra"; | ||
|
|
||
| const sb = await Sandbox.create({ plan: "medium", timeout: "10m" }); // resolves once running | ||
|
|
||
| const r = await sb.exec( | ||
| "bash", | ||
| "-c", | ||
| "pip install --break-system-packages pandas && python3 -c 'import pandas; print(pandas.__version__)'", | ||
| ); | ||
| console.log(r.stdout, r.stderr, r.returncode); | ||
|
|
||
| const out = await sb.runPython("print(21 * 2)"); | ||
| out.check(); // throws CommandFailedError on a non-zero exit code | ||
|
|
||
| await sb.fs.write("/workspace/in.csv", "a,b\n1,2\n"); | ||
| const data = await sb.fs.read("/workspace/in.csv"); // Buffer | ||
|
|
||
| await sb.stop(); // frees compute, keeps disk; resolves once stopped | ||
| await sb.start(); // resumes on the same disk; resolves once running | ||
| await sb.terminate(); // deletes the sandbox (stays fetchable by id as "deleted" briefly) | ||
| ``` | ||
|
|
||
| `/workspace` is the only path `fs.write`/`fs.read` accept, and it's the only | ||
| part of the filesystem that survives `stop()`/`start()` — everything else | ||
| (e.g. packages installed with `pip`/`apt` outside it) resets to the base | ||
| image on restart, and the sandbox's idle timeout has the same effect as an | ||
| explicit `stop()`. | ||
|
|
||
| Auto-terminate with `try`/`finally`: | ||
|
|
||
| ```typescript | ||
| const sb = await Sandbox.create({ plan: "small" }); | ||
| try { | ||
| await sb.runPython("open('/workspace/out.txt', 'w').write('hi')"); | ||
| console.log((await sb.fs.read("/workspace/out.txt")).toString()); | ||
| } finally { | ||
| await sb.terminate(); | ||
| } | ||
| ``` | ||
|
|
||
| Other lookups: | ||
|
|
||
| ```typescript | ||
| import fs from "node:fs"; | ||
|
|
||
| // Find existing sandboxes | ||
| const sb = await Sandbox.fromId("sb_..."); | ||
| const etlBoxes = await Sandbox.list({ tags: { job: "etl-42" } }); | ||
|
|
||
| // List available plans (id, vcpu, ram_gb, disk_gb, price_per_hour) | ||
| for (const plan of await Sandbox.catalog()) { | ||
| console.log(plan.id, plan.vcpu, plan.ram_gb, plan.price_per_hour); | ||
| } | ||
|
|
||
| // Large scripts: upload, then run | ||
| await sb.fs.write( | ||
| "/workspace/script.py", | ||
| await fs.promises.readFile("script.py", "utf8"), | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fs is not imported in this snippet. please copy and paste every readme snippet into a file and run it, same as we did for the docs. |
||
| ); | ||
| await sb.exec("python3", "/workspace/script.py", { timeout: "30m" }); | ||
| ``` | ||
|
|
||
| Errors are typed: `AuthenticationError` (401), `NotFoundError` (404), | ||
| `ConflictError` (409, e.g. exec on a stopped sandbox), `RateLimitError` | ||
| (429; for sandboxes that's the per-account cap — `TooManySandboxesError` is | ||
| an alias), `CapacityError` (503), plus SDK-side `SandboxTimeoutError` / | ||
| `SandboxFailedError` / `CommandFailedError`. If `Sandbox.create()` fails | ||
| while waiting for it to come up, the raised error carries `.sandboxId` so you | ||
| can inspect or terminate the sandbox it created. | ||
|
|
||
| See [`examples/sandbox-quickstart.ts`](examples/sandbox-quickstart.ts) for a runnable end-to-end example. | ||
|
|
||
| ## Usage | ||
|
|
||
| ### Use [text generation models](https://deepinfra.com/models/text-generation) | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| /** | ||
| * Sandbox quickstart: create, exec, file round-trip, terminate. | ||
| * | ||
| * Needs DEEPINFRA_API_KEY in the environment. Run with: | ||
| * npx ts-node -r tsconfig-paths/register examples/sandbox-quickstart.ts | ||
| */ | ||
| import { Sandbox } from "../src"; | ||
|
|
||
| async function main(): Promise<void> { | ||
| const sb = await Sandbox.create({ | ||
| plan: "small", | ||
| timeout: "10m", | ||
| tags: { demo: "quickstart" }, | ||
| }); | ||
| try { | ||
| console.log("sandbox:", sb.id, sb.state); | ||
|
|
||
| const uname = await sb.exec("uname", "-a"); | ||
| console.log("kernel:", uname.check().stdout.trim()); | ||
|
|
||
| const sum = await sb.runPython("print(sum(range(101)))"); | ||
| console.log("sum 0..100 =", sum.check().stdout.trim()); | ||
|
|
||
| await sb.fs.write("/workspace/hello.txt", "hello from the host\n"); | ||
| const readBack = await sb.fs.read("/workspace/hello.txt"); | ||
| console.log("read back:", readBack.toString("utf8").trim()); | ||
| } finally { | ||
| await sb.terminate(); | ||
| } | ||
| } | ||
|
|
||
| main(); |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
port the paragraph from the python readme here: /workspace is the only path fs accepts and the only one that survives stop/start, everything else comes back from the base image so runtime pip installs are gone after a restart, and the idle timeout stops the sandbox with the same effect. that is exactly the /work bug you hit, worth stating.