From 5f2f22f6d0090035412f48a07290060d981e4e4f Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 14:40:52 +0800 Subject: [PATCH 01/17] Add standalone Parsar Core landing page --- site/favicon.svg | 1 + site/index.html | 90 +++++++++++++++++++++++++++ site/styles.css | 155 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 246 insertions(+) create mode 100644 site/favicon.svg create mode 100644 site/index.html create mode 100644 site/styles.css diff --git a/site/favicon.svg b/site/favicon.svg new file mode 100644 index 000000000..94551801b --- /dev/null +++ b/site/favicon.svg @@ -0,0 +1 @@ + diff --git a/site/index.html b/site/index.html new file mode 100644 index 000000000..5986693aa --- /dev/null +++ b/site/index.html @@ -0,0 +1,90 @@ + + + + + + + + Parsar Core · Open-source Agents API infrastructure + + + + + +
+ Parsar Core + +
+ +
+
+
+

Open source. Self-deployed.

+

Agents API
infrastructure.
On your terms.

+

Run Codex, Claude Code and MiniMax Code through one Core API and a shared Runtime. Deploy it on your infrastructure, with a Web console included.

+
+ Install Core + Read the docs +
+

Your service. Your Runtime. Your model provider.

+
+
+
connect.pyREAD ONLY
+
Point an OpenAI client at your Core.
+ 
import os
+from openai import OpenAI
+
+client = OpenAI(
+    base_url="http://localhost:8091/v1",
+    api_key=os.environ["PARSAR_CORE_API_KEY"],
+)
+
+for agent in client.beta.agents.list():
+    print(agent.id)
+
Protocol baselineopenai-python 3.13.0 · agents=v1
+

Use your Core API key and endpoint. This reads saved Agents; it does not call a model. Check coverage →

+
+
+ +
+
+

01 / Architecture

One execution path.
Multiple native harnesses.

Core owns the API and execution state. The shared Runtime connects that contract to each native harness.

+
+
PUBLIC API

Parsar Core

Agents, Sessions & execution state

+ +
SHARED CONTRACT

Runtime

Daemon, tools & workspace

+ +
CodexClaude CodeMiniMax Code
+
+

Web console included. Connect to your Core through the same public API.

Native differences stay explicit. Supported capabilities depend on the harness and deployment profile.

+
+
+ +
+

02 / Installation

Start with a
local bundle.

Download or build a bundle for your host, extract it, then run the installer from that directory.

The default installs Core and the Web console, and prepares microsandbox and Runtime images. Core provisions the Runtime when you create an execution Session.

Installation guide
+
+

Core + Web

DEFAULT
./install.sh

Uses microsandbox. No model provider key is needed at install time.

+

Use Docker

Choose the Docker sandbox provider.

./install.sh --provider docker
+

Core only

Deploy the API without the Web console.

./install.sh --core-only
+

Web only

Connect the console to an existing Core.

./install.sh --web-only
+

Bundle availability and supported hosts are documented in the installation guide. Browse releases ↗

+
+
+ +
+

03 / Before you deploy

Know what you are running.

The pinned Agents API is the compatibility target. Coverage is tracked by operation and workflow; full compatibility is not claimed.

+
+ 01

Connect your first client

Configure your Core endpoint and authentication, then make your first API request.

Quickstart + 02

Check protocol coverage

Review verified workflows, harness differences and known gaps against the pinned baseline.

Coverage & limitations + 03

Operate your deployment

Self-deployment means owning service and container health, Runtime availability and recovery.

Operations guide +
+
+
+ + + + diff --git a/site/styles.css b/site/styles.css new file mode 100644 index 000000000..06659c3a7 --- /dev/null +++ b/site/styles.css @@ -0,0 +1,155 @@ +:root { + color-scheme: light; + --ink: #172033; + --muted: #526177; + --accent: #4338ca; + --line: #dbe2ec; + --soft: #f8fafc; + font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; + color: var(--ink); + background: #fff; + font-synthesis: none; + text-rendering: optimizeLegibility; +} +* { box-sizing: border-box; } +body { margin: 0; } +a { color: inherit; text-decoration: none; } +a:focus-visible { outline: 3px solid var(--accent); outline-offset: 5px; } +p { color: var(--muted); line-height: 1.65; } +h1, h2, h3, p { margin-top: 0; } +h1, h2, h3 { letter-spacing: -.035em; } +h2 { font-size: clamp(1.8rem, 3vw, 2.35rem); line-height: 1.15; margin-bottom: 0; } +h3 { font-size: 1.06rem; } +pre, code, .eyebrow, .node-kind, .panel-tag, .badge, .doc-number { font-family: "SFMono-Regular", Consolas, "Liberation Mono", monospace; } +pre { overflow-x: auto; margin: 0; } +.wrap { width: min(1120px, calc(100% - 64px)); margin-inline: auto; } +.skip-link { position: fixed; top: -100px; left: 16px; z-index: 2; padding: 12px 18px; background: var(--ink); color: white; } +.skip-link:focus { top: 16px; } +.site-header { min-height: 96px; display: flex; align-items: center; justify-content: space-between; border-bottom: 1px solid var(--line); gap: 24px; } +.brand { display: inline-flex; align-items: center; gap: 8px; font-size: 19px; font-weight: 700; letter-spacing: -.04em; white-space: nowrap; } +.brand-core { color: var(--muted); font-weight: 450; } +.brand-mark { display: inline-flex; align-items: center; justify-content: center; width: 32px; height: 32px; border-radius: 7px; background: var(--ink); color: #fff; font: 700 22px/1 monospace; margin-right: 3px; padding-bottom: 5px; } +.brand-mark span { color: #a5b4fc; } +nav { display: flex; align-items: center; gap: 30px; font-size: 14px; font-weight: 550; } +nav a:hover, .site-footer > a:last-child:hover { color: var(--accent); } +.nav-github { display: inline-flex; gap: 12px; } +.hero { display: grid; grid-template-columns: 1fr 1fr; gap: 64px; align-items: center; padding-block: 82px 76px; } +.eyebrow { text-transform: uppercase; letter-spacing: .1em; font-size: 11px; font-weight: 650; color: var(--accent); margin-bottom: 24px; display: flex; align-items: center; gap: 9px; } +.small-square { width: 6px; height: 6px; background: var(--accent); } +h1 { font-size: clamp(2.5rem, 4.7vw, 3.6rem); line-height: 1.075; margin-bottom: 24px; font-weight: 680; letter-spacing: -.055em; } +h1 > span { color: var(--accent); } +.hero-description { font-size: 16px; max-width: 455px; margin-bottom: 28px; } +.actions { display: flex; flex-wrap: wrap; gap: 12px; } +.button { min-height: 46px; padding: 12px 19px; border: 1px solid var(--line); border-radius: 6px; font-size: 14px; font-weight: 650; display: inline-flex; align-items: center; justify-content: center; gap: 24px; } +.primary { background: var(--accent); border-color: var(--accent); color: #fff; } +.primary:hover { background: #3730a3; } +.secondary:hover { background: var(--soft); border-color: #a9b4c5; } +.hero-note { font-size: 12px; margin: 18px 0 0; } +.code-panel { min-width: 0; border: 1px solid #cfd7e4; border-radius: 9px; background: var(--soft); box-shadow: 0 12px 28px -22px #33415580; } +.panel-header { padding: 16px 20px; border-bottom: 1px solid var(--line); display: flex; gap: 12px; justify-content: space-between; align-items: center; } +.file-label { font: 12px "SFMono-Regular", Consolas, monospace; } +.file-label > span { color: var(--muted); margin-right: 7px; } +.panel-tag { color: var(--muted); font-size: 9px; letter-spacing: .1em; } +.code-intro { font-size: 13px; padding: 23px 24px 18px; color: var(--muted); } +.code-panel pre { font-size: 12px; line-height: 1.85; padding: 0 24px 25px; color: #26334b; } +.syntax-keyword { color: #6d28d9; } +.syntax-string { color: #116b58; } +.syntax-function { color: #1d4ed8; } +.code-caption { padding: 16px 24px; border-block: 1px solid var(--line); font-size: 11px; display: flex; flex-wrap: wrap; gap: 6px 16px; } +.caption-label { color: var(--muted); } +.code-footnote { font-size: 11px; padding: 16px 24px; margin: 0; } +.code-footnote a { color: var(--accent); white-space: nowrap; text-decoration: underline; text-underline-offset: 3px; } +.architecture-section { padding-block: 56px; background: var(--soft); border-block: 1px solid var(--line); } +.section-heading { display: flex; justify-content: space-between; gap: 40px; align-items: end; margin-bottom: 30px; } +.section-heading .eyebrow { margin-bottom: 14px; } +.section-heading > p { max-width: 365px; font-size: 14px; margin: 0; } +.architecture { display: grid; grid-template-columns: 1fr 52px 1fr 52px 1fr; align-items: center; } +.architecture-node { background: #fff; border: 1px solid #cfd7e4; padding: 25px; border-radius: 7px; } +.node-kind { font-size: 10px; letter-spacing: .08em; color: var(--muted); } +.architecture-node h3 { font-size: 21px; margin: 14px 0 7px; } +.architecture-node p { font-size: 12px; margin: 0; } +.runtime-node { border-color: #a5b4fc; } +.runtime-node .node-kind { color: var(--accent); } +.connector { text-align: center; color: #64748b; font-size: 24px; } +.harnesses { display: grid; gap: 8px; } +.harnesses span { padding: 11px 20px; border: 1px solid var(--line); border-radius: 5px; background: #fff; font-size: 13px; font-weight: 600; } +.architecture-notes { display: grid; grid-template-columns: 1fr 1fr; gap: 52px; margin-top: 24px; } +.architecture-notes p { font-size: 12px; margin: 0; } +.architecture-notes strong { color: var(--ink); font-weight: 600; } +.install-section { padding-block: 72px; display: grid; grid-template-columns: .8fr 1.2fr; gap: 90px; } +.install-copy .eyebrow { margin-bottom: 15px; } +.install-copy h2 { margin-bottom: 24px; } +.install-copy > p:not(.eyebrow) { font-size: 14px; } +.text-link { color: var(--accent); display: inline-flex; gap: 16px; font-size: 14px; font-weight: 600; margin-top: 6px; } +.text-link:hover, .bundle-note a:hover { text-decoration: underline; text-underline-offset: 4px; } +.install-default { border: 1px solid #c7d2fe; border-radius: 7px; background: #f5f6ff; padding: 21px 24px; } +.install-label { display: flex; align-items: center; justify-content: space-between; gap: 12px; margin-bottom: 15px; } +.install-label h3 { margin: 0; } +.badge { color: var(--accent); font-size: 9px; letter-spacing: .06em; } +.install-default pre { font-size: 19px; } +.install-default p { font-size: 12px; margin: 13px 0 0; } +.install-option { display: flex; align-items: center; justify-content: space-between; gap: 16px; padding: 20px 0; border-bottom: 1px solid var(--line); } +.install-option h3 { font-size: 13px; margin: 0 0 3px; letter-spacing: 0; } +.install-option p { font-size: 11px; margin: 0; } +.install-option code { font-size: 11px; white-space: nowrap; } +.bundle-note { font-size: 11px; margin: 17px 0 0; } +.bundle-note a { color: var(--accent); } +.docs-section { border-top: 1px solid var(--line); padding-block: 55px 64px; } +.docs-grid { display: grid; grid-template-columns: repeat(3, 1fr); gap: 20px; } +.doc-card { border: 1px solid var(--line); padding: 25px; border-radius: 7px; display: flex; flex-direction: column; } +.doc-card:hover { border-color: #a5b4fc; background: var(--soft); } +.doc-number { color: var(--muted); font-size: 11px; display: flex; justify-content: space-between; margin-bottom: 30px; } +.doc-number > span { font: 18px sans-serif; } +.doc-card h3 { margin-bottom: 12px; font-size: 17px; } +.doc-card p { font-size: 13px; margin-bottom: 22px; } +.doc-link { margin-top: auto; color: var(--accent); font-size: 12px; font-weight: 600; } +.site-footer { border-top: 1px solid var(--line); display: flex; align-items: center; justify-content: space-between; gap: 24px; padding-block: 28px; } +.site-footer .brand { font-size: 15px; } +.site-footer .brand-mark { width: 25px; height: 25px; font-size: 18px; } +.site-footer p { font-size: 12px; margin: 0; } +.site-footer > a:last-child { font-size: 12px; } +section[id] { scroll-margin-top: 24px; } +@media (max-width: 1000px) { + .hero { gap: 32px; } + .code-panel pre { font-size: 11px; padding-inline: 18px; } + .install-section { gap: 45px; } + .install-option { align-items: start; flex-direction: column; gap: 10px; } +} +@media (max-width: 760px) { + .wrap { width: calc(100% - 40px); } + .site-header { min-height: 80px; } + nav { gap: 18px; font-size: 12px; } + nav a:first-child { display: none; } + .hero { grid-template-columns: 1fr; padding-block: 48px; gap: 36px; } + h1 { font-size: clamp(2.75rem, 9vw, 4rem); } + .hero-description { max-width: 520px; } + .code-panel pre { font-size: 12px; padding-inline: 24px; } + .section-heading { flex-direction: column; align-items: start; gap: 20px; } + .section-heading > p { max-width: 100%; } + .architecture-section { padding-block: 40px; } + .architecture { grid-template-columns: 1fr; } + .connector { transform: rotate(90deg); padding: 8px; } + .architecture-node { padding: 22px; } + .harnesses { grid-template-columns: repeat(3, 1fr); } + .harnesses span { text-align: center; padding: 14px 8px; font-size: 12px; } + .architecture-notes { grid-template-columns: 1fr; gap: 12px; margin-top: 22px; } + .install-section { grid-template-columns: 1fr; gap: 28px; padding-block: 44px; } + .install-copy h2 br { display: none; } + .install-copy h2 { margin-bottom: 18px; } + .install-option { flex-direction: row; align-items: center; gap: 16px; } + .docs-section { padding-block: 40px; } + .docs-grid { grid-template-columns: 1fr; gap: 12px; } + .doc-card { padding: 22px; } + .doc-number { margin-bottom: 16px; } + .doc-card p { margin-bottom: 16px; } + .site-footer { flex-wrap: wrap; gap: 18px; } + .site-footer p { order: 3; width: 100%; } +} +@media (max-width: 400px) { + .wrap { width: calc(100% - 32px); } + .brand { gap: 5px; font-size: 17px; } + nav { gap: 14px; } + .code-panel pre { font-size: 10px; padding-inline: 16px; } + .code-intro, .code-caption, .code-footnote { padding-inline: 16px; } + .install-option { flex-direction: column; align-items: start; gap: 10px; } +} From 9aed8fcd5e9c25063afc39c7de62f0cb9ee9a780 Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 14:43:08 +0800 Subject: [PATCH 02/17] Add authenticated production Core console proxy --- scripts/build-core-console.sh | 35 ++++ services/core-console/Dockerfile | 9 + services/core-console/config.go | 88 ++++++++++ services/core-console/config_test.go | 53 ++++++ services/core-console/main.go | 52 ++++++ services/core-console/server.go | 174 +++++++++++++++++++ services/core-console/server_test.go | 250 +++++++++++++++++++++++++++ 7 files changed, 661 insertions(+) create mode 100755 scripts/build-core-console.sh create mode 100644 services/core-console/Dockerfile create mode 100644 services/core-console/config.go create mode 100644 services/core-console/config_test.go create mode 100644 services/core-console/main.go create mode 100644 services/core-console/server.go create mode 100644 services/core-console/server_test.go diff --git a/scripts/build-core-console.sh b/scripts/build-core-console.sh new file mode 100755 index 000000000..35dde9a72 --- /dev/null +++ b/scripts/build-core-console.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash +set -euo pipefail + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +runtime_root="${PARSAR_HOME:-$HOME/.parsar}" +output_dir="${CORE_CONSOLE_BUILD_DIR:-$runtime_root/build/core-console}" +export GOCACHE="${GOCACHE:-$runtime_root/cache/go-build}" +export GOMODCACHE="${GOMODCACHE:-$runtime_root/cache/go-mod}" +for directory in "$runtime_root" "$output_dir" "$GOCACHE" "$GOMODCACHE"; do + case "$directory" in + "$HOME/.parsar"|"$HOME/.parsar/"*) ;; + *) printf 'Core console build directories must be absolute and under ~/.parsar\n' >&2; exit 1 ;; + esac + case "/$directory/" in + */../*|*/./*) printf 'Core console build directories must not contain dot segments\n' >&2; exit 1 ;; + esac +done + +mkdir -p "$runtime_root/cache/core-console-builds" +build_context="$(mktemp -d "$runtime_root/cache/core-console-builds/source.XXXXXX")" +trap 'rm -rf "$build_context"' EXIT +mkdir -p "$build_context/tmp" +export GOTMPDIR="$build_context/tmp" +# Keep the independent console build separate from API and frontend sources. +tar -C "$repo_root" -cf - go.mod go.sum internal/obs/log services/core-console \ + | tar -C "$build_context" -xf - +( + cd "$build_context" + export GOWORK=off CGO_ENABLED=0 + go build -mod=readonly -trimpath -buildvcs=false \ + -o "$build_context/core-console" ./services/core-console +) +mkdir -p "$output_dir" +mv -f "$build_context/core-console" "$output_dir/core-console" +printf 'Core console binary: %s/core-console\n' "$output_dir" diff --git a/services/core-console/Dockerfile b/services/core-console/Dockerfile new file mode 100644 index 000000000..5621b5f64 --- /dev/null +++ b/services/core-console/Dockerfile @@ -0,0 +1,9 @@ +# The build context contains only core-console and the existing Web dist. +FROM gcr.io/distroless/static-debian13:nonroot@sha256:e754765ad9e167b0677b41c617fd44afb7b9818a477f48f17bda08e12cfb98cb + +COPY --chmod=0555 core-console /usr/local/bin/core-console +COPY dist /www +ENV CORE_CONSOLE_ADDR=:8080 CORE_CONSOLE_DIST=/www +EXPOSE 8080 +USER 65532:65532 +CMD ["/usr/local/bin/core-console"] diff --git a/services/core-console/config.go b/services/core-console/config.go new file mode 100644 index 000000000..6c4f59779 --- /dev/null +++ b/services/core-console/config.go @@ -0,0 +1,88 @@ +package main + +import ( + "errors" + "io" + "net/url" + "os" + "path/filepath" + "strings" + "unicode" +) + +type config struct { + addr, origin, dist, token, password string + upstream *url.URL +} + +func loadConfig() (config, error) { + c := config{ + addr: envDefault("CORE_CONSOLE_ADDR", ":8080"), + origin: envDefault("CORE_CONSOLE_ORIGIN", "http://127.0.0.1:8080"), + dist: envDefault("CORE_CONSOLE_DIST", "/www"), + } + origin, err := serverURL(c.origin) + if err != nil || origin.Path != "" { + return config{}, errors.New("CORE_CONSOLE_ORIGIN must be an HTTP(S) origin without a path") + } + c.upstream, err = serverURL(envDefault("CORE_CONSOLE_UPSTREAM", "http://core:8091")) + if err != nil { + return config{}, errors.New("CORE_CONSOLE_UPSTREAM must be an HTTP(S) server URL without credentials, query or path") + } + if !filepath.IsAbs(c.dist) { + return config{}, errors.New("CORE_CONSOLE_DIST must be absolute") + } + c.token, err = readSecret(envDefault("CORE_CONSOLE_TOKEN_FILE", "/config/caller.key")) + if err != nil { + return config{}, errors.New("CORE_CONSOLE_TOKEN_FILE must name a private regular file containing one token") + } + c.password, err = readSecret(envDefault("CORE_CONSOLE_PASSWORD_FILE", "/config/console.password")) + if err != nil { + return config{}, errors.New("CORE_CONSOLE_PASSWORD_FILE must name a private regular file containing one password") + } + if c.password == c.token { + return config{}, errors.New("console password and Core bearer token must differ") + } + return c, nil +} + +func envDefault(key, fallback string) string { + if value, exists := os.LookupEnv(key); exists { + return value + } + return fallback +} + +func serverURL(value string) (*url.URL, error) { + u, err := url.Parse(value) + if err != nil || u.Host == "" || (u.Scheme != "http" && u.Scheme != "https") || + u.User != nil || u.RawQuery != "" || u.ForceQuery || u.Fragment != "" || + strings.ContainsAny(value, "?#") || (u.Path != "" && u.Path != "/") || u.RawPath != "" { + return nil, errors.New("invalid server URL") + } + return u, nil +} + +func readSecret(name string) (string, error) { + if !filepath.IsAbs(name) { + return "", errors.New("secret path must be absolute") + } + f, err := os.Open(name) + if err != nil { + return "", err + } + defer f.Close() + info, err := f.Stat() + if err != nil || !info.Mode().IsRegular() || info.Mode().Perm()&0o077 != 0 { + return "", errors.New("secret file must be private and regular") + } + data, err := io.ReadAll(io.LimitReader(f, 4097)) + if err != nil || len(data) > 4096 { + return "", errors.New("invalid secret size") + } + value := strings.TrimSpace(string(data)) + if value == "" || strings.IndexFunc(value, unicode.IsSpace) >= 0 || strings.ContainsAny(value, "\x00\r\n") { + return "", errors.New("invalid secret") + } + return value, nil +} diff --git a/services/core-console/config_test.go b/services/core-console/config_test.go new file mode 100644 index 000000000..beebaeba7 --- /dev/null +++ b/services/core-console/config_test.go @@ -0,0 +1,53 @@ +package main + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +func TestConfigRejectsUnsafeURLsAndSecretFiles(t *testing.T) { + directory := t.TempDir() + token, password := filepath.Join(directory, "caller.key"), filepath.Join(directory, "console.password") + for name, value := range map[string]string{token: "private-core-token\n", password: "private-console-password\n"} { + if err := os.WriteFile(name, []byte(value), 0o600); err != nil { + t.Fatal(err) + } + } + t.Setenv("CORE_CONSOLE_TOKEN_FILE", token) + t.Setenv("CORE_CONSOLE_PASSWORD_FILE", password) + t.Setenv("CORE_CONSOLE_ORIGIN", testOrigin) + t.Setenv("CORE_CONSOLE_UPSTREAM", "http://core:8091") + t.Setenv("CORE_CONSOLE_DIST", directory) + c, err := loadConfig() + if err != nil || c.token != "private-core-token" { + t.Fatalf("valid configuration failed: %v", err) + } + for _, value := range []string{"http://user:secret@core:8091", "http://core:8091/v1", "http://core:8091?token=secret", "http://core:8091#", "file:///config/caller.key", ""} { + t.Run(value, func(t *testing.T) { + t.Setenv("CORE_CONSOLE_UPSTREAM", value) + _, err := loadConfig() + if err == nil || strings.Contains(err.Error(), "secret") { + t.Fatal("unsafe URL accepted or exposed") + } + }) + } + for _, value := range []string{"", "token with spaces", strings.Repeat("x", 4097), "token\x00"} { + if err := os.WriteFile(token, []byte(value), 0o600); err != nil { + t.Fatal(err) + } + if _, err := loadConfig(); err == nil { + t.Fatal("invalid token accepted") + } + } + if err := os.WriteFile(token, []byte("valid-token"), 0o600); err != nil { + t.Fatal(err) + } + if err := os.Chmod(token, 0o644); err != nil { + t.Fatal(err) + } + if _, err := loadConfig(); err == nil { + t.Fatal("publicly readable secret accepted") + } +} diff --git a/services/core-console/main.go b/services/core-console/main.go new file mode 100644 index 000000000..9990cfe96 --- /dev/null +++ b/services/core-console/main.go @@ -0,0 +1,52 @@ +// Command core-console serves the Core Web build and its authenticated API proxy. +package main + +import ( + "context" + "errors" + "net/http" + "os" + "os/signal" + "syscall" + "time" + + "github.com/MiniMax-AI-Dev/parsar/internal/obs/log" +) + +func main() { + if err := run(); err != nil { + log.Bg().Error("Core console stopped", "error", err) + os.Exit(1) + } +} + +func run() error { + c, err := loadConfig() + if err != nil { + return err + } + handler, err := newConsole(c) + if err != nil { + return err + } + defer handler.Close() + ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) + defer stop() + server := &http.Server{Addr: c.addr, Handler: handler, ReadHeaderTimeout: 10 * time.Second, IdleTimeout: 60 * time.Second} + done := make(chan error, 1) + go func() { done <- server.ListenAndServe() }() + select { + case err := <-done: + if errors.Is(err, http.ErrServerClosed) { + return nil + } + return errors.New("console listener failed") + case <-ctx.Done(): + shutdown, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + if err := server.Shutdown(shutdown); err != nil { + return server.Close() + } + return nil + } +} diff --git a/services/core-console/server.go b/services/core-console/server.go new file mode 100644 index 000000000..a4c615f34 --- /dev/null +++ b/services/core-console/server.go @@ -0,0 +1,174 @@ +package main + +import ( + "crypto/sha256" + "crypto/subtle" + "errors" + "io" + stdlog "log" + "net/http" + "net/http/httputil" + "net/url" + "os" + "path" + "strings" +) + +type console struct { + config + root *os.Root + proxy *httputil.ReverseProxy + transport *http.Transport + host string + password [sha256.Size]byte +} + +func newConsole(c config) (*console, error) { + root, err := os.OpenRoot(c.dist) + if err != nil { + return nil, errors.New("cannot open console assets") + } + index, err := root.Stat("index.html") + if err != nil || !index.Mode().IsRegular() { + root.Close() + return nil, errors.New("console assets require index.html") + } + origin, _ := url.Parse(c.origin) + h := &console{config: c, root: root, host: origin.Host, password: sha256.Sum256([]byte(c.password))} + h.transport = http.DefaultTransport.(*http.Transport).Clone() + // Credentials go only to the configured Core, never an ambient HTTP proxy. + h.transport.Proxy = nil + h.proxy = &httputil.ReverseProxy{ + Transport: h.transport, + FlushInterval: -1, + ErrorLog: stdlog.New(io.Discard, "", 0), + Rewrite: func(r *httputil.ProxyRequest) { + r.SetURL(c.upstream) + r.Out.Header.Del("Authorization") + r.Out.Header.Del("Proxy-Authorization") + r.Out.Header.Del("Cookie") + r.Out.Header.Del("Origin") + r.Out.Header.Del("Referer") + r.Out.Header.Set("Authorization", "Bearer "+c.token) + }, + ModifyResponse: func(r *http.Response) error { + // Never send a browser to a different origin with its cached login. + if r.StatusCode >= 300 && r.StatusCode < 400 { + return errors.New("Core redirects are not supported") + } + r.Header.Del("Set-Cookie") + r.Header.Del("WWW-Authenticate") + r.Header.Del("Location") + r.Header.Del("Refresh") + for key := range r.Header { + if strings.HasPrefix(strings.ToLower(key), "access-control-") { + r.Header.Del(key) + } + } + return nil + }, + ErrorHandler: func(w http.ResponseWriter, _ *http.Request, _ error) { + http.Error(w, "Core is unavailable", http.StatusBadGateway) + }, + } + return h, nil +} + +func (h *console) Close() { + h.transport.CloseIdleConnections() + _ = h.root.Close() +} + +func (h *console) ServeHTTP(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Cache-Control", "no-store") + w.Header().Set("X-Content-Type-Options", "nosniff") + w.Header().Set("Referrer-Policy", "no-referrer") + w.Header().Set("Content-Security-Policy", "frame-ancestors 'none'") + if r.URL.Path == "/healthz" && (r.Method == http.MethodGet || r.Method == http.MethodHead) { + w.WriteHeader(http.StatusOK) + _, _ = io.WriteString(w, "ok\n") + return + } + if !h.sameOrigin(r) { + http.Error(w, "Forbidden", http.StatusForbidden) + return + } + username, password, ok := r.BasicAuth() + digest := sha256.Sum256([]byte(password)) + userDigest, adminDigest := sha256.Sum256([]byte(username)), sha256.Sum256([]byte("admin")) + if !ok || len(r.Header.Values("Authorization")) != 1 || + subtle.ConstantTimeCompare(digest[:], h.password[:])&subtle.ConstantTimeCompare(userDigest[:], adminDigest[:]) != 1 { + w.Header().Set("WWW-Authenticate", `Basic realm="Core console", charset="UTF-8"`) + http.Error(w, "Authentication required", http.StatusUnauthorized) + return + } + if !safePath(r.URL.Path) || r.URL.IsAbs() || r.Method == http.MethodConnect || r.Method == http.MethodTrace || r.Header.Get("Upgrade") != "" { + http.Error(w, "Invalid request", http.StatusBadRequest) + return + } + if r.URL.Path == "/v1" || strings.HasPrefix(r.URL.Path, "/v1/") { + h.proxy.ServeHTTP(w, r) + return + } + h.serveStatic(w, r) +} + +func (h *console) sameOrigin(r *http.Request) bool { + if r.Host != h.host { + return false + } + if origins := r.Header.Values("Origin"); len(origins) > 1 || (len(origins) == 1 && origins[0] != h.origin) { + return false + } + sites := r.Header.Values("Sec-Fetch-Site") + if len(sites) > 1 { + return false + } + if len(sites) == 1 && sites[0] != "same-origin" && sites[0] != "none" { + return false + } + // Modern browsers provide Fetch Metadata; older same-origin requests carry + // Origin or Referer. Reject an ambiguous browser mutation before proxying. + if r.Method != http.MethodGet && r.Method != http.MethodHead && + r.Header.Get("Origin") == "" && r.Header.Get("Sec-Fetch-Site") != "same-origin" { + referrer, err := url.Parse(r.Referer()) + if err != nil || referrer.Scheme+"://"+referrer.Host != h.origin { + return false + } + } + return true +} + +func safePath(value string) bool { + if !strings.HasPrefix(value, "/") || strings.ContainsAny(value, "\\\x00%") { + return false + } + return path.Clean(value) == strings.TrimSuffix(value, "/") || value == "/" +} + +func (h *console) serveStatic(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodGet && r.Method != http.MethodHead { + w.Header().Set("Allow", "GET, HEAD") + http.Error(w, "Method not allowed", http.StatusMethodNotAllowed) + return + } + name := strings.TrimPrefix(r.URL.Path, "/") + if name == "" { + name = "index.html" + } + file, err := h.root.Open(name) + if errors.Is(err, os.ErrNotExist) && path.Ext(name) == "" && !strings.HasPrefix(name, "api/") { + file, err = h.root.Open("index.html") + } + if err != nil { + http.NotFound(w, r) + return + } + defer file.Close() + info, err := file.Stat() + if err != nil || !info.Mode().IsRegular() { + http.NotFound(w, r) + return + } + http.ServeContent(w, r, info.Name(), info.ModTime(), file) +} diff --git a/services/core-console/server_test.go b/services/core-console/server_test.go new file mode 100644 index 000000000..dee0eabff --- /dev/null +++ b/services/core-console/server_test.go @@ -0,0 +1,250 @@ +package main + +import ( + "bufio" + "context" + "fmt" + "io" + "net/http" + "net/http/httptest" + "net/url" + "os" + "path/filepath" + "strings" + "sync/atomic" + "testing" + "time" +) + +const testOrigin = "http://127.0.0.1:8080" + +func testConsole(t *testing.T, backend http.Handler) (*httptest.Server, string) { + t.Helper() + upstream := httptest.NewServer(backend) + t.Cleanup(upstream.Close) + u, err := url.Parse(upstream.URL) + if err != nil { + t.Fatal(err) + } + dist := t.TempDir() + if err := os.WriteFile(filepath.Join(dist, "index.html"), []byte("existing web build"), 0o644); err != nil { + t.Fatal(err) + } + handler, err := newConsole(config{origin: testOrigin, upstream: u, dist: dist, token: "private-core-token", password: "private-console-password"}) + if err != nil { + t.Fatal(err) + } + t.Cleanup(handler.Close) + server := httptest.NewServer(handler) + t.Cleanup(server.Close) + return server, dist +} + +func consoleRequest(t *testing.T, server *httptest.Server, method, path string) *http.Request { + t.Helper() + r, err := http.NewRequest(method, server.URL+path, nil) + if err != nil { + t.Fatal(err) + } + r.Host = "127.0.0.1:8080" + r.SetBasicAuth("admin", "private-console-password") + r.Header.Set("Origin", testOrigin) + r.Header.Set("Sec-Fetch-Site", "same-origin") + return r +} + +func responseBody(t *testing.T, server *httptest.Server, r *http.Request) (*http.Response, string) { + t.Helper() + response, err := server.Client().Do(r) + if err != nil { + t.Fatal(err) + } + defer response.Body.Close() + body, err := io.ReadAll(response.Body) + if err != nil { + t.Fatal(err) + } + return response, string(body) +} + +func TestAuthenticationAndCrossSiteAdmission(t *testing.T) { + var calls atomic.Int32 + server, _ := testConsole(t, http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + calls.Add(1) + _, _ = io.WriteString(w, "[]") + })) + cases := []struct { + name string + method string + change func(*http.Request) + status int + }{ + {"missing login", "GET", func(r *http.Request) { r.Header.Del("Authorization") }, 401}, + {"wrong password", "GET", func(r *http.Request) { r.SetBasicAuth("admin", "wrong") }, 401}, + {"wrong user", "GET", func(r *http.Request) { r.SetBasicAuth("other", "private-console-password") }, 401}, + {"duplicate login", "GET", func(r *http.Request) { r.Header.Add("Authorization", r.Header.Get("Authorization")) }, 401}, + {"DNS rebinding host", "GET", func(r *http.Request) { r.Host = "attacker.example:8080" }, 403}, + {"cross origin", "POST", func(r *http.Request) { r.Header.Set("Origin", "https://attacker.example") }, 403}, + {"null origin", "POST", func(r *http.Request) { r.Header.Set("Origin", "null") }, 403}, + {"duplicate origin", "POST", func(r *http.Request) { r.Header.Add("Origin", testOrigin) }, 403}, + {"cross-site GET", "GET", func(r *http.Request) { r.Header.Set("Sec-Fetch-Site", "cross-site") }, 403}, + {"same-site subdomain", "POST", func(r *http.Request) { r.Header.Set("Sec-Fetch-Site", "same-site") }, 403}, + {"missing mutation provenance", "POST", func(r *http.Request) { + r.Header.Del("Origin") + r.Header.Del("Sec-Fetch-Site") + }, 403}, + {"upgrade", "GET", func(r *http.Request) { r.Header.Set("Upgrade", "websocket") }, 400}, + {"TRACE credential reflection", "TRACE", func(_ *http.Request) {}, 400}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + r := consoleRequest(t, server, tc.method, "/v1/agents") + tc.change(r) + response, body := responseBody(t, server, r) + if response.StatusCode != tc.status { + t.Fatalf("status = %d, body = %s", response.StatusCode, body) + } + if strings.Contains(body, "private-") { + t.Fatal("rejection exposed a credential") + } + }) + } + if calls.Load() != 0 { + t.Fatal("rejected request reached Core") + } + anonymous := consoleRequest(t, server, "GET", "/") + anonymous.Header.Del("Authorization") + response, body := responseBody(t, server, anonymous) + if response.StatusCode != 401 || strings.Contains(body, "existing web build") { + t.Fatal("static assets bypassed console authentication") + } + request := consoleRequest(t, server, "GET", "/healthz") + request.Header = make(http.Header) + request.Host = "healthcheck" + response, body = responseBody(t, server, request) + if response.StatusCode != 200 || body != "ok\n" { + t.Fatal("liveness requires authentication") + } +} + +func TestProxyUsesOnlyConfiguredCoreCredential(t *testing.T) { + observed := make(chan *http.Request, 1) + server, _ := testConsole(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + observed <- r.Clone(context.Background()) + w.Header().Set("Set-Cookie", "upstream=credential") + w.Header().Set("Access-Control-Allow-Origin", "*") + w.Header().Set("WWW-Authenticate", "Bearer") + w.Header().Set("Content-Type", "application/json") + _, _ = io.WriteString(w, `{"data":[]}`) + })) + r := consoleRequest(t, server, "POST", "/v1/agents?limit=5") + r.Header.Set("Cookie", "browser=private") + r.Header.Set("Proxy-Authorization", "Basic browser-secret") + r.Header.Set("OpenAI-Beta", "agents=v1") + r.Header.Set("Forwarded", "host=attacker.example") + r.Header.Set("X-Forwarded-Host", "attacker.example") + response, body := responseBody(t, server, r) + if response.StatusCode != 200 || body != `{"data":[]}` { + t.Fatalf("proxy response: %d %s", response.StatusCode, body) + } + for _, name := range []string{"Set-Cookie", "Access-Control-Allow-Origin", "WWW-Authenticate"} { + if response.Header.Get(name) != "" { + t.Errorf("upstream %s reached browser", name) + } + } + request := <-observed + if request.URL.RequestURI() != "/v1/agents?limit=5" || request.Header.Get("Authorization") != "Bearer private-core-token" || request.Header.Get("OpenAI-Beta") != "agents=v1" { + t.Fatal("proxy changed the public request or failed to inject the Core token") + } + for _, name := range []string{"Cookie", "Proxy-Authorization", "Origin", "Referer", "Forwarded", "X-Forwarded-Host"} { + if request.Header.Get(name) != "" { + t.Errorf("browser %s reached Core", name) + } + } +} + +func TestProxyRejectsRedirectWithoutFollowingOrExposingIt(t *testing.T) { + var destinationCalls atomic.Int32 + destination := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + destinationCalls.Add(1) + w.WriteHeader(200) + })) + defer destination.Close() + server, _ := testConsole(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, destination.URL+"/?token=private-core-token", http.StatusTemporaryRedirect) + })) + response, body := responseBody(t, server, consoleRequest(t, server, "GET", "/v1/agents")) + if response.StatusCode != 502 || response.Header.Get("Location") != "" || strings.Contains(body, "private-core-token") || destinationCalls.Load() != 0 { + t.Fatal("upstream redirect escaped the fixed proxy") + } +} + +func TestProxyFlushesSSEAndCancelsUpstream(t *testing.T) { + cancelled := make(chan struct{}) + server, _ := testConsole(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "text/event-stream") + _, _ = fmt.Fprint(w, "data: first\n\n") + w.(http.Flusher).Flush() + <-r.Context().Done() + close(cancelled) + })) + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + r := consoleRequest(t, server, "GET", "/v1/agents/sessions/session/events").WithContext(ctx) + response, err := server.Client().Do(r) + if err != nil { + t.Fatal(err) + } + line, err := bufio.NewReader(response.Body).ReadString('\n') + if err != nil || line != "data: first\n" { + t.Fatalf("SSE did not flush: %q %v", line, err) + } + response.Body.Close() + cancel() + select { + case <-cancelled: + case <-time.After(3 * time.Second): + t.Fatal("disconnect did not cancel Core request") + } +} + +func TestStaticAssetsStayInsideDistAndInternalRoutesStayLocal(t *testing.T) { + var calls atomic.Int32 + server, dist := testConsole(t, http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + calls.Add(1) + w.WriteHeader(500) + })) + outside := filepath.Join(t.TempDir(), "caller.key") + if err := os.WriteFile(outside, []byte("outside-secret"), 0o600); err != nil { + t.Fatal(err) + } + if err := os.Symlink(outside, filepath.Join(dist, "leak.key")); err != nil { + t.Fatal(err) + } + if err := os.Mkdir(filepath.Join(dist, "assets"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dist, "assets", "main.js"), []byte("console.log('existing');"), 0o644); err != nil { + t.Fatal(err) + } + for _, tc := range []struct { + path string + status int + }{ + {"/", 200}, {"/sessions/saved", 200}, {"/assets/main.js", 200}, + {"/assets/", 404}, {"/missing.js", 404}, {"/leak.key", 404}, + {"/../caller.key", 400}, {"/%2e%2e/caller.key", 400}, {"/%252e%252e/caller.key", 400}, + {"/v1/../api/v1/agent-daemon/ws", 400}, {"/v1//agents", 400}, + {"/api/v1/agent-daemon/ws", 404}, + } { + t.Run(tc.path, func(t *testing.T) { + response, body := responseBody(t, server, consoleRequest(t, server, "GET", tc.path)) + if response.StatusCode != tc.status || strings.Contains(body, "outside-secret") { + t.Fatalf("static response = %d %s", response.StatusCode, body) + } + }) + } + if calls.Load() != 0 { + t.Fatal("non-API path reached Core") + } +} From 2954570fa7716772b87beb65849a07bb4206c3cc Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 14:43:11 +0800 Subject: [PATCH 03/17] Package microsandbox Core and shared harness Runtime images --- deploy/distribution/Dockerfile | 15 +++++++++++++ deploy/distribution/Runtime.Dockerfile | 29 ++++++++++++++++++++++++++ 2 files changed, 44 insertions(+) create mode 100644 deploy/distribution/Dockerfile create mode 100644 deploy/distribution/Runtime.Dockerfile diff --git a/deploy/distribution/Dockerfile b/deploy/distribution/Dockerfile new file mode 100644 index 000000000..4367fa021 --- /dev/null +++ b/deploy/distribution/Dockerfile @@ -0,0 +1,15 @@ +# Context contains matched Core binaries and the verified microsandbox v0.7.2 release. +# The helper embeds its pinned SDK FFI; unlike the basic static image, this image +# supplies the glibc runtime needed to load that library. +FROM debian:bookworm-slim@sha256:a0982977ea1cf754c15281f48a7d5957cd14039a2a4f2aca9f23d74d226003d0 + +RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates libgcc-s1 \ + && rm -rf /var/lib/apt/lists/* + +COPY --chmod=0555 bin/agents-api bin/agents-api-migrate bin/agents-api-device bin/agents-api-environment-key bin/agents-api-microsandbox-provider /usr/local/bin/ +COPY --chmod=0555 microsandbox/msb microsandbox/libkrunfw.so.5.6.1 /opt/microsandbox/ + +ENV AGENTS_API_ADDR=:8091 +EXPOSE 8091 +USER 65532:65532 +CMD ["/usr/local/bin/agents-api"] diff --git a/deploy/distribution/Runtime.Dockerfile b/deploy/distribution/Runtime.Dockerfile new file mode 100644 index 000000000..97ab1d85a --- /dev/null +++ b/deploy/distribution/Runtime.Dockerfile @@ -0,0 +1,29 @@ +# Each input is an immutable Linux amd64 image built from the same Core revision. +# Reuse the native packages and isolation configuration from existing profiles. +ARG CODEX_IMAGE +ARG CLAUDE_IMAGE +ARG MCODE_IMAGE +FROM ${CODEX_IMAGE} AS codex +FROM ${CLAUDE_IMAGE} AS claude +FROM ${MCODE_IMAGE} + +# Keep the shared daemon, helpers and prebuilt tool-system seed from this base. +# Native harness packages remain outside the tool-system seed and workspace. +COPY --from=codex /usr/local/bin/codex /usr/local/bin/codex +COPY --from=codex /usr/local/codex-resources /usr/local/codex-resources +COPY --from=codex /etc/codex /etc/codex +COPY --from=claude /opt/claude-sdk /opt/claude-sdk +COPY --from=claude /usr/local/bin/agents-api-claude-shell-prefix /usr/local/bin/agents-api-claude-shell-prefix + +ENV PARSAR_CODEX_BIN=/usr/local/bin/codex \ + PARSAR_CODEX_PERMISSION_PROFILE=managed-workspace \ + PARSAR_CLAUDE_SDK_NODE=/usr/local/bin/node \ + PARSAR_CLAUDE_SDK_ENTRYPOINT=/opt/claude-sdk/dist/main.js \ + PARSAR_CLAUDE_SDK_WORKSPACE=managed + +USER 1000:1000 +RUN test "$(codex --version)" = "codex-cli 0.153.4" \ + && test -r /etc/codex/requirements.toml \ + && node /opt/claude-sdk/dist/runtime_check.js /opt/claude-sdk/dist/main.js \ + && node /opt/mcode-harness/check.mjs \ + && /opt/mcode-harness/native/cli.js --version From ac18d1409bc8cdabd292b2a4b909ac34f6f2e8ac Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 14:51:35 +0800 Subject: [PATCH 04/17] Build verified offline Core installation bundles --- scripts/build-core-distribution.sh | 183 +++++++++++++++++++++ scripts/core-distribution-manifest.py | 130 +++++++++++++++ scripts/core-distribution-manifest.test.py | 83 ++++++++++ 3 files changed, 396 insertions(+) create mode 100755 scripts/build-core-distribution.sh create mode 100644 scripts/core-distribution-manifest.py create mode 100644 scripts/core-distribution-manifest.test.py diff --git a/scripts/build-core-distribution.sh b/scripts/build-core-distribution.sh new file mode 100755 index 000000000..b47116647 --- /dev/null +++ b/scripts/build-core-distribution.sh @@ -0,0 +1,183 @@ +#!/usr/bin/env bash +set -euo pipefail + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +runtime_root="${PARSAR_HOME:-$HOME/.parsar}" +output_dir="${CORE_DISTRIBUTION_BUILD_DIR:-$runtime_root/build/core-distribution}" +export GOCACHE="${GOCACHE:-$runtime_root/cache/go-build}" +export GOMODCACHE="${GOMODCACHE:-$runtime_root/cache/go-mod}" +export GOOS=linux GOARCH=amd64 GOAMD64=v1 GOTOOLCHAIN=local +export GOFLAGS=-buildvcs=false +python3 - "$HOME/.parsar" "$runtime_root" "$output_dir" "$GOCACHE" "$GOMODCACHE" <<'PY' +import pathlib, sys +if sys.version_info < (3, 9): + sys.exit("Distribution builds require Python 3.9 or newer") +base = pathlib.Path(sys.argv[1]).resolve() +for value in sys.argv[2:]: + path = pathlib.Path(value) + if not path.is_absolute() or not path.resolve().is_relative_to(base): + sys.exit("Distribution build directories must be absolute and under ~/.parsar") +PY +if [[ "$(uname -s)" != Linux || "$(uname -m)" != x86_64 ]]; then + printf 'Build the distribution on Linux x86_64 with a glibc compatible with Debian 12\n' >&2 + exit 1 +fi +for command in docker go node pnpm python3 curl tar sha256sum; do + command -v "$command" >/dev/null +done + +require_clean_source() { + if [[ -n "$(git -C "$repo_root" status --porcelain --untracked-files=all)" ]]; then + printf 'Distribution builds require clean, committed source\n' >&2 + exit 1 + fi +} +require_clean_source +revision="$(git -C "$repo_root" rev-parse HEAD)" +source_tree="$(git -C "$repo_root" rev-parse "$revision^{tree}")" +source_epoch="$(git -C "$repo_root" show -s --format=%ct "$revision")" +mkdir -p "$output_dir" +stage="$(mktemp -d "$output_dir/.build.XXXXXX")" +image_tags=() +cleanup() { + if (( ${#image_tags[@]} )); then docker image rm "${image_tags[@]}" >/dev/null 2>&1 || true; fi + rm -rf "$stage" +} +trap cleanup EXIT +source_dir="$stage/source" +bundle="$stage/parsar-core-$revision-linux-amd64" +mkdir -p "$source_dir" "$bundle/images" "$stage/core/bin" "$stage/core/microsandbox" "$stage/web" "$stage/tmp" +export GOTMPDIR="$stage/tmp" +git -C "$repo_root" archive --format=tar.gz --output="$bundle/source.tar.gz" "$revision" +tar -xzf "$bundle/source.tar.gz" -C "$source_dir" +cd "$source_dir" +required_go="$(awk '$1 == "go" { print "go" $2; exit }' go.mod)" +if [[ "$(go env GOVERSION)" != "$required_go" ]]; then + printf 'Distribution build requires %s\n' "$required_go" >&2 + exit 1 +fi +for file in install.sh install.py configuration.py; do + cp "deploy/install/$file" "$bundle/$file" +done +mkdir -p "$bundle/docs" +cp docs/getting-started.md "$bundle/docs/" +mkdir -p "$bundle/runtime" +cp services/agents-api/deploy/codex/seccomp.json "$bundle/runtime/" +cp LICENSE "$bundle/" +cp -R site "$bundle/site" + +AGENTS_API_BUILD_DIR="$stage/core/bin" scripts/build-agents-api.sh +( + cd services/agents-api/tools/microsandbox-provider + GOWORK=off CGO_ENABLED=1 go build -mod=readonly -trimpath \ + -o "$stage/core/bin/agents-api-microsandbox-provider" . +) +msb_archive="${CORE_DISTRIBUTION_MICROSANDBOX_ARCHIVE:-$runtime_root/cache/microsandbox-v0.7.2-linux-x86_64.tar.gz}" +if [[ ! -f "$msb_archive" ]]; then + mkdir -p "$(dirname "$msb_archive")" + curl --fail --location --proto '=https' --tlsv1.2 \ + https://github.com/superradcompany/microsandbox/releases/download/v0.7.2/microsandbox-linux-x86_64.tar.gz \ + --output "$stage/microsandbox.download" + python3 scripts/core-distribution-manifest.py extract-runtime "$stage/microsandbox.download" "$stage/core/microsandbox" + mv "$stage/microsandbox.download" "$msb_archive" +else + python3 scripts/core-distribution-manifest.py extract-runtime "$msb_archive" "$stage/core/microsandbox" +fi +cp deploy/distribution/Dockerfile "$stage/core/Dockerfile" +docker build --platform linux/amd64 --iidfile "$stage/core.id" \ + --label "org.opencontainers.image.revision=$revision" "$stage/core" +core_image="$(cat "$stage/core.id")" +# Fail at packaging time if the helper or runtime requires unavailable host libraries. +docker run --rm --network none --entrypoint /bin/sh "$core_image" -ec \ + 'for p in /usr/local/bin/agents-api-microsandbox-provider /opt/microsandbox/msb /opt/microsandbox/libkrunfw.so.5.6.1; do ! ldd "$p" | grep "not found"; done; /opt/microsandbox/msb --version' + +CORE_CONSOLE_BUILD_DIR="$stage/web" scripts/build-core-console.sh +pnpm install --frozen-lockfile +AGENTS_CORE_WEB_OPENAI_HOSTED_SESSIONS=1 AGENTS_CORE_WEB_ENVIRONMENT_FILES=1 pnpm build:web +cp -R apps/web/dist "$stage/web/dist" +cp services/core-console/Dockerfile "$stage/web/Dockerfile" +docker build --platform linux/amd64 --iidfile "$stage/web.id" \ + --label "org.opencontainers.image.revision=$revision" "$stage/web" + +export AGENTS_EXECUTOR_BUILD_DIR="$stage/helpers" +scripts/build-agents-executor.sh +CGO_ENABLED=0 go build -mod=readonly -trimpath -o "$stage/parsar-daemon" ./apps/parsar-daemon/cmd/parsar-daemon +codex_image="${CORE_DISTRIBUTION_CODEX_IMAGE:-}" +claude_image="${CORE_DISTRIBUTION_CLAUDE_IMAGE:-}" +mcode_image="${CORE_DISTRIBUTION_MCODE_IMAGE:-}" +if [[ -n "$codex_image$claude_image$mcode_image" ]]; then + if [[ -z "$codex_image" || -z "$claude_image" || -z "$mcode_image" ]]; then + printf 'Provide all three CORE_DISTRIBUTION_*_IMAGE inputs or none\n' >&2 + exit 1 + fi +else + : "${AGENTS_RUNTIME_CODEX_PACKAGE:?Set the extracted pinned Codex Linux x64 package directory}" + : "${MCODE_HARNESS_BUILD_DIR:?Set the existing built pinned MiniMax Code companion directory}" + export CLAUDE_SDK_BUILD_DIR="$stage/claude-sdk" + scripts/build-claude-sdk-runtime.sh + for harness in codex claude mcode; do + script="scripts/build-$harness-runtime.sh" + if [[ "$harness" == codex ]]; then script=scripts/build-agents-runtime.sh; fi + AGENTS_RUNTIME_BUILD_DIR="$stage/$harness" "$script" + docker build --platform linux/amd64 --iidfile "$stage/$harness.id" \ + --label "org.opencontainers.image.revision=$revision" "$stage/$harness" + done + codex_image="$(cat "$stage/codex.id")" + claude_image="$(cat "$stage/claude.id")" + mcode_image="$(cat "$stage/mcode.id")" +fi +for image in "$codex_image" "$claude_image" "$mcode_image"; do + python3 scripts/core-distribution-manifest.py verify-runtime "$image" "$stage/parsar-daemon" "$stage/helpers" "$source_dir" +done +tag_suffix="${stage##*.}" +for harness in codex claude mcode; do + image_variable="${harness}_image" + tag="parsar-core-distribution:$harness-$revision-$tag_suffix" + docker image tag "${!image_variable}" "$tag" + image_tags+=("$tag") +done +mkdir "$stage/combined" +cp deploy/distribution/Runtime.Dockerfile "$stage/combined/Dockerfile" +docker build --platform linux/amd64 --iidfile "$stage/runtime.id" \ + --label "org.opencontainers.image.revision=$revision" \ + --build-arg "CODEX_IMAGE=${image_tags[0]}" --build-arg "CLAUDE_IMAGE=${image_tags[1]}" \ + --build-arg "MCODE_IMAGE=${image_tags[2]}" "$stage/combined" + +database_image="${CORE_DISTRIBUTION_DATABASE_IMAGE:-postgres:16-alpine}" +if [[ ! "$database_image" =~ ^sha256:[0-9a-f]{64}$ ]]; then + docker pull --platform linux/amd64 "$database_image" +fi +docker image inspect --format '{{.Id}}' "$database_image" > "$stage/database.id" +docker run --rm --network none --entrypoint postgres "$(cat "$stage/database.id")" --version \ + | python3 -c 'import sys; value=sys.stdin.read(); assert value.startswith("postgres (PostgreSQL) 16."), "Distribution requires PostgreSQL 16"' +for name in core web runtime database; do + image="$(cat "$stage/$name.id")" + python3 scripts/core-distribution-manifest.py verify-image "$image" + docker image save --output "$bundle/images/$name.tar" "$image" +done + +# OCI manifest digests differ from Docker config IDs. Import the exact offline +# archive through the pinned runtime in a temporary cache; no VM is started. +mkdir -m 0700 "$stage/msb-cache" +msb=(docker run --rm --network none --user "$(id -u):$(id -g)" \ + --mount "type=bind,src=$stage/msb-cache,dst=/cache" \ + --mount "type=bind,src=$bundle/images/runtime.tar,dst=/runtime.tar,readonly" \ + --env MSB_HOME=/cache --env MSB_BACKEND=local --env MSB_PATH=/opt/microsandbox/msb \ + --env MSB_LIBKRUNFW_PATH=/opt/microsandbox/libkrunfw.so.5.6.1 \ + --entrypoint /opt/microsandbox/msb "$core_image") +"${msb[@]}" image load --input /runtime.tar --tag parsar-core-runtime:distribution --quiet +"${msb[@]}" image inspect parsar-core-runtime:distribution --format json > "$stage/runtime-inspect.json" +python3 scripts/core-distribution-manifest.py manifest "$bundle" "$stage" "$revision" "$source_tree" +require_clean_source +if [[ "$(git -C "$repo_root" rev-parse HEAD)" != "$revision" ]]; then + printf 'Source changed during distribution build\n' >&2 + exit 1 +fi +python3 scripts/core-distribution-manifest.py archive "$bundle" "$source_epoch" +archive_name="$(basename "$bundle").tar.gz" +if [[ -e "$output_dir/$(basename "$bundle")" || -e "$output_dir/$archive_name" ]]; then + printf 'A distribution already exists for this revision; choose a fresh output directory\n' >&2 + exit 1 +fi +mv "$stage/$archive_name" "$stage/$archive_name.sha256" "$bundle" "$output_dir/" +printf 'Core distribution: %s/%s\n' "$output_dir" "$archive_name" diff --git a/scripts/core-distribution-manifest.py b/scripts/core-distribution-manifest.py new file mode 100644 index 000000000..b8c6cd295 --- /dev/null +++ b/scripts/core-distribution-manifest.py @@ -0,0 +1,130 @@ +#!/usr/bin/env python3 +"""Verify distribution inputs and write the offline bundle's public metadata.""" + +import gzip +import hashlib +import json +import pathlib +import re +import subprocess +import sys +import tarfile + + +RUNTIME_ARCHIVE_SHA256 = "47c223e3ef5298abf05f47ed9f87981106e400d99bb3f1d042d4d6881346b18b" +DIGEST = re.compile(r"sha256:[0-9a-f]{64}\Z") + + +def sha256(path): + digest = hashlib.sha256() + with pathlib.Path(path).open("rb") as stream: + for block in iter(lambda: stream.read(1024 * 1024), b""): + digest.update(block) + return digest.hexdigest() + + +def verify_image(image): + if not DIGEST.fullmatch(image): + raise ValueError("Distribution image inputs must be immutable sha256 image IDs") + details = json.loads(subprocess.check_output(["docker", "image", "inspect", image], text=True))[0] + if details["Id"] != image or details["Os"] != "linux" or details["Architecture"] != "amd64": + raise ValueError("Distribution images must be the selected Linux amd64 image") + return details + + +def verify_runtime(image, daemon, helpers, source): + details = verify_image(image) + helpers, source = pathlib.Path(helpers), pathlib.Path(source) + files = {"/usr/local/bin/parsar-daemon": pathlib.Path(daemon)} + for name in ("agents-api-codex-directory", "agents-api-codex-write", "agents-api-workspace-export"): + files["/usr/local/bin/" + name] = helpers / name + files["/usr/local/bin/agents-api-runtime-initialize"] = source / "services/agents-api/deploy/runtime/initialize.py" + files["/usr/local/bin/agents-api-tool-root"] = source / "services/agents-api/deploy/runtime/tool-root.py" + environment = dict(value.split("=", 1) for value in details["Config"]["Env"] if "=" in value) + if "PARSAR_CODEX_BIN" in environment: + files["/etc/codex/requirements.toml"] = source / "services/agents-api/deploy/codex/requirements.toml" + files["/etc/codex/tool-env.py"] = source / "services/agents-api/deploy/codex/tool-env.py" + if "PARSAR_CLAUDE_SDK_ENTRYPOINT" in environment: + files["/usr/local/bin/agents-api-claude-shell-prefix"] = source / "services/agents-api/deploy/claude/shell-prefix.py" + if "PARSAR_MCODE_BIN" in environment: + for name in ("launch.mjs", "bridge.mjs", "check.mjs", "tool-executor.mjs", "subagent-snapshot.mjs", "source.json"): + files["/opt/mcode-harness/" + name] = source / "packages/mcode-harness" / name + output = subprocess.check_output( + ["docker", "run", "--rm", "--network", "none", "--entrypoint", "sha256sum", image, *files], text=True + ) + actual = dict(reversed(line.split(None, 1)) for line in output.splitlines()) + for guest_path, local in files.items(): + if actual.get(guest_path) != sha256(local): + raise ValueError("Runtime image does not match the committed build: " + guest_path) + + +def extract_runtime(archive, destination): + if sha256(archive) != RUNTIME_ARCHIVE_SHA256: + raise ValueError("microsandbox v0.7.2 release checksum mismatch") + destination = pathlib.Path(destination) + destination.mkdir(parents=True, exist_ok=True) + with tarfile.open(archive, "r:gz") as bundle: + for name in ("msb", "libkrunfw.so.5.6.1"): + members = [member for member in bundle.getmembers() if pathlib.PurePosixPath(member.name).name == name and member.isfile()] + if len(members) != 1: + raise ValueError("Release must contain exactly one regular " + name) + with bundle.extractfile(members[0]) as stream, (destination / name).open("wb") as output: + for block in iter(lambda: stream.read(1024 * 1024), b""): + output.write(block) + (destination / name).chmod(0o555) + + +def manifest(bundle, stage, revision, source_tree): + bundle, stage = pathlib.Path(bundle), pathlib.Path(stage) + inspected = json.loads((stage / "runtime-inspect.json").read_text()) + digest = inspected.get("digest", "") + if not isinstance(digest, str) or not DIGEST.fullmatch(digest): + raise ValueError("msb did not return an immutable OCI manifest digest") + if inspected.get("architecture") != "amd64" or inspected.get("os") != "linux": + raise ValueError("msb imported an unexpected Runtime platform") + images = {name: (stage / (name + ".id")).read_text().strip() for name in ("core", "web", "runtime", "database")} + if any(not DIGEST.fullmatch(image) for image in images.values()): + raise ValueError("Missing immutable distribution image identity") + metadata = { + "source_commit": revision, + "source_tree": source_tree, + "platform": "linux/amd64", + "images": images, + "runtime_ref": "parsar-core-runtime@" + digest, + "microsandbox": { + "version": "0.7.2", + "runtime_sha256": sha256(stage / "core/microsandbox/msb"), + "firmware_sha256": sha256(stage / "core/microsandbox/libkrunfw.so.5.6.1"), + }, + } + (bundle / "manifest.json").write_text(json.dumps(metadata, indent=2, sort_keys=True) + "\n") + members = sorted(path for path in bundle.rglob("*") if path.is_file()) + (bundle / "SHA256SUMS").write_text("".join(sha256(path) + " " + path.relative_to(bundle).as_posix() + "\n" for path in members)) + + +def archive(bundle, epoch): + bundle = pathlib.Path(bundle) + output = bundle.with_name(bundle.name + ".tar.gz") + with output.open("wb") as raw, gzip.GzipFile(filename="", mode="wb", fileobj=raw, mtime=0) as compressed: + with tarfile.open(fileobj=compressed, mode="w", format=tarfile.PAX_FORMAT) as tar: + for path in sorted(bundle.rglob("*")): + if not path.is_file(): + continue + relative = path.relative_to(bundle) + info = tarfile.TarInfo(bundle.name + "/" + relative.as_posix()) + info.size = path.stat().st_size + info.mode = 0o755 if relative.as_posix() == "install.sh" else 0o644 + info.mtime = int(epoch) + with path.open("rb") as stream: + tar.addfile(info, stream) + output.with_name(output.name + ".sha256").write_text(sha256(output) + " " + output.name + "\n") + + +if __name__ == "__main__": + commands = {"extract-runtime": extract_runtime, "verify-runtime": verify_runtime, "verify-image": verify_image, "manifest": manifest, "archive": archive} + try: + commands[sys.argv[1]](*sys.argv[2:]) + except (KeyError, TypeError): + sys.exit("Usage: core-distribution-manifest.py extract-runtime|verify-runtime|verify-image|manifest|archive ARGS...") + except (OSError, ValueError, subprocess.CalledProcessError) as error: + sys.exit(str(error)) diff --git a/scripts/core-distribution-manifest.test.py b/scripts/core-distribution-manifest.test.py new file mode 100644 index 000000000..3468ecabd --- /dev/null +++ b/scripts/core-distribution-manifest.test.py @@ -0,0 +1,83 @@ +"""Regression checks for offline distribution identity and archive integrity.""" + +import hashlib +import importlib.util +import json +import pathlib +import tarfile +import tempfile +import unittest + + +spec = importlib.util.spec_from_file_location("distribution", pathlib.Path(__file__).with_name("core-distribution-manifest.py")) +distribution = importlib.util.module_from_spec(spec) +spec.loader.exec_module(distribution) + + +class DistributionTests(unittest.TestCase): + def setUp(self): + self.temporary = tempfile.TemporaryDirectory() + self.addCleanup(self.temporary.cleanup) + self.stage = pathlib.Path(self.temporary.name) + self.bundle = self.stage / "parsar-core-test-linux-amd64" + self.bundle.mkdir() + (self.bundle / "install.sh").write_text("#!/bin/sh\nexit 0\n") + (self.bundle / "source.tar.gz").write_bytes(b"source archive") + runtime = self.stage / "core/microsandbox" + runtime.mkdir(parents=True) + (runtime / "msb").write_bytes(b"runtime") + (runtime / "libkrunfw.so.5.6.1").write_bytes(b"firmware") + for number, name in enumerate(("core", "web", "runtime", "database"), 1): + (self.stage / (name + ".id")).write_text("sha256:" + str(number) * 64 + "\n") + self.inspection = {"digest": "sha256:" + "a" * 64, "architecture": "amd64", "os": "linux"} + self.write_inspection() + + def write_inspection(self): + (self.stage / "runtime-inspect.json").write_text(json.dumps(self.inspection)) + + def test_oci_manifest_identity_is_distinct_from_docker_config_identity(self): + distribution.manifest(self.bundle, self.stage, "commit", "tree") + metadata = json.loads((self.bundle / "manifest.json").read_text()) + self.assertEqual(metadata["runtime_ref"], "parsar-core-runtime@sha256:" + "a" * 64) + self.assertEqual(metadata["images"]["runtime"], "sha256:" + "3" * 64) + self.assertEqual(metadata["microsandbox"]["runtime_sha256"], hashlib.sha256(b"runtime").hexdigest()) + for line in (self.bundle / "SHA256SUMS").read_text().splitlines(): + digest, name = line.split(" ", 1) + self.assertEqual(digest, distribution.sha256(self.bundle / name)) + + def test_missing_manifest_digest_does_not_fall_back_to_config_id(self): + self.inspection.pop("digest") + self.inspection["config"] = {"digest": "sha256:" + "3" * 64} + self.write_inspection() + with self.assertRaisesRegex(ValueError, "manifest digest"): + distribution.manifest(self.bundle, self.stage, "commit", "tree") + + def test_wrong_guest_platform_rejected(self): + self.inspection["architecture"] = "arm64" + self.write_inspection() + with self.assertRaisesRegex(ValueError, "platform"): + distribution.manifest(self.bundle, self.stage, "commit", "tree") + + def test_archive_reproducible_and_installer_executable(self): + distribution.manifest(self.bundle, self.stage, "commit", "tree") + distribution.archive(self.bundle, "1700000000") + archive = self.bundle.with_name(self.bundle.name + ".tar.gz") + first = archive.read_bytes() + distribution.archive(self.bundle, "1700000000") + self.assertEqual(first, archive.read_bytes()) + self.assertEqual(archive.with_name(archive.name + ".sha256").read_text(), distribution.sha256(archive) + " " + archive.name + "\n") + with tarfile.open(archive) as contents: + self.assertEqual(contents.getmember(self.bundle.name + "/install.sh").mode, 0o755) + self.assertEqual(contents.getmember(self.bundle.name + "/manifest.json").mode, 0o644) + + def test_bad_upstream_checksum_does_not_extract(self): + archive = self.stage / "untrusted.tar.gz" + archive.write_bytes(b"not the pinned release") + destination = self.stage / "extracted" + with self.assertRaisesRegex(ValueError, "checksum mismatch"): + distribution.extract_runtime(archive, destination) + self.assertFalse(destination.exists()) + + +if __name__ == "__main__": + unittest.main() From 96b49b30d4265c10d23d59f72999e25c8a5e8671 Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 14:53:16 +0800 Subject: [PATCH 05/17] Test installer state preservation and deployment boundaries --- deploy/install/test_install.py | 321 +++++++++++++++++++++++++++++++++ 1 file changed, 321 insertions(+) create mode 100644 deploy/install/test_install.py diff --git a/deploy/install/test_install.py b/deploy/install/test_install.py new file mode 100644 index 000000000..c265786fc --- /dev/null +++ b/deploy/install/test_install.py @@ -0,0 +1,321 @@ +#!/usr/bin/env python3 +"""Check installation state and deployment boundaries without starting Docker.""" + +import base64 +import contextlib +import hashlib +import io +import json +import os +from pathlib import Path +import runpy +import shutil +import stat +import subprocess +import sys +import tempfile +from types import SimpleNamespace +import unittest +from unittest import mock + +import install + + +class InstallerTests(unittest.TestCase): + def setUp(self): + temporary_root = Path.home() / ".parsar/tests/install" + temporary_root.mkdir(parents=True, exist_ok=True) + self.temporary = tempfile.TemporaryDirectory(prefix="unit-", dir=temporary_root) + self.addCleanup(self.temporary.cleanup) + self.work = Path(self.temporary.name).resolve() + self.root = self.work / "deployment" + self.manifest = { + "source_commit": "a" * 40, + "images": {name: "sha256:" + digit * 64 for name, digit in ( + ("core", "1"), ("runtime", "2"), ("database", "3"), ("web", "4"))}, + "runtime_ref": "localhost/parsar-runtime:test-install", + "microsandbox": {"runtime_sha256": "5" * 64, "firmware_sha256": "6" * 64}, + } + self.ports = self.patched(mock.patch.object(install, "free_port")) + self.device_probes = [] + original_stat = os.stat + + def controlled_device_stat(name, *args, **kwargs): + if os.fspath(name) in ("/dev/kvm", "/var/run/docker.sock"): + self.device_probes.append(os.fspath(name)) + return SimpleNamespace(st_gid=1234, st_mode=stat.S_IFCHR | 0o660) + return original_stat(name, *args, **kwargs) + + self.patched(mock.patch.object(install.os, "stat", side_effect=controlled_device_stat)) + self.patched(mock.patch.object(install.os, "getuid", return_value=1000)) + self.patched(mock.patch.object(install.os, "getgid", return_value=1000)) + + def patched(self, patcher): + result = patcher.start() + self.addCleanup(patcher.stop) + return result + + def args(self, *values): + return install.arguments(["--install-dir", str(self.root), *values]) + + def initialize(self, *values): + return install.initialize(self.root, self.args(*values), self.manifest) + + def document(self, name): + return json.loads((self.root / name).read_text()) + + def snapshot(self): + return { + str(path.relative_to(self.root)): (stat.S_IMODE(path.stat().st_mode), + path.read_bytes() if path.is_file() else None) + for path in [self.root, *self.root.rglob("*")] + } + + def caller_file(self, contents="synthetic-existing-core-token", mode=0o600): + path = self.work / "existing-core.key" + path.write_text(contents) + path.chmod(mode) + return path + + def bundle(self): + bundle = self.work / "bundle" + (bundle / "images").mkdir(parents=True) + (bundle / "runtime").mkdir() + (bundle / "manifest.json").write_text(json.dumps(self.manifest)) + (bundle / "runtime/seccomp.json").write_text('{"defaultAction":"SCMP_ACT_ERRNO"}') + for name in ("install.py", "configuration.py", "install.sh"): + shutil.copyfile(Path(__file__).with_name(name), bundle / name) + for name in self.manifest["images"]: + (bundle / "images" / (name + ".tar")).write_bytes(("synthetic " + name).encode()) + self.write_checksums(bundle) + return bundle + + @staticmethod + def write_checksums(bundle): + files = sorted(path for path in bundle.rglob("*") if path.is_file() and path.name != "SHA256SUMS") + (bundle / "SHA256SUMS").write_text("".join( + hashlib.sha256(path.read_bytes()).hexdigest() + " " + str(path.relative_to(bundle)) + "\n" + for path in files)) + + def test_repeat_installation_preserves_execution_identity_and_all_secrets(self): + first = self.initialize() + keys = self.document("config/keys.json") + caller = (self.root / "config/caller.key").read_bytes() + encryption = (self.root / "config/credential.key").read_text() + self.assertEqual(hashlib.sha256(caller).hexdigest(), keys[0]["token_sha256"]) + self.assertEqual(len(base64.b64decode(encryption, validate=True)), 32) + self.assertEqual(first["installation_id"], self.document("config/managed-runtimes.json")["installation_id"]) + before = self.snapshot() + self.ports.reset_mock() + # Re-running against already listening services must not reserve their ports. + self.ports.side_effect = AssertionError("repeat installation re-probed an occupied port") + second = self.initialize() + self.assertEqual(first, second) + self.assertEqual(before, self.snapshot()) + self.assertEqual(keys, self.document("config/keys.json")) + + def test_configuration_changes_refuse_without_mutating_existing_deployment(self): + self.initialize() + before = self.snapshot() + for flags in (("--core-only",), ("--provider", "docker"), ("--core-port", "8092"), ("--web-port", "8081")): + with self.subTest(flags=flags), self.assertRaises(install.InstallError): + self.initialize(*flags) + self.assertEqual(before, self.snapshot()) + changed = dict(self.manifest, source_commit="b" * 40) + with self.assertRaises(install.InstallError): + install.initialize(self.root, self.args(), changed) + self.assertEqual(before, self.snapshot()) + + def test_default_microsandbox_exposes_only_kvm_to_core(self): + state = self.initialize() + self.assertEqual(state["provider"], "microsandbox") + managed = self.document("config/managed-runtimes.json") + self.assertIn("microsandbox", managed) + self.assertNotIn("docker", managed) + self.assertEqual(managed["microsandbox"]["network"]["default_ingress"], "deny") + self.assertEqual(managed["microsandbox"]["network"]["default_egress"], "deny") + self.assertEqual(managed["microsandbox"]["image"], self.manifest["runtime_ref"]) + services = self.document("compose.json")["services"] + self.assertEqual(set(services), {"core", "database", "migrate", "web"}) + self.assertEqual(services["core"]["devices"], ["/dev/kvm:/dev/kvm"]) + self.assertIn("1234", services["core"]["group_add"]) + for name, service in services.items(): + self.assertFalse(service.get("privileged", False)) + self.assertNotIn("docker.sock", json.dumps(service.get("volumes", []))) + if name != "core": + self.assertNotIn("devices", service) + self.assertNotIn("ports", services["database"]) + for name in ("core", "migrate", "web"): + self.assertEqual(services[name]["user"], "1000:1000") + self.assertTrue(services[name]["read_only"]) + self.assertIn("no-new-privileges:true", services[name]["security_opt"]) + for name in ("core", "web"): + self.assertTrue(all(port.startswith("127.0.0.1:") for port in services[name]["ports"])) + self.assertEqual(services["core"]["depends_on"]["migrate"]["condition"], "service_completed_successfully") + + def test_docker_provider_socket_and_runtime_network_belong_only_to_core(self): + state = self.initialize("--provider", "docker", "--core-only") + managed = self.document("config/managed-runtimes.json") + self.assertNotIn("microsandbox", managed) + self.assertEqual(managed["docker"]["host"], "unix:///var/run/docker.sock") + self.assertEqual(managed["docker"]["image"], self.manifest["images"]["runtime"]) + self.assertTrue(managed["docker"]["nested_sandbox"]) + compose = self.document("compose.json") + services = compose["services"] + self.assertNotIn("web", services) + self.assertFalse((self.root / "config/console.password").exists()) + self.assertEqual(compose["networks"]["runtime"]["name"], managed["docker"]["network"]) + self.assertEqual(managed["installation_id"], state["installation_id"]) + for name, service in services.items(): + self.assertNotIn("devices", service) + sockets = [mount for mount in service.get("volumes", []) + if isinstance(mount, dict) and mount["target"] == "/var/run/docker.sock"] + self.assertEqual(len(sockets), 1 if name == "core" else 0) + self.assertEqual("runtime" in service.get("networks", []), name == "core") + self.assertNotIn("/dev/kvm", self.device_probes) + + def test_web_only_uses_existing_local_core_without_database_or_provider(self): + source = self.caller_file() + flags = ("--web-only", "--core-url", "http://127.0.0.1:9091", "--core-token-file", str(source)) + state = self.initialize(*flags) + self.assertEqual(self.device_probes, []) + self.assertEqual(self.ports.call_args_list, [mock.call(8080)]) + compose = self.document("compose.json") + self.assertEqual(set(compose["services"]), {"web"}) + self.assertNotIn("volumes", compose) + web = compose["services"]["web"] + self.assertEqual(web["network_mode"], "host") + self.assertEqual(web["environment"]["CORE_CONSOLE_ADDR"], "127.0.0.1:8080") + self.assertEqual(web["environment"]["CORE_CONSOLE_UPSTREAM"], "http://127.0.0.1:9091") + self.assertNotIn("ports", web) + self.assertNotIn("devices", web) + self.assertEqual({path.name for path in (self.root / "config").iterdir()}, {"caller.key", "console.password"}) + self.assertEqual((self.root / "config/caller.key").read_bytes(), source.read_bytes()) + before = self.snapshot() + self.assertEqual(state, self.initialize(*flags)) + self.assertEqual(before, self.snapshot()) + + def test_private_files_are_owner_only_even_under_permissive_umask(self): + previous = os.umask(0) + try: + self.initialize() + finally: + os.umask(previous) + for path in [self.root, *self.root.rglob("*")]: + with self.subTest(path=path.relative_to(self.root)): + expected = 0o700 if path.is_dir() else 0o600 + self.assertEqual(stat.S_IMODE(path.stat().st_mode), expected) + self.assertNotEqual((self.root / "config/caller.key").read_bytes(), + (self.root / "config/console.password").read_bytes()) + + def test_web_only_rejects_exposed_or_malformed_caller_files(self): + for contents, mode in (("synthetic-token", 0o644), ("", 0o600), ("two tokens", 0o600), + ("token\x00", 0o600), ("x" * 4097, 0o600)): + with self.subTest(contents=contents, mode=mode): + source = self.caller_file(contents, mode) + with self.assertRaises(install.InstallError): + self.initialize("--web-only", "--core-url", "http://localhost:8091", + "--core-token-file", str(source)) + self.assertFalse(self.root.exists(), "invalid input left a non-retryable partial deployment") + + def test_web_only_rejects_directory_or_symlink_as_caller_file(self): + source = self.caller_file() + link = self.work / "linked.key" + link.symlink_to(source) + private_directory = self.work / "directory.key" + private_directory.mkdir(mode=0o700) + for path in (link, private_directory): + with self.subTest(path=path.name), self.assertRaises(install.InstallError): + self.initialize("--web-only", "--core-url", "http://localhost:8091", + "--core-token-file", str(path)) + self.assertFalse(self.root.exists()) + + def test_bundle_verifies_transferred_bytes_before_trusting_manifest(self): + bundle = self.bundle() + self.assertEqual(install.verify_bundle(bundle), self.manifest) + for name in ("images/core.tar", "images/runtime.tar", "manifest.json"): + with self.subTest(name=name): + path = bundle / name + original = path.read_bytes() + path.write_bytes(original + b"modified") + with self.assertRaises(install.InstallError): + install.verify_bundle(bundle) + path.write_bytes(original) + path.unlink() + with self.assertRaises(install.InstallError): + install.verify_bundle(bundle) + path.write_bytes(original) + + def test_bundle_rejects_omitted_or_duplicate_checksum_entries(self): + bundle = self.bundle() + checksums = bundle / "SHA256SUMS" + original = checksums.read_text() + checksums.write_text("".join(line + "\n" for line in original.splitlines() + if not line.endswith(" images/runtime.tar"))) + with self.assertRaises(install.InstallError): + install.verify_bundle(bundle) + checksums.write_text(original + original.splitlines()[0] + "\n") + with self.assertRaises(install.InstallError): + install.verify_bundle(bundle) + + def test_bundle_rejects_paths_outside_distribution(self): + bundle = self.bundle() + outside = self.caller_file() + checksums = bundle / "SHA256SUMS" + checksums.write_text(install.digest(outside) + " ../existing-core.key\n") + with self.assertRaises(install.InstallError): + install.verify_bundle(bundle) + + def test_main_web_only_never_imports_runtime_or_leaks_caller_password(self): + source = self.caller_file() + bundle = self.bundle() + calls = [] + + def external_command(args, **_kwargs): + calls.append(args) + return SimpleNamespace(stdout="", returncode=0) + + output = io.StringIO() + with mock.patch.object(install, "__file__", str(bundle / "install.py")), \ + mock.patch.object(install.platform, "system", return_value="Linux"), \ + mock.patch.object(install.platform, "machine", return_value="x86_64"), \ + mock.patch.object(install, "run", side_effect=external_command), \ + mock.patch.object(install, "wait_http", return_value=True) as health, \ + contextlib.redirect_stdout(output), contextlib.redirect_stderr(output): + install.main(["--install-dir", str(self.root), "--web-only", "--core-url", + "http://127.0.0.1:9091", "--core-token-file", str(source)]) + self.assertEqual(self.device_probes, []) + imports = [call for call in calls if call[:2] == ["docker", "load"]] + self.assertEqual(imports, [["docker", "load", "--input", str(bundle / "images/web.tar")]]) + self.assertFalse(any(call[:2] == ["docker", "run"] for call in calls)) + self.assertEqual([call.args[0] for call in health.call_args_list], ["http://127.0.0.1:8080/v1/agents"]) + for path in (self.root / "config").iterdir(): + self.assertNotIn(path.read_text(), output.getvalue()) + + def test_cli_failure_does_not_print_external_command_secrets(self): + secret = "synthetic-sensitive-command-value" + failure = subprocess.CalledProcessError(1, ["docker", secret], output=secret, stderr=secret) + output = io.StringIO() + with mock.patch.object(sys, "argv", [install.__file__, "--install-dir", str(self.root)]), \ + mock.patch.object(install.platform, "system", return_value="Linux"), \ + mock.patch.object(install.platform, "machine", return_value="x86_64"), \ + mock.patch.object(subprocess, "run", side_effect=failure), \ + contextlib.redirect_stdout(output), contextlib.redirect_stderr(output), \ + self.assertRaises(SystemExit) as raised: + runpy.run_path(install.__file__, run_name="__main__") + self.assertEqual(raised.exception.code, 1) + self.assertNotIn(secret, output.getvalue()) + self.assertFalse(self.root.exists()) + + def test_core_connection_rejects_remote_cleartext_and_embedded_credentials(self): + source = self.caller_file() + for url in ("http://remote.example:8091", "https://user:synthetic-secret@core.example", + "https://core.example/?token=synthetic-secret"): + output = io.StringIO() + with self.subTest(url=url), contextlib.redirect_stderr(output), self.assertRaises(SystemExit): + self.args("--web-only", "--core-url", url, "--core-token-file", str(source)) + self.assertNotIn("synthetic-secret", output.getvalue()) + + +if __name__ == "__main__": + unittest.main() From d93407221ea806755c80387f1406dc2be8e2ff76 Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 14:53:50 +0800 Subject: [PATCH 06/17] Install matched Core and console with managed sandbox defaults --- CONTRIBUTING.md | 57 ++++++ Makefile | 13 +- README.md | 101 ++++++----- deploy/install/configuration.py | 98 ++++++++++ deploy/install/install.py | 278 +++++++++++++++++++++++++++++ deploy/install/install.sh | 3 + docs/getting-started/README.md | 20 +++ docs/getting-started/install.md | 122 +++++++++++++ docs/getting-started/operations.md | 107 +++++++++++ docs/getting-started/quickstart.md | 103 +++++++++++ scripts/build-core-distribution.sh | 3 +- 11 files changed, 857 insertions(+), 48 deletions(-) create mode 100644 deploy/install/configuration.py create mode 100644 deploy/install/install.py create mode 100755 deploy/install/install.sh create mode 100644 docs/getting-started/README.md create mode 100644 docs/getting-started/install.md create mode 100644 docs/getting-started/operations.md create mode 100644 docs/getting-started/quickstart.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 95bf373b1..9d295d3af 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1306,6 +1306,63 @@ package with a fresh database, extracted binaries, loaded image and real public workflow. Keep model/operator credentials external and Provider ownership stable across upgrades. This is the same managed Runtime, not user-managed enrollment. +#### Matched Core and console distribution + +The installer milestone packages Core and the unchanged Web console together, +with independent `--core-only` and `--web-only` modes. `site/` is the public static +landing, separate from `apps/web`; it must not create an onboarding prerequisite, +call a model, or claim complete protocol compatibility. Operator installation, +optional API examples and service diagnostics live in `docs/getting-started/`. + +`make build-core-distribution` builds from clean committed source and reuses the +existing API, Runtime, SDK, helper and Web builders. Artifacts record source and +immutable image identities, the actual Runtime manifest digest, checksums and +microsandbox runtime/firmware hashes. Release generation is not publication or +qualification. A release must be tested from fresh extraction with real models; +no synthetic result may substitute for native execution acceptance. + +The first installer targets a trusted Linux amd64 Docker host. It installs a +private dedicated PostgreSQL service and separate Core and console services. +Default sandbox placement is microsandbox; `--provider docker` selects the +existing Docker provider. Missing KVM fails without changing that choice. +The distribution Core image contains the native glibc helper and pinned msb +runtime/firmware, with only KVM device access for microsandbox or the canonical +Docker socket for Docker. This does not put a harness or model loop in Core. +The basic distroless API image and binary builds remain independent artifacts. + +One Runtime image contains the existing daemon, shared helpers and three native +harness packages. Their differences remain in the adapters. Core keeps exclusive +ownership of Session allocation, initialization, cancellation, snapshots and +cleanup. The installer imports images and prepares running conditions; it never +creates an execution Session or supplies a model credential. Applications use the +existing write-only model execution extension, with the installation's persistent +credential encryption key. Provider identity/backend namespace and native history +must not change on a repeated install. + +`services/core-console` serves the existing production Web build and forwards only +public `/v1` requests to one configured Core. It uses the standard Go reverse +proxy with streaming/cancellation, a separate operator password, fixed origin and +cross-site checks. Only the server reads the Core bearer. It does not implement +product identity, resource semantics, Runtime discovery or an execution loop. +The console has neither KVM nor Docker authority; its static root contains no +secrets. Installation exposes only loopback API/console ports. Remote exposure +requires an operator-configured HTTPS/access boundary. Web-only mode can connect +to a loopback existing Core on the same Linux host or a remote HTTPS Core. + +Installation state and secrets live in a private directory under `~/.parsar/` by +default. No credential enters build arguments, image layers, browser bundles or +diagnostic output. Compose configuration is confidential. The generated database, +caller/tenant/provider identities and encryption key survive reruns; automatic +revision replacement and provider migration are outside this initial installer. +Do not delete data or issue broad container/volume pruning as recovery. + +`make check-distribution` covers the production proxy, installation rules and +release metadata. Real bundle validation covers default/provider selection, +component modes, existing Web connection, public native execution and restart +retention. Diagnostics report observed service health, not fabricated model or +complete environment readiness. Runtime observations are Core-owned; do not add +a duplicate monitoring/lifecycle framework to installation or the public landing. + #### Current implementation The constraints below describe existing code, not requirements to preserve legacy diff --git a/Makefile b/Makefile index f9142df47..6c05a5456 100644 --- a/Makefile +++ b/Makefile @@ -8,7 +8,7 @@ SWAG_VERSION ?= v1.16.4 help: @printf '%s\n' 'make build-agents-api Build standalone Core commands' 'make build-daemon Build the execution daemon' 'make check Run Core, persistence and runtime checks' 'See README.md for runtime prerequisites and deployment.' -check: check-database check-sqlc check-go check-microsandbox-provider check-agents-api check-claude-sdk check-web check-mcode-harness check-agents-executor +check: check-distribution check-database check-sqlc check-go check-microsandbox-provider check-agents-api check-claude-sdk check-web check-mcode-harness check-agents-executor @printf 'Parsar Core checks passed.\n' check-database: @@ -111,3 +111,14 @@ check-microsandbox-provider: else \ printf 'Skipping the Linux-only microsandbox SDK helper tests; the full Linux gate is required before release.\n'; \ fi + +.PHONY: check-distribution build-core-distribution +check-distribution: + go test ./services/core-console -count=1 + PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s deploy/install -p 'test_*.py' + PYTHONDONTWRITEBYTECODE=1 python3 scripts/core-distribution-manifest.test.py + bash -n deploy/install/install.sh scripts/build-core-console.sh scripts/build-core-distribution.sh + ./scripts/build-core-console.sh + +build-core-distribution: + ./scripts/build-core-distribution.sh diff --git a/README.md b/README.md index cc80ee81a..69456bef3 100644 --- a/README.md +++ b/README.md @@ -1,37 +1,47 @@ # Parsar Core -Standalone Agent API Core and its execution runtimes, copied from -[Parsar](https://github.com/MiniMax-AI-Dev/parsar) at -[`72ab4d37`](https://github.com/MiniMax-AI-Dev/parsar/commit/72ab4d37d49245f15b63d34f5741780e540bcec0). -The source repository retains both its product and its existing Core copy. - -This repository contains the API service, PostgreSQL migrations, pinned public -protocol, execution daemon, the Docker provider, native Harness adapters, -runtime image builders, Go and TypeScript client libraries, the standalone Core -Web console, tests and operator documentation. It does not contain the Parsar -product application, product backend, product database, business CLI or product -deployment stack. - -V1 user-managed deployments colocate our daemon, selected harness, tools and -`/workspace`. Core manages Docker only; users provision, renew and destroy E2B -through the official SDK. The returned `remote_url` uses our private daemon -transport, not stock `exec-server`. See the -[Runtime enrollment guide](services/agents-api/README.md#user-managed-runtime-enrollment) -for harness enablement and the [qualification record](contracts/agents-api/user-managed-runtime-v1.md) -for tested deployments and remaining limits. +**Open-source Agents API infrastructure, with your choice of native harness.** + +Run Codex, Claude Code and MiniMax Code behind one execution API. Parsar Core +owns Sessions, environments, files, credentials and execution history; each +native harness keeps its own model and tool loop. Core runs independently of the +Parsar product. + +Core and its Web console ship together. The installer prepares microsandbox by +default; use `--provider docker` for Docker. Core provisions the colocated +Runtime when a Session needs it. Model credentials are supplied through the +existing write-only API extension, not during installation. ## Start here -- [API setup, authentication and execution](services/agents-api/README.md) -- [Standalone containers](services/agents-api/CONTAINER.md) -- [Docker Runtime](services/agents-api/deploy/codex/README.md) -- [Protocol coverage and known gaps](contracts/agents-api/README.md) -- [Harness selection](contracts/agents-api/harness-selection.md) -- [Core Web overview](docs/web/README.md) -- [Core Web 中文说明](docs/web/README.zh-CN.md) -- [Connect Core Web to Core](docs/web/core-connection.md) -- [Contributor rules](CONTRIBUTING.md) -- [Copy provenance and validation](provenance/README.md) +- [Install Core and Web](docs/getting-started/install.md) +- [Make your first API request](docs/getting-started/quickstart.md) +- [Service health, data and operations](docs/getting-started/operations.md) +- [Protocol coverage and native differences](contracts/agents-api/README.md) +- [Add or select a harness](contracts/agents-api/harness-selection.md) +- [Public landing page source](site/index.html) + +After verifying and extracting a matching Linux amd64 distribution: + +```sh +./install.sh # Core + Web, microsandbox +./install.sh --provider docker # Core + Web, Docker +./install.sh --core-only # Core without the console +``` + +Web-only installation connects the unchanged console to an existing Core; see the +installation guide for its URL and private credential-file options. Installation +never creates a sample Session or calls a model. API examples are optional. + +The protocol baseline is `openai-python` 3.13.0 and `agents=v1`. Harness selection, +model execution configuration and our daemon transport are documented differences. +A passing workflow does not establish complete OpenAI Agents API compatibility. + +## Develop and build + +The repository includes the API, its independent PostgreSQL migrations, daemon, +Runtime/provider adapters, clients, Web console and distribution tools. It has no +Parsar product service, product database or business-user dependency. ```sh make build-agents-api @@ -39,23 +49,22 @@ make build-daemon pnpm dev:web ``` -These builds require the Go version pinned in `go.mod`. Output goes under -`~/.parsar/build/`; no product checkout, frontend or product database is needed. -Provision a dedicated Core PostgreSQL database and caller credentials using the -operator guide before starting the service. Native execution also needs the -appropriate Runtime image and provider configuration. - -Core Web lives in `apps/web` and talks only to the public `/v1/agents/**` -HTTP/SSE contract through the TypeScript implementation in -`packages/agents-client`. The Go client remains in -`packages/agents-client/v1`; both clients live next to the contract they consume -without coupling browser state to Core execution internals. - -The copied Go module/import paths, executable names and `PARSAR_*` environment -variables intentionally retain their existing names. They resolve to source in -this checkout, not a dependency on the Parsar product repository. This extraction -does not rename protocols or change execution behavior. Third-party native -sources and packages remain pinned dependencies, not vendored binaries. +Use the toolchain pinned in `go.mod`, Node 22 and pnpm 10.30.3. Build output goes +under `~/.parsar/build/`. For advanced deployment, see the +[service guide](services/agents-api/README.md), +[Docker Runtime](services/agents-api/deploy/codex/README.md), +[microsandbox provider](services/agents-api/deploy/microsandbox/README.md), and +[Web development guide](docs/web/README.md). + +Core-managed and user-managed environments reuse the colocated daemon, native +harness, tools and workspace. E2B uses caller-managed provisioning through the +official SDK; the returned `remote_url` connects our daemon, not `exec-server`. +See the [Runtime enrollment guide](services/agents-api/README.md#user-managed-runtime-enrollment). + +Read [CONTRIBUTING.md](CONTRIBUTING.md) before developing. Historical source-copy +provenance is retained in [provenance/README.md](provenance/README.md). Existing Go +import paths resolve inside this repository and do not require the product repo. +Third-party native packages remain pinned build dependencies. ## Validate diff --git a/deploy/install/configuration.py b/deploy/install/configuration.py new file mode 100644 index 000000000..623410e1d --- /dev/null +++ b/deploy/install/configuration.py @@ -0,0 +1,98 @@ +"""Deployment files for the existing Core, Runtime and production console.""" +from pathlib import Path + + +def bind(source, target, readonly=True): + return {"type": "bind", "source": str(source), "target": target, "read_only": readonly} + + +def managed_config(state, manifest): + result = {"installation_id": state["installation_id"], "provider": state["provider"], + "maintenance": False} + if state["provider"] == "docker": + result.update(core_url="http://core:8091/api/v1", docker={ + "host": "unix:///var/run/docker.sock", "image": manifest["images"]["runtime"], + "network": state["project"] + "-runtime", "seccomp_file": "/config/seccomp.json", + "nested_sandbox": True, + }) + else: + result.update(core_url="http://host.microsandbox.internal:8091/api/v1", microsandbox={ + "helper_path": "/usr/local/bin/agents-api-microsandbox-provider", + "runtime_path": "/opt/microsandbox/msb", + "firmware_path": "/opt/microsandbox/libkrunfw.so.5.6.1", + "runtime_sha256": manifest["microsandbox"]["runtime_sha256"], + "firmware_sha256": manifest["microsandbox"]["firmware_sha256"], + "runtime_home": "/state/msb", "image": manifest["runtime_ref"], + "memory_mib": 4096, "cpus": 2, "root_disk_mib": 8192, + "idle_seconds": 300, "retention_seconds": 86400, + "max_active": 4, "max_retained": 16, + "network": {"default_egress": "deny", "default_ingress": "deny", "rules": [ + {"action": "allow", "direction": "egress", "destination": "host", "protocol": "tcp", "port": "8091"}, + {"action": "allow", "direction": "egress", "destination": "host", "protocol": "udp", "port": "53"}, + {"action": "allow", "direction": "egress", "destination": "host", "protocol": "tcp", "port": "53"}, + {"action": "allow", "direction": "egress", "destination": "public"}, + ]}, + }) + return result + + +def compose_config(root, state, manifest, database_password): + root = Path(root) + config = root / "config" + identity = f'{state["uid"]}:{state["gid"]}' + doc = {"name": state["project"], "services": {}} + services = doc["services"] + if state["mode"] != "web-only": + services["database"] = { + "image": manifest["images"]["database"], "restart": "unless-stopped", + "environment": {"POSTGRES_USER": "agents_api", "POSTGRES_DB": "agents_api", + "POSTGRES_PASSWORD": database_password}, + "volumes": ["database:/var/lib/postgresql/data"], + "healthcheck": {"test": ["CMD-SHELL", "pg_isready -U agents_api -d agents_api"], + "interval": "2s", "timeout": "5s", "retries": 30}, + } + env = {"AGENTS_API_DATABASE_URL": f"postgres://agents_api:{database_password}@database:5432/agents_api?sslmode=disable", + "AGENTS_API_KEYS_FILE": "/config/keys.json", + "AGENTS_API_CREDENTIAL_KEY_FILE": "/config/credential.key", + "AGENTS_API_ADDR": ":8091", "AGENTS_API_ENGINE": "codex", + "AGENTS_API_HARNESSES": "codex,claude_sdk,mcode", + "AGENTS_API_MANAGED_RUNTIMES_FILE": "/config/managed-runtimes.json", + "AGENTS_API_DAEMON_WS_URL": managed_config(state, manifest)["core_url"].replace("http:", "ws:") + "/agent-daemon/ws"} + shared = {"image": manifest["images"]["core"], "user": identity, + "environment": env, "volumes": [bind(config, "/config")], + "read_only": True, "tmpfs": ["/tmp:mode=1777"], "init": True, + "security_opt": ["no-new-privileges:true"]} + services["migrate"] = dict(shared, command=["/usr/local/bin/agents-api-migrate"], + depends_on={"database": {"condition": "service_healthy"}}) + core = dict(shared, restart="unless-stopped", ports=[f'127.0.0.1:{state["core_port"]}:8091'], + depends_on={"migrate": {"condition": "service_completed_successfully"}}) + core["volumes"] = list(shared["volumes"]) + if state["provider"] == "microsandbox": + core["volumes"].append(bind(root / "state", "/state", False)) + core["environment"] = dict(env, HOME="/state", MSB_HOME="/state/msb") + core["devices"] = ["/dev/kvm:/dev/kvm"] + core["group_add"] = [str(state["device_gid"])] + else: + core["volumes"].append(bind("/var/run/docker.sock", "/var/run/docker.sock", False)) + core["group_add"] = [str(state["device_gid"])] + core["networks"] = ["default", "runtime"] + doc["networks"] = {"runtime": {"name": state["project"] + "-runtime"}} + services["core"] = core + doc["volumes"] = {"database": {}} + if state["mode"] != "core-only": + services["web"] = { + "image": manifest["images"]["web"], "user": identity, "restart": "unless-stopped", + "ports": [f'127.0.0.1:{state["web_port"]}:8080'], "read_only": True, + "security_opt": ["no-new-privileges:true"], + "volumes": [bind(config / "caller.key", "/config/caller.key"), + bind(config / "console.password", "/config/console.password")], + "environment": {"CORE_CONSOLE_ORIGIN": f'http://127.0.0.1:{state["web_port"]}', + "CORE_CONSOLE_UPSTREAM": state.get("core_url") or "http://core:8091", + "CORE_CONSOLE_TOKEN_FILE": "/config/caller.key", + "CORE_CONSOLE_PASSWORD_FILE": "/config/console.password"}, + } + if state["mode"] == "web-only": + services["web"].pop("ports") + services["web"]["network_mode"] = "host" + services["web"]["environment"]["CORE_CONSOLE_ADDR"] = f'127.0.0.1:{state["web_port"]}' + return doc diff --git a/deploy/install/install.py b/deploy/install/install.py new file mode 100644 index 000000000..71ffea352 --- /dev/null +++ b/deploy/install/install.py @@ -0,0 +1,278 @@ +#!/usr/bin/env python3 +"""Install one matched Core distribution without changing execution ownership.""" +import argparse +import base64 +import hashlib +import json +import os +from pathlib import Path +import platform +import secrets +import socket +import stat +import subprocess +import sys +import time +import urllib.error +import urllib.request +import uuid + +from configuration import compose_config, managed_config + + +class InstallError(Exception): + pass + + +def run(args, **kwargs): + # Never print a generated Compose file, process environment or secret value. + return subprocess.run(args, check=True, **kwargs) + + +def private_write(path, value): + descriptor = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o600) + with os.fdopen(descriptor, "w") as stream: + stream.write(value) + + +def write_json(path, value): + private_write(path, json.dumps(value, indent=2) + "\n") + + +def digest(path): + result = hashlib.sha256() + with path.open("rb") as stream: + for block in iter(lambda: stream.read(1024 * 1024), b""): + result.update(block) + return result.hexdigest() + + +def verify_bundle(bundle): + covered = set() + for line in (bundle / "SHA256SUMS").read_text().splitlines(): + expected, name = line.split(" ", 1) + if name in covered: + raise InstallError("Duplicate distribution checksum entry") + covered.add(name) + path = bundle / name + if not path.resolve().is_relative_to(bundle.resolve()) or path.is_symlink() or not path.is_file(): + raise InstallError("Invalid distribution path") + if digest(path) != expected: + raise InstallError("Distribution checksum mismatch: " + name) + required = {"manifest.json", "install.sh", "install.py", "configuration.py", "runtime/seccomp.json"} + required.update(f"images/{name}.tar" for name in ("core", "web", "runtime", "database")) + if not required.issubset(covered): + raise InstallError("Distribution checksum list is incomplete") + manifest = json.loads((bundle / "manifest.json").read_text()) + for image in manifest["images"].values(): + if not image.startswith("sha256:") or len(image) != 71: + raise InstallError("Distribution must select immutable images") + return manifest + + +def free_port(port): + with socket.socket() as sock: + try: + sock.bind(("127.0.0.1", port)) + except OSError: + raise InstallError(f"Port {port} is already in use; select another port") from None + + +def core_target(value): + from urllib.parse import urlsplit + parsed = urlsplit(value) + if (parsed.scheme not in ("http", "https") or not parsed.hostname or parsed.username or + parsed.password or parsed.query or parsed.fragment or parsed.path not in ("", "/")): + raise argparse.ArgumentTypeError("Core URL must be an HTTP(S) origin without credentials") + if parsed.scheme != "https" and parsed.hostname not in ("127.0.0.1", "localhost"): + raise argparse.ArgumentTypeError("Remote Core requires HTTPS") + return value.rstrip("/") + + +def arguments(argv=None): + parser = argparse.ArgumentParser(description=__doc__) + modes = parser.add_mutually_exclusive_group() + modes.add_argument("--core-only", action="store_true") + modes.add_argument("--web-only", action="store_true") + parser.add_argument("--provider", choices=("microsandbox", "docker"), default="microsandbox") + parser.add_argument("--install-dir", type=Path, default=Path.home() / ".parsar/core") + parser.add_argument("--core-port", type=int, default=8091) + parser.add_argument("--web-port", type=int, default=8080) + parser.add_argument("--core-url", type=core_target) + parser.add_argument("--core-token-file", type=Path) + parser.add_argument("--status", action="store_true", help="Read installation health; never invoke a model") + parser.add_argument("--stop", action="store_true", help="Stop installed services; retain all data") + args = parser.parse_args(argv) + if args.status and args.stop: + parser.error("Choose status or stop") + if not args.install_dir.is_absolute(): + parser.error("--install-dir must be absolute") + if any(not 1024 <= p <= 65535 for p in (args.core_port, args.web_port)): + parser.error("Ports must be between 1024 and 65535") + if not args.core_only and not args.web_only and args.core_port == args.web_port: + parser.error("Core and Web need different ports") + if args.web_only and not (args.core_url and args.core_token_file): + parser.error("--web-only requires --core-url and --core-token-file") + if not args.web_only and (args.core_url or args.core_token_file): + parser.error("Existing Core connection flags require --web-only") + return args + + +def compose(root, *args, **kwargs): + return run(["docker", "compose", "-f", str(root / "compose.json"), *args], **kwargs) + + +def wait_http(url, headers=None, attempts=60): + for attempt in range(attempts): + try: + request = urllib.request.Request(url, headers=headers or {}) + with urllib.request.urlopen(request, timeout=2) as response: + if response.status == 200: + return True + except (urllib.error.URLError, TimeoutError): + pass + if attempt + 1 < attempts: + time.sleep(1) + return False + + +def status(root, state): + output = compose(root, "ps", "--all", "--format", "json", capture_output=True, text=True).stdout + # Compose versions may return one array or one object per line. + rows = json.loads(output) if output.lstrip().startswith("[") else [json.loads(line) for line in output.splitlines() if line] + healthy = True + for row in rows: + print(f'{row["Service"]}: {row["State"]} {row.get("Health", "")}') + if state["mode"] != "web-only": + core_ok = wait_http(f'http://127.0.0.1:{state["core_port"]}/healthz', attempts=1) + healthy = healthy and core_ok + print("Core API: " + ("healthy" if core_ok else "unavailable")) + if state["mode"] != "core-only": + web_ok = wait_http(f'http://127.0.0.1:{state["web_port"]}/healthz', attempts=1) + healthy = healthy and web_ok + print("Web: " + ("healthy" if web_ok else "unavailable")) + print("Service health does not prove model execution. This check makes no model requests.") + if not healthy: + raise InstallError("One or more installed services are unavailable") + + +def initialize(root, args, manifest): + mode = "core-only" if args.core_only else "web-only" if args.web_only else "all" + if (root / "installation.json").exists(): + state = json.loads((root / "installation.json").read_text()) + wanted = (mode, args.provider, args.core_port, args.web_port, args.core_url) + actual = (state["mode"], state["provider"], state["core_port"], state["web_port"], state.get("core_url")) + if wanted != actual or state["source_commit"] != manifest["source_commit"]: + raise InstallError("Existing installation differs; preserve it and follow the upgrade/provider-change guide") + return state + if root.exists() and any(root.iterdir()): + raise InstallError("Installation directory is not empty; refusing to overwrite existing state") + if mode == "web-only": + source = args.core_token_file + info = source.stat() + if (not source.is_absolute() or source.is_symlink() or not stat.S_ISREG(info.st_mode) + or stat.S_IMODE(info.st_mode) & 0o077 or info.st_size > 4096): + raise InstallError("Core token file must be an absolute, private regular file") + token = source.read_text().strip() + if not token or any(c.isspace() for c in token) or "\x00" in token: + raise InstallError("Invalid Core token file") + else: + device_gid = os.stat("/dev/kvm" if args.provider == "microsandbox" else "/var/run/docker.sock").st_gid + token = secrets.token_hex(32) + if mode != "web-only": + free_port(args.core_port) + if mode != "core-only": + free_port(args.web_port) + root.mkdir(mode=0o700, parents=True, exist_ok=True) + os.chmod(root, 0o700) + for name in ("config", "state", "state/msb"): + (root / name).mkdir(mode=0o700) + state = {"version": 1, "source_commit": manifest["source_commit"], "mode": mode, + "provider": args.provider, "installation_id": str(uuid.uuid4()), + "project": "parsar-" + secrets.token_hex(5), "uid": os.getuid(), "gid": os.getgid(), + "core_port": args.core_port, "web_port": args.web_port, "core_url": args.core_url} + config = root / "config" + if mode != "web-only": + state["device_gid"] = device_gid + write_json(config / "keys.json", [{"tenant_id": str(uuid.uuid4()), "organization_id": "installation", + "project_id": "default", "subject_kind": "service_account", "subject_id": "operator", + "token_sha256": hashlib.sha256(token.encode()).hexdigest()}]) + private_write(config / "credential.key", base64.b64encode(secrets.token_bytes(32)).decode()) + private_write(config / "database.password", secrets.token_hex(32)) + write_json(config / "managed-runtimes.json", managed_config(state, manifest)) + private_write(config / "caller.key", token) + if mode != "core-only": + private_write(config / "console.password", secrets.token_hex(24)) + password = (config / "database.password").read_text() if mode != "web-only" else "" + write_json(root / "compose.json", compose_config(root, state, manifest, password)) + write_json(root / "installation.json", state) + return state + + +def import_runtime(root, state, manifest, bundle): + if state["provider"] != "microsandbox" or state["mode"] == "web-only": + return + run(["docker", "run", "--rm", "--init", "--user", f'{state["uid"]}:{state["gid"]}', + "--env", "MSB_HOME=/state/msb", "--env", "HOME=/state", + "--mount", f'type=bind,source={root / "state"},target=/state', + "--mount", f'type=bind,source={bundle / "images/runtime.tar"},target=/runtime.tar,readonly', + "--entrypoint", "/opt/microsandbox/msb", manifest["images"]["core"], + "image", "load", "--input", "/runtime.tar", "--tag", manifest["runtime_ref"], "--quiet"]) + + +def main(argv=None): + args = arguments(argv) + root = args.install_dir + if root.is_symlink() or root.resolve() != root: + raise InstallError("Installation directory must be canonical and not a symlink") + if args.status or args.stop: + state = json.loads((root / "installation.json").read_text()) + if args.stop: + compose(root, "stop") + print("Services stopped. Database and Runtime state retained.") + else: + status(root, state) + return + if platform.system() != "Linux" or platform.machine() not in ("x86_64", "amd64") or os.getuid() == 0: + raise InstallError("Run as a non-root user on Linux amd64 with Docker access") + run(["docker", "compose", "version"], stdout=subprocess.DEVNULL) + run(["docker", "info", "--format", "{{.ServerVersion}}"], stdout=subprocess.DEVNULL) + if not args.web_only and args.provider == "microsandbox" and not Path("/dev/kvm").exists(): + raise InstallError("microsandbox requires host KVM; enable virtualization or explicitly choose --provider docker") + bundle = Path(__file__).resolve().parent + manifest = verify_bundle(bundle) + state = initialize(root, args, manifest) + if state["mode"] != "web-only": + seccomp = bundle / "runtime/seccomp.json" + if not (root / "config/seccomp.json").exists(): + private_write(root / "config/seccomp.json", seccomp.read_text()) + images = ["web"] if state["mode"] == "web-only" else ["core", "runtime", "database"] + if state["mode"] == "all": + images.append("web") + for name in images: + run(["docker", "load", "--input", str(bundle / f"images/{name}.tar")], stdout=subprocess.DEVNULL) + import_runtime(root, state, manifest, bundle) + compose(root, "up", "--detach") + if state["mode"] != "web-only" and not wait_http(f'http://127.0.0.1:{state["core_port"]}/healthz'): + raise InstallError("Core did not become healthy. Use --status; retained state has not been removed") + if state["mode"] != "core-only": + url = f'http://127.0.0.1:{state["web_port"]}' + auth = base64.b64encode(("admin:" + (root / "config/console.password").read_text()).encode()).decode() + if not wait_http(url + "/v1/agents", {"Authorization": "Basic " + auth, "OpenAI-Beta": "agents=v1"}): + raise InstallError("Web could not authenticate to Core. Inspect private configuration; no model was called") + print("Console: " + url + " (user: admin)") + print("Console password file: " + str(root / "config/console.password")) + if state["mode"] != "web-only": + print(f'API: http://127.0.0.1:{state["core_port"]}/v1') + print("Caller key file: " + str(root / "config/caller.key")) + print("Provider: " + state["provider"] + ". Runtime image prepared; Core provisions Sessions on demand.") + print("Services installed. No model request was made. See docs/getting-started/quickstart.md.") + + +if __name__ == "__main__": + try: + main() + except (InstallError, OSError, ValueError, KeyError, subprocess.CalledProcessError) as error: + # Errors never include generated configuration or external process output. + print(str(error) if isinstance(error, InstallError) else "Installation failed; inspect prerequisites and private deployment files", file=sys.stderr) + sys.exit(1) diff --git a/deploy/install/install.sh b/deploy/install/install.sh new file mode 100755 index 000000000..6802833bc --- /dev/null +++ b/deploy/install/install.sh @@ -0,0 +1,3 @@ +#!/usr/bin/env bash +set -euo pipefail +exec python3 "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/install.py" "$@" diff --git a/docs/getting-started/README.md b/docs/getting-started/README.md new file mode 100644 index 000000000..e722a4a77 --- /dev/null +++ b/docs/getting-started/README.md @@ -0,0 +1,20 @@ +# Getting started + +Parsar Core is self-deployed, open-source Agents API infrastructure. It provides +an execution API and an optional Web console, with native harnesses behind one +Runtime contract. The Parsar product is not required. + +- [Install Core and Web](install.md) +- [Call the API](quickstart.md) +- [Operate the installation](operations.md) +- [Protocol coverage and native differences](https://github.com/MiniMax-AI/parsar-core/blob/main/contracts/agents-api/README.md) + +Core and Web ship together. A default installation prepares microsandbox and the +colocated Runtime; choose Docker with `--provider docker`. Core creates the +execution sandbox when a Session needs it. Installing the service does not require +a model key or run a model request. + +The Web console is a client of Core. API users work with Agents, Sessions and +Environments; operators also maintain the host, provider, Runtime images and +durable storage. Those operational needs extend beyond the hosted OpenAI +Platform experience, without redefining its public resource semantics. diff --git a/docs/getting-started/install.md b/docs/getting-started/install.md new file mode 100644 index 000000000..6c74e0385 --- /dev/null +++ b/docs/getting-started/install.md @@ -0,0 +1,122 @@ +# Install Core and Web + +Install one matching Parsar Core distribution. By default it starts PostgreSQL, +Core and the existing Web console, prepares microsandbox and imports the Runtime +image. Core creates a sandbox and starts its daemon when a Session needs it. +No model key, Environment wizard or sample task is required during installation. + +## Host requirements + +The first distribution targets Linux amd64 with Python 3.9+, Docker and Docker +Compose v2. Run the installer as a non-root user who can use Docker. The qualified +Docker host version is 29.1.3. Default microsandbox also requires an available +`/dev/kvm`; nested cloud hosts must expose hardware virtualization. The installer +passes that device and its group to Core, without a privileged container. +It does not silently fall back to Docker when KVM is missing. + +Reserve capacity for the native Runtime: the initial microsandbox profile uses +4 GiB RAM, 2 CPUs and an 8 GiB root disk per active sandbox, with at most 4 active +and 16 retained allocations. Limits are operator configuration, not model input. +Use a trusted, single-operator host and durable local storage. The installer does +not change host virtualization settings, install Docker, create an OS user or +expose a remote administration service. + +## Verify, extract and install + +Obtain the archive and checksum from a trusted distributor. Until a release is +published, build an archive using [Build a distribution](#build-a-distribution); +a source checkout alone is not an installable binary bundle. Do not substitute an +unpublished download URL. + +```sh +sha256sum -c parsar-core--linux-amd64.tar.gz.sha256 +mkdir -p "$HOME/.parsar/releases" +tar -xzf parsar-core--linux-amd64.tar.gz -C "$HOME/.parsar/releases" +cd "$HOME/.parsar/releases/parsar-core--linux-amd64" +./install.sh +``` + +The bundle contains the same-revision Core, unchanged Web build, production Web +proxy and colocated Runtime. It includes microsandbox's pinned runtime and +firmware. It also contains image archives, source provenance and checksums; +installation does not need Go, Node, Rust or a product checkout. + +Installation creates private configuration under `~/.parsar/core`, a dedicated +PostgreSQL volume, an API caller key and a credential encryption key. It also +creates a separate console password. Secret values are not printed. On success: + +- API: `http://127.0.0.1:8091/v1` +- Web console: `http://127.0.0.1:8080`, username `admin` +- Console password file: `~/.parsar/core/config/console.password` +- API caller key file: `~/.parsar/core/config/caller.key` + +Use exactly the displayed console address; the production proxy validates its +configured browser origin. The console password authenticates to the Web server; +the server holds the independent Core key. Model keys remain API execution input. + +## Installation choices + +```sh +./install.sh --provider docker +./install.sh --core-only +./install.sh --core-only --provider docker +``` + +`--provider` selects the deployment's sandbox provider. It does not select a +harness or alter the public `openai_hosted` discriminator. Both providers reuse +one colocated Runtime containing the native harnesses. The Docker option grants +only Core access to the Docker socket; microsandbox grants only Core access to +KVM. Web receives neither. + +To install only Web on a Linux host, provide the existing Core origin and a +private caller-key file. A loopback Core uses the same host network namespace; +a remote Core must use HTTPS. + +```sh +./install.sh --web-only \ + --install-dir "$HOME/.parsar/core-console" \ + --core-url http://127.0.0.1:8091 \ + --core-token-file "$HOME/.parsar/core/config/caller.key" +``` + +Web-only mode starts no database or Core and requires no KVM. Its key remains on +the server, outside the static Web files. The input file must be private (0600). + +Use `--install-dir /absolute/path`, `--core-port 8092` and `--web-port 8081` for +separate installations. Their database volumes, provider identities and Runtime +state are independent. Repeating the same installation command retains its +identities, secrets and data. Conflicting mode/provider/revision changes refuse +rather than silently replacing them. + +For remote browser or SDK access, put the intended endpoint behind your existing +TLS and access-control boundary. Update the console's trusted origin explicitly; +do not simply publish its port on every network interface. + +## After installation + +Start with an optional [API request](quickstart.md). Session creation supplies the +model, harness and write-only model credentials. Core owns sandbox preparation and +Runtime startup. Configuration is never injected into a public Agent instruction +or baked into a Runtime image. + +[Operations](operations.md) covers health, restarts, data and provider changes. +A service health check proves neither model availability nor complete protocol +compatibility. + +## Build a distribution + +Release builders need the repository's full Linux toolchain and Docker. Build from +clean, committed source, with the pinned Codex platform package and a matching +MiniMax companion prepared through the existing Runtime build instructions: + +```sh +export AGENTS_RUNTIME_CODEX_PACKAGE=/absolute/path/to/codex-linux-package +export MCODE_HARNESS_BUILD_DIR=/absolute/path/to/mcode-harness-artifact +make build-core-distribution +``` + +The builder reuses existing Core, Runtime, SDK and Web build scripts. It records +the commit, immutable image identities, microsandbox binary hashes and the actual +Runtime OCI manifest digest. Build output lives under `~/.parsar/build/`; it is +not automatically published to GitHub, an image registry or a website. Qualify the +exact bundle before distribution. See the [contributor guide](https://github.com/MiniMax-AI/parsar-core/blob/main/CONTRIBUTING.md). diff --git a/docs/getting-started/operations.md b/docs/getting-started/operations.md new file mode 100644 index 000000000..cee5d9230 --- /dev/null +++ b/docs/getting-started/operations.md @@ -0,0 +1,107 @@ +# Operate your Core + +The API abstracts execution environments for clients. The installation operator +also owns the host, container or microVM provider, storage and service availability. +Those are separate from a Session's public execution state. This guide adds that +self-deployment view without adding a second Runtime controller. + +## Read service health + +Run from the extracted bundle: + +```sh +./install.sh --status +# For a separate installation: +./install.sh --status --install-dir "$HOME/.parsar/core-console" +``` + +The command reads only this installation's service status and health endpoints. +It prints service names, running/exit state and available Docker health state; +it does not print Compose configuration, environment variables, credentials or +raw application logs. It never creates a Session or calls a model. + +Use these observations for distinct questions: + +| Observation | What it establishes | +| --- | --- | +| PostgreSQL container health | The dedicated database accepts its readiness check | +| Core `/healthz` | Core process liveness | +| Authenticated API resource read | Caller authentication and the requested resource operation | +| Environment connection | Runtime transport observation | +| Terminal Turn and queried results | The requested task's recorded execution outcome | + +Container liveness alone is not a healthy native harness or an available model. +Use public Session, Turn, Items, Environment and Usage reads for execution. Reuse +Core's Runtime observations for sandbox details when available; do not infer +execution truth from Docker or invent a second lifecycle collector. The existing +Web is unchanged by this installation batch. + +## Stop and restart + +Settle active work before a planned restart. Then: + +```sh +./install.sh --stop +./install.sh # Use the same component/provider/port flags as the initial install. +``` + +Stopping services retains the database, Runtime state and credentials. It does not +promise transparent continuation of an interrupted native tool. Query the same +Session after reconnecting; do not create a replacement Session to replay uncertain +work. The official SSE stream is live, and recovery uses durable resource queries. + +The generated Compose file is private because it contains database connection +credentials. Do not paste `docker compose config`, `docker inspect` or raw logs into +public issue reports. For local diagnosis, use the exact installation file: + +```sh +docker compose -f "$HOME/.parsar/core/compose.json" ps --all +``` + +## Data and upgrades + +Retain together: + +- the dedicated PostgreSQL volume, including large objects; +- `config/credential.key`, caller identity configuration and provider identity; +- microsandbox's private state/cache/disks/snapshots, or Docker-owned Runtime + volumes and histories; +- the exact distribution and private deployment configuration needed to recover. + +Never regenerate the encryption key to resolve an error: stored Session/model and +Vault credentials require it. Never prune Docker volumes or delete native history +to make a retry pass. Public Session deletion acknowledgement does not prove that +all physical provider resources have been reclaimed. + +This first installer supports fresh installation and same-release restart. It +refuses automatic replacement of an installed revision. For a reviewed upgrade, +back up the coordinated state, retain the previous distribution, settle execution, +apply the existing Core migration workflow and replace matched service/Runtime +artifacts while retaining identities and backend paths. Qualify recovery before +claiming the upgrade complete; there is no downgrade or history migration promise. + +Provider replacement is an operator operation, not a new `--provider` value on an +existing install. Follow the [maintenance and provider-switch procedure](https://github.com/MiniMax-AI/parsar-core/blob/main/services/agents-api/deploy/microsandbox/README.md#change-the-deployment-provider). +The installer never migrates Sessions between providers or deletes old compute. + +## Exposure and network policy + +API and console ports publish to host loopback. PostgreSQL has no published port. +The production Web proxy only forwards the public `/v1` surface to its configured +Core; it never receives the Docker socket, KVM device or provider/model secrets. +It requires an independent console password and rejects untrusted browser origins. +This is a single-operator console deployment, not a multi-user identity system. + +microsandbox uses an explicit policy: public egress, the Core/DNS host ports needed +for the colocated Runtime, and denied inbound/private-network access. Private +model/MCP endpoints require an explicit operator policy change. Native tool network +policy remains the Session's separate public configuration. + +Docker uses the existing qualified nested-sandbox Runtime policy. Only Core can +access the selected host Docker daemon. Install on a trusted service host and do +not share its Docker authority with untrusted users. + +Host virtualization, credentials, tenant isolation, durable state and actual +execution are release acceptance requirements. Other missing operational screens +or low-frequency improvements belong in the backlog; they do not turn this batch +into a Web redesign or a complete protocol-compatibility claim. diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md new file mode 100644 index 000000000..56073b255 --- /dev/null +++ b/docs/getting-started/quickstart.md @@ -0,0 +1,103 @@ +# Call your Core + +Install Core using the [installation guide](install.md). Your Core API credential +authenticates the caller to your installation. It is separate from the model +provider key used by a native harness. Neither key belongs in source control. + +Use the fixed client baseline: + +```sh +python3 -m venv .venv +. .venv/bin/activate +pip install openai==3.13.0 +``` + +Read the generated caller key from the private installation directory. For an +installation on another machine, use its authenticated HTTPS API endpoint. + +```python +from pathlib import Path +from openai import OpenAI + +client = OpenAI( + base_url="http://127.0.0.1:8091/v1", + api_key=Path.home().joinpath(".parsar/core/config/caller.key").read_text().strip(), + default_headers={"OpenAI-Beta": "agents=v1"}, +) +print(client.beta.agents.list().data) +``` + +This read verifies API access. It does not invoke a model or create an execution +environment. Installation has no mandatory sample task. + +## Run a Session when you are ready + +Core prepares the sandbox and starts its Runtime. You do not install or start a +separate daemon for a Core-managed Session. The public discriminator remains +`openai_hosted`; in this deployment it means the sandbox managed by Parsar Core. +The installation's provider can be microsandbox or Docker. + +For a Codex-compatible Responses endpoint, provide your actual model name, +endpoint and key through your application's private configuration: + +```python +import os + +session = client.beta.agents.sessions.create( + agent={"model": os.environ["MODEL_NAME"]}, + environment={"type": "openai_hosted"}, + input="Create /workspace/hello.txt with a short greeting, then describe it.", + extra_body={ + "agent": {"x_agents_core": {"harness": "codex"}}, + "x_agents_core": { + "model_provider": { + "protocol": "responses", + "base_url": os.environ["MODEL_BASE_URL"], + "api_key": os.environ["MODEL_API_KEY"], + } + }, + }, +) +print(session.id) +``` + +Running this example makes a real model request and may incur provider charges. +`extra_body` carries existing Core extensions: harness selection and write-only +model configuration. They are not fields in the official SDK 3.13.0 protocol. +The service encrypts model configuration with tenant/Session binding and never +returns the secret through public resource reads. Keep the installation's +credential encryption key and database together across restarts. + +The model must support the selected harness's native protocol: + +| Harness selector | Model protocol | Additional input | +| --- | --- | --- | +| `codex` | `responses` | Exact provider model ID | +| `claude_sdk` | `anthropic` | Exact provider model ID | +| `mcode` | `anthropic` | Actual `context_window` and `max_output_tokens` limits | + +See [harness selection](https://github.com/MiniMax-AI/parsar-core/blob/main/contracts/agents-api/harness-selection.md) and +[model execution](https://github.com/MiniMax-AI/parsar-core/blob/main/contracts/agents-api/model-execution.md) for the complete +extension contract. Native capabilities differ; selecting an engine does not make +an unsupported model or operation work. + +## Observe and recover + +Use the existing console or query the same resources from your application: + +```python +current = client.beta.agents.sessions.retrieve(session.id) +turns = client.beta.agents.sessions.turns.list(session.id) +items = client.beta.agents.sessions.items.list(session.id) +print(current.id, turns.data, items.data) +``` + +Wait for the Turn's terminal result before treating a task as complete. After a +client disconnect, recover through Session, Turn and Items queries. SSE is a live +stream, not a historical replay mechanism. Do not automatically submit the same +work as a new Session when a response is lost. + +Files, Artifacts, cancellation, credential management and their current limits +are documented in the [coverage ledger](https://github.com/MiniMax-AI/parsar-core/blob/main/contracts/agents-api/README.md). +MCP credentials use the separate Vault API; they are not Core caller keys or +model-provider credentials. diff --git a/scripts/build-core-distribution.sh b/scripts/build-core-distribution.sh index b47116647..e4d0dc701 100755 --- a/scripts/build-core-distribution.sh +++ b/scripts/build-core-distribution.sh @@ -60,7 +60,8 @@ for file in install.sh install.py configuration.py; do cp "deploy/install/$file" "$bundle/$file" done mkdir -p "$bundle/docs" -cp docs/getting-started.md "$bundle/docs/" +cp -R docs/getting-started "$bundle/docs/" +cp README.md "$bundle/" mkdir -p "$bundle/runtime" cp services/agents-api/deploy/codex/seccomp.json "$bundle/runtime/" cp LICENSE "$bundle/" From f6b063c4c07a3982c01f596db34934410f0afdca Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 15:03:11 +0800 Subject: [PATCH 07/17] Add public API release acceptance runner --- deploy/install/acceptance.py | 450 +++++++++++++++++++++++++++++++++++ 1 file changed, 450 insertions(+) create mode 100644 deploy/install/acceptance.py diff --git a/deploy/install/acceptance.py b/deploy/install/acceptance.py new file mode 100644 index 000000000..86bac3622 --- /dev/null +++ b/deploy/install/acceptance.py @@ -0,0 +1,450 @@ +#!/usr/bin/env python3 +"""Opt-in real-model acceptance of an installed Core through its public API. + +Run stages in order: run, optionally restart Core/Runtime externally, continue, +then cancel. Each stage uses the same private report directory. No service or +container restart is performed here. Sessions and files are retained for inspection. + +Private model configuration shape (all three harnesses are required): +{"codex": {"model": "...", "model_provider": {"protocol": "responses", ...}}, + "claude_sdk": {"model": "...", "model_provider": {"protocol": "anthropic", ...}}, + "mcode": {"model": "...", "model_provider": {"protocol": "anthropic", ...}}} +See contracts/agents-api/model-execution.md for provider fields and limits. + +Example: python3 deploy/install/acceptance.py run --base-url http://127.0.0.1:8091 + --caller-key-file /private/caller.key --model-config-file /private/models.json + --report-dir /private/release-acceptance + +This calls real models and leaves owned resources in place. A failed check remains +failed; this script does not translate native differences into synthetic success. +""" + +import argparse +import hashlib +import ipaddress +import json +import os +from pathlib import Path +import re +import shlex +import stat +import time +from urllib.error import HTTPError, URLError +from urllib.parse import urlencode, urlsplit +from urllib.request import HTTPRedirectHandler, ProxyHandler, Request, build_opener +import uuid + +HARNESSES = ("codex", "claude_sdk", "mcode") +TERMINAL = {"completed", "cancelled", "failed"} +MAX_RESPONSE = 8 * 1024 * 1024 + + +class Failure(Exception): + """Messages are fixed labels, never remote response text or private values.""" + + +def require(condition, label): + if not condition: + raise Failure(label) + + +def identifier(value): + require(isinstance(value, str) and re.fullmatch(r"[A-Za-z0-9_-]{1,200}", value), + "invalid_resource_identifier") + return value + + +def digest(value): + return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(",", ":")).encode()).hexdigest() + + +def private_file(path): + require(path.is_absolute() and not path.is_symlink(), "private_file_requires_absolute_regular_path") + info = path.stat() + require(stat.S_ISREG(info.st_mode) and info.st_mode & 0o077 == 0, + "private_file_permissions_must_exclude_group_and_others") + require(info.st_size <= MAX_RESPONSE, "private_file_too_large") + return path.read_text() + + +def validate_origin(raw, model=False): + require(isinstance(raw, str), "invalid_origin") + parsed = urlsplit(raw) + host = (parsed.hostname or "").lower().rstrip(".") + require(host and not parsed.username and not parsed.password and not parsed.query + and not parsed.fragment, "invalid_origin") + require(host not in {"api.openai.com", "platform.openai.com"}, "official_openai_origin_refused") + if model: + require(parsed.scheme == "https", "model_origin_requires_https") + else: + try: + loopback = ipaddress.ip_address(host).is_loopback + except ValueError: + loopback = host == "localhost" + require(parsed.scheme == "https" or (parsed.scheme == "http" and loopback), + "core_origin_requires_https_or_loopback") + return raw.rstrip("/") + + +def settings(args): + token = private_file(args.caller_key_file).strip() + require(token and not any(c in token for c in "\r\n\0"), "invalid_caller_key") + models = json.loads(private_file(args.model_config_file)) + require(isinstance(models, dict) and set(models) == set(HARNESSES), "require_all_three_harness_configs") + secrets = [token] + for harness, entry in models.items(): + require(isinstance(entry, dict) and set(entry) == {"model", "model_provider"}, "invalid_model_config_fields") + require(isinstance(entry["model"], str) and entry["model"].strip(), "invalid_model_name") + provider = entry["model_provider"] + require(isinstance(provider, dict), "invalid_model_provider") + allowed = {"protocol", "base_url", "api_key", "context_window", "max_output_tokens"} + require(set(provider) <= allowed and {"protocol", "base_url", "api_key"} <= set(provider), + "invalid_model_provider_fields") + validate_origin(provider["base_url"], model=True) + protocol = "responses" if harness == "codex" else "anthropic" + require(provider["protocol"] == protocol, "provider_protocol_does_not_match_harness") + key = provider["api_key"] + require(isinstance(key, str) and key and len(key) <= 16384 + and not any(c in key for c in "\r\n\0"), "invalid_model_provider_key") + for name in ("context_window", "max_output_tokens"): + require(type(provider.get(name, 0)) is int and provider.get(name, 0) >= 0, + "invalid_model_limits") + if harness == "mcode": + require(provider.get("context_window", 0) > 0 and provider.get("max_output_tokens", 0) > 0, + "mcode_requires_positive_model_limits") + if provider.get("context_window") and provider.get("max_output_tokens"): + require(provider["max_output_tokens"] <= provider["context_window"], "invalid_model_limits") + secrets.append(key) + return token, models, secrets + + +class NoRedirect(HTTPRedirectHandler): + def redirect_request(self, req, fp, code, msg, headers, newurl): + raise Failure("redirect_refused") + + +class API: + def __init__(self, base, token, evidence): + self.base = base.removesuffix("/v1") + "/v1/agents" + self.token = token + self.evidence = evidence + self.opener = build_opener(ProxyHandler({}), NoRedirect()) + + def request(self, method, path, operation, body=None, query=None, key=None, binary=False): + headers = {"Authorization": "Bearer " + self.token, "OpenAI-Beta": "agents=v1"} + data = None + if body is not None: + headers["Content-Type"] = "application/json" + data = json.dumps(body).encode() + if key: + headers["Idempotency-Key"] = key + url = self.base + path + ("?" + urlencode(query) if query else "") + entry = {"operation": operation, "method": method} + self.evidence.append(entry) + try: + with self.opener.open(Request(url, data=data, headers=headers, method=method), timeout=30) as response: + entry["http_status"] = response.status + request_id = response.headers.get("x-request-id", "") + if re.fullmatch(r"[A-Za-z0-9_.:-]{1,200}", request_id): + entry["request_id"] = request_id + raw = response.read(MAX_RESPONSE + 1) + except HTTPError as error: + entry["http_status"] = error.code + error.close() + raise Failure("public_api_http_error") from None + except (URLError, TimeoutError, OSError): + raise Failure("public_api_transport_error") from None + require(len(raw) <= MAX_RESPONSE, "response_limit_exceeded") + if binary: + return raw + try: + value = json.loads(raw) + except (ValueError, UnicodeError): + raise Failure("invalid_public_json") from None + require(isinstance(value, dict), "invalid_public_object") + return value + + def get(self, path, operation, query=None): + return self.request("GET", path, operation, query=query) + + def listing(self, path, operation, query=None, files=False): + query = {"limit": 100, "order": "asc", **(query or {})} + rows, cursors = [], set() + for _ in range(100): + page = self.get(path, operation, query) + require(isinstance(page.get("data"), list), "invalid_list_page") + rows.extend(page["data"]) + require(all(isinstance(row, dict) for row in page["data"]), "invalid_list_row") + more = page.get("has_more") is not False and bool(page.get("next")) if files else page.get("has_more") + require(page.get("has_more") is not True or not files or more, "missing_files_cursor") + if not more: + return rows + require(page["data"], "empty_continuing_page") + cursor = page.get("next") if files else identifier(page["data"][-1].get("id")) + require(isinstance(cursor, str) and cursor not in cursors, "invalid_list_cursor") + cursors.add(cursor) + query["page" if files else "after"] = cursor + raise Failure("pagination_limit_exceeded") + + +def session_path(record): + return "/sessions/" + identifier(record["session_id"]) + + +def wait_turn(api, record, previous, timeout, status="completed"): + deadline = time.monotonic() + timeout + path = session_path(record) + while time.monotonic() < deadline: + current = api.get(path, "session.retrieve") + require(current.get("status") != "failed", "session_failed") + require(not current.get("required_actions"), "unexpected_required_action") + new = [row for row in api.listing(path + "/turns", "turns.list") if row.get("id") not in previous] + require(len(new) <= 1, "input_created_multiple_turns") + if new and new[0].get("status") in TERMINAL: + turn = new[0] + record.setdefault("terminal_turns", {})[identifier(turn.get("id"))] = turn["status"] + require(turn["status"] == status, "unexpected_terminal_turn_status") + if current.get("status") == "idle": + return turn + time.sleep(1) + raise Failure("turn_timeout") + + +def submit(api, record, text): + event = {"type": "agent.session.input.message", "input": [ + {"role": "user", "content": [{"type": "input_text", "text": text}]}]} + api.request("POST", session_path(record) + "/events", "events.message", + body={"events": [event]}, key=uuid.uuid4().hex) + + +def items(api, record): + return api.listing(session_path(record) + "/items", "items.list") + + +def turn_ids(api, record): + return {identifier(row.get("id")) for row in api.listing(session_path(record) + "/turns", "turns.list")} + + +def file_rows(api, record, directory): + return api.listing("/environments/" + identifier(record["environment_id"]) + "/files", + "files.list", {"path": directory}, files=True) + + +def verify_output(api, record, turn_id, path, expected): + rows = file_rows(api, record, "/workspace/outputs") + matching = [row for row in rows if row.get("path") == path] + require(len(matching) == 1, "native_output_missing_from_files") + row = matching[0] + require(row.get("environment_id") == record["environment_id"] + and row.get("object") == "agent.environment.file" + and row.get("size_bytes") == len(expected), "native_output_metadata_mismatch") + artifacts = api.listing(session_path(record) + "/artifacts", "artifacts.list") + matches = [row for row in artifacts if row.get("path") == path and row.get("turn_id") == turn_id] + require(len(matches) == 1, "captured_artifact_missing_or_duplicate") + artifact = matches[0] + aid = identifier(artifact.get("id")) + require(artifact.get("session_id") == record["session_id"] + and artifact.get("environment_id") == record["environment_id"] + and artifact.get("object") == "agent.session.artifact" + and artifact.get("size_bytes") == len(expected), "artifact_metadata_mismatch") + endpoint = session_path(record) + "/artifacts/" + aid + require(api.get(endpoint, "artifact.retrieve") == artifact, "artifact_retrieve_mismatch") + content = api.request("GET", endpoint + "/content", "artifact.content", binary=True) + require(content == expected, "artifact_bytes_mismatch") + return {"id": aid, "path": path, "turn_id": turn_id, "size_bytes": len(content), + "sha256": hashlib.sha256(content).hexdigest()} + + +def write_prompt(path, value): + command = "mkdir -p /workspace/outputs && printf %s " + shlex.quote(value) + " > " + shlex.quote(path) + return ("Use your native shell tool to execute exactly once: " + command + + ". Do not delegate or repeat the command. Read the file with your native tools " + "to verify its exact contents, then reply with those contents.") + + +def verify_answer(api, record, turn_id, wanted): + stored = items(api, record) + text = "\n".join(part.get("text", "") for row in stored + if row.get("turn_id") == turn_id and row.get("type") == "message" + and row.get("role") == "assistant" for part in row.get("content", []) + if part.get("type") == "output_text") + require(wanted in text, "native_answer_did_not_contain_expected_nonce") + return stored + + +def run(api, record, config, timeout, save): + require("session_id" not in record, "run_already_created_session_use_fresh_report_directory") + nonce = uuid.uuid4().hex + record.update(nonce=nonce, memory="remember-" + uuid.uuid4().hex, + output_path="/workspace/outputs/release-" + nonce + ".txt") + payload = {"agent": {"model": config["model"], "x_agents_core": {"harness": record["harness"]}}, + "environment": {"type": "openai_hosted"}, + "x_agents_core": {"model_provider": config["model_provider"]}, + "input": "Remember this conversation-only token without writing it to any file: " + + record["memory"] + ". " + write_prompt(record["output_path"], nonce), "stream": False} + creation_key = uuid.uuid4().hex + record["creation_idempotency_key"] = creation_key + save() + session = api.request("POST", "/sessions", "session.create", body=payload, key=creation_key) + record["session_id"] = identifier(session.get("id")) + environment = session.get("environment") or {} + record["environment_id"] = identifier(environment.get("id")) + save() + require(environment.get("type") == "openai_hosted", "wrong_environment_type") + turn = wait_turn(api, record, set(), timeout) + stored = verify_answer(api, record, turn["id"], nonce) + record["artifact"] = verify_output(api, record, turn["id"], record["output_path"], nonce.encode()) + record["items_digest"] = digest(stored) + + +def continuation(api, record, timeout): + path = session_path(record) + current = api.get(path, "session.retrieve") + require(current.get("status") == "idle", "continuation_requires_idle_session") + deadline = time.monotonic() + min(timeout, 60) + while time.monotonic() < deadline: + environment = api.get("/environments/" + identifier(record["environment_id"]), "environment.retrieve") + if environment.get("status") == "connected": + break + time.sleep(1) + else: + raise Failure("runtime_not_reconnected") + require(digest(items(api, record)) == record["items_digest"], "committed_items_changed") + old = record["artifact"] + verify_output(api, record, old["turn_id"], old["path"], record["nonce"].encode()) + before = turn_ids(api, record) + resumed = "/workspace/outputs/resumed-" + record["nonce"] + ".txt" + prompt = ("Recall the exact conversation-only remember- token from our first turn without reading " + "files or rerunning earlier commands. Write only that token, with no newline, to " + resumed + + " using native tools. Then reply with that token. Do not delegate.") + submit(api, record, prompt) + turn = wait_turn(api, record, before, timeout) + stored = verify_answer(api, record, turn["id"], record["memory"]) + record["resumed_artifact"] = verify_output(api, record, turn["id"], resumed, record["memory"].encode()) + record["items_digest"] = digest(stored) + + +def cancellation(api, record, timeout): + path = session_path(record) + require(api.get(path, "session.retrieve").get("status") == "idle", "cancel_requires_idle_session") + before = turn_ids(api, record) + directory = "/workspace" + prefix = directory + "/release-cancel-" + record["nonce"] + command = ("printf start >> " + shlex.quote(prefix + "-starts") + + "; while true; do printf x >> " + shlex.quote(prefix + "-ticks") + "; sleep 1; done") + submit(api, record, "Run this command exactly once with your native shell tool in the foreground: " + + command + ". Wait for it; it runs until cancelled. Do not background, delegate or retry.") + deadline, observed = time.monotonic() + timeout, 0 + while time.monotonic() < deadline: + turns = [row for row in api.listing(path + "/turns", "turns.list") if row.get("id") not in before] + require(len(turns) <= 1 and not any(row.get("status") in TERMINAL for row in turns), + "cancel_work_finished_before_cancellation") + rows = file_rows(api, record, directory) + sizes = {row.get("path"): row.get("size_bytes") for row in rows} + observed = sizes.get(prefix + "-ticks", 0) + if type(observed) is int and observed >= 3: + require(sizes.get(prefix + "-starts") == 5, "cancel_command_repeated") + break + time.sleep(1) + else: + raise Failure("native_cancel_effects_not_observed") + api.request("POST", path + "/events", "events.cancel", + body={"events": [{"type": "agent.session.input.cancel"}]}, key=uuid.uuid4().hex) + wait_turn(api, record, before, timeout, "cancelled") + def effects(): + return {row.get("path"): row.get("size_bytes") for row in file_rows(api, record, directory)} + settled = effects() + require(settled.get(prefix + "-starts") == 5 and settled.get(prefix + "-ticks", 0) >= observed, + "cancel_effect_metadata_mismatch") + time.sleep(3) + require(effects() == settled, "native_effects_continued_after_cancel") + record["cancel_effects"] = {"ticks_before_cancel": observed, + "ticks_after_cancel": settled[prefix + "-ticks"], "stable_seconds": 3} + record["items_digest"] = digest(items(api, record)) + + +def settle_failed_work(api, record): + """Try to stop only this acceptance Session; preserve the original failure.""" + if "session_id" not in record: + return "no_known_session" + try: + path = session_path(record) + current = api.get(path, "failure_cleanup.session") + if current.get("status") in {"idle", "failed"}: + return "already_settled" + api.request("POST", path + "/events", "failure_cleanup.cancel", + body={"events": [{"type": "agent.session.input.cancel"}]}, key=uuid.uuid4().hex) + deadline = time.monotonic() + 30 + while time.monotonic() < deadline: + current = api.get(path, "failure_cleanup.session") + if current.get("status") in {"idle", "failed"}: + return "settled_after_cancel" + time.sleep(1) + except Exception: + pass + return "unresolved_operator_cleanup_required" + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument("stage", choices=("run", "continue", "cancel")) + parser.add_argument("--base-url", required=True) + parser.add_argument("--caller-key-file", required=True, type=Path) + parser.add_argument("--model-config-file", required=True, type=Path) + parser.add_argument("--report-dir", required=True, type=Path) + parser.add_argument("--timeout", type=int, default=300) + parser.add_argument("--after-restart", action="store_true", help="Record an operator-performed restart before continue.") + args = parser.parse_args() + require(args.timeout > 0 and (not args.after_restart or args.stage == "continue"), "invalid_stage_options") + base = validate_origin(args.base_url) + token, models, secrets = settings(args) + require(args.report_dir.is_absolute() and not args.report_dir.is_symlink(), "absolute_report_directory_required") + args.report_dir.mkdir(mode=0o700, parents=True, exist_ok=True) + require(args.report_dir.stat().st_mode & 0o077 == 0, "report_directory_must_be_private") + failed = False + for harness in HARNESSES: + location = args.report_dir / (harness + ".json") + record = json.loads(private_file(location)) if location.exists() else {"harness": harness, "stages": {}} + require(record.get("harness") == harness, "report_harness_mismatch") + require(args.stage not in record["stages"], "stage_already_attempted_use_new_report_directory") + stage = {"passed": False, "requests": []} + record["stages"][args.stage] = stage + def save(): + serialized = json.dumps(record, indent=2) + "\n" + require(not any(secret in serialized for secret in secrets), "credential_detected_in_evidence") + temp = location.with_suffix(".tmp") + fd = os.open(temp, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) + try: + with os.fdopen(fd, "w") as output: + output.write(serialized) + os.replace(temp, location) + finally: + temp.unlink(missing_ok=True) + api = API(base, token, stage["requests"]) + try: + save() + if args.stage == "run": + run(api, record, models[harness], args.timeout, save) + else: + require(record["stages"].get("run", {}).get("passed"), "initial_stage_did_not_pass") + if args.stage == "continue": + stage["operator_reported_restart"] = args.after_restart + continuation(api, record, args.timeout) + else: + cancellation(api, record, args.timeout) + stage["passed"] = True + except Exception as error: + failed = True + stage["failure"] = str(error) if isinstance(error, Failure) else "unexpected_local_or_response_error" + stage["cleanup"] = settle_failed_work(api, record) + save() + print(harness + ": " + args.stage + " " + ("passed" if stage["passed"] else "FAILED"), flush=True) + return 1 if failed else 0 + + +if __name__ == "__main__": + try: + raise SystemExit(main()) + except Exception as error: + label = str(error) if isinstance(error, Failure) else "invalid_configuration_or_local_io" + raise SystemExit("Acceptance stopped: " + label + "; private values and response bodies withheld.") from None From 55975e35f1df12019972588049351ea482924e1b Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 15:06:01 +0800 Subject: [PATCH 08/17] Invoke existing Runtime builders through Bash --- scripts/build-core-distribution.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/build-core-distribution.sh b/scripts/build-core-distribution.sh index e4d0dc701..fb8686300 100755 --- a/scripts/build-core-distribution.sh +++ b/scripts/build-core-distribution.sh @@ -119,7 +119,7 @@ else for harness in codex claude mcode; do script="scripts/build-$harness-runtime.sh" if [[ "$harness" == codex ]]; then script=scripts/build-agents-runtime.sh; fi - AGENTS_RUNTIME_BUILD_DIR="$stage/$harness" "$script" + AGENTS_RUNTIME_BUILD_DIR="$stage/$harness" bash "$script" docker build --platform linux/amd64 --iidfile "$stage/$harness.id" \ --label "org.opencontainers.image.revision=$revision" "$stage/$harness" done From cc9599b1c6eabed2e053810a792fa7866507a0e2 Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 15:06:50 +0800 Subject: [PATCH 09/17] Correct pinned SDK example and sandbox initialization wording --- docs/getting-started/install.md | 3 ++- docs/getting-started/quickstart.md | 13 +++++++++---- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/docs/getting-started/install.md b/docs/getting-started/install.md index 6c74e0385..3814931c5 100644 --- a/docs/getting-started/install.md +++ b/docs/getting-started/install.md @@ -2,7 +2,8 @@ Install one matching Parsar Core distribution. By default it starts PostgreSQL, Core and the existing Web console, prepares microsandbox and imports the Runtime -image. Core creates a sandbox and starts its daemon when a Session needs it. +image. When a Session needs a sandbox, Core asks its Provider to create one from +that image and initializes the colocated daemon, native harness and workspace. No model key, Environment wizard or sample task is required during installation. ## Host requirements diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md index 56073b255..06430d229 100644 --- a/docs/getting-started/quickstart.md +++ b/docs/getting-started/quickstart.md @@ -32,8 +32,9 @@ environment. Installation has no mandatory sample task. ## Run a Session when you are ready -Core prepares the sandbox and starts its Runtime. You do not install or start a -separate daemon for a Core-managed Session. The public discriminator remains +Core creates the sandbox through its Provider using the prepared Runtime image, +then initializes the daemon, native harness and workspace inside it. You do not +install or start a separate daemon for a Core-managed Session. The public discriminator remains `openai_hosted`; in this deployment it means the sandbox managed by Parsar Core. The installation's provider can be microsandbox or Docker. @@ -44,11 +45,13 @@ endpoint and key through your application's private configuration: import os session = client.beta.agents.sessions.create( - agent={"model": os.environ["MODEL_NAME"]}, environment={"type": "openai_hosted"}, input="Create /workspace/hello.txt with a short greeting, then describe it.", extra_body={ - "agent": {"x_agents_core": {"harness": "codex"}}, + "agent": { + "model": os.environ["MODEL_NAME"], + "x_agents_core": {"harness": "codex"}, + }, "x_agents_core": { "model_provider": { "protocol": "responses", @@ -67,6 +70,8 @@ model configuration. They are not fields in the official SDK 3.13.0 protocol. The service encrypts model configuration with tenant/Session binding and never returns the secret through public resource reads. Keep the installation's credential encryption key and database together across restarts. +Keep the complete `agent` object together: SDK 3.13.0 replaces an ordinary body +field with the corresponding `extra_body` field rather than merging nested fields. The model must support the selected harness's native protocol: From ef8652bb5bbb90da1bd88ec43ecd10630221b038 Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 15:12:03 +0800 Subject: [PATCH 10/17] Support proxy networks during distribution image builds --- scripts/build-core-distribution.sh | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/scripts/build-core-distribution.sh b/scripts/build-core-distribution.sh index fb8686300..f3ba2d344 100755 --- a/scripts/build-core-distribution.sh +++ b/scripts/build-core-distribution.sh @@ -25,6 +25,21 @@ fi for command in docker go node pnpm python3 curl tar sha256sum; do command -v "$command" >/dev/null done +build_network="${CORE_DISTRIBUTION_BUILD_NETWORK:-default}" +case "$build_network" in + default|host|none) ;; + *) printf 'CORE_DISTRIBUTION_BUILD_NETWORK must be default, host, or none\n' >&2; exit 1 ;; +esac +build_image() { + # Docker's predefined proxy arguments are build-only; no Dockerfile ARG or ENV + # declaration persists the operator's network configuration in the images. + docker build --network "$build_network" \ + --build-arg HTTP_PROXY --build-arg HTTPS_PROXY --build-arg ALL_PROXY --build-arg NO_PROXY \ + --build-arg "http_proxy=${http_proxy:-${HTTP_PROXY:-}}" \ + --build-arg "https_proxy=${https_proxy:-${HTTPS_PROXY:-}}" \ + --build-arg "all_proxy=${all_proxy:-${ALL_PROXY:-}}" \ + --build-arg "no_proxy=${no_proxy:-${NO_PROXY:-}}" "$@" +} require_clean_source() { if [[ -n "$(git -C "$repo_root" status --porcelain --untracked-files=all)" ]]; then @@ -85,7 +100,7 @@ else python3 scripts/core-distribution-manifest.py extract-runtime "$msb_archive" "$stage/core/microsandbox" fi cp deploy/distribution/Dockerfile "$stage/core/Dockerfile" -docker build --platform linux/amd64 --iidfile "$stage/core.id" \ +build_image --platform linux/amd64 --iidfile "$stage/core.id" \ --label "org.opencontainers.image.revision=$revision" "$stage/core" core_image="$(cat "$stage/core.id")" # Fail at packaging time if the helper or runtime requires unavailable host libraries. @@ -97,7 +112,7 @@ pnpm install --frozen-lockfile AGENTS_CORE_WEB_OPENAI_HOSTED_SESSIONS=1 AGENTS_CORE_WEB_ENVIRONMENT_FILES=1 pnpm build:web cp -R apps/web/dist "$stage/web/dist" cp services/core-console/Dockerfile "$stage/web/Dockerfile" -docker build --platform linux/amd64 --iidfile "$stage/web.id" \ +build_image --platform linux/amd64 --iidfile "$stage/web.id" \ --label "org.opencontainers.image.revision=$revision" "$stage/web" export AGENTS_EXECUTOR_BUILD_DIR="$stage/helpers" @@ -120,7 +135,7 @@ else script="scripts/build-$harness-runtime.sh" if [[ "$harness" == codex ]]; then script=scripts/build-agents-runtime.sh; fi AGENTS_RUNTIME_BUILD_DIR="$stage/$harness" bash "$script" - docker build --platform linux/amd64 --iidfile "$stage/$harness.id" \ + build_image --platform linux/amd64 --iidfile "$stage/$harness.id" \ --label "org.opencontainers.image.revision=$revision" "$stage/$harness" done codex_image="$(cat "$stage/codex.id")" @@ -139,7 +154,7 @@ for harness in codex claude mcode; do done mkdir "$stage/combined" cp deploy/distribution/Runtime.Dockerfile "$stage/combined/Dockerfile" -docker build --platform linux/amd64 --iidfile "$stage/runtime.id" \ +build_image --platform linux/amd64 --iidfile "$stage/runtime.id" \ --label "org.opencontainers.image.revision=$revision" \ --build-arg "CODEX_IMAGE=${image_tags[0]}" --build-arg "CLAUDE_IMAGE=${image_tags[1]}" \ --build-arg "MCODE_IMAGE=${image_tags[2]}" "$stage/combined" From e58094c0d5cd166172721a50e7790e499816d226 Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 15:18:14 +0800 Subject: [PATCH 11/17] Report unhealthy or missing installation services as failed status --- deploy/install/install.py | 7 ++++++- deploy/install/test_install.py | 18 ++++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/deploy/install/install.py b/deploy/install/install.py index 71ffea352..39201e457 100644 --- a/deploy/install/install.py +++ b/deploy/install/install.py @@ -140,7 +140,12 @@ def status(root, state): output = compose(root, "ps", "--all", "--format", "json", capture_output=True, text=True).stdout # Compose versions may return one array or one object per line. rows = json.loads(output) if output.lstrip().startswith("[") else [json.loads(line) for line in output.splitlines() if line] - healthy = True + required = {"web"} if state["mode"] == "web-only" else {"database", "core"} + if state["mode"] == "all": + required.add("web") + observed = {row["Service"]: row for row in rows} + healthy = all(name in observed and observed[name]["State"] == "running" + and observed[name].get("Health", "") in ("", "healthy") for name in required) for row in rows: print(f'{row["Service"]}: {row["State"]} {row.get("Health", "")}') if state["mode"] != "web-only": diff --git a/deploy/install/test_install.py b/deploy/install/test_install.py index c265786fc..942013886 100644 --- a/deploy/install/test_install.py +++ b/deploy/install/test_install.py @@ -316,6 +316,24 @@ def test_core_connection_rejects_remote_cleartext_and_embedded_credentials(self) self.args("--web-only", "--core-url", url, "--core-token-file", str(source)) self.assertNotIn("synthetic-secret", output.getvalue()) + def test_status_rejects_failed_database_even_while_http_processes_are_alive(self): + services = [{"Service": name, "State": "running", "Health": ""} + for name in ("database", "core", "web")] + for database in ({"State": "running", "Health": "unhealthy"}, {"State": "exited"}, None): + rows = services[1:] + ([{**services[0], **database}] if database else []) + with self.subTest(database=database), \ + mock.patch.object(install, "compose", return_value=SimpleNamespace(stdout=json.dumps(rows))), \ + mock.patch.object(install, "wait_http", return_value=True), \ + contextlib.redirect_stdout(io.StringIO()), self.assertRaises(install.InstallError): + install.status(self.root, {"mode": "all", "core_port": 8091, "web_port": 8080}) + + def test_status_accepts_web_only_without_database_or_core_services(self): + rows = [{"Service": "web", "State": "running", "Health": ""}] + with mock.patch.object(install, "compose", return_value=SimpleNamespace(stdout=json.dumps(rows))), \ + mock.patch.object(install, "wait_http", return_value=True), \ + contextlib.redirect_stdout(io.StringIO()): + install.status(self.root, {"mode": "web-only", "web_port": 8080}) + if __name__ == "__main__": unittest.main() From e8450a837840842639d6118c59759383c2687345 Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 15:26:49 +0800 Subject: [PATCH 12/17] Include native Core and microsandbox release payload --- scripts/build-core-distribution.sh | 2 ++ scripts/core-distribution-manifest.py | 5 ++++- scripts/core-distribution-manifest.test.py | 5 +++++ 3 files changed, 11 insertions(+), 1 deletion(-) diff --git a/scripts/build-core-distribution.sh b/scripts/build-core-distribution.sh index f3ba2d344..8d96d68b8 100755 --- a/scripts/build-core-distribution.sh +++ b/scripts/build-core-distribution.sh @@ -99,6 +99,8 @@ if [[ ! -f "$msb_archive" ]]; then else python3 scripts/core-distribution-manifest.py extract-runtime "$msb_archive" "$stage/core/microsandbox" fi +mkdir -p "$bundle/native" +cp -R "$stage/core/bin" "$stage/core/microsandbox" "$bundle/native/" cp deploy/distribution/Dockerfile "$stage/core/Dockerfile" build_image --platform linux/amd64 --iidfile "$stage/core.id" \ --label "org.opencontainers.image.revision=$revision" "$stage/core" diff --git a/scripts/core-distribution-manifest.py b/scripts/core-distribution-manifest.py index b8c6cd295..eaaa2eb39 100644 --- a/scripts/core-distribution-manifest.py +++ b/scripts/core-distribution-manifest.py @@ -113,7 +113,10 @@ def archive(bundle, epoch): relative = path.relative_to(bundle) info = tarfile.TarInfo(bundle.name + "/" + relative.as_posix()) info.size = path.stat().st_size - info.mode = 0o755 if relative.as_posix() == "install.sh" else 0o644 + if relative.parts[0] == "native": + info.mode = path.stat().st_mode & 0o777 + else: + info.mode = 0o755 if relative.as_posix() == "install.sh" else 0o644 info.mtime = int(epoch) with path.open("rb") as stream: tar.addfile(info, stream) diff --git a/scripts/core-distribution-manifest.test.py b/scripts/core-distribution-manifest.test.py index 3468ecabd..cfcf950e3 100644 --- a/scripts/core-distribution-manifest.test.py +++ b/scripts/core-distribution-manifest.test.py @@ -59,6 +59,10 @@ def test_wrong_guest_platform_rejected(self): distribution.manifest(self.bundle, self.stage, "commit", "tree") def test_archive_reproducible_and_installer_executable(self): + native = self.bundle / "native/bin/agents-api" + native.parent.mkdir(parents=True) + native.write_bytes(b"native executable") + native.chmod(0o555) distribution.manifest(self.bundle, self.stage, "commit", "tree") distribution.archive(self.bundle, "1700000000") archive = self.bundle.with_name(self.bundle.name + ".tar.gz") @@ -69,6 +73,7 @@ def test_archive_reproducible_and_installer_executable(self): with tarfile.open(archive) as contents: self.assertEqual(contents.getmember(self.bundle.name + "/install.sh").mode, 0o755) self.assertEqual(contents.getmember(self.bundle.name + "/manifest.json").mode, 0o644) + self.assertEqual(contents.getmember(self.bundle.name + "/native/bin/agents-api").mode, 0o555) def test_bad_upstream_checksum_does_not_extract(self): archive = self.stage / "untrusted.tar.gz" From 14cecbabd7d4d2c550b1dce80852805d1bbcbcb0 Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 15:36:21 +0800 Subject: [PATCH 13/17] Add native Core user service packaging draft --- deploy/install/native_service.py | 194 +++++++++++++++++++++++ deploy/install/test_native_service.py | 213 ++++++++++++++++++++++++++ 2 files changed, 407 insertions(+) create mode 100644 deploy/install/native_service.py create mode 100644 deploy/install/test_native_service.py diff --git a/deploy/install/native_service.py b/deploy/install/native_service.py new file mode 100644 index 000000000..dd6801e7e --- /dev/null +++ b/deploy/install/native_service.py @@ -0,0 +1,194 @@ +"""Install only the native Core user service; Core retains Runtime ownership.""" + +import hashlib +import os +from pathlib import Path +import platform +import re +import shutil +import subprocess +import tempfile + + +REQUIRED = ("bin/agents-api", "bin/agents-api-microsandbox-provider", + "microsandbox/msb", "microsandbox/libkrunfw.so.5.6.1") + + +def is_native(state): + return state["mode"] != "web-only" and state["provider"] == "microsandbox" + + +def _unit_name(state): + project = state.get("project", "") + if not isinstance(project, str) or not re.fullmatch(r"parsar-[0-9a-f]{10}", project): + raise RuntimeError("Native Core requires its installation's generated project name") + return project + "-core.service" + + +def _path(value): + path = Path(value) + text = str(path) + # EnvironmentFile accepts glob patterns and WorkingDirectory is not a shell + # word. Reject ambiguous paths instead of expanding another file or unit line. + if (not path.is_absolute() or path.resolve() != path or text != text.strip() + or any(ord(char) < 32 for char in text) or any(char in text for char in "\\*?[]")): + raise RuntimeError("Native Core paths must be canonical absolute paths without control characters, backslashes or wildcards") + return path + + +def _run(arguments, failure): + try: + return subprocess.run(arguments, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, text=True, timeout=30, check=False) + except (OSError, subprocess.SubprocessError): + raise RuntimeError(failure) from None + + +def _checked(arguments, failure): + result = _run(arguments, failure) + if result.returncode: + raise RuntimeError(failure) + return result.stdout.strip() + + +def _files(native): + if native.is_symlink() or not native.is_dir(): + raise RuntimeError("The distribution is missing its native Core payload") + files = {} + for path in native.rglob("*"): + if path.is_symlink() or not (path.is_dir() or path.is_file()): + raise RuntimeError("Native Core payload must contain only regular files and directories") + if path.is_file(): + files[str(path.relative_to(native))] = path + if not set(REQUIRED).issubset(files): + raise RuntimeError("The distribution is missing a required native Core executable or firmware") + return files + + +def preflight(bundle): + """Check host prerequisites without launching a helper operation or VM.""" + if platform.system() != "Linux": + raise RuntimeError("Native microsandbox installation requires Linux") + if not os.access("/dev/kvm", os.R_OK | os.W_OK): + raise RuntimeError("Native microsandbox requires read/write access to /dev/kvm; ask the host administrator to grant access") + _checked(["systemctl", "--user", "show", "--property=Version", "--value"], + "The systemd user manager is unavailable; establish a user session before installation") + linger = _checked(["loginctl", "show-user", str(os.getuid()), "--property=Linger", "--value"], + "Cannot check user lingering; ask the host administrator to configure it") + if linger != "yes": + raise RuntimeError("User lingering must be enabled by the host administrator before native Core installation") + try: + native = _path(bundle) / "native" + files = _files(native) + for name in REQUIRED[1:]: + with files[name].open("rb") as stream: + if stream.read(4) != b"\x7fELF": + raise RuntimeError("Native Core payload must contain Linux ELF binaries") + result = _run(["ldd", str(files[name])], "Cannot check native Core shared libraries") + diagnostic = result.stdout + result.stderr + static = "statically linked" in diagnostic or "not a dynamic executable" in diagnostic + if "not found" in diagnostic or (result.returncode and not static): + raise RuntimeError("Native Core shared libraries cannot load on this host; install the required host libraries") + except OSError: + raise RuntimeError("Cannot inspect the native Core distribution") from None + + +def _digest(path): + digest = hashlib.sha256() + with path.open("rb") as stream: + for block in iter(lambda: stream.read(1024 * 1024), b""): + digest.update(block) + return digest.digest() + + +def _environment(values): + lines = [] + for key, value in values.items(): + if (not isinstance(key, str) or not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", key) + or not isinstance(value, str) or any(char in value for char in "\x00\r\n")): + raise RuntimeError("Native Core environment requires valid names and single-line string values") + # EnvironmentFile double quotes preserve these shell metacharacters. + escaped = re.sub(r'([\\"`$])', r'\\\1', value) + lines.append(key + '="' + escaped + '"\n') + return "".join(sorted(lines)) + + +def _private_write(path, content): + if path.is_symlink() or (path.exists() and not path.is_file()): + raise RuntimeError("Native Core configuration must be a regular private file") + if path.exists() and path.read_text() == content: + os.chmod(path, 0o600) + return + descriptor, temporary = tempfile.mkstemp(prefix=".core-", dir=path.parent) + try: + with os.fdopen(descriptor, "w", encoding="utf-8") as stream: + stream.write(content) + os.replace(temporary, path) + finally: + if os.path.exists(temporary): + os.unlink(temporary) + + +def prepare(root, state, bundle, environment): + if not is_native(state): + return + try: + root, bundle = _path(root), _path(bundle) + unit_name = _unit_name(state) + environment_text = _environment(environment) + source, target = bundle / "native", root / "native" + incoming = _files(source) + if target.exists() or target.is_symlink(): + installed = _files(target) + if incoming.keys() != installed.keys() or any(_digest(path) != _digest(installed[name]) for name, path in incoming.items()): + raise RuntimeError("Installed native Core files differ; preserve the installation and follow the upgrade guide") + else: + root.mkdir(parents=True, mode=0o700, exist_ok=True) + with tempfile.TemporaryDirectory(prefix=".native-", dir=root) as temporary: + staged = Path(temporary) / "native" + shutil.copytree(source, staged) + os.replace(staged, target) + for path in [target, *target.rglob("*")]: + executable = path.is_dir() or path.parent == target / "bin" or path == target / "microsandbox/msb" + os.chmod(path, 0o700 if executable else 0o600) + config = root / "config" + if config.is_symlink(): + raise RuntimeError("Native Core configuration directory must not be a symlink") + config.mkdir(mode=0o700, exist_ok=True) + os.chmod(config, 0o700) + # ':' disables command-line environment substitution. The executable is + # still Core itself; no shell, wrapper or provider shutdown hook is used. + executable = str(target / "bin/agents-api").replace("%", "%%").replace('"', '\\"') + unit = ("[Unit]\nDescription=Parsar Core\n\n[Service]\nType=exec\n" + + 'ExecStart=:"' + executable + '"\n' + + "WorkingDirectory=" + str(root).replace("%", "%%") + "\n" + + "EnvironmentFile=" + str(config / "core.env").replace("%", "%%") + "\n" + + "Restart=on-failure\nKillMode=process\nUMask=0077\n\n[Install]\nWantedBy=default.target\n") + _private_write(config / "core.env", environment_text) + _private_write(config / unit_name, unit) + except (OSError, UnicodeError): + raise RuntimeError("Cannot prepare private native Core files; existing Runtime state was not removed") from None + + +def start(root, state): + if not is_native(state): + return + unit = _path(root) / "config" / _unit_name(state) + if unit.is_symlink() or not unit.is_file(): + raise RuntimeError("Native Core service must be prepared before starting it") + _checked(["systemctl", "--user", "daemon-reload"], "Cannot reload the systemd user manager") + _checked(["systemctl", "--user", "enable", "--now", str(unit)], "Cannot enable or start this installation's native Core service") + + +def stop(root, state): + if is_native(state): + _path(root) + _checked(["systemctl", "--user", "stop", _unit_name(state)], "Cannot stop this installation's native Core service") + + +def active(state): + if not is_native(state): + return False + result = _run(["systemctl", "--user", "is-active", "--quiet", _unit_name(state)], + "Cannot query this installation's native Core service") + return result.returncode == 0 diff --git a/deploy/install/test_native_service.py b/deploy/install/test_native_service.py new file mode 100644 index 000000000..3d97c2a09 --- /dev/null +++ b/deploy/install/test_native_service.py @@ -0,0 +1,213 @@ +"""Verify native packaging without starting a service or Runtime.""" + +import configparser +from pathlib import Path +import stat +import subprocess +import tempfile +import unittest +from unittest import mock + +import native_service as service + + +class NativeServiceTests(unittest.TestCase): + def setUp(self): + temporary_root = Path.home() / ".parsar/tests/install-native" + temporary_root.mkdir(parents=True, exist_ok=True) + temporary = tempfile.TemporaryDirectory(dir=temporary_root) + self.addCleanup(temporary.cleanup) + self.directory = Path(temporary.name).resolve() + self.root = self.directory / 'install space %n $HOME "quote"' + self.bundle = self.directory / "bundle" + self.state = {"mode": "all", "provider": "microsandbox", "project": "parsar-0123456789"} + self.environment = {"DATABASE_URL": 'synthetic:"quoted"\\path$HOME`value`%n', "PORT": "8091"} + for name in service.REQUIRED: + path = self.bundle / "native" / name + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(b"\x7fELFfixture-" + name.encode()) + self.commands = mock.patch.object(service.subprocess, "run") + self.run = self.commands.start() + self.addCleanup(self.commands.stop) + self.run.return_value = subprocess.CompletedProcess([], 0, "", "") + + def prepare(self): + service.prepare(self.root, self.state, self.bundle, self.environment) + + def unit_path(self): + return self.root / "config" / "parsar-0123456789-core.service" + + def host_ready(self): + for patch in (mock.patch.object(service.platform, "system", return_value="Linux"), + mock.patch.object(service.os, "access", return_value=True)): + patch.start() + self.addCleanup(patch.stop) + + def result(arguments, **kwargs): + output = "yes\n" if arguments[0] == "loginctl" else "" + return subprocess.CompletedProcess(arguments, 0, output, "") + + self.run.side_effect = result + + def test_only_core_microsandbox_uses_native_service(self): + for mode, provider, expected in (("all", "microsandbox", True), + ("core-only", "microsandbox", True), + ("web-only", "microsandbox", False), + ("all", "docker", False)): + with self.subTest(mode=mode, provider=provider): + state = dict(self.state, mode=mode, provider=provider) + self.assertEqual(service.is_native(state), expected) + if not expected: + service.prepare(self.root, state, self.bundle, {}) + service.start(self.root, state) + service.stop(self.root, state) + self.assertFalse(service.active(state)) + self.run.assert_not_called() + + def test_prepare_preserves_direct_core_and_runtime_process_lifetime(self): + self.prepare() + unit = configparser.ConfigParser(interpolation=None) + unit.read(self.unit_path()) + directives = unit["Service"] + executable = str(self.root / "native/bin/agents-api").replace("%", "%%").replace('"', '\\"') + self.assertEqual(directives["ExecStart"], ':"' + executable + '"') + self.assertEqual(directives["WorkingDirectory"], str(self.root).replace("%", "%%")) + self.assertEqual(directives["EnvironmentFile"], str(self.root / "config/core.env").replace("%", "%%")) + self.assertEqual(directives["KillMode"], "process") + self.assertEqual(directives["Restart"], "on-failure") + self.assertEqual(directives["UMask"], "0077") + self.assertNotIn("ExecStop", directives) + self.assertNotIn("ExecStopPost", directives) + self.assertNotIn("DATABASE_URL", self.unit_path().read_text()) + self.run.assert_not_called() + + def test_private_files_and_environment_literal_values(self): + self.prepare() + env = self.root / "config/core.env" + self.assertEqual(env.read_text(), 'DATABASE_URL="synthetic:\\"quoted\\"\\\\path\\$HOME\\`value\\`%n"\nPORT="8091"\n') + for path in (env, self.unit_path(), self.root / "native/microsandbox/libkrunfw.so.5.6.1"): + self.assertEqual(stat.S_IMODE(path.stat().st_mode), 0o600) + for path in (self.root / "config", self.root / "native/bin/agents-api", + self.root / "native/bin/agents-api-microsandbox-provider", self.root / "native/microsandbox/msb"): + self.assertEqual(stat.S_IMODE(path.stat().st_mode), 0o700) + + def test_repeat_keeps_binary_and_private_file_inodes(self): + self.prepare() + paths = [self.root / "native" / name for name in service.REQUIRED] + paths += [self.root / "config/core.env", self.unit_path()] + original = [(path.stat().st_ino, path.stat().st_mtime_ns, path.read_bytes()) for path in paths] + self.prepare() + self.assertEqual(original, [(path.stat().st_ino, path.stat().st_mtime_ns, path.read_bytes()) for path in paths]) + + def test_changed_payload_refuses_without_overwriting_installed_binary(self): + self.prepare() + installed = self.root / "native/bin/agents-api" + before = installed.read_bytes() + (self.bundle / "native/bin/agents-api").write_bytes(b"changed") + with self.assertRaisesRegex(RuntimeError, "differ"): + self.prepare() + self.assertEqual(installed.read_bytes(), before) + self.run.assert_not_called() + + def test_invalid_environment_is_rejected_before_writes(self): + for environment in ({"KEY": "secret\nInjected=value"}, {"KEY": "secret\x00"}, + {"KEY": "secret\r"}, {"BAD=KEY": "secret"}, {1: "secret", "OK": "value"}): + with self.subTest(environment=environment): + with self.assertRaises(RuntimeError) as error: + service.prepare(self.root, self.state, self.bundle, environment) + self.assertNotIn("secret", str(error.exception)) + self.assertFalse(self.root.exists()) + + def test_ambiguous_paths_and_foreign_units_refuse(self): + for suffix in ("bad\npath", "bad*path", "bad\\path"): + with self.assertRaises(RuntimeError): + service.prepare(self.directory / suffix, self.state, self.bundle, {}) + for project in ("other-service", "../parsar-0123456789", "parsar-0123456789\n"): + state = dict(self.state, project=project) + with self.assertRaises(RuntimeError): + service.prepare(self.root, state, self.bundle, {}) + with self.assertRaises(RuntimeError): + service.stop(self.root, state) + with self.assertRaises(RuntimeError): + service.active(state) + self.run.assert_not_called() + + def test_symlink_payload_is_not_followed(self): + path = self.bundle / "native/bin/agents-api" + path.unlink() + outside = self.directory / "outside" + outside.write_bytes(b"unchanged") + path.symlink_to(outside) + with self.assertRaisesRegex(RuntimeError, "regular"): + self.prepare() + self.assertEqual(outside.read_bytes(), b"unchanged") + self.assertFalse(self.root.exists()) + + def test_symlink_configuration_does_not_overwrite_target(self): + self.prepare() + env = self.root / "config/core.env" + env.unlink() + outside = self.directory / "outside" + outside.write_text("unchanged") + env.symlink_to(outside) + with self.assertRaisesRegex(RuntimeError, "regular"): + self.prepare() + self.assertEqual(outside.read_text(), "unchanged") + + def test_preflight_checks_host_and_libraries_without_running_provider(self): + self.host_ready() + service.preflight(self.bundle) + commands = [call.args[0] for call in self.run.call_args_list] + self.assertEqual(commands[:2], [["systemctl", "--user", "show", "--property=Version", "--value"], + ["loginctl", "show-user", str(service.os.getuid()), "--property=Linger", "--value"]]) + self.assertEqual(commands[2:], [["ldd", str(self.bundle / "native" / name)] for name in service.REQUIRED[1:]]) + + def test_preflight_requires_kvm_access_and_linger(self): + self.host_ready() + with mock.patch.object(service.os, "access", return_value=False): + with self.assertRaisesRegex(RuntimeError, "/dev/kvm"): + service.preflight(self.bundle) + self.run.assert_not_called() + self.run.side_effect = lambda arguments, **kwargs: subprocess.CompletedProcess(arguments, 0, "no\n", "") + with self.assertRaisesRegex(RuntimeError, "lingering"): + service.preflight(self.bundle) + self.assertFalse(any(call.args[0][0] == "ldd" for call in self.run.call_args_list)) + + def test_failed_commands_do_not_disclose_diagnostics(self): + self.host_ready() + self.run.side_effect = None + self.run.return_value = subprocess.CompletedProcess([], 1, "synthetic-secret", "synthetic-secret") + with self.assertRaises(RuntimeError) as error: + service.preflight(self.bundle) + self.assertNotIn("synthetic-secret", str(error.exception)) + self.prepare() + self.run.side_effect = OSError("synthetic-secret") + with self.assertRaises(RuntimeError) as error: + service.start(self.root, self.state) + self.assertNotIn("synthetic-secret", str(error.exception)) + + def test_missing_shared_library_refuses(self): + self.host_ready() + self.run.side_effect = lambda arguments, **kwargs: subprocess.CompletedProcess( + arguments, 0, "libc.so => not found" if arguments[0] == "ldd" else "yes\n", "") + with self.assertRaisesRegex(RuntimeError, "shared libraries"): + service.preflight(self.bundle) + + def test_commands_target_only_this_installation(self): + self.prepare() + service.start(self.root, self.state) + service.stop(self.root, self.state) + self.assertTrue(service.active(self.state)) + self.run.return_value = subprocess.CompletedProcess([], 3, "", "") + self.assertFalse(service.active(self.state)) + self.assertEqual([call.args[0] for call in self.run.call_args_list], [ + ["systemctl", "--user", "daemon-reload"], + ["systemctl", "--user", "enable", "--now", str(self.unit_path())], + ["systemctl", "--user", "stop", "parsar-0123456789-core.service"], + ["systemctl", "--user", "is-active", "--quiet", "parsar-0123456789-core.service"], + ["systemctl", "--user", "is-active", "--quiet", "parsar-0123456789-core.service"], + ]) + + +if __name__ == "__main__": + unittest.main() From 997cecb90d24f79c9dd2f51b3e29eac67d5a0a1d Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 15:39:53 +0800 Subject: [PATCH 14/17] Save native microsandbox installation integration draft --- CONTRIBUTING.md | 20 ++++++++--- README.md | 4 +-- deploy/install/configuration.py | 55 ++++++++++++++++++------------ deploy/install/install.py | 54 +++++++++++++++++++++-------- deploy/install/test_install.py | 47 +++++++++++++------------ docs/getting-started/install.md | 24 +++++++------ docs/getting-started/operations.md | 20 ++++++++--- scripts/build-core-distribution.sh | 2 +- 8 files changed, 146 insertions(+), 80 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9d295d3af..f919c455c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1317,7 +1317,7 @@ optional API examples and service diagnostics live in `docs/getting-started/`. `make build-core-distribution` builds from clean committed source and reuses the existing API, Runtime, SDK, helper and Web builders. Artifacts record source and immutable image identities, the actual Runtime manifest digest, checksums and -microsandbox runtime/firmware hashes. Release generation is not publication or +microsandbox runtime/firmware hashes and executable native payloads. Release generation is not publication or qualification. A release must be tested from fresh extraction with real models; no synthetic result may substitute for native execution acceptance. @@ -1325,9 +1325,16 @@ The first installer targets a trusted Linux amd64 Docker host. It installs a private dedicated PostgreSQL service and separate Core and console services. Default sandbox placement is microsandbox; `--provider docker` selects the existing Docker provider. Missing KVM fails without changing that choice. -The distribution Core image contains the native glibc helper and pinned msb -runtime/firmware, with only KVM device access for microsandbox or the canonical -Docker socket for Docker. This does not put a harness or model loop in Core. +The distribution supplies native Core/helper binaries and pinned msb runtime and +firmware. For microsandbox, Core is a native systemd user service with direct +`ExecStart` and `KillMode=process`: its restart must preserve the Provider's resident +microVM/helper processes. Never package those processes inside Core's container +PID namespace, kill their process group on Core stop, or add recovery mechanisms to +compensate for that packaging. User KVM access, the Linux runtime libraries and +linger are explicit prerequisites. For Docker, Core runs in Compose with the +canonical Docker socket. PostgreSQL/Web use Compose in either case; native Core +and its Web proxy use loopback, with a private PostgreSQL port. This packaging +choice does not change either Provider's execution contract. The basic distroless API image and binary builds remain independent artifacts. One Runtime image contains the existing daemon, shared helpers and three native @@ -1354,7 +1361,10 @@ default. No credential enters build arguments, image layers, browser bundles or diagnostic output. Compose configuration is confidential. The generated database, caller/tenant/provider identities and encryption key survive reruns; automatic revision replacement and provider migration are outside this initial installer. -Do not delete data or issue broad container/volume pruning as recovery. +Stopping control-plane services does not stop all Provider resources; use Core's +existing release operations for full cleanup. No native restart promise covers +host reboot or a lost running microVM. Do not delete data or issue broad +container/volume pruning as recovery. `make check-distribution` covers the production proxy, installation rules and release metadata. Real bundle validation covers default/provider selection, diff --git a/README.md b/README.md index 69456bef3..bd85b8e99 100644 --- a/README.md +++ b/README.md @@ -8,8 +8,8 @@ native harness keeps its own model and tool loop. Core runs independently of the Parsar product. Core and its Web console ship together. The installer prepares microsandbox by -default; use `--provider docker` for Docker. Core provisions the colocated -Runtime when a Session needs it. Model credentials are supplied through the +default; use `--provider docker` for Docker. Core creates each required sandbox from the +colocated Runtime image through its Provider, then initializes it. Model credentials are supplied through the existing write-only API extension, not during installation. ## Start here diff --git a/deploy/install/configuration.py b/deploy/install/configuration.py index 623410e1d..363a71337 100644 --- a/deploy/install/configuration.py +++ b/deploy/install/configuration.py @@ -6,7 +6,8 @@ def bind(source, target, readonly=True): return {"type": "bind", "source": str(source), "target": target, "read_only": readonly} -def managed_config(state, manifest): +def managed_config(root, state, manifest): + root = Path(root) result = {"installation_id": state["installation_id"], "provider": state["provider"], "maintenance": False} if state["provider"] == "docker": @@ -16,18 +17,18 @@ def managed_config(state, manifest): "nested_sandbox": True, }) else: - result.update(core_url="http://host.microsandbox.internal:8091/api/v1", microsandbox={ - "helper_path": "/usr/local/bin/agents-api-microsandbox-provider", - "runtime_path": "/opt/microsandbox/msb", - "firmware_path": "/opt/microsandbox/libkrunfw.so.5.6.1", + result.update(core_url=f'http://host.microsandbox.internal:{state["core_port"]}/api/v1', microsandbox={ + "helper_path": str(root / "native/bin/agents-api-microsandbox-provider"), + "runtime_path": str(root / "native/microsandbox/msb"), + "firmware_path": str(root / "native/microsandbox/libkrunfw.so.5.6.1"), "runtime_sha256": manifest["microsandbox"]["runtime_sha256"], "firmware_sha256": manifest["microsandbox"]["firmware_sha256"], - "runtime_home": "/state/msb", "image": manifest["runtime_ref"], + "runtime_home": str(root / "state/msb"), "image": manifest["runtime_ref"], "memory_mib": 4096, "cpus": 2, "root_disk_mib": 8192, "idle_seconds": 300, "retention_seconds": 86400, "max_active": 4, "max_retained": 16, "network": {"default_egress": "deny", "default_ingress": "deny", "rules": [ - {"action": "allow", "direction": "egress", "destination": "host", "protocol": "tcp", "port": "8091"}, + {"action": "allow", "direction": "egress", "destination": "host", "protocol": "tcp", "port": str(state["core_port"])}, {"action": "allow", "direction": "egress", "destination": "host", "protocol": "udp", "port": "53"}, {"action": "allow", "direction": "egress", "destination": "host", "protocol": "tcp", "port": "53"}, {"action": "allow", "direction": "egress", "destination": "public"}, @@ -36,12 +37,29 @@ def managed_config(state, manifest): return result +def core_environment(root, state, database_password): + native = state["provider"] == "microsandbox" + config = str(Path(root) / "config") if native else "/config" + database = f'127.0.0.1:{state["database_port"]}' if native else "database:5432" + daemon_host = f'host.microsandbox.internal:{state["core_port"]}' if native else "core:8091" + return { + "AGENTS_API_DATABASE_URL": f"postgres://agents_api:{database_password}@{database}/agents_api?sslmode=disable", + "AGENTS_API_KEYS_FILE": config + "/keys.json", + "AGENTS_API_CREDENTIAL_KEY_FILE": config + "/credential.key", + "AGENTS_API_ADDR": f'127.0.0.1:{state["core_port"]}' if native else ":8091", + "AGENTS_API_ENGINE": "codex", "AGENTS_API_HARNESSES": "codex,claude_sdk,mcode", + "AGENTS_API_MANAGED_RUNTIMES_FILE": config + "/managed-runtimes.json", + "AGENTS_API_DAEMON_WS_URL": f"ws://{daemon_host}/api/v1/agent-daemon/ws", + } + + def compose_config(root, state, manifest, database_password): root = Path(root) config = root / "config" identity = f'{state["uid"]}:{state["gid"]}' doc = {"name": state["project"], "services": {}} services = doc["services"] + native = state["provider"] == "microsandbox" and state["mode"] != "web-only" if state["mode"] != "web-only": services["database"] = { "image": manifest["images"]["database"], "restart": "unless-stopped", @@ -51,13 +69,7 @@ def compose_config(root, state, manifest, database_password): "healthcheck": {"test": ["CMD-SHELL", "pg_isready -U agents_api -d agents_api"], "interval": "2s", "timeout": "5s", "retries": 30}, } - env = {"AGENTS_API_DATABASE_URL": f"postgres://agents_api:{database_password}@database:5432/agents_api?sslmode=disable", - "AGENTS_API_KEYS_FILE": "/config/keys.json", - "AGENTS_API_CREDENTIAL_KEY_FILE": "/config/credential.key", - "AGENTS_API_ADDR": ":8091", "AGENTS_API_ENGINE": "codex", - "AGENTS_API_HARNESSES": "codex,claude_sdk,mcode", - "AGENTS_API_MANAGED_RUNTIMES_FILE": "/config/managed-runtimes.json", - "AGENTS_API_DAEMON_WS_URL": managed_config(state, manifest)["core_url"].replace("http:", "ws:") + "/agent-daemon/ws"} + env = core_environment(root, state, database_password) shared = {"image": manifest["images"]["core"], "user": identity, "environment": env, "volumes": [bind(config, "/config")], "read_only": True, "tmpfs": ["/tmp:mode=1777"], "init": True, @@ -67,17 +79,15 @@ def compose_config(root, state, manifest, database_password): core = dict(shared, restart="unless-stopped", ports=[f'127.0.0.1:{state["core_port"]}:8091'], depends_on={"migrate": {"condition": "service_completed_successfully"}}) core["volumes"] = list(shared["volumes"]) - if state["provider"] == "microsandbox": - core["volumes"].append(bind(root / "state", "/state", False)) - core["environment"] = dict(env, HOME="/state", MSB_HOME="/state/msb") - core["devices"] = ["/dev/kvm:/dev/kvm"] - core["group_add"] = [str(state["device_gid"])] + if native: + services["database"]["ports"] = [f'127.0.0.1:{state["database_port"]}:5432'] + services.pop("migrate") else: core["volumes"].append(bind("/var/run/docker.sock", "/var/run/docker.sock", False)) core["group_add"] = [str(state["device_gid"])] core["networks"] = ["default", "runtime"] doc["networks"] = {"runtime": {"name": state["project"] + "-runtime"}} - services["core"] = core + services["core"] = core doc["volumes"] = {"database": {}} if state["mode"] != "core-only": services["web"] = { @@ -87,11 +97,12 @@ def compose_config(root, state, manifest, database_password): "volumes": [bind(config / "caller.key", "/config/caller.key"), bind(config / "console.password", "/config/console.password")], "environment": {"CORE_CONSOLE_ORIGIN": f'http://127.0.0.1:{state["web_port"]}', - "CORE_CONSOLE_UPSTREAM": state.get("core_url") or "http://core:8091", + "CORE_CONSOLE_UPSTREAM": (f'http://127.0.0.1:{state["core_port"]}' if native + else state.get("core_url") or "http://core:8091"), "CORE_CONSOLE_TOKEN_FILE": "/config/caller.key", "CORE_CONSOLE_PASSWORD_FILE": "/config/console.password"}, } - if state["mode"] == "web-only": + if state["mode"] == "web-only" or native: services["web"].pop("ports") services["web"]["network_mode"] = "host" services["web"]["environment"]["CORE_CONSOLE_ADDR"] = f'127.0.0.1:{state["web_port"]}' diff --git a/deploy/install/install.py b/deploy/install/install.py index 39201e457..e531c14e5 100644 --- a/deploy/install/install.py +++ b/deploy/install/install.py @@ -17,7 +17,8 @@ import urllib.request import uuid -from configuration import compose_config, managed_config +from configuration import compose_config, core_environment, managed_config +import native_service class InstallError(Exception): @@ -59,8 +60,10 @@ def verify_bundle(bundle): raise InstallError("Invalid distribution path") if digest(path) != expected: raise InstallError("Distribution checksum mismatch: " + name) - required = {"manifest.json", "install.sh", "install.py", "configuration.py", "runtime/seccomp.json"} + required = {"manifest.json", "install.sh", "install.py", "configuration.py", "native_service.py", "runtime/seccomp.json"} required.update(f"images/{name}.tar" for name in ("core", "web", "runtime", "database")) + required.update("native/bin/" + name for name in ("agents-api", "agents-api-migrate", "agents-api-microsandbox-provider")) + required.update("native/microsandbox/" + name for name in ("msb", "libkrunfw.so.5.6.1")) if not required.issubset(covered): raise InstallError("Distribution checksum list is incomplete") manifest = json.loads((bundle / "manifest.json").read_text()) @@ -78,6 +81,12 @@ def free_port(port): raise InstallError(f"Port {port} is already in use; select another port") from None +def database_port(): + with socket.socket() as sock: + sock.bind(("127.0.0.1", 0)) + return sock.getsockname()[1] + + def core_target(value): from urllib.parse import urlsplit parsed = urlsplit(value) @@ -144,6 +153,9 @@ def status(root, state): if state["mode"] == "all": required.add("web") observed = {row["Service"]: row for row in rows} + if native_service.is_native(state): + observed["core"] = {"State": "running" if native_service.active(state) else "stopped"} + print("Core service: " + observed["core"]["State"]) healthy = all(name in observed and observed[name]["State"] == "running" and observed[name].get("Health", "") in ("", "healthy") for name in required) for row in rows: @@ -196,6 +208,8 @@ def initialize(root, args, manifest): "provider": args.provider, "installation_id": str(uuid.uuid4()), "project": "parsar-" + secrets.token_hex(5), "uid": os.getuid(), "gid": os.getgid(), "core_port": args.core_port, "web_port": args.web_port, "core_url": args.core_url} + if native_service.is_native(state): + state["database_port"] = database_port() config = root / "config" if mode != "web-only": state["device_gid"] = device_gid @@ -204,7 +218,7 @@ def initialize(root, args, manifest): "token_sha256": hashlib.sha256(token.encode()).hexdigest()}]) private_write(config / "credential.key", base64.b64encode(secrets.token_bytes(32)).decode()) private_write(config / "database.password", secrets.token_hex(32)) - write_json(config / "managed-runtimes.json", managed_config(state, manifest)) + write_json(config / "managed-runtimes.json", managed_config(root, state, manifest)) private_write(config / "caller.key", token) if mode != "core-only": private_write(config / "console.password", secrets.token_hex(24)) @@ -215,14 +229,13 @@ def initialize(root, args, manifest): def import_runtime(root, state, manifest, bundle): - if state["provider"] != "microsandbox" or state["mode"] == "web-only": + if not native_service.is_native(state): return - run(["docker", "run", "--rm", "--init", "--user", f'{state["uid"]}:{state["gid"]}', - "--env", "MSB_HOME=/state/msb", "--env", "HOME=/state", - "--mount", f'type=bind,source={root / "state"},target=/state', - "--mount", f'type=bind,source={bundle / "images/runtime.tar"},target=/runtime.tar,readonly', - "--entrypoint", "/opt/microsandbox/msb", manifest["images"]["core"], - "image", "load", "--input", "/runtime.tar", "--tag", manifest["runtime_ref"], "--quiet"]) + runtime = root / "native/microsandbox" + env = dict(os.environ, MSB_BACKEND="local", MSB_HOME=str(root / "state/msb"), + MSB_PATH=str(runtime / "msb"), MSB_LIBKRUNFW_PATH=str(runtime / "libkrunfw.so.5.6.1")) + run([str(runtime / "msb"), "image", "load", "--input", str(bundle / "images/runtime.tar"), + "--tag", manifest["runtime_ref"], "--quiet"], env=env) def main(argv=None): @@ -233,8 +246,10 @@ def main(argv=None): if args.status or args.stop: state = json.loads((root / "installation.json").read_text()) if args.stop: + if native_service.is_native(state): + native_service.stop(root, state) compose(root, "stop") - print("Services stopped. Database and Runtime state retained.") + print("Control-plane services stopped. Sandbox resources and data retained; running sandbox work may continue.") else: status(root, state) return @@ -246,18 +261,29 @@ def main(argv=None): raise InstallError("microsandbox requires host KVM; enable virtualization or explicitly choose --provider docker") bundle = Path(__file__).resolve().parent manifest = verify_bundle(bundle) + if args.provider == "microsandbox" and not args.web_only: + native_service.preflight(bundle) state = initialize(root, args, manifest) if state["mode"] != "web-only": seccomp = bundle / "runtime/seccomp.json" if not (root / "config/seccomp.json").exists(): private_write(root / "config/seccomp.json", seccomp.read_text()) images = ["web"] if state["mode"] == "web-only" else ["core", "runtime", "database"] + if native_service.is_native(state): + images = ["database"] + password = (root / "config/database.password").read_text() + environment = core_environment(root, state, password) + native_service.prepare(root, state, bundle, environment) if state["mode"] == "all": images.append("web") for name in images: run(["docker", "load", "--input", str(bundle / f"images/{name}.tar")], stdout=subprocess.DEVNULL) import_runtime(root, state, manifest, bundle) - compose(root, "up", "--detach") + compose(root, "up", "--detach", "--wait") + if native_service.is_native(state): + run([str(root / "native/bin/agents-api-migrate")], env=dict(os.environ, **environment), + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + native_service.start(root, state) if state["mode"] != "web-only" and not wait_http(f'http://127.0.0.1:{state["core_port"]}/healthz'): raise InstallError("Core did not become healthy. Use --status; retained state has not been removed") if state["mode"] != "core-only": @@ -277,7 +303,7 @@ def main(argv=None): if __name__ == "__main__": try: main() - except (InstallError, OSError, ValueError, KeyError, subprocess.CalledProcessError) as error: + except (InstallError, RuntimeError, OSError, ValueError, KeyError, subprocess.CalledProcessError) as error: # Errors never include generated configuration or external process output. - print(str(error) if isinstance(error, InstallError) else "Installation failed; inspect prerequisites and private deployment files", file=sys.stderr) + print(str(error) if isinstance(error, (InstallError, RuntimeError)) else "Installation failed; inspect prerequisites and private deployment files", file=sys.stderr) sys.exit(1) diff --git a/deploy/install/test_install.py b/deploy/install/test_install.py index 942013886..d84dd60cb 100644 --- a/deploy/install/test_install.py +++ b/deploy/install/test_install.py @@ -83,10 +83,15 @@ def bundle(self): (bundle / "runtime").mkdir() (bundle / "manifest.json").write_text(json.dumps(self.manifest)) (bundle / "runtime/seccomp.json").write_text('{"defaultAction":"SCMP_ACT_ERRNO"}') - for name in ("install.py", "configuration.py", "install.sh"): + for name in ("install.py", "configuration.py", "native_service.py", "install.sh"): shutil.copyfile(Path(__file__).with_name(name), bundle / name) for name in self.manifest["images"]: (bundle / "images" / (name + ".tar")).write_bytes(("synthetic " + name).encode()) + for name in ("bin/agents-api", "bin/agents-api-migrate", "bin/agents-api-microsandbox-provider", + "microsandbox/msb", "microsandbox/libkrunfw.so.5.6.1"): + path = bundle / "native" / name + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(b"synthetic native file") self.write_checksums(bundle) return bundle @@ -126,32 +131,30 @@ def test_configuration_changes_refuse_without_mutating_existing_deployment(self) install.initialize(self.root, self.args(), changed) self.assertEqual(before, self.snapshot()) - def test_default_microsandbox_exposes_only_kvm_to_core(self): + def test_default_microsandbox_keeps_core_and_vm_processes_outside_compose(self): state = self.initialize() self.assertEqual(state["provider"], "microsandbox") managed = self.document("config/managed-runtimes.json") - self.assertIn("microsandbox", managed) self.assertNotIn("docker", managed) - self.assertEqual(managed["microsandbox"]["network"]["default_ingress"], "deny") - self.assertEqual(managed["microsandbox"]["network"]["default_egress"], "deny") - self.assertEqual(managed["microsandbox"]["image"], self.manifest["runtime_ref"]) + micro = managed["microsandbox"] + self.assertEqual(micro["network"]["default_ingress"], "deny") + self.assertEqual(micro["network"]["default_egress"], "deny") + self.assertEqual(micro["image"], self.manifest["runtime_ref"]) + self.assertEqual(micro["runtime_home"], str(self.root / "state/msb")) + self.assertEqual(micro["helper_path"], str(self.root / "native/bin/agents-api-microsandbox-provider")) services = self.document("compose.json")["services"] - self.assertEqual(set(services), {"core", "database", "migrate", "web"}) - self.assertEqual(services["core"]["devices"], ["/dev/kvm:/dev/kvm"]) - self.assertIn("1234", services["core"]["group_add"]) - for name, service in services.items(): + self.assertEqual(set(services), {"database", "web"}) + for service in services.values(): self.assertFalse(service.get("privileged", False)) + self.assertNotIn("devices", service) self.assertNotIn("docker.sock", json.dumps(service.get("volumes", []))) - if name != "core": - self.assertNotIn("devices", service) - self.assertNotIn("ports", services["database"]) - for name in ("core", "migrate", "web"): - self.assertEqual(services[name]["user"], "1000:1000") - self.assertTrue(services[name]["read_only"]) - self.assertIn("no-new-privileges:true", services[name]["security_opt"]) - for name in ("core", "web"): - self.assertTrue(all(port.startswith("127.0.0.1:") for port in services[name]["ports"])) - self.assertEqual(services["core"]["depends_on"]["migrate"]["condition"], "service_completed_successfully") + self.assertEqual(services["database"]["ports"], [f'127.0.0.1:{state["database_port"]}:5432']) + web = services["web"] + self.assertEqual(web["network_mode"], "host") + self.assertEqual(web["environment"]["CORE_CONSOLE_ADDR"], "127.0.0.1:8080") + self.assertEqual(web["environment"]["CORE_CONSOLE_UPSTREAM"], "http://127.0.0.1:8091") + self.assertTrue(web["read_only"]) + self.assertIn("no-new-privileges:true", web["security_opt"]) def test_docker_provider_socket_and_runtime_network_belong_only_to_core(self): state = self.initialize("--provider", "docker", "--core-only") @@ -325,14 +328,14 @@ def test_status_rejects_failed_database_even_while_http_processes_are_alive(self mock.patch.object(install, "compose", return_value=SimpleNamespace(stdout=json.dumps(rows))), \ mock.patch.object(install, "wait_http", return_value=True), \ contextlib.redirect_stdout(io.StringIO()), self.assertRaises(install.InstallError): - install.status(self.root, {"mode": "all", "core_port": 8091, "web_port": 8080}) + install.status(self.root, {"mode": "all", "provider": "docker", "core_port": 8091, "web_port": 8080}) def test_status_accepts_web_only_without_database_or_core_services(self): rows = [{"Service": "web", "State": "running", "Health": ""}] with mock.patch.object(install, "compose", return_value=SimpleNamespace(stdout=json.dumps(rows))), \ mock.patch.object(install, "wait_http", return_value=True), \ contextlib.redirect_stdout(io.StringIO()): - install.status(self.root, {"mode": "web-only", "web_port": 8080}) + install.status(self.root, {"mode": "web-only", "provider": "microsandbox", "web_port": 8080}) if __name__ == "__main__": diff --git a/docs/getting-started/install.md b/docs/getting-started/install.md index 3814931c5..54b3c73de 100644 --- a/docs/getting-started/install.md +++ b/docs/getting-started/install.md @@ -9,11 +9,14 @@ No model key, Environment wizard or sample task is required during installation. ## Host requirements The first distribution targets Linux amd64 with Python 3.9+, Docker and Docker -Compose v2. Run the installer as a non-root user who can use Docker. The qualified -Docker host version is 29.1.3. Default microsandbox also requires an available -`/dev/kvm`; nested cloud hosts must expose hardware virtualization. The installer -passes that device and its group to Core, without a privileged container. -It does not silently fall back to Docker when KVM is missing. +Compose v2. Run the installer as a non-root user who can use Docker. +Default microsandbox also requires glibc, a running systemd user manager with +linger enabled, and user read/write access to `/dev/kvm`. Nested cloud hosts must +expose hardware virtualization. Core runs as a native user service so restarting +it does not terminate the Provider's microVM processes. The installer checks these +prerequisites; it does not grant host permissions or silently fall back to Docker. +The explicit Docker option runs Core in Compose and requires neither KVM nor +systemd user services. Reserve capacity for the native Runtime: the initial microsandbox profile uses 4 GiB RAM, 2 CPUs and an 8 GiB root disk per active sandbox, with at most 4 active @@ -37,8 +40,8 @@ cd "$HOME/.parsar/releases/parsar-core--linux-amd64" ./install.sh ``` -The bundle contains the same-revision Core, unchanged Web build, production Web -proxy and colocated Runtime. It includes microsandbox's pinned runtime and +The bundle contains the same-revision native Core binaries and service image, +unchanged Web build, production Web proxy and colocated Runtime. It includes microsandbox's pinned runtime and firmware. It also contains image archives, source provenance and checksums; installation does not need Go, Node, Rust or a product checkout. @@ -66,8 +69,8 @@ the server holds the independent Core key. Model keys remain API execution input `--provider` selects the deployment's sandbox provider. It does not select a harness or alter the public `openai_hosted` discriminator. Both providers reuse one colocated Runtime containing the native harnesses. The Docker option grants -only Core access to the Docker socket; microsandbox grants only Core access to -KVM. Web receives neither. +only Core access to the Docker socket; microsandbox uses the native service account's +KVM access. Web receives neither. To install only Web on a Linux host, provide the existing Core origin and a private caller-key file. A loopback Core uses the same host network namespace; @@ -85,7 +88,8 @@ the server, outside the static Web files. The input file must be private (0600). Use `--install-dir /absolute/path`, `--core-port 8092` and `--web-port 8081` for separate installations. Their database volumes, provider identities and Runtime -state are independent. Repeating the same installation command retains its +state are independent. Native Core connects to PostgreSQL through an automatically +selected loopback-only port, recorded in its private installation state. Repeating the same installation command retains its identities, secrets and data. Conflicting mode/provider/revision changes refuse rather than silently replacing them. diff --git a/docs/getting-started/operations.md b/docs/getting-started/operations.md index cee5d9230..b52d6cbea 100644 --- a/docs/getting-started/operations.md +++ b/docs/getting-started/operations.md @@ -16,7 +16,8 @@ Run from the extracted bundle: ``` The command reads only this installation's service status and health endpoints. -It prints service names, running/exit state and available Docker health state; +It prints service names, running/exit state, the native Core service state when +applicable, and available Docker health state; it does not print Compose configuration, environment variables, credentials or raw application logs. It never creates a Session or calls a model. @@ -45,8 +46,13 @@ Settle active work before a planned restart. Then: ./install.sh # Use the same component/provider/port flags as the initial install. ``` -Stopping services retains the database, Runtime state and credentials. It does not -promise transparent continuation of an interrupted native tool. Query the same +This stops the control-plane services and retains the database, Runtime state and +credentials. For microsandbox, the systemd user unit uses `KillMode=process`: +only Core stops; microVMs and their work may remain running. Docker-owned Runtime +containers likewise remain Provider resources. The command does not promise to +stop all compute. Release resources through the existing Core API before a full +shutdown; do not kill Provider processes directly. A Core restart does not promise +transparent continuation of an interrupted native tool. Query the same Session after reconnecting; do not create a replacement Session to replay uncertain work. The official SSE stream is live, and recovery uses durable resource queries. @@ -86,12 +92,18 @@ The installer never migrates Sessions between providers or deletes old compute. ## Exposure and network policy -API and console ports publish to host loopback. PostgreSQL has no published port. +API and console bind to host loopback. With native Core, PostgreSQL publishes an +installation-specific loopback port; with container Core it has no published port. The production Web proxy only forwards the public `/v1` surface to its configured Core; it never receives the Docker socket, KVM device or provider/model secrets. It requires an independent console password and rejects untrusted browser origins. This is a single-operator console deployment, not a multi-user identity system. +A native Core restart preserves resident microVM processes. Host reboot, user +manager termination and loss of a running microVM are not equivalent to that +restart and are not qualified cold-recovery workflows. Completed idle snapshots +retain their existing recovery contract. + microsandbox uses an explicit policy: public egress, the Core/DNS host ports needed for the colocated Runtime, and denied inbound/private-network access. Private model/MCP endpoints require an explicit operator policy change. Native tool network diff --git a/scripts/build-core-distribution.sh b/scripts/build-core-distribution.sh index 8d96d68b8..23704a5c0 100755 --- a/scripts/build-core-distribution.sh +++ b/scripts/build-core-distribution.sh @@ -71,7 +71,7 @@ if [[ "$(go env GOVERSION)" != "$required_go" ]]; then printf 'Distribution build requires %s\n' "$required_go" >&2 exit 1 fi -for file in install.sh install.py configuration.py; do +for file in install.sh install.py configuration.py native_service.py; do cp "deploy/install/$file" "$bundle/$file" done mkdir -p "$bundle/docs" From 2e84b37186a3168905b857c6094665473ffa6694 Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 18:14:40 +0800 Subject: [PATCH 15/17] fix(microsandbox): keep runtime workspace on a native owned disk --- CONTRIBUTING.md | 6 +++ deploy/install/acceptance.py | 2 + deploy/install/configuration.py | 2 +- deploy/install/test_install.py | 2 + docs/getting-started/install.md | 2 +- .../cmd/server/managed_microsandbox.go | 33 ++++++------- .../cmd/server/managed_microsandbox_test.go | 47 ++++++++++--------- .../agents-api/deploy/microsandbox/README.md | 5 +- .../managed-runtimes.example.json | 1 + .../internal/sandbox/microsandbox/identity.go | 2 +- .../sandbox/microsandbox/provider_test.go | 2 +- .../internal/sandbox/microsandbox/types.go | 25 +++++----- .../tools/microsandbox-provider/README.md | 9 ++++ .../tools/microsandbox-provider/bootstrap.go | 5 ++ 14 files changed, 87 insertions(+), 56 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 34c327a44..ab0c0cb88 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -598,6 +598,12 @@ pause does not release RAM. Suspension captures and verifies a full snapshot, stops the exact source, and removes its writable compute closure only after the artifact is durably identified. Explicit network policy applies on create and restore. Do not inherit undeclared host resources. +The native SDK owns a dedicated ext4 disk mounted at `/environment`, separately +bounded by `environment_disk_mib` alongside `root_disk_mib`. Workspace, staging +and outputs must share that filesystem; do not weaken cross-device or link +checks to accommodate the layered root. Creation uses `/` until bootstrap creates +the workspace. Existing full snapshots and sandbox cleanup own the disk, with +no external mount or separate storage lifecycle. Suspend only after at least one Turn is terminal, no queued/in-progress/waiting root or subagent Turn, pending input/file operation or initialization remains, diff --git a/deploy/install/acceptance.py b/deploy/install/acceptance.py index 86bac3622..303888afb 100644 --- a/deploy/install/acceptance.py +++ b/deploy/install/acceptance.py @@ -157,6 +157,8 @@ def request(self, method, path, operation, body=None, query=None, key=None, bina require(len(raw) <= MAX_RESPONSE, "response_limit_exceeded") if binary: return raw + if method == "POST" and path.endswith("/events") and entry["http_status"] == 202 and not raw: + return {} try: value = json.loads(raw) except (ValueError, UnicodeError): diff --git a/deploy/install/configuration.py b/deploy/install/configuration.py index 363a71337..71d2a36b2 100644 --- a/deploy/install/configuration.py +++ b/deploy/install/configuration.py @@ -24,7 +24,7 @@ def managed_config(root, state, manifest): "runtime_sha256": manifest["microsandbox"]["runtime_sha256"], "firmware_sha256": manifest["microsandbox"]["firmware_sha256"], "runtime_home": str(root / "state/msb"), "image": manifest["runtime_ref"], - "memory_mib": 4096, "cpus": 2, "root_disk_mib": 8192, + "memory_mib": 4096, "cpus": 2, "root_disk_mib": 8192, "environment_disk_mib": 8192, "idle_seconds": 300, "retention_seconds": 86400, "max_active": 4, "max_retained": 16, "network": {"default_egress": "deny", "default_ingress": "deny", "rules": [ diff --git a/deploy/install/test_install.py b/deploy/install/test_install.py index d84dd60cb..91399a3da 100644 --- a/deploy/install/test_install.py +++ b/deploy/install/test_install.py @@ -137,6 +137,8 @@ def test_default_microsandbox_keeps_core_and_vm_processes_outside_compose(self): managed = self.document("config/managed-runtimes.json") self.assertNotIn("docker", managed) micro = managed["microsandbox"] + self.assertEqual(micro["root_disk_mib"], 8192) + self.assertEqual(micro["environment_disk_mib"], 8192) self.assertEqual(micro["network"]["default_ingress"], "deny") self.assertEqual(micro["network"]["default_egress"], "deny") self.assertEqual(micro["image"], self.manifest["runtime_ref"]) diff --git a/docs/getting-started/install.md b/docs/getting-started/install.md index 54b3c73de..e9e3aad47 100644 --- a/docs/getting-started/install.md +++ b/docs/getting-started/install.md @@ -19,7 +19,7 @@ The explicit Docker option runs Core in Compose and requires neither KVM nor systemd user services. Reserve capacity for the native Runtime: the initial microsandbox profile uses -4 GiB RAM, 2 CPUs and an 8 GiB root disk per active sandbox, with at most 4 active +4 GiB RAM, 2 CPUs, an 8 GiB root disk and an 8 GiB environment disk per active sandbox, with at most 4 active and 16 retained allocations. Limits are operator configuration, not model input. Use a trusted, single-operator host and durable local storage. The installer does not change host virtualization settings, install Docker, create an OS user or diff --git a/services/agents-api/cmd/server/managed_microsandbox.go b/services/agents-api/cmd/server/managed_microsandbox.go index a306a8a9f..32266cf60 100644 --- a/services/agents-api/cmd/server/managed_microsandbox.go +++ b/services/agents-api/cmd/server/managed_microsandbox.go @@ -12,21 +12,22 @@ import ( // Single-host providers pin the helper, runtime, firmware, image and resource // limits explicitly. The helper owns local paths; no ambient backend is selected. type managedMicrosandboxConfig struct { - HelperPath string `json:"helper_path"` - RuntimeHome string `json:"runtime_home"` - RuntimePath string `json:"runtime_path"` - FirmwarePath string `json:"firmware_path"` - RuntimeSHA256 string `json:"runtime_sha256"` - FirmwareSHA256 string `json:"firmware_sha256"` - Image string `json:"image"` - MemoryMiB uint32 `json:"memory_mib"` - CPUs uint8 `json:"cpus"` - RootDiskMiB uint32 `json:"root_disk_mib"` - Network managedMicrosandboxNetwork `json:"network"` - IdleSeconds int64 `json:"idle_seconds"` - RetentionSeconds int64 `json:"retention_seconds"` - MaxActive int `json:"max_active"` - MaxRetained int `json:"max_retained"` + HelperPath string `json:"helper_path"` + RuntimeHome string `json:"runtime_home"` + RuntimePath string `json:"runtime_path"` + FirmwarePath string `json:"firmware_path"` + RuntimeSHA256 string `json:"runtime_sha256"` + FirmwareSHA256 string `json:"firmware_sha256"` + Image string `json:"image"` + MemoryMiB uint32 `json:"memory_mib"` + CPUs uint8 `json:"cpus"` + RootDiskMiB uint32 `json:"root_disk_mib"` + EnvironmentDiskMiB uint32 `json:"environment_disk_mib"` + Network managedMicrosandboxNetwork `json:"network"` + IdleSeconds int64 `json:"idle_seconds"` + RetentionSeconds int64 `json:"retention_seconds"` + MaxActive int `json:"max_active"` + MaxRetained int `json:"max_retained"` } type managedMicrosandboxNetwork struct { @@ -58,7 +59,7 @@ func configureManagedMicrosandbox(entry managedMicrosandboxConfig, result *execu provider, err := sandboxmicro.New(sandboxmicro.Config{ InstallationID: result.InstallationID, HelperPath: entry.HelperPath, RuntimeHome: entry.RuntimeHome, RuntimePath: entry.RuntimePath, FirmwarePath: entry.FirmwarePath, RuntimeSHA256: entry.RuntimeSHA256, FirmwareSHA256: entry.FirmwareSHA256, Image: entry.Image, - MemoryMiB: entry.MemoryMiB, CPUs: entry.CPUs, RootDiskMiB: entry.RootDiskMiB, Network: network, + MemoryMiB: entry.MemoryMiB, CPUs: entry.CPUs, RootDiskMiB: entry.RootDiskMiB, EnvironmentDiskMiB: entry.EnvironmentDiskMiB, Network: network, }) if err != nil { return errors.New("invalid managed microsandbox provider configuration") diff --git a/services/agents-api/cmd/server/managed_microsandbox_test.go b/services/agents-api/cmd/server/managed_microsandbox_test.go index a3d29839c..34d86cf86 100644 --- a/services/agents-api/cmd/server/managed_microsandbox_test.go +++ b/services/agents-api/cmd/server/managed_microsandbox_test.go @@ -20,7 +20,7 @@ func managedMicrosandboxFixture(t *testing.T) (managedRuntimeConfig, string, fun Microsandbox: &managedMicrosandboxConfig{ HelperPath: "/opt/parsar/microsandbox-provider", RuntimeHome: "/var/lib/parsar/microsandbox", RuntimePath: "/opt/parsar/msb", FirmwarePath: "/opt/parsar/libkrunfw.so", RuntimeSHA256: strings.Repeat("a", 64), FirmwareSHA256: strings.Repeat("b", 64), Image: "registry.example/parsar-runtime@sha256:" + strings.Repeat("c", 64), - MemoryMiB: 1024, CPUs: 1, RootDiskMiB: 4096, + MemoryMiB: 1024, CPUs: 1, RootDiskMiB: 4096, EnvironmentDiskMiB: 2048, Network: managedMicrosandboxNetwork{DefaultEgress: "deny", DefaultIngress: "deny", Rules: []managedMicrosandboxRule{{Action: "allow", Direction: "egress", Destination: "core.example", Protocol: "tcp", Port: "443"}}}, IdleSeconds: 300, RetentionSeconds: 86400, MaxActive: 4, MaxRetained: 8, }, @@ -64,28 +64,29 @@ func TestManagedMicrosandboxConfigurationAndPolicy(t *testing.T) { func TestManagedMicrosandboxRejectsUnboundedOrImplicitConfiguration(t *testing.T) { config, _, write := managedMicrosandboxFixture(t) cases := map[string]func(*managedMicrosandboxConfig){ - "retained_missing": func(c *managedMicrosandboxConfig) { c.MaxRetained = 0 }, - "retained_below_active": func(c *managedMicrosandboxConfig) { c.MaxRetained = c.MaxActive - 1 }, - "retained_negative": func(c *managedMicrosandboxConfig) { c.MaxRetained = -1 }, - "idle_missing": func(c *managedMicrosandboxConfig) { c.IdleSeconds = 0 }, - "retention_missing": func(c *managedMicrosandboxConfig) { c.RetentionSeconds = 0 }, - "capacity_missing": func(c *managedMicrosandboxConfig) { c.MaxActive = 0 }, - "negative_capacity": func(c *managedMicrosandboxConfig) { c.MaxActive = -1 }, - "negative_idle": func(c *managedMicrosandboxConfig) { c.IdleSeconds = -1 }, - "idle_overflow": func(c *managedMicrosandboxConfig) { c.IdleSeconds = 1<<63 - 1 }, - "retention_overflow": func(c *managedMicrosandboxConfig) { c.RetentionSeconds = 1<<63 - 1 }, - "memory_missing": func(c *managedMicrosandboxConfig) { c.MemoryMiB = 0 }, - "cpus_missing": func(c *managedMicrosandboxConfig) { c.CPUs = 0 }, - "disk_missing": func(c *managedMicrosandboxConfig) { c.RootDiskMiB = 0 }, - "relative_helper": func(c *managedMicrosandboxConfig) { c.HelperPath = "./helper" }, - "unclean_home": func(c *managedMicrosandboxConfig) { c.RuntimeHome = "/private/state/../msb" }, - "relative_home": func(c *managedMicrosandboxConfig) { c.RuntimeHome = ".cache" }, - "runtime_missing": func(c *managedMicrosandboxConfig) { c.RuntimePath = "" }, - "firmware_missing": func(c *managedMicrosandboxConfig) { c.FirmwarePath = "" }, - "runtime_hash_missing": func(c *managedMicrosandboxConfig) { c.RuntimeSHA256 = "" }, - "firmware_hash_missing": func(c *managedMicrosandboxConfig) { c.FirmwareSHA256 = "" }, - "image_unpinned": func(c *managedMicrosandboxConfig) { c.Image = "registry.example/parsar-runtime:latest" }, - "network_missing": func(c *managedMicrosandboxConfig) { c.Network = managedMicrosandboxNetwork{} }, + "retained_missing": func(c *managedMicrosandboxConfig) { c.MaxRetained = 0 }, + "retained_below_active": func(c *managedMicrosandboxConfig) { c.MaxRetained = c.MaxActive - 1 }, + "retained_negative": func(c *managedMicrosandboxConfig) { c.MaxRetained = -1 }, + "idle_missing": func(c *managedMicrosandboxConfig) { c.IdleSeconds = 0 }, + "retention_missing": func(c *managedMicrosandboxConfig) { c.RetentionSeconds = 0 }, + "capacity_missing": func(c *managedMicrosandboxConfig) { c.MaxActive = 0 }, + "negative_capacity": func(c *managedMicrosandboxConfig) { c.MaxActive = -1 }, + "negative_idle": func(c *managedMicrosandboxConfig) { c.IdleSeconds = -1 }, + "idle_overflow": func(c *managedMicrosandboxConfig) { c.IdleSeconds = 1<<63 - 1 }, + "retention_overflow": func(c *managedMicrosandboxConfig) { c.RetentionSeconds = 1<<63 - 1 }, + "memory_missing": func(c *managedMicrosandboxConfig) { c.MemoryMiB = 0 }, + "cpus_missing": func(c *managedMicrosandboxConfig) { c.CPUs = 0 }, + "disk_missing": func(c *managedMicrosandboxConfig) { c.RootDiskMiB = 0 }, + "environment_disk_missing": func(c *managedMicrosandboxConfig) { c.EnvironmentDiskMiB = 0 }, + "relative_helper": func(c *managedMicrosandboxConfig) { c.HelperPath = "./helper" }, + "unclean_home": func(c *managedMicrosandboxConfig) { c.RuntimeHome = "/private/state/../msb" }, + "relative_home": func(c *managedMicrosandboxConfig) { c.RuntimeHome = ".cache" }, + "runtime_missing": func(c *managedMicrosandboxConfig) { c.RuntimePath = "" }, + "firmware_missing": func(c *managedMicrosandboxConfig) { c.FirmwarePath = "" }, + "runtime_hash_missing": func(c *managedMicrosandboxConfig) { c.RuntimeSHA256 = "" }, + "firmware_hash_missing": func(c *managedMicrosandboxConfig) { c.FirmwareSHA256 = "" }, + "image_unpinned": func(c *managedMicrosandboxConfig) { c.Image = "registry.example/parsar-runtime:latest" }, + "network_missing": func(c *managedMicrosandboxConfig) { c.Network = managedMicrosandboxNetwork{} }, } for name, mutate := range cases { t.Run(name, func(t *testing.T) { diff --git a/services/agents-api/deploy/microsandbox/README.md b/services/agents-api/deploy/microsandbox/README.md index abaac2d0b..da78c67ab 100644 --- a/services/agents-api/deploy/microsandbox/README.md +++ b/services/agents-api/deploy/microsandbox/README.md @@ -115,7 +115,10 @@ installation UUID for `installation_id`. Keep that identity and its original backend while any allocation needs cleanup. Set `provider: "microsandbox"` and include the single `microsandbox` object; do not include a `docker` object. -Set VM memory, CPU and disk limits explicitly. `max_active` bounds active compute. +Set VM memory, CPU and disk limits explicitly. `root_disk_mib` bounds the root +disk; `environment_disk_mib` separately bounds the native owned ext4 disk at +`/environment`, including workspace, staging and initialization data. Both are +required; budget for both disks and their retained snapshots. `max_active` bounds active compute. `max_retained` bounds all retained allocations, including suspended snapshots, and must be at least `max_active`. `idle_seconds` is the sustained idle interval before suspension. diff --git a/services/agents-api/deploy/microsandbox/managed-runtimes.example.json b/services/agents-api/deploy/microsandbox/managed-runtimes.example.json index eb6aa69d1..ebbb31491 100644 --- a/services/agents-api/deploy/microsandbox/managed-runtimes.example.json +++ b/services/agents-api/deploy/microsandbox/managed-runtimes.example.json @@ -14,6 +14,7 @@ "memory_mib": 4096, "cpus": 2, "root_disk_mib": 8192, + "environment_disk_mib": 8192, "network": { "default_egress": "deny", "default_ingress": "deny", diff --git a/services/agents-api/internal/sandbox/microsandbox/identity.go b/services/agents-api/internal/sandbox/microsandbox/identity.go index dd3e89b4b..3c6cd2218 100644 --- a/services/agents-api/internal/sandbox/microsandbox/identity.go +++ b/services/agents-api/internal/sandbox/microsandbox/identity.go @@ -25,7 +25,7 @@ func validHash(s string) bool { return e == nil && len(b) == 32 && strings.ToLower(s) == s } func (c Config) Validate() error { - if !validID(c.InstallationID) || !filepath.IsAbs(c.HelperPath) || !filepath.IsAbs(c.RuntimeHome) || !filepath.IsAbs(c.RuntimePath) || !filepath.IsAbs(c.FirmwarePath) || !validHash(c.RuntimeSHA256) || !validHash(c.FirmwareSHA256) || c.MemoryMiB == 0 || c.CPUs == 0 || c.RootDiskMiB == 0 { + if !validID(c.InstallationID) || !filepath.IsAbs(c.HelperPath) || !filepath.IsAbs(c.RuntimeHome) || !filepath.IsAbs(c.RuntimePath) || !filepath.IsAbs(c.FirmwarePath) || !validHash(c.RuntimeSHA256) || !validHash(c.FirmwareSHA256) || c.MemoryMiB == 0 || c.CPUs == 0 || c.RootDiskMiB == 0 || c.EnvironmentDiskMiB == 0 { return sandbox.ErrInvalid } imageName, digest, pinned := strings.Cut(c.Image, "@sha256:") diff --git a/services/agents-api/internal/sandbox/microsandbox/provider_test.go b/services/agents-api/internal/sandbox/microsandbox/provider_test.go index 046aa2d2d..9aaee7aca 100644 --- a/services/agents-api/internal/sandbox/microsandbox/provider_test.go +++ b/services/agents-api/internal/sandbox/microsandbox/provider_test.go @@ -17,7 +17,7 @@ func testConfig() Config { return Config{ InstallationID: "11111111-1111-4111-8111-111111111111", HelperPath: "/helper", RuntimeHome: "/private/msb", RuntimePath: "/private/bin/msb", FirmwarePath: "/private/lib/libkrunfw.so", RuntimeSHA256: strings.Repeat("a", 64), FirmwareSHA256: strings.Repeat("b", 64), Image: "registry/runtime@sha256:" + strings.Repeat("c", 64), - MemoryMiB: 2048, CPUs: 2, RootDiskMiB: 4096, Network: NetworkPolicy{DefaultEgress: "deny", DefaultIngress: "deny", Rules: []NetworkRule{{Action: "allow", Direction: "egress", Destination: "host"}}}, + MemoryMiB: 2048, CPUs: 2, RootDiskMiB: 4096, EnvironmentDiskMiB: 2048, Network: NetworkPolicy{DefaultEgress: "deny", DefaultIngress: "deny", Rules: []NetworkRule{{Action: "allow", Direction: "egress", Destination: "host"}}}, } } func testRef() sandbox.Reference { diff --git a/services/agents-api/internal/sandbox/microsandbox/types.go b/services/agents-api/internal/sandbox/microsandbox/types.go index 5a1b7a3f7..d7f84beb9 100644 --- a/services/agents-api/internal/sandbox/microsandbox/types.go +++ b/services/agents-api/internal/sandbox/microsandbox/types.go @@ -18,18 +18,19 @@ const MaxResponseBytes = 16 * 1024 * 1024 // Config is trusted deployment configuration. Paths and hashes refer to one // immutable, qualified installation. Network is explicitly used on create and restore. type Config struct { - InstallationID string - HelperPath string - RuntimeHome string - RuntimePath string - FirmwarePath string - RuntimeSHA256 string - FirmwareSHA256 string - Image string - MemoryMiB uint32 - CPUs uint8 - RootDiskMiB uint32 - Network NetworkPolicy + InstallationID string + HelperPath string + RuntimeHome string + RuntimePath string + FirmwarePath string + RuntimeSHA256 string + FirmwareSHA256 string + Image string + MemoryMiB uint32 + CPUs uint8 + RootDiskMiB uint32 + EnvironmentDiskMiB uint32 + Network NetworkPolicy } type NetworkPolicy struct { diff --git a/services/agents-api/tools/microsandbox-provider/README.md b/services/agents-api/tools/microsandbox-provider/README.md index 484a44c0a..13d392f89 100644 --- a/services/agents-api/tools/microsandbox-provider/README.md +++ b/services/agents-api/tools/microsandbox-provider/README.md @@ -48,6 +48,15 @@ complete host network policy. This adapter does not install registry credentials ## Bootstrap and network +The SDK creates a private owned ext4 disk at `/environment`, with explicit +`environment_disk_mib` capacity. Workspace, staging and generated outputs share +that filesystem, preserving the existing cross-device and link checks. The +layered root filesystem can report different device IDs for directories and +upper-layer files and is not used for workspace storage. Native full snapshots +and sandbox removal capture, restore and reclaim the owned disk; no host path or +external volume lifecycle is introduced. Creation starts in `/` until bootstrap +creates the workspace directories. + VM creation does not implicitly run the OCI ENTRYPOINT. Before admitting native work, the helper uses confidential stdin to install the existing private `auth.json` format, create Runtime directories, bind the same workspace at diff --git a/services/agents-api/tools/microsandbox-provider/bootstrap.go b/services/agents-api/tools/microsandbox-provider/bootstrap.go index db856f5bf..6bfb12180 100644 --- a/services/agents-api/tools/microsandbox-provider/bootstrap.go +++ b/services/agents-api/tools/microsandbox-provider/bootstrap.go @@ -53,6 +53,11 @@ func (b backend) create(ctx context.Context) (wire.State, error) { live, e := sdk.CreateSandbox(ctx, c.Name, sdk.WithImage(b.q.Config.Image), sdk.WithMemory(b.q.Config.MemoryMiB), sdk.WithCPUs(b.q.Config.CPUs), sdk.WithRootDisk(sdk.RootDisk.Managed(b.q.Config.RootDiskMiB)), sdk.WithUser("1000:1000"), + // A native owned disk keeps workspace and staging on one filesystem. + // Bootstrap creates their directories before starting the daemon. + sdk.WithWorkdir("/"), sdk.WithMounts(map[string]sdk.MountConfig{ + "/environment": sdk.Mount.Owned(sdk.OwnedVolumeOptions{Kind: sdk.VolumeKindDisk, SizeMiB: b.q.Config.EnvironmentDiskMiB}), + }), sdk.WithLabels(labels), sdk.WithDetached(), sdk.WithQuietLogs(), sdk.WithNetwork(b.network()), sdk.WithEnv(map[string]string{ "HOME": "/home/runtime", "PARSAR_HOME": "/home/runtime/.parsar", From a8ac63625c537499074bbce4e1fbf625210674ea Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 18:36:58 +0800 Subject: [PATCH 16/17] feat(install): make local sandbox providers opt-in --- CONTRIBUTING.md | 29 ++++++++---- README.md | 16 ++++--- deploy/install/configuration.py | 15 ++++--- deploy/install/install.py | 36 +++++++++++---- deploy/install/test_install.py | 56 +++++++++++++++++++++--- deploy/install/test_native_service.py | 3 +- docs/getting-started/README.md | 10 +++-- docs/getting-started/install.md | 63 +++++++++++++++++---------- docs/getting-started/operations.md | 23 ++++++---- docs/getting-started/quickstart.md | 5 +++ site/index.html | 6 +-- 11 files changed, 188 insertions(+), 74 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ab0c0cb88..9af40d20b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -560,8 +560,8 @@ does not qualify its isolation or enable public creation. ### Optional single-host sandbox suspension -Each Core deployment enables exactly one sandbox provider, selected at setup: -Docker or microsandbox. Keep both adapters but reject multiple provider entries, +A Core deployment may run without a sandbox provider. When enabled, exactly one +sandbox provider is selected at setup: Docker or microsandbox. Keep both adapters but reject multiple provider entries, legacy default-provider maps and engine-based placement. Harness selection is independent. The configuration has one installation UUID, one provider kind and one backend object. No compatibility parser or parallel provider route remains. @@ -1351,17 +1351,25 @@ qualification. A release must be tested from fresh extraction with real models; no synthetic result may substitute for native execution acceptance. The first installer targets a trusted Linux amd64 Docker host. It installs a -private dedicated PostgreSQL service and separate Core and console services. -Default sandbox placement is microsandbox; `--provider docker` selects the -existing Docker provider. Missing KVM fails without changing that choice. +private dedicated PostgreSQL service and separate Core and console services in +Compose by default, with zero execution nodes. The default requires neither KVM +nor systemd user services, imports no Runtime image, mounts neither the Docker +socket nor host devices into Core, and generates no managed Provider configuration. +Local sandbox placement is opt-in: `--sandbox-provider true --provider microsandbox` +or `--sandbox-provider true --provider docker`. Enabling the option without naming +a provider selects microsandbox; `--provider` without enabling the option is an +error. Web-only mode cannot enable a sandbox provider. Core-only mode retains the +same opt-in rule. Missing KVM fails when microsandbox is selected without changing +that choice. The distribution supplies native Core/helper binaries and pinned msb runtime and firmware. For microsandbox, Core is a native systemd user service with direct `ExecStart` and `KillMode=process`: its restart must preserve the Provider's resident microVM/helper processes. Never package those processes inside Core's container PID namespace, kill their process group on Core stop, or add recovery mechanisms to compensate for that packaging. User KVM access, the Linux runtime libraries and -linger are explicit prerequisites. For Docker, Core runs in Compose with the -canonical Docker socket. PostgreSQL/Web use Compose in either case; native Core +linger are prerequisites only for the microsandbox option. With the Docker sandbox +option, Core runs in Compose with the canonical Docker socket. PostgreSQL/Web use +Compose in either case; native Core and its Web proxy use loopback, with a private PostgreSQL port. This packaging choice does not change either Provider's execution contract. The basic distroless API image and binary builds remain independent artifacts. @@ -1369,8 +1377,9 @@ The basic distroless API image and binary builds remain independent artifacts. One Runtime image contains the existing daemon, shared helpers and three native harness packages. Their differences remain in the adapters. Core keeps exclusive ownership of Session allocation, initialization, cancellation, snapshots and -cleanup. The installer imports images and prepares running conditions; it never -creates an execution Session or supplies a model credential. Applications use the +cleanup. When a sandbox provider is enabled, the installer imports its Runtime +image and prepares running conditions; it never creates an execution Session or +supplies a model credential. Applications use the existing write-only model execution extension, with the installation's persistent credential encryption key. Provider identity/backend namespace and native history must not change on a repeated install. @@ -1390,6 +1399,8 @@ default. No credential enters build arguments, image layers, browser bundles or diagnostic output. Compose configuration is confidential. The generated database, caller/tenant/provider identities and encryption key survive reruns; automatic revision replacement and provider migration are outside this initial installer. +Reruns also refuse enabling or disabling a sandbox provider on an existing +installation, including adding one to the default zero-node installation. Stopping control-plane services does not stop all Provider resources; use Core's existing release operations for full cleanup. No native restart promise covers host reboot or a lost running microVM. Do not delete data or issue broad diff --git a/README.md b/README.md index bd85b8e99..23b9cd9b2 100644 --- a/README.md +++ b/README.md @@ -7,10 +7,11 @@ owns Sessions, environments, files, credentials and execution history; each native harness keeps its own model and tool loop. Core runs independently of the Parsar product. -Core and its Web console ship together. The installer prepares microsandbox by -default; use `--provider docker` for Docker. Core creates each required sandbox from the -colocated Runtime image through its Provider, then initializes it. Model credentials are supplied through the -existing write-only API extension, not during installation. +Core and its Web console ship together. The default installation runs Core, Web +and PostgreSQL with zero execution nodes. A local sandbox provider is optional: +enable microsandbox or Docker explicitly when installing. With a provider enabled, +Core creates each required sandbox from the colocated Runtime image. Model +credentials are supplied through the existing write-only API extension. ## Start here @@ -24,9 +25,10 @@ existing write-only API extension, not during installation. After verifying and extracting a matching Linux amd64 distribution: ```sh -./install.sh # Core + Web, microsandbox -./install.sh --provider docker # Core + Web, Docker -./install.sh --core-only # Core without the console +./install.sh # Core + Web + PostgreSQL, no sandbox provider +./install.sh --core-only # Core + PostgreSQL, no sandbox provider +./install.sh --sandbox-provider true --provider microsandbox +./install.sh --sandbox-provider true --provider docker ``` Web-only installation connects the unchanged console to an existing Core; see the diff --git a/deploy/install/configuration.py b/deploy/install/configuration.py index 71d2a36b2..6a6364bb0 100644 --- a/deploy/install/configuration.py +++ b/deploy/install/configuration.py @@ -42,15 +42,17 @@ def core_environment(root, state, database_password): config = str(Path(root) / "config") if native else "/config" database = f'127.0.0.1:{state["database_port"]}' if native else "database:5432" daemon_host = f'host.microsandbox.internal:{state["core_port"]}' if native else "core:8091" - return { + result = { "AGENTS_API_DATABASE_URL": f"postgres://agents_api:{database_password}@{database}/agents_api?sslmode=disable", "AGENTS_API_KEYS_FILE": config + "/keys.json", "AGENTS_API_CREDENTIAL_KEY_FILE": config + "/credential.key", "AGENTS_API_ADDR": f'127.0.0.1:{state["core_port"]}' if native else ":8091", "AGENTS_API_ENGINE": "codex", "AGENTS_API_HARNESSES": "codex,claude_sdk,mcode", - "AGENTS_API_MANAGED_RUNTIMES_FILE": config + "/managed-runtimes.json", "AGENTS_API_DAEMON_WS_URL": f"ws://{daemon_host}/api/v1/agent-daemon/ws", } + if state["provider"]: + result["AGENTS_API_MANAGED_RUNTIMES_FILE"] = config + "/managed-runtimes.json" + return result def compose_config(root, state, manifest, database_password): @@ -83,10 +85,11 @@ def compose_config(root, state, manifest, database_password): services["database"]["ports"] = [f'127.0.0.1:{state["database_port"]}:5432'] services.pop("migrate") else: - core["volumes"].append(bind("/var/run/docker.sock", "/var/run/docker.sock", False)) - core["group_add"] = [str(state["device_gid"])] - core["networks"] = ["default", "runtime"] - doc["networks"] = {"runtime": {"name": state["project"] + "-runtime"}} + if state["provider"] == "docker": + core["volumes"].append(bind("/var/run/docker.sock", "/var/run/docker.sock", False)) + core["group_add"] = [str(state["device_gid"])] + core["networks"] = ["default", "runtime"] + doc["networks"] = {"runtime": {"name": state["project"] + "-runtime"}} services["core"] = core doc["volumes"] = {"database": {}} if state["mode"] != "core-only": diff --git a/deploy/install/install.py b/deploy/install/install.py index e531c14e5..7e96f04ce 100644 --- a/deploy/install/install.py +++ b/deploy/install/install.py @@ -103,7 +103,10 @@ def arguments(argv=None): modes = parser.add_mutually_exclusive_group() modes.add_argument("--core-only", action="store_true") modes.add_argument("--web-only", action="store_true") - parser.add_argument("--provider", choices=("microsandbox", "docker"), default="microsandbox") + parser.add_argument("--sandbox-provider", choices=("true", "false"), nargs="?", const="true", default="false", + help="Prepare a local sandbox provider (default: false)") + parser.add_argument("--provider", choices=("microsandbox", "docker"), + help="Local sandbox provider when enabled (default: microsandbox)") parser.add_argument("--install-dir", type=Path, default=Path.home() / ".parsar/core") parser.add_argument("--core-port", type=int, default=8091) parser.add_argument("--web-port", type=int, default=8080) @@ -112,6 +115,12 @@ def arguments(argv=None): parser.add_argument("--status", action="store_true", help="Read installation health; never invoke a model") parser.add_argument("--stop", action="store_true", help="Stop installed services; retain all data") args = parser.parse_args(argv) + args.sandbox_provider = args.sandbox_provider == "true" + if args.provider and not args.sandbox_provider: + parser.error("--provider requires --sandbox-provider true") + if args.web_only and args.sandbox_provider: + parser.error("--web-only cannot install a sandbox provider") + args.provider = (args.provider or "microsandbox") if args.sandbox_provider else None if args.status and args.stop: parser.error("Choose status or stop") if not args.install_dir.is_absolute(): @@ -194,7 +203,8 @@ def initialize(root, args, manifest): if not token or any(c.isspace() for c in token) or "\x00" in token: raise InstallError("Invalid Core token file") else: - device_gid = os.stat("/dev/kvm" if args.provider == "microsandbox" else "/var/run/docker.sock").st_gid + if args.provider: + device_gid = os.stat("/dev/kvm" if args.provider == "microsandbox" else "/var/run/docker.sock").st_gid token = secrets.token_hex(32) if mode != "web-only": free_port(args.core_port) @@ -202,7 +212,10 @@ def initialize(root, args, manifest): free_port(args.web_port) root.mkdir(mode=0o700, parents=True, exist_ok=True) os.chmod(root, 0o700) - for name in ("config", "state", "state/msb"): + directories = ["config"] + if args.provider == "microsandbox": + directories.extend(("state", "state/msb")) + for name in directories: (root / name).mkdir(mode=0o700) state = {"version": 1, "source_commit": manifest["source_commit"], "mode": mode, "provider": args.provider, "installation_id": str(uuid.uuid4()), @@ -212,13 +225,15 @@ def initialize(root, args, manifest): state["database_port"] = database_port() config = root / "config" if mode != "web-only": - state["device_gid"] = device_gid + if args.provider: + state["device_gid"] = device_gid write_json(config / "keys.json", [{"tenant_id": str(uuid.uuid4()), "organization_id": "installation", "project_id": "default", "subject_kind": "service_account", "subject_id": "operator", "token_sha256": hashlib.sha256(token.encode()).hexdigest()}]) private_write(config / "credential.key", base64.b64encode(secrets.token_bytes(32)).decode()) private_write(config / "database.password", secrets.token_hex(32)) - write_json(config / "managed-runtimes.json", managed_config(root, state, manifest)) + if args.provider: + write_json(config / "managed-runtimes.json", managed_config(root, state, manifest)) private_write(config / "caller.key", token) if mode != "core-only": private_write(config / "console.password", secrets.token_hex(24)) @@ -264,11 +279,13 @@ def main(argv=None): if args.provider == "microsandbox" and not args.web_only: native_service.preflight(bundle) state = initialize(root, args, manifest) - if state["mode"] != "web-only": + if state["provider"] == "docker": seccomp = bundle / "runtime/seccomp.json" if not (root / "config/seccomp.json").exists(): private_write(root / "config/seccomp.json", seccomp.read_text()) - images = ["web"] if state["mode"] == "web-only" else ["core", "runtime", "database"] + images = ["web"] if state["mode"] == "web-only" else ["core", "database"] + if state["provider"] == "docker": + images.append("runtime") if native_service.is_native(state): images = ["database"] password = (root / "config/database.password").read_text() @@ -296,7 +313,10 @@ def main(argv=None): if state["mode"] != "web-only": print(f'API: http://127.0.0.1:{state["core_port"]}/v1') print("Caller key file: " + str(root / "config/caller.key")) - print("Provider: " + state["provider"] + ". Runtime image prepared; Core provisions Sessions on demand.") + if state["provider"]: + print("Provider: " + state["provider"] + ". Runtime image prepared; Core provisions Sessions on demand.") + else: + print("No local sandbox provider configured. No execution node was installed.") print("Services installed. No model request was made. See docs/getting-started/quickstart.md.") diff --git a/deploy/install/test_install.py b/deploy/install/test_install.py index 91399a3da..82020d41f 100644 --- a/deploy/install/test_install.py +++ b/deploy/install/test_install.py @@ -109,7 +109,8 @@ def test_repeat_installation_preserves_execution_identity_and_all_secrets(self): encryption = (self.root / "config/credential.key").read_text() self.assertEqual(hashlib.sha256(caller).hexdigest(), keys[0]["token_sha256"]) self.assertEqual(len(base64.b64decode(encryption, validate=True)), 32) - self.assertEqual(first["installation_id"], self.document("config/managed-runtimes.json")["installation_id"]) + self.assertIsNone(first["provider"]) + self.assertFalse((self.root / "config/managed-runtimes.json").exists()) before = self.snapshot() self.ports.reset_mock() # Re-running against already listening services must not reserve their ports. @@ -122,7 +123,7 @@ def test_repeat_installation_preserves_execution_identity_and_all_secrets(self): def test_configuration_changes_refuse_without_mutating_existing_deployment(self): self.initialize() before = self.snapshot() - for flags in (("--core-only",), ("--provider", "docker"), ("--core-port", "8092"), ("--web-port", "8081")): + for flags in (("--core-only",), ("--sandbox-provider", "true", "--provider", "docker"), ("--core-port", "8092"), ("--web-port", "8081")): with self.subTest(flags=flags), self.assertRaises(install.InstallError): self.initialize(*flags) self.assertEqual(before, self.snapshot()) @@ -131,8 +132,8 @@ def test_configuration_changes_refuse_without_mutating_existing_deployment(self) install.initialize(self.root, self.args(), changed) self.assertEqual(before, self.snapshot()) - def test_default_microsandbox_keeps_core_and_vm_processes_outside_compose(self): - state = self.initialize() + def test_opt_in_microsandbox_keeps_core_and_vm_processes_outside_compose(self): + state = self.initialize("--sandbox-provider", "true") self.assertEqual(state["provider"], "microsandbox") managed = self.document("config/managed-runtimes.json") self.assertNotIn("docker", managed) @@ -159,7 +160,7 @@ def test_default_microsandbox_keeps_core_and_vm_processes_outside_compose(self): self.assertIn("no-new-privileges:true", web["security_opt"]) def test_docker_provider_socket_and_runtime_network_belong_only_to_core(self): - state = self.initialize("--provider", "docker", "--core-only") + state = self.initialize("--sandbox-provider", "true", "--provider", "docker", "--core-only") managed = self.document("config/managed-runtimes.json") self.assertNotIn("microsandbox", managed) self.assertEqual(managed["docker"]["host"], "unix:///var/run/docker.sock") @@ -271,6 +272,51 @@ def test_bundle_rejects_paths_outside_distribution(self): with self.assertRaises(install.InstallError): install.verify_bundle(bundle) + def test_default_has_no_provider_authority_or_runtime_configuration(self): + state = self.initialize() + self.assertIsNone(state["provider"]) + self.assertEqual(self.device_probes, []) + self.assertNotIn("device_gid", state) + self.assertFalse((self.root / "state").exists()) + self.assertFalse((self.root / "config/managed-runtimes.json").exists()) + compose = self.document("compose.json") + self.assertEqual(set(compose["services"]), {"database", "migrate", "core", "web"}) + self.assertNotIn("networks", compose) + for service in compose["services"].values(): + self.assertNotIn("devices", service) + self.assertNotIn("group_add", service) + self.assertNotIn("docker.sock", json.dumps(service.get("volumes", []))) + self.assertNotIn("AGENTS_API_MANAGED_RUNTIMES_FILE", service.get("environment", {})) + + def test_provider_requires_explicit_enablement_and_cannot_belong_to_web_only(self): + self.assertIsNone(self.args("--sandbox-provider", "false").provider) + self.assertEqual(self.args("--sandbox-provider").provider, "microsandbox") + for flags in (("--provider", "docker"), ("--provider", "microsandbox"), + ("--sandbox-provider", "false", "--provider", "docker"), + ("--web-only", "--sandbox-provider", "true")): + with self.subTest(flags=flags), contextlib.redirect_stderr(io.StringIO()), self.assertRaises(SystemExit): + self.args(*flags) + + def test_default_main_skips_kvm_native_service_and_runtime_import(self): + bundle = self.bundle() + calls = [] + output = io.StringIO() + with mock.patch.object(install, "__file__", str(bundle / "install.py")), \ + mock.patch.object(install.platform, "system", return_value="Linux"), \ + mock.patch.object(install.platform, "machine", return_value="x86_64"), \ + mock.patch.object(install, "run", side_effect=lambda args, **kw: calls.append(args)), \ + mock.patch.object(install, "wait_http", return_value=True), \ + mock.patch.object(install.native_service, "preflight", side_effect=AssertionError("native preflight on default")), \ + mock.patch.object(install.native_service, "prepare", side_effect=AssertionError("native install on default")), \ + contextlib.redirect_stdout(output): + install.main(["--install-dir", str(self.root)]) + self.assertEqual(self.device_probes, []) + self.assertEqual([call[-1] for call in calls if call[:2] == ["docker", "load"]], + [str(bundle / ("images/" + name + ".tar")) for name in ("core", "database", "web")]) + self.assertFalse((self.root / "native").exists()) + self.assertFalse((self.root / "config/seccomp.json").exists()) + self.assertIn("No execution node was installed", output.getvalue()) + def test_main_web_only_never_imports_runtime_or_leaks_caller_password(self): source = self.caller_file() bundle = self.bundle() diff --git a/deploy/install/test_native_service.py b/deploy/install/test_native_service.py index 3d97c2a09..036f7dad8 100644 --- a/deploy/install/test_native_service.py +++ b/deploy/install/test_native_service.py @@ -50,7 +50,8 @@ def result(arguments, **kwargs): self.run.side_effect = result def test_only_core_microsandbox_uses_native_service(self): - for mode, provider, expected in (("all", "microsandbox", True), + for mode, provider, expected in (("all", None, False), + ("all", "microsandbox", True), ("core-only", "microsandbox", True), ("web-only", "microsandbox", False), ("all", "docker", False)): diff --git a/docs/getting-started/README.md b/docs/getting-started/README.md index e722a4a77..bc0120ec0 100644 --- a/docs/getting-started/README.md +++ b/docs/getting-started/README.md @@ -9,10 +9,12 @@ Runtime contract. The Parsar product is not required. - [Operate the installation](operations.md) - [Protocol coverage and native differences](https://github.com/MiniMax-AI/parsar-core/blob/main/contracts/agents-api/README.md) -Core and Web ship together. A default installation prepares microsandbox and the -colocated Runtime; choose Docker with `--provider docker`. Core creates the -execution sandbox when a Session needs it. Installing the service does not require -a model key or run a model request. +Core and Web ship together. The default installation runs Core, Web and PostgreSQL +with zero execution nodes. To prepare a local sandbox provider, install with +`--sandbox-provider true --provider microsandbox` or +`--sandbox-provider true --provider docker`. Core then creates the execution sandbox +when a Session needs it. Installing the service does not require a model key or run +a model request. The Web console is a client of Core. API users work with Agents, Sessions and Environments; operators also maintain the host, provider, Runtime images and diff --git a/docs/getting-started/install.md b/docs/getting-started/install.md index e9e3aad47..d35b98964 100644 --- a/docs/getting-started/install.md +++ b/docs/getting-started/install.md @@ -1,24 +1,28 @@ # Install Core and Web Install one matching Parsar Core distribution. By default it starts PostgreSQL, -Core and the existing Web console, prepares microsandbox and imports the Runtime -image. When a Session needs a sandbox, Core asks its Provider to create one from -that image and initializes the colocated daemon, native harness and workspace. +Core and the existing Web console in containers, with zero execution nodes. +It does not import Runtime images, mount the Docker socket or host devices into +Core, or generate a managed Provider configuration. A local sandbox provider is +an explicit installation option. No model key, Environment wizard or sample task is required during installation. ## Host requirements The first distribution targets Linux amd64 with Python 3.9+, Docker and Docker Compose v2. Run the installer as a non-root user who can use Docker. -Default microsandbox also requires glibc, a running systemd user manager with +The default installation requires neither KVM nor systemd user services. +Optional microsandbox also requires glibc, a running systemd user manager with linger enabled, and user read/write access to `/dev/kvm`. Nested cloud hosts must -expose hardware virtualization. Core runs as a native user service so restarting -it does not terminate the Provider's microVM processes. The installer checks these +expose hardware virtualization. With this option, Core runs as a native user +service so restarting it does not terminate the Provider's microVM processes. +The installer checks these prerequisites; it does not grant host permissions or silently fall back to Docker. -The explicit Docker option runs Core in Compose and requires neither KVM nor -systemd user services. +The optional Docker sandbox provider keeps Core in Compose and requires neither +KVM nor systemd user services. -Reserve capacity for the native Runtime: the initial microsandbox profile uses +For optional microsandbox, reserve capacity for the native Runtime: its initial +profile uses 4 GiB RAM, 2 CPUs, an 8 GiB root disk and an 8 GiB environment disk per active sandbox, with at most 4 active and 16 retained allocations. Limits are operator configuration, not model input. Use a trusted, single-operator host and durable local storage. The installer does @@ -43,7 +47,9 @@ cd "$HOME/.parsar/releases/parsar-core--linux-amd64" The bundle contains the same-revision native Core binaries and service image, unchanged Web build, production Web proxy and colocated Runtime. It includes microsandbox's pinned runtime and firmware. It also contains image archives, source provenance and checksums; -installation does not need Go, Node, Rust or a product checkout. +installation does not need Go, Node, Rust or a product checkout. The default +installation loads only the Core, Web and PostgreSQL images; Runtime and +microsandbox payloads are used only when a sandbox provider is enabled. Installation creates private configuration under `~/.parsar/core`, a dedicated PostgreSQL volume, an API caller key and a credential encryption key. It also @@ -61,16 +67,24 @@ the server holds the independent Core key. Model keys remain API execution input ## Installation choices ```sh -./install.sh --provider docker +./install.sh --sandbox-provider true --provider microsandbox +./install.sh --sandbox-provider true --provider docker ./install.sh --core-only -./install.sh --core-only --provider docker +./install.sh --core-only --sandbox-provider true --provider docker ``` -`--provider` selects the deployment's sandbox provider. It does not select a -harness or alter the public `openai_hosted` discriminator. Both providers reuse -one colocated Runtime containing the native harnesses. The Docker option grants +`--sandbox-provider true` enables a local sandbox provider. If `--provider` is +omitted, it selects microsandbox. Supplying `--provider` without enabling the +sandbox provider is an error. `--core-only` installs Core and PostgreSQL without +Web and has no sandbox provider unless explicitly enabled. + +The provider choice does not select a harness or alter the public `openai_hosted` +discriminator. Both providers reuse one colocated Runtime containing the native +harnesses. The Docker option grants only Core access to the Docker socket; microsandbox uses the native service account's -KVM access. Web receives neither. +KVM access. Web receives neither. When a Session needs a sandbox, Core asks the +enabled Provider to create one from the prepared Runtime image and initializes +the colocated daemon, native harness and workspace. To install only Web on a Linux host, provide the existing Core origin and a private caller-key file. A loopback Core uses the same host network namespace; @@ -83,15 +97,18 @@ a remote Core must use HTTPS. --core-token-file "$HOME/.parsar/core/config/caller.key" ``` -Web-only mode starts no database or Core and requires no KVM. Its key remains on -the server, outside the static Web files. The input file must be private (0600). +Web-only mode cannot enable a sandbox provider. It starts no database or Core and +requires no KVM. Its key remains on the server, outside the static Web files. +The input file must be private (0600). Use `--install-dir /absolute/path`, `--core-port 8092` and `--web-port 8081` for separate installations. Their database volumes, provider identities and Runtime state are independent. Native Core connects to PostgreSQL through an automatically selected loopback-only port, recorded in its private installation state. Repeating the same installation command retains its -identities, secrets and data. Conflicting mode/provider/revision changes refuse -rather than silently replacing them. +identities, secrets and data. The installer refuses mode, sandbox-provider and +revision changes on an existing installation. This includes enabling a sandbox +provider on an installation originally created without one; rerunning with new +flags does not migrate it. For remote browser or SDK access, put the intended endpoint behind your existing TLS and access-control boundary. Update the console's trusted origin explicitly; @@ -99,8 +116,10 @@ do not simply publish its port on every network interface. ## After installation -Start with an optional [API request](quickstart.md). Session creation supplies the -model, harness and write-only model credentials. Core owns sandbox preparation and +Start with an optional [API request](quickstart.md). The read-only example works +with the default installation. The execution example requires an installation +created with a sandbox provider enabled. Session creation supplies the model, +harness and write-only model credentials. Core owns sandbox preparation and Runtime startup. Configuration is never injected into a public Agent instruction or baked into a Runtime image. diff --git a/docs/getting-started/operations.md b/docs/getting-started/operations.md index b52d6cbea..4353833ad 100644 --- a/docs/getting-started/operations.md +++ b/docs/getting-started/operations.md @@ -1,9 +1,10 @@ # Operate your Core -The API abstracts execution environments for clients. The installation operator -also owns the host, container or microVM provider, storage and service availability. -Those are separate from a Session's public execution state. This guide adds that -self-deployment view without adding a second Runtime controller. +The installation operator owns the host, storage and service availability. +The default installation has zero execution nodes. When a sandbox provider is +enabled during installation, the operator also maintains its containers or +microVMs. Provider operations below apply to that optional configuration. Service +health and provider state are separate from a Session's public execution state. ## Read service health @@ -86,8 +87,9 @@ apply the existing Core migration workflow and replace matched service/Runtime artifacts while retaining identities and backend paths. Qualify recovery before claiming the upgrade complete; there is no downgrade or history migration promise. -Provider replacement is an operator operation, not a new `--provider` value on an -existing install. Follow the [maintenance and provider-switch procedure](https://github.com/MiniMax-AI/parsar-core/blob/main/services/agents-api/deploy/microsandbox/README.md#change-the-deployment-provider). +The installer refuses to enable, disable or replace a sandbox provider on an +existing installation. Changing flags and rerunning is not a migration procedure. +For an installation with a provider, follow the [maintenance and provider-switch procedure](https://github.com/MiniMax-AI/parsar-core/blob/main/services/agents-api/deploy/microsandbox/README.md#change-the-deployment-provider). The installer never migrates Sessions between providers or deletes old compute. ## Exposure and network policy @@ -109,9 +111,12 @@ for the colocated Runtime, and denied inbound/private-network access. Private model/MCP endpoints require an explicit operator policy change. Native tool network policy remains the Session's separate public configuration. -Docker uses the existing qualified nested-sandbox Runtime policy. Only Core can -access the selected host Docker daemon. Install on a trusted service host and do -not share its Docker authority with untrusted users. +The optional Docker sandbox provider uses the existing qualified nested-sandbox +Runtime policy. Only Core can access the selected host Docker daemon. +Install on a trusted service host and do +not share its Docker authority with untrusted users. The default installation +does not mount the Docker socket or host devices into Core, import Runtime images +or generate managed Provider configuration. Host virtualization, credentials, tenant isolation, durable state and actual execution are release acceptance requirements. Other missing operational screens diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md index 06430d229..98d8de424 100644 --- a/docs/getting-started/quickstart.md +++ b/docs/getting-started/quickstart.md @@ -32,6 +32,11 @@ environment. Installation has no mandatory sample task. ## Run a Session when you are ready +This example requires an installation created with a local sandbox provider +enabled. The default installation has zero execution nodes. Choose microsandbox +or Docker using the [installation options](install.md#installation-choices); +adding these flags to an existing default installation is not a supported migration. + Core creates the sandbox through its Provider using the prepared Runtime image, then initializes the daemon, native harness and workspace inside it. You do not install or start a separate daemon for a Core-managed Session. The public discriminator remains diff --git a/site/index.html b/site/index.html index 5986693aa..ac687c8c4 100644 --- a/site/index.html +++ b/site/index.html @@ -65,10 +65,10 @@

Agents API
infrastructure.
On your terms.
-

02 / Installation

Start with a
local bundle.

Download or build a bundle for your host, extract it, then run the installer from that directory.

The default installs Core and the Web console, and prepares microsandbox and Runtime images. Core provisions the Runtime when you create an execution Session.

Installation guide
+

02 / Installation

Start with a
local bundle.

Download or build a bundle for your host, extract it, then run the installer from that directory.

The default runs Core, the Web console and PostgreSQL with zero execution nodes. A local sandbox provider is an explicit installation option.

Installation guide
-

Core + Web

DEFAULT
./install.sh

Uses microsandbox. No model provider key is needed at install time.

-

Use Docker

Choose the Docker sandbox provider.

./install.sh --provider docker
+

Core + Web

DEFAULT
./install.sh

Includes PostgreSQL. No sandbox provider, KVM or systemd user service required.

+

Local sandbox

Optional microsandbox. Add --provider docker to choose Docker.

./install.sh --sandbox-provider true

Core only

Deploy the API without the Web console.

./install.sh --core-only

Web only

Connect the console to an existing Core.

./install.sh --web-only

Bundle availability and supported hosts are documented in the installation guide. Browse releases ↗

From a5b4d140f5dd3ee6315baa5a90e6700882c73562 Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Wed, 23 Sep 2026 18:42:46 +0800 Subject: [PATCH 17/17] fix(distribution): produce readable non-root runtime payloads --- CONTRIBUTING.md | 3 +++ scripts/build-core-distribution.sh | 3 +++ 2 files changed, 6 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a28b03f07..42e54bf24 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1356,6 +1356,9 @@ microsandbox runtime/firmware hashes and executable native payloads. Release gen qualification. A release must be tested from fresh extraction with real models; no synthetic result may substitute for native execution acceptance. +The distribution build sets umask 022 for non-root-readable payloads; installation +credentials and state retain their explicit private permissions. + The first installer targets a trusted Linux amd64 Docker host. It installs a private dedicated PostgreSQL service and separate Core and console services in Compose by default, with zero execution nodes. The default requires neither KVM diff --git a/scripts/build-core-distribution.sh b/scripts/build-core-distribution.sh index 23704a5c0..528954681 100755 --- a/scripts/build-core-distribution.sh +++ b/scripts/build-core-distribution.sh @@ -1,6 +1,9 @@ #!/usr/bin/env bash set -euo pipefail +# Distribution payloads must remain readable by the non-root service users. +umask 022 + repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" runtime_root="${PARSAR_HOME:-$HOME/.parsar}" output_dir="${CORE_DISTRIBUTION_BUILD_DIR:-$runtime_root/build/core-distribution}"