diff --git a/CHANGELOG.md b/CHANGELOG.md index 158fbf7..b6ecda9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,3 +29,7 @@ fixture passes; semantic attack detection has not been independently evaluated. See `docs/deployment.md` for backup, restore, retention and SSO setup. SDKs remain source distributions. The release workflows prepare container images, a CLI archive, checksums and a draft GitHub release; publication is a separate action. + +### Standalone CLI + +CLI 0.2.0 now classifies with bundled local rules immediately after installation. Docker and a server are optional. Explicit `--semantic` calls TypeSafe directly; configured gateways keep the server client workflow. diff --git a/README.md b/README.md index 3b99918..6e07729 100644 --- a/README.md +++ b/README.md @@ -24,12 +24,12 @@ ## What Pyro does -Pyro is a self-hosted policy API and dashboard for teams adding LLM features or +Pyro is a standalone CLI, self-hosted policy API and dashboard for teams adding LLM features or tool-using agents. Call it before forwarding an untrusted prompt, retrieved passage, or tool payload. It returns `allow`, `review`, or `block`; your application must hold review decisions and reject blocked ones before executing work. -Local rules run on your server. Semantic detectors currently use **TypeSafe +Local rules run in the CLI process or on your server. Semantic detectors currently use **TypeSafe System One** and send inputs to that provider. Your application's LLM can be from any vendor, but the semantic classifier implementation is currently TypeSafe. Pyro does not automatically intercept a model or tool call and cannot guarantee @@ -54,62 +54,79 @@ See [deployment and data retention](docs/deployment.md), [release/support notes] ## Quick start -Pyro is early-beta software. The CLI connects to a server; installing it does not start one. Local rules need no provider account. Semantic screening uses TypeSafe's hosted API and sends the input there. +### Install and classify — no Docker required -### 1. Start a server, or use an existing instance - -Save [compose.yaml](https://delvisor.com/pyro/compose.yaml) and [.env.example](https://delvisor.com/pyro/pyro.env.example) in an empty folder. No source checkout is needed. With Docker Compose installed: +With Node.js 22.13+ and pnpm: ```sh -cp .env.example .env -# Fill in the four required credentials using the template's generation commands. -docker compose up --build -d +pnpm add --global @delvisor/pyro +pyro classify 'Summarize this document.' +pyro classify -- '-----BEGIN PRIVATE KEY-----' ``` -Open [the dashboard](http://localhost:3000) and sign in using `ADMIN_PASSWORD` from `.env`. The gateway listens on port 8080 and the management API on 8081. Keep those interfaces private; see [deployment guidance](./SECURITY.md). - -### 2. Install the published CLI - -Use Node.js 22.13+ and pnpm, or install the same package with your preferred npm-compatible package manager: +CLI 0.2+ bundles the classification engine and local-secrets policy. Expect +`allow` then `block`, with `execution: standalone`. No server, database, account, +policy download or provider key is needed. The synthetic header tests a specific +local rule; it is not a semantic detection benchmark. ```sh -pnpm add --global @delvisor/pyro -pyro config set gateway-url http://localhost:8080 -pyro config set control-url http://localhost:8081 -pyro auth login +pyro classify --file prompt.txt +pyro classify 'A document to inspect' --profile-file ./my-policy.yaml +pyro doctor --local ``` -Use your own server URLs if connecting to an existing instance. CLI 0.2.0 adds `pyro doctor` for connection diagnostics and `pyro doctor --semantic` to check provider configuration. These checks send no prompts; configured credentials do not prove that a provider is reachable or accurate. +Standalone results go to stdout; no input history or background services are +created. A configured gateway URL or `PYRO_API_KEY` selects the existing server +mode. `--local` overrides those settings; `--remote` explicitly uses the server. +See the [CLI guide](packages/cli/README.md) for files, stdin and structured inputs. -### 3. Get a decision without a provider key +### Optional semantic screening -Download [local-secrets.yaml](https://delvisor.com/pyro/profiles/local-secrets.yaml), then run from that folder: +Set `TYPESAFE_API_KEY` in your environment, then run: ```sh -pyro profiles import --file ./local-secrets.yaml -pyro playground 'Summarize this document.' --profile local-secrets -pyro playground 'Example: -----BEGIN PRIVATE KEY-----' --profile local-secrets +pyro classify 'Text to inspect' --semantic ``` -Expect `allow` for the first request and `block` for the second. Both use local rules and appear in Activity. The fake header is test data, not a real secret. This verifies integration, not general prompt-injection detection. Imports reject duplicate IDs; skip the import if already installed. +This calls TypeSafe directly using the bundled balanced-assistant policy. +`--semantic` explicitly authorizes sending inputs to that provider and its usage +charges. It needs no Docker or Pyro server. Missing keys fail before the request; +provider failures return an indeterminate verdict and the policy's failure action. +A fail-closed block is not evidence that an attack was detected. -### 4. Enable semantic screening explicitly +### Optional shared dashboard and team workflows -Get a key from [TypeSafe](https://console.typesafe.ai/) and add it in **Settings → Classifier provider**. Hosted screening sends inputs to TypeSafe; review its data terms and usage charges. Download and import [balanced-assistant.yaml](https://delvisor.com/pyro/profiles/balanced-assistant.yaml), then select that profile. Local patterns can flag quoted or educational text; evaluate representative benign and attack examples before enforcement. +For durable jobs, activity history, policy rollouts, evaluations and team access, +use an existing server or save [compose.yaml](https://delvisor.com/pyro/compose.yaml) +and [.env.example](https://delvisor.com/pyro/pyro.env.example) in an empty folder. +No source clone is needed. With Docker Compose installed: -Without a configured provider, a semantic request returns an `indeterminate` verdict and follows the profile's failure policy. A fail-closed `block` is not evidence of an attack. Keep fail-closed behavior for workloads that require it; use the explicit local-only preset to try Pyro without a key. +```sh +cp .env.example .env +# Fill in the four required credentials using the generation commands in the file. +docker compose up --build --wait --wait-timeout 180 +pyro config set gateway-url http://localhost:8080 +pyro config set control-url http://localhost:8081 +pyro auth login +``` -### 5. Connect an application +Open [the dashboard](http://localhost:3000) and sign in with `ADMIN_PASSWORD`. +Keep the interfaces private; see [deployment guidance](docs/deployment.md). +Download [local-secrets.yaml](https://delvisor.com/pyro/profiles/local-secrets.yaml), +then import it for server use and create an application key: ```sh -pyro apps create --name 'Support' --default-profile-id local-secrets -# Substitute the ID returned above. +pyro profiles import --file ./local-secrets.yaml +pyro apps create --name Support --default-profile-id local-secrets pyro keys create --name 'Support backend' --app-id APP_ID -# Set PYRO_API_KEY to the one-time key shown in the response. -pyro classify 'Summarize this document.' --profile local-secrets +# Set PYRO_API_KEY to the one-time key shown above; use the returned APP_ID. +pyro classify 'Summarize this document.' --remote --profile local-secrets ``` -Keep the key on your backend. Your application enforces `allow`, `review`, and `block`; Pyro does not automatically intercept model calls. Begin in staging, or record decisions without changing your existing controls. A successful CLI classification exits 0 for any decision; scripts must inspect `action` and `verdict`. +Server semantic profiles use the provider configured in dashboard Settings. +Standalone semantic checks use your local `TYPESAFE_API_KEY`. Your application +must enforce the returned action: continue only on allow, hold review, and reject +block. Successful classification exits 0 for any action; scripts must inspect it. ## Protection profiles diff --git a/docker-compose.yml b/docker-compose.yml index f89a1ce..5b47c9f 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -59,6 +59,7 @@ services: GATEWAY_API_KEY: ${GATEWAY_API_KEY:?Set GATEWAY_API_KEY in .env} CONTROL_PLANE_SECRET: ${CONTROL_PLANE_SECRET:?Set CONTROL_PLANE_SECRET in .env} ADMIN_PASSWORD: ${ADMIN_PASSWORD:?Set ADMIN_PASSWORD in .env} + EVENT_RETENTION_DAYS: ${EVENT_RETENTION_DAYS:-30} OIDC_ISSUER: ${OIDC_ISSUER:-} OIDC_CLIENT_ID: ${OIDC_CLIENT_ID:-} OIDC_CLIENT_SECRET: ${OIDC_CLIENT_SECRET:-} diff --git a/docs/releases.md b/docs/releases.md index ecc77c4..b8c1523 100644 --- a/docs/releases.md +++ b/docs/releases.md @@ -51,6 +51,11 @@ publication, then update website links and the compatibility notes together. ## Support policy +CLI 0.2+ supports standalone classification with no Docker or server. CLI 0.1 +was a server client. Publish the tested 0.2 archive before deploying the new +standalone-first website instructions; verify a fresh install runs both local +allow/block examples without a server. + During beta, fixes target the newest published beta. main is development code, not a release channel. Pin the CLI version, server image digest, PostgreSQL major version and policy revision in deployments. Keep the previous release archive diff --git a/docs/validation.md b/docs/validation.md index 707a309..1d66ad7 100644 --- a/docs/validation.md +++ b/docs/validation.md @@ -29,3 +29,72 @@ representative semantic evaluation with explicit provider-cost authorization, load testing at intended traffic, and verification of published artifacts after release. Release workflows are prepared; this record does not assert that an npm version, container image, GitHub release or website deployment was published. + +## Standalone CLI follow-up + +The packed CLI installs outside the repository with installation scripts disabled, +then returns local allow/block decisions without any server or provider key. +All 20 CLI tests pass, including files/stdin, saved-server overrides, explicit +semantic consent and a fixture intercepting direct TypeSafe calls. The full +workspace check also passes. No live TypeSafe request was made. + +## Earlier validation history + +# Validation for the dashboard, profiles, outgoing webhooks and SDKs + +Webhook form follow-up (2026-09-23): + +- Replaced comma-separated resource IDs with named, searchable application/profile checklists built from shadcn/ui Radix primitives. Added field help, explicit All scopes, inline risk/action validation and loading/retry feedback. +- Dashboard tests: 12 passed. New coverage checks ID serialization, explicit wildcard selection, preservation of unavailable saved references and disabled webhook state, unchanged destination handling when editing, and risk/action validation. Dashboard production build and whitespace checks passed. +- Verified live application/profile choices, search with and without matches, multiple selection, keyboard toggling, Escape dismissal, focus return and minimum-risk help on localhost. Clearing the final specific selection disables Save. Verified the migrated shared checkbox on Applications without saving changes. +- Checked both themes and a 390px viewport. Corrected nested dialog scrolling so only the form body scrolls; header/footer and checklist contents stay within the viewport. Restored light mode and desktop size, and cancelled test drafts without creating a webhook. +- Restarted Vite on `127.0.0.1:3000` to clear stale imports after adding dependencies. No new runtime errors were observed after restart. + +Webhook presentation follow-up (2026-09-23): + +- Renamed product copy and navigation references to Webhooks across the dashboard, website and guides; retained technical descriptions of outgoing delivery. +- Removed the local receiver help panel. Delivery history now uses the card width, separates timestamps, labels HTTP responses, shows status badges and only includes an Action column when a delivery can be retried. +- Dashboard and website production builds passed, along with `git diff --check`. Verified Refresh, Add webhook, light/dark presentation and a 390px viewport against existing local delivery history. The table scrolls inside its card without page overflow; no browser errors or warnings were observed. +- Restored light mode and the desktop viewport. Restarted the standalone website on `127.0.0.1:3100` and verified its updated home and documentation copy. All services remain local. + +Settings, navigation and library follow-up (2026-09-23): + +- Dashboard tests: 8 passed, covering preference migration/validation, library filtering and response defaults, stable detector identity, and historical API response handling. Production dashboard build and whitespace checks passed. +- Verified all six dashboard preference controls, persistence after reload, reset to defaults, System theme selection, compact table cell padding, full Activity timestamps and the neutral-black dark palette (`#050505` canvas). The existing classifier settings remain available separately. +- Verified the restored dropdown opening animation and 180ms sliding highlight, arrow-key selection and focus return inside a profile dialog. Reduced motion suppresses transitions. +- Verified profile library text search, local-only filtering, empty results, YAML inspection and customization into an editable draft. Cancelled the draft without modifying saved policies. +- Verified Playground is under Observe, there is no Test group, Settings has a separate sidebar entry and sidebar hover fills are removed. +- Settings and the library fit a 390px viewport without page overflow. Restored the desktop viewport and default personal preferences after testing. Everything remains on localhost. + +Dashboard redesign verification (2026-09-23): + +- Dashboard unit tests: 4 passed, including detector editor identity and payload serialization. The editor-only row key stays stable while its API ID changes and is omitted from saves. +- Production dashboard build and `git diff --check`: passed. +- Verified continuous character-by-character typing in both a new detector and an existing detector on `localhost:3000`; full values appeared and focus stayed in the ID input. Cancelled both drafts without modifying saved policies. +- Verified shared dropdown keyboard selection and focus restoration inside the profile dialog. Confirmed Activity filters and request trace dialogs still work. +- Verified neutral light and dark themes, self-hosted Open Sans, themed charts/dialogs, and dark preference persistence after reload. Returned the dashboard to light mode. +- Navigated all nine dashboard pages successfully. A fresh final reload produced no browser console errors or warnings. +- At a 390px viewport, checked collapsible navigation, profile dialogs, Overview, Usage, Applications, Activity and outgoing webhooks. Corrected Applications overflow; those pages fit the viewport, with tables scrolling within their containers. Restored the desktop viewport afterward. + +The shared UI conventions and repeatable browser checks are documented in `apps/dashboard/README.md`. + +Latest local verification (2026-09-23): + +- Reproduced the Protection Profiles white screen on the actual `localhost:3000` dashboard: older backend records omitted `localRules`. Schema defaults now normalize those records before rendering/editing. A page error boundary keeps navigation available if any page fails. +- Verified Activity against the existing database: the list, filters and an existing request trace render. Historical traces without detector arrays and responses without label catalogs have regression coverage; failed requests are shown in the page. +- Verified existing profile cards, the existing profile editor, all four curated presets, outgoing webhook configuration and delivered history in the browser after updating the running services. +- `npm run typecheck`: passed. +- `npm test` with `TEST_DATABASE_URL` pointing to an isolated PostgreSQL 17 instance: 42 tests passed, none skipped. This includes three dashboard compatibility regressions and a worker test ensuring unsupported persisted destinations cannot enqueue or send. +- `docker compose build gateway control-plane`: passed, including all package/application production builds. Both containers were recreated from the new images and report healthy. The existing Vite dashboard remains on port 3000; PostgreSQL data was retained. +- `npm run test:webhook` against the actual local Docker gateway/control plane: passed. Verified HMAC on `integration.test` and a real local-rule `decision.created` event; a deliberate HTTP 503 caused a retry and HTTP 204 completed delivery. Temporary destination/profile removed; audit records retained. All traffic stayed local, with no model request. +- The standalone website passed its Next.js production build and was restarted on `127.0.0.1:3100` with outgoing-webhook-only copy. +- `git diff --check`: passed. + +Earlier verification for the unchanged SDK/profile work: + +- PostgreSQL checks cover atomic event/outbox writes, duplicate suppression, concurrent claims, expired leases, stale-worker acknowledgements, manual retry and rollback on an invalid delivery. +- Rust SDK: four tests and Clippy with warnings denied passed. Existing Python SDK test passed. +- TypeScript SDK/contracts packed and installed in a separate temporary consumer; imports and a request succeeded. +- Website desktop/mobile layouts, interactive examples, documentation navigation and YAML downloads checked. + +The optional observability overlay and non-webhook adapter have been removed. Neither SDK has been published. The website now lives in the sibling `website` directory alongside Delvisor's homepage, with Pyro at `/pyro` and its quickstart at `/pyro/docs`. diff --git a/packages/cli/README.md b/packages/cli/README.md index b72e520..eab200f 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -1,21 +1,58 @@ # Pyro CLI -Classify inputs, inspect decisions, and manage the same applications, profiles, keys, -webhooks and provider settings as the dashboard. Requires Node.js 22 or newer and a -running Pyro server. The CLI does not start services. +Install once and classify immediately. Standalone checks run in the CLI process: +no Docker, server, database, account or sign-in is required. Server administration +commands remain available when you want a shared dashboard and team workflows. -## Install +## Install and get a decision -The CLI is published on npm. With Node.js 22.13+ and pnpm: +With Node.js 22.13+ and pnpm: ```sh pnpm add --global @delvisor/pyro -pyro --help +pyro classify 'Summarize this document.' +pyro classify -- '-----BEGIN PRIVATE KEY-----' ``` -A running Pyro server is required. Follow the [Docker quickstart](https://delvisor.com/pyro/docs#setup) to start one without cloning the repository, or connect to an existing instance. The CLI does not host the gateway, dashboard or database. +CLI 0.2+ includes the engine and local-secrets policy. Expect `allow` then `block` +with `execution: standalone`; no provider key or network call is needed. These +rules match specific credential shapes, not arbitrary semantic attacks. The +private-key header above is synthetic test data. Check `pyro --version` when +upgrading from the earlier server-client-only release. -CLI 0.2.0 adds `pyro doctor` (server checks) and `pyro doctor --semantic` (also requires classifier configuration). No prompts or credentials are sent to a model during diagnostics. Missing semantic configuration does not prevent local-only profiles from working. +```sh +pyro classify --file prompt.txt +printf '%s' 'A document to inspect' | pyro classify +pyro classify --input '{"messages":[{"role":"user","content":"Hello"}]}' +pyro classify --profile-file ./my-policy.yaml 'Text to inspect' +pyro doctor --local +``` + +Built-in policies need no download or import. Custom YAML/JSON policies use the +same profile schema as the server. Standalone results go to stdout (or a private +`--output` file); the CLI does not retain inputs or start background services. +Shadow policies and shared history/jobs/reviews require a server. + +### Optional semantic screening + +Set `TYPESAFE_API_KEY` in your environment, then opt into a direct provider call: + +```sh +pyro classify 'Text to inspect' --semantic +pyro classify --file prompt.txt --semantic --profile strict-tool-agent +``` + +`--semantic` authorizes sending the input to TypeSafe and its usage charges. It +uses the bundled balanced-assistant policy unless you select another policy. +A missing key is a setup error before any request. An unavailable provider +returns an indeterminate decision following the profile's failure policy, not a +claim that an attack was detected. There are no automatic provider retries in +standalone mode. `pyro doctor --local --semantic` checks key presence only. + +A configured gateway URL or `PYRO_API_KEY` preserves the existing server mode. +Use `--local` to force standalone execution despite saved server settings, or +`--remote` to explicitly use the gateway. `--semantic` and `--profile-file` select +standalone execution; they cannot be combined with `--remote`. For development from a checkout: @@ -25,9 +62,9 @@ pnpm --filter @delvisor/pyro pack --pack-destination artifacts # Install the generated tarball from artifacts/. ``` -## Start with your dashboard +## Optional: connect to a shared server -First download and import [local-secrets.yaml](https://delvisor.com/pyro/profiles/local-secrets.yaml) after signing in: `pyro profiles import --file ./local-secrets.yaml`. That preset checks credential shapes locally and needs no TypeSafe key. Semantic profiles require a key in Settings → Classifier provider and send inputs to that provider. A missing or unavailable classifier produces an indeterminate verdict and follows the configured fail mode; it does not mean an attack was detected. +For dashboard/team features, start the optional [Docker setup](https://delvisor.com/pyro/docs#setup) or use an existing server. Then download and import [local-secrets.yaml](https://delvisor.com/pyro/profiles/local-secrets.yaml) after signing in: `pyro profiles import --file ./local-secrets.yaml`. That preset checks credential shapes locally and needs no TypeSafe key. Semantic profiles require a key in Settings → Classifier provider and send inputs to that provider. A missing or unavailable classifier produces an indeterminate verdict and follows the configured fail mode; it does not mean an attack was detected. ```sh @@ -90,13 +127,14 @@ gateway trace headers. | Webhooks | `webhooks list`, `create`, `update`, `delete`, `test`, `rotate-secret`, `deliveries`, `retry` | | Provider settings | `settings provider get`, `settings provider update` | | Dashboard authentication | `auth login`, `auth status`, `auth logout` | -| Application gateway | `classify`, `jobs create`, `jobs get`, `events`, `gateway profiles` | +| Standalone | `classify`, `classify --semantic`, `classify --profile-file`, `doctor --local` | +| Application gateway | `classify --remote`, `jobs create`, `jobs get`, `events`, `gateway profiles` | | Service diagnostics | `health`, `ready`, `metrics`, `control health`, `gateway info` | | CLI configuration / API contracts | `config show`, `config set`, `spec gateway`, `spec control` | Browser-only preferences such as theme remain dashboard preferences; they are not server settings. `playground` uses the same session and gateway key as the -dashboard playground. `classify`, `jobs` and `events` use your application key and +dashboard playground. Remote `classify`, `jobs` and `events` use your application key and respect its permissions. Every HTTP operation and WebSocket endpoint in both OpenAPI contracts has a diff --git a/packages/cli/package.json b/packages/cli/package.json index 59acab1..df0fa85 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -1,7 +1,7 @@ { "name": "@delvisor/pyro", "version": "0.2.0", - "description": "Classify inputs, inspect decisions, and manage a self-hosted Pyro server.", + "description": "Classify inputs locally or with TypeSafe, and manage an optional Pyro server.", "type": "module", "license": "Apache-2.0", "bin": { @@ -16,7 +16,7 @@ "node": ">=22.0.0" }, "scripts": { - "build": "tsc -p tsconfig.json && node scripts/bundle-specs.mjs", + "build": "tsc -p tsconfig.json && node scripts/bundle-specs.mjs && node scripts/bundle-standalone.mjs", "typecheck": "tsc -p tsconfig.json --noEmit", "test": "pnpm run build && node --import tsx --test test/*.test.ts", "prepack": "pnpm run build", @@ -27,8 +27,10 @@ "ws": "^8.18.3" }, "devDependencies": { + "@pyro/classifiers": "workspace:*", "@pyro/contracts": "workspace:*", "@types/ws": "^8.18.1", + "esbuild": "^0.28.2", "tsx": "^4.20.6", "yaml": "^2.9.1", "zod": "^4.1.11" diff --git a/packages/cli/scripts/bundle-standalone.mjs b/packages/cli/scripts/bundle-standalone.mjs new file mode 100644 index 0000000..b956326 --- /dev/null +++ b/packages/cli/scripts/bundle-standalone.mjs @@ -0,0 +1,26 @@ +import { build } from 'esbuild'; +import { readFile, writeFile, readdir } from 'node:fs/promises'; +import YAML from 'yaml'; +// Runtime users get a self-contained engine, with no workspace packages, +// compiler, native addon, service or installation lifecycle script required. +await build({ entryPoints: ['src/standalone.ts'], outfile: 'dist/standalone.js', bundle: true, platform: 'node', format: 'esm', target: 'node22', minify: false, banner: { js: "import { createRequire as createStandaloneRequire } from 'node:module'; const require = createStandaloneRequire(import.meta.url);" } }); +const profiles = {}; +for (const name of await readdir(new URL('../../../profiles/', import.meta.url))) { + if (!name.endsWith('.yaml')) continue; + const document = YAML.parse(await readFile(new URL(`../../../profiles/${name}`, import.meta.url), 'utf8')); + profiles[document.profile.id] = document.profile; +} +await writeFile(new URL('../dist/profiles.json', import.meta.url), JSON.stringify(profiles)); +// Preserve full notices for the dependencies embedded in the standalone bundle. +const { createRequire } = await import('node:module'); +const { dirname } = await import('node:path'); +const { existsSync } = await import('node:fs'); +let notices = 'Third-party dependencies bundled into the Pyro standalone CLI\n'; +for (const [name, manifest] of [['yaml', '../package.json'], ['zod', '../package.json'], ['re2js', '../../classifiers/package.json']]) { + const require = createRequire(new URL(manifest, import.meta.url)); + let directory = dirname(require.resolve(name)); + while (!existsSync(`${directory}/package.json`)) directory = dirname(directory); + const metadata = JSON.parse(await readFile(`${directory}/package.json`, 'utf8')); + notices += `\n\n${metadata.name} ${metadata.version}\n\n${await readFile(`${directory}/LICENSE`, 'utf8')}`; +} +await writeFile(new URL('../dist/THIRD_PARTY_NOTICES.txt', import.meta.url), notices); diff --git a/packages/cli/scripts/test-install.mjs b/packages/cli/scripts/test-install.mjs index 1f679fe..392067d 100644 --- a/packages/cli/scripts/test-install.mjs +++ b/packages/cli/scripts/test-install.mjs @@ -25,7 +25,17 @@ try { assert.match(stdout, /Pyro/); const version = await exec(executable, ["--version"], options); assert.equal(version.stdout.trim(), manifest.version); + const cleanEnv = { ...options.env }; + for (const key of Object.keys(cleanEnv)) if (key.startsWith("PYRO_") || key === "TYPESAFE_API_KEY") delete cleanEnv[key]; + const localOptions = { ...options, env: { ...cleanEnv, PYRO_CONFIG: join(directory, "fresh-config.json") } }; + for (const [input, action] of [["hello", "allow"], ["-----BEGIN PRIVATE KEY-----", "block"]]) { + const result = await exec(executable, ["classify", "--local", "--", input], localOptions); + const decision = JSON.parse(result.stdout); + assert.equal(decision.action, action); assert.equal(decision.execution, "standalone"); + } + const doctor = await exec(executable, ["doctor", "--local"], localOptions); + assert.equal(JSON.parse(doctor.stdout).serverRequired, false); const spec = await exec(executable, ["spec", "control"], options); assert.equal(JSON.parse(spec.stdout).openapi, "3.1.0"); - console.log("Packed CLI installs globally and runs outside the repository."); + console.log("Packed CLI installs globally and classifies outside the repository without Docker or a server."); } finally { await rm(directory, { recursive: true, force: true }); } diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index d721f1e..2e4405b 100644 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -66,6 +66,21 @@ async function runEndpoint(endpoint: Endpoint, command: Command, args: string[]) const config = await readConfig(path); const resolved = settings(global, config); const baseUrl = resolved[endpoint.service]; + const wantsLocal = options.local || options.semantic || options.profileFile; + const hasServer = options.remote || global.gatewayUrl || process.env.PYRO_GATEWAY_URL || config.gatewayUrl || process.env.PYRO_API_KEY; + if (endpoint["x-cli-command"] === "classify" && (wantsLocal || !hasServer)) { + if (options.remote && wantsLocal) throw new Error("Choose standalone flags or --remote, not both."); + const { body, contentType } = await buildBody(endpoint, options, args); + const outputPath = options.output as string | undefined; + const outputFile = outputPath ? await open(outputPath, "wx", 0o600) : undefined; + let complete = false; + try { + const { classifyStandalone } = await import("./standalone.js"); + const result = await classifyStandalone(body!, contentType!, { profileFile: options.profileFile as string | undefined, semantic: Boolean(options.semantic), timeout: resolved.timeout, requestId: options.xRequestId as string | undefined }); + await output(result, { ...global, outputFile }); complete = true; + } finally { await outputFile?.close(); if (outputFile && !complete) await rm(outputPath!, { force: true }); } + return; + } const headers = credentials(endpoint, baseUrl, config); let route = endpoint.path; let position = 0; @@ -135,20 +150,31 @@ async function runEndpoint(endpoint: Endpoint, command: Command, args: string[]) export function createProgram(): Command { const { version } = JSON.parse(readFileSync(new URL("../package.json", import.meta.url), "utf8")) as { version: string }; const program = new Command().name("pyro").version(version) - .description("Pyro — classify inputs and manage a running Pyro server.") + .description("Pyro — classify inputs standalone, or connect to an optional Pyro server.") .option("--gateway-url ", "gateway URL (PYRO_GATEWAY_URL; default http://localhost:8080)") .option("--control-url ", "dashboard API URL (PYRO_CONTROL_URL; default http://localhost:8081)") .option("--config ", "config file (PYRO_CONFIG or ~/.config/pyro/config.json)") .option("--timeout ", "request/authentication timeout (PYRO_TIMEOUT_MS; default 130000)") .option("--json", "compact JSON output for scripts (streams always use NDJSON)") .showHelpAfterError() - .addHelpText("after", "\nGet started (a running server is required):\n pyro doctor Check server connectivity and setup\n pyro auth login Sign in with your dashboard password\n pyro profiles list Manage the same profiles as the dashboard\n pyro playground 'hello' --profile local-secrets After importing the local-only preset\n PYRO_API_KEY=pf_… pyro classify 'hello'\n\nObserve: overview, usage, activity, playground\nConfigure: profiles, apps, keys, webhooks, settings\nUse `pyro --help` for field flags and examples. No service is started automatically."); + .addHelpText("after", "\nGet started — no Docker, database or sign-in:\n pyro classify 'hello' Bundled local rules, no network calls\n pyro classify --file prompt.txt Check a text file\n pyro classify 'hello' --semantic TypeSafe directly; TYPESAFE_API_KEY required\n pyro doctor Verify the standalone installation\n\nOptional server: configure a gateway URL/key or use --remote.\nUse --local to override a saved server connection.\nServer commands: auth, profiles, apps, jobs, activity, team, reviews, evaluations.\nUse `pyro --help` for options."); const groups = new Map([["", program]]); - program.command("doctor").description("Check server connectivity and semantic configuration without sending prompts") + program.command("doctor").description("Check the standalone install or a configured server without sending prompts") + .option("--remote", "check server services") + .option("--local", "check standalone even with saved server settings") .option("--semantic", "also require semantic classifier configuration") - .action(async (options: { semantic?: boolean }, command: Command) => { + .action(async (options: { semantic?: boolean; remote?: boolean; local?: boolean }, command: Command) => { const global = command.optsWithGlobals(); - const report = await diagnose(settings(global, await readConfig(configPath(global)))); + const config = await readConfig(configPath(global)); + if (options.remote && options.local) throw new Error("Choose --local or --remote."); + const hasServer = options.remote || global.gatewayUrl || global.controlUrl || process.env.PYRO_GATEWAY_URL || process.env.PYRO_CONTROL_URL || config.gatewayUrl || config.controlUrl || process.env.PYRO_API_KEY; + if (options.local || !hasServer) { + const { standaloneProfiles } = await import("./standalone.js"); + await output({ mode: "standalone", localRulesReady: true, profiles: await standaloneProfiles(), semanticKeyConfigured: Boolean(process.env.TYPESAFE_API_KEY), serverRequired: false }, global); + if (options.semantic && !process.env.TYPESAFE_API_KEY) process.exitCode = 1; + return; + } + const report = await diagnose(settings(global, config)); await output(report, global); if (!report.localRulesReady) process.exitCode = 4; else if (options.semantic && !report.semantic.ok) process.exitCode = 1; @@ -188,6 +214,11 @@ export function createProgram(): Command { } } if (kind === "classification") command.argument("[text]", "text input; omit to read stdin").option("--file ", "read text input from a file or stdin; use --input for structured JSON"); + if (name === "classify") command.description("Classify standalone by default, or use a configured gateway") + .option("--local", "run in this process; override a saved gateway or PYRO_API_KEY") + .option("--remote", "use the configured Pyro gateway") + .option("--semantic", "standalone TypeSafe screening; sends input and can incur charges (TYPESAFE_API_KEY)") + .option("--profile-file ", "standalone policy YAML/JSON file; no import or server required"); if (kind === "profile-yaml") command.option("--file ", "read a portable profile YAML file or stdin"); if (endpoint["x-websocket"]) command.option("--count ", "stop after this many events; otherwise stream until Ctrl-C"); else command.option("-o, --output ", "write the response to a new file (never overwrite)"); diff --git a/packages/cli/src/doctor.ts b/packages/cli/src/doctor.ts index 140b3bf..ae43139 100644 --- a/packages/cli/src/doctor.ts +++ b/packages/cli/src/doctor.ts @@ -16,7 +16,7 @@ export async function diagnose(connection: { gateway: string; control: string; t gateway, control, semantic, localRulesReady: gateway.ok && control.ok, guidance: !gateway.ok || !control.ok - ? "Start the Pyro server with Docker, or set the URLs of an existing instance: https://delvisor.com/pyro/docs#setup. Installing the CLI does not start a server." + ? "These are server checks. For standalone checks use pyro doctor --local and pyro classify --local. For server features, start Docker or set an existing instance URL: https://delvisor.com/pyro/docs#setup. Installing the CLI does not start a server." : !semantic.ok ? "Local-only profiles work without a provider key. For semantic checks, add a TypeSafe key in Settings → Classifier provider, then run pyro doctor --semantic. Readiness checks configuration, not detector accuracy or provider availability." : "Server checks passed. Semantic configuration is present; test your policy against representative traffic before enforcement.", diff --git a/packages/cli/src/standalone.ts b/packages/cli/src/standalone.ts new file mode 100644 index 0000000..2b2c3d0 --- /dev/null +++ b/packages/cli/src/standalone.ts @@ -0,0 +1,48 @@ +import { randomUUID, createHash } from "node:crypto"; +import { readFile } from "node:fs/promises"; +import YAML from "yaml"; +import { evaluatePolicy } from "@pyro/classifiers"; +import { ClassificationEnvelopeSchema, ProfileSchema, createDefaultApp, createDefaultProviderSettings, type Profile } from "@pyro/contracts"; + +type LocalOptions = { profileFile?: string; semantic?: boolean; timeout: number; requestId?: string }; +export async function standaloneProfiles(): Promise { + return Object.keys(JSON.parse(await readFile(new URL("./profiles.json", import.meta.url), "utf8"))); +} +export async function classifyStandalone(body: string, contentType: string, options: LocalOptions) { + const raw = contentType === "application/json" ? JSON.parse(body) : body; + const envelope = ClassificationEnvelopeSchema.parse(raw && typeof raw === "object" && !Array.isArray(raw) && "input" in raw ? raw : { input: raw }); + if (options.profileFile && envelope.profile) throw new Error("Choose --profile or --profile-file, not both."); + const bundled = JSON.parse(await readFile(new URL("./profiles.json", import.meta.url), "utf8")) as Record; + const name = envelope.profile ?? (options.semantic ? "balanced-assistant" : "local-secrets"); + let input: unknown = bundled[name]; + if (options.profileFile) { + const file = YAML.parse(await readFile(options.profileFile, "utf8"), { maxAliasCount: 50 }); + if (file?.kind && (file.kind !== "Profile" || file.apiVersion !== "pyro/v1")) throw new Error("Expected a pyro/v1 Profile document."); + input = file?.profile ?? file; + } + if (!input || typeof input !== "object" || Array.isArray(input)) throw new Error(`Unknown bundled profile '${name}'. Use --profile-file for your own YAML/JSON policy, or --remote for server policies.`); + const now = new Date().toISOString(); + const profile = ProfileSchema.parse({ ...input, createdAt: now, updatedAt: now }); + if (profile.shadowProfileIds.length) throw new Error("Standalone classification does not run shadow policies. Remove shadowProfileIds or use a server."); + if ((typeof envelope.input === "string" ? envelope.input : JSON.stringify(envelope.input)).length > profile.maxInputChars) throw new Error(`Input exceeds this profile's ${profile.maxInputChars} character limit.`); + const semantic = profile.detectors.some((d) => d.enabled); + if (semantic && !options.semantic) throw new Error("This profile sends inputs to TypeSafe. Pass --semantic to authorize the provider call and its charges, or use the local-secrets profile."); + if (semantic && !process.env.TYPESAFE_API_KEY) throw new Error("Set TYPESAFE_API_KEY for standalone semantic checks. No Docker or Pyro server is needed."); + const provider = createDefaultProviderSettings(); + // Deliberately use the official endpoint. Server/provider URL settings cannot + // silently redirect a standalone prompt or provider credential. + provider.maxRetries = 0; + const policyHash = hashProfile(profile); + const result = await evaluatePolicy({ id: randomUUID(), traceId: randomUUID().replaceAll("-", ""), envelope, + profile: { ...profile, timeoutMs: Math.min(profile.timeoutMs, options.timeout) }, firewallApp: createDefaultApp(), provider, apiKey: async () => process.env.TYPESAFE_API_KEY }); + return { ...result.decision, labels: envelope.labels, requestId: options.requestId, policyHash, execution: "standalone" }; +} +function canonical(value: unknown): string { + if (Array.isArray(value)) return `[${value.map(canonical).join(",")}]`; + if (value && typeof value === "object") return `{${Object.entries(value).filter(([, v]) => v !== undefined).sort(([a], [b]) => a.localeCompare(b)).map(([k, v]) => `${JSON.stringify(k)}:${canonical(v)}`).join(",")}}`; + return JSON.stringify(value); +} +function hashProfile(profile: Profile): string { + const { createdAt, updatedAt, revision, contentHash, ...configuration } = profile; + return createHash("sha256").update(canonical(configuration)).digest("hex"); +} diff --git a/packages/cli/src/transport.ts b/packages/cli/src/transport.ts index c85fb76..b990575 100644 --- a/packages/cli/src/transport.ts +++ b/packages/cli/src/transport.ts @@ -26,7 +26,7 @@ export async function request(url: URL, method: string, headers: Record { + const directory = await temporary(t), config = join(directory, "fresh.json"); + const env = { TYPESAFE_API_KEY: "" }; + for (const [input, action] of [["hello", "allow"], ["-----BEGIN PRIVATE KEY-----", "block"], ["sk-" + "a".repeat(24), "review"]]) { + const result = await invoke(["classify", "--", input!], { config, env }); + assert.equal(result.code, 0, result.stderr); const decision = JSON.parse(result.stdout); + assert.equal(decision.action, action); assert.equal(decision.execution, "standalone"); assert.equal(decision.provider, "local-rules"); + } + const file = join(directory, "input.txt"); await writeFile(file, "-----BEGIN PRIVATE KEY-----"); + assert.equal(JSON.parse((await invoke(["classify", "--file", file], { config, env })).stdout).action, "block"); + assert.equal(JSON.parse((await invoke(["classify"], { config, env, input: "hello stdin" })).stdout).action, "allow"); + const custom = join(directory, "policy.yaml"); + const { readFile } = await import("node:fs/promises"); + await writeFile(custom, await readFile(new URL("../../../profiles/local-secrets.yaml", import.meta.url), "utf8")); + const customResult = await invoke(["classify", "Example: -----BEGIN PRIVATE KEY-----", "--profile-file", custom], { config, env }); + assert.equal(customResult.code, 0, customResult.stderr); assert.equal(JSON.parse(customResult.stdout).action, "block"); + const doctor = await invoke(["doctor"], { config, env }); assert.equal(doctor.code, 0); assert.equal(JSON.parse(doctor.stdout).serverRequired, false); + const missing = await invoke(["classify", "hello", "--semantic"], { config, env }); assert.equal(missing.code, 2); assert.match(missing.stderr, /TYPESAFE_API_KEY/); + const noConsent = await invoke(["classify", "hello", "--profile", "balanced-assistant"], { config, env }); assert.equal(noConsent.code, 2); assert.match(noConsent.stderr, /--semantic/); + const destination = join(directory, "decision.json"); + const missingFile = await invoke(["classify", "hello", "--profile-file", join(directory, "missing.yaml"), "--output", destination], { config, env }); assert.equal(missingFile.code, 2); + assert.equal((await invoke(["classify", "hello", "--output", destination], { config, env })).code, 0, "failed local output must not leave a reserved file"); +}); + +test("explicit local mode cannot leak to saved or environment gateway settings", async t => { + const config = join(await temporary(t), "config.json"); let calls = 0; + const { url } = await server(t, (_req, res) => { calls++; res.end('{}'); }); + await writeFile(config, JSON.stringify({ version: 1, gatewayUrl: url })); + const result = await invoke(["classify", "hello", "--local"], { config, env: { PYRO_API_KEY: "key", PYRO_GATEWAY_URL: url } }); + assert.equal(result.code, 0, result.stderr); assert.equal(JSON.parse(result.stdout).execution, "standalone"); assert.equal(calls, 0); + assert.equal((await invoke(["classify", "hello", "--remote", "--local"], { config })).code, 2); +}); + +test("standalone semantic calls require explicit consent and use the official provider directly", async t => { + const prior = process.env.TYPESAFE_API_KEY; process.env.TYPESAFE_API_KEY = "test-only-typesafe-key"; + t.after(() => { if (prior === undefined) delete process.env.TYPESAFE_API_KEY; else process.env.TYPESAFE_API_KEY = prior; }); + let calls = 0; + t.mock.method(globalThis, "fetch", async (url: string, options: RequestInit) => { + calls++; assert.equal(url, "https://api.typesafe.ai/v1/systemone"); + assert.equal((options.headers as Record).Authorization, "Bearer test-only-typesafe-key"); + const body = JSON.parse(options.body as string); assert.equal(body.state.payload, "synthetic semantic example"); + return Response.json({ model: "test-model", answers: Object.fromEntries(Object.keys(body.questions).map(id => [id, { probability: .05 }])) }); + }); + await classifyStandalone(JSON.stringify({ input: "local hello" }), "application/json", { timeout: 1000 }); assert.equal(calls, 0); + await assert.rejects(classifyStandalone(JSON.stringify({ input: "never sent", profile: "balanced-assistant" }), "application/json", { timeout: 1000 }), /--semantic/); assert.equal(calls, 0); + const decision = await classifyStandalone(JSON.stringify({ input: "synthetic semantic example" }), "application/json", { timeout: 1000, semantic: true }); + assert.equal(calls, 1); assert.equal(decision.action, "allow"); assert.equal(decision.execution, "standalone"); +}); diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index dec265d..0810d86 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -209,12 +209,18 @@ importers: specifier: ^8.18.3 version: 8.21.3 devDependencies: + '@pyro/classifiers': + specifier: workspace:* + version: link:../classifiers '@pyro/contracts': specifier: workspace:* version: link:../contracts '@types/ws': specifier: ^8.18.1 version: 8.18.1 + esbuild: + specifier: ^0.28.2 + version: 0.28.2 tsx: specifier: ^4.20.6 version: 4.23.15