From 9ac14def7b77366bbd1fb8f9cf8e7d795d52c166 Mon Sep 17 00:00:00 2001 From: Chaitanya Date: Tue, 22 Sep 2026 18:07:23 +0100 Subject: [PATCH] docs: consolidate environment variables reference in docs/configuration.md - Create docs/configuration.md with reference table for all environment variables. - Document variable name, reader model or subsystem, default value, and description. - Set accurate code defaults for TYPESAFE_API_URL and library defaults for LAYA_*. - Link configuration.md in Docs list of README.md. - Prune redundant per-page variable documentation in docs and evals down to pointers. - Update documentation to reference models rather than slots. - Add entry to CHANGELOG.md. --- CHANGELOG.md | 1 + README.md | 1 + docs/configuration.md | 41 +++++++++++++++++++++++++++++++++++++++++ docs/decision-models.md | 10 +++++----- docs/skills.md | 2 +- evals/README.md | 15 +++++---------- 6 files changed, 54 insertions(+), 16 deletions(-) create mode 100644 docs/configuration.md diff --git a/CHANGELOG.md b/CHANGELOG.md index bb80f85..2fde707 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); ver ### Added +- Consolidated configuration reference table in docs/configuration.md detailing every environment variable, default value, and reader subsystem. - Nine agents through the CLI and the MCP server: `allrecipes` and `flights` (browser use), `desktop` (computer use on Windows or macOS), `ticket_router` (30 labelled tickets to five queues), `blackjack`, `game2048`, `millionaire`, `alfworld` (games and embodied text), `injection_guard` (rail). diff --git a/README.md b/README.md index 5320a07..b35a240 100644 --- a/README.md +++ b/README.md @@ -140,6 +140,7 @@ interface fits: [docs/architecture.md](docs/architecture.md), [docs/decision-mod - [docs/agents.md](docs/agents.md): every agent with its flags, run command and extra. - [docs/architecture.md](docs/architecture.md) and [docs/decision-models.md](docs/decision-models.md): the fronts, the model slot, the model interface, adding a backend. - [docs/browser-front.md](docs/browser-front.md): the browser policy, decision by decision. +- [docs/configuration.md](docs/configuration.md): environment variables, defaults and reader subsystems in one table. - [docs/why.md](docs/why.md): the problem, the philosophy, the precedents. - [docs/roadmap.md](docs/roadmap.md) and [CHANGELOG.md](CHANGELOG.md). diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..7c71622 --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,41 @@ +# Configuration reference + +Environment variables configure decision models, the comparison chat model, logging paths, driver binaries, and evaluation datasets. + +Variables can be exported in your shell or placed in a `.env` file at the root of the repository. Exported environment variables take precedence over values in `.env`. + +## Environment variables + +| Variable | Who reads it | Default | What it does | +|:---|:---|:---|:---| +| `TYPESAFE_API_KEY` | `jev` model | *(unset)* | API key for direct TypeSafe decisions endpoint (`https://api.typesafe.ai/v1/systemone`). | +| `TYPESAFE_API_URL` | `jev` model | `https://openrouter.ai/api/alpha/decisions` | Endpoint URL for decisions; defaults to OpenRouter proxy, or can be overridden to a custom proxy URL. | +| `TYPESAFE_MODEL` | `jev` model | `typesafe/jev-1.13` | Model identifier when proxying Jev decisions through OpenRouter. | +| `OPENROUTER_API_KEY` | `jev` model (proxy), chat model fallback | *(unset)* | OpenRouter API key, used for proxying Jev decisions or as a fallback for `LLM_API_KEY`. | +| `OPENROUTER_BASE_URL` | chat model fallback | `https://openrouter.ai/api/v1` | Fallback base URL for the chat model when `LLM_BASE_URL` or `OPENAI_BASE_URL` is unset. | +| `MODEL_NAME` | chat model (`llm` model, rethink planner, browser agent) | *(unset)* | Model identifier for the chat model (e.g. `google/gemini-2.5-flash` or `claude-fable-5-1`). | +| `MODEL_PROVIDER` | chat model | `openai` | Provider protocol for the chat model (`openai` or `anthropic`). | +| `OPENAI_API_KEY` | chat model | *(unset)* | API key for OpenAI-compatible endpoints; checked alongside alias `LLM_API_KEY`. | +| `OPENAI_BASE_URL` | chat model | `https://api.openai.com/v1` | Base endpoint URL for OpenAI-compatible chat models; checked alongside alias `LLM_BASE_URL`. | +| `LLM_API_KEY` | chat model | *(unset)* | Alias for `OPENAI_API_KEY`; falls back to `OPENROUTER_API_KEY` when unset. | +| `LLM_BASE_URL` | chat model | *(unset)* | Alias for `OPENAI_BASE_URL`; falls back to `OPENROUTER_BASE_URL` when unset. | +| `ANTHROPIC_WORKSPACE_ID` | chat model | *(unset)* | Organization workspace identifier passed in the `anthropic-workspace-id` header when `MODEL_PROVIDER=anthropic`. | +| `S1A_HOME` | runtime, console, logging | Repository root | Root directory where agent runs, logs, and artifacts are written (`runs/logs/`). | +| `PLAYWRIGHT_MCP_ARGS` | browser agents (`allrecipes`, `flights`), `scripts/browser_showcase.sh` | `npx -y @playwright/mcp@0.0.78` | Command arguments passed when launching Playwright MCP, or custom flags like `--cdp-endpoint`. | +| `PLAYWRIGHT_MCP_COMMAND` | `scripts/browser_showcase.sh` | `node` | Executable used to launch the Playwright MCP server process in browser showcase scripts. | +| `ALFWORLD_DATA` | `alfworld` agent, replay | *(unset)* | Directory containing the downloaded ALFWorld benchmark dataset and game files. | +| `CHAT_USD_PER_M_INPUT` | pricing / accounting | *(catalogue)* | Price in USD per million prompt tokens when `MODEL_NAME` is not in OpenRouter's public catalogue. | +| `CHAT_USD_PER_M_OUTPUT` | pricing / accounting | *(catalogue)* | Price in USD per million completion tokens when `MODEL_NAME` is not in OpenRouter's public catalogue. | +| `CHAT_USD_PER_M_CACHED_INPUT` | pricing / accounting | `CHAT_USD_PER_M_INPUT` | Price in USD per million cached input tokens for non-catalogue models. | +| `LAYA_MODEL` | `laya` model | `convaiinnovations/laya` | Hugging Face repository ID or local path for the resident Laya decision model checkpoint. | +| `LAYA_SUBFOLDER` | `laya` model | *(unset)* | Optional subfolder in the checkpoint repo (e.g. `multilingual` or `typed-decisions`). | +| `LAYA_DEVICE` | `laya` model | `(library default)` | PyTorch device for Laya model evaluation; passes None so the library selects CUDA, MPS, or CPU. | +| `LAYA_MAX_LEN` | `laya` model | `(checkpoint default)` | Maximum token sequence length for Laya state representation; overrides checkpoint window only when set. | +| `LAYA_HEAD_MAX_LEN` | `laya` model | `(checkpoint default)` | Maximum token sequence length for Laya decision head options; overrides checkpoint window only when set. | +| `CUA_S1_CHECKPOINT` | `cua` model | `cua-ai/cua-s1-nano-0.1` | Hugging Face checkpoint ID or local directory for Cua-S1 Nano option scorer. | +| `CUA_S1_SUBFOLDER` | `cua` model | `text` | Subfolder within checkpoint directory containing text option scoring weights. | +| `CUA_S1_DEVICE` | `cua` model | `auto` | PyTorch device used for Cua-S1 Nano evaluation (`auto`, `cpu`, `cuda`, or `mps`). | +| `CUA_DRIVER_BIN` | `desktop` agent | `cua-driver` | Path to the `cua-driver` executable on Windows or macOS when not located on `PATH`. | +| `CUA_DRIVER_PERMISSION_MODE` | `desktop` agent | `standard` | Permission mode passed to `cua-driver mcp` (`standard`, or `bounded` for restricted capability manifests). | +| `HF_HOME` | Hugging Face runtime | `~/.cache/huggingface` | Cache directory where Laya and Cua-S1 checkpoints are downloaded on first run. | +| `HF_HUB_OFFLINE` | Hugging Face runtime | `0` | When set to `1`, prevents network requests and forces models to load exclusively from local cache. | diff --git a/docs/decision-models.md b/docs/decision-models.md index 915cd38..89b2e0d 100644 --- a/docs/decision-models.md +++ b/docs/decision-models.md @@ -42,9 +42,9 @@ shorthands; `warm()` and `close()` open and release the backend. | `--model` | class | `name` | notes | |---|---|---|---| -| `jev` | `JevModel(transport)` | `jev` | the request body every front sent before the layer existed, byte for byte; `from_env` picks TypeSafe or the OpenRouter proxy | -| `laya` | `LayaModel(agent, model=)` | `laya` | one forward pass per call on a thread; `MODEL_SERVICE_CONFIG_ERROR` when `input_tokens` fills the window (Laya cuts the state silently; `LAYA_MAX_LEN`, `LAYA_HEAD_MAX_LEN` widen it); `ValueError` and `RuntimeError` from the library become `MODEL_CALL_FAILED` | -| `cua` | `CuaS1Model(scorer, collator, model=, context_bytes=, option_bytes=)` | `cua` | Cua-S1 Nano, one `score_elements` pass per request on a thread; choice questions only, text only, deterministic; the context is header, state and rules; the checkpoint reads its first 256 bytes, and the first overflowing request logs one warning; `from_env` reads `CUA_S1_CHECKPOINT`, `CUA_S1_SUBFOLDER`, `CUA_S1_DEVICE` | +| `jev` | `JevModel(transport)` | `jev` | the request body every front sent before the layer existed, byte for byte; `from_env` picks TypeSafe or the OpenRouter proxy (see [configuration.md](configuration.md)) | +| `laya` | `LayaModel(agent, model=)` | `laya` | one forward pass per call on a thread; `MODEL_SERVICE_CONFIG_ERROR` when `input_tokens` fills the window (Laya cuts the state silently; see `LAYA_MAX_LEN` and `LAYA_HEAD_MAX_LEN` in [configuration.md](configuration.md)); `ValueError` and `RuntimeError` from the library become `MODEL_CALL_FAILED` | +| `cua` | `CuaS1Model(scorer, collator, model=, context_bytes=, option_bytes=)` | `cua` | Cua-S1 Nano, one `score_elements` pass per request on a thread; choice questions only, text only, deterministic; the context is header, state and rules; the checkpoint reads its first 256 bytes, and the first overflowing request logs one warning; `from_env` reads `CUA_S1_*` (see [configuration.md](configuration.md)) | | `random` | `RandomModel(seed)` | `random` | uniform over the offered keys, confidence 0, one seeded stream per episode; choice questions only | | `rule` | `RuleModel(name, rule)` | the rule's name | one-hot, confidence 1; a key outside the menu raises `RuntimeError`, a bug in the rule | @@ -53,8 +53,8 @@ shorthands; `warm()` and `close()` open and release the backend. TypeSafe Jev answers `--model jev`. One request holds a `state` and one or more questions over options the caller enumerates; the answer holds one option per question, a probability per option and a confidence, from one forward pass, with no free text. Three heads: `choice` picks one key among the options, `noul` gives the probability that a -statement holds, `score` places the state on an ordered rubric. Input is capped at 32K tokens; the endpoint is -`api.typesafe.ai` with `TYPESAFE_API_KEY`, or OpenRouter's `typesafe/jev-1.13` on `/api/alpha/decisions`. Latency and +statement holds, `score` places the state on an ordered rubric. Input is capped at 32K tokens; endpoint configuration +and API keys are referenced in [configuration.md](configuration.md). Latency and price: `benchmarks.md`. Laya (Convai Innovations, open weights, 0.4B parameters) and Cua-S1 Nano (Cua, 855K parameters) answer the same `choice` question in process. diff --git a/docs/skills.md b/docs/skills.md index 3056b6c..cbd7f1b 100644 --- a/docs/skills.md +++ b/docs/skills.md @@ -14,7 +14,7 @@ claude plugin marketplace add ThinkFlowLab/system1-agents && claude plugin insta The plugin's MCP server starts as `uv run --project s1a-mcp`, with `uv` on `PATH`; the first start runs `uv sync` in the plugin folder and takes minutes to resolve the `openjiuwen` git pin. Export `TYPESAFE_API_KEY` (or `OPENROUTER_API_KEY`) in the shell that launches Claude Code, since the plugin folder has no `.env`. Runs and logs -land under the plugin folder unless `S1A_HOME` names another root. Every host runs the command from a checkout of +land under the plugin folder unless `S1A_HOME` names another root ([configuration.md](configuration.md)). Every host runs the command from a checkout of this repository. ### A ticket through the skill diff --git a/evals/README.md b/evals/README.md index 256b11d..2983f5a 100644 --- a/evals/README.md +++ b/evals/README.md @@ -52,14 +52,10 @@ the score statistics over the `scored` episodes only, and `evals.table` skips tr ## Setup `uv sync`, plus `--extra blackjack` and `--extra alfworld` for those games (the README lists every extra), then -a `.env` with `TYPESAFE_API_KEY` or `OPENROUTER_API_KEY` for Jev. -`llm`, the rethink planner and the browser agent need the chat model: `OPENAI_API_KEY` (or `LLM_API_KEY`), -`OPENAI_BASE_URL` (or `LLM_BASE_URL`) and `MODEL_NAME`. `MODEL_PROVIDER=anthropic` talks Anthropic's own protocol, direct -(`OPENAI_BASE_URL=https://api.anthropic.com`, `MODEL_NAME=claude-fable-5-1`; an org-level key also needs -`ANTHROPIC_WORKSPACE_ID`) or through OpenRouter's `/v1/messages` (`MODEL_NAME=anthropic/claude-fable-5.1`). The browser agents launch a headless -Chromium through `@playwright/mcp` (Node); `--headed` shows it. ALFWorld needs `uv sync --extra alfworld` and -`ALFWORLD_DATA` in a Python 3.11 environment; the data comes from `python scripts/alfworld-download` in a clone -of alfworld/alfworld. The DeepAgent's workspace files land under `runs/evals/`. +a `.env` with `TYPESAFE_API_KEY` (or `OPENROUTER_API_KEY`) for Jev, and `MODEL_NAME` with provider credentials for the chat model. +ALFWorld needs `uv sync --extra alfworld` and `ALFWORLD_DATA` in a Python 3.11 environment; the data comes from +`python scripts/alfworld-download` in a clone of alfworld/alfworld. The DeepAgent's workspace files land under `runs/evals/`. +For the complete reference table of all environment variables, provider endpoints, and defaults, see [docs/configuration.md](../docs/configuration.md). ## Protocol @@ -71,8 +67,7 @@ seeds (same loop, swap the brain); `jev` with `--rethink on` against `off`; the act against a bare loop (loop overhead); and, later, Jev's top probability against the ALFWorld expert plan. `summary.json` also holds decisions, chat calls, tokens (`chat_input_tokens`, `chat_output_tokens`, `chat_cache_tokens`) and `cost_usd` (Jev at $0.042 per M input tokens; the chat model at OpenRouter's catalogue -price for `MODEL_NAME`, with cached input at the catalogue's cache-read rate; or `CHAT_USD_PER_M_INPUT`, -`CHAT_USD_PER_M_OUTPUT` and, optionally, `CHAT_USD_PER_M_CACHED_INPUT`). `python -m evals.table evals/results` prints one row per eval and model over every +price for `MODEL_NAME`, or custom rates from `CHAT_USD_PER_M_*`, see [docs/configuration.md](../docs/configuration.md)). `python -m evals.table evals/results` prints one row per eval and model over every job folder. ALFWorld's game files sort by task type; `--stride 11` from offset 0 takes twelve games across the six types. Every model plays the same tile draws because 2048 seeds the page's `Math.random`.