Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -343,13 +343,18 @@ putting it in a curried worker. A minimal per-device setup is:
# .caos-secrets/anthropic-api-key
name=anthropic-api-key
value:@=/absolute/path/to/anthropic-api-key
reader=DEEP-DEPS/llm-step
reader=DEEP-DEPS/llm-call
reader=--base:@=DEEP-DEPS/llm-step
reader=--base:@=DEEP-DEPS/llm-call
```

Run `caos-cli secrets` once to add the random `entropy=` used for cache
isolation. The file and value path stay local; only the entropy-derived identity
enters an ArgTree, while the value is carried out of band for the run.
enters an ArgTree, while the value is carried out of band for the run. Each
`reader=` is one physical line using the typed arguments accepted after
`curry`, with an explicit `--base`; the field itself denotes assembly, so it
has no `curry` token. Additional args constrain the grant, and separate reader
lines are independent. `:@=` paths resolve against the store's pinned source
tree.

`caos-cli` must run inside a git working tree with the server as its `caos`
remote — the remote's URL is also where compute is triggered and results are
Expand Down
57 changes: 34 additions & 23 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,22 +99,30 @@ If these become slow:
# Secrets

**Status:** partly built. The store is carried as ephemeral run context and
resolved client-side; injection (gated by the double-check below), superset
matching over path-only readers, the entropy/`secret-hash` cache-isolation tag,
the output-scrub assertion, log masking, and the `caos secrets` entropy tooling
all exist. **Cache isolation is now complete for the eval path**: the running
resolved client-side; constrained partial-ArgTree readers, injection (gated by
the double-check below), the entropy/`secret-hash` cache-isolation tag, the
output-scrub assertion, log masking, and the `caos secrets` entropy tooling all
exist. **Cache isolation is now complete for the eval path**: the running
worker, eval-path's `curry` returns, and — via the eval-path stripping rule —
a worker embedded through a `:@=` arg, which makes its embedder per-user too.
Builds on `.caos-expr` (eval-path, deep-deps) and map-then (server-mediated
worker starts).

**Since the ambient-`std` removal landed** (design/caos-expr.md, "Landed:
ambient `/std` is gone"), a reader is a **tree path and nothing else** — there
is no `/std/<name>` to name, so the two reader forms collapsed into one, which
is what this note always wanted. It also briefly *widened* the
caller-propagation gap: eval-path used to mark a `/std/<name>` `:@=` target, and
that was the only `:@=` marking there was. Closing it properly covers all of
`:@=` and needs no `/std` special case at all.
Every `reader=` is one physical line containing the arguments that assemble a
partial/curry ArgTree. It has no `curry` verb because `reader=` supplies that
operation, and it must contain an explicit typed `--base`. The rest uses the
same argument parser and resolution rules as `.caos-expr` curry commands;
notably, `:@=` paths resolve against the secret store's pinned source tree.
The assembled reader is never run. It is unwrapped to the existing name → oid
map and sent to the server, whose authorization remains a pure subset match.

This restores the inline pins removed in commit `91866bd94`. Moving constraints
into narrower expression wrappers conflated two policies: content-addressed
worker identity belongs in the source tree, while the secret owner's local
grant constraints belong on that device. Security-sensitive values such as a
credential destination belong beside the credential; requiring a wrapper would
make that policy repository-owned and needlessly proliferate expression
directories.

The agent harness carries the same store: conversation preparation resolves
`llm-step` with it, the admitted request includes the resulting isolation
Expand All @@ -133,26 +141,32 @@ Some tools need secrets: the github-push tool needs an auth token, and there wil

`.caos-secrets`:
- Secrets live in a git-ignored .caos-secrets directory
- Each secret file contains the secret's value and a list of workers that can read the secret. This is formatted as a repeated-key file. For example:
- Each secret file contains the secret's value and one or more independent
partial ArgTrees that may read it. This is formatted as a repeated-key file.
For example:
```
# Optional name. Defalts to the name of the file. This is the name that is used in the worker for /secret/<name>
# Optional name. Defaults to the filename. This is used at /secret/<name>.
name=<name>
entropy=...
# Inline secret
value=<secret key>
# External key
value:@=<file containing key>
# A reader is a PATH to an expression, without arguments. It is eval-path'd to
# an arg tree
reader=std/github-push
reader=tools/deploy
# Each reader is a curry argument list with its own explicit typed base.
reader=--base:@=DEEP-DEPS/github-push --repo=github.com/me/proj
reader=--base:@=tools/deploy --environment=production
```
- When a call stack is started, such as `caos-cli run`, we read the current source tree and the list of secrets. Readers in secrets are matched against the tree. Any worker named as a reader is granted access to the secret. These workers have a hash of the names and entropy of all exposed secrets injected into them as /cas/args/secret-hash
- Something is considered to be the same worker (ie, to have access to the secret) if it its arg tree is a superset of the reader's arg tree and secret-hash matches the set of secrets that the server computes for it
- When a call stack is started, such as `caos-cli run`, the client reads the
pinned source tree and the secret files. Each reader is assembled through the
normal argument/expression code, without executing the assembled request.
- A worker has access when its ArgTree is a superset of any one independently
assembled reader and its `secret-hash` matches the exact set of secrets the
server computes for that ArgTree. Arguments omitted from a reader remain
unconstrained.
- Each granted secret contributes its (worker-visible name, entropy) to a
`secret-hash` entry folded into the worker's arg tree (visible at
`/cas/args/secret-hash`). This makes two users with different secrets see
different cache keys — but keps the secret's *value* out (so rotating a value
different cache keys — but keeps the secret's *value* out (so rotating a value
doesn't bust the cache), and stores the *digest* of the entropy, never the
entropy itself (the entropy is a bearer capability for the cache: knowing it
reconstructs the key of any run that used it). The name is included because a
Expand Down Expand Up @@ -188,9 +202,6 @@ Note that this means that the server sees all secrets. We can revisit if this be

- **Binary `value:@=`.** Read but kept UTF-8 (binary/multiline later).

- **`run`-form `.caos-expr` grants** are deliberately unresolved (a grant must
never trigger compute); likely permanent.

- **Shared-server exposure.** Carrying the whole store means a shared server
sees values it never injects (sub-runs aren't known ahead of time, so the
client can't pre-filter to the granted subset). Moot for a per-user/local
Expand Down
4 changes: 2 additions & 2 deletions crates/caos-cli/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1117,8 +1117,8 @@ fn require_model_secret(store: &[ClientSecret]) -> Result<(), String> {
`.caos-secrets/{MODEL_API_SECRET}` with:\n\n\
name={MODEL_API_SECRET}\n\
value:@=/absolute/path/to/your/anthropic-api-key\n\
reader=DEEP-DEPS/llm-step\n\
reader=DEEP-DEPS/llm-call\n\n\
reader=--base:@=DEEP-DEPS/llm-step\n\
reader=--base:@=DEEP-DEPS/llm-call\n\n\
Then run `{invoked_as} secrets` to add cache-isolation entropy. \
See the README's Secrets section for details."
))
Expand Down
29 changes: 29 additions & 0 deletions crates/caos-eval/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -514,6 +514,21 @@ fn eval_value(
eval_command(host, input_tree, line, env)
}

/// Assemble `args` as one `curry` command against `input_tree`, without
/// dispatching the resulting request. Callers that already supply the
/// operation (such as a `.caos-secrets` `reader=` field) use this entry point
/// so they share the expression grammar without accepting a verb of their own.
pub fn assemble_curry(host: &dyn EvalHost, input_tree: &str, args: &str) -> Result<String, String> {
let command = format!("curry {args}");
let (kind, oid) = eval_command(host, input_tree, &command, &HashMap::new())?;
if kind != "tree" {
return Err(format!(
"curry assembly returned a {kind}, expected a partial ArgTree"
));
}
Ok(oid)
}

/// Evaluate a single `run --base:<t>=<image> …` or `curry --base:<t>=<image> …`
/// command against `input_tree`, returning the result's `(kind, oid)`. A `curry`
/// yields the curried ArgTree (a tree); a `run` triggers compute and yields its
Expand Down Expand Up @@ -809,6 +824,20 @@ fn build_curry(
new: Vec<Entry>,
) -> Result<gix::ObjectId, String> {
let (base, bound) = unwrap_curry(host, image_ref)?;

// Keep expression curries as strict as ordinary client curries. The merge
// helper is deliberately last-wins for overlays, but repeating one name on
// a curry command line is a malformed binding, not an override.
let mut new_names = std::collections::BTreeSet::new();
for entry in &new {
if !new_names.insert(entry.filename.to_vec()) {
return Err(format!(
"curry: arg {:?} was provided more than once",
String::from_utf8_lossy(&entry.filename)
));
}
}

for e in &new {
if bound.iter().any(|b| b.filename == e.filename) {
return Err(format!(
Expand Down
25 changes: 25 additions & 0 deletions crates/caos/src/eval.rs
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,31 @@ pub(crate) fn eval_path(
caos_eval::eval_path(&host, start_tree, path)
}

/// Assemble a `.caos-secrets` `reader=` value as a partial curry ArgTree.
///
/// A reader omits the `curry` verb because the key supplies it; everything on
/// the physical line is otherwise parsed and resolved by the ordinary
/// `.caos-expr` curry grammar. In particular, `:@=` paths are looked up in the
/// pinned source tree, never in the host filesystem. The empty secret store is
/// intentional: resolving a grant must not recursively mark that grant.
pub(crate) fn assemble_reader(
t: &dyn Transport,
input_tree: &str,
reader: &str,
) -> Result<String, String> {
// Apply the pinned tree's root expression first, exactly as the old
// path-only reader walk did. This exposes generated mounts such as
// `DEEP-DEPS/` while keeping every subsequent `:@=` inside that snapshot.
let (input_kind, input_tree) = eval_path(t, input_tree, "", &[])?;
if input_kind != "tree" {
return Err(format!(
"reader source evaluated to a {input_kind}, expected a tree"
));
}
let host = ClientEvalHost { t, store: &[] };
caos_eval::assemble_curry(&host, &input_tree, reader)
}

/// Resolve one of the WORKSPACE's declared entry points: evaluate the tracked
/// tree and descend to `DEEP-DEPS/<name>`.
///
Expand Down
87 changes: 47 additions & 40 deletions crates/caos/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3557,8 +3557,9 @@ pub fn cli_run(
let (bty, bval, kvs) = split_base_arg("run", kvs)?;
let image = resolve_base(t, None, bty, bval)?;
// Build the ephemeral secrets store from the caller's `.caos-secrets`
// (design/secrets.md), resolving each reader here — where eval-path is
// available — so the server never evals. Empty when there's no store.
// (SPEC.md, Secrets), assembling each constrained reader here — where the
// pinned-tree expression grammar is available — so the server never evals.
// Empty when there's no store.
let store = build_secret_store(t)?;
let (kind, result) = run_request(t, &image, None, trace, &kvs, &store)?;

Expand Down Expand Up @@ -3863,6 +3864,20 @@ fn curry_from_entries(

let (base, mut bound) = unwrap_curry(t, arg_tree)?;

// Reject duplicate new bindings before merging them. `merge_entries` is
// deliberately last-wins for run-time overlays, but curry is strict: two
// occurrences on one command line are as much a typo as rebinding an arg
// already present in the base ArgTree.
let mut new_names = std::collections::BTreeSet::new();
for entry in &new {
if !new_names.insert(entry.filename.to_vec()) {
return Err(format!(
"curry: arg {:?} was provided more than once",
String::from_utf8_lossy(&entry.filename)
));
}
}

// UNBIND first: drop the named args so they can be rebound. Currying is
// otherwise strict (below), so carrying a whole ArgTree forward and changing
// a few of its args — the self-recurry case — needs an explicit release. An
Expand Down Expand Up @@ -4145,10 +4160,10 @@ impl ClientSecret {
}
}

/// Read and resolve the caller's `.caos-secrets` store (design/secrets.md):
/// each reader resolved HERE (via eval-path, against the store's pinned tree)
/// to a partial arg tree of name → oid — so the server only subset-matches,
/// never evals. Empty when there is no store.
/// Read and resolve the caller's `.caos-secrets` store (SPEC.md, Secrets): each
/// constrained reader is assembled HERE through the curry/expression grammar,
/// against the store's pinned tree, into a partial name → oid ArgTree. The
/// server only subset-matches and never evals. Empty when there is no store.
pub fn build_secret_store(t: &dyn Transport) -> Result<Vec<ClientSecret>, String> {
let dir = Path::new(SECRETS_DIR);
if !dir.is_dir() {
Expand All @@ -4164,7 +4179,10 @@ pub fn build_secret_store(t: &dyn Transport) -> Result<Vec<ClientSecret>, String
let readers = spec
.readers
.iter()
.map(|r| resolve_reader_client(t, &pinned, r))
.map(|reader| {
resolve_reader_client(t, &pinned, reader)
.map_err(|error| format!("secret {file_name}: reader={reader}: {error}"))
})
.collect::<Result<_, _>>()?;
store.push(ClientSecret {
name: spec.name,
Expand Down Expand Up @@ -4404,57 +4422,46 @@ fn resolve_local_secret_value(
}
}

/// Resolve a reader — a single path/expression, no argument pins
/// (design/secrets.md: a reader names an *expression*; narrow by pointing at a
/// narrower one, not by pinning args here) — to the partial arg tree it stands
/// for: eval-path the path (so a flake/`.caos-expr` tool resolves to the same
/// arg tree the run uses), unwrap any curry layers, and take its entries. That
/// tree already carries whatever the expression bakes in (e.g. a curried
/// `worker1` script), so it is as specific as the expression is.
/// Assemble one `reader=` line into the partial name -> oid ArgTree used by the
/// existing subset matcher. The line is the ordinary curry argument list with
/// an explicit typed `--base`; `reader=` itself supplies the curry operation.
/// Resolution happens against the store's pinned source tree and never runs the
/// assembled request. Curry layers are unwrapped only after assembly, so an arg
/// already bound by the base is rejected by the shared strict-curry checks.
fn resolve_reader_client(
t: &dyn Transport,
pinned: &str,
reader: &str,
) -> Result<std::collections::BTreeMap<String, String>, String> {
if reader.split_whitespace().count() != 1 {
let partial = eval::assemble_reader(t, pinned, reader)?;
let (base, bound) = unwrap_curry(t, &partial)?;
if is_hex_hash(&base) {
let (kind, _) = t.get_object(&base)?;
if kind != "tree" {
return Err(format!("reader base {base} is a {kind}, not an image tree"));
}
} else if !base.starts_with(DOCKER_SCHEME) {
return Err(format!(
"reader {reader:?} must be a single path (argument pins are not supported — \
point at a narrower expression instead)"
"reader base {base:?} is not a git image or docker reference"
));
}
let image = resolve_reader_image(t, pinned, reader.trim())?;
let (base, bound) = unwrap_curry(t, &image)?;
let mut entries = std::collections::BTreeMap::new();
for entry in bound {
entries.insert(
String::from_utf8_lossy(entry_name(&entry)).into_owned(),
entry.oid.to_string(),
);
}
// The image entry wins over any like-named bound arg, mirroring assembly.
entries.insert("base".to_string(), base);
// Store the base entry's object id, like every other partial entry. Docker
// refs therefore match their blob oid rather than leaking a representation
// exception into the server's pure oid-equality matcher.
entries.insert(
"base".to_string(),
base_arg_entry(t, &base)?.oid.to_string(),
);
Ok(entries)
}

/// Resolve a reader's image token: a bare hash, or a path in the pinned tree
/// (via eval-path — so a flake/`.caos-expr` tool resolves to the same oid the
/// run uses).
///
/// A path only: there is no ambient library to name, so a reader says
/// `std/github-push` and it is read out of the tree, exactly as an expression
/// reaches a dependency. That is also why it converges with the run's own
/// resolution — the root `.caos-expr` deepens the tree, and the entry a reader
/// descends to is the same node a `DEEP-DEPS/<name>` mount points at.
fn resolve_reader_image(t: &dyn Transport, pinned: &str, expr: &str) -> Result<String, String> {
if is_hex_hash(expr) {
return Ok(expr.to_string());
}
// Empty store: a reader's own resolution must not be marked (its arg tree is
// what the match compares against; marking it would be circular).
let (_, oid) = eval::eval_path(t, pinned, expr, &[])?;
Ok(oid)
}

fn request_compute(base: &str, arg_tree: &str, secrets: &str) -> Result<(String, String), String> {
let url = run_url(base, arg_tree, None);
request_compute_url(&url, secrets)
Expand Down
Loading