Add a dry-run pool placement planner (lloom pool plan)

[data-angel]

Summary

Phase 1 of making LLooM a pool head. Every cluster node is treated as one pool, and lloom pool plan proposes:

• which model lanes run where;
• which residency each lane gets;
• how each alias routes;
• the diff from what runs now.

It is dry-run only: nothing is started, stopped or rerouted. Per-node admission (runtime-policy.mjs) stays the safety layer underneath. Full description in docs/pool-planning.md.

• Node inventory. GET /gateway/node?inventory=1 and GET /gateway/cluster?inventory=1 report:

  • structured compute archs (sm_121, apple-metal);
  • memory domains;
  • free disk per model root;
  • cached recipe weights;
  • installed commands and Docker images;
  • NVIDIA Sync fabric links.

  The inventory is collected on request and cached for 5 minutes, with in-flight requests sharing one collection. Older gateways fall back to their machine profile.

• Model-centric catalog (src/pool-catalog.mjs). Variants come from:

  • managed runtimes, fixed to their node/placement;
  • federated targets;
  • unmanaged lanes declared with a model pool block (e.g. a hand-run TP2 server);
  • external models;
  • with --consider-installs, bundled recipes.

  Each variant has structured requirements: node count, TP, archs, backend, context and sequences. Memory is the configured estimate raised to the measured RSS when that is larger. Performance comes from live gateway metrics, falling back to the best matching benchmark.

• Pure planner (src/pool-planner.mjs).

  • Hard constraints:
    • memory per domain, using the admission budget minus headroomGb;
    • TP only across directly linked nodes;
    • arch, platform and accelerator matching;
    • pins: node/placement, keepWarm, pool.pins, --pin, --profile;
    • cordons.
  • Plans are compared in order: coverage, then external models as a fallback only, then latency SLOs, then demand-weighted latency plus load-time churn.
  • Search is greedy placement plus single-workload local search. Models that share a runtime share one placement.

• Per-alias demand. /gateway/metrics (live snapshots and the persisted history file) adds requested buckets keyed by the requested id. Older gateways fall back to splitting each model's demand across the aliases routed to it.

• Recipe schema (additive, backward compatible): requirements.computeArchs, requirements.tensorParallel, models[].settings.maxSequences. Validator and interchange example updated. When the fields are absent, they are inferred from launch settings (CUTE_DSL_ARCH, TORCH_CUDA_ARCH_LIST, --tensor-parallel-size, MAX_NUM_SEQS).

• Shared helper. Admission and pool planning now use one memory-budget helper (resource-fit.mjs).

Evidence

test/fixtures/pool/spark-fleet.json (anonymized node names) reproduces the three-Spark fleet:

• With all nodes present, the plan matches the sparkqwen-pair-rc16 profile's routes and residency, with no diff.
• With the 01/02 pair cordoned or unreachable, it matches standalone. Cognition aliases move to the 27B on spark-node-03 with residency always, and the unmanaged pair lane gets a manual stop.

Against this Mac's installed config and live gateway, lloom pool plan reports no changes, which matches what is running. Offline mode (config plus local inventory) also works.

Tests

• node --test test/pool-planner.test.mjs test/pool-catalog.test.mjs test/pool-inventory.test.mjs test/pool-cli.test.mjs: 35 pass. These cover linked-TP sets, arch mismatch, headroom, per-domain memory, local search, external fallback, SLOs, churn hysteresis, shared runtimes, inventory probes and the cache, the gateway inventory flag, CLI fallback and offline JSON.
• Also passing:
  • test:cluster;
  • runtime-policy, runtime-policy-config and runtime-memory-safety;
  • metrics-persistence, stream-metrics, server-resilience, dashboard-status and security;
  • cli-diagnostics and config-profiles;
  • interchange and interchange:check;
  • resource-fit;
  • smoke;
  • check:js, lint (only pre-existing warnings), format:check and git diff --check.
• Not run: the full npm test.

Security notes

• The inventory is behind existing admin-read (/gateway/* GET). It adds model-root paths, image names and fabric peer addresses to what admins can already read: the profile already exposes x-local-modelRoot.
• Probes are argv-only runCommand calls (which, docker image ls) plus statfs. There is no shell interpolation.
• Plan reports carry no backend URLs or keys.

Not in this PR

There is no apply or reconcile step, no node self-enrollment or heartbeat, and nothing has been deployed or touched on the live fleet.

🤖 Generated with Claude Code

[data-angel]

Superseded by #74, which integrates the planner together with the rest of the pool feature set against current main.