feat: auto-switch mode — runtime manager, VRAM estimator, planner and scheduler - #144
Merged
Merged
Conversation
added 3 commits
September 13, 2026 23:09
Gap 1 of #143. Nothing in computing-provider could bring a model up or take one down: `models download` fetched weights, models.json was read once somebody had filled it in, and `setup` printed an sglang command for the operator to run by hand. A mode that re-plans which models to serve has to execute the plan, and this is the piece that executes it. Servers run as Docker containers because that is already how they run on provider hardware — both boxes this was written against serve llama.cpp and vLLM from ghcr.io and Docker Hub server images. The container is also what gives an instance an identity that outlives the computing-provider process: a pid does not, so a state file naming one can come back after a reboot pointing at an unrelated process. Ownership is decided by container labels and nothing else. `ps` lists only containers carrying io.swanchain.cp.managed, and `stop` will only touch one, so a backend the operator started themselves is invisible here and cannot be killed by this tool. The container name is a lossy slug and is never parsed back into a model ID; the label carries the ID verbatim. Three refusals guard an existing setup, all of them before a container is created: - a managed container already serving the model (--replace restarts it); - a models.json entry with no managed container behind it, which means something else is serving that model and repointing models.json would strand it, still holding VRAM and receiving nothing; - a name collision with a container this tool did not create. Backends contribute only an image, an entrypoint and an argument list. Each must serve under the Swan Inference model ID — --alias for llama.cpp, --served-model-name for vLLM and SGLang — or the hub routes requests the backend then rejects as unknown. Other decisions worth stating: - Published on 127.0.0.1. Model servers have no authentication of their own, and 0.0.0.0 would put an unauthenticated GPU on every interface of the host. - Readiness means the model is listed, not that the port answers. llama.cpp and vLLM both bind before the weights finish loading. - A failed load leaves the container running. Its logs are the only record of why, and the error points at them. - models.json is merged as raw JSON rather than rewritten through a struct, so an operator's api_key or hand-set override survives a restart and a field this binary does not know about is not dropped from an unrelated model. The write is atomic because the daemon watches the file with fsnotify; a truncate-then-write is seen as an empty file and deregisters everything for as long as it takes. - Flags precede the model ID and anything after it is passed to the server, as `docker run [options] IMAGE [command]` does. Flag parsing stops at the first positional, so `serve my/Model --dry-run` would otherwise have started a container and passed --dry-run to the inference server; a misplaced flag this command defines is now an error. - `--gpus none` omits --gpus entirely, for CPU-only models and hosts with no NVIDIA runtime. Verified end to end against a real daemon: serve, registration, a completion answered under the Swan model ID, --replace onto a new port with an operator-set api_key preserved, stop --rm, and deregistration — with two unmanaged production containers on the same host untouched throughout. 52 tests cover the argument lists, the ownership rules, the refusals, the models.json merge and the readiness loop.
… as a fit Step 4 of #143, promoted ahead of the scheduler because it is a blocker for it rather than a cleanup after it. The rule read a missing requirement as compatible: compatible := vramPerGPU == 0 || m.MinVRAMGB == 0 || m.MinVRAMGB <= totalVRAM The server publishes `vram_known: false` for a model whose requirement has never been established, and sends `min_vram_gb: 0` alongside. Measured against the live endpoint just now, that is **26 of 26 models** — not an edge case, the whole catalogue. So the middle clause fired for every model on the network and every one of them displayed a green "fits" against this node's 10 GB cards, DeepSeek-V3.2 and Kimi-K2.5 included. Fit is now three states, not two: fits, too-large, unknown. Unknown wins over every other answer, because a requirement nobody has measured cannot be compared, and a node that cannot detect its own hardware cannot do the comparing either. Neither is evidence that a model fits. The rule moves to internal/market so that the two callers who must agree on it cannot drift: recommend-models, which shows a human a column, and the auto-switch guardrails, which will decide whether to start a model with nobody watching. FitsKnown is the question an automatic decision has to ask — "not known to be too large" is a much weaker statement, and using it is how an unmeasured 600B model gets scheduled onto a 10 GB card. Display follows: the FIT column carries a glyph as well as a colour, the VRAM column reads "unknown" rather than "-", and a legend states how many requirements are unpublished and that such models are never selected automatically. --compatible-only now hides what is known not to fit rather than filtering on the old rule; it deliberately does not hide the unknowns, since with nothing published today that would leave an empty table and no clue why. select-model no longer calls its list "compatible models".
… switch Step 2 of #143, plus the switch notification. ## inference plan Runs one cycle of the auto-switch decision and prints it. Changes nothing: no server is started or stopped, models.json is untouched. The scheduler that would act on it is step 3 and is not built. The package splits down one line. The planner is a language model and holds the fuzzy judgement — a rising trend against a crowded model, a subscription-heavy standard-tier model against a thinner premium one. The guardrails are ordinary code and hold the operator's money. The planner proposes; code decides whether the proposal is allowed. It is called with json_schema constrained decoding, because the reply is parsed by code that acts on it and prose would otherwise be a refused cycle. Guardrails refuse a whole plan, never half of one: executing the half that passed leaves the node in a state the planner never proposed. Verified against the live market on a 4x RTX 3080 node — a proposal to serve DeepSeek-V3.2 was refused for an unpublished requirement, one to unload the planner for stops_a_pinned_model, an invented model ID for model_not_in_market, stopping a hand-started backend for stops_an_unmanaged_model, and prose for unparseable_decision. Two rules carry a distinction worth keeping: - current_earnings_unknown. The margin is measured against what the node actually earned over 24 hours, from the daemon's earnings *history* rather than its lifetime counters — those reset on restart and cover an unbounded window, so using them would compare a $/day projection against a total accumulated over however long the process happened to be up. When the figure is unavailable the switch is refused outright: substituting zero makes every proposal look profitable. A measured zero is a different answer and still allows a profitable switch. - The planner is always pinned, whatever the config says. A switch that unloads the model making the decision leaves no planner for the next cycle. Anything the planner is shown is observed, never inferred. Healthy is a *bool and stays nil when the daemon cannot be reached; the first live run had it as a plain bool, and the planner duly reasoned about an outage that was not happening. --decision '<json>' evaluates a decision you supply against the live snapshot instead of asking the planner, for testing a policy without waiting for the planner to propose something interesting. It goes through the same parser the planner's reply does. ## Switch notifications Whenever the set of served models changes — models serve, models stop, or the scheduler later — the operator is mailed through the [Alerts] transports they already configured, as model_serving_started / model_serving_stopped. Each carries who decided (operator or auto-switch) and why: whatever was passed to --reason, or the planner's own reason carried through unchanged. Announced after the model is serving and declared, not when the container is created; mailing about a model that then failed to load sends the operator looking for something that is not there. This needed Notifier.Flush. Fire is asynchronous so alerting can never block inference, which makes a short-lived command a hazard: it raises an alert and exits before the delivery worker runs, sending nothing at all. The daemon never needed this because it does not exit; every CLI path does. Without it the feature would have appeared to work and silently mailed nobody. Severity is info — a switch that worked is not a fault — so these bypass the failure cooldown. MaxSwitchesDay, not the alerting, is what stops a flapping planner filling an inbox. Confirmed end to end on this node: a serve and a stop each delivered mail over SMTP carrying the model, action, backend, endpoint, decider and reason. The model-demand fetch moves to internal/market so the display path and the guardrails read one struct from one call; two fetchers for one endpoint would drift, and the half that drifted would be the unattended one.
flyworker
changed the title
inference plan
added 3 commits
September 13, 2026 23:58
Groundwork for cp_agent, and for the gap #143 is currently blocked on: the marketplace publishes vram_known: false for 26 of 26 models, so a client that wants to pick models automatically has to work the requirement out itself. Findings: - No hosted API exists. Every VRAM calculator found is a web UI with no programmatic endpoint, so the arithmetic has to live in the node. - HuggingFace's ?expand=safetensors returns exact parameter counts broken down by dtype, unauthenticated, and works on gated repos. config.json carries the KV dimensions but answers 401 on gated repos, so a gated model yields a weight size and no KV term without a token. - gguf-parser-go (GPUStack, MIT, Go) reads GGUF metadata over HTTP range requests and reproduces llama.cpp's own allocation including --tensor-split. Being Go, it drops straight into this repo. The formula is validated against this node rather than asserted. Qwen3.8-27B at 65536 context with q8_0 KV: predicted 16.43 GiB against 17.42 measured, leaving 0.99 GiB of CUDA context and compute buffers — the overhead term the formula expects. Weights alone under-predicted actual usage by 18%, which is why capacity must never be planned on weight size. The note spends most of its length on where the formula fails, because that is the part that decides whether this is usable. The naive KV term over-predicted by 5.38 GiB, a 32% error, and the reason generalises: Qwen3.8-27B declares 48 linear_attention layers against 16 full_attention, so three quarters of its layers hold no KV cache at all. MLA (DeepSeek) and sliding-window (Gemma) break it in the same way. Here the error was in the safe direction; the dangerous one is under-predicting, which is an OOM partway through a load after the weights have been pulled. That is the case for reading real tensor shapes rather than assuming an architecture. Applied to the models the demand table ranks highest, on this 40 GiB node: DeepSeek-V3 needs 640+ GiB, Llama-3.3-70B at Q4 needs 42.9, and Qwen2.5-7B fits at 15.9. The best-earning models are the ones the hardware cannot run and nothing in the marketplace data says so. Recommends three tiers for cp_agent — measured, derived, unknown — with the rule that a lower tier's answer is never promoted to a higher tier's confidence, which is the rule internal/market.VRAMFit already applies.
The auto-switch guardrails were correct and useless: the marketplace publishes vram_known: false for 26 of 26 models, so nothing was ever known to fit and every proposed switch was refused. A scheduler built on that would have been well-behaved and permanently idle. This computes the requirement locally so the node can decide for itself. No hosted service does this — every VRAM calculator found is a web UI with no API — so the arithmetic lives here, from three primary sources: HuggingFace's ?expand=safetensors for parameter counts by dtype (unauthenticated, and it answers for gated repositories), config.json for the cache dimensions, and the node's own measurements where it has run the model before. The arithmetic is validated rather than asserted. Against this node's own Qwen3.8-27B at 65536 context with q8_0 cache: derived 17.43 GiB against 17.42 measured by nvidia-smi. That agreement depends entirely on corrections a generic formula does not make, and getting them wrong is the difference between a useful estimate and a dangerous one: - Hybrid attention. Qwen3.8-27B declares 48 linear_attention layers against 16 full_attention, so three quarters of its layers hold no KV cache. Counting all 64 over-estimates the cache four-fold. - Multi-head latent attention. DeepSeek V3 caches 512+64 elements per token per layer where the ordinary formula predicts 2 x 128 heads x 128 dims, about 25x too high. - Sliding windows, but only where enabled. Qwen publishes a sliding_window alongside use_sliding_window: false, and honouring it there would halve an estimate that should not move. - Quantisation byte sizes are measured, not nominal. Q4_K_S is 0.5528 bytes per parameter on the reference node's own file, not the 0.5 its name implies; taking the nominal figure under-states weights by 9.6%. Two failures found by running it against the live hub rather than fixtures: - Models must be sized at the precision they would be served in, not the one they are published at. Qwen3.8-27B is 60.2 GiB at bf16 and this node serves it in 17.4, because it serves a Q4 build. Sizing only the published precision reported 0 of 26 models as fitting while the node was demonstrably running two of them. FitWithin now walks down from the published precision and stops at the first that fits, refusing to go below q4_k_m because a provider that ships degraded output to win a fit is trading reputation for VRAM. - The hub's published parameter total is not reliable. Cydonia-24B reports a breakdown of 23,572,403,200 parameters alongside a total of 414,720 — five orders of magnitude out. Trusting it sized a 24B model at 0.0 GiB and reported it as a comfortable fit, which is precisely the under-estimate this package exists to prevent. The per-dtype breakdown is now summed instead. Every unresolved question resolves towards more memory or towards unknown, and derived figures carry a 10% margin. The asymmetry is the whole design: an over-estimate declines a model that would have fitted and costs some revenue, an under-estimate is an OOM partway through a load on a node that was serving traffic. On the live table this takes the node from 0 of 26 models decidable to 4, and the four are the right ones — they include both models the operator had already chosen by hand. The planner now argues economics instead of reporting that everything is unknown. Unknown stays unknown: 2 gated repositories could not be sized and remain unselectable, reported with the HF_TOKEN that would fix it.
Step 3, the last piece. A loop behind [Inference.AutoSwitch] Enable that re-plans every IntervalMin and acts when the guardrails allow it. Off by default, and it refuses to start rather than starting degraded: no planner configured, no endpoint for it, or a history it cannot read each stop it with an error rather than a warning. The three rules a single cycle could not judge are now judged, against a history persisted to $CP_PATH/autoswitch-history.json: dwell time, consecutive agreement, and the daily switch cap. Persisted because all three are defeated by a restart — a node that forgot its last switch on every start would honour no dwell time, and a crash loop would become a switch loop. A history that cannot be read is itself a refusal (scheduler_history_unavailable) rather than an empty one, since starting fresh would quietly discard exactly the state that exists to stop the node acting too often. EvaluateWithHistory stays separate from Evaluate so `inference plan` can run the stateless half honestly, naming what it has not checked, while the scheduler runs all of it. One function that silently skipped the stateful rules when handed no history would make a dry run read as a green light it is not. Details that are load-bearing rather than incidental: - A cycle records its proposal before the verdict, so "two consecutive cycles agree" counts the current one. Recording afterwards satisfies the rule a cycle late — half an hour on the default interval, about an opportunity the node had already decided on twice. This was a real bug, caught by the test. - Agreement is fingerprinted on the plan, not the prose. The same switch with a reworded reason or a projection a cent apart is the same plan; fingerprinting the text would mean a planner never agreed with itself twice. - A failed cycle records a keep, because two proposals either side of an outage did not agree consecutively. - A failed execution is not recorded as a switch. It has not changed what the node serves, and recording it would start a dwell period the node did not earn and block the retry. - Stops run before starts: the models being stopped usually hold the VRAM the new ones need. - The scheduler never downloads. Fetching 600 GB because a planner named a model would saturate the operator's connection on a decision that took thirty seconds. A model with no local weights is reported, not fetched. - Stop waits for an in-flight cycle and shuts down before the inference service, since a cycle mid-switch needs the node intact to finish and record what it did. History methods are nil-safe so a missing history refuses rather than crashing the daemon around it. Switches carry the planner's own reason through to execution, so the mail the operator receives records the argument the node acted on rather than only the outcome. Verified running in the daemon on this node: enabled at a one-minute interval, it logged its policy, fired a cycle, built the snapshot from the node's live earnings and health, recorded the proposal and persisted it, with no errors. The guardrail rules are mutation-checked — disabling dwell, agreement, the daily cap or the failed-switch rule each fails the test that names it.
flyworker
changed the title
inference plan
added 3 commits
September 14, 2026 00:49
…or an agent Published metadata says what a model should need. This records what it did need, here, with these settings, and where the weights came from. The two disagree often enough to matter. TheDrummer/Cydonia-24B-v4.3 derives to 30.7 GiB from its published bf16 parameters and runs on this node in 18.7, because it is served as a 4-bit AWQ build that nothing in the base repository mentions. Qwen3.8-27B derives to 31.8 at fp8 and runs in 17.4 as a Q4_K_S GGUF. Both were already working when the estimator would have refused them. Without this, a model the operator has made work is refused every cycle, forever. This fills in the measured tier that internal/vram specified and nothing wrote. Measured beats derived beats published, and a measurement carries no safety margin because it is not a prediction. Two files, one source of truth. model-memory.json holds the records. $CP_PATH/cp.md is operating context for an agent: the node's hardware and its limits, the rules that are enforced in code, every model known to work with what it cost, and the exact `models serve` command that produced each record — an agent re-serving a known-good model should run what worked rather than reconstruct it from fields and get one flag wrong. It is regenerated from the store on every write and never parsed back, so the two cannot drift and an accidental edit cannot corrupt anything. Measurement is per process, not per device: nvidia-smi --query-compute-apps gives usage by PID and `docker top` says which PIDs are the container's. A GPU shared between two models reports one total for both, and charging all of it to whichever was measured last would make the memory actively misleading. Verified against the two containers already running here — 17.40 GiB and 18.68 GiB, matching nvidia-smi exactly. Measurement waits for the backend to settle, since llama.cpp fills its KV cache lazily and vLLM's profiling run finishes after the server starts answering; measuring immediately records less than the model will hold. Five rules, each guarding a way a memory becomes worse than having none: - Records merge rather than replace. A start that did not measure must not erase last week's measurement, and one that did not know the quantisation must not blank a field the operator set by hand. - A failure never erases a working record. One failed start is usually transient, and discarding a configuration that took an afternoon to find over an upstream 502 is not acceptable. Only a model that has never worked here is marked failed. - --pin means the operator's record wins; automatic runs count the success and change nothing else. - A measurement does not transfer. It is tied to the hardware, context length and concurrency it was taken at, and reuse is asymmetric: a record taken at 32k answers a question about 8k, never the reverse. - A corrupt store is an error, not a silent reset. Also fixes the plan command labelling a measured figure as "derived", which inverted the point of keeping the measurement.
`computing-provider agent "<goal>"` gives a goal in plain language to a model this node already serves, along with cp.md and a fixed set of tools. Read-only unless the operator passes --allow-actions. Bounded by its tools rather than by its judgement. There is no shell and no way to reach anything outside the registry, so what a confused or manipulated model can do is exactly what those tools allow — which is why there are eight of them, each named and individually reviewable. The two that change the node are marked as such, refused without --allow-actions, and still go through every guardrail a person running the same command would hit. The agent is a way to reach the existing safety machinery conversationally, never a way around it. The grounding rule is the part that matters. Asked what the node was earning, the planner wrote "I need to check node_status", set done on the same turn, and returned a formatted per-model table reporting $142.80/day against a true $0.0683. It never called a tool. An operator reading that would have acted on fiction, and it was presented with more confidence than the correct answer. Prose in the system prompt does not fix this — a model that ignores one instruction ignores another — so finishing with zero successful tool results is refused in code. The attempt is surfaced to the operator rather than quietly retried, because a model willing to invent is worth knowing about even when the retry succeeds; the model is told it has observed nothing and given the tool names again; the run continues. A run that never looks at anything returns no figures at all, and says so. Three properties, each mutation-checked by disabling it and watching the test that names it fail: - No answer without evidence. - Acting tools need permission, decided per run and checked in the loop rather than by each tool remembering to check. - Required arguments are verified before a tool runs. Tool failures are fed back rather than ending the run: a wrong tool name or a missing argument is something a model can correct, and aborting would waste an episode on a typo. Unknown tools get the real names listed back. MaxSteps bounds a model that loops. Every step is printed with its arguments. An agent that changes a node while printing only its conclusions gives the operator no way to stop it part way and no way to reconstruct afterwards what it did. Verified against the live planner on this node. Asked what it earns, it now calls node_status and reports $0.0683/day with the correct per-model split. Asked to stop the worst model in a read-only run, it refused to act, was caught by the grounding rule about to guess which model that was, then looked it up and recommended rather than acted.
The grounding rule worked and then deadlocked. Asked whether any marketplace model would fit and pay better, the planner emitted a byte-identical step twelve times, was told the same thing each time, and burned the entire run without calling a single tool. A refusal that repeats itself is not a correction. At low temperature a model given the same context produces the same reply, so the rebuke has to change, and then it has to stop: - First refusal names the tools and says why the answer was discarded. - Second stops asking and instructs: do not set done, reply with exactly this shape, choosing a tool from this list. A model that ignored the general form of the request may still follow a specific one. - Third abandons the run and says plainly that the model would not use its tools, rather than spending the remaining steps to arrive at the same refusal. Also interrupts a repeated tool call, for the same reason in the general case: a model re-running a query it already has the result of is not making progress, and the remaining steps are better spent telling it so. With this the same question now produces a real investigation — market, then model_memory, then node_status — and an answer grounded in all three.
… actions Three gaps found by reviewing the agent design write-up against the code. The agent's actions were labelled decided_by: auto-switch in the operator's mail. An operator asked for the outcome but a model chose the action, and that is neither the scheduler nor the operator; the mail now says "agent" so a surprising change traces to the right cause. stop_model ignored the pin list. cp.md rule 4 promises the planner is never stopped, but that was enforced only in the scheduler's guardrails, so on a node whose planner was started with `models serve` the agent could be talked into stopping its own brain. The pin list — planner plus the operator's Pin — now binds on this path too, before the manager is asked anything. --quiet with --allow-actions is refused. A run that may change the node must show every step; one that prints only its conclusion gives the operator no way to stop it part way and no way to reconstruct what it did.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #143. All four steps, in the order that issue suggests, plus switch notifications and a VRAM estimator the issue did not anticipate needing.
Six commits, reviewable separately.
1.
models serve/stop/ps— the runtime manager (step 1)Nothing could bring a model up or take one down.
models downloadfetched weights, models.json was read once somebody filled it in, andsetupprinted a docker command to run by hand.Servers run as Docker containers — already how they run on provider hardware, and the container gives an instance an identity that outlives the CP process. A pid does not: pids are recycled, so a state file naming one can come back after a reboot pointing at an unrelated process.
Ownership is by container label and nothing else. A backend the operator started themselves is invisible here and cannot be killed by this tool. Three refusals fire before a container is created: already served; a models.json entry with nothing managed behind it; a name collision with a container this tool did not create.
Published on
127.0.0.1(model servers have no auth); readiness means the model is listed, not that the port answers; a failed load leaves the container so its logs survive; models.json is merged as raw JSON and written atomically because the daemon watches it with fsnotify.2. VRAM fit (step 4 — a prerequisite, not a follow-up)
The old rule read a missing requirement as compatible. The server sends
vram_known: falsewithmin_vram_gb: 0for 26 of 26 models, so every model on the network displayed a green ✓ against 10 GB cards. Fit is now three states, unknown winning over both neighbours, ininternal/marketso the human column and the unattended guardrail cannot drift.3.
internal/vram— deriving requirements locallyThe guardrails were correct and useless: nothing was ever known to fit, so a scheduler would have been well-behaved and permanently idle. No hosted service does this — every VRAM calculator is a web UI with no API (
docs/model.mdrecords the research) — so the arithmetic lives here.Validated, not asserted: against this node's Qwen3.8-27B at 65536 context with q8_0 cache, derived 17.43 GiB against 17.42 measured.
That rests on corrections a generic formula does not make: hybrid attention (Qwen3.8-27B has 48 linear layers against 16 full — counting all 64 over-estimates four-fold), MLA (DeepSeek, ~25x below the ordinary formula), sliding windows only where enabled, and measured quantisation sizes (Q4_K_S is 0.5528 bytes/param, not the 0.5 its name implies).
Two traps found only by running against the live hub:
totalis not reliable. Cydonia-24B reports 23,572,403,200 parameters alongside atotalof 414,720. Trusting it sized a 24B model at 0.0 GiB and called it a comfortable fit — the exact under-estimate this package exists to prevent.Effect on the live table: 0 of 26 decidable → 4, and the four include both models the operator had already chosen by hand.
4.
inference plan(step 2)One cycle, printed, changing nothing. The planner is a language model holding the fuzzy judgement; the guardrails are ordinary code holding the operator's money. Constrained decoding via
json_schema. A whole plan is refused, never half — executing the half that passed leaves the node in a state nobody proposed.--decision '<json>'evaluates a plan you supply against the live snapshot, for testing a policy without waiting for the planner to propose something interesting.5. The scheduler (step 3)
Behind
Enable, off by default, refusing to start rather than starting degraded. The three rules a single cycle cannot judge — dwell, consecutive agreement, daily cap — now run against a history persisted to disk, because all three are defeated by a restart and a crash loop would otherwise become a switch loop.A cycle records its proposal before the verdict so agreement counts the current cycle; agreement is fingerprinted on the plan, not the prose; a failed cycle records a keep; a failed execution is not recorded as a switch (it changed nothing, and recording it would start a dwell period the node did not earn). Stops run before starts. The scheduler never downloads.
6. Switch notifications
Any change to the served set mails the operator through their existing
[Alerts]transports with who decided and why. This neededNotifier.Flush—Fireis asynchronous so alerting can never block inference, which makes a short-lived CLI command raise an alert and exit before the worker runs. Without it the feature would have looked like it worked and mailed nobody.Verified on a live node
4x RTX 3080, real marketplace, two unmanaged production containers untouched throughout.
--replacepreserving an operator-setapi_key,stop --rm, deregistered.stopon an unmanaged model refused.vram_not_known_to_fit; unloading the planner →stops_a_pinned_model; invented ID →model_not_in_market; hand-started backend →stops_an_unmanaged_model; prose →unparseable_decision.Guardrail rules are mutation-checked: disabling each in turn fails the test that names it.
Known limitation
With requirements derived rather than published, the four models that fit this node earn $0.02–$0.05/day against a $0.05/day baseline, so the planner correctly answers
keep. The mode is now correct and able to act; whether it has anything worth acting on depends on the marketplace carrying models that fit mid-size nodes.