#305 - extract calypso-sim into a standalone MQTT simulator crate#326
Draft
bracyw wants to merge 30 commits into
Draft
#305 - extract calypso-sim into a standalone MQTT simulator crate#326bracyw wants to merge 30 commits into
bracyw wants to merge 30 commits into
Conversation
…ous modes Replace `simulate` with a single `calypso-sim` binary organized as a folder of focused modules under `src/bin/calypso-sim/`. Modes (can run together): - Autonomous heartbeat (`--auto`, default when no other mode is chosen) - Interactive keymap (`--key-map`) - Scripted keymap replay (`--script` with `--key-map`) - JSON-RPC 2.0 over stdio (`--stream`, for an agent) Per-topic ownership (`auto` / `stream` / `silenced`) lets stream and keymap modes claim/release/silence topics; the autonomous loop checks ownership before each publish. Stream protocol: NDJSON over stdin/stdout, methods `publish`, `claim`, `release`, `silence`, `status`, `list_topics`, `ping`. Tracing on stderr so stdout stays clean for JSON-RPC. Other: - Startup warning lists CAN topics with no `sim_freq` - README documents all four modes, keymap formats, and the stream protocol - Keymap supports Random / Pinned / Increment / Sequence per key, with optional `desc` per entry
Adds calypso-sim/scripts/: vcu_mimic.py (drives the sim's stream mode to emulate the VCU), the generated serverdata_pb2.py, and a README.
# Conflicts: # Cargo.toml # Odyssey-Definitions # src/bin/simulate.rs
Add `"unit": ""` to the 9 Wheel/Buttons/button_id entries so they publish without relying on a `sim` block in Odyssey-Definitions. We dropped the local-only OD sim-block commit and point the submodule at Develop's pointer, where button_id is not auto-simulatable; explicit units keep these keymap entries working. button_id is the only non-simulatable topic in this keymap (home_mode/nero_index/not_in_reverse remain simulatable via Develop's OD).
- autonomous: validate --enable/--disable-topic regexes up front so a bad pattern exits non-zero instead of silently disabling the heartbeat - script: fail-fast on malformed sleep / multi-char / unknown-key lines (deterministic replay shouldn't silently skip) - stream: async stdout so a stalled consumer can't block a runtime worker - build.rs: rerun-if-changed=Odyssey-Definitions so spec edits rebuild sim data - simulatable_message: guard SimValue::initialize against empty range/options to avoid a startup panic on a degenerate spec - drop set_topic_alias_max(600) to match the decoder and prior simulator - fix S5 README description (REVERSE is gateless) and vcu_mimic cleanup branch - remove Embedded-Base gitlink (not on Develop, no .gitmodules entry, unreferenced)
…st degenerate spec - Remove `crossterm` from the root Cargo.toml (zero refs in src/ or libs/; only the separate calypso-sim crate uses it, via its own manifest) and drop its now-orphaned transitive crates from Cargo.lock — pure removal, no other version changes. - Guard `SimValue::initialize` in the main crate against degenerate spec entries (empty min==max range, empty discrete options) so a malformed CAN spec can't panic at startup, bringing it to parity with the already-hardened calypso-sim copy. - Remove the now-stale "Divergence from upstream" note from the vendored calypso-sim copy, since upstream now carries the same guards.
…compile placeholder regex once
Centralize the per-topic publish/ownership policy that was re-derived inline at
four call sites across three mode files: add TopicRegistry::auto_may_publish
(the heartbeat publishes only while a topic is still Auto-owned) and
driver_may_publish (stream/keymap drivers publish unless a topic is Silenced),
co-located with the Owner enum, and route autonomous/keymap/stream through them.
Behavior-preserving.
In the vendored simulatable_message.rs, hoist the `{}` placeholder Regex in
topic_values_inject into a compile-once LazyLock and drop the per-call
intermediate Vec - a documented sim-local divergence from the upstream copy,
which still recompiles per call.
These per-repo Claude Code command/skill files were force-added in fdab310 despite `.claude` already being in .gitignore (since 7ac8de7). Untrack them with `git rm --cached` so the ignore rule takes effect — the files stay on disk locally and remain in history at fdab310; this only drops them from the branch tip going forward.
Replace scripts/vcu_mimic.py with native Rust tests that run under `cargo test` with no broker required: - tests/: spawn the real `calypso-sim --stream` binary and drive JSON-RPC over stdio — protocol conformance plus a VcuMock (mirroring Cerberus-2.0's u_statemachine.c) exercising boot/claim-all, the PIT brake+shutdown gate, and ownership isolation. - registry.rs: unit tests for the auto/stream/silenced ownership model. - publish.rs: extract encode_server_data so ServerData encoding is covered by a round-trip test, independent of any client/broker. - Add serde_json dev-dependency; document the suite in README.md. No broker is needed because publish only enqueues and ownership is answered from the JSON-RPC responses; broker-backed end-to-end value observation (drive-sweep/reverse/fault scenarios) is a planned follow-up.
…n, drop unused root serde dep
Cover the real, previously-untested logic: - keymap: advance_increment wrap/saturate semantics, the serde untagged entry disambiguation (step beats value; sequence wins), and build_topic_states skip/keep rules - cli: run_autonomous mode arbitration Drop encode_handles_empty_unit_and_values (it exercised protobuf, not our code; the field-mapping round-trip stays). Reframe the README + scenarios e2e notes from a "planned follow-up" to intentionally out of scope, since Siren in the compose stack covers real bytes-on-the-wire.
Replace the sprawling suite with a lean one where each assertion is independently verifiable: - delete scenarios.rs (a VcuMock mirroring unverifiable Cerberus pedal / fault constants; s3 only asserted the mock's own gate) and fold the protocol + ownership-isolation integration tests into one dedicated tests/stream.rs (harness inlined) - trim inline unit tests: registry 5->3, keymap 10->6, cli 3->1 (publish unchanged) Net 29 -> 18 tests. No JSON-RPC protocol coverage lost; the dropped cases are low-risk edges (snapshot sort order, empty-sequence / multi-char keymap guards, unbounded increment). README Testing section updated to match.
… a minimal regression-guarding set (18 -> 7)
…f a duplicate copy
…ares it DecodeData/EncodeData/FormatData now live in calypso-cangen; the calypso crate re-exports them through its own data module (macros unchanged) and calypso-sim imports DecodeData directly instead of vendoring a copy.
…docs) --auto becomes --mock, Owner::Auto becomes Owner::Mock, and the run_autonomous / modes::autonomous identifiers follow suit. The stream ownership wire string changes from "auto" to "mock" accordingly (the previous_owner / owner fields in claim/release/silence responses).
- Fmt CI step: drop `--all` so cargo-fmt stays scoped to the sim's detached
workspace instead of walking path deps up into the main calypso workspace
(whose generated `src/proto` never exists in the sim CI checkout). This was
failing the calypso-sim CI deterministically at the Fmt step.
- Stream mode: a request missing `method` is now an Invalid Request (-32600)
rather than a parse error (-32700), matching JSON-RPC 2.0.
- Docs: note that `publish` on a silenced topic returns `{skipped: "silenced"}`.
… Clippy The calypso-sim CI previously died at the Fmt step, so its Clippy step never ran. With Fmt fixed, Clippy flags two `.map(f).unwrap_or(a)` chains on `Result` (main.rs, publish.rs) under the CI toolchain; rewrite them as `.map_or(a, f)`.
…model An action is a named list of steps (publish | sleep | invoke another action). Interactive binds an action to a key; --play runs one by name (replacing --script); actions compose by invoking each other, so scenarios are one self-contained file. Removes the four keymap entry shapes, the parallel KeyEntry/KeyMode enums + increment cursor, and the separate script parser.
…tup partition Ownership is now resolved once at load instead of negotiated at runtime: the driver's topics (a scenario's, or none for --stream) are removed from the mock heartbeat's set up front and the split is printed, so the two publish disjoint topics with no locking. Deletes the registry (Owner enum / RwLock / per-publish predicates), the keymap lazy-claim, and the stream claim/release/silence/status RPCs -- the stream protocol is now publish/list_topics/ping, and a stream driver reserves topics with --disable-topic. Also adds a scenario example to the keymap model docs.
Review + /simplify follow-ups on the static-ownership work: - simulatable_message: guard inverted (min > max / inc_min > inc_max) sim ranges, not just degenerate (min == max), so a malformed spec can't panic random_range on a backwards range. update() already clamps via get_rand_offset. - keymap: move the duplicate-key check into validate() so it runs at load for --play too (not just interactive mode), and add a regression test. - stream: distinguish empty `values` from missing value/values in the JSON-RPC error message. - keymap: derive scenario_topics by unioning each action's own publish steps instead of flattening invokes -- equivalent (every invoke target is a top-level action), and drops a throwaway Vec<Prim> alloc and the acyclic precondition. - stream: cache the list_topics (name, unit) pairs in a LazyLock instead of rebuilding every component (with per-point RNG init) on each request.
…math into shared helpers
- Add `publish::resolve_values` as the single "exactly one of value/values,
non-empty -> Vec<f32>" policy; scenario validation, replay flattening, and
the stream RPC all call it (drops the bespoke `check_publish` and stream's
duplicate match arm).
- Add `SimValue::{quantize, sample_range_or_low}`; initialize/update/
get_rand_offset now share the snap-to-increment/round and the
range-sample panic-guard instead of each repeating them.
- Drop redundant `#[serde(default)]` on Option fields (serde already maps a
missing Option to None), a stale cross-file comment, dead commented-out
code, C-port section banners, and an over-long doc example.
- interactive: `classify()` + an `Input` enum isolate the crossterm `KeyEvent` destructuring, so the `select!` loop is a flat 4-arm match (behavior unchanged: Ctrl+C/EOF quit, bound key runs, error reports+quits). - mock: the `interval.tick()` arm calls `publish_due()`; the walk-and-publish logic moves into that helper. - main: the mock-heartbeat partition setup (build filter, resolve `Partition`, print split, spawn) moves into `spawn_mock()`, slimming `main` to a sequence.
…helpers - detect_cycle: add a shared "verified acyclic" set so an action reused by many others is walked once, not once per path that reaches it — avoids exponential blowup on diamond-shaped invoke graphs. Cycle detection and error messages are unchanged. - resolve_values: take Option<&[f32]> instead of Option<Vec<f32>>, dropping the throwaway clone in validate/flatten; it now clones once internally only when returning the accepted array. - stream: replace two #[allow(clippy::needless_pass_by_value)] with #[expect(..., reason = ...)] so the suppressions are self-checking and documented.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Extracts the MQTT simulator out of the main
calypsobinary (deleting the in-treesrc/bin/simulate.rs) into a standalone, detached-workspace crate atcalypso-sim/with four input modes — mock heartbeat, interactive keyboard, deterministic replay, and a JSON-RPC-over-stdio stream — kept from fighting over the same topic by a static ownership partition resolved once at startup (the heartbeat cedes any topic a driver owns, up front — no runtime negotiation). Interactive and replay share one recursive scenario model (an action is a list ofpublish/sleep/invokesteps), so there's no separate keymap-vs-script pair of files to coordinate. Also hoists the shared CAN data-model types (DecodeData/EncodeData/FormatData) intocalypso-cangenso the sim reuses them without depending on the Linux-onlycalypsocrate, removes the now-dead simulate path frommain, ports the old Python VCU-mimic harness to broker-free Rust tests, keeps thesimulateDocker entry point, and adds a path-filtered CI workflow for the crate.Notes
calypso-simis its own detached workspace (empty[workspace], ownCargo.lock/target). It does not build from a top-levelcargo build --all—cd calypso-simfirst. It has its own path-filtered CI (calypso-sim-ci.yml) that runs only when the crate or its path-deps change.publish·sleep· orinvokeanother action. Interactive binds an action to akey;--play <action>runs one by name (this replaces the old--script FILE); actions compose by invoking each other, so a scenario is one self-contained file. This collapses the previous four keymap entry shapes, the parallelKeyEntry/KeyModeenums, and the separate script parser into a single primitive — and steps are shape-disambiguated, so there's no order-dependentuntaggedmatching. Invalid/cyclic invokes, duplicate keys, and bad publish-arity are rejected at load. The keymap/script formats were new in this PR, so there's no external consumer to break.--key-map/--play, auto-derived; nothing for--stream, where you reserve topics with--disable-topic) is removed from the heartbeat's set up front, so the two publish disjoint topics by construction — and the split is printed at startup. This deleted theArc<RwLock<…>>ownership registry and the streamclaim/release/silence/statusRPCs; the stream protocol is now justpublish/list_topics/ping.calypso-cangen, notdaedalus— daedalus isproc-macro = trueand can't export plain types, whilecalypso-cangenis a normal lib both crates already depend on. Main'ssrc/data.rsis now a thin re-export shim, so the decode/encode codegen macros (crate::data::…) are unchanged.autonomous/--autotomock/--mock(flag, module, and docs).cargo fmt --check, not--all—--allmakes cargo-fmt walk the sim's path-deps up into the main workspace and try to format its generatedsrc/proto, which never exists in the sim CI checkout.Test Cases
All from inside
calypso-sim/:cargo run -- --list-topics→ prints every simulatable topic, then exits.cargo run→ mock heartbeat; publishes randomized values on every topic with asim_freqin the spec.cargo run -- --key-map manual_sim_buttons.keymap.json --mock→ interactive + heartbeat; prints anOwnership: mock heartbeat drives N topic(s); M reserved for the driver …line showing the scenario's topics are ceded, and each action fires on its bound key.cargo run -- --key-map manual_sim_buttons.keymap.json --play demo→ runs thedemoaction (its nested invokes +sleep_mswaits, in order), then exits.cargo run -- --stream, then feed{"jsonrpc":"2.0","id":1,"method":"publish","params":{"topic":"Wheel/Buttons/button_id","value":5}}→ responds{"jsonrpc":"2.0","id":1,"result":{"ts_us":...}}.cargo test→ 6 unit + 3 integration tests pass, no broker needed.cargo fmt --checkandcargo clippy --all -- -D warnings→ clean.Checklist
Closes #305