Port CT002/CT003 emulator to ESPHome as external component#385
Conversation
The _compute_smooth_target guard rejected any non-3-tuple input and returned the raw grid reading unchanged, so single-phase users got passive forwarding instead of balanced targets despite single-phase being a documented supported configuration (README and four powermeter backends explicitly support it). The function immediately collapses values to a scalar total via sum(parse_int(v, 0) for v in values), which works for any tuple length, and the balancer always returns a 3-element list, so the 3-tuple requirement was an over-eager defensive check that silently disabled real functionality. Keep the empty-values short-circuit. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
Lands the foundation for the C++ port of src/astrameter/ct002/ as an
ESPHome external component under esphome/components/ct002/:
- protocol.{h,cpp}: full port of ct002/protocol.py (SOH/STX/ETX framing,
build_payload, parse_request, calculate_checksum, RESPONSE_LABELS).
Replicates the firmware checksum-space-tolerance quirk on the decoder.
- ct002.{h,cpp}: minimal CT002Component skeleton — captures the
configuration surface and caches sensor values into a per-phase array.
UDP server, balancer, and filter pipeline land in subsequent commits.
- __init__.py: codegen schema with both-or-neither l2/l3 validation,
MAC normalization, and the standard cv.COMPONENT_SCHEMA extension.
Tests (parity-by-discipline, per the plan):
- tests/test_ct002_protocol_parity.py: Python parity test against the
shared golden-vector JSON. Every build_payload output matches the
canonical wire_hex, every parse_request round-trips, and the space-
tolerance decode mutation is covered.
- tests/components/ct002/host_protocol_test.cpp: 16-case gtest that
runs the same vectors through the C++ implementation. Built via
CMakeLists.txt that FetchContent's gtest 1.14.0 so no system
libgtest-dev is required. Driven from pytest by test_host_protocol.py;
skipped when cmake or a C++ toolchain is missing.
- tests/components/ct002/test_codegen.py: unit tests for the schema
validators (MAC normalization, both-or-neither l2/l3); skipped when
esphome is not installed.
- tests/components/ct002/test.*.yaml: compile matrix targets (host,
esp32-idf, esp32-arduino, single-phase). Validated by the planned
esphome-compile CI job once added.
CI: new ct002-host-protocol job runs the host-gcc parity test on every
push, providing a fast wire-format regression guard.
https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…ng/deadband/pid)
Adds the four sum-of-phases filter wrappers as plain helper classes under
esphome/components/ct002/powermeter/, mirroring
src/astrameter/powermeter/wrappers/*.py file-for-file:
- base.h: Powermeter interface + PowermeterWrapper decorator base. Wrappers
hold a non-owning pointer to upstream; CT002 owns the actual instances
via std::vector<std::unique_ptr<Powermeter>> at pipeline construction.
- hampel.{h,cpp}: rolling-median outlier filter with MAD-scaled threshold,
proportional phase redistribution, and median write-back to prevent a
single spike from poisoning future detections.
- smoothing.{h,cpp}: EMA smoother with sign-flip catchup alpha, optional
max_step clamp, and the dedup that prevents multi-consumer polling from
compounding the update. Plus DeadbandPowermeter (stateless |sum| clamp).
- pid.{h,cpp}: PID with anti-windup (integral paused while saturated unless
unwinding), bias/replace modes, injectable clock for deterministic tests.
Per the plan's numeric policy: float for inputs/outputs, double for the
Hampel median/MAD sort, the smoothing EMA accumulator, and the PID
integral — the long-running accumulators where float bias would drift.
Tests: 15 host-gcc gtest cases covering window-fill passthrough, outlier
rejection, threshold acceptance, proportional redistribution, EMA seed,
EMA convergence, max_step clamping, dedup, deadband threshold both sides,
PID zero-error, P-term sign, output clamping, and integral wind-up.
CT002_HOST_TEST=1 in the gtest build swaps the production millis()-backed
clock for the injectable test clock.
The Python wrappers retain their existing pytest battery; these gtests
are the C++-side guard.
https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…ipeline
Full line-by-line port of src/astrameter/ct002/balancer.py to
esphome/components/ct002/balancer.{h,cpp}. Method, struct, and field names
are preserved exactly so cross-language fixes map 1:1.
Includes:
- BalancerConfig + clamp() (matches Python __post_init__ clamps).
- ConsumerMode (auto/manual/inactive variant via enum + manual_value).
- BalancerConsumerState (per-consumer EMA + grace state) and ProbeState
(in-flight efficiency handoff).
- SaturationTracker: time-weighted EMA with grace handling, sign-reversal
detection, long-gap drop, stall-timeout escalation. EMA scores kept in
double per the plan's float/double policy.
- LoadBalancer: the full compute_target dispatch (inactive → manual →
auto), saturation feedback, manual override, lifecycle (remove/detach/
reset/force_rotation), observability (get_saturation/get_last_target).
- Auto-target pipeline: AC-chargeable charge-territory clamp (issue #338),
fading path during probe handoff, sign-disagreement clamp, fair-share
with balance correction, phase split by reporter weights.
- Efficiency deprioritization: hysteresis (1.2x), saturation-swap check
with cache-busting, probe begin/commit/reject with proof-sample
threshold, fade weights via efficiency_fade_alpha EMA with post-probe
ramp-down window.
Numeric policy: float for inputs/outputs/intermediate arithmetic; double
for the long-running accumulators where small bias would drift (the EMA
saturation_score, the time-weighted decay accumulator).
Tests: 7 host-gcc gtest cases covering AC-chargeable detection, inactive
steering, manual override, fair-share single/multi-phase, DC-only clamp
under surplus, remove_consumer state clearing, reset_consumer clearing.
The Python suite (tests/test_balancer*.py) remains the canonical
algorithmic specification; these gtests are the C++-side guard against
translation bugs.
https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…cer wiring
Expands the previously-scaffolded CT002Component into a functional CT002
emulator: UDP server, request parsing, sensor pipeline integration, and
balancer-driven response building.
Adds sensor_backed.{h,cpp}: the only file with no Python analog — bridges
ESPHome push-delivered sensor::Sensor* state into the Powermeter polling
interface that the wrapper pipeline expects. Cached raw_values/raw_stamp_ms
live on CT002Component; the bridge reads them and applies the
max_sensor_age timeout (rollover-safe via unsigned arithmetic, no
wrap-tracking needed).
ct002.{h,cpp} now mirrors src/astrameter/ct002/ct002.py for the UDP path:
- Consumer struct holds the per-battery state that the report handler
updates (phase, power, device_type, last_ip, timestamp, last_instructed_
power, active, manual_enabled, manual_target).
- consumer_key_/update_consumer_report_/get_consumer_ mirror the Python
helpers 1:1.
- validate_ct_mac_ replicates the case-insensitive lowercase-no-colons
compare from ct002.py:582.
- handle_request_ replicates the full Python path: parse → validate MAC →
build consumer_id → strip+upper phase → detect inspection mode (any
marker not in {A,B,C}) → update consumer → read filter pipeline → run
active control via compute_smooth_target_ → record net last_instructed_
power on phase index → build 24-field response → sendto.
- build_response_fields_ mirrors _build_response_fields exactly including
the info_idx_counter wrap, the chrg_nb activity flag derivation, and
the per-phase chrg/dchrg_power cross-talk aggregation. Pads to
RESPONSE_LABEL_COUNT to match Python's invariant.
- compute_smooth_target_ assembles the ReportMap + inactive/manual sets
for the balancer and dispatches via LoadBalancer::compute_target.
- collect_reports_by_phase_ aggregates last_instructed_power (not raw
reports) so PV-passthrough doesn't masquerade as discharge (#376).
UDP server uses esphome::socket primitives (socket_ip + bind +
non-blocking + setsockopt SO_REUSEADDR/SO_BROADCAST). The loop() pump
caps at 32 packets per tick to avoid starving other components on a
burst. The setBlocking(false) failure is logged-and-continued (some LWIP
variants are picky); SO_BROADCAST is best-effort.
The filter pipeline (Hampel → Smoothed → Deadband → PID, matching
config_loader.py order) is built in setup() based on per-filter enable_*
configuration setters. The YAML codegen wiring for those setters lands
in a follow-up alongside the README docs.
Host-gcc gtests are unchanged — they target protocol.cpp, balancer.cpp,
and the wrappers; ct002.cpp only compiles in the full ESPHome context
since it depends on socket/sensor/hal headers. Test coverage for the
request handler comes via the Docker E2E (Tier D) in a later commit.
https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…docs
Extends esphome/components/ct002/__init__.py with sub-block schemas for
the cross-phase filter pipeline and the balancer/saturation tracker so
users can configure them entirely from YAML:
- filters: hampel{window,n_sigma,min_threshold}, smoothing{alpha,max_step},
deadband{deadband}, pid{kp,ki,kd,output_max,mode}. Each sub-block is
optional; missing block = wrapper omitted from pipeline, matching
Python's fallback=0 → wrapper skipped semantics.
- balancer: full BalancerConfig surface (fair_distribution, balance_*,
error_*, max_*, min_efficient_power, probe_min_power, efficiency_*).
Codegen passes it as a cg.StructInitializer to set_balancer_config.
- saturation: alpha, min_target, decay_factor, grace_seconds,
stall_timeout_seconds, enabled. Dispatched via set_balancer_saturation.
Test YAMLs:
- test.esp32-idf.yaml now exercises the full filter + balancer +
saturation config so the compile matrix covers the new code paths.
- test.no-filters.yaml: minimal config with active_control=false to prove
the pipeline cleanly omits all wrappers and the balancer is bypassed.
User-visible docs:
- esphome.example.yaml at repo root — drop-in template with per-sensor
filters: (offset/multiply/throttle) commented as the ESPHome-idiomatic
per-phase substitute for Python's TransformedPowermeter/ThrottledPower-
meter, plus the ct002: sub-block with annotations on every knob.
- CHANGELOG.md ## Next bullet now describes the new ESPHome component
alongside the 1-phase Python fix (one bullet per branch per CLAUDE.md;
the bullet was scoped down earlier and is now expanded as the
ESPHome component reaches a user-runnable state).
https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…ADME + CI matrix
ESPHome's external_components loader doesn't recurse into arbitrary
subdirectories — only platform-split sub-components (e.g. ct002/sensor/).
The powermeter/ subdir caused the include of "powermeter/base.h" to fail
at compile time. Flattening: powermeter/base.h → wrapper_base.h,
powermeter/{hampel,pid,smoothing}.{h,cpp} → {hampel,pid,smoothing}.{h,cpp}.
All cross-includes and CMakeLists test wiring updated.
Two ESPHome-specific compile fixes:
- ct002.h now includes pid.h directly so PidMode is visible to the public
enable_pid() signature and the PidCfg member struct.
- ct002.cpp's std::stoi call would fault under ESPHome's no-exceptions
build (-fno-exceptions is the default). Replaced with strtol + end-
pointer check, matching ESPHome's parsing idioms.
Validated with a real `esphome compile tests/components/ct002/test.host.yaml`
end-to-end against esphome from PyPI — the host-platform binary builds
cleanly and links. Host-gcc gtest suite (31 cases) still passes.
CI: new ct002-esphome-compile matrix job runs `esphome compile` against
all five test YAMLs (host, esp32-idf, esp32-ard, 1phase, no-filters).
README: new "ESPHome External Component" subsection under Getting
Started — minimal YAML, link to esphome.example.yaml, ESP32-only
limitation, per-phase-filters-on-the-sensor divergence from Python.
https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…tterySimulator Adds the canonical-client end-to-end test the plan calls out as Tier D: the existing AstraMeter BatterySimulator drives real CT002 UDP polls against the freshly-built ESPHome host binary and asserts the response is a parseable 24-field CT002 frame with the right ct_type/ct_mac echo. This validates the full sensor → filter pipeline → balancer → response builder → wire flow against the same code real users run against their B2500s, in a single pytest. Any drift between Python's expectations and the C++ port's responses is caught here without needing a real ESP32. How it works: - test_host_e2e.py spawns the binary in a process group, waits for it to bind UDP 12345, runs BatterySimulator.step() a few times, asserts the response parses with the expected shape. Tears down with SIGTERM (escalating to SIGKILL after 2 s) and the killpg so any stray ESPHome internal threads die too. - A module-scoped host_binary fixture compiles the binary on first use via `esphome compile test.host.yaml` and reuses it across tests. - Skipped when the esphome CLI is not on PATH so local dev without a full ESPHome install is fine. CI: - New ct002-host-e2e job runs against ubuntu-latest; installs esphome via `uv tool install esphome`, runs the pytest. Build cache is per-job for now (no caching across runs yet — that lands when build time becomes a measurable bottleneck). Validated locally: BatterySimulator gets a 24-field response from the host binary, ct_type=HME-4 (as configured), meter_mac echoes back the requested AABBCCDDEEFF. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
Adds an "ESPHome external component (parity rule)" section to CONTRIBUTING.md so future contributors know that bug fixes in src/astrameter/ct002/ must be ported to the matching file under esphome/components/ct002/ in the same PR. Names file-by-file mappings, calls out the wrappers that have no C++ counterpart (delegated to ESPHome's standard sensor: filters:), and points at the host-gcc gtest suite as the C++-side regression guard. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
Adds the astrameter_mqtt_insights external component, mirroring
src/astrameter/mqtt_insights/ adapted for ESPHome's single-loop model:
- service.py → mqtt_insights.{h,cpp}. Drops the asyncio queue (events
fire synchronously from ct002's listener callbacks), the reconnect
loop (ESPHome's mqtt component owns reconnect; we detect transitions
by polling is_connected() so we don't steal the user's set_on_connect
slot), and the ARP lookup (lwIP doesn't expose /proc/net/arp). Republishes
device-level + per-consumer discovery on every reconnect rising edge.
- discovery.py → discovery.{h,cpp}. Same node_id / unique_id / state_topic
shape and same HA Jinja value_templates so a dashboard built against
the Python service works unchanged. Builds payloads through
json::build_json (stack-first allocation via ESPHome's SerializationBuffer).
- marstek_mqtt.py → marstek_responder.{h,cpp}. Standalone pure-C++ helpers
(no ESPHome deps) covering normalize_mac, parse_app_topic, parse_poll_payload
(cd=1 aggregate / cd=4 slave list), and the wire-format builders for both
the legacy core frame and the extended cd=1 runtime frame. Field order
matches the Python implementation exactly so the Marstek app and
hm2mqtt-style parsers can't tell which stack served the response.
ct002 gains the public API the insights component needs: snapshot_consumer
(returns the dict shape the Python service.py emits per event), an event
listener registration so insights gets notified after every successful
UDP poll-reply cycle, command-path setters (set_consumer_active /
manual_target / auto_target, force_balancer_rotation), and read-only
views of latest grid power / reporting consumer rows. The handle_request_
path captures pre- and post-balancer values into shared per-component
caches and fires listeners at the end of each successful round trip.
Tests:
- New host-gcc gtest suite host_marstek_responder_test (18 cases): poll
parsing for cd=1/cd=4 happy + rejection paths, topic templates for both
hame_energy and marstek_energy flavors, exact byte-level assertions on
the legacy and cd=1 aggregate response shapes, cd=4 CSV building
including the sanitize-out-of-band-chars escape rule. CMakeLists wires
it into the FetchContent-bound googletest harness; pytest fixture in
test_host_protocol.py picks it up automatically.
- The full host-platform `esphome compile` of test.host.yaml still
succeeds after the ct002 extensions (validated locally via the upstream
`esphome` CLI from PyPI).
- New CI matrix yaml test.insights.esp32-ard.yaml exercises the codegen
+ full C++ compile against esp32-arduino with both ct002 and the
insights sub-block wired up; runs in the existing ct002-esphome-compile
job.
Docs:
- README "ESPHome External Component" subsection expanded with the
insights YAML, what HA entities are published, and the Marstek topic
set the responder speaks.
- CONTRIBUTING parity-rule section gets file-level mappings for the
three new files (marstek_mqtt/discovery/service → responder/discovery/
mqtt_insights) with the architectural-diff caveat for service.py.
- CHANGELOG single-bullet update: insights is now in scope; only marstek
cloud registration is deferred.
Validated end-to-end: 711 pytest pass, 4 host-gcc gtest suites pass
(50 cases total across protocol/wrappers/balancer/marstek), host-platform
E2E against BatterySimulator still green.
https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…on first boot Adds the marstek_registration external component, mirroring src/astrameter/marstek_api.py adapted for ESPHome: - ensure_managed_fake_device's linear flow (token → list → maybe-add → confirm) runs as a state machine in loop() so the watchdog stays fed between HTTPS calls. Each HTTP step blocks for a few seconds via ESPHome's http_read_fully (which already feeds the watchdog mid-read), but the gaps between requests yield to other components. - Same wire format: same URL paths (/app/Solar/v2_get_device.php, /ems/api/v1/getDeviceList, /app/Solar/v2_add_device.php), same Dart/2.19 User-Agent, same MD5(password) salt, same 02b250 managed-MAC prefix. JSON parsing via the existing json component; password hashing via the md5 component (both AUTO_LOADed so users don't need to add the blocks manually). - MAC persisted to ESPPreferences (one stable slot per device_type variant) so the second boot loads from flash and skips the cloud round-trip entirely. Marstek's cloud is rate-limited; re-running the full flow on every boot would be both rude and slow. - New mac flows back into ct002 via the existing set_ct_mac() public setter — CT002 responses and Marstek MQTT topics use the cloud-side identity from that point forward. Documented limitation: users who enable marstek_registration AND astrameter_mqtt_insights should reboot once after first-time registration so mqtt_insights re-subscribes with the freshly-assigned MAC. The architectural diff from Python: - No "chunked HTTPS in loop()" — ESPHome's http_read_fully covers watchdog management. - No explicit cloud merge of solar+ems lists — we accept either source for the device records we use to detect "managed" entries; the merge only mattered to surface salt/version/access fields which Python doesn't use for decision-making either. - Configurable retry_interval (default 60s) for transient errors; fatal errors (bad config, mark_failed) stop the state machine entirely. - force_reregister option for the "I deleted the cloud-side record, redo the flow" workflow. Tests + CI: - Host-platform `esphome compile` of test.registration.host.yaml validated locally — the full C++ links cleanly against ESPHome's host http_request backend. - New CI matrix entries: test.registration.host.yaml (catches generic C++ regressions cheap) and test.registration.esp32-ard.yaml (catches ESP32-specific issues like MD5 backend / preferences). Docs: - README ESPHome subsection extended with the marstek_registration YAML and the reboot-once caveat for users who enable it alongside mqtt_insights. - esphome.example.yaml gets a commented marstek_registration block next to mqtt_insights. - CONTRIBUTING parity rule mapping for marstek_api.py → the new component. - CHANGELOG single-bullet update: cloud registration is now in scope; all four ESPHome sub-blocks (ct002, mqtt_insights, marstek_registration) are functional and just need wider field testing. All gates green: 711 pytest pass, 4 host-gcc gtest suites (50 cases), host-platform E2E against BatterySimulator still works. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…tters, EMA
Addresses the bugs/drift the post-merge review surfaced against the
Python source under src/astrameter/ct002/:
1. balancer.cpp:443 — probe-target backup_weights bug. The function had
the eff_part parameter declared anonymously, which forced the body
to use a literal 0.01f placeholder for every backup. Result: during
the probe window the fair-share denominator ignored per-consumer
efficiency entirely, so backups shared support load equally
regardless of how well they could actually follow targets. Now named
and seeded from eff_part.get(cid, 1.0f) with a floor at 0.01, exactly
matching balancer.py:612-614.
2. ct002.cpp — consumer eviction loop. Python's _cleanup_consumers
runs every 5 s and evicts anything older than consumer_ttl (120 s
default). Without this the consumers_ map grows unbounded across
battery turnover, the balancer keeps stale per-consumer state alive
forever, and mqtt_insights' "offline" availability never publishes.
Wired up via set_interval in setup(); fires the existing (but
previously-unused) consumer_removed_listeners_ before erasing the
entry so the listener can read the final snapshot. Codegen exposes
consumer_ttl (default 120s) so users can tune for short-lived
bench-test setups or long-poll-gap installs.
3. ct002.cpp — update_consumer_report_ now computes the poll_interval
EMA (alpha=0.3, round to 0.1s) that mqtt_insights publishes as a
diagnostic. Previously the snapshot.poll_interval optional was
always empty.
4. ct002.cpp — command setters (set_consumer_active /
set_consumer_manual_target / set_consumer_auto_target) now
auto-create the consumer entry like Python's _get_consumer, so HA
commands that arrive before the first UDP poll work instead of
silently no-op'ing.
5. ct002.cpp — set_consumer_active(true) calls balancer.reset_consumer
to drop stale saturation/last_target. set_consumer_auto_target(true)
does the same when leaving manual; set_consumer_auto_target(false)
calls balancer.detach_from_auto_pool to pull the consumer out of the
efficiency rotation immediately rather than waiting for the cache
to invalidate. Mirrors ct002.py:242-269.
6. ct002.cpp — force_balancer_rotation now filters the pool to
{active AND not manual_enabled}, matching Python's
force_efficiency_rotation. The earlier "all reporting consumers"
pool let inactive/manual batteries leak into the rotation.
Also adds an active_control() getter on CT002Component so
mqtt_insights can publish the real configured state instead of
hardcoding true (separate commit will use it).
All 711 pytest + 4 host-gcc gtest suites pass; host-platform
esphome compile + E2E both still green.
https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…ast_seen, IP re-publish
Addresses the bugs/regression the post-merge review surfaced against
Python's service.py:
1. mqtt_insights.cpp:200 — active_control hardcoded to true. The
device-level binary_sensor would always show "running" in HA even
when the user configured ct002 with active_control: false. Now
reads from the new ct002_->active_control() getter (added in the
ct002 sibling commit), matching service.py:427 which uses
data.get("active_control", False).
2. mqtt_insights.cpp:183 — last_seen sent monotonic millis() / 1000
where HA's device_class: timestamp expects Unix epoch seconds.
Result: HA renders the diagnostic timestamp as ~1970 + uptime,
showing "55 years ago" forever. Now reads std::time(nullptr) and
only publishes the value when it's past the 2020-01-01 sanity
threshold (i.e. SNTP / RTC has actually synced). Otherwise
publishes null, which HA shows as "unavailable" — clearly correct
semantics for "we don't have a wall clock yet".
3. mqtt_insights.cpp:206 — discovery published only on first sight,
never re-published when battery_ip arrives later. Python re-runs
discovery whenever its ARP lookup first succeeds for a previously-
unknown consumer (service.py:447-476), so the device.connections
array always picks up the ["ip", ...] entry. C++ now tracks
"discovered with IP" as a second set; an IP that becomes known
after first sight triggers exactly one re-publish (not one per
subsequent event).
All gates green: 711 pytest + 4 host-gcc gtest suites + E2E.
https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…dd-device headers Addresses the bugs the post-merge review surfaced against Python's marstek_api.py: 1. marstek_registration.cpp:158, :274 — `code` field was read strictly as const char *, which returns nullptr when the cloud emits an integer (`"code": 2` vs `"code": "2"`). Python casts via str() so it tolerates both shapes (marstek_api.py:98). On any Marstek backend that uses integer codes the entire token / add-device flow silently failed. New `read_code` helper in the anon namespace inspects the JSON type and stringifies the int variant. A docs-only regression test in host_marstek_responder_test.cpp pins the contract so future edits don't regress. 2. marstek_registration.cpp:268 — add-device call was missing the Marstek-app header set that Python sends: - Content-Type: application/json (declares the JSON-style payload) - token: <session-token> (mirrors what's also in the query string) Some backend versions reject the call when these are absent even though the wire payload is identical. http_get_json_ now takes an optional extra_headers vector and the add-device call wires the pair through. Mirrors marstek_api.py:210-215 exactly. Also fixes a latent bug flagged as a risk in the review: http_get_json_ now captures bytes_read BEFORE calling container->end(), since some http_request backends invalidate the container's counters once end() runs. All gates green: 711 pytest + 4 host-gcc gtest suites (19 cases now including the read_code documentation test) + host platform compile + E2E. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
WalkthroughAdds a full ESPHome CT002 external component with protocol, filters, balancer, MQTT insights, Marstek registration, host gtests, cross-backend E2E harness, CI jobs, YAML examples, docs, and a Python active-control tweak. ChangesCT002 ESPHome port and cross-backend parity
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related PRs
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
|
The CI failure on test.esp32-idf.yaml surfaced a latent codegen bug:
when the user includes a `balancer:` block, the schema fills
efficiency_rotation_interval with its default ("15min" →
TimePeriodSeconds), and the codegen's `float(bal.get(...))` call
crashes with "float() argument must be a string or a real number,
not 'TimePeriodSeconds'". Same bug applies to the `saturation:`
block's grace_seconds and stall_timeout_seconds.
`esphome config` (schema-only validation) didn't catch this because
the crash is in `to_code` which only runs on `esphome compile`. The
host YAML had no balancer block so the .get() fallback to a plain int
worked; the IDF YAML is the first matrix entry to actually exercise
the path.
Fix: introduce a small `_seconds(value)` helper that pulls
`.total_seconds` off a TimePeriod or passes a plain number through.
Apply to all three call sites.
https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…duino build) The registration.esp32-ard CI compile fails (host platform compiles fine, so it's esp32-arduino-specific). The most likely culprit is the ArduinoJson `is<long long>()` template probe in read_code — that specialization isn't stable across the various ArduinoJson revisions ESPHome bundles, and esp32-arduino is the strictest of the lot. Drop `is<long long>()`; probe only with `is<int>()` / `is<long>()` and read as `long`. Marstek codes are single-digit integers (the wire values we care about are "1", "2", "4"), so no overflow risk. The host gtest contract test still validates the int/string tolerance. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…lence -Wstringop-truncation
Best guess for the persistent registration.esp32-ard CI failure:
esp32-arduino's xtensa-gcc enables -Wall -Werror, and the
`std::strncpy(s.mac_hex, mac.c_str(), sizeof(s.mac_hex) - 1)` followed
by manual null-termination is the canonical pattern gcc 8+ flags with
-Wstringop-truncation ("the destination might not be null-terminated
even if you thought it was"). On host gcc the warning is off by default
so the compile passes; on esp32-arduino it's a hard error.
Replace with the explicit memcpy + null-terminate idiom that gcc
analyzes correctly. Same observable behavior — we already cap n at
sizeof(s.mac_hex) - 1 so the terminator always fits.
If this isn't the actual root cause, the next CI run will tell us via
the same webhook and we can iterate.
https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…rduino's Print.h macro The registration.esp32-ard CI failure was Arduino's Print.h defining `#define HEX 16` (used by `Serial.print(x, HEX)`). On esp32-arduino, the preprocessor mangles our local `static const char *HEX = "0..F"` into `static const char *16 = "..."`, which gcc rejects with "expected unqualified-id before numeric constant" — then the two HEX[c >> 4] / HEX[c & 0xF] subscripts fail as "invalid types 'int[int]' for array subscript". Rename to HEX_DIGITS. Same pitfall lurks for any short ALL-CAPS Arduino-flavored name (DEC, BIN, OCT, INPUT, OUTPUT, LOW, HIGH, LED_BUILTIN, ...), so adding a comment so the next contributor doesn't repeat the collision. Host platform build stays green (Arduino's Print.h isn't pulled in there, which is why the host test never caught this). https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…as sub-blocks
The plan called for mqtt_insights and marstek_registration to be
sub-blocks of `ct002:`, not separate top-level external components.
The earlier shipping split them across three component directories
(astrameter_mqtt_insights/, marstek_registration/, ct002/) with three
top-level YAML keys — my own README called them "sub-blocks" while the
schema disagreed.
Restructuring to match the plan:
# Before
external_components:
components: [ct002, astrameter_mqtt_insights, marstek_registration]
ct002: {...}
astrameter_mqtt_insights: { ct002_id: ct002_main, ... }
marstek_registration: { ct002_id: ct002_main, ... }
# After
external_components:
components: [ct002]
ct002:
id: ct002_main
...
mqtt_insights: { ... }
marstek_registration: { ... }
Mechanics:
- Move all .h/.cpp from esphome/components/astrameter_mqtt_insights/
and esphome/components/marstek_registration/ into ct002/. Rename
discovery.{h,cpp} -> ha_discovery.{h,cpp} so it doesn't clash with
the slightly different discovery.py file naming pattern Python uses.
- Drop the two stale __init__.py files; everything lives in ct002's
__init__.py now. Two new sub-schemas (MQTT_INSIGHTS_SCHEMA and
MARSTEK_REGISTRATION_SCHEMA) attach to CONFIG_SCHEMA and the
matching sub-codegen routines (_to_code_mqtt_insights /
_to_code_marstek_registration) run from the parent to_code when
the keys are present.
- C++ namespaces now nest: esphome::ct002::mqtt_insights and
esphome::ct002::marstek_registration.
Conditional compilation:
Putting the new .cpp files inside ct002/ means ESPHome compiles them
unconditionally for every ct002 user, regardless of whether the
sub-blocks are configured. Without guards, ct002-only host builds
would fail because mqtt_client.h doesn't exist on the host platform
and http_request.h isn't on the include path when http_request: is
not in the user YAML. Two guards address this:
* mqtt_insights.cpp body wraps in #ifdef USE_MQTT (set by the mqtt
component when configured). The .h forward-declares
mqtt::MQTTClientComponent so the class signature stays parseable
on builds without USE_MQTT.
* marstek_registration.cpp body wraps in
#ifdef USE_CT002_MARSTEK_REGISTRATION (added via cg.add_define
from _to_code_marstek_registration when the sub-block is present).
Same forward-declaration trick for http_request::HttpRequestComponent
in the header.
Dependency wiring:
- json and md5 move to ct002's AUTO_LOAD so users don't need to add
empty `json:` / `md5:` blocks just to enable the sub-blocks.
- mqtt and http_request stay user-declared in YAML (they carry
user-facing config — broker, timeout, verify_ssl) and the
sub-schemas validate their presence via cv.requires_component.
YAML and docs:
- All three test YAMLs (test.insights.esp32-ard.yaml,
test.registration.esp32-ard.yaml, test.registration.host.yaml)
updated to the nested form.
- README, esphome.example.yaml, CONTRIBUTING.md, CHANGELOG.md
rewritten for the nested schema. The CONTRIBUTING parity rule
now lists the file mappings against the new ct002/ paths.
- test.no-filters.yaml needed no change (no sub-blocks).
Validation: host platform compile passes for test.host.yaml and
test.registration.host.yaml; ESP32 configs (insights, registration,
idf) all validate via `esphome config`; 711 pytest + 4 host-gcc
gtest suites + Tier D BatterySimulator E2E all still green.
https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
The header comment claimed the e2e test "does not yet wire this test in" and misattributed the CI wiring to the ct002-esphome-compile matrix job. A dedicated ct002-host-e2e job (.github/workflows/ci.yml) has run this test since it was added — update the comment to match reality. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…ip only ct002-host-e2e failed because the cold `esphome compile` runs inside the module-scoped host_binary fixture and counts against the test timeout (global timeout=90 in pyproject; the test marker is 30s). The sub-block refactor added ha_discovery.cpp — which pulls in ArduinoJson's heavy template machinery — to the ct002 host build, pushing the cold compile past the timeout in CI (it passes locally only because the binary is already cached). Two fixes: - CI: build the host binary in a dedicated `esphome compile` step before pytest. A compile failure now surfaces with full gcc output in that step instead of as an opaque pytest timeout, and the fixture finds the binary already built so the test runs in ~1s. - Test: mark the timeout func_only=True so the 30s bound covers only the UDP round-trip (the thing we actually want to guard against hanging), leaving the fixture's compile unbounded. This keeps `pytest` working on cold local builds too, without pre-building. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…t types, empty-mac topic A deep field-by-field audit against src/astrameter/mqtt_insights/ found three more divergences beyond the earlier active_control/last_seen/IP fixes: 1. smooth_target was the WRONG VALUE. Python's _last_smooth_target (ct002.py:361-362) is the total INPUT grid power (post-filter, pre-balancer) — `sum(values)` computed before the balancer runs. The C++ published `sum(snap.target)`, i.e. the sum of the balancer's per-phase OUTPUT targets, which is a different quantity. Fixed: ct002 now stashes last_smooth_target_ = grid_total (the input sum) matching Python, exposes it on ConsumerSnapshot, and mqtt_insights publishes snap.smooth_target instead of recomputing from targets. 2. grid_power.* and target.* were published as ints (std::lround) but Python emits floats (ct002.py:721-730 wraps each in float()). The HA value_templates pass the value through unrounded, so lrounding silently truncated fractional watts vs the Python stack. Now emitted as floats. reported_power and smooth_target stay ints (Python uses parse_int / sum-of-parse_int for those); last_target now emitted as the balancer's raw float. 3. parse_app_topic accepted an empty ct_type or mac (`marstek_energy/HME-4/App//ctrl` → mac=""), but Python's regex captures are `[^/]+` and reject it (marstek_mqtt_test.py:149). Defused downstream (the empty topic is never subscribed and the caller compares against a non-empty configured mac) but the pure function diverged. Added the non-empty guard + two host-gcc test cases. Also documented (no behavior change) why ver_v is hardcoded to DEFAULT_VER_V on the ESPHome path: Python derives it from the cloud device list's `version`, but that value isn't plumbed from marstek_registration into the responder, and Python falls back to the same default when `version` is absent — so the common-case wire frame matches. Verified-correct items from the audit (no change needed): all HA discovery component keys / templates / device block / availability; all topic strings; command dispatch validation; the rest of the Marstek responder wire format; discovery dedupe + clear-on-reconnect; the discovered_consumers_with_ip_ substitute for Python's _pending_arp. Known untested gap (not addressed here): there's no host-gcc test for the discovery/state JSON output — adding one needs ArduinoJson in the CMake harness. Bugs #1/#2 above were found by audit, not by a test. 711 pytest + 4 host-gcc gtest suites (now incl. empty-mac topic cases) + host-platform compile + Tier D e2e all green. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…nt, request dedup
A focused parity audit of the emulator core (protocol + main request/
response path + consumer lifecycle) against src/astrameter/ct002/ found
three divergences beyond the prior review's fixes:
1. Rounding mode. Python uses round() (round-half-to-even / banker's) for
every wire value (ct002.py:406,498-501) and for poll_interval
(round(x,1) at :301,303). The C++ used std::lround / std::round
(round-half-away-from-zero), which differ on exact .5 ties
(round(2.5)==2 in Python, lround(2.5)==3). Reachable on the wire when
a filtered sensor value lands on x.5. Added a round_half_even() helper
(explicit, so it doesn't depend on the global FP rounding mode) and
used it in the response-field builder, the cross-phase chrg/dchrg
aggregation, and the poll_interval EMA.
2. parse_int semantics. Python's parse_int = int(), which strips
surrounding whitespace but REJECTS trailing garbage / float syntax
("5.7"→default, "5abc"→default). The C++ used bare strtol, which
accepts a numeric prefix ("5.7"→5, "5abc"→5). Added parse_int_strict()
mirroring int() and used it for reported_power. Only affects malformed
fields, but it's a real semantic difference.
3. Request deduplication was entirely absent. Python keys a
RequestDeduplicator by consumer_id and drops repeat polls inside
dedupe_time_window (ct002.py:636-644), with periodic purge alongside
the consumer-TTL sweep. The C++ processed every datagram. Ported:
new dedupe_window config knob (default 0s = disabled, matching
Python's dedupe_time_window=0.0 default), a dedup_should_process_
gate placed exactly where Python has it (after consumer_id derivation
+ inspection detection, before update_consumer_report_), window
measured from the last ACCEPTED poll (dropped polls don't refresh the
timestamp), and a purge in evict_stale_consumers_ at the consumer-TTL
cadence.
Audit-confirmed already-correct (no change): wire framing/checksum/
length/field-count, the full 24-field response construction + positions
+ info_idx rollover, MAC validate/mirror, consumer-key fallback,
inspection-mode detection, the #376 last_instructed_power cross-talk,
and the prior-review fixes (eviction, setters, poll_interval EMA,
smooth_target = input grid total).
Acceptable divergences left as-is (documented in audit): monotonic
millis() vs wall clock (delta-based, unobservable); 256-byte recv buffer
(oversize datagrams dropped, same as malformed); compute_length overflow
returns empty payload → reply skipped (same outcome, only log differs).
Known untested gap: the response builder + rounding + consumer lifecycle
still have no host-gcc test (golden vectors are all integer-valued, so
the .5 rounding path is unexercised). Flagged for a follow-up that adds
response-builder coverage to the host harness.
711 pytest + host gtests + host compile + Tier D e2e green.
https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…e (real-hardware bugs) Two bugs surfaced by running the firmware on an actual ESP32: 1. HA discovery silently broken. ESPHome's json::build_json hard-caps serialized output at 5120 bytes (json_util.cpp). The CT002 consumer discovery payload is ~6.9 KB (20 components inlined in one device-based config), so it was truncated to invalid JSON — HA never created the per-battery device. The device log shows "[W][json:146]: JSON payload too large, truncated to 5119 bytes". Fix: serialize the discovery payloads with ArduinoJson directly (new serialize_unbounded helper in ha_discovery.cpp) instead of via build_json. ArduinoJson has no such cap and ESP-IDF's MQTT client fragments large publishes, so the full device-based payload goes out intact — keeping exact parity with Python's single-payload structure (no abbreviations or per-entity splitting needed). Measured: consumer payload 6932 bytes, device payload 1453 bytes. 2. Marstek App-topic subscription never happened. The device log shows "[W]: Marstek MQTT enabled but ct_mac is unset — skipping App-topic subscribe" even though marstek_registration had the MAC (02b250b26777, loaded from prefs). Root cause: mqtt_insights captured ct002->ct_mac() at its setup() (setup_priority AFTER_WIFI), but marstek_registration applies the MAC at AFTER_CONNECTION — which runs AFTER. So the capture was always empty, and the documented "reboot once" workaround couldn't fix it (the ordering is deterministic regardless of persisted MAC). Fix: resolve the MAC lazily. New ensure_marstek_subscription_ reads ct002->ct_mac() at connect and every loop while still unsubscribed, subscribes once the MAC is known, and re-subscribes if it changes — mirroring Python's dynamic register_marstek. The marstek_mac_/marstek_ct_type_ members now hold the currently-subscribed identity (cleared on disconnect so we resubscribe on reconnect). No reboot needed; README + apply_mac_ comment updated to drop the stale caveat. The "mqtt took a long time (111 ms)" line in the same log is benign ESPHome core noise (first retained-discovery publish), not addressed. Host compile (exercises the ha_discovery serialize change) + 711 pytest + host gtests + Tier D e2e all green. The mqtt_insights.cpp changes are USE_MQTT-gated so they compile on the esp32 CI matrix targets (can't build esp32 in this sandbox — no toolchain network access). https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
|
Great idea! If this gets to happen I will definitely use it on my ESP32 based read-head, effectively making my own CT003. |
…mer join + phase switch
update_consumer_report_ was missing the INFO logs Python emits at
ct002.py:315-328 when a battery first reports a valid phase ("phase
detected") or moves to a different leg ("phase changed"). Flagged as a
cosmetic gap in the earlier emulator audit; this closes it for full
log parity.
Mirrors Python precisely:
- "prior phase known?" keys off timestamp>0 (not the phase string, which
defaults to "A" on a fresh Consumer), so a never-seen battery logs
"phase detected: X" rather than a spurious "changed".
- Only fires for a real A/B/C phase that differs from the prior one;
inspection-mode polls (phase forced to "A") don't trigger noise.
- Message strings are byte-identical:
"CT002 consumer %s phase detected: %s"
"CT002 consumer %s phase changed: %s -> %s"
Verified on the host binary: a phase-B poll from a new consumer logs
"phase detected: B", a repeat is silent, and a subsequent phase-C poll
logs "phase changed: B -> C".
Host compile + host gtests + Tier D e2e green.
https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…ed grid sensor The example repeated the full ct002: block three times (once active, twice in commented form to show the mqtt_insights and marstek_registration sub-blocks). Rewritten as one coherent config: - ct002: appears exactly once. The two optional sub-blocks (mqtt_insights, marstek_registration) are commented out INLINE within it, each with a note to uncomment alongside the top-level component it needs. - The optional top-level deps (mqtt:, http_request:) are commented out near the top with matching when-to-uncomment guidance, instead of being duplicated inside the repeated ct002 blocks. - The example grid sensor now uses `platform: homeassistant` (subscribes to an existing HA power entity over the native API) instead of a do-nothing template sensor — closer to a real first setup. Noted that api: is required for it, and that other platforms (modbus/mqtt/template) work too. - filters/balancer/saturation stay active as the feature showcase (they need no extra top-level component), with comments marking them optional. Validated with `esphome config` against the local component (homeassistant sensor + ct002 with all config sub-blocks parse clean). https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
|
@baal86 Feel free to test, it should mostly work. You can find an example config here: https://github.com/tomquist/AstraMeter/blob/claude/dazzling-cannon-vxuBk/esphome.example.yaml Make sure to replace this line: with this line: |
…ign inversion) A user reported the emulator sending the negated value to their battery. Traced it: not a parity bug (the C++ PID, balancer, response builder, and cross-talk all match Python exactly) but a config conflict that the example caused. esphome.example.yaml and the README minimal config both showed an ACTIVE PID filter (kp=0.5, ki=0.1, bias) next to active_control: true + balancer. The PID and the balancer are competing controllers (plus the battery's own loop). The PID error is -grid, so on a sustained import its integral winds up negative until the output saturates at -output_max; in bias mode the result is grid + pid_output, e.g. +300 + (-800) = -500 — the sign flips and the battery charges when it should discharge. Python never exhibits this by default because PID_KP defaults to 0 (PID off); the PID is intended for passive setups (active_control: false) where it works with the battery's internal loop, NOT alongside the ct002 balancer. The example wrongly bundled them and users copied it. Fix: comment out the `pid:` block in the example with an explanation that it's an alternative to active_control (use one or the other), and add the same warning to the README's minimal config. hampel/smoothing/deadband (signal conditioning) stay active — those are fine with the balancer. No code change — the implementations are correct and at parity. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
Adds a "Python ↔ ESPHome parity (REQUIRED)" section to AGENTS.md (also surfaced via the CLAUDE.md symlink) stating that any change affecting shared behavior must land on BOTH the Python canonical source and its C++ mirror under esphome/components/ct002/ in the same change. Includes the file-mapping table, the list of Python pieces that intentionally have no C++ counterpart (transform/throttle wrappers, Shelly path, ARP, asyncio reconnect), and the host-gcc gtest + esphome compile verification commands. Also notes in "Adding a powermeter" that powermeters are Python-only and exempt from the parity rule (the ESPHome component reads from a native ESPHome sensor instead). https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
The earlier parity section was too verbose for AGENTS.md. Reduce it to the rule + a pointer to CONTRIBUTING.md (which already holds the full file-mapping table and the no-counterpart list) + the one verify command. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…xample.yaml The README ESPHome section inlined three sizable YAML blocks (the full ct002 config plus the mqtt_insights and marstek_registration sub-blocks). Replace them with a truly minimal example (external_components + one sensor + ct002 with just id + power_sensor_l1) and a link to esphome.example.yaml for the complete annotated config. The two optional sub-blocks are now described in one bullet each (what they do + which top-level component they need) rather than duplicated as YAML — the example file is the single source of truth for actual config values. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
|
Compiles fine. There seems to be a name resolution issue with the Marstel registration: |
|
@baal86 This doesn't seem related to the ct002 component itself. What does your |
Resolves the CHANGELOG.md conflict (kept both ## Next bullets: the ESPHome ct002 component and develop's Modbus UDP TRANSPORT option from #387). develop had advanced past the PR's merge base, leaving the PR un-mergeable — which stopped GitHub from building the refs/pull/385/merge ref and silently halted the pull_request CI. Merging develop back in restores mergeability so CI runs again. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
|
|
If you add a manual ip section you should also provide a |
Very good point! Still not. Also tried just putting in the IP of eu.hamedata.com directly. Same effect. Sounds like a certificate problem? |
|
Oh, you should set dns1 to the IP of your gateway |
Had to clean build files. Now it registers! Thanks for the help. I'll continue testing. |
|
OK. Does work in real life! Great work! In my particular case it is not useable right now since I have literally no delay in my samples. Getting them straight from OBIS and the control loop is extra spicy. Will need some tuning on my side to get it stable. |
…ock hooks Wires the affordances a shared Python↔ESPHome e2e base needs into the existing host e2e test. The black-box binary previously could only be driven by what a battery polls; the Python e2e harness additionally controls grid power (via a before_send hook) and time (via a FakeClock). This adds the equivalent two controls to the binary, behind a build flag. Component (all gated by USE_CT002_TEST_HOOKS, added only when the YAML sets the test-only `test_control_port:` — never in production firmware): - new test_hooks.cpp: a second UDP control socket accepting text commands `grid <l1> [l2] [l3]` (inject grid power straight into the sensor cache, stamped now so the freshness check passes), `clock_set` / `clock_advance` / `clock_real` (engage a mock clock). Each command acks so the harness can synchronise. - now_seconds_() becomes an instance method that returns the mock clock when engaged; the balancer/saturation clock lambda now routes through it, so dedup / saturation / probe / eviction all honour mock time. Production path unchanged (millis()). - __init__.py: `test_control_port:` option → cg.add_define(USE_CT002_TEST_HOOKS) + set_control_port. tests/.../test.e2e.host.yaml enables it (with never-publishing template sensors so the injected grid isn't clobbered, and dedupe_window: 10s for the clock-gated test). Tests (test_host_e2e.py) — a _Ct002Control helper (control channel + raw UDP poll returning parsed fields or None on timeout) plus three scenarios that all pass against the real binary: - grid sign: +300 import → battery told to discharge (+); -300 export → charge (-). - convergence: once the battery reports it took the first target, grid is back to ~0 and the next target shrinks toward zero. - clock-gated dedup: a repeat poll inside the 10s window is dropped (no response); advancing the mock clock past it un-gates the poll — proving both the dedup feature and clock control are wired. CI: test.e2e.host.yaml added to the esphome-compile matrix, and the ct002-host-e2e job pre-builds it before pytest (so a compile failure surfaces with gcc output, not an opaque fixture timeout). The deeper scenario migration (running the Python efficiency/probe/ saturation suites against both backends) is still future work; this lands the binary affordances + harness those will build on. Production (non-hooks) build still compiles; 717 pytest + 4 host-gcc gtest suites + all host e2e green. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
|
The example config has all the possible parameters to tweak your values. I would start with |
…PHome backends Builds the shared e2e abstraction on top of the test-hooks control channel: test_shared_e2e.py defines a uniform poll / set_grid / set_clock / advance_clock interface with two implementations, and runs each scenario against both via a parametrized `backend` fixture. - PythonBackend drives the canonical astrameter.ct002.CT002 in-process: _handle_request is called with a fake transport that captures the reply, grid power comes from the before_send hook, and time is a controllable fake clock. No ESPHome toolchain needed, so these run everywhere. - EsphomeBackend drives the compiled host binary (test.e2e.host.yaml) over real UDP, with grid + mock clock via the test-hooks control channel. Skips when the esphome CLI is absent. Three scenarios now assert identical wire-observable behaviour on both stacks: grid-sign (import -> discharge, export -> charge), convergence (target shrinks toward zero as the battery covers the load), and clock-gated dedup (repeat poll dropped inside the window, un-gated after advancing the clock). This is the cross-stack parity guard — a divergence in the C++ port now fails the esphome parametrization while the python one passes. The three controllable scenarios that previously lived in test_host_e2e.py (esphome-only) moved here and now cover both backends; test_host_e2e.py keeps just the BatterySimulator round-trip. CI's ct002-host-e2e job runs both modules. Foundation laid for migrating the heavier Python suites (efficiency / probe / saturation): write new scenarios against the `backend` fixture. The remaining gap for those is an observability channel for assertions that currently read Python internals — most are already exposed via the wire response or mqtt_insights. 720 pytest pass (6 shared = 3 scenarios x 2 backends), mypy clean, host gtests + round-trip e2e green. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
Extends the test-hooks control channel with the three commands the existing Python e2e suites need to run against the binary: - force_rotation — calls force_balancer_rotation() (mirrors ct002.force_efficiency_rotation(), used by the rotation tests). - dedupe <ms> — sets the dedup window at runtime. The closed-loop sims poll many times per second of mock time and must NOT be deduplicated, so test.e2e.host.yaml now defaults dedupe_window to 0 and the dedup scenario turns it on at runtime. PythonBackend mirrors this via ct002._dedup._window. - dump — serializes the internal state the suites read directly (smooth_target + per-consumer phase / last_instructed_power / last_target / saturation / active / manual / reported) as a pipe-delimited line, so issue376's last_instructed_power and the probe-lockup _last_smooth_target assertions become observable on the black-box binary. All gated behind USE_CT002_TEST_HOOKS (test-only). The shared dedup scenario now enables dedup at runtime since the YAML default is 0; 11 component tests pass (round-trip + 6 shared + 4 gtest suites). Next: the backend-parametrized SimHarness migration of the efficiency / issue376 / probe-convergence suites onto these hooks. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
…ackends Adds a `cfg <key> <value>` test-hook control command that sets any balancer/saturation config field at runtime and rebuilds the LoadBalancer (build_balancer_ extracted from setup()). This is the missing piece for cross-backend e2e: the efficiency tests reconfigure the balancer per test (min_efficient_power, efficiency_rotation_interval, efficiency_fade_alpha, …), which on the binary is otherwise compile-time. All gated behind USE_CT002_TEST_HOOKS; production build unchanged. test_efficiency_e2e.py's _SimHarness is now backend-parametrized via an autouse fixture that flips _ACTIVE_BACKEND, so all 17 tests run once per backend (34 total) without touching the bodies: - python: in-process CT002 reading grid via before_send, FakeClock (unchanged behaviour). - esphome: spawns the test-hooks host binary; each step replicates BatterySimulator.step() and injects the grid AFTER the battery ramps but BEFORE it polls (matching Python's live before_send timing), drives the mock clock in lockstep via _HarnessClock.on_change, sets dedup off and pushes the scenario's balancer cfg at start. The closed-loop convergence/rotation/saturation assertions (battery powers, active count, grid total) are client-side observables that hold on both stacks. e2e YAML gets consumer_ttl: 100000s so long mock-time sims never evict a polling consumer. 34 passed (17 x 2). Production + e2e host builds compile. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
Extract the cross-backend e2e plumbing (HarnessClock, EsphomeSim, port helpers, backend switch) into a shared tests/_ct002_e2e_backend.py module and re-wire the efficiency suite onto it. Migrate the issue #376 PV- passthrough regression and the probe-handoff convergence test to run on both the in-process Python CT002 and the compiled ESPHome host binary; the before_send stale-meter and harness-lifecycle tests stay Python-only with inline rationale. CI's ct002-host-e2e job now runs the migrated suites so their esphome params are exercised where the CLI + test-hooks binary exist. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
Tag the five suites that drive the compiled ESPHome host binary with @pytest.mark.esphome_e2e (registered in pyproject) and have the ct002-host-e2e CI job run `pytest -m esphome_e2e` instead of enumerating files. New cross-backend suites are now picked up automatically. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
Replace the hardcoded test.*.yaml matrix with a discovery job that globs tests/components/ct002/test.*.yaml into the compile matrix, so adding a new compile target no longer requires editing the workflow. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
There was a problem hiding this comment.
Actionable comments posted: 10
🧹 Nitpick comments (2)
tests/components/ct002/_gen_protocol_test_vectors.py (1)
48-50: ⚡ Quick winAdd
<utility>to the generated header includes.The generated header uses
std::movebut does not explicitly include<utility>. Please avoid relying on transitive includes from other headers.Suggested fix
"`#include` <cstdint>", "`#include` <string>", + "`#include` <utility>", "`#include` <vector>",Also applies to: 80-80
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/components/ct002/_gen_protocol_test_vectors.py` around lines 48 - 50, The generated header uses std::move but the includes list that builds the header (the array of include strings in _gen_protocol_test_vectors.py) does not contain "<utility>"; add "`#include` <utility>" into that includes list (and the same includes set used at the other occurrence around the 80-range) so the generated C++ header explicitly includes <utility> instead of relying on transitive includes; ensure the include string matches the existing format ("`#include` <utility>") and is placed with the other standard headers.tests/components/ct002/host_marstek_responder_test.cpp (1)
181-204: 💤 Low valueAcknowledge the maintenance burden of the proxy parser.
The hand-rolled
proxy_read_code_strduplicates logic frommarstek_registration.cpp's anonymous namespace without compile-time enforcement. The comment correctly notes that "audit will catch it" if the real implementation diverges, but this approach requires manual synchronization.This is acceptable given the namespace constraints, but consider adding a build-time integration test that exercises the actual
marstek_registration.cpp::read_codefunction through a public wrapper if feasible in the future.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/components/ct002/host_marstek_responder_test.cpp` around lines 181 - 204, The test duplicates parsing logic in proxy_read_code_str which drifts from marstek_registration.cpp::read_code; replace the hand-rolled parser by invoking the real implementation (or a small public test-only wrapper around the anonymous read_code) so the test exercises the actual parsing logic, or alternatively add a build-time integration test that links against marstek_registration's read_code via a test-only public wrapper; locate proxy_read_code_str in the test and change it to call the wrapper/read_code, or add the wrapper in marstek_registration.cpp and use it from the test to eliminate duplication.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci.yml:
- Around line 87-88: The workflow passes the generated matrix.yaml unquoted into
a shell command (used by the step named "Compile ${{ matrix.yaml }}") which
allows filenames with metacharacters to break or inject commands; change the
compile invocation so the interpolated value is treated as a single literal
argument by adding a conventional option terminator and quoting the parameter
(i.e., ensure the esphome compile invocation uses -- and a quoted ${ matrix.yaml
}), and update the "Compile ${{ matrix.yaml }}" run line to use that safely
quoted/terminated form so filenames cannot influence shell parsing.
- Around line 50-61: Pin third-party actions to immutable commit SHAs and
prevent credential persistence: replace tag refs for actions/checkout@v4,
astral-sh/setup-uv@v5, actions/setup-python@v5, and actions/upload-artifact@v4
with their corresponding commit SHAs, and add with: persist-credentials: false
to every actions/checkout step to avoid leaving the checkout token in the runner
git config; also harden the esphome compile run command (esphome compile ${{
matrix.yaml }}) by quoting the matrix variable (e.g., " ${{ matrix.yaml }} " or
single-quoted) to ensure proper argument parsing.
In `@CHANGELOG.md`:
- Around line 5-8: The CHANGELOG.md "## Next" section contains multiple bullets
but must be reduced to a single continuously-edited branch-summary bullet; edit
the "## Next" block to remove the extra bullets and replace them with one
concise summary sentence that captures the overall outcome of this branch (e.g.,
key additions/fixes like ct002, Modbus UDP, HA powermeter fix, multi-Venus fix),
leaving that single bullet as the continuing summary for future edits.
In `@esphome/components/ct002/marstek_responder.cpp`:
- Around line 109-112: The current parsing of numeric fields accepts strings
with trailing junk because it only checks that strtol consumed at least one
digit; update the parsing logic for cd (where end and cd_val are used) and the
analogous p1 parsing (where p1_val/end are used) to also verify that *end ==
'\0' (i.e., the entire string was consumed) and return std::nullopt if not;
locate the strtol calls and their end-pointer checks in marstek_responder.cpp
(the cd parsing block around PollContext ctx and the p1 parsing block) and add
the trailing junk rejection so partially numeric inputs like "4x" or "2foo" are
treated as invalid.
In `@esphome/components/ct002/mqtt_insights.cpp`:
- Around line 89-96: The subscription gate currently checks marstek_mac_.empty()
so ensure_marstek_subscription_ is skipped once a MAC exists, which leaves
subscriptions stale if ct_mac changes; update the logic in the connected branch
(where connected && this->marstek_mqtt_enabled_ is evaluated) to always call
ensure_marstek_subscription_ while connected and enabled (remove the
marstek_mac_.empty() condition) and ensure ensure_marstek_subscription_ itself
is idempotent/safe to call repeatedly (or add a more robust check inside
ensure_marstek_subscription_ against the current marstek_mac_ to re-subscribe
only when it changes).
In `@esphome/components/ct002/sensor_backed.cpp`:
- Around line 28-35: The current freshness check uses max_stamp and so a fresh
phase can hide a stale one; change the logic to iterate each phase (using
this->num_phases_ and (*this->raw_stamp_ms_)[i]) and for each phase treat a zero
stamp or a stamp where (now - stamp) > this->max_sensor_age_ms_ as stale and
return {} (unavailable). Replace the max_stamp calculation and the single
comparison with a per-phase loop that validates each timestamp against now and
this->max_sensor_age_ms_ so any stale or missing phase causes the function to
return unavailable.
In `@esphome/components/ct002/test_hooks.cpp`:
- Around line 62-70: The pump_control_() loop dereferences control_socket_
without checking for null (start_control_server_ may leave it null); add an
early guard in CT002Component::pump_control_() to return immediately if
control_socket_ is null (or not open) before calling control_socket_->recvfrom,
so the method safely exits when the control socket wasn't created or failed to
bind.
In `@tests/_ct002_e2e_backend.py`:
- Around line 58-64: ensure_e2e_binary currently only compiles when E2E_BINARY
is missing, allowing stale artifacts to be used; change it to rebuild when
sources have changed by comparing modification times: if E2E_BINARY does not
exist OR E2E_BINARY.stat().st_mtime is older than E2E_YAML.stat().st_mtime (and
ideally older than any files under the e2e source dir like _E2E_DIR or its
children), invoke subprocess.run(["esphome", "compile", str(E2E_YAML)],
check=True, cwd=_E2E_DIR.parent.parent) to recompile; use pathlib to walk source
files and take the newest mtime to compare against E2E_BINARY to decide whether
to rebuild.
In `@tests/components/ct002/test_host_e2e.py`:
- Around line 95-116: Guard POSIX-only process-group handling around the
subprocess creation and teardown: when spawning the host binary, only pass
preexec_fn=os.setsid on POSIX (e.g. if os.name == "posix"), and on Windows use
subprocess.CREATE_NEW_PROCESS_GROUP via creationflags instead; in teardown,
protect os.killpg/os.getpgid with the same POSIX check and ensure the child is
still running (proc.poll() is None) before attempting to kill it to avoid
exceptions if it already exited. Update references in this fixture to the POSIX
symbols preexec_fn=os.setsid, os.killpg, and os.getpgid and add the non-POSIX
alternative using subprocess.CREATE_NEW_PROCESS_GROUP and safe existence checks
around kills.
In `@tests/components/ct002/test_host_protocol.py`:
- Around line 25-36: _pytestmark_ currently skips based on cmake/C++ but
_regen_vectors() always invokes "uv run python ..." which fails if uv is not
installed; update the _regen_vectors function to prefer using "uv" when
available but fall back to the system Python: detect uv with shutil.which("uv")
and if present call subprocess.run(["uv","run","python", str(HERE /
"_gen_protocol_test_vectors.py")], check=True, cwd=REPO_ROOT) otherwise call
subprocess.run([sys.executable, str(HERE / "_gen_protocol_test_vectors.py")],
check=True, cwd=REPO_ROOT); add imports for shutil and sys if not already
present.
---
Nitpick comments:
In `@tests/components/ct002/_gen_protocol_test_vectors.py`:
- Around line 48-50: The generated header uses std::move but the includes list
that builds the header (the array of include strings in
_gen_protocol_test_vectors.py) does not contain "<utility>"; add "`#include`
<utility>" into that includes list (and the same includes set used at the other
occurrence around the 80-range) so the generated C++ header explicitly includes
<utility> instead of relying on transitive includes; ensure the include string
matches the existing format ("`#include` <utility>") and is placed with the other
standard headers.
In `@tests/components/ct002/host_marstek_responder_test.cpp`:
- Around line 181-204: The test duplicates parsing logic in proxy_read_code_str
which drifts from marstek_registration.cpp::read_code; replace the hand-rolled
parser by invoking the real implementation (or a small public test-only wrapper
around the anonymous read_code) so the test exercises the actual parsing logic,
or alternatively add a build-time integration test that links against
marstek_registration's read_code via a test-only public wrapper; locate
proxy_read_code_str in the test and change it to call the wrapper/read_code, or
add the wrapper in marstek_registration.cpp and use it from the test to
eliminate duplication.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 617ba5de-dabd-4843-a4c6-667b1368ff50
📒 Files selected for processing (62)
.github/workflows/ci.ymlAGENTS.mdCHANGELOG.mdCONTRIBUTING.mdREADME.mdesphome.example.yamlesphome/components/ct002/__init__.pyesphome/components/ct002/balancer.cppesphome/components/ct002/balancer.hesphome/components/ct002/ct002.cppesphome/components/ct002/ct002.hesphome/components/ct002/ha_discovery.cppesphome/components/ct002/ha_discovery.hesphome/components/ct002/hampel.cppesphome/components/ct002/hampel.hesphome/components/ct002/marstek_registration.cppesphome/components/ct002/marstek_registration.hesphome/components/ct002/marstek_responder.cppesphome/components/ct002/marstek_responder.hesphome/components/ct002/mqtt_insights.cppesphome/components/ct002/mqtt_insights.hesphome/components/ct002/pid.cppesphome/components/ct002/pid.hesphome/components/ct002/protocol.cppesphome/components/ct002/protocol.hesphome/components/ct002/sensor_backed.cppesphome/components/ct002/sensor_backed.hesphome/components/ct002/smoothing.cppesphome/components/ct002/smoothing.hesphome/components/ct002/test_hooks.cppesphome/components/ct002/wrapper_base.hpyproject.tomlsrc/astrameter/ct002/ct002.pytests/_ct002_e2e_backend.pytests/components/ct002/.gitignoretests/components/ct002/CMakeLists.txttests/components/ct002/_gen_protocol_test_vectors.pytests/components/ct002/fixtures/_populate_wire_hex.pytests/components/ct002/fixtures/protocol_golden_vectors.jsontests/components/ct002/host_balancer_test.cpptests/components/ct002/host_marstek_responder_test.cpptests/components/ct002/host_protocol_test.cpptests/components/ct002/host_protocol_test_vectors.htests/components/ct002/host_wrappers_test.cpptests/components/ct002/test.1phase.yamltests/components/ct002/test.e2e.host.yamltests/components/ct002/test.esp32-ard.yamltests/components/ct002/test.esp32-idf.yamltests/components/ct002/test.host.yamltests/components/ct002/test.insights.esp32-ard.yamltests/components/ct002/test.no-filters.yamltests/components/ct002/test.registration.esp32-ard.yamltests/components/ct002/test.registration.host.yamltests/components/ct002/test_codegen.pytests/components/ct002/test_host_e2e.pytests/components/ct002/test_host_protocol.pytests/components/ct002/test_shared_e2e.pytests/test_ct002_active_control.pytests/test_ct002_protocol_parity.pytests/test_e2e_probe_lockup.pytests/test_efficiency_e2e.pytests/test_issue376_pv_passthrough.py
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@v5 | ||
| with: | ||
| enable-cache: true | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: "3.13" |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify unpinned action refs and missing persist-credentials in this workflow
rg -n 'uses:\s*[^@]+@v[0-9]+' .github/workflows/ci.yml
rg -n 'uses:\s*actions/checkout@' .github/workflows/ci.yml -A3Repository: tomquist/AstraMeter
Length of output: 1416
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/ci.yml"
echo "== persist-credentials occurrences =="
rg -n "persist-credentials\s*:" "$FILE" || true
echo
echo "== checkout blocks with context (first few) =="
rg -n "uses:\s*actions/checkout@v4" "$FILE" -n -C 6 || true
echo
echo "== checkout blocks around the originally cited ranges =="
sed -n '40,140p' "$FILE" | nl -ba | sed -n '1,120p'Repository: tomquist/AstraMeter
Length of output: 2347
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/ci.yml"
echo "== persist-credentials occurrences =="
rg -n "persist-credentials\s*:" "$FILE" || true
echo
echo "== esphome compile occurrences =="
rg -n "esphome\s+compile" "$FILE" || true
echo
echo "== matrix.yaml usage occurrences =="
rg -n "\bmatrix\.yaml\b" "$FILE" || true
echo
echo "== checkout steps (with nearby context) =="
rg -n "uses:\s*actions/checkout@v4" "$FILE" -C 4 || true
echo
echo "== ranges around where new jobs likely compile =="
sed -n '1,230p' "$FILE" | nl -ba | sed -n '40,220p'Repository: tomquist/AstraMeter
Length of output: 2101
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/ci.yml"
echo "== matrix generation + compile step =="
awk 'NR>=70 && NR<=160 {printf "%d:%s\n", NR, $0}' "$FILE" | sed -n '1,120p'
echo
echo "== all checkout@v4 blocks (full with nearby) =="
awk '
$0 ~ /uses:\s*actions\/checkout@v4/ { start=NR-2; if(start<1) start=1 }
{ if(start && NR>=start && NR<=NR+10) print }
' "$FILE"Repository: tomquist/AstraMeter
Length of output: 10937
🌐 Web query:
actions/checkout v4 persist-credentials default value
💡 Result:
The default value for the persist-credentials input in actions/checkout@v4 is true [1][2]. When set to true (the default), the action automatically configures the local git repository with the authentication token or SSH key used to perform the checkout [1][2]. This allows subsequent steps in the workflow to execute authenticated git commands directly [1][2]. For security-sensitive environments, it is often recommended to set persist-credentials: false to prevent the credentials from being stored in the local git configuration, which could otherwise be accessible to other steps or accidentally included in artifacts [3][4]. Example usage to disable this behavior: - uses: actions/checkout@v4 with: persist-credentials: false
Citations:
- 1: https://github.com/actions/checkout/tree/v4
- 2: https://github.com/actions/checkout
- 3: Remove
persist-credentialsor change the default tofalseactions/checkout#485 - 4: https://yossarian.net/til/post/actions-checkout-can-leak-github-credentials/
🏁 Script executed:
#!/bin/bash
# Verify unpinned action refs and missing persist-credentials in this workflow
FILE=".github/workflows/ci.yml"
rg -n 'uses:\s*[^@]+@v[0-9]+' "$FILE"
rg -n 'uses:\s*actions/checkout@' "$FILE" -A3
rg -n 'persist-credentials\s*:' "$FILE" || trueRepository: tomquist/AstraMeter
Length of output: 1416
🏁 Script executed:
#!/bin/bash
# Verify unpinned action refs and missing persist-credentials in this workflow
rg -n 'uses:\s*[^@]+@v[0-9]+' .github/workflows/ci.yml
rg -n 'uses:\s*actions/checkout@' .github/workflows/ci.yml -A3Repository: tomquist/AstraMeter
Length of output: 1416
🏁 Script executed:
#!/bin/bash
FILE=".github/workflows/ci.yml"
rg -n 'uses:\s*[^@]+@v[0-9]+' "$FILE"
rg -n 'uses:\s*actions/checkout@' "$FILE" -C 3
rg -n 'persist-credentials\s*:' "$FILE" || true
rg -n 'run:\s*esphome\s+compile\s+\$\{\{\s*matrix\.yaml\s*\}\}' "$FILE"Repository: tomquist/AstraMeter
Length of output: 2056
Pin third-party actions to commit SHAs and disable checkout credential persistence in CI.
.github/workflows/ci.yml uses tag-only action refs (actions/checkout@v4, astral-sh/setup-uv@v5, actions/setup-python@v5, actions/upload-artifact@v4) and every actions/checkout@v4 step omits with: persist-credentials: false (default is true, so the checkout token is persisted in the runner git config). Also harden ct002-esphome-compile: run: esphome compile ${{ matrix.yaml }} should quote the value.
🔧 Suggested hardening pattern
- - name: Checkout repository
- uses: actions/checkout@v4
+ - name: Checkout repository
+ uses: actions/checkout@<full_commit_sha>
+ with:
+ persist-credentials: false
- - name: Install uv
- uses: astral-sh/setup-uv@v5
+ - name: Install uv
+ uses: astral-sh/setup-uv@<full_commit_sha>
with:
enable-cache: true
- - name: Set up Python
- uses: actions/setup-python@v5
+ - name: Set up Python
+ uses: actions/setup-python@<full_commit_sha>
with:
python-version: "3.13"🧰 Tools
🪛 zizmor (1.25.2)
[warning] 50-51: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[error] 51-51: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 54-54: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 59-59: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 54-54: runtime artifacts potentially vulnerable to a cache poisoning attack (cache-poisoning): this step
(cache-poisoning)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci.yml around lines 50 - 61, Pin third-party actions to
immutable commit SHAs and prevent credential persistence: replace tag refs for
actions/checkout@v4, astral-sh/setup-uv@v5, actions/setup-python@v5, and
actions/upload-artifact@v4 with their corresponding commit SHAs, and add with:
persist-credentials: false to every actions/checkout step to avoid leaving the
checkout token in the runner git config; also harden the esphome compile run
command (esphome compile ${{ matrix.yaml }}) by quoting the matrix variable
(e.g., " ${{ matrix.yaml }} " or single-quoted) to ensure proper argument
parsing.
| matrix=$(ls tests/components/ct002/test.*.yaml | jq -R -s -c 'split("\n")[:-1]') | ||
| echo "matrix=$matrix" >> "$GITHUB_OUTPUT" |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Confirm direct matrix interpolation into shell execution points
rg -n 'esphome compile \${{\s*matrix\.yaml\s*}}' .github/workflows/ci.ymlRepository: tomquist/AstraMeter
Length of output: 205
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show the referenced lines for exact context
nl -ba .github/workflows/ci.yml | sed -n '70,130p'
# Find how matrix.yaml is used in esphome compile invocations
rg -n "matrix\.yaml" .github/workflows/ci.yml
rg -n "esphome compile" .github/workflows/ci.yml
rg -n "compile \\$\\{\\{\\s*matrix\\.yaml\\s*\\}\\}" .github/workflows/ci.yml || trueRepository: tomquist/AstraMeter
Length of output: 106
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Print the referenced region with line numbers via awk
awk 'NR>=70 && NR<=130 {printf "%d:%s\n", NR, $0}' .github/workflows/ci.yml
# Find all occurrences of matrix.yaml and esphome compile usage
rg -n "matrix\.yaml" .github/workflows/ci.yml
rg -n "esphome compile" .github/workflows/ci.yml
rg -n "esphome compile.*\\$\\{\\{\\s*matrix\\.yaml\\s*\\}\\}" .github/workflows/ci.yml || trueRepository: tomquist/AstraMeter
Length of output: 2290
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n "\\$\\{\\{\\s*matrix\\." .github/workflows/ci.yml || true
rg -n "esphome compile" .github/workflows/ci.ymlRepository: tomquist/AstraMeter
Length of output: 792
Quote and pass matrix.yaml safely to esphome compile to prevent filename-based command injection (CI).
matrix.yaml is built from repo filenames (lines 87-88) and then used unquoted in run: esphome compile ${{ matrix.yaml }} (lines 109-110); filenames containing shell metacharacters (e.g., ;, $(), backticks, spaces) can alter command parsing/execution.
🔒 Safer compile invocation
- - name: Compile ${{ matrix.yaml }}
- run: esphome compile ${{ matrix.yaml }}
+ - name: Compile ${{ matrix.yaml }}
+ run: |
+ set -euo pipefail
+ esphome compile -- "${{ matrix.yaml }}"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci.yml around lines 87 - 88, The workflow passes the
generated matrix.yaml unquoted into a shell command (used by the step named
"Compile ${{ matrix.yaml }}") which allows filenames with metacharacters to
break or inject commands; change the compile invocation so the interpolated
value is treated as a single literal argument by adding a conventional option
terminator and quoting the parameter (i.e., ensure the esphome compile
invocation uses -- and a quoted ${ matrix.yaml }), and update the "Compile ${{
matrix.yaml }}" run line to use that safely quoted/terminated form so filenames
cannot influence shell parsing.
| char *end = nullptr; | ||
| long cd_val = std::strtol(cd->c_str(), &end, 10); | ||
| if (end == cd->c_str()) return std::nullopt; | ||
| PollContext ctx; |
There was a problem hiding this comment.
Reject partially numeric cd/p1 values.
Line 110 and Line 121 accept values with trailing junk ("4x", "2foo"). That makes malformed control payloads look valid.
Suggested fix
+#include <cerrno>
+#include <climits>
@@
const std::string *cd = find(kv, "cd");
if (cd == nullptr) return std::nullopt;
+ errno = 0;
char *end = nullptr;
long cd_val = std::strtol(cd->c_str(), &end, 10);
- if (end == cd->c_str()) return std::nullopt;
+ if (end == cd->c_str() || *end != '\0' || errno == ERANGE || cd_val < INT_MIN || cd_val > INT_MAX)
+ return std::nullopt;
@@
const std::string *p1 = find(kv, "p1");
if (p1 == nullptr) return std::nullopt;
+ errno = 0;
char *e2 = nullptr;
long sid = std::strtol(p1->c_str(), &e2, 10);
- if (e2 == p1->c_str()) return std::nullopt;
+ if (e2 == p1->c_str() || *e2 != '\0' || errno == ERANGE || sid < INT_MIN || sid > INT_MAX)
+ return std::nullopt;Also applies to: 121-123
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@esphome/components/ct002/marstek_responder.cpp` around lines 109 - 112, The
current parsing of numeric fields accepts strings with trailing junk because it
only checks that strtol consumed at least one digit; update the parsing logic
for cd (where end and cd_val are used) and the analogous p1 parsing (where
p1_val/end are used) to also verify that *end == '\0' (i.e., the entire string
was consumed) and return std::nullopt if not; locate the strtol calls and their
end-pointer checks in marstek_responder.cpp (the cd parsing block around
PollContext ctx and the p1 parsing block) and add the trailing junk rejection so
partially numeric inputs like "4x" or "2foo" are treated as invalid.
| // Lazily subscribe to Marstek App topics once the MAC is known. Only | ||
| // poll while not yet subscribed (marstek_mac_ empty) so there's no | ||
| // steady-state per-loop cost; the MAC is applied once at boot by | ||
| // marstek_registration and cleared here only on disconnect, so a | ||
| // re-subscribe is driven by the reconnect path, not by polling. | ||
| if (connected && this->marstek_mqtt_enabled_ && this->marstek_mac_.empty()) { | ||
| this->ensure_marstek_subscription_(); | ||
| } |
There was a problem hiding this comment.
Always re-check Marstek identity while connected.
Line 94 gates ensure_marstek_subscription_() on marstek_mac_.empty(). If ct_mac changes later, subscriptions can stay on stale topics.
Suggested fix
- if (connected && this->marstek_mqtt_enabled_ && this->marstek_mac_.empty()) {
+ if (connected && this->marstek_mqtt_enabled_) {
this->ensure_marstek_subscription_();
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // Lazily subscribe to Marstek App topics once the MAC is known. Only | |
| // poll while not yet subscribed (marstek_mac_ empty) so there's no | |
| // steady-state per-loop cost; the MAC is applied once at boot by | |
| // marstek_registration and cleared here only on disconnect, so a | |
| // re-subscribe is driven by the reconnect path, not by polling. | |
| if (connected && this->marstek_mqtt_enabled_ && this->marstek_mac_.empty()) { | |
| this->ensure_marstek_subscription_(); | |
| } | |
| // Lazily subscribe to Marstek App topics once the MAC is known. Only | |
| // poll while not yet subscribed (marstek_mac_ empty) so there's no | |
| // steady-state per-loop cost; the MAC is applied once at boot by | |
| // marstek_registration and cleared here only on disconnect, so a | |
| // re-subscribe is driven by the reconnect path, not by polling. | |
| if (connected && this->marstek_mqtt_enabled_) { | |
| this->ensure_marstek_subscription_(); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@esphome/components/ct002/mqtt_insights.cpp` around lines 89 - 96, The
subscription gate currently checks marstek_mac_.empty() so
ensure_marstek_subscription_ is skipped once a MAC exists, which leaves
subscriptions stale if ct_mac changes; update the logic in the connected branch
(where connected && this->marstek_mqtt_enabled_ is evaluated) to always call
ensure_marstek_subscription_ while connected and enabled (remove the
marstek_mac_.empty() condition) and ensure ensure_marstek_subscription_ itself
is idempotent/safe to call repeatedly (or add a more robust check inside
ensure_marstek_subscription_ against the current marstek_mac_ to re-subscribe
only when it changes).
| uint32_t max_stamp = 0; | ||
| for (uint8_t i = 0; i < this->num_phases_; ++i) { | ||
| if ((*this->raw_stamp_ms_)[i] > max_stamp) max_stamp = (*this->raw_stamp_ms_)[i]; | ||
| } | ||
| // Unsigned subtraction wraps correctly across the 49.7-day boundary | ||
| // (matches the rollover-safe pattern documented in the plan). | ||
| if (max_stamp != 0 && (now - max_stamp) > this->max_sensor_age_ms_) { | ||
| return {}; // unavailable |
There was a problem hiding this comment.
Per-phase staleness is not enforced in freshness check.
Line 30 tracks only the newest timestamp; one fresh phase can mask another stale phase, and stale data is still emitted. Validate age per phase instead of using max_stamp.
💡 Suggested fix
- uint32_t max_stamp = 0;
- for (uint8_t i = 0; i < this->num_phases_; ++i) {
- if ((*this->raw_stamp_ms_)[i] > max_stamp) max_stamp = (*this->raw_stamp_ms_)[i];
- }
- // Unsigned subtraction wraps correctly across the 49.7-day boundary
- // (matches the rollover-safe pattern documented in the plan).
- if (max_stamp != 0 && (now - max_stamp) > this->max_sensor_age_ms_) {
- return {}; // unavailable
- }
+ // Unsigned subtraction wraps correctly across the 49.7-day boundary.
+ for (uint8_t i = 0; i < this->num_phases_; ++i) {
+ const uint32_t stamp = (*this->raw_stamp_ms_)[i];
+ if (stamp == 0 || (now - stamp) > this->max_sensor_age_ms_) {
+ return {}; // unavailable
+ }
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| uint32_t max_stamp = 0; | |
| for (uint8_t i = 0; i < this->num_phases_; ++i) { | |
| if ((*this->raw_stamp_ms_)[i] > max_stamp) max_stamp = (*this->raw_stamp_ms_)[i]; | |
| } | |
| // Unsigned subtraction wraps correctly across the 49.7-day boundary | |
| // (matches the rollover-safe pattern documented in the plan). | |
| if (max_stamp != 0 && (now - max_stamp) > this->max_sensor_age_ms_) { | |
| return {}; // unavailable | |
| // Unsigned subtraction wraps correctly across the 49.7-day boundary. | |
| for (uint8_t i = 0; i < this->num_phases_; ++i) { | |
| const uint32_t stamp = (*this->raw_stamp_ms_)[i]; | |
| if (stamp == 0 || (now - stamp) > this->max_sensor_age_ms_) { | |
| return {}; // unavailable | |
| } | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@esphome/components/ct002/sensor_backed.cpp` around lines 28 - 35, The current
freshness check uses max_stamp and so a fresh phase can hide a stale one; change
the logic to iterate each phase (using this->num_phases_ and
(*this->raw_stamp_ms_)[i]) and for each phase treat a zero stamp or a stamp
where (now - stamp) > this->max_sensor_age_ms_ as stale and return {}
(unavailable). Replace the max_stamp calculation and the single comparison with
a per-phase loop that validates each timestamp against now and
this->max_sensor_age_ms_ so any stale or missing phase causes the function to
return unavailable.
| void CT002Component::pump_control_() { | ||
| for (int i = 0; i < 8; ++i) { | ||
| char buf[128]; | ||
| struct sockaddr_storage from{}; | ||
| socklen_t from_len = sizeof(from); | ||
| ssize_t n = this->control_socket_->recvfrom(buf, sizeof(buf) - 1, | ||
| reinterpret_cast<struct sockaddr *>(&from), | ||
| &from_len); | ||
| if (n <= 0) break; |
There was a problem hiding this comment.
Guard pump_control_() against missing socket before recvfrom.
Line 67 dereferences control_socket_ unconditionally, but start_control_server_() can leave it null (disabled or bind failure). Add an early null check.
💡 Suggested fix
void CT002Component::pump_control_() {
+ if (this->control_socket_ == nullptr) return;
for (int i = 0; i < 8; ++i) {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@esphome/components/ct002/test_hooks.cpp` around lines 62 - 70, The
pump_control_() loop dereferences control_socket_ without checking for null
(start_control_server_ may leave it null); add an early guard in
CT002Component::pump_control_() to return immediately if control_socket_ is null
(or not open) before calling control_socket_->recvfrom, so the method safely
exits when the control socket wasn't created or failed to bind.
| def ensure_e2e_binary() -> None: | ||
| if not E2E_BINARY.exists(): | ||
| subprocess.run( | ||
| ["esphome", "compile", str(E2E_YAML)], | ||
| check=True, | ||
| cwd=_E2E_DIR.parent.parent, | ||
| ) |
There was a problem hiding this comment.
Rebuild the ESPHome e2e binary when sources change.
ensure_e2e_binary() only compiles when the binary is missing, so local runs can silently use stale artifacts after C++/YAML edits. That can produce false green/red e2e results.
Suggested fix
def ensure_e2e_binary() -> None:
- if not E2E_BINARY.exists():
- subprocess.run(
- ["esphome", "compile", str(E2E_YAML)],
- check=True,
- cwd=_E2E_DIR.parent.parent,
- )
+ subprocess.run(
+ ["esphome", "compile", str(E2E_YAML)],
+ check=True,
+ cwd=_E2E_DIR.parent.parent,
+ )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/_ct002_e2e_backend.py` around lines 58 - 64, ensure_e2e_binary
currently only compiles when E2E_BINARY is missing, allowing stale artifacts to
be used; change it to rebuild when sources have changed by comparing
modification times: if E2E_BINARY does not exist OR E2E_BINARY.stat().st_mtime
is older than E2E_YAML.stat().st_mtime (and ideally older than any files under
the e2e source dir like _E2E_DIR or its children), invoke
subprocess.run(["esphome", "compile", str(E2E_YAML)], check=True,
cwd=_E2E_DIR.parent.parent) to recompile; use pathlib to walk source files and
take the newest mtime to compare against E2E_BINARY to decide whether to
rebuild.
| proc = subprocess.Popen( | ||
| [str(host_binary)], | ||
| stdout=subprocess.DEVNULL, | ||
| stderr=subprocess.DEVNULL, | ||
| preexec_fn=os.setsid, | ||
| ) | ||
| # Wait for the binary to bind the port. | ||
| deadline = time.monotonic() + 5.0 | ||
| while not _port_in_use(12345) and time.monotonic() < deadline: | ||
| if proc.poll() is not None: | ||
| pytest.fail(f"host binary exited prematurely with code {proc.returncode}") | ||
| time.sleep(0.1) | ||
| if not _port_in_use(12345): | ||
| proc.terminate() | ||
| pytest.fail("host binary did not bind UDP 12345 within 5s") | ||
| yield proc | ||
| os.killpg(os.getpgid(proc.pid), signal.SIGTERM) | ||
| try: | ||
| proc.wait(timeout=2.0) | ||
| except subprocess.TimeoutExpired: | ||
| os.killpg(os.getpgid(proc.pid), signal.SIGKILL) | ||
| proc.wait() |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify POSIX-only APIs are used and whether there is a platform guard.
rg -n --type=py 'preexec_fn=os\.setsid|os\.killpg|pytest\.mark\.skipif|os\.name|sys\.platform' tests/components/ct002/test_host_e2e.pyRepository: tomquist/AstraMeter
Length of output: 239
Guard POSIX-only process-group handling in this fixture.
tests/components/ct002/test_host_e2e.py calls POSIX-only APIs (preexec_fn=os.setsid and unguarded os.killpg(os.getpgid(proc.pid), ...)) with no non-POSIX/platform skip, so it can fail off POSIX and teardown can raise if the child exits before the cleanup.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/components/ct002/test_host_e2e.py` around lines 95 - 116, Guard
POSIX-only process-group handling around the subprocess creation and teardown:
when spawning the host binary, only pass preexec_fn=os.setsid on POSIX (e.g. if
os.name == "posix"), and on Windows use subprocess.CREATE_NEW_PROCESS_GROUP via
creationflags instead; in teardown, protect os.killpg/os.getpgid with the same
POSIX check and ensure the child is still running (proc.poll() is None) before
attempting to kill it to avoid exceptions if it already exited. Update
references in this fixture to the POSIX symbols preexec_fn=os.setsid, os.killpg,
and os.getpgid and add the non-POSIX alternative using
subprocess.CREATE_NEW_PROCESS_GROUP and safe existence checks around kills.
| pytestmark = pytest.mark.skipif( | ||
| not (_have("cmake") and (_have("g++") or _have("clang++"))), | ||
| reason="cmake and a C++ compiler are required for host-gcc protocol tests", | ||
| ) | ||
|
|
||
|
|
||
| def _regen_vectors() -> None: | ||
| subprocess.run( | ||
| ["uv", "run", "python", str(HERE / "_gen_protocol_test_vectors.py")], | ||
| check=True, | ||
| cwd=REPO_ROOT, | ||
| ) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify current skip gate and uv invocation.
rg -n --type=py '_have\("cmake"\)|_have\("g\+\+"\)|_have\("clang\+\+"\)|_have\("uv"\)|\["uv",' tests/components/ct002/test_host_protocol.pyRepository: tomquist/AstraMeter
Length of output: 212
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE="tests/components/ct002/test_host_protocol.py"
echo "=== File header + skip gate + _regen_vectors ==="
sed -n '1,120p' "$FILE" | cat -n
echo
echo "=== All _regen_vectors references ==="
rg -n "_regen_vectors\(" "$FILE"
echo
echo "=== Any uv-related gating elsewhere in this file ==="
rg -n --type=py "_have\(\"uv\"\)|uv run|\\buv\\b" "$FILE"Repository: tomquist/AstraMeter
Length of output: 2878
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE="tests/components/ct002/_gen_protocol_test_vectors.py"
echo "=== $FILE (first 120 lines) ==="
sed -n '1,120p' "$FILE" | cat -n
echo
echo "=== External imports heuristic (non-stdlib) ==="
# crude heuristic: show import lines; will manually judge from them
rg -n "^import |^from " "$FILE" | head -n 80Repository: tomquist/AstraMeter
Length of output: 4048
Handle missing uv when regenerating protocol test vectors.
pytestmark skips only on cmake/C++ toolchain, but _regen_vectors() unconditionally runs uv run ..., so environments with cmake/compiler but no uv fail. _gen_protocol_test_vectors.py is stdlib-only, so falling back to sys.executable is sufficient.
💡 Proposed patch
@@
import shutil
import subprocess
+import sys
from pathlib import Path
@@
def _regen_vectors() -> None:
+ cmd = (
+ ["uv", "run", "python", str(HERE / "_gen_protocol_test_vectors.py")]
+ if _have("uv")
+ else [sys.executable, str(HERE / "_gen_protocol_test_vectors.py")]
+ )
subprocess.run(
- ["uv", "run", "python", str(HERE / "_gen_protocol_test_vectors.py")],
+ cmd,
check=True,
cwd=REPO_ROOT,
)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| pytestmark = pytest.mark.skipif( | |
| not (_have("cmake") and (_have("g++") or _have("clang++"))), | |
| reason="cmake and a C++ compiler are required for host-gcc protocol tests", | |
| ) | |
| def _regen_vectors() -> None: | |
| subprocess.run( | |
| ["uv", "run", "python", str(HERE / "_gen_protocol_test_vectors.py")], | |
| check=True, | |
| cwd=REPO_ROOT, | |
| ) | |
| pytestmark = pytest.mark.skipif( | |
| not (_have("cmake") and (_have("g++") or _have("clang++"))), | |
| reason="cmake and a C++ compiler are required for host-gcc protocol tests", | |
| ) | |
| def _regen_vectors() -> None: | |
| cmd = ( | |
| ["uv", "run", "python", str(HERE / "_gen_protocol_test_vectors.py")] | |
| if _have("uv") | |
| else [sys.executable, str(HERE / "_gen_protocol_test_vectors.py")] | |
| ) | |
| subprocess.run( | |
| cmd, | |
| check=True, | |
| cwd=REPO_ROOT, | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/components/ct002/test_host_protocol.py` around lines 25 - 36,
_pytestmark_ currently skips based on cmake/C++ but _regen_vectors() always
invokes "uv run python ..." which fails if uv is not installed; update the
_regen_vectors function to prefer using "uv" when available but fall back to the
system Python: detect uv with shutil.which("uv") and if present call
subprocess.run(["uv","run","python", str(HERE /
"_gen_protocol_test_vectors.py")], check=True, cwd=REPO_ROOT) otherwise call
subprocess.run([sys.executable, str(HERE / "_gen_protocol_test_vectors.py")],
check=True, cwd=REPO_ROOT); add imports for shutil and sys if not already
present.
…uv.lock The README and esphome.example.yaml external_components ref now tracks @develop between releases. release.sh pins it to the version tag (and finalizes uv.lock) on the release branch, then resets it back to @develop when syncing develop. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
Drive release.sh against a throwaway git sandbox (local bare repo as origin, no network) with uv/yq stubbed on PATH, asserting the released main state (version + uv.lock + pinned @Version component ref + finalized CHANGELOG + tag) and the develop sync-back (@develop ref, ## Next, addon "next"). Plus guards for non-semver input and a missing ## Next entry. https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
There was a problem hiding this comment.
♻️ Duplicate comments (1)
CHANGELOG.md (1)
5-8:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winConsolidate
## Nextto a single branch-summary bullet.
## Nextstill has multiple bullets (Lines 5-8). Keep one continuously edited bullet and fold these points into that single summary line.✏️ Suggested shape
- - **Added** ... - - **Added** ... - - **Fixed** ... - - **Fixed** ... + - **Added/Fixed** <single concise summary of this branch’s user-visible outcome>.As per coding guidelines: "For user-facing work, keep one bullet under
## Nextin CHANGELOG.md that summarizes the overall outcome of the branch; edit that bullet on later iterations rather than appending new bullets."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@CHANGELOG.md` around lines 5 - 8, CHANGELOG.md's "## Next" currently contains multiple bullets (the ESPHome ct002 addition, Modbus UDP TRANSPORT option, Home Assistant powermeter fix, and multi-Venus CT002 cross-talk fix). Replace those separate bullets with a single continuously edited branch-summary bullet under "## Next" that briefly summarizes the branch outcome and fold the individual points into that one line (mentioning the ct002 ESPHome external component, Modbus UDP/TRANSPORT option, Home Assistant powermeter startup fix, and multi‑Venus CT002 cross-talk / *_chrg_power *_dchrg_power fix) so the changelog has only one editable bullet going forward.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@CHANGELOG.md`:
- Around line 5-8: CHANGELOG.md's "## Next" currently contains multiple bullets
(the ESPHome ct002 addition, Modbus UDP TRANSPORT option, Home Assistant
powermeter fix, and multi-Venus CT002 cross-talk fix). Replace those separate
bullets with a single continuously edited branch-summary bullet under "## Next"
that briefly summarizes the branch outcome and fold the individual points into
that one line (mentioning the ct002 ESPHome external component, Modbus
UDP/TRANSPORT option, Home Assistant powermeter startup fix, and multi‑Venus
CT002 cross-talk / *_chrg_power *_dchrg_power fix) so the changelog has only one
editable bullet going forward.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 0c8ac0d5-e7bf-4004-a931-cd48e0529ac2
📒 Files selected for processing (5)
CHANGELOG.mdREADME.mdesphome.example.yamlrelease.shtests/test_release.py
Ports the AstraMeter CT002/CT003 grid-meter emulator, load balancer, and cross-phase filter pipeline from Python to C++ as an ESPHome external component, so the full stack can run directly on an ESP32 — no separate Home Assistant add-on or server required. Grid power comes from any ESPHome sensor source (Modbus, M-Bus, Tasmota, Shelly, Envoy, …).
What ships
A single ESPHome external component,
ct002, with two optional nested sub-blocks under the samect002:key:See
esphome.example.yamlfor an annotated template.Layout
Everything lives under
esphome/components/ct002/:ct002.{h,cpp}(UDP server, consumer lifecycle, response builder, balancer wiring),protocol.{h,cpp}(CT002 wire format),balancer.{h,cpp}(load balancer with saturation tracking + efficiency rotation),sensor_backed.{h,cpp}+hampel/smoothing/pid.{h,cpp}+wrapper_base.h(filter pipeline).mqtt_insights:sub-block —mqtt_insights.{h,cpp}(per-consumer + device-level state publishing, command dispatch, Marstek broadcast),ha_discovery.{h,cpp}(HA Device Discovery payloads),marstek_responder.{h,cpp}(Marstek poll wire helpers — no ESPHome deps, host-gcc testable). Gated by#ifdef USE_MQTT.marstek_registration:sub-block —marstek_registration.{h,cpp}(cloud token → list → add → confirm state machine, ESPPreferences-persisted MAC). Gated by#ifdef USE_CT002_MARSTEK_REGISTRATION.__init__.pyowns the parent schema plus both nested sub-schemas;mqtt/http_requestare enforced viacv.requires_component,json/md5are auto-loaded.Parity by discipline
Python under
src/astrameter/is canonical; the C++ is a mechanical mirror (same file/function/field names, same filter ordering). The rule is documented inCONTRIBUTING.md. Multiple parity audits were run against the Python source and its test suites; fixes landed for the balancer probe weights, consumer eviction, command-setter semantics,poll_intervalEMA,smooth_targetsource, MQTT-insights field types / discovery re-publish, banker's rounding on wire values,parse_intstrictness, and request dedup.Intentional divergences (documented): the Shelly discovery path is absent (an ESPHome build is a single CT002), there's no ARP lookup (lwIP has no ARP table),
origin.sw_versionis a literal instead of a git SHA, and the MQTT layer uses ESPHome's client + synchronous publish instead of Python's asyncio queue/reconnect loop.Testing
host_protocol_test(wire-format parity, golden-vector backed),host_balancer_test,host_wrappers_test(hampel/smoothing/pid),host_marstek_responder_test(Marstek poll/topic wire format).test_host_e2e.py): drives the realBatterySimulatorover UDP against the compiled host binary and asserts the response frame.Validated on real hardware
Flashed to an ESP32; fixed two bugs that only surface on-device: the HA consumer-discovery payload (~6.9 KB) exceeded ESPHome's
json::build_json5120-byte cap and was truncated to invalid JSON (now serialized via ArduinoJson directly), and the Marstek App-topic subscription missed the registration-applied MAC due to component setup ordering (now resolved lazily once the MAC is known).Status & limitations
ver_vin Marstek replies is hardcoded to the default rather than derived from the cloud device-list version (not plumbed through on the ESPHome path; matches Python's fallback when the field is absent).https://claude.ai/code/session_01TfTEE5DwAjqyassYprxaLp
Summary by CodeRabbit
New Features
Bug Fixes
Tests
Chores
Documentation