Skip to content

Stop publishing system stats for offline robots - #91

Merged
b-Tomas merged 2 commits into
mainfrom
skip-system-stats-when-offline
Aug 27, 2026
Merged

Stop publishing system stats for offline robots#91
b-Tomas merged 2 commits into
mainfrom
skip-system-stats-when-offline

Conversation

@b-Tomas

@b-Tomas b-Tomas commented Aug 27, 2026

Copy link
Copy Markdown
Member

Problem

__publish_pending_system_stats() runs once per execution loop iteration and publishes system stats for every robot with an active session — including robots the connector reports as offline.

That drives a loop in the platform:

  1. stats arrive for a robot InOrbit has marked offline
  2. InOrbit sends get_state to resolve the discrepancy
  3. the edge-sdk's _handle_get_state calls our online status callback, gets False, and publishes a retained offline status message
  4. InOrbit timestamps it

So the robot's offline timestamp keeps moving forward for as long as the connector runs, and an offline robot never looks like it has been offline for any length of time.

Fix

Gate the publish on _is_fleet_robot_online(robot_id) inside __publish_pending_system_stats, which is where the stored-stats path and the default-values path converge. One guard covers both, so stats explicitly stored via publish_robot_system_stats() are dropped for an offline robot too — not just the framework's zeroed/host defaults. Guarding only the public setter would have left the defaults path still refreshing the timestamp.

Nothing is lost by skipping: the forced state request only helps when the robot is online but InOrbit believes otherwise. For a robot we already consider offline, the reply just re-asserts offline.

Notes for review

  • The online check now runs once per robot per loop iteration, where before it only ran when InOrbit sent get_state. It is subclass code, so it is wrapped in try/except and falls back to publishing on error — matching the edge-sdk's own get_state fallback (robot.py:777-781), so a broken health check keeps today's behaviour instead of silently muting a robot. The _is_fleet_robot_online() / _is_robot_online() docstrings and the docs now require the implementation to be cheap and non-blocking.
  • Pre-existing gap, not addressed here: nothing pushes an online→offline transition. The edge-sdk only sends robot status on connect, on the MQTT will, and in response to get_state. InOrbit still detects offline through its own data timeout, and a connector that keeps publishing pose or key-values for a dead robot will keep it looking online regardless.
  • Behaviour change, so this wants a minor version bump on release.

Testing

  • test_publish_pending_system_stats_skips_offline_robots — offline robot gets no publish, its stored stats are dropped, pending dict still cleared
  • test_publish_pending_system_stats_publishes_when_online_check_fails — raising online check still publishes for every robot
  • Full suite: 237 passed. flake8 and black clean.

🤖 Generated with Claude Code

The framework publishes system stats for every robot on every execution
loop iteration, including robots the connector reports as offline. InOrbit
answers those stats with a get_state request, the online status callback
replies "offline", and the retained status message refreshes the robot's
offline timestamp -- so an offline robot's offline_ts keeps moving forward
for as long as the connector runs.

Gate the publish on _is_fleet_robot_online(robot_id) inside
__publish_pending_system_stats, where the stored-stats and default-values
paths converge, so stats explicitly stored via publish_robot_system_stats()
are dropped too rather than only the framework defaults. Nothing is lost by
skipping: the forced state request only helps when the robot is online but
InOrbit believes otherwise.

The online check is subclass code and now runs once per robot per loop
iteration instead of only on get_state, so it falls back to publishing if it
raises (matching the edge-sdk's own get_state fallback) and the docstrings
now require it to be cheap and non-blocking.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@b-Tomas b-Tomas self-assigned this Aug 27, 2026
@b-Tomas
b-Tomas requested a review from leandropineda August 27, 2026 15:51
Comment thread inorbit_connector/connector.py Outdated
Co-authored-by: Leandro <leandropineda.lp@gmail.com>
@b-Tomas
b-Tomas merged commit da7b14c into main Aug 27, 2026
13 checks passed
@b-Tomas
b-Tomas deleted the skip-system-stats-when-offline branch August 27, 2026 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants