Skip to content

Switch ingress/egress telemetry to pub/sub#28

Merged
codingjoe merged 7 commits into
mainfrom
pub-sub
Jul 8, 2026
Merged

Switch ingress/egress telemetry to pub/sub#28
codingjoe merged 7 commits into
mainfrom
pub-sub

Conversation

@codingjoe

Copy link
Copy Markdown
Owner

No description provided.

@codingjoe codingjoe self-assigned this Jul 8, 2026
Comment thread threadmill/backends/lua/reaper.lua Outdated
Comment thread threadmill/backends/base.py Outdated
Comment thread threadmill/inspector/app.py Outdated
Comment thread threadmill/inspector/app.py Outdated
Comment thread threadmill/backends/base.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR switches the inspector’s ingress/egress telemetry from Redis polling/time-series reads to a pub/sub-driven approach, adding live sparklines backed by an in-memory rolling buffer while keeping queue segment counts sourced from backend snapshots.

Changes:

  • Introduces an in-memory TelemetryBuffer and adds inspector sparklines that refresh on a timer from pub/sub events.
  • Replaces the backend telemetry polling API with queue_stats() (async counts snapshot) plus worker_telemetry() (async pub/sub stream).
  • Updates the Redis backend to publish ingress/egress events via Redis pub/sub and adjusts tests accordingly.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
threadmill/inspector/telemetry.py Adds a rolling per-second buffer to aggregate pub/sub telemetry into series/rates.
threadmill/inspector/inspector.scss Styles new ingress/egress sparkline widgets and adjusts layout sizing.
threadmill/inspector/app.py Wires up pub/sub listener + rolling buffer, adds sparklines, and migrates inspector refresh to async queue_stats().
threadmill/backends/redis.py Publishes telemetry via pub/sub; adds queue_stats() + worker_telemetry() implementations.
threadmill/backends/lua/acknowledge.lua Publishes egress telemetry on acknowledge via Redis PUBLISH.
threadmill/backends/base.py Introduces TelemetryDirection/TelemetryEvent and replaces telemetry API with async queue_stats() + worker_telemetry().
tests/test_inspector.py Updates inspector tests to use queue_stats() and adjusts timer expectations.
tests/backends/test_redis.py Replaces telemetry polling tests with pub/sub publish assertions and async queue_stats() usage.
tests/backends/test_base.py Updates base-backend contract tests for async queue_stats().
AGENTS.md Adds docstring style guidance for mkdocs (Google style; avoid RST directives).
Comments suppressed due to low confidence (1)

threadmill/inspector/app.py:626

  • If the backend telemetry stream raises an exception (network issue, decode error, etc.), the listener task will currently stop silently. Logging unexpected exceptions here would make failures visible and help avoid a “stuck at zero” UI with no clue why.
                try:
                    async for event in stream:
                        self.telemetry_buffer.record(event, now=time.monotonic())
                except asyncio.CancelledError:
                    raise

Comment thread threadmill/inspector/app.py
Comment thread threadmill/inspector/app.py Outdated
Comment thread threadmill/backends/redis.py Outdated
Comment thread threadmill/backends/redis.py Outdated
Comment thread threadmill/inspector/telemetry.py Outdated
@codingjoe codingjoe merged commit a188423 into main Jul 8, 2026
4 checks passed
@codingjoe codingjoe deleted the pub-sub branch July 8, 2026 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants