Skip to content

Improve jobs control plane and status API - #4

Merged
kraftaa merged 9 commits into
mainfrom
streaming
Apr 4, 2026
Merged

Improve jobs control plane and status API#4
kraftaa merged 9 commits into
mainfrom
streaming

Conversation

@kraftaa

@kraftaa kraftaa commented Mar 24, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add control-plane schema and CLI for scheduled sync jobs:
    • init-jobs, add-job, worker, force-job, reset-state
  • Add status API and UI:
    • JSON + HTML views, status filter, auto-refresh
    • job actions in UI: force run, retry failed, reset state
    • endpoints: /jobs, /jobs/html, /jobs/summary, /logs, /health, /health/worker
  • Record job runs with extra metadata:
    • severity, metrics_json, duration_ms
    • optional DQ hook via RUSTREAM_DQ_CMD
  • Improve reliability and migration behavior:
    • schema backfill on startup for control DB columns
    • invalid saved watermark/cursor state is detected and reset
    • --reset-state / reset-state --table ... for explicit recovery
  • Production template skeletons:
    • examples/production-template (docker-compose, Helm, Terraform stub)
    • GHCR Docker publish workflow

Testing

  • cargo fmt -- --check
  • cargo test
  • cargo clippy --all-targets -- -D warnings
  • Manual smoke:
    • init-jobs, add-job, worker, status-api
    • verified /jobs, /jobs/html, /logs, /health, /health/worker
    • verified UI actions (force/retry/reset)

@kraftaa kraftaa self-assigned this Mar 24, 2026
@kraftaa
kraftaa merged commit 9e12522 into main Apr 4, 2026
1 check passed
@kraftaa
kraftaa deleted the streaming branch April 4, 2026 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant