Projects Monitoring

A Cribl Stream app for monitoring the health and throughput of your Stream Projects & Subscriptions — for every Project in a worker group it rolls up delivery volume, dropped and blocked events, per-subscription and per-destination throughput, destination health, and the Packs each Project uses. It runs as an app inside a Cribl Stream instance and talks to the Cribl API to read Project configuration and the system-metrics store.

Installation

1. Log in to Cribl and then click on Apps->View All
2. Click Add App->Import from Git.
3. Paste the repo url and "latest" for the release tag.
4. Click Import.

What it does

Stream Projects and Subscriptions each move data independently, but Cribl's built-in monitoring is environment-wide — there's no single place to answer "how is this Project doing, and where is its data going?" Projects Monitoring assembles that per-Project picture:

1. Pick a worker group — choose a Stream worker group (internal Search and Lake-access groups are filtered out) and pick a time range (15m / 1h / 4h / 24h / 7d). Everything below is scoped to that group and window.
2. Scan the Projects table — every Project in the group, sorted by delivered volume, with its subscription count, connected-destination count, delivered bytes, dropped events, blocked outputs, and a health pill (Healthy / Drops / Blocked). A toolbar line shows how long the metric store has actually been recording throughput in the selected range.
3. Open a Project — drill into KPI tiles (delivery throughput, events in/out, dropped events, blocked outputs), a delivered-bytes-over-time chart, and a per-destination throughput breakdown.
4. Watch per-subscription throughput — a subscriptions table breaks out events and bytes in/out for each subscription in the Project.
5. Check destination health — each destination the Project's routes actually connect to is listed with a worst-observed health pill (Healthy / Degraded / Blocked / Down) derived from the health.outputs and backpressure.outputs gauges. Destinations that exist for the Project but have no live connection (for example an unconnected devnull) are hidden.
6. See which Packs are in use — Packs referenced by the Project are surfaced whether they're attached to a subscription's pipeline or to a project route (consumers[].connections[].pipeline), badged with how many subscriptions and routes wire each one, plus the Pack's event throughput.
7. Sample live data types — on demand, the app captures a short read-only sample of events at a Project's destination and classifies them into data types (sourcetype / dataset / structure-sniffed), so you can see what is flowing, not just how much.
8. Environment Overview — a compact group-wide summary (ingest/delivery throughput, events in/out, dropped, blocked, and a throughput-over-time chart) for context alongside the per-Project view.

How the metrics work

Projects, Subscriptions, and their config are worker-group-scoped and read under the /m/:gid API context (src/lib/api.ts). Throughput and health come from Cribl's system-metrics store, which is aggregated leader-side (bare /system/metrics/query, not group-routed) with every group's data tagged by a __worker_group dimension — so each query ANDs in a __worker_group == '<gid>' scope (src/lib/metrics.ts). Two gotchas the code handles: this endpoint silently ignores relative time strings like -1h/now and returns empty results, so ranges are resolved to absolute epoch-ms before every request; and dotted metric names (total.out_bytes) must be double-quoted inside aggregation expressions or they parse as field access and match nothing. Per-Project roll-ups (src/lib/projectMetrics.ts) join across the metric dimensions each series actually carries — the native project dim for egress, subscription id for ingest, pipeline id for drops, and the <type>:<id> output dim for blocked/health.

Project structure

• src/App.tsx — top-level shell: worker-group + range selection, tab switching (Projects & Subscriptions / Environment Overview), and Project drill-in
• src/lib/api.ts — Cribl API calls: groups, Projects, Subscriptions, Project destinations/pipelines, and NDJSON live capture
• src/lib/metrics.ts — system-metrics access layer: worker-group scoping, epoch-ms range resolution, quoted aggregation expressions, scalar/series/dataSpan helpers
• src/lib/projectMetrics.ts — rolls up throughput, drops, blocked, per-destination breakdown, destination health, and Pack usage for a single Project
• src/lib/classify.ts — buckets captured events into data types by their most telling field, with structure-sniffing fallback
• src/lib/format.ts — compact human-readable formatters (bytes, throughput, counts, durations)
• src/lib/types.ts — Group / Project / Subscription / Destination / metric type definitions
• src/components/ — UI: ProjectsList, ProjectDetail, SubscriptionsPanel, DataTypePanel, Overview, GroupSelect, RangePicker
• src/viz/ — self-contained SVG charts (LineChart, BarChart, Sparkline, StatTile)

Development

Clone this repo. Install dependencies and start the app.

npm install
npm run dev

Log into Cribl Cloud, Go to App Platform > Development > Live Preview

To build a distributable app bundle:

npm run package

Release Versions

Version	Changes
1.0.13	Per-subscription throughput, per-destination health, two-level Pack usage (subscription + project route), throughput recording-window display, connected-destinations-only filtering, and metric time-format/scoping fixes.

License

Licensed under the Apache License 2.0.