fix(gcp-billing): thread actual currency from BQ through API and frontend (IW-246)#36
Merged
Merged
Conversation
…tend (IW-246) - BQ query selects ANY_VALUE(currency) per (month, service) row; stored in attrs - _upsert_no_spend_data adds "currency": "USD" sentinel default - summary/by-service endpoints read currency from attrs instead of hardcoding USD - fmtUSD/fmtUSDShort accept currency param (default "USD") - fmtUSDShort rewritten with Intl.NumberFormat compact notation (handles non-USD symbols) - CostTooltip accepts currency prop; all YAxis tickFormatter and table call sites threaded - Sync submodule JSX with latest frontend: Check icon, showRefreshed state, SSE addEventListener
…resh Celery task (IW-246)
mguptahub
pushed a commit
to eyriehq/eyriehq
that referenced
this pull request
May 7, 2026
… fix refresh endpoint (IW-246) (#174) - plugins/oss bumped e7a4878 → 24c1b89 (release/0.14.0 tip post-PR#36) - Collector reads ANY_VALUE(currency) from BQ; stores ISO 4217 code in attrs - summary + by-service endpoints pass currency through instead of hardcoding USD - fmtUSD/fmtUSDShort accept currency param; Intl.NumberFormat handles symbol natively - Refresh endpoint passes GCP project ID (not billing account ID) to Celery task Plugins PR: eyriehq/plugins#36 Co-authored-by: Arun <arun@eyriehq.com>
mguptahub
added a commit
that referenced
this pull request
May 11, 2026
* feat(gcp): add gcp-monitoring plugin manifest with monitoring.read scope and viewer role
* feat(gcp-iam): GCP Cloud IAM plugin — service accounts, IAM bindings, custom roles (IW-223)
Add plugin.json (gcp_scopes + gcp_roles), collector (IAM API), FastAPI router
at /api/gcp/iam, GcpIamPanel.jsx with service-account/roles table and drawer,
manifest.js registration, and 15 unit tests (all passing).
* fix(gcp-iam): update to gcp_credentials v2 API (IW-216/IW-223 rebase)
- Replace are_gcp_credentials_configured → are_credentials_configured
- Replace get_active_gcp_collector_regions → get_gcp_projects()
(new API returns [{"project_id": str}] dicts; extract project_id field)
- Update test_router.py mocks to match new function names and return types
Aligns gcp-iam plugin with the credential provider refactor from PR #150.
Co-Authored-By: Arun <arun@eyriehq.com>
* fix(gcp-iam): fix MetaItem to use drawer-meta-item/key/val design system classes
Replace bare <span className="meta-label"> with the canonical MetaItem
pattern from DESIGN.md: drawer-meta-item wrapper div + drawer-meta-key
label + drawer-meta-val value span. Ensures the drawer meta grid renders
correctly with the shared CSS grid layout.
Co-Authored-By: Arun <arun@eyriehq.com>
* feat(gcp-compute): GCP Compute Engine plugin — VM instances and persistent disks (IW-219)
Squash merge of feat/iw-219-gcp-compute. Adds gcp-compute plugin: collector, router, GcpComputePanel.jsx, manifest, 19 unit tests. Valkey binary client fix for GCP credential cache.
* feat(ec2): add root_device_name + volumes to collector, Boot column to router (IW-235)
Rebased onto release/0.14.0 (includes gcp-compute). Adds root_device_name and full volumes array to collector DB upsert, delete_on_termination to router, and root_device to detail response.
* fix(gcp-compute): show zone instead of self_link URL in drawer and metrics modal headers (IW-219)
* feat(gcp-billing): GCP Billing & Cost Management plugin (IW-224)
Adds gcp-billing plugin: collector (Cloud Billing API + BigQuery export path), router (/summary, /by-service, /trend, /projects), GcpBillingPanel.jsx, 28 unit tests. SetupBanner shown until BigQuery billing export is configured.
* fix(gcp-billing): change group from 'cost' to 'system' so it appears in sidebar (IW-224)
* fix(gcp-billing): auto-discover billing account + BQ dataset — no env vars required (IW-224)
* feat(gcp-compute): add Cloud Monitoring metrics collection and API endpoint (IW-236)
- collector.py: add collect_metrics() — incremental pull from Cloud Monitoring API
for CPU, memory, network in/out, disk read/write per VM instance; stores into
CollectedMetric with service_type='gcp-compute', resource_id=selfLink, region=project_id.
Uses ALIGN_MEAN for CPU/memory, ALIGN_SUM for network/disk; 1-hour buckets; 72 h window.
- plugin.json: add metrics_collector='collector:collect_metrics' and
https://www.googleapis.com/auth/monitoring.read scope + roles/monitoring.viewer.
- router.py: add GET /instances/metrics?resource_id=<selfLink>&hours=N endpoint
returning EC2-compatible JSON shape {instance_id, hours, period_seconds, metrics{…}}.
Co-Authored-By: Arun Engineer <arun@eyriehq.com>
* feat(gcp-compute): replace MetricChartStub with real recharts charts (IW-236)
- GcpComputePanel.jsx: replace MetricChartStub stub components with real
MetricChart + ChartTooltip (recharts LineChart pattern, same as EC2Panel).
- GcpMetricsModal: wire api.getGcpComputeMetrics(inst.self_link, hours) on
mount and hours change; show loading/error states.
- Renders CPU, Memory (empty note if no guest agent), Network Traffic
(merged in/out), Disk I/O (merged read/write) — same 1h/6h/24h/72h tabs.
- Adds formatBytes helper for network/disk axes.
Co-Authored-By: Surya Engineer <surya@eyriehq.com>
* fix: use em dash for null bytes in formatBytes (design system compliance)
Replace 'N/A' with '—' in formatBytes per DESIGN.md rule: missing/unavailable
values must show em dash, never 'N/A' or empty string.
* feat(gcp-memorystore): add Memorystore (Valkey) plugin scaffold (IW-222)
- plugin.json: registered as gcp connector, order=35, group=data
- collector.py: uses memorystore.googleapis.com/v1 REST API via httpx + Bearer token;
collects instances across all locations; Cloud Monitoring metrics via googleapiclient
(memory_usage, connected_clients, ops_per_sec, keyspace_hits/misses, hit_ratio)
- router.py: GET /instances, GET /instances/{id}/metrics, POST /refresh, GET /refresh/stream
- GcpMemorystorePanel.jsx: sortable table (Name|IP:Port|Engine|Size|State|Zone|Shards/Replicas|Auth|TLS),
detail drawer with connection/config/overview sections, metrics modal (4 charts)
- manifest.js: exports panel + Database icon
* fix(gcp-memorystore): remove unconditional invalidate_cache() calls (IW-237)
collect_resources() and collect_metrics() were calling invalidate_cache()
before get_gcp_credentials() on every Celery task run, nuking the 55-min
Valkey token cache each cycle and forcing a fresh token mint every time.
get_gcp_credentials() already handles expiry internally via _is_expired();
remove the manual invalidations so the cache is used as intended.
Co-Authored-By: Arun <arun@eyriehq.com>
* fix(gcp-memorystore): design system polish — bg-card, order 36 (IW-222)
- Replace undefined var(--bg-app) with var(--bg-card) in Connection drawer
- Fix order conflict: was 35 (same as gcp-billing), now 36
* feat(widgets): add dashboard widget config to gcp-compute and gcp-memorystore (IW-222)
Adds widget sections to both plugin.json files so they appear in the
Add Widget dialog. Uses region_key: project_id since GCP plugins store
project_id in CollectedMetric.region.
* fix(gcp-compute): use metric.labels.instance_name instead of invalid resource label
gce_instance resource labels are project_id, zone, and instance_id (numeric).
instance_name is NOT a valid resource label — using it caused all Cloud Monitoring
queries to return 0 results silently.
Fix: scope per-instance queries via metric.labels.instance_name (a metric label
on compute.googleapis.com metrics) and drop the invalid resource.labels.instance_name
from resource_filter.
Co-Authored-By: Arun <arun@eyriehq.com>
* fix(gcp-memorystore): use memorystore.googleapis.com/* metric namespace for Valkey
- Replace redis.googleapis.com/* metrics with memorystore.googleapis.com/instance/* equivalents
- Fix resource filter: use memorystore.googleapis.com/Instance (not redis_instance)
- Drop hit_ratio computation — separate hits/misses metrics unavailable in Valkey namespace
- Add cpu_utilization metric instead
Fixes IW-222
* feat(gcp-cloudsql): add Cloud SQL plugin — collector, router, frontend panel (IW-240)
- sqladmin.googleapis.com/v1 REST API via httpx + Bearer token
- Collects instance name, DB version, region/zone, tier, state, private/public IPs,
storage config, backup config, maintenance window, replica info
- Cloud Monitoring metrics: cpu_utilization, memory_usage, disk_bytes_used,
active_connections, queries_per_sec (cloudsql_database resource type)
- Router: GET /api/gcp/cloudsql/instances, metrics, refresh, SSE stream
- Frontend panel: sortable table, detail drawer with collapsible sections,
metrics modal with 5 charts and time range selector
- plugin.json: gcp connector, roles/cloudsql.viewer, widget config with all 5 metrics
* fix: Cloud SQL metric type + GCP refresh button consistency (IW-240) (#31)
* fix(gcp-cloudsql): use postgresql/transaction_count instead of non-existent /database/queries metric (IW-240)
* fix(gcp): remove text label from refresh buttons — icon-only to match EC2 style
GcpComputePanel and GcpBillingPanel had "Refresh"/"Refreshing…" text
alongside the icon. Remove the text so all GCP list panels are consistent
with the icon-only refresh button used on the EC2 panel.
Co-Authored-By: Surya <surya@eyriehq.com>
* fix(gcp): add showRefreshed 3-state to Compute and Billing refresh buttons
GcpComputePanel and GcpBillingPanel were missing the third refresh state
(Check icon + "Refreshed" text shown for 1500ms after refresh_done SSE).
Also switches both from es.onmessage+string-compare to the correct
es.addEventListener("refresh_done", ...) pattern used by all other panels.
---------
Co-authored-by: Arun Engineer <arun@eyriehq.com>
Co-authored-by: Surya <surya@eyriehq.com>
* feat: plugin registry-driven api block + GCP detail endpoints (IW-243) (#32)
* feat: add api blocks to all plugin.json manifests + GCP detail endpoints (IW-243)
- Add api block to every service plugin's plugin.json (ec2, rds, docdb, elasticache,
lb, secrets, ses, iam, eks, cost, mq, opensearch, gcp-compute, gcp-cloudsql,
gcp-memorystore, gcp-billing, gcp-iam) — declares endpoint templates for the
generic api.plugin(slug) client
- Add GET /instances/{id} detail endpoints to gcp-compute, gcp-cloudsql, gcp-memorystore
routers (previously only list and metrics existed)
- Update GCP panel DrawerDetail components to fetch fresh detail via the new detail
endpoint on open instead of using list-state data
- Update all GCP panel components to use api.plugin() instead of local fetchPlugin
helpers and removed api.getGcp* method calls
- Add PLUGIN_DEVELOPMENT.md at plugins root — the canonical guide for building new plugins
Co-Authored-By: Arun Engineer <arun@eyriehq.com>
* fix: prefix all AWS routes with /api/aws/ and make CloudSQL metrics flavor-aware (IW-243)
- Rename api_prefix in all 12 AWS plugin.json from /api/{svc} to /api/aws/{svc}
- Update widget.list_path and api block paths accordingly (ec2, rds, docdb,
elasticache, lb, cost, eks, mq, opensearch, secrets, ses, iam)
- Add api blocks to submodule plugin.json files for cost, eks, mq, opensearch, iam
- Update all hardcoded panel paths in EKSPanel, IAMPanel, MQPanel,
OpenSearchPanel, CostPanel to use /api/aws/ prefix
- Fix gcp-cloudsql collector.py: replace single postgresql transaction_count
metric with flavor-aware dispatch (POSTGRES_, MYSQL_, SQLSERVER_ prefixes);
unknown flavors fall back to common metrics only without erroring
* fix: update api block paths to /api/aws/ prefix in all AWS plugin.json files (IW-243)
Cherry-pick from wrong branch included api blocks with old /api/{svc}/ paths.
Update all plugin.json api block values to use /api/aws/{svc}/ prefix, consistent
with the api_prefix field and MGupta's review feedback.
---------
Co-authored-by: Arun Engineer <arun@eyriehq.com>
* feat(gcp-billing): fix SSE stream channel alignment and wire refresh/stream endpoint (IW-238) (#33)
- POST /refresh: use apply_async(countdown=0) and resolve billing account from DB
so the Valkey channel key matches what GET /refresh/stream subscribes to
- GET /refresh/stream: use same DB-resolved billing account fallback ('default')
to guarantee publish/subscribe pair alignment
- Add tests for POST /refresh (503 guard, task dispatch, channel key fallback)
and GET /refresh/stream (content-type check)
Co-authored-by: Arun Engineer <arun@eyriehq.com>
* feat: migrate AWS panel components into plugin directories (IW-243) (#34)
* feat: migrate AWS panel components into plugin directories (IW-243)
Move all 7 AWS service frontend panels from the main app repo into
their plugin directories in the submodule, matching the GCP plugin
pattern. Each plugin directory now ships its own Panel JSX and
manifest.js for build-time discovery via setup-plugins.sh +
generate-plugin-registry.js.
Panels migrated:
- ec2/EC2Panel.jsx + manifest.js
- rds/RDSPanel.jsx + manifest.js
- docdb/DocumentDBPanel.jsx + manifest.js
- elasticache/ElastiCachePanel.jsx + manifest.js
- lb/LBPanel.jsx + manifest.js
- ses/SESPanel.jsx + manifest.js
- secrets/SecretsPanel.jsx + manifest.js
Import paths adjusted from ../x to ../../x to match the
frontend/src/plugins/<slug>/ location after setup-plugins.sh copies
them. AlertBanner and ResourceAlerts resolved via ../../components/.
* feat: add DatabasesPanel composite + fix rds manifest to export it (IW-243)
DatabasesPanel renders RDS + DocumentDB together under the "databases" service tab.
It imports sibling plugins (./RDSPanel, ../docdb/DocumentDBPanel) and AlertBanner
from the core app (../../components/AlertBanner), avoiding circular dependencies.
rds/manifest.js updated to export DatabasesPanel (the composite) instead of
RDSPanel directly, so the "databases" service tab renders both panels.
---------
Co-authored-by: Arun Engineer <arun@eyriehq.com>
* feat(iam): use ShieldCheck icon to differentiate AWS IAM from GCP Cloud IAM (IW-244) (#35)
Co-authored-by: Arun Engineer <arun@eyriehq.com>
Co-authored-by: Surya <surya@eyriehq.com>
* fix(gcp-billing): thread actual currency from BQ through API and frontend (IW-246) (#36)
* fix(gcp-billing): thread actual currency from BQ through API and frontend (IW-246)
- BQ query selects ANY_VALUE(currency) per (month, service) row; stored in attrs
- _upsert_no_spend_data adds "currency": "USD" sentinel default
- summary/by-service endpoints read currency from attrs instead of hardcoding USD
- fmtUSD/fmtUSDShort accept currency param (default "USD")
- fmtUSDShort rewritten with Intl.NumberFormat compact notation (handles non-USD symbols)
- CostTooltip accepts currency prop; all YAxis tickFormatter and table call sites threaded
- Sync submodule JSX with latest frontend: Check icon, showRefreshed state, SSE addEventListener
* fix(gcp-billing): pass GCP project ID (not billing account ID) to refresh Celery task (IW-246)
---------
Co-authored-by: Surya <surya@eyriehq.com>
Co-authored-by: Arun <arun@eyriehq.com>
* feat(gcp-gke): GKE plugin — clusters, nodes, metrics (IW-218) (#38)
* feat(gcp-gke): add GKE plugin — collector, router, frontend panel (IW-218)
Collects GKE clusters and node pools via container.googleapis.com v1
(projects.locations.clusters.list with locations/-). Cloud Monitoring
metrics: cpu/memory allocatable utilization, container restart rate,
pod volume total. Tested against cluster-1 in asia-south1-a
(glass-effect-495104-d3): 1 cluster, 1 node pool (e2-medium/default-pool),
56 metric points collected — CPU ~22.7%, Memory ~17.9% utilization.
GcpGkePanel: sortable cluster table, detail drawer with node pool table,
metrics modal with 1h/6h/24h/72h tabs, 3-state refresh button.
* feat(gcp-gke): add per-node drawer and metrics modal (IW-218)
- GET /clusters/{id}/nodes/{name}/metrics — k8s_node Cloud Monitoring
metrics per node (cpu/memory allocatable utilization, network rx/tx)
- GkeNodesModal: clickable rows open GkeNodeDetailDrawer
- GkeNodeDetailDrawer: overview (pool, machine, zone, IP, version) + Metrics button
- GkeNodeMetricsModal: 4 charts with 1h/6h/24h/72h range tabs
- Escape key sequence: metrics → node drawer → nodes list
---------
Co-authored-by: Arun <arun@eyriehq.com>
* feat(gcp-lb): Cloud Load Balancing plugin + cross-plugin fixes (IW-221) (#39)
* feat(gcp-lb): add GCP Cloud Load Balancing plugin (IW-221)
Collects forwarding rules with backend health via GCP Compute API,
exposes REST endpoints for list/detail/metrics, and renders a sortable
panel with health-status badges and a Cloud Monitoring metrics modal.
Live test against test-lb (glass-effect-495104-d3 / asia-south1):
- 1 LB returned: test-lb, IP 35.207.193.109, TCP, EXTERNAL, port 80
- 1 backend: gke-cluster-1-default-pool-c4a4de25-grp (asia-south1-a), UNHEALTHY
- Overall health: UNHEALTHY (backend health check correctly propagated)
Co-Authored-By: Arun <arun@eyriehq.com>
* fix(gcp-lb): use name as resource_id (not selfLink), move to compute group (IW-221)
* fix(gcp-compute): include resource_id in instance list response (metrics endpoint fix)
* fix: route order (compute metrics), lb_id path greed, gcp-iam requires_mfa
* feat: require MFA for IAM and Billing plugins across all cloud providers
---------
Co-authored-by: Arun <arun@eyriehq.com>
* feat(gcp-secret-manager): GCP Secret Manager plugin (IW-247) (#41)
* feat(gcp-secret-manager): add GCP Secret Manager plugin — collector, router, frontend panel (IW-247)
Collects all secrets and their versions via secretmanager.googleapis.com v1.
Stores replication policy, version count, latest version state, labels, and
rotation config. Panel shows sortable table with detail drawer including
versions table and labels section. No metrics collector (sparse data, v1).
* feat(gcp-secret-manager): AWS parity — Load Value, RelativeTime, CopyButton, AgeBadge; fix(gcp-gke): widget resource_id_key → full path so dashboard metrics resolve correctly (IW-247)
---------
Co-authored-by: Arun <arun@eyriehq.com>
* feat(iw-248): reorganise plugins into aws/, gcp/, observability/ vendor groups (#43)
* feat(iw-248): reorganise plugins into aws/, gcp/, observability/ groups; update names to group/plugin format; add READMEs
- Move 12 AWS plugins into aws/ group
- Move 5 GCP plugins into gcp/ (dropping gcp- prefix in dir names)
- Move 3 observability plugins into observability/
- hello-world and gcp-monitoring remain at root (stubs, no group yet)
- Update plugin.json name fields to group/name format (e.g. aws/ec2, gcp/compute)
- Update services arrays to match new names
- Fix JSX relative imports: ../../ -> ../../../ for api/client, hooks/useData, constants, components/
- Add README.md to every plugin directory
- Add group-level README.md for aws/, gcp/, observability/
* fix(iw-248): move gcp-gke, gcp-lb, gcp-secret-manager into gcp/ group; update names and JSX imports
* fix(iw-248): revert JSX imports to ../../ — plugins copy flat to src/plugins/{name}/ in the build
* fix(iw-248): observability plugins — use parent.parent.parent for _shared path after moving one level deeper
* fix: GCP collectors always use configured project_id, not AWS region arg
collect_tasks.collect_metrics() iterates AWS regions (ap-south-1, us-east-1)
and passes them as the `region` parameter to every plugin metrics collector,
including GCP ones. All GCP collectors used `region if region else gcp.get(
"project_id")` — so the AWS region string silently became the GCP project ID,
causing all GCP metric and resource queries to fail silently (empty results).
Fix: use gcp.get("project_id", "") directly in all 13 GCP collector call-sites
across billing, cloud-sql, compute, gke, gke-metrics, iam, lb, lb-metrics,
memorystore, memorystore-metrics, secret-manager. The region param is still
accepted in the function signature for API compatibility but is no longer used
as project_id.
* fix: update services field in aws/lb and aws/rds plugin.json to namespaced names
aws/lb: services was ['elb'] (historic ELB name) — update to ['aws/lb']
aws/rds: services was ['databases'] (historic name) — update to ['aws/rds']
The IW-248 reorg correctly updated the 'name' field but missed updating the
'services' field in these two plugins, causing the plugin loader to register
them under old service_type keys, mismatching the rest of the namespaced system.
* fix(iw-248): update SERVICE_TYPE constants and inline service_type strings to namespaced format
IW-248 reorg updated plugin.json name/services fields but missed the Python
code entirely. Every collector.py and router.py had hardcoded old strings:
gcp-compute, gcp-gke, gcp-lb, gcp-iam, gcp-billing, gcp-memorystore,
gcp-cloudsql, gcp-secret-manager (GCP)
elb, databases, opensearch, secrets, iam, eks, mq, docdb, elasticache (AWS)
These are used when storing CollectedResource/CollectedMetric rows and when
queuing collect_gcp_resources tasks on refresh — causing 'no collector for
gcp-compute' warnings and all collected data stored under old service_type
keys that no frontend panel could resolve.
26 files updated. No logic changed, string constants only.
* fix: add api block to memorystore and cloud-sql plugin.json
Both plugins were missing the 'api' block so plugin-registry.js never
registered them in _pluginApiMap — causing 'api block not loaded' in
the panel. Added correct api.list/detail/metrics/refresh/stream paths
matching their router endpoints.
* fix(billing): SSE channel mismatch + 5-min timeout on empty-DB refresh
post_refresh was dispatching collect_gcp_resources with project_id while the
SSE stream subscribed on billing_account_id (or 'default' when DB empty).
The publish/subscribe channels never matched on first refresh — user waited
5 minutes for the fallback timeout before data appeared.
Fix: dispatch task with bid ('default' when DB empty) so publish and subscribe
keys always match. Also reduce billing refresh timeout from 5 min to 30s —
BQ queries complete in seconds, not minutes.
* fix: AWS router collect_resources.delay calls updated to namespaced service names
All 10 AWS plugin routers were queuing collect_resources.delay with old
flat names (ec2, elb, databases, eks, iam, mq, docdb, elasticache, opensearch,
secrets). These were missed in the IW-248 reorg — same pattern as the
collector/collector.py SERVICE_TYPE constants fixed earlier.
Refresh button in any AWS panel was queuing the task with the old name,
causing 'no plugin collector for ec2' warnings on every manual refresh.
* fix: AWS router/collector service_type DB query strings updated to namespaced names
38 occurrences of service_type == "ec2", "elb", "databases", "eks", "iam",
"mq", "docdb", "elasticache", "opensearch", "secrets", "ses", "cost"
in AWS router and collector files updated to "aws/ec2", "aws/lb", "aws/rds"
etc.
These were used in CollectedResource/CollectedMetric DB filter queries and
in SSE stream channel keys. Mismatch with the namespaced service_type now
stored by the collector caused:
- Panels returning empty data (query found 0 rows)
- Refresh not updating frontend (SSE channel mismatch → no refresh_done event)
* fix: EC2 collector service_type assignments missed in batch fix — now aws/ec2
* fix: AWS router SSE refresh channel strings updated to namespaced names
All 10 AWS routers had hardcoded old service names in their SSE channel keys:
refresh:ec2:{region}, refresh:elb:{region}, refresh:databases:{region}, ...
The Celery task (_publish_refresh_done) was publishing to refresh:aws/ec2:...
but the stream endpoint was subscribing to refresh:ec2:... → events never
delivered → panel waits for timeout then needs page reload to show new data.
GCP routers were already correct (they use f'refresh:{SERVICE_TYPE}:{pid}'
with the variable, not hardcoded strings).
* fix: aws/rds plugin registers RDSPanel directly, not composite DatabasesPanel
DatabasesPanel was a pre-IW-248 composite that rendered both RDS and
DocumentDB in one view. Now that DocumentDB has its own aws/docdb plugin,
the Databases sidebar entry should show only RDS data.
manifest.js now imports RDSPanel instead of DatabasesPanel.
---------
Co-authored-by: Arun <arun@eyriehq.com>
* fix: _metrics_start_end service_type args updated to namespaced names in all AWS collectors
All 8 AWS metric collectors called _metrics_start_end(db, 'ec2', ...) with
old flat names. _last_metric_timestamp queries collected_metrics by service_type,
but data is now stored with 'aws/ec2' etc. This caused the timestamp lookup to
always return None → always fetches full 72h window → re-upserts stale data
instead of incrementally fetching new CloudWatch datapoints.
Fixed: ec2, lb (elb), rds, eks, mq, docdb, elasticache, opensearch
* fix: EC2 metrics collection period 3600→300 (5-min granularity)
Using 1-hour period with a 1h widget range produces exactly 1 datapoint —
a dot, not a line. All other AWS collectors use 300s (5-min). EC2 now matches,
giving 12 datapoints/hour for proper chart rendering.
---------
Co-authored-by: Arun <arun@infrawatchlabs.com>
Co-authored-by: Arun <arun@eyriehq.com>
Co-authored-by: eyriehq-bot[bot] <269943525+eyriehq-bot[bot]@users.noreply.github.com>
Co-authored-by: Surya Engineer <surya@eyriehq.com>
mguptahub
added a commit
to eyriehq/eyriehq
that referenced
this pull request
May 11, 2026
* chore: update plugins/oss submodule to release/0.14.0 (gcp-monitoring manifest) * chore: pin plugins/oss submodule to release/0.14.0 (b39e8be) * feat(gcp): GCP Credential Provider + Cloud Monitoring client + Onboarding step (IW-216, IW-217) (#150) * feat(gcp): GCP Credential Provider + Cloud Monitoring client + Onboarding step (IW-216, IW-217) Backend: - gcp_credentials.py: provider chain (cache → registered providers → DB → env), Valkey cache keyed by project_id (TTL from token expiry, 55 min default), invalidate_cache(), are_credentials_configured(), get_gcp_projects() - gcp_sts_service.py: SERVICE_SCOPES registry, get_scoped_credentials() with per-plugin OAuth scope restriction - gcp.py: request-scoped helpers — require_gcp_session(), get_or_create_gcp_session() - gcp_monitoring.py: GcpMonitoringClient wrapping Cloud Monitoring v3 API; list_time_series(), list_metric_descriptors(), list_monitored_resource_descriptors() - admin_gcp_settings.py: admin API (GET/PUT/POST validate/DELETE /api/admin/settings/gcp), SA key structure validation, encrypted DB storage (AppSetting namespace=gcp) - plugin_loader.py: _register_gcp_scopes() pass, wired into both init paths - admin_connectors.py: GCP connector type, validation, save/delete helpers - main.py: include admin_gcp_settings router; add GCP connector handling to POST /api/system/setup (mirrors AWS onboarding path) - requirements.txt: google-auth, google-api-python-client, google-cloud-resource-manager Frontend: - GcpCredentialsPanel.jsx: drag-and-drop SA key upload, WIF config textarea, project ID field, validate + save + remove, mirrors AwsCredentialsPanel pattern - client.js: adminGetGcpSettings, adminSaveGcpSettings, adminValidateGcpSettings, adminDeleteGcpSettings - AdminPage.jsx: gcp tab using ConnectorTab + GcpCredentialsPanel - OnboardingWizard: add GCP Infrastructure checkbox (Step 1) + Step 4 GCP config with SA key textarea, project ID, default region, Test Connection Tests: 60 unit tests (37 credential provider + 23 monitoring) — all pass. Co-Authored-By: Arun <arun@eyriehq.com> * fix(gcp): project name display, connector UI labels, docker-compose cleanup * feat(gcp): add collector_regions to GCP onboarding and config (IW-217) - GcpCredentialsBody model: add Optional[str] collector_regions field - _save_to_db / get_gcp_settings: persist and return collector_regions - _get_gcp_masked_config / save_connector_config: thread collector_regions - OnboardingWizard: Collector Regions input in GCP step, included in setup body - GcpCredentialsPanel: Collector Regions state, edit form field, display in current-state view Empty string or None → collect from all regions (same semantics as AWS). * fix(gcp): show project display name in connector status message on save - Status message now shows 'Valid (Project: <name>)' using enriched project name - Fixed isinstance check for sa_key_json before json.loads to handle dict vs string Co-Authored-By: MG <mg@eyriehq.com> * refactor(gcp): move monitoring OAuth scope to plugin manifest (plugin-driven permissions) - Add plugins/oss/gcp-monitoring/plugin.json with gcp_scopes and gcp_roles - Add GCP_ROLES registry to gcp_sts_service.py (mirrors SERVICE_SCOPES pattern) - Update _register_gcp_scopes() in plugin_loader.py to populate GCP_ROLES from manifest - Remove hardcoded SERVICE_SCOPES.setdefault() from gcp_monitoring.py - Update tests: verify scope comes from plugin loader, not hardcoded setdefault Co-Authored-By: Arun <arun@eyriehq.com> * fix(gcp): remove implicit broad scope fallback, enforce plugin manifest declaration Delete _DEFAULT_SCOPES constant and replace the silent fallback in _get_scopes_for_service() with an explicit RuntimeError, forcing all GCP plugins to declare gcp_scopes in their manifest rather than silently inheriting a broad cloud-platform.read-only scope. Update tests to register scopes explicitly and verify the error path. --------- Co-authored-by: Manish Gupta <manish@mgupta.me> Co-authored-by: Arun <arun@eyriehq.com> Co-authored-by: MG <mg@eyriehq.com> * feat(gcp): GCP Cloud IAM plugin — service accounts, IAM bindings, custom roles (IW-223) (#151) * feat(gcp): GCP Cloud IAM plugin — collector, router, frontend panel (IW-223) - Add GCP Celery task module (collect_tasks_gcp.py): collect_gcp_resources (single service/project) + collect_gcp_resources_all (periodic sweep) - Wire GCP beat schedule and Celery include in celery_app.py - Bump CORE_VERSION to 0.14.0 in plugin_loader.py - Add google-cloud-resource-manager to requirements.txt (IW-216, IW-223) - Bump plugins/oss submodule to gcp-iam plugin with updated API: are_credentials_configured + get_gcp_projects() (aligned with PR #150) gcp_credentials.py: uses release/0.14.0 foundation from PR #150 — provider chain (Valkey → registered providers → DB → env), WIF support, per-project cache, SA key validation, gcp_sts_service scope registry. Co-Authored-By: Arun <arun@eyriehq.com> * chore: add DESIGN.md, fix GcpIamPanel MetaItem classes, mount plugins in celery override - frontend/src/DESIGN.md: EyrieHQ design guide — canonical UI patterns for plugin panels, drawers, tables, badges. Read before building any plugin frontend. - plugins/oss: bump submodule to c77c6b4 — fixes MetaItem in GcpIamPanel from bare <span className="meta-label"> to drawer-meta-item/drawer-meta-key/drawer-meta-val divs per the design system spec. - docker-compose.override.yml: add volume mounts for celery_worker and celery_beat (./backend and ./plugins/oss) so backend and plugin code changes hot-reload without a container rebuild. Co-Authored-By: Arun <arun@eyriehq.com> --------- Co-authored-by: Manish Gupta <manish@mgupta.me> Co-authored-by: Arun <arun@eyriehq.com> * feat(gcp-compute): GCP Compute Engine plugin — VM instances and persistent disks (IW-219) (#152) * chore: pin plugins/oss submodule to feat/iw-219-gcp-compute (gcp-compute plugin) * fix(gcp): use binary Valkey client for pickle-based credential cache (IW-219) get_gcp_credentials() stores google.auth credentials via pickle.dumps(). The shared Valkey client uses decode_responses=True which tries to decode the binary pickle as UTF-8, raising UnicodeDecodeError in Celery workers (the backend hit the same code path but Valkey cache was still warm from a previous request, masking the bug). Add get_binary_client() to valkey_client.py (decode_responses=False) and switch gcp_credentials.py to use it for all pickle read/write/invalidate ops. Config-cache reads (JSON strings) continue to use the text client. * fix(dashboard): hide AWS region dropdown when AWS not configured; fix gcp-compute panel crash - password_auth.py: /api/auth/regions returns [] when AWS creds absent (was always returning all ~28 AWS regions regardless of connector state) - DashboardPage.jsx: gate region selector on awsRegions.length > 0 (cosmetic guard) - plugins/oss: bump submodule to include GcpComputePanel fetcher fix (useData crash) * feat(gcp-compute): UX parity — bump plugins/oss, update EC2 attached volumes drawer (IW-219) - plugins/oss: bump submodule to ef8e0ef (GCP panel rewrite + router disk enrichment) - EC2Panel.jsx: replace Storage section with Attached Volumes (NAME|SIZE|TYPE|BOOT|MODE|STATUS) matching GCP drawer layout exactly; boot detected by device name containing 'sda' or idx===0; repositioned after the Metrics section Co-Authored-By: Arun <arun@eyriehq.com> * revert(ec2): restore EC2Panel.jsx to release/0.14.0 — volumes section was incorrectly ported from GCP semantics (IW-219) * chore(submodule): bump plugins/oss to release/0.14.0 HEAD (aca40ce) Points to the squash-merged gcp-compute plugin on the release branch instead of the feature branch tip. (IW-219) --------- Co-authored-by: Manish Gupta <manish@mgupta.me> Co-authored-by: Arun <arun@eyriehq.com> * fix: bump plugins/oss to 5b98dee (IW-235 ec2 volumes fix, includes gcp-compute) Previous pointer (aca40ce) was set by the IW-219 merge but didn't include the IW-235 ec2 changes. New HEAD (5b98dee) has both gcp-compute and the ec2 volumes/root_device collector fix on the same release/0.14.0 line. * fix(gcp-compute): bump plugins/oss to 3b2ab6c — show zone not self_link in drawer (IW-219) * feat(ec2): rename Storage to Attached Volumes, add Boot column (IW-235) Renames Storage → Attached Volumes in EC2 detail drawer. Adds Boot column using root_device from API. Keeps Volume ID and Encrypted columns. * feat(gcp-billing): GCP Billing & Cost Management plugin — submodule bump (IW-224) (#154) * chore(plugins): bump plugins/oss submodule to include gcp-billing (IW-224) Points plugins/oss at 4d9e5b3 — adds the GCP Billing & Cost Management plugin with collector, router, GcpBillingPanel, and 28 unit tests. * fix(gcp-billing): add google-cloud-bigquery dep + GCP billing env var stubs (IW-224) * chore: bump plugins/oss to 23102f7 (gcp-billing merged to release/0.14.0) * fix(gcp-billing): bump plugins/oss to af09ad5 (sidebar group fix) * fix(gcp-billing): bump plugins/oss to 19bb3a0 (auto-discover BQ, no env vars) * chore: remove obsolete GCP billing env var stubs from .env.example (auto-discovered now) --------- Co-authored-by: Arun Engineer <arun@eyriehq.com> * feat(iw-237): GCP short-lived credentials + SECRET_KEY admin warning (#157) * feat(security): GCP short-lived tokens + secret-key posture endpoint (IW-237) Cache only token-only credential wrappers in Valkey — strips the RSA private-key signer from the SA Credentials object so a Valkey compromise leaks at most a 1-hour bearer token. Applied to all three paths: SA, WIF, and ADC env fallback. Widen _is_expired() / _cache_ttl_for_creds() margin from 60 s → 300 s (5 min) so the token-only wrapper is always re-minted before the google-cloud SDK would see a near-expired token and attempt its own refresh (which raises RefreshError on a non-refreshable token-only credential). Add GET /api/admin/security-config (admin-only) returning {"secret_key_set": bool} so the frontend can surface a warning band when SECRET_KEY is unset and credential encryption falls back to the weaker DATABASE_URL-derived key. Co-Authored-By: MG <mg@eyriehq.com> * feat(admin): show non-dismissable warning band when SECRET_KEY is not set (IW-237) Fetches GET /api/admin/security-config on AdminPage mount. If secret_key_set is false, renders a full-width red warning band above the nav tabs on every admin page. Band disappears automatically once the API returns true (after admin sets SECRET_KEY and restarts). No dismiss button — impossible to hide. Fails silently if the endpoint is not yet available. Co-Authored-By: surya <surya@eyriehq.com> * fix(admin): tighten secret_key_set check and add 60s auto-refresh - Change `cfg.secret_key_set !== false` → `cfg.secret_key_set === true` for clearer intent (only show warning when flag is explicitly true). - Extract security config fetch into a stable `useCallback` and wire a 60-second `setInterval` so the warning band auto-disappears when the admin sets SECRET_KEY and restarts the backend — no page reload needed. Interval is cleared on component unmount to avoid memory leaks. * docs(env): add SECRET_KEY to .env.example with generation instructions (IW-237) The admin warning band tells operators to set SECRET_KEY but the env example didn't document the variable at all. Adds it to a new Security section with a clear explanation of the fallback risk and a one-liner to generate a strong value. * fix(encryption): add key-rotation script for SECRET_KEY migration (IW-237) When SECRET_KEY is set for the first time on an existing install, the Fernet encryption key changes and all previously-encrypted DB fields (TOTP secrets, GCP SA key, AWS credentials, ClickHouse password) become unreadable. rotate_encryption_key.py re-encrypts all affected fields from the old DATABASE_URL-derived key to the new SECRET_KEY-derived key. Dry-runs with a confirmation prompt before committing. Safe to re-run (already- rotated fields are detected and skipped). Run once after setting SECRET_KEY: cd backend && python3 rotate_encryption_key.py --------- Co-authored-by: Arun Engineer <arun@eyriehq.com> Co-authored-by: MG <mg@eyriehq.com> Co-authored-by: surya <surya@eyriehq.com> * feat(iw-236): GCP Compute — Cloud Monitoring metrics charts (CPU, Memory, Network, Disk) (#158) * feat(gcp-compute): wire GCP metrics Celery task and submodule update (IW-236) - collect_tasks_gcp.py: add collect_gcp_metrics() Celery task mirroring the AWS collect_metrics pattern but gated on GCP credentials; update collect_gcp_resources_all() to call metrics collection after resource collection. - plugins/oss: bump submodule to feat/iw-236 (collector + router + plugin.json). Co-Authored-By: Arun Engineer <arun@eyriehq.com> * feat(gcp-compute): add getGcpComputeMetrics to API client + submodule bump (IW-236) - frontend/src/api/client.js: add getGcpComputeMetrics(resourceId, hours) using query params (resource_id URL-encoded; selfLink contains slashes). - plugins/oss: bump to commit with real recharts MetricChart implementation. Co-Authored-By: Surya Engineer <surya@eyriehq.com> * chore: bump plugins/oss submodule to df0eb6a (em dash fix) Picks up the formatBytes null guard fix — em dash instead of 'N/A'. * fix(connectors): purge collected_resources + metrics on AWS/GCP disconnect (IW-236) When credentials are disconnected and replaced with a new account, the old account's collected_resources rows persist — resource IDs are account-specific so on_conflict_do_update never fires on them, leaving stale data visible. _delete_aws_config and _delete_gcp_config now dynamically resolve the connector's service types from loaded plugin manifests and delete all matching rows from collected_resources and collected_metrics before clearing AppSetting credentials. Fixes the behaviour where switching AWS accounts caused old EC2/IAM resources to remain displayed alongside new-account data. * fix(session): invalidate all sessions on AWS credential/region change When power_aws_region changes, existing sessions keep the stale region baked in at login time. This causes collected_resources queries to filter by the wrong region — the user sees data from the old region or nothing. SessionStore.invalidate_all_sessions() scans and deletes all session:* keys from Valkey. Called from _save_aws_config and _delete_aws_config so any credential or region update forces a fresh login with the correct region. Also fixes the same gap for _delete_aws_config (disconnect path). --------- Co-authored-by: Arun Engineer <arun@eyriehq.com> Co-authored-by: Surya Engineer <surya@eyriehq.com> * feat(iw-222): GCP Memorystore (Valkey) plugin (#165) * feat(gcp-memorystore): add API client methods + submodule bump (IW-222) - client.js: getGcpMemorystore, getGcpMemorystoreMetrics, refreshGcpMemorystore, gcpMemorystoreRefreshStreamUrl - plugins/oss bumped to feat/iw-222 (1f2aacb) with full plugin scaffold * fix(gcp-memorystore): update plugins/oss submodule — design system polish (IW-222) Forward submodule to commit with bg-card fix and order=36 corrections. * feat(widgets): GCP widget support — region_key + submodule bump (IW-222) - plugin-registry.js: support region_key in widget config so GCP plugins can use project_id (stored in CollectedMetric.region) instead of item.region - Bump plugins/oss submodule to daccaf0 (gcp-compute + gcp-memorystore widget sections added) Unblocks PR #165. Co-Authored-By: Arun <arun@eyriehq.com> * fix(client): restore getGcpComputeMetrics dropped by IW-222 client.js rebase; add docker-compose.override.yml for dev hot reload (IW-241) * fix(gcp-compute): bump plugins/oss to fix Cloud Monitoring instance_name filter Moves instance scoping from invalid resource.labels.instance_name to metric.labels.instance_name so GCP Cloud Monitoring queries return actual data. Co-Authored-By: Arun <arun@eyriehq.com> * chore: bump plugins/oss submodule for IW-222 memorystore metric fix Points to b3dc327 — fix gcp-memorystore collector to use memorystore.googleapis.com/* namespace instead of redis.googleapis.com/* --------- Co-authored-by: Arun Engineer <arun@eyriehq.com> * feat(iw-240): GCP Cloud SQL plugin (#166) * feat(iw-240): add GCP Cloud SQL plugin — backend + frontend - plugins/oss submodule bumped to include gcp-cloudsql (collector, router, panel, plugin.json) - frontend/src/api/client.js: add getGcpCloudSql, getGcpCloudSqlMetrics, refreshGcpCloudSql, gcpCloudSqlRefreshStreamUrl Co-Authored-By: Arun Engineer <arun@eyriehq.com> * fix(iw-240): restore Memorystore/Compute exports, fix Cloud SQL metrics query - frontend/src/api/client.js: rebased onto origin/release/0.14.0 (brings in getGcpMemorystore, getGcpMemorystoreMetrics, gcpMemorystoreRefreshStreamUrl, refreshGcpMemorystore from IW-222 and getGcpComputeMetrics from IW-236); removed duplicate getGcpComputeMetrics entry and consolidated all GCP methods into the GCP section (Bugs 1 & 2) - plugins/oss/gcp-cloudsql/collector.py: replace non-existent cloudsql.googleapis.com/database/queries with cloudsql.googleapis.com/database/postgresql/transaction_count — the correct PostgreSQL transactions metric (Bug 3) - collect_tasks_gcp.py: collect_gcp_metrics task now present (from IW-236 rebase) so GCP metrics run via collect_gcp_resources_all on schedule Co-Authored-By: Arun Engineer <arun@eyriehq.com> * fix(iw-240): resolve widget region from collected_resources for GCP metric lookup GCP plugins store collected_metrics with region=project_id (e.g. glass-effect-495104-d3), not the GCP location string (e.g. asia-south1). When a Cloud SQL widget was created the region field was saved as the GCP location, so the dashboard widget data query compared "glass-effect-495104-d3" (metrics) against "asia-south1" (widget) and returned zero rows — silent empty charts. Fix: get_widget_data now looks up CollectedResource by service_type + resource_id to get the canonical region used by the collector, and falls back to widget.region if no resource row exists (e.g. AWS services where region is a real AWS region and there is no CollectedResource row). This fix is safe for all service types: AWS widgets have no CollectedResource row (region fallback), GCP widgets now resolve correctly regardless of what was stored in dashboard_widgets. Co-Authored-By: Arun Engineer <arun@eyriehq.com> * chore: bump plugins/oss to release/0.14.0 tip (7160b69) --------- Co-authored-by: Arun Engineer <arun@eyriehq.com> * fix(gcp): icon-only refresh buttons across all GCP panels (#167) * fix(gcp): bump plugins/oss submodule — icon-only refresh buttons across GCP panels Updates submodule to remove "Refresh" text from GcpComputePanel and GcpBillingPanel refresh buttons, matching the icon-only style on EC2. * fix(gcp): bump plugins/oss — 3-state refresh button for Compute and Billing panels Adds Check icon + "Refreshed" done state (1500ms) to GcpComputePanel and GcpBillingPanel, matching the pattern already in Memorystore, CloudSQL, IAM, and EC2. Also fixes both to use addEventListener("refresh_done") instead of the incorrect es.onmessage + string compare. * docs(design): document correct 3-state refresh button pattern - Icon-only at rest (no persistent text label) - Orange spinning icon during sync (.spinning CSS class) - Green tick + "Refreshed" for 1500ms on done (refresh-done span) Includes required imports (Check) and state setup (showRefreshed). * fix(gcp): add missing GCP API methods to client.js (Memorystore, Cloud SQL, Compute metrics) The Memorystore and Cloud SQL panels crashed on load because the api object in client.js had no methods for these services — api.getGcpMemorystore(), api.getGcpCloudSql(), and their refresh/metrics counterparts were all missing. This caused a TypeError at panel mount, which manifested as the panels not opening and the Compute metrics page erroring when trying to call api.getGcpComputeMetrics() which was also absent. Adds all 12 missing methods with correct URL patterns matching the backend router endpoints in plugins/oss (path param for Memorystore/CloudSQL metrics, query param for Compute metrics per its router spec). * chore: bump plugins/oss to release/0.14.0 tip (7160b69) --------- Co-authored-by: Arun Engineer <arun@eyriehq.com> * feat(IW-243): plugin-driven API client + AWS panels migrated to submodule (#168) * feat(IW-243): migrate AWS panels to plugins submodule, plugin-driven API client Move all 7 AWS service frontend panels out of frontend/src/components/ and into the plugins/oss submodule (eyriehq/plugins PR #34). Each plugin directory now ships its own Panel JSX + manifest.js, exactly matching the GCP plugin pattern. Changes in this commit: frontend/src/components/ (deleted): EC2Panel.jsx, RDSPanel.jsx, DocumentDBPanel.jsx, ElastiCachePanel.jsx, LBPanel.jsx, SESPanel.jsx, SecretsPanel.jsx, DatabasesPanel.jsx frontend/src/api/client.js: - Remove 40+ hardcoded per-plugin API methods (getEC2, getRDS, getDocDB, getElastiCache, getLBs, getGcpCompute*, getGcpMemorystore*, getGcpCloudSql*, getSES, getSecrets, refreshXxx, xxxStreamUrl, etc.) - Add api.plugin(slug) — reads endpoint templates from plugin.json api blocks at runtime, making new plugins zero-touch in client.js - Add interpolate() helper for {param} URL template expansion - Keep getSESIdentities, getSESSuppressionSearch, postSESSuppressionRemoveStream, getSecretValue as explicit methods — these don't fit the 5-key plugin() shape (paginated browser, streaming remove, ARN-keyed secret value fetch) - Update SES paths /api/ses/ -> /api/aws/ses/ and Secrets /api/secrets/ -> /api/aws/secrets/ frontend/src/plugin-registry.js: - Remove CORE_SERVICE_COMPONENTS / CORE_SERVICE_ICONS fallback — AWS panels are now full plugins discovered at build time via setup-plugins.sh - Add _pluginApiMap and getPluginApi(slug) — consumed by api.plugin() in client.js - Store full plugin metadata (including api block) in _pluginApiMap at loadActivePlugins time backend/app/core/plugin_loader.py: - Add "api" to safe_keys in get_loaded_plugins() and get_plugins_with_status() so the api block reaches the frontend /api/system/plugins response plugins/oss submodule: - Bumped to feat/iw-243-aws-components-to-submodule tip (4857f04) - Adds EC2Panel, RDSPanel, DocumentDBPanel, DatabasesPanel, ElastiCachePanel, LBPanel, SESPanel, SecretsPanel + manifest.js for each AWS plugin Result: frontend/src/components/ has zero plugin panel files. Adding a new plugin now requires zero changes to client.js, plugin-registry.js, or any app core file. Co-Authored-By: Arun Engineer <arun@eyriehq.com> * chore: bump plugins/oss submodule to f5d10e3 (IW-243) Picks up DatabasesPanel composite + rds/manifest.js fix from feat/iw-243-aws-components-to-submodule (plugins PR #34). * chore: bump plugins/oss to release/0.14.0 tip after PR #34 merge --------- Co-authored-by: Arun Engineer <arun@eyriehq.com> * feat(gcp-billing): fix refresh/stream channel alignment and countdown (IW-238) (#169) Bump plugins/oss submodule to include: - POST /refresh: use apply_async(countdown=0) and DB-resolved billing account ID so the Valkey publish key matches GET /refresh/stream - GET /refresh/stream: align channel key with same DB fallback ('default') to guarantee the publish/subscribe pair always connects - Tests: POST /refresh (503 guard, dispatch args, channel key fallback) and GET /refresh/stream (content-type) Frontend (GcpBillingPanel) was already correct on release/0.14.0 via IW-240. Co-authored-by: Arun Engineer <arun@eyriehq.com> * feat(iw-245): activity bar navigation — Dashboard, Observability, AWS, GCP (#171) - New ActivityBar component (56px) with 4 icon-only items - Dashboard and Observability: direct navigation - AWS and GCP: navigate to first available enabled service - Official AWS/GCP SVG logos with dark-theme filter - Sidebar filtered by connector (aws/gcp) based on URL prefix - /aws/<service> and /gcp/<service> URL structure - Activity bar highlight derived from URL on load/refresh - Instant JS-positioned tooltips reusing .sidebar-tooltip class - connector field added to plugin-registry.js tab map - Collapse toggle moved to bottom of secondary sidebar Co-authored-by: Surya <surya@eyriehq.com> * fix(gcp-billing): thread actual BQ currency through API and frontend, fix refresh endpoint (IW-246) (#174) - plugins/oss bumped e7a4878 → 24c1b89 (release/0.14.0 tip post-PR#36) - Collector reads ANY_VALUE(currency) from BQ; stores ISO 4217 code in attrs - summary + by-service endpoints pass currency through instead of hardcoding USD - fmtUSD/fmtUSDShort accept currency param; Intl.NumberFormat handles symbol natively - Refresh endpoint passes GCP project ID (not billing account ID) to Celery task Plugins PR: eyriehq/plugins#36 Co-authored-by: Arun <arun@eyriehq.com> * feat(gcp-gke): add GKE plugin — clusters, nodes, per-node metrics (IW-218) (#175) plugins/oss bumped 24c1b89 → 5d1906d (release/0.14.0 tip post-PR#38) Plugins PR: eyriehq/plugins#38 Co-authored-by: Arun <arun@eyriehq.com> * feat(gcp-lb): Cloud LB plugin + app fixes (IW-221) (#176) plugins/oss: 5d1906d → c704666 (PR #39) - GCP Cloud Load Balancing plugin (IW-221) - gcp-compute: resource_id in response, metrics route order fix - gcp-lb: {lb_id} not :path, resource_id = short name - requires_mfa: true on IAM + Billing for all cloud providers App fixes: - plugin_loader: MFA check at request time (not startup) so _loaded_plugins is complete - nginx.conf: Docker DNS resolver (127.0.0.11) prevents 502 after backend restarts Co-authored-by: Arun <arun@eyriehq.com> * feat(gcp-secret-manager): add Secret Manager plugin + GKE dashboard widget fix (IW-247) (#177) plugins/oss: c704666 → 4e4776d (plugins PR #41) - GCP Secret Manager plugin with Load Value, RelativeTime, CopyButton, AgeBadge - /value endpoint with secretAccessor permission gate - gcp-gke widget resource_id_key fixed to use full path (dashboard charts now load) Co-authored-by: Arun <arun@eyriehq.com> * feat(iw-248): plugin reorg — vendor groups, namespaced service types, build fixes (#178) * feat(iw-248): plugin reorganization — vendor groups, namespaced service types, build fixes Reorganize plugins/oss into aws/, gcp/, observability/ vendor groups with namespaced service_type names (aws/ec2, gcp/compute, observability/metrics). Includes all app-side fixes to handle both old flat names and new namespaced names. Backend: - plugin_loader.py: discover nested vendor group dirs; guard entry_point for manifest-only stubs; accept short and namespaced service names in require_mfa check - admin_plugins.py: use {name:path} route param so slashes in plugin names (aws/ec2) work - aws.py: accept both "ses" and "aws/ses" in STS service name check Frontend build: - setup-plugins.sh: wipe src/plugins/ before copy, scan vendor group subdirs - Dockerfile: explicit rm -rf src/plugins + vendor-aware copy loop - .dockerignore: exclude src/plugins/, node_modules, __pycache__ from build context - Remove src/plugins/.gitignore and .gitkeep (generated dir, not tracked) Frontend runtime: - plugin-registry.js: register short + hyphenated aliases in _pluginApiMap; use namespaced observability plugin names; fall back p.pluginName → p.id for component/icon lookup in getServiceTabs - dashboard-tab-filter.js: accept namespaced (aws/ec2) and short (ec2) service names; legacy "metrics" approval OR observability/* grants Explorer tab - ConnectorTab.jsx: encodeURIComponent plugin name in activate/deactivate API calls - AgentsPanel.jsx: treat 502/503/404 as not-configured rather than surfacing error toast Submodule: - plugins/oss: bump to a0ab7e4 (feat/iw-248-plugin-reorg-v2, merged to plugins/release/0.14.0) Co-authored-by: Arun <arun@eyriehq.com> * fix: IW-248 startup migration, GCP project_id collectors, CloudWatch Describe* - database.py: add IW-248 startup migration to rename flat service_type values (ec2, gcp-compute, metrics, …) to namespaced format (aws/ec2, gcp/compute, observability/metrics, …) across plugin_subscriptions, collected_resources, collected_metrics, dashboard_widgets, collected_alarms, collected_health_events, and ServiceRole.services / sts_services JSON arrays. Idempotent on restart. - sts_service.py: add cloudwatch:Describe* to _CLOUDWATCH_STATEMENT so collect_alarms can call DescribeAlarms without AccessDenied from the STS session policy. - plugins/oss → 0b341e6: GCP collectors now always read project_id from configured GCP credentials instead of using the AWS region arg passed by collect_tasks (which caused all GCP metric queries to use ap-south-1 as the GCP project ID). * fix: cover elb→aws/lb and databases→aws/rds in IW-248 migration; bump plugins submodule - database.py: add elb→aws/lb and databases→aws/rds to IW-248 startup migration (historic service names that were missed in the initial migration list) - plugins/oss → 28965ab: aws/lb and aws/rds plugin.json services field updated to namespaced names; GCP project_id fix included in previous commit * fix(iw-248): bump plugins submodule to 3bf0b1c — SERVICE_TYPE constants updated to namespaced format * fix: gcp-cloudsql alias in plugin-registry + memorystore/cloud-sql api blocks - plugin-registry.js: register stripped-hyphen alias (gcp-cloudsql) for plugins whose short name contains a hyphen (gcp/cloud-sql → gcp-cloudsql) so panel legacy slugs resolve correctly - plugins/oss → 97081de: api blocks added to memorystore and cloud-sql * fix(billing): bump plugins to f0ab0f1 — SSE channel fix + 30s timeout * fix(iw-248): GCP metrics schedule, startup migration cleanup, session handling celery_app.py: - Add collect-gcp-metrics-per-schedule beat entry so GCP Cloud Monitoring metrics run on the same cadence as AWS metrics (every ~5 min), not only during the 6h resource collection cycle. collect_tasks.py: - _get_collector_service_types(): filter out GCP service types so collect_metrics_all only dispatches AWS plugins. GCP metrics are now handled exclusively by collect_gcp_metrics_all. collect_tasks_gcp.py: - Add collect_gcp_metrics_all task: iterates all active GCP metrics collectors across configured projects; called by the new beat entry. database.py: - Move collected_metrics dedup migration to after create_all() so it is a no-op on fresh installs (table exists but is empty). - Trim GCP entries from _RENAMES — never shipped to prod before IW-248 so no real install has data under old gcp-* names. - Clarify comment on omitted GCP renames. aws_credentials.py: - _load_from_db: explicit rollback + safe close in except/finally so psycopg2 doesn't error on rollback when session is in a bad state (happens when workers start before DB tables are ready). * fix: split _get_collector_service_types into resource/metrics variants; fix GCP exclusion - _get_collector_service_types(): uses get_plugin_collectors() (resource collectors), filters out GCP — for collect_resources_all - _get_metrics_service_types(): uses get_plugin_metrics_collectors() (metrics collectors), filters out GCP — for collect_metrics_all and collect_metrics_all_finish - collect_metrics_all and finish callback updated to use _get_metrics_service_types() - plugins/oss → 2032b4d: AWS router refresh tasks use namespaced service names * fix: bump plugins to 0f25710 — AWS service_type query strings fixed * fix: bump plugins to 3301ceb — EC2 collector service_type fixed * fix: bump plugins to 7ecd7c9 — AWS SSE refresh channel keys fixed * fix: bump plugins to 95a016a — RDS manifest uses RDSPanel not composite * fix: prefix vendor group plugin dirs to prevent aws/iam vs gcp/iam collision Both aws/iam and gcp/iam were being copied to src/plugins/iam/ — GCP overwrote AWS since gcp/ sorts after aws/ alphabetically. Same for aws/lb vs gcp/lb. Result: IAMPanel and LBPanel were absent from the built JS bundle; aws/iam and aws/lb never appeared in the sidebar. Fix: when copying vendor group plugins, prefix the destination dir with the group name (aws-iam, gcp-iam, aws-lb, gcp-lb) in both Dockerfile and setup-plugins.sh. The codegen still uses plugin.json services field as the registry key (aws/iam, gcp/iam) — only the dir name changes. * fix: normalize STS service short name to namespaced before assume_role Routers call require_sts_session(request, 'ses') / 'cost' / 'secrets' with short names. assume_role_for_services does SERVICE_POLICIES[service] which keys on 'aws/ses' post-IW-248 — not found → ValueError → 500. Fix: after auth check passes (which already accepts short names via split('/').pop()), resolve the short name to its full namespaced form from the user's sts_services list before calling get_or_create_sts_session. * fix: replace slash in STS roleSessionName — aws/cost → aws-cost AWS requires roleSessionName to match [\w+=,.@-]* (no slashes). After IW-248 service names are namespaced (aws/cost, aws/ses) which produced invalid session names like iw-admin-aws/cost → ValidationError. * chore: bump plugins/oss submodule to 09cedc3 (squash merge of #43 into release/0.14.0) --------- Co-authored-by: Arun <arun@eyriehq.com> * fix(iw-249): plugin activate/deactivate updates sidebar without page reload After toggling a plugin, PluginsPanel and ConnectorTab dispatch a custom 'plugin-state-changed' event. DashboardPage listens, re-calls loadActivePlugins(), and flips pluginsLoaded to trigger a re-render — so the sidebar tab list reflects the new active plugin set immediately without a full page reload. * fix(iw-249): plugin toggle no longer flashes full loading state load() now accepts a silent flag — post-toggle refreshes skip setLoading(true) so the ConnectorTab doesn't unmount/remount (which looked like a full page reload). The plugin card updates its enabled state in-place. * feat(iw-249): spinner animation on plugin toggle button While activate/deactivate is in-flight the button shows a spinning indicator and 'Activating…' / 'Deactivating…' label, and is disabled to prevent double-clicks. Clears on completion. * chore: bump plugins submodule — _metrics_start_end service_type fix * fix: dispose DB connection pool in worker_process_init to prevent stale socket errors Forked Celery workers inherit the parent's connection pool. After a container restart the inherited connections are in an error state (PGRES_TUPLES_OK / tuple index out of range). engine.dispose() in worker_process_init forces each worker to open fresh connections on first use. * fix: point plugins submodule to correct release/0.14.0 tip (2d5859f) Previous bump accidentally resolved to 52c7453 (old README commit on main) instead of the IW-248 + _metrics_start_end fix branch. Corrected. * fix: bump plugins — EC2 metrics period 3600→300 * fix: pre-seed Admin connector tabs so AWS/GCP/Observability render immediately Previously connectors started as [] causing a flash where only Users/Roles showed until /api/admin/connectors returned (~200ms). Pre-seeding with the known three connectors (disconnected placeholder) makes all tabs render on mount; the API call then updates status/plugins in-place. * revert: remove connector pre-seed — caused duplicate API calls (IW-251) Pre-seeding connectors mounted all 3 ConnectorTabs simultaneously causing aws×2 and gcp×3 calls. Proper lazy-load fix tracked in IW-251. * feat(iw-250): group widget service picker by vendor (AWS / GCP) The 'Add widget' dropdown was a flat alphabetical list of all services. Now uses <optgroup> sections keyed by the namespace prefix (aws/, gcp/) so services are grouped as AWS and GCP — matching the sidebar layout. Both the Add and Edit modals are updated. * feat(iw-251): Admin lazy-load — hardcode connector tabs, remove upfront /connectors fetch (#180) * feat(iw-251): Admin lazy-load — hardcode connector tabs, remove upfront /connectors fetch Previously AdminPage called /api/admin/connectors on mount to build the tab list. This caused: - AWS+GCP+Observability ConnectorTabs all mounting simultaneously (wrong) - aws called ×2, gcp called ×3 when connectors state updated post-fetch - setup-status, security-config, connectors, plugins, me all firing on load Fix: hardcode the 3 connector tabs (always Observability, AWS, GCP — they never change). Each ConnectorTab now fetches /api/admin/connectors/{type} lazily only when that tab is first activated, via its own load() useEffect. On mount: only setup-status + security-config fire. On tab activation: exactly one /api/admin/connectors/{type} call. * fix: agents inventory returns 200 not_configured instead of 502 Any ClickHouse query failure (missing otel_logs table, schema not set up, transient error) now returns {status: not_configured} with 200 instead of raising HTTP 502. The AgentsPanel already handles not_configured gracefully; the 502 was showing as a red error in DevTools even though the UI was silent. --------- Co-authored-by: Arun <arun@eyriehq.com> * fix: upgrade path fixes — GCP project_id discovery, plugin routing collision, single-project OSS (#181) Discovered during v0.13.2 → v0.14.0 upgrade validation. gcp_credentials.py: - _get_primary_project_id(): auto-discover project_id from SA key JSON when no explicit projects list or primary_project_id is set. Supports the UI hint 'leave empty to use project from SA key'. - get_gcp_projects(): same fallback — if projects list is empty and no primary_project_id, extract from sa_key_json. Fixes 'GCP not configured or no project set' error after upgrade when user never filled the Additional Project IDs field. GcpCredentialsPanel.jsx: - Remove 'Additional Project IDs' field. Multi-project support is a future commercial feature; OSS always uses a single project from the SA key. The backend now auto-discovers it so no manual entry needed. plugin-registry.js: - Detect id collisions between vendors (aws/lb vs gcp/lb, aws/iam vs gcp/iam both resolve to short id 'lb'/'iam'). Resolve by using the hyphenated full name (aws-lb, gcp-lb) for all colliding plugins so sidebar routing is unambiguous. Previously clicking Cloud Load Balancing routed to AWS LB panel. dashboard-tab-filter.js: - Accept collision-resolved hyphenated ids (aws-lb, gcp-lb) by matching them back to their namespaced service entry in approvedServices. Co-authored-by: Arun <arun@eyriehq.com> * feat(iw-252): context-aware cloud picker in navbar (#182) * feat(iw-252): context-aware cloud picker in navbar Replace the unconditional AWS region dropdown with a cloud-aware picker: - Dashboard / Observability tabs → no picker (clean header) - AWS tabs → [AWS logo] + region dropdown (existing switch-region UX) - GCP tabs → [GCP logo] + project dropdown (single-project for OSS; dropdown shape ready for multi-project commercial) GCP project list is fetched on mount via adminGetGcpSettings and falls back silently for non-admin sessions or unconfigured GCP. Also completes GcpCredentialsPanel cleanup: removes all collectorRegions state, display, and body-builder references — GCP uses aggregatedList (project-wide), so per-region filtering is not applicable. * fix: aws-logo white text for dark mode (#252F3E → #FFFFFF) --------- Co-authored-by: MG <mg@eyriehq.com> * chore: bump plugins submodule to v0.14.0 squash tip (21bd611) --------- Co-authored-by: eyriehq-bot[bot] <269943525+eyriehq-bot[bot]@users.noreply.github.com> Co-authored-by: Arun <arun@eyriehq.com> Co-authored-by: MG <mg@eyriehq.com> Co-authored-by: surya <surya@eyriehq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
maintip (e7a4878)766e3ee, missing IW-238/IW-243/IW-244 mergesANY_VALUE(currency), router passes currency through, frontendfmtUSD/fmtUSDShortcurrency-aware viaIntl.NumberFormatTest plan