0.8.0-dev8: second sensory publisher — Meraki webhook → NATS

[stevenca]

Summary

First webhook-side sensory publisher. The web process now publishes sensory.* events when Meraki Dashboard webhooks arrive:

Meraki Dashboard
   │  POST /webhooks/meraki/<tenant>
   │  HMAC-SHA256 (X-Cisco-Meraki-Signature)
   ▼
web process (FastAPI)
   ├─ HMAC verify
   ├─ map_meraki_payload() → [MerakiSensoryEvent, ...]
   ├─ SensoryPublisher.publish(...) → NATS
   │    subject: sensory.link_down.meraki_webhook.meraki:<serial>|<port>
   └─ BackgroundTasks: targeted adapter sync (pre-dev8 behavior, retained)
                                                ▼
                                         worker process
                                            ReflexRunner
                                              ↓
                                     LinkDownHandler (dedup against SNMP!)
                                              ↓
                                     Neo4jReflexEventSink
                                              ↓
                                     (:ReflexEvent)-[:AFFECTS]->(:Device)

A port flap detected at the Meraki edge now reaches Neo4j in 50–200ms instead of waiting up to a full SNMP poll cycle (5 minutes).

This is the first release where the dedup store added in 0.8.0-dev3 actually does anything in production. The same port flap will now be observed by both the Meraki webhook (immediate) and the SNMP poller (next cycle), and the second arrival within the 60s dedup window collapses to a single :ReflexEvent.

What landed

New

• `netcortex/webhooks/event_publisher.py` — web-process bus coordination. One `NatsEventBus` per process on `app.state.event_bus`; one cached `SensoryPublisher` per source on `app.state._sensory_publishers`. Graceful degradation when `NATS_URL` is unset.
• `netcortex/webhooks/meraki_events.py` — pure Meraki-dialect-to-our-taxonomy mapper:
  • `Port connectivity` / `Switch port connection changed` → `link_up` / `link_down`
  • `Uplink status change` → `link_up` / `link_down`
  • `IDS alerted` / `Malware detected` → `security_alert`
  • `Settings changed` / `Configuration change` → `config_change`
  • Unknown alertTypes log `webhook.meraki.unmapped_alert_type` and skip (so we grow coverage based on real traffic)
• Target shape: `meraki:|` — matches `Device.id` so the dev7 `:AFFECTS` resolver lands the edge automatically.

Changed

• `webhooks/meraki.py::handle_meraki_webhook` accepts an optional `publisher`; pre-dev8 behavior preserved when absent.
• `webhooks/router.py` resolves the publisher per-request via `get_publisher(request.app, "meraki_webhook")`.
• `main.py` lifespan inits the NATS bus on startup and closes it on shutdown (both best-effort).
• Webhook response body now includes `sensory_events_published: ` for operator at-a-glance verification.

Tests (28 new)

• `tests/webhooks/test_meraki_events.py` — 18 mapper unit tests
• `tests/webhooks/test_meraki_webhook_route.py` — 10 HTTP integration tests (HMAC valid/invalid/missing, signature prefix variants, malformed JSON, no-secret degradation, unknown alertType, IDS publish, publisher-unavailable, bus-failure-during-publish)
• First FastAPI `TestClient` test module in the repo. Pattern (capturing bus + minimal app + secret fixture) is generalized via `conftest.py` so dev9 (ThousandEyes), dev10 (Nexus Dashboard), dev11 (cdFMC) can reuse it.

Operational notes

• Web deployment already has `NATS_URL` via Helm — no chart change required.
• HMAC secrets continue to live at `netcortex/webhooks/meraki/` in AWS SM.
• After deploy, find webhook-sourced events:
  ```cypher
  MATCH (e:ReflexEvent {source: 'meraki_webhook'})
  WHERE e.recorded_at > timestamp() - 3600000
  RETURN e.subject, e.target, e.outcome
  ORDER BY e.observed_at_ms DESC
  ```
• See dedup in action (webhook + SNMP poll observing the same flap, second one `skipped`):
  ```cypher
  MATCH (e1:ReflexEvent), (e2:ReflexEvent)
  WHERE e1.target = e2.target
  AND e1.source = 'meraki_webhook' AND e2.source = 'snmp_poll'
  AND e1.event_class = e2.event_class
  AND abs(e1.observed_at_ms - e2.observed_at_ms) < 60000
  RETURN e1.target, e1.outcome AS webhook_outcome, e2.outcome AS snmp_outcome
  ```

Deferred to subsequent releases

• dev9: ThousandEyes webhook receiver
• dev10: Nexus Dashboard webhook receiver (adds `nexus_dashboard_webhook` to SOURCES)
• dev11: cdFMC webhook receiver (adds `fmc_webhook` to SOURCES)
• Later: JetStream durable subscriptions for at-least-once across worker restarts

Test plan

• CI: all 8 jobs pass
• Build image `localhost:32000/netcortex:0.8.0-dev8`
• Helm upgrade on `cpn-ful-netcortex1`
• Verify web pod startup log shows `web.sensory_publisher_ready nats_url=...`
• Send a synthetic POST through one of the configured Meraki webhook tenants and verify a `:ReflexEvent` with `source: 'meraki_webhook'` lands
• Wait for a real port flap detected by both Meraki and SNMP within the dedup window — verify one is `outcome: 'logged'` and the second is `outcome: 'skipped'`

Made with Cursor