fix: align Cisco Meraki filter and rule contracts

Author: kryonsx

filters/audits/cisco-meraki.md

@@ -0,0 +1,42 @@
+# Cisco Meraki normalization and rule review
+
+Avoid fabricated success for observations/pending VPN; preserve rule input fields and handle retrospective AMP events without invented source IP.
+
+This draft targets UTMStack `v11`. It contains 1 filter changes
+and 1 rule changes for this technology only. Review covered
+1 filter configurations and 7 matching shipped rule files.
+Unchanged rules are listed in the regression manifest; they are not duplicated in the diff.
+
+## Contract and validation
+
+- Compared exact standard names/types with go-sdk v1.1.31 and the supplied UTMStack dictionaries.
+- Checked documented pipeline ordering, rename/move behavior, open vendor log fields,
+  event-side versus alert-side fields, and surviving fields used by affected rule predicates/history/grouping.
+- Strict SDK configuration decoding and actual CEL compilation pass for this scope.
+- 6 synthetic normalization cases pass, including SDK Event conversion and any
+  trigger predicate assertions recorded in the manifest.
+- The scoped alerts module tests and `git diff --check` pass with the shared contract runner applied.
+
+The shared alert-contract PR supplies the reusable Go runner for the manifest in
+`plugins/alerts/testdata/filter-contracts/cisco-meraki.json`. Apply that support before running `go test ./...` in `plugins/alerts`.
+
+The changed rules also require the shared alert-grouping fix to resolve `lastEvent.*` values correctly at runtime.
+
+The model starts from synthetic extraction results. It does not run complex grok,
+JSON/KV/XML/CSV extraction, time conversion, dynamic plugins, historical OpenSearch
+queries, or the closed EventProcessor. Raw vendor logs and resulting alerts must
+still be checked in staging before rollout. No customer false-positive reduction
+has been measured and no production rollout is included.
+
+Current Meraki security events retain their group/message for the AMP consumer. Retrospective malicious dispositions do not report a source IP, so the rule does not invent or require one; grouping includes dataSource. Source: https://documentation.meraki.com/General_Administration/Monitoring_and_Reporting/Syslog_Event_Types_and_Log_Samples
+
+## References
+
+- [SDK schema](https://github.com/threatwinds/go-sdk/blob/v1.1.31/plugins/plugins.proto)
+- [Filter steps](https://github.com/threatwinds/go-sdk/wiki/Filter-Steps-Reference)
+- [Standard event schema](https://github.com/threatwinds/go-sdk/wiki/Standard-Event-Schema)
+- [Rule implementation](https://github.com/threatwinds/go-sdk/wiki/Implementing-Rules)
+
+`afterEvents`, empty noncapturing grok names, supported numeric strings, and custom
+`log.*` fields are accepted. Existing textual protocol casing and vendor action names
+are preserved unless a concrete consumer mismatch requires correction.

filters/cisco/meraki.yml

@@ -103,10 +103,10 @@ pipeline:
           where: '!equals("log.controlFlag", "Init")'
       # Adding action result
       - add:
-          function: 'string'
+          function: string
           params:
             key: actionResult
-            value: 'accepted'
+            value: success
           where: equals("log.connectivity", "true")
       # ........................................
       # event uplink connectivity change
@@ -169,10 +169,10 @@ pipeline:
           where: '!equals("log.controlFlag", "Init") && contains("action", "Cellular connection")'
       # Adding action result
       - add:
-          function: 'string'
+          function: string
           params:
             key: actionResult
-            value: 'accepted'
+            value: success
           where: equalsIgnoreCase("log.connectionState", "up")
       # ........................................
       # Event: event, dhcp no offers
@@ -236,10 +236,10 @@ pipeline:
           where: '!equals("log.controlFlag", "Init") && contains("log.genericEvent", "dhcp lease")'
       # Adding action result
       - add:
-          function: 'string'
+          function: string
           params:
             key: actionResult
-            value: 'accepted'
+            value: success
           where: contains("log.genericEvent", "dhcp lease")
       # ........................................
       # Event: event, HTTP GET requests in Meraki MX Security Appliance
@@ -270,12 +270,6 @@ pipeline:
           source: log.genericEvent
           where: '!equals("log.controlFlag", "Init") && startsWith("log.genericEvent", "src") && log.merakiGroup=="urls"'
       # Adding action result
-      - add:
-          function: 'string'
-          params:
-            key: actionResult
-            value: 'accepted'
-          where: '!equals("log.controlFlag", "Init") && startsWith("log.genericEvent", "src") && log.merakiGroup=="urls"'
       # ........................................
       # Event: flows, IP session initiated in Meraki MX Security Appliance
       # First variant
@@ -342,10 +336,10 @@ pipeline:
           where: '!equals("log.controlFlag", "Init") && startsWith("log.genericEvent", "src") && equals("log.merakiGroup", "flows")'
       # Adding action result
       - add:
-          function: 'string'
+          function: string
           params:
             key: actionResult
-            value: 'accepted'
+            value: success
           where: '!equals("log.controlFlag", "Init") && startsWith("log.genericEvent", "src") && equals("log.merakiGroup", "flows") && (startsWith("log.pattern", "0") || startsWith("log.pattern", "allow") || startsWith("log.pattern", "Allow"))'
       - add:
           function: 'string'
@@ -501,31 +495,31 @@ pipeline:
           where: '!equals("log.controlFlag", "Init") && startsWith("log.genericEvent", "Site") && equals("log.merakiGroup", "events")'
       # Adding action result
       - add:
-          function: 'string'
+          function: string
           params:
             key: actionResult
-            value: 'accepted'
-          where: '!equals("log.controlFlag", "Init") && startsWith("log.genericEvent", "Site") && equals("log.merakiGroup", "events") && (contains("log.genericEvent", "queued due to no phase 1") || contains("log.genericEvent", "queued due to no phase1") || contains("log.genericEvent", "established"))'
+            value: success
+          where: '!equals("log.controlFlag", "Init") && startsWith("log.genericEvent", "Site") && equals("log.merakiGroup", "events") && contains("log.genericEvent", "established")'
       # ........................................
       # Event: event spanning-tree guard state change in Meraki MS Switches
       - grok:
           patterns:
-            - fieldName: log.irrelevant
-              pattern: 'Port'
-            - fieldName: origin.port
-              pattern: '{{.integer}}'
-            - fieldName: log.irrelevant
-              pattern: 'received an'
-            - fieldName: protocol
-              pattern: '{{.data}}BPDU'
-            - fieldName: log.irrelevant
-              pattern: 'from'
-            - fieldName: origin.mac
-              pattern: '{{.data}}so'
-            - fieldName: log.irrelevant
-              pattern: 'the port was'
-            - fieldName: actionResult
-              pattern: '{{.greedy}}'
+          - fieldName: log.irrelevant
+            pattern: Port
+          - fieldName: origin.port
+            pattern: '{{.integer}}'
+          - fieldName: log.irrelevant
+            pattern: received an
+          - fieldName: protocol
+            pattern: '{{.data}}BPDU'
+          - fieldName: log.irrelevant
+            pattern: from
+          - fieldName: origin.mac
+            pattern: '{{.data}}so'
+          - fieldName: log.irrelevant
+            pattern: the port was
+          - fieldName: log.merakiResult
+            pattern: '{{.greedy}}'
           source: log.genericEvent
           where: '!equals("log.controlFlag", "Init") && startsWith("log.genericEvent", "Port") && equals("log.merakiGroup", "events")'
       - trim:
@@ -853,10 +847,10 @@ pipeline:
           where: '!equals("log.controlFlag", "Init") && startsWith("log.genericEvent", "type") && equals("log.merakiGroup", "events") && contains("log.genericEvent", "radio") && contains("log.genericEvent", "channel") && contains("log.genericEvent", "auth_neg_dur") && contains("log.genericEvent", "last_auth_ago")'
       # Adding action result
       - add:
-          function: 'string'
+          function: string
           params:
             key: actionResult
-            value: 'accepted'
+            value: success
           where: '!equals("log.controlFlag", "Init") && startsWith("log.genericEvent", "type") && equals("log.merakiGroup", "events") && contains("log.genericEvent", "radio") && contains("log.genericEvent", "channel") && contains("log.genericEvent", "auth_neg_dur") && contains("log.genericEvent", "last_auth_ago")'
       # ........................................
       # Event: event 802.1X (all events) in Meraki MR Access Points
@@ -1008,10 +1002,10 @@ pipeline:
           where: '!equals("log.controlFlag", "Init") && startsWith("log.genericEvent", "type") && equals("log.merakiGroup", "events") && contains("log.genericEvent", "ip") && contains("log.genericEvent", "vap") && contains("log.genericEvent", "duration") && contains("log.genericEvent", "download") && contains("log.genericEvent", "upload")'
       # Adding action result
       - add:
-          function: 'string'
+          function: string
           params:
             key: actionResult
-            value: 'accepted'
+            value: success
           where: '!equals("log.controlFlag", "Init") && startsWith("log.genericEvent", "type") && equals("log.merakiGroup", "events") && contains("log.genericEvent", "ip") && contains("log.genericEvent", "vap") && contains("log.genericEvent", "duration") && contains("log.genericEvent", "download") && contains("log.genericEvent", "upload")'
       # ........................................
       # Event: event wireless packet flood detected in Meraki MR Access Points
@@ -1123,32 +1117,32 @@ pipeline:
       # Event: flows, flow allowed by Layer 3 firewall in Meraki MR Access Points
       - grok:
           patterns:
-            - fieldName: actionResult
-              pattern: '{{.word}}'
-            - fieldName: log.irrelevant
-              pattern: 'src(\s)?='
-            - fieldName: origin.ip
-              pattern: '{{.ipv4}}|{{.ipv6}}'
-            - fieldName: log.irrelevant
-              pattern: 'dst(\s)?='
-            - fieldName: target.ip
-              pattern: '{{.ipv4}}|{{.ipv6}}'
-            - fieldName: log.irrelevant
-              pattern: 'mac(\s)?='
-            - fieldName: origin.mac
-              pattern: '{{.data}}protocol'
-            - fieldName: log.irrelevant
-              pattern: '='
-            - fieldName: protocol
-              pattern: '{{.data}}sport'
-            - fieldName: log.irrelevant
-              pattern: '='
-            - fieldName: origin.port
-              pattern: '{{.integer}}'
-            - fieldName: log.irrelevant
-              pattern: 'dport(\s)?='
-            - fieldName: target.port
-              pattern: '{{.integer}}'
+          - fieldName: log.merakiResult
+            pattern: '{{.word}}'
+          - fieldName: log.irrelevant
+            pattern: src(\s)?=
+          - fieldName: origin.ip
+            pattern: '{{.ipv4}}|{{.ipv6}}'
+          - fieldName: log.irrelevant
+            pattern: dst(\s)?=
+          - fieldName: target.ip
+            pattern: '{{.ipv4}}|{{.ipv6}}'
+          - fieldName: log.irrelevant
+            pattern: mac(\s)?=
+          - fieldName: origin.mac
+            pattern: '{{.data}}protocol'
+          - fieldName: log.irrelevant
+            pattern: '='
+          - fieldName: protocol
+            pattern: '{{.data}}sport'
+          - fieldName: log.irrelevant
+            pattern: '='
+          - fieldName: origin.port
+            pattern: '{{.integer}}'
+          - fieldName: log.irrelevant
+            pattern: dport(\s)?=
+          - fieldName: target.port
+            pattern: '{{.integer}}'
           source: log.genericEvent
           where: '!equals("log.controlFlag", "Init") && (startsWith("log.genericEvent", "allow") || startsWith("log.genericEvent", "deny")) && equals("log.merakiGroup", "flows") && contains("log.genericEvent", "src") && contains("log.genericEvent", "dst") && contains("log.genericEvent", "sport") && contains("log.genericEvent", "dport") && contains("log.genericEvent", "mac")'
       - trim:
@@ -1165,10 +1159,10 @@ pipeline:
           where: '!equals("log.controlFlag", "Init") && (startsWith("log.genericEvent", "allow") || startsWith("log.genericEvent", "deny")) && equals("log.merakiGroup", "flows") && contains("log.genericEvent", "src") && contains("log.genericEvent", "dst") && contains("log.genericEvent", "sport") && contains("log.genericEvent", "dport") && contains("log.genericEvent", "mac")'
       # Adding action result
       - add:
-          function: 'string'
+          function: string
           params:
             key: actionResult
-            value: 'accepted'
+            value: success
           where: '!equals("log.controlFlag", "Init") && startsWith("log.genericEvent", "allow") && equals("log.merakiGroup", "flows") && contains("log.genericEvent", "src") && contains("log.genericEvent", "dst") && contains("log.genericEvent", "sport") && contains("log.genericEvent", "dport") && contains("log.genericEvent", "mac")'
       # Adding action result
       - add:
@@ -1388,12 +1382,6 @@ pipeline:
             - log.vap
           where: startsWith("log.genericEvent", "airmarshal_events") && contains("log.genericEvent", "type") && contains("log.genericEvent", "ssid") && contains("log.genericEvent", "bssid")
       # Adding action result
-      - add:
-          function: 'string'
-          params:
-            key: actionResult
-            value: 'accepted'
-          where: startsWith("log.genericEvent", "airmarshal_events") && contains("log.genericEvent", "type") && contains("log.genericEvent", "ssid") && contains("log.genericEvent", "bssid")
       # ........................................
       # Event: security_event ids_alerted, ids signature matched in Meraki MX Security Appliance
       - grok:
@@ -1469,12 +1457,6 @@ pipeline:
             - protocol
           where: startsWith("log.genericEvent", "signature") && contains("log.genericEvent", "priority") && contains("log.genericEvent", "timestamp") && contains("log.genericEvent", "direction") && contains("log.genericEvent", "dhost")
       # Adding action result
-      - add:
-          function: 'string'
-          params:
-            key: actionResult
-            value: 'accepted'
-          where: startsWith("log.genericEvent", "signature") && contains("log.genericEvent", "priority") && contains("log.genericEvent", "timestamp") && contains("log.genericEvent", "direction") && contains("log.genericEvent", "dhost")
       # ............................................................................
       # Cleaning common fields
       - trim:
@@ -1541,9 +1523,95 @@ pipeline:
             - target.port
           to: int
       #......................................................................#
+      # Keep the vendor event group and message available to correlation rules.
+      - grok:
+          source: log.merakiGroup
+          patterns:
+            - fieldName: log.eventType
+              pattern: '{{.greedy}}'
+          where: exists("log.merakiGroup")
+      - grok:
+          source: log.genericEvent
+          patterns:
+            - fieldName: log.message
+              pattern: '{{.greedy}}'
+          where: exists("log.genericEvent")
       # Removing unused fields
       - delete:
           fields:
             - log.controlFlag
             - log.irrelevant
-            - log.genericEvent
+            - log.genericEvent
+
+      # Normalize explicit outcomes; an unknown outcome remains unset.
+      - add:
+          function: string
+          params:
+            key: actionResult
+            value: denied
+          where: equalsIgnoreCase("log.merakiResult","blocked")
+
+      # Keep addresses in IP fields and retain other source values under log.
+      - rename:
+          from:
+          - origin.ip
+          to: log.unparsedOriginIp
+          where: exists("origin.ip") && (!(inCIDR("origin.ip","0.0.0.0/0") || inCIDR("origin.ip","::/0")) || oneOf("origin.ip",["0.0.0.0","::"]))
+      - rename:
+          from:
+          - target.ip
+          to: log.unparsedTargetIp
+          where: exists("target.ip") && (!(inCIDR("target.ip","0.0.0.0/0") || inCIDR("target.ip","::/0")) || oneOf("target.ip",["0.0.0.0","::"]))
+      - add:
+          function: string
+          params:
+            key: protocol
+            value: ICMP
+          where: equals("protocol",1)
+      - add:
+          function: string
+          params:
+            key: protocol
+            value: TCP
+          where: equals("protocol",6)
+      - add:
+          function: string
+          params:
+            key: protocol
+            value: UDP
+          where: equals("protocol",17)
+      - add:
+          function: string
+          params:
+            key: protocol
+            value: GRE
+          where: equals("protocol",47)
+      - add:
+          function: string
+          params:
+            key: protocol
+            value: ESP
+          where: equals("protocol",50)
+      - add:
+          function: string
+          params:
+            key: protocol
+            value: AH
+          where: equals("protocol",51)
+      - add:
+          function: string
+          params:
+            key: protocol
+            value: ICMPV6
+          where: equals("protocol",58)
+      - add:
+          function: string
+          params:
+            key: protocol
+            value: SCTP
+          where: equals("protocol",132)
+      - rename:
+          from:
+          - protocol
+          to: log.ipProtocolNumber
+          where: exists("protocol") && greaterOrEqual("protocol",0)