Skip to content

fix(filters): Suricata normalization and rule contracts #2618

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
rvald26 wants to merge 1 commit into
base: v11
Choose a base branch
from
Draft

fix(filters): Suricata normalization and rule contracts #2618

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions filters/audits/suricata.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Suricata normalization and rule review

Respect final packet/flow verdict, require evidence for success, correct priority and repair EVE DDoS rule consumers.

This draft targets UTMStack `v11`. It contains 1 filter changes
and 3 rule changes for this technology only. Review covered
1 filter configurations and 35 matching shipped rule files.
Unchanged rules are listed in the regression manifest; they are not duplicated in the diff.

## Contract and validation

- Compared exact standard names/types with go-sdk v1.1.31 and the supplied UTMStack dictionaries.
- Checked documented pipeline ordering, rename/move behavior, open vendor log fields,
event-side versus alert-side fields, and surviving fields used by affected rule predicates/history/grouping.
- Strict SDK configuration decoding and actual CEL compilation pass for this scope.
- 18 synthetic normalization cases pass, including SDK Event conversion and any
trigger predicate assertions recorded in the manifest.
- The scoped alerts module tests and `git diff --check` pass with the shared contract runner applied.

The shared alert-contract PR supplies the reusable Go runner for the manifest in
`plugins/alerts/testdata/filter-contracts/suricata.json`. Apply that support before running `go test ./...` in `plugins/alerts`.

The model starts from synthetic extraction results. It does not run complex grok,
JSON/KV/XML/CSV extraction, time conversion, dynamic plugins, historical OpenSearch
queries, or the closed EventProcessor. Raw vendor logs and resulting alerts must
still be checked in staging before rollout. No customer false-positive reduction
has been measured and no production rollout is included.



## References

- [SDK schema](https://github.com/threatwinds/go-sdk/blob/v1.1.31/plugins/plugins.proto)
- [Filter steps](https://github.com/threatwinds/go-sdk/wiki/Filter-Steps-Reference)
- [Standard event schema](https://github.com/threatwinds/go-sdk/wiki/Standard-Event-Schema)
- [Rule implementation](https://github.com/threatwinds/go-sdk/wiki/Implementing-Rules)

`afterEvents`, empty noncapturing grok names, supported numeric strings, and custom
`log.*` fields are accepted. Existing textual protocol casing and vendor action names
are preserved unless a concrete consumer mismatch requires correction.
151 changes: 107 additions & 44 deletions filters/suricata/suricata.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@ pipeline:
- json:
source: log.suricataJson
where: regexMatch("log.suricataJson", "^\\{.*\\}$")
- json:
source: raw
where: '!exists("log.suricataJson") && regexMatch("raw", "^\\s*\\{")'

# Rename fields to conversion v11 schema
- rename:
Expand Down Expand Up @@ -91,24 +94,24 @@ pipeline:

# Adding severity field based on log.alert.severity
- add:
function: "string"
function: string
params:
key: severity
value: "low"
value: critical
where: equals("log.alert.severity", 1)

- add:
function: "string"
function: string
params:
key: severity
value: "medium"
value: warning
where: equals("log.alert.severity", 2)

- add:
function: "string"
function: string
params:
key: severity
value: "high"
value: info
where: equals("log.alert.severity", 3)

#.......................................................................
Expand All @@ -124,46 +127,9 @@ pipeline:
to: log.fileInfo

# Implementing action field used for established connections
- add:
function: "string"
params:
key: action
value: "success"
where: (equals("log.eventType", "tls") && exists("log.tls.sessionresumed")) ||
(equals("log.eventType", "dns") && equals("log.dns.type", "answer")) ||
(equals("log.eventType", "flow") && exists("origin.ip") && exists("target.ip") && greaterThan("log.flow.bytestoserver", 0) && greaterThan("log.flow.bytestoclient", 0)) ||
(equals("log.eventType", "ssh") && exists("log.ssh.server") && exists("log.ssh.client")) ||
(equals("log.eventType", "alert") && exists("origin.ip") && exists("target.ip") && equals("log.alert.action", "allowed") && greaterThan("log.flow.bytestoserver", 0) && greaterThan("log.flow.bytestoclient", 0)) ||
(equals("log.eventType", "http") && exists("origin.ip") && exists("target.ip") && exists("log.http.status")) ||
(equals("log.eventType", "ftp") && exists("origin.ip") && exists("target.ip") && exists("log.ftp.completioncode")) ||
(equals("log.eventType", "tftp") && exists("origin.ip") && exists("target.ip") && !equals("log.tftp.packet", "error")) ||
(equals("log.eventType", "smb") && exists("origin.ip") && exists("target.ip") && oneOf("log.smb.command", ["NEGOTIATE"]) && oneOf("log.smb.status", ["SUCCESS", "GRANTED", "CONNECTED"])) ||
(equals("log.eventType", "rdp") && exists("origin.ip") && exists("target.ip") && (equals("log.rdp.eventType", "connectresponse") || equals("log.rdp.eventType", "tlshandshake"))) ||
(equals("log.eventType", "rfb") && exists("origin.ip") && exists("target.ip") && equals("log.rfb.authentication.security-result", "OK")) ||
(equals("log.eventType", "mqtt") && exists("origin.ip") && exists("target.ip") && (equals("log.mqtt.connack.returncode", "0") || equals("log.mqtt.connack.returncode", "0x00"))) ||
(equals("log.eventType", "pgsql") && exists("origin.ip") && exists("target.ip") && (exists("log.pgsql.request.simplequery") || exists("log.pgsql.response.commandcompleted") || equals("log.pgsql.response.sslaccepted", "true") || equals("log.pgsql.response.accepted", "true") || exists("log.pgsql.response.authenticationmd5password"))) ||
((equals("log.eventType", "ike") || equals("log.eventType", "ikev1") || equals("log.eventType", "ikev2")) && exists("origin.ip") && exists("target.ip")) ||
(equals("log.eventType", "modbus") && exists("origin.ip") && exists("target.ip")) ||
(equals("log.eventType", "sip") && exists("origin.ip") && exists("target.ip")) ||
(equals("log.eventType", "quic") && exists("origin.ip") && exists("target.ip")) ||
(equals("log.eventType", "fileinfo") && exists("origin.ip") && exists("target.ip")) ||
(equals("log.eventType", "snmp") && exists("origin.ip") && exists("target.ip")) ||
(equals("log.eventType", "dhcp") && exists("origin.ip") && exists("target.ip") && !equals("log.dhcp.assignedip", "0"))


# Adding action result
- add:
function: string
params:
key: actionResult
value: "allowed"
where: 'exists("log.alert.action") && equals("log.alert.action", "allowed")'
- add:
function: string
params:
key: actionResult
value: "blocked"
where: 'exists("log.alert.action") && equals("log.alert.action", "blocked")'

# Adding geolocation to origin.ip
- dynamic:
Expand All @@ -188,4 +154,101 @@ pipeline:
- log.syslogPri
- log.syslogTimestamp
- log.syslogProgram
- log.syslogPid
- log.syslogPid

# Normalize explicit outcomes; an unknown outcome remains unset.
- add:
function: string
params:
key: actionResult
value: denied
where: oneOf("log.verdict.action",["drop","reject"]) || exists("log.verdict.reject") || equals("log.flow.action","drop") || equals("log.alert.action","blocked")
- add:
function: string
params:
key: actionResult
value: success
where: '!exists("actionResult") && (equals("log.verdict.action","pass") || equals("log.flow.action","pass"))'
- add:
function: string
params:
key: actionResult
value: success
where: '!exists("actionResult") && equals("log.flow.state","established") && greaterThan("log.flow.bytestoserver",0) && greaterThan("log.flow.bytestoclient",0)'
- add:
function: string
params:
key: connectionStatus
value: established
where: equals("log.flow.state","established") && !equals("actionResult","denied")
- add:
function: string
params:
key: connectionStatus
value: closed
where: equals("log.flow.state","closed")

# Keep addresses in IP fields and retain other source values under log.
- rename:
from:
- origin.ip
to: log.unparsedOriginIp
where: exists("origin.ip") && (!(inCIDR("origin.ip","0.0.0.0/0") || inCIDR("origin.ip","::/0")) || oneOf("origin.ip",["0.0.0.0","::"]))
- rename:
from:
- target.ip
to: log.unparsedTargetIp
where: exists("target.ip") && (!(inCIDR("target.ip","0.0.0.0/0") || inCIDR("target.ip","::/0")) || oneOf("target.ip",["0.0.0.0","::"]))
- add:
function: string
params:
key: protocol
value: ICMP
where: equals("protocol",1)
- add:
function: string
params:
key: protocol
value: TCP
where: equals("protocol",6)
- add:
function: string
params:
key: protocol
value: UDP
where: equals("protocol",17)
- add:
function: string
params:
key: protocol
value: GRE
where: equals("protocol",47)
- add:
function: string
params:
key: protocol
value: ESP
where: equals("protocol",50)
- add:
function: string
params:
key: protocol
value: AH
where: equals("protocol",51)
- add:
function: string
params:
key: protocol
value: ICMPV6
where: equals("protocol",58)
- add:
function: string
params:
key: protocol
value: SCTP
where: equals("protocol",132)
- rename:
from:
- protocol
to: log.ipProtocolNumber
where: exists("protocol") && greaterOrEqual("protocol",0)
Loading
Loading