Skip to content

fix(filters): Palo Alto Networks normalization and rule contracts #2614

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
rvald26 wants to merge 1 commit into
base: v11
Choose a base branch
from
Draft

fix(filters): Palo Alto Networks normalization and rule contracts #2614

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions filters/audits/paloalto.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Palo Alto Networks normalization and rule review

Fix configuration, counter names/types and side meaning; honor explicit denial and leave submitted work pending.

This draft targets UTMStack `v11`. It contains 1 filter changes
and 0 rule changes for this technology only. Review covered
1 filter configurations and 7 matching shipped rule files.
Unchanged rules are listed in the regression manifest; they are not duplicated in the diff.

## Contract and validation

- Compared exact standard names/types with go-sdk v1.1.31 and the supplied UTMStack dictionaries.
- Checked documented pipeline ordering, rename/move behavior, open vendor log fields,
event-side versus alert-side fields, and surviving fields used by affected rule predicates/history/grouping.
- Strict SDK configuration decoding and actual CEL compilation pass for this scope.
- 4 synthetic normalization cases pass, including SDK Event conversion and any
trigger predicate assertions recorded in the manifest.
- The scoped alerts module tests and `git diff --check` pass with the shared contract runner applied.

The shared alert-contract PR supplies the reusable Go runner for the manifest in
`plugins/alerts/testdata/filter-contracts/paloalto.json`. Apply that support before running `go test ./...` in `plugins/alerts`.

The model starts from synthetic extraction results. It does not run complex grok,
JSON/KV/XML/CSV extraction, time conversion, dynamic plugins, historical OpenSearch
queries, or the closed EventProcessor. Raw vendor logs and resulting alerts must
still be checked in staging before rollout. No customer false-positive reduction
has been measured and no production rollout is included.



## References

- [SDK schema](https://github.com/threatwinds/go-sdk/blob/v1.1.31/plugins/plugins.proto)
- [Filter steps](https://github.com/threatwinds/go-sdk/wiki/Filter-Steps-Reference)
- [Standard event schema](https://github.com/threatwinds/go-sdk/wiki/Standard-Event-Schema)
- [Rule implementation](https://github.com/threatwinds/go-sdk/wiki/Implementing-Rules)

`afterEvents`, empty noncapturing grok names, supported numeric strings, and custom
`log.*` fields are accepted. Existing textual protocol casing and vendor action names
are preserved unless a concrete consumer mismatch requires correction.
207 changes: 131 additions & 76 deletions filters/paloalto/pa_firewall.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1543,15 +1543,15 @@ pipeline:
source: log.cefOrLeefMsgAll

- grok:
patterns:
- fieldName: log.irrelevant
pattern: '{{.data}}(panOSContainerNameSpace=)'
- fieldName: log.panOSContainerNameSpace
pattern: '{{.data}}({{.word}}=)'
- fieldName: log.irrelevant
pattern: '{{.greedy}}'
source: log.cefOrLeefMsgAll
patterns:
- fieldName: log.irrelevant
pattern: '{{.data}}(panOSContainerNameSpace=)'
- fieldName: log.panOSContainerNameSpace
pattern: '{{.data}}({{.word}}=)'
- fieldName: log.irrelevant
pattern: '{{.greedy}}'
source: log.cefOrLeefMsgAll

- grok:
patterns:
- fieldName: log.irrelevant
Expand Down Expand Up @@ -1743,14 +1743,14 @@ pipeline:
source: log.cefOrLeefMsgAll

- grok:
patterns:
- fieldName: log.irrelevant
pattern: '{{.data}}(panOSDGl1=)'
- fieldName: log.panOSDGl1
pattern: '{{.data}}({{.word}}=)'
- fieldName: log.irrelevant
pattern: '{{.greedy}}'
source: log.cefOrLeefMsgAll
patterns:
- fieldName: log.irrelevant
pattern: '{{.data}}(panOSDGl1=)'
- fieldName: log.panOSDGl1
pattern: '{{.data}}({{.word}}=)'
- fieldName: log.irrelevant
pattern: '{{.greedy}}'
source: log.cefOrLeefMsgAll

- grok:
patterns:
Expand Down Expand Up @@ -6381,7 +6381,7 @@ pipeline:
# Detect if its a TUNNEL INSPECTION log
# ..........................................................................#
- csv:
from: log.csvMsgAll
source: log.csvMsgAll
separator: ","
headers:
- log.receive_time
Expand Down Expand Up @@ -6472,7 +6472,7 @@ pipeline:
# Detect if its a SCTP log
# ..........................................................................#
- csv:
from: log.csvMsgAll
source: log.csvMsgAll
separator: ","
headers:
- log.receive_time
Expand Down Expand Up @@ -6545,7 +6545,7 @@ pipeline:
# Detect if its a CONFIG log
#......................................................................#
- csv:
from: log.csvMsgAll
source: log.csvMsgAll
separator: ','
headers:
- log.receive_time
Expand Down Expand Up @@ -6580,7 +6580,7 @@ pipeline:
# Detect if its a AUTHENTICATION log
#......................................................................#
- csv:
from: log.csvMsgAll
source: log.csvMsgAll
separator: ','
headers:
- log.receive_time
Expand Down Expand Up @@ -6633,7 +6633,7 @@ pipeline:
# Detect if its a SYSTEM log
#......................................................................#
- csv:
from: log.csvMsgAll
source: log.csvMsgAll
separator: ','
headers:
- log.receive_time
Expand Down Expand Up @@ -6667,7 +6667,7 @@ pipeline:
# Detect if its a CORRELATED EVENTS log
#......................................................................#
- csv:
from: log.csvMsgAll
source: log.csvMsgAll
separator: ","
headers:
- log.receive_time
Expand Down Expand Up @@ -6697,7 +6697,7 @@ pipeline:
# Detect if its a GTP log
#......................................................................#
- csv:
from: log.csvMsgAll
source: log.csvMsgAll
separator: ","
headers:
- log.receive_time
Expand Down Expand Up @@ -6810,22 +6810,22 @@ pipeline:
- rename:
from:
- log.bytessent
to: target.bytesSent
to: origin.bytesSent

- rename:
from:
- log.bytesreceived
to: target.bytesReceived
to: origin.bytesReceived

- rename:
from:
- log.pktssent
to: target.packagesSent
to: origin.packagesSent

- rename:
from:
- log.pktsreceived
to: target.packagesReceived
to: origin.packagesReceived

- rename:
from:
Expand All @@ -6850,69 +6850,27 @@ pipeline:
# ................................................#
# Adding action result
# ................................................#
- add:
function: string
params:
key: actionResult
value: "allow"
where: '!exists("log.status") && equalsIgnoreCase("action", "allow")'

- add:
function: string
params:
key: actionResult
value: "denied"
where: '!exists("log.status") && oneOf("action", ["deny", "Deny"])'

- add:
function: string
params:
key: actionResult
value: "blocked"
where: '!exists("log.status") && oneOf("action", ["drop", "Drop", "reset-client", "reset-server", "reset-both", "block-url", "block-ip", "random-drop", "sinkhole"])'

- add:
function: string
params:
key: actionResult
value: "Succeeded"
where: '!exists("log.status") && oneOf("log.result", ["Succeeded", "Submitted"])'

- add:
function: string
params:
key: actionResult
value: "failed"
where: '!exists("log.status") && oneOf("log.result", ["Failed", "Unauthorized"])'

- add:
function: string
params:
key: actionResult
value: "success"
where: 'equals("log.status", "success")'

- add:
function: string
params:
key: actionResult
value: "failed"
where: 'equals("log.status", "failure")'

# ................................................#
# Fileds conversions
# ................................................#
- cast:
fields:
- target.bytessent
- target.bytesreceived
- origin.bytesSent
- origin.bytesReceived
to: float

- cast:
fields:
- target.pktssent
- target.packagesReceived
to: int64
- origin.packagesSent
- origin.packagesReceived
to: int

- cast:
fields:
Expand Down Expand Up @@ -6941,4 +6899,101 @@ pipeline:
# ..........................................................................#
- delete:
fields:
- log.csvMsgAll
- log.csvMsgAll

# Normalize explicit outcomes; an unknown outcome remains unset.
- add:
function: string
params:
key: actionResult
value: denied
where: regexMatch("action", "(?i)^(deny|drop|drop ICMP|reset[- ]client|reset[- ]server|reset[- ]both|block-url|block-ip|random-drop|sinkhole)$")
- add:
function: string
params:
key: actionResult
value: success
where: '!exists("actionResult") && equalsIgnoreCase("action", "allow")'
- add:
function: string
params:
key: actionResult
value: denied
where: '!exists("actionResult") && equalsIgnoreCase("log.result", "Unauthorized")'
- add:
function: string
params:
key: actionResult
value: failure
where: '!exists("actionResult") && (equalsIgnoreCase("log.result", "Failed") || equalsIgnoreCase("log.status", "failure"))'
- add:
function: string
params:
key: actionResult
value: success
where: '!exists("actionResult") && (equalsIgnoreCase("log.result", "Succeeded") || equalsIgnoreCase("log.status", "success"))'

# Keep addresses in IP fields and retain other source values under log.
- rename:
from:
- origin.ip
to: log.unparsedOriginIp
where: exists("origin.ip") && (!(inCIDR("origin.ip","0.0.0.0/0") || inCIDR("origin.ip","::/0")) || oneOf("origin.ip",["0.0.0.0","::"]))
- rename:
from:
- target.ip
to: log.unparsedTargetIp
where: exists("target.ip") && (!(inCIDR("target.ip","0.0.0.0/0") || inCIDR("target.ip","::/0")) || oneOf("target.ip",["0.0.0.0","::"]))
- add:
function: string
params:
key: protocol
value: ICMP
where: equals("protocol",1)
- add:
function: string
params:
key: protocol
value: TCP
where: equals("protocol",6)
- add:
function: string
params:
key: protocol
value: UDP
where: equals("protocol",17)
- add:
function: string
params:
key: protocol
value: GRE
where: equals("protocol",47)
- add:
function: string
params:
key: protocol
value: ESP
where: equals("protocol",50)
- add:
function: string
params:
key: protocol
value: AH
where: equals("protocol",51)
- add:
function: string
params:
key: protocol
value: ICMPV6
where: equals("protocol",58)
- add:
function: string
params:
key: protocol
value: SCTP
where: equals("protocol",132)
- rename:
from:
- protocol
to: log.ipProtocolNumber
where: exists("protocol") && greaterOrEqual("protocol",0)
Loading
Loading