Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions filters/audits/gcp.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Google Cloud normalization and rule review

Normalize outcome/severity/address values and repair IAM/sink rule CEL functions and grouping.

This draft targets UTMStack `v11`. It contains 1 filter changes
and 2 rule changes for this technology only. Review covered
1 filter configurations and 43 matching shipped rule files.
Unchanged rules are listed in the regression manifest; they are not duplicated in the diff.

## Contract and validation

- Compared exact standard names/types with go-sdk v1.1.31 and the supplied UTMStack dictionaries.
- Checked documented pipeline ordering, rename/move behavior, open vendor log fields,
event-side versus alert-side fields, and surviving fields used by affected rule predicates/history/grouping.
- Strict SDK configuration decoding and actual CEL compilation pass for this scope.
- 13 synthetic normalization cases pass, including SDK Event conversion and any
trigger predicate assertions recorded in the manifest.
- The scoped alerts module tests and `git diff --check` pass with the shared contract runner applied.

The shared alert-contract PR supplies the reusable Go runner for the manifest in
`plugins/alerts/testdata/filter-contracts/gcp.json`. Apply that support before running `go test ./...` in `plugins/alerts`.

The model starts from synthetic extraction results. It does not run complex grok,
JSON/KV/XML/CSV extraction, time conversion, dynamic plugins, historical OpenSearch
queries, or the closed EventProcessor. Raw vendor logs and resulting alerts must
still be checked in staging before rollout. No customer false-positive reduction
has been measured and no production rollout is included.



## References

- [SDK schema](https://github.com/threatwinds/go-sdk/blob/v1.1.31/plugins/plugins.proto)
- [Filter steps](https://github.com/threatwinds/go-sdk/wiki/Filter-Steps-Reference)
- [Standard event schema](https://github.com/threatwinds/go-sdk/wiki/Standard-Event-Schema)
- [Rule implementation](https://github.com/threatwinds/go-sdk/wiki/Implementing-Rules)

`afterEvents`, empty noncapturing grok names, supported numeric strings, and custom
`log.*` fields are accepted. Existing textual protocol casing and vendor action names
are preserved unless a concrete consumer mismatch requires correction.
113 changes: 96 additions & 17 deletions filters/google/gcp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -380,24 +380,24 @@ pipeline:

# Adding severity field based on log.severity
- add:
function: "string"
function: string
params:
key: severity
value: "low"
value: info
where: oneOf("log.severity", ["INFO", "NOTICE"])

- add:
function: "string"
function: string
params:
key: severity
value: "medium"
value: warning
where: equals("log.severity", "WARNING")

- add:
function: "string"
function: string
params:
key: severity
value: "high"
value: error
where: equals("log.severity", "ERROR")

# Adding action field based on log.httpRequestMethod
Expand Down Expand Up @@ -445,10 +445,10 @@ pipeline:

# Adding actionResult field based on log.jsonPayloadEnforcedEdgeSecurityPolicyOutcome
- add:
function: "string"
function: string
params:
key: actionResult
value: "accepted"
value: success
where: equals("log.jsonPayloadEnforcedEdgeSecurityPolicyOutcome", "ACCEPT")

- add:
Expand Down Expand Up @@ -480,11 +480,11 @@ pipeline:
where: 'exists("log.protoPayloadMethodName") && equals("log.protoPayloadStatusCode", 0)'

- add:
function: "string"
function: string
params:
key: actionResult
value: "failed"
where: 'exists("log.protoPayloadMethodName") && greaterThan("log.protoPayloadStatusCode", 0)'
value: failure
where: exists("log.protoPayloadMethodName") && greaterThan("log.protoPayloadStatusCode", 0)

# Adding actionResult for Cloud DNS query logs (dns.googleapis.com).
- add:
Expand All @@ -502,10 +502,10 @@ pipeline:
where: '!exists("actionResult") && equals("log.jsonPayloadResponseCode", "REFUSED")'

- add:
function: "string"
function: string
params:
key: actionResult
value: "failed"
value: failure
where: '!exists("actionResult") && exists("log.jsonPayloadResponseCode")'

# Adding actionResult for plain HTTP request logs (Cloud Run, Load
Expand All @@ -525,11 +525,11 @@ pipeline:
where: 'exists("statusCode") && !exists("actionResult") && oneOf("statusCode", [401, 403])'

- add:
function: "string"
function: string
params:
key: actionResult
value: "failed"
where: 'exists("statusCode") && !exists("actionResult") && greaterOrEqual("statusCode", 400)'
value: failure
where: exists("statusCode") && !exists("actionResult") && greaterOrEqual("statusCode", 400)

# Adding geolocation to origin.ip
- dynamic:
Expand All @@ -547,6 +547,20 @@ pipeline:
destination: target.geolocation
where: exists("target.ip")

# Normalize the source event severity.
- add:
function: string
params:
key: severity
value: critical
where: oneOf("log.severity",["EMERGENCY","ALERT","CRITICAL"])
- add:
function: string
params:
key: severity
value: debug
where: equals("log.severity","DEBUG")

# Removing unused fields
- delete:
fields:
Expand All @@ -556,4 +570,69 @@ pipeline:
- log.httpRequestMethod
- log.jsonPayloadEnforcedEdgeSecurityPolicyOutcome
- log.severity
- log.jsonPayloadStructuredRdata
- log.jsonPayloadStructuredRdata

# Keep addresses in IP fields and retain other source values under log.
- rename:
from:
- origin.ip
to: log.unparsedOriginIp
where: exists("origin.ip") && (!(inCIDR("origin.ip","0.0.0.0/0") || inCIDR("origin.ip","::/0")) || oneOf("origin.ip",["0.0.0.0","::"]))
- rename:
from:
- target.ip
to: log.unparsedTargetIp
where: exists("target.ip") && (!(inCIDR("target.ip","0.0.0.0/0") || inCIDR("target.ip","::/0")) || oneOf("target.ip",["0.0.0.0","::"]))
- add:
function: string
params:
key: protocol
value: ICMP
where: equals("protocol",1)
- add:
function: string
params:
key: protocol
value: TCP
where: equals("protocol",6)
- add:
function: string
params:
key: protocol
value: UDP
where: equals("protocol",17)
- add:
function: string
params:
key: protocol
value: GRE
where: equals("protocol",47)
- add:
function: string
params:
key: protocol
value: ESP
where: equals("protocol",50)
- add:
function: string
params:
key: protocol
value: AH
where: equals("protocol",51)
- add:
function: string
params:
key: protocol
value: ICMPV6
where: equals("protocol",58)
- add:
function: string
params:
key: protocol
value: SCTP
where: equals("protocol",132)
- rename:
from:
- protocol
to: log.ipProtocolNumber
where: exists("protocol") && greaterOrEqual("protocol",0)
Loading
Loading