Skip to content

Commit de2ad9f

Browse files
committed
fix: align Google Cloud filter and rule contracts
1 parent 6c3af7e commit de2ad9f

5 files changed

Lines changed: 384 additions & 20 deletions

File tree

‎filters/audits/gcp.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Google Cloud normalization and rule review
2+
3+
Normalize outcome/severity/address values and repair IAM/sink rule CEL functions and grouping.
4+
5+
This draft targets UTMStack `v11`. It contains 1 filter changes
6+
and 2 rule changes for this technology only. Review covered
7+
1 filter configurations and 43 matching shipped rule files.
8+
Unchanged rules are listed in the regression manifest; they are not duplicated in the diff.
9+
10+
## Contract and validation
11+
12+
- Compared exact standard names/types with go-sdk v1.1.31 and the supplied UTMStack dictionaries.
13+
- Checked documented pipeline ordering, rename/move behavior, open vendor log fields,
14+
event-side versus alert-side fields, and surviving fields used by affected rule predicates/history/grouping.
15+
- Strict SDK configuration decoding and actual CEL compilation pass for this scope.
16+
- 13 synthetic normalization cases pass, including SDK Event conversion and any
17+
trigger predicate assertions recorded in the manifest.
18+
- The scoped alerts module tests and `git diff --check` pass with the shared contract runner applied.
19+
20+
The shared alert-contract PR supplies the reusable Go runner for the manifest in
21+
`plugins/alerts/testdata/filter-contracts/gcp.json`. Apply that support before running `go test ./...` in `plugins/alerts`.
22+
23+
The model starts from synthetic extraction results. It does not run complex grok,
24+
JSON/KV/XML/CSV extraction, time conversion, dynamic plugins, historical OpenSearch
25+
queries, or the closed EventProcessor. Raw vendor logs and resulting alerts must
26+
still be checked in staging before rollout. No customer false-positive reduction
27+
has been measured and no production rollout is included.
28+
29+
30+
31+
## References
32+
33+
- [SDK schema](https://github.com/threatwinds/go-sdk/blob/v1.1.31/plugins/plugins.proto)
34+
- [Filter steps](https://github.com/threatwinds/go-sdk/wiki/Filter-Steps-Reference)
35+
- [Standard event schema](https://github.com/threatwinds/go-sdk/wiki/Standard-Event-Schema)
36+
- [Rule implementation](https://github.com/threatwinds/go-sdk/wiki/Implementing-Rules)
37+
38+
`afterEvents`, empty noncapturing grok names, supported numeric strings, and custom
39+
`log.*` fields are accepted. Existing textual protocol casing and vendor action names
40+
are preserved unless a concrete consumer mismatch requires correction.

‎filters/google/gcp.yml‎

Lines changed: 96 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -380,24 +380,24 @@ pipeline:
380380

381381
# Adding severity field based on log.severity
382382
- add:
383-
function: "string"
383+
function: string
384384
params:
385385
key: severity
386-
value: "low"
386+
value: info
387387
where: oneOf("log.severity", ["INFO", "NOTICE"])
388388

389389
- add:
390-
function: "string"
390+
function: string
391391
params:
392392
key: severity
393-
value: "medium"
393+
value: warning
394394
where: equals("log.severity", "WARNING")
395395

396396
- add:
397-
function: "string"
397+
function: string
398398
params:
399399
key: severity
400-
value: "high"
400+
value: error
401401
where: equals("log.severity", "ERROR")
402402

403403
# Adding action field based on log.httpRequestMethod
@@ -445,10 +445,10 @@ pipeline:
445445

446446
# Adding actionResult field based on log.jsonPayloadEnforcedEdgeSecurityPolicyOutcome
447447
- add:
448-
function: "string"
448+
function: string
449449
params:
450450
key: actionResult
451-
value: "accepted"
451+
value: success
452452
where: equals("log.jsonPayloadEnforcedEdgeSecurityPolicyOutcome", "ACCEPT")
453453

454454
- add:
@@ -480,11 +480,11 @@ pipeline:
480480
where: 'exists("log.protoPayloadMethodName") && equals("log.protoPayloadStatusCode", 0)'
481481

482482
- add:
483-
function: "string"
483+
function: string
484484
params:
485485
key: actionResult
486-
value: "failed"
487-
where: 'exists("log.protoPayloadMethodName") && greaterThan("log.protoPayloadStatusCode", 0)'
486+
value: failure
487+
where: exists("log.protoPayloadMethodName") && greaterThan("log.protoPayloadStatusCode", 0)
488488

489489
# Adding actionResult for Cloud DNS query logs (dns.googleapis.com).
490490
- add:
@@ -502,10 +502,10 @@ pipeline:
502502
where: '!exists("actionResult") && equals("log.jsonPayloadResponseCode", "REFUSED")'
503503

504504
- add:
505-
function: "string"
505+
function: string
506506
params:
507507
key: actionResult
508-
value: "failed"
508+
value: failure
509509
where: '!exists("actionResult") && exists("log.jsonPayloadResponseCode")'
510510

511511
# Adding actionResult for plain HTTP request logs (Cloud Run, Load
@@ -525,11 +525,11 @@ pipeline:
525525
where: 'exists("statusCode") && !exists("actionResult") && oneOf("statusCode", [401, 403])'
526526

527527
- add:
528-
function: "string"
528+
function: string
529529
params:
530530
key: actionResult
531-
value: "failed"
532-
where: 'exists("statusCode") && !exists("actionResult") && greaterOrEqual("statusCode", 400)'
531+
value: failure
532+
where: exists("statusCode") && !exists("actionResult") && greaterOrEqual("statusCode", 400)
533533

534534
# Adding geolocation to origin.ip
535535
- dynamic:
@@ -547,6 +547,20 @@ pipeline:
547547
destination: target.geolocation
548548
where: exists("target.ip")
549549

550+
# Normalize the source event severity.
551+
- add:
552+
function: string
553+
params:
554+
key: severity
555+
value: critical
556+
where: oneOf("log.severity",["EMERGENCY","ALERT","CRITICAL"])
557+
- add:
558+
function: string
559+
params:
560+
key: severity
561+
value: debug
562+
where: equals("log.severity","DEBUG")
563+
550564
# Removing unused fields
551565
- delete:
552566
fields:
@@ -556,4 +570,69 @@ pipeline:
556570
- log.httpRequestMethod
557571
- log.jsonPayloadEnforcedEdgeSecurityPolicyOutcome
558572
- log.severity
559-
- log.jsonPayloadStructuredRdata
573+
- log.jsonPayloadStructuredRdata
574+
575+
# Keep addresses in IP fields and retain other source values under log.
576+
- rename:
577+
from:
578+
- origin.ip
579+
to: log.unparsedOriginIp
580+
where: exists("origin.ip") && (!(inCIDR("origin.ip","0.0.0.0/0") || inCIDR("origin.ip","::/0")) || oneOf("origin.ip",["0.0.0.0","::"]))
581+
- rename:
582+
from:
583+
- target.ip
584+
to: log.unparsedTargetIp
585+
where: exists("target.ip") && (!(inCIDR("target.ip","0.0.0.0/0") || inCIDR("target.ip","::/0")) || oneOf("target.ip",["0.0.0.0","::"]))
586+
- add:
587+
function: string
588+
params:
589+
key: protocol
590+
value: ICMP
591+
where: equals("protocol",1)
592+
- add:
593+
function: string
594+
params:
595+
key: protocol
596+
value: TCP
597+
where: equals("protocol",6)
598+
- add:
599+
function: string
600+
params:
601+
key: protocol
602+
value: UDP
603+
where: equals("protocol",17)
604+
- add:
605+
function: string
606+
params:
607+
key: protocol
608+
value: GRE
609+
where: equals("protocol",47)
610+
- add:
611+
function: string
612+
params:
613+
key: protocol
614+
value: ESP
615+
where: equals("protocol",50)
616+
- add:
617+
function: string
618+
params:
619+
key: protocol
620+
value: AH
621+
where: equals("protocol",51)
622+
- add:
623+
function: string
624+
params:
625+
key: protocol
626+
value: ICMPV6
627+
where: equals("protocol",58)
628+
- add:
629+
function: string
630+
params:
631+
key: protocol
632+
value: SCTP
633+
where: equals("protocol",132)
634+
- rename:
635+
from:
636+
- protocol
637+
to: log.ipProtocolNumber
638+
where: exists("protocol") && greaterOrEqual("protocol",0)

0 commit comments

Comments
 (0)