Skip to content

Commit a998a9b

Browse files
mykhailo-sindieievmykhailo-sindieiev
authored
Generic parser update (#13139)
* add new fields to generic parser * add test file * fix missing trailing comma * modify csv parser and add csv test file * remove spaces from blank lines * update parser documentation
1 parent f156277 commit a998a9b

5 files changed

Lines changed: 157 additions & 3 deletions

File tree

‎docs/content/en/connecting_your_tools/parsers/file/generic.md‎

Lines changed: 83 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
11
---
2-
title: "Generic Findings Import"
2+
title: 'Generic Findings Import'
33
toc_hide: true
44
---
5+
56
Import Generic findings in CSV or JSON format.
67

78
Attributes supported for CSV:
9+
810
- Date: Date of the finding in mm/dd/yyyy format.
911
- Title: Title of the finding
1012
- CweId: Cwe identifier, must be an integer value.
@@ -18,13 +20,79 @@ Attributes supported for CSV:
1820
- Verified: Indicator if the finding has been verified. Must be empty, TRUE, or FALSE
1921
- FalsePositive: Indicator if the finding is a false positive. Must be TRUE, or FALSE.
2022
- Duplicate:Indicator if the finding is a duplicate. Must be TRUE, or FALSE
21-
- IsMitigated: Indicator if the finding is mitigated. Must be TRUE, or FALSE
23+
- IsMitigated: Indicator if the finding is mitigated. Must be TRUE, or FALSE
2224
- MitigatedDate: Date the finding was mitigated in mm/dd/yyyy format or ISO format
25+
- epss_score: Finding [EPSS score](https://www.first.org/epss/)
26+
- epss_percentile: Finding [EPSS percentile](https://www.first.org/epss/articles/prob_percentile_bins)
27+
- CVSSV3: CVSSv3 verctor of the finding
28+
- CVSSV3_score: CVSSv3 score of the finding
29+
- CVSSV4: CVSSv4 vector of the finding
30+
- CVSSV4_score: CVSSv4 score of the finding
31+
- known_exploited: Indicator if the finding is listed in Known Exploited List. Must be TRUE, or FALSE
32+
- ransomware_used: Indicator if the finding is used in Ransomware. Must be TRUE, or FALSE
33+
- fix_available: Indicator if fix available for the finding. Must be TRUE, or FALSE
34+
- kev_date: Date the finding was added to Known Exploited Vulnerabilities list in mm/dd/yyyy format or ISO format.
2335

2436
The CSV expects a header row with the names of the attributes.
2537

2638
Date fields are parsed using [dateutil.parse](https://dateutil.readthedocs.io/en/stable/parser.html) supporting a variety of formats such a YYYY-MM-DD or ISO-8601.
2739

40+
The list of supported fields in JSON format:
41+
42+
- title: **Required.** String
43+
- severity: **Required.** One of the "Critical", "High", "Medium", "Low", "Info"
44+
- description: **Required.** String
45+
- date: Date
46+
- cwe: Int
47+
- cve: String
48+
- epss_score: Float
49+
- epss_percentile: Float
50+
- cvssv3: String
51+
- cvssv3_score: Float
52+
- cvssv4: String
53+
- cvssv4_score: Float
54+
- mitigation: String
55+
- impact: String
56+
- steps_to_reproduce: String
57+
- severity_justification: String
58+
- references: String
59+
- active: Bool
60+
- verified: Bool
61+
- false_p: Bool
62+
- out_of_scope: Bool
63+
- risk_accepted: Bool
64+
- under_review: Bool
65+
- is_mitigated: Bool
66+
- thread_id: String
67+
- mitigated: Bool
68+
- numerical_severity: Int
69+
- param: String
70+
- payload: String
71+
- line: Int
72+
- file_path: String
73+
- component_name: String
74+
- component_version: String
75+
- static_finding: Bool
76+
- dynamic_finding: Bool
77+
- scanner_confidence: Int
78+
- unique_id_from_tool: String
79+
- vuln_id_from_tool: String
80+
- sast_source_object: String
81+
- sast_sink_object: String
82+
- sast_source_line: Int
83+
- sast_source_file_path: String
84+
- nb_occurences: Int
85+
- publish_date: Date
86+
- service: String
87+
- planned_remediation_date: Date
88+
- planned_remediation_version: String
89+
- effort_for_fixing: One of the "High", "Medium", "Low"
90+
- tags: List of Strings
91+
- kev_date: Date
92+
- known_exploited: Bool
93+
- ransomware_used: Bool
94+
- fix_available: Bool
95+
2896
Example of JSON format:
2997

3098
```JSON
@@ -39,13 +107,23 @@ Example of JSON format:
39107
"cve": "CVE-2020-36234",
40108
"cwe": 261,
41109
"cvssv3": "CVSS:3.1/AV:N/AC:L/PR:H/UI:R/S:C/C:L/I:L/A:N",
110+
"cvssv4": "CVSS:4.0/AV:N/AC:L/AT:N/PR:H/UI:N/VC:L/VI:L/VA:N/SC:N/SI:N/SA:N",
111+
"cvssv4_score": 7.3,
112+
"known_exploited": true,
113+
"ransomware_used": true,
114+
"fix_available": true,
115+
"kev_date": "2024-05-01",
42116
"file_path": "src/first.cpp",
43117
"line": 13,
44118
"endpoints": [
45119
{
46120
"host": "exemple.com"
47121
}
48-
]
122+
],
123+
"tags": [
124+
"security",
125+
"myTag"
126+
],
49127
},
50128
{
51129
"title": "test title with endpoints as strings",
@@ -144,9 +222,11 @@ Example:
144222
```
145223

146224
### Sample Scan Data
225+
147226
Sample Generic Findings Import scans can be found [here](https://github.com/DefectDojo/django-DefectDojo/tree/master/unittests/scans/generic).
148227

149228
### Default Deduplication Hashcode Fields
229+
150230
By default, DefectDojo identifies duplicate Findings using these [hashcode fields](https://docs.defectdojo.com/en/working_with_findings/finding_deduplication/about_deduplication/):
151231

152232
- title

‎dojo/tools/generic/csv_parser.py‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,26 @@ def _get_findings_csv(self, filename):
8181
if len(cvss_objects) > 0:
8282
finding.cvssv3 = cvss_objects[0].clean_vector()
8383

84+
if "CVSSV4" in row:
85+
cvss4_objects = cvss_parser.parse_cvss_from_text(row["CVSSV4"])
86+
if len(cvss4_objects) > 0:
87+
finding.cvssv4 = cvss4_objects[0].clean_vector()
88+
89+
if "CVSSV4_score" in row:
90+
finding.cvssv4_score = float(row["CVSSV4_score"])
91+
92+
if "kev_date" in row:
93+
finding.kev_date = parse(row["kev_date"])
94+
95+
if "known_exploited" in row:
96+
finding.known_exploited = bool(row["known_exploited"])
97+
98+
if "ransomware_used" in row:
99+
finding.ransomware_used = bool(row["ransomware_used"])
100+
101+
if "fix_available" in row:
102+
finding.fix_available = bool(row["fix_available"])
103+
84104
# manage endpoints
85105
if "Url" in row:
86106
finding.unsaved_endpoints = [

‎dojo/tools/generic/json_parser.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,8 @@ def _get_test_json(self, data):
6565
"epss_percentile",
6666
"cvssv3",
6767
"cvssv3_score",
68+
"cvssv4",
69+
"cvssv4_score",
6870
"mitigation",
6971
"impact",
7072
"steps_to_reproduce",
@@ -102,6 +104,10 @@ def _get_test_json(self, data):
102104
"planned_remediation_version",
103105
"effort_for_fixing",
104106
"tags",
107+
"kev_date",
108+
"known_exploited",
109+
"ransomware_used",
110+
"fix_available",
105111
}.union(required)
106112
not_allowed = sorted(set(item).difference(allowed))
107113
if not_allowed:

‎unittests/scans/generic/generic_report_kev_cvssv4.csv‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Date,Title,CweId,epss_score,epss_percentile,Url,Severity,Description,Mitigation,Impact,References,Active,Verified,FalsePositive,Duplicate,CVSSV4,CVSSV4_score,known_exploited,ransomware_used,fix_available,kev_date,CVSSV3
2+
01/30/2018,"Test finding",0,.00042,.23474,https://192.168.1.1/,Low,"Test finding description",,,,False,False,False,False,"CVSS:4.0/AV:N/AC:L/AT:N/PR:H/UI:N/VC:L/VI:L/VA:N/SC:N/SI:N/SA:N","7.3",True,True,True,"09/11/2025","CVSS:3.1/AV:N/AC:L/PR:H/UI:R/S:C/C:L/I:L/A:N""

‎unittests/scans/generic/generic_report_kev_cvssv4.json‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
{
2+
"findings": [
3+
{
4+
"title": "test title",
5+
"description": "Some very long description with\n\n some UTF-8 chars à qu'il est beau",
6+
"active": true,
7+
"verified": true,
8+
"severity": "Medium",
9+
"impact": "Some impact",
10+
"date": "2021-01-06",
11+
"cve": "CVE-2020-36234",
12+
"cwe": 261,
13+
"cvssv3": "CVSS:3.1/AV:N/AC:L/PR:H/UI:R/S:C/C:L/I:L/A:N",
14+
"tags": [
15+
"security",
16+
"network"
17+
],
18+
"unique_id_from_tool": "3287f2d0-554f-491b-8516-3c349ead8ee5",
19+
"vuln_id_from_tool": "TEST1",
20+
"known_exploited": true,
21+
"ransomware_used": true,
22+
"fix_available": true,
23+
"kev_date": "2024-05-01",
24+
"cvssv4": "CVSS:4.0/AV:N/AC:L/PR:H/UI:R/S:C/C:L/I:L/A:N",
25+
"cvssv4_score": 7.3
26+
},
27+
{
28+
"title": "test title2",
29+
"description": "Some very long description with\n\n some UTF-8 chars à qu'il est beau2",
30+
"active": true,
31+
"verified": false,
32+
"severity": "Medium",
33+
"impact": "Some impact",
34+
"date": "2021-01-06",
35+
"cve": "CVE-2020-36235",
36+
"cwe": 287,
37+
"cvssv3": "CVSS:3.1/AV:N/AC:L/PR:H/UI:R/S:C/C:L/I:L/A:N",
38+
"tags": [
39+
"security",
40+
"network"
41+
],
42+
"unique_id_from_tool": "42500af3-68c5-4dc3-8022-191d93c2f1f7",
43+
"vuln_id_from_tool": "TEST2"
44+
}
45+
]
46+
}

0 commit comments

Comments
 (0)