Skip to content

Commit 3947924

Browse files
ggbeckerggbecker
committed
Add NIST 800-53 / CIS synchronization toolkit
Add toolkit for generating and maintaining product-specific NIST 800-53 control files with CIS benchmark mappings. Components: - Sync toolkit: Scripts to generate control files from CIS benchmarks and NIST OSCAL catalog - Documentation: README files explaining architecture and workflows - Weekly automation: GitHub Actions workflow for keeping files up to date Control files are organized by product (rhel8, rhel9, rhel10) and split into 21 family files (AC, AU, CM, IA, SC, SI, etc.).
1 parent dd86a53 commit 3947924

17 files changed

Lines changed: 6302 additions & 4 deletions

‎.github/workflows/cis-nist-sync.yml‎

Lines changed: 655 additions & 0 deletions
Large diffs are not rendered by default.

‎controls/README_nist_800_53.md‎

Lines changed: 286 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,286 @@
1+
# NIST 800-53 Control Files
2+
3+
This directory previously contained global NIST 800-53 control files. **These have been moved to product-specific locations.**
4+
5+
## 🔄 Architecture Change
6+
7+
NIST 800-53 control files are now **product-specific** instead of global:
8+
9+
### OLD Architecture (Deprecated)
10+
```
11+
controls/
12+
├── nist_800_53.yml # Global metadata with Jinja2 guards
13+
└── nist_800_53/ # Global family files with guards
14+
├── ac.yml # {{% if product == "rhel9" %}}
15+
├── au.yml # {{% if product.startswith('rhel') %}}
16+
└── ...
17+
```
18+
19+
### NEW Architecture (Current)
20+
```
21+
products/rhel8/controls/
22+
├── nist_800_53.yml # RHEL8-specific metadata
23+
└── nist_800_53/ # RHEL8-specific family files (NO guards)
24+
├── ac.yml # Only rules available in rhel8
25+
├── au.yml
26+
└── ... (21 family files)
27+
28+
products/rhel9/controls/
29+
├── nist_800_53.yml # RHEL9-specific metadata
30+
└── nist_800_53/ # RHEL9-specific family files (NO guards)
31+
├── ac.yml # Only rules available in rhel9
32+
├── au.yml
33+
└── ... (21 family files)
34+
35+
products/rhel10/controls/
36+
├── nist_800_53.yml # RHEL10-specific metadata
37+
└── nist_800_53/ # RHEL10-specific family files (NO guards)
38+
├── ac.yml # Only rules available in rhel10
39+
├── au.yml
40+
└── ... (21 family files)
41+
```
42+
43+
## Benefits of Product-Specific Controls
44+
45+
✅ **Cleaner files** - No Jinja2 conditional logic
46+
✅ **Better separation** - Each product can evolve independently
47+
✅ **Easier to read** - No guards cluttering the rule lists
48+
✅ **No build errors** - No NoneType errors from guards evaluating to None
49+
✅ **Smaller files** - Only rules that actually apply to the product
50+
51+
## File Structure
52+
53+
### Product-Specific Control Files
54+
55+
Each product has its own dedicated NIST 800-53 control files:
56+
57+
**Metadata File**: `products/{product}/controls/nist_800_53.yml`
58+
```yaml
59+
policy: NIST 800-53 Revision 5
60+
title: NIST Special Publication 800-53 Revision 5
61+
id: nist_800_53
62+
version: Revision 5
63+
product: rhel9 # Product-specific
64+
controls_dir: nist_800_53 # Points to family files
65+
levels:
66+
- id: low
67+
- id: moderate
68+
- id: high
69+
```
70+
71+
**Family Files**: `products/{product}/controls/nist_800_53/*.yml` (21 files)
72+
73+
Example: `products/rhel9/controls/nist_800_53/au.yml`
74+
```yaml
75+
# NIST 800-53 AU Family: Audit and Accountability
76+
controls:
77+
- id: au-2
78+
title: Event Logging
79+
levels:
80+
- low
81+
- moderate
82+
- high
83+
rules:
84+
- aide_build_database
85+
- aide_periodic_cron_checking
86+
- audit_rules_execution_chacl
87+
# ... only rules available in rhel9
88+
status: automated
89+
```
90+
91+
**Key Differences from Old Format**:
92+
- ❌ NO Jinja2 guards (`{{% if product == "rhel9" %}}`)
93+
- ❌ NO OSCAL metadata fields (description, parameters, guidance, related_controls)
94+
- ✅ Only rules/variables that exist for the specific product
95+
96+
### OSCAL Metadata
97+
98+
OSCAL metadata (description, parameters, guidance, related_controls) is **NOT included** in control files to keep them lean and focused on rule mappings.
99+
100+
**How to retrieve OSCAL metadata**:
101+
1. **Direct from NIST OSCAL Catalog**: Download from [NIST OSCAL](https://pages.nist.gov/OSCAL/)
102+
2. **On-demand retrieval**: Web application or API can load the OSCAL JSON catalog and extract metadata by control ID
103+
3. **Build-time enrichment**: If needed for documentation, retrieve during build process
104+
105+
**Benefits**:
106+
- Control files stay small (~50-200 lines per family vs. thousands)
107+
- OSCAL data retrieved from authoritative source (NIST catalog)
108+
- No duplication across products
109+
- Updates to OSCAL catalog don't require regenerating control files
110+
111+
## Reference Files (Auto-Generated)
112+
113+
Reference files are auto-generated weekly for comparison:
114+
115+
```
116+
shared/references/controls/
117+
├── nist_800_53_cis_reference_rhel8.yml
118+
├── nist_800_53_cis_reference_rhel8/ # 21 family files
119+
│ ├── ac.yml
120+
│ ├── au.yml
121+
│ └── ...
122+
├── nist_800_53_cis_reference_rhel9.yml
123+
├── nist_800_53_cis_reference_rhel9/ # 21 family files
124+
├── nist_800_53_cis_reference_rhel10.yml
125+
└── nist_800_53_cis_reference_rhel10/ # 21 family files
126+
```
127+
128+
- **Purpose**: Generated from CIS benchmark + NIST OSCAL for comparison
129+
- **Maintained By**: 🤖 **Weekly automation** (DO NOT edit manually)
130+
- **Usage**: Compare with product control files to detect changes
131+
132+
## Control File Content
133+
134+
Each control entry contains:
135+
136+
```yaml
137+
- id: au-2 # NIST control ID
138+
title: Event Logging # Control title
139+
levels: # Baseline applicability
140+
- low
141+
- moderate
142+
- high
143+
rules: # Rules that implement this control
144+
- var_audit_backlog_limit=8192
145+
- aide_build_database
146+
- audit_rules_execution_chacl
147+
status: automated # automated | manual | pending
148+
```
149+
150+
**Field Descriptions**:
151+
- `id`: NIST 800-53 control identifier (lowercase)
152+
- `title`: Short descriptive title from NIST catalog
153+
- `levels`: LOW/MODERATE/HIGH baseline applicability
154+
- `rules`: Rule IDs and variable assignments that implement the control
155+
- `status`:
156+
- `automated` - Has rules/checks
157+
- `pending` - No rules yet
158+
- `manual` - Requires manual verification
159+
160+
## How Profiles Use Control Files
161+
162+
Product profiles reference their own NIST control files:
163+
164+
**Profile**: `products/rhel9/profiles/cis_nist.profile`
165+
```yaml
166+
selections:
167+
- nist_800_53:all # References products/rhel9/controls/nist_800_53.yml
168+
```
169+
170+
The build system automatically:
171+
1. Loads `products/rhel9/controls/nist_800_53.yml` metadata
172+
2. Reads `controls_dir: nist_800_53` field
173+
3. Loads all `.yml` files from `products/rhel9/controls/nist_800_53/`
174+
4. Merges into single control tree
175+
5. Expands `nist_800_53:all` to include all rules from all controls
176+
177+
## Generating Control Files
178+
179+
Control files are generated using the NIST sync toolkit:
180+
181+
```bash
182+
cd utils/nist_sync
183+
184+
# Generate reference files for all products
185+
./generate_cis_nist_workflow.sh
186+
187+
# Generate for specific product
188+
python3 sync_nist_split.py --product rhel9
189+
190+
# Copy reference to product directory
191+
cp shared/references/controls/nist_800_53_cis_reference_rhel9.yml \
192+
products/rhel9/controls/nist_800_53.yml
193+
cp -r shared/references/controls/nist_800_53_cis_reference_rhel9/* \
194+
products/rhel9/controls/nist_800_53/
195+
```
196+
197+
See [utils/nist_sync/README.md](../../utils/nist_sync/README.md) for details.
198+
199+
## Workflow
200+
201+
### Weekly Automated Sync
202+
203+
Every Sunday at 2 PM UTC, GitHub Actions:
204+
1. Downloads latest NIST OSCAL catalog
205+
2. Generates product-specific reference files for rhel8, rhel9, rhel10
206+
3. Compares with previous version
207+
4. Creates PR if changes detected
208+
209+
**Reference files updated automatically. Product control files require manual review.**
210+
211+
### Manual Updates
212+
213+
To update product control files:
214+
215+
```bash
216+
# 1. Generate reference files
217+
cd utils/nist_sync
218+
./generate_cis_nist_workflow.sh
219+
220+
# 2. Compare reference vs. product files
221+
diff -ur shared/references/controls/nist_800_53_cis_reference_rhel9/ \
222+
products/rhel9/controls/nist_800_53/
223+
224+
# 3. Review differences and manually update if needed
225+
vim products/rhel9/controls/nist_800_53/au.yml
226+
227+
# 4. Test build
228+
./build_product rhel9 --datastream-only
229+
230+
# 5. Commit changes
231+
git add products/rhel9/controls/nist_800_53/
232+
git commit -m "Update RHEL9 NIST 800-53 controls"
233+
```
234+
235+
## Family Breakdown
236+
237+
| Family | Name | Controls | Description |
238+
|--------|------|----------|-------------|
239+
| AC | Access Control | 147 | User access, permissions, least privilege |
240+
| AT | Awareness and Training | 17 | Security awareness, role-based training |
241+
| AU | Audit and Accountability | 69 | Logging, monitoring, audit records |
242+
| CA | Assessment, Authorization, Monitoring | 32 | Security assessments, continuous monitoring |
243+
| CM | Configuration Management | 66 | Baseline configs, change control |
244+
| CP | Contingency Planning | 56 | Backup, disaster recovery, continuity |
245+
| IA | Identification and Authentication | 74 | User authentication, MFA, credentials |
246+
| IR | Incident Response | 42 | Incident handling, response procedures |
247+
| MA | Maintenance | 30 | System maintenance, tools, personnel |
248+
| MP | Media Protection | 30 | Removable media, sanitization, disposal |
249+
| PE | Physical and Environmental Protection | 59 | Physical access, environmental controls |
250+
| PL | Planning | 17 | Security planning, architecture |
251+
| PM | Program Management | 37 | Program-level controls, governance |
252+
| PS | Personnel Security | 18 | Background checks, termination procedures |
253+
| PT | PII Processing and Transparency | 21 | Privacy, personally identifiable information |
254+
| RA | Risk Assessment | 26 | Vulnerability scanning, risk analysis |
255+
| SA | System and Services Acquisition | 147 | SDLC, supply chain, developer controls |
256+
| SC | System and Communications Protection | 162 | Network security, encryption, boundaries |
257+
| SI | System and Information Integrity | 119 | Malware protection, security alerts |
258+
| SR | Supply Chain Risk Management | 27 | Supply chain security, provenance |
259+
| OTHER | CIS Items Without NIST Mapping | 102 | CIS rules not explicitly mapped to NIST |
260+
261+
**Total**: 1,196 NIST controls + 102 unmapped CIS items = **1,298 total controls**
262+
263+
## Troubleshooting
264+
265+
**Build fails with "Control file not found":**
266+
- Ensure `products/{product}/controls/nist_800_53.yml` exists
267+
- Check that `controls_dir: nist_800_53` points to correct subdirectory
268+
269+
**Profile shows missing rules:**
270+
- Verify rules exist in product's CIS control file
271+
- Regenerate control files: `./generate_cis_nist_workflow.sh`
272+
- Check that profile references `nist_800_53:all`
273+
274+
**Want to add OSCAL metadata back:**
275+
- OSCAL metadata is available in the NIST OSCAL catalog
276+
- Download: `utils/nist_sync/download_oscal.py`
277+
- Catalog location: `utils/nist_sync/data/nist_800_53_rev5_catalog.json`
278+
- Retrieve on-demand rather than duplicating in control files
279+
280+
## References
281+
282+
- **NIST OSCAL**: https://pages.nist.gov/OSCAL/
283+
- **NIST 800-53 Rev 5**: https://csrc.nist.gov/publications/detail/sp/800-53/rev-5/final
284+
- **CIS Benchmarks**: https://www.cisecurity.org/cis-benchmarks
285+
- **Sync Toolkit**: [utils/nist_sync/README.md](../../utils/nist_sync/README.md)
286+
- **GitHub Workflow**: [.github/workflows/cis-nist-sync.yml](../../.github/workflows/cis-nist-sync.yml)

‎ssg/controls.py‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -242,10 +242,10 @@ def from_control_dict(cls, control_dict, env_yaml=None, default_level=None):
242242
control.mitigation = control_dict.get('mitigation')
243243
control.fixtext = control_dict.get('fixtext')
244244
control.check = control_dict.get('check')
245-
control.tickets = control_dict.get('tickets')
245+
control.tickets = control_dict.get('tickets', []) # Default to empty list if not present
246246
control.original_title = control_dict.get('original_title')
247-
control.related_rules = control_dict.get('related_rules')
248-
control.rules = control_dict.get('rules')
247+
control.related_rules = control_dict.get('related_rules', []) # Default to empty list if not present
248+
control.rules = control_dict.get('rules', []) # Default to empty list if not present
249249

250250
if control.status == "automated":
251251
control.automated = "yes"
@@ -260,7 +260,7 @@ def from_control_dict(cls, control_dict, env_yaml=None, default_level=None):
260260
msg = "Levels for %s must be an array" % control.id
261261
raise ValueError(msg)
262262
control.notes = control_dict.get("notes", "")
263-
selections = control_dict.get("rules", {})
263+
selections = control_dict.get("rules", []) # Default to empty list, not dict
264264

265265
control.selections = selections
266266

‎utils/nist_sync/.gitignore‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Downloaded OSCAL data files
2+
data/nist_800_53_rev5_*.json
3+
4+
# Generated files (regenerated from source data)
5+
data/variable_to_products.json
6+
7+
# Generated profile files (derived from CIS profiles)
8+
../../products/rhel8/profiles/cis_nist.profile
9+
../../products/rhel9/profiles/cis_nist.profile
10+
../../products/rhel10/profiles/cis_nist.profile
11+
12+
# Note: data/cis_nist_mappings.json is COMMITTED (not ignored)
13+
# It's only regenerated when CIS benchmark PDFs are updated
14+
15+
# CIS Benchmark documents (large binary files)
16+
*.pdf
17+
18+
# Markdown conversions of PDFs
19+
CIS_Red_Hat_Enterprise_Linux_*.md
20+
21+
# Python virtual environment
22+
venv/
23+
__pycache__/
24+
*.pyc
25+
*.pyo
26+
27+
# Temporary files
28+
*.tmp
29+
*.bak
30+
31+
# Documentation (keep only workflow docs)
32+
!CIS_NIST_WORKFLOW.md
33+
!README.md
34+
35+
# Auto-generated CIS-NIST profiles (derived from CIS profiles)
36+
products/*/profiles/cis_nist.profile

0 commit comments

Comments
 (0)