Skip to content

feat: jc audit — cross-resource health checks (security/compliance/hygiene/identity)#47

Merged
jklaassenjc merged 2 commits into
mainfrom
juergen/kla-443-jc-audit
Jun 13, 2026
Merged

feat: jc audit — cross-resource health checks (security/compliance/hygiene/identity)#47
jklaassenjc merged 2 commits into
mainfrom
juergen/kla-443-jc-audit

Conversation

@jklaassenjc

@jklaassenjc jklaassenjc commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

A composable check registry that audits the whole org in one pass. 11 checks across 4 categories, each emitting severity-tagged findings with structured resource_ref and remediation hints. The same primitive backs jc-security-audit and jc-compliance-check skills, which now interpret structured JSON instead of scripting raw queries.

What's included

  • internal/audit/ (new package)
    • audit.go — Severity, Category, Finding, CheckResult, AuditCheck types + registry + Run loop + sort/filter helpers
    • data.go — parallel Fetcher interface + shared Data bundle (best-effort: sub-fetch failures become warnings, only all-fail returns an error)
    • checks.go — all 11 checks in one file for catalog discoverability
    • audit_test.go — registry, runner, per-check, fetch semantics
  • internal/cmd/audit.go — parent audit command now runs health checks on bare invoke; audit verify (KLA-411 MCP signed-log) remains as sibling subcommand. Includes clientFetcher adapter mapping v1/v2 to the Fetcher interface, and renderers for human/json/ndjson/table/csv.
  • Skillsjc-security-audit + jc-compliance-check rewritten to invoke jc audit --category X --output json and layer interpretation on top of structured findings instead of bash-scripted queries.
  • Schema + showcase site — adds jc audit + jc doctor to the manifest (both were missing from the hand-curated catalog), new "Diagnostics" sidebar category, duration flag-type added to the whitelist (jc doctor's --probe-timeout 5s).
  • docs/AUDIT.md — per-check reference: what it checks, why, severity scaling, roadmap notes.

The 11 checks

Category Check Severity
security admins-without-mfa CRITICAL
security users-without-mfa HIGH
security suspended-not-locked MEDIUM
security iplists-empty LOW
compliance mfa-adoption-rate scales (<50% CRITICAL → <95% MEDIUM → silent above)
compliance admin-mfa-coverage CRITICAL if <100%
compliance password-age MEDIUM
compliance fde-coverage scales (<50% CRITICAL → <90% HIGH → MEDIUM)
hygiene stale-devices MEDIUM
hygiene auth-policies-disabled LOW
identity recently-created-admins INFO

Deferred (N+1 fetch patterns): empty-groups, suspended-users-with-ssh-keys, policies-without-scope. Documented in docs/AUDIT.md roadmap section.

CLI surface

jc audit                              # everything, human-readable
jc audit --category security          # security only
jc audit --severity high              # high/critical only
jc audit --output json                # for skills + CI
jc audit --exit-code --threshold high # CI gate: exit 1 on threshold

Test plan

  • go test ./... — full sweep green (incl. new audit package + schema)
  • go vet ./... clean
  • make verify-site clean
  • Smoke against real org: surfaced 2 admins-without-mfa, 2 users-without-mfa, 5 stale devices, low FDE, disabled auth policy
  • Exit code wired: --threshold critical --severity critical exits 1; --threshold critical --category hygiene (no criticals there) exits 0
  • All output formats render: human/json/ndjson/table verified live
  • audit verify (existing KLA-411 subcommand) still resolves correctly

Closes KLA-443

🤖 Generated with Claude Code


Note

Medium Risk
Read-only org-wide API listing and security/compliance interpretation can mis-rank risk or fail CI gates if sub-fetches degrade; exit-code behavior was tightened to treat check errors as failures.

Overview
jc audit is now the primary entry point for org-wide health checks (security, compliance, hygiene, identity), not only MCP signed-log tooling. A bare jc audit runs a registered battery of checks after a single parallel JumpCloud list fetch; jc audit verify is unchanged for MCP manifest verification.

The new internal/audit package defines severities, categories, structured findings (resource_ref, remediation_hint), a check registry (11 checks in v1), and a runner with category/severity filters plus helpers for CI exit policy. The CLI wires v1/v2 clientFetcher, renders human/json/ndjson/table/csv/yaml, and --exit-code --threshold fails on threshold findings or check errors so partial API failures cannot pass a gate.

jc-security-audit and jc-compliance-check skills now call jc audit with JSON output instead of hand-rolled list queries. README, docs/AUDIT.md, schema, and the site manifest add a Diagnostics category documenting jc audit and jc doctor.

Reviewed by Cursor Bugbot for commit 5a3c381. Bugbot is set up for automated code reviews on this repo. Configure here.

…giene/identity)

A composable check registry that audits the whole org in one pass:
admins without MFA, MFA adoption rate, FDE coverage, stale devices,
disabled auth policies, suspicious admin lifecycle events, and more.
11 checks across 4 categories; adding a new check is one Register call
from init().

Each finding carries severity (info → critical), a resource_ref like
`admin:alice@acme.com`, and a remediation_hint that names the exact
`jc` command to fix it. The same primitive backs the jc-security-audit
and jc-compliance-check skills — they no longer script raw queries,
just `jc audit --category X --output json` and interpret the structured
findings.

CLI surface:
  jc audit                              # everything, grouped human output
  jc audit --category security          # security checks only
  jc audit --severity high              # high/critical findings only
  jc audit --output {json,ndjson,table,csv,human,yaml}
  jc audit --exit-code --threshold high # CI gate, exits 1 on threshold

The parent `audit` command now runs the health checks on bare
invocation; `audit verify` (KLA-411 MCP signed-log verification)
remains as a sibling subcommand.

Package layout:
  internal/audit/audit.go  — types, registry, runner, sort + filter
  internal/audit/data.go   — parallel Fetcher + shared Data bundle
  internal/audit/checks.go — all 11 checks in one file for discoverability
  internal/cmd/audit.go    — Cobra command + clientFetcher adapter +
                              human/json/ndjson/tabular renderers

Tests cover: severity ordering, category/severity filter, sort stability,
context cancellation, exit-code derivation, per-check positive/negative
cases, fetch parallel-soft-failure semantics. Stubbed Fetcher keeps
audit_test free of network deps.

Smoke-tested live against the configured org: surfaced 2 admins without
MFA, 2 active users without MFA, 5 stale devices, low FDE coverage, and
an old disabled auth policy — exit codes wire correctly with
--threshold.

Also adds `jc audit` + `jc doctor` to the showcase site (the hand-curated
schema manifest had drifted from the actual command set) under a new
"Diagnostics" sidebar category. Adds `duration` to the schema flag-type
whitelist so jc doctor's --probe-timeout 5s validates.

docs/AUDIT.md ships a per-check reference: what it checks, why it
matters, severity scaling, and roadmap notes for the N+1 checks
(empty-groups, ssh-keys-on-suspended) deferred to a future PR.

Closes KLA-443

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Reviewed by Cursor Bugbot for commit 5277379. Configure here.

Comment thread internal/cmd/audit.go
Bugbot review on PR #47 caught two related gaps in how per-check Errors
(returned when a check's required data wasn't fetched) flow through
the surface:

1. Human output printed "OK — N checks ran clean" when totalFindings
   was zero, even if half the checks had Error set. The summary
   bailed before rendering the [ERR] lines, so a partial fetch looked
   like an all-clear.
2. --exit-code only consulted findings ≥ threshold, so a check that
   failed to run (count: 0 findings) couldn't trigger the exit. A CI
   gate where "every check errored" reports the same exit code as
   "everything is healthy" is a lie the next time a sub-fetch flakes.

Fixes:

- New audit.AnyCheckError helper (mirrors AnyFindingAtLeast).
- runAuditHealth: --exit-code now also fails on AnyCheckError with a
  distinct error message so the operator can tell findings-failure
  apart from execution-failure.
- writeAuditHuman: count check errors alongside findings, walk the
  full result set when either is non-zero, and include error count in
  the summary line ("12 findings across 11 checks (1 check error).").
- TestAnyCheckError regression guard pins the new behavior.

Refs KLA-443, addresses Bugbot review on PR #47

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jklaassenjc jklaassenjc merged commit d775659 into main Jun 13, 2026
8 checks passed
@jklaassenjc jklaassenjc deleted the juergen/kla-443-jc-audit branch June 13, 2026 02:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

3 participants