Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

Large diffs are not rendered by default.

92 changes: 90 additions & 2 deletions docs/checks.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ This glossary is the quick map of every built-in check family and the main subse
| Reliability | `Reliability` | `checks.reliability` | missing outbound timeouts; unbounded retries; retries without backoff/jitter; non-idempotent retries; missing cancellation propagation; unbounded work; missing concurrency limits; resource leaks; hidden partial failures; missing graceful shutdown; swallowed errors; lost error context; recoverable panics/exceptions |
| Data Correctness | `Data Correctness` | `checks.data` | read-modify-write races; missing transaction boundaries; external side effects inside transactions; non-idempotent consumers; missing deduplication; unsafe dual writes; missing outbox strategy; unstable pagination; unbounded reads; exactly-once assumptions; cache writes without TTL/policy |
| Change Safety | `Change Safety`, `Change Safety / Testability`, `Change Safety / Refactors` | `checks.change` | implemented diff-size and mixed-concern detectors; behavior changes without tests; failure-path coverage gaps; hardwired or nondeterministic domain dependencies; safe-refactor confidence checks; PR-summary rollups |
| Observability | `Observability` | `checks.observability` | unstructured logs; errors without operation/request context; sensitive log data; high-cardinality metric labels; critical paths without instrumentation; log-and-ignore failures; shallow health checks |
| Operations | `Operations` | `checks.operations` | missing service ownership; missing runbook metadata for critical production paths |
| Delivery | `Delivery` | `checks.delivery` | missing rollback evidence; unsafe migration sequencing; high-risk changes without kill switches; missing post-deploy verification |
| API Contracts | `API Contracts` | `checks.contracts` | exported Go API breaks; public C++ header breaks; OpenAPI breaking changes; protobuf breaking changes; destructive migrations; non-expand/contract schema migration risk |
| Design | `Design Patterns` | `checks.design` | architecture boundaries; import/module cycles; god modules; graph reachability and stability; high-impact changes; public surface policy; production/test isolation; package/module naming; declarations per file; methods per type; interface/protocol size |
| Security | `Security` | `checks.security` | hardcoded secrets and credentials; private keys; insecure TLS; shell execution; dynamic code execution; unsafe HTML sinks; SSRF and taint-style flow; unsafe C string APIs; optional `govulncheck`; OWASP category metadata |
Expand Down Expand Up @@ -71,6 +74,9 @@ Related report artifacts:
"supply_chain": false,
"reliability": false,
"data": false,
"observability": false,
"operations": false,
"delivery": false,
"change": false,
"contracts": true,
"context": true
Expand All @@ -97,7 +103,7 @@ opt-in.
after both the recommended baseline and explicit section enables are resolved.
It is therefore the final precedence layer. Accepted names are `quality`,
`performance`, `design`, `security`, `prompts`, `ci`, `supply_chain`,
`reliability`, `data`, `change`, `context`, and `contracts`; blank, duplicate, unknown, and alias names are
`reliability`, `data`, `observability`, `operations`, `delivery`, `change`, `context`, and `contracts`; blank, duplicate, unknown, and alias names are
invalid.

When `use_recommended_defaults` is absent or `false`, section behavior is
Expand All @@ -115,6 +121,12 @@ baseline.

`data` covers distributed-system and data-correctness checks for Go, Python, TypeScript, JavaScript, and C++: read-modify-write race patterns, missing transaction boundaries, side effects in transaction callbacks, non-idempotent consumers, missing deduplication, unsafe dual writes, missing outbox strategy, unstable pagination, unbounded reads, exactly-once assumptions, and cache writes without TTL/policy evidence.

`observability` covers production operability checks for Go, Python, TypeScript, JavaScript, and C++: structured logging evidence, contextual errors, sensitive log payloads, high-cardinality metric labels, critical-path instrumentation, log-and-ignore failures, and health-check depth.

`operations` covers repository-level service readiness: ownership evidence and runbook metadata for critical production paths.

`delivery` covers safe rollout checks: rollback evidence, expand/backfill/contract migration sequencing, feature-flag or kill-switch evidence for high-risk changes, and post-deploy health/smoke/SLO verification.

`change` covers diff-mode change safety, testability, and refactor-confidence checks. The startup profile leaves it off unless explicitly enabled; strict, enterprise, and AI-safe enable it through their profile defaults. It is designed for PR review and expects a diff/base revision for the strongest evidence.

Set `output.format` to `cyclonedx` (or pass `codeguard scan -format cyclonedx`) to emit the normalized dependency artifacts as deterministic CycloneDX 1.6 JSON. The SBOM contains declared dependency versions or requirements when a resolver version is unavailable; it does not execute project code or contact a registry.
Expand Down Expand Up @@ -331,6 +343,9 @@ the configuration tests.
| `contracts` | scan-mode | scan-mode | true | true | scan-mode |
| `reliability` | false | false | true | true | true |
| `data` | false | false | false | true | true |
| `observability` | false | false | false | true | true |
| `operations` | false | false | false | true | false |
| `delivery` | false | false | false | true | true |
| `change` | false | false | true | true | true |
| `change_rules.max_changed_files` | 25 | 25 | 25 | 25 | 20 |
| `change_rules.max_changed_directories` | 8 | 8 | 8 | 8 | 6 |
Expand Down Expand Up @@ -955,6 +970,79 @@ Config keys:

Rules are implemented for Go, Python, TypeScript, JavaScript, and C++. Contracting database migrations are also surfaced as `contracts.non-expand-contract-migration` in the `API Contracts` section so production-risk scoring can treat unsafe rolling schema changes as data-correctness evidence without renaming the legacy destructive-migration rule.

## Observability

Purpose:
- Validate that production code is operable, not merely syntactically correct.
- Surface logging, metrics, instrumentation, and health-check gaps before they become incident-debugging gaps.

Config keys:

```json
{
"checks": {
"observability": true,
"observability_rules": {
"detect_unstructured_log": true,
"detect_error_without_context": true,
"detect_sensitive_log_data": true,
"detect_high_cardinality_label": true,
"detect_critical_path_uninstrumented": true,
"detect_log_and_ignore": true,
"detect_shallow_health_check": true
}
}
}
```

Rules are implemented for Go, Python, TypeScript, JavaScript, and C++. They use confidence-based source evidence for structured logging, contextual errors, sensitive payloads, metric label cardinality, instrumentation on critical paths, logged-and-ignored failures, and shallow health/readiness endpoints.

## Operations

Purpose:
- Ensure critical services have ownership and runbook evidence.
- Make production responsibility visible to reviewers and agents.

Config keys:

```json
{
"checks": {
"operations": true,
"operations_rules": {
"detect_missing_owner": true,
"detect_missing_runbook": true
}
}
}
```

Operations checks look for repository ownership files and runbook paths around critical production paths. Enterprise enables this family by default; AI-safe keeps it optional because ownership policy is often organization-specific.

## Delivery

Purpose:
- Catch rollout-safety gaps before production deployment.
- Surface migration ordering, rollback, kill-switch, and post-deploy verification risk.

Config keys:

```json
{
"checks": {
"delivery": true,
"delivery_rules": {
"detect_missing_rollback_strategy": true,
"detect_unsafe_migration_order": true,
"detect_high_risk_change_without_kill_switch": true,
"detect_missing_post_deploy_verification": true
}
}
}
```

Delivery checks combine repository-wide deployment/migration evidence with source-path scanning for high-risk behavior. The kill-switch detector covers Go, Python, TypeScript, JavaScript, and C++; rollback, migration-order, and post-deploy checks operate over workflows, deployment files, release files, and migration paths.

## Change Safety

Purpose:
Expand Down Expand Up @@ -1013,7 +1101,7 @@ Profile defaults:
Current detector rollout:

- Implemented `Change Safety` diff detectors: `change.oversized-diff`, `change.mixed-concerns`, `change.too-many-concerns`, `change.mixed-refactor-and-behavior`, `change.unnecessary-surface-area`, `change.one-use-abstraction`, `change.duplicate-helper`, `change.cleanup-regression`, `change.complexity-increased`, and `change.move-without-verification`.
- Implemented `Change Safety / Testability` detectors: `testing.behavior-change-without-test`, `testing.failure-path-missing`, `testing.hardwired-dependency`, and `testing.nondeterministic-domain-logic` for Go, Python, TypeScript, JavaScript, and C++ path/text evidence. `testing.legacy-hotspot-uncovered` is cataloged and configured, but intentionally skips when reliable history/hotspot inputs are unavailable.
- Implemented `Change Safety / Testability` detectors: `testing.behavior-change-without-test`, `testing.failure-path-missing`, `testing.hardwired-dependency`, `testing.nondeterministic-domain-logic`, and `testing.legacy-hotspot-uncovered` for Go, Python, TypeScript, JavaScript, and C++ path/text evidence. `testing.legacy-hotspot-uncovered` uses bounded local git history and skips when reliable history/hotspot inputs are unavailable.
- Implemented `Change Safety / Refactors` detectors: the direct `refactor.*` family below has stable metadata, language coverage, fix templates, config toggles, and diff-mode safe-refactor detector tests.
- Implemented local-quality support rules live in the `Code Quality` section: `naming.generic-identifier`, `function.excessive-parameters`, `function.mixed-abstraction-level`, `function.command-query-mix`, `error.logged-and-ignored`, `error.context-lost`, `defensive.unchecked-type-assertion`, `defensive.unsafe-numeric-conversion`, `maintainability.public-surface-growth`, and `maintainability.dependency-growth`.
- Implemented history-aware maintainability/smell rules live in `Code Quality`-adjacent report sections and skip when git history is unavailable: `maintainability.hotspot`, `maintainability.high-churn-hotspot`, `maintainability.repeat-defect-area`, `maintainability.unstable-interface`, `maintainability.change-amplification`, `smell.shotgun-surgery-history`, and `smell.divergent-change-history`.
Expand Down
11 changes: 10 additions & 1 deletion docs/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,18 @@ This page lists the current `codeguard` feature surface and the main config entr
- `data`
- distributed-system and data-correctness checks for Go, Python, TypeScript, JavaScript, and C++
- read-modify-write race patterns, missing transaction boundaries, side effects in transactions, consumer idempotency/deduplication gaps, unsafe dual writes, missing outbox strategy, unstable pagination, unbounded reads, exactly-once assumptions, and cache policy gaps
- `observability`
- production operability checks for Go, Python, TypeScript, JavaScript, and C++
- unstructured logs, errors without operation/request context, sensitive log data, high-cardinality metric labels, missing critical-path instrumentation, log-and-ignore failures, and shallow health checks
- `operations`
- ownership and runbook-readiness checks for critical production paths
- enterprise profile coverage for service ownership and operational handoff metadata
- `delivery`
- rollout-safety checks for workflows, deployment files, migrations, and high-risk source changes
- missing rollback strategies, unsafe migration ordering, high-risk behavior without feature flags or kill switches, and missing post-deploy verification
- `change`
- diff-mode change-safety, testability, and refactor-confidence checks for PR review
- implemented signals for oversized and mixed-concern diffs, too many concerns, mixed refactor/behavior diffs, broad public-surface edits, one-use abstractions, duplicate helpers, cleanup regressions, complexity increases, moves without verification, behavior changes without tests, failure-path coverage gaps, and hardwired or nondeterministic domain dependencies
- implemented signals for oversized and mixed-concern diffs, too many concerns, mixed refactor/behavior diffs, broad public-surface edits, one-use abstractions, duplicate helpers, cleanup regressions, complexity increases, moves without verification, behavior changes without tests, failure-path coverage gaps, legacy hotspots without characterization coverage, and hardwired or nondeterministic domain dependencies
- implemented direct `refactor.*` IDs for behavior preservation checks, public-contract checks, error-path checks, side-effect ordering, visibility expansion, dependency direction, duplicate implementations left behind, and dead paths left behind
- PR-summary signals for `change_safety`, `refactor_confidence`, and `maintainability_delta` when the change-summary postprocessor is available
- `contracts`
Expand Down
4 changes: 3 additions & 1 deletion docs/production.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ Use blocking failures for:
- unsafe dual writes, missing transaction boundaries, missing outbox strategy, non-idempotent consumers, or non-expand/contract migrations
- contract breaks
- architecture violations with clear ownership boundaries
- sensitive data in logs
- unsafe prompt or MCP config patterns
- CI policy requirements

Expand All @@ -115,6 +116,7 @@ Use warnings for:
- maintainability drift
- cleanup-oriented design heuristics
- confidence-based retry, concurrency, pagination, unbounded-read, cache-policy, or exactly-once-delivery signals that need repository-specific review
- missing structured logging, contextual error logging, critical-path instrumentation, ownership, runbook, rollback, kill-switch, or post-deploy verification evidence
- stability and reachability nudges
- performance smells that still need human review

Expand Down Expand Up @@ -143,7 +145,7 @@ For most teams:

- pull requests: `codeguard scan -mode diff`
- nightly or scheduled: `codeguard scan`
- release branches: `codeguard scan` plus reliability, data, contracts, and supply-chain enforcement
- release branches: `codeguard scan` plus reliability, data, observability, delivery, contracts, and supply-chain enforcement

Prefer SARIF or GitHub output when you want code-host annotations, and JSON when
another system or agent will consume the report programmatically.
Expand Down
11 changes: 10 additions & 1 deletion internal/codeguard/checks/change/change_smells.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ var (
goInterfaceDeclPattern = regexp.MustCompile(`^\s*type\s+([A-Za-z_][A-Za-z0-9_]*)\s+interface\b`)
scriptInterfaceDeclPattern = regexp.MustCompile(`^\s*(?:export\s+)?interface\s+([A-Za-z_][A-Za-z0-9_]*)\b`)
abstractClassDeclPattern = regexp.MustCompile(`^\s*(?:export\s+)?abstract\s+class\s+([A-Za-z_][A-Za-z0-9_]*)\b`)
scriptBoundaryClassPattern = regexp.MustCompile(`^\s*(?:export\s+)?class\s+([A-Za-z_][A-Za-z0-9_]*(?:Gateway|Provider|Port|Adapter|Boundary|Strategy))\b`)
pythonAbstractionDeclPattern = regexp.MustCompile(`^\s*class\s+([A-Za-z_][A-Za-z0-9_]*(?:Gateway|Provider|Port|Adapter|Boundary|Strategy|Protocol))\s*\((?:[^)]*(?:Protocol|ABC)[^)]*)\)\s*:`)
cppAbstractClassDeclPattern = regexp.MustCompile(`^\s*class\s+([A-Za-z_][A-Za-z0-9_]*)\b`)
functionDeclPattern = regexp.MustCompile(`^\s*(?:export\s+)?(?:async\s+)?(?:func|function|def)\s+([A-Za-z_][A-Za-z0-9_]*)\b|^\s*(?:export\s+)?(?:const|let|var)\s+([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(?:async\s*)?\([^)]*\)\s*=>`)
cppFunctionDeclPattern = regexp.MustCompile(`^\s*(?:[A-Za-z_][A-Za-z0-9_:<>,*&\s]+)\s+([A-Za-z_][A-Za-z0-9_]*)\s*\([^;{}]*\)\s*(?:const\s*)?\{`)
Expand Down Expand Up @@ -204,7 +206,14 @@ func abstractionNameForLine(rel string, line string) (string, bool) {
if m := abstractClassDeclPattern.FindStringSubmatch(line); len(m) == 2 {
return m[1], true
}
case ".h", ".hpp", ".hh":
if m := scriptBoundaryClassPattern.FindStringSubmatch(line); len(m) == 2 {
return m[1], true
}
case ".py":
if m := pythonAbstractionDeclPattern.FindStringSubmatch(line); len(m) == 2 {
return m[1], true
}
case ".h", ".hpp", ".hh", ".cpp", ".cc", ".cxx":
if m := cppAbstractClassDeclPattern.FindStringSubmatch(line); len(m) == 2 && strings.Contains(line, "virtual") {
return m[1], true
}
Expand Down
59 changes: 54 additions & 5 deletions internal/codeguard/checks/change/testability.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,17 @@ package change
import (
"context"
"errors"
"fmt"
"path/filepath"
"regexp"
"sort"
"strconv"
"strings"
"time"

"github.com/devr-tools/codeguard/internal/codeguard/checks/support"
"github.com/devr-tools/codeguard/internal/codeguard/core"
"github.com/devr-tools/codeguard/internal/codeguard/history"
)

var (
Expand All @@ -21,6 +25,13 @@ var (
nondeterministicPattern = regexp.MustCompile(`\b(time\.now|date\.now|new\s+date\(|math\.random|rand\.|random\.|uuid\.|datetime\.(now|today)|time\.time|os\.getenv|process\.env|std::chrono::system_clock::now|std::random_device|std::getenv|getenv\()`)
)

const (
legacyHotspotHistoryMaxCommits = 200
legacyHotspotMinCommits = 4
legacyHotspotMinChurn = 25
legacyHotspotMinDefectCommits = 1
)

type testabilityEvidence struct {
path string
line int
Expand Down Expand Up @@ -56,6 +67,10 @@ func testabilityTargetFindings(ctx context.Context, env support.Context, target

testFiles, testHasFailureEvidence := changedTestEvidence(ctx, env, target, changed)
hasChangedTests := len(testFiles) > 0
legacyHotspots := map[string]history.FileChangeMetrics{}
if enabled(env.Config.Checks.ChangeRules.DetectLegacyHotspotUncovered) && !hasChangedTests {
legacyHotspots = legacyHotspotMetrics(ctx, target)
}

findings := make([]core.Finding, 0)
for _, file := range changed {
Expand All @@ -81,16 +96,50 @@ func testabilityTargetFindings(ctx context.Context, env support.Context, target
},
}))
}
if metric, ok := legacyHotspots[path]; ok {
findings = append(findings, legacyHotspotUncoveredFinding(env, path, firstChangedLine(diffScope[path]), metric))
}
}

// TODO(testing.legacy-hotspot-uncovered): emit only after the change section
// receives reliable per-file history/churn inputs. The current diff context
// can identify touched files, but cannot distinguish genuine legacy hotspots
// from ordinary modified code without risking misleading findings.

return findings
}

func legacyHotspotMetrics(ctx context.Context, target core.TargetConfig) map[string]history.FileChangeMetrics {
historyCtx, cancel := context.WithTimeout(ctx, 3*time.Second)
defer cancel()
report, err := history.CollectChangeMetrics(historyCtx, history.ChangeMetricsOptions{
RepoPath: target.Path,
MaxCommits: legacyHotspotHistoryMaxCommits,
})
if err != nil || !report.Available {
return map[string]history.FileChangeMetrics{}
}
out := make(map[string]history.FileChangeMetrics)
for path, metric := range report.Files {
if metric.Commits >= legacyHotspotMinCommits && (metric.Churn >= legacyHotspotMinChurn || metric.DefectCommits >= legacyHotspotMinDefectCommits) {
out[filepath.ToSlash(path)] = metric
}
}
return out
}

func legacyHotspotUncoveredFinding(env support.Context, path string, line int, metric history.FileChangeMetrics) core.Finding {
return env.NewFinding(support.FindingInput{
RuleID: "testing.legacy-hotspot-uncovered",
Level: "warn",
Path: path,
Line: line,
Column: 1,
Message: fmt.Sprintf("touched legacy hotspot has no changed characterization or regression test evidence (%d commits, %d churn lines, %d defect-linked commits)", metric.Commits, metric.Churn, metric.DefectCommits),
Confidence: core.ConfidenceMedium,
Metadata: map[string]string{
"commits": strconv.Itoa(metric.Commits),
"churn": strconv.Itoa(metric.Churn),
"defect_commits": strconv.Itoa(metric.DefectCommits),
},
})
}

func fileTestabilityEvidence(env support.Context, path string, data []byte, ranges core.ChangedLineRanges, hasChangedTests bool, testHasFailureEvidence bool) []testabilityEvidence {
lines := strings.Split(string(data), "\n")
out := make([]testabilityEvidence, 0, 4)
Expand Down
Loading
Loading