Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .github/workflows/fuzz.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Fuzz

on:
workflow_dispatch:
schedule:
- cron: "0 8 * * *"

permissions:
contents: read

concurrency:
group: fuzz-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
fuzz:
name: Native Go fuzzing
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false

- name: Set up Go
uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
with:
go-version-file: go.mod
cache: true
cache-dependency-path: go.sum

- name: Run native Go fuzz targets
run: make fuzz FUZZTIME=2m

- name: Upload fuzz failures
if: failure()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: go-fuzz-failures
path: |
**/testdata/fuzz/**
if-no-files-found: ignore
retention-days: 7
16 changes: 16 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@ logger.Warn("cache miss", zap.Error(err))
- Add unit tests for new logic.
- Add integration tests for new commands and user-visible flows.
- Run `make test` before considering work complete.
- Run `make fuzz FUZZTIME=5s` after changing parsers, SBOM handling, plugin archive/path validation, or SDK transport JSON. Increase `FUZZTIME` for deeper local runs.
- Run `make smoke` if you touched a detector chain, matcher, auditor, or analyzer.
- If you change GitHub Actions workflows or release behavior, update [dev-docs/CI.md](dev-docs/CI.md) and any affected install guidance in [README.md](README.md).

Expand All @@ -110,6 +111,21 @@ logger.Warn("cache miss", zap.Error(err))

Do not add skipped tests without a recorded reason.

### Fuzzing

Native Go fuzz targets cover Bomly's highest-risk untrusted input boundaries. Run the committed fuzz target matrix with:

```bash
make fuzz
make fuzz FUZZTIME=5s
```

When Go reports a minimized failure, rerun it with the command printed by `go test`, for example:

```bash
go test ./internal/sbom -run=FuzzUnmarshalAutoJSON/<hash>
```

## Documentation

User-facing docs live in [`docs/`](docs/). Most pages are handwritten; some are generated.
Expand Down
6 changes: 5 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@ GO_LICENSES_VERSION=v1.6.0
GOPATH_BIN=$(shell go env GOPATH)/bin
EXE_SUFFIX=$(if $(filter Windows_NT,$(OS)),.exe,)
GOLANGCI_LINT=$(GOPATH_BIN)/golangci-lint$(EXE_SUFFIX)
FUZZTIME?=60s

.PHONY: build build-full build-lite fmt fmt-check lint install-hooks test run generate docs-config docs-schema docs-schema-md docs-support-matrix docs-components smoke benchmark benchmark-report licenses
.PHONY: build build-full build-lite fmt fmt-check lint install-hooks test fuzz run generate docs-config docs-schema docs-schema-md docs-support-matrix docs-components smoke benchmark benchmark-report licenses

build: build-full build-lite

Expand Down Expand Up @@ -34,6 +35,9 @@ install-hooks:
test:
go test ./...

fuzz:
FUZZTIME="$(FUZZTIME)" scripts/run-fuzz.sh

smoke:
go test -tags "smoke" ./test/smoke/ -v -count=1 -timeout 15m $(if $(ARGS),$(ARGS),)

Expand Down
19 changes: 19 additions & 0 deletions dev-docs/CI.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Bomly uses GitHub Actions for validation, security analysis, smoke coverage, and
| `Build & Test` | Pull requests, pushes to `main` | Fast validation split into parallel jobs: `lint`, `test`, `build`, `format`, `modules` (go.mod drift), and `generated-docs` (generated-doc drift) |
| `CodeQL` | Pull requests, pushes to `main`, weekly | Static security/quality analysis for Go; results surface in the Security tab |
| `Scorecard` | Pushes to `main`, weekly, manual dispatch | OpenSSF Scorecard supply-chain checks; publishes results and uploads SARIF |
| `Fuzz` | Nightly schedule, manual dispatch | Native Go fuzzing over lockfile parsers, SBOM JSON, SDK transport JSON, and plugin path/archive sanitizers |
| `Dependency review` | Pull requests | GitHub dependency-review of added/changed dependencies; fails on high-severity introductions |
| `Bomly Guard` | Pull requests | Dogfoods the Bomly Guard action to diff and audit dependency changes on each PR |
| `Smoke` | Merge queue, nightly schedule, manual dispatch | Slow end-to-end coverage against real repositories, SBOMs, and containers before merge, plus scheduled drift detection |
Expand Down Expand Up @@ -102,6 +103,24 @@ Smoke tests use the pinned `ref`. The local benchmark intentionally does not, be

The GitHub Actions `Smoke` workflow calls `make smoke`; it does not call package-specific scripts directly. Keep that pattern when adding smoke coverage so local and CI behavior stay aligned.

## Native Go Fuzzing

Bomly runs native Go fuzzing nightly and on demand through the `Fuzz` workflow. The target matrix is intentionally explicit so CI spends time on the highest-risk untrusted input boundaries:

- Node lockfile parsers for npm, pnpm, and Yarn
- SPDX/CycloneDX SBOM JSON detection and decoding
- SDK package URL canonicalization and transport JSON
- Managed plugin archive-name and relative-path sanitizers

Run the same matrix locally with:

```bash
make fuzz
make fuzz FUZZTIME=5s
```

`FUZZTIME` is passed to each `go test -fuzz` invocation and defaults to `60s`. When Go minimizes a failure into `testdata/fuzz/<FuzzName>/<hash>`, rerun the exact command printed by `go test`, then commit the reproducer only after confirming it is a useful regression seed.

## Local Dependency Graph Benchmark

Bomly ships a hidden, local-only `benchmark` command for comparing native-detector output with GitHub Dependency Graph and Syft SBOMs. It is intentionally not called by GitHub Actions.
Expand Down
36 changes: 36 additions & 0 deletions internal/detectors/node/nodetest/fuzz.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
package nodetest

import (
"testing"

"github.com/bomly-dev/bomly-cli/sdk"
)

// MaxFuzzInputSize caps fuzz payloads for lockfile parser targets.
const MaxFuzzInputSize = 1 << 20

// RequireFuzzGraphValid verifies graph invariants shared by node lockfile fuzz tests.
func RequireFuzzGraphValid(t *testing.T, graph *sdk.Graph) {
t.Helper()
if graph == nil {
t.Fatal("successful parse returned nil graph")
}
graph.WalkNodes(func(node *sdk.Dependency) bool {
if node == nil {
t.Fatal("graph contains nil node")
}
if node.ID == "" {
t.Fatalf("graph contains node with empty ID: %+v", node)
}
return true
})
graph.WalkEdges(func(from, to *sdk.Dependency) bool {
if from == nil || to == nil {
t.Fatalf("graph contains nil edge endpoint: from=%+v to=%+v", from, to)
}
if from.ID == "" || to.ID == "" {
t.Fatalf("graph contains edge with empty endpoint ID: from=%+v to=%+v", from, to)
}
return true
})
}
35 changes: 35 additions & 0 deletions internal/detectors/node/npm/npm_lockfile_fuzz_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
package npm

import (
"os"
"path/filepath"
"testing"

"github.com/bomly-dev/bomly-cli/internal/detectors/node/nodetest"
)

func FuzzDepGraphFromNPMLockfile(f *testing.F) {
for _, seed := range []string{
`{"name":"demo","version":"1.0.0","lockfileVersion":3,"packages":{"":{"name":"demo","version":"1.0.0","dependencies":{"left-pad":"1.3.0"}},"node_modules/left-pad":{"name":"left-pad","version":"1.3.0","resolved":"https://registry.npmjs.org/left-pad/-/left-pad-1.3.0.tgz","integrity":"sha512-seed"}}}`,
`{"name":"demo","version":"1.0.0","lockfileVersion":1,"dependencies":{"left-pad":{"version":"1.3.0","dependencies":{"repeat-string":{"version":"1.6.1"}}}}}`,
`{"name":"demo","lockfileVersion":3,"packages":{"":{"name":"demo","dependencies":{"benchmark":"1.0.0"}},"node_modules/benchmark":{"version":"1.0.0","engines":["node","rhino"]}}}`,
} {
f.Add([]byte(seed))
}

f.Fuzz(func(t *testing.T, raw []byte) {
if len(raw) > nodetest.MaxFuzzInputSize {
return
}
projectDir := t.TempDir()
if err := os.WriteFile(filepath.Join(projectDir, "package-lock.json"), raw, 0o644); err != nil {
t.Fatalf("write package-lock.json: %v", err)
}

graph, err := depGraphFromNPMLockfile(projectDir)
if err != nil {
return
}
nodetest.RequireFuzzGraphValid(t, graph)
})
}
38 changes: 38 additions & 0 deletions internal/detectors/node/pnpm/pnpm_lockfile_fuzz_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
package pnpm

import (
"os"
"path/filepath"
"testing"

"github.com/bomly-dev/bomly-cli/internal/detectors/node/nodetest"
)

func FuzzDepGraphFromPNPMLockfile(f *testing.F) {
for _, seed := range []string{
"lockfileVersion: '9.0'\nimporters:\n .:\n dependencies:\n left-pad:\n version: 1.3.0\npackages:\n left-pad@1.3.0:\n resolution:\n integrity: sha512-seed\nsnapshots:\n left-pad@1.3.0: {}\n",
"lockfileVersion: 5.4\ndependencies:\n react: 18.2.0\npackages:\n /react/18.2.0:\n resolution:\n integrity: sha512-seed\n dependencies:\n loose-envify: 1.4.0\n /loose-envify/1.4.0:\n resolution:\n integrity: sha512-seed\n",
"packages:\n /@scope/pkg/1.0.0:\n resolution:\n tarball: https://registry.npmjs.org/@scope/pkg/-/pkg-1.0.0.tgz\n",
} {
f.Add([]byte(seed))
}

f.Fuzz(func(t *testing.T, raw []byte) {
if len(raw) > nodetest.MaxFuzzInputSize {
return
}
projectDir := t.TempDir()
if err := os.WriteFile(filepath.Join(projectDir, "pnpm-lock.yaml"), raw, 0o644); err != nil {
t.Fatalf("write pnpm-lock.yaml: %v", err)
}
if err := os.WriteFile(filepath.Join(projectDir, "package.json"), []byte(`{"name":"demo","version":"1.0.0"}`), 0o644); err != nil {
t.Fatalf("write package.json: %v", err)
}

graph, err := depGraphFromPNPMLockfile(projectDir)
if err != nil {
return
}
nodetest.RequireFuzzGraphValid(t, graph)
})
}
38 changes: 38 additions & 0 deletions internal/detectors/node/yarn/yarn_lockfile_fuzz_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
package yarn

import (
"os"
"path/filepath"
"testing"

"github.com/bomly-dev/bomly-cli/internal/detectors/node/nodetest"
)

func FuzzDepGraphFromYarnLockfile(f *testing.F) {
for _, seed := range []string{
"left-pad@^1.3.0:\n version \"1.3.0\"\n resolved \"https://registry.yarnpkg.com/left-pad/-/left-pad-1.3.0.tgz\"\n integrity sha512-seed\n",
"\"@scope/pkg@npm:1.0.0\":\n version: 1.0.0\n resolution: \"@scope/pkg@npm:1.0.0\"\n dependencies:\n left-pad: ^1.3.0\nleft-pad@^1.3.0:\n version: 1.3.0\n",
"__metadata:\n version: 8\n cacheKey: 10\n",
} {
f.Add([]byte(seed))
}

f.Fuzz(func(t *testing.T, raw []byte) {
if len(raw) > nodetest.MaxFuzzInputSize {
return
}
projectDir := t.TempDir()
if err := os.WriteFile(filepath.Join(projectDir, "yarn.lock"), raw, 0o644); err != nil {
t.Fatalf("write yarn.lock: %v", err)
}
if err := os.WriteFile(filepath.Join(projectDir, "package.json"), []byte(`{"name":"demo","version":"1.0.0","dependencies":{"left-pad":"^1.3.0"}}`), 0o644); err != nil {
t.Fatalf("write package.json: %v", err)
}

graph, err := depGraphFromYarnLockfile(projectDir)
if err != nil {
return
}
nodetest.RequireFuzzGraphValid(t, graph)
})
}
67 changes: 67 additions & 0 deletions internal/plugin/path_fuzz_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
package plugin

import (
"os"
"path/filepath"
"strings"
"testing"
)

func FuzzPluginPathSanitizers(f *testing.F) {
for _, seed := range []string{
"bin/bomly-plugin",
"../escape",
"/absolute/path",
`.\\`,
`windows\path\plugin.exe`,
"plugin.tar.gz",
"plugin.zip",
"C:/escape/plugin.exe",
} {
f.Add(seed)
}

f.Fuzz(func(t *testing.T, raw string) {
if len(raw) > 4096 {
return
}

if name := safeDownloadArchiveName(raw); name != "" {
if strings.ContainsAny(name, `/\`) {
t.Fatalf("safe archive name contains path separator: %q from %q", name, raw)
}
if name == "." || name == ".." || strings.Contains(name, ":") {
t.Fatalf("safe archive name is unsafe: %q from %q", name, raw)
}
switch ext := archiveExtension(name); ext {
case "", ".zip", ".tar.gz", ".tgz":
default:
t.Fatalf("unexpected archive extension %q for %q", ext, name)
}
}

cleanPath, err := cleanRelativePluginPath(raw)
if err != nil {
return
}
if cleanPath == "" || cleanPath == "." || filepath.IsAbs(cleanPath) || strings.Contains(cleanPath, ":") {
t.Fatalf("clean relative path is unsafe: %q from %q", cleanPath, raw)
}
if cleanPath == ".." || strings.HasPrefix(cleanPath, ".."+string(os.PathSeparator)) {
t.Fatalf("clean relative path escapes base: %q from %q", cleanPath, raw)
}

root := t.TempDir()
fullPath, err := pathInPluginDir(root, raw)
if err != nil {
t.Fatalf("pathInPluginDir rejected path accepted by cleanRelativePluginPath: %q: %v", raw, err)
}
rel, err := filepath.Rel(root, fullPath)
if err != nil {
t.Fatalf("rel path for %q: %v", fullPath, err)
}
if rel == ".." || strings.HasPrefix(rel, ".."+string(os.PathSeparator)) {
t.Fatalf("pathInPluginDir returned escaping path: root=%q full=%q rel=%q", root, fullPath, rel)
}
})
}
3 changes: 3 additions & 0 deletions internal/plugin/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,9 @@ func cleanRelativePluginPath(value string) (string, error) {
if filepath.IsAbs(cleanPath) || cleanPath == ".." || strings.HasPrefix(cleanPath, ".."+string(os.PathSeparator)) {
return "", errors.New("path escapes base directory")
}
if cleanPath == "." {
return "", errors.New("invalid relative path")
}
return cleanPath, nil
}

Expand Down
33 changes: 33 additions & 0 deletions internal/sbom/codec_fuzz_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
package sbom

import "testing"

const maxFuzzInputSize = 1 << 20

func FuzzUnmarshalAutoJSON(f *testing.F) {
for _, seed := range []string{
`{"spdxVersion":"SPDX-2.3","SPDXID":"SPDXRef-DOCUMENT","name":"demo","documentNamespace":"https://example.com/spdx/demo","creationInfo":{"created":"2026-01-01T00:00:00Z","creators":["Tool: bomly-fuzz"]},"packages":[]}`,
`{"bomFormat":"CycloneDX","specVersion":"1.4","version":1,"components":[]}`,
`{"bomFormat":"CycloneDX","specVersion":"1.5","version":1,"components":[]}`,
`{"bomFormat":"CycloneDX","specVersion":"1.6","version":1,"components":[]}`,
`{"artifacts":[],"artifactRelationships":[],"source":{"type":"directory","target":"."},"descriptor":{"name":"syft","version":"seed"}}`,
} {
f.Add([]byte(seed))
}

f.Fuzz(func(t *testing.T, raw []byte) {
if len(raw) > maxFuzzInputSize {
return
}
doc, target, err := UnmarshalAutoJSON(raw)
if err != nil || target == TargetSyftJSON {
return
}
if doc == nil {
t.Fatalf("successful %s parse returned nil document", target)
}
if _, err := MarshalJSON(doc, target, EncodeOptions{}); err != nil {
return
}
})
}
Loading