Repository navigation
fix: harden CrowdSec preset apply and hub data persistence - #1525
Merged
Merged
Conversation
Hub preset apply and configuration import now take a copy-based snapshot instead of renaming the CrowdSec data directory, and restore it in place on failure. Snapshots exclude engine-owned state (live database, data/ and hub_cache/), are capped at 5 and pruned after every attempt, and preset apply, import and file writes are serialized on one handler lock. File writes back up only the replaced file (kept to the last 10).
Apply stricter file validation across the CrowdSec configuration file endpoints, with bounded, per-file backups and atomic writes.
Hub preset apply now surfaces the server result and error like curated presets, and file saves show the server's error message.
Redirect CrowdSec data_dir onto the persistent volume (migrating existing installs), decide the hub install guard on the JSON installed flag instead of the inspect exit code, and run a gated, bounded cscli hub upgrade only when installed items are missing their data files. Adds a recreate integration test wired into the CrowdSec CI job.
The crowdsec-inline stage injected the version through a symbol that CrowdSec v1.8.1 does not define, so the linker silently ignored it and cscli/crowdsec reported an empty version. Point both builds at go-cs-lib/version.Version, the symbol upstream's own Makefile uses, and correct the stale N5 comment. Note: this changes the toolchain recipe key, so verify-toolchain-pin stays red until the toolchain workflow pushes the new CHARON_TOOLCHAIN_TAG/DIGEST pin bump to the branch. Those ARG lines are bot-owned and are not edited here. The build-time version assertion and the explicit cscli hub branch follow in the next commit.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
Contributor
✅ Supply Chain Verification Results✅ PASSED 📦 SBOM Summary
🔍 Vulnerability Scan
📎 Artifacts
Generated by Supply Chain Verification workflow • View Details |
… hub branch deterministic
Assert in the final-stage N5 check that cscli reports v${CROWDSEC_VERSION}
(ARG redeclared in the stage), and pin cscli.hub_branch to master in the
entrypoint-managed config.yaml so hub commands no longer depend on
version.crowdsec.net or a version-named hub-cdn branch. Operator-chosen
values are preserved; the setting is idempotent.
Enable the hub preset apply and file editor specs, replace the tolerant preset assertions in the CrowdSec config spec with deterministic ones, and anchor the file write route stub so it no longer shadows the file list endpoint.
…wdSec builds The x/mod pin was a hard-coded downgrade (v0.40.0) whose go get cascade dragged x/net, x/crypto and x/text back below the pinned versions in the bundled caddy, crowdsec and cscli binaries. Make it a shared XMOD_VERSION arg (v0.41.0), re-assert x/crypto and x/net after it, and fail each builder stage if the built binaries embed x/net or x/crypto below the pinned versions.
Enable codecov.notify.manual_trigger and add a notify-codecov job that runs send-notifications after the backend, frontend and agent uploads, so codecov/patch is no longer evaluated against a partial report.
…paths Tighten archive extraction and configuration file endpoints: stricter destination checks, truncating writes, bounded reads and generic error responses. Tests added.
Wikid82
marked this pull request as ready for review
October 9, 2026 16:58
Codecov re-evaluates codecov/patch on every upload, so the quick agent and frontend uploads turned the check red until the ~15 min backend upload arrived; the manual_trigger/notify-codecov approach from 513440f did not prevent this. The backend, frontend and agent jobs now only stage their coverage file as a 1-day artifact. A final `upload-codecov` job downloads them and uploads the three reports back to back with their original flags, so Codecov no longer sees partial reports (only a seconds-wide window between the uploads remains). codecov.notify.manual_trigger and the notify-codecov job are removed. To restore the previous behavior, revert this commit together with 513440f.
This was referenced Oct 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-ups to #1513 (curated CrowdSec preset apply), delivered as one PR because they share one defect family: unsafe mutation and non-durable state in the persistent CrowdSec tree. Plan:
docs/plans/current_spec.md; QA report:docs/reports/qa_report.md.Closes #1514, closes #1515, closes #1516, closes #1517, closes #1523, closes #1524
Part of #1518 (the remaining Playwright-spec cleanup is tracked in #1526 and will be the next piece of work).
data/,hub_cache/) is restored on failure, and backups are pruned (5 snapshots, 10 per-file). One handler lock serializes apply, import and file saves.crowdsec_preset_handler.go(pure move).data_dirmoves onto the persistent volume (with migration); the install guard no longer trustscscli inspectexit codes; a bounded, non-fatalcscli hub upgraderuns only when installed items' data files are missing.cscli versionequalsCROWDSEC_VERSION, andcscli.hub_branch: masteris pinned so startup no longer depends on the version lookup.golang.org/x/netkept at the pinned version in the Caddy and CrowdSec builds (with build-time guards), file-handling hardening in the apply/import paths, extra failure-path tests.ci:): Codecov notifications are now sent by a final job after all three uploads finish, to stop the early redcodecov/patch. This can only be confirmed on this PR's runs; revert that commit if it misbehaves.Testing
internal/crowdsecandinternal/api/handlerspass; coverage 90.9% statements; full golangci-lint config clean (backend and agent).scripts/crowdsec_data_persistence_test.shpasses 11/11 on a rebuilt image, including recreate-with-same-volume.CROWDSEC_VERSIONfails the build.hub_branch: master, startup no longer queries the version lookup; offline, download commands still fail as expected.--project=security-tests).Note: the
playwright-cicompose mount belongs to the separate upstream sidecar and is intentionally unchanged.Claude Code was used to help produce this change.