ci: bound the ML-DSA fuzz corpus with periodic set-cover minimization - #265
Merged
Merged
Conversation
The nightly liboqs fuzz job grows a persisted corpus every run; over many nights it accumulates coverage-equivalent (redundant) inputs, bloating the corpus repo (now 14k+ files) and slowing each run's git clone. Before the nightly --generate, set-cover-minimize the persisted corpus (weekly on Sundays, or on demand). The merge keeps the minimal covering set — same coverage, far fewer files — and generate then regrows it from a lean base, so the corpus stays bounded instead of growing without limit. Reducing fuzz frequency was rejected: it would sacrifice coverage depth and still not bound the size. - 03_test_script.sh: run test_runner.py --m_dir (set_cover_merge) on the corpus when FUZZ_MINIMIZE=true or it is Sunday (date -u +%u == 7), then --generate. - 00_setup_env_native_fuzz_liboqs.sh: export FUZZ_MINIMIZE (default false) so a forced value is forwarded into the CI container. - fuzz-liboqs-nightly.yml: add a force_minimize workflow_dispatch input to shrink the corpus on demand (e.g. the initial one-off cleanup of the current backlog).
…try) The corpus push did a plain 'git push origin HEAD:main', which fails as a non-fast-forward if the corpus repo's main advanced relative to the shallow clone (e.g. another run pushed). The computed corpus (minimized and/or grown) is authoritative, so fetch the current remote main, soft-reset our commit onto it, and retry the push. Fixes the minimized corpus (14760 -> 435 inputs) failing to land.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
Fuzz ML-DSA (liboqs) nightlyjob grows a persisted corpus (avian-fuzz-corpus) on every run viatest_runner.py --generate. Over ~3 weeks of nightlies it has accumulated 14k+ files — the vast majority coverage-equivalent (redundant) inputs. That bloats the corpus repo (GitHub now truncates the directory listing) and, more importantly, slows every run: the nightlygit clones the entire corpus each time.Reducing the fuzz frequency (e.g. weekly) was considered and rejected — it sacrifices the coverage depth that is the whole point of the liboqs nightly, and it does not actually bound the corpus size, only slows its growth.
Change
Bound the corpus with periodic set-cover minimization, keeping full coverage while collapsing the file count (typically a large reduction). Before each nightly
--generate, when it is Sunday (or on demand), runtest_runner.py --m_dir— which uses libFuzzer's-set_cover_merge=1 -prefer_small=1— to reduce the corpus to its minimal covering set;--generatethen regrows it from a lean base. So the corpus oscillates around a bounded minimum instead of growing without limit, and clones stay fast.ci/test/03_test_script.sh: in theFUZZ_GENERATEbranch, set-cover-minimize the persisted corpus whenFUZZ_MINIMIZE=trueordate -u +%u == 7(Sunday), then run--generateas before.ci/test/00_setup_env_native_fuzz_liboqs.sh:export FUZZ_MINIMIZE(defaultfalse) so a forced value is forwarded into the CI container..github/workflows/fuzz-liboqs-nightly.yml: add aforce_minimizeworkflow_dispatchinput wired toFUZZ_MINIMIZE, for shrinking the corpus on demand.Rollout
After merge, trigger one manual run with
force_minimize: trueto shrink the current 14k backlog immediately; thereafter the Sunday minimization keeps it bounded automatically. Coverage is preserved throughout (set-cover keeps every distinct-coverage input). If the fuzzer ever finds a real bug, the job still fails as before — minimization runs a clean merge and does not mask crashes.