Skip to content

test(scoring): fix stale empty-tags reachability expectations (main is red)#123

Open
Amr-Saad wants to merge 1 commit into
Agent-Field:mainfrom
Amr-Saad:fix/scoring-empty-tags-reachability-default
Open

test(scoring): fix stale empty-tags reachability expectations (main is red)#123
Amr-Saad wants to merge 1 commit into
Agent-Field:mainfrom
Amr-Saad:fix/scoring-empty-tags-reachability-default

Conversation

@Amr-Saad

@Amr-Saad Amr-Saad commented Jun 9, 2026

Copy link
Copy Markdown

Two assertions in tests/test_scoring.py still encode the old reachability default (requires_auth, ×0.5) for findings with no reachability tags, so main is currently red on them.

src/sec_af/scoring.py was intentionally changed so an empty tag set defaults to externally_reachable (×1.0) — documented inline in _reachability_multiplier:

The previous default of requires_auth (0.5) severely penalised every finding when reachability assessment is not wired into the DAG path — causing critical CWEs to score 2.5/10.

The file’s other tests already use explicit reachability tags consistent with the new model; only these two lagged.

Changes (tests only)

Test was now math
…defaults_reachability_when_missing 1.05 2.1 LOW 3.0 × SANITIZATION_BYPASSABLE 0.7 × externally_reachable 1.0
…multipliers_and_default_behavior[set()] 2.5 5.0 MEDIUM 5.0 × FULL_EXPLOIT 1.0 × externally_reachable 1.0

Added inline comments so the empty-vs-non-empty default ({custom_tag} still falls back to requires_auth ×0.5) is explicit.

Verification

  • Full suite: 105 passed (was 2 failed), via pip install -e .[dev] + pytest.
  • ruff check tests/test_scoring.py: clean.
  • No production code touched.

🤖 Generated with Claude Code

Two assertions in test_scoring.py still encode the old reachability default
(requires_auth, x0.5) for findings with no reachability tags. scoring.py was
intentionally changed so an empty tag set defaults to externally_reachable
(x1.0) - documented in _reachability_multiplier: the old x0.5 default
'severely penalised every finding ... causing critical CWEs to score 2.5/10'.
The file's other tests already use explicit reachability tags consistent with
the new model; only these two lagged, leaving main red.

- defaults_reachability_when_missing: 1.05 -> 2.1
  (LOW 3.0 x SANITIZATION_BYPASSABLE 0.7 x externally_reachable 1.0)
- reachability_multipliers_and_default_behavior[set()]: 2.5 -> 5.0
  (MEDIUM 5.0 x FULL_EXPLOIT 1.0 x externally_reachable 1.0)

Full suite now green (105 passed); ruff clean on the file. Tests-only change;
no production code touched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants