Skip to content

Add delete-existing-comments option#57

Open
bertonjulian wants to merge 7 commits intoanthropics:mainfrom
bertonjulian:fix/post-new-findings-with-existing-comments
Open

Add delete-existing-comments option#57
bertonjulian wants to merge 7 commits intoanthropics:mainfrom
bertonjulian:fix/post-new-findings-with-existing-comments

Conversation

@bertonjulian
Copy link
Copy Markdown

@bertonjulian bertonjulian commented Jan 23, 2026

Problem

When using run-every-commit: true, the security review runs on every commit pushed to a PR. However, the comment posting logic skips ALL new comments if any existing security comments are found on the PR:

Found 8 existing security comments, skipping to avoid duplicates

This means:

  1. First commit: 8 security findings posted as comments ✓
  2. Second commit introduces 2 NEW vulnerabilities: 0 comments posted

The action detects the new issues but refuses to post them because existing comments are present. New vulnerabilities introduced in subsequent commits are silently ignored.

Solution

Adds a new delete-existing-comments input option that deletes existing security comments before posting new ones. This ensures all current findings are posted fresh on each run.

New Input

Input Description Default
delete-existing-comments Delete existing security comments before posting new ones. Ensures no duplicate or stale comments. false

Usage

- uses: anthropics/claude-code-security-review@main
  with:
    claude-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
    run-every-commit: true
    delete-existing-comments: true  # Recommended when using run-every-commit

Why Optional?

  • Default behavior remains unchanged (no breaking change)
  • Users can opt-in when using run-every-commit
  • Same approach used by reviewdog

What It Does

When enabled:

  • Fetches existing PR comments
  • Deletes comments matching 🤖 **Security Issue: from bot users
  • Posts all new findings fresh

Only security comments from bots are deleted - regular review comments are preserved.

Test plan

  • Added unit tests for delete behavior when flag is enabled
  • Added unit test to verify no deletion when flag is not set
  • Added unit test to verify non-security comments are preserved
  • Updated README documentation

🤖 Generated with Claude Code

bertonjulian and others added 4 commits January 23, 2026 16:07
Previously, the script would skip ALL new comments if it detected any
existing security comments on the PR. This meant that when new commits
introduced additional vulnerabilities, they would not be reported.

This fix changes the deduplication logic to:
1. Check each new finding against existing comments
2. Skip only findings that are true duplicates (same file + same message)
3. Post the remaining genuinely new findings

Includes test cases for the new behavior.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The description can vary between runs, so using it for deduplication
caused false negatives. Changed to use stable fields:
- file path
- line number
- category

This ensures findings are only considered duplicates when they refer
to the exact same location and type of vulnerability.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Instead of trying to deduplicate comments, post all findings fresh
and let GitHub's native "outdated" marking handle stale comments.

When code changes, GitHub automatically marks comments on affected
lines as "outdated" and hides them from the main view. This is
simpler and more reliable than custom deduplication logic.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Instead of trying to deduplicate or relying on GitHub's outdated marking,
delete all existing security comments from the bot before posting new ones.

This ensures:
- No duplicate comments
- No stale comments pointing to old line numbers
- Clean slate on each run

This is the same approach used by reviewdog.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@bertonjulian bertonjulian changed the title Fix: Post new findings even when existing comments exist Fix: Delete existing security comments before posting new ones Jan 23, 2026
bertonjulian and others added 2 commits January 23, 2026 16:42
Add new `delete-existing-comments` input (default: false) to optionally
delete existing security comments before posting new ones.

This allows users to opt-in to the clean slate behavior rather than
making it the default.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@bertonjulian bertonjulian changed the title Fix: Delete existing security comments before posting new ones Add delete-existing-comments option Jan 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant