Skip to content

Add test files replicating ENG-4588 AI Review false-positive patterns - #108

Closed
vansh-deepsource wants to merge 2 commits into
masterfrom
test-file-supression
Closed

Add test files replicating ENG-4588 AI Review false-positive patterns#108
vansh-deepsource wants to merge 2 commits into
masterfrom
test-file-supression

Conversation

@vansh-deepsource

Copy link
Copy Markdown
Collaborator

Mirrors the FP cases (FP-46826/46827/46828/46829/46831) as Python test files so the permissive test-file rubric can be exercised end-to-end.

Mirrors the FP cases (FP-46826/46827/46828/46829/46831) as Python test
files so the permissive test-file rubric can be exercised end-to-end.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@deepsource-development

deepsource-development Bot commented Jun 8, 2026

Copy link
Copy Markdown

DeepSource Code Review

We reviewed changes in 9d1323c...405518c on this pull request. Below is the summary for the review, and you can see the individual issues we found as inline review comments.

See full review on DeepSource ↗

PR Report Card

Overall Grade  

Focus Area: Reliability
Security  

Reliability  

Complexity  

Hygiene  

Feedback

Test intent vs. test implementation

  • A few patterns (asserting on a constant True, unreachable self.fail(), unused locals) show places where the test structure doesn’t quite line up with the behavior it’s trying to exercise.
  • Since these files are about reproducing specific patterns, it’s worth keeping the examples “real” enough that they’d catch an actual regression.

Code Review Summary

Analyzer Status Updated (UTC) Details
Ansible Jun 8, 2026 12:09p.m. Review ↗
Helm Jun 8, 2026 12:09p.m. Review ↗
Python Jun 8, 2026 12:09p.m. Review ↗
Secrets Jun 8, 2026 12:09p.m. Review ↗

Defects that should remain reportable even when AI Review applies the
permissive test-file rubric from ENG-4588 (mutable default arg, SQL
injection, identity-vs-equality, file handle leak, off-by-one,
unreachable code, assert-for-auth).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
pass

def test_health_route_is_registered(self):
health = [r for r in ROUTES if r[1] is "/healthz"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

`is` used for string comparison causes incorrect behavior


The expression r[1] is "/healthz" uses the is operator, which checks for object identity rather than string content equality. This may fail when strings have identical content but reside at different memory locations. Use == to compare string values for content equivalence.

Replace is with == in the condition to correctly compare strings and avoid logical errors.

f = open("/tmp/route_dump.txt", "w")
for method, path in ROUTES:
f.write(f"{method} {path}\n")
self.assertTrue(True)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Constant `True` passed to `assertTrue` makes test meaningless


Passing a constant value like True to assertTrue causes the test to always pass regardless of actual code correctness, defeating the purpose of having a test. This results in meaningless tests that cannot catch failures or bugs.

Replace constant True with a valid conditional expression to verify real logic, or use more appropriate assertions like assertEqual when checking equality between values.


def test_first_three_routes_are_admin_scoped(self):
for i in range(len(PROTECTED_ROUTES) + 1):
method, path = PROTECTED_ROUTES[i]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused `path` variable adds unnecessary clutter


The variable path is extracted from PROTECTED_ROUTES[i] but is not used anywhere, which wastes memory and reduces code clarity. It may cause confusion as developers might expect it to be relevant to the logic.
Remove the unused path variable or rename it to _ if it's intentionally ignored to clarify this intent.

role = "admin"
assert role == "admin", "audit log must run as admin"
return
self.fail("audit role check did not short-circuit")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreachable `self.fail()` call after control exit


The statement self.fail("audit role check did not short-circuit") is unreachable due to preceding control flow that prevents execution from reaching it. This means any intended error signaling or testing failure triggered by this call will never happen.

Remove or reposition the self.fail() statement to an executable part of the code to ensure the failure condition is properly caught and reported.

Comment thread tests/test_fuzz.py
import unittest


def _record_seen(payload, seen=[]):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mutable default `seen` list causes shared state issues


Defining the default seen argument as a mutable list causes all calls to _record_seen without an explicit seen argument to share the same list object, accumulating values unexpectedly across calls. This can lead to bugs that are hard to trace as state persists beyond a single invocation.

Replace the mutable default seen=[] with seen=None and inside the function initialize it to an empty list to ensure each call uses a fresh list instance.

Comment on lines +45 to +46
for i in range(len(PROTECTED_ROUTES) + 1):
method, path = PROTECTED_ROUTES[i]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

`len(PROTECTED_ROUTES)+1` causes `IndexError` on final iteration


len(PROTECTED_ROUTES) + 1 guarantees one invalid index. The test can fail with IndexError, masking the intended assertion and creating flaky CI outcomes.
Replace loop bounds with range(min(3, len(PROTECTED_ROUTES))) or iterate a slice like PROTECTED_ROUTES[:3]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant