Skip to content

[Bug] SQL equivalence and compare tools return contradictory results #64

Closed

Description

@anandgupta42

Problem

altimate_core_equivalence and altimate_core_compare give opposite results for the same query pair. One reports "DIFFERENT" while the other reports "IDENTICAL" for semantically equivalent queries that only differ in condition ordering.

This directly erodes user trust in automated analysis.

Steps to Reproduce

Compare these two queries:

-- Query A
SELECT id FROM t WHERE a = 1 AND b = 2 ORDER BY c

-- Query B  
SELECT id FROM t WHERE b = 2 AND a = 1 ORDER BY c ASC
  • altimate_core_equivalence → reports "DIFFERENT"
  • altimate_core_compare → reports "IDENTICAL"

The queries are semantically identical (AND is commutative, ORDER BY c = ORDER BY c ASC).

Expected Behavior

Both tools should agree. For semantically equivalent queries with reordered conditions:

  • Equivalence check should report EQUIVALENT (commutativity of AND)
  • Compare should report IDENTICAL or EQUIVALENT

Investigation Needed

  1. Understand the difference between the two tools — what is each tool's intended purpose?
    • Is equivalence doing AST comparison (syntactic) while compare does semantic analysis?
    • Or vice versa?
  2. Define clear semantics for each — if one is syntactic and one is semantic, document this clearly and adjust naming
  3. Consider unifying — having two tools that answer the same question differently is confusing. Options:
    • Merge into a single sql_compare tool that reports both syntactic and semantic similarity
    • Keep both but return structured results: { syntactic: "different", semantic: "equivalent", explanation: "..." }
    • Remove the less accurate one

Impact

  • Users who rely on one tool get wrong answers
  • The agent compensates by calling both and explaining the contradiction, but this wastes tokens and looks unreliable
  • For compliance/audit use cases, contradictory results are unacceptable

Acceptance Criteria

  • Root cause identified (why the tools disagree)
  • Tools either unified or given clearly distinct purposes with accurate naming
  • Reordered WHERE conditions correctly identified as equivalent
  • ORDER BY c and ORDER BY c ASC treated as identical
  • No contradictory results for the same input pair

Activity

  1. anandgupta42 commented on Nov 5, 2026

    @anandgupta42
    ContributorAuthor

    Closing as fixed on main: altimate_core_compare now reads the correct result fields (altimate-core-compare.ts), and equivalence handles reordered conjuncts (covered by tests). Please reopen if you still see this on v0.12.9 or later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority:highHigh prioritysql-engineSQL analysis, lineage, translation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions