Skip to content

Scan output is non-deterministic: array ordering and language classification both flip between runs #676

Description

@hyperpolymath

Found by gitar-bot on hyperpolymath/verisimdb-data#72 and verified against the actual diff before filing. Both findings are correct.

Symptom

A rescan with no upstream change produces a diff of +20,277 / −20,277 across 249 files — exactly symmetric, because essentially none of it is real.

1. Array ordering is unstable

Elements are re-emitted in a different order with identical content. Verified in scans/dafniser.json:

+    "disk",
-    "memory",
-    "disk"
+    "memory"

gitar cites the same pattern in recommended_attacks (Axiom.jl.json:916-921), dependency_graph.edges (hermeneia.json:385-437, cloudguard-server.json:161-310) and taint_matrix.rows (kea.json:240-281, chapeliser.json:152-196).

Suggested fix: sort before serialising — edges by from/to, rows by source_category+sink_axis, attack lists lexically. If these are built from a HashMap/HashSet, that is the likely cause; BTreeMap or an explicit sort at the emit boundary fixes it.

2. Language classification flips, and inconsistently

Verified in the diff:

-  "language": "idris"
+  "language": "rust"     # scans/dafniser.json
-  "language": "elixir"
+  "language": "scheme"   # anvomidav

gitar additionally reports halideiser idris→rust, lustreiser rust→idris (the opposite direction), awesome-idris2 scheme→idris, i-human zig→idris, unified-dataset-vocab idris→zig, refugia nickel→shell, rescript-ecosystem javascript→shell.

The mutual inconsistency is the tell: if this were a genuine reclassification the flips would share a direction. Opposite-direction flips in the same run point at an arbitrary tie-break on multi-language repos.

Suggested fix: make detection deterministic — stable ordering, with an explicit tie-break by file count and then by a fixed language priority.

Why this matters beyond noise

  • Any downstream analysis keyed on language is being fed a value that changes for no reason.
  • A 249-file symmetric diff cannot be reviewed, so a genuine regression arriving in the same rescan would pass unnoticed. That is a review gate defeated by noise rather than by a bug.

Filed from verisimdb-data#72; both gitar threads there point here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions