Skip to content

Fix the audit's weak spots: injection and hardcoded-value considers, TLS and escape reviews - #28

Merged
tauanbinato merged 9 commits into
mainfrom
weak-spots
Sep 26, 2026
Merged

tauanbinato merged 9 commits into
mainfrom
weak-spots

Conversation

@tauanbinato

@tauanbinato tauanbinato commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Follows #26 and #27. The audit left four weak spots in findings people read:

  • hardcoded-value considers that rest on a value needing a name (41% right)
  • injection considers that rest on a function's parameters (37% right)
  • TLS reviews on connections that use no TLS
  • an escape review on i18next in a React Native app

Each gets a follow-up asked only for such findings, or a reworded check.

What changes

  • Injection: a consider that rests on the function's parameters ("a caller passing outside input would make it exploitable") is asked, with the functions that call it, what the values it places can hold.
    • It becomes a note when the program's own values lead: text the program fixes (literals its callers pass, numbers, names from a fixed list), values it creates, or a local tool's own arguments.
    • Text another party can set, or callers that aren't shown, keep it.
    • Why a new question: the recheck's origin question answered "the function's parameters" at 0.9 even for a Rust helper whose four callers pass literal SQL fragments, and that answer isn't wrong.
  • Hardcoded values: the locate Choice lists, with each value, the other lines of its file that write it. A consider that rests only on a value's name, and whose file writes that value again, is then asked what the value is.
    • Copies that must change together, or a value nothing explains, keep it.
    • A value explained by the field or argument it fills or a comment beside it, an idiom (a tolerance near zero, a half, a unit conversion) or a hand-tuned number make it a note, when those options together reach 0.65, the threshold for a clear located part.
  • Unsafe settings: outside C#, PHP and Django, the TLS check asks whether code accepts a server's certificate, host key or host name without verifying it.
    • A connection made without TLS (gRPC's insecure credentials, an insecure OpenTelemetry exporter, sslmode=disable) presents nothing to verify.
    • Verification skipped only when a caller or the operator asks for it isn't turned off.
    • The escape check names i18next's escapeValue: false in a React app as escaping kept.

Results

Labeled findings on the tuned and fresh projects (not held out), main → this branch:

Right Wrong Share right
Injection considers resting on parameters 30 → 30 51 → 32 37% → 48%
Hardcoded "must guess" considers 53 → 32 77 → 28 41% → 53%
Unsafe-settings reviews 54 → 54 24 → 14 69% → 79%
All reviews 519 → 519 188 → 179 73% → 74%
All considers 850 → 830 404 → 337 68% → 71%
  • Against 0.20.0 on the 72 tuned projects: 76% of reviews right (142 wrong, from 192) and 73% of considers (239 wrong, from 355).
  • 14 fresh projects: 64% of reviews and 65% of considers right, against 53% and 47% with 0.20.0 and 59% and 57% with main.
  • 11 held-out projects: one fewer wrong review, otherwise unchanged. They hold few findings of these kinds.
  • 13 findings are new or changed and were labeled: 3 right, 9 wrong, 1 debatable. Most are hardcoded values the new locate evidence named. RailsGoat's API token check against a salt written in the code is a new right review, and a CORS setting in a development-only settings module is a new wrong one.
  • Corpus cost, all iterations included: about $0.4.

Review notes

  • What was lost:
    • pgweb's bookmark reader defaulting sslmode to disable for remote servers was a right review and is now a note. It drops TLS rather than verification, which the finding's message misdescribed.
    • 21 right hardcoded-value considers are notes. Most are values whose copies must stay equal, which the model also read as idioms.
  • What didn't work:
    • The first TLS rewording only added examples to the check's "no" (plaintext to services of the same deployment, then any plaintext). Online Boutique's gRPC clients stayed reviews with both, so the question itself changed.
    • For hardcoded values, offering the readable kinds as the locate's "none" turned 6 right considers into notes for 17 wrong ones.
    • Telling the kind Choice that copies win over the other kinds made it pick copies for wrong considers too: 22 notes instead of 42.
  • Request changes:
    • Injection: only the new follow-ups are asked. Rechecks share their state builder with them, and their requests are byte-identical, so the cache still answers them.
    • Hardcoded values: the locate follow-ups change.
    • Unsafe settings: unsafe-settings traces outside C#, PHP and Django change.
  • Self-check: the branch's self-check leaves no unbaselined review or consider. It had flagged the new follow-up's state as a copy of the recheck's, now shared. CI's review runs the released 0.20.0 and adds one consider on asked_checks in src/units/security.rs ("mixes separate jobs"). That function is unchanged here, and this branch's rules don't flag it.

…an hold

A consider that says "a caller passing outside input would make it
exploitable" is asked, with the functions that call it, what the values
it places can hold: text the program fixes, values it creates or a local
tool's own arguments make it a note; text another party can set, or
callers not shown, keep it. The recheck's origin question answered "the
function's parameters" at 0.9 even when every caller passed a literal.
Labeled such considers went from 30 right and 51 wrong to 30 and 32.
…lue what it is

The Choice that names a finding's value lists, with each value, the
other lines of its file that write it. A consider that rests only on a
value's name, whose file writes that value again, is then asked what the
value is: copies that must change together, or a value nothing explains,
keep it; a value its field, argument or comment explains, an idiom or a
hand-tuned number, clearly chosen, make it a note. Labeled such
considers went from 53 right and 77 wrong to 32 right and 28 wrong.
…i18next's escaping in React

Asked whether code turns certificate verification off, Online Boutique's
gRPC clients with insecure credentials, telemetry exporters and a
database behind the AlloyDB connector were seven reviews whatever the
check's examples said. The TLS check now asks whether code accepts a
server's certificate, host key or host name without verifying it: a
connection without TLS presents nothing, and verification skipped only
when a caller or the operator asks is not turned off. The escape check
names i18next's escapeValue: false in a React app as escaping kept.
Labeled unsafe-settings reviews went from 54 right and 24 wrong to 54
and 14.
A hardcoded-value review or consider that rests only on a value changing
between environments is asked, once its value or constant is named and
with the code that uses it, where the value would differ: each
installation or the author's own account keeps it; the same value in
every copy on purpose, a fallback used only when configuration gives
none, or code no deployment runs, at 0.80, make it a note. The Choice
that names a file's constant lists the lines that use each one. Labeled
such findings went from 17 right and 36 wrong to 15 right and 19 wrong.
A path check left undecided after its trace and recheck is asked, with
the functions that call it, where the paths come from: the program's own
directories or configuration, the command line or settings of the person
running a local program, a whole path its caller gives, or another
party's input. The program's own or the local user's clear it: 74 units
on the corpus, for about $0.02.
Parameters in a path, URL or redirect were a note once a check found
them, but stayed uncertain while the check was undecided. Once the
Choice that settles the check was asked and does not lean toward another
party's input, such a unit is a note naming the path, URL or redirect:
450 injection units had stayed uncertain that way.
A test whose hollow checks stay undecided once its recheck is asked is
clear when they lean below 0.50. Labeled from the code, 4 of 43 such
tests below 0.50 were hollow, against 10 of 35 above; undecided tests on
the corpus went from 792 to 156, with nothing asked again.
Undecided units on the corpus went from 1.39% to 0.83% of judged units
with these changes and the environment Choice before them; no review or
consider changed.
@tauanbinato
tauanbinato merged commit 253e702 into main Sep 26, 2026
9 checks passed
@tauanbinato
tauanbinato deleted the weak-spots branch September 26, 2026 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant