Audit 0.20.0 on 103 projects: crash and hang fixes, security checks, precision caps

CHANGELOG.md

@@ -4,6 +4,33 @@ Notable changes to JevGate. Versions follow [Semantic Versioning](https://semver
 
 ## [Unreleased]
 
+Measured on 103 pinned projects (24 new open-source ones of kinds not tried before, among them intentionally vulnerable Rails, Node, GraphQL, C# and Java apps, a Deno framework, a WordPress plugin, a cookiecutter template and projects in Kotlin, Swift, Elixir and C, and 8 more of the maintainer's own), with findings labeled by hand: on the 70 labeled projects JevGate was tuned on, 75% of reviews were right against 69% with 0.20.0 (136 wrong reviews against 192), and 72% of considers against 65% (254 wrong considers against 354); on 11 held-out projects, 61% of reviews against 57%, and 56% of considers against 54%. Undecided units went from 2.2% to 1.5% of judged units.
+
+- Server templates: ERB, EJS, JSP, Handlebars, Mustache, Nunjucks, Twig, Jinja and Go templates, and HTML under `templates/`, `views/`, `layouts/`, `partials/` or `includes/`, are judged. Their inline `<script>` code is parsed with the template's tags blanked and judged as the page's JavaScript in the visitor's browser, its top-level statements by every security rule, as a PHP page script is. Their `template code` is one more unit: each tag that writes request, cookie, session or signed-in-user data unescaped (ERB `raw` and `html_safe`, `<%== … %>`, EJS `<%- … %>`, `{{{ … }}}`, `|safe`, `|raw`), judged by injection, and every scriptlet and declaration of a JSP page once one reads the request, judged by every security rule. A template holding neither is not selected, so 900 of the corpus's 1,003 templates stay out of reports. On the intentionally vulnerable apps, 24 documented vulnerabilities that no rule read are found: RailsGoat's and DVNA's three XSS each, DVGA's paste-page XSS, DVJA's reflected XSS, and JavaVulnerableLab's JSP-only SQL, HQL and command injections, XSS, path traversal, SSRF and leaked stack traces. Of the labeled findings in templates, 42 of 49 reviews and 16 of 25 considers were right. About $0.1 on the corpus.
+- Node views: a handler that renders a view by name (`res.render('app/products', …)`, a view under `views/` named without its extension) is sent the view's lines that write values unescaped (EJS `<%- … %>`, Handlebars `{{{ … }}}`, Pug `!=`, Nunjucks's and Swig's `|safe`), as a Django view is sent its templates, and its presence question, markup check and markup Choice name such templates. DVNA's reflected XSS in the product search is a review and the stored XSS of its product list a consider; only such handlers' requests change.
+- A check no longer panics on text of several bytes: splitting SQL stepped into a character (pgweb's `booktown.sql` holds U+FFFD outside quotes), and locating a Python block that ends in a comment ending in `线` (vnpy) sliced inside it; both aborted the run with exit 101.
+- An agent instruction file is looked up by its exact name: on a case-insensitive file system (macOS, Windows), probing for `AGENTS.md` also opened refined-github's `agents.md`, and `CLAUDE.md` a React Native template's `claude.md`, whose read then failed as a symlinked path and left the whole run incomplete (exit 2). An instruction file that links to a file no harness reads by name, such as refined-github's `CLAUDE.md` pointing at that `agents.md`, now loads it: the target is judged as that harness's instructions and counted once toward what a session loads.
+- A document that is not text (NUL bytes, or not UTF-8) is skipped with a reason, as a source file is, instead of making the run incomplete: one Markdown file in dvja's docs failed the whole check with exit 2.
+- Planning works out which files import which once per file instead of once per function: jellyfin's dry run ran past half an hour and takes 17 seconds, and laravel/framework's took 254 seconds and takes 88. Requests are unchanged on every corpus project.
+- Planning keeps each follow-up request (traces, rechecks, settles, located parts) as its JSON text until it is asked, instead of as a JSON value, since most are never sent: a dry run on laravel/framework's 3,000 PHP files peaked at 3.2 GB for the security rules and 4.1 GB for all rules, and takes 1.5 GB and 2.2 GB. Requests are unchanged on every corpus project.
+- Access control: the SECURITY DEFINER question says that PostgreSQL lets every role execute a new function unless a revoke takes it from public, and asks about other users' rows or stored files: chatbot-ui's `delete_storage_object` and `delete_storage_object_from_bucket`, which let anyone delete any stored file with the service role key, were `search_path` considers and are reviews. A policy that lets others read rows their owners marked shared or public (`sharing <> 'private'`) is acceptable. Only access-control requests are asked again.
+- Unsafe settings: code outside C# and Django is asked the token and key checks C# asks, and every language whether code turns off HTML escaping or keeps passwords as plain text; a cookie added to a request or set empty to delete it is no session cookie without flags, and WordPress's `wp_rand` is cryptographic. The token, password and logging checks are settled by Choices that name what the code does, asked whenever they are not clear. A password or token finding stays a review only when the function itself hashes with a fast hash or turns a library's verification off; otherwise, since a callee, a model hook or the platform may hash or verify, it is a consider. On the corpus, 16 right findings in intentionally vulnerable apps became reviews or considers where they had been notes or nothing (NodeGoat's and JavaVulnerableLab's plain-text passwords, DVGA's, pygoat's, DVNA's and RailsGoat's unverified or forgeable tokens, govwa's unescaped templates and MD5 passwords), with 4 wrong ones; labeled unsafe-settings reviews went from 75% to 78% right. Only unsafe-settings traces and settles are asked again.
+- Sensitive data: what a function's logs write now tells who did what (an audit line naming who signed in) and values a command-line tool shows its operator on purpose from other personal data, and is asked whenever a logging signal is not clear: 17 logging reviews are gone, and every labeled one was wrong. Where a function's text goes is asked for reviews too, and names a program's own screens as local. Labeled sensitive-data reviews went from 43% to 49% right.
+- Desktop, game and terminal programs: a file whose package depends on an interface toolkit (`ratatui`, `egui`, `iced`, `bevy`, `tauri`, `electron`, `spacetimedb-sdk` and others) is named to Jev as a client application that runs on its user's machine, so the errors it shows its own screens are not sent to a remote client: a game client handing the server's error text to its own window over a channel of `Response` messages was 16 wrong reviews for sending internal details to a remote client, all gone. Only such packages' requests change.
+- GraphQL in Python: a file that imports graphene, strawberry or ariadne is named to Jev as GraphQL server code, so its resolvers' arguments read as client input: DVGA's SQL injection, SSRF and command injection through `resolve_*` and `mutate` arguments were considers or a note and are reviews.
+- Security findings in a file at a test path, judged as application code because it holds no tests (a test app's settings, a model only tests use), are one level lower, like code that runs only in development: devise's and clearance's dummy apps held the only three such reviews across the corpus, all wrong.
+- Hardcoded values: a finding that rests only on whether a value needs a name is at most a consider, since naming a value is a cleanup (17 such reviews were right and 18 wrong, most of those tuning in game, audio and animation code), and a note when its file writes the value once (19 of 52 such considers were right, against 34 of 49 for values the file repeats). Labeled hardcoded-value considers went from 43% to 52% right; 156 considers became notes. Nothing is asked again.
+- File organization: a file that writes out the same kind of code for each of several features, such as a mailer's function per email template, is one job, so when the split Score stays undecided, naming that kind clears it instead of raising a consider. Both such considers on the corpus were wrong (vaultwarden's mailer and a game's admin reducers per kind of map content), as was one on JevGate's own code; undecided file outlines went from 106 to 45. Nothing is asked again.
+- Large documents: a split finding is asked what kind of document it is, as an undecided split was, and the kinds gain a plan for one change and requirements; a kind that serves one subject clears it. Read from headings alone, dated release plans, READMEs, the RealWorld frontend instructions and a list of business rules held "several unrelated subjects": 14 split considers are gone, 6 of them labeled and all wrong. A large-docs note now says the document "may mainly record" past work. Only the kind follow-ups are asked, about $0.002 on the corpus.
+- Comments: a comment still undecided once its kind is asked leans on the kind: toward a kind a reader could do without (repeating the code, narrating an edit, past work, code turned off), a note; otherwise clear. Step headings such as `// update any single tag` above `this.addTag()` stayed between the thresholds on every ask: 1,153 comments on the corpus, each leaving its file uncertain. Nothing is asked again.
+- Agent instructions: a section of fewer than 15 tokens is a note: 1 of 10 labeled findings on such sections was right, most of them a title and a `Last updated` line read as a record of past work. Labeled agent-context considers went from 86% to 97% right.
+- Shared logic: a Go module (`go.mod`) is a package, as a Node, Rust or Python package is, so copies between modules that neither require each other nor share a local module are not compared. Online Boutique's Go services are each their own module, built on their own, and 9 of the 10 copies found between them were wrong. Only that project's requests change on the corpus.
+- Shared logic: copies in a directory named `deprecated`, `archive`, `attic`, `retired`, `obsolete` or proof of concept are not compared, as copies in code marked deprecated are not: a Unity project's retired proof builders were paired with its live scene builders in six wrong reviews.
+- Documentation: Claude Code skills, commands and subagent definitions (Markdown under `.claude/skills`, `.claude/commands` and `.claude/agents`) are project documentation, checked for stale paths, repetition and size; a session loads only their descriptions, so they do not count toward what loads at its start. One project's skill cited documentation paths a rename had removed, which nothing reported. A path holding a `$` placeholder, such as a command's `.kiro/specs/$1/spec.json`, names no file.
+- Project templates: a file under a directory whose name holds a `{{ … }}` placeholder, as a cookiecutter template's `{{cookiecutter.project_slug}}` does, is parsed without its Jinja tags (statements and comments blanked, each placeholder read as a name of the same length, so lines stay the file's) and judged; the evidence keeps the tags. 31 of cookiecutter-django's Python and JavaScript files, the generated application's settings, models, views and tests, were skipped for syntax errors and are judged; its Celery settings turning off Redis certificate checks (`ssl.CERT_NONE`) are a review.
+- Tests: a test said to assert internal details is asked, with the bodies of the functions it calls, what its assertions read: results, state the program shows or acts on next, or effects a caller observes clear the consider; stored input or calls between the program's own functions keep it. Asked of the test and the signatures it calls, the check read a debug panel's recorded queries (`panel._queries`, which the panel renders), Devise's documented hooks and an app's state after an action as internals: 49 of 66 such considers labeled on the corpus were wrong. A test that reads members through reflection or a cast to `any` keeps its consider without being asked. Labeled tests/value considers went from 29% to 77% right (20 right and 6 wrong, against 22 and 55), and on held-out projects from 2 right and 13 wrong to 1 wrong. About $0.006 of follow-ups on the corpus.
+- Tests: Deno tests are test cases, in each of their forms: `Deno.test("name", fn)`, `Deno.test({ name: "name", fn() {…} })` and `Deno.test(function name() {…})`, with `.only` and `.ignore`. oak writes its 266 tests in the object form, and none of them was judged: its test files got a file-purpose request each and the test rules found nothing to ask. Only Deno projects' requests change.
+
 ## [0.20.0] - 2026-09-26
 
 - Injection: code that names a deserializer that can build any object is asked whether it loads data another party sends, in Python, Ruby, Java, JavaScript and TypeScript: Python's `pickle`, `marshal`, `shelve`, `jsonpickle` and `yaml.load`, Ruby's `Marshal.load` and `YAML.load`, Java's `ObjectInputStream`, `XMLDecoder`, XStream and SnakeYAML, and node-serialize. Only Django views and PHP `unserialize` were asked before, so a Flask route passing `pickle.loads(request.get_data())` was clear; it is now a review (CWE-502). Other requests are unchanged, so cached answers stay valid.

README.md

@@ -33,7 +33,7 @@ Consider (2):
 | Security | Injection, sensitive data, unsafe settings, SQL access control, GitHub workflows; each finding names a CWE | `--rule security` |
 | Documentation | Agent instruction files, large and stale docs, duplicated sections, code comments | `--rule documentation` |
 
-It reads Rust, Python, JavaScript, TypeScript, Go, C#, Ruby, PHP and Java, the scripts of Astro, Vue and Svelte files, SQL for PostgreSQL and Supabase, GitHub Actions workflows, and Markdown, MDX, reStructuredText and AsciiDoc, and knows the routes, handlers and settings of frameworks from Express, Next.js and SvelteKit to Django, Laravel, ASP.NET Core and Spring MVC. [What it finds](https://tech-byte-frontier.github.io/jevgate/what-it-finds.html) and [supported languages and frameworks](https://tech-byte-frontier.github.io/jevgate/languages.html) have the details; `jevgate rules` prints every rule with the question it asks.
+It reads Rust, Python, JavaScript, TypeScript, Go, C#, Ruby, PHP and Java, the scripts of Astro, Vue and Svelte files and the inline scripts of server templates (ERB, EJS, JSP, Handlebars, Jinja and others), SQL for PostgreSQL and Supabase, GitHub Actions workflows, and Markdown, MDX, reStructuredText and AsciiDoc, and knows the routes, handlers and settings of frameworks from Express, Next.js and SvelteKit to Django, Laravel, ASP.NET Core and Spring MVC. [What it finds](https://tech-byte-frontier.github.io/jevgate/what-it-finds.html) and [supported languages and frameworks](https://tech-byte-frontier.github.io/jevgate/languages.html) have the details; `jevgate rules` prints every rule with the question it asks.
 
 ## Install