std.json: json.parse leaks 1 byte per call — empty-string error slot escapes the heap tracker (was: blamed on stringify)

[paul-hammant]

Symptom

Every json.stringify() call leaks exactly 1 byte. Valgrind, on an 8-line program that touches nothing but the JSON API:

import std.string
import std.json
main() {
    doc, perr = json.parse("{\"x\":3.14}")
    string.free(perr)
    out, serr = json.stringify(doc)
    string.free(out)
    string.free(serr)
    json.free(doc)
}

1 bytes in 1 blocks are definitely lost in loss record 1 of 1
   at malloc (vg_replace_malloc.c:381)
   by aether_caps_malloc
   by main
definitely lost: 1 bytes in 1 blocks

Every documented free is called. The leak is inside stringify.

Cause

std/json/module.ae:156:

stringify(value: ptr) -> string! {
    raw = json_stringify_raw(value)
    if raw == null {
        return string_concat("", ""), string_concat("stringify failed", "")
    }
    return string_concat(raw, ""), string_concat("", "")
}

json_stringify_raw returns an owned allocation; string_concat(raw, "") copies it into a fresh heap string and raw is then dropped without being freed. (The string_concat(x, "") idiom is deliberate — it mints a properly-owned AetherString from a plain char* so the heap tracker will free it — but the source still needs releasing.)

The 1-byte size is a red herring from the allocator's accounting; the point is one unfreed block per call, so a serialisation loop leaks linearly.

Not a regression

Confirmed on pristine main (fab571bd) built in a clean git worktree — identical result. Found incidentally while chasing the macOS leaks gate in #1459; explicitly not caused by that PR, which is why it is filed separately rather than fixed there.

Why it went unnoticed

• The .ae regression suite is not leak-gated on Linux; only the macOS ARM64 leaks job and the valgrind legs would see it, and no existing leak-gated test round-trips through stringify.
• 1 byte per call is below the noise floor of a casual look at a valgrind summary.

Suggested fix

Free raw after the copy, e.g. bind it and release it on both return paths (checking whether the correct call is string_free or string_release for a json_stringify_raw return — the two are not interchangeable here).

Worth a leak-gated regression test that does a parse → stringify → free round-trip, since nothing currently covers it.

Impact

Matters most for anything serialising in a loop — HTTP handlers returning JSON, avn-style commit walks, the tinyweb/schema showcase. A long-running server leaks one block per response.

[paul-hammant]

Correcting this issue after investigating. Three of my original claims were wrong, and the mis-attribution is the important one — anyone starting from the title would look in the wrong function.

1. It is json.parse, NOT json.stringify

The title and body blame stringify's string_concat(raw, ""). That is wrong. json.parse alone leaks, with no stringify anywhere:

import std.string
import std.json
main() {
    i = 0
    for (i = 0; i < 10; i++) {
        doc, e = json.parse("{\"x\":1}")
        string.free(e)
        json.free(doc)
    }
}

definitely lost: 10 bytes in 10 blocks

The likely culprit is std/json/module.ae:119 — the success path returns string_concat("", ""), minting a fresh 1-byte empty string per call for the error slot.

2. Severity is lower than I stated

I wrote "a long-running server leaks one block per response" and listed HTTP handlers and avn commit walks as impact. The block count is right; the implied magnitude is not.

It is exactly 1 byte per call, and does NOT scale with document size. Measured: 20 round-trips of a ~200-byte document leak 20 bytes, same as 20 round-trips of a 10-byte one. The serialized payload is freed correctly — only the empty-string sentinel escapes.

So: ~1 MB per million requests. Real and worth fixing, but nothing is on fire, and it should not be prioritised as if a server were bleeding memory proportional to traffic.

3. The suggested fix does not work

The issue proposes freeing raw after the copy. I tried the idiomatic version — adding @heap to the extern, exactly as std/os does for the same shape (a C function returning malloc'd memory, e.g. os_run_capture_status_raw at std/os/module.ae:74):

extern json_stringify_raw(value: ptr) -> string @heap

Still leaks, on a genuinely fresh build. (Worth flagging for anyone retrying: my first attempt reported a false pass because ae build said "Built (cache hit)" — clear ~/.aether/cache before trusting any result here.)

4. The stack implicates generated code, not the stdlib

20 bytes in 20 blocks are definitely lost
   at malloc
   by aether_caps_malloc
   by main            <- not json_parse_raw, not stringify

The allocation is attributed to main — i.e. the generated C at the call site — so this is about how a tuple-returned heap string is (not) freed at the destructure, rather than a missing free inside the module.

Two further data points that bound the scope:

• Not a general tuple/error-slot bug. The same shape via string.to_double("3.14") in a 10-iteration loop is completely clean (0 errors). So whatever the codegen does for that path, it works.
• The caller cannot fix it. Calling string.free(e) and string_release(e) on the returned error slot still leaks. This is not caller misuse.

Where to start

std/json/module.ae:119 (return value, string_concat("", "")) against the equivalent in a module that does not leak, and the emitted C for both destructures. The question is why the empty-string case escapes the heap tracker when the populated-string case does not.

Related: there are 41 instances of the string_concat(x, "") wrapper idiom across std/*/module.ae (std/os, std/fs, std/io, std/audit, …). If the root cause is the literal-empty-string case, several of those may leak identically — worth checking once the mechanism is understood, rather than assuming this is JSON-only.

Verified on main @ b2401b50, and previously reproduced on pristine main in a clean worktree, so it is not a regression from any recent PR.