json: numbers round-trip exactly — one number→text rule everywhere (#875) - #894
Merged
Merged
Conversation
…875) LANGUAGE_CONTRACT.md:108 promises `num of (str of x) == x`. That held for `str of` and for nothing else. json_encode, json_build and json_path each carried their own %.15g — one digit short of the 17 a double can need — so a value written as JSON and read back was a DIFFERENT number, silently, in the primary serialization format: str of : 3.141592653589793 json_encode : 3.14159265358979 (decodes to a different double) Worse than filed: each site also had its own integer fast path with its own bound (2^31, 1e9, 1e9), so every integer between that bound and 2^53 went through %.15g too. An ID of 1234567890123456 encoded as 1.23456789012346e+15 and decoded as 1234567890123460. There were four hand-copies of the number->text rule — the fourth in the SIGUSR1 observer dump (vm.c). There is now one, eigs_num_text, and value_to_string calls it as well, so `json_encode of x == str of x` for every number by construction and a fifth copy cannot appear with a fifth rule. One output change falls out of the unification: an exact integer below 2^53 renders bare rather than in exponent form, so json_build of ["v", 1.23e15] is now {"v": 1230000000000000}. That is what `str of` already produced — the two used to disagree, which is the drift this removes — and at 16 characters it is still nothing like the 22-char fixed-point blob #725 removed. JH97's golden was pinned to the old inconsistency; updated, with the exponent-form case above 2^53 pinned alongside it so the change is bounded. tests/test_json_roundtrip.eigs pins all three encoders against `str of` over the hard doubles and the exact-integer band. Both halves are planted-fault validated and they are independently load-bearing: capping the escalation at %.15g fails the round-trip checks, and narrowing the integer bound to 2^31 fails the exponent-form checks (the first plant alone did not catch the second fault — the ragged big integers round-trip through the escalation either way, so the round magnitudes that need the integer branch were added deliberately). Suite 3787/3787 release, 3785/3785 ASan+UBSan with detect_leaks=1, leak tally 0. Closes #875 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| * targets — and %.15g loses the 17th digit a double can need. */ | ||
| for (int prec = 15; prec <= 17; prec++) { | ||
| snprintf(buf, nbuf, "%.*g", prec, n); | ||
| if (strtod(buf, NULL) == n) return; |
There was a problem hiding this comment.
Pull request overview
Unifies EigenScript’s number→text formatting across str of and all JSON-related producers to guarantee lossless numeric round-tripping (json_decode of (json_encode of x) == x) and eliminate inconsistent integer fast-path bounds.
Changes:
- Introduces
eigs_num_text()as the single shared number formatting routine (15→17 digit escalation + exact-integer rendering up to 2^53). - Switches
json_encode,json_build,json_path,str of(viavalue_to_string), and the SIGUSR1 observer dump to useeigs_num_text(). - Adds/updates tests and documentation to pin JSON encoders against
str of, including the exact-integer band behavior change.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_json_roundtrip.eigs | Adds round-trip and “agrees with str of” assertions for JSON encode/build/path over hard doubles and exact-integer cases. |
| tests/test_json_hard.eigs | Updates golden expectation for json_build rendering of an exact integer below 2^53; pins agreement with str of. |
| src/vm.c | Removes the local number formatting copy from SIGUSR1 observer dump and delegates to eigs_num_text(). |
| src/eigenscript.h | Exposes eigs_num_text() to ensure all producers share one formatting rule. |
| src/eigenscript.c | Implements eigs_num_text() and routes value_to_string’s numeric formatting through it. |
| src/builtins.c | Replaces JSON numeric formatting in encode/build/path with eigs_num_text() to ensure exact round-trips. |
| docs/LANGUAGE_CONTRACT.md | Updates the contract to state that all number-text producers share the same implementation. |
| CHANGELOG.md | Documents the behavioral fix and the bounded output-format change for exact integers below 2^53. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+46
to
+47
| # encoders each carried their own %.15g — one digit short of the 17 a double | ||
| # needs — so a value written as JSON and read back was a different number. |
| * | ||
| * LANGUAGE_CONTRACT.md:108 promises `num of (str of x) == x`. That held for | ||
| * `str of` and for nothing else: the three JSON encoders each carried their | ||
| * own `%.15g`, one digit short of the 17 a double needs, so a value written |
| - **JSON is a lossless round-trip for every number (#875).** The | ||
| contract promises `num of (str of x) == x`. That held for `str of` and | ||
| for nothing else: `json_encode`, `json_build` and `json_path` each | ||
| carried their own `%.15g` — one digit short of the 17 a double can |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The contract promises
num of (str of x) == x. That held forstr ofand for nothing else.json_encode,json_buildandjson_patheach carried their own%.15g— one digit short of the 17 a double can need — so a value written as JSON and read back was a different number, silently, in the primary serialization format.Worse than filed: integers were corrupted too
Each site also had its own integer fast path, each with a different bound — 2^31, 1e9, 1e9 — so every integer between that bound and 2^53 went through
%.15gas well:An ID silently becomes a different ID. The issue cited only the float case.
The fix
There were four hand-copies of the number→text rule — the fourth in the SIGUSR1 observer dump (
vm.c). There is now one,eigs_num_text, andvalue_to_stringcalls it too. Sojson_encode of x == str of xfor every number by construction, and a fifth copy cannot appear with a fifth rule.One deliberate output change
An exact integer below 2^53 now renders bare rather than in exponent form:
That is what
str of 1.23e15already produced — the two disagreeing is exactly the drift this removes. At 16 characters it is still nothing like the 22-char fixed-point blob #725 was protecting against, and exponent form is retained above 2^53 (1.23e300→1.23e+300), pinned in the same test so the change is bounded.JH97's golden was pinned to the old inconsistency and is updated.Verification
tests/test_json_roundtrip.eigspins all three encoders againststr ofacross the hard doubles and the exact-integer band.Both halves are planted-fault validated, and they turn out to be independently load-bearing:
%.15g→ the round-trip checks fail;Worth stating: the first plant alone did not catch the second fault. The ragged big integers (
1234567890123456) round-trip through the precision escalation whether or not the integer branch exists, so I added the round magnitudes (1000000000000000) that genuinely depend on that bound. Without that, this PR would have shipped a test that looked like it covered the integer path and didn't.detect_leaks=1: 3785/3785, leak tally 0Closes #875
🤖 Generated with Claude Code