Skip to content

fix(package): parse manifests by key name and decode JSON escapes - #514

Merged
ss-o merged 4 commits into
nextfrom
bug-513
Sep 12, 2026
Merged

fix(package): parse manifests by key name and decode JSON escapes#514
ss-o merged 4 commits into
nextfrom
bug-513

Conversation

@ss-o

@ss-o ss-o commented Sep 12, 2026

Copy link
Copy Markdown
Member

Problem

Two independent defects in .zi-parse-json / .zi-get-package, both found while
reviewing whether zsh-string-lib should replace the vendored parser. The
library copy fails identically, so adoption does not fix either.

Key order. The parser matched the wanted key only as an object's first
key, and the caller then read fixed slots Strings[2/1] and Strings[2/2].
JSON objects are unordered. Swapping zi-ices before plugin-info in
z-shell/zsh's own manifest, with no other change, left the output variable
unassigned:

profiles count=0
zsh: bad set of key/value pairs for associative array

Escapes. The parser stored raw string bodies. Only \" was repaired, by a
blunt substitution in the caller, so the rest reached ice values that are later
executed:

JSON before after
"a\\b" a\\b a\b
"p\tq" literal \t tab
"l1\nl2" literal \n newline
"xAy" literal A xAy
"a\\\"b" a"b a\"b

The last row is the caller's substitution being wrong where it did fire.

Change

  • Select the smallest object that declares the key rather than one that opens
    with it.
  • Look the plugin-info and zi-ices slots up by name in Strings[1/1], using
    the index idiom the caller already uses for profile names.
  • Add .zi-unescape-json-string and decode escapes where a double-quoted body is
    captured; drop the now-redundant and incorrect \" substitution.

Surrogate pairs, unknown escapes such as \q, and malformed \u are left
exactly as written.

Verification

  • New tests/package-manifest-parsing.zsh, registered in zsh-n.yml (the
    zsh -n matrix discovers files itself; the paths filters and job are explicit).
    It fails on next and passes here.
  • Full suite: 27 files, no failures. zsh -n clean.
  • scripts/public-contract-impact.zsh against next: no public-contract changes.
  • Equivalence on the real z-shell/zsh manifest: every resolved ice is
    byte-identical to next once next's caller-side \" repair is applied. The
    only change is that the decoding now happens in the parser.
  • The unescape helper was unit-checked against 14 inputs with every backslash
    built at run time, after an earlier heredoc silently ate A and made that
    case pass without testing anything.

Also fixed here: profile ices shifted by nested members (#515)

Strings[level/n] numbers objects across the whole subtree in document order,
so an object nested in a member written before zi-ices (an npm-style bugs
block in plugin-info, say) shifted the profile bodies and resolved a different
profile's ices. Since ice values include atclone and atpull, that means
running the wrong commands. It was pre-existing, verified identical before and
after the first commit here.

Every such object leaves an --object-- marker in its own member's string, so
the marker count across the preceding members is the exact offset. No change to
.zi-parse-json or to the shape of the Strings hash.

Re-parsing keyed on the profile name also fixes the nesting case, but trades it
for another: if the profile name is reused as a key in a smaller object, the
smallest-object rule picks the decoy. Both shapes are covered by tests.

The lookup moved into .zi-read-package-manifest, because the test had been
mirroring the caller's slot arithmetic and so would have passed against the
unfixed code. Test and caller now call the same function, and removing the
offset from that function fails the suite.

Verified again after this commit: 27 test files pass, zsh -n clean, no
public-contract changes, and every profile of the real z-shell/zsh manifest
still resolves to its own ices (5.9 to zsh-5.9, 5.3.1 to zsh-5.3.1).

Added after the first review request

A second commit records the fork on .zi-parse-json: it began as
@str-parse-json in z-shell/zsh-string-lib and is now a maintained fork, with
three deliberate divergences named in the comment. Nothing recorded that before,
so a reader had no way to know that swapping in the library copy is a
regression. Comment only, no behavior change.

Refs #513
Refs #515

JSON objects are unordered, but `.zi-parse-json` selected the containing
object only when the wanted key was written first, and `.zi-get-package`
then read fixed slots. Writing `zi-ices` before `plugin-info` left the
output variable unassigned and aborted with "bad set of key/value pairs
for associative array". Select the smallest object that declares the key,
and look both slots up by name.

The parser also stored raw string bodies, leaving `\\`, `\t`, `\n`, `\/`
and `\uXXXX` in ice values that are later executed. Only `\"` was
repaired, by a substitution in the caller that corrupts `"a\\\"b"` into
`a"b` instead of `a\"b`. Decode the escapes where the string is captured
and drop that substitution.

Surrogate pairs, unknown escapes and malformed `\u` are left as written.

Refs #513
@ss-o
ss-o requested a review from a team as a code owner September 12, 2026 07:29
Copilot AI balanced review requested due to automatic review settings September 12, 2026 07:29

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical and moderate findings remain, and malformed-escape coverage is incomplete.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Fixes package manifest parsing for unordered keys and JSON escape decoding.

Changes:

  • Resolves manifest fields by key name.
  • Adds JSON string unescaping.
  • Adds regression tests and CI coverage.
File summaries
File Summary and findings
tests/package-manifest-parsing.zsh Adds parsing tests. Nit (1 vote): malformed \u escape coverage is missing.
lib/zsh/install.zsh Updates parser and field lookup. Critical (3 votes): raw text matching can mistake escaped string content for a key. Moderate (3 votes): surrogate pairs are decoded independently.
.github/workflows/zsh-n.yml Registers the new test in CI.
Review details

Suppressed comments (1)

tests/package-manifest-parsing.zsh:104

  • The acceptance criteria include malformed \u escapes, but this test only exercises an unknown \q escape. A regression in the separate malformed-escape fallback could therefore pass CI; add a parsed \u12 value and assert that it remains byte-for-byte unchanged.
# An unknown escape is not JSON; leave it exactly as written.
check ver    "a${bs}qb"   || return 1
  • Files reviewed: 3/3 changed files
  • Comments generated: 2
  • Review effort level: Lite (auto)

Note

Copilot is running an experiment and ran this review at Lite.


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread lib/zsh/install.zsh
___found="$___text"
# JSON objects are unordered, so the wanted object is the smallest one
# that declares the key, not merely one that opens with it.
if [[ $___text = [[:space:]]#\{*[\"\']${___key}[\"\'][[:space:]]#:* ]]; then
Comment thread lib/zsh/install.zsh
Comment on lines +25 to +27
if [[ $___esc == u && $___tail == (#b)([0-9a-fA-F](#c4))(*) ]] {
___code=16#$match[1]
___out+=${(#)___code} ___rest=$match[2]
The parser is a fork of `@str-parse-json', not a mirror, and the two have
diverged in three places. Nothing recorded that, so a future reader had no
way to know that swapping in the library copy is a regression.

Refs #513
ss-o and others added 2 commits September 12, 2026 09:59
`Strings[level/n]` numbers objects across the whole parsed subtree in
document order, but the profile's ice-list was addressed from its position
inside `zi-ices` alone. An object nested in a member written before
`zi-ices`, such as an npm-style `bugs` block in `plugin-info`, shifted the
numbering and resolved a different profile's ices. Those values include
`atclone` and `atpull`, so the failure mode was running the wrong commands.

Every such object leaves an `--object--` marker in its own member's string,
so the count of markers in the preceding members is the exact offset. No
change to `.zi-parse-json` or to the shape of the `Strings` hash.

Re-parsing keyed on the profile name also fixes the nesting case, but if
the profile name is reused as a key in a smaller object the
smallest-object rule selects the decoy. Both shapes are now covered.

Extract the lookup into `.zi-read-package-manifest`. The test previously
mirrored the caller's slot arithmetic, so it would have passed against the
unfixed code; now both call the same function.

Refs #515
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants