Parser

Signature: text → tokens

The core of the package. parser() is a factory that returns a function for use in a chain() pipeline. It consumes text and produces a stream of {name, value} tokens. It is always the first component in a pipeline, fed with text from a file, socket, or any other source.

parser.asStream() wraps the parser as a Duplex stream: the writable side accepts text (Buffer/string), the readable side emits token objects in object mode.

Parser assumes well-formed input. For error diagnostics use Verifier.

For strict line-delimited JSONL where individual items fit in memory, a dedicated line splitter is much faster than tokenizing — use stream-chain's JSONL.

Introduction

Using parser() with chain():

import chain from 'stream-chain';
import {parser} from 'stream-json/parser.js';
import fs from 'node:fs';

const pipeline = chain([fs.createReadStream('sample.json'), parser()]);

let objectCounter = 0;
pipeline.on('data', data => data.name === 'startObject' && ++objectCounter);
pipeline.on('end', () => console.log(`Found ${objectCounter} objects.`));

Using parser.asStream() with .pipe():

import {parser} from 'stream-json/parser.js';
import fs from 'node:fs';

const pipeline = fs.createReadStream('sample.json').pipe(parser.asStream());

let objectCounter = 0;
pipeline.on('data', data => data.name === 'startObject' && ++objectCounter);
pipeline.on('end', () => console.log(`Found ${objectCounter} objects.`));

API

The module exports a parser factory function. It produces a rigid token stream whose order is strictly defined — it is impossible to get a token out of sequence. All data items (strings, numbers, even object keys) are streamed in chunks and can be of any size.

The named export jsonParser is the raw inner tokenizer — the bare flushable() state machine without the fixUtf8Stream() front. parser is gen(fixUtf8Stream(), jsonParser()); reach for the raw jsonParser only to embed the tokenizer in a custom pipeline that handles cross-chunk UTF-8 itself. The same gen/raw split applies to the JSONC parser (jsoncParser), the JSONL parser (jsonlParser), and the verifiers (jsonVerifier/jsoncVerifier).

When individual data items fit in memory, the parser can pack them into single tokens for easier inspection.

parser(options)

options is an optional object. The following custom flags are recognized (all truthy/falsy):

• jsonStreaming controls the parsing algorithm. If truthy, a stream of JSON objects is parsed as described in JSON Streaming as "Concatenated JSON". Technically it will recognize "Line delimited JSON" AKA "JSON Lines" AKA JSONL as well. Otherwise, it will follow the JSON standard assuming a singular value. The default: false.
  • It allows streaming any number of values one after another. An empty stream produces no values.
  • Values are separated by any amount of whitespace (spaces, tabs, newlines, carriage returns) — or by nothing at all where the boundary is unambiguous: [1][2] and 1"a" are both two values. Anything else between values (e.g. a comma) is an error.
  • For strict line-delimited JSONL, a dedicated line splitter is much faster than tokenizing — use stream-chain's JSONL directly.
• Packing options control packing values. Each flag effectively defaults to true; packValues is a cascading initializer for the three individual flags (details and tables below).
  • packValues serves as the initial value for packing strings, numbers, and keys.
  • packKeys specifies, if we need to pack keys and send them as a value.
  • packStrings specifies, if we need to pack strings and send them as a value.
  • packNumbers specifies, if we need to pack numbers and send them as a value.
  • More details in the section below.
• Streaming options control sending unpacked values. Each flag effectively defaults to true; streamValues is a cascading initializer for the three individual flags (details and tables below).
  • streamValues serves as the initial value for other three options described above.
  • streamKeys specifies, if we need to send items related to unpacked keys.
  • streamStrings specifies, if we need to send items related to unpacked strings.
  • streamNumbers specifies, if we need to send items related to unpacked numbers.
  • More details in the section below.

By default, Parser follows a strict JSON format, streams all values by chunks and individual (packed) values.

Stream of tokens

This is the list of data objects produced by Parser in the correct order:

// a sequence can have 0 or more items
// a value is one of: object, array, string, number, null, true, false

// a parser produces a sequence of values

// object
{name: 'startObject'};
// sequence of object properties: key, then value
{name: 'endObject'};

// array
{name: 'startArray'};
// sequence of values
{name: 'endArray'};

// key
{name: 'startKey'};
// sequence of string chunks:
{name: 'stringChunk', value: 'string value chunk'};
{name: 'endKey'};
// when packing:
{name: 'keyValue', value: 'key value'};

// string
{name: 'startString'};
// sequence of string chunks:
{name: 'stringChunk', value: 'string value chunk'};
{name: 'endString'};
// when packing:
{name: 'stringValue', value: 'string value'};

// number
{name: 'startNumber'};
// sequence of number chunks (as strings):
{name: 'numberChunk', value: 'string value chunk'};
{name: 'endNumber'};
// when packing:
{name: 'numberValue', value: 'string value'};

// null, true, false
{name: 'nullValue', value: null};
{name: 'trueValue', value: true};
{name: 'falseValue', value: false};

Worked example

The document {"a": 1, "b": "c"} produces the following tokens with default options:

{name: 'startObject'}
{name: 'startKey'}
{name: 'stringChunk', value: 'a'}
{name: 'endKey'}
{name: 'keyValue', value: 'a'}
{name: 'startNumber'}
{name: 'numberChunk', value: '1'}
{name: 'endNumber'}
{name: 'numberValue', value: '1'}
{name: 'startKey'}
{name: 'stringChunk', value: 'b'}
{name: 'endKey'}
{name: 'keyValue', value: 'b'}
{name: 'startString'}
{name: 'stringChunk', value: 'c'}
{name: 'endString'}
{name: 'stringValue', value: 'c'}
{name: 'endObject'}

By default every key, string, and number appears in both forms: streamed as chunks (startXXX … endXXX) and packed (xxxValue). The packing and streaming options control which forms are emitted; the sections below show this same document under different settings.

All value chunks (stringChunk and numberChunk) should be concatenated in order to produce a final value. Empty string values may have no chunks. String chunks may have empty values.

Important: values of numberChunk and numberValue are strings, not numbers. It is up to a downstream code to convert it to a number using parseInt(x), parseFloat(x) or simply x => +x. Numbers are kept as text so values of any magnitude and precision survive intact (think BigInt-range integers — see Assembler's numberAsString option) and a round-trip through Stringer reproduces the source exactly.

The literal tokens (nullValue, trueValue, falseValue) carry their value even though name already identifies it: every xxxValue token holds its value in value, so downstream code can treat all packed tokens uniformly.

All items follow in the correct order. If something is going wrong, a parser will produce an error event. For example:

• All startXXX are balanced with endXXX.
• Between startKey and endKey can be zero or more stringChunk items. No other items can be seen.
• After startObject optional key-value pairs emitted in a strict pattern: a key-related item, then a value, and this cycle can be continued until all key-value pairs are streamed.
  • It is not possible for a key to be missing a value.
• All endObject are balanced with the corresponding startObject.
• endObject cannot close startArray.
• Between startString and endString can go 0 or more stringChunk, but no other items.
• endKey can be optionally followed by keyValue, then a new value will be started, but no endObject.

In short, the item sequence is always correctly formed. No need to do unnecessary checks.

Packing options

Parser packs keys, strings, and numbers separately. A frequent case when it is known that key and number values can fit in memory, but strings cannot.

Internally each type of value is controlled by a flag:

• By default, this flag is true.
• If packValues is set, it is assigned to each flag.
• If an individual option is set, it is assigned to the flag.

Examples:

Supplied options	packKeys	packStrings	packNumbers
{}	true	true	true
{packValues: false}	false	false	false
{packValues: false, packKeys: true}	true	false	false
{packKeys: true, packValues: false}	true	false	false
{packStrings: false}	true	false	true
{packKeys: true, packStrings: false, packNumbers: true}	true	false	true

The worked example's document {"a": 1, "b": "c"} with {packValues: false} — streamed chunks only:

{name: 'startObject'}
{name: 'startKey'}
{name: 'stringChunk', value: 'a'}
{name: 'endKey'}
{name: 'startNumber'}
{name: 'numberChunk', value: '1'}
{name: 'endNumber'}
{name: 'startKey'}
{name: 'stringChunk', value: 'b'}
{name: 'endKey'}
{name: 'startString'}
{name: 'stringChunk', value: 'c'}
{name: 'endString'}
{name: 'endObject'}

Streaming options

Parser can optionally skip streaming keys, strings, and/or numbers for optimization purposes, if a corresponding packing option is enabled. It means that only three configurations are supported for values and keys:

• The default: startXXX, 0 or more stringChunk (numberChunk for numbers), endXXX, xxxValue.
• packXXX is false: startXXX, 0 or more stringChunk (numberChunk for numbers), endXXX.
• packXXX is true, streamXXX is false: xxxValue.

Internally each type of value is controlled by a flag:

• By default, this flag is true.
• If streamValues is set, it is assigned to each flag.
• If an individual option is set, it is assigned to the flag.
• If a corresponding packing option is false, it is set to true.

Examples:

Supplied options	streamKeys	streamStrings	streamNumbers
{}	true	true	true
{packValues: true, streamValues: false}	false	false	false
{packKeys: true, streamKeys: false}	false	true	true
{packKeys: false, streamKeys: false}	true	true	true
{packValues: true, streamValues: false, streamKeys: true}	true	false	false
{streamStrings: false}	true	true	true
{packKeys: true, streamKeys: false, streamStrings: false, streamNumbers: true}	false	true	true

The worked example's document {"a": 1, "b": "c"} with {packValues: true, streamValues: false} — packed values only, the minimal token stream. This is the usual configuration when chunk-level access is not needed (e.g. when feeding streamers or path filters):

{name: 'startObject'}
{name: 'keyValue', value: 'a'}
{name: 'numberValue', value: '1'}
{name: 'keyValue', value: 'b'}
{name: 'stringValue', value: 'c'}
{name: 'endObject'}

Static methods and properties

parser.asStream(options)

Wraps the parser as a Duplex stream. Useful with .pipe() when not using chain():

import {parser} from 'stream-json/parser.js';
import fs from 'node:fs';

const pipeline = fs.createReadStream('sample.json').pipe(parser.asStream());

let objectCounter = 0;
pipeline.on('data', data => data.name === 'startObject' && ++objectCounter);
pipeline.on('end', () => console.log(`Found ${objectCounter} objects.`));

Web Streams

parser ships in two substrate-specific entries with the same factory shape:

• Node — stream-json/parser.js. Has asStream (Node Duplex) and asWebStream (Web {readable, writable} pair).
• Web — stream-json/web/parser.js. Has asWebStream only. Pulls in no Node-stream imports, safe for browser bundles.

Both factories return the same flushable, so chain on either substrate auto-wraps it. Use chain from stream-chain on Node and from stream-chain/web on Web.

// Web
import {chain} from 'stream-chain/web';
import {parser} from 'stream-json/web/parser.js';
import {streamArray} from 'stream-json/web/streamers/stream-array.js';

const pipeline = chain([source, parser(), streamArray()]);
for await (const item of pipeline.readable) console.log(item);

Or use asWebStream directly with pipeTo:

import {parser} from 'stream-json/web/parser.js';

const {readable, writable} = parser.asWebStream();
sourceReadable.pipeTo(writable);
for await (const tok of readable) console.log(tok);