go-ndjson

Parse newline-delimited JSON into typed Go slices with optional per-line validation and format auto-detection.

Why

NDJSON (one JSON object per line) is the standard streaming format for logs, events, and audit trails. Go's stdlib has no first-class reader for it. This library fills that gap with a generic, dependency-free API that validates each line as it parses and distinguishes empty input from all-blank input via distinct sentinel errors.

Installation

go get github.com/larsartmann/go-ndjson

Requires Go 1.26+ with GOEXPERIMENT=jsonv2 (the encoding/json/v2 package is currently experimental).

Quick start

package main

import (
	"fmt"
	"strings"

	"github.com/larsartmann/go-ndjson"
)

type Event struct {
	EventType string `json:"event_type"`
	Phase     string `json:"phase"`
}

func main() {
	input := `{"event_type":"start","phase":"before"}
{"event_type":"end","phase":"after"}`

	events, err := ndjson.Read[Event](strings.NewReader(input), nil)
	if err != nil {
		fmt.Println(err)
		return
	}

	fmt.Printf("%d events\n", len(events)) // 2 events
}

The explicit type parameter is required when validate is nil: type inference pins T from the callback, and nil gives it nothing to infer from.

Usage

With per-line validation

Pass a callback to validate each parsed object. The callback receives the 1-based line number:

events, err := ndjson.Read(reader, func(lineNum int, e Event) error {
	if e.EventType != "start" && e.EventType != "end" {
		return fmt.Errorf("line %d: unknown event_type %q", lineNum, e.EventType)
	}
	return nil
})

Format detection

The loader package detects whether raw bytes are a single JSON report or an NDJSON event stream. Detection probes the first non-blank line and decides by key presence, not value: any "version" key marks a report, any "event_type" key marks events; when both are present, "event_type" wins. Ambiguous input (neither key, or a first line that is not a JSON object) fails with ErrUnknownFormat instead of guessing:

import "github.com/larsartmann/go-ndjson/loader"

format, err := loader.Detect(data)
if err != nil {
	// ErrNoContent (no data) or ErrUnknownFormat (ambiguous input)
	return
}

switch format {
case loader.FormatJSON:
	// single JSON report (has a "version" key)
case loader.FormatNDJSON:
	// newline-delimited events (has an "event_type" key)
}

Sentinel errors

events, err := ndjson.Read[Event](reader, nil)
switch {
case errors.Is(err, ndjson.ErrEmpty):
	// input contained no data
case errors.Is(err, ndjson.ErrNoEvents):
	// all lines were blank
case errors.Is(err, ndjson.ErrOversizedLine):
	// a line exceeded 1 MB
}

License

Proprietary — see LICENSE.