Skip to content

feat(native): vectors of native data as a template over a PHP string - #15

Merged
lisachenko merged 4 commits into
mainfrom
claude/php-generics-vectors-vjhr1r
Aug 10, 2026
Merged

feat(native): vectors of native data as a template over a PHP string#15
lisachenko merged 4 commits into
mainfrom
claude/php-generics-vectors-vjhr1r

Conversation

@lisachenko

Copy link
Copy Markdown
Owner

What

Lisachenko\Generics\Native\NativeVector — a placeholder-form generic template whose storage is a PHP string acting as a raw block of memory:

$samples = new (NativeVector::of('int'))($blobFromTheWire);  // "cast" a binary string
$first   = $samples[0];          // zend_long read straight from the block
$samples[1] = 42;                // in-place native store
$samples->append(7);             // growth = string concatenation
$bytes = $samples->toBinary();   // hand the block back
$samples->destroy();             // explicit release
  • Type arguments are PHP builtinsNativeVector<int> maps elements to zend_long, NativeVector<float> to double (8 bytes each, naturally aligned in zend_string.val). Anything else is rejected at construction.
  • No pack()/unpack() anywhere on the hot path. Reads dereference a cached typed pointer (Core::cast('zend_long *'|'double *', $zstr->val)) so FFI initializes the PHP value straight from memory; writes store the value in place. Tests use pack() only as ground truth.
  • The engine enforces the element type. get(int): T, set(int, T) and append(T) are declared with the type parameter itself, so $intVector->append(1.5) is a real TypeError thrown by the Zend Engine — the ArrayAccess sugar delegates to those methods rather than re-checking anything.
  • Copy-on-write discipline. A write only goes in place when the buffer's zend_string is exclusively owned (refcount 1, not interned); otherwise the buffer is separated first, so a string returned by toBinary() is never retroactively mutated. Growth releases the pointer cache so .= keeps its in-place realloc fast path.
  • Consumes only z-engine's existing public API (StringEntry, Core::cast) — no z-engine changes. AGENTS.md decision 13 is amended to sanction the Native\ namespace as the second engine touchpoint, confined to this class's private helpers.

This is a direct answer to the array<T> entry in docs/limitations.md: a vector's elements are enforced by construction.

Also in this PR

  • NativeVectorException / NativeVectorBoundsException with named constructors (misaligned binary, unsupported element type, bounds, destroyed, fixed layout).
  • Stub generator growth: NativeVector is the first stubbed template with private helpers, static named constructors, constants, parameter defaults and PHP interfaces — the generator now reproduces all of those (Box/AttributeBox stubs regenerate byte-identical). PHPStan sees the stub (excluded via analyseAndScan), so $v->get(0) is int for NativeVector<int>.
  • docs/native-vectors.md (memory model, COW/destroy semantics, roadmap to sized scalar kinds and C-struct layouts), cross-links from docs/limitations.md, docs/static-analysis.md and README.
  • examples/native-vector.php + subprocess example test.
  • New documented limitation: a stub-described template does not narrow of() (stub vs GenericObject-extension exclusivity) — pre-existing for Box, now written down.

Verification

All run locally on PHP 8.5.9 (z-engine 8.5.x-dev):

  • engine suite (ffi.enable=1, opcache.jit=off): 179 tests, 510 assertions, 1 pre-existing skip
  • same under opcache.enable_cli=1 opcache.save_comments=0: pass (1 new deliberate skip: the stub generator's @implements test, which reads doc comments by design — tooling, not runtime)
  • --group internal --process-isolation: pass
  • composer test:analysis, composer phpstan (level max, no new baseline entries), composer cs:check, composer stubs:check, composer test:preload, composer bench:smoke: all pass

Roadmap (out of scope here)

Sized scalar kinds (int32, uint16, …) and C-struct layouts as follow-up phases; a zero-copy engine-side buffer was considered and deliberately not built (this class already writes in place through the existing API).

🤖 Generated with Claude Code

https://claude.ai/code/session_013qW1hct9mDKsFaHTvkLggx


Generated by Claude Code

claude added 4 commits August 10, 2026 12:26
`NativeVector<int>` and `NativeVector<float>` are fixed-layout blocks of native
scalars addressed as `$vector[$i]`, whose storage is an ordinary PHP string:
element `i` lives at byte `i * 8` of that string's `zend_string.val`, in the
machine's own layout. There is no encoding step, so pack()/unpack() appear
nowhere on the path - a read initializes a PHP value from bytes that are already
there, a write copies one into them, through a cached `zend_long *`/`double *`.

The element type is enforced because `get()`, `set()` and `append()` are declared
with the type parameter itself, which is a slot the engine really does check.
`ArrayAccess::offsetSet(mixed, mixed)` cannot narrow its parameters, so the array
syntax delegates to those three rather than replacing them, and `$vector[$i] = $x`
gets exactly the TypeError `$vector->set($i, $x)` gets.

Correctness rests on a small copy-on-write discipline: the class holds no engine
reference on its own buffer, reads the refcount off the cached `zend_string`
before every write, and separates the block - by assigning a string offset onto
itself, the userland spelling of the engine's own separation - whenever somebody
else is holding those bytes. That is what makes `toBinary()` safe to hand the
buffer out rather than copy it. Growth stays literal `.=` concatenation, which the
engine does in place precisely because nothing else holds a reference.

This answers the scalar half of docs/limitations.md's most important entry:
`array<T>` element types cannot be enforced, and a block of memory does not need
them to be.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013qW1hct9mDKsFaHTvkLggx
NativeVector is the first template with private helpers, named constructors,
class constants, a defaulted constructor parameter and interfaces, and the
generator described none of those: every method came out `public`, statics were
dropped entirely, defaults were lost, and the class implemented nothing. Analysed
code therefore could not call `fromString()`, could not write
`new (NativeVector::of('int'))()` without being told it passed too few arguments,
could not index or count or iterate a vector - and could call a private helper
and be told nothing.

The generator now reproduces visibility, class constants, parameter defaults and
own static methods (`of()` stays the one it writes out itself), and declares the
interfaces PHP itself owns. A generic interface has to say what it was
parameterized with, and only the template knows, so the `@implements` tags are
copied off the class doc comment - the static-analysis source of truth, which is
what decision 1 reserves doc comments for. The library's own interfaces still
cannot be named: a stub is reflected before the analysed paths are indexed.

The stub only wins if PHPStan is not also reading the real declaration, so
NativeVector.php is excluded from scanning rather than just from analysis. Box
and AttributeBox regenerate byte-identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013qW1hct9mDKsFaHTvkLggx
…release

NativeVectorTest is the sole owner of `NativeVector<int>`, `NativeVector<float>`
and `NativeVector<string>` across the suite (AGENTS.md section 6), which is why
the shipped example runs in a subprocess.

pack() is used throughout as the ground truth the block is checked against: the
production path never encodes anything, so an independent encoder is exactly what
makes a byte-level assertion worth having. The two that justify the design are the
engine's own TypeError arriving on `append(1.5)` and on `$vector[0] = 'x'` - the
second proving the array syntax really does delegate - and the copy-on-write case,
where a binary handed out by `toBinary()` is shown to survive a later `set()` and
a later `append()` unchanged. A literal cast into a vector is shown untouched by a
write for the same reason, since every literal is interned.

examples/native-vector.php casts a blob of native int64s, indexes it, grows it,
round-trips it and destroys it; its test asserts the shape of the output and the
TypeErrors, never wording.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013qW1hct9mDKsFaHTvkLggx
…t decision

docs/native-vectors.md covers the model in full: the block of memory is a PHP
string, both native scalar kinds are eight bytes and `zend_string.val` is
eight-aligned so `i * 8` is naturally aligned, the byte order is the machine's by
definition, the cast and round trip, growth by concatenation, the copy-on-write
discipline behind `toBinary()`, `destroy()`, and the roadmap towards sized scalar
kinds and C structures.

Two honest entries go with it. limitations.md's `array<T>` paragraph now points at
the way out for scalar elements, and a new entry records what the stub costs: a
stub cannot name this library's own interfaces, so a stub-described template is
not a `GenericObject` and `of()` stops narrowing for it. The two mechanisms are
exclusive; spell the specialization once at the boundary and everything downstream
follows. static-analysis.md carries the neon snippet.

AGENTS.md decision 13 is amended as the maintainer directed: `Native\` is the
second sanctioned z-engine touchpoint, may use StringEntry (getRawValue included)
and Core::cast, and confines both to NativeVector's private low-level methods, so
the property that made decision 13 worth having still holds - every line that
touches the engine can be named. `native` joins the commit scopes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013qW1hct9mDKsFaHTvkLggx
@lisachenko
lisachenko marked this pull request as ready for review August 10, 2026 12:55
@lisachenko
lisachenko merged commit dc625f7 into main Aug 10, 2026
16 checks passed
@lisachenko
lisachenko deleted the claude/php-generics-vectors-vjhr1r branch August 10, 2026 12:55
@lisachenko

Copy link
Copy Markdown
Owner Author

Session:

  • Fable 5: 175 in / 404 out / 9.8M cache read / 1.2M cache write
  • Cost: $2.79 | API 0m | Wall 1m

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants