From 07270e3cacb3d6a7db5211afd4c5d3f4576fafd6 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 10 Aug 2026 12:26:37 +0000 Subject: [PATCH 1/4] feat(native): reify native data vectors as a template over a PHP string `NativeVector` and `NativeVector` are fixed-layout blocks of native scalars addressed as `$vector[$i]`, whose storage is an ordinary PHP string: element `i` lives at byte `i * 8` of that string's `zend_string.val`, in the machine's own layout. There is no encoding step, so pack()/unpack() appear nowhere on the path - a read initializes a PHP value from bytes that are already there, a write copies one into them, through a cached `zend_long *`/`double *`. The element type is enforced because `get()`, `set()` and `append()` are declared with the type parameter itself, which is a slot the engine really does check. `ArrayAccess::offsetSet(mixed, mixed)` cannot narrow its parameters, so the array syntax delegates to those three rather than replacing them, and `$vector[$i] = $x` gets exactly the TypeError `$vector->set($i, $x)` gets. Correctness rests on a small copy-on-write discipline: the class holds no engine reference on its own buffer, reads the refcount off the cached `zend_string` before every write, and separates the block - by assigning a string offset onto itself, the userland spelling of the engine's own separation - whenever somebody else is holding those bytes. That is what makes `toBinary()` safe to hand the buffer out rather than copy it. Growth stays literal `.=` concatenation, which the engine does in place precisely because nothing else holds a reference. This answers the scalar half of docs/limitations.md's most important entry: `array` element types cannot be enforced, and a block of memory does not need them to be. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013qW1hct9mDKsFaHTvkLggx --- src/Exception/NativeVectorBoundsException.php | 43 ++ src/Exception/NativeVectorException.php | 114 +++++ src/Native/NativeVector.php | 436 ++++++++++++++++++ 3 files changed, 593 insertions(+) create mode 100644 src/Exception/NativeVectorBoundsException.php create mode 100644 src/Exception/NativeVectorException.php create mode 100644 src/Native/NativeVector.php diff --git a/src/Exception/NativeVectorBoundsException.php b/src/Exception/NativeVectorBoundsException.php new file mode 100644 index 0000000..e03558f --- /dev/null +++ b/src/Exception/NativeVectorBoundsException.php @@ -0,0 +1,43 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + * + */ +declare(strict_types=1); + +namespace Lisachenko\Generics\Exception; + +use OutOfBoundsException; + +/** + * Raised when an element index falls outside the block of memory + * + * Separate from NativeVectorException on purpose, and extending the SPL class callers already + * write `catch (OutOfBoundsException)` for: this is the one failure that ordinary, correct code + * runs into (a loop off by one), while everything on NativeVectorException is a mistake about + * the vector itself. + * + * It is also the check that stands between a userland typo and a wild pointer dereference, so + * it is made *before* every read and every write, never after. + */ +final class NativeVectorBoundsException extends OutOfBoundsException implements GenericsException +{ + public static function index(string $vectorClass, int $index, int $count): self + { + return new self(sprintf( + 'Index %d is outside the %d element(s) of %s. %s', + $index, + $count, + $vectorClass, + $count === 0 + ? 'The vector is empty, so no index is valid; append() first.' + : sprintf('Valid indices run from 0 to %d.', $count - 1), + )); + } +} diff --git a/src/Exception/NativeVectorException.php b/src/Exception/NativeVectorException.php new file mode 100644 index 0000000..dcad892 --- /dev/null +++ b/src/Exception/NativeVectorException.php @@ -0,0 +1,114 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + * + */ +declare(strict_types=1); + +namespace Lisachenko\Generics\Exception; + +use RuntimeException; + +/** + * Raised when a native vector cannot be built or can no longer be used + * + * These are all problems with the *block of memory*, never with an element: an element that is + * the wrong type is rejected by the engine as a TypeError, which is the entire point of the + * package and is never translated into one of these (AGENTS.md section 5). + * + * Element indices out of range get their own class, NativeVectorBoundsException, because a + * caller iterating a vector wants to catch that one and nothing else. + */ +final class NativeVectorException extends RuntimeException implements GenericsException +{ + /** + * The raw template has no element type, so it has no element size and no layout + */ + public static function notSpecialized(string $templateName): self + { + return new self(sprintf( + 'A %s cannot be constructed without a type argument: the element type is what fixes ' + . 'the layout of the memory block. Specialize it first, for example ' + . 'new (%s::of(\'int\'))().', + $templateName, + $templateName, + )); + } + + /** + * Only the two native scalar kinds have a machine layout this vector can address + */ + public static function unsupportedElementType(string $vectorClass, string $typeArgument): self + { + return new self(sprintf( + 'Native vector %s was specialized for "%s", but a native block of memory can only ' + . 'hold "int" (a zend_long) or "float" (a double) for now. Sized scalar kinds ' + . '(int32, uint16, ...) and C structures are the next phase - see docs/native-vectors.md.', + $vectorClass, + $typeArgument, + )); + } + + /** + * A binary string that is not a whole number of elements has no reading + * + * The element size is spelled out rather than imported: both native scalar kinds are + * 8 bytes wide, and an exception class that had to reach into the vector to phrase its + * own message would be the wrong dependency direction. + */ + public static function misalignedBinary(string $vectorClass, int $byteLength): self + { + return new self(sprintf( + 'Native vector %s holds 8-byte elements, so a binary block of %d byte(s) cannot be ' + . 'cast to it: %d byte(s) would be left over. Trim or pad the block before casting it.', + $vectorClass, + $byteLength, + $byteLength % 8, + )); + } + + public static function negativeCapacity(string $vectorClass, int $count): self + { + return new self(sprintf( + 'Native vector %s cannot be created with a capacity of %d: a block of memory has no ' + . 'negative size.', + $vectorClass, + $count, + )); + } + + /** + * Every element accessor is a pointer dereference, and destroy() dropped the pointer + */ + public static function destroyed(string $vectorClass): self + { + return new self(sprintf( + 'The memory block of this %s was released by destroy(), so it has no elements to ' + . 'read or write anymore. destroy() is final for an instance; build a new vector ' + . 'from a binary string instead.', + $vectorClass, + )); + } + + /** + * A native vector's layout is its whole identity, so an element cannot be removed + * + * Raised through a factory rather than as an inline LogicException because every failure + * mode in this package is named on its exception class (AGENTS.md section 8). + */ + public static function fixedLayout(string $vectorClass): self + { + return new self(sprintf( + 'Elements of a native vector cannot be unset: %s is a contiguous block of memory, ' + . 'not a hash table, and removing a slot from the middle of it has no meaning. ' + . 'Overwrite the element, or build a new vector from the bytes you want to keep.', + $vectorClass, + )); + } +} diff --git a/src/Native/NativeVector.php b/src/Native/NativeVector.php new file mode 100644 index 0000000..20bd325 --- /dev/null +++ b/src/Native/NativeVector.php @@ -0,0 +1,436 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + * + */ +declare(strict_types=1); + +namespace Lisachenko\Generics\Native; + +use ArrayAccess; +use Countable; +use FFI\CData; +use IteratorAggregate; +use Lisachenko\Generics\Attribute\TemplateParameter; +use Lisachenko\Generics\Exception\NativeVectorBoundsException; +use Lisachenko\Generics\Exception\NativeVectorException; +use Lisachenko\Generics\Generic; +use Lisachenko\Generics\GenericObject; +use Lisachenko\Generics\GenericTemplate; +use Traversable; +use ZEngine\Core; +use ZEngine\Type\StringEntry; + +/** + * A fixed-layout block of native scalars, addressed as `$vector[$i]`, stored in a PHP string + * + * ```php + * $samples = new (NativeVector::of('float'))($blobFromTheWire); + * $samples[0] = 1.5; // a double written straight into the block + * $samples->append(2.25); // the block grows by one element + * $bytes = $samples->toBinary(); + * ``` + * + * **The memory model.** The block of memory *is* a PHP string. Element `i` lives at byte + * `i * 8` of that string's `zend_string.val`, in the machine's own layout, and the accessors + * dereference it as `zend_long*` or `double*` - there is no encoding step anywhere, so + * `pack()`/`unpack()` appear nowhere in this file. Both native scalar kinds are 8 bytes wide on + * every platform PHP supports and `zend_string.val` starts 8-aligned, so `i * 8` is always a + * naturally aligned offset. A binary string from anywhere - a file, a socket, `pack()` in the + * caller's own code - is *cast* to a vector by handing it to the constructor, and `toBinary()` + * hands it back. + * + * **What the engine checks.** `get()`, `set()` and `append()` are declared with the type + * parameter itself, so the specialization carries `int`/`float` in those slots and it is the + * Zend Engine, not this class, that rejects a wrong element with a TypeError. That is the whole + * reason the element API is not `mixed`: `ArrayAccess::offsetSet(mixed, mixed)` cannot narrow + * its parameters, so the array-syntax sugar delegates to `set()`/`append()` and inherits their + * checking rather than replacing it. + * + * **Why a template rather than a class per type.** Monomorphization shares the compiled method + * bodies between specializations, so `NativeVector` and `NativeVector` cost one + * class entry each and no code at all. The element kind is therefore resolved once, in the + * constructor, from the specialization's own binding - never from `self::class`, which is + * folded into the shared opcodes. + * + * See docs/native-vectors.md for the memory model, the copy-on-write discipline and the + * roadmap towards sized scalar kinds and C structures. + * + * @template T + * + * @implements ArrayAccess + * @implements IteratorAggregate + */ +#[TemplateParameter('T')] +final class NativeVector implements ArrayAccess, Countable, IteratorAggregate, GenericObject +{ + use GenericTemplate; + + /** + * Both supported element kinds are exactly this wide: `zend_long` and `double` + * + * PHP has no narrower scalar to store, so this is a constant rather than a per-instance + * size. Sized kinds are the next phase, and that is the field this becomes. + */ + public const ELEMENT_SIZE = 8; + + /** + * One element's worth of zero bytes, the unit of growth + * + * A literal rather than `str_repeat()`: it is interned, so appending to an empty vector + * costs no allocation at all. That the result may then *be* the interned constant is + * handled where it matters - see acquire(). + */ + private const ZERO_ELEMENT = "\0\0\0\0\0\0\0\0"; + + /** + * The block of memory, which is a PHP string and nothing more + */ + private string $buffer; + + /** + * Element kind, resolved once from the binding; there are only two, so a bool carries it + */ + private bool $isFloat; + + private bool $destroyed = false; + + /** + * Cached `zend_string *` for $buffer, or null when the cache needs re-acquiring + * + * Held without a reference of its own: the property is what keeps the string alive, and a + * wrapper holding a second reference would push the refcount to 2 permanently, which would + * both defeat the in-place `.=` growth path and make the exclusivity test below useless. + */ + private ?CData $string = null; + + /** + * Cached typed pointer at the first element - `zend_long *` or `double *` + * + * This is the hot path: an element access is a bounds check and one dereference through + * this pointer. It is dropped whenever the string behind it may have moved (growth, + * separation, destroy) and lazily re-acquired. + */ + private ?CData $elements = null; + + /** + * Casts an existing binary block - the vector adopts its bytes, it does not decode them + * + * @param string $binary A whole number of native elements; anything else has no reading + */ + public function __construct(string $binary = '') + { + $binding = Generic::bindingOf(static::class); + if ($binding === null) { + throw NativeVectorException::notSpecialized(static::class); + } + + [$typeArgument] = $binding; + if ($typeArgument !== 'int' && $typeArgument !== 'float') { + // A bound cannot express "int|float", so this is the one check the specialization + // itself cannot make: the template is legal for any type argument, the *layout* + // is not + throw NativeVectorException::unsupportedElementType(static::class, $typeArgument); + } + $this->isFloat = $typeArgument === 'float'; + + if (strlen($binary) % self::ELEMENT_SIZE !== 0) { + throw NativeVectorException::misalignedBinary(static::class, strlen($binary)); + } + + $this->buffer = $binary; + // The caller's string may be interned, or shared with a variable they still hold. + // Writing into either would be a value-semantics violation, so the block is made + // exclusively ours before anything can point into it + $this->separate(); + } + + /** + * The cast, spelled as a named constructor + */ + public static function fromString(string $binary): static + { + return new static($binary); + } + + /** + * A zero-filled block of $count elements + */ + public static function withCapacity(int $count): static + { + if ($count < 0) { + throw NativeVectorException::negativeCapacity(static::class, $count); + } + + // Zero-filling is not an encoding step: every byte written is a literal zero + return new static(str_repeat("\0", $count * self::ELEMENT_SIZE)); + } + + public function count(): int + { + return intdiv(strlen($this->buffer), self::ELEMENT_SIZE); + } + + public function sizeInBytes(): int + { + return strlen($this->buffer); + } + + /** + * Reads element $index straight out of memory + * + * The return type is the type parameter, so the specialization returns a declared `int` or + * `float` and the engine verifies what came out of the block. + */ + public function get(int $index): T + { + $this->assertUsable(); + $this->assertInBounds($index); + + return $this->elements()[$index]; + } + + /** + * Stores $value into element $index, in place + * + * The parameter is the type parameter, so a wrong element never reaches this body: the + * engine rejects it at the call boundary with a TypeError, which is deliberately not + * caught anywhere in this package. + */ + public function set(int $index, T $value): void + { + $this->assertUsable(); + $this->assertInBounds($index); + + $this->exclusiveElements()[$index] = $value; + } + + /** + * Grows the block by one element and stores $item in it + * + * Growth is literal string concatenation, which is what makes it cheap: the buffer is held + * by nothing but this property, so the engine reallocates it in place instead of copying. + * The reallocation may move it, which is why the pointer cache is dropped first. + */ + public function append(T $item): void + { + $this->assertUsable(); + + $this->buffer .= self::ZERO_ELEMENT; + $this->invalidate(); + + $this->exclusiveElements()[$this->count() - 1] = $item; + } + + /** + * Grows the block by a whole binary blob at once + */ + public function appendFromString(string $binary): void + { + $this->assertUsable(); + + if (strlen($binary) % self::ELEMENT_SIZE !== 0) { + throw NativeVectorException::misalignedBinary(static::class, strlen($binary)); + } + + $this->buffer .= $binary; + $this->invalidate(); + } + + /** + * Hands the block back as an ordinary PHP string + * + * The buffer itself is returned rather than a copy, so this costs one reference and no + * bytes. What keeps that honest is the exclusivity test on the write path: the returned + * string now shares the block, the next write notices and separates first, and the value + * the caller is holding is never changed behind their back. + */ + public function toBinary(): string + { + $this->assertUsable(); + + return $this->buffer; + } + + /** + * @return Traversable + */ + public function getIterator(): Traversable + { + $this->assertUsable(); + + // Reading through get() rather than yielding the bytes keeps one bounds check and one + // dereference as the only way an element is ever read + for ($index = 0, $count = $this->count(); $index < $count; ++$index) { + yield $index => $this->get($index); + } + } + + public function offsetExists(mixed $offset): bool + { + return !$this->destroyed && is_int($offset) && $offset >= 0 && $offset < $this->count(); + } + + /** + * @return T + */ + public function offsetGet(mixed $offset): mixed + { + return $this->get($offset); + } + + /** + * `$vector[] = $x` appends, `$vector[$i] = $x` overwrites + * + * Both delegate rather than write, because these two parameters cannot be narrowed: + * ArrayAccess declares them `mixed` and PHP's contravariance rules forbid re-declaring + * them. Delegation is what puts the engine's checks back in front of the store - both of + * them, because this file is `strict_types=1` and so an offset that is not an `int` is + * rejected on the way into `set()` by the same engine that rejects a wrong element. + */ + public function offsetSet(mixed $offset, mixed $value): void + { + if ($offset === null) { + $this->append($value); + + return; + } + + $this->set($offset, $value); + } + + public function offsetUnset(mixed $offset): never + { + throw NativeVectorException::fixedLayout(static::class); + } + + /** + * Releases the block; idempotent, and final for this instance + * + * There is nothing to free at the FFI level - the bytes belong to the Zend memory manager, + * which reclaims them the moment the last reference goes - so this drops the pointers and + * the buffer, in that order, and marks the instance unusable so a stale index cannot be + * turned into a dereference of memory that has been handed back. + */ + public function destroy(): void + { + $this->invalidate(); + $this->buffer = ''; + $this->destroyed = true; + } + + /** + * The read pointer: valid whether or not the block is shared + * + * Sharing is only a problem for writes. A reader through a shared block sees exactly the + * bytes everyone else sees, which is what sharing means. + */ + private function elements(): CData + { + return $this->elements ?? $this->acquire(); + } + + /** + * The write pointer: the block is guaranteed to be ours alone before it is returned + * + * The test is a single field read on the cached `zend_string` - no allocation, no engine + * call - and it is what makes `toBinary()` safe to hand the buffer out: a refcount above + * one means somebody else is holding these bytes, and the block is separated before a byte + * of it is written. + */ + private function exclusiveElements(): CData + { + $elements = $this->elements ?? $this->acquire(); + if ($this->string !== null && $this->string->gc->refcount === 1) { + return $elements; + } + + $this->separate(); + $this->invalidate(); + + return $this->acquire(); + } + + /** + * Points the cache at the current buffer, making it exclusive-capable on the way + * + * An interned string has no meaningful refcount (a permanent one reuses the field as a + * class-entry cache slot), so a block that arrived interned - a literal, or `''` - is + * separated here rather than trusted. Everything downstream may then read + * `gc->refcount` and believe it. + * + * The StringEntry wrapper is released immediately: its only job is to hand over the + * `zend_string *`, and the property is what keeps that pointer alive afterwards. + */ + private function acquire(): CData + { + $entry = new StringEntry($this->buffer); + if ($entry->isInterned()) { + $entry->release(); + $this->separate(); + $entry = new StringEntry($this->buffer); + } + + $this->string = $entry->getRawValue(); + // Core::cast() restores the array-to-pointer decay PHP 8.3 took away, so this reads + // "the bytes of val, as native elements" and not "the first eight bytes, as a pointer" + $this->elements = Core::cast($this->isFloat ? 'double *' : 'zend_long *', $this->string->val); + $entry->release(); + + return $this->elements; + } + + /** + * Forces the engine to give this instance a block nobody else holds + * + * Assigning a string offset onto itself is the userland spelling of the engine's own + * string separation: `zend_assign_to_string_offset()` copies the string whenever it is + * shared or immutable and merely forgets the cached hash when it is not, so this is a + * no-op on a block that is already ours and a copy on one that is not. + * + * The guard is not defensive: `''` is an interned empty string with no offset zero to + * assign to, and an empty block has no elements to protect anyway. + */ + private function separate(): void + { + if ($this->buffer !== '') { + $this->buffer[0] = $this->buffer[0]; + } + } + + /** + * Drops the pointer cache after anything that may have moved the block + */ + private function invalidate(): void + { + $this->string = null; + $this->elements = null; + } + + /** + * @throws NativeVectorException + */ + private function assertUsable(): void + { + if ($this->destroyed) { + throw NativeVectorException::destroyed(static::class); + } + } + + /** + * @throws NativeVectorBoundsException + */ + private function assertInBounds(int $index): void + { + $count = $this->count(); + if ($index < 0 || $index >= $count) { + throw NativeVectorBoundsException::index(static::class, $index, $count); + } + } +} From c8e667cc0cc42555f71e2fa0204667e58f91d76a Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 10 Aug 2026 12:26:49 +0000 Subject: [PATCH 2/4] feat(phpstan): describe a template's full surface in its generated stub NativeVector is the first template with private helpers, named constructors, class constants, a defaulted constructor parameter and interfaces, and the generator described none of those: every method came out `public`, statics were dropped entirely, defaults were lost, and the class implemented nothing. Analysed code therefore could not call `fromString()`, could not write `new (NativeVector::of('int'))()` without being told it passed too few arguments, could not index or count or iterate a vector - and could call a private helper and be told nothing. The generator now reproduces visibility, class constants, parameter defaults and own static methods (`of()` stays the one it writes out itself), and declares the interfaces PHP itself owns. A generic interface has to say what it was parameterized with, and only the template knows, so the `@implements` tags are copied off the class doc comment - the static-analysis source of truth, which is what decision 1 reserves doc comments for. The library's own interfaces still cannot be named: a stub is reflected before the analysed paths are indexed. The stub only wins if PHPStan is not also reading the real declaration, so NativeVector.php is excluded from scanning rather than just from analysis. Box and AttributeBox regenerate byte-identical. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013qW1hct9mDKsFaHTvkLggx --- composer.json | 6 +- phpstan.dist.neon | 6 + src/StubGenerator/StubGenerator.php | 178 ++++++++++++++++-- tests/StubGenerator/StubGeneratorTest.php | 87 +++++++++ tests/phpstan/generated/nativevector-stub.php | 103 ++++++++++ tests/phpstan/generated/placeholders.php | 4 + 6 files changed, 367 insertions(+), 17 deletions(-) create mode 100644 tests/phpstan/generated/nativevector-stub.php diff --git a/composer.json b/composer.json index 704f650..58da53b 100644 --- a/composer.json +++ b/composer.json @@ -54,8 +54,8 @@ "cs:check": "php-cs-fixer fix --dry-run --diff", "cs:fix": "php-cs-fixer fix", "test:analysis": "phpunit --testsuite analysis", - "stubs:generate": "php bin/generics-stubs --out=tests/phpstan/generated 'Lisachenko\\Generics\\Fixture\\Box' 'Lisachenko\\Generics\\Fixture\\AttributeBox'", - "stubs:check": "php bin/generics-stubs --check --out=tests/phpstan/generated 'Lisachenko\\Generics\\Fixture\\Box' 'Lisachenko\\Generics\\Fixture\\AttributeBox'", + "stubs:generate": "php bin/generics-stubs --out=tests/phpstan/generated 'Lisachenko\\Generics\\Fixture\\Box' 'Lisachenko\\Generics\\Fixture\\AttributeBox' 'Lisachenko\\Generics\\Native\\NativeVector'", + "stubs:check": "php bin/generics-stubs --check --out=tests/phpstan/generated 'Lisachenko\\Generics\\Fixture\\Box' 'Lisachenko\\Generics\\Fixture\\AttributeBox' 'Lisachenko\\Generics\\Native\\NativeVector'", "test:preload": "phpunit --filter PreloadTest" }, "scripts-descriptions": { @@ -67,7 +67,7 @@ "cs:check": "Check coding standards without fixing", "cs:fix": "Fix coding standards", "test:analysis": "Run the PHPStan extension tests, which need no engine", - "stubs:generate": "Regenerate the committed PHPStan stubs for the placeholder-form fixtures", + "stubs:generate": "Regenerate the committed PHPStan stubs for the placeholder-form templates", "stubs:check": "Fail if the committed PHPStan stubs no longer match the generator", "test:preload": "Boot preload.php in a child process; must run, never silently skip" }, diff --git a/phpstan.dist.neon b/phpstan.dist.neon index ea75d92..b7f021c 100644 --- a/phpstan.dist.neon +++ b/phpstan.dist.neon @@ -11,6 +11,11 @@ parameters: - tests - benchmarks excludePaths: + analyseAndScan: + # The one placeholder-form template that ships in src/: its `T` slots are the same + # fiction the fixtures below declare, so analysing the declaration reports the + # fiction. It is described properly through stubFiles, exactly like Box. + - src/Native/NativeVector.php analyse: # Analysis-only declarations, deliberately outside the PSR-4 layout - tests/phpstan @@ -37,4 +42,5 @@ parameters: stubFiles: - tests/phpstan/generated/box-stub.php - tests/phpstan/generated/attributebox-stub.php + - tests/phpstan/generated/nativevector-stub.php treatPhpDocTypesAsCertain: false diff --git a/src/StubGenerator/StubGenerator.php b/src/StubGenerator/StubGenerator.php index 9df59cf..640f4de 100644 --- a/src/StubGenerator/StubGenerator.php +++ b/src/StubGenerator/StubGenerator.php @@ -20,6 +20,7 @@ use Lisachenko\Generics\Template\TemplateParameterDefinition; use Lisachenko\Generics\Template\TemplateParser; use ReflectionClass; +use ReflectionClassConstant; use ReflectionMethod; use ReflectionNamedType; use ReflectionParameter; @@ -60,13 +61,21 @@ public function generate(string $className): GeneratedStub $reflection = new ReflectionClass($className); $body = []; + foreach ($reflection->getReflectionConstants() as $constant) { + if ($this->isOwnedBy($reflection, $constant)) { + $body[] = $this->constant($constant); + } + } foreach ($reflection->getProperties() as $property) { if ($this->isOwnedBy($reflection, $property)) { $body[] = $this->property($property, $definition); } } foreach ($reflection->getMethods() as $method) { - if ($this->isOwnedBy($reflection, $method) && !$method->isStatic()) { + // of() is the one method written out below rather than reproduced, so it is the + // one method skipped here; every other own method - static ones included, because + // a template's named constructors are part of how it is used - is reproduced + if ($this->isOwnedBy($reflection, $method) && strtolower($method->getName()) !== 'of') { $body[] = $this->method($method, $definition); } } @@ -80,10 +89,11 @@ public function generate(string $className): GeneratedStub $className, $reflection->getNamespaceName(), sprintf( - "%s\n%sclass %s\n{\n%s\n}\n", - $this->classDocBlock($definition), + "%s\n%sclass %s%s\n{\n%s\n}\n", + $this->classDocBlock($definition, $reflection), $reflection->isFinal() ? 'final ' : '', $reflection->getShortName(), + $this->implementsClause($reflection), implode("\n\n", $body), ), $this->placeholderNames($definition), @@ -111,7 +121,113 @@ private function placeholderNames(TemplateDefinition $definition): array return array_keys($names); } - private function classDocBlock(TemplateDefinition $definition): string + /** + * The `implements` list, restricted to the interfaces PHP itself declares + * + * A template's *own* interfaces cannot be named here - stub files are reflected before the + * analysed paths are indexed, so `GenericObject` would be an unknown name. PHP's own are a + * different case entirely: they are always present, and leaving them out is what would be + * wrong, because `$vector[0]`, `count($vector)` and `foreach ($vector as ...)` are only + * legal in analysed code if the stub says the class is an ArrayAccess, a Countable and an + * IteratorAggregate. Interfaces implied by another kept interface (Traversable behind + * IteratorAggregate) are dropped so the list reads the way the class declared it. + * + * @param ReflectionClass $reflection + */ + private function implementsClause(ReflectionClass $reflection): string + { + $internal = array_filter( + $reflection->getInterfaceNames(), + static fn(string $name): bool => (new ReflectionClass($name))->isInternal(), + ); + + $direct = array_filter( + $internal, + static function (string $name) use ($internal): bool { + foreach ($internal as $other) { + if ($other !== $name && is_subclass_of($other, $name)) { + return false; + } + } + + return true; + }, + ); + sort($direct); + + return $direct === [] + ? '' + : ' implements ' . implode(', ', array_map(static fn(string $n): string => '\\' . $n, $direct)); + } + + /** + * The `@implements` tags the template wrote, with their interface names made absolute + * + * A generic interface named in an `implements` clause has to say what it was parameterized + * with, or PHPStan reports the stub itself. Only the template knows - `ArrayAccess` + * is a statement about the class, not something reflection can derive - so this reads the + * class doc comment, which is where this package keeps everything static analysis needs + * (AGENTS.md decision 1; the prohibition on doc comments is on the *runtime* path, and a + * stub generator is the other one). + * + * The name is rewritten absolute because the stub is emitted into the template's own + * namespace, where a bare `ArrayAccess` would resolve to a class that does not exist. + * + * @param ReflectionClass $reflection + * @return list + */ + private function implementsTags(ReflectionClass $reflection): array + { + $docComment = $reflection->getDocComment(); + if ($docComment === false) { + return []; + } + + $matched = preg_match_all( + '{@implements\s+\\\\?(?P[A-Za-z_\x80-\xff][\w\x80-\xff]*(?:\\\\[\w\x80-\xff]+)*)(?P<.*>)}', + $docComment, + $matches, + PREG_SET_ORDER, + ); + if ($matched === false) { + return []; + } + + $tags = []; + foreach ($matches as $match) { + foreach ($reflection->getInterfaceNames() as $interface) { + if (strcasecmp($interface, $match['name']) === 0 + || strcasecmp((new ReflectionClass($interface))->getShortName(), $match['name']) === 0 + ) { + $tags[] = sprintf(' * @implements \\%s%s', $interface, $match['arguments']); + break; + } + } + } + + return $tags; + } + + /** + * A class constant, reproduced verbatim + * + * Constants carry no type parameter and therefore need no rewriting, but they are part of + * the class's surface: analysed code naming one has to find it here. + */ + private function constant(ReflectionClassConstant $constant): string + { + return sprintf( + ' %s const %s = %s;', + $constant->isPublic() ? 'public' : ($constant->isProtected() ? 'protected' : 'private'), + $constant->getName(), + self::exported($constant->getValue()), + ); + } + + /** + * @param ReflectionClass $reflection + */ + private function classDocBlock(TemplateDefinition $definition, ReflectionClass $reflection): string { $lines = ['/**']; foreach ($definition->parameters as $parameter) { @@ -121,6 +237,9 @@ private function classDocBlock(TemplateDefinition $definition): string ? sprintf(' * @template %s', $parameter->name) : sprintf(' * @template %s of \\%s', $parameter->name, ltrim($parameter->bound, '\\')); } + foreach ($this->implementsTags($reflection) as $tag) { + $lines[] = $tag; + } $lines[] = ' */'; return implode("\n", $lines); @@ -138,8 +257,7 @@ private function property(ReflectionProperty $property, TemplateDefinition $defi // A typed property with no default is uninitialized, which is a different thing from // one defaulting to null - and PHPStan is right to treat them differently $default = $property->hasDefaultValue() - // var_export() writes NULL/TRUE/FALSE in capitals, which no PHP style guide wants - ? sprintf(' = %s', str_replace(['NULL', 'TRUE', 'FALSE'], ['null', 'true', 'false'], var_export($property->getDefaultValue(), true))) + ? sprintf(' = %s', self::exported($property->getDefaultValue())) : ''; return sprintf( @@ -175,26 +293,51 @@ private function method(ReflectionMethod $method, TemplateDefinition $definition $returnType = $returnSlot === null ? $this->nativeType($method->getReturnType()) : 'mixed'; return sprintf( - '%s public function %s(%s)%s {}', + '%s %s%s function %s(%s)%s {}', $tags === [] ? '' : sprintf(" /**\n%s\n */\n", implode("\n", $tags)), + // Visibility is reproduced rather than assumed: a stub that promoted a private + // helper to public would let analysed code call it and be told nothing + $method->isPublic() ? 'public' : ($method->isProtected() ? 'protected' : 'private'), + $method->isStatic() ? ' static' : '', $method->getName(), implode(', ', $parameters), $returnType === '' ? '' : ': ' . $returnType, ); } + /** + * One parameter, keeping its default value + * + * Dropping the default would make an optional parameter required for analysis, and every + * `new (Template::of('int'))()` on a template with a defaulted constructor would be + * reported as passing too few arguments. + */ private function parameter(ReflectionParameter $parameter, ?SlotDefinition $slot): string { - $type = $slot === null ? $this->nativeType($parameter->getType()) : 'mixed'; + $type = $slot === null ? $this->nativeType($parameter->getType()) : 'mixed'; + $default = !$parameter->isVariadic() && $parameter->isDefaultValueAvailable() + ? sprintf(' = %s', self::exported($parameter->getDefaultValue())) + : ''; return trim(sprintf( - '%s %s$%s', + '%s %s$%s%s', $type, $parameter->isVariadic() ? '...' : '', $parameter->getName(), + $default, )); } + /** + * A value written the way PHP source spells it + * + * var_export() writes NULL/TRUE/FALSE in capitals, which no PHP style guide wants. + */ + private static function exported(mixed $value): string + { + return str_replace(['NULL', 'TRUE', 'FALSE'], ['null', 'true', 'false'], var_export($value, true)); + } + /** * `of()` declared inline, returning the class-string the extension then narrows * @@ -245,7 +388,12 @@ private function nativeType(?\ReflectionType $type): string return (string) $type; } - $name = $type->isBuiltin() ? $type->getName() : '\\' . ltrim($type->getName(), '\\'); + // `static` and `self` are relative names rather than class names, so reflection calls + // them non-builtin while a leading backslash would turn them into a class that is not + // there. `static` in particular is what a named constructor returns, which is how + // `Template::of('int')::fromString(...)` keeps its specialization for analysis + $relative = in_array(strtolower($type->getName()), ['static', 'self', 'parent'], true); + $name = $type->isBuiltin() || $relative ? $type->getName() : '\\' . ltrim($type->getName(), '\\'); return $type->allowsNull() && $type->getName() !== 'mixed' && $type->getName() !== 'null' ? '?' . $name @@ -278,11 +426,13 @@ private function parameterSlot(TemplateDefinition $definition, string $methodNam } /** - * @param ReflectionClass $reflection - * @param ReflectionProperty|ReflectionMethod $member + * @param ReflectionClass $reflection + * @param ReflectionProperty|ReflectionMethod|ReflectionClassConstant $member */ - private function isOwnedBy(ReflectionClass $reflection, ReflectionProperty|ReflectionMethod $member): bool - { + private function isOwnedBy( + ReflectionClass $reflection, + ReflectionProperty|ReflectionMethod|ReflectionClassConstant $member, + ): bool { return $member->getDeclaringClass()->getName() === $reflection->getName(); } } diff --git a/tests/StubGenerator/StubGeneratorTest.php b/tests/StubGenerator/StubGeneratorTest.php index 691f4e4..4208042 100644 --- a/tests/StubGenerator/StubGeneratorTest.php +++ b/tests/StubGenerator/StubGeneratorTest.php @@ -17,7 +17,9 @@ use Lisachenko\Generics\Fixture\AttributeBox; use Lisachenko\Generics\Fixture\Box; use Lisachenko\Generics\Fixture\NotATemplate; +use Lisachenko\Generics\Native\NativeVector; use PHPUnit\Framework\TestCase; +use ReflectionClass; /** * The generator needs no engine: it reads declarations, it does not specialize anything @@ -95,6 +97,91 @@ public function testOneClassPerFile(): void self::assertSame(1, substr_count($stub->render(), 'class Box')); } + /** + * The shipped template is the first one with private members, named constructors and + * interfaces, so it is what the tests from here on keep honest. None of them specializes anything: + * the generator reads declarations, so naming the class here cannot collide with the + * specializations NativeVectorTest owns. + */ + public function testVisibilityIsReproducedRatherThanAssumed(): void + { + $stub = (new StubGenerator())->generate(NativeVector::class); + + // A stub that promoted a private helper to public would let analysed code call it + self::assertStringContainsString('private function acquire(): \\FFI\\CData', $stub->classDeclaration); + self::assertStringContainsString('public function get(int $index): mixed', $stub->classDeclaration); + } + + public function testNamedConstructorsSurviveButOfIsStillWrittenOut(): void + { + $stub = (new StubGenerator())->generate(NativeVector::class); + + self::assertStringContainsString('public static function fromString(string $binary): static', $stub->classDeclaration); + self::assertStringContainsString('public static function withCapacity(int $count): static', $stub->classDeclaration); + + // of() is reproduced once, by the generator, with the return type the extension narrows + self::assertSame(1, substr_count($stub->classDeclaration, 'function of(')); + } + + public function testDefaultValuesSurviveSoAnOptionalParameterStaysOptional(): void + { + $stub = (new StubGenerator())->generate(NativeVector::class); + + // Without this, `new (NativeVector::of('int'))()` reads as passing too few arguments + self::assertStringContainsString("public function __construct(string \$binary = '')", $stub->classDeclaration); + } + + /** + * `$vector[0]`, `count($vector)` and `foreach` are only legal in analysed code if the stub + * says the class is an ArrayAccess, a Countable and an IteratorAggregate + */ + public function testPhpsOwnInterfacesAreDeclaredAndTheLibrarysAreNot(): void + { + $stub = (new StubGenerator())->generate(NativeVector::class); + + self::assertStringContainsString( + 'final class NativeVector implements \\ArrayAccess, \\Countable, \\IteratorAggregate', + $stub->classDeclaration, + ); + + // Traversable is implied by IteratorAggregate, and the library's own marker cannot be + // named at all - a stub is reflected before the analysed paths are indexed + self::assertStringNotContainsString('Traversable,', $stub->classDeclaration); + self::assertStringNotContainsString('GenericObject', $stub->classDeclaration); + } + + /** + * A generic interface has to say what it was parameterized with, and only the template knows + * + * This is the one thing the generator reads out of a doc comment, and it is allowed to: + * the prohibition in AGENTS.md decision 1 is on the *runtime* path, and doc comments are + * where this package deliberately keeps everything static analysis needs. The skip is that + * rule showing its edge - with `opcache.save_comments=0` there is no doc comment to read, + * which is exactly why nothing on the runtime path may depend on one. + */ + public function testGenericInterfacesAreParameterizedFromTheTemplatesOwnImplementsTags(): void + { + if ((new ReflectionClass(NativeVector::class))->getDocComment() === false) { + self::markTestSkipped( + 'Doc comments are unavailable (opcache.save_comments=0), so the stub generator - ' + . 'which is tooling, not the runtime path - has nothing to read the @implements ' + . 'tags from. Regenerate stubs on a host that keeps doc comments.', + ); + } + + $stub = (new StubGenerator())->generate(NativeVector::class); + + self::assertStringContainsString('@implements \\ArrayAccess', $stub->classDeclaration); + self::assertStringContainsString('@implements \\IteratorAggregate', $stub->classDeclaration); + } + + public function testClassConstantsArePartOfTheSurfaceAndAreReproduced(): void + { + $stub = (new StubGenerator())->generate(NativeVector::class); + + self::assertStringContainsString('public const ELEMENT_SIZE = 8;', $stub->classDeclaration); + } + public function testATemplateTheRuntimeWouldRejectIsRejectedHere(): void { // The generator parses through TemplateParser, so it accepts exactly what the runtime diff --git a/tests/phpstan/generated/nativevector-stub.php b/tests/phpstan/generated/nativevector-stub.php new file mode 100644 index 0000000..c17d332 --- /dev/null +++ b/tests/phpstan/generated/nativevector-stub.php @@ -0,0 +1,103 @@ + + * @implements \IteratorAggregate + */ +final class NativeVector implements \ArrayAccess, \Countable, \IteratorAggregate +{ + public const ELEMENT_SIZE = 8; + + private const ZERO_ELEMENT = '' . "\0" . '' . "\0" . '' . "\0" . '' . "\0" . '' . "\0" . '' . "\0" . '' . "\0" . '' . "\0" . ''; + + private string $buffer; + + private bool $isFloat; + + private bool $destroyed = false; + + private ?\FFI\CData $string = null; + + private ?\FFI\CData $elements = null; + + public function __construct(string $binary = '') {} + + public static function fromString(string $binary): static {} + + public static function withCapacity(int $count): static {} + + public function count(): int {} + + public function sizeInBytes(): int {} + + /** + * @return T + */ + public function get(int $index): mixed {} + + /** + * @param T $value + */ + public function set(int $index, mixed $value): void {} + + /** + * @param T $item + */ + public function append(mixed $item): void {} + + public function appendFromString(string $binary): void {} + + public function toBinary(): string {} + + public function getIterator(): \Traversable {} + + public function offsetExists(mixed $offset): bool {} + + public function offsetGet(mixed $offset): mixed {} + + public function offsetSet(mixed $offset, mixed $value): void {} + + public function offsetUnset(mixed $offset): never {} + + public function destroy(): void {} + + private function elements(): \FFI\CData {} + + private function exclusiveElements(): \FFI\CData {} + + private function acquire(): \FFI\CData {} + + private function separate(): void {} + + private function invalidate(): void {} + + private function assertUsable(): void {} + + private function assertInBounds(int $index): void {} + + /** + * @return class-string> + */ + public static function of(string ...$typeArguments): string {} +} diff --git a/tests/phpstan/generated/placeholders.php b/tests/phpstan/generated/placeholders.php index c4c864c..7455148 100644 --- a/tests/phpstan/generated/placeholders.php +++ b/tests/phpstan/generated/placeholders.php @@ -14,3 +14,7 @@ namespace Lisachenko\Generics\Fixture; class T {} + +namespace Lisachenko\Generics\Native; + +class T {} From daa7ec8a3746ed4e3a16248455825a7fdc605187 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 10 Aug 2026 12:27:01 +0000 Subject: [PATCH 3/4] test(native): cover the block, its bounds, its copy-on-write and its release NativeVectorTest is the sole owner of `NativeVector`, `NativeVector` and `NativeVector` across the suite (AGENTS.md section 6), which is why the shipped example runs in a subprocess. pack() is used throughout as the ground truth the block is checked against: the production path never encodes anything, so an independent encoder is exactly what makes a byte-level assertion worth having. The two that justify the design are the engine's own TypeError arriving on `append(1.5)` and on `$vector[0] = 'x'` - the second proving the array syntax really does delegate - and the copy-on-write case, where a binary handed out by `toBinary()` is shown to survive a later `set()` and a later `append()` unchanged. A literal cast into a vector is shown untouched by a write for the same reason, since every literal is interned. examples/native-vector.php casts a blob of native int64s, indexes it, grows it, round-trips it and destroys it; its test asserts the shape of the output and the TypeErrors, never wording. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013qW1hct9mDKsFaHTvkLggx --- examples/native-vector.php | 106 ++++++ tests/Example/NativeVectorExampleTest.php | 77 ++++ tests/Native/NativeVectorTest.php | 423 ++++++++++++++++++++++ 3 files changed, 606 insertions(+) create mode 100644 examples/native-vector.php create mode 100644 tests/Example/NativeVectorExampleTest.php create mode 100644 tests/Native/NativeVectorTest.php diff --git a/examples/native-vector.php b/examples/native-vector.php new file mode 100644 index 0000000..0059f1b --- /dev/null +++ b/examples/native-vector.php @@ -0,0 +1,106 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + * + */ +declare(strict_types=1); + +namespace Lisachenko\Generics\Example; + +use Lisachenko\Generics\Exception\NativeVectorException; +use Lisachenko\Generics\Generic; +use Lisachenko\Generics\Native\NativeVector; +use TypeError; + +/** + * A block of memory you can index, whose storage is an ordinary PHP string + * + * Run it: + * + * php -d ffi.enable=1 -d opcache.jit=off examples/native-vector.php + * + * The blob below stands in for anything that arrives as bytes - a file, a socket, a mmap'd + * region. `pack()` builds it here because the example has to produce its own input; the vector + * itself never packs or unpacks anything, which is the entire point: reading element 3 is a + * bounds check and one `zend_long *` dereference into those very bytes. + */ +require_once __DIR__ . '/../vendor/autoload.php'; + +Generic::bootstrap(); + +$blob = pack('q*', 10, 20, 30, 40); + +// 1. The cast. `NativeVector` is a real class, minted here, whose element slots say `int`. +$samples = new (NativeVector::of('int'))($blob); + +echo '1. get_class() : ', $samples::class, PHP_EOL; +echo ' type arguments : ', implode(', ', Generic::bindingOf($samples) ?? []), PHP_EOL; +echo ' count / bytes : ', count($samples), ' / ', $samples->sizeInBytes(), PHP_EOL; +echo PHP_EOL; + +// 2. Indexing reads and writes the block in place. +$samples[1] = -20; +$samples[] = 50; +$samples->append(60); + +echo '2. elements : ', implode(', ', iterator_to_array($samples)), PHP_EOL; +echo ' element 3 : ', $samples->get(3), PHP_EOL; +echo PHP_EOL; + +/* + * 3. The element type is enforced by the engine, on a class that did not exist when this file + * started running. Caught here only in order to print it. + */ +try { + $samples->append(1.5); + echo '3. UNEXPECTED : the engine accepted a float', PHP_EOL; +} catch (TypeError $error) { + echo '3. engine TypeError : ', $error->getMessage(), PHP_EOL; +} +echo PHP_EOL; + +/* + * 4. Back to a PHP string, byte for byte. The block is handed out rather than copied, so the + * next write separates it first and the value printed here cannot change afterwards. + */ +$roundTripped = $samples->toBinary(); +$samples->set(0, 999); + +echo '4. toBinary() round trip : ', implode(', ', unpack('q*', $roundTripped)), PHP_EOL; +echo ' still the same bytes : ', var_export($roundTripped === pack('q*', 10, -20, 30, 40, 50, 60), true), PHP_EOL; +echo ' vector moved on : ', $samples->get(0), PHP_EOL; +echo PHP_EOL; + +// 5. A second element kind is a second class, with `double` in the same slots. +$readings = new (NativeVector::of('float'))(pack('d*', 1.5, -2.25)); +$readings->append(M_PI); + +echo '5. a second specialization: ', $readings::class, PHP_EOL; +echo ' elements : ', implode(', ', iterator_to_array($readings)), PHP_EOL; + +try { + $readings->append('not a double'); + echo ' UNEXPECTED : the engine accepted a string', PHP_EOL; +} catch (TypeError $error) { + echo ' engine TypeError : ', $error->getMessage(), PHP_EOL; +} +echo PHP_EOL; + +// 6. destroy() releases the block; it is idempotent, and the instance is done afterwards. +$samples->destroy(); +$samples->destroy(); + +echo '6. destroyed count : ', count($samples), PHP_EOL; + +try { + $samples->get(0); + echo ' UNEXPECTED : a destroyed vector still read an element', PHP_EOL; +} catch (NativeVectorException $error) { + echo ' access after destroy : ', $error->getMessage(), PHP_EOL; +} diff --git a/tests/Example/NativeVectorExampleTest.php b/tests/Example/NativeVectorExampleTest.php new file mode 100644 index 0000000..1de48c2 --- /dev/null +++ b/tests/Example/NativeVectorExampleTest.php @@ -0,0 +1,77 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + * + */ +declare(strict_types=1); + +namespace Lisachenko\Generics\Example; + +use Lisachenko\Generics\RequiresEngine; +use PHPUnit\Framework\TestCase; + +/** + * Keeps the native-vector example from rotting + * + * It runs in a subprocess because it mints `NativeVector` and `NativeVector`, which + * NativeVectorTest already owns in the test process - two tests registering the same canonical + * name collide, and the second one is the one that fails (AGENTS.md section 6). + * + * The assertions are on the shape of the output and on the engine's TypeError actually + * arriving, never on wording that is free to change. + */ +final class NativeVectorExampleTest extends TestCase +{ + use RequiresEngine; + + public function testTheExampleRunsAndShowsWhatItClaims(): void + { + [$status, $output] = self::runExample(); + + self::assertSame(0, $status, sprintf("the example exited with %d:\n%s", $status, $output)); + + // Real, registered classes named after the specializations + self::assertStringContainsString('NativeVector', $output); + self::assertStringContainsString('NativeVector', $output); + + // The engine rejected the wrong element type - the entire point of the package + self::assertStringContainsString('engine TypeError', $output); + self::assertStringContainsString('must be of type int, float given', $output); + self::assertStringContainsString('must be of type float, string given', $output); + + // The block round-tripped back to a PHP string, and stayed the bytes it was handed out as + self::assertStringContainsString('still the same bytes : true', $output); + self::assertStringContainsString('vector moved on : 999', $output); + + // ...and a destroyed vector is done + self::assertStringContainsString('destroyed count : 0', $output); + self::assertStringContainsString('access after destroy', $output); + + // The branches that would mean the example silently stopped demonstrating anything + self::assertStringNotContainsString('UNEXPECTED', $output); + } + + /** + * @return array{int, string} + */ + private static function runExample(): array + { + $command = sprintf( + '%s -d ffi.enable=1 -d opcache.jit=off %s 2>&1', + escapeshellarg(PHP_BINARY), + escapeshellarg(dirname(__DIR__, 2) . '/examples/native-vector.php'), + ); + + $lines = []; + $status = 0; + exec($command, $lines, $status); + + return [$status, implode(PHP_EOL, $lines)]; + } +} diff --git a/tests/Native/NativeVectorTest.php b/tests/Native/NativeVectorTest.php new file mode 100644 index 0000000..0454077 --- /dev/null +++ b/tests/Native/NativeVectorTest.php @@ -0,0 +1,423 @@ + + * + * This source file is subject to the license that is bundled + * with this source code in the file LICENSE. + * + */ +declare(strict_types=1); + +namespace Lisachenko\Generics\Native; + +use ArrayAccess; +use Countable; +use IteratorAggregate; +use Lisachenko\Generics\Exception\NativeVectorBoundsException; +use Lisachenko\Generics\Exception\NativeVectorException; +use Lisachenko\Generics\GenericObject; +use Lisachenko\Generics\RequiresEngine; +use PHPUnit\Framework\TestCase; +use ReflectionMethod; +use TypeError; + +/** + * The sole owner of every NativeVector specialization the suite mints + * + * `NativeVector`, `NativeVector` and `NativeVector` are registered in the + * class table for the rest of the process, so AGENTS.md section 6 makes this the only test + * class allowed to name them. The shipped example uses the first two as well, which is why it + * runs in a subprocess. + * + * `pack()` appears throughout as the *ground truth* the vector is checked against: the + * production path never encodes or decodes anything, so an independent encoder is exactly what + * a byte-level assertion needs. If the two ever disagree, one of them is wrong about the + * machine's layout, and that is the assertion worth having. + */ +final class NativeVectorTest extends TestCase +{ + use RequiresEngine; + + /** + * The two names this class owns, behind a helper each so the rule is visible in one place + * + * The element type is spelled out rather than inferred, and that is the documented way to + * do it: a template described to PHPStan by a generated stub loses the `implements + * GenericObject` the `of()` return-type extension keys on, so the analyser cannot derive + * `NativeVector` from `of('int')` on this class and must be told (docs/limitations.md, + * "A stub-described template does not narrow `of()`"). Told once, here, every + * `$vector->get(0)` below analyses as `int`. + * + * @return class-string> + */ + private static function ints(): string + { + /** @var class-string> $specialized */ + $specialized = NativeVector::of('int'); + + return $specialized; + } + + /** + * @return class-string> + */ + private static function floats(): string + { + /** @var class-string> $specialized */ + $specialized = NativeVector::of('float'); + + return $specialized; + } + + /** + * The same two classes, with the element type deliberately left unspoken + * + * The engine-rejection tests below pass values that are wrong on purpose. Told the element + * type, PHPStan would report every one of them - correctly, and uselessly, because the + * run-time rejection *is* the assertion. Told nothing, it stands back and lets the engine + * do the work it is there to do. + * + * @return class-string> + */ + private static function unchecked(string $typeArgument): string + { + /** @var class-string> $specialized */ + $specialized = NativeVector::of($typeArgument); + + return $specialized; + } + + public function testASpecializationIsARealClassWithTheExpectedIdentity(): void + { + $vector = new (self::ints())(); + + self::assertSame(NativeVector::class . '', $vector::class); + self::assertInstanceOf(GenericObject::class, $vector); + self::assertInstanceOf(ArrayAccess::class, $vector); + self::assertInstanceOf(Countable::class, $vector); + self::assertInstanceOf(IteratorAggregate::class, $vector); + } + + public function testABinaryBlockIsCastAndHandedBackUnchanged(): void + { + $blob = pack('q*', 1, 2, 3, -4); + $vector = (self::ints())::fromString($blob); + + self::assertCount(4, $vector); + self::assertSame(32, $vector->sizeInBytes()); + self::assertSame($blob, $vector->toBinary()); + } + + public function testElementsAreReadAndWrittenThroughTheMethods(): void + { + $vector = (self::ints())::fromString(pack('q*', 10, 20, 30)); + + self::assertSame(20, $vector->get(1)); + + $vector->set(1, -20); + self::assertSame(-20, $vector->get(1)); + self::assertSame(pack('q*', 10, -20, 30), $vector->toBinary()); + } + + public function testArraySyntaxDelegatesToTheSameSlots(): void + { + $vector = (self::ints())::fromString(pack('q*', 7, 8)); + + self::assertSame(7, $vector[0]); + + $vector[1] = 99; + $vector[] = 100; + + self::assertSame(99, $vector[1]); + self::assertSame(100, $vector[2]); + self::assertSame(pack('q*', 7, 99, 100), $vector->toBinary()); + } + + public function testOffsetExistsAnswersForTheBlockAndNothingElse(): void + { + $vector = (self::ints())::fromString(pack('q*', 1, 2)); + + self::assertTrue(isset($vector[0])); + self::assertTrue(isset($vector[1])); + self::assertFalse(isset($vector[2])); + self::assertFalse(isset($vector[-1])); + self::assertFalse(isset($vector['nope'])); + } + + public function testIterationYieldsEveryElementInOrder(): void + { + $vector = (self::ints())::fromString(pack('q*', 5, 6, 7)); + + self::assertSame([5, 6, 7], iterator_to_array($vector)); + } + + public function testTheFullRangeOfANativeIntegerSurvivesTheRoundTrip(): void + { + $extremes = [PHP_INT_MIN, -1, 0, 1, PHP_INT_MAX]; + $vector = (self::ints())::fromString(pack('q*', ...$extremes)); + + self::assertSame($extremes, iterator_to_array($vector)); + self::assertSame(pack('q*', ...$extremes), $vector->toBinary()); + } + + public function testDoublesAreStoredAsDoubles(): void + { + $vector = (self::floats())::fromString(pack('d*', 1.5, -2.25, M_PI)); + + self::assertSame(1.5, $vector->get(0)); + self::assertSame(-2.25, $vector->get(1)); + self::assertSame(M_PI, $vector->get(2)); + + $vector->set(1, INF); + $vector->append(-0.0); + + self::assertSame(pack('d*', 1.5, INF, M_PI, -0.0), $vector->toBinary()); + } + + public function testTheEngineRejectsAFloatInAVectorOfIntegers(): void + { + $vector = new (self::unchecked('int'))(); + + $this->expectException(TypeError::class); + $this->expectExceptionMessage('must be of type int, float given'); + $vector->append(1.5); + } + + public function testTheEngineRejectsAStringWrittenThroughArraySyntax(): void + { + $vector = (self::unchecked('int'))::fromString(pack('q*', 1)); + + // The sugar delegates, so the TypeError is raised on set() rather than on offsetSet() + $this->expectException(TypeError::class); + $this->expectExceptionMessage(NativeVector::class . '::set()'); + $vector[0] = 'not an int'; + } + + public function testTheEngineRejectsAStringInAVectorOfFloats(): void + { + $vector = new (self::unchecked('float'))(); + + $this->expectException(TypeError::class); + $this->expectExceptionMessage('must be of type float, string given'); + $vector->append('1.5'); + } + + public function testAnIntegerIsAcceptedByAVectorOfFloats(): void + { + // Not a hole in the enforcement: int-to-float widening is allowed by the language even + // under strict_types, and what lands in the block is a double + $vector = new (self::unchecked('float'))(); + $vector->append(3); + + self::assertSame(3.0, $vector->get(0)); + self::assertSame(pack('d', 3.0), $vector->toBinary()); + } + + public function testANegativeIndexIsOutOfBounds(): void + { + $vector = (self::ints())::fromString(pack('q*', 1, 2)); + + $this->expectException(NativeVectorBoundsException::class); + $this->expectExceptionMessage('Index -1 is outside the 2 element(s)'); + $vector->get(-1); + } + + public function testTheIndexEqualToTheCountIsOutOfBounds(): void + { + $vector = (self::ints())::fromString(pack('q*', 1, 2)); + + $this->expectException(NativeVectorBoundsException::class); + $this->expectExceptionMessage('Index 2 is outside the 2 element(s)'); + $vector->set(2, 3); + } + + public function testAnEmptyVectorHasNoValidIndexAtAll(): void + { + $vector = new (self::ints())(); + + $this->expectException(NativeVectorBoundsException::class); + $this->expectExceptionMessage('The vector is empty'); + $vector->get(0); + } + + public function testAnElementCannotBeUnset(): void + { + $vector = (self::ints())::fromString(pack('q*', 1)); + + $this->expectException(NativeVectorException::class); + $this->expectExceptionMessage('cannot be unset'); + unset($vector[0]); + } + + public function testABinaryHandedOutIsNotChangedByALaterWrite(): void + { + $vector = (self::ints())::fromString(pack('q*', 1, 2, 3)); + $snapshot = $vector->toBinary(); + + $vector->set(0, 999); + + // The block was shared the moment it was handed out, so the write separated first + self::assertSame(pack('q*', 1, 2, 3), $snapshot); + self::assertSame(pack('q*', 999, 2, 3), $vector->toBinary()); + } + + public function testASharedBinaryIsNotChangedByALaterAppendEither(): void + { + $vector = (self::ints())::fromString(pack('q*', 4, 5)); + $snapshot = $vector->toBinary(); + + $vector->append(6); + + self::assertSame(pack('q*', 4, 5), $snapshot); + self::assertSame(pack('q*', 4, 5, 6), $vector->toBinary()); + } + + public function testTheStringACastWasMadeFromIsNeverWrittenInto(): void + { + // A literal is interned, which is the case that must never be written through + $source = "\x01\x00\x00\x00\x00\x00\x00\x00"; + $vector = (self::ints())::fromString($source); + + $vector->set(0, 42); + + self::assertSame("\x01\x00\x00\x00\x00\x00\x00\x00", $source); + self::assertSame(42, $vector->get(0)); + } + + public function testContentSurvivesTheReallocationsOfManyAppends(): void + { + $vector = new (self::ints())(); + $expected = []; + for ($index = 0; $index < 2_000; ++$index) { + $value = $index * -7; + $expected[] = $value; + $vector->append($value); + } + + self::assertCount(2_000, $vector); + self::assertSame($expected, iterator_to_array($vector)); + self::assertSame(pack('q*', ...$expected), $vector->toBinary()); + } + + public function testWithCapacityZeroFillsTheBlock(): void + { + $vector = (self::ints())::withCapacity(3); + + self::assertCount(3, $vector); + self::assertSame([0, 0, 0], iterator_to_array($vector)); + self::assertSame(str_repeat("\0", 24), $vector->toBinary()); + } + + public function testWithCapacityRejectsANegativeCount(): void + { + $this->expectException(NativeVectorException::class); + $this->expectExceptionMessage('no negative size'); + + (self::ints())::withCapacity(-1); + } + + public function testAppendFromStringGrowsTheBlockByAWholeBlob(): void + { + $vector = (self::ints())::fromString(pack('q*', 1)); + $vector->appendFromString(pack('q*', 2, 3)); + + self::assertSame([1, 2, 3], iterator_to_array($vector)); + self::assertSame(pack('q*', 1, 2, 3), $vector->toBinary()); + } + + public function testAppendFromStringRejectsAMisalignedBlob(): void + { + $vector = new (self::ints())(); + + $this->expectException(NativeVectorException::class); + $this->expectExceptionMessage('byte(s) would be left over'); + $vector->appendFromString('abc'); + } + + public function testCastingAMisalignedBinaryIsRejected(): void + { + $this->expectException(NativeVectorException::class); + $this->expectExceptionMessage('holds 8-byte elements'); + + (self::ints())::fromString('abcdefghij'); + } + + public function testDestroyIsIdempotent(): void + { + $vector = (self::ints())::fromString(pack('q*', 1, 2)); + + $vector->destroy(); + $vector->destroy(); + + self::assertCount(0, $vector); + self::assertSame(0, $vector->sizeInBytes()); + self::assertFalse(isset($vector[0])); + } + + public function testReadingAfterDestroyThrows(): void + { + $vector = (self::ints())::fromString(pack('q*', 1, 2)); + $vector->destroy(); + + $this->expectException(NativeVectorException::class); + $this->expectExceptionMessage('released by destroy()'); + $vector->get(0); + } + + public function testWritingAfterDestroyThrows(): void + { + $vector = (self::ints())::fromString(pack('q*', 1, 2)); + $vector->destroy(); + + $this->expectException(NativeVectorException::class); + $this->expectExceptionMessage('released by destroy()'); + $vector->append(3); + } + + public function testTheBinaryOfADestroyedVectorCannotBeAskedForEither(): void + { + $vector = (self::ints())::fromString(pack('q*', 1)); + $vector->destroy(); + + $this->expectException(NativeVectorException::class); + $this->expectExceptionMessage('released by destroy()'); + $vector->toBinary(); + } + + public function testTheRawTemplateCannotBeConstructed(): void + { + $this->expectException(NativeVectorException::class); + $this->expectExceptionMessage('cannot be constructed without a type argument'); + + new NativeVector(); + } + + public function testATypeArgumentWithNoNativeLayoutIsRejectedAtConstruction(): void + { + // The specialization itself is legal - it is a real class with `string` in its slots - + // and it is the layout that has no meaning, so this is the constructor's rejection + $specialized = NativeVector::of('string'); + self::assertTrue(class_exists($specialized, false)); + + $this->expectException(NativeVectorException::class); + $this->expectExceptionMessage('can only hold "int" (a zend_long) or "float" (a double)'); + new $specialized(); + } + + public function testTheDeclaredElementTypesFollowTheTypeArgument(): void + { + $ints = self::ints(); + $floats = self::floats(); + + self::assertSame('int', (string) (new ReflectionMethod($ints, 'get'))->getReturnType()); + self::assertSame('int', (string) (new ReflectionMethod($ints, 'append'))->getParameters()[0]->getType()); + self::assertSame('float', (string) (new ReflectionMethod($floats, 'get'))->getReturnType()); + self::assertSame('float', (string) (new ReflectionMethod($floats, 'set'))->getParameters()[1]->getType()); + + // ...and the index parameter was never a slot, so it is untouched + self::assertSame('int', (string) (new ReflectionMethod($floats, 'set'))->getParameters()[0]->getType()); + } +} From 22726edc5379d9f06813751366bce96833d54534 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 10 Aug 2026 12:27:17 +0000 Subject: [PATCH 4/4] docs: write up the native vector memory model and amend the touchpoint decision docs/native-vectors.md covers the model in full: the block of memory is a PHP string, both native scalar kinds are eight bytes and `zend_string.val` is eight-aligned so `i * 8` is naturally aligned, the byte order is the machine's by definition, the cast and round trip, growth by concatenation, the copy-on-write discipline behind `toBinary()`, `destroy()`, and the roadmap towards sized scalar kinds and C structures. Two honest entries go with it. limitations.md's `array` paragraph now points at the way out for scalar elements, and a new entry records what the stub costs: a stub cannot name this library's own interfaces, so a stub-described template is not a `GenericObject` and `of()` stops narrowing for it. The two mechanisms are exclusive; spell the specialization once at the boundary and everything downstream follows. static-analysis.md carries the neon snippet. AGENTS.md decision 13 is amended as the maintainer directed: `Native\` is the second sanctioned z-engine touchpoint, may use StringEntry (getRawValue included) and Core::cast, and confines both to NativeVector's private low-level methods, so the property that made decision 13 worth having still holds - every line that touches the engine can be named. `native` joins the commit scopes. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013qW1hct9mDKsFaHTvkLggx --- AGENTS.md | 37 +++++-- README.md | 41 ++++++- docs/limitations.md | 28 +++++ docs/native-vectors.md | 238 ++++++++++++++++++++++++++++++++++++++++ docs/static-analysis.md | 28 +++++ 5 files changed, 356 insertions(+), 16 deletions(-) create mode 100644 docs/native-vectors.md diff --git a/AGENTS.md b/AGENTS.md index 27b2124..eafbc86 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,15 +73,27 @@ needs a reason, not a refactor. Each is explained in full in the README. analyses itself with its own `extension.neon` for the same reason: a rule that crashes or a generator that drifts fails here rather than in somebody's project. -13. **`Monomorphizer` is the only class that talks to z-engine.** Everything else describes - what should happen in this package's own vocabulary (`SubstitutionRequest`, `SlotAddress`); - `Monomorphizer` translates that into z-engine's value objects at the single point where the - engine is asked. Keeping the dependency in one place is what makes it possible to say - exactly when engine state is touched. Two corollaries: z-engine is consumed through its - documented API only — never `Core::$executor`/`Core::$compiler`, never a method marked - `@internal` — and nothing here re-derives z-engine's own environment rules. The one - sanctioned exception outside `Monomorphizer` is `EngineCapabilities`, whose - `class_exists()` on a public z-engine class name is deliberate feature detection. +13. **`Monomorphizer` is the only class that talks to z-engine for *specialization*.** + Everything else describes what should happen in this package's own vocabulary + (`SubstitutionRequest`, `SlotAddress`); `Monomorphizer` translates that into z-engine's + value objects at the single point where the engine is asked. Keeping the dependency in one + place is what makes it possible to say exactly when engine state is touched. Two + corollaries: z-engine is consumed through its documented API only — never + `Core::$executor`/`Core::$compiler`, never a method marked `@internal` — and nothing here + re-derives z-engine's own environment rules. The one sanctioned exception for feature + detection is `EngineCapabilities`, whose `class_exists()` on a public z-engine class name + is deliberate. + + **Amended (maintainer-directed):** `Lisachenko\Generics\Native\` is the *second* sanctioned + touchpoint, and the only one that is not about specialization at all — a native vector's + hot path is memory access, which has nothing to translate into `SubstitutionRequest` terms. + It may use `ZEngine\Type\StringEntry` (including `getRawValue()`, which z-engine's own + AGENTS discourages for dependents: the maintainer sanctioned consuming the existing API + here rather than adding a class to z-engine) and `ZEngine\Core::cast`, and nothing else. + Both stay confined to `NativeVector`'s private low-level methods — `acquire()` is the only + place in the package outside `Monomorphizer` where FFI is reached for — so the same + property still holds: you can name every line that touches the engine. Adding a third + touchpoint is a design decision, not a refactor. 14. **Nothing in this package boots the engine.** Z-Engine initializes itself from its Composer bootstrap, including during the `opcache.preload` stage, so `require vendor/autoload.php` @@ -184,7 +196,7 @@ hand-written message at the call site. Add a factory rather than an inline ## 9. Conventional commits See [conventionalcommits.org](https://www.conventionalcommits.org/). Common scopes here: `runtime`, -`template`, `type`, `naming`, `phpstan`, `bench`, `ci`, `docs`. +`template`, `type`, `naming`, `native`, `phpstan`, `bench`, `ci`, `docs`. ``` feat(runtime): reify generic templates through class specialization @@ -197,6 +209,7 @@ test(template): cover promoted properties producing two slots ``` src/Attribute/ the template and slot attributes users write +src/Native/ NativeVector - the one shipped template, and the second engine touchpoint src/Template/ parsing a template class into a TemplateDefinition src/Type/ type-argument grammar, validation and resolution src/Naming/ specialized class-name mangling and parsing @@ -211,8 +224,8 @@ extension.neon wires the extension up, auto-loaded by phpstan/extension-insta benchmarks/ the monomorphization cost harness examples/ runnable, and covered by a test that runs them preload.php opcache.preload entry point - templates yes, specializations never -docs/ design.md, limitations.md, long-running.md, static-analysis.md - and the generated benchmarks.md +docs/ design.md, limitations.md, long-running.md, static-analysis.md, + native-vectors.md and the generated benchmarks.md tests/phpstan/generated/ committed generator output - regenerate, never edit ``` diff --git a/README.md b/README.md index b258b95..f3ce557 100644 --- a/README.md +++ b/README.md @@ -32,8 +32,9 @@ from a hand-written class, on a class that did not exist a microsecond ago.** - [Identity: sibling, not subclass](#identity-sibling-not-subclass) - [Static analysis](#static-analysis) and the [extension guide](docs/static-analysis.md) - [Long-running processes](#long-running-processes) and the [deployment guide](docs/long-running.md) -- [Known limitations](#known-limitations), in full in [docs/limitations.md](docs/limitations.md) +- [Native data vectors](#native-data-vectors) and the [memory model](docs/native-vectors.md) - [A runnable example](#a-runnable-example) +- [Known limitations](#known-limitations), in full in [docs/limitations.md](docs/limitations.md) - [What it costs](#what-it-costs) and the full [benchmark report](docs/benchmarks.md) - [Design notes](#design-notes) and the full [design document](docs/design.md) - [Contributing](#contributing) @@ -435,16 +436,47 @@ registration lives until the request (or worker) ends. Nothing survives shutdown [`docs/long-running.md`](docs/long-running.md) has the per-request budget, the worker and FPM recipes and a deployment checklist. +## Native data vectors + +`array` element types are the one thing this package cannot enforce at run time (see +[Known limitations](#known-limitations)). For scalars there is now a data structure that can: + +```php +use Lisachenko\Generics\Native\NativeVector; + +$samples = new (NativeVector::of('int'))($blobFromTheWire); + +echo $samples[0]; // a zend_long read straight out of the block +$samples[1] = -20; // written straight back into it +$samples->append(50); // the block grows by eight bytes +$samples->append(1.5); // TypeError, from the engine + +$bytes = $samples->toBinary(); // back to a PHP string, byte for byte +``` + +**The block of memory is a PHP string.** Element `i` lives at byte `i * 8` of that string's +`zend_string.val`, and every accessor reaches it as a `zend_long *` or a `double *` — there is +no encoding step, so `pack()`/`unpack()` appear nowhere on the path. The element type is +enforced because `get()`, `set()` and `append()` are declared with the type parameter, which is +a slot the engine really does check; the array syntax delegates to them rather than replacing +them. + +[`docs/native-vectors.md`](docs/native-vectors.md) has the memory model, the copy-on-write +discipline that makes `toBinary()` safe to hand out, the static-analysis setup, and the roadmap +towards sized scalar kinds and C structures. + ## A runnable example ```bash php -d ffi.enable=1 -d opcache.jit=off examples/collection.php +php -d ffi.enable=1 -d opcache.jit=off examples/native-vector.php ``` [`examples/collection.php`](examples/collection.php) specializes a collection template, prints `get_class()`, shows the engine's own `TypeError` rejecting the wrong element type, and shows the -template left exactly as it was. It is covered by a test that runs it, so it cannot quietly stop -working. +template left exactly as it was. [`examples/native-vector.php`](examples/native-vector.php) casts +a binary blob to a `NativeVector`, indexes it, grows it and hands it back as a string. Both +are covered by tests that run them, so they cannot quietly stop working. ## Known limitations @@ -457,7 +489,8 @@ anything else: array type, so a slot declared `array` is checked for being an array and nothing more. The doc tag still carries the element type and PHPStan still enforces it — but statically only. This is the one place where less is checked at run time than it looks, and therefore the most likely - source of false confidence in the package. + source of false confidence in the package. For scalar elements, + [native data vectors](#native-data-vectors) are the way out. - **A specialization is a sibling, not a subclass.** `$box instanceof Box` is `false` and cannot be made true. Type-hint an interface or an abstract base; both are preserved. - **Everything is request-scoped.** Class entries are request memory. Specialize at worker boot — diff --git a/docs/limitations.md b/docs/limitations.md index 124a77f..04355a5 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -125,6 +125,13 @@ for you and will not tell you it did not. This is the most likely source of false confidence in the package, which is why it is stated this bluntly. +**For scalars, there is now a way out.** A +[native data vector](native-vectors.md) puts the elements in a block of memory instead of an +array, which moves the element type from a slot the engine cannot check (`array`) to method slots +it can: `NativeVector` really does reject `1.5`, with the engine's own `TypeError`. It +covers `int` and `float` today. It is not a general answer — an `array` is still an +`array` — and it is a different data structure rather than a fix to this entry. + --- ## Loud: rejected at specialization time @@ -263,6 +270,27 @@ case, since their type-argument names are long by construction. Tracked upstream Everything else — method dispatch, class-typed *parameters*, builtin-typed properties — measured at parity with a hand-written class. +### A stub-described template does not narrow `of()` + +Affects static analysis only; nothing about the run time changes. A placeholder-form template +declares its type parameter as a native type, which an analyser resolves to a class that does not +exist, so the package ships a **generated stub** describing the class the way it behaves. Once +that stub is in your PHPStan configuration, `NativeVector::of('int')` stays +`class-string>` instead of narrowing to `class-string>`. + +**Why.** A stub cannot name this library's own interfaces — stub files are reflected before the +analysed paths are indexed — so the stubbed class is not a `GenericObject`, and +`TemplateOfReturnTypeExtension` is registered against exactly that marker (it has to be: `of()` +is a *trait* method, and a trait is not in any class's ancestry). The two mechanisms are +therefore exclusive: the stub gives you element types, the marker gives you `of()` narrowing. + +**What to do instead.** Spell the specialization where it enters your code and let inference do +the rest — `/** @var NativeVector $vector */` once, and `$vector->get(0)` is `int`, +`iterator_to_array($vector)` is `array` and `$vector[0]` is `int|null` from there on. +The setup and the exact `neon` snippet are in +[`native-vectors.md`](native-vectors.md#static-analysis). Attribute-form templates are unaffected: +they need no stub, so `of()` narrows for them as it always did. + --- ## Environment diff --git a/docs/native-vectors.md b/docs/native-vectors.md new file mode 100644 index 0000000..06fc799 --- /dev/null +++ b/docs/native-vectors.md @@ -0,0 +1,238 @@ +# Native data vectors + +`Lisachenko\Generics\Native\NativeVector` is a fixed-layout block of native scalars that you +index like an array, whose storage is an ordinary PHP string. + +```php +use Lisachenko\Generics\Native\NativeVector; + +$samples = new (NativeVector::of('int'))($blobFromTheWire); + +echo $samples[0]; // a zend_long read straight out of the block +$samples[1] = -20; // written straight back into it +$samples->append(50); // the block grows by eight bytes +$bytes = $samples->toBinary(); +``` + +It exists because of the entry in [`limitations.md`](limitations.md) that says the most: +[`array` element types are not enforced](limitations.md#arrayt-and-iterablet-element-types-are-not-enforced). +`zend_type` has no parametric array type, so a slot declared `array` and documented `array` +is checked for being an array and nothing else. A native vector answers that for the scalar +case by not being an array at all: the element type lives in the *method* slots, which the +engine does check, and the storage is a block of memory whose layout the element type fixes. + +--- + +## Contents + +- [The memory model](#the-memory-model) +- [Casting and the round trip](#casting-and-the-round-trip) +- [Growth](#growth) +- [What the engine enforces](#what-the-engine-enforces) +- [Copy-on-write, and why `toBinary()` is safe](#copy-on-write-and-why-tobinary-is-safe) +- [`destroy()`](#destroy) +- [Static analysis](#static-analysis) +- [What it is not, yet](#what-it-is-not-yet) + +--- + +## The memory model + +**The block of memory *is* a PHP string.** Element `i` lives at byte `i * 8` of that string's +`zend_string.val`, in the machine's own layout, and every accessor reaches it as a native +pointer: + +| Type argument | C type | Width | How an element is reached | +|---|---|---|---| +| `int` | `zend_long` | 8 bytes | `Core::cast('zend_long *', $zstr->val)`, then `$ptr[$i]` | +| `float` | `double` | 8 bytes | `Core::cast('double *', $zstr->val)`, then `$ptr[$i]` | + +Three facts make that a definition rather than a trick: + +- **PHP scalars are fixed-width.** `int` is a `zend_long` and `float` is a `double` on every + platform PHP supports, both 8 bytes. There is no narrower scalar to store, which is why the + element size is a constant (`NativeVector::ELEMENT_SIZE`) rather than a per-instance field. +- **`zend_string.val` starts 8-aligned,** so `i * 8` is always a naturally aligned offset. No + padding arithmetic, no unaligned loads. +- **There is no encoding step.** `pack()` and `unpack()` appear nowhere in the production path. + A read initializes a PHP value from the bytes that are already there; a write copies a PHP + value into them. The byte order, the sign representation and the float format are the + machine's, by definition rather than by choice — which is exactly what you want when the bytes + came from `mmap`, from a socket, or from another process on the same host, and exactly what + you must convert for yourself when they came from a different architecture. + +`pack()` appears throughout the *tests* for the opposite reason: it is an independent encoder, +and checking the block against one is how a byte-level assertion is made worth having. + +## Casting and the round trip + +The constructor is the cast. It adopts a binary string as the block, rejecting one that is not a +whole number of elements: + +```php +$vector = NativeVector::of('int')::fromString(pack('q*', 1, 2, 3)); // or new (…)($blob) +$vector->toBinary() === pack('q*', 1, 2, 3); // true, byte for byte +``` + +`fromString()` and `new (…)($binary)` are the same thing; `withCapacity($count)` is the +zero-filled variant. `toBinary()` hands the block back as an ordinary PHP string and costs one +reference rather than a copy. + +The string a vector was cast from is never written into. It may be interned (every string +literal is), or it may still be held by the caller — either way the constructor makes the block +exclusively the vector's before anything can point into it. + +## Growth + +Growth is literal string concatenation: + +```php +$vector->append(60); // $this->buffer .= 8 zero bytes, then store +$vector->appendFromString(pack('q*', 7, 8)); // a whole blob at once +``` + +That is cheaper than it looks. The buffer is held by the vector's own property and nothing else, +so the engine extends it in place instead of copying — which is precisely why nothing in the +class holds a second reference on it. The reallocation may move the block, so the cached pointer +is dropped across every growth and re-acquired on the next access. + +## What the engine enforces + +`get()`, `set()` and `append()` are declared with the type parameter itself, so a specialization +carries `int` or `float` in those slots and the rejection comes from the Zend Engine: + +```php +$ints = new (NativeVector::of('int'))(); +$ints->append(1.5); // TypeError: …NativeVector::append(): Argument #1 ($item) must be of type int, float given +$ints[0] = 'nope'; // TypeError, on set(): the array syntax delegates +``` + +That delegation is the reason the class has both an element API and array syntax. +`ArrayAccess::offsetSet(mixed, mixed)` cannot narrow its parameters — PHP's contravariance rules +forbid re-declaring them — so `offsetSet()` calls `set()`, and `$vector[$i] = $x` gets exactly +the check `$vector->set($i, $x)` gets. `$vector[] = $x` appends the same way. A non-integer +offset is rejected by the same engine, for the same reason: `set()` declares `int $index` and +this package is `strict_types=1`. + +Two failures are the vector's own rather than the engine's, and both are named exceptions: + +- `NativeVectorBoundsException` — an index below zero or at/after `count()`. Checked before every + read and every write, because it is what stands between a userland off-by-one and a wild + pointer dereference. +- `NativeVectorException` — the raw template constructed with no type argument, a type argument + with no native layout (`NativeVector` is a legal class with an impossible layout), a + misaligned binary, a negative capacity, use after `destroy()`, and `unset($vector[$i])`, which + a contiguous block has no meaning for. + +`int`-to-`float` widening is allowed by the language even under `strict_types`, so a +`NativeVector` accepts `3` and stores the double `3.0`. That is not a hole in the +enforcement; it is the same rule every `float` parameter in PHP follows. + +## Copy-on-write, and why `toBinary()` is safe + +Writing through a pointer into a PHP string's bytes is only correct while nobody else holds that +string. `toBinary()` hands the block out, which means somebody does: + +```php +$snapshot = $vector->toBinary(); +$vector->set(0, 999); + +$snapshot; // unchanged - the write separated the block first +$vector->toBinary(); // the new bytes +``` + +The discipline behind that is deliberately small: + +1. The vector caches the `zend_string *` and the typed element pointer, and holds **no engine + reference** on the string. The property is what keeps it alive; a wrapper holding a second + reference would push the refcount to 2 for good, defeat the in-place growth path, and make + step 2 useless. +2. Before every write, the refcount on the cached `zend_string` is read — one field access, no + allocation and no engine call. `1` means the block is the vector's alone and the write goes + straight through. +3. Anything else means the block is shared, and it is separated first, by assigning a string + offset onto itself (`$buffer[0] = $buffer[0]`). That is the userland spelling of the engine's + own separation: `zend_assign_to_string_offset()` copies a string that is shared or immutable + and merely forgets its cached hash when it is not, so the operation is a no-op on a block + that is already exclusive. +4. The pointer cache is dropped whenever the block may have moved — a separation, a growth, a + `destroy()` — and re-acquired lazily. + +Reads never separate. A reader through a shared block sees the bytes everybody else sees, which +is what sharing means. + +An interned string has no meaningful refcount at all — a permanent one reuses the field as the +engine's class-entry cache slot — so a block that arrives interned (a literal, or the empty +string) is separated when the pointer is acquired rather than trusted. That is why `''` is the +one buffer value the separation trick is skipped for: it has no offset zero, and no elements to +protect. + +## `destroy()` + +```php +$vector->destroy(); // idempotent +$vector->destroy(); +$vector->get(0); // NativeVectorException: …released by destroy() +``` + +There is nothing to free at the FFI level: the bytes belong to the Zend memory manager, which +reclaims them when the last reference goes. What `destroy()` does is drop the pointers and the +buffer and mark the instance unusable, so a stale index cannot be turned into a dereference of +memory that has been handed back. `count()` and `sizeInBytes()` answer `0` afterwards; every +element accessor throws. + +## Static analysis + +`NativeVector` is a **placeholder-form** template: it declares `get(int $index): T` natively, +because that native `T` is what the engine keys substitution on. For an analyser that +declaration is a fiction, so the package ships a generated stub — +`tests/phpstan/generated/nativevector-stub.php`, produced by `composer stubs:generate` — which +describes the class the way it behaves: `mixed` where the fiction was, with `@param T`/`@return +T` carrying the meaning, plus `@implements ArrayAccess` and +`@implements IteratorAggregate`. + +To get element types in your own project, point PHPStan at that stub **and** keep it from +reading the real declaration, which would otherwise win: + +```neon +parameters: + excludePaths: + analyseAndScan: + - vendor/lisachenko/userland-php-generics/src/Native/NativeVector.php + stubFiles: + - vendor/lisachenko/userland-php-generics/tests/phpstan/generated/nativevector-stub.php +``` + +With that in place, a spelled specialization infers all the way through: + +```php +/** @var NativeVector $vector */ +$vector->get(0); // int +iterator_to_array($vector); // array +$vector[0]; // int|null - the nullability PHPStan gives every ArrayAccess read +$vector->toBinary(); // string +``` + +The specialization does have to be *spelled*. A stub cannot name this library's own interfaces +(stub files are reflected before the analysed paths are indexed), so the stubbed class is not a +`GenericObject`, and the return-type extension that narrows `Template::of('int')` is registered +against exactly that marker. `NativeVector::of('int')` therefore stays +`class-string>` once the stub is in play — write the type at the boundary, in the +`@var` or the `@param` where the vector enters your code, and everything downstream follows. This +is a documented trade-off rather than a defect on either side: with the stub you get element +types and no `of()` narrowing, without it you get `of()` narrowing and `get()` typed as the +placeholder class. `get()` is the accessor to prefer either way — it is the precisely typed one. + +## What it is not, yet + +- **Only `int` and `float`.** Every other type argument builds a perfectly real class whose + constructor then refuses, because there is no layout to give it. The next phase is **sized + scalar kinds** — `int32`, `uint16`, `float32` and friends — which turns `ELEMENT_SIZE` from a + constant into a property and the two-way element-kind flag into a descriptor. +- **No C structures.** The phase after that is a layout description for records, so a vector of + structs can be indexed the same way. +- **One dimension, and no slicing.** `appendFromString()` and `toBinary()` are the two bulk + operations; anything else is done on the binary string, which is the point of it being one. +- **Request-scoped, like every specialization.** See + [`long-running.md`](long-running.md): mint `NativeVector` at worker boot, never during + `opcache.preload`. diff --git a/docs/static-analysis.md b/docs/static-analysis.md index 7336845..b47c94b 100644 --- a/docs/static-analysis.md +++ b/docs/static-analysis.md @@ -113,6 +113,34 @@ Both were learned the hard way and are not negotiable: analysed paths are indexed. That is why the generated `of()` is written out in full rather than inherited from `GenericTemplate`. +PHP's *own* interfaces are the exception to the second one, and the generator does declare them: +`$vector[0]`, `count($vector)` and `foreach` are only legal in analysed code if the stub says +the class is an `ArrayAccess`, a `Countable` and an `IteratorAggregate`. A generic interface has +to say what it was parameterized with, and only the template knows — so the generator copies the +`@implements` tags off the template's own class doc comment, which is where this package keeps +everything static analysis needs. Everything else about a stub is read from reflection: +visibility, class constants, parameter defaults and static named constructors are all +reproduced, because analysed code that names one has to find it. + +### The stub replaces the declaration, so keep PHPStan from reading both + +A stub only wins if the analyser is not also reading the real file. In your own project the +template lives in a path you analyse, so exclude it from **scanning**, not just from analysis: + +```neon +parameters: + excludePaths: + analyseAndScan: + - src/Box.php + stubFiles: + - var/generics-stubs/box-stub.php +``` + +The price of doing that is stated in +[`limitations.md`](limitations.md#a-stub-described-template-does-not-narrow-of): the stubbed +class is not a `GenericObject`, so `Box::of('int')` no longer narrows and the specialization has +to be spelled once, in the `@var` or `@param` where it enters your code. + ### The placeholders file The placeholder types never exist at run time — an undefined class name is precisely what gives