Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions .agents/skills/develop-typerb-native/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
name: develop-typerb-native
description: Implement and review TypeRB Native compiler, MIR, runtime, bootstrap, gate, fixture, and benchmark work. Use for changes in type-rb/type-rb-native, especially when deciding the current gate scope, preserving TypeRB semantics, or updating the pinned reference compiler.
---

# Develop TypeRB Native

Work on one recorded experiment gate at a time.

## Establish the boundary

1. Read `README.md`, `docs/architecture.md`, and `docs/experiment-plan.md`.
2. Read the decisions relevant to the change.
3. Treat `type-rb/type-rb` at `TYPE_RB_REVISION` as the language, compiler, and
conformance source of truth.
4. State the current gate and its exit condition before expanding scope.

## Implement

- Write repository-owned executable compiler and runtime source in TypeRB.
- Allow external code generators, assemblers, linkers, SDKs, and system
libraries only behind explicit boundaries whose time and distribution cost
can be measured.
- Preserve source origins and exact TypeRB semantics through every lowering.
- Reject unknown, malformed, unsupported, or unverifiable input with stable,
deterministic diagnostics. Never add a semantic fallback or `Any` escape
hatch to improve a benchmark.
- Keep bootstrap snapshots, Native MIR, ABI profiles, and runtime interfaces
internal and unstable until a decision explicitly promotes them.
- Add only the feature set required by the active gate. Record a new decision
before changing language semantics, ownership boundaries, self-hosting
criteria, or backend selection policy.

## Verify

From the repository root, use the compiler revision in `TYPE_RB_REVISION` and
run:

```sh
trb fmt --check .
trb check
TYPE_RB_NATIVE_ROOT="$PWD" trb test
```

For executable gates, run the same differential corpus through the optimized
Go reference baseline and every active native candidate. Count frontend,
serialization, lowering, code generation, assembly, linking, runtime, sidecar,
and distribution costs according to `docs/experiment-plan.md`.

When the active gate passes, stop before starting the next gate. Report the
implemented subset, evidence for every exit condition, measurements, known
limitations, discarded paths, and decisions that need maintainer discussion.
44 changes: 44 additions & 0 deletions .github/workflows/gate-zero.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Gate 0

on:
pull_request:
push:
branches:
- main

permissions:
contents: read

jobs:
verify:
runs-on: ubuntu-latest
steps:
- name: Check out TypeRB Native
uses: actions/checkout@v5

- name: Check out the pinned TypeRB compiler
uses: actions/checkout@v5
with:
repository: type-rb/type-rb
ref: 30b5f5206680a39a02966d1579f76de55760a349
path: .type-rb

- name: Set up Go for the bootstrap compiler
uses: actions/setup-go@v6
with:
go-version-file: .type-rb/go.mod
cache-dependency-path: .type-rb/go.sum

- name: Verify the compiler revision pin
run: test "$(cat TYPE_RB_REVISION)" = "$(git -C .type-rb rev-parse HEAD)"

- name: Verify formatting
run: .type-rb/trb fmt --check .

- name: Check TypeRB sources
run: .type-rb/trb check

- name: Run Gate 0 tests and fixtures
env:
TYPE_RB_NATIVE_ROOT: ${{ github.workspace }}
run: .type-rb/trb test
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
.DS_Store
/.trb/
/bin/
/build/
/dist/
/tmp/
6 changes: 6 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,9 @@ TypeRB backend or product commitment.
unchecked lowering, or `Any`-shaped escape hatches.
- Add only the structure required by the current experiment gate. Do not add
release, package, or compatibility machinery before a real consumer needs it.
- Write repository-owned compiler and runtime implementation source in TypeRB.
External code generators, assemblers, linkers, SDKs, and system libraries are
allowed when their role and cost are explicit.
- Treat reproducible TypeRB self-hosting as a required promotion outcome. The
Go reference compiler is an early bootstrap and differential oracle, not part
of the ordinary final release path.
17 changes: 17 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,23 @@ Performance does not justify changing portable integer behavior, Unicode
behavior, failure behavior, initialization order, source attribution, or other
TypeRB guarantees.

Repository-owned compiler and runtime implementation source must be TypeRB.
External backend and platform tools are allowed when their revisions, licenses,
invocations, and distribution costs are explicit. Do not introduce a permanent
Go, Rust, Zig, or C host implementation as an intermediate shortcut.

Use the pinned reference compiler revision recorded in `TYPE_RB_REVISION` for
gate verification. A revision update is a reviewed compatibility change, not
an incidental tool upgrade.

Run the Gate 0 checks from the repository root:

```sh
trb fmt --check .
trb check
TYPE_RB_NATIVE_ROOT="$PWD" trb test
```

## Backend changes

Backend candidates share Native MIR, conformance inputs, and benchmark policy.
Expand Down
55 changes: 41 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@
TypeRB Native explores whether a TypeRB-specific native compiler and runtime
can improve end-to-end build time, generated-program performance, and deployed
binary size relative to an optimized release executable produced by the
reference compiler's Go backend.
reference compiler's Go backend. Its long-term objective is a self-hosted
compiler whose repository-owned implementation is written in TypeRB and whose
ordinary release/bootstrap path does not require Go or another host language.

The [TypeRB repository](https://github.com/type-rb/type-rb) remains the source
of truth for the language specification, reference compiler, supported
Expand All @@ -19,6 +21,8 @@ semantics; it does not define a native-only TypeRB dialect.

- Test a native AOT pipeline without requiring the Go toolchain to compile a
TypeRB application.
- Reach reproducible self-hosting: a native TypeRB compiler builds the next
equivalent native TypeRB compiler from TypeRB source.
- Design a small Native MIR, target ABI profiles, data layout, and runtime.
- Compare multiple machine-code strategies behind the same MIR and semantics.
- Measure complete toolchains, including code generation, linking, runtime,
Expand All @@ -28,11 +32,13 @@ semantics; it does not define a native-only TypeRB dialect.

## Current status

The repository currently contains design and experiment policy only. It does
not compile TypeRB programs and does not provide a production runtime, stable
ABI, stable artifact format, or compatibility guarantee.
Gate 0 implements the experimental boundary in TypeRB: strict decoding of
versioned, data-only bootstrap snapshots, lowering to Native MIR, MIR
verification, deterministic diagnostics, and source-origin preservation. It
does not yet generate or execute machine code and provides no production
runtime, stable ABI, stable artifact format, or compatibility guarantee.

The first implementation milestone will use a small, heap-free corpus with
Gate 1 will use a small, heap-free corpus with
functions, direct calls, control flow, scalar values, exact TypeRB integer and
failure behavior, simple static-layout values, and observable output.

Expand All @@ -57,15 +63,33 @@ validate -> Native MIR -> optimize -> codegen -> object -> link

The bootstrap snapshot is a temporary, data-only bridge. It is not the public
compiler tooling protocol, a package-extension API, or a stable serialization
of the reference compiler's internal typed IR. Native MIR remains internal to
this repository.
of the reference compiler's internal typed IR. During early gates the Go
reference compiler may produce that bridge. Later gates replace the bridge's
frontend side with a TypeRB implementation in this repository. Native MIR
remains internal here.

The intended bootstrap sequence is:

```text
Go reference compiler -> B0 native compiler from TypeRB source
B0 native compiler -> B1 native compiler
B1 native compiler -> B2 native compiler
B1 and B2 -> reproducibly equivalent artifacts
```

The Go compiler remains a differential oracle, but it is not part of the
ordinary self-hosted release/bootstrap chain. External code generators,
assemblers, linkers, SDKs, and system libraries may remain explicit toolchain
dependencies.

See [Architecture](docs/architecture.md) for the ownership and pipeline
boundaries.

## Backend experiments

Candidate strategies under consideration are:
QBE is the first planned executable path because it gives the lowest-cost test
of the TypeRB runtime and ABI hypothesis. Candidate roles under consideration
are:

- [Cranelift](https://cranelift.dev/) as a balanced fast-codegen candidate;
- [LLVM](https://llvm.org/) as a high-optimization comparison;
Expand All @@ -85,11 +109,13 @@ measurement rules, and abandonment criteria.

## Non-goals

The initial experiment does not attempt to:
The initial gates do not attempt to:

- replace the Go implementation of the reference compiler;
- port the compiler to Rust, Zig, or another host implementation language;
- make self-hosting a prerequisite for native-code feasibility;
- replace external code generators, assemblers, linkers, SDKs, or system
libraries merely to claim self-hosting;
- implement the full TypeRB frontend before native execution feasibility has
passed its earlier gates;
- commit TypeRB to a supported native mode;
- expose mutable compiler internals or backend hooks as a package API;
- support the full standard library, Web, ORM, Jobs, or native package
Expand All @@ -99,15 +125,16 @@ The initial experiment does not attempt to:
- claim an advantage over Go without reproducible end-to-end measurements.

External code generators, assemblers, and linkers may be used as experimental
components. This experiment remains responsible for preserving TypeRB semantics
and for implementing its MIR, ABI profiles, and runtime. Normative semantics
remain in the reference repository.
components. Repository-owned compiler, MIR, ABI, and runtime implementation
source is written in TypeRB. Normative semantics remain in the reference
repository.

## Documentation

- [Architecture](docs/architecture.md)
- [Experiment plan](docs/experiment-plan.md)
- [Decision 0001: Experimental native toolchain boundary](docs/decisions/0001-experimental-native-toolchain.md)
- [Decision 0002: TypeRB-owned self-hosting](docs/decisions/0002-typerb-owned-self-hosting.md)
- [Contributing](CONTRIBUTING.md)
- [Security](SECURITY.md)

Expand Down
1 change: 1 addition & 0 deletions TYPE_RB_REVISION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
30b5f5206680a39a02966d1579f76de55760a349
56 changes: 45 additions & 11 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,12 @@ question is whether a native pipeline can improve the practical tradeoff among
build time, execution performance, and deployed binary size after all required
tooling is counted.

The experiment is not a host-language rewrite. Self-hosting and native code
generation are independent concerns: the Go implementation of the reference
compiler can bootstrap a native backend, and a hypothetical TypeRB-authored
compiler could still target Go. The latter is an architectural example, not a
planned milestone.
The experiment is not a port to a different host language. Native execution
and self-hosting are separate checkpoints, but both belong to the intended
path. The Go reference compiler bootstraps early artifacts and remains a
differential oracle. The completed compiler and runtime owned by this
repository are written in TypeRB and reproduce themselves without Go in the
ordinary release/bootstrap path.

## Ownership boundary

Expand All @@ -26,6 +27,7 @@ The [reference TypeRB repository](https://github.com/type-rb/type-rb) owns:

This repository owns only experimental native concerns:

- an independent TypeRB-authored frontend when the self-hosting gates reach it;
- bootstrap snapshot validation and lowering;
- Native MIR and its verifier;
- native data layout and target ABI profiles;
Expand All @@ -34,9 +36,23 @@ This repository owns only experimental native concerns:
- the experimental runtime; and
- native correctness, portability, and performance measurements.

The normal TypeRB build, test, and release paths must not depend on this
repository. A language-level change discovered here belongs in the reference
repository's normal design and review process.
The normal reference TypeRB build, test, and release paths must not depend on
this repository. A language-level change discovered here belongs in the
reference repository's normal design and review process. Until the independent
frontend exists, the reference implementation may provide a narrow snapshot
producer on a short-lived, removable experimental surface.

## Implementation-language boundary

Repository-owned executable compiler and runtime source is written in TypeRB.
Go, Rust, Zig, C, or another existing implementation language is not introduced
as the permanent host for those components. Generated C, assembly, object
files, or backend IR are outputs rather than maintained implementation source.

External tools remain allowed and must be accounted for. QBE or LLVM, an
assembler, a linker, an SDK, and system libraries do not violate self-hosting;
they are explicit dependencies of a TypeRB-authored compiler in the same way a
linker can be a dependency of another self-hosted language implementation.

## Pipeline

Expand All @@ -62,6 +78,22 @@ TypeRB source
Each boundary must preserve source origins so diagnostics and runtime failures
can eventually refer to authored TypeRB source.

The bootstrap snapshot is intentionally transitional. After native execution
and runtime viability are established, the repository gains its own
TypeRB-authored parser, resolver, checker, and lowering. The reproducible
self-hosting sequence is:

```text
reference Go compiler -> B0 from TypeRB compiler sources
B0 -> B1
B1 -> B2
compare(B1, B2) -> equivalent under the reproducibility policy
```

Published native releases use a previously released native compiler as their
seed. Building the bootstrap seed from Go is a recovery/development path, not
an ordinary release requirement.

## Bootstrap snapshot

The bootstrap snapshot is a deterministic, versioned, target-neutral, data-only
Expand Down Expand Up @@ -111,8 +143,9 @@ concepts into portable TypeRB source.

## Backend adapters

Candidate adapters consume the same verified, target-neutral MIR subset. Target
lowering selects a versioned ABI profile for an operating system and
Candidate adapters consume the same verified, target-neutral MIR subset. QBE
is tried first to minimize the cost of the initial executable experiment.
Target lowering selects a versioned ABI profile for an operating system and
architecture. Backend comparisons on the same target use the same profile.

| Candidate | Experimental role |
Expand Down Expand Up @@ -181,7 +214,8 @@ repository is stable. Official TypeRB packages must not depend on it.
Promotion to a supported TypeRB target is a separate decision. It would require
representative portable conformance, source-mapped diagnostics and failures,
runtime and package boundaries, reproducible builds, primary-platform support,
and an end-to-end advantage after the complete toolchain is counted.
an end-to-end advantage after the complete toolchain is counted, and a
reproducible self-hosted compiler build whose ordinary path does not use Go.

If the experiment is abandoned, the native implementation and any bootstrap
bridge should remain removable. Reusable benchmark and conformance findings may
Expand Down
8 changes: 8 additions & 0 deletions docs/decisions/0001-experimental-native-toolchain.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

Status: accepted for the initial experiment

Self-hosting scope is amended by
[Decision 0002](0002-typerb-owned-self-hosting.md). This decision still defines
the repository and experimental compiler boundary.

## Context

The reference TypeRB compiler emits Go, Ruby, and TypeScript from one checked
Expand Down Expand Up @@ -78,6 +82,10 @@ Rejected because it duplicates a working parser, resolver, checker, diagnostics,
and tooling before answering the native runtime and backend feasibility
question.

This rejection concerns sequencing, not the final implementation boundary.
Decision 0002 requires a TypeRB-authored frontend after earlier native
execution and runtime gates have justified that investment.

### Publish the existing typed IR as a stable API

Rejected because it would freeze reference compiler implementation details and
Expand Down
Loading