Update vendored kernel to Tarver's S41.2 (2026-07-11 refresh)

[pyrex41]

Summary

Mark Tarver re-uploaded shenlanguage.org/Download/S41.2.zip on 2026-07-11 (Last-Modified; sha256 51becbfd60fa8c93c3f8ae5b20b948eaa84c4b1d14ad2f5d2a056002a53ee836) with a restructured kernel under the same "41.2" version number — a different lineage from the community ShenOSKernel-41.2 (shen-sources tag shen-41.2) shen-lua had been vendoring byte-identical. This PR rolls the refreshed kernel in. (version) still reports "41.2".

Full delta and dual-lineage provenance are documented in klambda/PROVENANCE.md.

Upstream delta (refresh vs community 41.2)

Kernel proper — 15 files, byte-identical to the zip's KLambda/: yacc core load prolog reader sequent sys t-star toplevel track types writer backend declarations macros

• New backend.kl — a cl.* KLambda→Common-Lisp backend. On the upstream boot list; pure defuns, never called under Lua.
• Removed compiler.kl — shen-cl's generated KL→Lisp compiler (a build artifact, never part of a ShenOSKernel release). shen-lua uses its own compiler.lua; never referenced at runtime.
• Removed dict.kl — property vectors are now plain vectors. *property-vector* is (vector 20000); get/put index it via hash + shen.change-pointer-value/shen.remove-pointer instead of shen.dict/shen.<-dict/shen.dict->. Internal to get/put; callers unaffected.
• Removed init.kl — folded into declarations.kl/toplevel.kl. There is no more shen.initialise function: the kernel initialises itself at load time via top-level forms in declarations.kl ((set *property-vector* (vector 20000)), the env sets, (shen.initialise-arity-table …), (put shen shen.external-symbols …), (shen.build-lambda-table …)). Renames: hush→shen.hush, input+→shen.input-h+, shen.initialise-lambda-forms→shen.initialise-lambda-tables, dropped shen.set-lambda-form-entry.
• Removed stlib.kl — upstream now ships the standard library as lazy Shen sources under Lib/StLib/, not as precompiled KLambda.

Standard library + extensions — retained byte-identical from community ShenOSKernel-41.2 (so the stdlib + CLI launcher stay available and certification holds): stlib.kl, extension-features, extension-expand-dynamic, extension-launcher (booted), extension-programmable-pattern-matching (opt-in, not booted). All pure defuns/defmacros referencing only public kernel functions, so they load unchanged.

What I did (runtime adaptations)

• boot.lua — new 19-module FILES boot order. It is not upstream make.shen order (that relies on the factorise pass + a macros bootstrap that runs last); shen-lua compiles KLambda directly, so order is driven by load-time side effects. Tail is … macros declarations t-star types: the refresh moved shen.rectify-type into t-star, and types.kl's 161 (declare …) forms run the type checker at load, so t-star must precede types, and both macros (for shen.prolog-vector) and declarations (for shen.*sigf* + the arity table) must precede types. initialise() no longer requires shen.initialise, but still calls it when present (older kernels via SHEN_KL_DIR keep working).
• lua_interop.lua — shen_register now routes through the kernel's own update-lambda-table (put arity + cons (shen.lambda-entry name) onto shen.*lambdatable*), since shen.set-lambda-form-entry is gone. The refresh's shen.lambda-entry stores the curried function, so the old tc+ "apply a non-function" hazard the previous code worked around no longer exists.
• typecheck_native.lua — the 161 native kernel signatures are now harvested from types.kl's (declare NAME TYPE) forms (the same rcons trees the existing evalrcons decodes); init.kl's shen.initialise-signedfuncs is gone. Falls back to the init.kl harvest for pre-refresh kernels. Native typecheck fast path stays active: 161/161 signatures harvested.
• repl.lua — "kernel loaded?" probe now checks version (was shen.initialise, which the refresh removed — would have forced a kernel reload on every REPL start).
• build/make-bundle.lua — expect 20 vendored .kl (was 21). Single-file bundle rebuilt and boots standalone.

Test status (LuaJIT 2.1)

Tier	Result
Port spec suite (make test)	471 / 0 across 12 specs
Canonical kernel certification (make certify)	134 / 134 (100%)
Single-file bundle	builds (5.3 MB) + boots standalone ((version) → 41.2)
Kernel bytecode cache + user fasl	validated (cached boot ~0.13 s; define+call works)

Remaining work / follow-ups (not blocking)

• Lazy Lib/StLib loader. The clean end state is to load the refresh's Shen-source standard library on demand instead of vendoring the community stlib.kl. Blocked on a source-loader; not attempted here.
• Pre-existing stdlib limitation (unchanged by this PR). stlib functions compile as direct calls and pass the suite, but their runtime arity property is -1 (because stlib.initialise is never called), so a bare (fn filter) / top-level (filter …) through the lambda table fails with "fn: filter is undefined". Verified identical on the pre-refresh kernel. The lazy loader above fixes this too.
• bin/shen fallback branch calls shen.initialise_environment (removed, and already a no-such-symbol name pre-refresh). Dead code — only reached if require("repl") fails, which it does not. Left as-is; flagged for a future cleanup.
• bin/ratatoskr-build.lua references shen.initialise in a comment describing its generated manifest's init step. The ratatoskr build path is out of scope here (separate tracked task) and will need its manifest updated for the load-time-init model.

Note for sibling Shen ports

The load-time-init restructuring is the headline for any port that tracks this kernel: there is no shen.initialise to call post-load — initialisation is top-level forms in declarations.kl, so a port's boot driver must stop calling shen.initialise and must load declarations after sys and before types, with macros + t-star also before types. Ports with a native/precomputed signature table should re-source it from types.kl's (declare …) forms. Ports that assumed a shen.dict property vector must accept a plain (vector 20000).

🤖 Generated with Claude Code