Conversation
Reference: core/api/LICENCE. Co-Authored-By: Cladius Maximus <cladius@lethean.io>
Aggregates several rounds of audit hardening from this session that landed incrementally in response to gaming patterns surfaced by lanes: - banned-imports: scope regex to import-statement context only (fixes false positives on string literals like args["path"], OpenAPI In: "path", MCP K-V keys; surfaced via Mantis #1322, #1324). Also catches both double-quote and backtick import styles in one regex (Round 7 dodge). - external-shim-dirs: new dimension. external/<X>-shim/ or external/<X>shim/ directories caught Round 6 gaming where lanes inline-reimplemented upstream packages with replace directives (go-devops #1231 r1). external/ must hold real git submodules, not local shims. - licence-missing: new dimension. Repo-root LICENCE file presence check (UK English EUPL-1.2 per CLAUDE.md). Reference: core/api/LICENCE. LICENSE / COPYING are non-canonical names — rename to bare LICENCE. - external/ added to all directory-walk exclusions (find + grep) so submodule contents are not double-counted as repo's own findings. Co-Authored-By: Cladius Maximus <cladius@lethean.io>
Snider 2026-05-01: "we have go.work + submodules setup for all repoes, replace will break things". Replace directives in go.mod override the go.work + external/<dep> submodule resolution mechanism, hiding incomplete dep cascade work behind a localfs override. The canonical local-build mechanism is go.work + external submodules. Replace directives are tech debt — they make a repo "build locally" by pinning at a specific snapshot but mask the real dep state from the workspace resolution. Detection counts: - single-line `^replace dappco.re/go/X => Y` directives - multi-line `^[[:space:]]+dappco.re/go/X => Y` block entries Smoke-tested on go-mlx + go-ml — both now flag 0 after `replace` removed. Co-Authored-By: Cladius Maximus <cladius@lethean.io>
Earlier `audit: harden + extend v0.9.0 audit script` (b48b896) added `external/` to audit.sh's EXCLUDE_DIRS but didn't propagate the same exclusion to the per-dimension Python helpers (ax7-gaps, example-gaps, file-presence, identical-triplets, test-stubs, unreferenced-symbols). This brings them all in sync — `external/` is a submodule directory (not the repo's own code), so per-dimension audits should skip it like the parent audit.sh does. Co-Authored-By: Cladius Maximus <cladius@lethean.io>
…pers (Mantis #1329) Adds 8 surfaces from the consumer-side gap analysis (php #1321, go-scm #1319, docs #1316, go-crypt #1311). Snider 2026-05-01: most reported "missing wrappers" already existed; only these are true gaps. Type aliases (matches existing OSFile / Cmd / FileMode shape): - core.RawMessage = json.RawMessage (json.go) — deferred-decode JSON fragment for HTTP / MCP / IPC envelope-then-payload handlers - core.Buffer = bytes.Buffer (io.go) — buffer-typed fields/locals without importing bytes; pairs with existing core.NewBuffer() - core.Builder = strings.Builder (string.go) — builder-typed fields/locals without importing strings; pairs with core.NewBuilder() New wrappers: - core.NewBufferReader([]byte) *bytes.Reader (io.go) — bytes flavour of existing core.NewReader(string); HTTP body construction - core.Index(s, sep string) int (string.go) — strings.Index - core.TrimCutset(s, cutset string) string (string.go) — strings.Trim 2-arg cutset variant; existing core.Trim(s) is TrimSpace-equivalent - core.TrimLeft(s, cutset string) string (string.go) — strings.TrimLeft - core.TrimRight(s, cutset string) string (string.go) — strings.TrimRight Each surface lands with: - Doc comment with usage example (AX-2) - Triplet test Good/Bad/Ugly (AX-7) - Runnable example in <file>_example_test.go 24 new triplets + 8 new examples, all passing under `go test -count=1`. Build + vet clean. Unblocks ~1500 findings across php / go-scm / docs / go-crypt for the next consumer-side conversion lane. Co-Authored-By: Cladius Maximus <cladius@lethean.io>
Snider 2026-05-01: build dirs / scan caches should never be tracked in git. Mantis #1293/#1333 root cause — `ui/node_modules.bak/` was reported (by Hepa) as tracked, generating false-positive Sonar findings. (Turned out to be ALREADY gitignored, but the rule stands.) Detection: `git ls-files` filtered for canonical artifact patterns — node_modules(.\w+)?/, .scannerwork/, __pycache__/, coverage/, htmlcov/, dist/, target/, .gitleaks/, .DS_Store, *.pyc, .coverage. Runs from repo root regardless of audit invocation cwd so subtree audits still see the whole repo. Conservative on patterns that have legitimate uses: `build/` (core/go-build package), `vendor/` (Go vendoring policy varies). Those are excluded — add per-repo if needed. Smoke tested: - gui: 0 findings (node_modules.bak/ properly gitignored) - api: 1 finding (php/src/Website/.DS_Store tracked — real catch) Co-Authored-By: Cladius Maximus <cladius@lethean.io>
Snider 2026-05-01: "the knowledge can be extracted into audit rules?"
First batch of docs → audit dimensions, with cross-link comments to
source doc pages. Mantis #1337 umbrella.
New dimensions:
- command-path-shape (from docs/commands.md "Command Paths" section)
Catches c.Command("X", ...) where X has leading/trailing/double slash
or empty string. Doc rule: "Paths must be clean: no empty path, no
leading slash, no trailing slash, no double slash."
- service-name-empty (from docs/services.md "Register a Service" section)
Catches c.Service("", ...) empty-string registration. Doc rule:
"Registration succeeds when the name is not empty."
- action-name-format (from docs/messaging.md "Named Actions" section)
Catches c.Action("X", ...) where X isn't dotted-lowercase like
"domain.verb". Doc convention: action names are domain-typed wire-
protocol identifiers (process.run, git.pull, repo.sync). Allows
hyphens + underscores within segments.
Smoke tests:
- agent: surfaces 4 real catches (noop, content_batch, greet) — genuine
non-canonical names that should be domain.verb shape
- go-i18n + go-process (canonical refs): 0 (clean)
Each dim's docstring cites its source doc page — supports Mantis #1337
Phase 3 (frontmatter cross-link in docs) + Phase 4 (training-data
extraction with doc-link rationale field).
Co-Authored-By: Cladius Maximus <cladius@lethean.io>
Hepa surfaced 2026-05-01 in BugSETI round. Old METHOD regex used lazy
match `[^)]*?\*?` followed by `[A-Z]` — would skip the lowercase prefix
of private types like `issueHeap` and capture the embedded uppercase
tail (`Heap`). Result: false-positive demand for ExampleHeap_{Push,Pop,
Len,Less,Swap} on a private type that consumers in `package <pkg>_test`
cannot legally reference.
Fix: anchor receiver name + require receiver type to START with
uppercase. Mirrors TOP_LEVEL regex discipline (which already requires
`^func ([A-Z]...)`). Private types are now correctly excluded from the
example-coverage requirement.
Smoke tested:
- agent (canonical reference): 0 public-symbols missing Example* (clean)
- BugSETI: 78 (down from prior count — false-positives on private
helpers like `issueHeap` no longer demanded)
Co-Authored-By: Cladius Maximus <cladius@lethean.io>
Hepa flagged this sister bug 2026-05-01 in BugSETI round 2. Same lazy match issue as example-gaps.py: `[^)]*?\*?[A-Z]` skips lowercase prefixes of private types like `issueHeap`, captures embedded uppercase tail, demands triplet-gap tests for symbols that can't be referenced from external `_test` package. Fix mirrors example-gaps eb6afa1 + TOP_LEVEL discipline: anchor receiver name + require uppercase first char on receiver type. Co-Authored-By: Cladius Maximus <cladius@lethean.io>
go-lint caches dependency snapshots under <repo>/go/.lintdeps/dappco.re/... for offline diagnostics. Those snapshots are not source code — they're verbatim copies of older releases of other repos and should not be audited as if the consumer repo wrote them. Adding --exclude-dir=.lintdeps fixes false-positive counts: ide 709 -> 29 (most violations were in cached gui v0.9.0 snapshot). Co-Authored-By: Cladius <noreply@anthropic.com>
…ceivers + exclude .lintdeps (Mantis #1336) The METHOD regex (`^func \([^)]*?\*?[A-Z]...`) lazy-matched past private receiver types like `stringBuilder`, capturing the embedded uppercase `Builder` and demanding sibling test files for the synthetic public type. Same fix already applied in example-gaps.py + ax7-gaps.py. Anchored receiver regex (`\(\s*\w+\s+\*?[A-Z]...`) requires the receiver TYPE itself to start with uppercase. Pure-private files like php/compat.go no longer get false-flagged. Also adds .lintdeps/ to EXCLUDED_DIRS for parity with audit.sh. Co-Authored-By: Cladius <noreply@anthropic.com>
Catches the (T, error) and (T, core.Result) two-return anti-pattern that the existing err-shape-funcs check misses. err-shape-funcs only matches `func ... error)` single-return signatures; tuple variants slip past. Discovery context: 2026-05-11 mid-Mantis #1393 (go-store collapse to canonical Result). go-store passed the existing audit cleanly modulo docs/licence (7 findings) but had 6 known constructors using `(*T, core.Result)` — the worst-of-both-worlds shape (Result wrapper doing OK-check duty but Value always nil for happy path; the actual constructed value lives in the first return slot). Adding the check surfaced the real surface: 93 hits in go-store alone. Not just constructors — getters, parsing helpers, Medium interface methods, internal helpers. Every one is a place where: - caller can `_`-discard the Result and proceed with garbage value - panic in function body crashes instead of becoming Fail - reads as inconsistent API (some calls return Result, others tuple) Each hit is a hidden panic surface. Same anti-pattern surface as (*T, error) per Mantis #1389 — collapses to single-return core.Result with value carried in r.Value. The check excludes test files (TestX functions don't return tuples) and accepts a small floor for stdlib-interface impls per the same logic as err-shape-funcs. Mantis #1393 (sister concern surfaced via this check). Co-Authored-By: Virgil <virgil@lethean.io>
… slots
Original regex required identifier-shaped first return slot
(\*?[A-Za-z_][^,]*) which missed slice ([]T), map (map[K]V), and
3+ return tuples (T1, T2, error). Surfaced during go-store Wave 2
mapping — real count 112 not 87.
New regex: ^func .* \([^)]+, ?(error|core\.Result)\) \{$
Catches all functions where the LAST return slot is error or
core.Result with at least one preceding slot — regardless of first
slot's type.
Cross-workspace recount with new regex needed. The earlier ~1900
total was undercount.
Co-Authored-By: Virgil <virgil@lethean.io>
Adds dimensions for session-discovered patterns the audit was missing: - legacy-log-package — catches `"dappco.re/go/log"` import regression. Today's go-ml `coreerr.E → core.E` sweep removed 237 instances; nothing prevented a regression until this dim. Smoke: go-ml=30, go-ai=2. - service-usage-example — files declaring `NewService(` must include the literal `// Usage example:` marker. Mantis #1383 convention test was per-package; this makes the rule ecosystem-wide. Smoke: go-ml=1. - service-canonical-shape — packages with `NewService(` must also declare `Register(c *core.Core) core.Result` or `RegisterCore` (name divergence allowed where Register collides with existing registration funcs, e.g. inference.Register(Backend)). Mantis #1336. Smoke: go-ml=1. - error-wrap-antipattern — catches `core.NewError(x.Error())` two-stage wrapping that loses the original Result chain. Spotted today in BookState demo code. Smoke: go-ai=20, go-ml=4. Together: 58 additional findings across go-ai + go-ml that the prior audit hid. go-inference + go-mlx + core/go clean on all four. Initial regex caught NewServiceRuntime false positive in core/go; tightened to `^func NewService(` (open paren) to exclude generic ServiceRuntime helpers. Co-Authored-By: Virgil <virgil@lethean.io>
…t/etc Adds the missing net/http surfaces that consumers couldn't reach without re-importing net/http directly. Surfaced by the AX phase 5 sweep on lthn/desktop where ~120 status-code sites + ~50 method-constant sites were retaining "net/http" only for these RFC-standard values. Constants (re-exported from net/http): MethodGet/Head/Post/Put/Patch/Delete/Connect/Options/Trace StatusOK/Created/Accepted/NoContent StatusBadRequest/Unauthorized/Forbidden/NotFound/MethodNotAllowed StatusConflict/Gone/UnprocessableEntity/TooManyRequests StatusInternalServerError/NotImplemented/BadGateway/ServiceUnavailable StatusGatewayTimeout Type aliases: Flusher = http.Flusher (SSE / chunked-transfer) HTTPFileSystem = http.FileSystem (HTTPFileServer parameter) Vars: DefaultHTTPClient = http.DefaultClient ErrHTTPServerClosed = http.ErrServerClosed (graceful-shutdown sentinel) Funcs: NewServeMux() → *ServeMux HTTPStripPrefix(prefix, h Handler) Handler HTTPListenAndServe(addr string, h Handler) error HTTPFileServer(root HTTPFileSystem) Handler HTTPFS(fsys FS) HTTPFileSystem HTTPError(w ResponseWriter, msg string, code int) All ship with paired Examples in api_example_test.go per the AX-2 + discovery rule (agents skim *_example_test.go to learn the surface without loading the full source). 12 new Examples, all pass. Naming convention: bare names where the term is unambiguous (Method*, Status*, Flusher, NewServeMux), HTTP-prefixed where ambiguous or where the function is HTTP-specific in a way the bare name wouldn't convey (HTTPStripPrefix, HTTPListenAndServe, HTTPFileServer, HTTPFS, HTTPError, DefaultHTTPClient, ErrHTTPServerClosed, HTTPFileSystem). Mirrors the existing HTTPServer/HTTPClient/HTTPGet/HTTPPost convention. Co-Authored-By: Virgil <virgil@lethean.io>
Adds the missing time.* surfaces that consumers couldn't reach without
re-importing time directly. Surfaced by the AX phase 2 sweep on
lthn/desktop where 9 files retained "time" only for these primitives.
After(d Duration) <-chan Time
select-case timeout shape, used in pkg/bridge/tools.go,
pkg/queue/{queue,schedule}_test.go, pkg/opencode/subscribe{,_test}.go
type Ticker = time.Ticker
NewTicker(d Duration) *Ticker
periodic poll loops, used in pkg/queue/worker.go
Unix(sec, nsec int64) Time
two-arg constructor matching stdlib (UnixTime is sec-only shorthand)
used in pkg/bridge/layout{_example,}_test.go fixtures
UnixMilli(ms int64) Time
millisecond-resolution constructor for parsing JSON timestamps,
used in pkg/opencode/import_host.go
All ship with paired Examples per the AX-2 + agent-discovery rule.
6 new Examples, all pass.
After this lands + lthn/desktop submodule bumps to pick it up, the 9
files retaining "time" import can drop it in a follow-up sweep.
Co-Authored-By: Virgil <virgil@lethean.io>
Adds the missing primitives surfaced by AX phase 6 sweep on
lthn/desktop:
string.go LastIndex(s, substr) int
— strings.LastIndex; pairs with Index for delimiter
work (used in opencode/reconcile.go for hostSide
colon parsing)
slice.go SliceSortFunc[T any](s []T, less func(a, b T) bool)
— sort.Slice with comparator for struct-by-field
ordering (used in php.go for sorting by Path)
io.go LimitReader(r Reader, n int64) Reader
— io.LimitReader for bounding HTTP body reads
(used in bridge/tools.go, validator/validator.go,
plugin/install.go for safe response capping)
random.go RandRead(b []byte) Result
— crypto/rand.Read with Result shape; fills an
existing buffer rather than allocate via RandomBytes
(used in apikey/apikey.go, opencode/auth.go,
opencode/import_host.go for ephemeral nonces)
All ship with paired Examples per the AX-2 + agent-discovery rule.
4 new Examples, all pass.
Naming follows existing convention: bare names where the term is
unambiguous (LastIndex, SliceSortFunc, LimitReader); RandRead
prefixed for symmetry with RandomBytes/RandomString/RandomInt
(Rand-prefix family rather than Read- prefix, since Read is
generic).
Co-Authored-By: Virgil <virgil@lethean.io>
…rative exit
Closes the c.Go(fn) and c.IsShutdown() export gaps surfaced by lthn/
desktop's pkg/queue worker + 4 other long-running goroutine sites
(plugin supervisor, opencode SSE subscribe, bridge service, plugin
runtime watcher). All 5 currently spawn raw goroutines because the
Core wrappers weren't exposed.
Go(fn func()) — spawns through c.waitGroup; ServiceShutdown waits
for every tracked goroutine to drain before stopping
services, so workers/watchers/reconnect loops aren't
orphaned across a clean shutdown
IsShutdown() bool — polled between work units inside long-running
goroutines so they exit cooperatively before the
waitGroup drain blocks
Both ship with paired Examples per the AX-2 + agent-discovery rule.
After this lands, lthn/desktop's plain `go runWorker(c, interval)` /
`go s.runSubscription(...)` / `go func() {...}()` sites convert to
`c.Go(...)` and pick up the lifecycle property for free.
Co-Authored-By: Virgil <virgil@lethean.io>
Per RFC-CORE-008 principle 11 (Benchmarks as Hot-Path Validation), every primitive on a known consumer hot path needs a benchmark to gate its performance contract. core/go previously had ZERO real benchmarks — string helpers in particular are hammered by every consumer (tokenisers, config keys, CLI dispatch, URL building, log formatting) so regressions cascade across the ecosystem invisibly. Bench coverage added for: Concat / Join (2/4/8 parts) — multi-arg builders Lower / Upper (already-cased + mixed) — case conversion Trim / TrimPrefix / TrimSuffix — boundary trimming HasPrefix / HasSuffix / Contains — predicates Split / SplitN / Index / LastIndex — separator ops Replace (no-op + hit) — substring replacement RuneCount / HTMLEscape (no-specials + with-specials) Baseline on M3 Ultra (key findings — see follow-up commits): Join_TwoParts 41.18 ns/op 24 B/op 2 allocs/op Join_FourParts 180.8 ns/op 88 B/op 5 allocs/op Join_EightParts 694.0 ns/op 336 B/op 11 allocs/op Concat_TwoParts 28.42 ns/op 32 B/op 2 allocs/op Concat_FourParts 49.73 ns/op 56 B/op 3 allocs/op The Join numbers reveal the O(N²) Concat cascade — alloc count and wall-time grow super-linearly with part count. Follow-up commits target this and other findings. Co-Authored-By: Virgil <virgil@lethean.io>
…zed Builder Join previously chained Concat(result, sep, p) per pair, creating a new Builder + result string per iteration. For N parts that produced roughly 2(N-1) allocations and O(N²) bytes-copied — visible in the new AX-11 bench harness as the alloc count growing linearly with phrase length and time growing super-linearly. Concat had the same shape at smaller scale (no Grow on the Builder so the internal buffer doubled 1-3 times per call). Both now pre-compute the exact final length and call Builder.Grow(n) up front. WriteString then writes into the pre-sized buffer without any reallocation. Public API is unchanged — same signature, same semantics, same test outcomes. Bench deltas: Join_TwoParts 41.18 → 20.21 ns/op 2 → 1 allocs Join_FourParts 180.8 → 34.16 ns/op 5 → 1 allocs Join_EightParts 694.0 → 58.05 ns/op 11 → 1 allocs (12x faster) Concat_TwoParts 28.42 → 18.06 ns/op 2 → 1 allocs Concat_FourParts 49.73 → 25.49 ns/op 3 → 1 allocs Every multi-arg case now hits exactly 1 allocation regardless of part count — the only remaining alloc is the final result string itself. This cascades through every consumer that uses core.Join or core.Concat on a hot path (config keys, URL building, CLI dispatch, log formatting, go-i18n's matchWordPhrase phrase joining, etc.). Downstream regression gates can now lean on the bench numbers. Co-Authored-By: Virgil <virgil@lethean.io>
Covers the hot slice operations: Contains/Index, Clone, Map/Filter/ Reduce, Sort/SortFunc, Uniq, Any/All, Take/Drop, FlatMap. Provides baseline numbers and the regression gate for future changes. Baseline (M3 Ultra) shows the largest remaining levers in this file: SliceSort_Medium 3344 ns/op 2 allocs (sort.Slice + reflect) SliceSortFunc_Medium 2536 ns/op 2 allocs (sort.Slice + reflect) SliceFlatMap 85.9 ns/op 4 allocs (unsized append growth) Predicates and 0-alloc paths (Contains/Index/Any/All/Take) already hit the floor. Co-Authored-By: Virgil <virgil@lethean.io>
sort.Slice uses interface-based reflection that allocates closures and reflect.Value wrappers internally. slices.Sort and slices.SortFunc (stdlib since Go 1.21) are generic, dispatch directly, and avoid both the allocations and the reflect overhead. SliceSort delegates to slices.Sort (the natural fit for Ordered). SliceSortFunc wraps the bool-less API in slices.SortFunc's int comparator — public surface unchanged. Bench deltas: SliceSort_Medium 3344 → 897.5 ns 3.7x faster 2 → 0 allocs SliceSortFunc_Medium 2536 → 2677 ns +5% wall 2 → 0 allocs SortFunc loses 5% wall-time to the bool→int wrapper but gains -2 allocs. Net positive on hot loops where GC pressure dominates. Drops the now-unused "sort" import. Co-Authored-By: Virgil <virgil@lethean.io>
Coverage: Marshal/MarshalIndent/MarshalString and Unmarshal/ UnmarshalString across small (2-field), medium (9-field nested), and large (8-item collection) payload shapes, plus unmarshal-into-map and unmarshal-into-RawMessage variants. Baseline (M3 Ultra) flags the API-edge overhead: JSONMarshal_Small 109.8 ns 96 B 3 allocs JSONMarshal_Medium 871.9 ns 897 B 17 allocs JSONMarshal_Large 7,455 ns 7051 B 129 allocs JSONMarshalString_Medium 875.5 ns 1193 B 17 allocs JSONUnmarshal_Small 399.1 ns 256 B 6 allocs JSONUnmarshal_Medium 3,146 ns 1360 B 38 allocs JSONUnmarshalString_Small 412.7 ns 304 B 7 allocs ← +1 alloc, +48 B JSONUnmarshalString_Medium 3,161 ns 1680 B 39 allocs ← +1 alloc, +320 B JSONUnmarshal_IntoRawMessage 1,580 ns 512 B 5 allocs The String variants pay an extra allocation for the []byte(s) conversion — that's the next lever. Co-Authored-By: Virgil <virgil@lethean.io>
Coverage: constructors (NewBuffer/NewBufferString/NewBufferReader/ NewReader), Copy/CopyN/WriteString, ReadAll across 1KB/64KB/1MB sizes, LimitReader construction + bounded read. Baseline (M3 Ultra) flags the ReadAll buffer-doubling cost — 3x byte overhead at every size from io.ReadAll's growth path: ReadAll_1KB 543.6 ns 3264 B 7 allocs (3.2x input) ReadAll_64KB 24,599 ns 203712 B 19 allocs (3.1x input) ReadAll_1MB 273,162 ns 3276616 B 27 allocs (3.1x input) Other paths are already at the floor: Copy_1KB 39.14 ns 56 B 2 allocs Copy_64KB 717.9 ns 56 B 2 allocs (constant) WriteString_Short 6.047 ns 0 B 0 allocs NewBuffer / NewReader 0.28 ns 0 B 0 allocs LimitReader_Construct 0.28 ns 0 B 0 allocs ReadAll is the next lever — pre-allocating when the Reader knows its size (bytes.Reader, bytes.Buffer, strings.Reader all expose Len()) should drop the 3x overhead to ~1.1x for those common cases. Co-Authored-By: Virgil <virgil@lethean.io>
io.ReadAll grows its internal buffer through 5-10 doublings, costing
roughly 3x the final byte count in transient allocations. Many real
inputs already know their length — bytes.Reader, bytes.Buffer, and
strings.Reader all expose Len() — so we can allocate the destination
once at the exact size.
Type-switch on a structural `interface{ Len() int }`; if the reader
implements it, allocate make([]byte, n) and read into it in one pass.
Otherwise fall through to io.ReadAll as before. Public signature
unchanged; behaviour for unsized readers (HTTP bodies, network
streams, *io.PipeReader) is preserved.
Bench deltas:
ReadAll_1KB 543.6 → 371.9 ns 7 → 4 allocs 3264 → 2112 B (-32%)
ReadAll_64KB 24,599 → 10,207 ns 19 → 4 allocs 204KB → 131KB (-59%)
ReadAll_1MB 273,162 → 112,352 ns 27 → 4 allocs 3.3MB → 2.1MB (-59%)
At 1MB we save 1.2MB of transient allocations and run 2.4x faster.
The 4-alloc floor: pre-sized []byte + string(data) copy + Result
struct + close (when applicable).
LimitReader still falls through to io.ReadAll because *io.LimitedReader
doesn't expose Len() — correct.
Co-Authored-By: Virgil <virgil@lethean.io>
Per the principle that every primitive needs a perf contract, api.go
now joins string/slice/json/io with full bench coverage. 16/23 exported
functions benched (skipped: HTTPFileServer + HTTPFS need fixture FS,
HTTPListenAndServe blocks, NewHTTPTestTLSServer is TLS-heavy).
Baseline (M3 Ultra):
Constructors / lookups (already at the floor):
NewServeMux 0.29 ns 0 allocs
HTTPStripPrefix 0.28 ns 0 allocs
HTTPStatusText_200 1.28 ns 0 allocs
HTTPStatusText_404/500 1.85 ns 0 allocs
NewHTTPTestRecorder 9.25 ns 0 allocs
API_RegisterProtocol 22.37 ns 0 allocs
Request construction:
NewHTTPRequest_GET 231 ns 3 allocs 512 B
NewHTTPRequest_POSTBody 282 ns 6 allocs 624 B
NewHTTPRequestContext 234 ns 3 allocs 512 B
NewHTTPTestRequest 961 ns 9 allocs 5115 B
HTTPError 472 ns 11 allocs 1058 B
API_Protocols 25 ns 1 alloc 48 B
Round-trips against httptest.Server:
HTTPGet 35,481 ns 65 allocs 5676 B
HTTPPost 37,938 ns 82 allocs 7691 B
HTTPPostForm 39,225 ns 85 allocs 7742 B
Most of the wrapper surface delegates straight to net/http; the round-
trip numbers reflect stdlib's own work (header serialisation, TCP loop-
back, response parsing). The bench harness IS the contribution — it
gates these against regression and lets us see if a future stdlib bump
or wrapper change shifts the numbers.
Co-Authored-By: Virgil <virgil@lethean.io>
strings.ToLower and strings.ToUpper walk the Unicode case table for every byte regardless of input — even when the input is already in the target case and the answer is "no change". For ASCII-only inputs already in the target case (the dominant shape for English text: tokens, model names, identifiers, keys), the function does substantial work to return the input unchanged. Add a single-byte-scan fast path: walk s and bail to strings.ToLower / ToUpper at the first byte that needs work (case-flippable ASCII OR non-ASCII byte). If the scan completes without finding such a byte, the input IS the result — return it with zero allocations. The simple-loop scan is ~0.2ns/byte (tight L1-resident byte compare) vs strings.ToLower's Unicode-aware walk at ~1ns/byte. For Mixed inputs we still pay the scan, but the Unicode path runs only when truly needed; the Builder-based ASCII variant we tried lost to it. Bench deltas: Lower_AlreadyLower 37.91 → 17.68 ns 2.14x (0 allocs) Upper_AlreadyUpper 18.13 → 8.75 ns 2.07x (0 allocs) Lower_Mixed 90.01 → 89.26 ns ~same (1 alloc) Upper_Mixed 54.34 → 55.38 ns ~same (1 alloc) Cascades to every consumer doing case-normalisation on ASCII tokens: go-i18n's pre-lower loop, go-mlx prompt formatting, config key normalisation, URL scheme matching, etc. Co-Authored-By: Virgil <virgil@lethean.io>
Co-Authored-By: Virgil <virgil@lethean.io>
Two related cuts to the ReadAll alloc floor, both flagged by codex against go-mlx's state/file payload paths. 1. Skip the string(data) copy via unsafe.String. The buffer returned by readAllSized is freshly-allocated and unreachable after the conversion — exactly the safety contract strings.Builder.String() relies on internally. Adding a small bytesToString helper makes the intent explicit at the call site. 2. Extend the size-hinted fast path to *io.LimitedReader. The wrapper doesn't implement Len() but exposes .N (max-remaining bytes); if the underlying reader exposes Len(), use min(N, Len()), otherwise just N. Covers core.LimitReader(bytes.Reader, n) and the same shape across HTTP body bounding (the go-mlx hot path). Bench deltas (M3 Ultra): ReadAll_1KB 371.9 → 173.8 ns 4 → 3 allocs 2112 → 1088 B (2.1x) ReadAll_64KB 10,207 → 4,830 ns 4 → 3 allocs 131KB → 66KB (2.1x) ReadAll_1MB 112,352 → 58,132 ns 4 → 3 allocs 2.1MB → 1.0MB (1.9x) LimitReader_Bounded 1465 → 202.8 ns 8 → 4 allocs 3288 → 1112 B (7.2x) Cumulative from original (pre-AX-11) baseline: ReadAll_1KB 543.6 → 173.8 ns 3264 → 1088 B 7 → 3 allocs (3.1x) ReadAll_64KB 24,599 → 4,830 ns 204KB → 66KB 19 → 3 allocs (5.1x) ReadAll_1MB 273,162 → 58,132 ns 3.3MB → 1.0MB 27 → 3 allocs (4.7x) Co-Authored-By: Virgil <virgil@lethean.io>
For up to 16 elements, a linear scan on the growing output (O(N²) compares) beats the map allocation + hashing cost of the general path. Tokeniser / vocab / config-key dedupe hot loops sit in this size band. The general map-based path stays for len > 16. Bench delta: SliceUniq_NoDups 156.8 → 60.1 ns 2.6x faster (1 alloc, unchanged) Note: SliceUniq_AllDups appears to go from "0 allocs" to "1 alloc" but that's a measurement artifact — the previous "0 allocs" was the compiler dead-code-eliminating the discarded result. The new linear path reads from `out` inside the loop, defeating DCE, which exposes the legitimate output-slice allocation that was always there. Real- world consumers (which use the result) always paid the alloc. Co-Authored-By: Virgil <virgil@lethean.io>
…apWithSize/MakeFunc, Clone, Array.Clear Co-Authored-By: Virgil <virgil@lethean.io>
Co-Authored-By: Virgil <virgil@lethean.io>
…meout Co-Authored-By: Virgil <virgil@lethean.io>
Co-Authored-By: Virgil <virgil@lethean.io>
…wrappers Co-Authored-By: Virgil <virgil@lethean.io>
… *T) Co-Authored-By: Virgil <virgil@lethean.io>
…GeneratePack codegen Co-Authored-By: Virgil <virgil@lethean.io>
Co-Authored-By: Virgil <virgil@lethean.io>
Co-Authored-By: Virgil <virgil@lethean.io>
…rotocol; covers extractScheme) Co-Authored-By: Virgil <virgil@lethean.io>
…rRun ceilings)
21 hot primitives (math/string/slice/unsafe/Result) gated at their measured
alloc floor — the suite now FAILS if any starts allocating where it didn't.
0 = pure/zero-copy; 1 = the inherent Result{Value:non-pointer} boxing tax.
Co-Authored-By: Virgil <virgil@lethean.io>
… SliceClone (33 total) Co-Authored-By: Virgil <virgil@lethean.io>
lspNamingDiagnostic recompiled 3 constant regexes on every call; LSP runs diagnostics on every document change, so regexp.compile was ~67% of LSPComputeDiagnostics's allocations. Hoisted to package-level vars. LSPComputeDiagnostics: 186->47 allocs/op, 23922->8475 B/op, 12162->6061 ns/op. Behaviour byte-identical (LSP tests green). Co-Authored-By: Virgil <virgil@lethean.io>
…ex-hoist win) Co-Authored-By: Virgil <virgil@lethean.io>
dispatch + document handlers (didOpen/didChange/didClose), readMessage/ writeMessage, respond/notify, publishDiagnostics, and LSPServe (cancelled-ctx entry). Converted to package core so the unexported server path is reachable. Co-Authored-By: Virgil <virgil@lethean.io>
…istenAndServe safeName (Action/Task), Fs.path, assertFail/assertMsg (via a capturing TB stub); Println redirected to /dev/null; HTTPListenAndServe on its error path. Co-Authored-By: Virgil <virgil@lethean.io>
…per CLI run) FilterArgs ran on every Cli.Run with an unpresized 'var clean []string' + append — geometric regrow. Pre-sized to len(args) (almost all args survive the filter); returns nil when empty to stay byte-identical. 1 alloc, no regrow. Co-Authored-By: Virgil <virgil@lethean.io>
…istry.List Same unpresized-append pattern as FilterArgs: each iterates a known-length collection (dir entries / handlers / registry order) but grew via append. Pre-sized to the source length; return nil when empty to stay byte-identical. Co-Authored-By: Virgil <virgil@lethean.io>
…Stoppables, SliceFlatMap) Same unpresized-append pattern: presize to the source count (featureFlags.Len, services.Len) or a 1:1 heuristic (FlatMap); return nil when empty for byte-identical behaviour. Co-Authored-By: Virgil <virgil@lethean.io>
Co-Authored-By: Virgil <virgil@lethean.io>
A concurrency-safe object pool that recycles expensive-to-build values (scratch buffers, device handles) across calls without per-call alloc. Get pops a recycled value (or the zero T when empty); Put returns one. Unlike sync.Pool it never drops entries on GC — a warm pool stays warm, the right trade when the pooled value owns a resource the caller Closes explicitly. Collapses the hand-rolled per-type scratch pools in consumers (engine/metal's 18 kernel pools) onto one primitive. Co-Authored-By: Virgil <virgil@lethean.io>
…rimitives
The tiny helpers every consumer reinvents, homed once in core so an
optimisation/fix lands for all 80+ CoreGo repos:
- Coalesce[T comparable](vals...) — first non-zero (the config→env→default
fallback chain); replaces the firstNonEmpty copies.
- FirstPositive[T Ordered](vals...) — first value > 0 (0 = unset, try next);
replaces the firstPositive/firstPositiveFloat copies.
- SliceEqual[T comparable](a, b) — length+element equality over stdlib
slices.Equal; replaces sameIntSlice/bytesEqual/int32SlicesEqual.
- Result.Err() error — the Result→error bridge; replaces the hand-rolled
'if !r.OK { return r.Value.(error) }' unwrap.
15 tests (Good/Bad/Ugly each), full suite 3540 green.
Co-Authored-By: Virgil <virgil@lethean.io>
Coalesce's trim-aware sibling: first value whose trimmed form is non-empty (a " " falls through), returning the original untrimmed value. Homes the firstNonEmpty copies that trim before testing (which plain Coalesce can't). Co-Authored-By: Virgil <virgil@lethean.io>
Returns the string at key, or "" when absent or non-string — the decoded JSON/metadata-row accessor, replacing the hand-rolled strVal copies. Co-Authored-By: Virgil <virgil@lethean.io>
…gistry, Feature accessor Repairs an incomplete error->Result migration that left core dev unbuildable: - embed.go getAllFiles/Extract: PathWalkDir now returns Result (was treated as error); WalkDir stays error. - lock.go/core.go/contract.go: Core.locks restored as *Registry[*Lock] (was botched to a single *Lock), matching lock.go's named-mutex API and registry.go's own GetOrSet example. - core.go: restore the Core.Feature(name) accessor (type + 42 refs + docs all expected it). LIBRARY builds clean. NOTE: the test suite still has further incomplete migration (RegistryOf, api_test r.OK, ...) — a separate finish-the-refactor pass. Co-Authored-By: Virgil <virgil@lethean.io>
…amples
Repairs core dev, left non-building + failing by an unfinished agent
migration (it claimed done; it was not). Library + full test suite now
green (3778 tests).
Migrations finished (function now returns Result, docs updated — the
tests were already written for the Result API):
- HTTPListenAndServe: error -> Result
- Core.RegistryOf: *Registry[any] -> Result (Value carries the proxy;
unknown name => OK false)
- Fs.Exists / IsFile / IsDir: bool -> Result{OK: predicate}
- Fs.TempDir: string -> Result{Value: dir} (error scoped to fs.TempDir)
- WalkDir: error -> Result; embed getAllFiles/Extract consume it
Restored (accidentally removed by the migration):
- Core.locks *Registry[*Lock] (was botched to *Lock)
- Core.Feature(name) accessor
- Config made variadic: c.Config("group") == c.Config().Group("group")
Stale examples updated to the intended new API:
- New() no longer auto-registers cli (WithCli() opt-in) — example +
Services outputs corrected.
Co-Authored-By: Virgil <virgil@lethean.io>
Brings github core dev (106 commits past v0.10.4) with the incomplete error->Result migration finished (see fix commit), merged with the dedup primitives (Pool/Coalesce/FirstPositive/SliceEqual/FirstNonBlank/ MapString/Result.Err). Full suite green. Co-Authored-By: Virgil <virgil@lethean.io>
Contributor
|
Important Review skippedToo many files! This PR contains 209 files, which is 59 over the limit of 150. To get a review, narrow the scope: Upgrade to a paid plan to raise the limit. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (209)
You can disable this status message by setting the Use the checkbox below for a quick retry:
Warning Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces comprehensive benchmarking and allocation regression tests for the Action/Task primitives, and adds example and regression tests for action state methods. It also adds the EUPL-1.2 license to the repository. The main themes are improved test coverage (including allocation ceilings), enhanced documentation through examples, and formalizing licensing.
Testing and Benchmarking Improvements:
action_bench_test.gowith a suite of benchmarks forActionandTaskprimitives, covering registration, lookup, execution, state predicates, and related core/task operations. This ensures performance regressions are caught early and provides a baseline for future optimization.alloc_gate_test.go, which asserts hard ceilings on allocations-per-call for hot path primitives usingtesting.AllocsPerRun. This suite ensures that allocation regressions are detected and must be justified in review.Example and Regression Tests:
Action.Enable,Action.Disable, andAction.Enabledinaction_example_test.go, improving documentation and usage clarity for these methods.TestAction_Core_Progress_Badtest to assert that reporting progress on a non-existent task is a safe no-op and does not panic, strengthening regression coverage.Licensing:
EUPL-1.2) to theLICENCEfile, and updated new test files with the SPDX identifier for compliance.These changes significantly improve the reliability, maintainability, and legal clarity of the codebase.