RegisterFunc: hidden struct-return pointer undercounts stack arguments on amd64/loong64

[hajimehoshi]

PureGo Version

main (5f49e7c, around v0.11.0-alpha.6)

Operating System

• Windows
• macOS
• Linux
• FreeBSD
• NetBSD
• Android
• iOS

(This is an architecture-level ABI bug, not OS-specific: it affects the System V amd64 path — macOS, Linux, Android/iOS on amd64 — and loong64. It is reachable wherever ensureStructSupported allows struct returns on those arches. Windows/amd64 is not affected; see below.)

Go Version (go version)

go version go1.26.5 darwin/arm64 (not Go-version specific)

What steps will reproduce the problem?

On amd64 or loong64, register a Go function with maxArgs integer parameters (32 on these arches) that returns a struct larger than two eightbytes (returned in memory):

package main

import "github.com/ebitengine/purego"

// 24 bytes: returned in memory via a hidden pointer on amd64/loong64.
type Big struct{ A, B, C uint64 }

func main() {
	var fn func(
		uintptr, uintptr, uintptr, uintptr, uintptr, uintptr, uintptr, uintptr,
		uintptr, uintptr, uintptr, uintptr, uintptr, uintptr, uintptr, uintptr,
		uintptr, uintptr, uintptr, uintptr, uintptr, uintptr, uintptr, uintptr,
		uintptr, uintptr, uintptr, uintptr, uintptr, uintptr, uintptr, uintptr, // 32
	) Big
	// cfn is a dummy; the argument-limit check runs at registration time.
	purego.RegisterFunc(&fn, 1)
}

Run with GOARCH=amd64.

What is the expected result?

RegisterFunc should reject the registration with panic: purego: too many stack arguments. The struct is returned in memory, so its hidden pointer is prepended as the first integer argument; with 32 integer arguments already occupying every slot, the hidden pointer needs a 33rd slot that sysargs cannot hold.

What happens instead?

RegisterFunc accepts the registration. The preflight argument count does not reserve a slot for the hidden pointer once the integer registers are full, so the stack > sizeOfStack guard never fires. If such a function is then called, addStack writes at sysargs[numOfIntegerRegisters()+numStack] with an index equal to maxArgs, i.e. one past the end of the fixed-size [maxArgs]uintptr array — an out-of-bounds write.

Anything else you feel useful to add?

Root cause. In the preflight counting in RegisterFunc (func.go), the hidden struct-return pointer is counted with a bare ints++. Because the per-argument counting caps ints at numOfIntegerRegisters() and spills to stack beyond that, ints++ fails to model the spill: when the integer registers are already full, prepending the hidden pointer should push one regular argument onto the stack, but stack is left one too small.

Scope.

• Reachable on amd64 and loong64 (the arches where ensureStructSupported permits struct returns and the return pointer is passed as an ordinary integer argument).
• arm64 is not affected — it returns the pointer in the dedicated R8 indirect-result register, which is not counted against the argument registers.
• Windows/amd64 is not affected — its preflight guard uses the ints+floats+stack sum, which ints++ keeps correct.
• It is an extreme edge case in practice: it requires a function near the argument-slot limit and a >16-byte struct return.

History. The flaw is pre-existing on amd64 (the ints++ form predates the recent multi-arch struct work); #473 later routed additional arches through the same counting.

Fix. #476 replaces ints++ with a spill-aware count (if ints < numOfIntegerRegisters() { ints++ } else { stack++ }) and adds a regression test (TestABI_StructReturnHiddenPointer) that registers maxArgs integer arguments plus a large struct return and asserts the registration is rejected.

---

Filed by Claude (Claude Code), on behalf of @hajimehoshi.