Repository navigation
processgroup: macOS reads process groups, so left-running services and the job bound work there - #2039
Merged
Conversation
AbirAbbas
added a commit
that referenced
this pull request
Oct 29, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AbirAbbas
force-pushed
the
feat/darwin-process-inspection
branch
from
October 29, 2026 23:17
dfd4641 to
3406119
Compare
Collaborator
Author
|
Closing unmerged: the Mac push stopped early to save usage. What this fixes and what is left is tracked in #2055. The branch |
Members lists a group through kern.proc.pgrp with a bounded retry, and Group.Usage walks the recorded leader's descendants through the kernel's proc_info interface, converting task time with hw.tbfrequency. The new darwinproc package owns the raw trap, the SDK layouts and the argv and listening-socket readers the executor uses next. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cmdline reads kern.procargs2 and the ports come from each member's TCP listeners. describe reads each member's sockets once for both the naming member and the port set. The two service tests run on Linux and macOS, and the listening-service test polls the record until the backgrounded shell has exec'd its server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fixes: - A queued parent is re-read after its children are listed and its start time compared, so a pid handed to another process cannot bring that process's children into the job's reading. - A child listing that fills the largest buffer, and a walk that reaches 1024 processes, count as lower bounds; any unexpected listing failure makes the reading unknown instead of partial. - kern.procargs2 is parsed at the kernel's eight-byte boundary after the executable path, so an empty argv[0] can never shift an environment string into argv (checked for path lengths 1-40 on arm64 and under Rosetta). - The listening-service test's shell now waits for the server's port before it exits, so the record the call returns is asserted directly; the membersOf helpers it no longer needs are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fix: a reading that left part of the tree out (the walk limit, a truncated child listing, a process that could not be read) still counts processes as a lower bound, but its CPU sum is marked partial. The job bound takes no rate from it and does not use it as the next baseline, so a process missing from one reading and present in the next cannot read as a burst of its whole lifetime's CPU and invent a strike. Linux readings are always complete, so nothing changes there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fix: when a queued parent read once but its re-read after the child listing fails for a reason other than having exited, its children are left out of this reading only, so the CPU sum is marked partial and no rate is taken from it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fix: the macOS walk now reads a parent's listed children first and re-reads the parent after them. A pid is never handed on while its process lives, so a parent with the same start time after its children were read was their parent while they were read; a parent and child whose pids both changed hands just after the parent's own re-read can no longer bring a stranger's busy child into the job's reading. The partial-sum regression now starts from one complete, quiet reading, so it also fails if a partial reading ever keeps that earlier baseline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AbirAbbas
force-pushed
the
feat/darwin-process-inspection
branch
from
November 2, 2026 23:32
096e996 to
0cf7152
Compare
AbirAbbas
marked this pull request as ready for review
November 3, 2026 00:33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found in the macOS run of the engine suite (Apple M4 Pro, macOS 15.6):
TestAShellCallTellsTheObserverWhatItLeftRunningandTestLocalRecordsAListeningServiceFullyfail at app/v312dadd64abecause darwin had no process-group inspection at all.What was wrong (macOS)
processgroup.Membersreturned nil (members_other.go),Group.Usagereturned "cannot say" (usage_other.go), and the executor'scmdline/listenPortsreturned nil (service_other.go).executor.Leftsaw no members), a service record was named after the shell with no ports, and inventory refreshes had no ports.internal/session/jobbound.go: more than 128 processes, or more than three fifths of the machine's cores where nothing caps them, for 15 readings 2 s apart). A runaway or fork-storming bash job ran unbounded on macOS.Reproduced in the real engine on this Mac (
codeaf appspawned as the desktop spawns it, fake provider issuing onebashcall):12dadd64a(shipped sidecar)CGO_ENABLED=0)python3 -m http.server 0 --bind 127.0.0.1 &left runninginventory.jsonhas noservicesand norunningservices: [{name: python3, version: Python 3.10.19}],running: [{command: "python3 -m http.server 0 --bind 127.0.0.1", ports: [63451]}]— the port lsof showsfor i in $(seq 140); do sleep 90 & done; waitThe same checks in the real desktop app (UI2_CODEAF_BIN pointing at the branch sidecar) are queued behind the machine's performance baseline; results will be added here.
What changes
darwin readers, no cgo (new
internal/darwinproc, darwin-only files):kern.proc.pgrpthroughunix.SysctlRawwith a bounded retry (x/sys'sSysctlKinfoProcSliceretries forever while the table grows); only the pid, state and pgid fields are read from each record; zombies excluded; sorted.proc_listpids(PROC_PPID_ONLY)+proc_pidinfo(PROC_PIDTASKALLINFO)), so a child that changed process groups is still the job's, withowned()checked before and after.kern.procargs2, parsed at the kernel's eight-byte boundary after the executable path, stopping before the environment.PROC_PIDLISTFDS+PROC_PIDFDSOCKETINFO, IPv4/IPv6 TCP in LISTEN only.Why the raw
proc_infotrap and not a//go:cgo_import_dynamiclibproc bridge. Both work without cgo. The trap (unix.Syscall6(unix.SYS_PROC_INFO, …), call 1 listpids, 2 pidinfo, 3 pidfdinfo) goes through Go's own, maintained per-architecture syscall stubs (arm64x16+svc 0x80; amd64 adds the0x2000000class), so there is no assembly to write or test per architecture and no//go:linknameintosyscallfor garble or the linkname rules to break. It also returns errno directly; libproc's wrappers return 0 and leave errno in thread-local storage, which the runtime's libc bridge does not capture for a 0 return, so "gone" and "empty" could not be told apart. Caveat: Apple documents neither the trap's call numbers (XNU'sproc_info_private.h) nor libproc (its header calls the interface private and subject to change); both have been unchanged since macOS 10.5. Verified on this Mac natively on arm64, as aGOARCH=amd64binary under Rosetta, and in the garbled, stripped sidecar (make app-sidecar).CPU units.
pti_total_user/pti_total_systemare Mach absolute-time ticks, not nanoseconds: 0.267 s of CPU read as 6,406,888 ticks here. Under Rosetta the kernel still returns 24 MHz ticks whilemach_timebase_inforeports 1/1, so the conversion divides byhw.tbfrequency(24,000,000 here; 1e9 on Intel Macs, where ticks are ns).TestDarwinTaskMatchesSelfchecks it againstgetrusageon both architectures.Robustness of a reading (all from the adversarial review):
Usage.CPUPartial(zero value = complete, so Linux is unchanged): the job bound takes no rate from a partial sum and does not use it as the next baseline, so a process missing from one reading and present in the next cannot read as a burst and invent a strike.Executor.
describereads each member's sockets once and derives both the naming member and the port set from that (it used to read every member twice). Naming, argv fallback,DataDirand the per-platform nil-vs-emptyPortsare unchanged.Tags.
members_other.go,usage_other.go,service_other.goare now!linux && !darwin(Windows/other BSDs keep "cannot say"); the two service tests run onlinux || darwin.TestLocalRecordsAListeningServiceFully's shell now waits (bounded) for the server's port before it exits, so the record the call returns is asserted directly; on macOS the call otherwise often returns while the backgrounded child is still the forked shell.Behaviour change on macOS. A registered
bashjob (abackground: truecommand, or a foreground command kept as a job) that sustains more than three fifths of the logical cores (7.2 of 12 on this M4 Pro) for 30 s, or holds more than 128 processes for 30 s, is now ended, as on Linux. Thresholds are unchanged. The manual gains a section on the bound (what-i-can-do.md) and the change entry says it.Cost (benchmarks on this Mac,
-cpu 1, arm64 / amd64 under Rosetta)Task(one TASKALLINFO)Children(one PPID listing)ListenPorts(test process)ArgvMembers(10-process group)Usage(10-process tree)Usageruns every 2 s per registered bash job; 16 such jobs cost about 1 ms of CPU per second (≈0.1% of one core). Nops, no polling beyond what callers already do (Leftonce per finished shell call,GroupPortsper live item at a seal).Tests
internal/darwinproc:TestDarwinProcArgsStopsBeforeEnvironment(exact padding, emptyargv[0]with environment after it, malformed layouts),TestDarwinArgvKeepsAnEmptyFirstArgument(live, four path lengths),TestDarwinArgvReadsALiveChild,TestDarwinTaskMatchesSelf,TestDarwinChildrenListsASpawnedChild,TestDarwinListeningPorts(v4 + v6 listeners; connected TCP, UDP and unix sockets excluded), benchmarks.internal/processgroup:TestDarwinMembersExcludeZombies,TestDarwinMembersReadsALiveGroup,TestDarwinUsageReadsDescendants,TestDarwinUsageRefusesUnownedGroup, and scripted-tree walk tests (…CountsTheWholeTreeOnce,…KeepsAReusedParentsChildrenOut,…CountsATruncatedListingAndRefusesAFailedOne,…StopsAtTheWalkLimit,…MarksAnUnreadableChildPartial,…MarksAnUnreadableParentPartial), benchmarks.internal/session:TestAPartialCPUSumNeverMakesARate(fails on the old strike code),TestAPartialReadingStillCountsTowardsTheProcessBackstop.internal/executor:TestDescribeInspectsEachMemberOnce(portable).Gates (on the Mac)
gofmt;go vetof./internal/darwinproc ./internal/processgroup ./internal/executor ./internal/session ./internal/inventory(darwin), and of the first three for linux, windows and freebsd (session too for linux).CGO_ENABLED=0 go build ./cmd/codeaffor darwin/arm64, darwin/amd64, linux/amd64, linux/arm64 and windows/amd64; the garbled, stripped desktop sidecar (make app-sidecar, darwin/arm64) builds and is the binary the manual checks used.internal/darwinproc,internal/processgroup,internal/inventorypass;internal/executorpasses exceptTestWorkspaceEnvironmentRoutesRealTemporaryFilesThroughBothRunnerDoors(fails at12dadd64atoo: BSDmktemp). The two R4 tests pass, 10/10 in a row. The same darwin tests pass asGOARCH=amd64binaries under Rosetta.internal/session: the job-bound tests and the two new ones;-run Manualininternal/sessionandinternal/tui3;internal/manual,make test-packed-manual,internal/namelawpass.internal/commentlawfails only on the two files it fails on at12dadd64a(factory_pipeline_copy.go,pipeline_copy.go).go test -count=1 -p 6 ./...compared by test name with the12dadd64aMac baseline: running (results will be added here).12dadd64a: PR gate 40934512927 fails exactly app/v3's 5 (commentlaw, guard, session ×2, taxonomy); its touched-packages job (processgroup, executor, session, …) passes on Linux. Full check 40934517251: first run: 3 tests outside the baseline failed, all intermittent (two reproduced at12dadd64alocally, 1 in 30; the third is the known Mac/CI flakeTestFactoryAssigneesRunnerPublishesTeamAndSolo); failed jobs re-run, results will be added here.12dadd64a;git merge-treeonto the new app/v3 (1c4ec8a25, pair: merge a computer's own chats when it joins, instead of replacing them #2031) is clean and pair: merge a computer's own chats when it joins, instead of replacing them #2031 touches none of these files.Codex review (adversarial, read-only, 3 rounds)
argv[0]could shift an environment string into argv → fixed (kernel boundary parse, verified for path lengths 1–40 × threeargv[0]values, 0/120 mismatches, both arches); the service test repaired its result outside production → fixed (fixture handshake);pr: 0→ kept.CPUPartial, baseline cleared; regression test).…MarksAnUnreadableParentPartial); not re-reviewed by Codex. Everything else accepted.The change entry follows app/v3's convention (
docs/changes/unreleased/0-app-<slug>.md,pr: 0, as in #2014–#2016).codeaf-changes checkreports "pr is missing" for every such entry on app/v3 (249 at the base).Not covered
TestWorkspaceEnvironmentRoutesRealTemporaryFilesThroughBothRunnerDoorsstill fails on macOS (BSDmktempignoresTMPDIR; a test assumption, unrelated).🤖 Generated with Claude Code