Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
b842a49
fork-kit: pc-agents walls and their check (part 1 of 2)
scuttletoby Nov 15, 2026
9d46a34
fork-kit: pc-agents setup, bootstrap and owner steps (part 2 of 2)
scuttletoby Nov 15, 2026
364edb6
fork-kit: pc-agents fix round 1 (walls)
scuttletoby Nov 16, 2026
6f840d6
Merge part 1 round 1 into pc-agents setup; setup fix round 1
scuttletoby Nov 16, 2026
b6ae1ea
fork-kit: pc-agents fix round 2 (walls)
scuttletoby Nov 16, 2026
60b8f79
Merge part 1 fix round 2 into pc-agents setup
scuttletoby Nov 16, 2026
315aa3b
fork-ci-tests: run test files in parallel so the step fits its 10-min…
scuttletoby Nov 16, 2026
b542146
Merge remote-tracking branch 'origin/fork/pc-agents-walls' into fork/…
scuttletoby Nov 16, 2026
519449e
fork-ci-tests: format with vp fmt
scuttletoby Nov 16, 2026
ec6f349
Merge remote-tracking branch 'origin/fork/pc-agents-walls' into fork/…
scuttletoby Nov 16, 2026
f446ea1
fork-ci-tests: prove parallel runs without clocks; shorter header
scuttletoby Nov 16, 2026
f62ac93
Merge remote-tracking branch 'origin/fork/pc-agents-walls' into fork/…
scuttletoby Nov 16, 2026
d6f9e20
fork tests: no pipe into a reader that stops early (SIGPIPE race)
scuttletoby Nov 16, 2026
b72b5de
Merge remote-tracking branch 'origin/fork/pc-agents-walls' into fork/…
scuttletoby Nov 16, 2026
4f70d86
pc-agents setup: restart wall, stays-up wait, env lookup, round 2 fixes
scuttletoby Nov 16, 2026
8867a14
Merge remote-tracking branch 'origin/main' into fork/pc-agents-walls
scuttletoby Nov 16, 2026
d9ce015
fork-run-alert test: read calls fully before grep -q
scuttletoby Nov 16, 2026
cae2529
Merge remote-tracking branch 'origin/fork/pc-agents-walls' into fork/…
scuttletoby Nov 16, 2026
4a656cc
Merge remote-tracking branch 'origin/main' into fork/pc-agents-kit
scuttletoby Nov 16, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/fork-ci-tests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ fork-kit/heavy/tests/sweep.test.py
# A real systemd user manager (the step starts one for the runner user).
fork-kit/heavy/tests/heavy.test.py run by .github/workflows/fork-tests.yml
fork-kit/pc-agents/tests/check.test.py
fork-kit/pc-agents/tests/setup.test.py

# Upstream's tests, run where fork-tests.yml follows upstream's ci.yml.
.github/scripts/check-nightly-release.test.cjs run by .github/workflows/fork-tests.yml
Expand Down
13 changes: 7 additions & 6 deletions fork-kit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,13 @@ Every kit follows the same pattern:
- safe to run again, and safe to stop at any step;
- an uninstall that leaves nothing.

| Kit | What |
| ---------------------------- | ------------------------------------------------------------------------------------------------------- |
| [desk](desk/README.md) | A desktop's side of a fork fleet desk pool: runner users, walls, desk agent; one folder (Linux, WSL2) |
| [gh-apps](gh-apps/README.md) | GitHub App tokens for agents through `gh-as`, without giving agents the keys (Linux) |
| [golive](golive/README.md) | Move a T3 boot service to a new build with a cold copy first; core and `check` so far (Linux) |
| [heavy](heavy/README.md) | `heavy <command>`: run builds and tests in a capped slice, stop what they leave behind (Linux, systemd) |
| Kit | What |
| -------------------------------- | ------------------------------------------------------------------------------------------------------- |
| [desk](desk/README.md) | A desktop's side of a fork fleet desk pool: runner users, walls, desk agent; one folder (Linux, WSL2) |
| [gh-apps](gh-apps/README.md) | GitHub App tokens for agents through `gh-as`, without giving agents the keys (Linux) |
| [golive](golive/README.md) | Move a T3 boot service to a new build with a cold copy first; core and `check` so far (Linux) |
| [heavy](heavy/README.md) | `heavy <command>`: run builds and tests in a capped slice, stop what they leave behind (Linux, systemd) |
| [pc-agents](pc-agents/README.md) | Run T3's agents on a Windows PC in a walled WSL2 distro, checked before every start (WSL2) |

[common](common/README.md) is code the kits share, copied into each kit's version folder: the loader of the one
private machine file (`/etc/fork-kit/machine.json`) where the facts of one machine live.
13 changes: 13 additions & 0 deletions fork-kit/common/machine.example.json
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,19 @@
"quietWaitSeconds": 7200,
"stopLimitSeconds": 1800
},
"pc-agents": {
"organization": "Example home lab",
"t3Version": "0.0.30",
"t3ReleaseBaseUrl": "https://github.com/example/t3code/releases/download",
"port": 3773,
"maxRuntimeMode": "full-access",
"lockedSettings": { "agentSlots.enabled": true, "agentSlots.maxPerMachine": 2 },
"memoryMax": "8G",
"cpuQuota": "400%",
"nameservers": ["1.1.1.1", "9.9.9.9"],
"peers": ["100.101.102.103/32"],
"tailscaleTag": "tag:pc-agents"
},
"desk": {
"pool": "pc",
"host": "fleet-host.example",
Expand Down
143 changes: 134 additions & 9 deletions fork-kit/pc-agents/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,17 @@ so the agent cannot undo it:
- no Windows drive is mounted;
- the loaded firewall table is the one from the kit's `firewall.nft`: `pc-agents load-firewall` saves the file's hash
and what the kernel then lists, and any later change, or an older file left loaded, differs;
- the default gateway (the Windows host) is in a blocked range, and no `allow` or `peers` entry lets it through.
- the default gateway (the Windows host) is in a blocked range, and no `allow` or `peers` entry lets it through;
- no T3 runs that started before setup's last change: while setup's marker `/etc/pc-agents/restart-pending` is there
(see "Runs again" below), a T3 process started before the time in it fails this wall, and so does a marker or a
start time it cannot read. So the 2-minute check stops a T3 that a stopped or failed setup left running with old
files, and `pc-agents as-agent` refuses meanwhile. Setup's own first check skips it.

The fleet's `mounts`, `sockets` and `userns` walls are
about the CI launcher's job units, so they are not reported here; the unit's own sandbox is checked instead (it must
be the kit's file, byte for byte, with no drop-in).

Linux (WSL2, systemd, Debian or Ubuntu) only. Python 3 standard library only. This folder has the walls and their
check so far; `pc-agents setup`, which writes them into a fresh distro, and its owner steps are still to come.
Linux (WSL2, systemd, Debian or Ubuntu) only. Python 3 standard library only.

## Config

Expand Down Expand Up @@ -90,11 +93,119 @@ refuses what is cheap to tell (a field of a provider instance, an object deeper
two settings, a blank `organization`); T3 refuses the rest, like a field inside a setting that can be empty (lock the
whole setting), says why in its log (`journalctl -u t3code-agents`) and does not start. T3 exits 1 on a policy it
cannot apply, the same code as any other failure, so the unit cannot tell it apart (`RestartPreventExitStatus=`);
it gives up after 5 starts in 10 minutes instead (`StartLimitBurst=5`).
it gives up after 5 starts in 10 minutes instead (`StartLimitBurst=5`). Setup clears that count
(`systemctl reset-failed`) before it starts T3, so running it again within the 10 minutes still starts T3.

A wall that fails stops T3, and nothing starts it again by itself: fix what `pc-agents check` names (or run
`pc-agents setup` again), then start it as root: `systemctl start t3code-agents.service` (its own start runs the
check first and refuses on a failed wall).
check first and refuses on a failed wall). If it says `start request repeated too quickly`, run
`systemctl reset-failed t3code-agents.service` first.

## Owner steps (once per PC)

Commands are for Windows PowerShell unless a step says otherwise. `<distro>` is a name of your choice (for example
`t3-agents`); `<fork>` is the fork's `OWNER/NAME`, `<sha>` a commit of its `ci-release` branch.

1. **Make a fresh distro** (not the CI one, not your own): `wsl --install -d Ubuntu-24.04 --name <distro> --no-launch`.
2. **Write the machine file** inside it as root: `wsl -d <distro> -u root`, then
`mkdir -p /etc/fork-kit && nano /etc/fork-kit/machine.json` (the example above, your values), and leave with `exit`.
Always open this distro with `-u root`. If a window ever asks you for a new user name and password (Ubuntu's
first-launch setup), that makes a login user, which setup refuses: remove it as root (`userdel -r <name>`) or
start again from step 1. The kit makes `root` the default user, so this cannot happen after step 4.
3. **Run the setup** as root:
`wsl -d <distro> -u root -- sh -c 'curl -fsSL https://raw.githubusercontent.com/<fork>/ci-release/fork-kit/pc-agents/bootstrap.sh -o /root/pc-agents-bootstrap.sh && sh /root/pc-agents-bootstrap.sh --repo <fork> --sha <sha>'`.
The bootstrap comes from the protected `ci-release` branch itself, not from `<sha>`, so it is a reviewed one; it
refuses a `<sha>` that is not on that branch before it downloads or runs anything of it.
The first run writes everything, then says a wall does not hold (Windows' drives are still mounted) and does
not start T3.
4. **Restart the distro** so `wsl.conf` applies: `wsl --terminate <distro>`.
5. **Run the setup again** (the same command as step 3). Now every wall holds and T3 starts. It prints the next step.
6. **Join Tailscale** with the tag (as root in the distro): install it
(`curl -fsSL https://tailscale.com/install.sh | sh`), then `tailscale up --advertise-tags=<tag> --accept-dns=false`
(keep the kit's public nameservers; MagicDNS is on the tailnet, which the agent cannot reach) and open the link it
prints in a browser on any machine. The linking T3's Tailscale address is already in `peers` (step 2's file needs
it; setup refuses a file without it); if that address changes, edit the file and run step 3 again. The ACL is
below.
7. **Log in to Claude as the agent:** never open a shell as `agent` (`wsl -u agent` would start the agent's own
session, outside T3's walls). Run each command through the kit instead, which runs it as `agent` under the same
walls as T3 (a transient system unit, `systemd-run`):
`wsl -d <distro> -u root -- /opt/pc-agents/current/sbin/pc-agents as-agent -- sh -c '<the install command from Claude Code's docs>'`,
then the same with `-- claude`; open the link it prints in your browser. The kit runs the command through `env`
with the agent's `PATH` (`/home/agent/.local/bin` first), so a bare `claude` is found there, and with only
`HOME`, `T3CODE_HOME`, `PATH` and `SHELL` set. Codex: its browser login waits on `localhost` port 1455, outside
`devPorts`, so the walls refuse it: use its device-code login instead, the same with `-- codex login --device-auth`
(`codex login --help` names it).
8. **Log in to GitHub as the agent:** install `gh` (as root, from GitHub CLI's docs), then
`wsl -d <distro> -u root -- /opt/pc-agents/current/sbin/pc-agents as-agent -- gh auth login --with-token` and paste
a fine-grained token for the agents' repos into that window only.
9. **Link from the other T3:** on this PC's T3 (`http://<tailscale address>:<port>`), Settings → Sharing & access →
make a pairing code. In the other T3: Settings → Environments → Linked environments → Link, the same address
and the code, **Full access**.
10. **Keep the distro running:** WSL stops a distro soon after its last window closes, and does not start one when
Windows starts. A logon task of your own (no administrator needed) keeps this one running while you are logged
in. In PowerShell, as yourself (the same task as the desk starter's, `fork-kit/desk/windows/start-desk.ps1`):
```powershell
$a = New-ScheduledTaskAction -Execute 'conhost.exe' -Argument '--headless wsl.exe -d <distro> -u root --exec sleep infinity'
Register-ScheduledTask -TaskName 'pc-agents <distro>' -Action $a -Trigger (New-ScheduledTaskTrigger -AtLogOn -User ([Security.Principal.WindowsIdentity]::GetCurrent().Name)) -Settings (New-ScheduledTaskSettingsSet -ExecutionTimeLimit 0 -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries) -Force
Start-ScheduledTask -TaskName 'pc-agents <distro>'
```
`conhost.exe --headless` shows no window. Running it again replaces the task. A distro set up before this step
only needs these lines; nothing in the distro changes. To remove it:
`Unregister-ScheduledTask -TaskName 'pc-agents <distro>' -Confirm:$false`.

Run step 3 again for a new release or a changed machine file (and steps 4 and 5 if it asks); T3 restarts only when
something changed. To remove everything: `wsl --unregister <distro>`.

## Tailscale tag and ACL

The PC's node is tagged, so it is not you: it gets no access of its own, only what the ACL grants. Grant one thing,
the linking T3 reaching T3's port here, and nothing from the PC:

```json
{
"tagOwners": { "tag:pc-agents": ["autogroup:admin"], "tag:orchestrator": ["autogroup:admin"] },
"grants": [{ "src": ["tag:orchestrator"], "dst": ["tag:pc-agents"], "ip": ["tcp:3773"] }]
}
```

With no grant whose `src` is `tag:pc-agents`, the PC starts no connection into the tailnet; and the agent's uid cannot
reach any tailnet address anyway (100.64.0.0/10 and `fc00::/7` are dropped for it). Replies to the linking T3 are
allowed (connection tracking).

## Runs again, stops half-way, meets old or empty state

- **Again:** every file is compared before it is written; an unchanged file is not touched, and T3 restarts only
when a file, the release or the T3 version changed. The tests run the setup twice and find no change.
- **Stopped half-way:** each file is written whole (a new file, then one rename). Before and after every change
(and after a T3 install, and once more right before its restart), setup writes the root-owned marker
`/etc/pc-agents/restart-pending` with this boot's id and the time (`CLOCK_MONOTONIC`, the clock of systemd's
`ExecMainStartTimestampMonotonic`, which no clock change moves). It is removed only after T3 restarted and stayed
up for 15 seconds (the same process, active, no automatic restart). So a run stopped anywhere, or a failed T3
install, leaves the marker, and the next good run restarts T3 even when it has nothing left to write. Meanwhile
the `restart` wall fails while the T3 that runs started before the marker's time, so the 2-minute check stops it
(an old T3 never keeps running under new files and a passing check). A run stopped during setup's own restart or
its 15 seconds leaves a T3 started after the marker: the wall passes, and the next run restarts it once more. A
marker of an earlier boot passes (every T3 now started after it); one an older kit wrote, with no time in it,
fails until setup runs again. A T3 that starts and then exits within the 15 seconds is reported (`T3 did not
start or stay up`), and the marker stays. The record
(`/etc/pc-agents/installed.tsv`) holds a file's old and new hash while it is replaced, so a run stopped right after
a write, followed by an edit of the machine file, is not mistaken for a hand edit. A new T3 version is installed by
root after the new firewall is loaded, so a slow or failed download never leaves the old firewall in place.
- **Timeouts and signals:** every program setup runs has a time limit (30 minutes for T3's download, 15 for
`apt-get`); past it the whole process group is stopped and setup says so. Run it again. Ctrl-C, `SIGTERM` or
`SIGHUP` (a closed window) stop setup the same way, the running download's whole group too.
- **Start limit:** a rerun within 10 minutes of five failed starts still starts T3 (`systemctl reset-failed` first).
- **An older kit's distro:** an `agent` made with a login shell gets `/usr/sbin/nologin` (`usermod`), once.
- **A distro that is not fresh:** refused before anything is written, with the reasons: another login user, an
`agent` with another uid or in another group, a file the kit would write that it did not write (or that was
edited since, for example a hand-written `wsl.conf` or `policy.json`), a cron, at or ssh unit where the kit masks
one. A fresh distro's own `wsl.conf` (systemd only) and WSL's generated `resolv.conf` are replaced.
- **Old releases:** each release has its own folder under `/opt/pc-agents/`, switched by the `current` link; old ones
stay (small) until the distro is removed. A T3 that an older kit installed in the agent's home is left there and
no longer run (the unit runs root's copy under `/opt/pc-agents/t3`).
- **Unsafe folders:** setup refuses a distro whose `/`, `/opt`, `/opt/pc-agents`, `/etc`, `/etc/systemd`,
`/etc/systemd/system`, `/etc/pc-agents` or `/etc/t3code` is not root's alone.
- **No or a broken machine file:** setup and check refuse, naming the key; the unit does not start.

## Limits

Expand Down Expand Up @@ -128,19 +239,33 @@ check first and refuses on a failed wall).
- **DNS** goes to the public `nameservers` only. A `systemd-resolved` stub (`127.0.0.53`), which may resolve LAN
names through Windows, is out of reach: the agent's localhost traffic outside T3's port and `devPorts` is dropped.
- **Terminals:** what the agent prints in a terminal can use escape codes, such as OSC 52, which some terminals turn
into a write to the Windows clipboard. Do not paste from the clipboard blindly after watching agent output.
into a write to the Windows clipboard. `pc-agents as-agent` runs with `--pty`, so the owner steps' output reaches
your terminal that way too. Do not paste from the clipboard blindly after watching agent output.
- **Runs only while you are logged in:** the logon task (step 10) keeps the distro running while you are logged in
and the PC is awake. Logging out, sleep, a Windows restart or `wsl --shutdown` stop T3 (the linking T3 sees this
PC offline); the task starts it again at your next logon. Without the task, T3 stops soon after the last WSL
window closes.
- **No namespaces in the unit** (`RestrictNamespaces=yes`): provider sandboxes that need them (bubblewrap) do not
run; the distro is the sandbox, so use full access.
- **Tailscale, Claude Code and `gh` are yours to install** (steps 6 to 8): their installers and logins need you.
Claude Code and logins go through `pc-agents as-agent`, never a shell as `agent`.
- **No uninstall** but removing the distro; the distro holds nothing else.
- **No code hash manifest yet:** the kit's code is checked for root ownership and its written files byte for byte,
but the code files are not hashed (T3's `t3` is).
but the code files are not hashed (T3's `t3` is). The bootstrap duplicates the heavy kit's (a shared bootstrap is
a follow-up).

## Tests

```
python3 -I -B fork-kit/pc-agents/tests/setup.test.py
python3 -I -B fork-kit/pc-agents/tests/check.test.py
```

A fake distro in a temporary folder (`PC_AGENTS_ROOT`, refused for root; `tests/fake_distro.py`), with fake `nft` and
`systemctl` (it keeps which units are enabled and running): every wall passes on a walled distro, and each one
A fake distro in a temporary folder (`PC_AGENTS_ROOT`, refused for root; `tests/fake_distro.py`), with fake
`useradd`, `usermod`, T3's installer, `systemctl` (it keeps which units are enabled and running, their main process
and start time, a start limit, and a T3 that dies a moment after it starts), `systemd-run` (a bare command is looked
up in the caller's `PATH`, as the real one does) and `nft`: every file and mode, a second run changes nothing, a run
stopped at every write finishes on the next and restarts T3, an upgrade stopped at every write fails `check` until
setup runs again, `SIGTERM` and `SIGHUP` end the download too, each kind of foreign distro is refused, and each wall
fails on its own when its part is taken away.
`apps/server/src/policy/managedSettings.forkKeys.test.ts` proves the managed policy locks the fork's settings.
Loading
Loading