Skip to content

boot: what the disk costs a restore, the first login's lane, and the lab's scratch on NVMe - #275

Merged
aledbf merged 3 commits into
mainfrom
restore-and-login-lane
Nov 19, 2026
Merged

aledbf merged 3 commits into
mainfrom
restore-and-login-lane

Conversation

@aledbf

@aledbf aledbf commented Nov 19, 2026

Copy link
Copy Markdown
Member

Two probes and a lab change, for the two numbers' next iteration.

boot:restore-first says what the disk costs a restore

Each row is also restored a second way, interleaved with the first: its state evicted, read back whole in one sequential pass (READ), and restored from the page cache. The table also gives the state's size.

fleet mode state read restore → running login
nested c8i.4xlarge (44542990433) from disk 163 MiB – 781/842 837/892
from the page cache 162 MiB 530/540 112/115 164/170
metal c8i.metal-48xl (44542999076) from disk 162 MiB – 891/954 940/1001
from the page cache 162 MiB 777/1228 115/119 159/162
  • A restore is ~160 ms to the first login once its state is in memory. The rest is reading 163 MiB from the runner's gp3 volume.
  • QEMU reads it ~250 ms slower than one sequential pass does.
  • Metal is slower only because its volume is.

The lab's scratch on local NVMe

spin's hosts keep checkpoints on local NVMe, so the lab now puts TMPDIR on the instance store when the host has one (formatted and mounted if nothing has), and the run's summary names the disk. spin-stack/ci#21 moves the bench to c8id.4xlarge (the same CPU, with NVMe).

boot:login-lane

Where the first login's own time goes between sshd's accept and the command running, on the guest's clock:

  • authenticated;
  • PAM's session open (pam_systemd comes after it);
  • logind's new session;
  • the session scope active;
  • the command.

The first login's command is the probe; a second login reads the journal. sshLogin becomes sshRun, which returns the command's output.

Also

kernel/README.md loses the paragraph that waited for systemd upstream: the image's systemd has no BPF framework (#271).

🤖 Generated with Claude Code

aledbf and others added 3 commits November 19, 2026 11:22
Each row is restored a second way, interleaved: its state evicted, read
back whole in one sequential pass (READ, what the disk gives), and
restored from the page cache. The table gives the state's size too. A
restore took ~790 ms to running on nested and on metal alike
(2026-11-19), and nothing said which part of it was the disk.

kernel/README.md: the paragraph that waited for systemd upstream goes;
the image's systemd has no BPF framework.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
boot:ssh-first says accept to login is ~125-145 ms on the bench fleet,
and docs/log.md puts ~56 of it in pam_systemd without saying which hop.
The first login's command marks the journal and prints its sshd@ unit's
and session scope's starts; a second login reads the journal. The marks:
accept, authenticated, PAM's session open, logind's new session, the
scope active, the command. sshLogin becomes sshRun, which returns the
command's output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
spin's hosts keep checkpoints on local NVMe; on a c8i the lab read a
restore's state from gp3, ~680 of its ~840 ms (lab 44542990433). An
instance store is used, formatted and mounted if nothing has, and the
run's summary names the disk TMPDIR is on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aledbf
aledbf merged commit 6cd42c1 into main Nov 19, 2026
3 checks passed
@aledbf
aledbf deleted the restore-and-login-lane branch November 19, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant