ssh: standardize authorized_keys on /media/fat/config, with a one-time migration

.github/workflows/lint.yml

@@ -18,6 +18,7 @@
 #   shellcheck -s sh -x package/azcopy/azcopy-profile.sh
 #   scripts/shellcheck-composite-actions.sh # .github/actions/*/action.yml run: bodies
 #   scripts/test-installer-splash.sh        # SD-installer first-boot splash (ADR 0020 §6)
+#   scripts/test-authorized-keys-migration.sh  # S50sshd's authorized_keys move (#183)
 #
 ################################################################################
 
@@ -371,3 +372,24 @@ jobs:
         run: |
           set -euo pipefail
           scripts/test-installer-splash.sh
+
+      # Same shape again, for S50sshd's one-time authorized_keys migration
+      # (issue #183). It deletes the user's old key file once it has copied it,
+      # so a bug there does not mean "the migration did not run", it means "the
+      # key is gone" on a box whose password nobody remembers.
+      #
+      # BusyBox is installed if the runner image lacks it, because the sharpest
+      # edge in that function -- `grep -F -x -v -f` against an EMPTY pattern
+      # file -- behaves the OPPOSITE way under BusyBox and GNU grep, and only
+      # the BusyBox reading is the one the image ships. Without it, case 12
+      # cannot fail. The test reports a loud SKIP rather than a pass if the
+      # install does not work out, so a broken package name degrades the check
+      # visibly instead of silently.
+      - name: authorized_keys migration unit test (scripts/test-authorized-keys-migration.sh)
+        run: |
+          set -euo pipefail
+          if ! command -v busybox >/dev/null 2>&1; then
+            sudo apt-get update -qq && sudo apt-get install -y -qq busybox-static \
+              || echo "::warning::could not install busybox -- the BusyBox pass of the authorized_keys migration test will be SKIPPED"
+          fi
+          scripts/test-authorized-keys-migration.sh

README.md

@@ -415,11 +415,15 @@ Full write-up with the reasoning for each: [`docs/patch-provenance.md` §10](doc
   update discards it. (The root filesystem is read-only at boot and only becomes writable
   once you log in, so putting a key there also means logging in first, which is circular
   when the key *is* the login method.) `sshd` here also
-  reads **`/media/fat/linux/authorized_keys`**, on the exFAT partition an update never
+  reads **`/media/fat/config/authorized_keys`**, on the exFAT partition an update never
   touches: drop your `.pub` file there from any PC with a card reader and key login keeps
   working across every future update. No shell access, no script to edit, and
   `StrictModes` stays on — the initramfs mounts that partition `fmask=0022,dmask=0022`,
-  which is exactly what `sshd` requires. ([FAQ](docs/user/faq.md#ssh-key-persist))
+  which is exactly what `sshd` requires. That is the same file
+  [`security_fixes.sh`](https://github.com/MiSTer-devel/Scripts_MiSTer) has read since
+  2021, so a key set up for stock already works here — the difference is that stock
+  copies it *into* `linux.img` and so needs the script re-run after every update, while
+  this image reads it in place. ([FAQ](docs/user/faq.md#ssh-key-persist))
 - **OpenSSH 8.6p1 → 10.5p1**, **Samba ~4.14 → 4.24.6**, **BlueZ → 5.86**,
   **wpa_supplicant 2.9 → 2.12** — the network-facing surface, several release cycles of
   hardening each.

board/mister/de10nano/rootfs-overlay/etc/init.d/S50sshd

@@ -2,15 +2,8 @@
 #
 # sshd        Starts sshd.
 #
-# P2.3 / ADR 0015: SSH host keys are NOT shipped in this image. Every stock
-# MiSTer bakes the SAME ssh_host_* private keys into a read-only rootfs (all
-# dated 2016-12-31) -- a real vulnerability: trivial host impersonation,
-# and no host-key-mismatch warning to tip a user off, since every box's key
-# "matches". We generate a unique key set per device, on first boot, and
-# persist it to a small ext4 image on the FAT data partition -- mirroring
-# stock's OWN mechanism for exactly this shape of problem (Bluetooth
-# pairing keys also need real Unix permissions on a permission-less FAT
-# card): see bin/bluetoothd and docs/decisions/0015-per-device-ssh-host-keys.md.
+# Per-device host keys (ADR 0015) + the one-time user authorized_keys move
+# (#183). Full reasoning: docs/init-parity.md, docs/ssh-ftp-parity.md §1.3.
 #
 
 KEYIMG=/media/fat/linux/ssh.ext4
@@ -21,7 +14,118 @@ KEYDIR=/etc/ssh_keys
 
 umask 077
 
+# A USER's key, on the exFAT partition an update never touches. AUTHKEYS_LEGACY
+# is the pre-#183 location, migrated below. See docs/ssh-ftp-parity.md §1.3.
+AUTHKEYS=/media/fat/config/authorized_keys
+AUTHKEYS_LEGACY=/media/fat/linux/authorized_keys
+
+# A failed migration is survivable: sshd still starts, and start() keeps the old
+# path in its AuthorizedKeysFile list for this boot.
+migrate_warn() {
+	echo "WARNING: could not migrate $AUTHKEYS_LEGACY to $AUTHKEYS." >&2
+	echo "WARNING: sshd will keep reading the OLD path for this boot, so key" >&2
+	echo "WARNING: login still works; move the file by hand to make it stick." >&2
+}
+
+# grep into a file, distinguishing "matched nothing" (status 1, fine) from "went
+# wrong" (2+), which must never reach the caller as an empty file -- see §1.3.
+filter() { # <outfile> <grep args...>
+	_out=$1
+	shift
+	# The `else` is load-bearing: after a bare `if ...; fi`, `$?` is the IF
+	# STATEMENT's status, not grep's. Inside the else it is still grep's.
+	if grep "$@" > "$_out"; then
+		return 0
+	else
+		_st=$?
+		[ "$_st" -eq 1 ]
+	fi
+}
+
+# Move a user's authorized_keys to the #183 location, once. COPY, VERIFY, THEN
+# DELETE -- never mv: that file may be the only way into the box.
+migrate_authorized_keys() {
+	[ -f "$AUTHKEYS_LEGACY" ] || return 0
+
+	echo "Migrating $AUTHKEYS_LEGACY -> $AUTHKEYS"
+
+	# Scratch on tmpfs, as a DIRECTORY: creating it proves /run is writable
+	# before anything below trusts an empty file.
+	scratch=/run/authorized_keys.$$
+	if ! mkdir "$scratch"; then
+		migrate_warn
+		return 1
+	fi
+	legacy_lines="$scratch/legacy-lines"
+	legacy_keys="$scratch/legacy-keys"
+	dest_lines="$scratch/dest-lines"
+	merged="$scratch/merged"
+
+	# Non-blank lines are what gets carried over (a user's comments included);
+	# the key lines alone decide whether there is anything worth keeping.
+	if ! filter "$legacy_lines" -v '^[[:space:]]*$' "$AUTHKEYS_LEGACY" ||
+	   ! filter "$legacy_keys" -v '^[[:space:]]*#' "$legacy_lines"; then
+		migrate_warn
+		rm -rf "$scratch"
+		return 1
+	fi
+
+	if [ ! -s "$legacy_keys" ]; then
+		rm -f "$AUTHKEYS_LEGACY"
+		rm -rf "$scratch"
+		echo "  old file held no keys -- removed"
+		return 0
+	fi
+
+	# Test $dest_lines for content, not $AUTHKEYS for existence: an EMPTY
+	# pattern file matches every line under BusyBox grep and none under GNU.
+	: > "$dest_lines"
+	if [ -f "$AUTHKEYS" ] &&
+	   ! filter "$dest_lines" -v '^[[:space:]]*$' "$AUTHKEYS"; then
+		migrate_warn
+		rm -rf "$scratch"
+		return 1
+	fi
+
+	if [ -s "$dest_lines" ]; then
+		# MERGE, never clobber: two PCs can mean two keys. Base is $dest_lines,
+		# not `cat "$AUTHKEYS"`, which splices when the card file lacks a \n.
+		cat "$dest_lines" > "$merged"
+		grep -F -x -v -f "$dest_lines" "$legacy_lines" >> "$merged"
+	else
+		mkdir -p "${AUTHKEYS%/*}"
+		cat "$legacy_lines" > "$merged"
+	fi
+
+	# Rename so no reader sees a partial file, then verify what landed ON THE
+	# CARD. `-s` guards the empty-pattern-file trap described above.
+	if cp "$merged" "$AUTHKEYS.new" && mv "$AUTHKEYS.new" "$AUTHKEYS" &&
+	   [ -s "$AUTHKEYS" ] &&
+	   ! grep -F -x -v -f "$AUTHKEYS" "$legacy_lines" | grep -q .; then
+		# sync BEFORE the delete: exFAT is not mounted sync, so the new data
+		# must reach the card before the old file's unlink can.
+		sync
+		rm -f "$AUTHKEYS_LEGACY"
+		echo "  OK -- $(wc -l < "$legacy_keys") key(s) migrated"
+		rm -rf "$scratch"
+		return 0
+	fi
+
+	migrate_warn
+	# Quiet: whatever failed the migration tends to fail these too, and the
+	# errors would only bury the warning above.
+	rm -rf "$scratch" 2>/dev/null
+	rm -f "$AUTHKEYS.new" 2>/dev/null
+	return 1
+}
+
 start() {
+	if migrate_authorized_keys; then
+		authkeys_fallback=
+	else
+		authkeys_fallback=yes
+	fi
+
 	echo 'Checking for SSH host key storage'
 
 	if [ ! -f "$KEYIMG" ]; then
@@ -31,40 +135,36 @@ start() {
 	mkdir -p "$KEYDIR"
 	mount -o sync,dirsync,noatime,nodiratime "$KEYIMG" "$KEYDIR"
 
-	# Persistence is best-effort; sshd starting is NOT. If the ext4 mount did
-	# not take (missing /media/fat, corrupt image, no free loop device), then
-	# $KEYDIR is the READ-ONLY rootfs and ssh-keygen below cannot write there
-	# -- sshd would then come up with no host key and refuse every connection.
-	# That is the one outcome we must never ship, because serial is then the
-	# only way back in. Fall back to a tmpfs keydir so sshd ALWAYS starts;
-	# keys are non-persistent in that mode (regenerated each boot), which is a
-	# tolerable degradation, and we say so loudly on the console so it is
-	# diagnosable rather than silent.
+	# Persistence is best-effort; sshd starting is NOT. No mount means $KEYDIR
+	# is the read-only rootfs, so fall back to tmpfs and say so loudly.
 	if ! mountpoint -q "$KEYDIR"; then
 		echo "WARNING: could not mount $KEYIMG at $KEYDIR -- SSH host keys" >&2
 		echo "WARNING: will be EPHEMERAL this boot (regenerated, not persisted)." >&2
 		KEYDIR=/run/ssh_keys
 		mkdir -p "$KEYDIR"
 	fi
 
-	# Create any missing keys -- first boot only, each type generated
-	# individually so it lands FLAT in $KEYDIR (ssh-keygen -A -f instead
-	# uses its argument as a prefix ahead of the whole compiled-in
-	# /etc/ssh/... path, not a replacement for it, which would not match
-	# the flat HostKey paths below in sshd_config).
+	# First boot only, per type: `ssh-keygen -A -f` treats its argument as a
+	# PREFIX, so keys would not land flat in $KEYDIR (docs/init-parity.md).
 	for kt in rsa ecdsa ed25519; do
 		key="$KEYDIR/ssh_host_${kt}_key"
 		[ -f "$key" ] || ssh-keygen -q -t "$kt" -N '' -f "$key"
 	done
 
 	printf "Starting sshd: "
-	# -o HostKey overrides sshd_config's compiled paths, so the tmpfs
-	# fallback above is actually honoured (otherwise sshd would read the
-	# persistent /etc/ssh_keys paths from the config and find nothing).
-	/usr/sbin/sshd \
-		-o "HostKey=$KEYDIR/ssh_host_rsa_key" \
+	# -o HostKey overrides sshd_config's paths, so the tmpfs fallback above is
+	# actually honoured.
+	set -- -o "HostKey=$KEYDIR/ssh_host_rsa_key" \
 		-o "HostKey=$KEYDIR/ssh_host_ecdsa_key" \
 		-o "HostKey=$KEYDIR/ssh_host_ed25519_key"
+
+	# Migration failed, so keep reading the old path this boot. -o REPLACES the
+	# config's list, hence both shipped paths are restated.
+	if [ -n "$authkeys_fallback" ]; then
+		set -- "$@" -o "AuthorizedKeysFile=.ssh/authorized_keys $AUTHKEYS $AUTHKEYS_LEGACY"
+	fi
+
+	/usr/sbin/sshd "$@"
 	touch /var/lock/sshd
 	echo "OK"
 }

board/mister/de10nano/rootfs-overlay/etc/ssh/sshd_config

@@ -15,11 +15,8 @@
 #ListenAddress 0.0.0.0
 #ListenAddress ::
 
-# P2.3 / ADR 0015: host keys are per-device, generated on first boot by
-# /etc/init.d/S50sshd into a persistent ext4 image mounted at /etc/ssh_keys
-# (NOT the compiled-in /etc/ssh/ssh_host_* default -- "/" is read-only and
-# these must survive reboots/reflashes). The image ships NO ssh_host_*
-# private keys anywhere -- see docs/decisions/0015-per-device-ssh-host-keys.md.
+# ADR 0015: per-device host keys, generated on first boot by S50sshd into an
+# ext4 image mounted here. No ssh_host_* private key ships in this image.
 HostKey /etc/ssh_keys/ssh_host_rsa_key
 HostKey /etc/ssh_keys/ssh_host_ecdsa_key
 HostKey /etc/ssh_keys/ssh_host_ed25519_key
@@ -46,45 +43,9 @@ PermitRootLogin yes
 # The default is to check both .ssh/authorized_keys and .ssh/authorized_keys2
 # but this is overridden so installations will only check .ssh/authorized_keys
 #
-# SECOND PATH ADDED: /media/fat/linux/authorized_keys -- a user's key that
-# SURVIVES AN IMAGE UPDATE. An update replaces linux.img wholesale, so anything
-# under /root/.ssh -- which lives inside that file -- is destroyed by it. That
-# is the durable reason for this second path, and it holds regardless of how
-# the root filesystem happens to be mounted at the time.
-#
-# Getting a key into the first path is awkward besides: / is mounted READ-ONLY
-# at boot (`ro` on the cmdline; inittab's remount-rw line is deliberately left
-# commented, ADR 0011) and a fresh image ships NO /root/.ssh at all. It does
-# become writable later -- /etc/profile ends with `mount -o remount,rw /` on
-# interactive login, which is how / ever becomes writable at all (stock parity,
-# docs/init-parity.md) -- so a key CAN be placed there by hand. It just needs a
-# login first, which is circular when the key IS the login method, and it does
-# not survive the next update either way. The FAT partition is not reflashed,
-# so a key here persists across every update. This
-# is the same "persist it on /media/fat" principle as ADR 0015's host keys,
-# and sshd's native multi-path support means it needs no init script, no
-# bind-mount and no user-startup.sh hook.
-#
-# WHY NOT ssh.ext4 (ADR 0015's mechanism)? Considered and rejected. That is an
-# ext4 image INSIDE a file, right for HOST keys because the DEVICE writes them
-# -- but an authorized_keys file is written by the USER, and an ext4-in-a-file
-# cannot be opened from Windows or macOS with a card reader, while editing it
-# on the box needs the very shell access the key is meant to grant. The split
-# is who writes the file: machine-written state in ssh.ext4, user-supplied
-# state on exFAT.
-#
-# StrictModes STAYS ON (default yes) and this path satisfies it: the initramfs
-# mounts the FAT partition with fmask=0022,dmask=0022 and no uid/gid options
-# (board/mister/common/initramfs-overlay/init:27), so the file lands
-# root-owned 0755 and its parents 0755 -- owner-writable only, which is what
-# sshd requires. Those mount options are OURS and fixed, not user-tunable, so
-# this cannot be silently invalidated by a differently-mounted card. Verified
-# end-to-end on hardware (exFAT, StrictModes=yes, key auth from this path
-# alone).
-#
-# ORDER MATTERS ONLY FOR PRECEDENCE, NOT FUNCTION: sshd tries every listed
-# file, so a key in either location works.
-AuthorizedKeysFile	.ssh/authorized_keys /media/fat/linux/authorized_keys
+# Second path added: a user's key on the exFAT partition, which an image update
+# does not touch. Why there, and the migration: docs/ssh-ftp-parity.md §1.3.
+AuthorizedKeysFile	.ssh/authorized_keys /media/fat/config/authorized_keys
 
 #AuthorizedPrincipalsFile none
 

docs/ci.md

@@ -2191,11 +2191,22 @@ line — which, for a `runs-on:` typo or an unquoted glob in a rarely-hit
 branch, can be months. Cheap (a two-binary download, no build) and scoped by
 `paths:` so it never fires on the 3-hour image build's pushes.
 
-It has since picked up one non-shell job for the same reason: the last step
-runs `scripts/sbom-to-dependency-snapshot.py --self-test`. That script is
-pure logic over a CSV that `release.yml` only exercises on a tag, so this is
-the only place a regression in it surfaces on the PR that caused it. See
-[`#dependency-graph-submission`](#dependency-graph-submission).
+It has since picked up three jobs that are not linting at all, for the same
+reason — each is pure logic that nothing else reaches on a PR, and each is
+fast enough to belong in the cheap workflow rather than the 3-hour one:
+
+- `scripts/sbom-to-dependency-snapshot.py --self-test` — logic over a CSV that
+  `release.yml` only exercises on a tag. See
+  [`#dependency-graph-submission`](#dependency-graph-submission).
+- `scripts/test-installer-splash.sh` — the SD-installer's first-boot splash,
+  whose only other coverage needs a built `sdcard.img` and two QEMU boots.
+- `scripts/test-authorized-keys-migration.sh` — `S50sshd`'s one-time move of a
+  user's `authorized_keys` to `/media/fat/config` (issue #183). The step
+  installs `busybox-static` if the runner image has no BusyBox, because the
+  migration's sharpest edge (`grep -f` against an empty pattern file) behaves
+  the *opposite* way under GNU grep, and the BusyBox reading is the one the
+  image ships. The test prints a loud SKIP rather than a pass if BusyBox is
+  unavailable.
 
 <a id="push-pr-trigger-split"></a>
 ### push/pull_request split, applied even though this job is cheap

docs/decisions/0031-secure-by-default-network-posture.md

@@ -89,7 +89,8 @@ Everything below was verified on 2026-09-11 against the HIL rig (beta `260904`,
   the MiSTer update ecosystem; that is a parity fact, not a gap this project can close
   alone.
 - `authorized_keys` on the FAT partition already survives updates, satisfies `StrictModes`,
-  and is CI-asserted (`docs/ssh-ftp-parity.md` §1.3).
+  and is CI-asserted (`docs/ssh-ftp-parity.md` §1.3). Since issue #183 it lives at
+  `/media/fat/config/authorized_keys`, the location `security_fixes.sh` already used.
 
 ### The tension
 
@@ -110,7 +111,7 @@ the owner's answer to Q1 below; Tier 3 is recorded so it is not re-discovered.
 
 ### Tier 1 — invisible to a stock-style user
 
-1. **Key present ⇒ password auth off.** If `/media/fat/linux/authorized_keys` (or
+1. **Key present ⇒ password auth off.** If `/media/fat/config/authorized_keys` (or
    `/root/.ssh/authorized_keys`) is non-empty at sshd start, `S50sshd` passes
    `-o PasswordAuthentication=no -o KbdInteractiveAuthentication=no`. Implemented in the
    init script, not `sshd_config`, because a `Match` block cannot test for a file. Lockout