Serve a first boot page that loads nothing from a third party

COVERAGE.md

@@ -67,6 +67,56 @@ project-authored code, with the inherited modules without tests omitted in
 `pyproject.toml` until their tests land. Both thresholds are only ever
 raised. The sections that follow record the state before the merges.
 
+## Branch fix/no-third-party-script: the first boot page (2026-09-28)
+
+`firstboot.d/29tagid` and `lib/tagid.sh` stop putting a third party script
+on the page the appliance serves before anybody has logged in
+(Keel-Linux/inithooks#13). `tests/test-tagid.bats` grows from 14 to 27
+tests. Measured with kcov 43 and bats 1.11, 124 bats over the six files:
+
+| File | Before | After |
+| --- | --- | --- |
+| `firstboot.d/29tagid` | 19/19 | 17/17, 100 percent |
+| `lib/tagid.sh` | 8/8 | 2/2, 100 percent |
+
+Both files shrank because four functions went away with the scripts they
+built. Total 99.53 (was 99.40). The lowest file is still
+`lib/init-fence.sh` at 98.44, so the shell gate stays at 98. Python is
+untouched: 187 passed, 3 skipped.
+
+**What is asserted, and what is not.** The verdict is taken from the
+rendered page, not from the source of the hook: the page is parsed for
+every host a `<script>`, `<link>`, `<img>` or `<iframe>` would fetch from,
+and that set has to be empty. A grep for one known hostname would pass the
+day somebody adds a different one, which is how the jQuery from
+`ajax.googleapis.com` survived next to the two scripts the issue named. An
+`<a href>` is deliberately not in that set: a link is somewhere the
+operator may choose to go, not something the page loads.
+
+Both the packaged page and a page inherited in `/var` from an older image
+are covered, because `/var` survives and `29tagid` is the only thing that
+looks at that copy again. The failure path of the rewrite has a test too:
+with `perl` stubbed to fail, the index is left byte for byte as it was and
+no `.tmp` is left behind, which is the `docs/traps.md` entry "gpg truncates
+its output file before asking for the passphrase".
+
+Refutations use `run !`, never a bare `! cmd`: bash does not apply errexit
+to a negated command, so a bare one passes whatever happens.
+
+After review (2026-09-29): the strip and the detector shared a blind spot
+(double quoted attributes of four elements), so an unquoted
+`<script src=...>`, a remote stylesheet, image or frame got past both.
+`29tagid` no longer strips anything: it rebuilds the `/var` copy from the
+packaged pages on every run, beside the served directory, and swaps it in;
+`lib/tagid.sh` is down to `tagid_app_name`, 1/1, and `29tagid` is 29/29.
+The pages are now judged by two detectors that share no parser:
+`tests/remote_loads.py` (HTMLParser, every attribute and style construct
+that fetches, with its own 9 Python tests and 48 cases) and a tag level
+regular expression that knows no HTML, and each hostile page the review
+measured, plus parser differentials such as `<!--><img src=...>`, is
+served as an inherited page and must be gone. `tests/test-tagid.bats` has
+27 tests, 125 bats in all, total 99.55; Python 196 passed, 3 skipped.
+
 ## Branch feat/ip6-preseed: IPv6 preseed keys in 01ipconfig (2026-09-26)
 
 Adds `IP6_CONFIG`, `IP6_ADDRESS`, `IP6_GW`, `IP6_DNS1` and `IP6_DNS2` to

debian/changelog

@@ -1,3 +1,30 @@
+inithooks (2.3.6+keel8) trixie; urgency=high
+
+  * 29tagid: the first boot page carries no third party script. It loaded two
+    from ajax.turnkeylinux.org, naming the build, the version and the
+    appliance in the URL path, and a jQuery from ajax.googleapis.com that
+    nothing on the page used. Both ran in the operator's browser on a machine
+    nobody had logged into yet, and neither could be attested by keel verify,
+    which can account for every byte in the layer and nothing fetched at view
+    time. The turnkeylinux endpoints answer 404 today, so what shipped was a
+    beacon to a service that no longer exists.
+  * tagid.sh: drop tagid_render_scripts, tagid_is_tagged, tagid_version and
+    tagid_build, which existed only to build those URLs.
+  * 29tagid: the fence pages in /var are rebuilt from the packaged ones on
+    every run, not edited, so pages inherited from an older image lose
+    whatever they loaded from another host, however it was written. The
+    new copy is built beside the served directory and swapped in; if that
+    fails the /var copy is removed and the fence serves the packaged pages.
+    It runs under turnkey-init too, which is how a machine whose first boot
+    did not complete, or one initialised before this version, gets its
+    pages replaced; until then such a machine keeps the old page, which is
+    served only while the fence is up.
+  * 29tagid: no longer reads /etc/apt/apt.conf.d/01turnkey. The build tag it
+    parsed out of the apt User-Agent had no other consumer, and common
+    removes that file (Keel-Linux/common#6).
+
+ -- Marcos Mendez <mendez.foto@gmail.com>  Tue, 29 Sep 2026 04:39:56 +0000
+
 inithooks (2.3.6+keel7) trixie; urgency=high
 
   * lib/init-fence.sh: the fence no longer reports a fence it did not put up.

firstboot.d/29tagid

@@ -1,41 +1,73 @@
 #!/bin/bash
 
-# tag the init-fence index page with the appliance name, version and build;
-# parsing and rendering live in lib/tagid.sh
-
-[ -n "$_TURNKEY_INIT" ] && exit 0
+# Serve the packaged init fence pages, named after the appliance, and
+# nothing else.
+#
+# HTDOCS is in /var, and /var survives: an image built before
+# Keel-Linux/inithooks#13 left an index there that loads scripts from
+# another host, and this hook is the only thing that ever looks at it again.
+# So HTDOCS is not edited, it is rebuilt from the packaged copy on every
+# run: a page the package did not ship is not served, whatever it contains
+# and however it is written. An earlier version of this hook stripped remote
+# <script> elements with a regular expression, and review measured what got
+# past it (unquoted attributes, stylesheets, images, frames); an HTML parser
+# would have its own differences from a browser. Nothing in the tree writes
+# a page into HTDOCS but this hook, so rebuilding it costs nothing.
+#
+# It runs under turnkey-init as well: that is what an operator runs when the
+# first boot did not complete, with the fence up and the old page in /var.
+#
+# The new copy is built beside HTDOCS, not in it (the fence serves HTDOCS),
+# and swapped in with two renames. If the build fails, HTDOCS is removed:
+# the fence serves the packaged htdocs when the writable copy is missing, so
+# a failure serves the page with '@APP_NAME@' in it rather than an old one.
 
 INITHOOKS_PATH="${INITHOOKS_PATH:-/usr/lib/inithooks}"
 INITFENCE_DEFAULT="${INITFENCE_DEFAULT:-/etc/default/turnkey-init-fence}"
 TURNKEY_VERSION_FILE="${TURNKEY_VERSION_FILE:-/etc/turnkey_version}"
-APT_CONF_TURNKEY="${APT_CONF_TURNKEY:-/etc/apt/apt.conf.d/01turnkey}"
 
 # shellcheck source=lib/tagid.sh
 source "$INITHOOKS_PATH/lib/tagid.sh"
 
 APP_NAME=$(tagid_app_name "$(< "$TURNKEY_VERSION_FILE")")
-VERSION=$(tagid_version "$(< "$TURNKEY_VERSION_FILE")")
-BUILD=$(tagid_build "$(< "$APT_CONF_TURNKEY")")
-
-### initfence 
 
 # shellcheck source=default/turnkey-init-fence
 source "$INITFENCE_DEFAULT"
 
-if [[ ! -d "$HTDOCS" ]]; then
-    mkdir -p "$(dirname "$HTDOCS")"
-    cp -R "$INITHOOKS_PATH/turnkey-init-fence/htdocs" "$(dirname "$HTDOCS")"
-fi
+PACKAGED_HTDOCS="$INITHOOKS_PATH/turnkey-init-fence/htdocs"
+HTDOCS="${HTDOCS:?HTDOCS is not set in $INITFENCE_DEFAULT}"
+PARENT=$(dirname "$HTDOCS")
+NEW=
+OLD=
 
-INITFENCE_INDEX="$HTDOCS/index.html"
-sed -i "s|@APP_NAME@|$APP_NAME|" $INITFENCE_INDEX
+# install_htdocs
+# Builds the pages in NEW beside HTDOCS and swaps them in. Returns non-zero
+# at the first step that fails, with HTDOCS either untouched or gone, never
+# half written.
+install_htdocs() {
+    mkdir -p "$PARENT" || return
+    NEW=$(mktemp -d "$PARENT/.htdocs.new.XXXXXX") || return
+    OLD="$PARENT/.htdocs.old.${NEW##*.}"
+    cp -R "$PACKAGED_HTDOCS/." "$NEW/" || return
+    sed -i "s|@APP_NAME@|$APP_NAME|" "$NEW/index.html" || return
+    chmod 0755 "$NEW" || return
+    if [[ -e "$HTDOCS" ]]; then
+        mv -T "$HTDOCS" "$OLD" || return
+    fi
+    mv -T "$NEW" "$HTDOCS" || return
+    rm -rf "$OLD"
+}
 
-if ! tagid_is_tagged "$INITFENCE_INDEX"; then
-    tagid_render_scripts "$BUILD" "$VERSION" "$APP_NAME" >> "$INITFENCE_INDEX"
+if ! install_htdocs; then
+    echo "<3>cannot render the fence pages; serving the packaged ones" >&2
+    [[ -z "$NEW" ]] || rm -rf "$NEW"
+    [[ -z "$OLD" ]] || rm -rf "$OLD"
+    rm -rf "$HTDOCS"
+    exit 1
 fi
 
-# reload init-fence to pick up the updated index (if it is not running there
-# is nothing to reload; it reads the index when it starts)
+# reload init-fence to pick up the updated pages (if it is not running there
+# is nothing to reload; it reads them when it starts)
 if systemctl is-active --quiet turnkey-init-fence; then
     systemctl reload turnkey-init-fence
 fi

lib/tagid.sh

@@ -1,43 +1,11 @@
-# Parsing and rendering for firstboot.d/29tagid.
+# Parsing for firstboot.d/29tagid.
 #
 # Sourced by the hook and by tests/test-tagid.bats. Every function here only
-# reads its arguments and prints; the hook applies the results.
+# reads its arguments or stdin and prints; the hook applies the results.
 
 # tagid_app_name TURNKEY_VERSION
 # Prints the appliance name of a turnkey_version string, e.g. core for
 # turnkey-core-19.0-trixie-amd64.
 tagid_app_name() {
     perl -pe 's/^turnkey-//; s/-[^-]+(-[^-]+){2}$//' <<< "$1"
 }
-
-# tagid_version TURNKEY_VERSION
-# Prints the version of a turnkey_version string, e.g. 19.0-trixie-amd64.
-tagid_version() {
-    perl -pe 's/.*-([^-]+(-[^-]+){2})$/\1/' <<< "$1"
-}
-
-# tagid_build APT_CONF_LINE
-# Prints the build tags found after the version inside the parentheses of
-# the apt User-Agent line (01turnkey), sorted, unique and joined with '-';
-# iso when there is none.
-tagid_build() {
-    perl -ne 'chomp; s/.*\((.*)\).*/\1/; s/^\S+ ?//; $tags=$_ ? $_ : "iso"; system("echo $tags | xargs -n 1 | sort -u | xargs echo | sed \"s/ /-/g\"");' <<< "$1"
-}
-
-# tagid_is_tagged INDEX
-# Succeeds when the fence index page already loads the initfence scripts.
-tagid_is_tagged() {
-    grep -q ajax.turnkeylinux.org "$1"
-}
-
-# tagid_render_scripts BUILD VERSION APP_NAME
-# Prints the script tags appended to the fence index page.
-tagid_render_scripts() {
-    local build=$1
-    local version=$2
-    local app_name=$3
-    cat <<EOT
-<script src="https://ajax.turnkeylinux.org/initfence/$build/$version/$app_name.js" async></script>
-<script src="https://ajax.turnkeylinux.org/initfence/$build/$version/$app_name.direct" async></script>
-EOT
-}

tests/remote_loads.py

@@ -0,0 +1,133 @@
+#!/usr/bin/python3
+# Copyright (c) 2026 TurnKey GNU/Linux <admin@turnkeylinux.org>
+"""Print every URL on another host that an HTML page fetches on its own
+
+    remote_loads.py PAGE
+
+Exit status: 0 the page fetches nothing from another host, 1 it does (each
+URL printed on a line of its own), 2 the page could not be read.
+
+"On its own" is what a browser does while it renders the page, before
+anybody clicks: scripts, stylesheets and every other link element, images
+and srcsets, frames, objects, media and their posters and tracks, a base
+URL, a meta refresh, and url() and @import in style attributes and style
+elements. A link (<a>, <area>) and a form action are somewhere the operator
+may choose to go, and are not reported. Inline script is not parsed.
+
+"Another host" is any URL with a scheme other than data, or with no scheme
+and two leading slashes. A browser strips tabs and newlines from a URL and
+reads a backslash as a slash, and so does this, so neither is a way past
+it. When in doubt, a URL is reported.
+
+It is a test instrument, not a defence: tests/test-tagid.bats asks it about
+every page the fence serves. It is built on Python's HTMLParser, which does
+not parse malformed markup the way a browser does (an abruptly closed
+comment such as '<!-->' hides what follows from it, and not from a
+browser), so it says what a well formed page loads, and the hook does not
+rely on it: firstboot.d/29tagid serves the packaged page and nothing else.
+"""
+
+import re
+import sys
+from html.parser import HTMLParser
+
+# attributes that make the element fetch, whatever the element is
+FETCHING = {"src", "srcset", "poster", "data", "background", "lowsrc"}
+# href fetches on every element but these, which navigate on a click
+NAVIGATING = {"a", "area"}
+# a scheme that does not reach a host
+LOCAL_SCHEMES = {"data", "about", "blob", "javascript"}
+
+CSS_URL = re.compile(r"""url\(\s*(['"]?)(.*?)\1\s*\)""", re.I | re.S)
+CSS_IMPORT = re.compile(r"""@import\s+(['"])(.*?)\1""", re.I | re.S)
+REFRESH_URL = re.compile(r"""^\s*\d*\s*[;,]?\s*url\s*=\s*['"]?([^'"]*)""",
+                         re.I)
+SCHEME = re.compile(r"^([a-z][a-z0-9+.-]*):", re.I)
+
+
+def is_remote(url: str) -> bool:
+    """Whether a browser would reach another host for URL"""
+    url = re.sub(r"[\t\n\r]", "", url).strip().replace("\\", "/")
+    if url.startswith("//"):
+        return True
+    match = SCHEME.match(url)
+    return bool(match) and match.group(1).lower() not in LOCAL_SCHEMES
+
+
+def srcset_urls(value: str) -> list:
+    """The URLs of a srcset: each candidate is a URL and a descriptor"""
+    return [part.split()[0] for part in value.split(",") if part.split()]
+
+
+def css_urls(text: str) -> list:
+    """The URLs a stylesheet or a style attribute fetches"""
+    return ([m.group(2) for m in CSS_URL.finditer(text)]
+            + [m.group(2) for m in CSS_IMPORT.finditer(text)])
+
+
+class _Finder(HTMLParser):
+    def __init__(self):
+        super().__init__(convert_charrefs=True)
+        self.found = []
+        self._in_style = False
+
+    def _check(self, urls):
+        self.found.extend(url for url in urls if is_remote(url))
+
+    def handle_starttag(self, tag, attrs):
+        tag = tag.lower()
+        values = {}
+        for name, value in attrs:
+            values[name.lower()] = value or ""
+        for name, value in values.items():
+            if name in FETCHING:
+                self._check(srcset_urls(value) if name == "srcset"
+                            else [value])
+            elif name in ("href", "xlink:href") and tag not in NAVIGATING:
+                self._check([value])
+            elif name == "style":
+                self._check(css_urls(value))
+        if tag == "meta" and values.get("http-equiv", "").lower() == "refresh":
+            match = REFRESH_URL.match(values.get("content", ""))
+            if match:
+                self._check([match.group(1)])
+        if tag == "style":
+            self._in_style = True
+
+    handle_startendtag = handle_starttag
+
+    def handle_endtag(self, tag):
+        if tag.lower() == "style":
+            self._in_style = False
+
+    def handle_data(self, data):
+        if self._in_style:
+            self._check(css_urls(data))
+
+
+def remote_loads(page: str) -> list:
+    """Every URL on another host PAGE fetches on its own, in page order"""
+    finder = _Finder()
+    finder.feed(page)
+    finder.close()
+    return [url.strip() for url in finder.found]
+
+
+def main(argv: list) -> int:
+    if len(argv) != 1:
+        print("usage: remote_loads.py PAGE", file=sys.stderr)
+        return 2
+    try:
+        with open(argv[0], "rb") as fob:
+            page = fob.read().decode("utf-8", errors="replace")
+    except OSError as exc:
+        print(f"remote_loads.py: {exc}", file=sys.stderr)
+        return 2
+    found = remote_loads(page)
+    for url in found:
+        print(url)
+    return 1 if found else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv[1:]))