Serve a first boot page that loads nothing from a third party

[marcos-mendez]

The page the appliance serves on first boot, before anybody has logged in,
loaded three scripts from two other hosts. It now loads none.

Closes #13.

What was there

Measured on the wordpress-demo container, 2026-09-28, on the page that is
actually served (/var/lib/inithooks/turnkey-init-fence/htdocs/index.html):

74:<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
87:<script src="https://ajax.turnkeylinux.org/initfence/iso/19.0-trixie-amd64/wordpress.js" async></script>
88:<script src="https://ajax.turnkeylinux.org/initfence/iso/19.0-trixie-amd64/wordpress.direct" async></script>

Two are the ones the issue names, appended by 29tagid at first boot. The
third is not in the issue: a jQuery from ajax.googleapis.com, in the
packaged page itself, that nothing on the page uses: the inline script
below it is plain document.getElementById. It was there to support the
remote script. That is exactly why the test parses the page for every host
it would fetch from instead of grepping for a hostname: a hostname grep is
what let this one sit next to the two that were noticed.

What the tag was for, established before removing it

The issue asked for this, so it was settled first rather than after.

Introduced in 2015 (52e7c8e, "include script support for AJAX updates")
and extracted into 29tagid a week later (dc9fd92):

• the .js let TurnKey change the fence page's content after an image
  had shipped, without shipping anything;
• the .direct is a beacon. It has no content to return; requesting it
  is the point. It told TurnKey the fence page had been hit directly, broken
  down by build, version and appliance: which is the same triple the URL
  path announces.

The same pair was once applied to the web control panel's index as well
(webcp/$build/$version/$app.js); that half was deleted in 2016
(50101fc).

Both endpoints answer 404 today:

$ curl -sS -w 'code=%{http_code}\n' -o /dev/null \
    https://ajax.turnkeylinux.org/initfence/iso/19.0-trixie-amd64/core.js
code=404
$ curl ... core.direct    -> code=404
$ curl ... wordpress.js   -> code=404

So what every Keel appliance shipped was a beacon to a service that no
longer exists, and a page that is unverifiable by construction: keel verify can attest to every byte in the layer and say nothing about a
script fetched at view time.

What depended on it

Only itself, which is what the issue asked to establish up front.

• tagid_version and tagid_build existed solely to fill in those
  URLs. Both are gone, with tagid_is_tagged and tagid_render_scripts.
• tagid_build was the only reader of /etc/apt/apt.conf.d/01turnkey
  in this repository: it parsed the build tags out of the apt User-Agent.
  The appliance still resolves apt, security and its vendor through TurnKey common#6 removes that file, so the two changes fit together;
  a test asserts the hook is fine without it.
• bin/secalerts.sh reads the same file, independently, for the version
  it POSTs to hub.turnkeylinux.org. It is guarded
  ([ -e "$f" ] && ...) and falls back to turnkey-version. Measured that
  it survives the file's removal and does not trip its own set -e. It is
  untouched here; that it talks to the Hub at all is a separate subject.
• tagid_app_name stays. The @APP_NAME@ substitution is the hook's
  other job and is not the tag. 29tagid still does it and still reloads
  the fence.

Why this is not a pure deletion

HTDOCS is /var/lib/inithooks/turnkey-init-fence/htdocs, and /var
survives. 29tagid used to copy the packaged pages only when that directory
did not exist, so an index tagged by an older image stays tagged, and this
hook is the only thing that ever looks at it again. The measurement above is
that page.

The first version of this branch stripped remote <script> elements from
it with a regular expression. Review measured an unquoted src, a
stylesheet, an image and a frame getting past it. So 29tagid no longer
judges the page at all: on every run, under turnkey-init too, it rebuilds
the /var copy from the packaged pages beside the served directory, names
the appliance in it, and swaps it in. A page the package did not ship is
not served, however it is written. Nothing in the tree writes into that
copy but this hook. If the rebuild fails, the copy is removed and the fence
serves the packaged pages (fence_htdocs), and the hook exits 1 with a
<3> message. The commit message has the three part justification for the
change of approach.

How it is asserted

Not by reading the hook's source, and not by grepping for
ajax.turnkeylinux.org. Every page the fence serves is judged by two
detectors that share no parser: tests/remote_loads.py (HTMLParser, every
attribute and style construct that fetches, 48 cases of its own) and a tag
level regular expression that knows no HTML and catches what HTMLParser
misses on malformed markup. Each hostile page the review measured, and two
parser differentials, are served as inherited pages and must be gone.

Tests

tests/test-tagid.bats: 27 tests; against the first version of the branch,
8 fail. 125 bats in all, 29tagid 29/29, lib/tagid.sh 1/1, total 99.55;
Python 196 passed, 3 skipped.

Changelog: 2.3.6+keel8, after #19 (keel6) and #18 (keel7).

Noticed, not changed

The page still carries <a href> links to www.turnkeylinux.org (the
appliance product page and the contact page) and calls the appliance a
TurnKey one in its prose. Those are branding and vocabulary, not a script
and not a fetch (nothing is requested until an operator clicks), so they
are out of this issue's scope and left for the vocabulary work.

Test plan

• tests/coverage.sh: six files, both touched ones at 100 percent
• pytest: 187 passed, 3 skipped, unchanged
• shellcheck -S warning on the hook and the library (the library's
  SC2148 is pre-existing and matches lib/ipconfig.sh and
  lib/init-fence.sh, which are sourced the same way)
• the hook run against a scratch tree and the rendered page inspected
• both ajax.turnkeylinux.org endpoints probed, 404
• bin/secalerts.sh checked against a missing 01turnkey
• every assertion mutation-checked
• a built appliance booted to see the served page. Not run: the build
  host's core.tar.zst is in a known-inconsistent state and a repair is
  pending, so the build lock was deliberately not taken. The measurement
  above was taken read-only with lxc-attach.

Fits with Keel-Linux/common#6, which removes the 01turnkey this hook
stopped reading. Org-wide context: Keel-Linux/tracker#15.

[marcos-mendez]

Review of fix/no-third-party-script. Findings ordered by severity, each says whether I got it by running something or by reading. No CRITICAL and no HIGH.

Checked first, because the branch rests on them

• Both endpoints are gone. core.js, core.direct, wordpress.js and wordpress.direct under https://ajax.turnkeylinux.org/initfence/iso/19.0-trixie-amd64/ all answer 404. Confirmed.

• The strip cannot truncate the page. This is the claim I most wanted to fail, because docs/traps.md has a gpg entry that is exactly this shape, and perl does not check write errors on many paths. It holds. Driven against /dev/full:

  $ tagid_strip_remote_scripts < big.html > /dev/full
  -p destination: No space left on device      exit=28
  $ printf '...' | tagid_strip_remote_scripts > /dev/full
  Unable to flush stdout: No space left on device   exit=1
  

  perl reports a short write and exits non-zero on both the streaming path and the flush path, so firstboot.d/29tagid:37-38 takes the else branch, removes the temp file and leaves the index untouched. The redirection also targets $INITFENCE_INDEX.tmp, never the index, so even a failure before perl starts cannot touch it. Test 9 covers the perl-exits-1 case; the short-write case is the one that would have bitten and it behaves the same.

• The survival-across-upgrade reasoning is right. HTDOCS is /var/lib/inithooks/turnkey-init-fence/htdocs (default/turnkey-init-fence), 29tagid:22-25 copies the packaged page only when that directory is absent, and fab-apply-overlay only ever adds files. So an index tagged by an older image does survive and this hook is the only thing that looks at it again. Deleting the generator alone would not have been enough.

• bin/secalerts.sh survives. Lines 53-55 are [ -e "$f" ] && turnkey_version=$(...) followed by [ -n "$turnkey_version" ] || turnkey_version=$(turnkey-version). Under #!/bin/bash -e a failing && list does not terminate the script — confirmed by running the shape — so with 01turnkey gone it falls through to turnkey-version, which is still installed. It is guarded and it does survive.

• 27/27 of tests/test-tagid.bats pass here on bats 1.11.1, and the suite uses run ! throughout, so its refutations really do refute.

MEDIUM — lib/tagid.sh:25 and tests/test-tagid.bats:56-64: the stripper and the detector share a blind spot, so the test cannot catch the stripper's gap

The strength claimed for the test is that it parses the page for every host it would fetch from rather than grepping a hostname — which is what caught the jQuery. Two things narrow that in practice, and they narrow it in the same direction, so nothing fails.

Driving the function directly:

<script src="https://evil.example/x.js"></script>      -> stripped
<script src='https://evil.example/x.js'></script>      -> stripped
<script src=https://evil.example/x.js></script>        -> KEPT
<link rel="stylesheet" href="https://evil.example/x.css">  -> KEPT
<img src="https://evil.example/x.png">                 -> KEPT
<iframe src="https://evil.example/"></iframe>          -> KEPT

The <link>, <img> and <iframe> cases are by design — the function is named for scripts. But loaded_hosts() at line 56 is what asserts the broader property, and it matches only double-quoted attributes ((src|href)[[:space:]]*=[[:space:]]*"[^"]+"). So the one form the stripper misses is also the one form the detector cannot see, and the three element types the stripper does not handle are only ever tested on pages that do not contain them: the inherited-page test at line 96 uses script_hosts, not loaded_hosts, and test 2 runs on a fresh tree.

The scenario: an image whose /var index came from a build that carried a remote <link rel="stylesheet"> or a remote <img> — a rebranded fence page, a downstream edit, or simply the next thing somebody appends to that page for the same reason TurnKey appended the scripts. It survives the strip, it serves from the fence before anybody has logged in, and the suite stays green. The fix is small in both places: widen the element set the strip handles, or rename the function and say plainly that scripts are the only class defended; and make loaded_hosts() accept '...' and bare attribute values so the detector is strictly wider than the defence rather than exactly as narrow.

Nothing that has ever actually been written to that page slips through — the two generated tags and the packaged jQuery were all double-quoted <script src>, and I checked all three are stripped. That is why this is MEDIUM and not higher.

LOW — lib/tagid.sh:25: a remote script with a body leaves its body behind as page text

<script src="https://evil.example/x.js">var a=1;</script>
-> var a=1;</script>

The optional </script> in the pattern only matches when it follows the opening tag immediately. The fetch is removed, which is the point, so this is cosmetic — but the residue lands in the rendered page as visible text and an orphan closing tag, on the first thing an operator sees.

LOW — firstboot.d/29tagid:6: the strip never runs under turnkey-init

[ -n "$_TURNKEY_INIT" ] && exit 0 is inherited and correct for the @APP_NAME@ half. It now also means the new defence is skipped on exactly the path an operator takes when the first boot did not complete — and docs/traps.md entry 1 is a first boot that hangs forever, hit three times in this project. On that machine the fence is up, the index in /var is the tagged one, and running turnkey-init does not strip it. Worth either moving the strip above the early exit or saying in the comment why it does not need to be.

LOW — firstboot.d/29tagid:35: the temporary file is written inside the directory the fence serves

$HTDOCS/index.html.tmp is under HTDOCS, and bin/simplehttpd.py serves that directory statically. During the write, and permanently if the hook is killed between the redirect and the mv, /index.html.tmp is fetchable. The content is the stripped page so nothing leaks, and rm -f covers the ordinary failure — a kill is what it does not cover. mktemp in the parent directory, or a name the server will not serve, costs nothing.

LOW — an already-initialised machine keeps its tagged page

The strip runs from a first-boot hook, so a machine that has already completed init and is then upgraded to 2.3.6+keel6 keeps the two <script src> tags in /var for good. Impact is small — the fence is disabled by 97turnkey-init-fence-disable and both endpoints 404 — but "the first boot page carries no third party script" in debian/changelog is true of new machines only, and the changelog is where someone will look for the answer.

Noted, not a finding

• Establishing what the tag was for before removing it, and finding the jQuery the issue never named, is the part of this that could not have been done by grepping. The .direct beacon reading is right: no content to return, the request is the payload, and the URL path carries build, version and appliance.
• The <a href> links to www.turnkeylinux.org left in the page are correctly out of scope — I confirmed nothing is requested until a click.
• tests/test-tagid.bats uses run ! correctly throughout. The rest of this repository does not: 20 bare negations in other suites assert nothing, 17 of them in tests/test-ipconfig.bats, where 11 of the 13 rejection cases of ipconfig_ip6_syntax are inert — an IPv6 validator whose negative tests do not run, in a project with a docs/traps.md entry about IPv6 literals. Out of scope here, list handed over separately, wants its own issue.

Approve. The removal is complete for everything that page has ever carried, the inherited-copy problem was found rather than assumed, and the temp-file path holds up under the failure mode that would actually have caused the trap it cites. The MEDIUM is that the guarantee stated is wider than the regex delivers and the test cannot tell the difference; I would widen the detector at least, so the next remote element added to that page fails something.

[marcos-mendez]

Review addressed in 7ab1826 and e314934, then master merged in (c3371c8). 29tagid no longer strips anything. It rebuilds the /var copy from the packaged pages next to the served directory, runs under turnkey-init as well, and falls back to the packaged pages if it fails. The commit message has the justification. Two detectors that share no parser now judge the page. The changelog is now keel7, so this lands after #18.