diff --git a/.gitignore b/.gitignore index 7bb91f7..2b91c59 100644 --- a/.gitignore +++ b/.gitignore @@ -24,3 +24,4 @@ tmp tags *.tar.bz2 .coverage +coverage/ diff --git a/COVERAGE.md b/COVERAGE.md index 91c5e51..cac243a 100644 --- a/COVERAGE.md +++ b/COVERAGE.md @@ -4,6 +4,42 @@ Measured on 2026-09-24 against upstream master (33c43b8), following the project decision 0003 (90 percent floor per repository, 95 percent for every file our changes touch). +## Branch fix/fence-no-silent-skip: shell 99.57 percent (2026-09-28) + +`lib/init-fence.sh` gains `fence_close_port` and `fence_open_port`, and +`iptables_add_redirect` refuses a port it cannot redirect instead of leaving +it open. `tests/test-init-fence.bats` grows from 27 to 31 tests (116 bats +over the six files): the refusal for one family and for both, the insertion +first in the INPUT chain so it precedes the appliance's own ACCEPT rules, +the removal loop, the fatal path when a port can be neither redirected nor +refused, and the same path through the script, which is the one that proves +the library stops its caller rather than dying invisibly. + +Measured with kcov 43: `bin/turnkey-init-fence` 28/28, `lib/init-fence.sh` +78/79 (98.73), `firstboot.d/01ipconfig` 29/29, `lib/ipconfig.sh` 72/72, +`firstboot.d/29tagid` 19/19, `lib/tagid.sh` 8/8, total 99.57. + +The one uncovered line of `lib/init-fence.sh` is line 154, the first line of +the multi-line `simplehttpd.py` invocation, which kcov attributes to a later +line. It is the same line that was uncovered before this branch, and it is +not part of this change. Every line this branch adds is covered. + +The shell gate stays at 98, the lowest file rounded down; 98.73 clears it +and decision 0003's 95 percent bar for a file a change touches. + +After review (2026-09-29): the stubs say which commands the fence issues, +not what they do to traffic, so `tests/test-init-fence-netfilter.bats` +(8 tests) asks the real netfilter. `tests/netns-sandbox` runs the library +against real iptables and ip6tables in a network namespace of its own, with +a client namespace on the other end of a veth pair, and connects to the +ports over IPv6, IPv4 and loopback; the only stand-in is the answer +"there is no nat table" (or "the filter table refuses the insert") for one +family. It found that the default ICMP refusal left an IPv6 client waiting +for its timeout, that the refusal also refused loopback, and that a +restore of the appliance firewall's own rules removes the whole fence. +125 bats in all; `lib/init-fence.sh` 83/84 (98.81), total 99.58, the same +one line uncovered. + ## bin/keel-host-keys (2026-09-29, keel-core#8) `tests/test-host-keys.bats`, 22 tests, measures `bin/keel-host-keys` at diff --git a/debian/changelog b/debian/changelog index 45cdcf4..8ff213b 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,32 @@ +inithooks (2.3.6+keel7) trixie; urgency=high + + * lib/init-fence.sh: the fence no longer reports a fence it did not put up. + A family whose nat table cannot be used, which is the normal case in an + LXC container whose host has not loaded (ip6)table_nat, had its REDIRECT + rule skipped with a warning, and the service then started and reported + success with the application ports reachable over that family. The port + is now refused in the filter table instead, inserted ahead of the rules + the appliance firewall accepts, so what the fence promises holds by + another means: the fence page stays reachable on its own ports, as it + became in 2.3.6+keel1, and the application behind the fenced port does + not answer. The refusal is a TCP reset, because an IPv6 client given + the default ICMP error waits for its timeout instead of failing, and it + does not apply to loopback, which a REDIRECT never did either. + * lib/init-fence.sh: a port that can be neither redirected nor refused is + fatal, with a message, because then nothing is fencing it. Fatal means + the unit fails and its stop-post removes every rule the fence + installed, so the application answers on every port again; it does not + mean the port is left closed. + * lib/init-fence.sh: stopping removes the refusals with the redirects, so + a stopped or failed fence leaves no port closed behind it. + * turnkey-init-fence.service: ordered after netfilter-persistent and + Webmin's webmin-iptables and webmin-ip6tables, which restore a saved + firewall at boot. A restore flushes the tables it names and every fence + rule in them, redirects and refusals alike; ordering covers boot, and a + restore while the fence is up still unfences every port. + + -- Marcos Mendez Tue, 29 Sep 2026 04:34:01 +0000 + inithooks (2.3.6+keel6) trixie; urgency=high * keel-host-keys.service: generate the SSH host keys, the TLS pair in diff --git a/debian/inithooks.turnkey-init-fence.service b/debian/inithooks.turnkey-init-fence.service index 137b08c..f2550e2 100644 --- a/debian/inithooks.turnkey-init-fence.service +++ b/debian/inithooks.turnkey-init-fence.service @@ -2,6 +2,8 @@ Description=TurnKey Initialization web interface fence After=network.target network-online.target After=iptables.service firewalld.service ip6tables.service ipset.service nftables.service +# a restore of a saved firewall flushes the fence rules: start after them +After=netfilter-persistent.service webmin-iptables.service webmin-ip6tables.service Before=apache2.service nginx.service lighttpd.service tomcat10.service tomcat11.service [Service] diff --git a/lib/init-fence.sh b/lib/init-fence.sh index 1353cd4..d90ea8d 100644 --- a/lib/init-fence.sh +++ b/lib/init-fence.sh @@ -25,9 +25,71 @@ fence_nat_available() { "$1" -t nat -n -L PREROUTING >/dev/null 2>&1 } +# fence_refusal DPORT +# The rule fence_close_port inserts and fence_open_port deletes, after the +# chain name. Traffic that arrives on loopback is left alone, because a +# REDIRECT in PREROUTING never applied to locally originated traffic, and a +# refused port must behave like a redirected one for anything on the +# appliance itself. tcp-reset, not the default ICMP error: measured in a +# network namespace, an IPv6 client that gets icmp6-port-unreachable for +# its SYN keeps retrying until it times out, so the port looked silent +# rather than refused. +fence_refusal() { + echo "! -i lo -p tcp -m tcp --dport $1 -j REJECT --reject-with tcp-reset" +} + +# fence_close_port IPT DPORT +# Refuses DPORT on IPT's filter INPUT chain, inserted first so that it comes +# before whatever the appliance firewall accepts. This is what the fence +# does for a port it cannot redirect: the fence page is not reachable on +# that port over that family, and neither is the application standing behind +# it. +# +# First in the chain as it stands when the fence starts. A restore of a +# saved firewall (iptables-restore of /etc/iptables/rules.v4 or rules.v6, +# which is what Webmin's Firewall module applies) flushes the tables it +# names, and takes this rule with it, as it takes the REDIRECT rules in the +# nat table. The unit is ordered after the services that restore one at +# boot; a restore while the fence is up unfences every port, and nothing +# here can prevent it. +# +# Fails, with a message, when the port can be neither redirected nor +# refused, because then nothing is fencing it and the caller must not report +# that anything is. What follows a failure is not a closed port: the start +# fails, the unit's stop-post removes every rule the fence installed, and +# the application answers on every port again, with a failed unit to say so. +fence_close_port() { + local ipt=$1 + local dport=$2 + local -a rule + read -r -a rule <<< "$(fence_refusal "$dport")" + fence_open_port "$ipt" "$dport" + if ! "$ipt" -t filter -I INPUT 1 "${rule[@]}"; then + echo "<3>$ipt cannot refuse $dport either, nothing is fencing it" >&2 + return 1 + fi + echo "Refusing port $dport on $ipt for as long as the fence is up" +} + +# fence_open_port IPT DPORT +# Removes every refusal fence_close_port left behind for DPORT. Stopping has +# to succeed, so this never fails: a family that cannot be asked at all ends +# the loop exactly as a family with nothing left to remove does, and the +# caller cannot tell the two apart. +fence_open_port() { + local ipt=$1 + local dport=$2 + local -a rule + read -r -a rule <<< "$(fence_refusal "$dport")" + while "$ipt" -t filter -D INPUT "${rule[@]}" 2>/dev/null; do + : + done +} + iptables_delete_redirect() { local dport=$1 local to_port=$2 + local ipt echo "Removing REDIRECT firewall rule: $dport => $to_port" while iptables -t nat -D PREROUTING -p tcp --dport "$dport" \ -j REDIRECT --to-port "$to_port" 2>/dev/null; do @@ -37,6 +99,9 @@ iptables_delete_redirect() { -j REDIRECT --to-port "$to_port" 2>/dev/null; do : done + for ipt in iptables ip6tables; do + fence_open_port "$ipt" "$dport" + done } iptables_add_redirect() { @@ -46,10 +111,13 @@ iptables_add_redirect() { echo "Adding REDIRECT firewall rule: $dport => $to_port" iptables_delete_redirect "$dport" "$to_port" for ipt in iptables ip6tables; do - # skip a family whose nat table can't be used rather than failing, - # the fence is still reachable on $to_port + # a family whose nat table can't be used gets the port refused + # instead of redirected. Skipping it was worse than doing nothing: + # the caller went on to report a fence that was not there, over the + # family an LXC container is most likely to be reachable on. if ! fence_nat_available "$ipt"; then - echo "WARNING: $ipt nat table unavailable, skipping REDIRECT rule: $dport => $to_port" >&2 + echo "<4>$ipt nat table unavailable, refusing $dport rather than redirecting it to $to_port" >&2 + fence_close_port "$ipt" "$dport" || return 1 continue fi $ipt -t nat -A PREROUTING -p tcp --dport "$dport" -j REDIRECT --to-port "$to_port" diff --git a/tests/fixtures/webmin-fw.rules.v4 b/tests/fixtures/webmin-fw.rules.v4 new file mode 100644 index 0000000..40fae51 --- /dev/null +++ b/tests/fixtures/webmin-fw.rules.v4 @@ -0,0 +1,24 @@ +*nat +:PREROUTING ACCEPT [0:0] +:POSTROUTING ACCEPT [0:0] +:OUTPUT ACCEPT [0:0] +COMMIT +*mangle +:PREROUTING ACCEPT [0:0] +:INPUT ACCEPT [0:0] +:FORWARD ACCEPT [0:0] +:OUTPUT ACCEPT [0:0] +:POSTROUTING ACCEPT [0:0] +COMMIT +*filter +:FORWARD ACCEPT [0:0] +:INPUT DROP [0:0] +:OUTPUT ACCEPT [0:0] +-A INPUT -i lo -j ACCEPT +-A INPUT -p icmp -m icmp --icmp-type echo-request -j ACCEPT +-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT +-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT +-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT +-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT +-A INPUT -p tcp -m tcp --dport 12321 -j ACCEPT +COMMIT diff --git a/tests/fixtures/webmin-fw.rules.v6 b/tests/fixtures/webmin-fw.rules.v6 new file mode 100644 index 0000000..664f46f --- /dev/null +++ b/tests/fixtures/webmin-fw.rules.v6 @@ -0,0 +1,24 @@ +*nat +:PREROUTING ACCEPT [0:0] +:POSTROUTING ACCEPT [0:0] +:OUTPUT ACCEPT [0:0] +COMMIT +*mangle +:PREROUTING ACCEPT [0:0] +:INPUT ACCEPT [0:0] +:FORWARD ACCEPT [0:0] +:OUTPUT ACCEPT [0:0] +:POSTROUTING ACCEPT [0:0] +COMMIT +*filter +:FORWARD ACCEPT [0:0] +:INPUT DROP [0:0] +:OUTPUT ACCEPT [0:0] +-A INPUT -i lo -j ACCEPT +-A INPUT -p ipv6-icmp -j ACCEPT +-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT +-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT +-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT +-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT +-A INPUT -p tcp -m tcp --dport 12321 -j ACCEPT +COMMIT diff --git a/tests/netns-sandbox b/tests/netns-sandbox new file mode 100755 index 0000000..186c202 --- /dev/null +++ b/tests/netns-sandbox @@ -0,0 +1,196 @@ +#!/bin/bash +# Runs a test body against the real netfilter, in a network namespace of +# its own. +# +# tests/netns-sandbox [NO_NAT [NO_FILTER]] < BODY +# +# Re-executes itself in a new network and mount namespace: an unprivileged +# user namespace where the kernel lets one mount (Debian), otherwise +# 'sudo -n unshare' where sudo needs no password (the Ubuntu CI runner, +# whose AppArmor policy leaves an unprivileged user namespace without the +# right to mount). Either way iptables and ip6tables are the real ones, +# acting on a network namespace nobody else uses. A second namespace plays +# a client on the other end of a veth pair, over IPv6 (fd00:fe::/64) and +# IPv4 (10.254.0.0/30), so traffic reaches the appliance on an interface +# that is not loopback, the way it does in production. +# +# A listener answers "app" on the ports the fence stands in front of (80, +# 443, 12321, 12322) and "fence" on the fence's own (60080, 60443). BODY is +# bash, sourced with lib/init-fence.sh and these loaded: +# +# probe FAMILY PORT from the client: prints app, fence, refused or +# silent (FAMILY is 6 or 4) +# probe_local FAMILY PORT +# the same from the appliance itself, over loopback +# FIXTURES tests/fixtures +# +# NO_NAT (iptables, ip6tables, both, or none) names a family whose nat +# table is made unavailable, the way it is in a container whose host has not +# loaded (ip6)table_nat: that command answers the nat table with an error, +# and everything else goes to the real binary. NO_FILTER does the same for +# inserting a rule into the filter INPUT chain, which is how the fence fails +# when it can neither redirect a port nor refuse it. Those two answers are +# the only ones stood in for; what the rules do to traffic is always real. +# +# Exits 77 when no namespace can be made here, so that a caller can tell +# "cannot ask" from any answer. + +set -euo pipefail + +here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +REPO=$(cd "$here/.." && pwd) + +if [[ -z "${NETNS_SANDBOX_INSIDE:-}" ]]; then + probe_ns='mount -t tmpfs none /mnt && ip link set lo up' + if unshare --user --map-root-user --net --mount -- \ + /bin/sh -c "PATH=/usr/sbin:/sbin:\$PATH; $probe_ns" 2>/dev/null; then + exec env NETNS_SANDBOX_INSIDE=1 \ + unshare --user --map-root-user --net --mount -- "$0" "$@" + elif sudo -n unshare --net --mount -- \ + /bin/sh -c "PATH=/usr/sbin:/sbin:\$PATH; $probe_ns" 2>/dev/null; then + exec sudo -n env NETNS_SANDBOX_INSIDE=1 \ + unshare --net --mount -- "$0" "$@" + fi + echo "netns-sandbox: no network and mount namespace available here" >&2 + exit 77 +fi + +no_nat=${1:-none} +no_filter=${2:-none} +PATH=/usr/sbin:/sbin:$PATH +scratch=$(mktemp -d) +pids=() +cleanup() { + local pid + for pid in "${pids[@]}"; do + kill "$pid" 2>/dev/null || true + done + rm -rf "$scratch" +} +trap cleanup EXIT + +# xtables takes its lock under /run, which an ordinary user cannot write. +mount -t tmpfs tmpfs /run + +# stand_in IPT FAIL_NAT FAIL_FILTER +# Puts a wrapper for IPT first on PATH that fails the nat table and/or the +# filter insertion as a container that lacks them does, and passes every +# other call to the real binary. +stand_in() { + local ipt=$1 + mkdir -p "$scratch/bin" + { + echo '#!/bin/bash' + if [[ "$2" == yes ]]; then + echo 'if [[ " $* " == *" -t nat "* ]]; then' + echo " echo \"$ipt: can't initialize $ipt table 'nat': Table does not exist\" >&2" + echo ' exit 3' + echo 'fi' + fi + if [[ "$3" == yes ]]; then + echo 'if [[ " $* " == *" -I INPUT "* ]]; then' + echo " echo \"$ipt: can't initialize $ipt table 'filter': Permission denied\" >&2" + echo ' exit 4' + echo 'fi' + fi + echo "exec /usr/sbin/$ipt \"\$@\"" + } > "$scratch/bin/$ipt" + chmod +x "$scratch/bin/$ipt" +} + +for ipt in iptables ip6tables; do + fail_nat=no + fail_filter=no + [[ "$no_nat" == both || "$no_nat" == "$ipt" ]] && fail_nat=yes + [[ "$no_filter" == both || "$no_filter" == "$ipt" ]] && fail_filter=yes + if [[ "$fail_nat$fail_filter" != nono ]]; then + stand_in "$ipt" "$fail_nat" "$fail_filter" + fi +done +PATH=$scratch/bin:$PATH + +ip link set lo up +unshare --net sleep infinity & +client=$! +pids+=("$client") +client_net=/proc/$client/ns/net +# the client namespace exists once sleep runs in it +for _ in $(seq 50); do + [[ "$(readlink "$client_net")" != "$(readlink /proc/self/ns/net)" ]] && break + sleep 0.02 +done +ip link add fence0 type veth peer name client0 netns "$client" +ip addr add fd00:fe::1/64 dev fence0 nodad +ip addr add 10.254.0.1/30 dev fence0 +ip link set fence0 up +nsenter --net="$client_net" ip link set lo up +nsenter --net="$client_net" ip addr add fd00:fe::2/64 dev client0 nodad +nsenter --net="$client_net" ip addr add 10.254.0.2/30 dev client0 +nsenter --net="$client_net" ip link set client0 up + +python3 - "$scratch/listening" <<'EOF' & +import selectors, socket, sys +banners = {80: b"app", 443: b"app", 12321: b"app", 12322: b"app", + 60080: b"fence", 60443: b"fence"} +sel = selectors.DefaultSelector() +for port, banner in banners.items(): + s = socket.socket(socket.AF_INET6, socket.SOCK_STREAM) + s.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 0) + s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + s.bind(("::", port)) + s.listen(16) + sel.register(s, selectors.EVENT_READ, banner) +open(sys.argv[1], "w").close() +while True: + for key, _ in sel.select(): + conn, _ = key.fileobj.accept() + conn.sendall(key.data) + conn.close() +EOF +pids+=("$!") +for _ in $(seq 100); do + [[ -e "$scratch/listening" ]] && break + sleep 0.02 +done + +_connect=' +import socket, sys +family = socket.AF_INET6 if sys.argv[1] == "6" else socket.AF_INET +s = socket.socket(family, socket.SOCK_STREAM) +s.settimeout(1) +try: + s.connect((sys.argv[2], int(sys.argv[3]))) + print(s.recv(16).decode() or "closed") +except ConnectionRefusedError: + print("refused") +except (socket.timeout, TimeoutError): + print("silent") +' + +probe() { + local family=$1 + local port=$2 + local addr=fd00:fe::1 + [[ "$family" == 6 ]] || addr=10.254.0.1 + nsenter --net="$client_net" python3 -c "$_connect" "$family" "$addr" "$port" +} + +probe_local() { + local family=$1 + local port=$2 + local addr=::1 + [[ "$family" == 6 ]] || addr=127.0.0.1 + python3 -c "$_connect" "$family" "$addr" "$port" +} + +# shellcheck disable=SC2034 # read by BODY +FIXTURES=$here/fixtures +HTTP_PORTS=(80) +HTTPS_PORTS=(443 12321 12322) +HTTP_FENCE_PORT=60080 +HTTPS_FENCE_PORT=60443 +# shellcheck source=lib/init-fence.sh +source "$REPO/lib/init-fence.sh" + +# shellcheck source=/dev/null +source /dev/stdin diff --git a/tests/test-init-fence-netfilter.bats b/tests/test-init-fence-netfilter.bats new file mode 100644 index 0000000..0add3e5 --- /dev/null +++ b/tests/test-init-fence-netfilter.bats @@ -0,0 +1,134 @@ +#!/usr/bin/env bats +# What the init fence does to traffic, asked of the real netfilter. +# +# tests/test-init-fence.bats drives lib/init-fence.sh against stubs and +# asserts the commands it issues. This file runs the same functions against +# real iptables and ip6tables in a network namespace of its own +# (tests/netns-sandbox) and connects to the ports from a client on the other +# end of a veth pair, over IPv6 and IPv4, and from the appliance itself over +# loopback. The answer is what a connection gets: the application, the +# fence page, a refusal, or silence. +# +# The rules the appliance firewall restores are tests/fixtures/webmin-fw.*, +# rendered from Keel-Linux/common conf/turnkey.d/webmin-fw (ae064ba) with its +# defaults. The appliance applies them with iptables-legacy and the sandbox +# with the nf_tables backend; what a restore does to a table it names, and +# what the rules do to a packet, is the same in both. + +bats_require_minimum_version 1.5.0 + +SANDBOX=$BATS_TEST_DIRNAME/netns-sandbox + +# fence [NO_NAT [NO_FILTER]] < BODY +# Runs BODY in the sandbox. A sandbox that cannot be made fails the test; +# it is never read as an answer. +fence() { + local rc=0 + "$SANDBOX" "$@" || rc=$? + if [[ $rc -eq 77 ]]; then + echo "no network namespace here: nothing was asked" >&2 + fi + return "$rc" +} + +@test "with both nat tables, every fenced port gets the fence page, over both families" { + run fence none <<'EOF' +iptables_redirect start >/dev/null +for port in 80 443 12321 12322; do + echo "$port $(probe 6 "$port") $(probe 4 "$port")" +done +EOF + [ "$status" -eq 0 ] + [ "$output" = "$(printf '%s\n' '80 fence fence' '443 fence fence' \ + '12321 fence fence' '12322 fence fence')" ] +} + +@test "without the IPv6 nat table, IPv6 is refused on every fenced port and IPv4 still gets the fence" { + run fence ip6tables <<'EOF' +iptables_redirect start >/dev/null 2>&1 +for port in 80 443 12321 12322; do + echo "$port $(probe 6 "$port") $(probe 4 "$port")" +done +EOF + [ "$status" -eq 0 ] + [ "$output" = "$(printf '%s\n' '80 refused fence' '443 refused fence' \ + '12321 refused fence' '12322 refused fence')" ] +} + +@test "without the IPv6 nat table, the fence page stays reachable on its own ports over IPv6" { + run fence ip6tables <<'EOF' +iptables_redirect start >/dev/null 2>&1 +echo "$(probe 6 60080) $(probe 6 60443)" +EOF + [ "$status" -eq 0 ] + [ "$output" = "fence fence" ] +} + +@test "a refusal comes before the appliance firewall restored ahead of the fence" { + run fence ip6tables <<'EOF' +ip6tables-restore < "$FIXTURES/webmin-fw.rules.v6" +iptables-restore < "$FIXTURES/webmin-fw.rules.v4" +echo "restored $(probe 6 443) $(probe 4 443)" +iptables_redirect start >/dev/null 2>&1 +echo "fenced $(probe 6 443) $(probe 6 12321) $(probe 4 443) $(probe 6 60443)" +EOF + [ "$status" -eq 0 ] + [ "${lines[0]}" = "restored app app" ] + [ "${lines[1]}" = "fenced refused refused fence fence" ] +} + +@test "loopback reaches the application behind a refused port, as it does behind a redirected one" { + run fence ip6tables <<'EOF' +ip6tables-restore < "$FIXTURES/webmin-fw.rules.v6" +iptables-restore < "$FIXTURES/webmin-fw.rules.v4" +iptables_redirect start >/dev/null 2>&1 +echo "$(probe_local 6 443) $(probe_local 4 443) $(probe_local 6 12321)" +EOF + [ "$status" -eq 0 ] + [ "$output" = "app app app" ] +} + +@test "stop gives every port back to the application, over both families" { + run fence ip6tables <<'EOF' +iptables_redirect start >/dev/null 2>&1 +iptables_redirect stop >/dev/null 2>&1 +for port in 80 443 12321 12322; do + echo "$port $(probe 6 "$port") $(probe 4 "$port")" +done +EOF + [ "$status" -eq 0 ] + [ "$output" = "$(printf '%s\n' '80 app app' '443 app app' \ + '12321 app app' '12322 app app')" ] +} + +# The start stops at the first port it cannot fence (80, over IPv6), as the +# script does under errexit, so the ports after it were never touched. Then +# stop-post runs 'iptables_redirect stop', which is the second half here: +# what a failed fence ends with is every port open, not closed. +@test "a port that can be neither redirected nor refused fails the start, and the cleanup leaves everything open" { + run fence ip6tables ip6tables <<'EOF' +set +e +(set -e; iptables_redirect start >/dev/null 2>&1) +echo "start $?" +echo "failed $(probe 6 80) $(probe 4 80) $(probe 6 443) $(probe 4 443)" +iptables_redirect stop >/dev/null 2>&1 +echo "cleaned $(probe 6 80) $(probe 4 80) $(probe 6 443) $(probe 4 443)" +EOF + [ "$status" -eq 0 ] + [ "${lines[0]}" = "start 1" ] + [ "${lines[1]}" = "failed app fence app app" ] + [ "${lines[2]}" = "cleaned app app app app" ] +} + +@test "a restore while the fence is up removes it, redirect and refusal alike" { + run fence ip6tables <<'EOF' +iptables_redirect start >/dev/null 2>&1 +echo "fenced $(probe 6 443) $(probe 4 443)" +ip6tables-restore < "$FIXTURES/webmin-fw.rules.v6" +iptables-restore < "$FIXTURES/webmin-fw.rules.v4" +echo "restored $(probe 6 443) $(probe 4 443)" +EOF + [ "$status" -eq 0 ] + [ "${lines[0]}" = "fenced refused fence" ] + [ "${lines[1]}" = "restored app app" ] +} diff --git a/tests/test-init-fence.bats b/tests/test-init-fence.bats index 32b5c8c..045a4e7 100644 --- a/tests/test-init-fence.bats +++ b/tests/test-init-fence.bats @@ -12,10 +12,12 @@ load helpers REPO=$BATS_TEST_DIRNAME/.. FENCE=$REPO/bin/turnkey-init-fence -# stub_tables NAME NAT_STATUS DELETE_OK +# stub_tables NAME NAT_STATUS DELETE_OK [INSERT_STATUS] # Stubs iptables or ip6tables: 'nat -n -L' exits NAT_STATUS (0 means the # table is available), the first DELETE_OK rule deletions succeed and the -# next ones fail (the rule is gone), everything else succeeds. +# next ones fail (the rule is gone), a rule insertion exits INSERT_STATUS +# (default 0, a filter table that cannot be used exits non-zero), +# everything else succeeds. stub_tables() { stub "$1" "if [[ \"\$*\" == *'-t nat -n -L PREROUTING'* ]]; then exit $2 @@ -28,6 +30,9 @@ if [[ \" \$* \" == *' -D '* ]]; then fi exit 1 fi +if [[ \" \$* \" == *' -I '* ]]; then + exit ${4:-0} +fi exit 0" } @@ -115,23 +120,52 @@ teardown() { [ "$(count ip6tables '-t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 60443')" -eq 1 ] } -@test "add_redirect skips a family whose nat table is unavailable" { +@test "add_redirect refuses the port of a family whose nat table is unavailable" { stub_tables ip6tables 3 0 run iptables_add_redirect 443 60443 [ "$status" -eq 0 ] - [ "${lines[2]}" = 'WARNING: ip6tables nat table unavailable, skipping REDIRECT rule: 443 => 60443' ] - [ "$(count iptables ' -A ')" -eq 1 ] - [ "$(count ip6tables ' -A ')" -eq 0 ] + [ "${lines[2]}" = '<4>ip6tables nat table unavailable, refusing 443 rather than redirecting it to 60443' ] + [ "${lines[3]}" = 'Refusing port 443 on ip6tables for as long as the fence is up' ] + [ "$(count iptables ' -A PREROUTING ')" -eq 1 ] + [ "$(count ip6tables ' -A PREROUTING ')" -eq 0 ] + [ "$(count ip6tables '-t filter -I INPUT 1 ! -i lo -p tcp -m tcp --dport 443 -j REJECT --reject-with tcp-reset')" -eq 1 ] + [ "$(count iptables ' -I ')" -eq 0 ] } -@test "add_redirect adds nothing when neither nat table is available" { +@test "add_redirect refuses the port on both families when neither has nat" { stub_tables iptables 3 0 stub_tables ip6tables 3 0 run iptables_add_redirect 80 60080 [ "$status" -eq 0 ] - [ "$(count iptables ' -A ')" -eq 0 ] - [ "$(count ip6tables ' -A ')" -eq 0 ] - [ "$(printf '%s\n' "${lines[@]}" | grep -c '^WARNING: ip6\?tables nat table unavailable')" -eq 2 ] + [ "$(count iptables ' -A PREROUTING ')" -eq 0 ] + [ "$(count ip6tables ' -A PREROUTING ')" -eq 0 ] + [ "$(count iptables '-t filter -I INPUT 1 ! -i lo -p tcp -m tcp --dport 80 -j REJECT --reject-with tcp-reset')" -eq 1 ] + [ "$(count ip6tables '-t filter -I INPUT 1 ! -i lo -p tcp -m tcp --dport 80 -j REJECT --reject-with tcp-reset')" -eq 1 ] + [ "$(printf '%s\n' "${lines[@]}" | grep -c '^<4>ip6\?tables nat table unavailable')" -eq 2 ] +} + +@test "add_redirect fails when a port can be neither redirected nor refused" { + stub_tables ip6tables 3 0 4 + run iptables_add_redirect 12321 60443 + [ "$status" -eq 1 ] + [ "${lines[-1]}" = '<3>ip6tables cannot refuse 12321 either, nothing is fencing it' ] + # the journal never claims a refusal that was not installed + [ "$(printf '%s\n' "${lines[@]}" | grep -c '^Refusing port')" -eq 0 ] +} + +@test "close_port inserts the refusal first, before the appliance's own rules" { + run fence_close_port ip6tables 443 + [ "$status" -eq 0 ] + [ "$output" = 'Refusing port 443 on ip6tables for as long as the fence is up' ] + [ "$(calls ip6tables | tail -1)" = '-t filter -I INPUT 1 ! -i lo -p tcp -m tcp --dport 443 -j REJECT --reject-with tcp-reset' ] +} + +@test "open_port removes the refusal until it is gone" { + stub_tables ip6tables 0 2 + run fence_open_port ip6tables 443 + [ "$status" -eq 0 ] + [ -z "$output" ] + [ "$(count ip6tables '-t filter -D INPUT ! -i lo -p tcp -m tcp --dport 443 -j REJECT --reject-with tcp-reset')" -eq 3 ] } @test "unensure_accept removes the ACCEPT rules until they are gone" { @@ -163,13 +197,17 @@ teardown() { [ "$(count ip6tables ' -A INPUT ')" -eq 2 ] } -@test "redirect stop only deletes" { +@test "redirect stop only deletes, the redirects and the refusals alike" { run iptables_redirect stop [ "$status" -eq 0 ] [ "$(count iptables ' -A ')" -eq 0 ] [ "$(count ip6tables ' -A ')" -eq 0 ] + [ "$(count iptables ' -I ')" -eq 0 ] + [ "$(count ip6tables ' -I ')" -eq 0 ] [ "$(count iptables ' -D PREROUTING ')" -eq 4 ] - [ "$(count iptables ' -D INPUT ')" -eq 2 ] + [ "$(count iptables '-t filter -D INPUT ! -i lo -p tcp -m tcp --dport 443 -j REJECT --reject-with tcp-reset')" -eq 1 ] + [ "$(count ip6tables '-j REJECT')" -eq 4 ] + [ "$(count iptables '-j ACCEPT')" -eq 2 ] } @test "start_mini_server prepares the files and runs the server" { @@ -233,34 +271,47 @@ teardown() { [ "$(count ip6tables ' -A PREROUTING ')" -eq 4 ] [ "$(count iptables ' -A INPUT ')" -eq 2 ] [ "$(count ip6tables ' -A INPUT ')" -eq 2 ] - run ! grep -q WARNING <<<"$output" + [ "$(count iptables ' -I ')" -eq 0 ] + [ "$(count ip6tables ' -I ')" -eq 0 ] + [ "$(printf '%s\n' "${lines[@]}" | grep -c '^<[0-9]>')" -eq 0 ] [[ "$(calls simplehttpd.py)" == *"$WRITABLE_HTDOCS 60080 60443 "* ]] } -@test "start with only the IPv4 nat table warns and still serves" { +@test "start without the IPv6 nat table refuses what it cannot redirect" { stub_tables ip6tables 3 0 run "$FENCE" start [ "$status" -eq 0 ] [ "${lines[-1]}" = 'Started turnkey-init-fence' ] [ "$(count iptables ' -A PREROUTING ')" -eq 4 ] [ "$(count ip6tables ' -A PREROUTING ')" -eq 0 ] + [ "$(count ip6tables ' -I INPUT 1 ')" -eq 4 ] [ "$(count ip6tables ' -A INPUT ')" -eq 2 ] - [ "$(printf '%s\n' "${lines[@]}" | grep -c '^WARNING: ip6tables nat table unavailable')" -eq 4 ] + [ "$(printf '%s\n' "${lines[@]}" | grep -c '^<4>ip6tables nat table unavailable')" -eq 4 ] [ -n "$(calls simplehttpd.py)" ] } -@test "start with neither nat table warns eight times and still serves" { +@test "start with neither nat table refuses every port it was given" { stub_tables iptables 3 0 stub_tables ip6tables 3 0 run "$FENCE" start [ "$status" -eq 0 ] [ "$(count iptables ' -A PREROUTING ')" -eq 0 ] [ "$(count ip6tables ' -A PREROUTING ')" -eq 0 ] - [ "$(printf '%s\n' "${lines[@]}" | grep -c '^WARNING: ')" -eq 8 ] + [ "$(count iptables ' -I INPUT 1 ')" -eq 4 ] + [ "$(count ip6tables ' -I INPUT 1 ')" -eq 4 ] + [ "$(printf '%s\n' "${lines[@]}" | grep -c '^<4>')" -eq 8 ] [ "$(count iptables ' -A INPUT ')" -eq 2 ] [ -n "$(calls simplehttpd.py)" ] } +@test "start fails and serves nothing when a port cannot even be refused" { + stub_tables ip6tables 3 0 4 + run "$FENCE" start + [ "$status" -ne 0 ] + [ "${lines[-1]}" = '<3>ip6tables cannot refuse 80 either, nothing is fencing it' ] + [ -z "$(calls simplehttpd.py)" ] +} + @test "start serves the packaged htdocs when the writable copy is missing" { rm -r "$WRITABLE_HTDOCS" run "$FENCE" start @@ -286,7 +337,9 @@ teardown() { [ "$(count iptables ' -A ')" -eq 0 ] [ "$(count iptables ' -D PREROUTING ')" -eq 4 ] [ "$(count ip6tables ' -D PREROUTING ')" -eq 4 ] - [ "$(count iptables ' -D INPUT ')" -eq 2 ] + [ "$(count iptables '-j REJECT')" -eq 4 ] + [ "$(count ip6tables '-j REJECT')" -eq 4 ] + [ "$(count iptables '-t filter -D INPUT -p tcp -m tcp --dport 60080 -j ACCEPT')" -eq 1 ] [ ! -e "$PIDFILE" ] sleep 0.2 run ! kill -0 "$pid" 2>/dev/null @@ -331,3 +384,25 @@ teardown() { [ "$status" -eq 1 ] printf '%s\n' "${lines[@]}" | grep -qx 'Unknown command: bogus' } + +# A restore of a saved firewall flushes the tables it names, and with them +# every rule the fence installed (tests/test-init-fence-netfilter.bats has +# it happen). At boot the only defence is order: start after whatever does +# the restoring. Webmin's Firewall module restores /etc/iptables/rules.v4 +# and rules.v6 through webmin-iptables.service and webmin-ip6tables.service +# when "activate at boot" is set; iptables-persistent restores the same +# files through netfilter-persistent.service. +@test "the unit starts after everything that restores a saved firewall at boot" { + local unit=$REPO/debian/inithooks.turnkey-init-fence.service + local after + after=" $(sed -n 's/^After=//p' "$unit" | tr '\n' ' ') " + local restorer + for restorer in iptables.service ip6tables.service nftables.service \ + firewalld.service ipset.service netfilter-persistent.service \ + webmin-iptables.service webmin-ip6tables.service; do + [[ "$after" == *" $restorer "* ]] || { + echo "not ordered after $restorer" >&2 + return 1 + } + done +}