Skip to content

sec(networking): host-firewall egress backstop for Windows job containers - #136

Merged
luthermonson merged 1 commit into
mainfrom
fix/windows-egress-firewall
Aug 7, 2026
Merged

sec(networking): host-firewall egress backstop for Windows job containers#136
luthermonson merged 1 commit into
mainfrom
fix/windows-egress-firewall

Conversation

@luthermonson

Copy link
Copy Markdown
Contributor

Problem

Windows job containers had unrestricted LAN egress. The containment suite (#134) reached the fleet's Incus daemon (192.168.12.113:8443) and Grafana (192.168.10.45:3000) from a Hyper-V-isolated job. Linux jobs are contained by firewall_linux.go's iptables FORWARD chain; installFirewallRules on Windows was a no-op, and the per-endpoint HNS/VFP ACLs applied in setup() did not contain the job on the real runner.

Mechanism: host-side Windows Defender Firewall rules

firewall_windows.go is the counterpart of firewall_linux.go: installFirewallRules programs Windows Defender Firewall on the host via netsh advfirewall, removeFirewallRules deletes the same set.

Why host firewall rather than (more) HNS ACLs:

  • Every container flow is routed and NATed by the host network stack (WinNAT), so host firewall rules sit in the path regardless of what the vSwitch port enforces. The containment run showed vSwitch/VFP endpoint ACLs cannot be the only line of defense for Hyper-V-isolated endpoints.
  • Like the Linux FORWARD chain, the rules are host-global: one set covers every endpoint, including stale endpoints leaked by a crashed run — no per-job step can be skipped.
  • HNS network-level ACLs were considered and rejected: they use the same VFP enforcement point as the endpoint ACLs that just failed, and they cannot express "this range minus the gateway".

The per-endpoint ACLs stay in place as defense in depth.

What is allowed through, and why

  • The container subnet 10.88.0.0/16 (which contains the NAT gateway 10.88.0.1 and the other containers) is subtracted from each blocked range with IPv4 range arithmetic (subtractCIDR), e.g. 10.0.0.0/8 becomes 10.0.0.0-10.87.255.255,10.89.0.0-10.255.255.255. Windows Firewall has no rule ordering — Block always beats Allow — so the Linux "allow gateway above the deny" pattern cannot be ported; a blocked gateway could not be rescued by an allow rule, and blocking it would brick all container networking (DNS and outbound NAT both go through it). Carving it out of the block set is the only correct shape.
  • Everything non-RFC1918/link-local (the internet) is untouched: the block rules enumerate only 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16 (minus the subnet), so GitHub, Docker Hub, and package registries are unaffected.
  • Gateway control ports are re-tightened inbound: container→gateway traffic terminates at the host and never hits the outbound blocks, so hostFirewallRules also emits inbound TCP blocks (source = container subnet, dest = gateway, one specific port each) for cfg.ControlPorts — the mirror of controlPlaneInputRules on Linux. DNS (53) and NAT are deliberately not matched.
  • Outbound blocks are scoped localip=<container subnet>: forwarded container traffic is evaluated pre-NAT with its 10.88/16 source, so the host's own management-LAN traffic can never match — a mis-scoped rule degrades to a no-op, never to cutting the fleet host off its own LAN.
  • IPv4 only, deliberately: the HCN NAT network has no v6 IPAM (containers have no IPv6 path), and a host-wide v6 link-local block without a container-source scope would break the host's neighbor discovery.

Idempotency and cleanup

Every rule has a deterministic name under the ephemerd-egress- prefix (ephemerd-egress-block-10.0.0.0_8, ephemerd-egress-control-10000, …). Install is delete-before-add per name, so re-running installFirewallRules never accumulates duplicates; removeFirewallRules recomputes the same set and deletes each name (best-effort, Debug-logged), same as the Linux removal path. The set is findable with netsh advfirewall firewall show rule name=all | findstr ephemerd-egress.

Failure mode

installFirewallRules returns an error on the first failed netsh add; every caller (cmd/ephemerd/main.go:271,424) already logs a warning and continues, so a host where the daemon lacks firewall privileges degrades to the per-endpoint ACLs instead of refusing daemon startup — identical posture to Linux.

Tests

  • TestSubtractCIDR — table test of the range subtraction (split, no-overlap, full-cover, edge-aligned, malformed, IPv6-rejected).
  • TestHostFirewallRules_* — pure rule-set assertions without invoking netsh, mirroring firewall_linux_test.go: every denied range becomes an outbound block; the subnet/gateway never appear in a blocked range; control-port rules are narrow (TCP, one port, subnet→gateway, never 53); names/argv contract for idempotent delete-before-add.
  • go build ./..., go test ./pkg/networking/..., GOOS=windows go vet ./pkg/networking/, GOOS=linux go vet ./pkg/networking/ all pass (built and run natively on Windows).

The real regression test is the containment suite's Windows "Fleet management planes must be unreachable" step (.github/workflows/containment.yml, #134), which fails today and should pass with this deployed on the Windows node.

Closes #135

…ners

Windows jobs could reach the fleet's management planes (Incus, Grafana)
because installFirewallRules was a no-op on Windows and the per-endpoint
HNS/VFP ACLs alone did not contain a Hyper-V-isolated job on the real
runner (found by the containment suite, #134).

Implement the Windows counterpart of firewall_linux.go: Windows Defender
Firewall rules installed on the host, where WinNAT routes every container
flow regardless of vSwitch policy. Outbound blocks cover RFC1918 +
link-local with the container subnet (gateway, DNS, NAT, container-to-
container) subtracted via range arithmetic — Windows Firewall has no rule
ordering and Block beats Allow, so the gateway can never appear inside a
blocked range. Inbound rules block container→gateway traffic on the
ephemerd control ports, mirroring controlPlaneInputRules.

Rules are named ephemerd-egress-* and installed delete-before-add, so
reinstallation is idempotent and Cleanup removes the exact set. Install
failure surfaces as an error that callers already treat as a warning, so
a host without firewall privileges degrades to the endpoint ACLs instead
of refusing to start.

Closes #135
@luthermonson
luthermonson merged commit 3d61eb8 into main Aug 7, 2026
4 checks passed
luthermonson added a commit that referenced this pull request Aug 8, 2026
…wall (#140)

#136 blocked container->LAN egress with host Windows Defender Firewall
rules (netsh advfirewall) scoped to the container subnet as source. On
real hardware this is a no-op: the host firewall (WFP/MPSSVC) evaluates
NATed container egress post-NAT, where the source is the host's own LAN
address, so a rule scoped to 10.88/16 matches nothing. Confirmed against
a live v0.1.6 Windows node -- the containment suite still reached
Proxmox, Incus, and Grafana.

Replace it with Hyper-V firewall rules (New-NetFirewallHyperVRule),
which filter at the container vNIC boundary BEFORE NAT, where the packet
still carries the container's own address. Rules are scoped by
-VMCreatorId (discovered at runtime via Get-NetFirewallHyperVVMCreator,
excluding the well-known WSL creator) so they apply to job containers,
not the host or unrelated Hyper-V workloads.

The gateway (DNS/NAT/default route) and the container-to-container range
are carved out of every blocked range up front (subtractCIDR), so they
never appear in a Block rule -- no reliance on rule-priority ordering,
which is the kind of unverified firewall semantic that sank #136. The
internet, including GitHub/Docker Hub/registries, is untouched.

Idempotent (remove-before-add), findable/removable by the ephemerd-
name prefix, and graceful: hosts without the Hyper-V firewall cmdlets
fall back to the netsh rules, and any failure warns instead of failing
daemon startup. netsh construction is retained as that fallback.

Needs a live run on the Server 2025 node to confirm; the containment
suite's Windows step is the regression gate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows job containers have unrestricted LAN egress (no InstallFirewallRules equivalent)

1 participant