Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ jobs:
fi
done
bazelisk test //runtime:preload_browser_test --test_output=errors
- name: Verify VRT with only declared tools and Docker
run: bash tests/preloaded-vrt.sh
- name: Verify generated capture bounds and pixels
run: bazelisk test //runtime:capture_browser_test --test_output=errors
- name: Run visual tests and container isolation coverage
Expand Down
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

## Unreleased

### Breaking compatibility: preload VRT images

VRT no longer pulls images or authenticates to registries during execution.
Preload every image in the runtime manifest before testing or updating baselines.
Existing Ryuk containers must match the pinned image; unverifiable or mismatched
reapers fail without being stopped. See [setup](docs/api.md#vrt-image-manifest-and-ci-preloading).

### Breaking compatibility: host browser execution

`web_e2e_test` and `component_browser_test` now launch on the host. Provision a
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,8 @@ attributes; see [migration](docs/getting-started.md#migrating-from-100).
## Try it

Install Bazelisk and provision host Chromium first (with the locked Playwright version).
Start Docker only for VRT. From this checkout:
Start Docker and [preload the pinned images](docs/api.md#vrt-image-manifest-and-ci-preloading)
for VRT. From this checkout:

```sh
export PLAYWRIGHT_BROWSERS_PATH="$(pwd)/.playwright-browsers"
Expand Down
21 changes: 11 additions & 10 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -303,9 +303,9 @@ done

Then run the test using the **same daemon**, exact image references, and correct
platforms, with an empty `DOCKER_CONFIG` and registry credentials unavailable.
Testcontainers uses already-present images before attempting registry authentication.
The runtime requires locally available images and never attempts registry authentication or pulls.
The manifest is not an image archive; transfer/load workflows must preserve the
exact digest references. Missing images may still trigger a pull and fail.
exact digest references. Missing or uninspectable images fail with a preload error before resource creation.

Supported discovery is a host runner's normal Docker discovery, or a containerized
runner with an explicit TCP/HTTP(S) `DOCKER_HOST` (and optional TLS settings).
Expand All @@ -322,15 +322,16 @@ browser infrastructure, not images launched by consumer fixtures or servers.

### Existing reapers on shared daemons

Testcontainers may reuse an already-running Ryuk container without checking its
image digest against this manifest. The pinned requirement governs **fresh reaper
creation**, not the identity of every reused reaper. The preload regression hides
existing reapers and therefore verifies fresh startup only.
Before connecting to an existing Ryuk, the runtime verifies that its actual
container image ID matches the locally resolved pinned Ryuk image. Labels and
requested image names are not sufficient. Mismatched or unverifiable identities
fail with an actionable error; the runtime never stops another invocation's
reaper. Use a dedicated daemon or coordinate cleanup with that reaper's owner.

For strict image identity today, use a dedicated fresh daemon with the manifest's
images preloaded and no running reaper from another invocation. Digest-checked
reuse or rejection of mismatched reapers on shared daemons is tracked in
[#20](https://github.com/perplexityai/rules_web_e2e/issues/20).
A pinned [dependency patch](../patches/README.md) enforces verification before
reuse and disables pulls/auth inside Testcontainers itself, including after
preflight. Tests cover real fresh and reused reapers, a different Ryuk image,
unverifiable identity, missing images, and public comparison/update failures.


## Host browser execution
Expand Down
3 changes: 2 additions & 1 deletion docs/component-vrt.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ and the [API reference](api.md) for all supported attributes.

## Try the standalone example

Install Bazelisk and Docker, start a local Docker daemon, then:
Install Bazelisk and Docker, start a local Docker daemon, and
[preload the pinned images](api.md#vrt-image-manifest-and-ci-preloading), then:

```sh
cd examples/react
Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ flowchart LR
## Run the example

Install Bazelisk and [provision host Chromium](host-browsers.md) for E2E/component
tests. Start Docker only for VRT; Linux amd64 is the validated screenshot platform.
tests. Start Docker and [preload images](api.md#vrt-image-manifest-and-ci-preloading) for VRT; Linux amd64 is the validated screenshot platform.

```sh
cd examples/react
Expand Down
3 changes: 2 additions & 1 deletion docs/host-browsers.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,4 +123,5 @@ AGI can retain its ECR runtime image; FormatJS can supply a custom image with it
fonts and rendering dependencies. This image is ignored by host interaction
tests that share the runtime. Ryuk is still a separate helper requirement for
VRT, exposed by `playwright_images`; constructing a browser image does not remove
that helper or the documented existing-reaper reuse limitation.
that helper. Preload all manifest images before execution; existing Ryuk image
identity must match the pin before reuse.
24 changes: 23 additions & 1 deletion docs/testcontainers-vrt.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ control relay per invocation. Both use the same digest-pinned Playwright image
and explicit `linux/amd64` platform. The runner verifies the browser's platform
and the declared Playwright package version before running tests. Ryuk, the
cleanup helper, uses a pinned image digest on the daemon-selected platform.
All images must be preloaded; execution never pulls images or authenticates to
registries. Existing Ryuk image identity is verified before reuse.

```mermaid
flowchart LR
Expand Down Expand Up @@ -58,7 +60,7 @@ state. Pixel tolerance should not hide uncontrolled inputs.
This is reproducible local browser testing, not a fully sandboxed Bazel action.
The host Node processes execute trusted consumer config, plugins, and tests;
those can explicitly read host files or access the network. Docker discovery,
credentials, daemon/kernel behavior, image availability, and machine resources
daemon/kernel behavior, preloaded image availability, and machine resources
remain external inputs. Tests therefore remain manual, local, and uncached.
Remote Docker daemons are not supported by the loopback control binding.

Expand All @@ -67,3 +69,23 @@ blocked unrelated host ports, and blocked direct public-network access. The
standalone example checks committed screenshot baselines. The built-in server reads only compiled assets; dotenv loading and source
transformation are absent during execution. Consumer builds remain responsible
for their own environment and dependency discovery.

## Enforced runtime dependency contract

Beyond Bazel and standard OS facilities, Docker is the only additional installed
VRT prerequisite. Node, Playwright packages, and compiled application inputs come
from Bazel; Chromium and fonts come from the pinned browser image. Preload both
browser and Ryuk images using the [manifest](api.md#vrt-image-manifest-and-ci-preloading)
before testing. Image acquisition remains a caller-owned setup step.

CI runs `tests/preloaded-vrt.sh` in a pinned OS-only Linux container with no
installed Node, Chromium, Docker CLI, or registry credentials. It executes built
public compare/update targets using their declared runfiles and a proxy that
rejects Docker pull/auth requests. Missing-image updates must preserve baselines;
completed and failed invocations must remove their browser/relay/network resources.
The envelope uses Linux host networking so Docker's loopback relay is reachable;
it is a regression environment, not a new consumer execution requirement.

This verifies tool provisioning, not full hermeticity of consumer code. Host-side
Node code is still trusted and unsandboxed, and application clocks, randomness,
and opted-in external services remain consumer-controlled inputs.
23 changes: 23 additions & 0 deletions examples/react/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -247,3 +247,26 @@ visual_test(
shell = ":app_shell",
baseline_dir = "__isolation_screenshots__",
)

js_test(
name = "preloaded_vrt_test",
copy_data_to_bin = False,
data = [
"package.json",
":native_visual_test",
":native_visual_test.update",
"__native_screenshots__/saved.png",
"@rules_web_e2e//runtime:typecheck",
"@rules_web_e2e//runtime:images",
],
entry_point = "@rules_web_e2e//runtime:preload_lifecycle_entry",
env = {
"PRELOAD_COMPARE": "$(rlocationpath :native_visual_test)",
"PRELOAD_UPDATE": "$(rlocationpath :native_visual_test.update)",
"PRELOAD_BASELINE": "$(rlocationpath __native_screenshots__/saved.png)",
"PRELOAD_IMAGES": "$(rlocationpath @rules_web_e2e//runtime:images)",
},
env_inherit = ["DOCKER_HOST"],
tags = ["manual", "external", "no-sandbox", "no-remote", "no-cache"],
timeout = "long",
)
1 change: 1 addition & 0 deletions patches/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
exports_files(["testcontainers-12.1.0.patch"])
23 changes: 23 additions & 0 deletions patches/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Testcontainers 12.1.0 execution guard

The pnpm-locked patch combines two independent changes, enabled by `startBrowser`
in the rules' private runtime process:

- `TESTCONTAINERS_PULL_POLICY=never` uses the implementation submitted in
[testcontainers-node#1457](https://github.com/testcontainers/testcontainers-node/pull/1457)
(commit `42c7675`). The image-client and utility hunks are the exact compiled
output of that upstream commit (`npm run build -w testcontainers`). Image startup
inspects local availability and returns without registry authentication or pulls,
including if an image disappears after preflight. Both pull paths are covered.
- `TESTCONTAINERS_PRELOADED_IMAGES_ONLY=true` retains our separate Ryuk identity
check. Existing containers are inspected before opening a cleanup connection.
Docker's actual container image ID must equal the locally resolved pinned image
ID. Cached reapers are checked too. Unverifiable or mismatched identities fail;
foreign reapers are never stopped by this check.

This patches the pinned package rather than replacing its cleanup lifecycle or
monkey-patching shared client methods. Default Testcontainers behavior is unchanged
when the flags are absent. Replace the upstream hunks with a released dependency
once available; retain the Ryuk check until upstream supports identity verification.
Real-daemon coverage lives in `runtime/preload-browser.test.ts` and the React
example's `preloaded_vrt_test`.
110 changes: 110 additions & 0 deletions patches/testcontainers-12.1.0.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
diff --git a/build/container-runtime/clients/image/docker-image-client.js b/build/container-runtime/clients/image/docker-image-client.js
--- a/build/container-runtime/clients/image/docker-image-client.js
+++ b/build/container-runtime/clients/image/docker-image-client.js
@@ -12,6 +12,7 @@
const tar_fs_1 = __importDefault(require("tar-fs"));
const common_1 = require("../../../common");
const get_auth_config_1 = require("../../auth/get-auth-config");
+const use_local_image_1 = require("../../utils/use-local-image");
class DockerImageClient {
dockerode;
indexServerAddress;
@@ -124,6 +125,9 @@
}
async pull(imageName, opts) {
try {
+ if (await (0, use_local_image_1.useLocalImage)(this.dockerode, imageName)) {
+ return;
+ }
if (!opts?.force && (await this.exists(imageName))) {
common_1.log.debug(`Image "${imageName.string}" already exists`);
return;
diff --git a/build/container-runtime/utils/pull-image.js b/build/container-runtime/utils/pull-image.js
--- a/build/container-runtime/utils/pull-image.js
+++ b/build/container-runtime/utils/pull-image.js
@@ -8,8 +8,12 @@
const common_1 = require("../../common");
const get_auth_config_1 = require("../auth/get-auth-config");
const image_exists_1 = require("./image-exists");
+const use_local_image_1 = require("./use-local-image");
const pullImage = async (dockerode, indexServerAddress, options) => {
try {
+ if (await (0, use_local_image_1.useLocalImage)(dockerode, options.imageName)) {
+ return;
+ }
if (!options.force && (await (0, image_exists_1.imageExists)(dockerode, options.imageName))) {
common_1.log.debug(`Not pulling image "${options.imageName.string}" as it already exists`);
return;
diff --git a/build/container-runtime/utils/use-local-image.js b/build/container-runtime/utils/use-local-image.js
new file mode 100644
--- /dev/null
+++ b/build/container-runtime/utils/use-local-image.js
@@ -0,0 +1,16 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.useLocalImage = useLocalImage;
+async function useLocalImage(dockerode, imageName) {
+ if (process.env.TESTCONTAINERS_PULL_POLICY !== "never") {
+ return false;
+ }
+ // Bypass the existence cache: an image may have been removed since the last check.
+ try {
+ await dockerode.getImage(imageName.string).inspect();
+ }
+ catch (cause) {
+ throw new Error(`Cannot use local image "${imageName.string}" with TESTCONTAINERS_PULL_POLICY=never; preload it before starting containers`, { cause });
+ }
+ return true;
+}
diff --git a/build/container-runtime/utils/use-local-image.d.ts b/build/container-runtime/utils/use-local-image.d.ts
new file mode 100644
--- /dev/null
+++ b/build/container-runtime/utils/use-local-image.d.ts
@@ -0,0 +1,3 @@
+import Dockerode from "dockerode";
+import { ImageName } from "../image-name";
+export declare function useLocalImage(dockerode: Dockerode, imageName: ImageName): Promise<boolean>;
diff --git a/build/reaper/reaper.js b/build/reaper/reaper.js
--- a/build/reaper/reaper.js
+++ b/build/reaper/reaper.js
@@ -25,6 +25,7 @@
let sessionId;
async function getReaper(client) {
if (reaper) {
+ await verifyReaperImage(client, reaper.containerId);
return reaper;
}
const userId = (0, os_1.userInfo)().uid;
@@ -35,6 +36,7 @@
return new DisabledReaper(sessionId, "");
}
for (const reaperContainer of reaperContainers) {
+ await verifyReaperImage(client, reaperContainer.Id);
const existingSessionId = reaperContainer.Labels[labels_1.LABEL_TESTCONTAINERS_SESSION_ID] ?? new common_1.RandomUuid().nextUuid();
try {
sessionId = existingSessionId;
@@ -52,6 +54,24 @@
});
reaper.addSession(sessionId);
return reaper;
+}
+// Verify before opening a cleanup connection; never remove a foreign reaper.
+async function verifyReaperImage(client, containerId) {
+ if (process.env.TESTCONTAINERS_PRELOADED_IMAGES_ONLY !== "true") return;
+ const reference = getReaperImage();
+ if (!/@sha256:[a-f0-9]{64}$/.test(reference)) {
+ throw new Error("Ryuk requires a digest-pinned image in preloaded-only mode");
+ }
+ let expected, actual;
+ try {
+ expected = await client.image.inspect(container_runtime_1.ImageName.fromString(reference));
+ actual = await client.container.inspect(client.container.getById(containerId));
+ } catch {
+ throw new Error(`Cannot verify existing Ryuk ${containerId} against ${reference}; preload the image and use a dedicated daemon`);
+ }
+ if (!expected.Id || actual.Image !== expected.Id) {
+ throw new Error(`Existing Ryuk ${containerId} does not match ${reference}; use a dedicated daemon or ask its owner to stop it`);
+ }
}
async function findReaperContainers(client) {
const containers = await client.container.list();
7 changes: 5 additions & 2 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions pnpm-workspace.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,6 @@ minimumReleaseAgeExclude:
- "playwright@1.63.0"
- "playwright-core@1.63.0"
- "@types/node@26.5.0"

patchedDependencies:
testcontainers@12.1.0: patches/testcontainers-12.1.0.patch
6 changes: 6 additions & 0 deletions runtime/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -237,3 +237,9 @@ js_test(
data = ["package.json", ":typecheck"],
entry_point = "host-browser.test.js",
)

filegroup(
name = "preload_lifecycle_entry",
srcs = ["preload-lifecycle.test.js"],
visibility = ["//visibility:public"],
)
13 changes: 13 additions & 0 deletions runtime/container.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import {
GenericContainer,
getContainerRuntimeClient,
getReaper,
ImageName,
LABEL_TESTCONTAINERS_SESSION_ID,
StartedNetwork,
Wait,
Expand All @@ -25,7 +26,19 @@ export async function startBrowser(
fs.existsSync(path.join(os.homedir(), '.testcontainers.properties'))
)
if (host) process.env.TESTCONTAINERS_HOST_OVERRIDE = host
// The pinned Testcontainers patch enforces this before auth, pulls, or reuse.
process.env.TESTCONTAINERS_PULL_POLICY = 'never'
process.env.TESTCONTAINERS_PRELOADED_IMAGES_ONLY = 'true'
const client = await getContainerRuntimeClient()
for (const reference of [image, process.env.RYUK_CONTAINER_IMAGE || '']) {
if (!/@sha256:[a-f0-9]{64}$/.test(reference))
throw new Error(`VRT requires a digest-pinned image: ${reference}`)
try {
await client.image.inspect(ImageName.fromString(reference))
} catch {
throw new Error(`Preload required image ${reference} before running VRT; registry access is disabled`)
}
}
const reaper = await getReaper(client)
const name = `vrt-${randomUUID()}`
const network = new StartedNetwork(
Expand Down
Loading