Release 2.5.0

[defunctl]

Release testing

This is a release PR for 2.5.0, prepared from #253. CLI_VERSION is set to 2.5.0; pushing this branch triggers the Slic and WordPress image builds with that tag. Wait for both image workflows to finish before testing consuming projects against release/2.5.0.

Warning

Playwright suites may require changes before upgrading. This release changes browser installation and execution behavior; use the migration instructions below before testing existing suites.

The release date remains TBD. No Git tag or GitHub Release has been published, and #252 and #253 remain open while this release is evaluated.

---

Main Changes

The main reason for this change is to remove the browser installation step from CI without losing the PHP tooling that existing Playwright suites depend on.

PR #252 addresses browser installation by moving the entire Playwright test runner into Microsoft's image. That works for suites that only interact with WordPress through HTTP, but some of our projects also run WP-CLI commands directly from their tests:

• Events Calendar Pro: the Calendar Embed suite calls db.reset() and db.loadDump('blocks') in test.beforeAll(). These execute wp db reset --yes and wp db import. Test setup, database helper.
• Event Tickets: the Seating suite uses the same approach to reset the database and import its fixture before running browser tests. Test setup, database helper.

Those commands run wherever the Playwright test process runs. Microsoft's image provides the browsers and their dependencies, but not Slic's PHP, WP-CLI, or Composer environment.

This implementation keeps the tests and setup hooks inside slic, and runs the browsers through a separate Playwright server using Microsoft's image. This release branch includes the implementation from #253, including its Docker-context fix. It targets main for release testing. #252 and #253 remain open until that testing is complete.

---

Compared with current Slic and #252

Behavior	Current Slic (main)	Original PR (#252)	This implementation
Test runner and setup hooks	Existing slic container	Microsoft Playwright container	Existing slic container
Browsers	Installed inside slic	Microsoft Playwright container	Microsoft Playwright container
Browser installation	slic playwright install runs playwright install chromium --with-deps inside slic	Browsers supplied by the Microsoft image	Browsers supplied by the Microsoft image
PHP, WP-CLI, and Composer inside hooks	Available	Unavailable without additional setup	Remain available
Automatic image selection	No separate browser image; the installed CLI selects browser builds during installation	Exact version declared in package.json	Version reported by the project's installed Playwright CLI
Dependency ranges such as ^1.60.0	Supported	Rejected without an override	Supported
Separate version setting	None; uses the project's installed dependency	SLIC_PLAYWRIGHT_VERSION	Removed; the installed dependency determines the version
Direct chromium.launch() calls	Launch locally inside slic, after browser installation	Run in the browser image	Need to connect remotely or use built-in fixtures
Node.js running the tests	Version supplied by Slic	Version supplied by Microsoft's image	Version supplied by Slic

Each test invocation gets its own browser container. Slic waits for it to become ready, supplies Playwright's native PW_TEST_CONNECT_WS_ENDPOINT, and removes the container afterward.

---

Fixes and other changes

• Preserve PHP-based setup hooks. Existing database preparation and other WP-CLI calls continue running inside Slic.
• Preserve the test runner's environment. The original service supplied WP_URL, while Slic exposes WORDPRESS_URL alongside database connection details and Slic context variables. Keeping test execution inside Slic preserves that environment without duplicating it in the browser service.
• Select browsers from the installed dependency. Reading node_modules/.bin/playwright --version supports dependency ranges and avoids selecting an image from a manifest that no longer matches node_modules.
• Preserve command arguments. The original PR uses the legacy Compose helpers, which join arguments into a command string and remove duplicate values in noninteractive mode. The new argument-array helpers preserve spaces, quotes, regular expressions, empty arguments, and repeated options.
• Handle existing installation commands. Both slic playwright install and supported forms such as slic playwright install chromium --with-deps skip installation successfully. Unsupported browser channels and installation options produce an error.
• Keep informational commands lightweight. Help, version, report, and test-listing commands run without starting a browser server.
• Add startup diagnostics and cancellation cleanup. Failed browser startup prints container logs. On hosts with PCNTL, Ctrl+C and termination signals trigger browser cleanup.
• Add developer documentation. The README links to docs/playwright.md, with fixture examples, setup migration guidance, and download handling.

The new Docker and process helpers are loaded from the root bootstrap. Existing command-string helpers retain their current behavior.

---

Faster container readiness checks

Removed start_period from the Playwright, Selenium, Redis, WordPress, and MariaDB health checks. Each service now uses its existing one-second interval from startup, avoiding Docker's default five-second startup polling interval. The checks themselves, timeouts, and retry counts are unchanged. Startup failures now count toward the retry limit immediately.

This requires no new configuration or newer Docker/Compose features. Selenium still starts for existing suites. These changes are in slic-stack.yml, so checking out the updated branch applies them without rebuilding images.

LearnDash CI confirmed shorter startup waits:

Measurement	Before the relevant adjustment	After
Playwright container ID printed to tests starting	5.66s	1.76s
Redis start to healthy	5.5s	1.5s
Selenium start to healthy	5.5s	2.5s
WordPress start to healthy	5.5s	2.5s
MariaDB start to healthy	5.5s	4.5s
slic use operation, before and after the Selenium/Redis and WordPress/MariaDB adjustments	18.9s	11.7s

These are observations across successive runs, not controlled benchmarks. Redis and MariaDB start together, so their savings do not add together. The slic use measurement includes image pulls and initial container startup; Slic prints “services restarted” even on this fresh CI stack.

Sources: before health-check changes, Playwright adjustment, Selenium/Redis adjustments, all adjustments.

The latest run passed all three setup/infrastructure tests. Compared with the preceding run, WordPress and MariaDB readiness waits decreased by about four seconds combined. The whole job dropped from 4m22s to 3m42s, but asset building also dropped from 103s to 72s, so the full improvement cannot be attributed to these changes.

---

Developer experience

Projects continue declaring Playwright in package.json. For example:

{
  "devDependencies": {
    "@playwright/test": "^1.60.0"
  }
}

Commit the dependency lockfile and install normally:

slic npm ci

# Run the project's normal WordPress preparation first.
slic playwright test
slic playwright test --grep 'login|checkout'

If the installed CLI reports 1.60.0, Slic selects mcr.microsoft.com/playwright:v1.60.0. There is no second version to maintain in Slic configuration and no browser installation command to run.

Tests using Playwright's fixtures connect automatically. PHP and WP-CLI hooks remain local to Slic:

import { execFileSync } from 'node:child_process';
import { expect, test } from '@playwright/test';

test.beforeAll(() => {
    execFileSync('wp', ['plugin', 'is-active', 'event-tickets'], {
        stdio: 'inherit',
    });
});

test('shows the WordPress login form', async ({ page }) => {
    await page.goto('http://wordpress.test/wp-login.php');

    await expect(
        page.getByRole('button', { name: 'Log In', exact: true })
    ).toBeVisible();
});

---

Changes needed in existing suites

LearnDash Core's original setup launches Chromium directly in tests/playwright/global-setup.ts. There are two ways to prepare that setup for this change.

Minimal fix: adapt the existing global setup.

Warning

This is the smallest change to get existing setup code working, but it keeps manual browser connection and cleanup logic. Prefer the fixture-based setup project shown below when you can make that migration. This follows Playwright’s recommended setup-project approach, which supports built-in fixtures and automatic browser management.

If the project needs a smaller migration first, keep its current configuration and global-setup.ts. Replace:

const browser = await chromium.launch();

with:

const endpoint = process.env.PW_TEST_CONNECT_WS_ENDPOINT;
const browser  = endpoint
    ? await chromium.connect( endpoint )
    : await chromium.launch();

Keep the existing warmup, login, storage-state, and finally cleanup code. Slic supplies the endpoint automatically; the fallback supports running outside Slic with locally installed browsers.

Warning

Direct chromium.launch(), firefox.launch(), and webkit.launch() calls bypass the automatic remote connection. Use built-in fixtures where possible, or explicitly connect in existing setup code.

---

Recommended: move authentication into a Playwright setup project.

Use the built-in page fixture so Playwright manages the browser connection and cleanup.

1. Preserve the existing WordPress warmup helper.

Move warmUpWordPress() and its three WARMUP_* constants from tests/playwright/global-setup.ts into:

tests/playwright/_helpers/warm-up-wordpress.ts

Keep its implementation unchanged, but export the function:

export async function warmUpWordPress( baseURL: string ): Promise< void > {
    // Keep the existing retry, timeout, and HTTP-status checks.
}

This preserves LearnDash's protection against WordPress cold starts.

2. Create tests/playwright/auth.setup.ts.

Use Playwright's built-in page fixture alongside the existing WPAdmin and ADMIN_AUTH_FILE helpers:

import { test as setup } from '@playwright/test';
import { mkdirSync } from 'node:fs';
import { dirname } from 'node:path';
import { warmUpWordPress } from './_helpers/warm-up-wordpress';
import { WPAdmin } from './_page_objects/wp-admin';
import { ADMIN_AUTH_FILE } from './fixtures';

setup( 'authenticate the administrator', async ( { page, baseURL } ) => {
    // Allow time for WordPress warmup retries and the subsequent login.
    setup.setTimeout( 180_000 );

    await warmUpWordPress( baseURL ?? 'http://wordpress.test' );

    mkdirSync( dirname( ADMIN_AUTH_FILE ), { recursive: true } );

    await new WPAdmin( page ).login();
    await page.context().storageState( { path: ADMIN_AUTH_FILE } );
} );

The page fixture connects to Slic's remote browser automatically. Playwright also closes its page and context afterward, so this file needs no chromium.launch(), explicit endpoint handling, or manual browser cleanup.

3. Update playwright.config.ts.

Remove the existing registration:

globalSetup: './tests/playwright/global-setup',

Replace the projects array with:

projects: [
    {
        name: 'setup',
        testMatch: /auth\.setup\.ts/,
        use: { ...devices[ 'Desktop Chrome' ] },
    },
    {
        name: 'infra',
        testDir: './tests/playwright/infra',
        dependencies: [ 'setup' ],
        use: { ...devices[ 'Desktop Chrome' ] },
    },
],

Keep the rest of LearnDash's configuration, including its existing testDir, baseURL, worker count, and artifact settings.

The setup project inherits tests/playwright as its test directory and runs auth.setup.ts. The infra project waits for successful authentication before running its tests.

4. Remove the old global setup file and keep the existing authenticated fixture.

Once the warmup helper has been moved, delete tests/playwright/global-setup.ts.

LearnDash's existing adminPage fixture already reads ADMIN_AUTH_FILE and uses Playwright's built-in browser fixture. Its implementation and the tests consuming it can remain unchanged. Update its comments to refer to the setup project instead of global setup.

Developers can continue running:

# After the project's normal dependency installation and WordPress preparation:
slic playwright test --project=infra

Playwright runs the setup dependency first, writes the administrator's authentication state, and then runs the infrastructure tests. Avoid --no-deps, which skips that setup.

This follows Playwright's recommended setup-project approach, including fixture management and setup results in reports and traces.

LearnDash has implemented the fixture-based migration in PR #3340. Its setup/infrastructure tests have passed against this release branch; this does not validate its full suite.

Other compatibility checks:

• Use http://wordpress.test for WordPress. Browser requests to localhost now address the browser container.
• Use download.saveAs() instead of download.path() for downloads.
• Check custom browser launch options and persistent browser profiles separately.
• The test runner still uses Slic's Node.js version. This change does not resolve newer Node.js requirements.
• Headed and UI debugging have not been validated.

---

How Playwright version detection works

The project controls its Playwright version through its npm dependency and lockfile. Slic uses the version actually installed in node_modules.

When a developer runs slic playwright test:

1. Slic ensures the slic service is running. The project's dependencies must already be installed.
2. It queries the project's installed CLI, running node_modules/.bin/playwright --version inside Slic, from the project directory.
3. It validates the result. Missing dependencies, a failed command, or an unsupported version format stop the run with an error.
4. It selects the matching image. For example, Version 1.60.0 selects mcr.microsoft.com/playwright:v1.60.0. An explicit SLIC_PLAYWRIGHT_IMAGE overrides this selection and must contain matching browsers.
5. It starts a temporary browser server in that image, using the project's mounted Playwright CLI, and waits for readiness.
6. It runs the tests inside Slic, supplying PW_TEST_CONNECT_WS_ENDPOINT so Playwright's built-in fixtures connect to that server.
7. It removes the browser container after the run and returns the test exit status.

flowchart TD
    A["package.json declares a Playwright dependency"] --> B["npm ci installs the version recorded in the lockfile"]
    B --> C["Developer runs slic playwright test"]
    C --> D["Ensure slic is running"]
    D --> E["Read the project's installed Playwright CLI version"]
    E --> F{"Valid stable version?"}
    F -->|No| G["Stop with an actionable error"]
    F -->|Yes| H["Select matching Microsoft image or configured image override"]
    H --> I["Start browser server and wait for readiness"]
    I --> J["Run tests and PHP hooks inside slic"]
    J <-->|"Playwright WebSocket connection"| K["Browsers in the Microsoft container"]
    J --> L["Remove browser container and return test status"]

Loading

For example, package.json might allow ^1.60.0, while the lockfile installs 1.60.1. Slic reads 1.60.1 from the installed CLI and selects that image. Developers can keep normal dependency ranges without maintaining a separate Slic version setting.

Help, version, report, and test --list commands skip browser-server startup.

---

Testing

During development, isolated Docker smoke checks exercised remote Chromium alongside PHP/WP-CLI hooks, screenshots, traces, videos, uploads, downloads, failure statuses, concurrent runs against an already-started stack, and cancellation.

Temporary checks also cover argument preservation, environment inheritance, large output, version selection, and orchestration. The strengthened cancellation check verifies that child processes stop before test cleanup; disabling shutdown termination correctly makes it fail in both output modes. These temporary tests are not included in this PR.

LearnDash's setup/infrastructure tests passed against release/2.5.0, including the latest health-check changes in 9ff64ed. The CI observations are recorded above.

Fresh-container checks also passed for Selenium and Redis. MariaDB and WordPress checks used an empty database and fresh WordPress directory, with PHP 7.4 and 8.3, and verified a database query and an HTTP response after readiness. Compose configuration validation passed. Temporary verification scripts are not included in this PR.

Full LearnDash and TEC suites, including existing Codeception browser suites, still need release validation. Controlled cold and warm CI comparisons remain outstanding; a first run still needs to pull the browser image.

Summary by CodeRabbit

• New Features
  • Playwright tests can run through a separate browser container, using a browser version matched to the project’s installed Playwright CLI.
  • Added support for configuring the browser image and shared memory, plus guidance for test setup, fixtures, URLs, and browser connections.
• Changes
  • slic playwright install no longer installs browsers; browser dependencies are provided separately.
  • Updated Slic to version 2.5.0.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Enterprise

Run ID: 71245e15-a983-4707-bfad-6a40938dfe83

📥 Commits

Reviewing files that changed from the base of the PR and between 30595bd and 9ff64ed.

📒 Files selected for processing (12)

• .env.slic
• README.md
• changelog.md
• docs/playwright.md
• skills/slic/references/slic-commands.md
• slic-stack.yml
• slic.php
• src/commands/playwright.php
• src/docker-argv.php
• src/playwright.php
• src/process-argv.php
• src/services.php

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

📝 Walkthrough

Walkthrough

The CLI adds remote browser support for Playwright tests. It runs tests in the Slic service, connects browser fixtures to a temporary Playwright server, and adds process and Docker Compose argument-vector helpers. Documentation describes setup and migration considerations.

Changes

Playwright execution

Layer / File(s)	Summary
Process and Compose argument helpers src/process-argv.php, src/docker-argv.php	Adds shell-free process execution and argument parsing. Compose helpers apply Slic environment and terminal options and preserve configured executable prefixes.
Remote browser orchestration and stack wiring src/playwright.php, src/commands/playwright.php, slic-stack.yml, src/services.php, slic.php, .env.slic	Routes Playwright commands through the new runner. Test runs start a temporary browser server, wait for health, pass its WebSocket endpoint to tests, and clean up the server. The stack adds a profiled Playwright service.
Setup and migration guidance README.md, docs/playwright.md, skills/slic/references/slic-commands.md, changelog.md	Documents Playwright setup, remote browser connections, test hooks, download handling, migration considerations, and the 2.5.0 changes.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~50 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant PlaywrightCommand
  participant SlicService
  participant BrowserServer
  PlaywrightCommand->>SlicService: Check installed Playwright CLI version
  PlaywrightCommand->>BrowserServer: Start temporary browser server
  PlaywrightCommand->>BrowserServer: Poll health status
  PlaywrightCommand->>SlicService: Run tests with browser WebSocket endpoint
  PlaywrightCommand->>BrowserServer: Remove browser container after test run

Loading

Merge Risk: ⚪ Minimal · up to 9ff64

No confirmed issue currently blocks merging. Release testing should still exercise browser startup and readiness.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 15 functions across 6 files. (6 skipped: 6…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately identifies this changeset as the Slic 2.5.0 release, which matches the version update, changelog entry, and release objectives.

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

_{Comment @coderabbitai help to get the list of available commands.}

[d4mation]

https://github.com/stellarwp/learndash-core/pull/3340 is approved. I'll merge the LearnDash PR once we have this one tagged and released :)