Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/workflows/browser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,21 @@ jobs:
- name: Check desktop and mobile website
working-directory: tests/browser
run: npm test
- name: Record visual capture environment
if: ${{ !cancelled() }}
run: |
mkdir -p tests/results/visual
{
cat /etc/os-release
node --version
npm --version
fc-list --format='%{family}: %{style}\n' | sort -u
cat tests/browser/node_modules/playwright-core/browsers.json
} > tests/results/visual/environment.txt
- name: Compare website visual baselines twice
if: ${{ !cancelled() }}
working-directory: tests/browser
run: npm run test:visual -- --repeat-each=2
- name: Retain bounded browser evidence
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9
Expand All @@ -41,3 +56,10 @@ jobs:
tests/results/browser/results.json
tests/results/browser/artifacts
retention-days: 7
- name: Retain bounded visual evidence
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9
with:
name: website-visual-evidence
path: tests/results/visual
retention-days: 7
7 changes: 5 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,7 @@ Identify significant generated material or reused sources in the PR description.
## Workflow

Maintainers track both repositories in the shared [TendUI project](https://github.com/orgs/Tend-Stack/projects/2).
The board is currently private while publication review is completed; use this
repository's issues to propose work or report bugs.
Use this repository's issues to propose website work or report website bugs.

1. Fork the repository and create a focused branch from current `main`.
2. Make the smallest complete change; keep unrelated cleanup separate.
Expand Down Expand Up @@ -69,6 +68,10 @@ CI supplies native-platform and browser coverage. Do not claim a check ran when
it did not; explain unavailable checks in the PR. Keep generated test evidence
under ignored `tests/results/` and remove owned disposable artifacts after use.

The separate [visual baseline suite](tests/browser/VISUAL.md) compares four key
pages in both themes and viewport sizes. Baseline changes require visual review;
do not automatically accept screenshots to clear a failing check.

## Design and compatibility

- Semantic HTML and native browser behavior first; no bundled client JavaScript,
Expand Down
64 changes: 64 additions & 0 deletions tests/browser/VISUAL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Website visual baselines

The visual suite covers the first viewport of home, documentation, catalog and
playground in light and dark themes at 1920 × 1080 and 390 × 844. Its 16 approved
PNGs live in `baselines/desktop/` and `baselines/mobile/`. These bounded captures
protect the shared header, typography, page hierarchy and responsive layout;
they do not replace the behavioral, accessibility or below-the-fold checks in
`site.spec.js`.

## Canonical environment

GitHub Actions on Ubuntu 24.04 is the capture authority. `package-lock.json` pins
Playwright and its Chromium version. Both viewport projects use Chromium for
image comparison; the existing behavior suite also exercises mobile WebKit.
The site keeps its real system fonts. CI records OS, browser and font information
in `environment.txt` so runner updates can be distinguished from source changes.
OS images and font packages can change: review environment changes rather than
increasing tolerance to hide them.

A desktop negative control hides the home-page heading and requires the screenshot
matcher to reject the result with a pixel-difference error. It is skipped only
while the initial baseline is missing; the normal page checks still fail then.

Each test uses a fresh browser context and explicit theme cookie, locale, timezone,
scale and reduced-motion preference. It waits for fonts and images, disables
animations, hides the caret, and compares viewport screenshots. At most 100 pixels
may differ, with a per-pixel color threshold of 0.1. Do not raise these thresholds
to accept layout changes. CI repeats the suite twice to check capture stability.

## Run and review

Build the website binary from the repository root, then use the existing locked
development-only browser tools:

```sh
go build -mod=vendor -o tests/results/browser/site .
cd tests/browser
npm ci --ignore-scripts
npx playwright install --with-deps chromium
npm run test:visual
```

Missing baselines and changed images **fail**. Normal runs cannot update approved
images (`updateSnapshots: 'none'`). CI retains actual, expected and difference
images, traces, results, capture environment and candidate PNGs in
`website-visual-evidence` for seven days. No visual tooling ships with the site.

For an intentional change:

1. Open a PR describing the affected page/state and why its appearance should change.
2. Download `website-visual-evidence` from that PR's exact commit. Inspect candidates
and difference images at desktop and mobile sizes. Check light and dark themes,
text wrapping, clipping, header alignment and visible focus behavior separately.
3. Copy only reviewed `candidates/<project>/<page>-<theme>.png` files into matching
`tests/browser/baselines/<project>/` paths. Never copy the entire artifact into Git.
4. Commit the selected images with the source change and record the capture run,
source revision, reviewer and reason in the PR. Obtain human review before merge.
5. Require both repeated comparisons to pass. A blank or broken page is never an
acceptable replacement simply because it makes CI green.

For initial establishment, review all 16 images before merging. A local screenshot
from a different OS is useful for diagnosis but is not an approved CI baseline.
Remove disposable downloaded artifacts after review; retain concise provenance
under ignored `tests/results/` and the approved PNGs in Git.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/browser/baselines/desktop/docs-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/browser/baselines/desktop/home-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/browser/baselines/mobile/catalog-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/browser/baselines/mobile/docs-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/browser/baselines/mobile/docs-light.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/browser/baselines/mobile/home-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added tests/browser/baselines/mobile/home-light.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 2 additions & 1 deletion tests/browser/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"version": "2026.1.0",
"description": "Development-only browser verification; never shipped with TendUI.",
"scripts": {
"test": "playwright test"
"test": "playwright test",
"test:visual": "playwright test --config=playwright.visual.config.js"
},
"devDependencies": {
"@playwright/test": "1.65.0",
Expand Down
2 changes: 1 addition & 1 deletion tests/browser/playwright.config.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
const { defineConfig } = require('@playwright/test');
module.exports = defineConfig({
testDir: '.', testMatch: '*.spec.js',
testDir: '.', testMatch: 'site.spec.js',
outputDir: '../results/browser/artifacts',
reporter: [['line'], ['json', { outputFile: '../results/browser/results.json' }]],
timeout: 60000, retries: 0, workers: 2,
Expand Down
43 changes: 43 additions & 0 deletions tests/browser/playwright.visual.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
const { defineConfig } = require('@playwright/test');

// Canonical captures run on the Ubuntu 24.04 CI runner with the locked browser.
// Other platforms can help inspect changes but must not replace these baselines.
module.exports = defineConfig({
testDir: '.',
testMatch: 'visual.spec.js',
snapshotPathTemplate: '{testDir}/baselines/{projectName}/{arg}{ext}',
updateSnapshots: 'none',
outputDir: '../results/visual/artifacts',
reporter: [['line'], ['json', { outputFile: '../results/visual/results.json' }]],
timeout: 30000,
retries: 0,
workers: 1,
expect: {
toHaveScreenshot: {
animations: 'disabled',
caret: 'hide',
scale: 'css',
threshold: 0.1,
maxDiffPixels: 100,
},
},
use: {
baseURL: 'http://127.0.0.1:8080',
browserName: 'chromium',
locale: 'en-US',
timezoneId: 'UTC',
deviceScaleFactor: 1,
reducedMotion: 'reduce',
serviceWorkers: 'block',
trace: 'retain-on-failure',
},
projects: [
{ name: 'desktop', use: { viewport: { width: 1920, height: 1080 } } },
{ name: 'mobile', use: { viewport: { width: 390, height: 844 }, isMobile: true, hasTouch: true } },
],
webServer: {
command: '../results/browser/site',
url: 'http://127.0.0.1:8080/healthz',
reuseExistingServer: false,
},
});
62 changes: 62 additions & 0 deletions tests/browser/visual.spec.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
const { test, expect } = require('@playwright/test');
const { mkdir } = require('node:fs/promises');
const { existsSync } = require('node:fs');
const path = require('node:path');

async function waitForVisibleAssets(page) {
await page.evaluate(async () => {
await document.fonts.ready;
const visibleImages = Array.from(document.images).filter(image => {
const rect = image.getBoundingClientRect();
return rect.width > 0 && rect.height > 0 && rect.bottom > 0 && rect.right > 0 &&
rect.top < innerHeight && rect.left < innerWidth;
});
await Promise.all(visibleImages.map(image => image.decode()));
});
}

const pages = [
['home', '/'],
['docs', '/docs'],
['catalog', '/catalog'],
['playground', '/playground'],
];

for (const theme of ['light', 'dark']) {
for (const [name, route] of pages) {
test(`${name} ${theme}`, async ({ page, context }, testInfo) => {
await context.addCookies([{ name: 'tendui_theme', value: theme, url: 'http://127.0.0.1:8080' }]);
await page.emulateMedia({ colorScheme: theme, reducedMotion: 'reduce' });
const response = await page.goto(route);
expect(response.status()).toBe(200);
await expect(page.locator('h1')).toBeVisible();
await waitForVisibleAssets(page);
await expect(page.locator('html')).toHaveAttribute('data-theme', theme);
expect(await page.evaluate(() => document.documentElement.scrollWidth <= innerWidth + 1)).toBe(true);

// Candidate captures are evidence only. CI never writes approved baselines.
const directory = path.resolve(__dirname, '../results/visual/candidates', testInfo.project.name);
await mkdir(directory, { recursive: true });
await page.screenshot({ path: path.join(directory, `${name}-${theme}.png`), animations: 'disabled', caret: 'hide', scale: 'css' });
await expect(page).toHaveScreenshot(`${name}-${theme}.png`);
});
}
}

test('comparison rejects a missing heading', async ({ page, context }, testInfo) => {
test.skip(testInfo.project.name !== 'desktop', 'One desktop negative control is sufficient.');
test.skip(!existsSync(testInfo.snapshotPath('home-light.png')), 'Requires the reviewed initial baseline.');
await context.addCookies([{ name: 'tendui_theme', value: 'light', url: 'http://127.0.0.1:8080' }]);
await page.goto('/');
await waitForVisibleAssets(page);
await expect(page).toHaveScreenshot('home-light.png');
await page.locator('h1').evaluate(heading => { heading.style.visibility = 'hidden'; });
let failure = '';
try {
await expect(page).toHaveScreenshot('home-light.png', { timeout: 1000 });
} catch (error) {
failure = error.message;
}
// Accept only an actual image difference, never a browser or network failure.
expect(failure).toMatch(/\d+ pixels.*different/s);
});