diff --git a/specs/010-java-services/tasks.md b/specs/010-java-services/tasks.md index fedf5663..3c7c2cae 100644 --- a/specs/010-java-services/tasks.md +++ b/specs/010-java-services/tasks.md @@ -21,7 +21,7 @@ - [ ] T001 [EXT] Ask the Reactome organisation's owners to create the library-service repository, licensed AGPL-3.0 (research R2), and record its name in this file and in `plan.md` - [ ] T002 [EXT] Ask the owners of `reactome/content-service` and `reactome/analysis-service` to merge the deployed branches (`7b9eceb` on `add-content-page-endpoints`; `9b147e1`, PR #25). The pins stay on these commits either way (research R1). -- [x] T003 [P] Record the pinned commits of research R1 in one place, `deploy/java/PINS.md`, each with the reason it is pinned and how it was verified (byte comparison of classes) +- [x] T003 [P] Record the pinned commits of research R1 in one place, `deploy/java/PINS.md`, each with the reason it is pinned and how it was verified (byte comparison of classes) — #505 --- @@ -31,22 +31,22 @@ ### Tests first -- [x] T004 Extend `deploy/nginx/routes.spec.ts`: +- [x] T004 Extend `deploy/nginx/routes.spec.ts`: — #505 - parse `deploy/nginx/common/routes.conf` as well as `routes-core.conf`, in nginx's order; - classify each `proxy_pass` as node, Java service or Tomcat; - assert that no URL in node's endpoint samples or the proxy-order node list reaches a Java service or Tomcat. - Show it failing by adding a location that sends a node URL to Tomcat. -- [x] T005 Add a test in `deploy/nginx/routes.spec.ts` that an exporter address routed away from Tomcat keeps the exporter location's rate limit. Show it failing with a `routes-core.conf` location for `/ContentService/exporter/event/` (research R6 trap). +- [x] T005 Add a test in `deploy/nginx/routes.spec.ts` that an exporter address routed away from Tomcat keeps the exporter location's rate limit. Show it failing with a `routes-core.conf` location for `/ContentService/exporter/event/` (research R6 trap). — #505 ### Implementation -- [ ] T006 Create the parity harness in `tools/java-parity/`, per research R7: +- [x] T006 Create the parity harness in `tools/java-parity/`, per research R7: — #507 - same request to a reference and a candidate, same Host; - compare status, a fixed header set and the SHA-256 of the body; - per-class normalisers, declared in `tools/java-parity/normalisers.mjs`; - JSON report of identical, different and both-error counts, with no bodies or token values in the output. -- [ ] T007 Add a self-test to `tools/java-parity/parity.spec.mjs`. Two stub servers that differ in one byte must report a difference; identical stubs must not. Show it failing with the comparison disabled. -- [ ] T008 [P] Add the PDF normaliser to `tools/java-parity/normalisers.mjs`: +- [x] T007 Add a self-test to `tools/java-parity/parity.spec.mjs`. Two stub servers that differ in one byte must report a difference; identical stubs must not. Show it failing with the comparison disabled. — #507 +- [ ] T008 [P] (With the first PDF comparison, T030: it needs `qpdf` in a container.) Add the PDF normaliser to `tools/java-parity/normalisers.mjs`: - `qpdf --qdf --object-streams=disable --static-id`, run in a throwaway container; - mask CreationDate, ModDate and the cover-page date. - Test it in `tools/java-parity/parity.spec.mjs`: two PDFs differing only in those fields compare equal, and a PDF with a changed body does not. @@ -205,12 +205,11 @@ Each cutover is one PR. It adds the group's patterns to node's table, one nginx ## Phase 8: Polish and side findings -- [ ] T058 [EXT] Decide the analysis report's species overview figure. The PDF report embeds the retired Fireworks image, drawn by Java's fireworks-exporter; the site's overview is now the Reacfoam (Voronoi) map. Showing that in the report needs analysis-report changed upstream (its owners) or the figure supplied to it. - - [ ] T054 [P] Set `client_max_body_size` in `deploy/nginx/local.conf` to match the other deployments (64 MB), so local uploads over 1 MB are not refused. Test it with `deploy/nginx/test-config.sh`. - [ ] T055 [P] Declare the proxy package that `projects/website-angular/src/scripts/serve-prod.js` uses in `package.json` (it is only present as a dev-only transitive dependency), and check the lockfile changes only that entry - [ ] T056 Update `specs/006-content-service-to-node/tasks.md` Phase 9 rows with the PR that moved each Java-homed operation, as each moves - [ ] T057 Update the curator-facing documents if any reader-visible behaviour is touched by a cutover (constitution: Development Workflow) +- [ ] T058 [EXT] Decide the analysis report's species overview figure. The PDF report embeds the retired Fireworks image, drawn by Java's fireworks-exporter; the site's overview is now the Reacfoam (Voronoi) map. Showing that in the report needs analysis-report changed upstream (its owners) or the figure supplied to it. --- diff --git a/tools/java-parity/normalisers.mjs b/tools/java-parity/normalisers.mjs new file mode 100644 index 00000000..e40db6eb --- /dev/null +++ b/tools/java-parity/normalisers.mjs @@ -0,0 +1,35 @@ +/** + * The normalisers parity may apply, each declared here with what it removes + * and why. A difference is only ever hidden by one of these, by name, in the + * run that uses it -- never by loosening the comparison itself. + * + * Each removes exactly its target's bytes and leaves every other byte as it + * came. A JSON parse-and-print round trip would not: it loses digits past + * 2^53, folds -0 into 0, drops duplicate keys, reorders integer-like keys and + * rewrites number formats and escapes -- differences the API's R and Python + * users would see, hidden from the comparison. + */ +import { gunzipSync } from 'node:zlib'; + +/** + * An analysis token holds the time and a per-process counter, so two runs + * never share one: its value -- only its value -- is blanked inside + * `summary`. + */ +function withoutToken(bytes) { + // latin1 maps bytes to code units one to one, so nothing else is re-encoded. + const text = bytes.toString('latin1'); + const token = /("summary"\s*:\s*\{[^{}]*?"token"\s*:\s*)"(?:[^"\\]|\\.)*"/; + // No token to blank is not an answer this normaliser is for: an error, not a silent pass-through. + if (!token.test(text)) throw new Error('no summary.token'); + return Buffer.from(text.replace(token, '$1""'), 'latin1'); +} + +export const normalisers = { + /** Byte for byte. */ + identity: (body) => body, + /** An analysis result (JSON) with its token blanked. */ + analysisResult: (body) => withoutToken(body), + /** The gzipped result download, decompressed, with its token blanked. Not gzip: an error. */ + analysisResultGz: (body) => withoutToken(gunzipSync(body)), +}; diff --git a/tools/java-parity/parity.mjs b/tools/java-parity/parity.mjs new file mode 100644 index 00000000..1b28d3d2 --- /dev/null +++ b/tools/java-parity/parity.mjs @@ -0,0 +1,287 @@ +#!/usr/bin/env node +/** + * Java-against-Java parity (spec 010, research R7): the same requests to a + * reference -- what answers today -- and a candidate -- what replaces it -- + * compared on status, every response header but named noise, and the bytes + * of the body after a declared normaliser. + * + * Over node:http, not fetch: fetch drops a Host set by the caller, so each + * side saw its own address and every answer built from Host differed for a + * reason that was not a difference; and fetch hides Transfer-Encoding. + * + * Reports carry digests and verdicts, never bodies -- some answers hold + * people's records -- and errors carry only their kind, never a message, + * which can quote a body. With --redact not even the request, because some + * paths are tokens anyone holding them can read. + * + * Usage: + * node tools/java-parity/parity.mjs --reference URL --candidate URL \ + * --requests requests.json [--normalise NAME] [--host HOST] [--timeout MS] + * [--ignore-header NAME]... [--calibrate] [--redact] [--simultaneous] + * requests.json: [{ "method": "GET", "path": "/...", "headers": {}, + * "body": "text" | "bodyBase64": "..." | "bodyFile": "path" }] + * Exit: 0 all identical; 1 differences; 2 errors, unstable requests, or + * nothing compared -- a run that compared nothing is not a pass; 3 usage. + */ +import { createHash } from 'node:crypto'; +import { readFileSync } from 'node:fs'; +import { request as httpRequest } from 'node:http'; +import { request as httpsRequest } from 'node:https'; +import { brotliDecompressSync, gunzipSync, inflateSync } from 'node:zlib'; +import { normalisers } from './normalisers.mjs'; + +/** Headers that differ between any two servers by nature, and only these. */ +const NOISE = new Set(['date', 'connection', 'keep-alive', 'server']); + +function requestBody({ body, bodyBase64, bodyFile }) { + if (bodyFile) return readFileSync(bodyFile); + if (bodyBase64) return Buffer.from(bodyBase64, 'base64'); + return body === undefined ? undefined : Buffer.from(body); +} + +/** + * A base URL as the place to send a path: scheme, host (an IPv6 one without + * its brackets), port, and any context path, which is kept -- dropping it + * sent both sides to the same wrong resource, whose matching 404s compared + * as a pass. Anything it cannot honour is refused, never ignored. + */ +function baseOf(base) { + const url = new URL(base); + if (url.protocol !== 'http:' && url.protocol !== 'https:') + throw new Error(`base must be http or https: ${url.protocol}`); + if (url.search || url.hash) throw new Error('base must not carry a query or fragment'); + return { + send: url.protocol === 'https:' ? httpsRequest : httpRequest, + hostname: url.hostname.replace(/^\[(.*)\]$/, '$1'), + port: url.port || (url.protocol === 'https:' ? 443 : 80), + prefix: url.pathname.replace(/\/$/, ''), + }; +} + +/** + * Headers as sent: from rawHeaders, every copy kept and joined in order -- + * the parsed map keeps one Content-Type or Location of two, so a side that + * sent two compared equal to one that sent one. + */ +function rawHeaders(raw) { + const headers = {}; + for (let at = 0; at < raw.length; at += 2) { + const name = raw[at].toLowerCase(); + headers[name] = name in headers ? `${headers[name]}, ${raw[at + 1]}` : raw[at + 1]; + } + return headers; +} + +const failure = (kind) => Object.assign(new Error(kind), { kind }); + +/** One answer: status, headers as sent, and the body as sent -- within `timeout` in all. */ +function answer(base, req, { host, timeout }) { + const { send, hostname, port, prefix } = baseOf(base); + const body = requestBody(req); + return new Promise((resolve, reject) => { + const outgoing = send( + { + hostname, + port, + method: req.method ?? 'GET', + path: `${prefix}${req.path}`, + headers: { + ...(req.headers ?? {}), + ...(host ? { host } : {}), + ...(body ? { 'content-length': body.length } : {}), + }, + }, + (response) => { + const chunks = []; + response.on('data', (chunk) => chunks.push(chunk)); + response.on('end', () => { + clearTimeout(timer); + resolve({ + status: response.statusCode, + headers: rawHeaders(response.rawHeaders), + body: Buffer.concat(chunks), + }); + }); + // A reset part way through: the run goes on at once, and no timer + // is left armed to keep the process alive until it fires. + response.on('error', (error) => { + clearTimeout(timer); + reject(error); + }); + } + ); + // The whole request, not a silence: a body trickled a byte at a time + // never goes quiet, and held the run open. + const timer = setTimeout(() => outgoing.destroy(failure('timeout')), timeout); + outgoing.on('error', (error) => { + clearTimeout(timer); + reject(error); + }); + outgoing.end(body); + }); +} + +/** The body a client reads: Content-Encoding undone (the header itself is still compared). */ +function decoded({ headers, body }) { + const encoding = (headers['content-encoding'] ?? '').trim().toLowerCase(); + try { + if (encoding === 'gzip') return gunzipSync(body); + if (encoding === 'br') return brotliDecompressSync(body); + if (encoding === 'deflate') return inflateSync(body); + } catch { + throw failure('body decoding'); + } + return body; +} + +/** A body through the normaliser; a body it cannot take is the normaliser's failure. */ +function normalised(normalise, answer) { + const body = decoded(answer); + try { + return normalise(body); + } catch { + throw failure('normaliser'); + } +} + +const digest = (bytes) => createHash('sha256').update(bytes).digest('hex'); + +/** What differs between two answers, bodies normalised the same way. */ +function differences(a, b, normalise, ignored = new Set()) { + const found = []; + if (a.status !== b.status) found.push('status'); + const names = new Set([...Object.keys(a.headers), ...Object.keys(b.headers)]); + for (const name of [...names].sort()) { + if (NOISE.has(name) || ignored.has(name)) continue; + // Content-Length follows the body; under a normaliser the body is compared instead. + if (name === 'content-length' && normalise !== normalisers.identity) continue; + if (a.headers[name] !== b.headers[name]) found.push(name); + } + // Different statuses are a difference whatever the bodies are: an error + // page the normaliser cannot take must not turn that into an error. + if (found.includes('status')) { + const raw = (answer) => { + try { + return decoded(answer); + } catch { + // An error page that claims an encoding it lacks: compare it as sent. + return answer.body; + } + }; + if (digest(raw(a)) !== digest(raw(b))) found.push('body'); + return found; + } + if (digest(normalised(normalise, a)) !== digest(normalised(normalise, b))) found.push('body'); + return found; +} + +/** Only the kind of a failure: a message can quote the body or the URL. */ +const kindOf = (error) => error?.kind ?? (error?.code ? `network ${error.code}` : 'failed'); + +/** + * Compare every request. With `calibrate`, the reference is asked again: + * where it disagrees with itself (live third-party data), the request is + * "unstable" and counted neither way. + */ +export async function compareAll( + requests, + { + reference, + candidate, + normalise = normalisers.identity, + host, + timeout = 120_000, + calibrate = false, + redact = false, + simultaneous = false, + ignoreHeaders = [], + } +) { + // Refused before anything is sent: a base it cannot honour is a usage error. + baseOf(reference); + baseOf(candidate); + const ignored = new Set(ignoreHeaders.map((name) => name.toLowerCase())); + const results = []; + const counts = { identical: 0, different: 0 }; + const count = (key) => (counts[key] = (counts[key] ?? 0) + 1); + const ask = (base, req) => answer(base, req, { host, timeout }); + for (const [index, req] of requests.entries()) { + const label = redact ? `#${index}` : `${req.method ?? 'GET'} ${req.path}`; + try { + const [ref, cand] = simultaneous + ? await Promise.all([ask(reference, req), ask(candidate, req)]) + : [await ask(reference, req), await ask(candidate, req)]; + const result = { request: label, status: [ref.status, cand.status] }; + if (calibrate) { + result.calibrated = + differences(ref, await ask(reference, req), normalise, ignored).length === 0; + if (!result.calibrated) { + count('unstable'); + results.push(result); + continue; + } + } + result.differs = differences(ref, cand, normalise, ignored); + count(result.differs.length ? 'different' : 'identical'); + results.push(result); + } catch (error) { + count('errors'); + results.push({ request: label, error: kindOf(error) }); + } + } + // A declared exclusion is part of the result, never silent. + return { counts, ...(ignored.size ? { ignoredHeaders: [...ignored] } : {}), results }; +} + +/** The exit code a report earns: 0 only when something was compared and all of it was identical. */ +export function verdict({ counts }) { + if (counts.errors || counts.unstable || counts.identical + counts.different === 0) return 2; + return counts.different ? 1 : 0; +} + +// Started directly rather than imported by a test. +if (process.argv[1] && process.argv[1].endsWith('parity.mjs')) { + const args = process.argv.slice(2); + const option = (name) => { + const at = args.indexOf(`--${name}`); + return at === -1 ? undefined : args[at + 1]; + }; + const flag = (name) => args.includes(`--${name}`); + const normalise = normalisers[option('normalise') ?? 'identity']; + if (!option('reference') || !option('candidate') || !option('requests') || !normalise) { + console.error( + 'usage: parity.mjs --reference URL --candidate URL --requests FILE [--normalise NAME] [--host HOST] [--timeout MS] [--ignore-header NAME]... [--calibrate] [--redact] [--simultaneous]' + ); + process.exit(3); + } + // Usage errors exit 3, never 1 ("differences"): an unusable base, a + // dangling --ignore-header. + const usage = (message) => { + console.error(`parity.mjs: ${message}`); + process.exit(3); + }; + for (const base of [option('reference'), option('candidate')]) { + try { + baseOf(base); + } catch { + usage('each base must be an http or https URL without a query'); + } + } + args.forEach((arg, at) => { + if (arg === '--ignore-header' && (!args[at + 1] || args[at + 1].startsWith('--'))) + usage('--ignore-header needs a header name'); + }); + const report = await compareAll(JSON.parse(readFileSync(option('requests'), 'utf8')), { + reference: option('reference'), + candidate: option('candidate'), + normalise, + host: option('host'), + timeout: option('timeout') ? Number(option('timeout')) : undefined, + calibrate: flag('calibrate'), + redact: flag('redact'), + simultaneous: flag('simultaneous'), + ignoreHeaders: args.flatMap((arg, at) => (arg === '--ignore-header' ? [args[at + 1]] : [])), + }); + console.log(JSON.stringify(report, null, 2)); + process.exit(verdict(report)); +} diff --git a/tools/java-parity/parity.spec.mjs b/tools/java-parity/parity.spec.mjs new file mode 100644 index 00000000..a0435a45 --- /dev/null +++ b/tools/java-parity/parity.spec.mjs @@ -0,0 +1,300 @@ +// @vitest-environment node +// +// The Java-against-Java parity harness (spec 010, research R7), against two +// stub servers standing in for the reference (what runs today) and the +// candidate (what replaces it). +import { createServer } from 'node:http'; +import { setInterval, clearInterval } from 'node:timers'; +import { spawnSync } from 'node:child_process'; +import { writeFileSync, mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { gzipSync } from 'node:zlib'; +import { afterAll, beforeAll, describe, expect, it } from 'vitest'; +import { compareAll, verdict } from './parity.mjs'; +import { normalisers } from './normalisers.mjs'; + +/** A server answering from a table of path -> { status, headers, body }. */ +function stub(table) { + let calls = 0; + const server = createServer((request, response) => { + calls += 1; + // Live third-party data: a different answer every time. + if (request.url === '/live') return response.end(`live ${calls}`); + // An answer built from the Host it was asked with. + if (request.url === '/host') return response.end(`host ${request.headers.host}`); + // A server that never answers. + if (request.url === '/hang') return; + // One that never stops answering, a byte at a time. + if (request.url === '/trickle') { + const timer = setInterval(() => response.write('.'), 50); + response.on('close', () => clearInterval(timer)); + return; + } + // Two Content-Types, which a parsed header map shows as one. + if (request.url === '/twice' && !table.name) { + response.setHeader('content-type', ['text/plain', 'text/html']); + return response.end('x'); + } + // A body cut off by a reset, part way. + if (request.url === '/reset') { + response.writeHead(200, { 'content-length': '100' }); + response.write('partial'); + return setTimeout(() => request.socket.destroy(), 20); + } + // A status difference whose error page claims gzip and is not. + if (request.url === '/badgz' && table.name) + return response.writeHead(500, { 'content-encoding': 'gzip' }).end('not gzip'); + // Context-path aware: only /ctx/x exists. + if (request.url === '/ctx/x') return response.end(`ctx ${table.name ?? ''}`); + const answer = table[request.url] ?? { status: 404, body: 'none' }; + response.writeHead(answer.status ?? 200, answer.headers ?? { 'content-type': 'text/plain' }); + response.end(answer.body); + }); + return new Promise((resolve) => + server.listen(0, '127.0.0.1', () => + resolve({ server, url: `http://127.0.0.1:${server.address().port}` }) + ) + ); +} + +const json = { 'content-type': 'application/json;charset=UTF-8' }; +const reference = { + '/same': { body: 'abc' }, + '/byte': { body: 'abc' }, + '/status': { status: 200, body: 'x' }, + '/header': { + headers: { 'content-type': 'text/plain', 'content-disposition': 'a.txt' }, + body: 'x', + }, + '/redirect': { status: 302, headers: { location: '/a' }, body: '' }, + '/vary': { headers: { 'content-type': 'text/plain', vary: 'Accept' }, body: 'x' }, + '/token': { + headers: json, + body: '{"summary":{"token":"T1","type":"X"},"n":9007199254740993}', + }, + '/gz': { + headers: { 'content-type': 'application/gzip' }, + body: gzipSync('{"summary":{"token":"T1"},"n":1}'), + }, + '/text404': { status: 404, body: 'secret-body-text' }, + '/badgz': { headers: { 'content-type': 'text/plain' }, body: 'ok' }, + '/secret/ABC123': { body: 'x' }, +}; +const candidate = { + ...reference, + name: 'candidate', + '/twice': { headers: { 'content-type': 'text/plain' }, body: 'x' }, + '/byte': { body: 'abd' }, + '/status': { status: 500, body: 'x' }, + '/header': { + headers: { 'content-type': 'text/plain', 'content-disposition': 'b.txt' }, + body: 'x', + }, + '/redirect': { status: 302, headers: { location: '/b' }, body: '' }, + '/vary': { headers: { 'content-type': 'text/plain' }, body: 'x' }, + '/token': { + headers: json, + body: '{"summary":{"token":"T2","type":"X"},"n":9007199254740993}', + }, + '/gz': { + headers: { 'content-type': 'application/gzip' }, + body: gzipSync('{"summary":{"token":"T9"},"n":1}'), + }, +}; + +let ref; +let cand; +beforeAll(async () => { + ref = await stub(reference); + cand = await stub(candidate); +}); +afterAll(async () => { + ref.server.closeAllConnections(); + cand.server.closeAllConnections(); + await new Promise((resolve) => ref.server.close(resolve)); + await new Promise((resolve) => cand.server.close(resolve)); +}); + +const run = (paths, options = {}) => + compareAll( + paths.map((path) => (typeof path === 'string' ? { method: 'GET', path } : path)), + { reference: ref.url, candidate: cand.url, ...options } + ); + +describe('the parity harness', () => { + it('finds identical answers identical', async () => { + const report = await run(['/same']); + expect(report.counts).toEqual({ identical: 1, different: 0 }); + expect(verdict(report)).toBe(0); + }); + + it('finds a one-byte, a status and a header difference', async () => { + const report = await run(['/byte', '/status', '/header']); + expect(report.counts).toEqual({ identical: 0, different: 3 }); + expect(report.results.map((r) => r.differs)).toEqual([ + ['body'], + ['status'], + ['content-disposition'], + ]); + expect(verdict(report)).toBe(1); + }); + + it('compares every header but named noise: a redirect target, a Vary', async () => { + const report = await run(['/redirect', '/vary']); + expect(report.results.map((r) => r.differs)).toEqual([['location'], ['vary']]); + }); + + it('asks both sides with the same Host', async () => { + const report = await run([{ method: 'GET', path: '/host' }], { host: 'reactome.example' }); + expect(report.counts.identical).toBe(1); + }); + + it('removes the analysis token only through a declared normaliser, leaving every other byte', async () => { + expect((await run(['/token'])).counts.different).toBe(1); + expect( + (await run(['/token'], { normalise: normalisers.analysisResult })).counts.identical + ).toBe(1); + expect((await run(['/gz'], { normalise: normalisers.analysisResultGz })).counts.identical).toBe( + 1 + ); + // Byte for byte apart from the token: no JSON round trip to lose a digit. + const strip = (text) => normalisers.analysisResult(Buffer.from(text)).toString(); + expect(strip('{"summary":{"token":"T1"},"n":9007199254740993}')).toBe( + '{"summary":{"token":""},"n":9007199254740993}' + ); + expect(strip('{"summary":{"token":"T1"},"n":-0}')).not.toBe( + strip('{"summary":{"token":"T1"},"n":0}') + ); + }); + + it('records a failure as an error, with no body or secret in it, and does not stop', async () => { + const report = await run(['/text404', '/same', '/hang'], { + normalise: normalisers.analysisResultGz, + redact: true, + timeout: 300, + }); + expect(report.results.map((r) => r.error !== undefined)).toEqual([true, true, true]); + expect(report.counts.errors).toBe(3); + expect(JSON.stringify(report)).not.toContain('secret-body-text'); + expect(verdict(report)).toBe(2); + const plain = await run(['/same', '/hang'], { timeout: 300 }); + expect(plain.counts).toEqual({ identical: 1, different: 0, errors: 1 }); + }); + + it('never puts a body, and with redaction no path, in its report', async () => { + const report = await run(['/secret/ABC123'], { redact: true }); + expect(JSON.stringify(report)).not.toContain('ABC123'); + expect(report.results[0].request).toBe('#0'); + expect(Object.keys(report.results[0]).sort()).toEqual(['differs', 'request', 'status']); + }); + + it('calibrates against the reference itself: a difference counts only where the reference agreed with itself', async () => { + const report = await run(['/same', '/byte'], { calibrate: true }); + expect(report.results.map((r) => r.calibrated)).toEqual([true, true]); + expect(report.counts).toEqual({ identical: 1, different: 1 }); + // A reference that disagrees with itself is unstable, counted neither way, + // and a run that compared nothing is not a pass. + const live = await run(['/live'], { calibrate: true }); + expect(live.counts).toEqual({ identical: 0, different: 0, unstable: 1 }); + expect(live.results[0].calibrated).toBe(false); + expect(verdict(live)).toBe(2); + }); + + it('asks both at once with --simultaneous, with the same result', async () => { + const report = await run(['/same', '/byte'], { simultaneous: true }); + expect(report.counts).toEqual({ identical: 1, different: 1 }); + }); + + it("keeps a base URL's path, and refuses a scheme or query it cannot honour", async () => { + // Without the prefix both sides would answer the same 404 -- a false pass. + const report = await compareAll([{ method: 'GET', path: '/x' }], { + reference: `${ref.url}/ctx`, + candidate: `${cand.url}/ctx`, + }); + expect(report.counts).toEqual({ identical: 0, different: 1 }); + for (const bad of ['ftp://127.0.0.1:1', `${ref.url}/?q=1`]) + await expect( + compareAll([{ path: '/x' }], { reference: bad, candidate: cand.url }) + ).rejects.toThrow(/base/); + }); + + it('times out the whole request, not just a silence', async () => { + const started = Date.now(); + const report = await run(['/trickle'], { timeout: 300 }); + expect(report.results[0].error).toBe('timeout'); + expect(Date.now() - started).toBeLessThan(2000); + }); + + it('sees a header sent twice', async () => { + const report = await run(['/twice']); + expect(report.results[0].differs).toContain('content-type'); + // And how the body was framed: Content-Length on one side, chunked on the other. + expect(report.results[0].differs).toContain('transfer-encoding'); + }); + + it('names what failed: the normaliser, not the network', async () => { + const report = await run(['/same'], { normalise: normalisers.analysisResultGz }); + expect(report.results[0].error).toBe('normaliser'); + // A token normaliser that finds no token is an error, not a silent pass. + expect(() => normalisers.analysisResult(Buffer.from('{"n":1}'))).toThrow(); + }); + + it('reports a status difference even when the body cannot be normalised', async () => { + const report = await run(['/status'], { normalise: normalisers.analysisResult }); + expect(report.results[0].differs).toContain('status'); + }); + + it('ignores a header only when the run declares it, and says so', async () => { + const report = await run(['/header'], { ignoreHeaders: ['content-disposition'] }); + expect(report.counts.identical).toBe(1); + expect(report.ignoredHeaders).toEqual(['content-disposition']); + }); + + it('lets a reset part way through end at once, leaving no timer behind', async () => { + // Timers of the run's own: the process may hold others (the test runner's). + const timers = () => + process.getActiveResourcesInfo().filter((kind) => kind === 'Timeout').length; + const before = timers(); + const started = Date.now(); + const report = await run(['/reset'], { timeout: 60_000 }); + expect(report.results[0].error).toMatch(/^network/); + expect(Date.now() - started).toBeLessThan(2000); + // A timer left armed would keep the process alive for the full minute. + expect(timers() - before).toBe(0); + }); + + it('reports a status difference even when the error page cannot be decoded', async () => { + const report = await run(['/badgz']); + expect(report.results[0].differs).toEqual(expect.arrayContaining(['status', 'body'])); + expect(verdict(report)).toBe(1); + }); + + it('exits 3, not 1, on a usage error', () => { + const dir = mkdtempSync(join(tmpdir(), 'parity-')); + try { + const requests = join(dir, 'requests.json'); + writeFileSync(requests, JSON.stringify([{ path: '/x' }])); + const cli = join(import.meta.dirname, 'parity.mjs'); + for (const args of [ + ['--reference', 'ftp://x', '--candidate', 'http://127.0.0.1:1', '--requests', requests], + ['--reference', 'not a url', '--candidate', 'http://127.0.0.1:1', '--requests', requests], + [ + '--reference', + 'http://127.0.0.1:1', + '--candidate', + 'http://127.0.0.1:1', + '--requests', + requests, + '--ignore-header', + ], + ]) + expect([args.join(' '), spawnSync(process.execPath, [cli, ...args]).status]).toEqual([ + args.join(' '), + 3, + ]); + } finally { + rmSync(dir, { recursive: true }); + } + }); +});