fix: don't infer the date from response headers

[Kikobeats]

What

addHead injected a <meta name="date"> tag built from the response headers — last-modified, falling back to date, falling back to now - age.

Only last-modified says anything about the content. date is when the response was generated and age is how long it sat in a cache, so every origin that does not send last-modified (most dynamically served pages) got a date equal to the moment of the request.

Seen through microlink:

$ curl "https://api.microlink.io/?url=https://primary.miltoneducation.com"
"date": "2026-08-10T16:40:16.000Z"    # == the response `date` header, i.e. now

Reproduced with no browser involved:

last-modified: undefined
date header  : Mon, 10 Aug 2026 16:44:50 GMT
injected     : <meta name="date" content="2026-08-10T16:44:50.000Z">

metascraper-date reads meta[name="date"] as its first rule, so the injected value won over everything the page actually declares.

Why remove it rather than restrict it to last-modified

Consumers run their own date detection over the returned markup, so there is nothing here to compensate for. Inferring metadata from transport headers only creates values the page never claimed.

The insert was also guarded by meta[property="article:published_time"] while inserting meta[name="date"] — a page carrying its own date got a second, header-derived one appended. That goes away with the tag.

Tests

• test/html/add-head.js — no date is inferred even when last-modified, date and age are all present; a date already in the markup survives untouched.
• test/url.js / test/snapshots/index.js.md — expectations drop the injected tag.
• test/index.js, test/pdf.js — drop the $('meta[name="date"]').remove() scrubbing that existed only to make the injected tag deterministic.
• getDate is gone; it had no consumer outside its own test.

Full suite green, except the pre-existing getaddrinfo ENOTFOUND notexisturl.dev uncaught exception in test/index.js, which fails the same way on master.

🤖 Generated with Claude Code

https://claude.ai/code/session_01E1yARH3MU7jQYm5J8cAhtR

---

Note

Medium Risk
Changes metadata emitted for downstream scrapers (e.g. microlink/metascraper-date); behavior is intentional but may shift published-date fields for URLs that relied on the old injection.

Overview
Stops injecting <meta name="date"> from HTTP response headers during HTML rewrite (addHead). The removed getDate helper and its export are deleted entirely.

Previously, last-modified, date, or age could produce a date tag that metascraper-date treated as authoritative—even when it only reflected fetch/cache timing, not page content. Dates already in the markup are unchanged.

Tests add test/html/add-head.js for the new behavior, drop test/html/get-date.js, and refresh snapshots/integrations that no longer strip an injected date for determinism.

^{Reviewed by Cursor Bugbot for commit 7634516. Bugbot is set up for automated code reviews on this repo. Configure here.}

Summary by CodeRabbit

• Bug Fixes

  • HTML pages no longer automatically include article:published_time metadata derived from response headers.
  • Existing date metadata in page markup remains unchanged.

• Tests

  • Updated rendering and snapshot coverage to reflect the removal of automatically generated date metadata.
  • Added coverage confirming response headers do not create date metadata.

[coderabbitai]

📝 Walkthrough

Walkthrough

Removed response-header date extraction and generated date metadata from HTML rendering. Removed the getDate export. Updated HTML, URL, image, and PDF tests to validate the new output.

Changes

Date Metadata Removal

Layer / File(s)	Summary
Remove date extraction and validate HTML output src/html.js, test/html/add-head.js, test/url.js, test/index.js, test/pdf.js	addHead no longer derives dates from response headers or inserts article:published_time. Existing markup dates remain unchanged. Updated snapshots omit generated date metadata.
Formatting-only cleanup and test updates src/html.js, test/url.js, test/index.js	Reformatted unchanged HTML processing code and simplified test callbacks and URL setup.

Estimated code review effort: 2 (Simple) | ~15 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ResponseHeaders
  participant addHead
  participant RenderedHTML
  ResponseHeaders->>addHead: provide response headers
  addHead->>RenderedHTML: preserve existing date metadata
  addHead-->>RenderedHTML: omit generated date metadata

Loading

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: removing date inference from response headers.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/date-header-timestamp

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/html.js`:
- Around line 23-35: Remove getDate and PUBLISHED_TIME_SELECTOR from src/html.js
(lines 23-35), and remove the header-derived date insertion block at src/html.js
(lines 54-56). Delete the obsolete getDate tests in test/html/get-date.js (lines
12-30), and update test/html/add-date.js (lines 16-18) so the last-modified case
asserts that no date meta tag is injected.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 9b74ac5a-0f1a-41d7-a08a-711966ca81f3

📥 Commits

Reviewing files that changed from the base of the PR and between c8100df and 279a3e0.

📒 Files selected for processing (4)

• src/html.js
• test/html/add-date.js
• test/html/get-date.js
• test/url.js

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/html/add-head.js`:
- Line 19: Update the header assertions in the relevant test cases to directly
verify the exact date metadata element, including both name="date" and its
expected content, rather than checking only the timestamp text or absence of a
name attribute. Preserve the existing coverage for non-injection and
preservation behavior.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 1a3ffe1a-35ed-46b9-a403-f658d27dec9c

📥 Commits

Reviewing files that changed from the base of the PR and between 279a3e0 and 6ecedbb.

⛔ Files ignored due to path filters (1)

• test/snapshots/index.js.snap is excluded by !**/*.snap

📒 Files selected for processing (6)

• src/html.js
• test/html/add-head.js
• test/html/get-date.js
• test/index.js
• test/pdf.js
• test/snapshots/index.js.md

💤 Files with no reviewable changes (2)

• test/html/get-date.js
• test/snapshots/index.js.md