fix(web-search): reduce domain entries on every route, not just site:

[sroussey]

What was wrong

normalizeDomain — which strips a scheme, a www. prefix and trailing slashes — was applied in exactly one place: normalizeAll inside queryOperators.ts, on the way into a site: operator. A provider declaring domainFilter: "native" received the caller's entry verbatim.

unusableDomainEntries validates against the normalized form on purpose (" https://a.com/ " is a perfectly good domain), so a scheme-bearing, www.-prefixed or trailing-slash entry passes validation on every route and is then reduced on one route only. The raw value is forwarded to body.include_domains (Tavily), tool.filters.allowed_domains (OpenAI), tool.allowed_domains / blocked_domains (Anthropic) and plugin.include_domains (OpenRouter) — all of which document bare hosts.

The concrete failure

{ provider: "auto", includeDomains: ["https://arxiv.org/"] }:

• routed to Brave/SearXNG, the query becomes cats site:arxiv.org and the restriction holds;
• routed to Tavily/OpenAI/Anthropic/OpenRouter, the vendor is handed "https://arxiv.org/" — either zero results for a filter that should have matched, or, for a vendor that skips an entry it cannot parse, an unrestricted web search reported as successful, with provider set and nothing saying the filter was discarded.

That is the trade this package refuses everywhere else — the date-filter rule and the empty-entry refusal both say a bound reported as honored on a search that ran unfiltered is worse than a refused request. It is also not reproducible from the caller's side, because "auto" picks the route at run time.

What changed

• adaptRequest reduces includeDomains / excludeDomains once, before either direction is decided, and both the native pass-through and the site: translation read that same value. The two directions stay decided separately, so a split provider (domainFilter: "native", excludeDomainFilter: "query-operator") gets a reduced native include list and a reduced -site: clause.
• Refusal behaviour is untouched: assertUsableDomains still runs first, for every provider, and an entry that names no domain is still an error rather than a silent drop. normalizeAll's filter stays in queryOperators as the structural backstop it was added to be — nothing spliced into a query can restructure it whether or not a call site validated first.
• domainInput is now exported from common.ts, so a host writing its own native provider can apply the same reduction instead of re-deriving it.

No new refusals and no new normalization rule — the same reduction, on both routes.

Tests

Two new cases in WebSearchTask.test.ts, both run red against the unfixed code and green after (confirmed by running them):

• a native provider and a query-operator provider given the identical includeDomains: ["https://ArXiv.org/"] / excludeDomains: ["www.spam.example"], asserting the native adapter sees ["arxiv.org"] / ["spam.example"] and the operator route builds cats site:arxiv.org -site:spam.example;
• a split provider, asserting the natively-filtered direction is reduced too.

One existing test was renamed from "passes domain lists through untouched to a native provider" to "…rather than into the query" — its assertion is unchanged, but "untouched" now contradicts the contract.

Verified

• bunx vitest run packages/web-search/src — 15 files, 137 tests, all pass (includes the existing "domain entries that name no domain" refusal block).
• bunx vitest run over the four vendor *WebSearchProvider.test.ts suites (gemini, anthropic, openai, openrouter) — 50 tests, all pass.
• bunx turbo run build-types for @workglow/web-search and the four vendor packages — clean.
• bunx oxfmt --check packages/web-search/src and bunx oxlint packages/web-search/src — clean.

Not verified here: the full repo test suite, bun run lint across the whole tree, and the SearXNG integration test (it is skipped without WEB_SEARCH_SEARXNG_URL). No live vendor API was called — the adapter assertions are on the request objects, as the existing suites already are.

🤖 Generated with Claude Code

https://claude.ai/code/session_01T8DU24G9GWuRUJncc5TJFZ

---

Generated by Claude Code