Skip to content

fix(fetch): add fallback for streaming SSR content and support configurable timeout - #4659

Open
AbhiPra24 wants to merge 1 commit into
modelcontextprotocol:mainfrom
AbhiPra24:fix/fetch-ssr-fallback-and-configurable-timeout
Open

fix(fetch): add fallback for streaming SSR content and support configurable timeout#4659
AbhiPra24 wants to merge 1 commit into
modelcontextprotocol:mainfrom
AbhiPra24:fix/fetch-ssr-fallback-and-configurable-timeout

Conversation

@AbhiPra24

Copy link
Copy Markdown

Summary

Fixes #3878 and #4448.

1. Fallback for Progressive/Streaming SSR Content (Fixes #3878)

  • Problem: Mozilla Readability (readabilipy) strips HTML elements styled with visibility: hidden or off-screen positioning commonly used in modern SSR skeleton hydration (Next.js, Remix). Only the loading shell text survived, silently dropping up to 85% of the page.
  • Fix: In extract_content_from_html(), if Readability returns empty content or extracted content that is significantly shorter than the source HTML (< len(html) * 0.05), fall back to markdownifying the source HTML directly via markdownify.

2. Configurable Timeout (Fixes #4448)

  • Problem: 30-second network timeout was hardcoded in fetch_url.
  • Fix: Added DEFAULT_TIMEOUT = 30.0, support for MCP_FETCH_TIMEOUT environment variable, --timeout CLI flag, and an optional timeout parameter in the Fetch tool input schema.

Verification

  • Added 16 new test cases in src/fetch/tests/test_server.py covering SSR fallback, timeout resolution precedence, robots.txt timeout, and Fetch model validation.
  • pytest: 36/36 passed.
  • ruff check and pyright: 0 errors.

cc @olaservo for review when available. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fetch: hardcoded 30s timeout with no CLI/env/per-request override mcp-server-fetch drops SSR content from streaming/progressive rendering sites

1 participant