fix: make Markdown alternates resolve and page moves permanent

[josemontesdeoca]

Why the change

Every docs page advertises a Markdown copy at <slug>.mdx, but six of the seven answered 404 and Accept: text/markdown only worked on /, so agents that followed the advertised link got an HTML error page; now every page serves Markdown both ways, and the old /start and /openprose URLs redirect permanently so search engines can retire them.

Special things to note

• Once the pattern is fixed, the negotiated rewrite matches every path, and Fumadocs's isMarkdownPreferred is true for Accept: text/plain as well as text/markdown. So negotiation only applies to paths with no dot that are outside /_next. /llms.txt, /robots.txt, /sitemap.xml, /llms.mdx/**, /og/**, and /.well-known/** pass through unchanged for any Accept. If a future page slug contains a dot, markdown-alternates.test.ts fails, so the page can't quietly fall back to HTML.
• Only the three page-move redirects become 308. The four harness redirects (/reactor, /sdk, /cli, /reactor-devtools) stay 307 because those routes may host docs again. Browsers cache a 308 for a long time, which is fine because the pages are staying at the root.
• The smoke server now starts with HOSTNAME=0.0.0.0, the same value the Dockerfile uses. With 127.0.0.1, Next treated every proxy rewrite as external and proxied it back to itself, which dropped the Vary header. Production never takes that path.

Change outline

What a client sees:

 GET /setup.mdx                               (every page's advertised alternate)
-  404 text/html
+  200 text/markdown
 GET /setup   Accept: text/markdown | text/plain
-  200 text/html
+  200 text/markdown, Vary: Accept
 <link rel="alternate" type="text/markdown"> on /
-  /.mdx
+  /index.mdx                                 (/.mdx still resolves)
 GET /openprose/contracts
-  307 → /contracts
+  308 → /contracts
 GET /openprose/contracts.mdx
-  307 → /contracts.mdx → 404
+  308 → /contracts.mdx → 200 text/markdown

The cause: proxy.ts put docsRoute ("/") in front of Fumadocs's optional {/*path} group, which already starts with a slash, so both patterns only matched / and double-slash paths. Removing the prefix's trailing slash fixes both patterns and still works if the docs ever move back under a prefix like /docs.

+const docsPrefix = docsRoute.replace(/\/$/, '');
-`${docsRoute}{/*path}.mdx`     // "/{/*path}.mdx" matched "/.mdx" and "//setup.mdx"
+`${docsPrefix}{/*path}.mdx`    // "{/*path}.mdx"  matches "/setup.mdx"

The rewrite decision moves into one exported pure helper, like resolveDocsHostRedirect and hasEncodedPathname. The encoded-path guard from #15 still runs first.

 proxy(request)
   if hasEncodedPathname(pathname)        → 404
   if host is docs.openprose.ai           → 301 docs.prose.md
-  if rewriteSuffix(pathname)             → rewrite
-  if isMarkdownPreferred(request)
-     and rewriteDocs(pathname)           → rewrite
+  target = resolveMarkdownRewrite(pathname, isMarkdownPreferred(request))
+  if target                              → rewrite, with Vary: Accept when negotiated
   otherwise                              → next()

resolveMarkdownRewrite(pathname, prefersMarkdown)
  pathname contains "//"                        → null
  "/index.mdx" or "/.mdx"                       → /llms.mdx/content.md
  "/<slug>.mdx"                                 → /llms.mdx/<slug>/content.md
  prefersMarkdown, no ".", not under /_next     → /llms.mdx/<slug>/content.md
  otherwise                                     → null

A rewrite can only land on a file the build prerendered, or on a 404, because /llms.mdx is fallback: false (#16). A bad guess never serves the wrong body or writes to disk.

 proxy.ts                               ~ prefix fix, resolveMarkdownRewrite(), Vary on negotiated rewrites
 lib/canonical.ts                       ~ root page advertises /index.mdx
 lib/shared.ts                          ~ comment: docsRoute is "/" or a prefix without a trailing slash
 next.config.mjs                        ~ three page-move redirects → permanent: true
 __tests__/proxy.test.ts                + rewrite, negotiation, and non-page pass-through cases
 __tests__/canonical.test.ts            ~ fixtures move to root-mounted paths, root case added
+__tests__/markdown-alternates.test.ts   for every page: alternate and HTML URL → proxy → /llms.mdx handler URL
 scripts/smoke-standalone.ts            + .mdx, negotiation, and redirect probes; records Vary and Location