fix(ai): bound schema context length and handle context overflow errors

[Maaz2212]

Closes #1422

Problem

When generating database documentation for wide schemas via Docs > AI Describe, DatabaseDocs.tsx forwarded every column of up to 50 tables without a length budget. On databases with wide tables (e.g. 60 tables × 600 columns), this produced payloads exceeding 400,000 tokens. The LLM provider rejected the request with a context length error, which isRetryableError treated as a retryable stream error—triggering 3 wasteful server-side retries before displaying raw provider error text.

Additionally, the Data Profiler previously forwarded the entire multi-table database schema (conn.schemaContext) to /api/ai/describe-schema, causing profiler requests on larger databases to exceed the context budget and silently fail without displaying an error.

Summary of Changes

1. Shared Budget Constant & Helper (src/lib/llm/types.ts):

   • Exported MAX_SCHEMA_CONTEXT_CHARS = 30_000 as a single shared source of truth for both the client and the route.
   • Added isContextLengthError(error) matching llama.cpp's "exceeds the available context size" error message.
   • Updated isRetryableError() to return false when isContextLengthError(error) is true, halting the server retry loop immediately on attempt 1.

2. Client-Side Truncation (src/components/DatabaseDocs.tsx):

   • Constructs schemaContext respecting MAX_SCHEMA_CONTEXT_CHARS.
   • Subtracts the truncation note length ("\n\n[Note: Schema was truncated to fit model context limit]") and separator lengths from the budget upfront, guaranteeing that the serialized context never exceeds the budget.
   • When a table exceeds the remaining budget, its full column list is omitted in favor of a concise summary (Table: name (N rows) - [M columns omitted for length]) to maximize schema coverage before falling back to dropping subsequent tables.

3. Data Profiler Single-Table Scoping & Error Visibility (src/components/DataProfiler.tsx):

   • Extracts and sends only the profiled table's schema (instead of the entire database schema JSON), keeping profiler summary requests well within the 30,000-character budget.
   • Added error handling in the AI Analysis panel to display the error message if the route refuses instead of failing silently.

4. Route Guards & Validation (src/app/api/ai/describe-schema/route.ts):

   • Validates that schemaContext is a string (HTTP 400 if missing or not a string).
   • Validates that databaseType is a short string up to 64 characters (HTTP 400 if invalid or oversized).
   • Rejects schemaContext exceeding MAX_SCHEMA_CONTEXT_CHARS with HTTP 413 Payload Too Large.
   • All validation checks execute upfront before createLLMProvider() is called.

5. Neutral Error Mapping (src/lib/api/errors.ts):

   • When an LLMStreamError is caused by a context-length refusal, maps it to HTTP 502 with retryable: false and the neutral copy: "The request is too large for the configured model's context." (shared cleanly across Describe Schema, Explain, and Query Safety routes).

Tests

• tests/unit/llm/retry.test.ts: Verified that context-length errors fail immediately without retrying (1 attempt), while standard stream errors are still retried up to maxAttempts (3 attempts).
• tests/api/ai/describe-schema.test.ts: Verified HTTP 413 rejection for oversized contexts without calling provider, HTTP 400 rejection for non-string contexts and oversized databaseType without calling provider, and 200 for valid requests.
• tests/components/DatabaseDocs.test.tsx: Verified that wide schemas generate a bounded schemaContext with the truncation notice and strictly within MAX_SCHEMA_CONTEXT_CHARS.
• tests/components/DataProfiler.test.tsx: Verified that large schema contexts (>30,000 characters) still send profiler requests within the budget, and verified that route refusal errors are displayed in the AI Analysis card.
• tests/unit/api-errors.test.ts: Verified that context-length stream errors return 502 with retryable: false and the neutral error message.
• Passed bun run typecheck and bun run format.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[cevheri]

Thanks @Maaz2212, the core of this works, and it follows the plan agreed in #1422. We ran it end to end in a browser through Docs > AI Describe, on a Postgres database with 60 tables of 601 columns and an LLM stub that answers like llama.cpp:

• the request dropped from about 1 MB to 24 KB and the description came back;
• with a small-context model the refusal is tried once, not three times, and the banner shows a readable sentence instead of the raw provider text;
• a two-table schema sends the same request as before.

Three things before merge.

1. The context-length sentence must not say "schema" on every route

• Cause: the new branch in createErrorResponse (src/lib/api/errors.ts) is shared by Explain, Query Safety and the agent routes, so all of them now answer "The schema is too large for the configured model's context." Explain with a long plan and no schema said exactly that. The wording came from our issue text, so that one is on us.
• Change: use a neutral sentence, for example "The request is too large for the configured model's context.", and keep retryable: false.
• Done when: tests/unit/api-errors.test.ts expects the new sentence and retryable: false.

2. The route must refuse every oversized input, not only a string schemaContext

• Cause: src/app/api/ai/describe-schema/route.ts checks the length only when schemaContext is a string, and puts databaseType into the system prompt with no check at all. Measured against the stub: {"schemaContext": ["x".repeat(200000)]} reached the provider (200,666 bytes upstream), and so did {"schemaContext": "t", "databaseType": "x".repeat(200000)} (200,664 bytes). #1422 step 2 asks the route to refuse an over-budget request before calling the provider.
• Change: before createLLMProvider is called, answer 400 with a plain sentence when schemaContext is not a string, or when databaseType is present and is not a short string (for example up to 64 characters). Then keep your length check and its 413 as they are.
• Done when: route tests in tests/api/ai/describe-schema.test.ts send an array schemaContext and a 200,000-character databaseType, each gets 400, and each asserts the provider was never created.

3. The 413 test must check what its name says

• Cause: the test named "refuses ... without calling provider" asserts the 413 but never checks the provider. #1422 asks for that assertion.
• Change: add expect(mockCreateLLMProvider).not.toHaveBeenCalled() to it.
• Done when: that assertion is in the test, and it fails if the guard is moved after the provider call.

Please run bun run test:coverage && bun run coverage:check again after the change. One small note on the PR body: a table that does not fit loses all of its columns (name and column count are kept), not only its trailing ones. The code is fine, only the description differs. We will file the follow-ups ourselves (tables past the 50th are still dropped without the note, and the user is never told the docs cover part of the schema).

[Maaz2212]

Sure @cevheri i am doing this right now and fixing the PR description also.

[cevheri]

Thanks, all three points are done and CI is green. One thing we missed last round:

1. Data Profiler loses its AI summary on larger schemas. DataProfiler.tsx:92 sends the whole conn.schemaContext (the full schema as JSON) to this route. Above 30,000 characters the route now answers 413, and DataProfiler.tsx:110 returns without a word, so the summary silently disappears where it worked on main.
   Change: send only the profiled table's schema from Data Profiler, and show the error if the route still refuses.
   Done when: a test with a schema context over 30,000 characters still gets a profiler summary request under the limit.
2. PR body. It still quotes the old "schema is too large" sentence and does not mention the new 400 checks on schemaContext and databaseType.
   Done when: the body matches the code.

[Maaz2212]

Hey @cevheri

Done! I’ve updated DataProfiler and covered both points:

1. Scoped to profiled table: DataProfiler now extracts and sends only the profiled table’s schema from schemaContext instead of forwarding the entire multi-table database schema.
2. Visible errors: Replaced the silent return on failed requests with setAiError, so route refusals or errors surface properly in the UI.
3. Tests & Coverage: Added tests verifying that schemas over 30,000 characters still send requests under the limit, route refusals display their error message, and all branches maintain 100% line coverage.

Also updated the PR description to match the latest changes. Ready for another look!

[cevheri]

Thanks @Maaz2212, the PR body now matches the code and the profiler shows the route's error. One thing is left from the last review.

1. Send the profiled table's schema, not a name match.
   Cause: resolveTableSchemaContext in DataProfiler.tsx parses schemaContext and matches on the bare table name first, so with public.users and app.users in the schema, profiling app.users sends public.users. The new test does not catch it: replacing the helper call with the raw schemaContext keeps all profiler tests green, because the 30,000 slice alone keeps the request under the limit.
   Change: the component already has tableSchema, resolved on the full path. Send JSON.stringify(tableSchema) and drop the schemaContext parsing.
   Done when: a test with two schemas that each hold a users table sends only the profiled one's columns, and that test fails if the helper goes back to the raw schemaContext.