Skip to content

Fix and accelerate LLM tabular background runs - #1178

Merged
Paul Lizer (paullizer) merged 3 commits into
microsoft:feature/tabular-unbounded-analysisfrom
paullizer:fix/tabular-direct-source-backed-runs
Aug 10, 2026
Merged

Fix and accelerate LLM tabular background runs#1178
Paul Lizer (paullizer) merged 3 commits into
microsoft:feature/tabular-unbounded-analysisfrom
paullizer:fix/tabular-direct-source-backed-runs

Conversation

@paullizer

@paullizer Paul Lizer (paullizer) commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Summary - Fixes streaming direct source-backed tabular queue calls by passing required model and settings arguments with explicit keywords. - Automatically retries LLM row-count/schema/source-token validation failures, then preserves manual Continue from durable checkpoints after automatic retries are exhausted. - Keeps every row-level answer LLM-generated while increasing independent model-call concurrency through adaptive 4/16/64/128 tiers; explicit administrator concurrency still overrides the tier. - Sizes source batches from selected-model metadata or optional input/output token-limit fields in the local model capability catalog, with bounded fallback budgets, 50% input and 60% output targets, and a 180k soft input cap for contexts over 500k tokens. - Calculates ETA from recent wall-clock rows/minute across parallel windows instead of summing concurrent request durations as serial work. - Persists progress once per completed parallel window and exposes throughput, configured concurrency, and effective concurrency on the chat status card. - Bumps the application to 0.250.136 and updates feature/test documentation. ## Production evidence - The 30,000-row source staged as 909 batches of about 33 rows under the prior 60k-character budget. - Runtime used concurrency 3 and reported about 35 hours remaining because concurrent request durations were serialized in ETA math. - At the observed roughly 155 seconds per 33-row model call, 100-128 effective asynchronous calls project the same LLM-only workload into roughly the 18-25 minute range before validation retries. Actual time remains model- and output-volume-dependent. ## Validation - python -m py_compile application/single_app/config.py application/single_app/functions_tabular_generated_exports.py functional_tests/test_tabular_row_orchestration_scale.py ui_tests/test_chat_background_generated_export_status.py - python functional_tests/test_tabular_row_orchestration_scale.py - node --check application/single_app/static/js/chat/chat-messages.js - python -m json.tool application/single_app/static/json/model_capabilities.json - python -m pytest ui_tests/test_chat_background_generated_export_status.py -q (5 skipped locally because authenticated UI target variables are not configured) - VS Code diagnostics: clean for all changed code/test files - git -c core.whitespace=blank-at-eol,blank-at-eof,space-before-tab,cr-at-eol diff --check

Refs #1031

@paullizer

Copy link
Copy Markdown
Contributor Author

Added retry capability for failed tabular generated exports:

  • Model-output validation failures now auto-requeue up to abular_generated_output_model_validation_auto_retries (default 3).
  • After the auto budget is exhausted, the run is marked failed but remains retryable so the existing manual Continue action can resume from checkpoints.
  • Scheduler auto-claim skips exhausted failures; manual Continue clears the exhausted state and queues another attempt.
  • Added regression coverage for auto retries followed by manual-only continuation.

Validation rerun:

  • .\.venv\Scripts\python.exe -m py_compile application\single_app\functions_tabular_generated_exports.py application\single_app\functions_settings.py application\single_app\config.py functional_tests\test_tabular_row_orchestration_scale.py
  • .\.venv\Scripts\python.exe functional_tests\test_tabular_row_orchestration_scale.py

@paullizer Paul Lizer (paullizer) changed the title Fix streaming direct tabular queue call sites Fix and accelerate LLM tabular background runs Aug 9, 2026
@paullizer
Paul Lizer (paullizer) merged commit 7f15e05 into microsoft:feature/tabular-unbounded-analysis Aug 10, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant