Skip to content

common : reject non-positive batch size (#28525) - #28639

Closed
goodruyas wants to merge 1 commit into
ggml-org:masterfrom
goodruyas:fix-batch-size-zero
Closed

common : reject non-positive batch size (#28525)#28639
goodruyas wants to merge 1 commit into
ggml-org:masterfrom
goodruyas:fix-batch-size-zero

Conversation

@goodruyas

@goodruyas goodruyas commented Sep 9, 2026

Copy link
Copy Markdown

Overview

Currently, if you pass --batch-size 0 or a negative number to llama-server, it triggers a GGML assertion crash deep inside context initialization – not a great user experience. This PR fixes that by:

  • Adding validation in the CLI parser (common/arg.cpp) to reject non-positive values outright, with a clear error message.

  • Strengthening the check inside llama_init_from_model so it now explicitly rejects n_batch == 0, instead of only complaining when both n_batch and n_ubatch are zero.

  • Adding a regression test (test-arg-parser).

  • Fixes Eval bug: --batch-size 0 reaches an uncaught GGML assertion #28525

Additional information

  • --ubatch-size 0 is left unchanged because it already means "follow n_batch" in llama_init_from_model.

Test Plan

  • Build: CPU backend, commit 304665f, GCC 16.2.1, Arch Linux x86_64

  • ctest --test-dir build -R test-arg-parser --output-on-failure: 1/1 passed

  • Manual: llama-server --batch-size 0 exits with a clean argument error; no GGML_ASSERT.

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES – I used AI assistance for brainstorming and to sketch out the initial validation logic and test boilerplate. However, I personally reviewed every single line, fully understand the changes, and can explain the reasoning behind each modification to reviewers. The core decisions – where to put the checks, the exact error wording, and test coverage – were made by me. This description and any future comments are my own, not AI-generated.

@goodruyas
goodruyas requested review from a team and ggerganov as code owners September 9, 2026 09:08
@github-actions github-actions Bot added the testing Everything test related label Sep 9, 2026
@am17an am17an closed this Sep 9, 2026
@goodruyas

Copy link
Copy Markdown
Author

Thanks for taking a look. I noticed this PR was closed without comments — could you point me to the preferred direction for fixing #28525? If someone is already handling it, I'll leave it to them. I'm new to this repo and want to learn the right way to contribute

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Eval bug: --batch-size 0 reaches an uncaught GGML assertion

2 participants