Skip to content

model: fix all granite family parameter counts - #28643

Open
taronaeo wants to merge 3 commits into
ggml-org:masterfrom
taronaeo:fix/granite-3.3-parameter-count
Open

model: fix all granite family parameter counts#28643
taronaeo wants to merge 3 commits into
ggml-org:masterfrom
taronaeo:fix/granite-3.3-parameter-count

Conversation

@taronaeo

@taronaeo taronaeo commented Sep 9, 2026

Copy link
Copy Markdown
Member

Overview

cont: #28632

After #28632, I realised that quite a lot of our Granite family parameter counts are either inaccurate (2B model reported as 3B), missing, or using dense model naming schemes when the model is MoE or hybrid.

This PR fixes all Granite family parameter counts reported in llama-bench.

Additional information

-hf Original Updated
bartowski/granite-3.0-2b-instruct-GGUF:Q2_K granite 2B Q2_K - Medium No changes
bartowski/granite-3.0-8b-instruct-GGUF:Q2_K granite 8B Q2_K - Medium No changes
bartowski/granite-3.0-1b-a400m-instruct-GGUF:Q3_K_S granitemoe 1B.A400M Q3_K - Small No changes
bartowski/granite-3.0-3b-a800m-instruct-GGUF:Q2_K granitemoe 3B Q2_K - Medium granitemoe 3B.A800M Q2_K - Medium
bartowski/granite-3.1-2b-instruct-GGUF:Q2_K granite 2B Q2_K - Medium No changes
bartowski/granite-3.1-8b-instruct-GGUF:Q2_K granite 8B Q2_K - Medium No changes
bartowski/granite-3.1-1b-a400m-instruct-GGUF:Q2_K granitemoe 1B.A400M Q2_K - Medium No changes
bartowski/granite-3.1-3b-a800m-instruct-GGUF:Q2_K granitemoe 3B Q2_K - Medium granitemoe 3B.A800M Q2_K - Medium
bartowski/ibm-granite_granite-3.2-2b-instruct-GGUF:Q2_K granite 2B Q2_K - Medium No changes
bartowski/ibm-granite_granite-3.2-8b-instruct-GGUF:Q2_K granite 8B Q2_K - Medium No changes
ibm-granite/granite-3.3-2b-instruct-GGUF:Q2_K granite 2B Q2_K - Medium No changes
ibm-granite/granite-3.3-8b-instruct-GGUF:Q2_K granite 8B Q2_K - Medium No changes
ibm-granite/granite-4.0-micro-GGUF:Q2_K granite 3B Q2_K - Medium No changes
ibm-granite/granite-4.0-h-micro-GGUF:Q2_K granitehybrid 3B Q2_K - Medium No changes
ibm-granite/granite-4.0-h-tiny-GGUF:Q2_K granitehybrid 7B.A1B Q2_K - Medium No changes
ibm-granite/granite-4.0-h-small-GGUF:Q2_K granitehybrid 32B Q2_K - Medium granitehybrid 32B.A9B Q2_K - Medium
unsloth/granite-4.0-h-350m-GGUF:Q3_K_S granitehybrid 350M Q3_K - Small No changes
unsloth/granite-4.0-h-1b-GGUF:Q2_K granitehybrid 1B Q2_K - Medium No changes
ibm-granite/granite-4.1-3b-GGUF:Q2_K granite 2B Q2_K - Medium granite 3B Q2_K - Medium
ibm-granite/granite-4.1-8b-GGUF:Q2_K granite 8B Q2_K - Medium No changes
ibm-granite/granite-4.1-30b-GGUF:Q2_K granite ?B Q2_K - Medium granite 30B Q2_K - Medium
ibm-granite/granite-4.2-3b-GGUF:Q2_K granite 2B Q2_K - Medium granite 3B Q2_K - Medium
ibm-granite/granite-4.2-8b-GGUF:Q2_K granite 8B Q2_K - Medium No changes
ibm-granite/granite-4.2-30b-GGUF:Q2_K granite ?B Q2_K - Medium granite 30B Q2_K - Medium

cc: @gabe-l-hart

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: No, painstakingly checked every model 🙃

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
@taronaeo
taronaeo requested a review from CISC as a code owner September 9, 2026 10:26
@github-actions github-actions Bot added the model Model specific label Sep 9, 2026
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
Comment thread src/models/granite-moe.cpp Outdated
Comment thread src/llama-model.cpp Outdated
Comment thread src/llama-model.h Outdated
Comment thread src/models/granite.cpp Outdated

@gabe-l-hart gabe-l-hart left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for cleaning all this up! This has been on my TODO list for a while, but never come to the top. Your question about GraniteMoe w/ 40 layers surfaced that the source of that line was an accidental copy over during refactors, so it would be great to strip it out now while this cleanup is going on.

Comment thread src/models/granite-moe.cpp Outdated
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
@taronaeo

taronaeo commented Sep 9, 2026

Copy link
Copy Markdown
Member Author

Latest push addressed review comments + removed the unused Granite MoE 40 layers check.

@taronaeo taronaeo added the merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. label Sep 9, 2026

@gabe-l-hart gabe-l-hart left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. model Model specific

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants