Skip to content

feat: add Granite Docling model support - #13

Open
reacher-z wants to merge 1 commit into
InternScience:mainfrom
reacher-z:feat/granite-docling-support
Open

feat: add Granite Docling model support#13
reacher-z wants to merge 1 commit into
InternScience:mainfrom
reacher-z:feat/granite-docling-support

Conversation

@reacher-z

Copy link
Copy Markdown

Summary

  • add a Transformers adapter for IBM Granite Docling 258M document conversion
  • register granite-docling-258M in supported_VLM
  • preserve image/text ordering and return DocTags without stripping structural special tokens
  • support CUDA, MPS, CPU, custom model-loading kwargs, and optional device_map
  • add focused regression tests for multimodal message construction, model loading, device placement, generation slicing, and DocTags decoding

This is a third independent new foundation-model integration, one of the major-contribution categories described in the SciEvalKit README. It is separate from the LFM2-VL and nanoVLM integrations in #11 and #12.

Official model card and inference API: https://huggingface.co/ibm-granite/granite-docling-258M

Validation

  • python3 -m pytest -q tests/test_granite_docling.py — 2 passed
  • python3 -m py_compile scieval/config.py scieval/vlm/__init__.py scieval/vlm/granite_docling.py tests/test_granite_docling.py
  • pre-commit hooks pass for the new adapter, import, and tests
  • real CPU smoke test with transformers==4.57.1, torch==2.8.0, the public 258M weights, and the official sample arXiv page: the adapter loaded a cached checkpoint in 10.69s, generated 8 tokens in 40.53s, and returned the expected DocTags prefix <doctag><page_header><loc_100><loc
  • a longer uncached official-API run generated <doctag><page_header>...Energy Budget of WASP-121 b..., confirming page-content extraction

The full config file has pre-existing lint violations outside this diff, so validation also included git diff --check and compilation of the modified config.

AI assistance disclosure

OpenAI Codex assisted with implementation, tests, and PR drafting. I ran and inspected both the focused tests and real-model smoke inference before submission.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant