Skip to content
This repository was archived by the owner on Oct 16, 2026. It is now read-only.

Enable OV gguf frontend via llama.cpp openvino backend - #3

Open
preetha-intel wants to merge 2 commits into
ravi9:dev_backend_openvinofrom
preetha-intel:ov-gguf-frontend
Open

preetha-intel wants to merge 2 commits into
ravi9:dev_backend_openvinofrom
preetha-intel:ov-gguf-frontend

Conversation

@preetha-intel

@preetha-intel preetha-intel commented Oct 15, 2026 •

Copy link
Copy Markdown

Add GGML decoder to support OV GGUF Frontend.

Overview

This PR removes the legacy in-tree OpenVINO translation layer from ggml-openvino and moves the backend onto the OpenVINO GGUF frontend (openvino::frontend::gguf).

Until now the backend carried its own copy of the GGML→OpenVINO conversion: a decoder interface, node context, per-op translators, graph-fusion passes and a translate session, all under ggml/src/ggml-openvino/openvino/.
After this change, the frontend owns the generic op translations and passes. The backend keeps a GgmlOvDecoder bridge that exposes the ggml graph to the frontend, plus the runtime pieces (device handling, KV cache, compile and cache logic).

Changes

  • Removed the legacy bridge. The whole ggml-openvino/openvino/ tree is deleted: decoder.h, frontend., input_model., node_context.h, translate_session., utils., all of op/* (about 55 translators) and the fusion passes in pass/*. These include fuse_to_sdpa, fuse_to_conv, fuse_moe_router, squeeze_matmul and the dequantization marking passes.
  • Reworked GgmlOvDecoder (ggml-decoder.cpp/.h). It is now the bridge that exposes the ggml cgraph, tensor metadata and model-level info to the GGUF frontend.
  • Simplified ggml-openvino.cpp and utils.cpp. Graph conversion now goes through the frontend, and the per-op support logic that duplicated the translators is gone.
  • Moved the KV-state sequence-axis helper. It went from openvino/pass/kv_state_seq_axis.{h,cpp} to kv-state-seq-axis.{h,cpp} in the backend directory, because it is runtime logic and not a frontend pass.

CMake changes

  • ggml-openvino now links openvino::frontend::gguf.
  • Configuration fails with a clear error if that target isn't found.
  • The source globs are no longer recursive, since the subdirectories are gone.

OpenVINO PR enabling this flow : openvinotoolkit/openvino#38769

This PR is dependent on OV PR to be merged.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant