You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
onnx: a public ONNX-native int8 export cannot even create a session (no ConvInteger kernel in the pinned ORT) #1436
Found while measuring the TGPNPU vision rows against our ORT plugin EP (#647). It is not an EP problem: the session never
starts.
Reproduced
Xenova/yolov8n-pose (AGPL-3.0), the onnx/model_int8.onnx export, 3,755,740 B — a very common shape of public
"quantized ONNX" (Hugging Face Optimum / Xenova exports). Through our runner:
$ build/ort_run --model model_int8.onnx --input images=in.bin:float32:1x3x640x640
onebit-ort-run: FAIL create session: Could not find an implementation for
ConvInteger(10) node with name '/model.0/conv/Conv_quant'
The graph is 739 nodes, all in the default ai.onnx domain, opset 17: ConvInteger 73, DynamicQuantizeLinear 65, Mul/Add/Cast/Sigmoid/Reshape, and no MatMul. So it is not a graph our EP could serve either way — but it does not
get that far, because ORT 1.23.2 (our pin, docs/onnx.md "The pin") has no CPU kernel for ConvInteger.
Why it matters beyond one row
"Quantized ONNX" is at least two formats, and our docs and boards do not currently distinguish them:
Xenova/yolov8n-posemodel_int8.onnx, many Optimum exports
ConvInteger, MatMulInteger, DynamicQuantizeLinear
does not load at all
So a row that "fails" can be the runtime's op coverage rather than the model or our EP, and a reader cannot tell from the current
board. That is the class of misattribution this project keeps trying to avoid.
Options (not decided here)
Say it out loud and route around it: the OOB path for such repos uses the fp32 export when the repo ships one (it does
here: onnx/model.onnx, 13,484,153 B, 311 nodes, 73 Conv, runs at 62 ms CPU), and the board records the int8 variant as
"runtime has no ConvInteger" instead of "model fails".
Build the pinned ORT with the quantized-op kernels — needs a check whether 1.23.2 ships ConvInteger for int8
activations at all, or only uint8 (the ONNX spec allows both; the error does not say).
Support it in our EP — out of scope for a plugin EP: ConvInteger is a conv, and our seam is MatMul.
Acceptance for this issue
A row that ships an ONNX-native int8 export either runs, or says in one line that the pinned runtime has no kernel for it and
which export of the repo does run. No silent "does not load".
Found in: ~/amd-oss/ONNX-VISION-OOB.md (§4a's trap note) and the GPNPU-PARITY.md YOLOv8-pose row, both of which now record
both exports separately.
Second data point, different model, same exporter family — so this is not one repo's quirk.
onnx-community/yolo26n-ONNX, onnx/model_int8.onnx, 2,848,000-ish B (the same optimum/onnx-community export pipeline that produced the Xenova file):
graph: 999 nodes, ConvInteger 102, DynamicQuantizeLinear 92, MatMulInteger 0, Q/DQ 0 (opset 18, all ai.onnx)
$ build/ort_run --model model_int8.onnx --input pixel_values=in.bin:float32:1x3x640x640
onebit-ort-run: FAIL create session: Could not find an implementation for ConvInteger(10) node with name '/model.0/conv/Conv_quant'
Identical failure, identical message, at session creation — and note the counts: every conv in the model is a ConvInteger, so for these exports there is nothing to fall back to inside the graph.
Two things this sharpens for the fix chosen in this issue:
the fp32 export of the same repo runs fine (onnx/model.onnx, 9,890,454 B, 397 nodes, 102 Conv, 4 MatMul): 1,270 ms on the CPU EP, and 1,797 ms with our plugin EP registered — which claimed 0 of 400 offered nodes because those four matmuls are dynamically shaped. So option 1 in the issue (route the row to the repo's fp32 export, say so on the board) is measured to work on a second row now.
the failure is a runtime op-coverage question, not a model or an EP question: ConvInteger is ai.onnx opset 10, and our pin is 1.23.2. Whether the pin implements it only for uint8 activations (the ONNX spec allows int8 and uint8) is still unchecked — that is the one thing worth knowing before deciding between "document and route around it" and "build the pin differently".
No action requested; recorded so the next lane that meets a model_int8.onnx from this exporter does not spend an hour on it.
Answering this issue's own open question (option 2): the pinned ORT 1.23.2 does implement ConvInteger — but only for uint8 × uint8. Measured with three minimal single-node graphs, each run through our own runner with the vendored pinned ORT:
probe (ConvInteger, opset 10, IR 10)
session
x uint8, W uint8
creates and runs (run: 97.93 ms, int32 output)
x uint8, W int8
fails: Could not find an implementation for ConvInteger(10)
x int8, W int8
fails, same message
So it is not "the pin has no ConvInteger" — it is type-combination coverage, and the combination the public exports use is the unsupported one. Checked in the failing file itself (Xenova/yolov8n-posemodel_int8.onnx):
i.e. unsigned activations × signed weights, which is precisely probe row 2. The same holds for the onnx-community/yolo26n-ONNX file (102 ConvInteger, same exporter family).
What that changes for the fix
Option 1 stands and is the cheap path: route the OOB row to the repo's fp32 export (measured to run on both repos: 62 ms and 1,270 ms CPU) and say on the board that the int8 variant is a runtime op-coverage limit, naming this issue.
A workaround now exists and is bounded: re-quantize the same weights to uint8 × uint8 — ONNX's QOperator with QUInt8 weights — which the pin executes. It costs a quantisation step per repo, so it is a "when a row really needs int8" answer rather than the default.
Option 2 (build the pin differently) is unnecessary on this evidence; the kernel is there, the registration is narrow.
Nothing here changes the EP's own scope: ConvInteger is a conv, our seam is MatMul, so even a working ConvInteger would run on the CPU EP — recorded so nobody reads this as an EP gap.
Evidence reproducible from ~/lb/onebp-ort2/ort_run plus three 300-byte models; the probes are ~50 lines of onnx.helper and I can commit them as a ctest if you want this pinned rather than commented.
Closed as completed — recording what that does and does not cover, so the closure is not read as "no residual".
Done and pinned in the tree: the runtime behaviour, as a ctest. #1450 (merged 1b480031) runs three single-node graphs and asserts that uint8 × uint8ConvInteger works — including its arithmetic — while uint8 × int8 and int8 × int8 are reported rather than asserted, so a runtime that gains them says so instead of turning the test red. That is the fact every row in this area depends on, and it no longer lives in an issue comment.
Not done, and not a code defect: the policy this issue was really about — what an OOB row does when a repo ships only an ONNX-native int8 export and no fp32 export. Today's answer is a convention, not a mechanism: route to the fp32 export when the repo has one (measured on two repos: Xenova/yolov8n-pose, onnx-community/yolo26n-ONNX), otherwise record "runtime has no kernel for this op/type combination" on the row. Whether that becomes a registry field, a refusal at fetch time, or nothing at all is a product call on the ONNX rows.
That residual belongs to TAMDCOL's ONNX kinds, which are deferred by OWNER FOCUS ("AMDGPU ONNX (DirectML)" and the non-NPU items), so closing this now is the right sequencing — it is parked with its evidence attached rather than lost. If the ONNX rows come back off deferral, the policy question should be filed fresh against whoever owns those rows, with this issue as the reference: ~/amd-oss/ONNX-VISION-OOB.md §6b has the per-row state (8 of my 12 rows measured, 4 blocked on an artifact) and §4a has the two-flavour trap with the measured table.
Activity
bong-water-water-bong commentedon Nov 8, 2026
Claiming this: lane cl-1077. It was unowned (no comments, no board entry; the filer's state file records it as filed, not taken). Plan:
ConvIntegerexists only for uint8 x uint8, and what this export actually feeds it.ort_runturn "Could not find an implementation for (N)" into one line that names the missing kernel and points to the repo's float/QDQ export.docs/onnx.md.CPU only.
bong-water-water-bong commentedon Nov 8, 2026
Second data point, different model, same exporter family — so this is not one repo's quirk.
onnx-community/yolo26n-ONNX,onnx/model_int8.onnx, 2,848,000-ish B (the sameoptimum/onnx-communityexport pipeline that produced theXenovafile):Identical failure, identical message, at session creation — and note the counts: every conv in the model is a
ConvInteger, so for these exports there is nothing to fall back to inside the graph.Two things this sharpens for the fix chosen in this issue:
onnx/model.onnx, 9,890,454 B, 397 nodes, 102Conv, 4MatMul): 1,270 ms on the CPU EP, and 1,797 ms with our plugin EP registered — which claimed 0 of 400 offered nodes because those four matmuls are dynamically shaped. So option 1 in the issue (route the row to the repo's fp32 export, say so on the board) is measured to work on a second row now.ConvIntegerisai.onnxopset 10, and our pin is 1.23.2. Whether the pin implements it only foruint8activations (the ONNX spec allows int8 and uint8) is still unchecked — that is the one thing worth knowing before deciding between "document and route around it" and "build the pin differently".No action requested; recorded so the next lane that meets a
model_int8.onnxfrom this exporter does not spend an hour on it.onnx: the sibling-exports line says the files were not checked (revie…
bong-water-water-bong commentedon Nov 8, 2026
Answering this issue's own open question (option 2): the pinned ORT 1.23.2 does implement
ConvInteger— but only foruint8 × uint8. Measured with three minimal single-node graphs, each run through our own runner with the vendored pinned ORT:ConvInteger, opset 10, IR 10)run: 97.93 ms, int32 output)Could not find an implementation for ConvInteger(10)So it is not "the pin has no
ConvInteger" — it is type-combination coverage, and the combination the public exports use is the unsupported one. Checked in the failing file itself (Xenova/yolov8n-posemodel_int8.onnx):i.e. unsigned activations × signed weights, which is precisely probe row 2. The same holds for the
onnx-community/yolo26n-ONNXfile (102ConvInteger, same exporter family).What that changes for the fix
uint8 × uint8— ONNX's QOperator withQUInt8weights — which the pin executes. It costs a quantisation step per repo, so it is a "when a row really needs int8" answer rather than the default.Nothing here changes the EP's own scope:
ConvIntegeris a conv, our seam isMatMul, so even a workingConvIntegerwould run on the CPU EP — recorded so nobody reads this as an EP gap.Evidence reproducible from
~/lb/onebp-ort2/ort_runplus three 300-byte models; the probes are~50lines ofonnx.helperand I can commit them as a ctest if you want this pinned rather than commented.bong-water-water-bong commentedon Nov 8, 2026
Closed as completed — recording what that does and does not cover, so the closure is not read as "no residual".
Done and pinned in the tree: the runtime behaviour, as a ctest. #1450 (merged
1b480031) runs three single-node graphs and asserts thatuint8 × uint8ConvIntegerworks — including its arithmetic — whileuint8 × int8andint8 × int8are reported rather than asserted, so a runtime that gains them says so instead of turning the test red. That is the fact every row in this area depends on, and it no longer lives in an issue comment.Not done, and not a code defect: the policy this issue was really about — what an OOB row does when a repo ships only an ONNX-native int8 export and no fp32 export. Today's answer is a convention, not a mechanism: route to the fp32 export when the repo has one (measured on two repos:
Xenova/yolov8n-pose,onnx-community/yolo26n-ONNX), otherwise record "runtime has no kernel for this op/type combination" on the row. Whether that becomes a registry field, a refusal at fetch time, or nothing at all is a product call on the ONNX rows.That residual belongs to TAMDCOL's ONNX kinds, which are deferred by OWNER FOCUS ("AMDGPU ONNX (DirectML)" and the non-NPU items), so closing this now is the right sequencing — it is parked with its evidence attached rather than lost. If the ONNX rows come back off deferral, the policy question should be filed fresh against whoever owns those rows, with this issue as the reference:
~/amd-oss/ONNX-VISION-OOB.md§6b has the per-row state (8 of my 12 rows measured, 4 blocked on an artifact) and §4a has the two-flavour trap with the measured table.