/v1/batches refuses any model served through Amazon Bedrock Mantle: _resolve_model
forces the resolution to None for a Mantle model, and the caller gets a 400 saying the
model is not available for batched requests.
That was the safe assumption when the Batch API landed, but it is wrong. AWS's own
supported-models list for batch inference includes DeepSeek, Qwen 3, gpt-oss, Gemma 3,
MiniMax, Moonshot, NVIDIA Nemotron and Z.AI — which are precisely the families this
gateway routes through Mantle. So the largest gap between what the gateway allows in a
batch and what the backend allows is one the gateway imposes on itself.
Establish empirically, per family, whether a CreateModelInvocationJob naming the
Mantle-served model id is accepted and returns usable output, then widen the resolution
to whatever actually works. Where a family is genuinely unbatchable, the backend's own
rejection already produces a clean 400, so the client-side refusal earns nothing.
Note that batch requests are translated to the Converse shape before submission, so part
of this is establishing what shape a Mantle-served batch job expects.
/v1/batchesrefuses any model served through Amazon Bedrock Mantle:_resolve_modelforces the resolution to
Nonefor a Mantle model, and the caller gets a 400 saying themodel is not available for batched requests.
That was the safe assumption when the Batch API landed, but it is wrong. AWS's own
supported-models list for batch inference includes DeepSeek, Qwen 3, gpt-oss, Gemma 3,
MiniMax, Moonshot, NVIDIA Nemotron and Z.AI — which are precisely the families this
gateway routes through Mantle. So the largest gap between what the gateway allows in a
batch and what the backend allows is one the gateway imposes on itself.
Establish empirically, per family, whether a
CreateModelInvocationJobnaming theMantle-served model id is accepted and returns usable output, then widen the resolution
to whatever actually works. Where a family is genuinely unbatchable, the backend's own
rejection already produces a clean 400, so the client-side refusal earns nothing.
Note that batch requests are translated to the Converse shape before submission, so part
of this is establishing what shape a Mantle-served batch job expects.