What
Sign into LLMAO (llm.apache.org), generate a committer personal access token, point Claude Code at the gateway, and confirm what actually works today.
ASF Tooling asked for this directly: real usage from Magpie is more useful to them than the synthetic load they have been running, and it feeds their capacity planning.
Why now
organizations/ASF/organization.md records everything we know about the gateway second-hand, from a discuss@rai.apache.org post. Nothing in it has been verified by us against a live endpoint. This issue is the cheapest way to turn a written record into a tested one.
Steps
-
Sign into https://llm.apache.org as a committer, generate a PAT.
-
Point a session at it:
export ANTHROPIC_BASE_URL=https://llm.apache.org
export ANTHROPIC_AUTH_TOKEN=<your PAT>
export ANTHROPIC_MODEL=gemma4-26b
-
Confirm plain conversation works end to end.
-
Re-test tool use. known_limitations records anthropic_tool_use_broken — LiteLLM routes the Anthropic-compatible path to vLLM's /v1/responses with a tool_choice shape vLLM rejects. Every Magpie skill is tool-driven, so this decides whether the measurement work is possible at all. Record whether it still reproduces.
-
Note observed throughput against the published figures (~128 tok/s for gemma4-26b, ~46–54 tok/s for the others — all from synthetic load).
-
If tool use works, smoke-test against real Apache Airflow data — a issue-triage pass and a security-family skill over actual Airflow issues. Contrived prompts tell Tooling very little; Airflow's queue is a genuine maintainer workload of the kind the gateway will have to carry, and it is the load ASF Tooling specifically said would be most useful to them. Keep it to a handful of items at this stage; the full measurement run is a separate issue.
Done when
- The gateway is confirmed reachable with a PAT, or the failure is recorded.
- The tool-use limitation is confirmed as still-present or resolved, with the date.
organizations/ASF/organization.md is updated where reality differs from the written record — including removing known_limitations entries that no longer hold.
Notes
There is no budgeting, quota or access control on the gateway today; the $100 figure visible in the UI is a placeholder, and spend_usd reports 0.00 for self-hosted models. The real cost of a request is capacity other communities cannot use at that moment, and no fairness mechanism exists yet. Keep this smoke test small.
What
Sign into LLMAO (
llm.apache.org), generate a committer personal access token, point Claude Code at the gateway, and confirm what actually works today.ASF Tooling asked for this directly: real usage from Magpie is more useful to them than the synthetic load they have been running, and it feeds their capacity planning.
Why now
organizations/ASF/organization.mdrecords everything we know about the gateway second-hand, from adiscuss@rai.apache.orgpost. Nothing in it has been verified by us against a live endpoint. This issue is the cheapest way to turn a written record into a tested one.Steps
Sign into
https://llm.apache.orgas a committer, generate a PAT.Point a session at it:
Confirm plain conversation works end to end.
Re-test tool use.
known_limitationsrecordsanthropic_tool_use_broken— LiteLLM routes the Anthropic-compatible path to vLLM's/v1/responseswith atool_choiceshape vLLM rejects. Every Magpie skill is tool-driven, so this decides whether the measurement work is possible at all. Record whether it still reproduces.Note observed throughput against the published figures (~128 tok/s for
gemma4-26b, ~46–54 tok/s for the others — all from synthetic load).If tool use works, smoke-test against real Apache Airflow data — a
issue-triagepass and a security-family skill over actual Airflow issues. Contrived prompts tell Tooling very little; Airflow's queue is a genuine maintainer workload of the kind the gateway will have to carry, and it is the load ASF Tooling specifically said would be most useful to them. Keep it to a handful of items at this stage; the full measurement run is a separate issue.Done when
organizations/ASF/organization.mdis updated where reality differs from the written record — including removingknown_limitationsentries that no longer hold.Notes
There is no budgeting, quota or access control on the gateway today; the
$100figure visible in the UI is a placeholder, andspend_usdreports0.00for self-hosted models. The real cost of a request is capacity other communities cannot use at that moment, and no fairness mechanism exists yet. Keep this smoke test small.