Skip to content

Add GLM-5.3 Flash extraction pipeline - #30

Merged
boyang-zhang1 merged 1 commit into
mainfrom
boyang/glm-5-3-flash-extract
Aug 28, 2026
Merged

Add GLM-5.3 Flash extraction pipeline#30
boyang-zhang1 merged 1 commit into
mainfrom
boyang/glm-5-3-flash-extract

Conversation

@boyang-zhang1

Copy link
Copy Markdown
Member

What

Adds the Z.ai GLM-5.3 Flash one-shot structured extraction pipeline and its public leaderboard results.

Why

This makes the hosted model directly runnable in ExtractBench and publishes its value quality, cost, and latency characteristics.

How

  • Sends documents inline through Z.ai's OpenAI-compatible chat API.
  • Embeds the requested JSON schema in the prompt and requests JSON object output.
  • Accounts for standard and cached input tokens using public list pricing.
  • Loads credentials from GLM_ZAI_API_KEY.

Changes

  • Added the GLM Z.ai extraction provider and pipeline registration.
  • Documented setup and the required environment variable.
  • Added the benchmark row and regenerated the README leaderboard blocks.

Testing

  • Full test suite: 788 passed.
  • Ruff and mypy passed on the changed Python files.
  • Pipeline discovery confirmed the new pipeline is registered.
  • Mocked inference verified file input, JSON output, cached-token cost accounting, and normalization.
  • README regeneration and git diff checks passed.

Notes

  • Requires a Z.ai API key in GLM_ZAI_API_KEY.
  • Unified value F1 is 80.75 overall: 96.30 Short, 51.56 Medium, and 27.83 Long.
  • Average cost is 0.34 cents per page.
  • The pipeline does not emit evidence citations, so grounding scores are zero.
  • No breaking changes; the pipeline is opt-in.

Register the z.ai one-shot document extraction provider and publish its benchmark performance, pricing, latency, and setup requirements.
@boyang-zhang1
boyang-zhang1 merged commit f9a8ec8 into main Aug 28, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant