Skip to content

Add Qwen3.8-Flash-Next-FP8 one-shot extract pipeline - #34

Merged
boyang-zhang1 merged 2 commits into
mainfrom
boyang/add-qwen3-8-flash-next-extract
Aug 31, 2026
Merged

Add Qwen3.8-Flash-Next-FP8 one-shot extract pipeline#34
boyang-zhang1 merged 2 commits into
mainfrom
boyang/add-qwen3-8-flash-next-extract

Conversation

@boyang-zhang1

Copy link
Copy Markdown
Member

What

Adds qwen3_8_flash_next_fp8_vllm_extract_oneshot_structured_output_file, a self-hosted vLLM one-shot structured-output extract pipeline for Qwen3.8-Flash-Next-FP8, and its leaderboard result.

Why

Qwen3.8-Flash-Next is a strong open-weight model; this lands it on the ExtractBench leaderboard alongside the other self-hosted Qwen entries so the family is comparable end to end.

How

  • Registers the pipeline via _pipeline_spec with provider_name="vllm_extract", mirroring the existing qwen3_8_27b_fp8_vllm_extract_oneshot_structured_output_file entry: json_object mode (structured_output: False), 32k output budget, 1h timeout, additional_properties_false.
  • The endpoint is read from QWEN3_8_FLASH_NEXT_SERVER_URL (via endpoint_env_var); there is no default endpoint, consistent with the other open-weight pipelines.
  • Regenerates the README leaderboard tables from leaderboard.csv with scripts/update_readme.py.

Changes

  • src/extract_bench/inference/pipelines/extract.py — new pipeline registration.
  • .env.exampleQWEN3_8_FLASH_NEXT_SERVER_URL entry.
  • docs/pipelines.md — Extract Pipelines table row.
  • leaderboard.csv — one new row (Overall 89.88; Short 94.82 / Medium 87.81 / Long 37.74).
  • README.md — regenerated leaderboard/grounding tables.

Testing

  • uv run extract-bench pipelines lists the new name.
  • uv run ruff check passes on the changed file.
  • uv run python scripts/update_readme.py reproduces the README tables from the CSV with no manual edits.

Notes

  • Self-hosted open-weight model: set QWEN3_8_FLASH_NEXT_SERVER_URL to your vLLM deployment before running; cost columns are blank and grounding columns are 0.00 (json_object mode emits no bounding boxes), matching the sibling Qwen rows.

Register qwen3_8_flash_next_fp8_vllm_extract_oneshot_structured_output_file,
a self-hosted vLLM one-shot structured-output pipeline mirroring the
Qwen3.8-27B entry (json_object mode, 32k output budget, 1h timeout). The
endpoint is read from QWEN3_8_FLASH_NEXT_SERVER_URL with no default. Adds
the leaderboard result (Overall 89.88) and regenerates the README tables.
# Conflicts:
#	README.md
#	leaderboard.csv
@boyang-zhang1
boyang-zhang1 merged commit c7c79df into main Aug 31, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant