diff --git a/README.md b/README.md index d1f1299ede..f0ce360fa1 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,14 @@ ## Quick Start +### Native Transformers + +For Fun-ASR-Nano transcription with the Hugging Face API, start with the [Transformers 5.17.0 CPU quickstart](./docs/transformers_native.md). No FunASR toolkit or remote Python code is needed. + +[Space](https://huggingface.co/spaces/FunAudioLLM/Fun-ASR-Nano) · [Notebook](https://colab.research.google.com/github/QwenAudio/Fun-ASR/blob/main/examples/colab/fun_asr_nano_transformers.ipynb) · [Python / batch examples](https://github.com/QwenAudio/Fun-ASR/tree/main/examples/transformers) + +### FunASR toolkit and pipelines + [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/modelscope/FunASR/blob/main/examples/colab/funasr_quickstart.ipynb) No local setup? Open the [Colab quickstart](./examples/colab/) to transcribe a public sample or upload your own audio in a browser. @@ -55,7 +63,7 @@ PY Only use `device="cuda"` when this prints `True`; otherwise use `device="cpu"` or reinstall PyTorch with the correct CUDA wheel. -**Flagship model — Fun-ASR-Nano** (LLM-ASR for Chinese, English, and Japanese, plus Chinese dialect groups and regional accents; needs a GPU): +**FunASR toolkit GPU example: Fun-ASR-Nano** (Chinese, English, Japanese, and Chinese dialect groups and regional accents; the separate native Transformers CPU path is linked above): ```python from funasr import AutoModel @@ -357,7 +365,7 @@ recordings with the same evaluation scope. - **MOSS-Transcribe-Diarize** brings long-form ASR, timestamps, and anonymous speaker labels to FunASR services, Docker, Kubernetes, vLLM/SGLang workflows, and FunClip. [Deploy MOSS ->](./docs/moss_transcribe_diarize.md) - **FunASR 1.4.15** adds tested NumPy 2 compatibility and fixes streaming KWS/VAD boundaries and checkpoint ranking. Install with `python -m pip install -U "funasr==1.4.15"`. [Release and verification scope ->](https://github.com/modelscope/FunASR/releases/tag/v1.4.15) -- **Native Transformers:** Fun-ASR-Nano [merged upstream](https://github.com/huggingface/transformers/pull/46180). Use the official `-hf` checkpoint and pinned source; stable 5.16.1 does not include it. [Installation and inference ->](./docs/transformers_native.md) +- **Native Transformers:** Released **5.17.0** supports Fun-ASR-Nano with the official `-hf` checkpoint, CPU examples and a notebook. [Get started ->](./docs/transformers_native.md) > See [GitHub Releases](https://github.com/modelscope/FunASR/releases) for the complete changelog and downloadable assets. diff --git a/README_ja.md b/README_ja.md index ed65e1f79b..fc430971a9 100644 --- a/README_ja.md +++ b/README_ja.md @@ -30,6 +30,14 @@ ## クイックスタート +### ネイティブ Transformers + +Hugging Face API で Fun-ASR-Nano を使う場合は [Transformers 5.17.0 CPU ガイド(英語)](./docs/transformers_native.md) から開始できます。FunASR toolkit とリモート Python コードは不要です。 + +[Space](https://huggingface.co/spaces/FunAudioLLM/Fun-ASR-Nano) · [Notebook](https://colab.research.google.com/github/QwenAudio/Fun-ASR/blob/main/examples/colab/fun_asr_nano_transformers.ipynb) · [Python / batch examples](https://github.com/QwenAudio/Fun-ASR/tree/main/examples/transformers) + +### FunASR toolkit とパイプライン + ```bash python -m pip install torch torchaudio python -m pip install funasr @@ -105,7 +113,7 @@ CER/WER をそろえて比較してください。オフラインのスループ - **MOSS-Transcribe-Diarize** を FunASR service、Docker、Kubernetes、vLLM/SGLang workflow、FunClip に統合し、長時間 ASR、timestamp、匿名 speaker label を一度に処理できます。[MOSS をデプロイ ->](./docs/moss_transcribe_diarize.md) - **FunASR 1.4.15** はテスト済みの NumPy 2 互換性を追加し、ストリーミング KWS/VAD の境界処理と checkpoint の順位付けを修正します。`python -m pip install -U "funasr==1.4.15"`。[リリースと検証範囲 ->](https://github.com/modelscope/FunASR/releases/tag/v1.4.15) -- **ネイティブ Transformers:** Fun-ASR-Nano が[上流にマージ](https://github.com/huggingface/transformers/pull/46180)されました。公式 `-hf` checkpoint と固定ソースを使用します。安定版 5.16.1 には未収録です。[導入ガイド(英語) ->](./docs/transformers_native.md) +- **ネイティブ Transformers:** 正式版 **5.17.0** が Fun-ASR-Nano に対応。公式 `-hf` checkpoint、CPU サンプル、Notebook:[導入ガイド(英語) ->](./docs/transformers_native.md) > 完全な変更履歴と download asset は [GitHub Releases](https://github.com/modelscope/FunASR/releases) を参照してください。 diff --git a/README_ko.md b/README_ko.md index 4005ab2e5f..10a6fc75cd 100644 --- a/README_ko.md +++ b/README_ko.md @@ -30,6 +30,14 @@ ## 빠른 시작 +### 네이티브 Transformers + +Hugging Face API로 Fun-ASR-Nano를 사용하려면 [Transformers 5.17.0 CPU 가이드(영문)](./docs/transformers_native.md)에서 시작하세요. FunASR toolkit이나 원격 Python 코드가 필요 없습니다. + +[Space](https://huggingface.co/spaces/FunAudioLLM/Fun-ASR-Nano) · [Notebook](https://colab.research.google.com/github/QwenAudio/Fun-ASR/blob/main/examples/colab/fun_asr_nano_transformers.ipynb) · [Python / batch examples](https://github.com/QwenAudio/Fun-ASR/tree/main/examples/transformers) + +### FunASR toolkit 및 파이프라인 + ```bash python -m pip install torch torchaudio python -m pip install funasr @@ -104,7 +112,7 @@ checkpoint/revision, 오디오 집합, 하드웨어, 배치, 워밍업, 측정 - **MOSS-Transcribe-Diarize**를 FunASR service, Docker, Kubernetes, vLLM/SGLang workflow, FunClip에 통합해 긴 오디오 ASR, timestamp, 익명 speaker label을 한 번에 처리합니다. [MOSS 배포 ->](./docs/moss_transcribe_diarize.md) - **FunASR 1.4.15**는 테스트를 거친 NumPy 2 호환성을 추가하고 스트리밍 KWS/VAD 경계 처리와 checkpoint 순위 산정을 수정합니다. `python -m pip install -U "funasr==1.4.15"`. [릴리스 및 검증 범위 ->](https://github.com/modelscope/FunASR/releases/tag/v1.4.15) -- **네이티브 Transformers:** Fun-ASR-Nano가 [업스트림에 병합](https://github.com/huggingface/transformers/pull/46180)되었습니다. 공식 `-hf` checkpoint와 고정 소스를 사용하세요. 안정 버전 5.16.1에는 아직 포함되지 않았습니다. [설치 가이드(영문) ->](./docs/transformers_native.md) +- **네이티브 Transformers:** 정식 버전 **5.17.0**이 Fun-ASR-Nano를 지원합니다. 공식 `-hf` checkpoint, CPU 예제, Notebook: [설치 가이드(영문) ->](./docs/transformers_native.md) > 전체 변경 기록과 download asset은 [GitHub Releases](https://github.com/modelscope/FunASR/releases)에서 확인할 수 있습니다. diff --git a/README_zh.md b/README_zh.md index cab445800b..8e34f491c1 100644 --- a/README_zh.md +++ b/README_zh.md @@ -28,6 +28,14 @@ ## 快速开始 +### 原生 Transformers + +使用 Hugging Face API 转写 Fun-ASR-Nano,先看 [Transformers 5.17.0 CPU 快速开始](./docs/transformers_native_zh.md),不需要安装 FunASR 工具库或执行远程 Python 代码。 + +[Space](https://huggingface.co/spaces/FunAudioLLM/Fun-ASR-Nano) · [Notebook](https://colab.research.google.com/github/QwenAudio/Fun-ASR/blob/main/examples/colab/fun_asr_nano_transformers.ipynb) · [Python / batch examples](https://github.com/QwenAudio/Fun-ASR/tree/main/examples/transformers) + +### FunASR 工具库与流水线 + [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/modelscope/FunASR/blob/main/examples/colab/funasr_quickstart.ipynb) 不想先配置本地环境?可以打开 [Colab 快速体验](./examples/colab/README_zh.md) 在浏览器里转写公开样例或上传自己的音频。 @@ -150,7 +158,7 @@ checkpoint/revision、音频集、硬件、批量大小、预热、计时范围 - **MOSS-Transcribe-Diarize** 已接入 FunASR 服务、Docker、Kubernetes、vLLM/SGLang 工作流和 FunClip,一次完成长音频转写、时间戳与匿名说话人标注。[部署 MOSS ->](./docs/moss_transcribe_diarize_zh.md) - **FunASR 1.4.15** 新增经过测试的 NumPy 2 兼容支持,修复流式 KWS/VAD 边界处理和 checkpoint 排序。升级命令:`python -m pip install -U "funasr==1.4.15"`。[发布说明与验证范围 ->](https://github.com/modelscope/FunASR/releases/tag/v1.4.15) -- **原生 Transformers:** Fun-ASR-Nano [已合入上游](https://github.com/huggingface/transformers/pull/46180)。使用官方 `-hf` checkpoint 与固定源码;稳定版 5.16.1 尚未包含。[安装与推理 ->](./docs/transformers_native_zh.md) +- **原生 Transformers:** 正式版 **5.17.0** 已支持 Fun-ASR-Nano。官方 `-hf` 权重、CPU 示例与 Notebook:[安装与推理 ->](./docs/transformers_native_zh.md) > 完整改动记录和可下载资产请查看 [GitHub Releases](https://github.com/modelscope/FunASR/releases)。 diff --git a/docs/deployment_matrix.md b/docs/deployment_matrix.md index 4aa7810e40..899b0184e6 100644 --- a/docs/deployment_matrix.md +++ b/docs/deployment_matrix.md @@ -6,7 +6,7 @@ Use this page to choose the shortest deployment path for a product, demo, benchm | Path | Best for | Start here | Operational notes | |---|---|---|---| -| Native Transformers | Python applications using Hugging Face processors and generation | [Native Fun-ASR-Nano guide](./transformers_native.md) | Official `-hf` checkpoint and pinned source; stable 5.16.1 lacks support at the 2026-09-09 check. Python inference, not an HTTP or realtime server. | +| Native Transformers | First Nano transcript, notebooks and Hugging Face Python applications | [Native Fun-ASR-Nano guide](./transformers_native.md) | Released **5.17.0**, official `-hf` checkpoint, CPU examples and batching. Python inference, not an HTTP or realtime server. | | Colab notebook | Browser smoke tests, first evaluation, shareable demos | [Colab quickstart](../examples/colab/) | No local setup; first run downloads model files, GPU runtime is faster. | | Python API | Notebooks, offline jobs, first model evaluation | [README quick start](../README.md#quick-start) | Lowest ceremony; caller owns batching, retries, and files. | | OpenAI-compatible API | Private speech API, agents, Dify/LangChain/AutoGen-style clients | [OpenAI API example](../examples/openai_api/) | Easiest integration for apps that already support OpenAI audio APIs. | diff --git a/docs/deployment_matrix_ja.md b/docs/deployment_matrix_ja.md index 63c9e5a36f..cd767c92de 100644 --- a/docs/deployment_matrix_ja.md +++ b/docs/deployment_matrix_ja.md @@ -1,5 +1,9 @@ # FunASR デプロイ選択マトリクス +## Transformers で Nano を試す + +中国語・英語・日本語の文字起こしには [Transformers 5.17.0 ガイド(英語)](./transformers_native.md) と公式 `FunAudioLLM/Fun-ASR-Nano-2512-hf` checkpoint を使えます。CPU サンプルがあり、toolkit とサービスの経路は別です。ネイティブ出力はテキストで、タイムスタンプ・話者・HTTP サーバーを追加しません。 + プロダクト、デモ、ベンチマーク、社内ワークフローに合わせて最短のデプロイ経路を選ぶためのガイドです。まずは要件を満たす最小構成から始め、throughput、latency、integration 要件が明確になったら重い runtime に移行してください。 ## クイック判断表 diff --git a/docs/deployment_matrix_ko.md b/docs/deployment_matrix_ko.md index 586b600603..67ace54f65 100644 --- a/docs/deployment_matrix_ko.md +++ b/docs/deployment_matrix_ko.md @@ -1,5 +1,9 @@ # FunASR 배포 선택 매트릭스 +## Transformers로 Nano 시작하기 + +중국어·영어·일본어 전사는 [Transformers 5.17.0 가이드(영문)](./transformers_native.md)와 공식 `FunAudioLLM/Fun-ASR-Nano-2512-hf` checkpoint로 시작할 수 있습니다. CPU 예제가 있으며 toolkit과 서비스 경로는 별도입니다. 네이티브 출력은 텍스트이며 타임스탬프·화자·HTTP 서버를 추가하지 않습니다. + 제품, 데모, 벤치마크, 내부 워크플로에 맞는 가장 짧은 배포 경로를 고르기 위한 가이드입니다. 먼저 요구를 만족하는 최소 구성에서 시작하고, throughput, latency, integration 요구가 명확해질 때 더 무거운 runtime으로 이동하세요. ## 빠른 결정 표 diff --git a/docs/deployment_matrix_zh.md b/docs/deployment_matrix_zh.md index b4468a8816..038a50e29b 100644 --- a/docs/deployment_matrix_zh.md +++ b/docs/deployment_matrix_zh.md @@ -6,7 +6,7 @@ | 路径 | 适合场景 | 从这里开始 | 运维提示 | |---|---|---|---| -| 原生 Transformers | 已使用 Hugging Face 处理器与生成接口的 Python 应用 | [原生 Fun-ASR-Nano 指南](./transformers_native_zh.md) | 官方 `-hf` checkpoint 与固定源码;2026-09-09 核验的稳定版 5.16.1 尚未包含。是 Python 推理入口,不是 HTTP 或实时服务器。 | +| 原生 Transformers | Nano 首次转写、Notebook 与 Hugging Face Python 应用 | [原生 Fun-ASR-Nano 指南](./transformers_native_zh.md) | 正式版 **5.17.0**,官方 `-hf` checkpoint,CPU 与批处理示例。是 Python 推理入口,不是 HTTP 或实时服务器。 | | Colab Notebook | 浏览器 smoke test、首次评估、可分享 demo | [Colab 快速体验](../examples/colab/README_zh.md) | 不需要本地环境;首次运行会下载模型,GPU runtime 更快。 | | Python API | Notebook、离线任务、首次模型评测 | [README 快速开始](../README_zh.md#快速开始) | 最简单;调用方自己负责批处理、重试和文件管理。 | | OpenAI 兼容 API | 私有语音 API、Agent、Dify/LangChain/AutoGen 风格客户端 | [OpenAI API 示例](../examples/openai_api/README_zh.md) | 已支持 OpenAI audio API 的应用最容易接入。 | diff --git a/docs/index.rst b/docs/index.rst index 654738f993..2090c14fe3 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -12,6 +12,8 @@ entry to these guides. Source Markdown remains in this repository. :maxdepth: 1 :caption: Get Started + transformers_native + transformers_native_zh installation/installation installation/installation_zh installation/docker diff --git a/docs/installation/installation.md b/docs/installation/installation.md index fdb5a0fd16..a0906f0657 100755 --- a/docs/installation/installation.md +++ b/docs/installation/installation.md @@ -2,6 +2,8 @@ # Install the Python SDK +> **Only need native Fun-ASR-Nano transcription?** Start with [Transformers 5.17.0](../transformers_native.md). It loads the separate `-hf` checkpoint without the FunASR toolkit. This page covers the `funasr.AutoModel` toolkit path; do not mix dependencies, parameters or output contracts. + Use this guide for `from funasr import AutoModel`. For a packaged C++ service, start with [Docker and runtime images](./docker.md). After installation, continue to the [SDK tutorial](../tutorial/README.md). ## 1. Create an isolated environment diff --git a/docs/installation/installation_zh.md b/docs/installation/installation_zh.md index cc1b740621..7424d9cf48 100755 --- a/docs/installation/installation_zh.md +++ b/docs/installation/installation_zh.md @@ -2,6 +2,8 @@ # 安装 Python SDK +> **只需 Fun-ASR-Nano 原生推理?** 使用 [Transformers 5.17.0 快速开始](../transformers_native_zh.md)。它加载独立的 `-hf` 权重,不要求 FunASR 工具库。本页保留 `funasr.AutoModel` 工具库路径,两者的依赖、参数和输出不可混用。 + 本页适用于 `from funasr import AutoModel`。需要打包好的 C++ 服务时,请先看 [Docker 与运行时镜像](./docker_zh.md)。安装完成后进入 [SDK 教程](../tutorial/README_zh.md)。 ## 1. 创建独立环境 diff --git a/docs/model_selection.md b/docs/model_selection.md index ac706a41d5..69cc2f5e77 100644 --- a/docs/model_selection.md +++ b/docs/model_selection.md @@ -1,5 +1,9 @@ # FunASR Model Selection Guide +## Transformers-first Nano evaluation + +For Chinese, English and Japanese transcription through Hugging Face APIs, start with [native Transformers 5.17.0](./transformers_native.md), the official `FunAudioLLM/Fun-ASR-Nano-2512-hf` checkpoint and the CPU example. The toolkit and service paths below remain separate; the native export does not add timestamps, speakers or an HTTP endpoint. + Use this guide when you are choosing a first model, comparing FunASR with Whisper or a cloud ASR provider, or deciding which model alias to expose through the OpenAI-compatible API. ## Fast default path diff --git a/docs/model_selection_ja.md b/docs/model_selection_ja.md index 0a1489a820..5f693b6056 100644 --- a/docs/model_selection_ja.md +++ b/docs/model_selection_ja.md @@ -1,5 +1,9 @@ # FunASR モデル選択ガイド +## Transformers で Nano を試す + +中国語・英語・日本語の文字起こしには [Transformers 5.17.0 ガイド(英語)](./transformers_native.md) と公式 `FunAudioLLM/Fun-ASR-Nano-2512-hf` checkpoint を使えます。CPU サンプルがあり、toolkit とサービスの経路は別です。ネイティブ出力はテキストで、タイムスタンプ・話者・HTTP サーバーを追加しません。 + 初めて FunASR を試すとき、Whisper やクラウド ASR から移行するとき、または OpenAI 互換 API で公開するモデル alias を決めるときに使ってください。 ## 迷ったらここから diff --git a/docs/model_selection_ko.md b/docs/model_selection_ko.md index 6d11d9a100..23a9d0d233 100644 --- a/docs/model_selection_ko.md +++ b/docs/model_selection_ko.md @@ -1,5 +1,9 @@ # FunASR 모델 선택 가이드 +## Transformers로 Nano 시작하기 + +중국어·영어·일본어 전사는 [Transformers 5.17.0 가이드(영문)](./transformers_native.md)와 공식 `FunAudioLLM/Fun-ASR-Nano-2512-hf` checkpoint로 시작할 수 있습니다. CPU 예제가 있으며 toolkit과 서비스 경로는 별도입니다. 네이티브 출력은 텍스트이며 타임스탬프·화자·HTTP 서버를 추가하지 않습니다. + 처음 FunASR을 사용할 때, Whisper나 클라우드 ASR에서 전환할 때, 또는 OpenAI 호환 API에서 노출할 model alias를 정할 때 참고하세요. ## 고민된다면 여기서 시작 diff --git a/docs/model_selection_zh.md b/docs/model_selection_zh.md index 3e762c7d3c..e8088d2366 100644 --- a/docs/model_selection_zh.md +++ b/docs/model_selection_zh.md @@ -1,5 +1,9 @@ # FunASR 模型选择指南 +## 先用 Transformers 体验 Nano + +通过 Hugging Face API 转写中、英、日语音,先用 [原生 Transformers 5.17.0](./transformers_native_zh.md)、官方 `FunAudioLLM/Fun-ASR-Nano-2512-hf` 权重和 CPU 示例。下文的工具库与服务路径仍独立;原生导出不自动提供时间戳、说话人或 HTTP 接口。 + 当你第一次选择模型、评估是否从 Whisper 或云端 ASR 迁移,或者准备通过 OpenAI 兼容 API 暴露模型别名时,可以先看这份指南。 ## 默认快速路径 diff --git a/docs/python_api.md b/docs/python_api.md index bc67f1884d..97ad49d96d 100644 --- a/docs/python_api.md +++ b/docs/python_api.md @@ -1,5 +1,7 @@ # Python SDK: AutoModel +> **Only need native Fun-ASR-Nano transcription?** Start with [Transformers 5.17.0](./transformers_native.md). It loads the separate `-hf` checkpoint without the FunASR toolkit. This page covers the `funasr.AutoModel` toolkit path; do not mix dependencies, parameters or output contracts. + [简体中文](python_api_zh.md) | [Installation](installation/installation.md) | [Model selection](model_selection.md) `from funasr import AutoModel` runs models inside your Python process. It is not an HTTP client and does not implement the entire OpenAI API. This guide describes the implementation in this checkout, not every historical FunASR release or upstream checkpoint. diff --git a/docs/python_api_zh.md b/docs/python_api_zh.md index c4ed878c1c..1a35a628b0 100644 --- a/docs/python_api_zh.md +++ b/docs/python_api_zh.md @@ -1,5 +1,7 @@ # Python SDK:AutoModel +> **只需 Fun-ASR-Nano 原生推理?** 使用 [Transformers 5.17.0 快速开始](./transformers_native_zh.md)。它加载独立的 `-hf` 权重,不要求 FunASR 工具库。本页保留 `funasr.AutoModel` 工具库路径,两者的依赖、参数和输出不可混用。 + [English](python_api.md) | [安装](installation/installation_zh.md) | [模型选择](model_selection_zh.md) `from funasr import AutoModel` 在当前 Python 进程内运行模型。它不是 HTTP 客户端,也不实现完整的 OpenAI API。本文说明当前代码版本的实现,不代表所有历史 FunASR 版本或上游 checkpoint 的行为。 diff --git a/docs/transformers_native.md b/docs/transformers_native.md index 3415b62554..51548a5e20 100644 --- a/docs/transformers_native.md +++ b/docs/transformers_native.md @@ -1,208 +1,184 @@ -# Fun-ASR-Nano with native Transformers +# Fun-ASR-Nano with Transformers [简体中文](./transformers_native_zh.md) | English -Use this path when your Python application already works with Hugging Face -processors and generation APIs. It runs Fun-ASR-Nano through native Transformers -classes, without the FunASR toolkit or checkpoint-provided remote Python code. -For a managed HTTP service, start with the [deployment matrix](./deployment_matrix.md); -this guide does not add an OpenAI endpoint, request queue or realtime server. +Transcribe Chinese, English and Japanese with `AutoProcessor` and +`AutoModelForSpeechSeq2Seq`. **Transformers 5.17.0 is a released package with +native Fun-ASR-Nano support.** No source checkout, FunASR toolkit or +checkpoint-provided Python code is needed. -## What was merged, and what was released? +[Try the Space](https://huggingface.co/spaces/FunAudioLLM/Fun-ASR-Nano) · +[Open the notebook](https://colab.research.google.com/github/QwenAudio/Fun-ASR/blob/main/examples/colab/fun_asr_nano_transformers.ipynb) · +[Runnable examples](https://github.com/QwenAudio/Fun-ASR/tree/main/examples/transformers) -On **2026-09-09**, [Transformers PR #46180](https://github.com/huggingface/transformers/pull/46180) -was merged as `fc501343edfccdc840eb8594a6cafa8185c2de53`. -At verification time that day, the latest stable **5.16.1 did not contain this -model**: both its release tag and actual wheel were inspected. A merge is not -a stable release. The source build identifies itself as `5.17.0.dev0`; that is -not a promised release version or date. Pin the tested source below, or check -that a later released package really contains `fun_asr_nano` before migrating. +## Get your first transcript -## Choose the checkpoint and the interface together - -| Path | Checkpoint | Entry point | -| --- | --- | --- | -| FunASR toolkit | `FunAudioLLM/Fun-ASR-Nano-2512` | `funasr.AutoModel`; separate documented split-engine path | -| Native Transformers | `FunAudioLLM/Fun-ASR-Nano-2512-hf` | `AutoProcessor` + `AutoModelForSpeechSeq2Seq` | -| Native vLLM | `FunAudioLLM/Fun-ASR-Nano-2512-vllm` | [Official native vLLM guide](./vllm_official_native_validation.md) | -| Native C++ | Converted GGUF files for the chosen runtime | [llama.cpp guide](../runtime/llama.cpp/README.md) | - -These are different artifacts and API contracts. Renaming a model ID or passing -`backend="vllm"` does not convert weights. Native Transformers support does not -make the `-hf` checkpoint a vLLM, GGUF or FunASR HTTP-server checkpoint. - -The official [native HF checkpoint](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512-hf/tree/d93b302ee7fd505e1b3576120fc142fc6f7820e1) -is public and ungated at revision `d93b302ee7fd505e1b3576120fc142fc6f7820e1`. -Its configuration maps to `FunAsrNanoForConditionalGeneration`. Its feature -extractor is embedded in `processor_config.json`; a separate -`preprocessor_config.json` is not required for this snapshot. - -## Independent CPU environment - -Use a new directory for the virtual environment; do not upgrade an existing -FunASR or vLLM service environment in place. The following is **Linux x86-64, -Python 3.12, CPU-only**, not a CUDA or macOS installation recipe. +Start in an isolated environment. This reproducible installation is for Linux +x86-64, Python 3.12 and **CPU**; do not upgrade an existing vLLM or FunASR service +environment in place. Torch and torchaudio must match because the native +feature extractor uses `torchaudio.compliance.kaldi.fbank`. ```bash python3.12 -m venv .venv-funasr-native . .venv-funasr-native/bin/activate python -m pip install --index-url https://download.pytorch.org/whl/cpu \ 'torch==2.10.0+cpu' 'torchaudio==2.10.0+cpu' -python -m pip install 'numpy==1.26.4' 'librosa==0.11.0' 'soundfile==0.13.1' \ - 'huggingface-hub==1.30.0' 'tokenizers==0.23.2' \ - 'https://github.com/huggingface/transformers/archive/fc501343edfccdc840eb8594a6cafa8185c2de53.zip' +python -m pip install 'transformers==5.17.0' 'numpy==1.26.4' \ + 'librosa==0.11.0' 'soundfile==0.13.1' \ + 'huggingface-hub==1.30.0' 'tokenizers==0.23.2' python -m pip check ``` -The verification environment installed a wheel built from the exact source -commit above. Matching **torch and torchaudio** are required for this native -feature extractor: it calls `torchaudio.compliance.kaldi.fbank`. The toolkit's -optional audio dependencies do not imply that torchaudio is optional here. -Older prepared environments failed dependency gates; do not disable those gates -or mix incompatible Hub/tokenizers versions to get an import to succeed. -This is a tested environment, not a complete lock of every transitive dependency; -record `python -m pip freeze`, platform details and model revision in your own run. - -## Check preprocessing without loading weights +Run this Python example. It downloads about 1.66 GB of model weights on first +use and transcribes a short, pinned official English sample. CPU float32 is +explicit; no GPU is needed for this example. -This downloads the public configuration, tokenizer and chat template, **not -model weights**. One second of synthetic silence checks the entry point; it is -not a speech-recognition quality test. - - + ```python -import numpy as np -from transformers import AutoProcessor +import torch +from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor +torch.set_num_threads(4) model_id = "FunAudioLLM/Fun-ASR-Nano-2512-hf" revision = "d93b302ee7fd505e1b3576120fc142fc6f7820e1" +audio = "https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512/resolve/272c57b82523ada6fd87095e955f8e29100979ab/example/en.mp3" + processor = AutoProcessor.from_pretrained( model_id, revision=revision, trust_remote_code=False, token=False ) +model = AutoModelForSpeechSeq2Seq.from_pretrained( + model_id, revision=revision, trust_remote_code=False, token=False, + dtype=torch.float32, +).to("cpu").eval() inputs = processor.apply_transcription_request( - audio=np.zeros(16000, dtype=np.float32), - language="en", + audio=audio, language="en", processor_kwargs={ "return_tensors": "pt", "audio_kwargs": {"sampling_rate": 16000}, + "text_kwargs": {"padding": True}, }, ) -print({name: tuple(value.shape) for name, value in inputs.items()}) +with torch.inference_mode(): + generated = model.generate(**inputs, max_new_tokens=128, do_sample=False) +new_tokens = generated[:, inputs.input_ids.shape[1]:] +print(processor.batch_decode(new_tokens, skip_special_tokens=True)[0]) ``` -In the verified environment the text tensors have shape `(1, 42)`, audio features -`(1, 17, 560)`, and feature mask `(1, 17)`. There are 17 audio placeholder tokens -and 17 valid feature frames. These shapes depend on input length and template; -do not hard-code them for real recordings. +The model returns the prompt and generated tokens together. Remove the full +input tensor width before decoding. For this sample, the observed raw output +was: “The tribal chieftain called for the boy, and presented him with fifty +pieces of gold.” This is a functional example, not a quality guarantee. -## Transcribe one recording +## Your files, keywords and batches -Prepare a non-empty **mono 16 kHz WAV** named `audio.wav` in your working directory. -The code deliberately rejects a different sample rate or channel layout rather -than silently assigning 16 kHz to arbitrary samples. Resample/downmix with an -explicit policy before this step and preserve the original recording for review. +The [command-line example](https://github.com/QwenAudio/Fun-ASR/tree/main/examples/transformers) +adds explicit input validation, sample-rate conversion and completion checks. +Clone the **QwenAudio/Fun-ASR code repository** (not an HF weights repository), +install the [independent native requirements](https://github.com/QwenAudio/Fun-ASR/tree/main/examples/transformers), +and run these commands from its root directory: -The first model load downloads the weights unless the pinned snapshot is already -cached. It needs substantially more memory and time than preprocessing. This -example uses CPU float32 explicitly; GPU placement, mixed precision, attention -backends and service concurrency need separate validation. +```bash +python examples/transformers/transcribe.py recording.wav --language zh --keywords 开放时间 +python examples/transformers/transcribe.py chinese.wav english.wav --language zh en +``` - +It averages multichannel audio to mono and resamples to 16 kHz with `soxr_hq`; +it never overwrites the original file. It accepts 1-4 files with at most +60 seconds of total audio. Longer recordings, empty input and non-finite samples +are rejected, not silently trimmed. + +For your own application, pass a waveform, local path or supported URL to +`processor.apply_transcription_request`. A raw waveform must have its true +sample rate; do not label 48 kHz samples as 16 kHz. Use `language="zh"`, +`"en"` or `"ja"`. For a batch, use an audio list, matching language list and +`text_kwargs={"padding": True}`. Preserve input order when decoding. + +`keywords` supplies vocabulary hints and `prompt` supplies context. They are +not enforced vocabulary and are not the toolkit's `hotword` parameter. In our +Chinese sample, adding `开放时间` did not correct the raw output `开饭时间`. +Use representative recordings and human references to assess quality. + +The short example bounds generation at 128 new tokens. Missing EOS can indicate +truncation; increasing the limit does not prove coverage of a long recording. +The CLI reports `reached_eos` and exits unsuccessfully for missing EOS or empty +text. EOS itself is not proof that all words were recognized. + +## From an example to a service + +| Need | Checkpoint and interface | +| --- | --- | +| Native Transformers in Python | `FunAudioLLM/Fun-ASR-Nano-2512-hf`, `AutoProcessor` + `AutoModelForSpeechSeq2Seq` | +| FunASR pipelines and HTTP adapters | `FunAudioLLM/Fun-ASR-Nano-2512`, `funasr.AutoModel`; [deployment matrix](./deployment_matrix.md) | +| Native vLLM | `FunAudioLLM/Fun-ASR-Nano-2512-vllm`; [native vLLM guide](./vllm_official_native_validation.md) | +| C++ / edge | Converted GGUF files; [llama.cpp](../runtime/llama.cpp/README.md) | + +These are different artifacts, not interchangeable model IDs. Native +Transformers support does not add an HTTP server, queue, streaming protocol, +word timestamps or speaker identities. This native export omits the CTC +timestamp branch. The base Nano checkpoint covers zh/en/ja and Chinese +dialects/accents; the **31-language MLT** checkpoint is separate. + +For transcription with anonymous speakers and segment timestamps, see the +third-party OpenMOSS [MOSS guide](./moss_transcribe_diarize.md). For other tasks, +use the [Model Zoo](../model_zoo/readme.md). Validate GPU dtype, attention +backends, memory, concurrency and quality on the actual target hardware. + +## Troubleshooting without loading weights + +If loading fails, check the active interpreter, `transformers.__version__`, +the `-hf` model ID and model revision first. Version 5.16.1 lacks this model; +upgrade in the new environment, not an unrelated running service. The native +classes live in `transformers.models.fun_asr_nano`. + +This optional synthetic-silence check downloads the processor, not model weights: + + ```python -import soundfile as sf -import torch -from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor +import numpy as np +from transformers import AutoProcessor model_id = "FunAudioLLM/Fun-ASR-Nano-2512-hf" revision = "d93b302ee7fd505e1b3576120fc142fc6f7820e1" -audio, sample_rate = sf.read("audio.wav", dtype="float32") -if sample_rate != 16000 or audio.ndim != 1 or audio.size == 0: - raise ValueError("Use a non-empty mono 16 kHz WAV file") - processor = AutoProcessor.from_pretrained( model_id, revision=revision, trust_remote_code=False, token=False ) -model = AutoModelForSpeechSeq2Seq.from_pretrained( - model_id, revision=revision, trust_remote_code=False, token=False, - dtype=torch.float32, -).to("cpu").eval() inputs = processor.apply_transcription_request( - audio=audio, language="zh", + audio=np.zeros(16000, dtype=np.float32), language="en", processor_kwargs={ "return_tensors": "pt", - "audio_kwargs": {"sampling_rate": sample_rate}, + "audio_kwargs": {"sampling_rate": 16000}, }, ) -with torch.inference_mode(): - generated = model.generate(**inputs, max_new_tokens=128, do_sample=False) -new_tokens = generated[:, inputs.input_ids.shape[1]:] -print(processor.batch_decode(new_tokens, skip_special_tokens=True)[0]) +print({name: tuple(value.shape) for name, value in inputs.items()}) ``` -`generated` includes the input prompt. Slice off `inputs.input_ids.shape[1]` -before decoding so prompt framing is not mistaken for recognized speech. -`max_new_tokens=128` bounds this short-file example; hitting the limit can -truncate output. A longer limit is not proof of complete coverage. -Use `language="en"` or `language="ja"` for the corresponding target language; -language selection is a transcription instruction, not a promise of translation. - -## Context, hotwords and batches - -`apply_transcription_request` accepts `prompt` for context and `keywords` for -hotwords. These are native processor arguments, not the toolkit's `hotword` -parameter or a vLLM HTTP field. The source API accepts one language for the batch -or a language list matching its length; per-recording prompt and nested keyword -lists must also match the batch. Keep the decoded results in input order. - -Do not infer quality gains from a template accepting a keyword. Evaluate names, -numbers and unusual terms against authorized, representative recordings. -Batch padding, memory use and generated-text correctness must be tested together -before turning a single-file example into a production batch worker. - -## Evidence and limits - -The documented single-recording recipe was also executed on the official Chinese -sample, followed by an English single recording, a Chinese keyword request and a -Chinese/English batch. All returned non-empty text and EOS before the 128-token -limit. These are functional checks on two short public samples, not a CER/WER or -capacity evaluation. CPU: Intel Xeon Platinum 8480+, float32, four Torch threads; -matching torch/torchaudio 2.10.0+cpu and librosa 0.11.0. Model weights were loaded -from a previously downloaded official cache whose SHA/size were rechecked. - -Samples came from the original model's +Reject empty recordings and batches before calling the processor. Synthetic +preprocessing success is **not an accuracy or capacity result**. + +## Reproducibility and sources + +Verified on **2026-09-09** using the actual released 5.17.0 wheel, Python 3.12.3, +matching torch/torchaudio 2.10.0+cpu, float32 and four Torch threads on an Intel +Xeon Platinum 8480+. Official English URL, Chinese waveform, Chinese keywords +and a padded Chinese/English batch all returned non-empty text and EOS before +128 tokens. These two short public samples do not establish CER/WER, GPU +compatibility or serving capacity. + +The public native revision is +`d93b302ee7fd505e1b3576120fc142fc6f7820e1`; it resolves to +`FunAsrNanoForConditionalGeneration` with `trust_remote_code=False`. +`model.safetensors` is 1,659,773,320 bytes, SHA256 +`1bbb6dcc5d8b75084a399d48c4d4b0f3aa1d3f09f2616ae40ed2c4fba03d89c9`. +Audio samples come from the original repository's [fixed example directory](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512/tree/272c57b82523ada6fd87095e955f8e29100979ab/example), -not from an invented -hf example directory. The mono 48 kHz Chinese MP3 was -explicitly resampled with librosa's default soxr_hq and written as float32 16 kHz -WAV before running the unmodified recipe. Its raw text was -`开饭时间早上九点至下午五点。`; adding the candidate keyword `开放时间` did not force that -spelling into the output. A keyword is a hint, not an enforced vocabulary. - -Synthetic processor tests exercised language aliases, context/keyword lists and -two-length batches. The empty-list case still raises an upstream `IndexError` -before its intended validation; reject empty batches in your application. -The guide's explicit non-empty waveform check avoids that entry path. Successful -cases do not erase this observed limitation. Initial resampling also exposed the -old librosa 0.10.1 dependency on removed `pkg_resources`; the final environment -uses 0.11.0 and was retested. - -The 2026-09-09 isolated CPU verification passed package dependency checks, -native configuration/class resolution and synthetic preprocessing with the -pinned official revision. Installed configuration, modeling, processing and -feature-extraction files were byte-identical to the merged upstream source. -A preprocessing pass is **not an accuracy or capacity result**. It does not -establish word timestamps, diarization, realtime behavior, GPU compatibility, -or performance equivalence with vLLM. - -For anonymous speakers and segment timestamps, see the third-party OpenMOSS -[MOSS guide](./moss_transcribe_diarize.md); for other choices see the -[Model Zoo](../model_zoo/readme.md). Do not infer timestamp or identity support -from a model returning text. Preserve input hashes, versions, raw responses and -an authorized human review when assessing an application. Never attach private -recordings or credentials to a public issue. - -## Sources and next steps - -- [Merged implementation and usage documentation](https://github.com/huggingface/transformers/blob/fc501343edfccdc840eb8594a6cafa8185c2de53/docs/source/en/model_doc/fun_asr_nano.md). -- [Native processor](https://github.com/huggingface/transformers/blob/fc501343edfccdc840eb8594a6cafa8185c2de53/src/transformers/models/fun_asr_nano/processing_fun_asr_nano.py) and [audio feature extractor](https://github.com/huggingface/transformers/blob/fc501343edfccdc840eb8594a6cafa8185c2de53/src/transformers/models/fun_asr_nano/feature_extraction_fun_asr_nano.py). -- [Why the checkpoint suffix matters](https://www.funasr.com/en/blog/fun-asr-nano-transformers.html), an application-oriented walkthrough. -- [Fun-ASR model project](https://github.com/QwenAudio/Fun-ASR) and [FunASR toolkit](https://github.com/modelscope/FunASR). +not an invented `-hf/example` path. + +[PR #46180](https://github.com/huggingface/transformers/pull/46180) merged as +`fc501343edfccdc840eb8594a6cafa8185c2de53` before the stable package was +published. Earlier source-only instructions described that interval; new +installations should use the released package above. +See the [official 5.17.0 model documentation](https://huggingface.co/docs/transformers/v5.17.0/en/model_doc/fun_asr_nano) +and [native model card](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512-hf). + +Keep versions, input hashes and raw results with application evaluations. +Never attach private recordings or credentials to a public issue. diff --git a/docs/transformers_native_zh.md b/docs/transformers_native_zh.md index 0c06ac22d5..ba34ea2b71 100644 --- a/docs/transformers_native_zh.md +++ b/docs/transformers_native_zh.md @@ -1,181 +1,167 @@ -# 用原生 Transformers 接入 Fun-ASR-Nano +# 用 Transformers 转写 Fun-ASR-Nano 简体中文 | [English](./transformers_native.md) -如果 Python 应用已经使用 Hugging Face 的处理器和生成接口,可以选择这条路径。 -它通过 Transformers 原生类运行 Fun-ASR-Nano,不依赖 FunASR 工具包或 checkpoint -中的远程 Python 代码。需要 HTTP 服务时先看[部署矩阵](./deployment_matrix_zh.md): -本指南不会自动提供 OpenAI 接口、请求队列或实时服务器。 +用 `AutoProcessor` 和 `AutoModelForSpeechSeq2Seq` 转写中、英、日语音。 +**正式发布的 Transformers 5.17.0 已包含 Fun-ASR-Nano 原生支持。** +不需要克隆源码、安装 FunASR 工具库,也不需要执行模型仓库的远程 Python 代码。 -## 合入主线,不等于已经发布 +[在线 Space](https://huggingface.co/spaces/FunAudioLLM/Fun-ASR-Nano) · +[打开 Notebook](https://colab.research.google.com/github/QwenAudio/Fun-ASR/blob/main/examples/colab/fun_asr_nano_transformers.ipynb) · +[可运行示例](https://github.com/QwenAudio/Fun-ASR/tree/main/examples/transformers) -**2026-09-09**,[Transformers PR #46180](https://github.com/huggingface/transformers/pull/46180) -合入提交 `fc501343edfccdc840eb8594a6cafa8185c2de53`。 -当天核验时,最新稳定版 **5.16.1 不包含这个模型**,其 tag 和实际 wheel 均已检查。 -源码构建显示 `5.17.0.dev0`,不代表下一稳定版的版本或发布日期承诺。 -请固定下方经过验证的源码提交;改用后续稳定版前,实际确认安装包包含 `fun_asr_nano`。 +## 先得到第一条转写 -## 模型格式和接口一起选 - -| 路径 | Checkpoint | 入口 | -| --- | --- | --- | -| FunASR 工具包 | `FunAudioLLM/Fun-ASR-Nano-2512` | `funasr.AutoModel`;拆分引擎另见对应文档 | -| 原生 Transformers | `FunAudioLLM/Fun-ASR-Nano-2512-hf` | `AutoProcessor` + `AutoModelForSpeechSeq2Seq` | -| 原生 vLLM | `FunAudioLLM/Fun-ASR-Nano-2512-vllm` | [官方原生 vLLM 指南](./vllm_official_native_validation_zh.md) | -| 原生 C++ | 对应运行时转换后的 GGUF 文件 | [llama.cpp 指南](../runtime/llama.cpp/README.md) | - -这些产物及 API 契约不同。改模型 ID 的名字或传 `backend="vllm"` 不会转换权重; -原生 Transformers 支持不意味着 `-hf` 可以交给 vLLM、GGUF 运行时或 FunASR HTTP 服务。 - -官方[原生 HF checkpoint](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512-hf/tree/d93b302ee7fd505e1b3576120fc142fc6f7820e1) -在 revision `d93b302ee7fd505e1b3576120fc142fc6f7820e1` 公开且无需申请访问。 -配置映射到 `FunAsrNanoForConditionalGeneration`,特征提取器嵌在 -`processor_config.json` 中。此快照不需要另一个 `preprocessor_config.json`。 - -## 准备独立的 CPU 环境 - -使用新目录创建虚拟环境,不要直接升级正在运行的 FunASR/vLLM 服务环境。 -下面对应 **Linux x86-64、Python 3.12、仅 CPU**,不是 CUDA 或 macOS 安装方案。 +使用独立环境。下面是已验证的 Linux x86-64、Python 3.12 **CPU** 安装路径, +不要直接升级已有的 FunASR 或 vLLM 服务环境。torch 与 torchaudio 必须匹配, +因为原生特征提取器使用 `torchaudio.compliance.kaldi.fbank`。 ```bash python3.12 -m venv .venv-funasr-native . .venv-funasr-native/bin/activate python -m pip install --index-url https://download.pytorch.org/whl/cpu \ 'torch==2.10.0+cpu' 'torchaudio==2.10.0+cpu' -python -m pip install 'numpy==1.26.4' 'librosa==0.11.0' 'soundfile==0.13.1' \ - 'huggingface-hub==1.30.0' 'tokenizers==0.23.2' \ - 'https://github.com/huggingface/transformers/archive/fc501343edfccdc840eb8594a6cafa8185c2de53.zip' +python -m pip install 'transformers==5.17.0' 'numpy==1.26.4' \ + 'librosa==0.11.0' 'soundfile==0.13.1' \ + 'huggingface-hub==1.30.0' 'tokenizers==0.23.2' python -m pip check ``` -验证环境安装的是从上述精确提交构建的 wheel。原生特征提取器调用 -`torchaudio.compliance.kaldi.fbank`,因此需要匹配的 **torch 和 torchaudio**。 -FunASR 工具包的可选音频依赖策略不能套用到这里。旧环境曾被过旧的 Hub/tokenizers -依赖挡住;不要关闭依赖检查或混搭版本来强行导入。 -这是经过验证的环境,不是所有传递依赖的完整锁文件;自己的运行还应记录 -`python -m pip freeze`、平台信息和模型 revision。 - -## 先验证预处理,不加载权重 +运行以下 Python 示例。首次加载需要下载约 1.66 GB 模型权重,然后转写一段 +固定版本的官方英文样例。明确使用 CPU float32,不要求 GPU。 -这一步仅下载公开的配置、tokenizer 和聊天模板,**不下载模型权重**。 -一秒合成静音用来检查入口,不是识别质量测试。 - - + ```python -import numpy as np -from transformers import AutoProcessor +import torch +from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor +torch.set_num_threads(4) model_id = "FunAudioLLM/Fun-ASR-Nano-2512-hf" revision = "d93b302ee7fd505e1b3576120fc142fc6f7820e1" +audio = "https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512/resolve/272c57b82523ada6fd87095e955f8e29100979ab/example/en.mp3" + processor = AutoProcessor.from_pretrained( model_id, revision=revision, trust_remote_code=False, token=False ) +model = AutoModelForSpeechSeq2Seq.from_pretrained( + model_id, revision=revision, trust_remote_code=False, token=False, + dtype=torch.float32, +).to("cpu").eval() inputs = processor.apply_transcription_request( - audio=np.zeros(16000, dtype=np.float32), - language="en", + audio=audio, language="en", processor_kwargs={ "return_tensors": "pt", "audio_kwargs": {"sampling_rate": 16000}, + "text_kwargs": {"padding": True}, }, ) -print({name: tuple(value.shape) for name, value in inputs.items()}) +with torch.inference_mode(): + generated = model.generate(**inputs, max_new_tokens=128, do_sample=False) +new_tokens = generated[:, inputs.input_ids.shape[1]:] +print(processor.batch_decode(new_tokens, skip_special_tokens=True)[0]) ``` -验证环境中文字张量形状是 `(1, 42)`,音频特征是 `(1, 17, 560)`, -特征 mask 是 `(1, 17)`;17 个音频占位 token 与 17 帧有效特征对应。 -这些形状随输入时长和模板变化,不应在真实应用里写死。 +模型输出同时包含输入提示和生成结果,因此解码前需要去掉完整输入张量宽度。 +这段音频实测的原始输出为: +“The tribal chieftain called for the boy, and presented him with fifty pieces of gold.” +这是功能示例,不是质量保证。 -## 转写一份录音 +## 换成自己的录音,再做批处理 -在工作目录准备非空、**单声道 16 kHz WAV** 文件 `audio.wav`。 -代码明确拒绝其他采样率或声道布局,而不是把任意采样直接标成 16 kHz。 -重采样和混音请使用明确的策略,在这一步之前完成,并保留原录音便于回听。 +[命令行示例](https://github.com/QwenAudio/Fun-ASR/tree/main/examples/transformers) +补齐了输入校验、采样率转换和生成完成检查。克隆 **QwenAudio/Fun-ASR 代码仓库** +(不是 HF 权重仓库),按该目录说明安装独立依赖,再在代码仓库根目录运行: -首次加载模型会下载权重,已有固定 revision 缓存时复用缓存。 -其内存和耗时明显高于仅预处理。下面显式使用 CPU float32; -GPU 放置、混合精度、attention 后端和服务并发需要分别验证。 +```bash +python examples/transformers/transcribe.py recording.wav --language zh --keywords 开放时间 +python examples/transformers/transcribe.py chinese.wav english.wav --language zh en +``` - +示例将多声道取均值为单声道,再用 `soxr_hq` 显式重采样到 16 kHz, +不会覆盖原始录音。接受 1-4 个文件,总音频时长最多 60 秒;过长、空输入 +和非有限采样值会被拒绝,不会静默截断。 + +在自己的应用中,可向 `processor.apply_transcription_request` 传入波形、 +本地路径或支持的 URL。原始数组必须使用真实采样率,不能把 48 kHz 数组标成 +16 kHz。语言使用 `zh`、`en` 或 `ja`。批量时传入音频列表、对应语言列表, +并设置 `text_kwargs={"padding": True}`,解码后保留输入顺序。 + +`keywords` 是词汇提示,`prompt` 是上下文,不是强制词典,也不是工具库的 +`hotword` 参数。中文样例加入 `开放时间` 后,实测仍输出 `开饭时间`; +需要用代表性录音和人工参考文本评估质量。 + +上面的短样例最多生成 128 个新 token。没有 EOS 可能意味着截断,单纯提高 +上限不等于长录音已完整覆盖。CLI 输出 `reached_eos`,缺少 EOS 或文本为空时 +以非零状态结束;出现 EOS 也不代表每个字都识别正确。 + +## 从示例走向服务 + +| 需求 | checkpoint 与接口 | +| --- | --- | +| Python 原生 Transformers | `FunAudioLLM/Fun-ASR-Nano-2512-hf`,`AutoProcessor` + `AutoModelForSpeechSeq2Seq` | +| FunASR 流水线和 HTTP 适配器 | `FunAudioLLM/Fun-ASR-Nano-2512`,`funasr.AutoModel`;[部署矩阵](./deployment_matrix_zh.md) | +| 原生 vLLM | `FunAudioLLM/Fun-ASR-Nano-2512-vllm`;[原生 vLLM 指南](./vllm_official_native_validation_zh.md) | +| C++ / 边缘部署 | 转换后的 GGUF;[llama.cpp](../runtime/llama.cpp/README.md) | + +这些是不同格式,不能只替换模型 ID。Transformers 原生支持不自动提供 HTTP +服务、队列、流式协议、字级时间戳或说话人身份。该原生导出不包含 CTC 时间戳 +分支。基础 Nano 支持中、英、日及中文方言、口音;**31 语言 MLT 是独立 checkpoint**。 + +需要转写、匿名说话人和片段时间戳时,查看第三方 OpenMOSS +[MOSS 指南](./moss_transcribe_diarize_zh.md);其他任务见 +[Model Zoo](../model_zoo/readme_zh.md)。GPU dtype、attention 后端、显存、 +并发和质量都要在实际目标硬件上另行验证。 + +## 不加载权重的排查方法 + +加载失败时,先检查当前解释器、`transformers.__version__`、`-hf` 模型 ID +和 revision。5.16.1 不包含此模型,请在新环境升级,不要改动无关运行服务。 +原生类位于 `transformers.models.fun_asr_nano`。 + +下面是可选的合成静音预处理检查,只下载 processor,不加载模型权重: + + ```python -import soundfile as sf -import torch -from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor +import numpy as np +from transformers import AutoProcessor model_id = "FunAudioLLM/Fun-ASR-Nano-2512-hf" revision = "d93b302ee7fd505e1b3576120fc142fc6f7820e1" -audio, sample_rate = sf.read("audio.wav", dtype="float32") -if sample_rate != 16000 or audio.ndim != 1 or audio.size == 0: - raise ValueError("Use a non-empty mono 16 kHz WAV file") - processor = AutoProcessor.from_pretrained( model_id, revision=revision, trust_remote_code=False, token=False ) -model = AutoModelForSpeechSeq2Seq.from_pretrained( - model_id, revision=revision, trust_remote_code=False, token=False, - dtype=torch.float32, -).to("cpu").eval() inputs = processor.apply_transcription_request( - audio=audio, language="zh", + audio=np.zeros(16000, dtype=np.float32), language="en", processor_kwargs={ "return_tensors": "pt", - "audio_kwargs": {"sampling_rate": sample_rate}, + "audio_kwargs": {"sampling_rate": 16000}, }, ) -with torch.inference_mode(): - generated = model.generate(**inputs, max_new_tokens=128, do_sample=False) -new_tokens = generated[:, inputs.input_ids.shape[1]:] -print(processor.batch_decode(new_tokens, skip_special_tokens=True)[0]) +print({name: tuple(value.shape) for name, value in inputs.items()}) ``` -`generated` 包含输入 prompt。解码前去掉 `inputs.input_ids.shape[1]`, -避免把提示模板当作识别结果。`max_new_tokens=128` 限定这份短文件示例的生成上限; -触顶可能截断输出,调大也不保证整段覆盖。对应语言可改为 `language="en"` 或 -`language="ja"`;这里是转写语言指令,不是翻译能力承诺。 - -## 上下文、热词和批处理 - -`apply_transcription_request` 使用 `prompt` 传上下文、`keywords` 传热词, -不是工具包的 `hotword` 参数或 vLLM HTTP 字段。源码 API 接受全批统一语言, -或与批大小一致的语言列表;逐录音的 prompt 和嵌套关键词列表也必须与批大小匹配。 -结果应保持输入顺序。 - -模板接收了关键词,不等于识别更准确。应在获授权且有代表性的录音中检查专名、 -数字和生僻词。把单文件示例变成生产批处理之前,应一起测试 padding、内存和生成文本, -而不是只检查请求张量的形状。 - -## 验证范围与边界 - -本指南的单录音代码还实际运行了官方中文样本,随后测试了英文单录音、中文关键词请求及 -中英混合批量。各请求均返回非空文本,并在 128 token 上限之前生成 EOS。 -这只是两份公开短样本的功能检查,不是 CER/WER 或容量评测。 -CPU 为 Intel Xeon Platinum 8480+,float32、Torch 四线程;torch/torchaudio -2.10.0+cpu、librosa 0.11.0。模型来自先前下载的官方缓存,本轮重新检查了 SHA 和文件大小。 - -样本来自原始模型的[固定 example 目录](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512/tree/272c57b82523ada6fd87095e955f8e29100979ab/example), -不是假定存在的 -hf 示例目录。中文源文件是单声道 48 kHz MP3,先用 librosa 默认的 -soxr_hq 明确重采样,再写成 float32 16 kHz WAV,运行未改写的指南代码。 -原始文本为 `开饭时间早上九点至下午五点。`;传入候选关键词 `开放时间` 也没有强制产生该词形。 -热词是提示,不是强制词表约束。 - -合成处理器测试覆盖了语言别名、上下文/关键词列表和不同长度的批量输入。 -空列表仍会在上游预期校验前抛 `IndexError`,应用应提前拒绝空批量;指南已显式拒绝空波形, -不经过这一路径。成功用例不能抹去这个限制。最初的重采样也暴露出 librosa 0.10.1 -依赖已移除的 `pkg_resources`;最终环境使用 0.11.0 并重新验证。 - -2026-09-09 的独立 CPU 验证通过依赖检查、原生配置/类解析和固定官方 revision 的合成预处理。 -已安装的配置、模型、处理器、特征提取器源码逐字节匹配上游合并提交。 -预处理通过**不代表准确率或容量结论**,也不证明字级时间戳、说话人分离、实时行为、 -GPU 兼容性或与 vLLM 的性能一致。 - -需要匿名说话人与段级时间戳时,参考第三方 OpenMOSS 的 -[MOSS 指南](./moss_transcribe_diarize_zh.md);其他路径见 -[Model Zoo](../model_zoo/readme_zh.md)。不能从“返回了文本”推导时间戳或身份识别能力。 -应用验收应保留输入散列、固定版本、原始输出和获授权的人工回听记录; -不要把私有录音或凭据附到公开 issue 中。 - -## 来源与下一步 - -- [合并后的实现与官方用法](https://github.com/huggingface/transformers/blob/fc501343edfccdc840eb8594a6cafa8185c2de53/docs/source/en/model_doc/fun_asr_nano.md)。 -- [原生处理器](https://github.com/huggingface/transformers/blob/fc501343edfccdc840eb8594a6cafa8185c2de53/src/transformers/models/fun_asr_nano/processing_fun_asr_nano.py)与[音频特征提取器](https://github.com/huggingface/transformers/blob/fc501343edfccdc840eb8594a6cafa8185c2de53/src/transformers/models/fun_asr_nano/feature_extraction_fun_asr_nano.py)。 -- [为什么 checkpoint 后缀重要](https://www.funasr.com/blog/fun-asr-nano-transformers.html):面向应用的专题解读。 -- [Fun-ASR 模型项目](https://github.com/QwenAudio/Fun-ASR)与[FunASR 工具包](https://github.com/modelscope/FunASR)。 +应用层先拒绝空录音和空批次。预处理成功**不代表准确率或容量验证**。 + +## 复现记录与来源 + +**2026-09-09** 使用实际发布的 5.17.0 wheel 验证:Python 3.12.3, +torch/torchaudio 2.10.0+cpu、float32、四个 Torch 线程,CPU 为 +Intel Xeon Platinum 8480+。官方英文 URL、中文波形、中文热词和中英 padding +批次均返回非空文本,并在 128 token 之前出现 EOS。两段公开短样例的功能 +检查不等于 CER/WER、GPU 兼容性或服务容量评测。 + +公开原生 revision 为 `d93b302ee7fd505e1b3576120fc142fc6f7820e1`, +解析到 `FunAsrNanoForConditionalGeneration`,使用 +`trust_remote_code=False`。`model.safetensors` 大小 1,659,773,320 字节, +SHA256:`1bbb6dcc5d8b75084a399d48c4d4b0f3aa1d3f09f2616ae40ed2c4fba03d89c9`。 +音频来自原始模型仓库的 +[固定样例目录](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512/tree/272c57b82523ada6fd87095e955f8e29100979ab/example), +不是虚构的 `-hf/example` 路径。 + +[PR #46180](https://github.com/huggingface/transformers/pull/46180) 的合并提交为 +`fc501343edfccdc840eb8594a6cafa8185c2de53`,早于正式包发布。旧版源码安装说明 +对应那个时间窗口;现在新安装应使用上面的正式包。 +参阅 [5.17.0 官方模型文档](https://huggingface.co/docs/transformers/v5.17.0/en/model_doc/fun_asr_nano) +与 [原生 model card](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512-hf)。 + +评估应用时保留版本、输入摘要与原始结果,不要向公开 issue 上传私人录音或凭证。 diff --git a/docs/tutorial/README.md b/docs/tutorial/README.md index aeeae4a957..8ba1e3f726 100644 --- a/docs/tutorial/README.md +++ b/docs/tutorial/README.md @@ -2,6 +2,8 @@ # Python SDK tutorial +> **Only need native Fun-ASR-Nano transcription?** Start with [Transformers 5.17.0](../transformers_native.md). It loads the separate `-hf` checkpoint without the FunASR toolkit. This page covers the `funasr.AutoModel` toolkit path; do not mix dependencies, parameters or output contracts. + Start with [installation and environment verification](../installation/installation.md). This path covers one transcript, result inspection, VAD, batching, and model-specific options. For model selection, languages, dependencies, and model cards, use the [Model Zoo](../../model_zoo/readme.md), not a universal capability list. No local setup yet? See the [Colab quickstart](../../examples/colab/README.md). FunASR software uses the [MIT license](../../LICENSE). Each model weight has its own license: record the full model ID and revision, and follow its model card. The [FunASR Model License Agreement](../../MODEL_LICENSE) applies only when the model card explicitly links to it. Third-party integrations remain third-party models; for example, MOSS-Transcribe-Diarize is from OpenMOSS, not a FunASR-trained checkpoint. diff --git a/docs/tutorial/README_zh.md b/docs/tutorial/README_zh.md index 5e1ea05b30..c8b1e09c7d 100644 --- a/docs/tutorial/README_zh.md +++ b/docs/tutorial/README_zh.md @@ -2,6 +2,8 @@ # Python SDK 教程 +> **只需 Fun-ASR-Nano 原生推理?** 使用 [Transformers 5.17.0 快速开始](../transformers_native_zh.md)。它加载独立的 `-hf` 权重,不要求 FunASR 工具库。本页保留 `funasr.AutoModel` 工具库路径,两者的依赖、参数和输出不可混用。 + 先完成[安装与环境验证](../installation/installation_zh.md)。本页按首次转写、结果检查、VAD、批处理、模型专属选项的顺序介绍。模型选择、语言、依赖和模型卡请查阅[模型仓库](../../model_zoo/readme_zh.md),不要将某个示例当成所有模型的通用能力。尚未配置本地环境时,可先看 [Colab 快速体验](../../examples/colab/README_zh.md)。 FunASR 软件采用 [MIT 许可](../../LICENSE)。每个模型权重都有各自的许可:请记录完整模型 ID 与 revision,并以对应模型卡为准。只有模型卡明确链接 [FunASR 模型许可协议](../../MODEL_LICENSE) 时,该协议才适用。第三方集成仍属于第三方模型,例如 MOSS-Transcribe-Diarize 来自 OpenMOSS,不是 FunASR 训练的权重。 diff --git a/examples/colab/README.md b/examples/colab/README.md index 819400f89e..76b6c7668c 100644 --- a/examples/colab/README.md +++ b/examples/colab/README.md @@ -1,5 +1,7 @@ # FunASR Colab Quickstart +**Fun-ASR-Nano / Transformers:** [Open the native notebook](https://colab.research.google.com/github/QwenAudio/Fun-ASR/blob/main/examples/colab/fun_asr_nano_transformers.ipynb) for released 5.17.0, CPU, the official sample and your recordings. No toolkit is needed; the toolkit notebook below is a separate environment. + English | [简体中文](README_zh.md) | [日本語](README_ja.md) | [한국어](README_ko.md) Run FunASR in a browser without preparing a local Python environment. diff --git a/examples/colab/README_ja.md b/examples/colab/README_ja.md index dcc69acdce..8f8b4c4bdb 100644 --- a/examples/colab/README_ja.md +++ b/examples/colab/README_ja.md @@ -1,5 +1,7 @@ # FunASR Colab クイックスタート +**Fun-ASR-Nano / Transformers:** [ネイティブ Notebook](https://colab.research.google.com/github/QwenAudio/Fun-ASR/blob/main/examples/colab/fun_asr_nano_transformers.ipynb) は正式版 5.17.0、CPU、公式サンプルとローカル音声を使います。下記 toolkit Notebook とは別環境です。 + [English](README.md) | [简体中文](README_zh.md) | 日本語 | [한국어](README_ko.md) ローカルの Python 環境を準備せずに、ブラウザだけで FunASR を実行できます。 diff --git a/examples/colab/README_ko.md b/examples/colab/README_ko.md index cd46207d4f..728bf3c657 100644 --- a/examples/colab/README_ko.md +++ b/examples/colab/README_ko.md @@ -1,5 +1,7 @@ # FunASR Colab 빠른 시작 +**Fun-ASR-Nano / Transformers:** [네이티브 Notebook](https://colab.research.google.com/github/QwenAudio/Fun-ASR/blob/main/examples/colab/fun_asr_nano_transformers.ipynb)은 정식 5.17.0, CPU, 공식 샘플과 로컬 녹음을 사용합니다. 아래 toolkit Notebook과 환경이 다릅니다。 + [English](README.md) | [简体中文](README_zh.md) | [日本語](README_ja.md) | 한국어 로컬 Python 환경을 준비하지 않고 브라우저에서 바로 FunASR을 실행할 수 있습니다. diff --git a/examples/colab/README_zh.md b/examples/colab/README_zh.md index db2b7420ad..d5277e9272 100644 --- a/examples/colab/README_zh.md +++ b/examples/colab/README_zh.md @@ -1,5 +1,7 @@ # FunASR Colab 快速体验 +**Fun-ASR-Nano 原生 Transformers:** [打开新 Notebook](https://colab.research.google.com/github/QwenAudio/Fun-ASR/blob/main/examples/colab/fun_asr_nano_transformers.ipynb)。正式版 5.17.0、CPU、官方样例与自己的录音,无需 FunASR 工具库。下方保留独立的工具库 Notebook。 + [English](README.md) | 简体中文 | [日本語](README_ja.md) | [한국어](README_ko.md) 无需提前配置本地 Python 环境,直接在浏览器里运行 FunASR。 diff --git a/examples/industrial_data_pretraining/fun_asr_nano/README.md b/examples/industrial_data_pretraining/fun_asr_nano/README.md index 0a57eb8f20..2d86c78e5c 100644 --- a/examples/industrial_data_pretraining/fun_asr_nano/README.md +++ b/examples/industrial_data_pretraining/fun_asr_nano/README.md @@ -1,5 +1,7 @@ # Fun-ASR +> **Only need native Fun-ASR-Nano transcription?** Start with [Transformers 5.17.0](../../../docs/transformers_native.md). It loads the separate `-hf` checkpoint without the FunASR toolkit. This page covers the `funasr.AutoModel` toolkit path; do not mix dependencies, parameters or output contracts. + 「[简体中文](README_zh.md)」|「English」 Fun-ASR is an end-to-end speech recognition large model launched by Tongyi Lab. It is trained on tens of millions of hours of real speech data, possessing powerful contextual understanding capabilities and industry adaptability. The Nano checkpoint supports low-latency real-time transcription for Chinese, English, and Japanese, plus Chinese dialect groups and regional accents; the separate MLT-Nano checkpoint extends coverage to 31 languages. It excels in vertical domains such as education and finance, accurately recognizing professional terminology and industry expressions, effectively addressing challenges like "hallucination" generation and language confusion, achieving "clear hearing, understanding meaning, and accurate writing." diff --git a/examples/industrial_data_pretraining/fun_asr_nano/README_zh.md b/examples/industrial_data_pretraining/fun_asr_nano/README_zh.md index 52d51fad8a..6fb09df018 100644 --- a/examples/industrial_data_pretraining/fun_asr_nano/README_zh.md +++ b/examples/industrial_data_pretraining/fun_asr_nano/README_zh.md @@ -1,5 +1,7 @@ # Fun-ASR +> **只需 Fun-ASR-Nano 原生推理?** 使用 [Transformers 5.17.0 快速开始](../../../docs/transformers_native_zh.md)。它加载独立的 `-hf` 权重,不要求 FunASR 工具库。本页保留 `funasr.AutoModel` 工具库路径,两者的依赖、参数和输出不可混用。 + 「简体中文」|「[English](README.md)」 Fun-ASR 是通义实验室推出的端到端语音识别大模型,是基于数千万小时真实语音数据训练而成,具备强大的上下文理解能力与行业适应性,支持低延迟实时听写。Fun-ASR-Nano 支持中文、英文、日语,以及 7 种中文方言和 26 种地域口音;独立的 Fun-ASR-MLT-Nano checkpoint 则覆盖 31 个语种。在教育、金融等垂直领域表现出色,能准确识别专业术语与行业表达,有效应对"幻觉"生成和语种混淆等挑战,实现"听得清、懂其意、写得准"。 diff --git a/model_zoo/huggingface_models.md b/model_zoo/huggingface_models.md index fa09a52143..008ffc892b 100644 --- a/model_zoo/huggingface_models.md +++ b/model_zoo/huggingface_models.md @@ -15,6 +15,21 @@ For model selection and deployment boundaries, start with the Here we provided several pretrained models on different datasets. The details of models and datasets can be found on [ModelScope](https://www.modelscope.cn/models?page=1&tasks=auto-speech-recognition). ### Speech Recognition Models + +#### Fun-ASR-Nano: choose the checkpoint for your interface + +| Checkpoint | Interface | Scope | +| --- | --- | --- | +| [Fun-ASR-Nano-2512-hf](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512-hf) | Native Transformers **5.17.0**, `AutoProcessor` + `AutoModelForSpeechSeq2Seq` | Chinese, English, Japanese transcription; no remote Python code, native word timestamps or speaker identities | +| [Fun-ASR-Nano-2512](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512) | FunASR toolkit `AutoModel` | Original toolkit artifact; not interchangeable with the native export | +| [Fun-ASR-Nano-2512-vllm](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512-vllm) | Native vLLM | Separate serving conversion and runtime | +| [Fun-ASR-MLT-Nano-2512](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512) | FunASR toolkit | Separate 31-language checkpoint, not the zh/en/ja native export | + +Start with the [native Transformers guide](../docs/transformers_native.md), +[online Space](https://huggingface.co/spaces/FunAudioLLM/Fun-ASR-Nano), or +[batch examples and notebook](https://github.com/QwenAudio/Fun-ASR/tree/main/examples/transformers). +For a service, follow the [deployment matrix](../docs/deployment_matrix.md). + #### Paraformer Models | Model Name | Language | Training Data | Vocab Size | Parameter | Offline/Online | Notes | diff --git a/model_zoo/readme.md b/model_zoo/readme.md index 8c18140dc2..cfa33adea8 100644 --- a/model_zoo/readme.md +++ b/model_zoo/readme.md @@ -26,8 +26,9 @@ the [deployment matrix](../docs/deployment_matrix.md) for serving options. Use [FunAudioLLM/Fun-ASR-Nano-2512-hf](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512-hf) with `AutoProcessor` and `AutoModelForSpeechSeq2Seq`, not the original toolkit, native vLLM or GGUF loader. [Installation and inference](../docs/transformers_native.md) -pin the merged source and official revision. As checked on 2026-09-09, stable -Transformers 5.16.1 does not contain this native model. Matching torchaudio is required. +use released Transformers **5.17.0** and a pinned official revision. CPU examples, +batching and a notebook are available; matching torchaudio is required. This +zh/en/ja export returns text, not word timestamps or speaker identities. ### FunASR toolkit diff --git a/model_zoo/readme_zh.md b/model_zoo/readme_zh.md index 6caaf8ec6a..77608ec7dd 100644 --- a/model_zoo/readme_zh.md +++ b/model_zoo/readme_zh.md @@ -24,9 +24,9 @@ [FunAudioLLM/Fun-ASR-Nano-2512-hf](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512-hf) 使用 `AutoProcessor` 和 `AutoModelForSpeechSeq2Seq`,不是原始工具包、原生 vLLM 或 -GGUF 的加载路径。[安装与推理指南](../docs/transformers_native_zh.md)固定了上游提交和官方 -模型 revision。2026-09-09 核验的稳定版 Transformers 5.16.1 尚未包含该原生模型, -并且这条路径需要匹配的 torchaudio。 +GGUF 的加载路径。[安装与推理指南](../docs/transformers_native_zh.md)使用正式版 +Transformers **5.17.0** 与固定模型 revision,提供 CPU、批处理和 Notebook 示例。 +需要匹配的 torchaudio;该中英日原生导出返回文本,不提供字级时间戳或说话人身份。 ### FunASR 工具包 diff --git a/web-pages/product-site/content/legacy-manifest.json b/web-pages/product-site/content/legacy-manifest.json index a4433029e6..1c59b5497f 100644 --- a/web-pages/product-site/content/legacy-manifest.json +++ b/web-pages/product-site/content/legacy-manifest.json @@ -3,8 +3,8 @@ "source": "https://www.funasr.com/ public static corpus", "captured": "2026-07-26", "files": { - "blog/fun-asr-nano-transformers.html": "dd8e53b59e502fd6ff54237943564fbc05886eae747275b97dc862281cf80bbe", - "en/blog/fun-asr-nano-transformers.html": "9619a7fae30c444a67e566fe6bbe44fea1805f2778e5e8003f0387aacb33e29e", + "blog/fun-asr-nano-transformers.html": "139401dd9790362bb7c8d935eb1241f51af67ae6135e5c268699f64707941719", + "en/blog/fun-asr-nano-transformers.html": "0cb7a1cab4210a29622b17154b3ad59b0f2d6be67829ef12060f5d5780738c8f", "img/fun-asr-nano-native-waveform.png": "930517464ffb47bc7f9051dbdcb5422e1b3daa5c6abaa455a8bf072359ae0f0b", "7ec404429c825fe9a9a030731bbca986.txt": "e0e4c945481cb03f52426d8420c3407ea18cd3b46cb2482590bd50e5aab117d2", "blog/cantonese-speech-recognition.html": "f4a674f2b8f15963a386d586bb79d62f9f75c20d704ca2eafba21a800f52d4df", @@ -88,7 +88,7 @@ "en/ecosystem.html": "f02ee516440dfc3757165df801fbf93b403b2ef1fdbee1a2dee26cd2a3ef9fa3", "en/index.html": "d86effc3bf4d218eabd465cbbe026f62c98dd50110ddd58251d9d271a64882a2", "en/llama-cpp.html": "dbfbd125ebbf07f7d923c91b122818f8b09484ce54c1e80eba66a89d22dfb5b4", - "en/models.html": "5802ec4902b145d2ea398e172a07680cbef61188a3dc611f4d313db1dc12b1d8", + "en/models.html": "9806d01ba1807c2a6a88f7232102e185426fb8458bf51e1d7a8ce872352e1fd1", "en/quickstart.html": "7453cd7d582079eb4082373bce6e5de26492458b73c41617bbb1bd4248bda6ac", "en/vs-whisper.html": "4833b93195957210436f791a9d8ebc726f0af93b809d740a5cee1e53de0ac390", "favicon.ico": "e5edd9f12088fa920b848485c62c72a093f27bfba7b6a46b4e7b63c34eb5126a", @@ -113,7 +113,7 @@ "lang-redirect.js": "e1189080a8e81eb76cf39e0538d0378b6631f5aa87e4679c0fd4fd34d1f6c197", "llama-cpp.html": "892b5c05e3375bcc7cd0af343b629f069d8ec4f372e6470cd830cdf2181a11d5", "logo.png": "ac0ff97f2b0203c3fad4ac5e90231d93e36d98cc8e90d383483de150b0dd0728", - "models.html": "1e16794d389275a1ecca05bcc218a9c89311582aebcec7e62ec4b1ecd4379d40", + "models.html": "fd4179a53735b8e034a1ee8e79fb78c6c66b6a6205c7124fcef7f4e0aaf92437", "quickstart.html": "a543bf07c2cfc9a69ef3831f15c9e9d3e571a826a73cf7e4bdc8e9eb7b910254", "robots.txt": "1305b0f680b3e9f73b01301dc474423230dffab85731a379a694f30c73c07f2d", "sitemap.xml": "1fc88be1d337d7fac6acf0dd61e6b3416f6808f5d6e8fc0006fb9b15cba6e039", diff --git a/web-pages/product-site/data/documentation.json b/web-pages/product-site/data/documentation.json index 97c7a93ffd..1f7ee243f4 100644 --- a/web-pages/product-site/data/documentation.json +++ b/web-pages/product-site/data/documentation.json @@ -15,7 +15,7 @@ {"id": "reference", "zh": "协议与参考", "en": "Protocols & reference"} ], "pages": [ - {"slug": "native-transformers", "group": "models", "zh": "Transformers 原生 Fun-ASR-Nano", "en": "Native Transformers Fun-ASR-Nano", "source_zh": "docs/transformers_native_zh.md", "source_en": "docs/transformers_native.md"}, + {"slug": "native-transformers", "group": "start", "zh": "Transformers 快速开始", "en": "Transformers quickstart", "source_zh": "docs/transformers_native_zh.md", "source_en": "docs/transformers_native.md"}, {"slug": "installation", "group": "start", "zh": "安装与环境验证", "en": "Installation & environment checks", "source_zh": "docs/installation/installation_zh.md", "source_en": "docs/installation/installation.md"}, {"slug": "quickstart", "group": "start", "zh": "第一次转写", "en": "Your first transcription", "source_zh": "docs/tutorial/README_zh.md", "source_en": "docs/tutorial/README.md"}, {"slug": "python-api", "group": "start", "zh": "Python SDK: AutoModel", "en": "Python SDK: AutoModel", "source_zh": "docs/python_api_zh.md", "source_en": "docs/python_api.md"}, diff --git a/web-pages/product-site/legacy/blog/fun-asr-nano-transformers.html b/web-pages/product-site/legacy/blog/fun-asr-nano-transformers.html index e426073edd..d56311b821 100644 --- a/web-pages/product-site/legacy/blog/fun-asr-nano-transformers.html +++ b/web-pages/product-site/legacy/blog/fun-asr-nano-transformers.html @@ -16,6 +16,7 @@

接入 Transformers,先选对权重

如果你已经在用 Transformers,接入语音识别时先确认权重属于这条接口。同一个模型名字,不代表工具包、服务端和原生 Python 能互换文件。

例如,文件下载完了,应用却提示“不认识模型类型”。先检查安装包有没有实现、权重是不是对应格式,不要急着把问题归为文件损坏。本篇选择官方 -hf 权重。

这条路径生成转写文本,不是现成服务。加载成功不等于拿到了字级时间戳、说话人分离或实时流式能力。

+

想直接试一段录音?查看 Transformers 5.17.0 快速开始,或打开 SpaceNotebook。先得到一次转写,再扩展批处理与服务部署。

用于功能检查的官方中文样本波形,横轴为秒,纵轴为振幅
官方中文样本原始波形:约 5.62 秒、单声道 48 kHz。运行指南前应明确重采样为 16 kHz。波形图不是准确率或性能结果。

同一模型家族,四条不同路径

@@ -30,7 +31,7 @@

接入 Transformers,先选对权重

一次真实短录音检查说明了什么?

-

在固定源码、官方 -hf revision 和 CPU float32 环境中,我们运行了中文单录音、英文单录音、中英混合批量,以及一个带关键词的中文请求。四个请求均返回文本并在上限前生成 EOS;批量结果顺序正确。这是两份公开短音频的功能检查,不是准确率排行榜。

+

在正式版 Transformers 5.17.0、官方 -hf revision 和 CPU float32 环境中,我们运行了中文单录音、英文单录音、中英混合批量,以及一个带关键词的中文请求。四个请求均返回文本并在上限前生成 EOS;批量结果顺序正确。这是两份公开短音频的功能检查,不是准确率排行榜。

@@ -46,7 +47,7 @@

接入 Transformers,先选对权重

附录:环境与接口细节

先确认安装包,不只看合并状态

-

PR #46180 于 2026-09-09 合并。当天核验的稳定版 5.16.1 还没有原生 fun_asr_nano 文件;源码包显示的 5.17.0.dev0 也不是下一稳定版发布承诺。请从固定版本接入指南安装明确的源码提交,不要只执行一次无版本约束的升级就假定已经兼容。

+

PR #46180 于 2026-09-09 合并,随后发布的 Transformers 5.17.0 已包含原生实现。我们检查了实际 wheel,并在 CPU 上复测真实中英文、热词和批处理。现在可以直接使用正式包快速开始,不再要求从源码安装。

这种区别影响排障方向:未知模型类型可能是安装包未包含实现,不一定是 checkpoint 损坏;模型能下载也不代表所选运行时认识它。

从声音到文本,中间实际经过什么?

  1. 音频样本。采样率与声道是数据含义的一部分。48 kHz 的数组不能只改参数标成 16 kHz;那会改变模型看到的时间尺度。指南先要求明确的单声道 16 kHz WAV。
  2. diff --git a/web-pages/product-site/legacy/en/blog/fun-asr-nano-transformers.html b/web-pages/product-site/legacy/en/blog/fun-asr-nano-transformers.html index 30e1ef1e66..dc43d310d9 100644 --- a/web-pages/product-site/legacy/en/blog/fun-asr-nano-transformers.html +++ b/web-pages/product-site/legacy/en/blog/fun-asr-nano-transformers.html @@ -16,6 +16,7 @@

    Choose the right checkpoint for Transformers

    If you are adding speech recognition to an existing Transformers application, choose the checkpoint for that interface first. A shared model name does not make toolkit, serving and native Python weights interchangeable.

    For example, a checkpoint may download successfully while the application reports an unknown model type. Check both the installed implementation and the selected artifact before assuming the weights are damaged. The native path here uses the official -hf checkpoint.

    This is a text-generation interface, not a ready-made service. It does not supply word timestamps, speaker separation or streaming simply because loading succeeds.

    +

    Ready to try it? Follow the Transformers 5.17.0 quickstart, open the Space or run the notebook. Start with one recording, then add batches or a deployment backend.

    Official Chinese functional-test sample waveform, time in seconds and amplitude
    The official Chinese sample: approximately 5.62 seconds, mono, originally 48 kHz. Explicitly resample it to 16 kHz before the guide's input step. A waveform is not an accuracy or performance result.

    One family, four different paths

    @@ -30,7 +31,7 @@

    Choose the right checkpoint for Transformers

    What did the short-recording check establish?

    -

    With pinned source, official -hf revision and CPU float32, we ran a Chinese recording, an English recording, a mixed Chinese/English batch and a Chinese keyword request. All four requests returned text and EOS before the limit, with the batch in input order. These are functional checks on two public short recordings, not an accuracy ranking.

    +

    With released Transformers 5.17.0, official -hf revision and CPU float32, we ran a Chinese recording, an English recording, a mixed Chinese/English batch and a Chinese keyword request. All four requests returned text and EOS before the limit, with the batch in input order. These are functional checks on two public short recordings, not an accuracy ranking.

同一份中文音频原始输出
不传关键词开饭时间早上九点至下午五点。
候选关键词:开放时间开饭时间:早上九点至下午五点。
@@ -46,7 +47,7 @@

Choose the right checkpoint for Transformers

Appendix: environment and interface details

Check the package, not just the merge

-

PR #46180 merged on 2026-09-09. The stable 5.16.1 package checked that day did not contain native fun_asr_nano files. The source build's 5.17.0.dev0 label is not a promise of the next stable version or release date. Use the explicit source commit in the pinned setup guide; an unversioned upgrade alone is not evidence of compatibility.

+

PR #46180 merged on 2026-09-09, and released Transformers 5.17.0 now contains the native implementation. We inspected the published wheel and repeated real Chinese/English inference, keywords and batching on CPU. Use the stable-package quickstart; a source build is no longer required.

This distinction changes troubleshooting: an unknown model type may mean that the installed package lacks the implementation, not that the checkpoint is corrupt. A successful model download does not prove that the selected runtime can load it.

What happens between audio and text?

  1. Audio samples. Sample rate and channel layout are part of the data's meaning. Labeling a 48 kHz array as 16 kHz changes the time scale seen by the model. The guide requires an explicitly prepared mono 16 kHz WAV.
  2. diff --git a/web-pages/product-site/legacy/en/models.html b/web-pages/product-site/legacy/en/models.html index e660afcfb5..7fe0b08d46 100644 --- a/web-pages/product-site/legacy/en/models.html +++ b/web-pages/product-site/legacy/en/models.html @@ -88,14 +88,14 @@

    ASR Models

    Fun-ASR-Nano ⭐ Flagship / default
    -
    800M params · LLM-based (SenseVoice encoder + Qwen3-0.6B) · GitHub · HuggingFace
    -

    Native Transformers support is merged upstream. Use the official -hf checkpoint, distinct from original, -vllm and GGUF artifacts. The pinned installation and inference guide explains why stable 5.16.1 is not sufficient.

    -
    Next-generation LLM-based ASR model. Combines SenseVoice's audio encoder with Qwen3-0.6B language model for superior context understanding. Supports vLLM acceleration for high-throughput batch inference and real-time streaming. The released model.pt checkpoint does not provide reliable checkpoint-native character timestamps (issue #106).
    +
    800M params · LLM-based (SenseVoice encoder + Qwen3-0.6B) · GitHub · HuggingFace
    +

    Start with released Transformers 5.17.0 and the official -hf checkpoint. CPU quickstart and batches · Try online · Notebook. Text transcription for zh/en/ja; original, -vllm and GGUF artifacts use different interfaces.

    +
    SenseVoice audio encoder plus Qwen3-0.6B decoder. Native Transformers produces text; service and edge paths use their own vLLM or GGUF artifacts. Timestamp and speaker capabilities depend on the exact checkpoint and pipeline: see the model capability boundaries.
    -vLLM accelerated +vLLM runtime zh/en/ja + Chinese dialects/accents -Streaming -LLM-quality +Transformers +Context hints

    When to use

    Best for high-throughput batch processing, real-time subtitles, and scenarios where LLM-quality context understanding improves output (e.g., proper nouns and code-switching). For reliable character-level timestamps, use Paraformer.

    diff --git a/web-pages/product-site/legacy/models.html b/web-pages/product-site/legacy/models.html index bd11a2d947..80ae538399 100644 --- a/web-pages/product-site/legacy/models.html +++ b/web-pages/product-site/legacy/models.html @@ -88,14 +88,14 @@

    ASR 模型

    Fun-ASR-Nano ⭐ 旗舰 / 默认推荐
    -
    800M 参数 · LLM 架构 (SenseVoice 编码器 + Qwen3-0.6B) · GitHub · HuggingFace
    -

    原生 Transformers 路径已合入上游:使用 官方 -hf checkpoint,不要与原始权重、-vllm 或 GGUF 混用。固定版本的安装与推理指南说明 5.16.1 稳定版尚未包含的边界。

    -
    新一代 LLM 架构 ASR 模型。融合 SenseVoice 音频编码器与 Qwen3-0.6B 语言模型,具备更强的上下文理解能力。支持 vLLM 加速的高吞吐批量推理与实时流式服务。发布的 model.pt checkpoint 不提供可靠的原生字级时间戳(issue #106)。
    +
    800M 参数 · LLM 架构 (SenseVoice 编码器 + Qwen3-0.6B) · GitHub · HuggingFace
    +

    从正式版 Transformers 5.17.0官方 -hf 权重开始。CPU 快速上手与批处理 · 在线体验 · Notebook。支持中英日文本转写;原始、-vllm 和 GGUF 权重的接口不同。

    +
    融合 SenseVoice 音频编码器与 Qwen3-0.6B 解码器。原生 Transformers 返回文本;服务与端侧分别使用对应的 vLLM、GGUF 权重。时间戳和说话人能力取决于具体 checkpoint 与流水线,见模型能力边界
    -vLLM 加速 +vLLM 服务 中/英/日 + 中文方言/口音 -实时流式 -LLM 级别 +Transformers +上下文提示

    适用场景

    适合高吞吐批处理、实时字幕,以及 LLM 级别上下文理解可提升输出质量的场景(如专有名词、代码混合语音);需要可靠字级时间戳时请使用 Paraformer。

    diff --git a/web-pages/product-site/tests/browser/native-transformers.spec.ts b/web-pages/product-site/tests/browser/native-transformers.spec.ts index 0efedf89eb..e666ce3703 100644 --- a/web-pages/product-site/tests/browser/native-transformers.spec.ts +++ b/web-pages/product-site/tests/browser/native-transformers.spec.ts @@ -43,6 +43,8 @@ for (const prefix of ['', 'en/']) { 'href', new RegExp(`/docs/transformers_native${prefix ? '' : '_zh'}\\.md$`), ); await expect(page.locator('.docs-article')).toContainText('torchaudio==2.10.0+cpu'); + await expect(page.locator('.docs-article')).toContainText('transformers==5.17.0'); + await expect(page.locator('.docs-article a[href*="fun_asr_nano_transformers.ipynb"]')).toBeVisible(); expect(await page.evaluate(() => document.documentElement.scrollWidth - document.documentElement.clientWidth)).toBeLessThanOrEqual(1); await page.screenshot({ path: testInfo.outputPath('guide-top.png') }); await page.goto(`/${prefix}models.html`); diff --git a/web-pages/product-site/tests/test_native_transformers_docs.py b/web-pages/product-site/tests/test_native_transformers_docs.py index e2fda7b700..b09f44fce7 100644 --- a/web-pages/product-site/tests/test_native_transformers_docs.py +++ b/web-pages/product-site/tests/test_native_transformers_docs.py @@ -48,10 +48,12 @@ def test_catalogue_and_pages_export_own_both_native_guides(): def test_native_guide_pins_format_runtime_and_audio_dependency(suffix): text = guide(suffix) for expected in (MODEL, REVISION, HEAD, "torchaudio==2.10.0+cpu", - "torch==2.10.0+cpu", "5.16.1", "2026-09-09", + "torch==2.10.0+cpu", "transformers==5.17.0", "2026-09-09", "vllm", "GGUF", "46180"): assert expected in text - assert re.search(r"not.*stable|stable.*not|稳定版.*不|不.*稳定版", text, re.I) + assert "huggingface/transformers/archive/" not in text + assert "examples/transformers" in text + assert text.index("native-example: transcribe") < text.index("native-example: processor") assert re.search(r"not.*(?:capacity|accuracy)|不.*(?:容量|准确率)", text, re.I) @@ -77,7 +79,28 @@ def test_bilingual_recipes_are_identical_and_use_fixed_native_checkpoint(name): assert "inference_mode" in en and "max_new_tokens" in en assert "generated[:, inputs.input_ids.shape[1]:]" in en assert "batch_decode" in en - assert "16000" in en and "ndim" in en + assert "16000" in en and "example/en.mp3" in en + assert "272c57b82523ada6fd87095e955f8e29100979ab" in en + + +@pytest.mark.parametrize("suffix", ["", "_zh", "_ja", "_ko"]) +def test_all_language_entry_points_reach_native_examples(suffix): + for name in [f"README{suffix}.md", f"docs/model_selection{suffix}.md", + f"docs/deployment_matrix{suffix}.md", f"examples/colab/README{suffix}.md"]: + text = (ROOT / name).read_text() + assert "5.17.0" in text, name + assert "5.16.1" not in text, name + text = (ROOT / f"README{suffix}.md").read_text() + assert "examples/transformers" in text + assert "fun_asr_nano_transformers.ipynb" in text + + +def test_hf_catalogue_and_sphinx_reach_native_model(): + text = (ROOT / "model_zoo/huggingface_models.md").read_text() + assert MODEL in text and "transformers_native.md" in text + text = (ROOT / "docs/index.rst").read_text() + assert " transformers_native\n" in text + assert " transformers_native_zh\n" in text @pytest.mark.parametrize("name,heading", [
The same Chinese audioRaw output
No keywords开饭时间早上九点至下午五点。
Candidate keyword: 开放时间开饭时间:早上九点至下午五点。