Xiaoe Audio Pipeline

An original local-first pipeline for authorized Xiaoe courses:

Xiaoe catalog -> audio-only download -> ASR -> selectable structuring provider -> Markdown

The CLI owns all write operations. A loopback read API is available for a future optional frontend.

The desktop launcher shows courses in a ten-row paginated picker. Menu items 5, 6, and 7 accept a course number and pass the course ID automatically.

Double-click Xiaoe Audio Pipeline.command in the project root, or use the desktop forwarding launcher, to open the complete terminal menu.

Install

cd "/Users/cimu_lumi/Desktop/【項目】小工具/xiaoe-tools/xiaoe-audio-pipeline"
/usr/bin/python3 -m pip install --user -e .

For development without installing the command:

PYTHONPATH=src /usr/bin/python3 -m xiaoe_cli --help

Runtime data defaults to ~/.xiaoe-audio-pipeline. Use --data-dir or XIAOE_DATA_DIR to change it.

First Run

1. Open the managed system Chrome profile and complete Xiaoe login once:

xiaoe auth start --url "YOUR_AUTHORIZED_COURSE_URL"

The project launches the installed Google Chrome application with a dedicated local profile. Chrome stores its own session cookies locally. Optional automatic login credentials are read from macOS Keychain first, with a private local file available as a fallback.

Microsoft Edge is supported through the same Chromium/CDP implementation while keeping its managed login data in a separate local profile:

xiaoe browser use edge --json
xiaoe auth start --url "YOUR_AUTHORIZED_COURSE_URL"

Ego Lite is also supported. It runs the automation in an isolated Task Space and inherits the login state stored by Ego:

xiaoe browser use ego --json
xiaoe auth start --url "YOUR_AUTHORIZED_COURSE_URL"

Complete interactive login in the handed-off Ego Task Space, then run the normal auth check, course refresh, download, or run commands. Switch back at any time:

xiaoe browser use chrome --json
xiaoe browser current --json

Use --browser chrome, --browser edge, or --browser ego before a command for a one-off override without changing the saved selection.

2. Scan the purchased-course list, or add one authorized course manually:

xiaoe course scan-account --json
xiaoe course add "YOUR_AUTHORIZED_COURSE_URL" --title "Course Name" --json
xiaoe course refresh COURSE_ID --json

scan-account opens the saved Xiaoe store session, reads every page under “我的课程”, and imports course containers into the local course list. Individual live-session purchases remain in ~/.xiaoe-audio-pipeline/account-catalog.raw.json for auditing and are not imported as standalone courses. If several Xiaoe store origins already exist in the local list, each origin is scanned once.

3. Prepare the isolated local ASR runtime. It reads the Qwen3-ASR model already downloaded by OpenLess:

./scripts/setup_local_asr.sh

The runtime lives in .local-asr-venv. It does not change PATH or shell configuration and does not copy the model weights. Desktop menu item 15 selects another ASR service and stores its credentials in macOS Keychain.

Supported ASR providers:

Provider	Default model/interface	Credential
Local	Qwen3-ASR 1.7B on Apple MPS	None
Alibaba	Qwen3-ASR-Flash	DashScope API Key
Volcengine	BigModel AUC Turbo	X-Api-Key
Tencent Cloud	Flash file transcription	AppID + SecretID + SecretKey
Baidu Cloud	Short speech recognition	API Key
OpenAI	GPT-4o mini Transcribe	API Key
Groq	Whisper Large V3 Turbo	API Key
Deepgram	Nova-3	API Key
Custom	OpenAI-compatible transcription	API Key + Base URL

Inspect or change the non-secret selection from the CLI:

xiaoe asr providers --json
xiaoe asr current --json
xiaoe asr use openai --model gpt-4o-mini-transcribe

4. Select a structuring provider. Desktop menu item 16 provides the same choices:

Provider	Mode	Default model	Thought control	Credential
Codex	Local agent CLI	Account default	minimal to xhigh	Existing Codex login
Claude Code	Local agent CLI	sonnet	low to max	Existing Claude Code login
OpenAI	Responses API	gpt-5-mini	none to xhigh, model-dependent	API Key
Anthropic	Messages API	claude-sonnet-4-6	Levels returned by model capabilities	API Key
DeepSeek	OpenAI-compatible API	deepseek-chat	Select a reasoning model	API Key
Alibaba Model Studio	OpenAI-compatible API	qwen-plus	Model-dependent effort	DashScope API Key
Custom	OpenAI-compatible API	User-selected	Generic reasoning effort	API Key + Base URL

xiaoe structurer providers
xiaoe structurer current
xiaoe structurer models openai
xiaoe structurer models openai --refresh
xiaoe structurer use claude-code --model sonnet --effort medium
xiaoe structurer use openai --model gpt-5-mini --effort medium
xiaoe structurer prompt init
xiaoe structurer prompt show
xiaoe structurer prompt open

Provider and model choices are saved in settings.json. API keys are kept in macOS Keychain and never written to that settings file. Local agents run as single-use, non-interactive jobs with tool access disabled or read-only.

The desktop selector shows ten models per page and always allows a manually entered model ID. API model catalogs are fetched with a three-second timeout and stored in ~/.xiaoe-audio-pipeline/structure-models.json. A fresh cache is used for six hours. Later refreshes send ETag or Last-Modified validators when the provider supplies them. A failed refresh uses the cached list and suppresses repeat network waits for 30 minutes. Local-agent aliases use a built-in list because their CLIs do not expose a stable model-list endpoint.

structurer prompt init creates and enables ~/.xiaoe-audio-pipeline/structure-prompt.txt. Edit that file to customize the role, organization rules, tone, cleanup policy, or level of detail. {{LESSON_TITLE}} and {{TRANSCRIPT}} are optional placement markers. When either marker is absent, the program appends that input to the end of the template automatically. Changes take effect on the next structuring run. Use structurer prompt use --file /absolute/path/prompt.txt for another file, structurer prompt open to launch the selected template in the system default text editor, or structurer prompt reset to restore the built-in faithful-note prompt. The desktop provider configuration offers the same choices without opening another application.

5. Run the complete workflow:

xiaoe run COURSE_ID --language zh --json

Run a small learning batch first:

xiaoe run COURSE_ID --limit 1 --language zh --json

Stage Commands

xiaoe download COURSE_ID --limit 1 --json
xiaoe transcribe COURSE_ID --limit 1 --language zh --json
xiaoe structure COURSE_ID --limit 1 --json
xiaoe structure COURSE_ID --limit 1 --provider claude-code --model sonnet --effort medium
xiaoe status --json

Completed audio artifacts are matched by their persisted metadata and file size, then skipped on later runs without re-reading gigabytes of media. If the database record is missing, download.json is used to restore it automatically. Use --force-transcription or --force-structure with xiaoe run when regeneration is intentional.

During a course download, the browser resolves the next incomplete lesson while ffmpeg processes the current lesson. Short-lived signed media URLs are cached in the lesson directory with 600 permissions, so an interrupted run can resume without reopening the lesson page while the URL is valid. Browser cookies and authorization headers are never written to this cache.

Authentication checks:

xiaoe auth login --json
xiaoe auth check --url "YOUR_AUTHORIZED_COURSE_URL" --json
xiaoe auth check --url "YOUR_AUTHORIZED_COURSE_URL" --recover --json
xiaoe auth credentials --template --json
xiaoe auth stop

When --recover finds an expired login, it first reads xiaoe.login.username and xiaoe.login.password from macOS Keychain. If they are absent, it reads the private ~/.xiaoe-audio-pipeline/xiaoe-login.json file. The desktop launcher prints that path when credentials are missing and never opens another app. Its permissions are forced to 600.

The password form at https://study.xiaoe-tech.com/#/acount is submitted in the selected browser's managed profile. Automated browser work always stays in the background. When QR code, image code, slider, SMS, or device verification is required, the CLI prints the login URL so the user can open it manually later. Desktop menu item 1 runs auth login directly and never asks for a course URL.

Processing Details

• Account discovery reads the authorized “我的课程” list and imports columns, large columns, camps, memberships, and course catalogs.
• Course discovery listens to authorized page JSON responses and uses Xiaoe resource_id as the stable lesson key.
• Catalog metadata classifies lessons as text, audio, video, live replay, or probe-required. Explicit text/no-replay items skip browser, ASR, and structuring waits immediately; unknown legacy types still use browser detection.
• A suspected login failure is confirmed against the Xiaoe learning-center account page. Valid media/player evidence wins, and a lesson-specific capture failure does not invalidate the whole account session.
• Collapsed catalog chapters are expanded in bounded batches, and the active browser tab and authenticated Gateway are reused across the course run.
• Direct audio preserves original bytes and supports HTTP Range resume.
• HLS prefers an independent audio rendition; mixed video streams are reduced to their first audio track.
• HLS and video-file handling uses two explicit stages: network audio download to audio.downloaded.mka, then local remux/transcoding to m4a. A completed network stage is reused after interruption.
• Download and conversion progress show percentages and estimated remaining time when duration or content length is available. Network progress also shows transferred size and current throughput; conversion progress shows ffmpeg's media-time speed.
• Before a course download starts, the CLI prints every lesson's catalog type and local state: complete, downloaded/awaiting conversion, interrupted/partial, pending, cached source, or no media.
• CLI tables calculate terminal display width for CJK text so Chinese and Latin columns remain aligned.
• Standard AES-128 HLS is handled by ffmpeg. SAMPLE-AES and DRM are reported as unsupported.
• Long audio is split locally into provider-sized, 16 kHz mono chunks. Baidu uses 55-second WAV chunks; the other current providers use four-minute MP3 chunks. The local Qwen worker loads the model once and transcribes all chunks through Apple MPS.
• Raw provider responses, normalized transcripts, plain text, structured JSON, and Markdown notes are all retained.
• Codex runs ephemerally in a read-only sandbox. Claude Code runs without tools or session persistence. Both return schema-constrained note data.
• API structurers receive the same fidelity prompt and JSON schema. The OpenAI adapter disables response storage.

Per-lesson output:

audio.source.<ext>
download.json
asr_chunks/chunk_0000.mp3
transcript.raw.json
transcript.json
transcript.txt
notes.data.json
notes.md

Background Course Monitoring

Each saved course can be checked on its own interval. A due check validates the saved Xiaoe session, refreshes the course catalog, and runs the existing incremental pipeline. Completed lessons are skipped, while newly discovered lessons continue through audio download, ASR, and structured Markdown.

Add a monitor and start the native background service:

xiaoe schedule add COURSE_ID --every 30m
xiaoe schedule add ANOTHER_COURSE_ID --every 2h

Supported frequency suffixes are m for minutes, h for hours, and d for days. The minimum interval is one minute. The operating system wakes one small worker every minute; only courses whose own interval has expired are processed.

Manage monitoring:

xiaoe schedule list
xiaoe schedule disable COURSE_ID
xiaoe schedule enable COURSE_ID
xiaoe schedule run-now COURSE_ID
xiaoe schedule remove COURSE_ID
xiaoe schedule logs --limit 30

Control the shared background service while keeping every course setting:

xiaoe schedule start
xiaoe schedule stop
xiaoe schedule status
xiaoe schedule uninstall

macOS uses a per-user launch agent at ~/Library/LaunchAgents/com.ecai.course-monitor.plist. Windows uses a per-user Task Scheduler entry named ECAI Course Monitor. The main menu and Terminal may remain closed. Schedule state and recent results are stored under ~/.xiaoe-audio-pipeline/ in schedules.json and scheduler.log.

Background browser operations request hidden/background mode. If login expires, credentials are missing, or an Ego Task Space is under user control, that run is recorded as failed and the next scheduled check can try again. Interactive verification remains available through the normal login menu.

Local API

xiaoe serve --port 8765

Read endpoints:

• GET /api/health
• GET /api/status
• GET /api/courses
• GET /api/courses/{course_id}/lessons

The service binds only to 127.0.0.1. It is intended for a local frontend and has no remote-production security model.

Current Verification Boundary

Automated tests cover database migration, paginated course selection, Chrome session helpers, direct and HLS downloads, AES-128, cloud and persistent local ASR adapters, cloud request signing/normalization, chunk merging, local-agent and API structuring, pipeline orchestration, and the local API.

The authorized Xiaoe course catalog, paginated picker, media-source capture, download start, and local Qwen inference have been live-tested. Cloud ASR and structuring adapters have protocol-level tests; each cloud service still needs a first live request with your own credential. Codex and Claude Code executable availability is checked locally; their first real structuring job uses the login already held by each CLI. Interactive login challenges still require user action when Xiaoe expires the saved session.

When a user stops the dedicated Ego Agent during an interactive course run, the CLI preserves completed artifacts and pauses at the current task. Finish the browser action and press Enter in the CLI to reclaim the same Ego Task Space and resume. JSON and other non-interactive runs return the control error immediately so unattended callers never wait for terminal input.

Long-running catalog, media-source, ASR, and structuring operations show a TTY-only spinner with elapsed seconds. Redirected and JSON output stays clean.

Safety

Use this project only with content you own or are explicitly authorized to download. Runtime files, browser profile data, cookies, media, transcripts, and notes are excluded from Git.