Lets AI agents watch videos, locally on Apple Silicon. Give it a file or any URL yt-dlp handles; it returns a transcript and, for video, overview sheets (small, evenly spaced pictures) with an index.txt mapping each one to what was being said. Videos without an audio track get the same sheets and an empty transcript. The agent then fetches full-size frames at the moments that matter. Runs on-device with MLX: no API keys, no cloud.
Agents answered 156 questions about 21 videos, 36 of them about moments the speech never names: once from the transcript alone, once with vidscribe (overview sheets, --at and --scan). A blind grader scored the answers.
| Transcript only | vidscribe | |
|---|---|---|
| Score | 15% | 96% |
Method and caveats: docs/measurements.md. Questions, grades and scripts: eval/.
Needs an Apple Silicon Mac and Homebrew. The default model (~2.5 GB) downloads on first run.
git clone https://github.com/SamuelZ12/vidscribe.git
cd vidscribe && ./install.shMake sure ~/.local/bin is on your PATH, then check with vidscribe -h. Update with git pull. Manual install and uninstall: docs/usage.md.
vidscribe talk.mp4 # transcript + overview sheets
vidscribe "https://www.youtube.com/watch?v=<id>" -a # transcript only
vidscribe memo.m4a -f srt -o ~/transcripts # srt, kept outside /tmp
vidscribe /tmp/vidscribe/frames/<name> --at 8:14,7:50-8:50 # full-size frames from a run
vidscribe /tmp/vidscribe/frames/<name> --scan 50:00-60:00 # find an unnamed momenttxt transcripts print to stdout; the transcript path and frames folder go to stderr. Outputs stay in /tmp/vidscribe/ for 3 days. The default model is Qwen3-ASR (~30 languages); for others, use -m mlx-community/whisper-large-v3-turbo-asr-fp16. Full reference: docs/usage.md.
npx skills add SamuelZ12/vidscribeNeeds Node.js 22.20+. Without Node, copy skill/ into your agent's skills folder (Claude Code: .claude/skills/vidscribe).
- Speech only: music and other sounds aren't described.
- No OCR or word timestamps; agents read on-screen text from the frames.
Design decisions: docs/decisions/.
MIT. The models, ffmpeg and yt-dlp come under their own licenses: docs/usage.md.