feat(sources): add YouTube search connector (yt-dlp)

[matthewdonsemail-lab]

Summary

Adds YouTube search capability to OpenMagpie using yt-dlp library, following the same patterns as the Twitter connector.

What's Changed

New Files

• apps/core/sources/connectors/youtube/__init__.py
• apps/core/sources/connectors/youtube/client.py (YtDlpClient wrapper)
• apps/core/sources/connectors/youtube/connector.py (YouTubeSearchConnector)
• apps/core/sources/connectors/youtube/errors.py (YouTubeError taxonomy)
• apps/core/sources/connectors/youtube/payloads.py (NewVideoPayload)

Modified Files

• packages/openmagpie-schema/src/openmagpie_schema/configs.py - Added YouTubeSearchSourceSpec
• apps/core/pyproject.toml - Added yt-dlp>=2026.07.04 dependency
• apps/core/sources/connectors/__init__.py - Exported YouTubeSearchConnector

Design Decisions

1. Search API: Uses yt-dlp's ytsearch<N>:<query> URI scheme - no authentication required for public content
2. Error Handling: 6 error codes with retry semantics (video_unavailable, rate_limited, js_runtime_missing, network_error, date_parse_error, yt_dlp_error)
3. Payload Mapping: YouTube metadata -> SourcePayload with title, description, metrics (views/likes/comments), thumbnails, duration
4. Watermark Filtering: Same dedup pattern as Twitter connector using last_event_at

Testing

Live search verified:

Found 2 videos for "AI news" query
- Zuckerberg makes 'STUNNING' announcement on AI race... (Fox News Clips, 69742 views)
- The AI Safety Tests Are Broken. All Of Them... (AI Revolution, 26092 views)

Full specification in: RESEARCH/YT_DLP_OPENMAGPIE_INTEGRATION.md

This PR stacks on top of PR #175 (X/Twitter connector). Merge that first.

[joshpanka]

@matthewdonsemail-lab Merged, thanks again for the contribution! I added some changes on top before merging: a starter example, a YOUTUBE_COOKIES_FILE env var (with a credentials dir convention that covers the twitter cookie files too), and a fix to the search itself. YouTube removed sort-by-upload-date entirely (why yt-dlp dropped ytsearchdate), so it now uses the "this week" upload filter to keep results recent. Got it running locally end to end through a feed and watch and things look good.

Will look at the facebook piece next.

[matthewdonsemail-lab]

@matthewdonsemail-lab Merged, thanks again for the contribution! I added some changes on top before merging: a starter example, a YOUTUBE_COOKIES_FILE env var (with a credentials dir convention that covers the twitter cookie files too), and a fix to the search itself. YouTube removed sort-by-upload-date entirely (why yt-dlp dropped ytsearchdate), so it now uses the "this week" upload filter to keep results recent. Got it running locally end to end through a feed and watch and things look good.

Will look at the facebook piece next.

good to see you're finally checking the repo! @joshpanka I dont know how keen you are on sending this to the moon but we could make a good bit of money working together. I am about to set this up https://x.com/matthewsoldit/status/2094814046654759086 we're going to use openmagpie to send out 100k sms's to book meetings and we have a sales team. would be cool to collab because alot of this is going to be built around openmagpie.

I saw someone on twitter steal this exact idea https://www.stalkr.ai/ and I dont want to sell it as a saas product. I want to make it more of a service based thing. I am really f*ckin good at building lead acquisition systems so if we combined forces and made alot of unofficial api's I think we could really do something big with this!

https://listeningkit-frontend-web.vercel.app/ check it out will be up next week and we'll start our campaign.