Repository navigation
Sync videos from YouTube and fix the published transcripts - #487
Merged
Merged
Conversation
Produced with the new video scripts during testing: - data/videos.toml: 43 new videos, 270 changed (mostly YouTube no longer reporting an audio language) - 43 new video pages, 25 renamed after title changes, 283 updated - New transcripts for 2026-11-10 (Florent Bled hangout) and 2026-11-11 (AI voice agents in Shiny), and a complete re-formatted transcript for 'Let the agent cook' (the published one stopped at 59 of 75 minutes)
Ran format-transcriptions.py --repair-timestamps: every <a data-t> now gets the start time of the VTT cue its text comes from, instead of the model's often miscalculated value. Only data-t values change (1,484 transcripts).
These short videos (6-79 seconds) have no speech, so Whisper returned an empty VTT and the model made up a 'transcript' from the title (one reads 'Video title: What can you do with Positron Assistant?'). Remove both files and set transcribe: false, so the pipeline leaves them alone.
These transcripts lost more than 10% of the spoken words (mostly videos of 45-90 minutes that were formatted in one request) or had malformed tags. Re-formatted with format-transcriptions.py --reformat-invalid: 15-minute chunks, an explicit instruction not to skip anything, timestamps taken from the VTT, and validation before writing. Median coverage of the new transcripts: 99.7% of the spoken words (minimum 90%).
…alid ones - Transcripts for the 41 new videos without one (Whisper, then Claude) - Re-formatted the transcripts the first pass rejected (malformed tags, network errors, Whisper repeats) - Repaired a link inserted inside another link (broken HTML on the 'End-to-end data science workflow' page) 1,748 of 1,749 transcripts now pass validation; the remaining one (2014-06-26_rstudio-overview-130) keeps its previous version.
✅ Deploy Preview for posit-open-source canceled.
|
Contributor
Publishing checklist
|
5 of 6 tasks
Contributor
Blog YAML Checks
|
Contributor
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prerequisite check
Related Issue
None. Companion of #486 (nightly video sync); merge this one first.
Description of changes
Brings the videos up to date (the last update was in September) and fixes the problems found in the published transcripts while building #486. Everything was produced with the new scripts from #486, run locally.
Catch-up with YouTube
data/videos.toml: 43 new videos and 270 changed entries (mostly YouTube no longer reporting an audio language).Fixed transcripts
Incomplete transcripts re-formatted (about 500). About 500 published transcripts had lost more than 10% of the spoken words: mostly videos of 45–90 minutes, formatted in one request that ran into the output limit or got condensed (e.g. "Let the agent cook" stopped at minute 59 of 75). Re-formatted in 15-minute chunks, with an explicit instruction not to skip anything; median coverage is now 99.7% of the spoken words.
Timestamps repaired (1,484 transcripts). The model often miscalculated the
data-ttimestamps (only 64% were within a second). They're now taken from the VTT cues: 97% exact. Onlydata-tvalues changed in these files.Made-up transcripts removed (4). Short videos without speech, for which the model had invented a transcript from the title; now marked
transcribe: false.One transcript redone that had an 11-minute stretch of empty Whisper cues.
One broken link fixed (a link inserted inside another link on the "End-to-end data science workflow" page; the cause is fixed in Sync videos nightly with GitHub Actions #486).
1,748 of 1,749 transcripts now pass the new validation (all tags allowed, timestamps within the video, at least 90% of the spoken words). The remaining one,
2014-06-26_rstudio-overview-130, keeps its previous version, with repaired timestamps. One video published today isn't included; the first nightly run will add it.Screenshots or previews (if applicable)
See the deploy preview, e.g. the video pages under
/resources/videos/.Checklist
CONTRIBUTING.mdguidelines.