diff --git a/AGENTS.md b/AGENTS.md index ccece3b..44a7b68 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -41,8 +41,12 @@ swift test # full suite swift test --filter TimestampUnwrapperTests # one suite swift test --filter DecoderTests/testName # one test swift run LumeEngineDemo # macOS demo app with diagnostics HUD +swift run lumectl probe # tracks, codecs, live/seekable, HDR tags +swift run lumectl play --duration 60 # headless run: events, heartbeat, startup, exit status ``` +`lumectl` reproduces a provider issue without the app: it drives a `PlayerSession` headlessly (muted) and prints what the engine reports — typed events, the `Diagnostics` heartbeat with the delivery rate, the `StartupTimeline`, `lastStallReason` on every stall, the `EngineError` (with its HTTP class) on failure. Exit status: 0 ok, 3 open failed, 4 playback failed, 5 stalled. URLs are redacted to scheme and host unless `--show-urls`, since IPTV URLs carry credentials — keep it that way. Its logic lives in `Sources/LumeCtlCore` (tested from `LumeCtlTests.swift`); `lumectl --help` lists the options. When a Lume report arrives, a `lumectl play` run against the same URL (with `--json` or `--record-events run.jsonl`) is usually the quickest way to tell an engine problem from a provider one. + Prerequisites: - `BinaryDependencies/FFmpeg.xcframework` must exist before anything compiles. Build it once (or after bumping `build/versions.json`): ```bash @@ -51,7 +55,7 @@ Prerequisites: build/scripts/build-ffmpeg.sh macos-arm64 # one slice is enough for local dev build/scripts/make-xcframework.sh ``` -- Tests need a host `ffmpeg` CLI (Homebrew) — fixtures are generated lazily on first test run by `Tests/LumeEngineCoreTests/Fixtures/generate-fixtures.sh` into `TestStreams/generated/` (gitignored, idempotent). `Fixtures.swift` finds Homebrew ffmpeg automatically; override with the `FFMPEG` env var. The closed-caption fixture (`captions.ts`) also needs `python3` (the one Xcode's command-line tools provide is enough), because no ffmpeg CLI option writes captions into the video; without it that fixture is skipped and its tests fail as missing. +- Tests need a host `ffmpeg` CLI (Homebrew) — fixtures are generated lazily on first test run by `Tests/LumeEngineCoreTests/Fixtures/generate-fixtures.sh` into `TestStreams/generated/` (gitignored, idempotent). `Fixtures.swift` finds Homebrew ffmpeg automatically; override with the `FFMPEG` env var. The closed-caption and bitmap-subtitle fixtures (`captions.ts`, `pgs.mkv`, `dvbsub.ts`, `dvdsub.mkv`) also need `python3` (the one Xcode's command-line tools provide is enough), because no ffmpeg CLI option writes captions into the video or renders subtitle bitmaps; without it those fixtures are skipped and their tests fail as missing. CI (`.github/workflows/ci.yml`) builds the macOS FFmpeg slice (cached on `build/versions.json` + scripts + patches hashes) and runs `swift test` with a strict one-thread cooperative pool (see above), checks the Markdown docs offline (`.github/scripts/check-docs.py`: relative links and in-repo anchors resolve, `CLAUDE.md` is still the `AGENTS.md` symlink — run it after editing docs), then cross-compiles the library for iOS and tvOS via `xcodebuild` (compile-only, one cached slice each — visionOS is left to local verification because that matrix entry was never confirmed on a runner — the xrOS SDK ships with Xcode, but a visionOS *destination* also needs Xcode's visionOS platform component); tagged releases build all 10 platform slices and attach the xcframework. @@ -89,6 +93,7 @@ This split is the core invariant of the codebase: - **Dolby Vision with an IPT base layer is converted, not tagged.** Profiles 5, 10.0 and 20 (compatibility ID 0) carry IPTPQc2, which shows pink/green as YCbCr. No colour attachment fixes that — the RPU reshaping changes per scene and Apple's decoders don't apply it to decoded frames — so `Decode/DolbyVisionConverter` turns each frame into HDR10 on the GPU in `VideoDecoder.deliver`, the one point every frame (hardware, software, deinterlaced) passes with its side data. Detection is the stream's DV configuration record *or* an `AVCOL_SPC_IPT_C2` frame; many P5 encodes leave the VUI matrix unspecified, so don't reduce it to the tag. HDR10-compatible profiles (8.1 etc.) must stay on the zero-copy path. GPU refusals are per-frame (iOS refuses a backgrounded app); only a converter that can't exist gives up for the session. - **Audio is fitted to the negotiated route width, never the route's capability.** `AudioDecoder` reads `AudioOutputRoute.outputChannels` (the session's `outputNumberOfChannels`, followed through route-change notifications) per frame and remixes per `AudioLayoutPolicy` with an explicit `DownmixMatrix` — a fold-down mixes centre and surrounds in, it never drops channels. Don't switch it to `maximumOutputNumberOfChannels`: a renderer fed more channels than the session outputs failed outright on tvOS. The engine only observes the audio session; setting its category, activation or preferred channel count is the host's job (Lume does it). When the route narrows, `SystemRenderer` keeps audio decoded for the wider route out of the renderer. - **How long a subtitle shows is the timeline's decision, not the decoder's.** Decoders hand `Subtitle/SubtitleStore` a `SubtitleEvent` (start, end *or none*, text, whether the track replaces the whole screen with each update); `Subtitle/SubtitleTimeline` decides the rest: an open cue ends at the next update of its track (capped at 16 s), an empty update on a closed-caption/DVB/PGS/DVD track is a clear that ends what is showing, a seek ends open cues, re-read ranges are deduplicated, and the store is bounded. Don't turn `end_display_time == UINT32_MAX` into a number in a decoder or drop empty updates there — both put stuck or stacked cues on screen. +- **Bitmap subtitles (DVB, PGS, DVD) are images on the cue, positioned by their own canvas.** `Subtitle/SubtitleBitmap` turns `SUBTITLE_BITMAP` rects into premultiplied sRGB `CGImage`s with a rect normalized to the canvas the decoder reports *after* decoding (DVB display definition, PGS presentation size, DVD `size:`), never to the video's pixel size — an HD DVB page over SD video, or PGS authored at 1080p over a 720p encode, would land off screen. The DVB context's size is zeroed before opening so the stream's display definition sets it. Bitmap cues carry empty `text` (text consumers like Lume show nothing rather than garbage); drawing them is the host's job, since `LumePlayerView` is bare video. The timeline bounds bitmap memory separately (`imageByteBudget`). - **Closed captions leave the video lane through a tap, never a wait.** CEA-608 in A/53 side data (`AV_FRAME_DATA_A53_CC`) is lifted off each decoded frame in `VideoDecoder.receiveFrames` (before deinterlacing) by `Subtitle/ClosedCaptionTap`, stamped with the frame's PTS, and `trySend`-ed to an ordinary `SubtitleDecoder` running `ccaption` in real-time mode — a slow caption lane drops captions, never frames. The track is synthetic (`TrackInfo.embeddedClosedCaptionsIndex`) and appears in `PlayerSession.mediaInfo` once caption content is seen; `.opened` keeps the open-time track list, because no new `PlayerEvent` case may be added for it. Only CEA-608 field 1 (CC1/CC2) is decoded: FFmpeg has no CEA-708 (DTVCC) decoder, and teletext needs libzvbi, which the build does not include. - **Zero global mutable state:** all configuration is `PlayerConfiguration` passed at session creation. No `static var` knobs. - **No force unwraps / `as!` / `fatalError` in engine paths**; failures return typed `EngineError`. No `assertionFailure` in production channel/data paths — misuse degrades, never crashes. diff --git a/PLAN.md b/PLAN.md index e8a5ccc..31a4e99 100644 --- a/PLAN.md +++ b/PLAN.md @@ -116,7 +116,8 @@ LumeEngine/ │ ├── LumeEngineRender/ # SystemRenderer, MetalRenderer, shaders │ ├── LumeEngineSubtitles/ # libass bridge, parsers, cue model, styling │ ├── LumeEngineMedia/ # Now Playing, PiP, audio session, route handling -│ └── LumeEngine/ # public facade: LumePlayer, LumePlayerView, config, events +│ ├── LumeEngine/ # public facade: LumePlayer, LumePlayerView, config, events +│ └── LumeCtlCore/, lumectl/ # macOS triage CLI: headless session against a URL, events + diagnostics ├── BinaryDependencies/ # generated Package targets for ffmpeg/libass/dav1d xcframeworks ├── build/ # FFmpeg build pipeline (scripts, patches, configs) ├── DemoApp/ # multiplatform SwiftUI demo (streams menu, diagnostics HUD) @@ -142,7 +143,7 @@ Brightness/contrast/saturation and MetalFX spatial upscaling operate CVPixelBuff **D6 — One subtitle engine, composited as overlay.** libass renders ASS/SSA (full effects, embedded fonts via attachment streams, word-by-word \k tags); PGS/DVB/VobSub decoded by FFmpeg to bitmaps; SRT/VTT parsed to styled cues rendered through the same layout box. Output = time-tagged `SubtitleImage` cues composited on a CALayer above the video surface (and into the PiP content when active; into Metal for HDR subtitle mode). Dual subtitle slots (main + secondary). External files attach as additional demux sessions. -**Cue lifecycle and closed captions (as built).** Every subtitle source — demuxed stream, sidecar file, or captions lifted from the video — reaches one time-indexed store as display updates; the store's `SubtitleTimeline` alone decides how long each shows (open cues end at the next update or a clear, capped; seeks end open cues; bounded size). CEA-608 captions carried as A/53 side data are tapped off decoded video frames without ever blocking the video lane and surface as a synthetic "CC1" subtitle track once seen, additively through `PlayerSession.mediaInfo` (no new event). +**Cue lifecycle and closed captions (as built).** Every subtitle source — demuxed stream, sidecar file, or captions lifted from the video — reaches one time-indexed store as display updates; the store's `SubtitleTimeline` alone decides how long each shows (open cues end at the next update or a clear, capped; seeks end open cues; bounded size). CEA-608 captions carried as A/53 side data are tapped off decoded video frames without ever blocking the video lane and surface as a synthetic "CC1" subtitle track once seen, additively through `PlayerSession.mediaInfo` (no new event). PGS/DVB/DVD bitmaps reach the same store as `SubtitleImage`s (premultiplied `CGImage` + rect normalized to the subtitle's authoring canvas) on cues with empty text; compositing them is still the host's job until the overlay layer described above exists. **System caption respect:** map `MACaptionAppearance` (user's Style settings) onto text-cue styling by default. **D7 — Buffering is PTS-math, not heuristics.** diff --git a/Package.swift b/Package.swift index a2a7966..392cfee 100644 --- a/Package.swift +++ b/Package.swift @@ -85,6 +85,9 @@ let package = Package( // namespace), so apps can also link engines embedding other FFmpeg builds // without symbol collisions. .library(name: "LumeEngine", type: .dynamic, targets: ["LumeEngine"]), + // macOS triage tool: `swift run lumectl --help`. Drives a session + // headlessly against a URL; nothing a consumer app links. + .executable(name: "lumectl", targets: ["lumectl"]), ], targets: [ // FFmpeg 9.0.x static libraries: released artifact, or a local build when @@ -116,11 +119,24 @@ let package = Package( path: "DemoApp/Sources" ), + // lumectl's logic (argument parsing, URL redaction, the headless run, + // output), kept out of the executable so the tests can reach it. + .target( + name: "LumeCtlCore", + dependencies: ["LumeEngineCore"] + ), + .executableTarget( + name: "lumectl", + dependencies: ["LumeCtlCore"] + ), + .testTarget( name: "LumeEngineCoreTests", // CFFmpeg directly: some tests build FFmpeg structs (e.g. Dolby // Vision metadata) to exercise the engine's readers of them. - dependencies: ["LumeEngineCore", "LumeEngine", "CFFmpeg"], + // LumeCtlCore and lumectl: the tool's tests reuse the fixtures and + // the torture server, and the Ctrl-C test runs the built binary. + dependencies: ["LumeEngineCore", "LumeEngine", "CFFmpeg", "LumeCtlCore", "lumectl"], exclude: ["Fixtures/generate-fixtures.sh"] ), ], diff --git a/README.md b/README.md index 849cb14..f536497 100644 --- a/README.md +++ b/README.md @@ -101,10 +101,27 @@ swift run LumeEngineDemo # macOS demo with diagnostics H Testing needs a host `ffmpeg` CLI (`brew install ffmpeg`) to generate fixtures on first run — a build-time tool, unrelated to the FFmpeg the engine links. The full 10-slice matrix (`build/scripts/platforms.sh`) is only needed for release binaries, which CI builds on tags. +## Debugging streams with lumectl + +`lumectl` (macOS) plays a URL headlessly through the engine and prints what it reports, so a stream problem — a slow start, stalls, a reconnect that replays media, timestamp seams, a hardware decoder falling back, an HTTP error — can be reproduced from a terminal without the app: + +```bash +swift run lumectl probe 'http://provider.example:8080/live/user/pass/1234.ts' +swift run lumectl play 'http://provider.example:8080/live/user/pass/1234.ts' --duration 60 +swift run lumectl play stream.m3u8 --json --record-events run.jsonl --interval 1 +``` + +`probe` prints the container, duration, start time, live/seekable flags and every track (codec, language, resolution, frame rate, bit depth, HDR transfer and colour tags). `play` runs a muted session and prints every `PlayerEvent` with a timestamp, a diagnostics heartbeat (queue depths, renderer health, delivery rate, timeline seams, replayed packets), the startup timeline once the first frame shows, the stalled stage on every stall, and the typed error — with its HTTP status class — on failure. `--json` switches stdout to JSON Lines; `--record-events FILE` keeps a JSON Lines copy next to the readable output. + +The exit status tells the outcome apart for scripted repro loops: `0` ok, `3` the source never opened, `4` playback failed, `5` stalled (without failing), `2` bad arguments. Ctrl-C ends a run cleanly with a summary. + +URLs are printed as scheme and host only (`http://provider.example:8080/[redacted].ts`), because IPTV URLs carry the account; header values are never printed. Pass `--show-urls` to see them in full. `lumectl --help` lists every option: start position, mid-stream seek, software decoding, preferred audio languages, a single lane, custom headers, user agent, referer, reconnect and timeouts. + ## Layout - `Sources/LumeEngineCore` — demux, decode, render, session, subtitles, media integration (no UI) - `Sources/LumeEngine` — public facade: `LumePlayer`, `LumePlayerView` +- `Sources/lumectl`, `Sources/LumeCtlCore` — the `lumectl` command-line tool - `build/` — FFmpeg build pipeline (versions manifest, scripts, patches) - `DemoApp/` — macOS demo with diagnostics HUD - `Tests/` — including a fault-injecting HTTP torture server diff --git a/Sources/LumeCtlCore/Arguments.swift b/Sources/LumeCtlCore/Arguments.swift new file mode 100644 index 0000000..44c99b4 --- /dev/null +++ b/Sources/LumeCtlCore/Arguments.swift @@ -0,0 +1,349 @@ +import Foundation +import LumeEngineCore + +/// What the command line asked for. Parsing is pure (`Arguments.parse`), +/// so every flag is unit-tested without opening a stream. +public enum Command: Equatable, Sendable { + case help + case version + case probe(ProbeOptions) + case play(PlayOptions) +} + +/// How to reach the source: everything `DemuxerOptions` exposes that a +/// provider issue tends to hinge on. +public struct SourceOptions: Equatable, Sendable { + public var url: String + /// In command-line order; a repeated name keeps the last value. + public var headers: [Header] = [] + public var userAgent: String? + public var referer: String? + public var reconnect = true + /// Seconds; nil keeps the engine default. + public var ioTimeout: Double? + /// Seconds; nil keeps the engine default. + public var openTimeout: Double? + + public struct Header: Equatable, Sendable { + public var name: String + public var value: String + } + + public init(url: String) { + self.url = url + } + + func apply(to options: inout DemuxerOptions) { + for header in headers { + options.httpHeaders[header.name] = header.value + } + if let userAgent { options.userAgent = userAgent } + if let referer { options.referer = referer } + options.enableReconnect = reconnect + if let ioTimeout { options.ioTimeout = Int64((ioTimeout * 1_000_000).rounded()) } + if let openTimeout { options.openTimeout = openTimeout } + } +} + +public struct OutputOptions: Equatable, Sendable { + /// JSON Lines on stdout instead of text. + public var json = false + /// Print URLs as given instead of redacted. + public var showURLs = false + /// Also write every record as JSON Lines to this file. + public var recordPath: String? + + public init() {} +} + +public struct ProbeOptions: Equatable, Sendable { + public var source: SourceOptions + public var output = OutputOptions() + + public init(source: SourceOptions) { + self.source = source + } + + public var configuration: PlayerConfiguration { + var configuration = PlayerConfiguration() + configuration.muted = true + source.apply(to: &configuration.demuxer) + return configuration + } +} + +public struct PlayOptions: Equatable, Sendable { + public var source: SourceOptions + public var output = OutputOptions() + /// Wall-clock seconds to play from `play()`; nil plays to the end (or + /// until Ctrl-C, for a live source). + public var duration: Double? + /// Seek here (seconds) once the first frame shows — a mid-stream seek, + /// as opposed to `start`. + public var seek: Double? + /// `PlayerConfiguration.startPosition`: the resume path, which seeks + /// before the first read. + public var start: Double? + public var software = false + public var audioLanguages: [String] = [] + public var noAudio = false + public var noVideo = false + /// Seconds between diagnostics heartbeats; 0 turns them off. + public var interval: Double = 2 + /// Seconds; nil keeps the engine default. + public var stallThreshold: Double? + + public init(source: SourceOptions) { + self.source = source + } + + public var configuration: PlayerConfiguration { + var configuration = PlayerConfiguration() + // Nothing should come out of the speakers while triaging; the audio + // lane still decodes and renders, so its timing is the real one. + configuration.muted = true + source.apply(to: &configuration.demuxer) + if let start { configuration.startPosition = start } + if software { configuration.hardwareDecode = .software } + configuration.preferredAudioLanguages = audioLanguages + configuration.enableAudio = !noAudio + configuration.enableVideo = !noVideo + if let stallThreshold { configuration.stallThreshold = stallThreshold } + return configuration + } +} + +public struct UsageError: Error, Equatable, CustomStringConvertible { + public let message: String + public var description: String { message } + + init(_ message: String) { + self.message = message + } +} + +public enum Arguments { + /// Parses the arguments after the executable name. + public static func parse(_ arguments: [String]) throws(UsageError) -> Command { + guard let first = arguments.first else { return .help } + // `lumectl play --help` asks for help, not for a URL. + let options = arguments.prefix(while: { $0 != "--" }) + if options.contains("-h") || options.contains("--help") { return .help } + switch first { + case "help": + return .help + case "--version", "version": + return .version + case "probe": + var reader = Reader(Array(arguments.dropFirst())) + return .probe(try parseProbe(&reader)) + case "play": + var reader = Reader(Array(arguments.dropFirst())) + return .play(try parsePlay(&reader)) + default: + throw UsageError("unknown command '\(first)' (expected probe or play)") + } + } + + private static func parseProbe(_ reader: inout Reader) throws(UsageError) -> ProbeOptions { + var source = SourceOptions(url: "") + var output = OutputOptions() + var url: String? + while let argument = try reader.next() { + switch argument { + case .positional(let value): + try setURL(&url, value) + case .option(let name, let inline): + if try parseSourceOption(name, inline, &source, &reader) { continue } + if try parseOutputOption(name, inline, &output, &reader) { continue } + throw UsageError("unknown option '--\(name)' for probe") + } + } + guard let url else { throw UsageError("probe needs a URL or file path") } + source.url = url + var options = ProbeOptions(source: source) + options.output = output + return options + } + + private static func parsePlay(_ reader: inout Reader) throws(UsageError) -> PlayOptions { + var options = PlayOptions(source: SourceOptions(url: "")) + var url: String? + while let argument = try reader.next() { + switch argument { + case .positional(let value): + try setURL(&url, value) + case .option(let name, let inline): + if try parseSourceOption(name, inline, &options.source, &reader) { continue } + if try parseOutputOption(name, inline, &options.output, &reader) { continue } + switch name { + case "duration": + options.duration = try seconds(name, reader.value(for: name, inline), allowZero: false) + case "seek": + options.seek = try seconds(name, reader.value(for: name, inline)) + case "start": + options.start = try seconds(name, reader.value(for: name, inline)) + case "interval": + options.interval = try seconds(name, reader.value(for: name, inline)) + case "stall-threshold": + options.stallThreshold = try seconds(name, reader.value(for: name, inline), allowZero: false) + case "audio-lang": + options.audioLanguages = try languages(reader.value(for: name, inline)) + case "software": + try noValue(name, inline) + options.software = true + case "no-audio": + try noValue(name, inline) + options.noAudio = true + case "no-video": + try noValue(name, inline) + options.noVideo = true + default: + throw UsageError("unknown option '--\(name)' for play") + } + } + } + guard let url else { throw UsageError("play needs a URL or file path") } + guard !(options.noAudio && options.noVideo) else { + throw UsageError("--no-audio and --no-video together leave nothing to play") + } + options.source.url = url + return options + } + + /// Options both commands share. Returns false when `name` is not one. + private static func parseSourceOption( + _ name: String, _ inline: String?, _ source: inout SourceOptions, _ reader: inout Reader + ) throws(UsageError) -> Bool { + switch name { + case "header": + source.headers.append(try header(reader.value(for: name, inline))) + case "user-agent": + source.userAgent = try nonEmpty(name, reader.value(for: name, inline)) + case "referer": + source.referer = try nonEmpty(name, reader.value(for: name, inline)) + case "no-reconnect": + try noValue(name, inline) + source.reconnect = false + case "io-timeout": + source.ioTimeout = try seconds(name, reader.value(for: name, inline), allowZero: false) + case "open-timeout": + source.openTimeout = try seconds(name, reader.value(for: name, inline), allowZero: false) + default: + return false + } + return true + } + + private static func parseOutputOption( + _ name: String, _ inline: String?, _ output: inout OutputOptions, _ reader: inout Reader + ) throws(UsageError) -> Bool { + switch name { + case "json": + try noValue(name, inline) + output.json = true + case "show-urls": + try noValue(name, inline) + output.showURLs = true + case "record-events": + output.recordPath = try nonEmpty(name, reader.value(for: name, inline)) + default: + return false + } + return true + } + + private static func setURL(_ url: inout String?, _ value: String) throws(UsageError) { + guard url == nil else { throw UsageError("unexpected extra argument '\(value)'") } + guard !value.isEmpty else { throw UsageError("the URL is empty") } + url = value + } + + private static func seconds(_ name: String, _ text: String, allowZero: Bool = true) throws(UsageError) -> Double { + guard let value = Double(text), value.isFinite, value >= 0, allowZero || value > 0 else { + let expected = allowZero ? "a number of seconds ≥ 0" : "a number of seconds > 0" + throw UsageError("--\(name) expects \(expected), got '\(text)'") + } + return value + } + + private static func header(_ text: String) throws(UsageError) -> SourceOptions.Header { + guard let colon = text.firstIndex(of: ":") else { + throw UsageError("--header expects 'Name: value', got '\(text)'") + } + let name = text[.. [String] { + let tags = text.split(separator: ",", omittingEmptySubsequences: false).map { $0.trimmingCharacters(in: .whitespaces).lowercased() } + guard !tags.isEmpty, tags.allSatisfy({ !$0.isEmpty }) else { + throw UsageError("--audio-lang expects a comma-separated list such as 'de,en', got '\(text)'") + } + return tags + } + + private static func nonEmpty(_ name: String, _ text: String) throws(UsageError) -> String { + guard !text.isEmpty else { throw UsageError("--\(name) expects a value") } + return text + } + + private static func noValue(_ name: String, _ inline: String?) throws(UsageError) { + if inline != nil { throw UsageError("--\(name) takes no value") } + } +} + +/// Splits arguments into positionals and `--name` / `--name=value` options; +/// everything after `--` is positional (a URL that starts with a dash). +private struct Reader { + enum Argument { + case positional(String) + case option(name: String, inline: String?) + } + + private var arguments: [String] + private var index = 0 + private var positionalOnly = false + + init(_ arguments: [String]) { + self.arguments = arguments + } + + mutating func next() throws(UsageError) -> Argument? { + guard index < arguments.count else { return nil } + let argument = arguments[index] + index += 1 + if positionalOnly { return .positional(argument) } + if argument == "--" { + positionalOnly = true + return try next() + } + guard argument.hasPrefix("--"), argument.count > 2 else { + if argument.hasPrefix("-"), argument.count > 1 { + throw UsageError("unknown option '\(argument)' (options are spelled --name)") + } + return .positional(argument) + } + let body = argument.dropFirst(2) + if let equals = body.firstIndex(of: "=") { + return .option(name: String(body[.. String { + if let inline { return inline } + guard index < arguments.count else { throw UsageError("--\(name) expects a value") } + let value = arguments[index] + index += 1 + return value + } +} diff --git a/Sources/LumeCtlCore/Describe.swift b/Sources/LumeCtlCore/Describe.swift new file mode 100644 index 0000000..207c7b4 --- /dev/null +++ b/Sources/LumeCtlCore/Describe.swift @@ -0,0 +1,267 @@ +import Foundation +import LumeEngineCore + +/// Text and JSON renderings of the engine's public types. Anything that can +/// carry a URL goes through `redact` first. +struct Describe { + /// Redacts URLs inside free text unless the user asked to see them. + let showURLs: Bool + + func url(_ url: String) -> String { + showURLs ? url : URLScrubber.redact(url) + } + + func text(_ text: String) -> String { + showURLs ? text : URLScrubber.redactText(text) + } + + // MARK: Media + + func media(_ info: MediaInfo) -> (text: String, json: JSON) { + var lines: [String] = [] + let duration = info.duration.map { String(format: "%.3f s", seconds($0)) } ?? "unknown (live)" + lines.append("format \(info.formatName)") + lines.append("duration \(duration)") + lines.append(String(format: "start %.3f s", seconds(info.startTime))) + lines.append("live \(info.isLive ? "yes" : "no") seekable \(info.isSeekable ? "yes" : "no")") + if info.bitrate >= 1_000_000 { + lines.append(String(format: "bitrate %.2f Mbit/s", Double(info.bitrate) / 1_000_000)) + } else if info.bitrate >= 1000 { + lines.append(String(format: "bitrate %.0f kbit/s", Double(info.bitrate) / 1000)) + } else if info.bitrate > 0 { + lines.append("bitrate \(info.bitrate) bit/s") + } + lines.append("tracks") + for track in info.tracks { + lines.append(" " + trackLine(track)) + } + if !info.chapters.isEmpty { + lines.append("chapters") + for chapter in info.chapters { + lines.append(String( + format: " %.3f–%.3f s %@", + seconds(chapter.start), seconds(chapter.end), text(chapter.title ?? "") + )) + } + } + if !info.metadata.isEmpty { + lines.append("metadata") + for key in info.metadata.keys.sorted() { + lines.append(" \(key): \(text(info.metadata[key] ?? ""))") + } + } + + let json = JSON.object([ + ("format", .string(info.formatName)), + ("duration", .optional(info.duration.map(seconds))), + ("startTime", .double(seconds(info.startTime))), + ("live", .bool(info.isLive)), + ("seekable", .bool(info.isSeekable)), + ("bitrate", .int(info.bitrate)), + ("tracks", .array(info.tracks.map(trackJSON))), + ("chapters", .array(info.chapters.map { chapter in + .object([ + ("title", .optional(chapter.title.map(text))), + ("start", .double(seconds(chapter.start))), + ("end", .double(seconds(chapter.end))), + ]) + })), + ("metadata", .object(info.metadata.keys.sorted().map { ($0, .string(text(info.metadata[$0] ?? ""))) })), + ]) + return (lines.joined(separator: "\n"), json) + } + + private func trackLine(_ track: TrackInfo) -> String { + var parts = ["#\(track.index)", track.kind.rawValue, track.codecName] + if let language = track.language { parts.append(language) } + if let video = track.video { + parts.append("\(video.width)x\(video.height)") + if video.fps > 0 { parts.append(String(format: "%.6g fps", video.fps)) } + if video.bitDepth > 0 { parts.append("\(video.bitDepth)-bit") } + if let pixelFormat = video.pixelFormatName { parts.append(pixelFormat) } + parts.append(colour(video)) + if video.rotation != 0 { parts.append("rotated \(video.rotation)°") } + } + if let audio = track.audio { + parts.append("\(audio.channels) ch") + parts.append("\(audio.sampleRate) Hz") + } + if track.bitrate > 0 { + parts.append(String(format: "%.0f kbit/s", Double(track.bitrate) / 1000)) + } + if let title = track.title, !title.isEmpty { parts.append("\"\(text(title))\"") } + if track.isDefault { parts.append("[default]") } + if track.isForced { parts.append("[forced]") } + return parts.joined(separator: " ") + } + + /// `HDR PQ bt2020/bt2020nc full` — enough to tell HDR10 from HLG from + /// SDR, and to spot an IPT base layer (Dolby Vision profile 5) that the + /// matrix tag gives away. + private func colour(_ video: TrackInfo.Video) -> String { + let range = video.colorRangeFull ? "full" : "limited" + let unspecified: UInt32 = 2 + if video.colorPrimaries == unspecified, video.colorTransfer == unspecified, video.colorSpace == unspecified { + return "\(video.isHDR ? "HDR" : "SDR") untagged \(range)" + } + let dynamicRange: String + switch ColourNames.transfer(video.colorTransfer) { + case "pq": dynamicRange = "HDR PQ" + case "hlg": dynamicRange = "HDR HLG" + default: dynamicRange = video.isHDR ? "HDR" : "SDR" + } + return "\(dynamicRange) \(ColourNames.primaries(video.colorPrimaries))/\(ColourNames.matrix(video.colorSpace))" + + "/\(ColourNames.transfer(video.colorTransfer)) \(range)" + } + + private func trackJSON(_ track: TrackInfo) -> JSON { + var members: [(String, JSON)] = [ + ("index", .int(Int64(track.index))), + ("kind", .string(track.kind.rawValue)), + ("codec", .string(track.codecName)), + ("language", .optional(track.language)), + ("title", .optional(track.title.map(text))), + ("default", .bool(track.isDefault)), + ("forced", .bool(track.isForced)), + ("bitrate", .int(track.bitrate)), + ] + if let video = track.video { + members.append(("video", .object([ + ("width", .int(Int64(video.width))), + ("height", .int(Int64(video.height))), + ("fps", .double(video.fps)), + ("bitDepth", .int(Int64(video.bitDepth))), + ("pixelFormat", .optional(video.pixelFormatName)), + ("rotation", .int(Int64(video.rotation))), + ("hdr", .bool(video.isHDR)), + ("primaries", .string(ColourNames.primaries(video.colorPrimaries))), + ("transfer", .string(ColourNames.transfer(video.colorTransfer))), + ("matrix", .string(ColourNames.matrix(video.colorSpace))), + ("fullRange", .bool(video.colorRangeFull)), + ]))) + } + if let audio = track.audio { + members.append(("audio", .object([ + ("channels", .int(Int64(audio.channels))), + ("sampleRate", .int(Int64(audio.sampleRate))), + ]))) + } + return .object(members) + } + + // MARK: Errors + + func error(_ error: EngineError) -> (text: String, json: JSON) { + var detail: [String] = [] + if let ffmpegCode = error.ffmpegCode { detail.append("ffmpeg \(ffmpegCode)") } + if let http = error.httpFailure { detail.append("http \(http.rawValue)") } + let suffix = detail.isEmpty ? "" : " (\(detail.joined(separator: ", ")))" + let message = text(error.message) + let json = JSON.object([ + ("code", .string(error.code.rawValue)), + ("ffmpegCode", error.ffmpegCode.map { .int(Int64($0)) } ?? .null), + ("http", .optional(error.httpFailure?.rawValue)), + ("message", .string(message)), + ]) + return ("\(error.code.rawValue)\(suffix): \(message)", json) + } + + // MARK: Session state + + func startup(_ startup: StartupTimeline) -> JSON { + .object([ + ("opened", .optional(startup.opened)), + ("firstPacket", .optional(startup.firstPacket)), + ("firstVideoFrame", .optional(startup.firstVideoFrame)), + ("firstAudioFrame", .optional(startup.firstAudioFrame)), + ("firstFrameShown", .optional(startup.firstFrameShown)), + ]) + } + + func diagnostics(_ diagnostics: PlayerSession.Diagnostics, inputMbps: Double?) -> (text: String, json: JSON) { + func lane(_ queue: (count: Int, seconds: Double)?) -> JSON { + guard let queue else { return .null } + return .object([("count", .int(Int64(queue.count))), ("seconds", .double(queue.seconds))]) + } + let health = diagnostics.rendererHealth + var line = text(diagnostics.description) + if let inputMbps { line += String(format: " in=%.2fMbit/s", inputMbps) } + let json = JSON.object([ + ("state", .string(diagnostics.state)), + ("playhead", .double(diagnostics.playheadSeconds)), + ("videoFrames", lane(diagnostics.videoQueue)), + ("videoPackets", lane(diagnostics.videoPacketQueue)), + ("videoLead", .double(diagnostics.videoLeadSeconds)), + ("audioFrames", lane(diagnostics.audioQueue)), + ("audioPackets", lane(diagnostics.audioPacketQueue)), + ("audioLead", .double(diagnostics.audioLeadSeconds)), + ("renderers", .object([ + ("video", .int(Int64(health.videoStatus))), + ("audio", .int(Int64(health.audioStatus))), + ("videoError", .optional(health.videoError.map(text))), + ("audioError", .optional(health.audioError.map(text))), + ])), + ("demuxEOF", .bool(diagnostics.demuxAtEOF)), + ("sourceLost", .bool(diagnostics.sourceLost)), + ("deliveredBytes", .int(diagnostics.deliveredBytes)), + ("inputMbps", .optional(inputMbps)), + ("timelineSeams", .int(Int64(diagnostics.timelineSeams))), + ("replayedPackets", .int(Int64(diagnostics.replayedPackets))), + ("lastStallReason", .optional(diagnostics.lastStallReason?.rawValue)), + ("startup", startup(diagnostics.startup)), + ]) + return (line, json) + } + + private func seconds(_ microseconds: Int64) -> Double { + Double(microseconds) / 1_000_000 + } +} + +/// Names for the ITU-T H.273 code points `TrackInfo.Video` carries raw. +enum ColourNames { + static func primaries(_ value: UInt32) -> String { + switch value { + case 1: "bt709" + case 2: "unspecified" + case 5: "bt470bg" + case 6: "smpte170m" + case 9: "bt2020" + case 11: "dci-p3" + case 12: "display-p3" + default: "pri\(value)" + } + } + + static func transfer(_ value: UInt32) -> String { + switch value { + case 1: "bt709" + case 2: "unspecified" + case 6: "smpte170m" + case 8: "linear" + case 13: "srgb" + case 14: "bt2020-10" + case 15: "bt2020-12" + case 16: "pq" + case 18: "hlg" + default: "trc\(value)" + } + } + + static func matrix(_ value: UInt32) -> String { + switch value { + case 0: "rgb" + case 1: "bt709" + case 2: "unspecified" + case 5: "bt470bg" + case 6: "smpte170m" + case 9: "bt2020nc" + case 10: "bt2020c" + case 14: "ictcp" + // Dolby Vision's IPT base layer (profiles 5, 10.0, 20) when the + // stream tags it; many such encodes leave the matrix unspecified. + case 15: "ipt-c2" + default: "spc\(value)" + } + } +} diff --git a/Sources/LumeCtlCore/ExitCode.swift b/Sources/LumeCtlCore/ExitCode.swift new file mode 100644 index 0000000..d87da64 --- /dev/null +++ b/Sources/LumeCtlCore/ExitCode.swift @@ -0,0 +1,41 @@ +/// lumectl's exit status: scripts reproducing a provider issue in a loop +/// branch on these, so each failure class gets its own code. +public enum ExitCode: Int32, Sendable { + /// Played for the requested time, to the end, or until Ctrl-C, with no + /// stall and no failure. Probe: the source opened. + case success = 0 + /// lumectl itself could not run (e.g. the record file). + case toolFailure = 1 + /// Bad command line. + case usage = 2 + /// The source never opened (DNS, connect, HTTP status, probe, timeout). + case openFailed = 3 + /// The session opened and then failed. + case playbackFailed = 4 + /// The session stalled at least once and did not fail. + case stalled = 5 + /// Ctrl-C before the source opened: no verdict to report. + case interrupted = 130 + + /// The verdict of a playback that got past `open`: a failure outranks a + /// stall (the stall usually preceded it), and Ctrl-C is how a live run + /// ends, so it is not a verdict of its own. + static func playback(failed: Bool, stalls: Int) -> ExitCode { + if failed { return .playbackFailed } + if stalls > 0 { return .stalled } + return .success + } + + /// Short name used in the summary record. + var outcome: String { + switch self { + case .success: "ok" + case .toolFailure: "tool-failure" + case .usage: "usage" + case .openFailed: "open-failed" + case .playbackFailed: "playback-failed" + case .stalled: "stalled" + case .interrupted: "interrupted" + } + } +} diff --git a/Sources/LumeCtlCore/JSON.swift b/Sources/LumeCtlCore/JSON.swift new file mode 100644 index 0000000..b1f6682 --- /dev/null +++ b/Sources/LumeCtlCore/JSON.swift @@ -0,0 +1,87 @@ +import Foundation + +/// A JSON value with ordered object keys. +/// +/// lumectl's JSON Lines are read by people as often as by `jq`, and a record +/// that starts with `t` and `type` scans far better than one sorted +/// alphabetically — which is all `JSONSerialization` offers. +public enum JSON: Equatable, Sendable { + case null + case bool(Bool) + case int(Int64) + case double(Double) + case string(String) + case array([JSON]) + case object([(String, JSON)]) + + public static func == (lhs: JSON, rhs: JSON) -> Bool { + lhs.encoded == rhs.encoded + } + + /// Compact single-line encoding (one record per line). + public var encoded: String { + var output = "" + write(to: &output) + return output + } + + private func write(to output: inout String) { + switch self { + case .null: + output += "null" + case .bool(let value): + output += value ? "true" : "false" + case .int(let value): + output += String(value) + case .double(let value): + // JSON has no NaN or infinity; a missing measurement is null. + if value.isFinite { + output += String(format: "%.3f", value) + } else { + output += "null" + } + case .string(let value): + Self.writeString(value, to: &output) + case .array(let values): + output += "[" + for (offset, value) in values.enumerated() { + if offset > 0 { output += "," } + value.write(to: &output) + } + output += "]" + case .object(let members): + output += "{" + for (offset, member) in members.enumerated() { + if offset > 0 { output += "," } + Self.writeString(member.0, to: &output) + output += ":" + member.1.write(to: &output) + } + output += "}" + } + } + + private static func writeString(_ value: String, to output: inout String) { + output += "\"" + for scalar in value.unicodeScalars { + switch scalar { + case "\"": output += "\\\"" + case "\\": output += "\\\\" + case "\n": output += "\\n" + case "\r": output += "\\r" + case "\t": output += "\\t" + case _ where scalar.value < 0x20: + output += String(format: "\\u%04x", scalar.value) + default: + output.unicodeScalars.append(scalar) + } + } + output += "\"" + } +} + +extension JSON { + /// `nil` becomes `null`, so optional fields keep their key. + static func optional(_ value: Double?) -> JSON { value.map(JSON.double) ?? .null } + static func optional(_ value: String?) -> JSON { value.map(JSON.string) ?? .null } +} diff --git a/Sources/LumeCtlCore/LumeCtl.swift b/Sources/LumeCtlCore/LumeCtl.swift new file mode 100644 index 0000000..4db49df --- /dev/null +++ b/Sources/LumeCtlCore/LumeCtl.swift @@ -0,0 +1,411 @@ +import Foundation +import LumeEngineCore +import Synchronization + +/// lumectl: drives a `PlayerSession` headlessly against one URL, so a +/// provider issue (startup stall, reconnect, replay, seam, decoder downgrade, +/// HTTP error) can be reproduced from a terminal instead of the app. +/// +/// It only observes. Like the app, it uses the session's public surface — +/// events, `diagnostics`, `startup`, `lastStallReason` — and never retries: +/// what the engine reports is what gets printed. +public enum LumeCtl { + public static let usage = """ + usage: lumectl probe [source options] [output options] + lumectl play [play options] [source options] [output options] + + probe open the source, print what it carries, close it + play play it headlessly (muted) and print every event, a diagnostics + heartbeat, the startup timeline and the outcome + + play options: + --duration S stop after S seconds of playback (default: to the + end, or Ctrl-C for a live source) + --start S open at S seconds (the resume path: seeks before + the first read) + --seek S seek to S seconds once the first frame shows + --interval S heartbeat every S seconds, 0 for none (default 2) + --software decode video in software (no VideoToolbox) + --audio-lang de,en preferred audio languages, in order + --no-audio video lane only + --no-video audio lane only + --stall-threshold S seconds without progress before a stall + + source options: + --header 'Name: value' extra HTTP header (repeatable) + --user-agent UA + --referer URL + --no-reconnect disable FFmpeg's HTTP auto-reconnect + --io-timeout S blocking I/O timeout (default 15) + --open-timeout S deadline for open + probe (default 30) + + output options: + --json JSON Lines on stdout instead of text + --record-events FILE also write every record to FILE as JSON Lines + --show-urls print URLs as given (default: scheme and host + only, since IPTV URLs carry credentials) + + exit status: + 0 ok 1 lumectl failed 2 usage 3 open failed + 4 playback failed 5 stalled (and did not fail) 130 Ctrl-C before open + + Ctrl-C ends a run cleanly (the session is shut down and the summary + printed); a second Ctrl-C exits at once. + """ + + /// Runs a parsed command to completion. + /// + /// - Parameter interrupts: yields once per Ctrl-C. The first one ends the + /// run the way `--duration` would; handling a second is the caller's. + public static func run( + _ command: Command, + interrupts: AsyncStream, + stdout: @escaping Reporter.Sink, + stderr: @escaping Reporter.Sink + ) async -> ExitCode { + switch command { + case .help: + stdout(usage) + return .success + case .version: + stdout("lumectl (LumeEngine) — \(FFmpegRuntime.versions)") + return .success + case .probe(let options): + guard let reporter = makeReporter(options.output, stdout: stdout, stderr: stderr) else { + return .toolFailure + } + defer { reporter.close() } + return await Run( + source: options.source, output: options.output, + configuration: options.configuration, reporter: reporter + ).probe(interrupts: interrupts) + case .play(let options): + guard let reporter = makeReporter(options.output, stdout: stdout, stderr: stderr) else { + return .toolFailure + } + defer { reporter.close() } + return await Run( + source: options.source, output: options.output, + configuration: options.configuration, reporter: reporter + ).play(options, interrupts: interrupts) + } + } + + private static func makeReporter( + _ output: OutputOptions, stdout: @escaping Reporter.Sink, stderr: Reporter.Sink + ) -> Reporter? { + do { + return try Reporter(output: output, stdout: stdout) + } catch { + stderr("lumectl: \(error)") + return nil + } + } +} + +/// One probe or play run. +private struct Run { + let source: SourceOptions + let output: OutputOptions + let configuration: PlayerConfiguration + let reporter: Reporter + + private var describe: Describe { Describe(showURLs: output.showURLs) } + + // MARK: probe + + func probe(interrupts: AsyncStream) async -> ExitCode { + reportStart("probe", extra: []) + let session = PlayerSession(configuration: configuration) + let control = RunControl() + let forwarder = forward(interrupts, to: control) + defer { forwarder.cancel() } + + let result = await open(session, control: control) + control.stop(.ended) + await shutdown(session) + switch result { + case .opened(let info): + let media = describe.media(info) + reporter.emit("media", text: media.text, fields: [("media", media.json)]) + return summarize(.success, control: control, position: nil) + case .failed(let error): + // Probe does not follow the event stream; tally the failure here. + control.setSessionState(.failed) + _ = control.recordError(error) + let described = describe.error(error) + reporter.emit("error", text: "open failed: \(described.text)", fields: [ + ("fatal", .bool(true)), ("error", described.json), + ]) + return summarize(.openFailed, control: control, position: nil) + case .interrupted: + return summarize(.interrupted, control: control, position: nil) + } + } + + // MARK: play + + func play(_ options: PlayOptions, interrupts: AsyncStream) async -> ExitCode { + reportStart("play", extra: [ + ("duration", .optional(options.duration)), + ("start", .optional(options.start)), + ("seek", .optional(options.seek)), + ("software", .bool(options.software)), + ("audioLanguages", .array(options.audioLanguages.map(JSON.string))), + ("audio", .bool(!options.noAudio)), + ("video", .bool(!options.noVideo)), + ("interval", .double(options.interval)), + ("stallThreshold", .double(configuration.stallThreshold)), + ]) + let session = PlayerSession(configuration: configuration) + let control = RunControl() + let forwarder = forward(interrupts, to: control) + defer { forwarder.cancel() } + + // Subscribed before `open`, so the opening state changes print too. + let events = Task { await consumeEvents(of: session, control: control) } + + switch await open(session, control: control) { + case .opened: + break + case .failed: + // The session reported the failure on its event stream already. + control.stop(.failed) + await shutdown(session, draining: events) + return summarize(.openFailed, control: control, position: nil) + case .interrupted: + await shutdown(session, draining: events) + return summarize(.interrupted, control: control, position: nil) + } + + await session.play() + var helpers: [Task] = [] + if let duration = options.duration { + helpers.append(Task { + try? await Task.sleep(for: .seconds(duration)) + control.stop(.duration) + }) + } + if options.interval > 0 { + helpers.append(Task { await heartbeat(session, every: options.interval) }) + } + helpers.append(Task { await watchStartup(session, thenSeekTo: options.seek) }) + + let reason = await control.waitForStop() + for helper in helpers { helper.cancel() } + if reason == .interrupted { + reporter.emit("note", text: "interrupted, shutting down", fields: [("note", .string("interrupted"))]) + } + let diagnostics = await session.diagnostics + let position = await session.position + let line = describe.diagnostics(diagnostics, inputMbps: nil) + reporter.emit("diagnostics", text: "final \(line.text)", fields: [("diagnostics", line.json)]) + await shutdown(session, draining: events) + + let tally = control.snapshot + return summarize( + .playback(failed: tally.failure != nil, stalls: tally.stalls.count), + control: control, position: position + ) + } + + // MARK: Steps + + private enum OpenResult { + case opened(MediaInfo) + case failed(EngineError) + case interrupted + } + + /// `open`, abandoned on Ctrl-C: cancelling the opening task aborts the + /// FFmpeg I/O, which can otherwise sit out the whole open timeout. + private func open(_ session: PlayerSession, control: RunControl) async -> OpenResult { + let url = source.url + let opening = Task { try await session.open(url: url) } + // Waits for the run's stop, which every run reaches; by then the + // open has long finished unless it was the Ctrl-C that stopped it. + Task { + _ = await control.waitForStop() + opening.cancel() + } + do { + return .opened(try await opening.value) + } catch { + if control.stopReason == .interrupted { return .interrupted } + let engineError = (error as? EngineError) ?? EngineError(code: .openFailed, message: "\(error)") + return .failed(engineError) + } + } + + /// Prints events as they come and turns the terminal ones into a stop. + private func consumeEvents(of session: PlayerSession, control: RunControl) async { + for await event in session.events { + switch event { + case .stateChanged(let state): + control.setSessionState(state) + reporter.emit("state", text: "state \(state)", fields: [("state", .string("\(state)"))]) + if state == .ended { control.stop(.ended) } + + case .opened(let info): + // Printed from here rather than where `open` returns, so the + // records keep the order the session reported them in. + let media = describe.media(info) + reporter.emit("opened", text: "opened\n\(media.text)", fields: [("media", media.json)]) + await reportSelection(session) + + case .decoderDowngraded(let error): + control.countDowngrade() + let described = describe.error(error) + reporter.emit( + "decoderDowngraded", text: "decoder downgraded to software: \(described.text)", + fields: [("error", described.json)] + ) + + case .didSeek(let position): + reporter.emit( + "didSeek", text: String(format: "seeked to %.3f s", position), + fields: [("position", .double(position))] + ) + + case .stalled(let position): + let reason = await session.lastStallReason + control.countStall(reason) + reporter.emit( + "stalled", + text: String(format: "STALLED at %.3f s, stage: %@", position, reason?.rawValue ?? "unknown"), + fields: [("position", .double(position)), ("reason", .optional(reason?.rawValue))] + ) + + case .error(let error): + // A session that fails switches to `.failed` before it + // reports why; any other error left the session running. + let fatal = control.recordError(error) + let described = describe.error(error) + reporter.emit( + "error", text: "\(fatal ? "FAILED" : "error") \(described.text)", + fields: [("fatal", .bool(fatal)), ("error", described.json)] + ) + if fatal { control.stop(.failed) } + } + } + } + + private func heartbeat(_ session: PlayerSession, every interval: Double) async { + // Seeded here, so even the first heartbeat has a delivery rate. + var previous = (bytes: await session.diagnostics.deliveredBytes, at: reporter.elapsed) + while !Task.isCancelled { + try? await Task.sleep(for: .seconds(interval)) + if Task.isCancelled { return } + let diagnostics = await session.diagnostics + let now = reporter.elapsed + // What the source delivered since the last beat: the number to + // hold against a provider's per-connection cap. + let mbps = now > previous.at + ? Double(diagnostics.deliveredBytes - previous.bytes) * 8 / 1_000_000 / (now - previous.at) + : nil + previous = (diagnostics.deliveredBytes, now) + let line = describe.diagnostics(diagnostics, inputMbps: mbps) + reporter.emit("heartbeat", text: "heartbeat \(line.text)", fields: [("diagnostics", line.json)]) + } + } + + /// Prints the startup timeline once the first frame shows, then makes + /// the `--seek` (a seek that lands mid-stream, which is the case that + /// differs from `--start`). + private func watchStartup(_ session: PlayerSession, thenSeekTo seek: Double?) async { + while !Task.isCancelled { + let startup = await session.startup + if startup.firstFrameShown != nil { + reporter.emit("startup", text: "startup \(startup)", fields: [("startup", describe.startup(startup))]) + if let seek { + reporter.emit( + "seek", text: String(format: "seeking to %.3f s", seek), + fields: [("position", .double(seek))] + ) + await session.seek(to: seek) + } + return + } + try? await Task.sleep(for: .milliseconds(50)) + } + } + + private func reportSelection(_ session: PlayerSession) async { + let video = await session.selectedVideoTrackIndex + let audio = await session.selectedAudioTrackIndex + let subtitle = await session.selectedSubtitleTrackIndex + func name(_ index: Int32?) -> String { index.map { "#\($0)" } ?? "none" } + func json(_ index: Int32?) -> JSON { index.map { .int(Int64($0)) } ?? .null } + reporter.emit( + "selected", + text: "selected video \(name(video)), audio \(name(audio)), subtitles \(name(subtitle))", + fields: [("video", json(video)), ("audio", json(audio)), ("subtitle", json(subtitle))] + ) + } + + /// Shuts the session down, bounded: a wedged teardown must not keep the + /// terminal hostage. The event stream ends with the session, so the + /// events still queued are printed before the summary. + private func shutdown(_ session: PlayerSession, draining events: Task? = nil) async { + let finished = await finishes(within: 5) { await session.shutdown() } + if !finished { + reporter.emit( + "note", text: "session shutdown did not finish within 5 s", + fields: [("note", .string("shutdown-timeout"))] + ) + } + if let events { + _ = await finishes(within: 2) { await events.value } + events.cancel() + } + } + + private func reportStart(_ command: String, extra: [(String, JSON)]) { + let headerNames = source.headers.map(\.name) + var text = "lumectl \(command) \(describe.url(source.url))" + if !headerNames.isEmpty { + text += "\n headers: \(headerNames.joined(separator: ", ")) (values hidden)" + } + reporter.emit("start", text: text, fields: [ + ("command", .string(command)), + ("url", .string(describe.url(source.url))), + // Header values are where tokens and cookies travel. + ("headers", .array(headerNames.map(JSON.string))), + ("userAgent", .optional(source.userAgent)), + ("referer", .optional(source.referer.map(describe.url))), + ("reconnect", .bool(source.reconnect)), + ("ioTimeout", .optional(source.ioTimeout)), + ("openTimeout", .optional(source.openTimeout)), + ("ffmpeg", .string("\(FFmpegRuntime.versions)")), + ] + extra) + } + + private func summarize(_ code: ExitCode, control: RunControl, position: Double?) -> ExitCode { + let tally = control.snapshot + let stallReasons = tally.stalls.map { $0?.rawValue ?? "unknown" } + var text = "summary \(code.outcome) (exit \(code.rawValue))" + if let position { text += String(format: ", position %.3f s", position) } + text += ", stalls \(tally.stalls.count)" + if !stallReasons.isEmpty { text += " [\(stallReasons.joined(separator: ", "))]" } + text += ", errors \(tally.errors), downgrades \(tally.downgrades)" + if let failure = tally.failure { text += "\n failure: \(describe.error(failure).text)" } + reporter.emit("summary", text: text, fields: [ + ("outcome", .string(code.outcome)), + ("exitCode", .int(Int64(code.rawValue))), + ("position", .optional(position)), + ("stalls", .array(stallReasons.map(JSON.string))), + ("errors", .int(Int64(tally.errors))), + ("downgrades", .int(Int64(tally.downgrades))), + ("failure", tally.failure.map { describe.error($0).json } ?? .null), + ]) + return code + } + + private func forward(_ interrupts: AsyncStream, to control: RunControl) -> Task { + Task { + for await _ in interrupts { + control.stop(.interrupted) + } + } + } +} diff --git a/Sources/LumeCtlCore/Reporter.swift b/Sources/LumeCtlCore/Reporter.swift new file mode 100644 index 0000000..3084c32 --- /dev/null +++ b/Sources/LumeCtlCore/Reporter.swift @@ -0,0 +1,66 @@ +import Foundation +import Synchronization + +/// Where lumectl's records go: text or JSON Lines on stdout, and optionally +/// JSON Lines into a file (`--record-events`), so a run can be watched live +/// and attached to an issue afterwards. +/// +/// Every record carries `t`, seconds since the run started — the one clock +/// that lines up events, heartbeats and the startup timeline. +public final class Reporter: Sendable { + /// Receives one finished line (no trailing newline). + public typealias Sink = @Sendable (String) -> Void + + private let json: Bool + private let started: TimeInterval + private let stdout: Sink + private let recorder: Mutex + + /// - Throws: when the record file cannot be created. + public init(output: OutputOptions, stdout: @escaping Sink) throws { + json = output.json + started = ProcessInfo.processInfo.systemUptime + self.stdout = stdout + var file: FileHandle? + if let path = output.recordPath { + let url = URL(fileURLWithPath: path) + guard FileManager.default.createFile(atPath: url.path, contents: nil) else { + throw UsageError("cannot create --record-events file '\(path)'") + } + file = try FileHandle(forWritingTo: url) + } + recorder = Mutex(file) + } + + /// Seconds since the run started. + public var elapsed: Double { + ProcessInfo.processInfo.systemUptime - started + } + + /// One record: `text` for the terminal, `fields` for JSON (after `t` + /// and `type`). Multi-line text keeps the timestamp on its first line. + func emit(_ type: String, text: String, fields: [(String, JSON)] = []) { + let t = elapsed + let record = JSON.object([("t", .double(t)), ("type", .string(type))] + fields).encoded + if json { + stdout(record) + } else { + let stamp = String(format: "[%8.3f] ", t) + let indent = String(repeating: " ", count: stamp.count) + let lines = text.split(separator: "\n", omittingEmptySubsequences: false) + stdout(lines.enumerated().map { ($0.offset == 0 ? stamp : indent) + $0.element }.joined(separator: "\n")) + } + recorder.withLock { file in + file?.write(Data((record + "\n").utf8)) + } + } + + /// Flushes and closes the record file. + func close() { + recorder.withLock { file in + try? file?.synchronize() + try? file?.close() + file = nil + } + } +} diff --git a/Sources/LumeCtlCore/RunControl.swift b/Sources/LumeCtlCore/RunControl.swift new file mode 100644 index 0000000..b7a78c2 --- /dev/null +++ b/Sources/LumeCtlCore/RunControl.swift @@ -0,0 +1,123 @@ +import LumeEngineCore +import Synchronization + +/// What a run has seen so far, and the one-shot signal that ends it. +/// +/// Several tasks race to end a run — the duration timer, Ctrl-C, the event +/// stream reporting `.ended` or a failure — and the first one wins; the +/// others find the run already stopping. +final class RunControl: Sendable { + enum Stop: Sendable, Equatable { + case duration + case ended + case failed + case interrupted + } + + struct Tally: Sendable { + /// One entry per `.stalled`, with the stage `lastStallReason` named. + var stalls: [StallReason?] = [] + var errors = 0 + var downgrades = 0 + /// The error the session failed with. + var failure: EngineError? + } + + private struct State { + var stop: Stop? + var waiters: [CheckedContinuation] = [] + var sessionState: PlayerSession.State = .idle + var tally = Tally() + } + + private let state = Mutex(State()) + + /// Ends the run; only the first reason counts. + func stop(_ reason: Stop) { + let waiters = state.withLock { state -> [CheckedContinuation] in + guard state.stop == nil else { return [] } + state.stop = reason + let waiters = state.waiters + state.waiters.removeAll() + return waiters + } + for waiter in waiters { + waiter.resume(returning: reason) + } + } + + var stopReason: Stop? { + state.withLock { $0.stop } + } + + func waitForStop() async -> Stop { + await withCheckedContinuation { continuation in + let stopped = state.withLock { state -> Stop? in + if let stop = state.stop { return stop } + state.waiters.append(continuation) + return nil + } + if let stopped { continuation.resume(returning: stopped) } + } + } + + var snapshot: Tally { + state.withLock { $0.tally } + } + + func setSessionState(_ sessionState: PlayerSession.State) { + state.withLock { $0.sessionState = sessionState } + } + + /// Counts an error; true when it is the one the session failed with. + func recordError(_ error: EngineError) -> Bool { + state.withLock { state in + state.tally.errors += 1 + guard state.sessionState == .failed, state.tally.failure == nil else { return false } + state.tally.failure = error + return true + } + } + + func countStall(_ reason: StallReason?) { + state.withLock { $0.tally.stalls.append(reason) } + } + + func countDowngrade() { + state.withLock { $0.tally.downgrades += 1 } + } +} + +/// Runs `operation` and waits at most `seconds` for it. True when it +/// finished in time; when it did not, it keeps running unobserved. +func finishes(within seconds: Double, _ operation: @escaping @Sendable () async -> Void) async -> Bool { + let outcome = Once() + return await withCheckedContinuation { continuation in + outcome.arm(continuation) + Task { + await operation() + outcome.resolve(true) + } + Task { + try? await Task.sleep(for: .seconds(seconds)) + outcome.resolve(false) + } + } +} + +/// A continuation resumed by whichever side gets there first. +private final class Once: Sendable { + private let continuation = Mutex?>(nil) + + func arm(_ continuation: CheckedContinuation) { + self.continuation.withLock { $0 = continuation } + } + + func resolve(_ value: Bool) { + let continuation = self.continuation.withLock { stored in + defer { stored = nil } + return stored + } + continuation?.resume(returning: value) + } +} diff --git a/Sources/LumeCtlCore/URLScrubber.swift b/Sources/LumeCtlCore/URLScrubber.swift new file mode 100644 index 0000000..ac9c1af --- /dev/null +++ b/Sources/LumeCtlCore/URLScrubber.swift @@ -0,0 +1,77 @@ +/// Keeps credentials out of what lumectl prints. +/// +/// IPTV providers put the account in the URL itself — Xtream-style paths +/// (`/live///.ts`), `get.php?username=…&password=…`, +/// tokens in the query, userinfo before the host — and terminal output ends +/// up pasted into issues and chats. So a URL is reduced to what triage needs: +/// scheme, host and port (which provider, which edge), plus the file +/// extension (HLS playlist vs raw TS), and everything else is withheld. +/// +/// Deliberately small and private to the tool: it only has to be right for +/// the strings this tool prints. +public enum URLScrubber { + static let placeholder = "[redacted]" + + /// The URL reduced to `scheme://host[:port]`, with `/[redacted]` (plus a + /// short file extension when the path has one) standing for anything + /// after it. Strings without a scheme — local paths — come back as given: + /// a path names a file on the developer's own machine, not an account. + public static func redact(_ url: String) -> String { + guard let separator = url.range(of: "://") else { return url } + let scheme = url[.. String { + guard text.contains("://") else { return text } + // A scheme followed by `://` and everything up to whitespace or a + // character that commonly closes a quoted URL in a message. Built per + // call: `Regex` is not Sendable, so it cannot be a shared constant. + let embeddedURL = #/[A-Za-z][A-Za-z0-9+.\-]*://[^\s'"<>()\[\]{}]+/# + return text.replacing(embeddedURL) { match in + redact(String(text[match.range])) + } + } + + private static func isScheme(_ candidate: Substring) -> Bool { + guard let first = candidate.first, first.isASCII, first.isLetter else { return false } + return candidate.allSatisfy { $0.isASCII && ($0.isLetter || $0.isNumber || "+-.".contains($0)) } + } + + /// `.m3u8`, `.ts`, `.mkv` … of the last path component, when it looks + /// like a real extension (1–5 alphanumerics): it tells an HLS playlist + /// from a raw stream without saying which stream. + private static func fileExtension(of rest: Substring) -> String { + let pathEnd = rest.firstIndex(where: { "?#".contains($0) }) ?? rest.endIndex + let path = rest[.. [SubtitleImage] { + var rects: [UnsafeMutablePointer] = [] + for index in 0.. SubtitleImage? { + let width = Int(rect.w) + let height = Int(rect.h) + let stride = Int(rect.linesize.0) + guard width > 0, height > 0, stride >= width, + let indices = rect.data.0, let paletteData = rect.data.1, + rect.nb_colors > 0 + else { return nil } + + let palette = UnsafeRawPointer(paletteData).withMemoryRebound( + to: UInt32.self, capacity: Int(rect.nb_colors) + ) { Array(UnsafeBufferPointer(start: $0, count: Int(rect.nb_colors))) } + let pixels = premultipliedPixels( + indices: UnsafeBufferPointer(start: indices, count: stride * height), + width: width, height: height, stride: stride, palette: palette + ) + guard pixels.contains(where: { $0 >> 24 != 0 }), + let image = makeImage(pixels: pixels, width: width, height: height) + else { return nil } // fully transparent: nothing to draw + + return SubtitleImage( + image: image, + rect: normalizedRect( + x: Int(rect.x), y: Int(rect.y), width: width, height: height, + canvasWidth: canvasWidth, canvasHeight: canvasHeight + ) + ) + } + + /// Palette-indexed pixels to premultiplied ARGB words. FFmpeg's palettes + /// are straight-alpha `0xAARRGGBB`; Core Animation and Core Graphics + /// composite premultiplied, and straight alpha drawn as premultiplied + /// shows a bright fringe around every glyph's anti-aliased edge. An index + /// beyond the palette is transparent. + static func premultipliedPixels( + indices: UnsafeBufferPointer, + width: Int, + height: Int, + stride: Int, + palette: [UInt32] + ) -> [UInt32] { + let premultiplied = palette.map(premultiply) + var pixels = [UInt32](repeating: 0, count: width * height) + for row in 0.. UInt32 { + let alpha = argb >> 24 + func channel(_ shift: UInt32) -> UInt32 { + ((argb >> shift & 0xFF) * alpha + 127) / 255 + } + return alpha << 24 | channel(16) << 16 | channel(8) << 8 | channel(0) + } + + /// ARGB words in native (little-endian) order are BGRA bytes in memory: + /// `byteOrder32Little` + `premultipliedFirst`, Core Graphics' native layout. + static func makeImage(pixels: [UInt32], width: Int, height: Int) -> CGImage? { + guard pixels.count == width * height, + let space = CGColorSpace(name: CGColorSpace.sRGB) + else { return nil } + let data = pixels.withUnsafeBytes { Data($0) } + guard let provider = CGDataProvider(data: data as CFData) else { return nil } + return CGImage( + width: width, + height: height, + bitsPerComponent: 8, + bitsPerPixel: 32, + bytesPerRow: width * 4, + space: space, + bitmapInfo: CGBitmapInfo(rawValue: CGBitmapInfo.byteOrder32Little.rawValue + | CGImageAlphaInfo.premultipliedFirst.rawValue), + provider: provider, + decode: nil, + shouldInterpolate: true, + intent: .defaultIntent + ) + } + + static func normalizedRect( + x: Int, y: Int, width: Int, height: Int, + canvasWidth: Int, canvasHeight: Int + ) -> CGRect { + guard canvasWidth > 0, canvasHeight > 0 else { return .zero } + let canvasW = Double(canvasWidth) + let canvasH = Double(canvasHeight) + return CGRect( + x: Double(x) / canvasW, + y: Double(y) / canvasH, + width: Double(width) / canvasW, + height: Double(height) / canvasH + ) + } +} diff --git a/Sources/LumeEngineCore/Subtitle/SubtitleCue.swift b/Sources/LumeEngineCore/Subtitle/SubtitleCue.swift index 9e4f274..3e53b56 100644 --- a/Sources/LumeEngineCore/Subtitle/SubtitleCue.swift +++ b/Sources/LumeEngineCore/Subtitle/SubtitleCue.swift @@ -1,8 +1,14 @@ import Foundation import Synchronization -/// One subtitle cue on the engine timeline. Text-only for now; bitmap cues -/// (PGS/DVB) and full libass styling land with the subtitle pack (PLAN.md P5+). +/// One subtitle cue on the engine timeline: text, bitmaps, or (rarely) both. +/// Full libass styling lands with the subtitle pack (PLAN.md P5+). +/// +/// Bitmap formats (DVB, PGS, DVD) arrive as `images` with empty `text`, so a +/// consumer that only reads `text` shows nothing for them rather than +/// something wrong. Drawing them is the host's job: `LumePlayerView` is bare +/// video. Draw each image at its `rect` scaled to the video's display area, +/// above the video. public struct SubtitleCue: Sendable, Equatable, Identifiable { public let id: UInt64 /// Engine µs. @@ -10,12 +16,19 @@ public struct SubtitleCue: Sendable, Equatable, Identifiable { public let end: Int64 /// Plain display text (override tags stripped, line breaks preserved). public let text: String + /// Bitmaps to draw over the video, positioned relative to it. + public let images: [SubtitleImage] - public init(id: UInt64, start: Int64, end: Int64, text: String) { + public init(id: UInt64, start: Int64, end: Int64, text: String, images: [SubtitleImage] = []) { self.id = id self.start = start self.end = end self.text = text + self.images = images + } + + func ending(at time: Int64) -> SubtitleCue { + SubtitleCue(id: id, start: start, end: time, text: text, images: images) } } diff --git a/Sources/LumeEngineCore/Subtitle/SubtitleDecoder.swift b/Sources/LumeEngineCore/Subtitle/SubtitleDecoder.swift index 4878e26..cda9a33 100644 --- a/Sources/LumeEngineCore/Subtitle/SubtitleDecoder.swift +++ b/Sources/LumeEngineCore/Subtitle/SubtitleDecoder.swift @@ -7,7 +7,8 @@ import Foundation /// so does the closed-caption lane, whose packets come from the video decoder /// (`ClosedCaptionTap`) rather than the demuxer. Each decoded update goes to /// the store as a `SubtitleEvent`; how long it shows is the store's policy. -/// Bitmap codecs (PGS/DVB/VobSub) are recognized but skipped for now. +/// Bitmap codecs (DVB, PGS, VobSub) produce cues with `images` instead of +/// text (`SubtitleBitmap`). public final class SubtitleDecoder: @unchecked Sendable { public let store: SubtitleStore @@ -204,9 +205,15 @@ public final class SubtitleDecoder: @unchecked Sendable { lines.append(String(cString: text)) } default: - break // bitmap subtitle — deferred (libass/PGS pack) + break // bitmaps below } } + // Read after decoding: the bitmap decoders set the context's size to + // their authoring canvas as they parse it (DVB display definition, + // PGS presentation, DVD `size:`). + let images = SubtitleBitmap.images( + of: subtitle, canvasWidth: context.pointee.width, canvasHeight: context.pointee.height + ) var text = lines.joined(separator: "\n") if isClosedCaptions { // Caption rows are indented by column position, which a text @@ -227,7 +234,8 @@ public final class SubtitleDecoder: @unchecked Sendable { start: span.start, end: span.end, text: text.trimmingCharacters(in: .whitespacesAndNewlines), - replacesPrevious: replacesPrevious + replacesPrevious: replacesPrevious, + images: images )) } @@ -246,6 +254,14 @@ public final class SubtitleDecoder: @unchecked Sendable { avcodec_free_context(&pointer) return false } + if context.pointee.codec_id == AV_CODEC_ID_DVB_SUBTITLE { + // The DVB decoder takes its canvas from the first display + // definition only while the context has no size; the stream probe + // may have left one (often the 720×576 default), which would pin + // an HD page's bitmaps to an SD canvas. + context.pointee.width = 0 + context.pointee.height = 0 + } context.pointee.pkt_timebase = lume_av_time_base_q() var options: OpaquePointer? for (key, value) in codecOptions { diff --git a/Sources/LumeEngineCore/Subtitle/SubtitleTimeline.swift b/Sources/LumeEngineCore/Subtitle/SubtitleTimeline.swift index d3f8381..cbcee35 100644 --- a/Sources/LumeEngineCore/Subtitle/SubtitleTimeline.swift +++ b/Sources/LumeEngineCore/Subtitle/SubtitleTimeline.swift @@ -8,14 +8,19 @@ struct SubtitleEvent: Sendable, Equatable { /// exactly that — "until the next update" — and a decoder that turned it /// into a number would leave the text up for 49 days (`UINT32_MAX` ms). var end: Int64? - /// Display text, trimmed. Empty on a replacing track means "clear the - /// screen", which is how those formats end what is showing. + /// Display text, trimmed. Empty — with no images either — on a replacing + /// track means "clear the screen", which is how those formats end what is + /// showing. var text: String /// The track describes the whole screen with every update (closed /// captions, DVB, PGS, DVD): an update ends everything shown before it. /// Text formats (SRT, ASS, WebVTT, mov_text) overlap freely instead, each /// cue carrying its own end. var replacesPrevious: Bool + /// Bitmaps (DVB, PGS, DVD). + var images: [SubtitleImage] = [] + + var isEmpty: Bool { text.isEmpty && images.isEmpty } /// Where an `AVSubtitle` sits on the engine timeline. `base` is the /// subtitle's PTS in engine µs, display times are milliseconds after it. @@ -66,8 +71,8 @@ struct SubtitleEvent: Sendable, Equatable { /// seek, backfill seek) does not double its lines. On a replacing track a /// different update at the same start replaces the stored one: the screen /// at one instant has one state, and the latest decode of it wins. -/// - At most `capacity` cues are kept; the ones farthest from where cues are -/// being added go first. +/// - At most `capacity` cues, holding at most `imageByteBudget` of bitmaps, +/// are kept; the ones farthest from where cues are being added go first. struct SubtitleTimeline: Sendable { /// How long an open cue stays without a successor. Caption decoders on /// televisions erase a caption left standing by about this long without @@ -80,6 +85,12 @@ struct SubtitleTimeline: Sendable { /// for a live track it is half an hour or more of roll-up captions. static let capacity = 8_192 + /// Bitmap memory kept. A full-HD PGS line is the better part of a + /// megabyte decoded, so a cue count alone would let a film's bitmap track + /// grow to gigabytes; this keeps the last minute or more of a live DVB + /// page, and a backward seek re-decodes whatever was dropped. + static let imageByteBudget = 64 << 20 + /// How far back from an insertion point active cues are looked for. /// Overlap is rare and short-lived; a replacing track has at most one. private static let scanLimit = 64 @@ -89,15 +100,19 @@ struct SubtitleTimeline: Sendable { /// The end is the limit, not the source's: the next update or a seek /// may still bound it. var provisional: Bool + + var imageBytes: Int { cue.images.reduce(0) { $0 + $1.byteCount } } } private var entries: [Entry] = [] private var nextID: UInt64 = 0 + /// Sum of `imageBytes` over `entries`. + private(set) var imageBytes = 0 var count: Int { entries.count } mutating func apply(_ event: SubtitleEvent) { - if event.text.isEmpty { + if event.isEmpty { // A clear. Text tracks have no such thing; an empty text cue is // just nothing to show. guard event.replacesPrevious else { return } @@ -119,13 +134,12 @@ struct SubtitleTimeline: Sendable { } guard end > event.start else { return } - entries.insert( - Entry( - cue: SubtitleCue(id: nextID, start: event.start, end: end, text: event.text), - provisional: event.end == nil - ), - at: index + let entry = Entry( + cue: SubtitleCue(id: nextID, start: event.start, end: end, text: event.text, images: event.images), + provisional: event.end == nil ) + entries.insert(entry, at: index) + imageBytes += entry.imageBytes nextID += 1 trim(keeping: index) } @@ -140,6 +154,7 @@ struct SubtitleTimeline: Sendable { mutating func removeAll() { entries.removeAll() + imageBytes = 0 } /// All cues covering `time` (engine µs), ordered by start. @@ -175,7 +190,11 @@ struct SubtitleTimeline: Sendable { var index = insertionIndex(for: event.start) - 1 while index >= 0 && entries[index].cue.start == event.start { let entry = entries[index] - if entry.cue.text == event.text { + // Bitmaps decoded twice are different objects; where and how big + // they are identifies them. + if entry.cue.text == event.text, + entry.cue.images.map(\.rect) == event.images.map(\.rect), + entry.cue.images.map(\.image.width) == event.images.map(\.image.width) { // A stored end may have been bounded by a successor since, so // it only has to match where both ends came from the source. if event.replacesPrevious || event.end == nil || entry.provisional { return true } @@ -189,7 +208,7 @@ struct SubtitleTimeline: Sendable { private mutating func removeEntries(startingAt start: Int64) { var index = insertionIndex(for: start) - 1 while index >= 0 && entries[index].cue.start == start { - entries.remove(at: index) + imageBytes -= entries.remove(at: index).imageBytes index -= 1 } } @@ -201,26 +220,40 @@ struct SubtitleTimeline: Sendable { while index >= 0 && scanned < Self.scanLimit { let entry = entries[index] if entry.cue.start < time, entry.cue.end > time, !onlyProvisional || entry.provisional { - entries[index].cue = SubtitleCue( - id: entry.cue.id, start: entry.cue.start, end: time, text: entry.cue.text - ) + entries[index].cue = entry.cue.ending(at: time) } index -= 1 scanned += 1 } } - /// Drops cues beyond `capacity` from the end farther from `index`, where - /// cues are being added — the front on a live timeline, the back after a - /// seek far into the past of a long file. Drops an eighth more than - /// needed so the shift is not paid on every insert. + /// Drops cues beyond `capacity` or `imageByteBudget` from the end farther + /// from `index`, where cues are being added — the front on a live + /// timeline, the back after a seek far into the past of a long file. Over + /// capacity it drops an eighth more than needed, so the shift is not paid + /// on every insert. The cue just added always stays. private mutating func trim(keeping index: Int) { - guard entries.count > Self.capacity else { return } - let excess = entries.count - Self.capacity + Self.capacity / 8 - if index < entries.count / 2 { - entries.removeLast(excess) + var kept = index + if entries.count > Self.capacity { + drop(entries.count - Self.capacity + Self.capacity / 8, keeping: &kept) + } + while imageBytes > Self.imageByteBudget, entries.count > 1 { + drop(1, keeping: &kept) + } + } + + private mutating func drop(_ count: Int, keeping kept: inout Int) { + let count = min(count, entries.count - 1) + guard count > 0 else { return } + let removed: ArraySlice + if kept < entries.count / 2 { + removed = entries[(entries.count - count)...] + entries.removeLast(count) } else { - entries.removeFirst(excess) + removed = entries[...makeStream() + let sigint = installInterruptHandler(continuation) + let code = await LumeCtl.run( + command, + interrupts: interrupts, + stdout: { writeLine($0, to: .standardOutput) }, + stderr: { writeLine($0, to: .standardError) } + ) + sigint.cancel() + exit(code.rawValue) + } + + /// The first Ctrl-C ends the run cleanly (bounded session shutdown, then + /// the summary); a second one means the user is done waiting. + private static func installInterruptHandler( + _ continuation: AsyncStream.Continuation + ) -> DispatchSourceSignal { + // Dispatch delivers the signal only once the default action is off. + signal(SIGINT, SIG_IGN) + let source = DispatchSource.makeSignalSource(signal: SIGINT, queue: DispatchQueue(label: "lumectl.sigint")) + let count = InterruptCount() + source.setEventHandler { + if count.increment() == 1 { + continuation.yield() + } else { + writeLine("lumectl: interrupted again, exiting without cleanup", to: .standardError) + exit(ExitCode.interrupted.rawValue) + } + } + source.resume() + return source + } + + /// Unbuffered, so piped output (`| tee`, `| jq`) shows each record as it + /// happens rather than when a buffer fills. + private static func writeLine(_ line: String, to handle: FileHandle) { + handle.write(Data((line + "\n").utf8)) + } +} +/// Ctrl-C presses so far; the handler is a `@Sendable` closure. +private final class InterruptCount: Sendable { + private let count = Mutex(0) + + func increment() -> Int { + count.withLock { count in + count += 1 + return count + } + } +} +#else +@main +enum LumeCtlMain { + static func main() { + print("lumectl is macOS-only") + } +} +#endif diff --git a/Tests/LumeEngineCoreTests/AudioRouteTests.swift b/Tests/LumeEngineCoreTests/AudioRouteTests.swift index f792888..0570fed 100644 --- a/Tests/LumeEngineCoreTests/AudioRouteTests.swift +++ b/Tests/LumeEngineCoreTests/AudioRouteTests.swift @@ -505,7 +505,7 @@ struct AudioRouteSessionTests { #expect(await session.audioHealth == .healthy) // The renderer stops taking audio; the clock and the picture go on. - session.renderer.audioRenderer.stopRequestingMediaData() + session.renderer.stopFeedingAudio() let lost = await eventually(timeout: 15) { await session.audioHealth == .lost(.starved) } let health = await session.audioHealth #expect(lost, "lost audio must be detected, health is \(health)") diff --git a/Tests/LumeEngineCoreTests/BitmapSubtitleTests.swift b/Tests/LumeEngineCoreTests/BitmapSubtitleTests.swift new file mode 100644 index 0000000..6ea7db1 --- /dev/null +++ b/Tests/LumeEngineCoreTests/BitmapSubtitleTests.swift @@ -0,0 +1,178 @@ +import CoreGraphics +import Foundation +import Testing +@testable import LumeEngineCore + +/// Reads one pixel of a `SubtitleBitmap` image as (r, g, b, a), premultiplied. +private func pixel(of image: CGImage, x: Int, y: Int) -> (r: Int, g: Int, b: Int, a: Int)? { + guard let data = image.dataProvider?.data as Data? else { return nil } + let offset = y * image.bytesPerRow + x * 4 + guard offset + 4 <= data.count else { return nil } + // byteOrder32Little + premultipliedFirst: B, G, R, A in memory. + return (Int(data[offset + 2]), Int(data[offset + 1]), Int(data[offset]), Int(data[offset + 3])) +} + +@Suite("Bitmap subtitle conversion") +struct BitmapSubtitleConversionTests { + @Test("palette colours are premultiplied") + func premultiply() { + #expect(SubtitleBitmap.premultiply(0xFF_FF_FF_FF) == 0xFF_FF_FF_FF) + #expect(SubtitleBitmap.premultiply(0x00_FF_FF_FF) == 0) + // Half-transparent red: colour scaled by alpha, alpha kept. + #expect(SubtitleBitmap.premultiply(0x80_FF_00_00) == 0x80_80_00_00) + #expect(SubtitleBitmap.premultiply(0x40_80_40_20) == 0x40_20_10_08) + } + + @Test("indexed rows honour the stride; indices past the palette are transparent") + func indexedPixels() { + // Two rows of three pixels, stride four (one byte of padding each). + let indices: [UInt8] = [0, 1, 2, 9, 2, 1, 7, 9] + let palette: [UInt32] = [0x00_00_00_00, 0xFF_FF_FF_FF, 0x80_FF_00_00] + let pixels = indices.withUnsafeBufferPointer { + SubtitleBitmap.premultipliedPixels(indices: $0, width: 3, height: 2, stride: 4, palette: palette) + } + #expect(pixels == [0, 0xFF_FF_FF_FF, 0x80_80_00_00, 0x80_80_00_00, 0xFF_FF_FF_FF, 0]) + } + + @Test("an image round-trips as premultiplied BGRA") + func imageLayout() throws { + let image = try #require(SubtitleBitmap.makeImage(pixels: [0xFF_FF_FF_FF, 0x80_80_00_00], width: 2, height: 1)) + #expect(image.width == 2 && image.height == 1) + #expect(image.alphaInfo == .premultipliedFirst) + let white = try #require(pixel(of: image, x: 0, y: 0)) + #expect(white == (255, 255, 255, 255)) + let red = try #require(pixel(of: image, x: 1, y: 0)) + #expect(red == (128, 0, 0, 128)) + } + + @Test("rects are normalized to the authoring canvas") + func normalizedRect() { + let rect = SubtitleBitmap.normalizedRect(x: 480, y: 900, width: 960, height: 108, canvasWidth: 1920, canvasHeight: 1080) + #expect(rect == CGRect(x: 0.25, y: 0.8333333333333334, width: 0.5, height: 0.1)) + #expect(SubtitleBitmap.normalizedRect(x: 0, y: 0, width: 1, height: 1, canvasWidth: 0, canvasHeight: 0) == .zero) + } + + private func image(width: Int, height: Int) throws -> SubtitleImage { + let image = try #require(SubtitleBitmap.makeImage( + pixels: [UInt32](repeating: 0xFF_FF_FF_FF, count: width * height), width: width, height: height + )) + return SubtitleImage(image: image, rect: CGRect(x: 0.4, y: 0.9, width: 0.1, height: 0.05)) + } + + @Test("a bitmap update is a cue, not a clear, and a re-read does not double it") + func bitmapCuesInTimeline() throws { + var timeline = SubtitleTimeline() + let bitmap = try image(width: 8, height: 4) + timeline.apply(SubtitleEvent(start: 1_000_000, end: nil, text: "", replacesPrevious: true, images: [bitmap])) + timeline.apply(SubtitleEvent(start: 3_000_000, end: nil, text: "", replacesPrevious: true)) + let shown = timeline.activeCues(at: 2_000_000) + #expect(shown.count == 1) + #expect(shown.first?.images.map(\.rect) == [bitmap.rect]) + #expect(shown.first?.text == "") + #expect(timeline.activeCues(at: 3_000_000).isEmpty) + + // The same update decoded again makes a new CGImage of the same size. + let again = try image(width: 8, height: 4) + timeline.apply(SubtitleEvent(start: 1_000_000, end: nil, text: "", replacesPrevious: true, images: [again])) + #expect(timeline.count == 1) + } + + @Test("bitmap memory is bounded, dropping the oldest cues") + func imageBudget() throws { + var timeline = SubtitleTimeline() + // 1 MB per cue: the budget holds 64 of them. + let bitmap = try image(width: 512, height: 512) + for index in 0..<100 { + timeline.apply(SubtitleEvent( + start: Int64(index) * 1_000_000, end: nil, text: "", replacesPrevious: true, images: [bitmap] + )) + } + #expect(timeline.imageBytes <= SubtitleTimeline.imageByteBudget) + #expect(timeline.count == SubtitleTimeline.imageByteBudget / bitmap.byteCount) + #expect(timeline.activeCues(at: 99_500_000).count == 1) + #expect(timeline.activeCues(at: 500_000).isEmpty) + timeline.removeAll() + #expect(timeline.imageBytes == 0) + } +} + +@Suite("Bitmap subtitles in playback", .serialized) +struct BitmapSubtitlePlaybackTests { + private func eventually( + timeout: TimeInterval = 10, + _ condition: () async -> Bool + ) async -> Bool { + let deadline = Date(timeIntervalSinceNow: timeout) + while Date() < deadline { + if await condition() { return true } + try? await Task.sleep(for: .milliseconds(100)) + } + return false + } + + /// The fixtures carry the same two bitmaps (see `generate-fixtures.sh`): + /// 64x16 at (928, 1000) for 1–3 s, 48x8 at (936, 960) for 4–6 s, on a + /// 1920x1080 canvas over a 320x180 video — so a canvas taken from the + /// video instead of the subtitle would put them far off screen. + @Test( + "bitmap subtitles become positioned image cues", + .timeLimit(.minutes(1)), + arguments: ["pgs.mkv", "dvbsub.ts", "dvdsub.mkv"] + ) + func bitmapCues(fixture: String) async throws { + var configuration = PlayerConfiguration() + configuration.muted = true + configuration.bufferTarget = 0.3 + let session = PlayerSession(configuration: configuration) + let info = try await session.open(url: try Fixtures.path(fixture)) + let track = try #require(info.subtitleTracks.first) + + await session.selectSubtitleTrack(track.index) + let start = info.startTime + let decoded = await eventually { + session.subtitles.activeCues(at: start + 5_000_000).first?.images.isEmpty == false + } + #expect(decoded, "\(fixture): cues should decode, store has \(session.subtitles.count)") + + let first = try #require(session.subtitles.activeCues(at: start + 2_000_000).first, "\(fixture): first cue") + #expect(first.text.isEmpty, "bitmap cues carry no text, so text consumers show nothing") + let image = try #require(first.images.first) + #expect(first.images.count == 1) + #expect(abs(first.start - (start + 1_000_000)) <= 40_000, "\(fixture): starts at \(first.start - start)") + #expect(abs(first.end - (start + 3_000_000)) <= 40_000, "\(fixture): ends at \(first.end - start)") + + // Decoders may crop fully transparent borders (FFmpeg's DVD decoder + // drops the 8 transparent columns); the visible part must still land + // within a few canvas pixels of the original. + let tolerance = 4.0 / 1080 + #expect(abs(image.rect.maxX - 992.0 / 1920) <= tolerance, "\(fixture): \(image.rect)") + #expect(abs(image.rect.maxY - 1016.0 / 1080) <= tolerance, "\(fixture): \(image.rect)") + #expect(image.rect.minX >= 928.0 / 1920 - tolerance, "\(fixture): \(image.rect)") + #expect(image.rect.minX <= 936.0 / 1920 + tolerance, "\(fixture): \(image.rect)") + #expect(abs(image.rect.minY - 1000.0 / 1080) <= tolerance, "\(fixture): \(image.rect)") + #expect(image.image.width == Int((image.rect.width * 1920).rounded()), "\(fixture): one image pixel per canvas pixel") + + // Column 2 is transparent, column 16 opaque white, column 48 + // half-transparent red (premultiplied). The DVB and DVD encoders + // requantize the palette, hence the tolerances. + let offsetX = Int(((image.rect.minX - 928.0 / 1920) * 1920).rounded()) + let offsetY = Int(((image.rect.minY - 1000.0 / 1080) * 1080).rounded()) + let row = 8 - offsetY + if offsetX <= 2 { + let clear = try #require(pixel(of: image.image, x: 2 - offsetX, y: row)) + #expect(clear.a <= 16, "\(fixture): \(clear)") + } + let white = try #require(pixel(of: image.image, x: 16 - offsetX, y: row)) + #expect(white.a >= 240 && white.r >= 220 && white.g >= 220 && white.b >= 220, "\(fixture): \(white)") + let red = try #require(pixel(of: image.image, x: 48 - offsetX, y: row)) + #expect((100...160).contains(red.a) && red.r >= red.a - 30 && red.g <= 30 && red.b <= 30, "\(fixture): \(red)") + #expect(red.r <= red.a, "\(fixture): premultiplied colour never exceeds alpha") + + #expect(session.subtitles.activeCues(at: start + 3_500_000).isEmpty, "\(fixture): cleared at 3 s") + let second = try #require(session.subtitles.activeCues(at: start + 5_000_000).first?.images.first) + #expect(abs(second.rect.width - 48.0 / 1920) <= tolerance, "\(fixture): \(second.rect)") + #expect(session.subtitles.activeCues(at: start + 6_500_000).isEmpty, "\(fixture): cleared at 6 s") + + await session.shutdown() + } +} diff --git a/Tests/LumeEngineCoreTests/Fixtures/generate-fixtures.sh b/Tests/LumeEngineCoreTests/Fixtures/generate-fixtures.sh index 9edace4..30a27fe 100644 --- a/Tests/LumeEngineCoreTests/Fixtures/generate-fixtures.sh +++ b/Tests/LumeEngineCoreTests/Fixtures/generate-fixtures.sh @@ -20,6 +20,8 @@ # hdr10.mp4 — HEVC Main 10, BT.2020 PQ, mastering display + content light level SEI, HDR10+ per frame # surround51.ts — 20 s H.264 + AC-3 5.1 MPEG-TS, tone in the centre channel only # captions.ts — H.264 TS with CEA-608 captions in A/53 SEI (1–3 s, 4–6 s; needs python3) +# pgs.mkv, dvbsub.ts, dvdsub.mkv — the same two bitmap subtitles (1–3 s, 4–6 s) +# as PGS, DVB and DVD subtitles on a 1920x1080 canvas (needs python3) set -euo pipefail FFMPEG="${FFMPEG:-ffmpeg}" @@ -428,4 +430,77 @@ PY fi fi +# Bitmap subtitles in the three formats broadcast and disc rips carry: PGS +# (Blu-ray), DVB (European broadcast) and DVD. ffmpeg cannot render text into +# subtitle bitmaps, but it converts between bitmap formats, so a short script +# writes the PGS original by hand: a 64x16 bitmap (8 transparent, 24 opaque +# white and 32 half-transparent red pixels per row) at (928, 1000) of a +# 1920x1080 canvas shown 1–3 s, then a 48x8 white one at (936, 960) shown +# 4–6 s, each ended by an empty presentation. A leading empty presentation at +# 0 keeps ffmpeg from shifting the subtitles to start at 0, and +# -fix_sub_duration turns PGS's "until the next one" into the durations the +# DVB and DVD encoders need. The DVB conversion keeps the canvas as a display +# definition segment. +if [ ! -f "$OUT/pgs.mkv" ] || [ ! -f "$OUT/dvbsub.ts" ] || [ ! -f "$OUT/dvdsub.mkv" ]; then + if command -v python3 >/dev/null 2>&1; then + python3 -I - "$OUT/bitmap-subs.sup" <<'PY' +import struct +import sys + +def segment(kind, pts, payload): + return b'PG' + struct.pack('>IIBH', pts, 0, kind, len(payload)) + payload + +def display_set(pts, composition, position=None, palette=None, bitmap=None): + objects = 1 if bitmap else 0 + pcs = struct.pack('>HHBHBBBB', 1920, 1080, 0x10, composition, 0x80 if bitmap else 0x00, 0, 0, objects) + if bitmap: + x, y = position + width, height, rle = bitmap + pcs += struct.pack('>HBBHH', 0, 0, 0, x, y) + out = segment(0x16, pts, pcs) + out += segment(0x17, pts, struct.pack('>BBHHHH', 1, 0, x, y, width, height)) + pds = struct.pack('>BB', 0, 0) + b''.join(struct.pack('>BBBBB', *entry) for entry in palette) + out += segment(0x14, pts, pds) + ods = struct.pack('>HBB', 0, 0, 0xC0) + struct.pack('>I', len(rle) + 4)[1:] + out += segment(0x15, pts, ods + struct.pack('>HH', width, height) + rle) + else: + out = segment(0x16, pts, pcs) + out += segment(0x17, pts, struct.pack('>BBHHHH', 1, 0, 0, 0, 1, 1)) + return out + segment(0x80, pts, b'') + +# (index, Y, Cr, Cb, alpha): transparent, opaque white, half-transparent red. +palette = [(0, 16, 128, 128, 0), (1, 235, 128, 128, 255), (2, 63, 240, 102, 128)] +# PGS run-length lines: 00 0n = n of colour 0, 00 8n c = n of colour c, 00 00 = end. +first = (64, 16, bytes([0, 0x08, 0, 0x80 | 24, 1, 0, 0x80 | 32, 2, 0, 0]) * 16) +second = (48, 8, bytes([0, 0x80 | 48, 1, 0, 0]) * 8) + +stream = display_set(0, 0) +stream += display_set(90_000 * 1, 1, (928, 1000), palette, first) +stream += display_set(90_000 * 3, 2) +stream += display_set(90_000 * 4, 3, (936, 960), palette, second) +stream += display_set(90_000 * 6, 4) +open(sys.argv[1], 'wb').write(stream) +PY + "$FFMPEG" -hide_banner -loglevel error -y \ + -f lavfi -i "testsrc2=duration=8:size=320x180:rate=25" \ + -i "$OUT/bitmap-subs.sup" -map 0:v -map 1:s \ + -c:v libx264 -preset ultrafast -pix_fmt yuv420p -c:s copy \ + "$OUT/pgs.mkv" + "$FFMPEG" -hide_banner -loglevel error -y \ + -f lavfi -i "testsrc2=duration=8:size=320x180:rate=25" \ + -fix_sub_duration -i "$OUT/bitmap-subs.sup" -map 0:v -map 1:s \ + -c:v libx264 -preset ultrafast -pix_fmt yuv420p -c:s dvbsub \ + -muxdelay 0 -f mpegts "$OUT/dvbsub.ts" + "$FFMPEG" -hide_banner -loglevel error -y \ + -f lavfi -i "testsrc2=duration=8:size=320x180:rate=25" \ + -fix_sub_duration -i "$OUT/bitmap-subs.sup" -map 0:v -map 1:s \ + -c:v libx264 -preset ultrafast -pix_fmt yuv420p -c:s dvdsub \ + "$OUT/dvdsub.mkv" + rm -f "$OUT/bitmap-subs.sup" + echo "generated: $OUT/pgs.mkv $OUT/dvbsub.ts $OUT/dvdsub.mkv" + else + echo "skipped: bitmap subtitle fixtures (needs python3)" >&2 + fi +fi + echo "fixtures ready in $OUT" diff --git a/Tests/LumeEngineCoreTests/LumeCtlTests.swift b/Tests/LumeEngineCoreTests/LumeCtlTests.swift new file mode 100644 index 0000000..db858f4 --- /dev/null +++ b/Tests/LumeEngineCoreTests/LumeCtlTests.swift @@ -0,0 +1,383 @@ +import Foundation +import Testing +@testable import LumeCtlCore +import LumeEngineCore + +// lumectl lives in this test target rather than one of its own: its runs +// need the generated fixtures and the torture server, both of which live here. + +@Suite("lumectl arguments") +struct LumeCtlArgumentTests { + private func play(_ arguments: String...) throws -> PlayOptions { + guard case .play(let options) = try Arguments.parse(["play"] + arguments) else { + throw UsageError("not a play command") + } + return options + } + + private func usageError(_ arguments: [String]) -> String? { + do { + _ = try Arguments.parse(arguments) + return nil + } catch { + return error.message + } + } + + @Test("no arguments, help and version") + func helpAndVersion() throws { + #expect(try Arguments.parse([]) == .help) + #expect(try Arguments.parse(["--help"]) == .help) + #expect(try Arguments.parse(["help"]) == .help) + #expect(try Arguments.parse(["play", "http://h/x.ts", "-h"]) == .help) + #expect(try Arguments.parse(["--version"]) == .version) + } + + @Test("play with every option, both spellings") + func playEverything() throws { + let options = try play( + "http://host/live.ts", + "--duration", "12.5", "--seek=30", "--start", "60", "--interval", "0", + "--software", "--audio-lang", "DE, en", "--no-video", "--stall-threshold=4", + "--header", "Authorization: Bearer a:b", "--header=X-Device: tv ", + "--user-agent", "Lume/1", "--referer", "http://ref/", "--no-reconnect", + "--io-timeout", "2.5", "--open-timeout", "10", + "--json", "--show-urls", "--record-events", "/tmp/run.jsonl" + ) + #expect(options.source.url == "http://host/live.ts") + #expect(options.duration == 12.5) + #expect(options.seek == 30) + #expect(options.start == 60) + #expect(options.interval == 0) + #expect(options.software) + #expect(options.audioLanguages == ["de", "en"]) + #expect(options.noVideo && !options.noAudio) + #expect(options.stallThreshold == 4) + #expect(options.source.headers == [ + .init(name: "Authorization", value: "Bearer a:b"), + .init(name: "X-Device", value: "tv"), + ]) + #expect(options.source.userAgent == "Lume/1") + #expect(options.source.referer == "http://ref/") + #expect(!options.source.reconnect) + #expect(options.source.ioTimeout == 2.5) + #expect(options.source.openTimeout == 10) + #expect(options.output.json && options.output.showURLs) + #expect(options.output.recordPath == "/tmp/run.jsonl") + } + + @Test("defaults: muted, engine defaults kept") + func playDefaults() throws { + let options = try play("/path/basic.mp4") + #expect(options.duration == nil && options.seek == nil && options.start == nil) + #expect(options.interval == 2) + let configuration = options.configuration + let engineDefaults = PlayerConfiguration() + #expect(configuration.muted) + #expect(configuration.hardwareDecode == .videoToolbox) + #expect(configuration.enableAudio && configuration.enableVideo) + #expect(configuration.startPosition == 0) + #expect(configuration.stallThreshold == engineDefaults.stallThreshold) + #expect(configuration.demuxer.ioTimeout == engineDefaults.demuxer.ioTimeout) + #expect(configuration.demuxer.openTimeout == engineDefaults.demuxer.openTimeout) + #expect(configuration.demuxer.enableReconnect) + #expect(configuration.demuxer.httpHeaders.isEmpty) + } + + @Test("options map onto PlayerConfiguration") + func configurationMapping() throws { + let configuration = try play( + "u", "--io-timeout", "2.5", "--open-timeout", "7", "--no-reconnect", "--software", + "--start", "42", "--audio-lang", "de,en", "--no-audio", "--stall-threshold", "3", + "--header", "A: 1", "--header", "A: 2", "--user-agent", "UA", "--referer", "R" + ).configuration + #expect(configuration.demuxer.ioTimeout == 2_500_000, "seconds become microseconds") + #expect(configuration.demuxer.openTimeout == 7) + #expect(!configuration.demuxer.enableReconnect) + #expect(configuration.hardwareDecode == .software) + #expect(configuration.startPosition == 42) + #expect(configuration.preferredAudioLanguages == ["de", "en"]) + #expect(!configuration.enableAudio) + #expect(configuration.stallThreshold == 3) + #expect(configuration.demuxer.httpHeaders == ["A": "2"], "a repeated header keeps the last value") + #expect(configuration.demuxer.userAgent == "UA") + #expect(configuration.demuxer.referer == "R") + } + + @Test("probe takes source and output options only") + func probe() throws { + let command = try Arguments.parse(["probe", "--json", "http://h/a.m3u8", "--user-agent", "UA"]) + guard case .probe(let options) = command else { + Issue.record("expected probe, got \(command)") + return + } + #expect(options.source.url == "http://h/a.m3u8") + #expect(options.output.json) + #expect(options.configuration.demuxer.userAgent == "UA") + #expect(usageError(["probe", "u", "--duration", "3"])?.contains("unknown option '--duration'") == true) + } + + @Test("-- ends options, so a URL may start with a dash") + func doubleDash() throws { + #expect(try play("--json", "--", "-odd-name.ts").source.url == "-odd-name.ts") + } + + @Test("bad command lines say what is wrong") + func usageErrors() { + let cases: [([String], String)] = [ + (["stream", "u"], "unknown command"), + (["play"], "needs a URL"), + (["play", "a", "b"], "unexpected extra argument 'b'"), + (["play", "u", "--duration"], "--duration expects a value"), + (["play", "u", "--duration", "-1"], "> 0"), + (["play", "u", "--duration", "0"], "> 0"), + (["play", "u", "--seek", "abc"], "≥ 0"), + (["play", "u", "--interval", "nan"], "≥ 0"), + (["play", "u", "--io-timeout", "inf"], "> 0"), + (["play", "u", "--header", "no colon"], "'Name: value'"), + (["play", "u", "--header", ": empty name"], "'Name: value'"), + (["play", "u", "--header", "A: 1\r\nB: 2"], "'Name: value'"), + (["play", "u", "--audio-lang", "de,,en"], "comma-separated"), + (["play", "u", "--software=yes"], "takes no value"), + (["play", "u", "--frobnicate"], "unknown option '--frobnicate'"), + (["play", "u", "-d", "3"], "spelled --name"), + (["play", "u", "--no-audio", "--no-video"], "nothing to play"), + (["play", ""], "empty"), + ] + for (arguments, expected) in cases { + let message = usageError(arguments) + #expect(message?.contains(expected) == true, "\(arguments) → \(message ?? "no error")") + } + } +} + +@Suite("lumectl URL redaction") +struct URLScrubberTests { + @Test("credentials in path, query and userinfo are withheld") + func redactsCredentials() { + let cases: [(String, String)] = [ + ("http://provider.tv:8080/live/alice/s3cret/1234.ts", "http://provider.tv:8080/[redacted].ts"), + ("http://provider.tv/get.php?username=alice&password=s3cret&type=m3u", "http://provider.tv/[redacted].php"), + ("https://alice:s3cret@cdn.example.com/hls/index.m3u8?token=abc#t=1", "https://cdn.example.com/[redacted].m3u8"), + ("http://a:p@ss@host/x", "http://host/[redacted]"), + ("http://host:80/movie/alice/s3cret/99", "http://host:80/[redacted]"), + ("http://host/stream?auth=xyz", "http://host/[redacted]"), + ("rtmp://ingest.example.com/app/streamkey", "rtmp://ingest.example.com/[redacted]"), + ("HTTP://Host/path.with.dots/file.TS", "HTTP://Host/[redacted].TS"), + ("http://host/clip.verylongext", "http://host/[redacted]"), + ("http://host/v1.2/stream", "http://host/[redacted]"), + ] + for (input, expected) in cases { + #expect(URLScrubber.redact(input) == expected, "\(input)") + } + } + + @Test("bare hosts and local paths are left alone") + func keepsWhatIsNotSecret() { + #expect(URLScrubber.redact("http://host:8080") == "http://host:8080") + #expect(URLScrubber.redact("http://host/") == "http://host/") + #expect(URLScrubber.redact("/Users/dev/TestStreams/basic.mp4") == "/Users/dev/TestStreams/basic.mp4") + #expect(URLScrubber.redact("file:///Users/dev/a.ts") == "file:///Users/dev/a.ts") + #expect(URLScrubber.redact("not a url") == "not a url") + #expect(URLScrubber.redact("1http://x/y") == "1http://x/y", "not a scheme") + } + + @Test("URLs inside messages are redacted where they stand") + func redactsText() { + let message = "Failed to open segment 'http://cdn.tv/live/alice/s3cret/seg7.ts?t=1' (via https://u:p@edge.tv/k) for reading" + #expect(URLScrubber.redactText(message) + == "Failed to open segment 'http://cdn.tv/[redacted].ts' (via https://edge.tv/[redacted]) for reading") + #expect(URLScrubber.redactText("Connection refused") == "Connection refused") + #expect(URLScrubber.redactText("") == "") + } +} + +@Suite("lumectl output") +struct LumeCtlOutputTests { + @Test("JSON keeps key order and escapes") + func jsonEncoding() { + let value = JSON.object([ + ("t", .double(1.23456)), + ("type", .string("a\"b\\c\nd\u{01}é")), + ("n", .null), ("ok", .bool(true)), ("i", .int(-5)), + ("nan", .double(.nan)), ("list", .array([.int(1), .optional(nil as String?)])), + ]) + #expect(value.encoded + == #"{"t":1.235,"type":"a\"b\\c\nd\u0001é","n":null,"ok":true,"i":-5,"nan":null,"list":[1,null]}"#) + } + + @Test("exit status: failure outranks stall, Ctrl-C is no verdict") + func exitCodes() { + #expect(ExitCode.playback(failed: false, stalls: 0) == .success) + #expect(ExitCode.playback(failed: false, stalls: 2) == .stalled) + #expect(ExitCode.playback(failed: true, stalls: 2) == .playbackFailed) + #expect(ExitCode.playback(failed: true, stalls: 0) == .playbackFailed) + let codes = [ExitCode.success, .toolFailure, .usage, .openFailed, .playbackFailed, .stalled, .interrupted] + #expect(Set(codes.map(\.rawValue)).count == codes.count, "every outcome has its own code") + } + + @Test("errors carry the HTTP class and no URL") + func errorRendering() { + let forbidden = EngineError( + code: .openFailed, ffmpegCode: -(0xF8 | 0x34 << 8 | 0x30 << 16 | 0x33 << 24), + message: "open http://h/live/u/p/1.ts: Forbidden" + ) + let rendered = Describe(showURLs: false).error(forbidden) + #expect(rendered.text.contains("http forbidden")) + #expect(!rendered.text.contains("/u/p/")) + #expect(rendered.json.encoded.contains(#""http":"forbidden""#)) + #expect(Describe(showURLs: true).error(forbidden).text.contains("/live/u/p/1.ts")) + } +} + +/// Collects what a run prints. +private final class Output: Sendable { + private let lines = LockedBox<[String]>([]) + + var sink: Reporter.Sink { + { [lines] line in lines.set(lines.get() + [line]) } + } + + var all: [String] { lines.get() } + + /// JSON records of a `--json` run, by type. + func records(_ type: String) -> [[String: Any]] { + all.compactMap { line in + guard let data = line.data(using: .utf8), + let object = try? JSONSerialization.jsonObject(with: data) as? [String: Any], + object["type"] as? String == type else { return nil } + return object + } + } +} + +@Suite("lumectl runs", .serialized) +struct LumeCtlRunTests { + private func run(_ arguments: [String], interrupts: AsyncStream = AsyncStream { _ in }) async throws + -> (ExitCode, Output) + { + let output = Output() + let code = await LumeCtl.run( + try Arguments.parse(arguments), interrupts: interrupts, stdout: output.sink, stderr: output.sink + ) + return (code, output) + } + + @Test("probe prints the tracks", .timeLimit(.minutes(1))) + func probe() async throws { + let (code, output) = try await run(["probe", try Fixtures.path("multitrack.mkv"), "--json"]) + #expect(code == .success) + let media = output.records("media").first?["media"] as? [String: Any] + let tracks = media?["tracks"] as? [[String: Any]] ?? [] + #expect(tracks.map { $0["kind"] as? String } == ["video", "audio", "audio", "subtitle"]) + #expect(tracks.compactMap { $0["language"] as? String }.contains("ger")) + #expect((media?["chapters"] as? [Any])?.count == 2) + #expect(output.records("summary").first?["exitCode"] as? Int == 0) + } + + @Test("play prints startup, heartbeats and a clean summary", .timeLimit(.minutes(1))) + func play() async throws { + let (code, output) = try await run([ + "play", try Fixtures.path("basic.mp4"), "--duration", "2", "--interval", "0.5", "--json", + ]) + #expect(code == .success, "\(output.all.joined(separator: "\n"))") + #expect(output.records("startup").first.flatMap { $0["startup"] as? [String: Any] }?["firstFrameShown"] is Double) + #expect(output.records("heartbeat").count >= 2) + let states = output.records("state").compactMap { $0["state"] as? String } + #expect(states.contains("playing")) + let summary = output.records("summary").first + #expect(summary?["outcome"] as? String == "ok") + #expect((summary?["position"] as? Double ?? 0) > 1, "two seconds of playback") + } + + @Test("a refused open exits 3 and never prints the credentials", .timeLimit(.minutes(1))) + func openFailure() async throws { + let server = try TortureHTTPServer(mode: .forbidden) + defer { server.stop() } + let url = "http://alice:pw@127.0.0.1:\(server.port)/live/alice/s3cret/1.ts?token=t0k" + let (code, output) = try await run(["play", url, "--no-reconnect", "--io-timeout", "2"]) + #expect(code == .openFailed) + let text = output.all.joined(separator: "\n") + #expect(text.contains("http forbidden"), "\(text)") + for secret in ["alice", "pw", "s3cret", "t0k"] { + #expect(!text.contains(secret), "leaked '\(secret)': \(text)") + } + } + + @Test("a source that goes silent exits 5 with the stage", .timeLimit(.minutes(1))) + func stall() async throws { + // ~6 s of a 12 s TS, then nothing on an open connection. Less than + // FFmpeg's ~5 s MPEG-TS probe window would hold `open` until its + // timeout instead. The watchdog reports the source within the + // threshold of it going quiet, while the buffer still plays out. + let server = try TortureHTTPServer(mode: .stallAfter(prefix: 700_000, file: Fixtures.url("wrap.ts"))) + defer { server.stop() } + let (code, output) = try await run([ + "play", server.url, "--duration", "5", "--stall-threshold", "2", + "--io-timeout", "60", "--no-reconnect", "--interval", "0", "--json", + ]) + #expect(code == .stalled, "\(output.all.joined(separator: "\n"))") + #expect(output.records("stalled").first?["reason"] as? String == "source") + } + + @Test("a live channel that ends exits 4", .timeLimit(.minutes(1))) + func liveSourceEnded() async throws { + let directory = try Fixtures.url("hls-live/index.m3u8").deletingLastPathComponent() + let server = try TortureHTTPServer(mode: .hlsLiveThenEnded(directory: directory)) + defer { server.stop() } + let (code, output) = try await run([ + "play", server.hlsURL, "--duration", "30", "--stall-threshold", "30", "--interval", "0", "--json", + ]) + #expect(code == .playbackFailed, "\(output.all.joined(separator: "\n"))") + let failure = output.records("summary").first?["failure"] as? [String: Any] + #expect(failure?["code"] as? String == "sourceEnded") + } + + @Test("an interrupt ends the run cleanly", .timeLimit(.minutes(1))) + func interrupt() async throws { + let (interrupts, continuation) = AsyncStream.makeStream() + let started = Date() + Task { + try? await Task.sleep(for: .seconds(1.5)) + continuation.yield() + } + let (code, output) = try await run( + ["play", try Fixtures.path("basic.mp4"), "--interval", "0", "--json"], interrupts: interrupts + ) + #expect(code == .success) + #expect(Date().timeIntervalSince(started) < 8, "a 10 s file must not play to its end") + #expect(output.records("note").first?["note"] as? String == "interrupted") + #expect(output.records("summary").count == 1) + } + + #if os(macOS) + @Test("Ctrl-C on the real binary shuts down and exits 0", .timeLimit(.minutes(1))) + func sigint() async throws { + // The test bundle sits next to the products it was built with. + let binary = Bundle(for: BundleMarker.self).bundleURL + .deletingLastPathComponent().appendingPathComponent("lumectl") + try #require(FileManager.default.isExecutableFile(atPath: binary.path), "no lumectl at \(binary.path)") + + let process = Process() + process.executableURL = binary + process.arguments = ["play", try Fixtures.path("basic.mp4"), "--interval", "0"] + let stdout = Pipe() + process.standardOutput = stdout + process.standardError = FileHandle.nullDevice + try process.run() + try await Task.sleep(for: .seconds(2)) + process.interrupt() + let deadline = Date(timeIntervalSinceNow: 10) + while process.isRunning, Date() < deadline { + try await Task.sleep(for: .milliseconds(50)) + } + if process.isRunning { process.terminate() } + let text = String(decoding: stdout.fileHandleForReading.readDataToEndOfFile(), as: UTF8.self) + #expect(process.terminationStatus == 0, "\(text)") + #expect(text.contains("interrupted, shutting down")) + #expect(text.contains("summary ok (exit 0)")) + } + #endif +} + +private final class BundleMarker {}