Skip to content

Latest commit

 

History

History
74 lines (41 loc) · 16.5 KB

report-comparison.md

File metadata and controls

74 lines (41 loc) · 16.5 KB

Offline report details and comparison

audio report summary REPORT --metrics --evidence-limits adds recorded measurements and a flat index of limits to the existing summary. audio report compare LEFT RIGHT compares two saved reports by recorded identity and time overlap. Both read report files only; they never open recorded source/output media paths, run detection, change processing decisions, or rewrite reports. The public callables audio_cli.pipeline.summarize_report and audio_cli.pipeline.compare_reports own this offline enhancement-report policy in pipeline/reports.

Optional summary details

summarize_report(path: Path, *, include_metrics: bool = False, include_evidence_limits: bool = False, navigation: bool = False) -> dict[str, Any]

The default audio_enhancement_summary version "1" projection remains unchanged. Each of the two detail keywords independently adds the following fields to the existing summary:

Keyword Additional fields
include_metrics=True measurements with the recorded before, predicted, and after blocks; each contains its block pointer and any recorded program_actual, regional, duration_delta_ms, and duration_basis values. regions is included only when recorded, providing the original intervals for regional measurements.
include_evidence_limits=True evidence_limits, a flat index of recorded limits under stages, unresolved rows, rule evaluations, timeline verification, and peak validation.

A copied metric block or region uses {"value": <recorded value>, "report_pointer": "/exact/location"}. Pointers refer to the original file named by report, including escaped JSON Pointer segments. Values are copied without rounding or remeasurement. Raw loudnorm program.output_* fields describe a hypothetical normalization pass and are never substituted for actual measurements. Legacy reports missing program_actual retain only their available blocks and pointers; no actual measurement is inferred from their raw diagnostics. Absent phases, regions, or metrics stay absent, and prediction never becomes encoded-output evidence.

Each evidence-limit row has report_pointer and the complete recorded evidence object. The flat index includes abstained, abstained_overlap, outside_target, bounded_outside_target, failed, rejected, not_run, skipped, and unmeasured statuses, plus a recorded target_attained: false. It includes nested limits beneath applied parents, such as RNNoise speech-preservation abstention and rejected noise-reference intervals. It does not change the parent's operation status. Stage operations can repeat component evidence; their distinct pointers are retained rather than silently deduplicated.

Where available, context retains the nearest recorded stage/component, time/frequency scope, start/end, measurement phase, reference basis, and speech reference, each with its own pointer. These pointers distinguish inherited treatment scope from a candidate reference interval or a child-specific scope. region_basis and region_scopes link to the recorded basis and manifest intervals when a report-local region ID resolves. No missing scope, measurement phase, or reference is guessed. An empty index means no indexed recorded limits, not an overall quality verdict.

Compare two reports

compare_reports(left: Path, right: Path, *, navigation: bool = False) -> dict[str, Any]

Inputs may be two saved audio_inspection or audio_enhancement_report documents, schema version "1", in either order. The result is audio_report_comparison, schema version "1", with compatibility, left, right, and, when both reports supply region manifests, region_comparison. Recorded scope evidence can additionally supply measurement_scope_comparison, described below. The function raises PipelineError for invalid reports or unestablished/incompatible identity; callers can use the existing enhancement error envelope and exit 2.

Compatibility requires recorded SHA256 identity and decoded timeline metadata: duration_basis: "decoded_pcm", time_origin: "first_decoded_sample", nonnegative integer sample_count, and positive integer sample_rate_hz. Recorded duration seconds, when present, must agree with the sample ratio to its published microsecond precision. Same-source reports must have the same source digest and decoded duration. Historical or moved media paths do not affect compatibility. Legacy reports without sufficient identity/timeline evidence refuse comparison; they remain eligible for the existing summary projection.

An enhancement report also bridges to an inspection of its delivered output when enhancement.output.sha256 == inspection.source.sha256. That bridge requires a rendered, non-dry-run report, timeline_preserved: true, matching decoded output/inspection durations, and a recorded passing decoded_audio_duration_only verification. The source/output duration difference must satisfy that report's recorded tolerance. The comparator consumes this saved check; it does not set a new media policy. compatibility.identities and compatibility.timeline_verification carry side-qualified exact evidence pointers. The latter preserves any recorded content-alignment and A/V-sync abstentions. A digest link and duration check establish neither content alignment nor A/V synchronization.

Each side retains its report path, source identity, profile and region basis when present, actual metric views, evidence limits, and recorded rule evaluations. Enhancement measurements retain their before/predicted/after names. Inspection measurements use inspected, with program_actual and pointed observations; inspection regional evidence is in observations rather than an invented enhancement-style regional block. Recorded relative-to-speech values and their reference names are copied exactly. Each side's speech_reference independently retains its speech intervals, available aggregate speech measurements, and detection basis.

region_comparison.basis is positive_time_interval_overlap. overlaps lists every positive intersection with its original left/right region, available measurements, intersection interval, same_interval, same_kind, and ambiguous. Ambiguity means either interval overlaps multiple intervals on the other side; no arbitrary best match is chosen. left_without_overlap and right_without_overlap retain regions without any positive intersection. Touching endpoints alone do not overlap. Region IDs join a measurement to its own report's manifest only; IDs never match regions across reports. Cross-kind overlaps remain visible, so changed classification does not disappear from the comparison.

These are comparisons of recorded measurements and detection scopes. A changed interval or classification is not a semantic regression or proof that an event appeared/disappeared. Regional values over different intervals, or relative to different aggregate speech scopes, are retained separately without a fabricated metric delta or quality verdict. Missing region manifests suppress region_comparison rather than manufacturing an empty match result.

measurement_scope_comparison.basis is recorded_time_intervals. Its two independent findings are speech_reference_intervals and non_speech_region_intervals, each same or different when the corresponding evidence is available. Speech reference intervals compare the union of the recorded speech spans: touching or overlapping spans merge, and their order, segmentation and report-local IDs do not matter. Non-speech regions compare the exact collection of individual start/end intervals, ignoring order and IDs while retaining duplicate intervals. Splitting one non-speech interval changes this finding even when total coverage stays the same, because regional RMS measurements describe each individual interval.

Both findings require region manifests and the recorded source-timeline, fixed-region basis with the report kind's original or inspected detection basis. The speech finding additionally requires nonempty speech spans and a recorded aggregate reference measurement; inspection references must identify the full-band speech-region scope. Missing, unknown or unavailable evidence leaves that finding absent, and an empty finding set omits the entire block. Two empty non-speech candidate collections can have same intervals; two empty speech collections do not establish a speech reference. These findings compare the published time coordinates exactly, without a matching tolerance or claims about decoded sample masks, equal RMS values, equal detection policies, perceptual quality or acoustic identity. Measurement phases remain in the existing side-specific measurement blocks: equal speech coverage across before and after does not mean the reference level stayed equal.

Use a report's fixed before and after regional measurements to inspect changes over its original intervals. A separately inspected delivered file has newly detected candidates and its own speech reference. For example, nine fixed source intervals can all be inside target while five fresh candidates are outside their inspection target. If either interval finding is different, those counts cannot substitute for one another or form a before/after success rate. The fresh findings remain measured diagnostics; neither collection establishes an overall audio failure or success. Even when both interval findings are same, compare the recorded phases, targets and evaluation policies before interpreting outcome changes.

Both functions reject malformed JSON, duplicate keys at any depth, nonfinite numbers (including exponent overflow), unencodable Unicode, unsupported kinds/versions, and nonregular input files. Requested metric views additionally validate actual metric/region structures. Validation completes before returning machine output.

Compact navigation

audio report summary REPORT --navigation emits audio_report_navigation, schema version "1". audio report compare LEFT RIGHT --navigation emits audio_report_comparison_navigation, schema version "1". These opt-in projections replace the usual stdout view for that invocation; the default projections and canonical report files are unchanged. Summary navigation cannot be combined with --metrics or --evidence-limits: it already contains compact measurements and a limit index, and conflicting flags refuse at exit 2. The matching Python keyword is navigation=True.

Start with measurements.after.program_actual for delivered program metrics. before, predicted, after, and inspected remain separate recorded measurement blocks. Regional aggregate values have individual pointers; the machine_regions count and pointer lead to the full original interval measurements. No legacy loudnorm diagnostic is promoted into an actual metric. regions gives the recorded manifest count and kind counts, while speech_reference retains that report's aggregate levels and pointers to its own speech intervals. Each comparison side keeps its own region basis and speech reference.

outcome_counts counts separate recorded collections by exact report_pointer, retaining their status counts where supplied. /rule_evaluations belongs to before in an enhancement report and inspected in an inspection report. Stage-local inside_target_regions, abstained_regions, and final_region_evaluations each have their own row; an intermediate count never substitutes for a final collection count. The /unresolved collection may contain several measurement phases and is labeled mixed_or_unknown; inspect its individual limit groups for the actual phases. A stage collection without recorded measured_at is unknown, even in a rendered report. A final collection's name and pointer remain visible without inventing its missing phase field.

The /rule_evaluations count row also has by_rule when entries supply string rule names. Each group retains the exact rule, its count, and its statuses, in first-occurrence order. Grouping uses only those recorded names; legacy entries with missing or non-string names remain in the full collection count and statuses without acquiring a group. This makes five machine-relative-level evaluations distinguishable from an inspection's seven total evaluations that also include channel balance and program loudness. The enclosing collection pointer and exact rule name identify the contributing original rows.

limits.groups groups indexed occurrences by recorded phase and contextual scope. Explicit measured_at values are copied exactly, including intermediate names such as after_voice_enhance; their original pointers stay in each occurrence's context. Original/inspected rule collections use their schema-defined phase. Other missing phases remain unknown: neither an applied parent, membership in unresolved, nor a render flag makes an unmarked child an encoded-output measurement. Missing scope is an empty object, never an inferred treatment interval. Local context, region scopes, and every original evidence pointer remain on their occurrences. An exact report-local observation_id link, when available, supplies observation_scopes with pointers to all matching recorded observations; it is not a cross-report region match.

Each occurrence's zero-based finding_index links into limits.findings. A finding exposes scalar recorded fields in preview and the names of nested detail_fields; its complete evidence is available at every occurrence's report_pointer. Only exact equality of the entire recorded evidence object shares a finding index, and scopes and phases remain attached to the separate occurrences. indexed_occurrences counts index entries, and distinct_recorded_values counts exact-equal evidence values; neither counts distinct delivered failures. Identical not_measured evidence can concern speech preservation, content alignment and A/V sync at different pointers. Unknown phase stays unknown even when an equal value appears elsewhere with a measured phase.

Comparison navigation retains the usual compatibility checks and adds comparison_overview.counts before the per-report and interval details: all positive overlap_pairs, ambiguous_overlap_pairs, same_interval_pairs, same_kind_pairs, each side's regions_with_multiple_overlaps, and left_without_overlap / right_without_overlap. Pair counts are not region counts. overlaps retains every intersection and its flags, using left_report_pointer and right_report_pointer to address the corresponding original reports. Each side's regions.intervals lists each interval once. Unmatched lists retain side-specific original pointers. Missing manifests still suppress the comparison rather than manufacturing zero counts. No best match, numeric delta, quality score or overall audio-success field is introduced.

When the detailed comparison supplies measurement_scope_comparison, navigation copies it unchanged into comparison_overview.measurement_scope_comparison. This adds scope findings to the existing interval and reference pointers without copying another measurement or phase block. The ordinary summary projection is unchanged, and neither comparison view rewrites the saved reports.

For full detail, follow a pointer into the named original report or rerun the same saved-report command without --navigation. This performs no audio work. Counts reflect only what the reports recorded, and comparison still establishes neither content alignment nor A/V sync.

RNNoise guide calibration metadata

calibrate_guide_input retains its exact existing gain calculation: min(target_rms_dbfs - level, peak_limit_dbfs - peak, maximum_gain_db). It adds target_attained, determined from the unrounded resolved gain, and limiting_reasons, listing every bound that equals the chosen gain and is strictly below the requested gain. The reason names are peak_headroom and maximum_gain, in that deterministic order. Tied limiting bounds both appear; bounds tied at the target do not prevent attainment and yield an empty list. Metadata neither feeds back into processing nor claims delivered-media quality.

Attenuation and a no-op can attain the target. A no-op constrained below the target reports the applicable limit. An unattained target can round to the displayed target at six decimals; attainment still reflects the original calculation. The no-energy path retains its existing status: "abstained", reason: "no_speech_energy" with no fabricated RMS, target-attainment result, or limiting-bound list. Calibration concerns only the model guide; it is not delivered speech leveling or measured denoising effectiveness.