Repository navigation
feat(v1): run-2 tooling: image degradations, paired landmark error, pilot - #10
Merged
Merged
Conversation
…ilot ImageDegradation applies graded, deterministic darken/downscale/blur/ noise to real frames; extract_rollout_from_video takes frame_transform and records it in metadata. paired_landmark_error measures distance from the same frame's clean detection in hand sizes, a quality signal the presence mask does not contain. run_v1_pilot.py reports dropout, landmark error and trivial presence baselines per severity and computes no gap scores by design. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Tooling for run 2's pilot and design (no change to existing behaviour unless the new options are used).
worldgap.data.degradations.ImageDegradation: graded, deterministic image degradations (darken,downscale,blur,noise, the last seeded per frame). Each is a frozen, JSON-describable value, so a pre-registered condition can be re-created exactly. These are simulated degradations of real footage and must be labelled so.extract_rollout_from_video(..., frame_transform=...): applies the transform to each decoded frame before MediaPipe and records it inmetadata["frame_transform"].paired_landmark_error(reference, degraded): for paired data (same video, one degraded), the distance between the degraded hand landmarks and the clean detection of the same frame, in units of the clean hand's size. A left/right label swap isn't counted as error, and frames lacking a hand on either side are skipped (dropout is measured separately). It measures tracking quality, which the presence mask (a world-model input) doesn't contain. That's why it's the candidate ground truth for comparing the gap score against a trivial dropout-counting baseline: with dropout as the ground truth, that baseline is the ground truth (ρ = 1).scripts/run_v1_pilot.py: for one clean recording and a severity grid, reports dropout, landmark error and the trivial baselines (no_hand_fraction,presence_density), plus how well the dropout baseline alone ranks landmark error. By design it computes no gap scores or gap correlations, so severities can't be chosen by looking at the tool's performance.v1_pilot/is gitignored.Tests:
tests/test_degradations.py(12):Verification
pytest -q: 175 passedruff check .: clean; the new script is100755🤖 Generated with Claude Code