diff --git a/SPEC-ISSUES.md b/SPEC-ISSUES.md index 9d4d3e7..342d4a8 100644 --- a/SPEC-ISSUES.md +++ b/SPEC-ISSUES.md @@ -431,3 +431,45 @@ measured in a shared feature space. **Two consequences the spec should price in: feature (here: commit `primitive_frequency_mix`, SI-008) or state explicitly that its enrolment distance is single-dimensional and therefore that same-family collision is unguarded — because a one-feature signature has no minimum distance to defend once that feature is shared. + +## SI-023 · §11/§10 · decided-here — The field battery runs the pipeline UNMODIFIED on raw photos, and puts its human-readable label in the page margin + +Spec §11 makes L3 depend on recognition "under the hostile-conditions test battery (print, camera, +light, angle, damage)" but does not say (a) whether a conforming recogniser may *preprocess* a +photograph — deskew, colour-correct, denoise — before the "identical pipeline" of §8, or (b) where the +human-readable fallback §10 requires on an identity-bearing artefact may sit relative to the pattern. +The print-and-photograph path (`battery/printpack.py`, `battery/ingest/`) makes both concrete. +**Choice:** (a) v0 ingestion runs `recogniser.claim.recognise` on each photo with **no** photo-special +preprocessing — byte-for-byte the pipeline the synthetic battery uses — so a raw-phone-photo failure is +a *reported result* (a per-row status and a "not_recognised"/insufficient line in `summary.md`), never +silently repaired. A v0 that quietly pre-corrected photos could not measure what real capture costs +recognition, which is the entire L3 question; any preprocessing should be an explicit, declared, +reproducible stage the spec names, not an implementation convenience. (b) Each print sheet carries a +small label (surface id + seed + "print at 100%") in the **page margin**, whose bounding box is asserted +disjoint from the pattern rectangle (`test_ingest`), so nothing is added *inside* the surface — honouring +principle 2 / §2 ("no bounded marks, no fiducial code regions; the whole surface is the pattern") while +still providing the §10 human-readable fallback. Nothing on the label is measured or fed to the +recogniser; the photographer crops the pattern, not the label. **Spec consequence:** §11 should state +(i) whether field-battery recognition is defined on the raw capture or on a declared preprocessing +front-end (and if the latter, fix that stage so two conforming recognisers agree), and (ii) that a +required human-readable fallback lives outside the signature-bearing surface — a margin/label region is +not part of the pattern and MUST NOT be recovered or scored as if it were. + +## SI-024 · §11/§5 · decided-here — The field battery must gather ink-rich framings, or the diagonal-white-balance question is unanswerable + +`experiments/exp-002-cross-grammar` §6 sets three questions Phase 4b must answer, and the ingest +`summary.md` answers each from data or marks it "insufficient data". Question (a) — *is real camera white +balance diagonal?* — is answered from the iso-002 ink two-path working: the implied applied gain (1/g), +its in-bounds flag, and the per-channel rank correlation (SI-020). But the relationship colour path only +runs when a photo shows at least `MIN_INKS_FOR_GAIN=3` distinct inks (SI-020's credibility floor against +a per-channel gain overfitting a handful of colours). A field battery shot mostly at distance, or on +small fragments, can therefore recognise 002 surfaces yet **never accumulate a single row able to answer +(a)** — the capture protocol silently gates the conclusion. **Choice:** the printed 002 sheets are dense +12×8 grids (many inks per frame) and the INSTRUCTIONS ask explicitly for at least the "fills the frame" +condition per surface, so the ink-rich framings that (a) needs are captured by construction; the summary +still degrades to "insufficient data" honestly when they are absent. **Spec consequence:** §11 should +recognise that a hostile-conditions battery's *coverage requirements are feature-dependent* — a +relationship/ordering colour signature (Milestone 2, §5) can only be validated for illuminant-robustness +from captures rich enough to estimate the correction, so the field-battery protocol must state a minimum +per-feature framing (here: ≥3 inks in frame for the WB-diagonal check) rather than treating "photograph +the surface under varied conditions" as sufficient for every question the battery is meant to answer. diff --git a/battery/ingest/__init__.py b/battery/ingest/__init__.py index 3b66279..9635de9 100644 --- a/battery/ingest/__init__.py +++ b/battery/ingest/__init__.py @@ -1,111 +1,482 @@ -"""Phase-4b real-photo ingestion path -- STRUCTURE ONLY for this session. +"""Phase-4b real-photo ingestion path -- the field half of the L3 battery. The synthetic battery (``battery.run``) degrades generated fragments; the *field* -battery (spec s11 L3) needs real printed-and-photographed surfaces. This module -is the seam for that: given a folder of photos and a manifest describing each -one, it runs the IDENTICAL recognise pipeline and writes the SAME -``raw_results.csv`` shape, so synthetic and real rows are directly comparable. +battery (spec s11 L3: "print, camera, light, angle, damage") needs real +printed-and-photographed surfaces. This module is that path: given a folder of +photos and a manifest describing each one, it runs the IDENTICAL recognise +pipeline the synthetic battery uses -- **no photo-special preprocessing in v0**. +If recognition fails on a raw phone photo, that is a RESULT to record and report +(the summary marks it), never something to silently fix here; a v0 that quietly +pre-corrected photos could not tell us what real capture does to recognition, +which is the whole question (spec s11: "criteria deliberately unfinished until +that data exists"). -Print-and-photograph capture is explicitly out of scope this session; here the -plumbing is defined and unit-tested with a generated PNG standing in for a photo. +Each photo becomes one row of ``raw_results.csv``: the manifest fields verbatim, +a per-row status, the recogniser's top verdict, the target-sheet aggregate / +coverage / renormalised / verdict, the four bar-cascade-001 feature agreements +(for 001 rows), and -- for iso-002 rows -- the ink two-path detail INCLUDING the +implied applied gain (1/g). That gain block is what answers Phase-4b question (a): +the relationship colour path (SI-020) assumes a real illuminant is a per-channel +DIAGONAL gain; the implied applied gain recovers the white balance it removed, and +``ink_gain_in_bounds`` / ``ink_rank_correlation`` say whether a single diagonal +gain actually explained the cast. If those are consistently out of bounds or the +rank correlation is low across real photos, the diagonal model is wrong. + +A ``summary.md`` is auto-written alongside: per-condition tables plus the three +Phase-4b questions (from ``experiments/exp-002-cross-grammar`` s6), each answered +from the data present or explicitly marked "insufficient data". + +Robustness (a broken capture set must never crash the run): a manifest entry +whose file is missing, a file in the folder not named in the manifest, and an +unreadable/corrupt image are each recorded as a row ``status`` -- never an +exception. Only a malformed manifest file (not loadable YAML) is fatal. Manifest format (``manifest.yaml``):: - surface_id: bar-cascade-001 # optional default for every entry + surface_id: null # optional default surface_id for every entry photos: - - file: shot_0001.jpg # path relative to the photo folder - surface_id: bar-cascade-001 # which enrolled surface this depicts - conditions: # free-form capture metadata (logged) - light: office - angle_deg: 15 - print: laser - -Each photo becomes one CSV row: the recogniser's verdict + three numbers against -bar-cascade-001, plus the recorded capture conditions. Ground-truth fields that -only a generated fragment can know (``boundaries_spanned``, ``frac``) stay blank. + - file: IMG_0001.jpg # filename inside the photo folder + surface_id: 001-s0 # matches the printed label (printpack) + grammar: bar-cascade-001 # bar-cascade-001 | iso-002 (the target sheet) + conditions: + lighting: daylight + angle_deg: 0 + distance: fills_frame + printer: "Brand Model" + paper: plain + notes: "" """ from __future__ import annotations import argparse import csv +import subprocess +from datetime import datetime, timezone from pathlib import Path import yaml from recogniser.claim import recognise -from battery.run import CSV_FIELDS, GRAMMARS, TARGET_SHEET_ID, _result_for, _feature_agreement - - -def _row_from_claim(file_name, surface_id, conditions, claim) -> dict: - """One CSV row (battery.run.CSV_FIELDS shape) for a recognised real photo.""" - top = claim["results"][0] - r001 = _result_for(claim, TARGET_SHEET_ID) - cond_str = ";".join(f"{k}={v}" for k, v in sorted((conditions or {}).items())) - row = {k: "" for k in CSV_FIELDS} - row.update({ - "arm": "real", - "impostor_id": "", - "surface_seed": surface_id or "", - "degradation": "photo", - "degradation_param": cond_str, - "top_sheet": top["sheet_id"], - "top_aggregate": top["aggregate_confidence"], - "verdict_001": r001["verdict"] if r001 else "", - "aggregate_001": r001["aggregate_confidence"] if r001 else "", - "coverage_001": r001["coverage"] if r001 else "", - "renormalised_001": r001["renormalised_score"] if r001 else "", - "agr_cascade_ratio": _feature_agreement(r001, "cascade_ratio") if r001 else "", - "agr_duty": _feature_agreement(r001, "duty") if r001 else "", - "agr_phase_duty_identity": _feature_agreement(r001, "phase_duty_identity") if r001 else "", - "agr_colour_pair": _feature_agreement(r001, "colour_pair") if r001 else "", - }) - row["rotation_deg"] = (conditions or {}).get("angle_deg", "") + +REPO_ROOT = Path(__file__).resolve().parent.parent.parent +GRAMMARS = REPO_ROOT / "grammars" + +# Recognised image extensions when scanning the folder for stray files. +IMAGE_EXTS = {".jpg", ".jpeg", ".png", ".tif", ".tiff", ".bmp", ".webp", ".heic"} + +# The four bar-cascade-001 identification/normalisation feature agreements we log +# (mirrors battery.run.FEATURE_COLUMNS so 001 real rows line up with synthetic). +BAND_FEATURES = ["cascade_ratio", "duty", "phase_duty_identity", "colour_pair"] + +# Manifest condition keys logged verbatim (order fixes the CSV column order). +CONDITION_KEYS = ["lighting", "angle_deg", "distance", "printer", "paper"] + +CSV_FIELDS = [ + # provenance + status + "file", "status", "surface_id", "grammar", "notes", + *CONDITION_KEYS, + # recognition summary + "image_shape", "top_sheet", "top_aggregate", + "target_sheet", "verdict", "aggregate", "coverage", "renormalised", + # bar-cascade-001 per-feature agreements (blank for grid rows) + "agr_cascade_ratio", "agr_duty", "agr_phase_duty_identity", "agr_colour_pair", + # iso-002 ink two-path detail incl. implied applied gain (blank for band rows) + "ink_agreement_absolute", "ink_agreement_relationship", + "ink_mean_delta_e_absolute", "ink_mean_delta_e_relationship", + "ink_correction_gain_b", "ink_correction_gain_g", "ink_correction_gain_r", + "ink_implied_applied_gain_b", "ink_implied_applied_gain_g", + "ink_implied_applied_gain_r", + "ink_gain_in_bounds", "ink_relationship_applicable", + "ink_n_inks", "ink_n_clipped_inks", "ink_gain_fallback_channels", + "ink_rank_correlation", + # anything that went wrong (unreadable image message, etc.) + "error", +] + + +# --------------------------------------------------------------------------- # +# Claim -> row extraction +# --------------------------------------------------------------------------- # + +def _result_for(claim: dict, sheet_id: str): + for r in claim.get("results", []): + if r.get("sheet_id") == sheet_id: + return r + return None + + +def _feature_agreement(result: dict, fid: str): + if not result: + return None + for f in result.get("per_feature", []): + if f.get("id") == fid: + return f.get("agreement") + return None + + +def _ink_two_path_detail(result: dict): + """Return the grid ink-set two-path ``detail`` dict for ``result`` or None. + + Scans the per-feature working for the ``ink_set_match`` measure carrying the + white-balance relationship path (a ``correction_gain_bgr`` marks the grid + two-path detail; band sheets score colour absolute-only and have no gain). + """ + if not result: + return None + for f in result.get("per_feature", []): + if f.get("measure") == "ink_set_match": + detail = f.get("detail") or {} + if "correction_gain_bgr" in detail: + return detail + return None + + +def _base_row() -> dict: + return {k: "" for k in CSV_FIELDS} + + +def _row_from_claim(entry_row: dict, claim: dict, target_sheet: str) -> dict: + """Fill ``entry_row`` (already carrying manifest fields) from a recognise claim.""" + row = dict(entry_row) + row["status"] = "ok" + row["image_shape"] = "x".join(str(d) for d in claim.get("image_shape", [])) + results = claim.get("results", []) + if results: + row["top_sheet"] = results[0].get("sheet_id", "") + row["top_aggregate"] = results[0].get("aggregate_confidence", "") + + target = _result_for(claim, target_sheet) if target_sheet else None + if target is None and results: + # No declared grammar (or it is not an enrolled sheet): fall back to the + # recogniser's own top result so the row still carries a verdict. + target = results[0] + row["target_sheet"] = target.get("sheet_id", "") + else: + row["target_sheet"] = target_sheet or "" + + if target: + row["verdict"] = target.get("verdict", "") + row["aggregate"] = target.get("aggregate_confidence", "") + row["coverage"] = target.get("coverage", "") + row["renormalised"] = target.get("renormalised_score", "") + for fid in BAND_FEATURES: + agr = _feature_agreement(target, fid) + if agr is not None: + row[f"agr_{fid}"] = agr + + detail = _ink_two_path_detail(target) + if detail: + gain = detail.get("correction_gain_bgr", [None, None, None]) + impl = detail.get("implied_applied_gain_bgr", [None, None, None]) + clip = detail.get("clipping", {}) + row.update({ + "ink_agreement_absolute": detail.get("agreement_absolute", ""), + "ink_agreement_relationship": detail.get("agreement_relationship", ""), + "ink_mean_delta_e_absolute": detail.get("mean_delta_e_absolute", ""), + "ink_mean_delta_e_relationship": detail.get("mean_delta_e_relationship", ""), + "ink_correction_gain_b": gain[0], "ink_correction_gain_g": gain[1], + "ink_correction_gain_r": gain[2], + "ink_implied_applied_gain_b": impl[0], + "ink_implied_applied_gain_g": impl[1], + "ink_implied_applied_gain_r": impl[2], + "ink_gain_in_bounds": detail.get("gain_in_bounds", ""), + "ink_relationship_applicable": detail.get("relationship_applicable", ""), + "ink_n_inks": detail.get("n_inks", ""), + "ink_n_clipped_inks": clip.get("n_clipped_inks", ""), + "ink_gain_fallback_channels": ",".join(clip.get("gain_fallback_channels", [])), + "ink_rank_correlation": detail.get("rank_correlation", ""), + }) + return row + + +# --------------------------------------------------------------------------- # +# Ingest +# --------------------------------------------------------------------------- # + +def _entry_row(entry: dict, default_surface: str) -> dict: + """Build the manifest-field portion of a row (no recognition yet).""" + row = _base_row() + row["file"] = entry.get("file", "") + row["surface_id"] = entry.get("surface_id", default_surface) or "" + row["grammar"] = entry.get("grammar", "") or "" + row["notes"] = entry.get("notes", "") or "" + conditions = entry.get("conditions") or {} + for k in CONDITION_KEYS: + v = conditions.get(k, "") + row[k] = "" if v is None else v return row -def ingest(folder, manifest_path, out_csv=None, grammars_dir=None) -> list[dict]: - """Recognise every photo listed in ``manifest_path`` under ``folder``. +def ingest(folder, manifest_path, out_dir=None, *, grammars_dir=None, + timestamp=None) -> list[dict]: + """Recognise every photo in ``manifest_path`` under ``folder``; return rows. + + Runs the identical ``recognise`` pipeline used by the synthetic battery. When + ``out_dir`` is given, writes ``raw_results.csv`` and ``summary.md`` there. - Runs the identical ``recognise`` pipeline used by the synthetic battery and - (when ``out_csv`` is given) writes rows in the same ``raw_results.csv`` shape. - Returns the list of row dicts. Raises ``FileNotFoundError`` for a missing - photo so a broken manifest fails loudly rather than silently skipping. + A missing photo, a folder file absent from the manifest, and an unreadable + image are each recorded as a row ``status`` (``missing`` / ``not_in_manifest`` + / ``unreadable``) rather than raising -- a broken capture set never crashes + the run. A manifest that will not parse as YAML IS fatal (that is operator + error, not capture data). """ folder = Path(folder) + grammars = str(grammars_dir) if grammars_dir else str(GRAMMARS) with open(manifest_path) as fh: manifest = yaml.safe_load(fh) or {} default_surface = manifest.get("surface_id") - grammars = str(grammars_dir) if grammars_dir else str(GRAMMARS) - rows = [] - for entry in manifest.get("photos", []): - file_name = entry["file"] - path = folder / file_name - if not path.exists(): - raise FileNotFoundError(f"photo listed in manifest not found: {path}") - surface_id = entry.get("surface_id", default_surface) - conditions = entry.get("conditions", {}) - claim = recognise(str(path), grammars) - rows.append(_row_from_claim(file_name, surface_id, conditions, claim)) - - if out_csv is not None: - with open(out_csv, "w", newline="") as fh: + rows: list[dict] = [] + listed_files = set() + for entry in manifest.get("photos", []) or []: + entry_row = _entry_row(entry, default_surface) + file_name = entry_row["file"] + listed_files.add(file_name) + target_sheet = entry_row["grammar"] + path = folder / file_name if file_name else None + + if not file_name: + entry_row["status"] = "missing" + entry_row["error"] = "manifest entry has no 'file'" + rows.append(entry_row) + continue + if path is None or not path.exists(): + entry_row["status"] = "missing" + entry_row["error"] = f"file not found under {folder}" + rows.append(entry_row) + continue + try: + claim = recognise(str(path), grammars) + except Exception as exc: # unreadable/corrupt image, decode failure, etc. + entry_row["status"] = "unreadable" + entry_row["error"] = f"{type(exc).__name__}: {exc}" + rows.append(entry_row) + continue + rows.append(_row_from_claim(entry_row, claim, target_sheet)) + + # Files present in the folder but never named in the manifest -> flagged. + for p in sorted(folder.glob("*")) if folder.exists() else []: + if p.is_file() and p.suffix.lower() in IMAGE_EXTS and p.name not in listed_files: + stray = _base_row() + stray["file"] = p.name + stray["status"] = "not_in_manifest" + stray["error"] = "image in folder not listed in manifest" + rows.append(stray) + + if out_dir is not None: + out_dir = Path(out_dir) + out_dir.mkdir(parents=True, exist_ok=True) + with open(out_dir / "raw_results.csv", "w", newline="") as fh: writer = csv.DictWriter(fh, fieldnames=CSV_FIELDS) writer.writeheader() writer.writerows(rows) + ts = timestamp or datetime.now(timezone.utc).isoformat() + (out_dir / "summary.md").write_text(build_summary(rows, ts)) + return rows +# --------------------------------------------------------------------------- # +# summary.md +# --------------------------------------------------------------------------- # + +def _git_commit() -> str: + try: + return subprocess.check_output( + ["git", "rev-parse", "HEAD"], cwd=str(REPO_ROOT), text=True).strip() + except Exception: + return "unknown" + + +def _num(v): + """Coerce a CSV cell to float, or None if blank/non-numeric.""" + if v is None or v == "" or isinstance(v, bool): + return None + try: + return float(v) + except (TypeError, ValueError): + return None + + +def _ok_rows(rows): + return [r for r in rows if r.get("status") == "ok"] + + +def _fmt(v, nd=3): + return "--" if v is None else f"{v:.{nd}f}" + + +def _mean(values): + vals = [v for v in values if v is not None] + return sum(vals) / len(vals) if vals else None + + +def _verdict_counts(rows): + counts = {"identified": 0, "candidate": 0, "not_recognised": 0, "other": 0} + for r in rows: + v = r.get("verdict", "") + counts[v if v in counts else "other"] += 1 + return counts + + +def _condition_table(rows, key, title): + """A markdown table of mean aggregate + verdict mix grouped by one condition.""" + groups: dict = {} + for r in rows: + groups.setdefault(str(r.get(key, "")), []).append(r) + lines = [f"**By {title}**", "", + f"| {title} | n | mean aggregate | identified | candidate | not_recognised |", + "|---|---|---|---|---|---|"] + for g in sorted(groups): + grp = groups[g] + vc = _verdict_counts(grp) + magg = _mean([_num(r.get("aggregate")) for r in grp]) + lines.append(f"| {g or '(blank)'} | {len(grp)} | {_fmt(magg)} | " + f"{vc['identified']} | {vc['candidate']} | {vc['not_recognised']} |") + lines.append("") + return "\n".join(lines) + + +def _answer_wb_diagonal(grid_rows): + """Question (a): is real camera white balance diagonal?""" + usable = [r for r in grid_rows + if str(r.get("ink_relationship_applicable")).lower() == "true"] + if not usable: + return ("**(a) Is real camera white balance diagonal?** insufficient data " + "-- no iso-002 photo recognised with the relationship colour path " + "applicable (need >= 3 inks in frame; SI-020).") + in_bounds = sum(1 for r in usable + if str(r.get("ink_gain_in_bounds")).lower() == "true") + mean_rank = _mean([_num(r.get("ink_rank_correlation")) for r in usable]) + frac = in_bounds / len(usable) + verdict = ("consistent with a per-channel diagonal gain" + if frac >= 0.8 and (mean_rank or 0) >= 0.8 + else "NOT well explained by a diagonal gain (the SI-020 model is " + "too weak for real illuminants)") + return (f"**(a) Is real camera white balance diagonal?** {verdict}. " + f"{in_bounds}/{len(usable)} recognised iso-002 photos had an in-bounds " + f"single diagonal correction gain (GAIN_MIN..GAIN_MAX), mean per-channel " + f"rank correlation {_fmt(mean_rank)}. A positive diagonal gain is " + f"order-preserving, so high rank correlation corroborates the diagonal " + f"model; low values (or many out-of-bounds gains) say real white balance " + f"is not diagonal and the relationship path needs a richer transform.") + + +def _answer_clipping(grid_rows): + """Question (b): real clipping behaviour on bright inks.""" + counts = [_num(r.get("ink_n_clipped_inks")) for r in grid_rows] + counts = [c for c in counts if c is not None] + if not counts: + return ("**(b) Real clipping on bright inks?** insufficient data -- no " + "iso-002 photo produced a measured ink set to inspect for clipping.") + total_clipped = sum(int(c) for c in counts) + n_with_clip = sum(1 for c in counts if c > 0) + return (f"**(b) Real clipping on bright inks?** across {len(counts)} recognised " + f"iso-002 photos, {n_with_clip} showed >= 1 clipped ink " + f"(mean {_fmt(_mean(counts), 2)} clipped inks/photo, {total_clipped} total). " + f"Clipping destroys a bright ink's channel asymmetrically (only a lower " + f"bound survives; SI-020), so a rising clipped-ink count under warm/bright " + f"lighting is the s2 strong-warm edge appearing on real ink -- cross-check " + f"against the by-lighting table above.") + + +def _answer_gamut(grid_rows): + """Question (c): does 002's ink set survive the print gamut?""" + if not grid_rows: + return ("**(c) Does 002's ink set survive the print gamut?** insufficient " + "data -- no iso-002 photo was successfully recognised.") + vc = _verdict_counts(grid_rows) + survived = vc["identified"] + vc["candidate"] + mean_abs = _mean([_num(r.get("ink_agreement_absolute")) for r in grid_rows]) + mean_rel = _mean([_num(r.get("ink_agreement_relationship")) for r in grid_rows]) + return (f"**(c) Does 002's ink set survive the print gamut?** {survived}/" + f"{len(grid_rows)} recognised iso-002 photos reached candidate or better " + f"(identified {vc['identified']}, candidate {vc['candidate']}, " + f"not_recognised {vc['not_recognised']}). Mean ink agreement: absolute " + f"path {_fmt(mean_abs)}, relationship path {_fmt(mean_rel)}. If the " + f"absolute path collapses but the relationship path holds, the print " + f"shifted the inks by a recoverable cast; if BOTH collapse, the print " + f"gamut moved the inks out of delta-E tolerance and 002's single " + f"colour-borne identity (SI-008/SI-022) does not survive this print.") + + +def build_summary(rows, timestamp: str) -> str: + """Render summary.md from the ingested rows (deterministic given ``timestamp``).""" + ok = _ok_rows(rows) + band_rows = [r for r in ok if r.get("grammar") == "bar-cascade-001"] + grid_rows = [r for r in ok if r.get("grammar") == "iso-002"] + + status_counts: dict = {} + for r in rows: + status_counts[r.get("status", "")] = status_counts.get(r.get("status", ""), 0) + 1 + + L = [] + L.append("# exp-003 print-and-photograph -- ingest summary") + L.append("") + L.append(f"- generated: {timestamp}") + L.append(f"- git commit: {_git_commit()}") + L.append(f"- recogniser: v0 (identical pipeline; no photo-special preprocessing)") + L.append(f"- rows: {len(rows)} " + f"(" + ", ".join(f"{k}={v}" for k, v in sorted(status_counts.items())) + ")") + L.append(f"- recognised: bar-cascade-001 {len(band_rows)}, iso-002 {len(grid_rows)}") + L.append("") + + if not ok: + L.append("No photo was successfully recognised, so every Phase-4b question " + "below is answered **insufficient data**. Check the per-row status " + "in raw_results.csv (missing / unreadable / not_in_manifest).") + L.append("") + + L.append("## Per-condition tables") + L.append("") + if ok: + L.append(_condition_table(ok, "lighting", "lighting")) + L.append(_condition_table(ok, "angle_deg", "angle (deg)")) + L.append(_condition_table(ok, "distance", "distance")) + L.append(_condition_table(ok, "surface_id", "surface")) + else: + L.append("_(no recognised rows to tabulate)_") + L.append("") + + L.append("## The three Phase-4b questions (exp-002 s6)") + L.append("") + L.append(_answer_wb_diagonal(grid_rows)) + L.append("") + L.append(_answer_clipping(grid_rows)) + L.append("") + L.append(_answer_gamut(grid_rows)) + L.append("") + L.append("---") + L.append("") + L.append("Each answer is computed only from photos that recognised; a question " + "with no supporting rows is marked *insufficient data* rather than " + "guessed. L3 (spec s11) stays open until this runs on a real, " + "sufficiently-covered capture set (SI-017, SI-023).") + L.append("") + return "\n".join(L) + + +# --------------------------------------------------------------------------- # +# CLI +# --------------------------------------------------------------------------- # + def _main(argv=None) -> int: parser = argparse.ArgumentParser( description="Ingest photographed surfaces and recognise them (Phase 4b).") parser.add_argument("folder", help="folder of photo files") parser.add_argument("manifest", help="manifest.yaml listing the photos") - parser.add_argument("--out", default=None, help="output raw_results.csv path") + parser.add_argument("--out", default=None, + help="output directory for raw_results.csv + summary.md") + parser.add_argument("--grammars", default=None, + help="grammar sheet directory (default: repo grammars/)") args = parser.parse_args(argv) - rows = ingest(args.folder, args.manifest, out_csv=args.out) - print(f"ingested {len(rows)} photo(s)") + rows = ingest(args.folder, args.manifest, out_dir=args.out, + grammars_dir=args.grammars) + ok = sum(1 for r in rows if r["status"] == "ok") + print(f"ingested {len(rows)} row(s); {ok} recognised") + if args.out: + print(f"wrote {args.out}/raw_results.csv and {args.out}/summary.md") return 0 diff --git a/battery/ingest/__main__.py b/battery/ingest/__main__.py new file mode 100644 index 0000000..e608ac8 --- /dev/null +++ b/battery/ingest/__main__.py @@ -0,0 +1,6 @@ +"""``python -m battery.ingest`` entry point (Phase 4b real-photo ingestion).""" + +from battery.ingest import _main + +if __name__ == "__main__": + raise SystemExit(_main()) diff --git a/battery/printpack.py b/battery/printpack.py new file mode 100644 index 0000000..a1f9366 --- /dev/null +++ b/battery/printpack.py @@ -0,0 +1,364 @@ +"""Print-pack generator for the print-and-photograph field battery (Phase 4b). + +Spec s11 L3 ("Field-proven") depends on recognition "under the hostile-conditions +test battery (print, camera, light, angle, damage)". The synthetic battery +(``battery.run``) stands in for those axes deterministically; this module produces +the *fixed physical input* for the real thing: a small set of A4 sheets a person +prints at 100% and photographs. It is the experiment's committed input, so the +generated ``print-pack/`` is version-controlled, not gitignored. + +What it emits (deterministically -- README principle 4): + + * six single-page PNGs sized EXACTLY for A4 at 300 dpi (2480 x 3508 px): + - three bar-cascade-001 surfaces (seeds 0, 1, 2), ~170 mm square, 5 bands; + - three iso-002 surfaces (seeds 0, 1, 2; each seed picks a different ink + subset), a 12 x 8 cell grid ~170 mm wide. + Each surface is centred with a white margin, and a small human-readable label + is placed in the BOTTOM PAGE MARGIN, strictly outside the pattern rectangle. + * ``print-pack/INSTRUCTIONS.md`` -- the print/photograph/manifest protocol. + * ``photos/manifest.template.yaml`` (+ an empty ``photos/`` for the captures). + +On the label vs the spec's "no marks inside the pattern" rule (spec s2, principle +2: "no payload encoding, no bounded marks, no fiducial code regions -- the whole +surface is the pattern"): the label is NOT part of the surface. It sits in the +page margin, never touching the pattern rectangle (``test_ingest`` asserts the two +bounding boxes are disjoint), and carries only human bookkeeping (which seed this +is, and "print at 100%"). Nothing about it is measured or fed to the recogniser -- +the recogniser only ever sees a photograph of the pattern rectangle, cropped by +the photographer. It is exactly the "human-readable fallback" the spec calls a +conformance requirement for identity-bearing artefacts (spec s10 Accessibility), +kept off the signature-bearing surface on purpose. +""" + +from __future__ import annotations + +import argparse +from pathlib import Path + +import cv2 +import numpy as np + +from sheets import load_sheet +from generator import cascade +from generator import grid + +REPO_ROOT = Path(__file__).resolve().parent.parent +GRAMMARS = REPO_ROOT / "grammars" +SHEET_001 = GRAMMARS / "bar-cascade-001.yaml" +SHEET_002 = GRAMMARS / "iso-002.yaml" + +# --- A4 at 300 dpi (portrait) ------------------------------------------------ +DPI = 300 +PAGE_W = 2480 # 210 mm * 300 / 25.4, rounded +PAGE_H = 3508 # 297 mm * 300 / 25.4, rounded +MM = DPI / 25.4 # pixels per millimetre + +# Pattern target width ~170 mm at 300 dpi. +PATTERN_PX = round(170 * MM) # 2008 px + +# 001: 5 bands across ~170 mm -> a band module of ~402 px (5 * 402 = 2010 px). +BANDS_001 = 5 +MODULE_001 = round(PATTERN_PX / BANDS_001) # 402 + +# 002: 12 x 8 cells, ~170 mm across the 12 columns -> module ~167 px. +COLS_002, ROWS_002 = 12, 8 +MODULE_002 = round(PATTERN_PX / COLS_002) # 167 + +# A bottom band of the page reserved for the label (never overlaps the pattern). +LABEL_BAND_PX = 380 +SEEDS = (0, 1, 2) + +_FONT = cv2.FONT_HERSHEY_SIMPLEX + + +def _surface_specs(): + """Return the ordered list of surface specs (grammar x seed) to emit. + + Each spec is a plain dict; ``render`` is a zero-arg callable returning the + (H, W, 3) BGR pattern array so page assembly is generator-agnostic. + """ + sheet_001 = load_sheet(SHEET_001) + sheet_002 = load_sheet(SHEET_002) + specs = [] + for seed in SEEDS: + specs.append({ + "surface_id": f"001-s{seed}", + "grammar": "bar-cascade-001", + "seed": seed, + "render": (lambda s=seed: cascade.render( + sheet_001, n_bands=BANDS_001, module_px=MODULE_001, seed=s)), + }) + for seed in SEEDS: + specs.append({ + "surface_id": f"002-s{seed}", + "grammar": "iso-002", + "seed": seed, + "render": (lambda s=seed: grid.render( + sheet_002, cols=COLS_002, rows=ROWS_002, module_px=MODULE_002, seed=s)), + }) + return specs + + +def _label_lines(spec) -> list[str]: + """The two human-readable label lines drawn in the bottom page margin.""" + return [ + f"{spec['surface_id']} seed {spec['seed']} grammar {spec['grammar']}", + "PRINT AT 100% / ACTUAL SIZE - no fit-to-page, no scaling", + ] + + +def build_page(spec) -> tuple[np.ndarray, dict]: + """Compose one A4 page for ``spec``; return (page_bgr, meta). + + The pattern is centred horizontally and centred vertically in the region + ABOVE a reserved bottom label band, so the label bounding box is always + strictly below (disjoint from) the pattern rectangle. ``meta`` records both + bounding boxes so the disjointness is machine-checkable (test requirement). + """ + pattern = spec["render"]() + ph, pw = pattern.shape[:2] + + page = np.full((PAGE_H, PAGE_W, 3), 255, np.uint8) + + avail_h = PAGE_H - LABEL_BAND_PX + x0 = (PAGE_W - pw) // 2 + y0 = (avail_h - ph) // 2 + if x0 < 0 or y0 < 0: + raise ValueError( + f"pattern {pw}x{ph} does not fit A4 with the reserved label band") + x1, y1 = x0 + pw, y0 + ph + page[y0:y1, x0:x1] = pattern + pattern_bbox = (x0, y0, x1, y1) + + # --- label in the bottom band, horizontally centred, disjoint from pattern + lines = _label_lines(spec) + scale, thickness = 1.4, 3 + sizes = [cv2.getTextSize(t, _FONT, scale, thickness) for t in lines] + line_gap = 26 + text_h = sum(s[0][1] for s in sizes) + line_gap * (len(lines) - 1) + band_top = avail_h + block_top = band_top + (LABEL_BAND_PX - text_h) // 2 + + ly = block_top + label_x0, label_x1 = PAGE_W, 0 + label_y0 = block_top + for text, ((tw, th), _base) in zip(lines, sizes): + tx = (PAGE_W - tw) // 2 + ly += th + cv2.putText(page, text, (tx, ly), _FONT, scale, (0, 0, 0), thickness, + cv2.LINE_AA) + label_x0 = min(label_x0, tx) + label_x1 = max(label_x1, tx + tw) + ly += line_gap + label_bbox = (label_x0, label_y0, label_x1, block_top + text_h) + + meta = { + "surface_id": spec["surface_id"], + "grammar": spec["grammar"], + "seed": spec["seed"], + "pattern_bbox": pattern_bbox, + "label_bbox": label_bbox, + "pattern_shape": [int(ph), int(pw)], + } + return page, meta + + +def _bboxes_disjoint(a, b) -> bool: + """True if axis-aligned boxes (x0,y0,x1,y1) do not overlap.""" + ax0, ay0, ax1, ay1 = a + bx0, by0, bx1, by1 = b + return ax1 <= bx0 or bx1 <= ax0 or ay1 <= by0 or by1 <= ay0 + + +INSTRUCTIONS = """\ +# Print pack -- Phase 4b (print-and-photograph field battery) + +These six sheets are the fixed physical input for the L3 "field-proven" test +(spec s11). You print them, photograph them under a spread of real conditions, +and the recogniser is run on the photos UNCHANGED. The point is to find out what +real print + camera + light + angle do to recognition -- so photograph honestly, +including the awkward conditions. A photo that fails to recognise is a RESULT, not +a mistake; do not retake it to "make it work". + +## 1. Print (do this once, note what you used) + +- Print every PNG in this folder at **100% / actual size**. In the print dialog: + turn **OFF** "fit to page", "shrink to fit" and any scaling -- it MUST say 100%. +- Use **plain white paper** for the first pass. (If you want a second pass on + matte photo paper later, do it -- just record `paper:` in the manifest.) +- Write down your **printer model** -- you will put it in the manifest. +- The label in the bottom margin of each sheet says which surface it is + (e.g. `001-s0`) and "print at 100%". That label is NOT part of the pattern; it + is only there so you can tell the sheets apart. Do not photograph it as if it + were the pattern -- frame the pattern square/rectangle itself. + +## 2. Photograph (the condition matrix) + +For **each** of the six printed sheets, take a spread of photos covering: + +- **3 lightings:** `daylight` (near a window, no direct sun on the paper) · + `warm_indoor` (a warm bulb / tungsten-ish room light) · + `cool_led_or_shade` (cool white LED, or open shade outdoors). +- **3 angles:** `0` (straight on, phone parallel to paper) · `30` (~30 deg + tilt) · `60` (~60 deg tilt -- a steep, hostile angle). +- **2 distances:** `fills_frame` (the pattern fills most of the frame) · + `far_2m` (stand back ~2 m; the pattern is small in the frame). + +That is 3 x 3 x 2 = 18 photos per sheet if you do the full matrix. If time is +short, prioritise: all three lightings straight-on and filling the frame first, +then add angles and distance. Partial coverage is fine -- the summary marks any +question it lacks data for as "insufficient data" rather than guessing. + +Phone camera: use the **default** camera app, **auto** everything (auto WB, auto +exposure, HDR as it comes). Do NOT use a "document scan" mode -- that flattens +lighting and defeats the point. Hold steady; a blurred shot is a blurred shot. + +## 3. File naming + +Names carry NO meaning -- the manifest does. Let your phone name them +(`IMG_0001.jpg` ...), or use anything you like. Just make each filename unique and +put every file in the `photos/` folder. + +## 4. Fill the manifest (this is the important part) + +Copy `photos/manifest.template.yaml` to `photos/manifest.yaml` and add **one +entry per photo**. Each entry ties a filename to the surface it shows and the +conditions you shot it under: + +```yaml +photos: + - file: IMG_0001.jpg # exact filename in photos/ + surface_id: 001-s0 # the id printed on the sheet's bottom label + grammar: bar-cascade-001 # bar-cascade-001 for 001-* sheets, iso-002 for 002-* + conditions: + lighting: daylight # daylight | warm_indoor | cool_led_or_shade + angle_deg: 0 # 0 | 30 | 60 + distance: fills_frame # fills_frame | far_2m + printer: "Brand Model 123" # your printer + paper: plain # plain | matte_photo + notes: "" # anything worth remembering; free-form +``` + +`surface_id` MUST match the printed label exactly, and `grammar` MUST be +`bar-cascade-001` or `iso-002`. Everything else is logged verbatim. + +## 5. Run the recogniser over the photos + +From the repo root: + +```sh +uv run python -m battery.ingest photos/ photos/manifest.yaml --out experiments/exp-003-print-photo/ +``` + +That reads the manifest, runs the **identical** recognise pipeline on each photo +(no photo-special preprocessing), and writes `raw_results.csv` (one row per photo, +with the recogniser's verdict and, for the 002 sheets, the white-balance two-path +detail) and `summary.md` (per-condition tables plus the three Phase-4b questions, +each answered from the data or marked "insufficient data"). A missing file, a file +not in the manifest, or an unreadable image is recorded as a row status -- the run +never crashes. +""" + +MANIFEST_TEMPLATE = """\ +# Manifest for the print-and-photograph field battery (Phase 4b). +# +# Copy this file to photos/manifest.yaml and add one entry per photo you take. +# - file: the exact filename in the photos/ folder (names carry no meaning). +# - surface_id: MUST match the label printed in the sheet's bottom margin +# (001-s0..001-s2 for the bar-cascade-001 sheets, 002-s0..002-s2 +# for the iso-002 sheets). +# - grammar: bar-cascade-001 (for the 001-* sheets) or iso-002 (for the 002-*). +# - conditions: how you shot it; logged verbatim into raw_results.csv. +# lighting: daylight | warm_indoor | cool_led_or_shade +# angle_deg: 0 | 30 | 60 (straight-on / ~30 deg / ~60 deg) +# distance: fills_frame | far_2m +# printer: your printer model (free text) +# paper: plain | matte_photo | ... +# - notes: free-form. +# +# The two entries below are EXAMPLES -- delete them and add your own. + +surface_id: null # optional default surface_id for entries that omit their own +photos: + - file: IMG_0001.jpg + surface_id: 001-s0 + grammar: bar-cascade-001 + conditions: + lighting: daylight + angle_deg: 0 + distance: fills_frame + printer: "" + paper: plain + notes: "" + - file: IMG_0002.jpg + surface_id: 002-s0 + grammar: iso-002 + conditions: + lighting: warm_indoor + angle_deg: 30 + distance: fills_frame + printer: "" + paper: plain + notes: "example second entry" +""" + + +def generate(out_dir, photos_dir=None) -> dict: + """Generate the whole print pack. Returns a small summary dict. + + Writes the six page PNGs and INSTRUCTIONS.md under ``out_dir`` (default + ``print-pack/``) and the manifest template + an empty capture folder under + ``photos_dir`` (default ``/photos``). Deterministic: identical bytes on + every run for identical grammars/generators. + """ + out_dir = Path(out_dir) + out_dir.mkdir(parents=True, exist_ok=True) + photos_dir = Path(photos_dir) if photos_dir else (REPO_ROOT / "photos") + photos_dir.mkdir(parents=True, exist_ok=True) + + pages = [] + for spec in _surface_specs(): + page, meta = build_page(spec) + # Cheap invariant: the label never touches the pattern (spec s2). + if not _bboxes_disjoint(meta["pattern_bbox"], meta["label_bbox"]): + raise AssertionError( + f"label overlaps pattern for {meta['surface_id']}") + path = out_dir / f"{spec['surface_id']}.png" + if not cv2.imwrite(str(path), page): + raise IOError(f"failed to write {path}") + meta["png"] = str(path) + pages.append(meta) + + (out_dir / "INSTRUCTIONS.md").write_text(INSTRUCTIONS) + (photos_dir / "manifest.template.yaml").write_text(MANIFEST_TEMPLATE) + # Keep the (otherwise empty) capture folder in version control. + (photos_dir / ".gitkeep").write_text("") + + return { + "out_dir": str(out_dir), + "photos_dir": str(photos_dir), + "pages": pages, + "page_size": [PAGE_W, PAGE_H], + } + + +def _main(argv=None) -> int: + parser = argparse.ArgumentParser( + description="Generate the print-and-photograph print pack (Phase 4b).") + parser.add_argument("--out", default="print-pack", + help="output directory for the page PNGs + INSTRUCTIONS") + parser.add_argument("--photos", default=None, + help="folder for the manifest template + captures " + "(default: /photos)") + args = parser.parse_args(argv) + summary = generate(args.out, args.photos) + print(f"wrote {len(summary['pages'])} pages to {summary['out_dir']} " + f"(A4 {summary['page_size'][0]}x{summary['page_size'][1]} px)") + for p in summary["pages"]: + print(f" {p['surface_id']}: {p['png']}") + print(f"manifest template + captures: {summary['photos_dir']}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(_main()) diff --git a/photos/.gitkeep b/photos/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/photos/manifest.template.yaml b/photos/manifest.template.yaml new file mode 100644 index 0000000..42f26a1 --- /dev/null +++ b/photos/manifest.template.yaml @@ -0,0 +1,40 @@ +# Manifest for the print-and-photograph field battery (Phase 4b). +# +# Copy this file to photos/manifest.yaml and add one entry per photo you take. +# - file: the exact filename in the photos/ folder (names carry no meaning). +# - surface_id: MUST match the label printed in the sheet's bottom margin +# (001-s0..001-s2 for the bar-cascade-001 sheets, 002-s0..002-s2 +# for the iso-002 sheets). +# - grammar: bar-cascade-001 (for the 001-* sheets) or iso-002 (for the 002-*). +# - conditions: how you shot it; logged verbatim into raw_results.csv. +# lighting: daylight | warm_indoor | cool_led_or_shade +# angle_deg: 0 | 30 | 60 (straight-on / ~30 deg / ~60 deg) +# distance: fills_frame | far_2m +# printer: your printer model (free text) +# paper: plain | matte_photo | ... +# - notes: free-form. +# +# The two entries below are EXAMPLES -- delete them and add your own. + +surface_id: null # optional default surface_id for entries that omit their own +photos: + - file: IMG_0001.jpg + surface_id: 001-s0 + grammar: bar-cascade-001 + conditions: + lighting: daylight + angle_deg: 0 + distance: fills_frame + printer: "" + paper: plain + notes: "" + - file: IMG_0002.jpg + surface_id: 002-s0 + grammar: iso-002 + conditions: + lighting: warm_indoor + angle_deg: 30 + distance: fills_frame + printer: "" + paper: plain + notes: "example second entry" diff --git a/print-pack/001-s0.png b/print-pack/001-s0.png new file mode 100644 index 0000000..f616080 Binary files /dev/null and b/print-pack/001-s0.png differ diff --git a/print-pack/001-s1.png b/print-pack/001-s1.png new file mode 100644 index 0000000..0b28949 Binary files /dev/null and b/print-pack/001-s1.png differ diff --git a/print-pack/001-s2.png b/print-pack/001-s2.png new file mode 100644 index 0000000..bef22f3 Binary files /dev/null and b/print-pack/001-s2.png differ diff --git a/print-pack/002-s0.png b/print-pack/002-s0.png new file mode 100644 index 0000000..aa41cfc Binary files /dev/null and b/print-pack/002-s0.png differ diff --git a/print-pack/002-s1.png b/print-pack/002-s1.png new file mode 100644 index 0000000..f4ea390 Binary files /dev/null and b/print-pack/002-s1.png differ diff --git a/print-pack/002-s2.png b/print-pack/002-s2.png new file mode 100644 index 0000000..89bfaa0 Binary files /dev/null and b/print-pack/002-s2.png differ diff --git a/print-pack/INSTRUCTIONS.md b/print-pack/INSTRUCTIONS.md new file mode 100644 index 0000000..1478dfd --- /dev/null +++ b/print-pack/INSTRUCTIONS.md @@ -0,0 +1,86 @@ +# Print pack -- Phase 4b (print-and-photograph field battery) + +These six sheets are the fixed physical input for the L3 "field-proven" test +(spec s11). You print them, photograph them under a spread of real conditions, +and the recogniser is run on the photos UNCHANGED. The point is to find out what +real print + camera + light + angle do to recognition -- so photograph honestly, +including the awkward conditions. A photo that fails to recognise is a RESULT, not +a mistake; do not retake it to "make it work". + +## 1. Print (do this once, note what you used) + +- Print every PNG in this folder at **100% / actual size**. In the print dialog: + turn **OFF** "fit to page", "shrink to fit" and any scaling -- it MUST say 100%. +- Use **plain white paper** for the first pass. (If you want a second pass on + matte photo paper later, do it -- just record `paper:` in the manifest.) +- Write down your **printer model** -- you will put it in the manifest. +- The label in the bottom margin of each sheet says which surface it is + (e.g. `001-s0`) and "print at 100%". That label is NOT part of the pattern; it + is only there so you can tell the sheets apart. Do not photograph it as if it + were the pattern -- frame the pattern square/rectangle itself. + +## 2. Photograph (the condition matrix) + +For **each** of the six printed sheets, take a spread of photos covering: + +- **3 lightings:** `daylight` (near a window, no direct sun on the paper) · + `warm_indoor` (a warm bulb / tungsten-ish room light) · + `cool_led_or_shade` (cool white LED, or open shade outdoors). +- **3 angles:** `0` (straight on, phone parallel to paper) · `30` (~30 deg + tilt) · `60` (~60 deg tilt -- a steep, hostile angle). +- **2 distances:** `fills_frame` (the pattern fills most of the frame) · + `far_2m` (stand back ~2 m; the pattern is small in the frame). + +That is 3 x 3 x 2 = 18 photos per sheet if you do the full matrix. If time is +short, prioritise: all three lightings straight-on and filling the frame first, +then add angles and distance. Partial coverage is fine -- the summary marks any +question it lacks data for as "insufficient data" rather than guessing. + +Phone camera: use the **default** camera app, **auto** everything (auto WB, auto +exposure, HDR as it comes). Do NOT use a "document scan" mode -- that flattens +lighting and defeats the point. Hold steady; a blurred shot is a blurred shot. + +## 3. File naming + +Names carry NO meaning -- the manifest does. Let your phone name them +(`IMG_0001.jpg` ...), or use anything you like. Just make each filename unique and +put every file in the `photos/` folder. + +## 4. Fill the manifest (this is the important part) + +Copy `photos/manifest.template.yaml` to `photos/manifest.yaml` and add **one +entry per photo**. Each entry ties a filename to the surface it shows and the +conditions you shot it under: + +```yaml +photos: + - file: IMG_0001.jpg # exact filename in photos/ + surface_id: 001-s0 # the id printed on the sheet's bottom label + grammar: bar-cascade-001 # bar-cascade-001 for 001-* sheets, iso-002 for 002-* + conditions: + lighting: daylight # daylight | warm_indoor | cool_led_or_shade + angle_deg: 0 # 0 | 30 | 60 + distance: fills_frame # fills_frame | far_2m + printer: "Brand Model 123" # your printer + paper: plain # plain | matte_photo + notes: "" # anything worth remembering; free-form +``` + +`surface_id` MUST match the printed label exactly, and `grammar` MUST be +`bar-cascade-001` or `iso-002`. Everything else is logged verbatim. + +## 5. Run the recogniser over the photos + +From the repo root: + +```sh +uv run python -m battery.ingest photos/ photos/manifest.yaml --out experiments/exp-003-print-photo/ +``` + +That reads the manifest, runs the **identical** recognise pipeline on each photo +(no photo-special preprocessing), and writes `raw_results.csv` (one row per photo, +with the recogniser's verdict and, for the 002 sheets, the white-balance two-path +detail) and `summary.md` (per-condition tables plus the three Phase-4b questions, +each answered from the data or marked "insufficient data"). A missing file, a file +not in the manifest, or an unreadable image is recorded as a row status -- the run +never crashes. diff --git a/tests/test_battery.py b/tests/test_battery.py index 83c363d..39ed450 100644 --- a/tests/test_battery.py +++ b/tests/test_battery.py @@ -1,6 +1,6 @@ -"""Tests for the Phase-4 test battery: degradations, harness, ingest. +"""Tests for the Phase-4 test battery: degradations and the synthetic harness. -Three parts, mirroring the project rule that machinery is tested before it is +Two parts, mirroring the project rule that machinery is tested before it is trusted: 1. Degradation unit tests -- each transform preserves shape/dtype, is @@ -10,8 +10,9 @@ 2. A ``--quick`` end-to-end battery run into a tmp dir: asserts the manifest, CSV and curve PNGs exist, the CSV has the declared columns and a non-trivial row count, and the run is reproducible. - 3. Ingest structure: a generated PNG stands in for a photo and is recognised - through the identical pipeline, producing a raw_results-shaped row. + +The Phase-4b real-photo ingestion path (``battery.ingest``) and the print pack +(``battery.printpack``) have their own suite in ``tests/test_ingest.py``. """ import csv @@ -25,7 +26,6 @@ from generator.fragments import sample_fragment from battery import degrade from battery.run import BatteryConfig, quick_config, run_battery, CSV_FIELDS -from battery.ingest import ingest REPO_ROOT = Path(__file__).resolve().parent.parent SHEET_PATH = REPO_ROOT / "grammars" / "bar-cascade-001.yaml" @@ -176,44 +176,3 @@ def test_impostors_do_not_reach_identified_in_quick_run(tmp_path): rows = list(csv.DictReader(fh)) impostor_verdicts = {r["verdict_001"] for r in rows if r["arm"] == "impostor"} assert "identified" not in impostor_verdicts - - -# ========================================================================= -# 3. Ingest structure (a generated PNG stands in for a photo) -# ========================================================================= - -def test_ingest_recognises_photo_and_writes_csv(tmp_path): - sheet = load_sheet(SHEET_PATH) - photo = tmp_path / "shot_0001.png" - # Render at module 200 (as the canonical recogniser test does) so a clean - # full surface reaches 'identified'; 160 tops out at 'candidate'. - cascade.render_png(sheet, photo, n_bands=N_BANDS, module_px=200, seed=0, - orientation_deg=0.0) - manifest = tmp_path / "manifest.yaml" - manifest.write_text( - "surface_id: bar-cascade-001\n" - "photos:\n" - " - file: shot_0001.png\n" - " surface_id: bar-cascade-001\n" - " conditions:\n" - " light: office\n" - " angle_deg: 0\n" - ) - out_csv = tmp_path / "raw_results.csv" - rows = ingest(tmp_path, manifest, out_csv=out_csv) - assert len(rows) == 1 - row = rows[0] - assert row["arm"] == "real" - assert row["top_sheet"] == "bar-cascade-001" - assert row["verdict_001"] == "identified" # a clean full render is identified - # CSV shape matches the synthetic battery exactly. - with open(out_csv, newline="") as fh: - reader = csv.DictReader(fh) - assert reader.fieldnames == CSV_FIELDS - - -def test_ingest_missing_photo_raises(tmp_path): - manifest = tmp_path / "manifest.yaml" - manifest.write_text("photos:\n - file: nope.png\n surface_id: bar-cascade-001\n") - with pytest.raises(FileNotFoundError): - ingest(tmp_path, manifest) diff --git a/tests/test_ingest.py b/tests/test_ingest.py new file mode 100644 index 0000000..a0bea1b --- /dev/null +++ b/tests/test_ingest.py @@ -0,0 +1,236 @@ +"""Tests for the Phase-4b print pack + real-photo ingestion path. + +Two machines, tested before they are trusted (project rule): + + 1. ``battery.printpack`` -- the fixed physical input. Asserts determinism + (byte-identical pages on re-run), exact A4-at-300dpi dimensions, and that the + bottom-margin label bounding box is DISJOINT from the pattern rectangle (the + spec-s2 "no marks inside the pattern" guarantee, machine-checked). + 2. ``battery.ingest`` -- the real-photo path, exercised with SYNTHETIC stand-in + "photos": generated surfaces run through perspective_warp + white_balance + + jpeg_roundtrip and saved as .jpg, then ingested through the IDENTICAL + recognise pipeline. Asserts the CSV/summary shape, that mild-condition + stand-ins recognise, that the recovered implied-applied gain tracks the cast + that was applied (question (a) plumbing), and the never-crash robustness + (missing file / stray file / unreadable image -> row status, not exception). +""" + +import csv + +import cv2 +import numpy as np +import pytest + +from pathlib import Path + +from sheets import load_sheet +from generator import cascade, grid +from battery import degrade +from battery import printpack +from battery.ingest import ingest, CSV_FIELDS, build_summary + +REPO_ROOT = Path(__file__).resolve().parent.parent +SHEET_001 = REPO_ROOT / "grammars" / "bar-cascade-001.yaml" +SHEET_002 = REPO_ROOT / "grammars" / "iso-002.yaml" +TS = "2026-01-01T00:00:00+00:00" + + +# ========================================================================= +# 1. Print pack +# ========================================================================= + +def test_printpack_pages_are_a4_at_300dpi(): + for spec in printpack._surface_specs(): + page, meta = printpack.build_page(spec) + assert page.shape == (printpack.PAGE_H, printpack.PAGE_W, 3) + assert (printpack.PAGE_W, printpack.PAGE_H) == (2480, 3508) + assert page.dtype == np.uint8 + + +def test_printpack_emits_six_surfaces(): + specs = printpack._surface_specs() + ids = [s["surface_id"] for s in specs] + assert ids == ["001-s0", "001-s1", "001-s2", "002-s0", "002-s1", "002-s2"] + grammars = {s["grammar"] for s in specs} + assert grammars == {"bar-cascade-001", "iso-002"} + + +def test_printpack_label_bbox_disjoint_from_pattern(): + """Spec s2: no marks inside the pattern. The bottom-margin label must never + touch the pattern rectangle -- assert the two bounding boxes are disjoint.""" + for spec in printpack._surface_specs(): + _page, meta = printpack.build_page(spec) + pat = meta["pattern_bbox"] + lab = meta["label_bbox"] + assert printpack._bboxes_disjoint(pat, lab), spec["surface_id"] + # And specifically: the label sits strictly BELOW the pattern. + assert lab[1] >= pat[3], spec["surface_id"] + + +def test_printpack_is_deterministic(tmp_path): + a, b = tmp_path / "a", tmp_path / "b" + printpack.generate(a, photos_dir=a / "photos") + printpack.generate(b, photos_dir=b / "photos") + for name in ("001-s0.png", "001-s1.png", "002-s0.png", "002-s2.png"): + assert (a / name).read_bytes() == (b / name).read_bytes(), name + + +def test_printpack_generate_writes_pack_and_manifest(tmp_path): + photos = tmp_path / "photos" + summary = printpack.generate(tmp_path / "pack", photos_dir=photos) + assert len(summary["pages"]) == 6 + assert (tmp_path / "pack" / "INSTRUCTIONS.md").exists() + assert (photos / "manifest.template.yaml").exists() + for p in summary["pages"]: + assert Path(p["png"]).exists() + + +# ========================================================================= +# 2. Ingest -- synthetic stand-in photos +# ========================================================================= + +def _save_jpg(path, image, quality=92): + cv2.imwrite(str(path), image, [int(cv2.IMWRITE_JPEG_QUALITY), quality]) + + +def _make_photos(folder): + """Write four synthetic stand-in 'photos' and return the manifest text. + + Two mild-condition captures (one per grammar) that must recognise, plus a + harsher 001 capture, plus a manifest entry for a file that does not exist. + """ + folder = Path(folder) + s1 = load_sheet(SHEET_001) + s2 = load_sheet(SHEET_002) + + # Render at test-friendly sizes (fast) that still recognise clean. + surf1 = cascade.render(s1, n_bands=5, module_px=200, seed=0) + surf2 = grid.render(s2, cols=12, rows=8, module_px=72, seed=0) + + # Mild 001: tiny warp, near-neutral cast, high-quality jpeg. + mild1 = degrade.jpeg_roundtrip( + degrade.white_balance(degrade.perspective_warp(surf1, 0.01, seed=1), + (1.05, 1.0, 0.95)), 92) + # Mild 002: tiny warp, a warm cast the relationship path should recover. + mild2 = degrade.jpeg_roundtrip( + degrade.white_balance(degrade.perspective_warp(surf2, 0.01, seed=2), + (1.1, 1.0, 0.9)), 92) + # Harsher 001: stronger warp + heavier compression. + harsh1 = degrade.jpeg_roundtrip( + degrade.perspective_warp(surf1, 0.05, seed=3), 40) + + _save_jpg(folder / "IMG_1.jpg", mild1) + _save_jpg(folder / "IMG_2.jpg", mild2) + _save_jpg(folder / "IMG_3.jpg", harsh1) + + manifest = folder / "manifest.yaml" + manifest.write_text( + "surface_id: null\n" + "photos:\n" + " - file: IMG_1.jpg\n" + " surface_id: 001-s0\n" + " grammar: bar-cascade-001\n" + " conditions: {lighting: daylight, angle_deg: 0, distance: fills_frame," + " printer: TestPrinter, paper: plain}\n" + " notes: mild\n" + " - file: IMG_2.jpg\n" + " surface_id: 002-s0\n" + " grammar: iso-002\n" + " conditions: {lighting: warm_indoor, angle_deg: 0, distance: fills_frame," + " printer: TestPrinter, paper: plain}\n" + " notes: mild\n" + " - file: IMG_3.jpg\n" + " surface_id: 001-s0\n" + " grammar: bar-cascade-001\n" + " conditions: {lighting: cool_led_or_shade, angle_deg: 60, distance: far_2m," + " printer: TestPrinter, paper: plain}\n" + " notes: harsh\n" + " - file: MISSING.jpg\n" + " surface_id: 001-s1\n" + " grammar: bar-cascade-001\n" + ) + return manifest + + +def test_ingest_end_to_end_shape_and_recognition(tmp_path): + manifest = _make_photos(tmp_path) + out = tmp_path / "out" + rows = ingest(tmp_path, manifest, out_dir=out, timestamp=TS) + + # One row per manifest entry (4). + by_file = {r["file"]: r for r in rows} + assert set(by_file) >= {"IMG_1.jpg", "IMG_2.jpg", "IMG_3.jpg", "MISSING.jpg"} + + # CSV + summary exist with the declared shape. + with open(out / "raw_results.csv", newline="") as fh: + reader = csv.DictReader(fh) + assert reader.fieldnames == CSV_FIELDS + csv_rows = list(reader) + assert len(csv_rows) == len(rows) + summary = (out / "summary.md").read_text() + assert "Phase-4b questions" in summary + for tag in ("(a)", "(b)", "(c)"): + assert tag in summary + + # Mild stand-ins recognise: 001 identified, 002 at least candidate. + assert by_file["IMG_1.jpg"]["status"] == "ok" + assert by_file["IMG_1.jpg"]["top_sheet"] == "bar-cascade-001" + assert by_file["IMG_1.jpg"]["verdict"] in ("identified", "candidate") + assert by_file["IMG_2.jpg"]["status"] == "ok" + assert by_file["IMG_2.jpg"]["top_sheet"] == "iso-002" + assert by_file["IMG_2.jpg"]["verdict"] in ("identified", "candidate") + + # Missing file recorded as a status, never a crash. + assert by_file["MISSING.jpg"]["status"] == "missing" + assert by_file["MISSING.jpg"]["error"] + + +def test_ingest_records_implied_gain_for_grid(tmp_path): + """Question (a) plumbing: the recovered implied-applied gain tracks the cast + that was applied to the 002 stand-in (warm cast (1.1, 1.0, 0.9)).""" + manifest = _make_photos(tmp_path) + rows = ingest(tmp_path, manifest, timestamp=TS) + grid_row = next(r for r in rows if r["file"] == "IMG_2.jpg") + b = float(grid_row["ink_implied_applied_gain_b"]) + g = float(grid_row["ink_implied_applied_gain_g"]) + r = float(grid_row["ink_implied_applied_gain_r"]) + # Recovered cast is in the right direction: blue boosted, red cut, green ~1. + assert b > 1.0 > r + assert g == pytest.approx(1.0, abs=0.1) + assert str(grid_row["ink_gain_in_bounds"]).lower() == "true" + + +def test_ingest_flags_stray_and_unreadable(tmp_path): + # A stray image not in the manifest, and a corrupt file that will not decode. + (tmp_path / "stray.jpg").write_bytes(cv2.imencode( + ".jpg", np.full((32, 32, 3), 127, np.uint8))[1].tobytes()) + (tmp_path / "broken.png").write_bytes(b"not a real image") + manifest = tmp_path / "manifest.yaml" + manifest.write_text( + "photos:\n" + " - file: broken.png\n" + " surface_id: 001-s0\n" + " grammar: bar-cascade-001\n" + ) + rows = ingest(tmp_path, manifest) # must not raise + by_file = {r["file"]: r for r in rows} + assert by_file["broken.png"]["status"] == "unreadable" + assert by_file["stray.jpg"]["status"] == "not_in_manifest" + + +def test_ingest_empty_manifest_gives_insufficient_summary(tmp_path): + manifest = tmp_path / "manifest.yaml" + manifest.write_text("photos: []\n") + out = tmp_path / "out" + rows = ingest(tmp_path, manifest, out_dir=out, timestamp=TS) + assert rows == [] + summary = (out / "summary.md").read_text() + assert "insufficient data" in summary + + +def test_build_summary_marks_insufficient_when_nothing_recognised(): + rows = [{"file": "x.jpg", "status": "missing", "grammar": "iso-002", + "verdict": "", "aggregate": ""}] + summary = build_summary(rows, TS) + # All three questions fall back to insufficient data with no ok rows. + assert summary.count("insufficient data") >= 3