Skip to content

Phase 2C.1: run SD 1.5 teaching-image parameter ablation - #30

Draft
xueyang-dev wants to merge 5 commits into
mainfrom
codex/phase2c1-sd15-ablation
Draft

xueyang-dev wants to merge 5 commits into
mainfrom
codex/phase2c1-sd15-ablation

Conversation

@xueyang-dev

Copy link
Copy Markdown
Owner

Summary

This Draft PR adds an evaluation-only SD 1.5 ablation experiment. It carries the controlled benchmark runner needed from the earlier Phase 2C.1 slice, then adds a fixed 6-case × 3-configuration × 3-seed experiment (54 images).

  • A: current Euler/normal, 20 steps, CFG 7, fixed production prompt profile.
  • B: DPM++ 2M/Karras, 30 steps, CFG 6, fixed production prompt profile.
  • C: B plus the evaluation-only ablation-clean-modern-v1 prompt profile.
  • Uses only the controlled TeachingImageRequest entry point and the fixed core-node graph.
  • Supports pause/resume, retries, idempotent artifact checks, identity fail-closed, provenance binding, technical prechecks, and ignored output directories.
  • Produces a 54-image comparison HTML page plus blank teacher-review JSON/CSV exports.

Real results

  • Pilot: 9/9 real technical successes.
  • Full run phase2c1-ablation-1785135763-338802ab: 54/54 real technical successes, 0 failures, 0 pending/invalidated.
  • A: 18/18, mean 35.011s; B: 18/18, mean 49.012s; C: 18/18, mean 45.585s.
  • All PNG, dimension, hash, provenance, request/config, identity, nonblank, and Asset Manifest checks passed; 0 near-solid warnings.
  • Review package contains 54 images and 54 pending_review records; 0 teacher records imported and no teacher scores or failure tags were fabricated.
  • Preliminary engineering spot-checks are documented as provisional only; no automatic teaching-quality conclusion is claimed.

Review package

runtime/phase2c1-sd15-ablation-full/review-package/ (ignored):

  • comparison.html — A/B/C comparison by case and seed;
  • experiment-manifest.json — fixed contract and artifact/provenance index;
  • teacher-reviews.pending.json and teacher-reviews.csv — blank manual review fields;
  • images/ — 54 generated review copies.

Validation

  • npm test: 659 API tests passed, 3 skipped; Vite production build passed; Web state tests passed.
  • npm run test:e2e: 14 passed on the rerun after one existing route-fetch race.
  • Focused ablation/benchmark tests: 11 passed; Ruff passed; git diff --check passed.
  • Real macOS arm64 opt-in Runtime/model lifecycle exercised with the fixed SD 1.5 model.

Scope boundaries

No production Model Package, Workflow Pack, Provider Hub default, sampler, or prompt profile was changed. No model replacement, LoRA, ControlNet, custom node, arbitrary graph, cloud fallback, generated PNG, Runtime, model, cache, or report is committed. The decision between prompt/sampler changes, restricting SD 1.5, or evaluating a stronger model remains gated on real teacher review and is not applied by this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant