Skip to content

集成 dev 到 main:KohakuMossTrainer 与训练引擎更新(待 IryNeko 审查) - #519

Merged
wochenlong merged 121 commits into
mainfrom
dev
Nov 18, 2026
Merged

wochenlong merged 121 commits into
mainfrom
dev

Conversation

@wochenlong

@wochenlong wochenlong commented Nov 3, 2026 •

Copy link
Copy Markdown
Owner

将 dev 的已集成更新提交到 main 审查,包含 #507 的 KohakuMossTrainer 适配。

改动范围

  • 接入可选 KohakuMossTrainer 引擎:独立运行环境、安装/修复/卸载、MOSS-Image LoRA 表单、预检、任务进度与导出;来源与许可声明随代码保留。现有 Kohya 默认路径保留。
  • 同步 dev 中的引擎安装与下载源管理、训练工作台及任务页面更新、相关测试和打包调整。
  • 同步文档和来源致谢。此 PR 是 dev → main 的集成审查,并非仅 适配 KohakuMossTrainer(接续 #506,待审核) #507 的单独补丁。

审查与合并条件

@IryNeko 请审查此 PR。合并 main 属于重要操作,请审查确认没有问题后再合并;本 PR 不启用自动合并。

重点请检查:

  • KohakuMossTrainer 安装、配置映射、预检、训练启动/停止、日志和导出链路,以及独立环境与其它引擎的隔离。
  • main 已有的训练入口、保存配置、更新器及用户数据保留行为有无回归。
  • 分支冲突、CI 结果、前端产物与源码是否一致,及上游许可证/署名是否保留。

冲突解决与验证

已在 dev 合入 main,通过合并提交 549c898f799951d01cf994d7e52ad4b3b485f3fa 解决分支冲突。GitHub 当前已确认无合并冲突。

  • 保留 dev 的 KohakuMossTrainer 接入、训练工作台、独立环境升级及服务端下载源设置。
  • 保留 main 的新版下载/更新说明、Windows 路径兼容与 MOSS 补丁源码的 CRLF 校验修复;README 同时保留 KohakuMossTrainer 介绍及模型对应表。
  • 使用 Node 22.17.1 和现有 lockfile 重新构建前端,生成产物与 dev 原有产物一致。
  • npm run check 通过:类型检查、lint、75 个测试文件 / 505 项测试及生产构建。构建有已有的大 bundle / PURE annotation 提示。
  • 核心后端合同检查:135 passed / 1 skipped;跨模块回归:119 passed / 5 skipped。覆盖 KohakuMoss 生命周期、引擎注册/路由、Kohya/MOSS 环境、下载源、配置导入、DiffSynth 修复、任务页面及打包契约。
  • 补充 CRLF 内容校验回归测试;旧代理测试改为明确验证直连/手动策略,避免依赖宿主自动代理状态。
  • 大型合成权重测试在 Windows 上耗时较长,已中止,不计入通过项;本次未重新安装训练引擎、执行真实 GPU 训练或推理验收。适配 KohakuMossTrainer(接续 #506,待审核) #507 的历史短训范围仍以该 PR 和 docs/MOSS.md 为准。

仍需 @IryNeko 审查确认后再合并到 main;未启用自动合并。

关联:#507(已合并到 dev);早期方案 #488 已关闭且未合并。

wochenlong and others added 30 commits October 12, 2026 18:26
The 3.2.0 release merged dev into main, so dev now carries the release
history; VERSION moves to the next milestone (AI Toolkit, see docs/roadmap.md).
…izer_config (#416)

transformers >= 4.5x expects a mapping; a bare list aborted tokenizer loading
with an opaque TypeError. Normalize on download (including a custom asset path),
and again in the krea2 / ideogram4 source patches so manually placed tokenizers
are healed before launch too.
vendor/sd-scripts builds the Anima DiT with num_blocks=28, so a 2.9B checkpoint
only failed later with a wall of 'Unexpected keys ... blocks.28.*' names and the
user could not tell they picked the wrong engine. validate_model now reads the
safetensors header (no weights, no GPU), counts DiT blocks and returns an
actionable message pointing at Anima Fast; unreadable headers are not treated as
2.9B and the Anima Fast path is untouched.
…on (#410)

The fingerprint mixed in size+mtime_ns, so cloud images, Docker layers, backups
and directory copies reported a healthy install as broken and pushed users into
a multi-GB repair. It now hashes content (timestamps excluded, files >8 MiB
pinned by size, symlinks by target), stores the scheme alongside the hash, and
migrates a legacy state once instead of failing it; content changes are still
detected.
fix: 近一个月 issue 第一批(#416 tokenizer 兼容 / #408 Anima 2.9B 拦截 / #410 指纹去 mtime)
…fix check (#390)

absolute() called Path.resolve(), so an AutoDL-style '*.safetensors' name that
points at an extension-less real file lost its suffix and weight_files() rejected
a perfectly valid path. Normalise without following symlinks instead; the suffix
check stays strict for genuinely wrong paths.
/api/run named the snapshot from the second alone, so two submissions inside the
same second shared one file and the later write silently replaced the config the
earlier task was about to read (agents submit back-to-back). The batch endpoint
already carried a random suffix; single runs now use the same scheme.
…images (#338)

Upstream bucket sampling only batches inside one bucket, so small aspect-ratio
heavy datasets silently lose most images (one WARNING deep in the launch log)
while the run finishes normally with far fewer steps. The preflight now says so
up front, with the usual mitigations. #333 (caption_dropout_rate baked into the
TE cache) is documented as a known limitation until the upstream cache key is
versioned.
fix: 近一个月 issue 第二批(#390 软链路径 / #334 autosave 重名 / #338 丢图提示 / #333 限制写明)
#435)

A thin C# (NativeAOT, ~5 MB, no .NET runtime needed) shell over the existing
portable layout: it delegates app startup to launch_portable.bat (which owns the
embedded-Python environment policy), opens the browser once the GUI port
answers, and owns the process tree. Keys: Q stops the tree, U stops + runs the
package updater + offers to restart. Missing pieces produce a readable Chinese
error instead of a console that vanishes. --selftest starts, waits for the port,
stops and reports an exit code so packaging CI can assert it end-to-end.

The .bat entries stay as the fallback entry points.
feat(portable): Windows 启动器 NextTrainer.exe(EXE 整合包第一阶段)
musubi-only, kohya-only and the resume flow now build and place NextTrainer.exe
just like the full package. Packaging tests assert the wiring so a future build
script cannot silently drop the launcher.
feat(portable): 启动器接入全部 2026 打包分支(#435 第二阶段)
… the controlled open prop (#440)

DatasetEditorPage hides the picker's own button and drives the dialog through
v-model:open, so toggle() never ran: the dialog opened with an empty dataset
list even though the API returned the datasets. The controlled open transition
now runs the same initialisation (source/serverPath/browse), and reopening
refreshes without stacking requests.
fix(dataset): 标签编辑受控弹窗打开时不加载数据集(#440)
…443)

The picker only distinguished "model does not support this engine" (greyed out),
so an installed and a missing engine looked identical. Each option now carries a
state suffix (ready / not installed / needs repair / installing / disabled /
unverified), the unsupported hint stays, and a new actionable hint lists engines
this model supports but that are missing or broken. enginesApi.list() failures
degrade to plain engine names.
feat(ui): 训练引擎下拉区分「未安装」与「模型不支持」(#443)
…le input (#446)

Both models keep the Comfy-format files the user already has; no manual
diffusers conversion and no weight copies.

- SD 1.5 (arch sd1): single_file mode loads Comfy/LDM checkpoints
  (model.diffusion_model.* / cond_stage_model.* / first_stage_model.*)
  through StableDiffusionPipeline.from_single_file. The driver binds a
  project-local diffusers config dir (configs + tokenizer only, default
  sd-models/sd15/config, downloadable from HF/ModelScope) and drops the
  legacy safety-checker fetch so the offline launch never touches the Hub.
- Ideogram 4 (arch ideogram4): comfyui_files mode feeds the Comfy-Org
  conditional DiT + Flux2 VAE straight to the pinned upstream extension.
  The driver normalizes the FP8 weight_scale layout, drops comfy_quant
  markers and tolerates the original Flux2 VAE key layout in memory, so
  nothing is written to disk (no conversion cache). The frozen text
  encoder is the local bf16 Qwen3-VL-8B directory; non-commercial license
  labeling and no bundled weights are preserved.
- Schemas, frontend module rows, i18n, asset registry entries, preflight
  facts/warnings and tests for both variants.
…gram4

feat(ai-toolkit): SD 1.5 与 Ideogram 4 LoRA(直接吃 Comfy 单文件,#446)
…ochs (#417 #418)

- fp8_base_hint() reads only the safetensors header; an already-quantized DiT
  makes musubi reject --fp8_base/--fp8_scaled, so the adapter closes both and
  says why (both-off was always a legal state)
- when max_train_steps and max_train_epochs are both filled, steps now win and
  epochs are dropped with an explicit warning (musubi prefers epochs upstream,
  which made steps=100 run far longer); form copy states the contract
- 9 new unit tests cover both rules, unreadable checkpoints and the unchanged
  pairing behaviour
fix(musubi): FP8 底模守卫与 steps/epochs 优先级(#417 #418)
…mfy FP8 single file (#446)

Upstream hands one path to both the tokenizer and the encoder, so the Comfy FP8
file could not be used; its generic mixin, however, already loads local
single-file checkpoints with comfy_quant markers. Split the two paths in the
pack driver (tokenizer/config from the local directory, weights from the file)
and mirror it in the assets (TE single file + a weightless tokenizer dir), the
schema and the preflight wording. Users no longer need the 16 GB bf16 TE.

Fallback stays available: the model_directory branch and the asset override in
config/model_assets.toml keep the bf16 directory path working.
…fy-te

feat(ai-toolkit): Ideogram 4 文本编码器改用 Comfy FP8 单文件(#446 跟进)
Upstream's ideogram4 extension reads model_config.unconditional_lora_path and
activates that LoRA only for the unconditional (negative) CFG pass, falling back
to single-DiT CFG when it is null. Wire it as an optional asset (ostris'
ideogram_4_unconditional_lora, 103.9 MB, labelled as third-party), an optional
form field and a model_inputs passthrough; whitelist that field and
text_encoder_tokenizer_path in the adapter (the latter used to produce a
spurious unsupported-field warning) and keep both in the import round trip.
feat(ai-toolkit): Ideogram 4 无条件采样 LoRA 设为可选(#446 跟进)
…keys, notes)

- TaskManager.wait_for_process: tolerate tasks the worker has not started yet
  (used to raise AttributeError) and wait for the recorded terminal state;
  unify the never-started check on getattr(task, 'process', None), which also
  removes an AttributeError path that could kill the compute worker when a
  Popen fails outright.
- config/ai_toolkit_backend.toml: drop keys nothing reads for this pack
  (source_commit/external_root/external_python/preflight_level/
  allow_unsupported/extension_root); the pin lives in manifest.UPSTREAM.
- FIELD_NOTES: Windows install + sd15 real-machine rows, train_data_dir note.
chore(ai-toolkit): 真机遗留清理(wait_for_process / 死配置键 / FIELD_NOTES)
IryNeko and others added 27 commits October 21, 2026 00:23
Disk reads now ignore and preserve unknown fields at every level of
settings.json and auth.json, so future field additions no longer break
older readers. Writes stay strict: patch payloads with unknown keys or
wrong types are still rejected, and schema_version remains the only
breaking-change gate.
ComfyUI model conversion wrote tens of GB into the fixed project-side
extensions/diffsynth/cache directory, filling AutoDL's ~30GB system
disk even when source models live on a data disk.

- paths.diffsynth_cache in user_data settings, NEXT_DIFFSYNTH_CACHE_DIR
  env var, and a per-task training form field; resolution order:
  task config > env > settings.json > project default
- ensure_cache_space() estimates conversion bytes (skipping identity
  components and existing manifests) and fails with a readable message
  before training when the target filesystem is short
- a submitted form value is remembered into settings.json so the next
  task inherits it server-side
…tings

- training form gains the diffsynth_cache field via the schema, next to
  the model input mode where the conversion actually happens
- engine manage dialog offers the persisted machine-level default via
  the existing user-data settings patch flow
- zh-CN/en-US labels and a manage-dialog save test
…ions

The tolerant-by-default _keys helper silently disabled the preset field
allow-list, letting credential-looking top-level fields bypass the
config-only credential check. Restore strict as the helper default so
save_preset is protected again; settings validation keeps passing its
own strictness explicitly.

Patch-mode validation also required complete lastByModel selections,
so updating only the target of a remembered model was rejected. Fields
are now required only when validating the complete document; patches
validate the fields they supply.
…codings

The encoding cache derives its location from cache_dir's parent, so a
custom path like /root/autodl-tmp placed encodings in /root/encodings
and could fill the system disk again. The configured path is now a
cache root: conversion writes to <root>/models and encodings land in
<root>/encodings. The default layout is unchanged.
…ck restart

On Linux the probe bound without SO_REUSEADDR, so recently closed
client connections left the GUI port in TIME_WAIT and the next launch
failed with PortConflictError for ~60s even though nothing was
listening. Active listeners still fail the probe; only TIME_WAIT is
ignored. Windows keeps SO_EXCLUSIVEADDRUSE.
Selectively reuse wochenlong/lora-scripts-next KohakuMoss@9ba0a081
and retain the pinned KohakuMossTrainer 5bb1a81 runtime. Preserve the
existing engines, Next installer lifecycle and isolated environment.
Expose Kohaku parameters and lifecycle UI; support literal-only task
archives without executing configuration code.

Validation: frontend Node 22 check (501 tests), targeted backend
212 passed / 2 skipped, additional regressions 64 passed with one
pre-existing DiffSynth fixture failure reproduced on upstream dev.
Real HTTP parameter smoke stops at task execution: no training.
Windows install/GPU runtime and actual browser click are not verified.

Co-authored-by: Codex <codex@openai.com>
chore(frontend): rebuild dist for MOSS-Image and dev UI updates
…-dev

fix(portable): bundle uv in requirements and resolve uv executable for engine installs
@wochenlong
wochenlong requested a review from IryNeko November 3, 2026 14:31
@wochenlong
wochenlong merged commit f9e40aa into main Nov 18, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants