Repository navigation
集成 dev 到 main:KohakuMossTrainer 与训练引擎更新(待 IryNeko 审查) - #519
Merged
Merged
Conversation
The 3.2.0 release merged dev into main, so dev now carries the release history; VERSION moves to the next milestone (AI Toolkit, see docs/roadmap.md).
…tus on the roadmap
…izer_config (#416) transformers >= 4.5x expects a mapping; a bare list aborted tokenizer loading with an opaque TypeError. Normalize on download (including a custom asset path), and again in the krea2 / ideogram4 source patches so manually placed tokenizers are healed before launch too.
vendor/sd-scripts builds the Anima DiT with num_blocks=28, so a 2.9B checkpoint only failed later with a wall of 'Unexpected keys ... blocks.28.*' names and the user could not tell they picked the wrong engine. validate_model now reads the safetensors header (no weights, no GPU), counts DiT blocks and returns an actionable message pointing at Anima Fast; unreadable headers are not treated as 2.9B and the Anima Fast path is untouched.
…on (#410) The fingerprint mixed in size+mtime_ns, so cloud images, Docker layers, backups and directory copies reported a healthy install as broken and pushed users into a multi-GB repair. It now hashes content (timestamps excluded, files >8 MiB pinned by size, symlinks by target), stores the scheme alongside the hash, and migrates a legacy state once instead of failing it; content changes are still detected.
…fix check (#390) absolute() called Path.resolve(), so an AutoDL-style '*.safetensors' name that points at an extension-less real file lost its suffix and weight_files() rejected a perfectly valid path. Normalise without following symlinks instead; the suffix check stays strict for genuinely wrong paths.
/api/run named the snapshot from the second alone, so two submissions inside the same second shared one file and the later write silently replaced the config the earlier task was about to read (agents submit back-to-back). The batch endpoint already carried a random suffix; single runs now use the same scheme.
…images (#338) Upstream bucket sampling only batches inside one bucket, so small aspect-ratio heavy datasets silently lose most images (one WARNING deep in the launch log) while the run finishes normally with far fewer steps. The preflight now says so up front, with the usual mitigations. #333 (caption_dropout_rate baked into the TE cache) is documented as a known limitation until the upstream cache key is versioned.
#435) A thin C# (NativeAOT, ~5 MB, no .NET runtime needed) shell over the existing portable layout: it delegates app startup to launch_portable.bat (which owns the embedded-Python environment policy), opens the browser once the GUI port answers, and owns the process tree. Keys: Q stops the tree, U stops + runs the package updater + offers to restart. Missing pieces produce a readable Chinese error instead of a console that vanishes. --selftest starts, waits for the port, stops and reports an exit code so packaging CI can assert it end-to-end. The .bat entries stay as the fallback entry points.
feat(portable): Windows 启动器 NextTrainer.exe(EXE 整合包第一阶段)
musubi-only, kohya-only and the resume flow now build and place NextTrainer.exe just like the full package. Packaging tests assert the wiring so a future build script cannot silently drop the launcher.
feat(portable): 启动器接入全部 2026 打包分支(#435 第二阶段)
… the controlled open prop (#440) DatasetEditorPage hides the picker's own button and drives the dialog through v-model:open, so toggle() never ran: the dialog opened with an empty dataset list even though the API returned the datasets. The controlled open transition now runs the same initialisation (source/serverPath/browse), and reopening refreshes without stacking requests.
fix(dataset): 标签编辑受控弹窗打开时不加载数据集(#440)
…443) The picker only distinguished "model does not support this engine" (greyed out), so an installed and a missing engine looked identical. Each option now carries a state suffix (ready / not installed / needs repair / installing / disabled / unverified), the unsupported hint stays, and a new actionable hint lists engines this model supports but that are missing or broken. enginesApi.list() failures degrade to plain engine names.
feat(ui): 训练引擎下拉区分「未安装」与「模型不支持」(#443)
…le input (#446) Both models keep the Comfy-format files the user already has; no manual diffusers conversion and no weight copies. - SD 1.5 (arch sd1): single_file mode loads Comfy/LDM checkpoints (model.diffusion_model.* / cond_stage_model.* / first_stage_model.*) through StableDiffusionPipeline.from_single_file. The driver binds a project-local diffusers config dir (configs + tokenizer only, default sd-models/sd15/config, downloadable from HF/ModelScope) and drops the legacy safety-checker fetch so the offline launch never touches the Hub. - Ideogram 4 (arch ideogram4): comfyui_files mode feeds the Comfy-Org conditional DiT + Flux2 VAE straight to the pinned upstream extension. The driver normalizes the FP8 weight_scale layout, drops comfy_quant markers and tolerates the original Flux2 VAE key layout in memory, so nothing is written to disk (no conversion cache). The frozen text encoder is the local bf16 Qwen3-VL-8B directory; non-commercial license labeling and no bundled weights are preserved. - Schemas, frontend module rows, i18n, asset registry entries, preflight facts/warnings and tests for both variants.
…gram4 feat(ai-toolkit): SD 1.5 与 Ideogram 4 LoRA(直接吃 Comfy 单文件,#446)
…ochs (#417 #418) - fp8_base_hint() reads only the safetensors header; an already-quantized DiT makes musubi reject --fp8_base/--fp8_scaled, so the adapter closes both and says why (both-off was always a legal state) - when max_train_steps and max_train_epochs are both filled, steps now win and epochs are dropped with an explicit warning (musubi prefers epochs upstream, which made steps=100 run far longer); form copy states the contract - 9 new unit tests cover both rules, unreadable checkpoints and the unchanged pairing behaviour
…mfy FP8 single file (#446) Upstream hands one path to both the tokenizer and the encoder, so the Comfy FP8 file could not be used; its generic mixin, however, already loads local single-file checkpoints with comfy_quant markers. Split the two paths in the pack driver (tokenizer/config from the local directory, weights from the file) and mirror it in the assets (TE single file + a weightless tokenizer dir), the schema and the preflight wording. Users no longer need the 16 GB bf16 TE. Fallback stays available: the model_directory branch and the asset override in config/model_assets.toml keep the bf16 directory path working.
…fy-te feat(ai-toolkit): Ideogram 4 文本编码器改用 Comfy FP8 单文件(#446 跟进)
Upstream's ideogram4 extension reads model_config.unconditional_lora_path and activates that LoRA only for the unconditional (negative) CFG pass, falling back to single-DiT CFG when it is null. Wire it as an optional asset (ostris' ideogram_4_unconditional_lora, 103.9 MB, labelled as third-party), an optional form field and a model_inputs passthrough; whitelist that field and text_encoder_tokenizer_path in the adapter (the latter used to produce a spurious unsupported-field warning) and keep both in the import round trip.
feat(ai-toolkit): Ideogram 4 无条件采样 LoRA 设为可选(#446 跟进)
…keys, notes) - TaskManager.wait_for_process: tolerate tasks the worker has not started yet (used to raise AttributeError) and wait for the recorded terminal state; unify the never-started check on getattr(task, 'process', None), which also removes an AttributeError path that could kill the compute worker when a Popen fails outright. - config/ai_toolkit_backend.toml: drop keys nothing reads for this pack (source_commit/external_root/external_python/preflight_level/ allow_unsupported/extension_root); the pin lives in manifest.UPSTREAM. - FIELD_NOTES: Windows install + sd15 real-machine rows, train_data_dir note.
chore(ai-toolkit): 真机遗留清理(wait_for_process / 死配置键 / FIELD_NOTES)
Feat/kohya main env fallback
Disk reads now ignore and preserve unknown fields at every level of settings.json and auth.json, so future field additions no longer break older readers. Writes stay strict: patch payloads with unknown keys or wrong types are still rejected, and schema_version remains the only breaking-change gate.
ComfyUI model conversion wrote tens of GB into the fixed project-side extensions/diffsynth/cache directory, filling AutoDL's ~30GB system disk even when source models live on a data disk. - paths.diffsynth_cache in user_data settings, NEXT_DIFFSYNTH_CACHE_DIR env var, and a per-task training form field; resolution order: task config > env > settings.json > project default - ensure_cache_space() estimates conversion bytes (skipping identity components and existing manifests) and fails with a readable message before training when the target filesystem is short - a submitted form value is remembered into settings.json so the next task inherits it server-side
…tings - training form gains the diffsynth_cache field via the schema, next to the model input mode where the conversion actually happens - engine manage dialog offers the persisted machine-level default via the existing user-data settings patch flow - zh-CN/en-US labels and a manage-dialog save test
…ions The tolerant-by-default _keys helper silently disabled the preset field allow-list, letting credential-looking top-level fields bypass the config-only credential check. Restore strict as the helper default so save_preset is protected again; settings validation keeps passing its own strictness explicitly. Patch-mode validation also required complete lastByModel selections, so updating only the target of a remembered model was rejected. Fields are now required only when validating the complete document; patches validate the fields they supply.
…codings The encoding cache derives its location from cache_dir's parent, so a custom path like /root/autodl-tmp placed encodings in /root/encodings and could fill the system disk again. The configured path is now a cache root: conversion writes to <root>/models and encodings land in <root>/encodings. The default layout is unchanged.
…ck restart On Linux the probe bound without SO_REUSEADDR, so recently closed client connections left the GUI port in TIME_WAIT and the next launch failed with PortConflictError for ~60s even though nothing was listening. Active listeners still fail the probe; only TIME_WAIT is ignored. Windows keeps SO_EXCLUSIVEADDRUSE.
Selectively reuse wochenlong/lora-scripts-next KohakuMoss@9ba0a081 and retain the pinned KohakuMossTrainer 5bb1a81 runtime. Preserve the existing engines, Next installer lifecycle and isolated environment. Expose Kohaku parameters and lifecycle UI; support literal-only task archives without executing configuration code. Validation: frontend Node 22 check (501 tests), targeted backend 212 passed / 2 skipped, additional regressions 64 passed with one pre-existing DiffSynth fixture failure reproduced on upstream dev. Real HTTP parameter smoke stops at task execution: no training. Windows install/GPU runtime and actual browser click are not verified. Co-authored-by: Codex <codex@openai.com>
chore(frontend): rebuild dist for MOSS-Image and dev UI updates
chore: bump VERSION to 3.3.1-dev
…r engine installs
…-dev fix(portable): bundle uv in requirements and resolve uv executable for engine installs
…precheck env lock
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
将 dev 的已集成更新提交到 main 审查,包含 #507 的 KohakuMossTrainer 适配。
改动范围
审查与合并条件
@IryNeko 请审查此 PR。合并 main 属于重要操作,请审查确认没有问题后再合并;本 PR 不启用自动合并。
重点请检查:
冲突解决与验证
已在 dev 合入 main,通过合并提交
549c898f799951d01cf994d7e52ad4b3b485f3fa解决分支冲突。GitHub 当前已确认无合并冲突。npm run check通过:类型检查、lint、75 个测试文件 / 505 项测试及生产构建。构建有已有的大 bundle / PURE annotation 提示。仍需 @IryNeko 审查确认后再合并到 main;未启用自动合并。
关联:#507(已合并到 dev);早期方案 #488 已关闭且未合并。