[Bug] TAESD Bug

[lastrosade]

Git commit

git rev-parse HEAD
6103d86

Operating System & Version

Windows 11

GGML backends

HIP

Command-line arguments used

-M img_gen -t 6 -m "uncannyValley_Noob3dV3.safetensors" --vae "sdxl_vae.safetensors" --taesd "diffusion_pytorch_model.safetensors" -i "20200817145326_1.jpg" -o image.png -p "masterpiece, best quality, amazing quality, very aesthetic, absurdres, 1girl, long hair, photoshop (medium), glowing, blue theme, dark, waterfall, cave" -n "" --cfg-scale 1 --scheduler gits --sampling-method euler_a --steps 12 --strength 0.35 -H 768 -W 1344 --seed 42 -b 1 --clip-skip 2 --canny --color

Steps to reproduce

Ran ti2i over different builds of sd.cpp using joytag generated prompts
Older build: compiled 2025.09.16
Newer build: 6103d86

Using this image:

What you expected to happen

The older build yields:

What actually happened

The newer build yields:

Logs / error messages / stack trace

No response

Additional context / environment details

GPU: 7900 XT; gfx1100
I build using this script:

set CC=%HIP_PATH%\bin\clang.exe
set CXX=%HIP_PATH%\bin\clang++.exe

set PATH=%HIP_PATH%\bin;%PATH%

cmake .. -G "Ninja" -DGGML_OPENMP=OFF -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DSD_HIPBLAS=ON -DCMAKE_BUILD_TYPE=Release -DAMDGPU_TARGETS=gfx1100
cmake --build . --config Release

xcopy "%HIP_PATH%\bin\hipblas.dll" "bin\hipblas.dll*" /Y
xcopy "%HIP_PATH%\bin\rocblas.dll" "bin\rocblas.dll*" /Y
Robocopy "%HIP_PATH%\bin\rocblas" bin/rocblas /MIR

pause

[lastrosade]

To note, I have only had this issue for a short while, I waited for c42826b (Issue: #925) before trying again, it did not fix the issue.

[stduhpf]

Can you share the full logs (with -v flag)? Does this same model work in txt2img mode?

[lastrosade]

This model works in txt2img mode, this issue is repeatable with every model I have tried (5 or so, all SDXL).

Full Arguments: -M img_gen -t 6 -m "<redacted>\novaRealityXL_illustriousV80.safetensors" --vae "<redacted>\sdxl_vae.safetensors" --taesd "<redacted>\diffusion_pytorch_model.safetensors" --lora-model-dir "<redacted>LoRa" -i "<redacted>_\temp.png" -o "<redacted>image.png" -p "<lora:sdxl/dmd2_sdxl_4step_lora_fp16:1> masterpiece, best quality, amazing quality, very aesthetic, high resolution, ultra-detailed, absurdres, scenery, photorealistic, 1girl, long hair, photoshop (medium), sword, official art, water, night, glowing, nature, forest, blue theme, lantern, dark, waterfall, cave" -n "modern, old, oldest, cartoon, graphic, text, painting, crayon, graphite, abstract, glitch, deformed, mutated, ugly, disfigured, long body, lowres, bad anatomy, bad hands, missing fingers, extra digits, fewer digits, very displeasing, (worst quality, bad quality:1.2), bad anatomy, sketch, jpeg artifacts, signature, watermark, username, simple background, conjoined, bad ai-generated" --cfg-scale 1 --scheduler karras --sampling-method lcm --steps 8 --strength 0.45 -H 768 -W 1344 --seed 42 -b 1 --clip-skip 2 --canny --color --verbose

Older version

Option:
n_threads:                         6
    mode:                              img_gen
    model_path:                        <redacted>\novaRealityXL_illustriousV80.safetensors
    wtype:                             unspecified
    clip_l_path:
    clip_g_path:
    clip_vision_path:
    t5xxl_path:
    diffusion_model_path:
    high_noise_diffusion_model_path:
    vae_path:                          <redacted>\sdxl_vae.safetensors
    taesd_path:                        <redacted>\diffusion_pytorch_model.safetensors
    esrgan_path:
    control_net_path:
    embedding_dir:
    stacked_id_embed_dir:
    input_id_images_path:
    style ratio:                       20.00
    normalize input image:             false
    output_path:                       <redacted>image.png
    init_image_path:                   <redacted>_\temp.png
    end_image_path:
    mask_image_path:
    control_image_path:
    ref_images_paths:
    offload_params_to_cpu:             false
    clip_on_cpu:                       false
    control_net_cpu:                   false
    vae_on_cpu:                        false
    diffusion flash attention:         false
    diffusion Conv2d direct:           false
    vae_conv_direct:                   false
    control_strength:                  0.90
    prompt:                            <lora:sdxl/dmd2_sdxl_4step_lora_fp16:1> masterpiece, best quality, amazing quality, very aesthetic, high resolution, ultra-detailed, absurdres, scenery, photorealistic, 1girl, long hair, photoshop (medium), sword, official art, water, night, glowing, nature, forest, blue theme, lantern, dark, waterfall, cave
    negative_prompt:                   modern, old, oldest, cartoon, graphic, text, painting, crayon, graphite, abstract, glitch, deformed, mutated, ugly, disfigured, long body, lowres, bad anatomy, bad hands, missing fingers, extra digits, fewer digits, very displeasing, (worst quality, bad quality:1.2), bad anatomy, sketch, jpeg artifacts, signature, watermark, username, simple background, conjoined, bad ai-generated
    clip_skip:                         2
    width:                             1344
    height:                            768
    sample_params:                     (txt_cfg: 1.00, img_cfg: 1.00, distilled_guidance: 3.50, slg.layer_count: 3, slg.layer_start: 0.01, slg.layer_end: 0.20, slg.scale: 0.00, scheduler: karras, sample_method: lcm, sample_steps: 8, eta: 0.00)
    high_noise_sample_params:          (txt_cfg: 7.00, img_cfg: 7.00, distilled_guidance: 3.50, slg.layer_count: 3, slg.layer_start: 0.01, slg.layer_end: 0.20, slg.scale: 0.00, scheduler: default, sample_method: euler_a, sample_steps: 20, eta: 0.00)
    strength(img2img):                 0.45
    rng:                               cuda
    seed:                              42
    batch_count:                       1
    vae_tiling:                        false
    upscale_repeats:                   1
    chroma_use_dit_mask:               true
    chroma_use_t5_mask:                false
    chroma_t5_mask_pad:                1
    video_frames:                      1
    fps:                               16
System Info:
    SSE3 = 1
    AVX = 1
    AVX2 = 1
    AVX512 = 0
    AVX512_VBMI = 0
    AVX512_VNNI = 0
    FMA = 1
    NEON = 0
    ARM_FMA = 0
    F16C = 1
    FP16_VA = 0
    WASM_SIMD = 0
    VSX = 0
crop input image from 2560x1440 to 2520x1440, image_path = <redacted>_\temp.png
resize input image from 2520x1440 to 1344x768
[DEBUG] stable-diffusion.cpp:143  - Using CUDA backend
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon RX 7900 XT, gfx1100 (0x1100), VMM: no, Wave Size: 32
[INFO ] stable-diffusion.cpp:200  - loading model from '<redacted>\novaRealityXL_illustriousV80.safetensors'
[INFO ] model.cpp:1038 - load <redacted>\novaRealityXL_illustriousV80.safetensors using safetensors format
[DEBUG] model.cpp:1145 - init from '<redacted>\novaRealityXL_illustriousV80.safetensors', prefix = ''
[INFO ] stable-diffusion.cpp:254  - loading vae from '<redacted>\sdxl_vae.safetensors'
[INFO ] model.cpp:1038 - load <redacted>\sdxl_vae.safetensors using safetensors format
[DEBUG] model.cpp:1145 - init from '<redacted>\sdxl_vae.safetensors', prefix = 'vae.'
[INFO ] stable-diffusion.cpp:266  - Version: SDXL
[INFO ] stable-diffusion.cpp:300  - Weight type:                 f16
[INFO ] stable-diffusion.cpp:301  - Conditioner weight type:     f16
[INFO ] stable-diffusion.cpp:302  - Diffusion model weight type: f16
[INFO ] stable-diffusion.cpp:303  - VAE weight type:             f32
[DEBUG] stable-diffusion.cpp:305  - ggml tensor size = 400 bytes
[DEBUG] stable-diffusion.cpp\clip.hpp:171  - vocab size: 49408
[DEBUG] stable-diffusion.cpp\clip.hpp:182  - trigger word img already in vocab
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1595 - clip params backend buffer size =  307.44 MB(VRAM) (196 tensors)
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1595 - clip params backend buffer size =  1449.92 MB(VRAM) (517 tensors)
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1595 - unet params backend buffer size =  4900.07 MB(VRAM) (1680 tensors)
[DEBUG] stable-diffusion.cpp:558  - loading weights
[DEBUG] model.cpp:1985 - loading tensors from <redacted>\novaRealityXL_illustriousV80.safetensors
  |=============>                                    | 695/2642 - 465.82it/s[INFO ] model.cpp:2193 - unknown tensor 'cond_stage_model.1.model.transformer.text_model.embeddings.position_ids | f16 | 2 [77, 1, 1, 1, 1]' in model file
  |==================================================| 2642/2642 - 444.63it/s
[DEBUG] model.cpp:1985 - loading tensors from <redacted>\sdxl_vae.safetensors
  |==================================================| 2642/2642 - 0.00it/s
[INFO ] stable-diffusion.cpp\tae.hpp:226  - loading taesd from '<redacted>\diffusion_pytorch_model.safetensors', decode_only = false
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1595 - taesd params backend buffer size =   4.67 MB(VRAM) (134 tensors)
[INFO ] model.cpp:1038 - load <redacted>\diffusion_pytorch_model.safetensors using safetensors format
[DEBUG] model.cpp:1145 - init from '<redacted>\diffusion_pytorch_model.safetensors', prefix = ''
[DEBUG] model.cpp:1985 - loading tensors from <redacted>\diffusion_pytorch_model.safetensors
  |==================================================| 134/134 - 6700.00it/s
[INFO ] stable-diffusion.cpp\tae.hpp:248  - taesd model loaded
[INFO ] stable-diffusion.cpp:656  - total params memory size = 6662.10MB (VRAM 6662.10MB, RAM 0.00MB): text_encoders 1757.36MB(VRAM), diffusion_model 4900.07MB(VRAM), vae 4.67MB(VRAM), controlnet 0.00MB(VRAM), pmid 0.00MB(VRAM)
[INFO ] stable-diffusion.cpp:660  - loading model from '<redacted>\novaRealityXL_illustriousV80.safetensors' completed, taking 6.03s
[INFO ] stable-diffusion.cpp:705  - running in eps-prediction mode
[DEBUG] stable-diffusion.cpp:716  - finished loaded file
[DEBUG] stable-diffusion.cpp:2152 - generate_image 1344x768
[INFO ] stable-diffusion.cpp:728  - running with Karras scheduler
[INFO ] stable-diffusion.cpp:2199 - IMG2IMG
[INFO ] stable-diffusion.cpp:2204 - target t_enc is 3 steps
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1422 - taesd compute buffer size: 1954.97 MB(VRAM)
[DEBUG] stable-diffusion.cpp:1297 - computing vae encode graph completed, taking 0.34s
[INFO ] stable-diffusion.cpp:2328 - encode_first_stage completed, taking 0.34s
[DEBUG] stable-diffusion.cpp:861  - lora sdxl/dmd2_sdxl_4step_lora_fp16:1.00
[INFO ] stable-diffusion.cpp:846  - attempting to apply 1 LoRAs
[INFO ] model.cpp:1038 - load <redacted>LoRa/sdxl/dmd2_sdxl_4step_lora_fp16.safetensors using safetensors format
[DEBUG] model.cpp:1145 - init from '<redacted>LoRa/sdxl/dmd2_sdxl_4step_lora_fp16.safetensors', prefix = ''
[INFO ] stable-diffusion.cpp\lora.hpp:118  - loading LoRA from '<redacted>LoRa/sdxl/dmd2_sdxl_4step_lora_fp16.safetensors'
[DEBUG] model.cpp:1985 - loading tensors from <redacted>LoRa/sdxl/dmd2_sdxl_4step_lora_fp16.safetensors
  |==================================================| 2364/2364 - 1182000.00it/s
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1595 - lora params backend buffer size =  375.37 MB(VRAM) (2364 tensors)
[DEBUG] model.cpp:1985 - loading tensors from <redacted>LoRa/sdxl/dmd2_sdxl_4step_lora_fp16.safetensors
  |==================================================| 2364/2364 - 4361.62it/s
[DEBUG] stable-diffusion.cpp\lora.hpp:161  - lora type: ".lora_down"/".lora_up"
[DEBUG] stable-diffusion.cpp\lora.hpp:163  - finished loaded lora
[DEBUG] stable-diffusion.cpp\lora.hpp:861  - (2364 / 2364) LoRA tensors will be applied
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1422 - lora compute buffer size: 236.56 MB(VRAM)
[DEBUG] stable-diffusion.cpp\lora.hpp:861  - (2364 / 2364) LoRA tensors will be applied
[INFO ] stable-diffusion.cpp:823  - lora 'sdxl/dmd2_sdxl_4step_lora_fp16' applied, taking 1.44s
[INFO ] stable-diffusion.cpp:866  - apply_loras completed, taking 1.44s
[DEBUG] stable-diffusion.cpp:867  - prompt after extract and remove lora: " masterpiece, best quality, amazing quality, very aesthetic, high resolution, ultra-detailed, absurdres, scenery, photorealistic, 1girl, long hair, photoshop (medium), sword, official art, water, night, glowing, nature, forest, blue theme, lantern, dark, waterfall, cave"
[DEBUG] stable-diffusion.cpp\conditioner.hpp:359  - parse ' masterpiece, best quality, amazing quality, very aesthetic, high resolution, ultra-detailed, absurdres, scenery, photorealistic, 1girl, long hair, photoshop (medium), sword, official art, water, night, glowing, nature, forest, blue theme, lantern, dark, waterfall, cave' to [[' masterpiece, best quality, amazing quality, very aesthetic, high resolution, ultra-detailed, absurdres, scenery, photorealistic, 1girl, long hair, photoshop ', 1], ['medium', 1.1], [', sword, official art, water, night, glowing, nature, forest, blue theme, lantern, dark, waterfall, cave', 1], ]
[DEBUG] stable-diffusion.cpp\clip.hpp:311  - token length: 77
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1422 - clip compute buffer size: 1.40 MB(VRAM)
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1422 - clip compute buffer size: 2.33 MB(VRAM)
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1422 - clip compute buffer size: 2.33 MB(VRAM)
[DEBUG] stable-diffusion.cpp\conditioner.hpp:487  - computing condition graph completed, taking 67 ms
[INFO ] stable-diffusion.cpp:1942 - get_learned_condition completed, taking 79 ms
[INFO ] stable-diffusion.cpp:1965 - sampling using LCM method
[INFO ] stable-diffusion.cpp:2014 - generating image: 1/1 - seed 42
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1422 - unet compute buffer size: 808.81 MB(VRAM)
  |==================================================| 4/4 - 2.68it/s
[INFO ] stable-diffusion.cpp:2049 - sampling completed, taking 1.77s
[INFO ] stable-diffusion.cpp:2057 - generating 1 latent images completed, taking 1.84s
[INFO ] stable-diffusion.cpp:2060 - decoding 1 latents
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1422 - taesd compute buffer size: 1890.25 MB(VRAM)
[DEBUG] stable-diffusion.cpp:1439 - computing vae decode graph completed, taking 0.20s
[INFO ] stable-diffusion.cpp:2070 - latent 1 decoded, taking 0.20s
[INFO ] stable-diffusion.cpp:2074 - decode_first_stage completed, taking 0.20s
[INFO ] stable-diffusion.cpp:2356 - generate_image completed in 3.91s
save result PNG image to '<redacted>image.png'

Newer version

Option:
    n_threads:                         6
    mode:                              img_gen
    model_path:                        <redacted>\novaRealityXL_illustriousV80.safetensors
    wtype:                             unspecified
    clip_l_path:
    clip_g_path:
    clip_vision_path:
    t5xxl_path:
    qwen2vl_path:
    qwen2vl_vision_path:
    diffusion_model_path:
    high_noise_diffusion_model_path:
    vae_path:                          <redacted>\sdxl_vae.safetensors
    taesd_path:                        <redacted>\diffusion_pytorch_model.safetensors
    esrgan_path:
    control_net_path:
    embedding_dir:
    photo_maker_path:
    pm_id_images_dir:
    pm_id_embed_path:
    pm_style_strength:                 20.00
    output_path:                       <redacted>image.png
    init_image_path:                   <redacted>_\temp.png
    end_image_path:
    mask_image_path:
    control_image_path:
    ref_images_paths:
    control_video_path:
    auto_resize_ref_image:             true
    increase_ref_index:                false
    offload_params_to_cpu:             false
    clip_on_cpu:                       false
    control_net_cpu:                   false
    vae_on_cpu:                        false
    diffusion flash attention:         false
    diffusion Conv2d direct:           false
    vae_conv_direct:                   false
    control_strength:                  0.90
    prompt:                            <lora:sdxl/dmd2_sdxl_4step_lora_fp16:1> masterpiece, best quality, amazing quality, very aesthetic, high resolution, ultra-detailed, absurdres, scenery, photorealistic, 1girl, long hair, photoshop (medium), sword, official art, water, night, glowing, nature, forest, blue theme, lantern, dark, waterfall, cave
    negative_prompt:                   modern, old, oldest, cartoon, graphic, text, painting, crayon, graphite, abstract, glitch, deformed, mutated, ugly, disfigured, long body, lowres, bad anatomy, bad hands, missing fingers, extra digits, fewer digits, very displeasing, (worst quality, bad quality:1.2), bad anatomy, sketch, jpeg artifacts, signature, watermark, username, simple background, conjoined, bad ai-generated
    clip_skip:                         2
    width:                             1344
    height:                            768
    sample_params:                     (txt_cfg: 1.00, img_cfg: 1.00, distilled_guidance: 3.50, slg.layer_count: 3, slg.layer_start: 0.01, slg.layer_end: 0.20, slg.scale: 0.00, scheduler: karras, sample_method: lcm, sample_steps: 8, eta: 0.00, shifted_timestep: 0)
    high_noise_sample_params:          (txt_cfg: 7.00, img_cfg: 7.00, distilled_guidance: 3.50, slg.layer_count: 3, slg.layer_start: 0.01, slg.layer_end: 0.20, slg.scale: 0.00, scheduler: default, sample_method: default, sample_steps: -1, eta: 0.00, shifted_timestep: 0)
    moe_boundary:                      0.875
    prediction:                        default
    flow_shift:                        inf
    strength(img2img):                 0.45
    rng:                               cuda
    seed:                              42
    batch_count:                       1
    vae_tiling:                        false
    force_sdxl_vae_conv_scale:         false
    upscale_repeats:                   1
    chroma_use_dit_mask:               true
    chroma_use_t5_mask:                false
    chroma_t5_mask_pad:                1
    video_frames:                      1
    vace_strength:                     1.00
    fps:                               16
System Info:
    SSE3 = 1
    AVX = 1
    AVX2 = 1
    AVX512 = 0
    AVX512_VBMI = 0
    AVX512_VNNI = 0
    FMA = 1
    NEON = 0
    ARM_FMA = 0
    F16C = 1
    FP16_VA = 0
    WASM_SIMD = 0
    VSX = 0
crop input image from 2560x1440 to 2520x1440, image_path = <redacted>_\temp.png
resize input image from 2520x1440 to 1344x768
[DEBUG] stable-diffusion.cpp:139  - Using CUDA backend
[INFO ] stable-diffusion.cpp\ggml_extend.hpp:69   - ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
[INFO ] stable-diffusion.cpp\ggml_extend.hpp:69   - ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
[INFO ] stable-diffusion.cpp\ggml_extend.hpp:69   - ggml_cuda_init: found 1 ROCm devices:
[INFO ] stable-diffusion.cpp\ggml_extend.hpp:69   -   Device 0: AMD Radeon RX 7900 XT, gfx1100 (0x1100), VMM: no, Wave Size: 32
[INFO ] stable-diffusion.cpp:196  - loading model from '<redacted>\novaRealityXL_illustriousV80.safetensors'
[INFO ] model.cpp:1112 - load <redacted>\novaRealityXL_illustriousV80.safetensors using safetensors format
[DEBUG] model.cpp:1219 - init from '<redacted>\novaRealityXL_illustriousV80.safetensors', prefix = ''
[INFO ] stable-diffusion.cpp:264  - loading vae from '<redacted>\sdxl_vae.safetensors'
[INFO ] model.cpp:1112 - load <redacted>\sdxl_vae.safetensors using safetensors format
[DEBUG] model.cpp:1219 - init from '<redacted>\sdxl_vae.safetensors', prefix = 'vae.'
[INFO ] stable-diffusion.cpp:285  - Version: SDXL
[INFO ] stable-diffusion.cpp:312  - Weight type stat:                      f32: 248  |     f16: 2514
[INFO ] stable-diffusion.cpp:313  - Conditioner weight type stat:          f16: 586
[INFO ] stable-diffusion.cpp:314  - Diffusion model weight type stat:      f16: 1680
[INFO ] stable-diffusion.cpp:315  - VAE weight type stat:                  f32: 248  |     f16: 248
[DEBUG] stable-diffusion.cpp:317  - ggml tensor size = 400 bytes
[DEBUG] stable-diffusion.cpp\clip.hpp:171  - vocab size: 49408
[DEBUG] stable-diffusion.cpp\clip.hpp:182  - trigger word img already in vocab
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1785 - clip params backend buffer size =  235.06 MB(VRAM) (196 tensors)
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1785 - clip params backend buffer size =  1329.29 MB(VRAM) (517 tensors)
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1785 - unet params backend buffer size =  4900.07 MB(VRAM) (1680 tensors)
[DEBUG] stable-diffusion.cpp:592  - loading weights
[DEBUG] model.cpp:2047 - using 6 threads for model loading
[DEBUG] model.cpp:2130 - loading tensors from <redacted>\novaRealityXL_illustriousV80.safetensors
  |============>                                     | 641/2642 - 773.22it/s[INFO ] model.cpp:2400 - unknown tensor 'cond_stage_model.1.model.transformer.text_model.embeddings.position_ids | f16 | 2 [77, 1, 1, 1, 1]' in model file
  |=============================================>    | 2394/2642 - 648.08it/s
[DEBUG] model.cpp:2130 - loading tensors from <redacted>\sdxl_vae.safetensors
  |==================================================| 2642/2642 - 677.78it/s
[INFO ] model.cpp:2374 - loading tensors completed, taking 3.92s (process: 0.02s, read: 2.81s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.72s)
[INFO ] stable-diffusion.cpp\tae.hpp:215  - loading taesd from '<redacted>\diffusion_pytorch_model.safetensors', decode_only = false
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1785 - taesd params backend buffer size =   4.67 MB(VRAM) (134 tensors)
[INFO ] model.cpp:1112 - load <redacted>\diffusion_pytorch_model.safetensors using safetensors format
[DEBUG] model.cpp:1219 - init from '<redacted>\diffusion_pytorch_model.safetensors', prefix = ''
[DEBUG] model.cpp:2047 - using 6 threads for model loading
[DEBUG] model.cpp:2130 - loading tensors from <redacted>\diffusion_pytorch_model.safetensors
  |==================================================| 134/134 - 626.17it/s
[INFO ] model.cpp:2374 - loading tensors completed, taking 0.22s (process: 0.00s, read: 0.00s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.04s)
[INFO ] stable-diffusion.cpp\tae.hpp:237  - taesd model loaded
[INFO ] stable-diffusion.cpp:690  - total params memory size = 6469.10MB (VRAM 6469.10MB, RAM 0.00MB): text_encoders 1564.36MB(VRAM), diffusion_model 4900.07MB(VRAM), vae 4.67MB(VRAM), controlnet 0.00MB(VRAM), pmid 0.00MB(VRAM)
[INFO ] stable-diffusion.cpp:790  - running in eps-prediction mode
[DEBUG] stable-diffusion.cpp:802  - finished loaded file
[DEBUG] stable-diffusion.cpp:2480 - generate_image 1344x768
[INFO ] stable-diffusion.cpp:814  - running with Karras scheduler
[INFO ] stable-diffusion.cpp:2515 - IMG2IMG
[INFO ] stable-diffusion.cpp:2520 - target t_enc is 3 steps
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1600 - taesd compute buffer size: 1954.97 MB(VRAM)
[DEBUG] stable-diffusion.cpp:1597 - computing vae encode graph completed, taking 0.33s
[INFO ] stable-diffusion.cpp:2694 - encode_first_stage completed, taking 0.35s
[DEBUG] stable-diffusion.cpp:963  - lora sdxl/dmd2_sdxl_4step_lora_fp16:1.00
[INFO ] stable-diffusion.cpp:948  - attempting to apply 1 LoRAs
[INFO ] model.cpp:1112 - load <redacted>LoRa/sdxl/dmd2_sdxl_4step_lora_fp16.safetensors using safetensors format
[DEBUG] model.cpp:1219 - init from '<redacted>LoRa/sdxl/dmd2_sdxl_4step_lora_fp16.safetensors', prefix = ''
[INFO ] stable-diffusion.cpp\lora.hpp:120  - loading LoRA from '<redacted>LoRa/sdxl/dmd2_sdxl_4step_lora_fp16.safetensors'
[DEBUG] model.cpp:2047 - using 6 threads for model loading
[DEBUG] model.cpp:2130 - loading tensors from <redacted>LoRa/sdxl/dmd2_sdxl_4step_lora_fp16.safetensors
  |==================================================| 2364/2364 - 11702.97it/s
[INFO ] model.cpp:2374 - loading tensors completed, taking 0.23s (process: 0.03s, read: 0.01s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.00s)
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1785 - lora params backend buffer size =  375.37 MB(VRAM) (2364 tensors)
[DEBUG] model.cpp:2047 - using 6 threads for model loading
[DEBUG] model.cpp:2130 - loading tensors from <redacted>LoRa/sdxl/dmd2_sdxl_4step_lora_fp16.safetensors
  |==================================================| 2364/2364 - 5751.82it/s
[INFO ] model.cpp:2374 - loading tensors completed, taking 0.44s (process: 0.03s, read: 0.14s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.17s)
[DEBUG] stable-diffusion.cpp\lora.hpp:175  - lora type: ".lora_down"/".lora_up"
[DEBUG] stable-diffusion.cpp\lora.hpp:177  - finished loaded lora
[DEBUG] stable-diffusion.cpp\lora.hpp:875  - (2364 / 2364) LoRA tensors will be applied
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1600 - lora compute buffer size: 236.56 MB(VRAM)
[DEBUG] stable-diffusion.cpp\lora.hpp:875  - (2364 / 2364) LoRA tensors will be applied
[INFO ] stable-diffusion.cpp:928  - lora 'sdxl/dmd2_sdxl_4step_lora_fp16' applied, taking 1.40s
[INFO ] stable-diffusion.cpp:968  - apply_loras completed, taking 1.40s
[DEBUG] stable-diffusion.cpp:969  - prompt after extract and remove lora: " masterpiece, best quality, amazing quality, very aesthetic, high resolution, ultra-detailed, absurdres, scenery, photorealistic, 1girl, long hair, photoshop (medium), sword, official art, water, night, glowing, nature, forest, blue theme, lantern, dark, waterfall, cave"
[DEBUG] stable-diffusion.cpp\conditioner.hpp:350  - parse ' masterpiece, best quality, amazing quality, very aesthetic, high resolution, ultra-detailed, absurdres, scenery, photorealistic, 1girl, long hair, photoshop (medium), sword, official art, water, night, glowing, nature, forest, blue theme, lantern, dark, waterfall, cave' to [[' masterpiece, best quality, amazing quality, very aesthetic, high resolution, ultra-detailed, absurdres, scenery, photorealistic, 1girl, long hair, photoshop ', 1], ['medium', 1.1], [', sword, official art, water, night, glowing, nature, forest, blue theme, lantern, dark, waterfall, cave', 1], ]
[DEBUG] stable-diffusion.cpp\clip.hpp:311  - token length: 77
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1600 - clip compute buffer size: 1.40 MB(VRAM)
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1600 - clip compute buffer size: 2.33 MB(VRAM)
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1600 - clip compute buffer size: 2.33 MB(VRAM)
[DEBUG] stable-diffusion.cpp\conditioner.hpp:484  - computing condition graph completed, taking 64 ms
[INFO ] stable-diffusion.cpp:2265 - get_learned_condition completed, taking 1477 ms
[INFO ] stable-diffusion.cpp:2283 - sampling using LCM method
[INFO ] stable-diffusion.cpp:2377 - generating image: 1/1 - seed 42
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1600 - unet compute buffer size: 808.81 MB(VRAM)
  |==================================================| 4/4 - 2.45it/s
[INFO ] stable-diffusion.cpp:2414 - sampling completed, taking 1.70s
[INFO ] stable-diffusion.cpp:2422 - generating 1 latent images completed, taking 1.77s
[INFO ] stable-diffusion.cpp:2425 - decoding 1 latents
[DEBUG] stable-diffusion.cpp\ggml_extend.hpp:1600 - taesd compute buffer size: 1890.25 MB(VRAM)
[DEBUG] stable-diffusion.cpp:1723 - computing vae decode graph completed, taking 0.20s
[INFO ] stable-diffusion.cpp:2435 - latent 1 decoded, taking 0.20s
[INFO ] stable-diffusion.cpp:2439 - decode_first_stage completed, taking 0.20s
[INFO ] stable-diffusion.cpp:2728 - generate_image completed in 3.80s
save result PNG image to '<redacted>image.png'

[stduhpf]

a lovely cat.png	.\build\bin\sd.exe --model .\models\dreamshaper_8.safetensors -p "a cute dog" -i '.\assets\a lovely cat.png' --strength 0.35	.\build\bin\sd.exe --model .\models\dreamshaper_8.safetensors -p "a cute dog" -i '.\assets\a lovely cat.png' --strength 0.35 --taesd ./models/taesd.safetensors

I'm thinking this could be related to taesd encoding not working as expected.

[stduhpf]

I think I found the cause of the taesd issue. @lastrosade can you try to see if #935 fixes it?

[lastrosade]

It would appear that this works! Thanks.
Should I close the issue now or when it is merged?

[stduhpf]

Maybe you could rename the issue to clarify that it's related to taesd encoding and not inpainting, and close it when it's merged.