Unable to inference using Segmind Tiny SD model

[Rehaman1429]

Hi there,

I am trying to run Segmind's Distilled Diffusion model (segmind/tiny-sd) from Hugging Face on my machine with the following specifications:

• Processor: 12th Gen Intel(R) Core(TM) i5-1235U 1.30 GHz
• Installed RAM: 16.0 GB (15.7 GB usable)
• System Type: 64-bit operating system, x64-based processor

I successfully converted the model to a single checkpoint file (.safetensors, .ckpt) as shown below:

Then, I used the convert function in stable-diffusion.cpp to convert the .safetensors file to the GGUF format. While the conversion completed without any errors, the resulting GGUF file is significantly smaller than the original model, as shown here:

When I attempt to run inference with the converted GGUF file, I encounter the following error:

It appears that the issue may lie in the conversion process to GGUF, possibly due to the fact that the model in question is a distilled version (Tiny-SD). I am wondering if anyone has worked with distilled models in this context and found a fix for this issue.

Any insights or suggestions would be greatly appreciated.

Thank you for your time and assistance!

[stduhpf]

You don't need to convert the .safetensors to gguf, sdcpp supports loading safetensors files directly. You could try loading it and see if you get the same error? (I'm pretty sure you would)

I think there are two problems here:

• the model file doesn't contain the text encoder, which would explain why It's unable to detect the version. If it was included, I believe it would be detected as a sd1.x model.
• Even if the text encoder was included, there would be another problem causing a crash when loading, because It would try loading it as a standard sd1.x model, even though the architecture is actually fairly different.

So to "fix" this you would first need to make sure the text_encoder is included in the model file, and then you'd still have to add support for this model (I think this would require making changes to model.h, model.cpp, and unet .cpp mostly).

[Rehaman1429]

Thanks for your reply @stduhpf ,

The tiny-sd model(https://huggingface.co/segmind/tiny-sd/tree/main) has text encoder:

And hence the error is not because of the absence of text encoder, it would be something else.

[stduhpf]

Yes, there is a text encoder on the hf repo, I already knew that (and that's how I knew it should be detected as a sd1.x model), I was only saying the text encoder might be missing from the safetensors file. But it's also possible it has been included under a different name than the one expected.
Anyways this model has an unsupported architecture, so it won't work regardless.

[Naveen7217]

@stduhpf what is the meaning of unsupported architecture? , is there any alternative is there to run?

[stduhpf]

@Naveen7217 Sdcpp expects very specific model architectures, with the right number of layers and tensor sizes. The Tiny SD model has a different architecture that doesn't match SD1.x, SD2.x or SDXL architectures. So the model cannot be loaded.
The alternative is to use something else than sdcpp to run it, or to try and add support for this specific model with a PR.

[stduhpf]

Another thing that might be worth trying is something like #490 but for unet models.

[akleine]

I have just commited my work on that kind of TINY SD 1.x models:

https://github.com/akleine/stable-diffusion.cpp/tree/tiny-unet-in-sd-cpp

A Pull-Request will be added later. Perhaps I'll add some DOC about that and about creating some ckpt files.
At all this opens up some new perspectives for us on low-power hardware like Raspberry Pi and some Android phones.

Some output:
[INFO ] model.cpp:1016 - load models/bk-sdm-tiny.ckpt using checkpoint format
[DEBUG] model.cpp:1593 - init from 'models/bk-sdm-tiny.ckpt'
[INFO ] stable-diffusion.cpp:244 - Version: SD 1.x tiny UNet <---------------- NEW
....
[DEBUG] ggml_extend.hpp:1193 - unet params backend buffer size = 617.20 MB(RAM) (358 tensors)
....
[DEBUG] stable-diffusion.cpp:1879 - generate_image 512x512
[INFO ] stable-diffusion.cpp:2009 - TXT2IMG
......
One step needs here approximately 32.37s on a RaspberryPi 5 - and the picture quallity is quite okay.

For comparisation: SD1.5 needs here:
[DEBUG] ggml_extend.hpp:1193 - unet params backend buffer size = 2155.33 MB(RAM) (686 tensors)

At last: here you can read more about that kind of tiny SD models https://arxiv.org/pdf/2305.15798.pdf

[ProgerXP]

I am trying to run inference on this SD 2.x Tiny UNet model:

https://huggingface.co/IDKiro/sdxs-512-0.9

No matter what I do - clone and run on sd-cli -m sdxs-512-0.9 directly, use -M convert, use convert_diffusers_to_original_stable_diffusion.py or use a script for SD1.x from distilled_sd.md - I always end up with this error:

[ERROR] model.cpp:1620 - tensor
'model.diffusion_model.input_blocks.1.1.transformer_blocks.0.attn1.to_k.weight'
has wrong shape in model file: got [1024, 320, 1, 1], expected [320, 320, 1, 1]

I have tried other Tiny U-Net models from distilled_sd.md - they work fine, as do regular SD 1.x/2.x models from the README.

I am willing to put up a bounty for fixing this or even just assisting me in running this inference. @akleine Perhaps you would be wiling to help?

# sd-cli --version
stable-diffusion.cpp version master-432-60abda5-3-gca5b196, commit ca5b196

[akleine]

At first: Happy New Year 2026 to everyone!

Thank you for your interest in sd.cpp.

You are correct: the SDXS-512 model is not compatible with the current releases of sd.cpp.
I have not yet integrated it into my work on TINY models. This is primarily because, as @IDKiro previously noted with the unfortunate news: "The subsequent open source program has been terminated by the Company." As a result, the SDXS-1024 model was never publicly released.

I completely agree, however; SDXS is renowned for its incredible speed.
Also my project sdcpp-on-android would benefit greatly from the performance of SDXS.

Therefore, I will look into this for a future release of sd.cpp.

[akleine]

Hi @ProgerXP ,
take a look at my fork https://github.com/akleine/stable-diffusion.cpp .

For the first time you could use SDXS with sd.cpp an it runs really, really fast! !!

PR will come later. For testing I downloaded the model using Python:

#!/usr/bin/env python
from diffusers import StableDiffusionPipeline
pipe = StableDiffusionPipeline.from_pretrained("IDKiro/sdxs-512-dreamshaper")
pipe.save_pretrained(save_directory="sdxs")

...continued after downloading....

~/stable-diffusion.cpp/build/bin/sd-cli -m sdxs  -p "portrait of a lovely cat"  --cfg-scale 1 --steps 1 \
   --taesd  sdxs/vae/diffusion_pytorch_model.safetensors  -o output_diffusers_sdxs_vae_sdxs.png

Today the option --taesd sdxs/vae/diffusion_pytorch_model.safetensors is very important
because SDXS does not use AutoencoderKL as usual, but AutoencoderTiny.
Also consider both options: --cfg-scale 1 --steps 1 .

By the way, if you get an error message like this model.cpp:663 - Couldn't find working second text encoder in sdxs
you can simply ignore that.

You wrote:

I am willing to put up a bounty for fixing this or even just assisting me in running this inference.

👍 Just lend me a star :-;
( ... and don't forget @leejet and @IDKiro )

[akleine]

Hi @ProgerXP ,
have you already tested sd.cpp with my extensions for SDXS ?

Meanwhile I have continued that work at my fork.
Soon I will add a PR to @leejet .

• most important: you don't need the taesd option anymore,
• use the SDXS model now as a safetensors file as usual. Create it this way:

python convert_diffusers_to_original_stable_diffusion.py \
        --model_path  sdxs  --checkpoint_path ./sdxs.safetensors --half --use_safetensors

• use the safetensors file as follows:

~/stable-diffusion.cpp/build/bin/sd-cli -m sdxs.safetensors  -p "portrait of a lovely cat" \
   --cfg-scale 1 --steps 1  -o output_safetensors_sdxs.png

You'll find that SD.CPP using SDXS is incredibly fast !!

Any feedback is much appreciated. Thank you.