[Feature] Prebuilt binaries support of AMD CPUs without FMA3

[klkr1]

Feature Summary

Prebuilt binaries should support AMD CPUs without FMA3

Detailed Description

I want to use stable diffusion prebuilt binaries on an AMD FX(tm)-6100, but it fails with Illegal instruction.
After some debugging I found that vfmadd213ss (FMA3 instruction) is used in libstable-diffusion.so, which is not supported by my CPU.
It would be great if you could provide a variant for type of CPU.

Alternatives you considered

No response

Additional context

No response

[klkr1]

I downloaded build master-665 of commit 02f0637 from #1591. Illegal instructions still occurs in ggml_cpu_init due to unsupported FMA3 instruction, so it does not solve this issue

[wbruna]

I downloaded build master-665 of commit 02f0637 from #1591 . Illegal instructions still occurs in ggml_cpu_init due to unsupported FMA3 instruction, so it does not solve this issue

As I've explained in that PR, it opened up the possibility of a released binary to support several CPU arches at the same time; a proper fix still needs changes to the Github release configuration.

I've pushed a tentative change to my fork; could you give it a try?

https://github.com/wbruna/stable-diffusion.cpp/releases/tag/master-674-fd19aaa

It shall pick the appropriate backend for your CPU at the beginning:

$ /tmp/user/1000/test-sdcpp/sd-cli -m dreamshaper_8.safetensors -p sunset -v 
[DEBUG] main.cpp:597  - version: stable-diffusion.cpp version unknown, commit fd19aaa
ggml_vulkan: Found 2 Vulkan devices:
(...)
load_backend: loaded Vulkan backend from /tmp/user/1000/test-sdcpp/libggml-vulkan.so
load_backend: loaded CPU backend from /tmp/user/1000/test-sdcpp/libggml-cpu-haswell.so
[DEBUG] main.cpp:598  - System Info: 
   SSE3 = 1 |    SSSE3 = 1 |    AVX = 1 |    AVX2 = 1 |    F16C = 1 |    FMA = 1 |    BMI2 = 1 |    OPENMP = 1 |    REPACK = 1 | 
[DEBUG] main.cpp:599  - SDCliParams {
(...)

[klkr1]

This version runs and does not terminate with illegal instruction any more.
The generated image is not OK however.

Log Output:

./sd-cli --model /root/.cache/huggingface/hub/models--stabilityai--sdxl-turbo/snapshots/71153311d3dbb46851df1931d3ca6e939de83304/sd_xl_turbo_1.0_fp16.safetensors -p sunset -v
[DEBUG] main.cpp:597 - version: stable-diffusion.cpp version unknown, commit fd19aaa
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon RX 6600 XT (RADV NAVI23) (radv) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 0 | matrix cores: none
load_backend: loaded Vulkan backend from /root/.cache/lemonade/bin/sd-cpp/rocm-stable/libggml-vulkan.so
load_backend: loaded CPU backend from /root/.cache/lemonade/bin/sd-cpp/rocm-stable/libggml-cpu-sandybridge.so
[DEBUG] main.cpp:598 - System Info:
SSE3 = 1 | SSSE3 = 1 | AVX = 1 | OPENMP = 1 | REPACK = 1 |
...

[wbruna]

This version runs and does not terminate with illegal instruction any more.
The generated image is not OK however.

I was able to reproduce something like that on my iGPU (3400G):

versus the dGPU (7600 XT):

With SD1.5, both the dGPU and the iGPU work:

except with the iGPU + --vae-conv-direct:

So I suspect this is a separate issue: likely a bug on the Vulkan backend (ggml, or the driver), because it seems to be triggered only by certain device+model+flag combinations.

Could you check if other models work for you?

[klkr1]

I did some checks with both SD15 and SDXL both with rocm 7.13 backend and vulkan backend.

SD15 on rocm: OK
SD15 on vulkan: OK
SDXL on rocm: OK
SDXL on vulkan: garbled
SDXL on vulkan --offload-to-cpu: OK
SDXL on vulkan --vae-conv-direct: OK

Seems indeed to be a problem of model+options

[leejet]

This should be resolved now.

[wbruna]

@klkr1 , could you verify if the garbled image issue still happens on the latest release (master-737-3b6c9ca)? I cannot reproduce it anymore on my iGPU.

[klkr1]

@wbruna: with this version for me it is still garbled for SDXL without options:
SD15 on rocm: OK
SD15 on vulkan: OK
SDXL on rocm: now outOfMemory, unless I use --offload-to-cpu, then it is OK
SDXL on vulkan: garbled
SDXL on vulkan --offload-to-cpu: OK
SDXL on vulkan --vae-conv-direct: OK
Edit: I mixed up SD15 on vulkan and SDXL on rocm SD15 is OK in both cases, SDXL on rocm was outOfMemory
Is there anything I can do for debugging?