Error when using Lora with quantization q4_K

[pro777s]

I get this error when using Lora:

stable-diffusion.cpp\ggml\src\ggml-cuda\getrows.cu:197: ggml_cuda_get_rows_switch_src0_type: unsupported src0 type: q4_K

Is there a solution to the problem?
My device is RTX30xx, backend is CUDA, SD.CPP is 162 release.

[stduhpf]

I think the only solution right now is to avoid using K quants if you want to use LoRAs. Q4_0 might be more likely to work. Maybe it would be a good idea to add a way to apply LoRAs on CPU backend to bypass any compatibility issues.

[pro777s]

I wonder if it's possible to convert Lora from q4_k to q4_0?

[wbruna]

It should, but quantizing the original weights would be better anyway, to avoid cumulative errors.

[leejet]

This PR #969 should fix the issue.