[draft] Attempt to implement the Flux Klein bonsai model

[Juste-Leo2]

This draft adds the necessary mappings to make the dequantized 4B model work.

Opencode was used, but I am noticing significant differences during inference that I haven't been able to patch using DeepSeek Flash on Opencode.

Here are some avenues to explore:

• The issue could potentially stem from the scheduler behaving differently.
• Re-analyze the VAE for the umpteenth time :)

Note: The text encoder seems to use <think> tags, but it doesn't appear to have much of an impact.

I suspect the problem lies either in the denoising process or in the VAE conversion. However, despite comparing it with the Python implementation, I haven't been able to pinpoint the issue yet. Any help would be greatly appreciated!

Here is an inference example:

git clone --recursive https://github.com/Juste-Leo2/stable-diffusion.cpp
cd stable-diffusion.cpp
 
git checkout bonsai

git submodule init
git submodule update

cmake -B build -DCMAKE_BUILD_TYPE=Release -DSD_CUDA=ON

# Compilation
cmake --build build -j

cd /home/leo/stable-diffusion.cpp && timeout 180 ./build/bin/sd-cli --cfg-scale 1 --width 512 --height 512 --steps 4 --seed 42 -p "a cat sitting on a window sill" -o /tmp/vae_fp32_test.png --diffusion-model /tmp/hf_cache_bf16/prism-ml_bonsai-image-ternary-4B-unpacked/transformer/diffusion_pytorch_model.safetensors --vae /tmp/hf_cache_bf16/prism-ml_bonsai-image-ternary-4B-unpacked/vae/diffusion_pytorch_model.safetensors --llm /tmp/hf_cache_bf16/prism-ml_bonsai-image-ternary-4B-unpacked/text_encoder/ 2>&1 | tail -30

Here is the result using the Python reference:

Here is the result using the fork:

[Juste-Leo2]

The conversation below is interesting and might provide some insights for the future (see leejet#1580)

[Juste-Leo2]

I don't think this approach will work. So I'm going to fall back on the method suggested in this discussion leejet#1580 . I'm closing this PR and will open another one for potential operational optimizations.

[Juste-Leo2]

Well, I got this branch working again :)

[Juste-Leo2]

In the end, I think I'll close this branch, even though it works. It's best to keep the original implementation; modifying Safetensor makes the code easier to read.