Skip to content

Add Heun and DPM-Solver-2-like samplers((high-order solvers widely used in diffusion-based image generation)) - #11

Open
Fangjiage-1 wants to merge 5 commits into
lillian039:pytorch_elffrom
Fangjiage-1:pytorch_elf
Open

Add Heun and DPM-Solver-2-like samplers((high-order solvers widely used in diffusion-based image generation))#11
Fangjiage-1 wants to merge 5 commits into
lillian039:pytorch_elffrom
Fangjiage-1:pytorch_elf

Conversation

@Fangjiage-1

Copy link
Copy Markdown

Summary

This PR adds two deterministic second-order sampling baselines to the PyTorch ELF implementation:

  • Heun: a second-order Runge–Kutta predictor-corrector method.
  • DPM-Solver-2-like: an exponential midpoint method adapted to ELF's flow-matching ODE.

Together with the existing ODE/Euler method, the sampling code now exposes three methods:

  • ode
  • heun
  • dpm_solver_2_like

Changes

  • Add _heun_step() to src/utils/sampling_utils.py.
  • Add _dpm_solver_2_like_step() to src/utils/sampling_utils.py.
  • Remove the previous SDE-style sampling step.
  • Update src/utils/generation_utils.py to import and dispatch the three supported methods.
  • Update conditional and unconditional sampling configurations.
  • Keep the final sampling interval as an ODE/Euler step, following the original endpoint handling.

Sampler details

Heun

The Heun sampler uses two network evaluations per regular interval:

  1. An Euler predictor estimates the next latent state.
  2. A corrector averages the velocity estimates at the current and predicted states.

DPM-Solver-2-like

The DPM-Solver-2-like sampler performs:

  1. An exponential half-step using the prediction at the current time.
  2. A model evaluation at the midpoint.
  3. An exponential full-step using the midpoint prediction.

This method is inspired by DPM-Solver-2, but it is specifically adapted to ELF's flow-matching formulation and is not a direct reproduction of the original image-diffusion implementation.

Compute matching

The provided configurations use:

Method Sampling steps Effective NFE
ODE/Euler 64 64
Heun 32 63
DPM-Solver-2-like 32 63

The second-order methods use two network evaluations for each regular interval, while the final interval uses the original ODE/Euler update.

Files changed

  • src/utils/sampling_utils.py
  • src/utils/generation_utils.py
  • src/configs/sampling_configs/uncond_sampling_configs.yml
  • src/configs/sampling_configs/cond_sampling_configs.yml

Validation

  • Python syntax validation passed.
  • Both YAML configuration files parse successfully.
  • Every configured sampling method has a corresponding generation dispatch branch.
  • No stale _sde_step import or dispatch remains.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant