LoRAs often work with models they were not designed for.
People sometimes experience a placebo effect - using LoRAs that neither enhance nor hinder good results.
If several LoRa-s are used together the order of application does not matter.
Re unianimate LoRa: “you have to load the unianimate lora on the model loader, not the set node because it patches the model too”
Normal ComfyUI behaviour with loras is to merge the weights to the model before inference, which can’t be done to GGUF models as it would be too slow operation, so instead the lora weights are just added to the dequantized GGUF weight when it’s used
merging loras:
- no VRAM hit
- slower first run init after changing lora/lora strength
- precision loss if using lower main model precision than the lora
unmerged loras (GGUF or the option in the wrapper):
- uses more VRAM
- no init time ever
- better lora quality as it’s used directly at base precision
- allows runtime lora strength adjusting
Q: Is it possible to merge a lora into a GGUF model ?
A: Would need to do that to bf or fp16 model and then conver that to GGUF
Lora = low-rank matrix factorisation
Loha = hadamard product matrix factorisation
Lokr = kronecker product matrix decomposition
In bongmath language
Wan 2.1 and 2.2 LoRA-s: