GGUF support and comfy dev teams take
Hi
I recently updated to the new version of Comfyui with the dynamic VRAM features.
My experience with a 3080 10GB VRAM card, is that it does not work as good with Dynamic VRAM as GGUF. Because you introduce dependency on disk and regular ram.
So in the console when disabling this feature you are meet with this message:
>[WARNING] Dynamic vram disabled with argument. If you have any issues with dynamic vram enabled please give us a detailed reports as this argument will be removed soon. If you use gguf we recommend keeping dynamic vram enabled and using native ComfyUI model formats instead. ComfyUI native formats like fp8 will be faster even if they are larger than your memory.
I am baffled by this message, there is no way fitting a whole model in my 10GB VRAM is outperformed by Dynamic VRAM. When your model don't fit in the limited VRAM, you are forced to load either from disk or regular RAM. Both are slower. Because the devices are slower and now depending on each other.
Reading this take: https://github.com/Comfy-Org/ComfyUI/issues/13110#issuecomment-4107008389
It's seems Comfy team don't like GGUF's even though it is to me always preferable to fit a entire model in VRAM. Regardless of its format.
So my question is. Why this rather "aggressive" take on GGUF's?
Another question why would you even consider removing the user friendly option of disabling dynamic VRAM to allow users continue to use GGUF's?
With VRAM, RAM and Storage being more expensive than ever, even just for the file size alone GGUF's is worth considering for some people.
I am hoping the Comfy team will allow us to continue to use GGUF's.