Why hasn't TurboQuant been implemented in llama.cpp yet? (Genuine question from a hobbyist)
Hi everyone,
I've been following the local LLM scene for a while, but I lack the deep technical background in C++ or low-level CUDA programming to understand the inner workings of quantization frameworks.
Recently, I’ve been reading about **TurboQuant** and its performance claims. I know there are repos out there with implementations, like the one by **TheTom**, but it got me wondering: **Why hasn't it been integrated or ported into the main llama.cpp project yet?**
Is there a fundamental architectural incompatibility between how llama.cpp (GGML) handles inference and how TurboQuant is designed? Or is it simply a matter of community priority, given that formats like GGUF (with IQ/Q quantizations) are already highly optimized and widely adopted?
Thanks for the answers!