u/henk717

HF blocks download threads

I want to signal this early to the community since this seems to be a very recent change.
Upon starting a KoboldCpp instance I was greeted with 403 errors on every thread except for one.

This kind of download acceleration is needed in the current way the GGUF ecosystem is handled, its large single files and the single thread download speed is often capped at 40mb/s.

With tools like hf cli downloading an entire repo is accelerated by downloading multiple separate files at once and in safetensors repositories its common place to have a lot of smaller split files to achieve parallelism. The GGUF repositories however are not suitable for this currently, they contain large files and every single quant in the same branch. This makes downloading an entire branch pointless, and it also means that currently most models you can't apply the kind of acceleration we have been doing.

I hope that huggingface reverses this change since I see very little difference between the impact of us all downloading 16 files at once, or one file 16 times at once. But with this in place downloads will be significantly slower unless the GGUF ecosystem also transitions into large amounts of split files which most people do not find desirable.

If this is not reversed one option would be uploading quants in large amounts of split files each in their own branch of a repository, similar to how the exl uploaders have been doing it. This would help people download them much faster, but would then mean they have to manually merge them if they wish to keep them as a single file. This is much less desirable than HF helping the community out by reversing this.