Unsloth isnt offloading enough to my gpu ?
After trying out unsloth studio for a bit, i was really happy with the results, but then i decided to switch to linux (Unbuntu). MOE models are running flawlessly (Qwen 3.6 35B A3B for example), but for small models (Qwen 3.5 4B ), unsloth is lefting out a lot of vram when loading the model, and when using it, gpu usage doesnt go above 30%.
When running the same model with max gpu offloading through llama.cpp, all my vram is used correctly, i get 100% gpu usage and over twice the output speed. Is this an issue related to the auto fit system of unsloth, that doesnt work with my hardware ?
For context i have 16go of ram and a rtx4050 with 6gb vram
I get around 22 token/s with Q2 Qwen 3.6 35B A3B and only 12 /s with Qwen 3.5 4B ( up to 32/s when running 4B through command line )