u/MaleficentCrab4672

I just find out that my PC can support model with 122B but at 2/3bit

As u can see on this picture my pc can support this model is anyone use this model on his local machine and what do you think about it?

I just love to see what my local machine can do with all this

I ask Chat GPT and even Gemini they both told me that this version is so smart but with 2Bit will be slower however for the writing and artistic stuff and prompting it will be like a God mod on my local machine the speed is going to be betwewn 3 to 5 tok/sec is anyone try this model in the lowest quality?

u/MaleficentCrab4672 — 4 days ago

▲ 13 r/Qwen_AI

Model with f16 4B worth it?

I'm using model Qwen 4B f16 what do you think guys is it worth it?

My pc can support 80B but for the seck of the speed i go straight to 4b but not 4bit so i choice the full precise of it like Q8 or f16 and sometimes if i find bf16 i download as well the speed is about 34Toke/sec for the default tokens when i go to the full tokens the speed goes around 6 but i'm here to ask for the intelligence i'm not a developers or anything like that but i just need a model that cannot halucinate and go for long as i need all the information.

I also using the rolling windows for the settings so the model won't stuck.

reddit.com

u/MaleficentCrab4672 — 21 days ago