Ladies and gentlemen I present to you Qwen3.8 27b 1bit brain damage quant
I wanted to just test the unsloth 1bit quant of qwen 3.8 27b as I have just 8gb vram and ngl it gave me a good laugh
u/Ok-Health-7096 — 1 day ago
▲ 1.3k r/LocalLLM
Every Second post rn
Maybe someday I'll get a system to run it but hey definitely another w for the open weights community
u/Ok-Health-7096 — 7 days ago
Llama bench and real performance wayy different(Help)
I had been using qwen 3.6-35b-a3b iq3xxs for past couple of days at 900tk/s prefil and ~40tk/s gen but it hallucinated alot would get facts wrong and what not. I decided to switch to iq4xs for better accuracy and thought even if I get 25tk/s it would be fine and it was way better than that(atleast in llama bench)
llama banch settings:
-m /home/arfat00/Projects/Models/Qwen3.6-35B-A3B-UD-IQ4_XS.gguf \
-ngl -1 \
-ncmoe 31 \
--cache-type-k q8_0 \
--cache-type-v q8_0 \
-b 4096 -ub 2048 \
-t 14 \
-p 4096 -n 512 \
-r 10 \
-o md
performance:
Device 0: NVIDIA GeForce RTX 4060 Laptop GPU, compute capability 8.9, VMM: yes, VRAM: 7807 MiB
| model | size | params | backend | ngl | n_cpu_moe | threads | n_batch | n_ubatch | type_k | type_v | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ---------: | ------: | ------: | -------: | -----: | -----: | --------------: | -------------------: |
| qwen35moe 35B.A3B IQ4_XS - 4.25 bpw | 16.50 GiB | 34.66 B | CUDA | -1 | 31 | 14 | 4096 | 2048 | q8_0 | q8_0 | pp4096 | 753.97 ± 122.26 |
| qwen35moe 35B.A3B IQ4_XS - 4.25 bpw | 16.50 GiB | 34.66 B | CUDA | -1 | 31 | 14 | 4096 | 2048 | q8_0 | q8_0 | tg512 | 36.29 ± 1.63 |
as you can see it gives me ~700tk/s prefil and ~35 tk/s gen speed, but when I use it actually this is the performance:
8.09.323.365 I slot print_timing: id 0 | task 0 | prompt eval time = 144460.53 ms / 18535 tokens ( 7.79 ms per token, 128.30 tokens per second)
8.09.323.370 I slot print_timing: id 0 | task 0 | eval time = 23056.43 ms / 184 tokens ( 125.31 ms per token, 7.98 tokens per second)
(keep in mind this was the first message too)
what do I do is there something im doing wrong this is the llama server:
-ngl -1 \
--n-cpu-moe 31 \
--cache-type-k q8_0 \
--cache-type-v q8_0 \
-b 4096 -ub 2048 \
-t 14 \
--ctx-size 65536 \
--host 0.0.0.0 \
--port 4060 \
--jinja \
--flash-attn on \
--reasoning-budget 4096 \
--repeat-penalty 1.1 \
--cache-reuse 0 \
--parallel 1 \
--temp 0.6 \
--top-k 20 \
--top-p 0.95
specs:
rtx 4060 laptop gpu 8gb vram+16 gb ddr5 ram
u/Ok-Health-7096 — 2 months ago
I should really be studying for my endsems right now
u/Ok-Health-7096 — 3 months ago
▲ 0 r/gaming
Made a game rating + review website with game price aggression
Hello guys I'm an undergrad student who loves gaming and keep track of games I played, playing and want to play ,also to leave behind reviews of the games I play.
I was learning backend development and thought of making a project for myself to do all this and made GameOn, I thought there would be many people like me who want to do the same things.
You guys can try the website
it's totally free, no ads, no nothing just a hobby project of mine.
You can review games on it manage your played games through list with drag and drop features with which you can simply drag the playing game to played(on desktops).
I also implemented a game price aggregator which is still a little rough in its logic and shows wrong results some time but I'll work on it after my end sem.
A feature which I love in this project and one of the reason I started this project was the ability to view your snapshot of the review which you can then share on your story for other people to see.
Please feel free to tell me what's wrong and what can be better in this along with more features I could add.
Link in comments
u/Ok-Health-7096 — 3 months ago