u/PlasticRevenue4601

I have a dilemma — I can run Qwen 3.8 flash next only in aggressive 2 bit quantisation, specifically GSQ-RCO-IQ2_XS while its dense counterpart runs perfectly fine at 4 bits.

Application
I use local models for agentic coding— main drivers for my pet projects and auxiliary brains for my job

Context
Context quantity are exactly the same — 160k, except that flash next uses 8 bit kv cache while 27b can only fit using 4 bit kv cache

Speed
Speed wise flash next is noticeably better with this new Strata inference engine, decode is faster(flash 50 ts vs dense 40ts), pre-fill is slower (flash 450ts vs 750ts dense) but flash is roughly the same for all the context length while dense with llama and vulkan degrades severely as context groves — in the end pre-fill drops up to 150 ts, decode to 25-30 ts

Quality
That is the very point of my post — I don’t know, they both seem pretty fine, my homemade tests seem to be insufficient to reliably determine the winner. Have anybody tested 2 bits of flash next? What was your experience with it?

My old iPhone 13 lagged like hell after “upgrading” to iOS 26, which is absolutely not normal. Don’t get me wrong, I don’t expect a four-year-old phone to perform as well as a new one, but I really can’t understand how it can’t handle basic apps and scrolling. The funny thing is that even the iPhone 17 lagged a little bit before I disabled those Liquid Glass BS visuals. After that, the phone worked flawlessly. But the very fact that the new OS is so heavy that even the current flagship struggles with it is insane. Apple didn’t do a good job here.
And just a tiny pinch of conspiracy—Apple’s generous support terms might have a completely different purpose than people think. I’m not saying they deliberately do this to render older phones useless with newer updates, but the pattern has been pretty consistent throughout the years: you have a 3–4-year-old iPhone, you accept the new “update,” and after that you basically have a lagging, barely breathing brick that struggles with basic stuff. I find this consistency pretty suspicious. My brother’s old Poco X3 Pro that is leagues behind an IPhone 13 still works flawlessly, it just smooth and fast as it was in the beginning while my old, fellow iPhone 13 is basically useless after this new updates

I’ve just switched from an iPhone 13 to an iPhone 17, and apart from the 120 Hz display and a fresh battery, it’s basically the same phone.
Yeah, it’s noticeably better — the display, battery, RAM capacity, and performance are all improved, but nothing feels revolutionary. The cameras are basically the same; you won’t notice much of a difference unless you really take a close look. Both phones can comfortably last a full day with a fresh battery, both handle all of my favorite games pretty well, and Face ID is flawless on both devices.
I would’ve continued using my iPhone 13 if the battery hadn’t given up and if I hadn’t severely damaged the frame, which eventually led to an unpleasant rattling noise whenever I used the phone.
I miss the time when every new release brought something genuinely new and meaningful, something that actually justified upgrading. I perfectly understand that this is happening because of physical constraints, but deep down, I’m still somehow disappointed.

I was excited to try out new Ornith 1.5 35B because Alibaba seems not to care much about people’s champ Qwen 35B and on the paper Ornith looks just as something I’d expect from Qwen 3.8 35b — solid improvements in agentic workflows, much better autonomy and context retention, Qwen 3.8 27b-like think-plan-act-repeat approach, but the more I tested it against plain old Qwen 3.6 35B the lesser and lesser became my excitement. Here are key observations:
— it’s really sensitive to KV cache quantisation. My stock Qwen runs at q4 without any problems while Ornith was consistently messing up approximately 10-30% of tool calls, especially edits before I removed cache quantisation at all. Considering the fact that small MoE models are peculiarly interesting for weak hardware users this can be a problem for the low vram users
— it isn’t noticeably better than the stock version. While I really liked it web design taste, Ornith hasn’t shown anything in my test suite, consisting from both real life tasks and synthesis to conclude it is a better model overall than Qwen. It was better in some tasks, worse in anothers, no consistent supremacy detected
— thinking „doom loops”. It lever actually looped while I was using it but occasionally, in the difficult tasks it would start incredibly long and watery thinkings in which it would do back and forth not being to able to come up with anything. And it’s difficult to pull it out from such a loop, it continued to overthink even after being stopped and told direct, simple instructions by following which it could easily finish the task. It just doesn’t care, if it decides to overthink there’s very little you can do about it
— Unsloth studio ocmpletely ignored it, didn't even bothered to leave some comments about it in their social media. I know it sounds ridiculous, but Unsloth is always one of the first out there to quantise a model when something good is released. And they ignored both Ornith models, it may be just a "no go for finetune" policy but it's also may be that they don't think that the model is worthy of their time which would be quite a strong signal

Conclusions — my first impressions were quite different from my initial expectations. Not in a good way. It looks like we should stick around with Qwen 3.6 35b for another while. Not the bad news though, it’s still hands off the best model available for 6-12 gb vram users

Not for the first time I stumble across advices like „keep kv cache at bf9999, that is a bare minimum or the Universe collapses” however whenever I tried to switch to higher bits kv cache I simply cannot notice any differences at all and soon return to my plain, old q4. My typical daily activities— 90% of agentic coding and 10% of some minor researches such as GPU prices analysis etc. I wonder whether there are some data that I might have missed that would confidently prove that „kv cache mattes” point or the opposite one