Qwen3.8 Flash Next GSQ-RCO IQ3_XXS running Strata on 2x Intel B70s
Flash-Next GSQ-RCO IQ3_XXS via Strata running on 3975wx Threadripper with 256gb DDR4 with 131k context. No speculative decoding. 4096 chunk prefill 1,563.4 tok/s and decode up to 58 tok/s.
Speed is great, prefill could probably be tuned even higher and MTP/Dflash should be tested eventually but the correctness of the IQ3_XXS quant is no bueno, it only passed 4/8 quality tests in my testing. Not safe as a local agent without supervision IMO. May test IQ3_XS or maybe a small q4 if it can fit.
Prefill chunk | Cold ~8K prefill | Cold 32K prefill | 256-token decode
128 | 270.55 tok/s | 263.6 tok/s | 53.95 tok/s
512 | 660.15 tok/s | 690.6 tok/s | 53.00 tok/s
1,024 | 687.05 tok/s | 754.7 tok/s | 51.70 tok/s
2,048 | 976.15 tok/s | 1,123.3 tok/s | 52.60 tok/s
4,096 | 1,226.00 tok/s | 1,563.4 tok/s | 52.00 tok/s