
Trying to make sense of Model Benchmarks
I'll preface by saying i'm not a developer.
i'm just curious and eager to learn more on LLMs and coding.
I have opencode setup wit oMLX on a m1 max (40c) 64GB
i've been going through the oMLX benchmarks and looking through best options for Qwen (general coding) and Gemma (general research/reasoning)
https://omlx.ai/benchmarks
This is where i think i'm getting confused.
I'll apologize in advance if my qtns are somewhat amateurish.
i get i should be looking at the larger models (e.g 30B)
I understand a higher quant is preferred for coding (e.g 8bit)
with context though, shouldn't i be looking at higher context for coding sessions. If that is the case, doesn't that in turn lead to a larger KV cache size and chew in more onto memory.