▲ 136 r/LocalLLaMA
If you are wondering why Ornith 1.5 35B A3B with MTP is so slow, this is why
The author of the HuggingFace post discovered that Ornith-1.5-35B-A3B is currently being shipped with a MTP head that was never actually trained — it's just random initialization.
u/Max-_-Power — 23 hours ago