u/LegitimateWolf6611

Wrote a blog about running Qwen2.5-7B-Instruct served with vLLM on a Cisco UCS Spinifex cluster, comparing Intel AMX-accelerated CPU serving with NVIDIA L4 GPU. Go check it out!

https://docs.mulgadc.com/docs/cisco-ucs-llm-serving

Hey! I've been working for a very early stage startup called Mulga. We've been continuously working on our flagship product, Spinifex, which recreates core AWS services such as EC2, EBS and S3, locally (and even air-gapped if you want/need!).

Have a poke around our GitHub: https://github.com/mulgadc/spinifex

Feedback/criticism welcome!