u/Fun-Meaning-6474

Qwen released the 2.4T Max weights and I was curious how well it can re-create COD in one prompt

I ran the model on a rented B200 cluster and used roughly 1.1M output tokens over a 5 hour time span

Realistically barely anyone can run this model locally, but the fact that it's open weights unlocks a lot of opportunities for the local AI community even regarding quantization options that might bring us closer to running frontier level intelligence on consumer-ish hardware 

For consumer hardware there is a 27B model from the same release and it's really capable for its size, many people have posted 3D games made with it on X

I'm from the atomic.chat team, so If you wanna try running the new Qwen locally you can do so inside our app (any feedback is appreciated, since we want to make our product as good as possible for you guys) 

We have our own and other community quants of the 27B model available to download directly inside the app

P.S. I am a noob in shooters so pls no hate my low skill gameplay

Liquid AI put out LFM2.5-VL-3B today, which is a 3.1B vision model that weighs roughly 2GB and fits well on a phone

Benchmarks are benchmarks so I tried something sillier. Took a photo of a little Steve toy I have, gave it to the model and asked it what it was looking at

It ended up thinking for around 2 minutes and 31 seconds on an iPhone 17, which is a bit too lengthy, but it did end up recognizing Steve and gave a detailed description of him

The main diff from the last gen is that it got much better at spotting where things are. ScreenSpot-v2 desktop went from 6 to 78.7. That's why it describes Steve part by part rather than just naming him

LFM2.5-VL-3B HF card: https://huggingface.co/LiquidAI/LFM2.5-VL-3B

The model was run through atomic.chat mobile app (I'm the founder, so any feedback is welcome)

we're experimenting with our own dynamic GGUF quants of kimi k3, made from the original weights with our llama.cpp fork. Q3_K_S is done and works 1114.76 GiB on disk. Q1 and Q2 are in progress, results on those tomorrow

rented box hardware:

- AMD EPYC 9554P, 64 cores
- 1.5 TB of DDR5
- NVMe in raid0 to store the weights (inference runs fully from ram)
- no GPU

the run:

- 110 threads
- pp512: 4.21 t/s

we ran a short test for text coherence and image understanding to make sure the quant isn't lobotomized. loaded the 1969 NYT "men walk on moon" front page and asked the model to describe what's going on. it listed the masthead, the "all the news that's fit to print" slogan, the date, the 10 cent price, the headline, the sub-headline about astronauts collecting rocks and the "voice from moon" column. we haven't noticed any hallucinated text

wdyt about running quants of giant models like this on cpu instead of going with smth smaller but with normal tps and zero extra costs?

disclaimer: we're the team behind atomic chat ( atomic.chat )