u/big-in-jap

Trying to figure out where newer Llama models actually become practical in Ollama. ollama pull works on a small card; living with it daily doesn't.

Ollama tag sizes (Q4_K_M) — official library, not my benchmarks:

Model	Ollama tag	On-disk (Q4_K_M)	Params	Source
Llama 3.3 70B	llama3.3:70b	43 GB	70.6B dense	Ollama
Llama 3.1 70B	llama3.1:70b	43 GB	70.6B dense	Ollama
Llama 4 Scout	llama4:16x17b	67 GB	109B total, 17B active (MoE)	Ollama
Llama 4 Maverick	llama4:128x17b	245 GB	402B total, 17B active (MoE)	Ollama
Llama 3.1 405B	llama3.1:405b	243 GB	406B dense	Ollama

MoE gotcha (Llama 4): Scout has 109B weights on disk but only ~17B participate in each token's computation. You still need ~67 GB VRAM because Ollama loads all experts — the router can call any of them on the next token. Maverick is the same pattern at 245 GB. (Meta Llama 4 post). KV cache stacks on top, of course.

Hardware ladder — which tags fit where:

Tier	VRAM	Example	3.3 70B (43 GB)	4 Scout (67 GB)	4 Maverick (245 GB)
Consumer	24 GB	RTX 4090 / 3090	No — offload	No	No
Consumer	32 GB	RTX 5090	No	No	No
Workstation	48 GB	RTX 6000 Ada	Tight	No	No
Multi-GPU	96 GB	3× RTX 5090	Possible (split)	Tight / split	No
Unified	128 GB	DGX Spark / GB10	Comfortable	Fits	No — need ~245 GB+

I'm aiming for the 128 GB tier: first unified-memory class where llama3.3:70b (43 GB) is comfortable and llama4:16x17b / Scout (67 GB) actually fits, with headroom for context. Still not enough for Maverick (245 GB) or 405B (243 GB). Looking at DGX Spark / GB10, Mac Studio-class boxes, or renting hourly when I need a weekend on Scout rather than buying.

Anyone actually running Scout locally? Or still on 3.3 70B?

• 24–32 GB and living with offload / smaller models?
• 48 GB (6000 Ada) for 70B-class dense?
• 96 GB+ or 128 GB for Scout?
• Renting when you want Maverick / 405B-class sizes?
• Accepting the quantisation lower accuracy?

Especially curious if you're NOT on the box 24/7; does buying 128 GB hardware pencil out vs hourly rent?

Hello locals,

Kept seeing "where do I actually rent a DGX Spark" questions with no good answer, so I went and catalogued every option I could find. Posting it here in case it saves someone the search.

Remote access (cloud — you rent the GPU, connect over SSH)

• Enverge — from $0.65/hr, 128GB, SSH + Docker, hourly pay-as-you-go, no commitment
• gb10.studio - mostly for inference
• VFX Now (US) — rudimentary cloud access; also offers physical
• Primcast — dedicated/monthly hosting rather than hourly

Physical rental (the box ships to you — per week, UK)

• HardSoft
• Scan — per-week, includes a clunky cloud-access option too

Quick takeaways

• For a weekend experiment, hourly cloud is the cheapest by a mile.
• If you need it physically on your desk (data residency, air-gapped, privacy), the UK per-week physical rentals are the only real route right now.
• Buying is ~$3–4k; rough breakeven vs $0.65/hr is ~5,000+ hours, so unless you're running it near-constantly, renting is the call.

What did I miss? Will edit the list with anything good in the comments.

Anyone DIY-ing this?