u/DivineEggs

I'll start by saying that I'm pretty illiterate when it comes to tech, and I've never used API or any cloud services aside from chatgpt and grok through the websites and apps.

I want to have a local llm in the future when I can afford the hardware, and I've gained more understanding of the tech.

I recently learned about Mimo 2.5v, and I need to learn how to go about it. I'm not expecting anyone to hold my hand and guide me step by step, but I'm hoping you can point me in the right direction where I can learn more and hopefully understand it.

I've tried to Google it, and I just get more confused.

Do I need some kind of specific app or software? Because, when I try to click the links on Xiaomi's website, the url seems to be blocked.

From what I've gathered, Mimo 2.5v is available on openrouter.ai, but I still don't really understand if it's a cloud service or API or a combination of both. And I don't understand if I need some kind of app to use it.

I want to use a cloud service so that I'm able to retain larger projects with memory and continuity. It also seems simpler for someone who is somewhat technologically illiterate.

Where do I even start?

Thank you for reading🙏💜

Hi, I'm generally not well versed when it comes to tech/hardware, so please bear with me🙏!

I currently have a Lenovo T490s Thinkpad, i5-8365U, with 16gb ram. Intel (R) UHD Graphics 620.

When I go to task manager and look at processes and "GPU 0" it says that I have 7,9GB of shared GPU memory, which Google claims works as VRAM when the computer only has an integrated graphics card🫣.

Would I be able to run Gemma 4 31B on this laptop?

Or at least Mistral Small 24B or Gemma 4 26B?

I don't really understand how to use "aggressive 3-bit quantization (such as IQ3_XXS or IQ3_XS)", at this point... so I'm thinking it may not be worth it to begin with?

I will mainly use it for chatting and writing. No coding or image/video generation.

I just need a (preferably heretic/uncensored, not pearl clutching at the very least) model that understands nuance and has a sense of humor while also being able to handle a large context window without melting. From what I've read, my current setup would require a very small (nearly useless) context window if I were to run Gemma 4 31B?

I understand that there are new smaller models that outperform some of the older larger ones when it comes to writing, but I'm also curious to know how I can know how much Shared GPU memory/vram a computer actually has *before* buying it. Is it safe to assume that roughly half of the RAM is shared GPU memory if it's an integrated graphics card and nothing is specified, generally speaking?

Last question, would I be able to run the even larger 70b models on a laptop with 32gb RAM (integrated GPU) or is 64gb necessary to not have to wait an eternity for every answer since I'm dealing with a lot of text?

Thank you for your patience and for reading this far💜.

I appreciate any clarity and help y'all can provide me with because this is making my brain melt a little🫠!