u/ApplicationOk9465

Hey everyone,

I’ve been thinking about a project idea and I’d love to get your feedback.

The idea is to take a 1TB SSD and turn it into a fully portable AI system. Basically:

• Install Ubuntu (or another Linux distro) on the SSD
• Set up tools like Ollama or llama.cpp
• Load it with open-source LLMs (Mistral, Gemma, TinyLlama, etc.)

The goal would be:

• Plug-and-play SSD that boots into Linux
• Run AI locally (offline, more privacy)
• Use it across different machines

What I’m trying to figure out:

• Is this setup actually practical?
• What are the best lightweight models for a portable setup like this?
• Any recommendations for optimizing performance (quantization, memory usage, etc.)?
• Are there better tools or stacks than Ollama/llama.cpp for this use case?

Also thinking ahead — could this be turned into a small product (like pre-configured SSDs with local AI)?

Has anyone here tried something similar or seen a setup like this?

Appreciate any advice 🙏