New to Local LLMs, need some advice
Looking to temper my expectations a bit with local LLMs. I've recently just started experimenting with hosting models on my own hardware with Odysseus, and compared to some of the web models I've used, I'm really running into some roadblocks.
I know that hardware is limiting what I can run compared to Anthropic and GPT models, but is the attached picture really the standard for local models?
I'm using gemma4-9B with 16k context, was testing out the Agent method by asking it create a "hello world" webpage using python. It executed it flawlessly, and when I asked it to stop the script, it's as if it lost all memory of it's previous work.
Is this user error? Or limitations of my model/hardware? Is the model running out of context? Thanks for any insight/roasting