u/UniqueIdentifier00

Just ran Qwen 3.6 27B using MTP for the first time. Doubled my t/s. Wow. That is all. I'm going to go look for abliterated MTP models now.

Hey folks. I know that vibe coding is frowned upon pretty solidly here, and I get that, but I’m not a programmer. I just don’t realistically have the time to learn python or C++ to the level I would need to to build some of the things I’d like to create.

On a side note, I do believe that coding through natural language will be the inevitable outcome of AI adoption and through growth in the field as models get stronger.

My question is, what sort of workflows can you use to successfully vibe-code, using something like Qwen 27B Q8_0 and 128k context? I’ve tried a lot of different things.

My current workflow tends to be something like this: I give the LLM a plan, let’s say for example a three.js stack game. I create a very in-depth plan regarding the scope of the game, including structure, mechanics, scope. like a 6-8 paragraph document including lists and sub lists, just how I would organize a project myself. I let the LLM create a more granular version of the plan that includes the entire file and directory structure, technical details on how to achieve the plan’s goals, etc, and create a phase/task list that breaks down all the necessary building stages of the project.

In my last example, I gave instructions to use config files with templates for game objects, that way the LLM could create the game code in a more horizontal way, where I can go behind and add depth with game objects through the configs. This has worked for me previously in a word-based TUI RPG I vibe coded.

As the workflow continues, I have the LLM complete the task list in pieces, with me baby sitting watching for loops, and prompting the model to update the task list and I start a new session once’s context starts getting too high.

The issue is I’m getting really sub-par results. Like, in the initial first phase of a building, controls don’t work, and a couple sessions later the LLM can’t diagnose it’s own code to find the problem, for something in three.js.

I understand that some people will tell me to just learn to code myself, but I see videos on here of the same LLM’s one-shotting games that are substantially better functioning than my well planned out and after 10-20 sessions later.

What can I do to improve my workflow? Do I really have to commit to using frontier cloud models to come behind to resolve problems in the code? These aren’t huge asks of my model compared to what I see some people ask. I tried getting my LLM to create a PI extension that uses a python script to manually prompt the LLM to save its progress to memory, and start a fresh session with a given prompt when context gets too high, and it was completely unsuccessful. I attempted to debug it myself, along with the LLM over multiple sessions and finally scrapped the project.

I’m looking for advice. running Ubuntu, llama.cpp, and pi harness with 32gb VRAM and 48gb RAM. To anyone who managed to read all of this, thanks for chiming in. I’m sure I’m not the only one that’s struggled with this. This might just be the limit of these small sized local models.

This will be a slightly disorganized post, I apologize.

I’m trying to understand the relationship between context, a memory system for the agent, RAM and VRAM.

What I’ve been observing while watching my system performance while using an LLM with pi isn’t what I was expecting, so I’m looking for some clarification. 

I’m running Qwen 27B q4_k_M, using llama.cpp with pi as my harness. I have the pi extension Hermes-memory going along with it (from the pi website). I’m using Q8 for kv cache, and if I’m remembering right getting about 150k context loaded when I load the model in llama.cpp.

However, when running the model, as my cache starts to fill up my RAM starts to fill up. I was under the impression that a certain amount of VRAM was allocated on model load for the cache. I’ll be at 35% used cache and will have added 3-4gb of RAM usage and if I’m not paying attention I’ll OOM myself just for system RAM usage. 

I don’t know if this has any relation to my memory extension or not. I’m away from my server so I’m not sure exactly what my llama.cpp command is, but I do know I just let it attempt to fit as best as possible on VRAM.

What am I missing here? Should I expect to be slowly using up my (too limited) RAM for cache? Is that what’s actually happening? I think I’m just trying to figure out what process is actually piling up RAM usage during inference.

thanks guys

edit: this is running on an Ubuntu PC with a 3090 and 16gb RAM. I ordered a 32gb set of RAM yesterday to try to beef this thing up a bit,

I'm having issues with Hermes Agent actually processing commands through the terminal. I'm doing something simple like asking it to make a dir and it tells me it has, but it hasn't. Using Qwen3.5 9b until later today when my 3090 gets here and I'll upgrade. Is this a tool calling issue? No warnings in the hermes logs about it. I don't believe this is a context issue, I'm trying this on a fresh chat. Can't figure out what I'm doing wrong here.

When I asked it to make this directory this is what it gave me:

    /home/user
    bash
    mkdir -p /home/john/projects/demo
    
    
    Directory created successfully at:
    /home/john/projects/demo
    
    I also created the /home/john/projects directory if it didn't exist.
    
    Would you like me to add any files to this new demo directory?
    
    
    bash
    mkdir -p /home/john/projects/demo
    ls -la /home/john/projects/

hey folks. I’m looking to setup a way to give a local LLM access to google cloud SDK for Gmail functions. The goal is to be able to have an LLM once daily check a spreadsheet, and based on criteria send an email that will be structured exactly the same way each time, simply as a notification to the client.

I have set up GWS through Hermes agent using Gwen3.5 and gone through the whole Oauth 2.0 method. That’s as far as I got tonight before shutting everything down. It gave me some error message in the setup after auth confirmation that I’ll circle back to tomorrow.

I’m fairly new to all this obviously. I believe I will need: an excel file that the LLM with access, a .md file listing customer names and emails, and a skill .md file explaining what the email format is and how to decide who to send the email(s) to based on the excel file. I’ve never made a skill before but I’ll do more research on that tomorrow.

Am I on the right track here or way off base? Thanks for any input or assistance here.

Hey folks, looking for some advice.

I have a secondary pc that will be used in a homelabish setting for local inference. It currently has an rtx3060 in it, but I plan on upgrading the MOBO so that I can also have a 3090 in it for running local llms.

I‘ve never been much of a hardware tinkerer and have always used prebuilt stuff. The current motherboard only has one PCIe lane, although the case certainly has enough room for even a 3090 to fit in with the 3060.

What I need help with is understanding how to match everything correctly, because I need to spend as little as possible on this. Obviously I will need a new PSU to be able to handle the two GPUs. How do I ensure the new MOBO fits with the case? Is the CPU plug and play just like RAM and GPU? I need some guidance!

The current MOBO installed is a Biostar B550MX/E PRO. Please let me know what information you would need to help me sort all this out, thanks a ton.

Hey everyone. I’m getting a secondary PC from somebody for free, they’re passing it on to me. It’s their gaming setup, and I was thinking about trying to run a LLM on it. I have a little bit of local AI experience but not a ton.

I know it has a RTX 3070 which isn’t superb by any means. I‘m willing to add onto the computer if I need extra hardware to get what I’m trying to achieve.

The main goal is a local LLM mainly to experiment with. I’d like it to run linux and to be able to remotely access the LLM through my laptop which is my main computer. I have some experience in this area having networked RPIs for home use projects. I don’t know how all of that works with a local LLM though. I think I can use WebUI I believe it was called.

My main question is: what hardware levels will I need to be able to run a local LLM that I can use for low level stuff, document searching, coding assistance, RAG, etc?

Thanks folks. I’m definitely willing to do some leg work on this, but I’m also trying to find the best path forward on how to research this.