u/recro69

I've spent a lot of time fixing RAG systems. I think "our RAG is inaccurate" problems are actually about finding the right information, not about the model generating answers.

The model usually isn't making things up. Its answering based on what it was given.

The real issue is that it's getting the piece of information to work with.

The biggest improvements I've seen come from:

• Breaking up documents into chunks based on how they're structured not just using fixed sizes.
• Adding a step to reorder the results after searching for vectors.
• Creating a test set from questions people asked instead of just guessing what would work.

What surprised me most was how difference it made to switch models compared to improving how we find the right information.

Models that can reason make this even clearer. They don't fix information. They just give a more convincing answer based on the wrong idea.

GraphRAG definitely has its use for complex questions that involve many connected documents. For simple questions about a document, I've often found that fixing how we break up documents and find the right information solves the problem before we need a more complicated system.

For those who've shipped RAG to production:

What ended up making the biggest difference for accuracy? Was it the model, or was it finding the right information that was the real problem?

I've spent a lot of time fixing issues with our RAG system. I think most problems with it not being accurate are actually problems with getting the information not with generating answers.

When I looked into the issues people reported I found that the model wasn't making things up. It was answering based on what it had.

The real problem was that it was given the part of the document to work with.

The biggest improvements we made weren't changes to the model. They were:

• Chunking documents based on how they're structured, like headings and sections of just using a set number of tokens.
• Adding a reranker after searching for vectors.
• Creating a set of test questions from users to see if getting the right information was actually improving, instead of just relying on instinct.

I was surprised by how little changing the model helped compared to improving how we get information.

Models that reason make this even clearer. They don't recover from getting information. They just produce a very convincing answer based on the wrong idea.

I'm curious what others have seen after putting RAG into production.

Did the biggest gains, in accuracy come from changing the model. Did you find that getting the right information was the real problem?

I saw someone test Qwen3.6-27B with a 3-critic harness. The harness included code review, test review and Playwright e2e. Each critic had context. The result was that the model is usable for coding work. This matches what I have come to believe from running agents in production. The harness around the model is more important than the model itself.

A smaller model makes mistakes. That is real and expected.. A good critic pipeline catches the extra mistakes. Suddenly the gap between a 27B model and a frontier model gets smaller. The reliability comes from the process, not the model size.

The mistake I see teams make is that they focus on model selection and prompt-tuning.. They do not verify the results. They blame the model when it is flaky. The model is not your reliability layer. The harness is.

Fresh context per critic is important. A reviewer that has not seen the code catches things a self-review never will.

For people running models in production I ask: where does your reliability come from? Is it the model or the scaffolding, around it?

While building AI agents I noticed something that really surprised me.

When I started out, I thought that using a model would make a huge difference in reducing mistakes and wrong tool usage... That wasn't the case. What actually made the biggest difference was giving the agent information from the start.

Some things that helped were:

• Clearly defining what the agent's job is and what its supposed to do.
• Specifying where the agent works and what rules it has to follow.
• Stating what actions, the agent can take and what tools it has.
• Limiting the tools. Only giving the agent relevant information.

Once I had those things in place the agent seemed better at understanding what I wanted it to do. It was less likely to think I was asking it to do something or choose the wrong tool.

I'm curious if other people building AI agents have seen the thing. Have you found that giving the agent information and guidance makes a bigger difference than just using a bigger model? Or did switching to a model make a bigger difference for you?

I'm especially interested, in hearing from people who are already using AI agents in real-world situations or have workflows set up.

There are a lot of posts about the models and benchmarks, but I am more interested in the workflows that people use. What is one workflow that really saved you time or made your local LLM more useful?

It could be anything—RAG, MCP, coding agents, organizing prompt, document indexing, automation or something else entirely. What was it, and why did it make such a big difference in your day-to-day workflow?

I have been spending a lot of time looking at founder content on LinkedIn recently.

One thing I keep noticing is that the posts getting the engagement are not always the founder content posts that generate the most business for the founders.

The founders who seem to attract the meaningful conversations usually share specific experiences instead of generic advice.

They talk about problems they faced as founders.

They start conversations of broadcasting their expertise as founders.

My takeaway from looking at founder content is that people engage with insights. They trust experiences of the founders.

For those people who are managing founder-led content I was wondering if you have noticed something with the founder content you are managing.

What type of founder content has actually generated business results, for you from the founder content you have been managing?

I'm wondering if anyone else has come across this.

I've tested the same model on llama.cpp and vLLM with similar settings and quantizations. The performance and concurrency in vLLM are much noticeably better, but sometimes the model feels less reliable.

Some things I've noticed:

* More mistakes with formatting and tool calls

* Forgetting context suddenly

* Sometimes acting like messages didn't exist

* Lower quality code even with similar parameters

I'm not trying to start a comparison. I just want to know if others have seen differences in quality between inference backends... Is it usually because of quantization, chat templates, parser problems or configuration errors.

What has your experience been, like?

Every day I see posts about startup founders who are doing well they are making a lot of money getting a lot of funding or creating something new in just a few days.

I know that these posts are not about people they are about a few startup founders who are doing well.

When I see these posts every day I start to think that everyone else is doing better than me that they are moving forward faster than me.

Do you think that social media is giving people the idea, about what it means to have a successful startup or am I just looking at social media in the wrong way I mean about startup success?

I am curious to know what kinds of behaviors people have seen that were not programmed into the language model agents.

I do not mean mistakes or things that are not true. I am talking about patterns that seem to happen on their own.

For example:

* Agents creating their own workflows

* Unexpected tool-use habits

* Persistent personalities

* Strange total dynamics between agents

* Recurring beliefs or preferences

What is the weirdest thing you have seen a language model agent do that you did not tell it to do?

What kind of language model and setup were you using?

Building a SaaS is technically not that hard anymore.

With AI made templates and modern tools I can turn an idea into a working product really fast.

Getting someone who does not know my product to actually pay for it?

That still feels just as tough.

What has been the difficult part, other, than technical stuff of building your SaaS?

AI is getting better at a lot of marketing tasks. I still find myself spending more time fixing some things that's AI produces than I would if I did the work manually.

What is one thing you still would not trust AI to handle on its own?

Local models have gotten a lot better in the year but it seems like there are still some things that models on the internet can do better.

For me it is usually things like thinking through problems writing that sounds good and doing tasks the same way every time.

I want to know what other people think.

What is the one thing that local language models are not good at that stops you from using them all the time, as your helper, like a personal assistant?

I have been curious about models because they make sense in theory. Local models are good for privacy there are no API limits. I have more control over them. However I am not sure if local models are actually good enough for use.

When I say use I mean things like writing and brainstorming and summarizing and explaining concepts and planning and general back-and-forth thinking with local models. I do not mean using models for coding or benchmarks.

Every time I try to use models I eventually go back to using ChatGPT. This is because ChatGPT is faster and the quality is higher and it is more convenient to use ChatGPT.

Maybe I am not setting up models correctly or maybe local models are just not good enough yet.

So I am curious about models.

Has anyone fully switched to using models for daily work, with local models?

What models are you using and what makes these local models good enough to replace ChatGPT?

Is it still mostly a hybrid setup that uses local models and ChatGPT for most people?

It seems like every SaaS product is adding AI these days.

But I'm curious, how many of those features are solving a real customer problem vs. just following the trend?

Have you found an AI feature that really made you stick with a product?

I saw a great checklist today for validating B2B SaaS ideas: existing market, monetizable, large market. These are fundamental truths, and they apply just as much to AI-powered SaaS as to any other.

My take: AI engineering studios like Skygnosis accelerate the *build* part. Our 28-day process gets a production-ready AI MVP out the door quickly. This isn't just about speed; it's about enabling founders to test their assumptions against these checklist items in the real world, much faster than traditional development. 'Vibecoding' can get you to a prototype, but getting to a $10k MRR requires a production-grade system that can actually deliver on the market need. What's missing from this checklist for AI-first products?

I need a plumber in vesu area do any one have contact let me know