My introduction to evaluation has not been very pleasant. To me it is a choice between investing time or spending money.

By that I mean you can either:

1. Spend a lot of time hand-crafting Q/A pairs and then perform manual checks to see if your system retrieves the correct chunks. And you repeat this for every single query. Suppose my dataset has 100 Q/A pairs, is this not a lot of work for a single person to do?
2. Use LLM-as-a-judge which automates much of what I described but you run into API rate limits pretty quick. I did figure out a workaround where I evaluate my dataset in batches (e.g: 5 questions per run) but again, it takes time.

I am not trying to complain here. I believe there is a more efficient way to run evaluation but I am new to RAG which is why I do not know proper evaluation techniques. Most metrics on DeepEval use LLM-as-a-judge which comes with API costs but I think I could try RAGAS and it's deterministic evaluation metrics.

Is evaluation in itself a time consuming process?

I need direction and guidance, advice would be much appreciated because I have basically put my project to a halt.

Hello everyone, as the title says I wanna start learning about RAG, at this moment I know absolutely nothing even though I'm doing a masters in AI (I know...) that's why I need help on how to actually start learning about this topic ? I know the best way to learn something is to build a project but I find it hard to build something when I have no idea where to start. I heard LangChain, Vector Databases, etc... but I don't know anything about these frameworks.

Hi guys, I would like to land premium clients for developing RAG systems. The thing is that I also need to make my master’s thesis (I study data science), and I was wondering what project could I build so that it fulfills my masters thesis and also helps me find premium clients afterwards.

I want to build something that creates real value for a company. Which problems could I solve for firms using RAG?

Thanks a lot in advance!!

I've been reading about RAG and I keep seeing people mention the retrieval layer like it's some separate thing you build, not just a vector db you query. Can someone explain what actually happens there? My current understanding is this: user asks question, question gets turned into an embedding, that embedding gets compared against a bunch of stored embeddings in a vector db, top matches come back, and those get stuffed into the prompt before it goes to the LLM.

Now, what I don't understand yet is: why does everyone say RAG with cosine similarity search isn't good enough. What's going wrong there in practice, is it pulling irrelevant stuff or missing the right things altogether? I keep seeing hybrid search mentioned, combining keyword search and vector search. Why would you need both if embeddings are supposed to capture meaning already? On top of that, I also saw someone mention reranking as a separate step after retrieval, so you retrieve like 50 chunks and then a reranker picks the best 10. Wouldn't that just make retrieval happening twice?

One more thing I keep wondering about, where does the data come from in the first place. Like if you're building a knowledge base from web content, how do people keep that fresh as possible? Constant scraping/crawling?

Thank you in advance

I've just completed the corpus on a RAG I'm building for local lawyers, realtors, and contractors who want fast information about every parcel in town without having to go through town hall's antiquated records system. I'm not trying to make a buck - just trying to learn how to make AI more precise in domains where that's important.

The corpus is tiny (only a couple hundred PDFs). I plan to use structure-aware chunking with contextual enrichment. Here are the steps:

• Parse PDF into text
• Detect structure (agenda items, sections, paragraphs)
• Split at natural boundaries
• For each chunk, generate a context summary via Claude
• Prepend context to chunk before embedding
• Store both the enriched chunk (for embedding) and the raw chunk (for citation)

Am I on the right track? Any land mines to watch out for?

Hey everyone, I’ve been thinking about a problem with AI knowledge systems and was hoping to get somewhat of a sanity check from people actually building in this space.

From my understanding, most RAG setups seem tied to a particular app, vendor, or index. You often end up ingesting the same docs again for different agents or runtimes and and some basic questions can be difficult to answer consistently like:

- What version of this knowledge is the agent using?

- Where exactly did this information come from?

- Has the underlying source changed since it was last ingested?

- Can I move the same body of knowledge to another runtime without rebuilding it?

- Can multiple agents use the exact same knowledge?

The idea I’m exploring is something I'm calling a Durable Knowledge Base (DKB).

The basic concept:

- Compile source docs, code, or structured data into a portable, versioned knowledge artifact

- Preserve source paths, hashes, citations, and provenance

- Sign and publish releases through a registry

- Allow knowledge packages to be installed, updated, pinned, and removed

- Let agents search, find, and read the same knowledge base across different runtimes

- Keep the artifact retrieval-agnostic rather than baking one specific top-K/RAG strategy into the format

Basically, I'm wondering whether knowledge should have something closer to a package lifecycle, rather than every application maintaining another disconnected RAG index.

I'm also very aware that things like Azure AI Search, GCP, vector databases, MCP servers, Agent skills, etc. already cover pieces of this problem, sometimes extremely well.

So Im specifically not asking: "Can I build a better enterprise search engine here?"

I'm trying to figure out whether the portable knowledge artifact itself is useful.

Would this solve an actual problem for you? Or is this mostly reinventing existing search/RAG infrastructure with some packaging and provenance added on?

I would especially like to know:

- What do you currently do when multiple agents/apps need the same knowledge?

- Do versioning and provenance actually matter to you?

- Would you ever install someone else's curated knowledge package?

- What would this need to do that existing solutions don't before you woukd bother using it?

Feel free to poke some holes. I'm actually looking for reasons not to build this further before I sink more time into it.

Thank you.

I haven’t seen RAG come up in agent architectures in over 6 months due to Agentic Search (letting the model use Bash/grep/glob/read), which seems to work pretty well. Wondering what others are experiencing. I’m sure there’s still a time and place for RAG, exposing semantic search as a tool… but where do we draw the line? When the corpus is too large to let the model comb through it progressively?

Automation+Systems engineer, years of machine commissioning. I built a RAG system for on-site troubleshooting: you're at the plant, machine won't run, whoever wrote the logic is asleep in another country.

Corpus: Siemens SCL / TIA Portal XML, Rockwell L5X, HMI config and alarm tables, schematics, I/O lists, FAT/SAT protocols.

Stack (client PoC, deliberately minimal infra):

• bge-small-en-v1.5, 384-dim, CPU via sentence-transformers. Corpus embedded offline, only queries embedded at runtime — customers won't send proprietary PLC code to an external API, and plant connectivity is whatever you get.
• Docs: Chroma + LlamaIndex BM25, fused with RRF. Immutable bundle built offline.
• PLC code and alarms: Postgres 16 + pgvector, Postgres FTS (tsvector), plus exact entity/tag lookup, fused with RRF in SQL. Code graph in plain Postgres tables — no Neo4j for the PoC.
• HMI: exact structured lookup, no embeddings. Embedding this layer made results worse.
• LangGraph multi-agent orchestration, LangChain Core init_chat_model, Vertex AI with Bedrock as portability path.

Three things this corpus taught me that generic RAG advice misses: exact match is the backbone not a fallback (MOT_CONV_03_FLT and MOT_CONV_08_FLT are vector-space neighbours and different machines); semantic chunking is actively wrong for control code, because the meaning lives in the cross-reference graph, not the block; and not everything deserves embeddings.

Now the actual problem.

I have no customers. Two industrial prospects agreed the problem is real, then said "we'll build it internally." Both have now started, and neither has anyone who has done this before. The pattern is identical in both: take a good engineer who knows Python, hand them the project, assume it's a weekend of work.

I understand why they think that. The naive version demos. Dense-only retrieval with fixed-size chunking over twenty documents looks finished. What isn't visible at that stage is how badly it degrades on a real corpus — and there's no eval set, so nobody finds out. The failure arrives later, at 2am, when someone is deciding whether to bypass an interlock based on an answer that sounded confident.

So I've been building PoCs for free to get in the door. I'm now suspecting that's the mistake: nobody had to get budget, so nobody owns it internally, and free reads as unproven rather than generous. But I also don't feel I can charge with zero track record.

What I'm asking:

1. How did you find your first paying customer? Not how you closed them, how you found them. Cold outreach, network, community, partnering with integrators/OEMs who already had the relationship?
2. Do free PoCs ever convert, or do they just train the market to expect this for free? How did you get paid for the first one with no history?
3. How do you beat "we'll build it internally"? Has anyone led with an eval harness or retrieval-quality audit as a wedge — proving their internal system is broken before proposing a replacement?
4. Product or implementation? Is a productized offering realistic here, or is the honest business bespoke implementation that gets productized slowly?
5. How do you find companies already spending money on this, badly? I've been targeting companies that have the problem. Every machine builder has it. Wrong filter — what's the right signal?

I am a decent engineer and a terrible salesman, and I'm running out of ideas before I run out of runway. Any of the five above, even a partial answer, would help.

Hey All, A few months ago I had published the "21 RAG Strategies" Book Here. And it was downloaded about 2500 times across subreddits. I made 2 revisions from the feedback.

This week I published it on Amazon and it became a best seller. I am getting ready to publish the next editition. Help me improve the content. What am I missing. what would you add?

Table of Contents

• RAG and the Reference Architecture
  • 02 The Evolution of RAG
  • 03 Foundations of RAG Systems
  • 04 Reference Architecture
  • P A R T I I I Data Extraction
  • 05 Data Extraction
  • P A R T I V Chunking
  • 06 Chunking Strategies
  • P A R T V RAG Strategies
  • 07 Baseline RAG Pipeline
  • 08 Context-Aware RAG
  • 09 Dynamic RAG
  • 10 Hybrid RAG
  • 11 Multi-Stage Retrieval
  • 12 Graph-Based RAG
  • 13 Hierarchical RAG
  • 14 Agentic RAG
  • 15 Multi-Agent RAG Systems
  • 16 Streaming RAG
  • P A R T V I Memory and Content Management
  • 17 Memory-Augmented RAG
  • 18 Knowledge Graph IntegrationP A R T V I I Evaluation
  • 19 Evaluation Metrics
  • 20 Synthetic Data Generation
• Fine-Tuning
  • 21 Domain-Specific Fine-Tuning
• Security
  • 22 Privacy & Compliance in RAG
• Production
  • 23 Real-Time Evaluation & Monitoring
  • 24 Human-in-the-Loop RAG
• Twig RAG Strategies
  • 25 RAG Strategies in Twig
• P A R T X I I Conclusion
  • 26 Conclusion & Future Directions

Hi everyone,

I’m working on a project where I want to build an AI agent for customer support. The idea is that customers can ask questions about technical issues such as SSH, IP addresses, DNS, VPS, Outlook, etc., and the LLM should help them diagnose and solve their problems.

I’m using my own knowledge base + RAG, but I’m still a beginner and I’m not sure what the best way is to structure my data for retrieval. I already have some old support tickets that I’d like to add to the knowledge base. These tickets usually contain:

• The customer’s initial problem/question
• A conversation between the customer and the human support agent
• Troubleshooting steps
• The final diagnosis
• The solution that was applied

For example, if a customer previously had an SSH connection problem and the support agent solved it by identifying a specific configuration/firewall issue, I’d like the RAG system to retrieve that previous case when the AI encounters a similar problem, so the LLM can use the previous solution to help the new customer.

My question is: how should I transform and structure these old support tickets before putting them into the RAG?

Should I keep the conversations as they are, or should I transform each ticket into something more structured, for example:

• Problem / symptoms
• Environment
• Diagnostic steps
• Root cause
• Solution
• Verification
• Similar scenarios
• Keywords / metadata

And how should I handle chunking these tickets so that the RAG retrieves useful parts without losing the context of the original conversation?

I’d really appreciate advice on how you would structure this kind of knowledge base, especially if you’ve built a RAG system for customer/technical support before.

Thanks!

Hi guys,

I've spent the last few weeks learning the basics about RAG. Howerver, I think one of the best ways to actually learn is by implementing a real solution for a company.

But, I just wanted to know, what do you recommend me learning/impelementing before offering my serivices (for free) to a company? any project I should build first?

I've been working around LLM/RAG systems and I'm curious about something from people who are actually running them in production.
When an LLM can retrieve information from internal company data, which problem has caused you the most trouble?
PII / sensitive information reaching the model
Compliance / privacy requirements
The model retrieving data it shouldn't see
Sensitive information appearing in the generated response
I'm especially interested in practical experiences — even small examples are useful.
Is there another RAG security problem that you think is more important than these?

We built a news search API for RAG — looking for feedback

We have been working on a new News Search API at Webz.io, specifically with RAG and AI agents in mind.

The basic idea is simple as you know is using natural-language query like:

>

The API searches Webz.io’s news data and returns the most relevant articles and matching content chunks.

You can also filter by date, country, language, source, sentiment, and category.

For RAG, the content chunks are probably the part I’m most interested in. You can feed the relevant section into your context rather than passing an entire article through the pipeline.

We also built an MCP server, so agents can search the news directly without having to build a separate API integration.

Docs: https://docs.webz.io/docs/webz/news-search

I’d be interested in feedback from people building RAG systems: when retrieving news, what matters more to you: retrieval quality, freshness, source coverage, or getting smaller relevant chunks back?

Has anyone dealt with a RAG system that performs really well during testing but starts returning irrelevant or incomplete answers in production?

We’re facing an interesting problem.

The knowledge base contains thousands of documents with overlapping information, different versions, tables, PDFs, and occasionally conflicting data. On a small test dataset, retrieval accuracy looks good. But once the document volume increases, the system sometimes retrieves a related document instead of the correct document.

The confusing part is that the LLM itself seems to be working fine. The problem appears to be somewhere between document ingestion, chunking, embeddings, retrieval, and re-ranking.

We’re considering several approaches:

• Hybrid search using vector + keyword retrieval
• Metadata-based filtering
• Better chunking strategies
• Re-ranking retrieved results
• Query rewriting before retrieval
• Adding document/version awareness

But improving one part sometimes seems to negatively affect another.

If you’ve solved a similar production RAG problem, what was the actual bottleneck?

Was it the embedding model, chunking strategy, retrieval architecture, metadata, or something else?

Would really appreciate practical suggestions or lessons learned from real implementations.