u/Helpful_Regular_30

I started learning generative AI thinking most of my time would go into understanding models.

Ended up spending time on completely different things.

One day I was reading about prompts, then embeddings, then vector databases, then RAG, then trying to understand why a model was giving weird outputs even though everything looked fine.

I also realized building something yourself feels very different from watching tutorials. I'll watch a 20 minute video and think "okay that looks straightforward", then spend the next few hours trying to figure out why something isn't working.

Not complaining or anything, I actually like it. I just didn't expect the learning process to go like this.

Curious if anyone else had the same experience or if I just went down a weird path.

I've been spending some time learning agentic AI lately, and it feels pretty different from how I learned ML or even basic LLM applications.

When I was learning ML, I was mostly thinking about datasets, training models, evaluation metrics, and improving performance. With a lot of basic LLM projects, I spent more time around prompts and connecting APIs.

But with agentic AI, I noticed I started running into different questions:

• Should the agent use a tool here or not?
• How much information should it keep in memory?
• How do you stop agents from taking unnecessary actions?
• How do people usually structure these workflows?

I thought the coding part would be the difficult part, but for me it wasn't really that. Most of my time was going into understanding how the whole system should behave rather than writing code.

Still figuring things out, but curious if anyone else felt the same while getting started.

What confused you the most in the beginning?

[ Removed by Reddit on account of violating the content policy. ]

Built a basic RAG setup a few months ago. Retrieval looked fine, model was decent, but the answers were consistently half-wrong or weirdly incomplete.

Spent way too long suspecting the LLM. Swapped models twice. Still bad.

Turned out the issue was how I was chunking documents.

I was using fixed 512-token chunks with no overlap. Clean, simple, felt logical. But the retrieved chunks kept cutting sentences mid-thought, sometimes right before the actual answer, sometimes right after. The model was working with literally incomplete information and hallucinating the rest.

What actually helped:

1. Adding overlap (obvious in hindsight) Went from 0 overlap to ~50 tokens. Retrieval quality jumped immediately. The "answer" wasn't getting split across two chunks anymore.

2. Respecting natural document boundaries Splitting by paragraph or section instead of raw token count made a huge difference for structured documents like PDFs and docs with headers.

3. Smaller chunks + more of them Counterintuitive but retrieving 6 small clean chunks beat retrieving 3 large messy ones. Less noise in the context window.

4. Checking what actually got retrieved I wasn't logging retrieved chunks at all early on. Once I started printing them, I immediately saw the problem. Obvious step I skipped because I assumed retrieval was working.

The model was never the bottleneck. The garbage-in-garbage-out problem was upstream the whole time.

Curious if others ran into this, especially with PDFs. Those feel like a special kind of painful.