u/Prize_Carpenter5423

I've been working around LLM/RAG systems and I'm curious about something from people who are actually running them in production.
When an LLM can retrieve information from internal company data, which problem has caused you the most trouble?
PII / sensitive information reaching the model
Compliance / privacy requirements
The model retrieving data it shouldn't see
Sensitive information appearing in the generated response
I'm especially interested in practical experiences — even small examples are useful.
Is there another RAG security problem that you think is more important than these?

I’ve been digging into the math of running an AI automation agency, and I think a lot of us are accidentally operating at a negative gross margin without realizing it.

When we build an agent loop for a client (especially with retries on failure), a single task can trigger 10+ LLM calls. If a client’s volume spikes, a $300/mo retainer can easily swallow $400 in hidden API tokens.

Right now, the tools out there (like LiteLLM or Helicone) either require you to self-host complex dev infrastructure, or they just show a dashboard without letting you actually bill the client or put a hard stop-cap on their usage.

The Question: How are you guys passing these costs onto clients? Are you just eating the cost in your retainers, flat-charging, or did you build a custom internal solution?

P.S. I got tired of guessing, so I built a dead-simple proxy relay that lets me add a markup and automatically bill clients via Stripe based on their actual token usage. If anyone running a real agency is losing sleep over this and wants to try it out for free to see their actual client margins, drop a comment and I'll DM you access. No selling, just need real feedback.