u/kondamuri

Curious how teams are handling this. Between OpenAI/Anthropic API bills, GPU instances on RunPod or EC2, and vector DB costs, it seems like most places have one big number and no idea which feature or model is driving it.

• Do you know your cost per request, or per user, for anything AI-powered?
• Is anyone tracking token spend by feature, or is it all one line item?
• If you self-host, do you know your actual GPU utilisation, or is it "the box is up"?
• Has anyone gone through and actually cut this — what worked?

Genuinely curious how other teams handle this, because at every place I've seen it works differently and mostly badly.

The bill comes in, someone says it's too high, and then... what? Is there one person whose job it is to dig in? Does it fall on whoever is least busy that week? Does it just get ignored until finance escalates?

Specifically wondering:

- Is anyone on your team formally responsible for cloud spend, or does it float?

- When the bill jumps 20% month over month, how do you find out why? Dashboards, a tool, or someone manually clicking through Cost Explorer for two hours?

- Do you review it on a schedule, or only when someone panics?

- Has anyone actually done a proper line-by-line audit, and did it stick or did spend creep back up in six months?

Mostly asking because I keep seeing the same pattern — everyone knows the bill is too high, nobody owns it, so nothing happens. Curious whether that's universal or whether some teams have solved it.