How are people managing AI costs?
Just like everyone else, I've been seeing the recent news about how AI bills have been skyrocketing for companies. I've been seeing people Reddit posts / comments about how their companies have done a full 180 from "use AI for everything" to "limit AI usage as much as possible".
So I've been wondering - what mechanisms are folks actually using to monitor and control AI costs intelligently? I know the most basic version of this is just seeing your bill at the end of the month, having a heart attack, and then telling employees to stop using AI. But there must be a smarter way to do this right?
Is there some way to track AI usage across departments, task types, and employees (across different LLM providers?). Can managers set limits on what they want their AI budget to be so that you don't get an unexpectedly high bill? Maybe then you could switch low-priority departments or tasks to cheaper model or just stop allowing AI usage for that department for the rest of the month
Just curious on why AI bills are so shocking to people - I assume people are setting hard caps on token usage.