AI Gateways???
With all the expanded partnerships and capabilities between the labs and cloud providers, can someone explain to me why a company should spend 6 figures on an AI gateway? I get if you're multi cloud or have on-prem bullshit to deal with, but if your just down the middle using AWS, why are you not keeping things simple with Bedrock and just building any thin layers needed internally? All your compliance is already covered with AWS...?
Internally, if your company is running rampant using all LLMs from God knows where and all these agents, etc, etc, blowing through tokens, and there's that much variation and the train has left the station and all that....I'd assume by even having that problem usage is somewhat more sophisticated and would suggest a more thoughtful solution? But it also seems like a lot of these situations are just from people blasting through chat interfaces? I'm just confused about who needs this outside of the obvious cases? A lot of this is just structural internally and can be fixed without buying something? And if it's truly a shit show buying something that just gives you a better view to the chaos isnt a real solution..?
Not to mention, what is this even going to look like in 3 months? Any gaps will likely just fully close...?
Someone please help me. Tell me I'm an idiot. What am I missing?