Weirdness with GLM 5.2 on Nvidia Nim (and a potential cause for it?)
So, let me preface this with that I've been a NIM user for quite some time now. It's an amazing API... when it works, which is the issue I'm having with it.
Now, I know this is a free endpoint and realistically, complaining about it is going to fall on deaf ears... but over the past 3 to 4 weeks, I've noticed a pattern:
The endpoint works... okay, nothing wrong with it, but it seems to be quantized decently. It's also slightly slow, but nothing a bit of waiting can't fix.
Then, sometime during the week, it works PERFECTLY; like I'm being given an FP8 with BLAZING speeds.
Sometime later, it slows down slightly, but it's still very much fine.
Then, it just COMPLETELY shits out; I can't get a single message through through the API... and usually, it comes with the error of 429, or 'Too Many Requests'. This is weird... since I'm not running any agentic processing, and I send one message over the timespan of 4 minutes. (Also note, my tokens sent is around 100000, which is fairly standard for this endpoint from what I've seen due to the heavy coding/agentic emphasis)
Rinse and repeat, this is the cycle I'm constantly seeing on my end... and I have a sneaking suspicion why, and surprisingly (if my theory is correct), it isn't even due to the fact GLM 5.2 is going away soon. I've noticed when I can't use it and others are complaining... some people will say they are having ZERO issues with it.
Huh... sounds familiar, doesn't it?
It gets even stranger, the message on this sub from Plus-Switch 13 hours ago that states THEY were having the same issues I'm having right now, where they can't get a single message through; had me QUITE intrigued when I was using GLM at the BLAZING speeds I had at that same time.
If you're not picking up what I'm putting down: I'm 90% sure Nvidia has a system in place that is gating access based on some level of factors. I saw some post earlier (that I can't find...) that mentioned there's a hidden amount of requests you can do before you hit this limit, but I think it's even simpler than that; where they have a set amount of time with any specific endpoint, ESPECIALLY if you use it a lot.
That's just what I've noticed though, but with any of my posts here, I want to get to the bottom of it! So please, if you are a heavy NIM user like I am, please tell me YOU'RE findings/perspective on this topic! We're all LLM enjoyers, and this is the one place we can gather our information to see how we can progress things... even if it's just seeing what providers are doing behind the scenes!