My work has implemented AI to count our every word. It is programmed to catch only key words, so if we do not say something exactly how the AI is programmed, it assumes we did not have the conversation at all. I work from home in inbound sales for a travel company. Today we were told we must say “Vacation Ambassador Sweepstakes” on every call—orders from upper management, the usual.
The problem is, we legally cannot offer this sweepstakes to residents of California, Florida, Colorado, and Rhode Island due to state laws. I took four calls and got a message from a supervisor on day one of this new expectation saying, “You took four calls but only said it once.” I responded, “That is correct. Three callers were from NY, FL, and CO.” He joked that I’d win a prize if I got Rhode Island next.
So your AI clearly can not distinguish that I legally could not offer it on three of the four calls.
I then asked the head of sales how that was being accounted for in the metrics and whether those calls were being removed. He wasn’t sure. The head of the sales department was not sure how that was impacting our metrics. He said he thinks it isn’t counting against us, but he wasn’t sure, called it a good question, and said he’d get back to me.
That alone is concerning.
Now their AI also grades our calls. Let’s say I take 10 calls and 4 of those are customer service calls. Again, I am in sales—my job is to close sales—but I’m also expected to take service calls, which takes me away from selling. Their AI does not know it was a service call; it only knows I didn’t close a sale. I’m probably not closing a sale when Ethel is calling to cancel because her husband died.
So 4 out of 10 calls are customer service. Now let’s say I get 2 ghost calls—dead air. That’s 6 invalid calls and only 4 real sales opportunities. Yet AI reads that as 60% of my calls not closed. AI can’t tell the difference between those calls.
Now that 60% counts against me and I have to climb mountains to overcome it. I could go on. There are so many flawed ways they count numbers against us that AI is removing the human interface from.
For example, if I have a budget traveler calling for a last-minute deal—say they got a two-week cruise for $1,295—then I run a flight search and I’m expected to pitch premium for $5,897 or business for $9,432, which is clearly insulting to a budget traveler. They are likely not paying $9,000 for an upgrade, and somehow that is my fault because AI cannot decipher quality over quantity. They are using AI and forgetting we are humans. Instead of measuring close rate on qualified leads, it is close rate on every interaction, at the mercy of AI. AI has turned into an absolute nightmare for someone like me.
Do you know what it feels like to have your every move and word poorly graded by AI? Demoralizing. I feel as though I’m walking on eggshells trying to keep up with expectations that are not realistic. It has impacted my mental health. It has caused undue stress. My livelihood is being determined by AI. It is stressful beyond belief.
And I feel this rises to a level of abuse and manipulation on the company’s part for wage avoidance. They are not removing or properly factoring invalid interactions. It all counts against us, which impacts my pay and could result in disciplinary action, including termination. My wages and employment are directly tied to these AI metrics.
I’d also like to address informed consent—something my employer did not provide.
We found out AI was being used this way only after it was already being used. We were never asked about it, never consented to it, never had disclosed how our data is being used, and even now it is still unclear what is being tracked, how it is being used, or what is being excluded.
And what about consumers?
Sure, they hear “your call may be recorded for training and quality purposes,” but they have not been told AI is analyzing every call in this way, and I have a feeling many consumers would not be comfortable with that.
I can literally ask our AI tool detailed questions about what a customer said or did, and they have no idea their data is being collected, interpreted, and queried this way.
That should concern everyone.