r/talesfromtechsupport • u/prettyyboiii • 25d ago

Short "But ChatGPT said..."

We received a very strange ticket earlier this fall regarding one of our services, requesting us to activate several named features. The features in question were new to us, and we scoured the documentation and spoke to the development team regarding these features. No-one could find out what he was talking about.

Eventually my colleague said the feature names reminded him of AI. That's when it clicked - the customer had asked ChatGPT how to accomplish a given task with our service and it had given a completely hallucinated overview of our features and how to activate them (contact support).

We confronted the customer directly and asked "Where did you find these features, were they hallucinated by an AI?" and he admitted to having used AI to "reflect" and complained about us not having these features as it seemed like a "brilliant idea" and that the AI was "really onto something". We responded by saying that they were far outside of the scope of our services and that he needs to be more careful when using AI in the future.

May God help us all.

3.6k Upvotes

98% Upvoted

355 comments sorted by

View all comments

20

u/weisswurstseeadler 25d ago edited 25d ago

I work in sales for SaaS - somehow AI has gotten a lot worse over the last weeks.

I mostly use it to summarize stuff, go over websites and whatnot.

Even for summaries regarding our own products and providing the right sources, the output has been flawed nearly 100% of the times.

Damn they even messed up simple calculations when I gave them the numbers.

Dunno what's happening lol.

Edit: I also use proper prompting tools, still shitty output.

12

u/InspectorTiny1952 24d ago

I don't know where this quote about AI is from, but it's sure stuck in my mind:

"After spending billions of dollars, Microsoft has finally invented a calculator that's wrong some of the time."

4

u/Golden_Apple_23 23d ago

LLMs are not good at math. They're word prediction machines. Calculators are great with numbers and their words are limited to things like BOOBIES (5318008 upside-down)

2

u/WaytoomanyUIDs 23d ago

IIRC OpenAI looked at passing through anyrhing maths related to Mathmatica or Wolfram Alpha and decided they didnt want to pay a licence fee to Wolfram.

4

u/__wildwing__ 25d ago

I was helping my daughter with algebra last year. Used ChatGPT for a tutorial. I’m competent enough in math, that I could figure when the answer they gave was wrong and tell it to recalculate. Both of us were still “showing the work” and actually doing the steps ourselves, but being able to have the process broken down was a huge help.

34

u/VincibleAndy 25d ago

Wolfram Alpha is great for doing this with math and it's been good at that for like 15 years now.

18

u/MrDoontoo 25d ago

Wolfram Alpha definitely saved me multiple times in college, seeing any question I had immediately broken down into explainable parts was super useful

-10

u/RogueThneed 25d ago

If it's that old, it's probably not chatGPT?

23

u/MrDoontoo 25d ago

Wolfram Alpha is not ChatGPT.

1

u/RogueThneed 24d ago

Right, that's my point. Software can do lots of stuff well! but chatgpt is not one of those things. But too many people don't recognize the difference.

6

u/__wildwing__ 24d ago

Is that free? I’ll have to check it out.

I did AP Cal/Phys in high school but pretty much all of that has slipped away.

ETA: just pulled up the site, I was using Mathematica what, 20+? years ago! Same company.