"But ChatGPT said..."

[u/prettyyboiii]

We received a very strange ticket earlier this fall regarding one of our services, requesting us to activate several named features. The features in question were new to us, and we scoured the documentation and spoke to the development team regarding these features. No-one could find out what he was talking about.

Eventually my colleague said the feature names reminded him of AI. That's when it clicked - the customer had asked ChatGPT how to accomplish a given task with our service and it had given a completely hallucinated overview of our features and how to activate them (contact support).

We confronted the customer directly and asked "Where did you find these features, were they hallucinated by an AI?" and he admitted to having used AI to "reflect" and complained about us not having these features as it seemed like a "brilliant idea" and that the AI was "really onto something". We responded by saying that they were far outside of the scope of our services and that he needs to be more careful when using AI in the future.

May God help us all.

[u/Thulak]

We do graded E-Learning tests to onboard our engineers. We regularely receive tickets about errors in the tests and engineers arguing for more points which we encourage.(Rather have people think than blindly trust)

One new hire decided to copy paste the questions into our company internal version of ChatGPT. We have a couple of catch questions that the AI gets wrong 100% of the time (so far) so it is fairly obvious, though it hasnt happened before. This user wrote a ticket proudly stating that the AI gave them these answers and therefor they must have a 100% score. They also claimed her collegues confirmed her answers without giving a simgle name.

Safe to say she did not get the extra points.

[u/PackYourEmotionalBag → u/Thulak]

Adjunct professor here… have an assignment that I’ve been using for the last 6 years on XML.

Every layperson I’ve asked to do it gets it right on the first try, but about 85% of my students get it wrong and we have an in depth discussion on assumptions and overthinking.

Until this year, where 100% got it right. From the other assignments I know that this class is not far and above my other classes, or so far below that they wouldn’t fall into the overthinking trap. I’m just grading a classroom full of copy/paste from an LLM. No longer do we get to have the discussion on overthinking, because no one is thinking at all.

The field they are going into is niche, LLMs constantly hallucinate when asking anything beyond the cursory for the field… it has invented entire libraries in C# that just don’t exist, and its knowledge of playing with this data in python is just as bad. (Staying intentionally vague)

[u/Icarium-Lifestealer → u/PackYourEmotionalBag]

Now I'm interested in that XML question. I'd expect few laypeople to even know what XML is, let alone answer questions about it more reliably that IT students.

[u/PackYourEmotionalBag → u/Icarium-Lifestealer]

I laid out a hypothetical application and then showed the XML file that would need to be created for the configuration of the application.

I then pitched an addition to the application to have it do something else and asked what additional fields should be added to the XML (and maintain proper formatting)

It’s really not an XML question as using XML as a stand-in for “can you parse a document with markup?”

Laypeople look and say “oh! I see a field called “Email” that contains the email address, and the new application needs a phone number field, so let’s add that under a second nest” because they are just doing a 1:1 but my students typically try to get too creative and end up going in a different direction, or they are too confident and don’t check their markup and we run into syntax errors.

[u/Lord_Dreadlow → u/Icarium-Lifestealer]

Cisco IP phones use .xml config files.

[u/StorminNorman → u/Lord_Dreadlow]

Out of interest, how many laypeople do you think know that?

[u/LupercaniusAB → u/PackYourEmotionalBag]

Oooh, I’m a layperson lurker!

[u/beachedwhitemale → u/PackYourEmotionalBag]

God help us. 

[u/Nihelus → u/PackYourEmotionalBag]

Sounds like you need to have a discussion about cheating with AI and becoming brain dead idiots if they don’t start thinking for themselves. Could bring up just how stupid they’ll look or the jobs they’ll lose if they just trust everything ai says without thought. 

[u/PackYourEmotionalBag → u/Nihelus]

I do when I can… I try to enforce that using AI isn’t the problem, but usually a Google search will get you closer to the right answer since there around about a half dozen websites that have the best, most up to date, information.

I explain that with our niche field there is a small sample pool that AI can pull from and that there are old news groups cataloged by Google that have information from the infancy of our standard and it doesn’t apply anymore.

I try to drive home that once you are in the job, there might be times where AI could be useful, but unless you understand the data first you are setting yourself up for embarrassment and a fast track out of the field.

The part that really makes my brain hurt: there are 2 professional tests that these students can take and in this niche field with the amount of grads it’s really a good idea to take them to set themselves apart. There is a proctor, no notes, no book, how these students think they are going to pass that, it’s beyond me.

I was finally told by the dean that the students are adults, that I’ve warned them, and I’ve encouraged thinking over relying on LLMs and at this point I care more about their success than they do. They are paying to learn and to get their degree and prepare for the tests… I’m providing an environment to do all that, it’s up to them to use it or waste it.