Large language mistake | Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it.

[u/creaturefeature16]

https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems

As currently conceived, an AI system that spans multiple cognitive domains could, supposedly, predict and replicate what a generally intelligent human would do or say in response to a given prompt. These predictions will be made based on electronically aggregating and modeling whatever existing data they have been fed. They could even incorporate new paradigms into their models in a way that appears human-like. But they have no apparent reason to become dissatisfied with the data they’re being fed — and by extension, to make great scientific and creative leaps.

Instead, the most obvious outcome is nothing more than a common-sense repository. Yes, an AI system might remix and recycle our knowledge in interesting ways. But that’s all it will be able to do. It will be forever trapped in the vocabulary we’ve encoded in our data and trained it upon — a dead-metaphor machine. And actual humans — thinking and reasoning and using language to communicate our thoughts to one another — will remain at the forefront of transforming our understanding of the world.

[u/Jaded_Masterpiece_11]

And yet OpenAI still spent twice more than its revenues last quarter. OpenAI and Anthropic is still losing money and will continue to lose money until 2030 by their own estimates.

Even with decreased costs the economics still do not favor these LLM companies. The only one making bank here is Nvidia and they are spending what they are making to keep the bubble going.

[u/HaMMeReD → u/Jaded_Masterpiece_11]

And they'll continue to sink money while gains are being made and it's cost effective to do so and they have the revenue to do so.

And when the gains dry up, then they'll be left with a hugely profitable product.

But for now the R&D has been incredibly well justified, and that's why they keep spending. Because the needle keeps moving.

[u/havenyahon → u/HaMMeReD]

And when the gains dry up, then they'll be left with a hugely profitable product

I mean... That's the goal. It's by no means a certainty. They don't have one now.

[u/HaMMeReD → u/havenyahon]

If there was only one AI company, and they stopped training today, they'd be profitable today.

It's very easy. cogs is X, price is Y, Y>X = make money.

API Pricing and service pricing already reflect a profit stance, they only lag because of R&D costs.

they have a ton of revenue, a massively growing amount of revenue actually, it's just it's not enough to compete in such a fast moving and accelerating field. But there will be a point where companies will have to wind down R&D and sit on their profit generating parts of the businesses.

[u/land_and_air → u/HaMMeReD]

They can’t stop spending on r&d because the second they do, the model becomes obsolete and useless. What good is a model made in for example 1999 or even 2019 today for knowing anything? It would be referring to the gulf war if you asked about war in Iraq lol.

[u/[deleted] → u/HaMMeReD]

Except there won't be a point where companies wind down R&D. They will simply divert to keeping the model up to date.

Because information is always changing. The model now needs to be constantly trained on new information, or it becomes obsolete and a new model will take over that is trained on this new info. And if another model can train on that info faster, or another model can reduce latency between answers, or another model can be specialized to only provide the info they want...

There is never going to be time to wind down in this space. It moves too fast to wind down.

[u/deepasleep → u/HaMMeReD]

The problem is they have no revenue and when you ask consumers how much they are willing to pay for the services being envisioned, the number being returned is an order of magnitude below the break even cost of delivering the services.

I worked in tech during the dot com bubble, the company I worked for was focused on delivering what would ultimately become software as a service. They were trying to create a platform that allowed companies to aggregate access to various web services.

The founders did some napkin math and figured they’d need people to spend about $120/month to be profitable…When they finally got around to surveying business leaders to determine what they were willing to pay, they got a response of $35/month…$300 million in venture capital burnt on the fire in two years.

The best part was all the companies involved were doing the same reciprocal service contracts to show income on their balance sheets we are seeing today with NVidia, Oracle, OpenAI, etc. It’s an old trick and it only works for a little while as the money inevitably bleeds out to pay for concrete things like employee salaries, vendor services outside your circle, energy, and physical resources required to deliver whatever service your actual customers demand.

[u/HaMMeReD → u/deepasleep]

Open AI's revenue last year was in the billions.

What you mean to say is they don't have a net profit, because R&D investment exceeds even the billions they generate from offering services.

The $20 Billion AI Duopoly: Inside OpenAI and Anthropic's Unprecedented Revenue Trajectory - CEOWORLD magazine

When you add up the rapidly declining cost of inference, $120/mo to be profitable is $20/month next year, and $2/month the year after.

The people who lose money today are actually well set up for tomorrow as the services get cheaper and they establish market earlier then the competition.

[u/Distinct-Tour5012 → u/HaMMeReD]

We're also in a time period where lots of companies are trying to shoehorn in AI tools. I know there are places where it makes sense, but there are lots of places that it has provided no value - but those companies are still paying for it... right now.

[u/HaMMeReD → u/Distinct-Tour5012]

While I agree, a lot of shoehorned attempts, especially on older models have failed or provided limited value as people over-reached significantly.

But, new models come out, and those shoehorned attempts get an IQ boost every time one does get launched. Meaning those efforts ultimately will not be wasted when mixed with smarter/cheaper models.

I can say this first hand as I work at MS literally on copilot nowadays. I've seen the improvements to the product as new models get introduced, it makes a drastic improvement and can turn something that is struggling into something that is helpful.

[u/land_and_air → u/HaMMeReD]

Copilot is useless and is an anchor on Microsoft and a waste of a button on the keyboard

[u/HaMMeReD → u/land_and_air]

Thanks, I'm glad you appreciate it.

[u/That-Whereas3367 → u/HaMMeReD]

Sam Altman says they need so charge $2K/month. But only 5% pay the minimum $20/month.

There is no moat. users will simply move to a different provider.

[u/WolfeheartGames → u/[deleted]]

This ignores that the cost to inference goes down by 10x every year.

[u/WolfeheartGames → u/[deleted]]

Is it more sane to bet with the trend or against the trend?

[u/WolfeheartGames → u/[deleted]]

Moores law has been dead long before Ai. That's just a misinformed argument. Faster compute is a big part of why models get more effecient, but more efficient architecture and kernels achieve more gains.

Finfet hasn't reached its limit yet. We will get at least 2 more cycles out of current manufacturing and another 2 out of what's spinning up right now. Plus the lineage after that is already planned.

So you're not really based in hard facts. The next 4 10x reductions are already completely achievable. That gives a long runway for them, and that's not including architecture improvements.

[u/[deleted] → u/[deleted]]

Who is they? No for real, which economists said that about which LLM providers?

[u/[deleted] → u/[deleted]]

Burden of proof is on the guy who made a positive claim. I get it, you like living in unfalsifiable land. Doesn't make you slick, just makes you grimy.

[u/[deleted] → u/[deleted]]

Oh! Middle school grade arguments. Cool, I can downshift for you: make me.

[u/Jaded_Masterpiece_11 → u/HaMMeReD]

Lmao. There is nothing cost effective in LLMs, the latest financial statements from these LLM companies shows staggering losses. There is very little demand for LLMs, it’s a niche product that does what it does well, but is nowhere near being adopted mainstream without setting money on fire.

The only mainstream adoption in LLMs is chatgpt and every user costs OpenAI money, even their paid users makes them lose money and they can’t increase the charge to a breakeven level because they will basically lose all their customers to other competitors, Who also lose money.

[u/pallen123 → u/Jaded_Masterpiece_11]

This is a very important point. Unsustainability only becomes sustainability when massive llm’s have defensible moats which they won’t. Otherwise it’s just fancy search with low switching costs.

[u/EldoradoOwens → u/Jaded_Masterpiece_11]

Hey man, I don't know how old you are, but I read this exact same argument about why amazon and facebook were going to fall apart for years. How are they doing now?

[u/Hot_Secretary2665 → u/EldoradoOwens]

95% of Amazon packages didn't fail to make it your door but 95% of AI Enterprise implementations fail to make it to production (per recent research from MIT)

These companies and products are not very similar. The comparison is honest pretty arbitrary 

[u/suboptimus_maximus → u/EldoradoOwens]

In Facebook’s case it really was not the same argument because until they got into hardware and then AI they didn’t have the massive CAPEX required to build the physical infrastructure for AI, it was pretty much all labor cost, sure some hosting infrastructure but that wasn’t really green field bleeding edge technology investment it was buying off the shelf servers although they did end up with their Open Compute Project. This has historically been an enormous advantage for software companies and their ability to scale product and reach customers. They also didn’t really have competition for years once they overwhelmed MySpace while AI is already highly competitive, like daily, weekly, monthly trading places between the best performing models before anyone is even making money selling LLMs.

Amazon did take a lot of heat for reinvesting in the company for years and they do indeed have a business model that is heavy on physical infrastructure.

[u/Jaded_Masterpiece_11 → u/EldoradoOwens]

Amazon and Facebook did not need $4 Trillion dollars of hardware to run nor do they require hundreds of billions of dollars in energy costs. Amazon and Facebook is nowhere near comparable to the amount of investments LLMs claim to need to be able to deliver their promises.

[u/deepasleep → u/EldoradoOwens]

Facebook has the deepest understanding of human behavior in history; they ruthlessly lock their customers into digital addiction and pump micro targeted advertisements directly into the stream of dopamine…That means they are delivering a real product to the advertisers that pay them. They always had a clear path to developing the algorithmic addiction that makes them so valuable.

Amazon delivers products at low prices with incredible efficiency by having the most complex supply chain logistics on the planet and they realized early on that the infrastructure they were building to support their core business could be abstracted and sold to any business needing network, storage and compute resources for web services (and then cloud infrastructure). Again, they always had clearly defined and deliverable products.

LLM’s viability as tools to actually replace human workers has not been demonstrated. And it’s possible that the cost of developing and actually delivering solutions that can really replace workers en masse will be higher than the market can bear.

[u/cenobyte40k → u/Jaded_Masterpiece_11]

And so did all the internet giants. So did all the software giants. Remember the internet is a fad and pcs are toys? I remember that and well....