from Hacker News

Open AI in Trouble

by a_tartaruga on 3/1/25, 5:38 PM with 106 comments

• by Kapura on 3/1/25, 6:11 PM

  It absolutely blows my mind that people swallowed "AGI is just around the corner" hook, line and sinker. We don't understand human cognition despite literally thousands of years of thought and study, but some nerds with a thousand GPUs are going to make a machine think? Ridiculous. People were misled by graphs that they were told pointed to a singularity right around the corner, despite obvious errors in the extant systems.

  Concentrated capital is truly a wild thing.

• by Chance-Device on 3/1/25, 6:17 PM

  I don’t think I’d be this pessimistic. There’s still o3 that hasn’t been seen in its full form, and whatever improvements are made to it now that multi head latent attention is out in the wild.

  Orion does seem to have been a failure, but I also find it a bit weird that they seemingly decided to release the full model rather than a distillation, which is the pattern we now usually see with foundation models.

  So, did they simply decide that it wasn’t worth the effort and dedicate the compute to other, better things? Were they pushed by sama to release it anyway, to look like they were still making progress while developing something really next gen?

• by RDaneel0livaw on 3/1/25, 6:03 PM

  IF this is the beginning of the bubble bursting, I welcome it with open arms. Burn baby burn.

• by redwood on 3/1/25, 6:04 PM

  These are powerful building blocks, they're just not the be all end all. The building blocks up stack that use these as part of a broader architecture, that shape these and many other traditional techniques in software... this is where the real value will be unlocked.

  This layer itself will inevitably see its cost come down per unit of use on a long road towards commoditization. It will probably get better and better and more sophisticated but again the value will be primarily up stack, not accrued primarily from a company like this. It's not to say they couldn't be a great company... even Google is a great company that has enabled countless other companies to bloom. The myopic way people look to these one size fit all companies is just so disconnected from our economy works.

• by cs702 on 3/1/25, 6:32 PM

  Gary Marcus. By all accounts, he doesn't understand how LLMs work, so usually he's wrong about technical matters.[a]

  But here, I think he's right about business matters. The massive investment in computing capacity we've seen in recent years, by Open AI and others, can generate positive returns only if the technology continues to improve rapidly so it can overcome its limitations and failure modes in the short run.

  If the rate of improvement has slowed down, even temporarily, OpenAI and others like Anthropic are likely to face financial difficulties.

  ---

  [a] In the words of Geoff Hinton: https://www.youtube.com/watch?v=d7ltNiRrDHQ

  ---

  Note: At the moment, the OP is flagged. To the mods: It shouldn't be, because it conforms to the HN guidelines.

• by qntmfred on 3/1/25, 5:58 PM

  gary marcus = automatic thanks but no thanks

• by tananaev on 3/1/25, 6:02 PM

  I wonder if we reached the limit of what a single-shot large models can do? Or we're just waiting for another breakthrough? If it's the limit, then probably the chain of thought is the answer. I think models are so good now that given the right context, tools and time, they can pretty much do anything.

• by Willingham on 3/1/25, 6:04 PM

  The pressure to hit quarterly targets seems to, in this case, caused quite the opposite outcome intended. Gotta love corporate America and the overvalued AI sector. (:

• by fancyfredbot on 3/1/25, 6:09 PM

  Open AI has a brand, it has talent, it has some really solid models, and it has the compute to serve inference at scale. They stand as good a chance of covering their inference costs as anyone else.

  The compute for training is beginning to seem a poor investment since it is depreciating fast and isn't producing value in this case. That's a seriously big investment to make if it's not productive but since a lot of it actually belongs to Azure they could cut back here fast if they had to. I hope they won't because in the hands of good researchers there is still a real possibility that they'll use the compute to find some kind of technical innovation to give them a bigger edge.

• by paulpauper on 3/1/25, 6:21 PM

  I used to be a skeptic. Chat GPT is awesome. Similar to Google, it's become a staple of my everyday life, like writing, or composing latex, or even doing math. I cannot imagine not having it without losing productivity. Is the business model sustainable? Probably not; I already see some signs of throttling in an attempt to cut costs, like if you overuse computation for math. Just the ability to render the latex from wolfram alpha is great. Writing latex used to be such a chore.

• by armchairhacker on 3/1/25, 6:28 PM

  Humans learn. LLM context windows are vastly larger than our short-term memory, but vastly smaller than our long-term recollection. LLMs can recall vastly more information than our long-term memory, but only from their static training data.

  Also, even though LLMs can generate text much faster than humans, we may be internally thinking much faster. Each adult human brain has over 100 billion neurons and 100 trillion synapses, and each has been working every moment, for decades.

  This is what separates human reasoning from LLM reasoning, and it can’t be solved by scaling the latter to anything feasible.

  I wish AI companies would take a decent chunk of their billions, and split it into 1000+ million-dollar projects that each try a different idea to overcome these issues, and others (like emotion and alignment). Many of these projects would certainly fail, but some may produce breakthroughs. Meanwhile, spending the entire billion on scaling compute has failed and will continue to fail, because everyone else does that, so the resulting model has no practical advantages and makes less money than it cost to train before it becomes obsoleted by other people’s breakthroughs.

• by returnInfinity on 3/1/25, 9:59 PM

  Gary Marcus and Ed Zitron are very bearish on LLMs.

  Disclosure - I am neither bearish or a mega bull on LLMs. LLMs useful in some cases.

• by startupsfail on 3/1/25, 6:09 PM

  While OpenAI is certainly in trouble (zero Hinton students, partnership with Azure is straining, while Brin is back at Google!), its demise is still uncertain. They are still absolutely amazing lab, pushing both product and research. Execute. And make all these naysayers obsolete ;)

• by adamgordonbell on 3/1/25, 6:19 PM

  Gary Marcus has been saying for 3 years ( or more ) that LLMs are at the limit and will never get better and also are useless.

  A smart enough AI would summarize each of his posts as "I still hate the current AI boom".

  There must be a term for such writers? He's certainly consistently on message.

• by eliothmonroy on 3/1/25, 6:44 PM

  Why this was flagged?

• by 1vuio0pswjnm7 on 3/2/25, 3:14 AM

  "Gary Marcus has been warning the field for years that pure statistical approaches like LLMs probably wouldn't suffice for AGI. Half a trillion dollars later, it looks like maybe he had a point."

  D'oh!

• by stuartjohnson12 on 3/1/25, 5:58 PM

  Has there been a single transformative technology that was any good 2.5 years after mainstream public buy-in?

• by 1970-01-01 on 3/1/25, 6:36 PM

  This is going to sound disrespectful, but nobody cares. Both bloggers and CEOs will continue to argue they have the bestest AI. Our goal should simply be making sure AI is helping more than it harms. Much like nuclear research 75 years ago, we've setup this century where AI will be simultaneously massively weaponized and massively assistive technology. There's also an ultimate goal of having AGI solve all our problems like fusion was supposed to do. Let me know when your AI CEO is ready to discuss how all this scaling is helping the kids sleep better instead of making quarterly profits rise faster than the competition.

• by agnishom on 3/1/25, 7:29 PM

  It is not "in trouble". It just has more competition. That's what _should_ happen when capitalism functions correctly.

• by computerex on 3/1/25, 6:22 PM

  Gary Marcus is an insufferable cretin, who likes nothing more than to hear himself talk.