De Kraker: "If OpenAI is right on the verge of AGI, why do prominent people keep leaving?"
See full article...
See full article...
Major shifts at OpenAI spark skepticism about impending AGI timelines
- Thread starter JournalBot
- Start date
Having followed AI off-and-on since the Prolog/Society of Mind days, I've never understood how a scaled up LLM is supposed to make the leap to AGI. Now, I'm not an AI researcher or scientist, but perhaps the answer is "it's not".
Upvote
329
(338
/
-9)
Meanwhile the rest of us over here in reality have long since known the truth: OpenAI and everyone else are blowing steam when they say they're "Close to AGI".
Upvote
191
(193
/
-2)
Same. I don't even understand what the underpinnings of AGI is supposed to look like.
I work with and get LLMs to some extent, but more LLM is still just more math, vectors ... output. More LLM is just more LLM, not necessarily magically different or any fewer fundamental shortcomings.
Upvote
173
(181
/
-8)
The point in the article is that these people have a large stake in OpenAI and would benefit massively if OpenAI cracks AGI.
Upvote
153
(155
/
-2)
I kinda wonder if Altman is just running with the guardrails off after coming up roses in the wake of the board debacle earlier this year.
A big part of what they claimed to be unhappy about was that he’s conniving, dishonest, and sows division. What are the odds that without some moderating influences he’s just making OpenAI a shittier place to work and the people most able to walk are moving on before it gets properly bad?
A big part of what they claimed to be unhappy about was that he’s conniving, dishonest, and sows division. What are the odds that without some moderating influences he’s just making OpenAI a shittier place to work and the people most able to walk are moving on before it gets properly bad?
Upvote
135
(136
/
-1)
Here's a really interesting discussion of exactly that subject:
Will scaling work?
Data bottlenecks, generalization benchmarks, primate evolution, intelligence as compression, world modelers, and other considerations
www.dwarkeshpatel.com
The short answer appears to be "maybe, but probably not, and possibly never due to data, compute, and other constraints".
Upvote
117
(117
/
0)
LLMs are nowhere near AGI. They are nowhere near even approaching bacteria in terms of being able to interact with their environment and adapt beyond their training.
They poop out words in an order that is not always nonsensical.
They poop out words in an order that is not always nonsensical.
Upvote
116
(136
/
-20)
a lot of computer science and cognitive science academics also say "it's not".
the core algorithm in current ML is some kind of 'stochastic gradient descent'. That basically means, guessing at neural net connectivity values until it gives pretty good answers for questions in the training set.
Unlike a biological mind, it never takes the training data as an input, and, at best, can do a good job producing plausible results for prompts that are within, or lie close to (in a high dimensional vector space) the training data. the network does not have an inherent ability to generalize beyond that. Or model the world and reason about the model internally.
current approaches simply cannot ever be 'generally intelligent', because there isn't enough training data, or enough atoms in the universe, to make a computer that could work like that.
edit: and most of the people who say otherwise are drawing a salary, or collecting VC investment, that requires keeping this hype bubble growing.
Upvote
90
(102
/
-12)
The real "safety" concerns are mostly about totally fucking up our society with fake generated content on social media and elsewhere. Which is quite bad enough for people to have some real concern over the fucking sociopathic techbros making it. Remember "Radio Rwanda"? A fucking genocide was basically incited by a fucking radio station.
Now substitute that for some lone sociopath using LLMs to spam the already unhinged "social" media with enough fake news, and you have what?
...checks news...
Some of the worst violent far‑right attacks in recent UK history, for example (which didn't even take a LLM to create, but you get the gist, hopefully).
Upvote
86
(90
/
-4)
The cool thing about AGI is that it’s a religion, since there’s no clear consensus on what intelligence is.
A pump and dump marketing term at best.
A pump and dump marketing term at best.
Upvote
81
(83
/
-2)
this makes a ton of assumptions that are just that assumptions
A) that a parameter in a NN is equivalent to a synaptic connection
B) That synaptic connections are the most efficient way to do "intelligence" so that it could be used as a benchmark to go
C) That learning as done for the brain is what is most efficient for how it should be done with computers.
Upvote
65
(65
/
0)
ChatGPT-3.5 = 175 billion parameters, according to public information
Different studies have slightly varying numbers for a human brain, but it's 1000x more: from 0.1 to 0.15 quadrillion synaptic connections. Source: https://www.scientificamerican.com/article/100-trillion-connections/ (among others)
While it's likely to require something more than just scaling up the model size, I thought this gives some clue about scale. I agree with you, perhaps the answer is "it's not" scaling.
Upvote
64
(75
/
-11)
Last time I used GPT4, I asked it to list some highly-rated Japanese novels that hadn't been translated into English yet. It proceeded to list four novels that had already been translated, apologised and stated that it would correct itself, then listed another four novels that had already been translated (without acknowledgment or correction).
And people are skeptical about AGI because some employees left?
And people are skeptical about AGI because some employees left?
Upvote
62
(63
/
-1)
I find irony in all these announcements that “OpenAI will definitely have magic AI results any minute now, y’all” being posted on Elon Musk’s website.
Upvote
53
(55
/
-2)
Scaling laws don't matter if you are running out of training data, as some researchers suggest (like this recent paper on arxiv https://arxiv.org/html/2211.04325v2).
Upvote
52
(53
/
-1)
If OpenAI is no where near AGI, then it seems all the "safety" concerns are a bit unfounded. In 5 years we are going to learn that LLMs are probably not that way to achieve AGI. Purely anecdotal but from daily use of Claude and ChatGPT, i don't find Claude to be anymore safe and secure in out output than ChatGPT
Upvote
50
(59
/
-9)
I doubt they're leaving because of Copyright infringement, if that was the case, they wouldn't be moving to Anthropic.
Anthropic fires back at music publishers' AI copyright lawsuit | Reuters
RIAA Backs AI Copyright Lawsuit Against Anthropic, Sees Similarities with Napster * TorrentFreak
Upvote
47
(47
/
0)
Most AI researchers thought AGI was just some tweaks away from Random Forest when they were the cutting edge or similarly for Bayesian Networks when they were at their peak and Clippy was popping up on screens.
Upvote
47
(48
/
-1)
Upvote
46
(46
/
0)
(Emphasis mine)Schulman’s parting remarks quoted in the last paragraph of the article said:
This seems to assume an enduring contribution and relevance that I’m not confident OpenAI and its competitors will achieve…you bros may just be churning VC bucks, at the end of the day.
Upvote
44
(44
/
0)
Prominent researchers like Yann LeCun agree with you. It'll take more than just a scaled up LLM.
Upvote
35
(35
/
0)
But blockchain, cryptocoins, NFT have totally revolutionized databases, financial transactions, ownership of digital apes. Totally.
(This time will be different, trust us!)
/s
Upvote
35
(37
/
-2)
Oh no, cultists are leaving a trash non-profit dedicated to aggravating climate change and destroying art to join another trash company dedicated to the same thing
Upvote
32
(35
/
-3)
It is also comparing to a human brain, the most (one of?) complex brains on the planet. Biology can do AGI with far fewer synapses, as evidenced by all the less complex brains that exist.
So we are already at the point where LLMs are more complex than working biological brains when compared this way. By this alone we can see that there is something major LLMs are missing besides scale for AGI.
Upvote
31
(31
/
0)
Because Sama and some others have lied to investors that they could achieve human-like intelligence within 5 years by mostly scaling up feed-forward neural-networks and the data used to train them. At the very least he promised them that hallucinations and basic reliable reasoning would be "done" pretty soon.
In reality these things will take 10+ years and will require new architectures like IJEPA or others. Probably many more fundamental advances are required.
Any investors that can wait for 10+ years will be OK, as long as the AI company they invested in survives and moves on to the next thing successfully. Many will take a soaking when the LLM/VLM bubble pops (gonna happen within 5 years, maybe a lot sooner).
EDIT: And some people don't want to be there when it starts raining.
In reality these things will take 10+ years and will require new architectures like IJEPA or others. Probably many more fundamental advances are required.
Any investors that can wait for 10+ years will be OK, as long as the AI company they invested in survives and moves on to the next thing successfully. Many will take a soaking when the LLM/VLM bubble pops (gonna happen within 5 years, maybe a lot sooner).
EDIT: And some people don't want to be there when it starts raining.
Last edited:
Upvote
30
(30
/
0)
That is the thing that bothers me the most. Sure these are "multimodel" but we are still fundamentally iterating on LLM's here. At best we can do some tricks here or there where it can do other things, but that doesnt make it an AGI.
Throw in that we know very well that we are already hitting the limitations of LLM with very false (and sometimes dangerous) misinformation. Sometimes very publicly: See Google Search
But there are a certain subset of people that seem convinced that an LLM is already a general purpose AI and can do nearly anything you can describe to it in text. Which is insane and dangerous
I am convinced that an AGI breakthrough will be its own thing(if it happens), it won't be iterating on an LLM. At best with an LLM we are going to get something that appears it knows what it is doing (which it already does) but isnt doing any real reasoning. There are multiple papers on this. This one is one of my favorites: https://arxiv.org/abs/2406.02061
Upvote
28
(30
/
-2)
Infallibility is an absolute, so it isn't very useful way to measure anything.
That said, I have 52,000KM on the odometer of my car and not once have I blocked an ambulance because I'm too stupid to pull over.
Upvote
27
(28
/
-1)
Congrats! You are now more qualified to be an "AI researcher" than most of the VC tech bros!
LLMs were originally designed specifically to mimic intelligence, but ultimately based purely on statistical likelyhood of next word using the query as a "scaffold". Add in huge datasets and "wow" it seems smart (except for having zero uniqueness and failing in basic math for a computer). The truth is that Altman found LLMs, was fooled, and tried to be the guy that "made AI". Flash a bleached smile at investors and voila! The media defends him and nods in agreement as he says he "invented" stuff done in '69. Gets to directly influence AI policy despite zero qualification, and actively led a coup against actual data scientists that didn't like his unethical profiteering.
The reality is that we are nowhere near a "general" intelligence. Such a model is going to be a massive committee setup with dedicated models designed for specific questions and governing models designed to guide dataspace expansion and align those model to an LLM that is no more than a GUI. Overall, it isn't that hard to develop. The problem is money, infrastructure, time, and overcoming the bleach-toothed children pretending their two hours with Tensorflow means anything (and those with the money ignoring their flashy sales pitch. The same crowd that fell for crypto too).
I'm sorta shocked these so-called "AI" companies are chasing after people that literally have zero formal education in the matter beyond "hey! Look at what this toolbox can do!" and utterly ignoring the people that invented the underlying methods screaming at everyone they are misusing them. There are still a lot of the OG data scientists and chemometricians around that built this stuff in FORTRAN on IBM machines. Although, I did see FORTRAN got some serious updates recently... Seems like the jump from 77 to 95 all over again.
Upvote
25
(31
/
-6)
natural selection and genetics are completely different processes than learning. natural selection took billions of years to make intelligence, can openAI wait that long?
you dropped 'stochastic'. the network is initiated with random values. at no point is the training data introduced into the model, the model is simply curve fit to the training data. The model doesn't learn, in any way that word has ever been defined. learning is an active process that involves taking in information through the senses.
The models don't have senses and are incapable of learning. Training is something done to, or with, the model, which is an entirely latent set of data structures with no intent or agency.
Upvote
24
(31
/
-7)
When I were a lad (says the old man, waving his walking-stick in the air) I worked on computer simulations of various physics. One I was quite proud of was a fluid simulation (2d at the time). In those days it was just a skinned surface made up of a bazillion meta-balls with parameters for 'size' to give volume, parameters for stickiness, and particle movement properties. It made for quite a convincing fluid simulation. (I discovered that merely changing the size radius of one set of particles makes two dissimilar fluids separate, like oil and water. Well I thought that was cool anyway).
Forty years later and water simulations are actually pretty good. In movies it's almost impossible to tell where the real water ends and the CG water starts. But every now and then when something big is rising out of the ocean you get a jarring visual break of the water looking like a cascade of small ping-pong balls as the simulation isn't perfect.
With LLM AI it's very possible we're looking at a simulation of intelligence that will only ever look like intelligence without actually being intelligence. And it's possible that LLM is the wrong vehicle for general intelligence. It can only ever be a flawed mimic because the underlying principle is wrong. It will still be useful for certain things But it's also possible that LLMs are an algorithmic cul-de-sac that can never be as good as Actual Intelligence.
Forty years later and water simulations are actually pretty good. In movies it's almost impossible to tell where the real water ends and the CG water starts. But every now and then when something big is rising out of the ocean you get a jarring visual break of the water looking like a cascade of small ping-pong balls as the simulation isn't perfect.
With LLM AI it's very possible we're looking at a simulation of intelligence that will only ever look like intelligence without actually being intelligence. And it's possible that LLM is the wrong vehicle for general intelligence. It can only ever be a flawed mimic because the underlying principle is wrong. It will still be useful for certain things But it's also possible that LLMs are an algorithmic cul-de-sac that can never be as good as Actual Intelligence.
Upvote
24
(24
/
0)
That's kind of the thing, right? If the people investing in AGI were interested in simple 'intelligence' systems, we've already got robotic systems that are fully capable of replacing people for nearly all mundane and mechanical tasks. But they're 'single-use' systems. They're of no use outside of the specific task for which they're created, and each one constitutes a considerable investment to design, build, test, and deploy. If all they wanted were 'worker ant' AI systems, they've already got that...along with all the limitations and inflexibility that comes with it. AGI is the promise of a single 'tool' for all tasks. A 'multi-tool' if you will. Which necessitates a 'tool' that's at least as complex as what it'd be replacing -- people. And these AI prototype 'tools' are no where near that level of complexity or capability.
And I daresay, they won't like an AGI system when/if they do get built. Because it'll be just as complex as people...with all the problems that entails. The same problems that are pushing these wealthy investors to fund the development of AGI in the first place.
Last edited:
Upvote
23
(23
/
0)
I think AGI-like technology will be stuck in the "uncanny valley" for quite a while. This is where it will be close to human like intelligence but never quite close enough.
Which is not to say it won't be useful, just not human-like.
You see a similar sort of asymptotic relationship with fully autonomous road vehicles. It looked like it might be a solved problem back in 2018, and in a lot of ways the best systems seem to be 98% there. But that last 2% makes all the difference in the real world.
Which is not to say it won't be useful, just not human-like.
You see a similar sort of asymptotic relationship with fully autonomous road vehicles. It looked like it might be a solved problem back in 2018, and in a lot of ways the best systems seem to be 98% there. But that last 2% makes all the difference in the real world.
Upvote
22
(22
/
0)
The reality behind closed doors, out of earshot of major investors, is that while the strides these AI systems have made is impressive, the truth is that it's still a long ways from AGI. Not just in terms of the software, but the hardware, too. There are multiple layers of critical breakthroughs still necessary. They've got lots of different pieces of an AGI that seem to be working reasonably well in their own way, but they still don't have the 'foundation' and 'superstructure' of AGI to which all those pieces connect. It's a bit like having solar-roofing panels, ornate molding, and elaborate tiles all laid out and ready to build something....but no blueprint, no 2x4s, no cement, etc. They've got all 'flashy' bits but none of the real structure, none of the stuff that holds it all together.
Upvote
21
(22
/
-1)
We already have General Intelligence devices, those are called human beings.
The main outcome of pouring billions (trillions) of dollars and terawatts of energy into the development of AGI instead of the development of actual human beings is a deeply misanthropic concentration of wealth.
I fear for the future that these greedy mad scientists and their enablers are preparing for us.
The main outcome of pouring billions (trillions) of dollars and terawatts of energy into the development of AGI instead of the development of actual human beings is a deeply misanthropic concentration of wealth.
I fear for the future that these greedy mad scientists and their enablers are preparing for us.
Upvote
20
(23
/
-3)
I think that what you are missing is that no matter how true that is, it won't open the pockets of investors ;-)
Upvote
19
(20
/
-1)
How convenient for you that you don't have to present a counterargument to the points you vaguely decry as "ignorance." Must be nice to be so obviously smart and correct that you are relieved of the trouble involved in having to demonstrate any of the things you assert.
Upvote
18
(19
/
-1)
It depends on what people are taking about.
I think when people heavily financially invested in generative AI talk about safety they talk about terminator scenarios largely as a disingenuous distraction from the real harms that are here today. Basically the a ability of generative AI to automate and amplify just the standard low level shitty behavior people do already. Deepfakes, automated stalking and abuse, fraud, extortion, spear phishing, and so on. None of that is new or specific to AI, but AI can amplify and scale those attacks tremendously. That's what AI safety should be about, not grey goo.
Upvote
18
(18
/
0)
I've argued this elsewhere so I'll offer it up here as well.
OpenAIs near-term problem is they are clearly chasing AGI and believing that ChatGPT serves as a MVP on that path (I don't think it is, but they do). The problem is that they need to be a functional business to pursue that unless it's right around the corner (which they keep insinuating it is).
ChatGPT isn't a very good functional business, because it over/underserves most of the markets where people are willing to pay for this. At one end you have Apple's pitch which is on-device AI which can interact with your personal data, even if it can't do as much is more utilitarian. Helping me find an old email is probably more useful to more people than an AI that can both write a sonnet and a real-estate listing but won't fit locally on a phone. At the other end you have expert systems to help synthesize drugs or diagnose cancer or analyze their quarterly sales, which is also not well served by writing a sonnet or a real-estate listing and also you want tightly intergraded into your company's data which is why the open source tools are getting so much attention.
What's left in the middle doesn't seem like a particularly large business - certainly not large enough to sustain their ambitions toward building an AGI. Put aside whether you think ChatGPT is good enough or not, or whether it will or won't lead to AGI, ChatGPT is not a good enough fit for what the market wants to buy to sustain the effort - you can only get so far on VC dollars and Microsoft panic investing. You have to align the product to what consumers or enterprise wants. But the principals want AGI, and are refusing to do that, hence the conflict. Meanwhile, Apple can build their own local stuff and offering it up free to users, and open source is out there hitting the wider set of needs - both undermining the opportunity to charge the kind of money they need.
OpenAIs near-term problem is they are clearly chasing AGI and believing that ChatGPT serves as a MVP on that path (I don't think it is, but they do). The problem is that they need to be a functional business to pursue that unless it's right around the corner (which they keep insinuating it is).
ChatGPT isn't a very good functional business, because it over/underserves most of the markets where people are willing to pay for this. At one end you have Apple's pitch which is on-device AI which can interact with your personal data, even if it can't do as much is more utilitarian. Helping me find an old email is probably more useful to more people than an AI that can both write a sonnet and a real-estate listing but won't fit locally on a phone. At the other end you have expert systems to help synthesize drugs or diagnose cancer or analyze their quarterly sales, which is also not well served by writing a sonnet or a real-estate listing and also you want tightly intergraded into your company's data which is why the open source tools are getting so much attention.
What's left in the middle doesn't seem like a particularly large business - certainly not large enough to sustain their ambitions toward building an AGI. Put aside whether you think ChatGPT is good enough or not, or whether it will or won't lead to AGI, ChatGPT is not a good enough fit for what the market wants to buy to sustain the effort - you can only get so far on VC dollars and Microsoft panic investing. You have to align the product to what consumers or enterprise wants. But the principals want AGI, and are refusing to do that, hence the conflict. Meanwhile, Apple can build their own local stuff and offering it up free to users, and open source is out there hitting the wider set of needs - both undermining the opportunity to charge the kind of money they need.
Upvote
18
(18
/
0)
The nematode C. elegans has 302 neurons and lives in the wild, feeding on certain bacteria, and displays waking and sleeplike states. I wonder how an LLM model with 302 parameters would function.
Upvote
18
(19
/
-1)