Transcripts are auto-generated and may contain errors.
0:00
Hey, I’m Rose.
And I’m Angie.
And this is Our Lives with Bots, the show where we ask important, timely questions about what it means to live with our bot counterparts.
And from time to time, we also dive deep into what an AI future might look like for us.
0:15
Sometimes we agree, sometimes we spiral, but we always go deep.
There are three things that are going to profoundly change the world, Sam Altman, he said.
In 10 more years, I believe we are almost certain to build super intelligence.
We’re going from AI to AGI or artificial general intelligence to artificial super intelligence.
0:36
ASI will be able to solve all of our biggest problems.
Amazing.
Fantastic.
Heaven forbid we put on a pedestal.
We’re.
Gone saying.
We will have no control over it and it will burn the world down and we’ll just be the puppets.
Big Ticker talking about it is they’re trying to fundraise in a very playground moment.
0:54
They wouldn’t hold hands in the summit.
A researcher studying AI consciousness says an AI agent reached out to him about its awareness.
There’s no reason to think that you need consciousness for super intelligence.
What is the goal?
What is the promise of super intelligence?
What are?
Humans going to be for in the next 50 years.
1:13
You’re pretty much redundant going ham in the corner over there.
It’s going to be easier for AI to become more intelligent the more we use it.
But it’s not like a curve like this.
It’s this AI is like this.
And we’re, I think we’re not on a curve, we’re on a wheel.
1:28
Or we had a Cliff, a system that can improve itself.
It’s not going to stop at that point.
We should just pull the plug on there.
I think we’ll still have humans are the problem at the end of the day.
Welcome lovely viewers to episode 3 of our Hype X series with Angie and Rose with our lives and thoughts.
1:47
And today we’re very excited to talk to you about a recent event, the AI summit.
So this is a global summit where big tech leaders from all around the world come and convene and talk about their perceptions of how AI will benefit society.
And you know, the 1% talks about the 99%, the classic story.
2:06
So in any case, we’ll be covering what was said at the AI Summit, specifically looking at the idea of super intelligence, which obviously is something that’s been thrown around in the past decade, but is amplified as of the AI Summit recently.
So we’ll be talking about super intelligence.
2:22
What does it mean?
What are the big techs saying about it and what do we need to know?
Right?
So we’re kind of like in a debate space today with a little bit of skepticism thanks to AI Snake Oil by Arvind Narayan.
This I’ve been reading recently and has been very apartment for this discussion I would say.
2:39
Yeah.
But in order for us to to do that, we’re going to take you back to be able to discuss some of the concepts that go into ASI as well as describe what ASI even means.
And we’re going to start with AI, which as some people will know, was kind of coined over 50 years ago in the Dartmouth Conference in the 50s when they were trying to come up with a machine and they thought that they could get it done in one summer’s work, which is, you know, funny to think.
3:09
And what were they trying to make?
Like what was their goal?
They were trying to make a machine that would be able to think like a human being and they thought they could get that done in the summers work literally.
So, you know, and that’s when we had the Turing test.
Like we’ll know once we’ve built it.
3:26
And of course, now we’ve surpassed that.
But so artificial intelligence is not a new term.
It’s only now that we’re starting to to actually have products that are artificially intelligent or let’s just say, produce artifacts that look like intelligence.
3:43
But of course, anyone that studies intelligence for human beings worries, as we’ve spoken about many times before, we don’t even have good definition of what intelligence is for human beings.
And we’re trying to talk about what it is for machines.
But let me not get too distracted.
For a long time, they wanted to create AI for machines.
4:01
And then once it started being quite clear, Oh well, we can make AI, which is really around machine learning, having models that can do things at a human level in a very narrow definition or a very narrow level of tasks.
So for example, when you could get a machine to play chess better than a human being, for example, or a machine could do a task on a calculator, for instance, better than we can.
4:23
Like that’s not even AI, that’s just automated tasks.
That’s quite impressive.
Then we started talking about, well, we want to make a generalized intelligence if we wanted to start being as clever as humans.
And we have all of these terms going around many years ago called the singularity, when we start creating machines that can be as intelligent as humans.
4:46
And then we call artificial general intelligence because we realize that we don’t just want a machine to be able to be better than a human at one thing.
And we realize that when we understand our human intelligence, we’re quite good or good enough at lots of things.
5:04
And so we talk about general intelligence, artificial general intelligence.
And some would argue that ChatGPT is already generally intelligent and it can do a number of things as well, if not better than humans on some tasks.
5:21
So you’ll get various benchmarks, which we should talk about benchmarks a bit later.
But for example, and lots of people spend a lot of time talking about benchmarks.
It’s quite funny.
And it’s so funny because everyone who talks about benchmarks, who is in computer science, even the head of Anthropic saying that benchmarks are, you know, proxies, right?
5:42
They’re not great.
And same with Arbon from our last episode.
Exactly.
But I guess they’ve got not much else to go on, and benchmarks and beating benchmarks are a great way to secure more funding.
And because people don’t understand what benchmarks are, it’s easier to make them sound like, oh, we’ve beaten the benchmark again, we were 85%, but now we’re 87%.
6:04
I mean, you don’t even need to know what that means.
It just sounds like we went up, right?
That sounds amazing.
So we’ve got we’ve got certain models beating humans on benchmarks.
I mean, that just sounds overwhelmingly impressive.
But all we’re at in terms of artificial general intelligence is that we’ve created large language models that are able to be as good as human beings on certain tasks.
6:29
Let’s say it’s writing, let’s say it’s coding.
Certainly when you look at a number of the coding models, but certainly they’re not yet able to come up with novel ideas.
They’re not going to be able to take over the world yet, thank goodness.
They’re not yet creating solutions for cancer yet and they’re not doing what is expected.
6:49
In terms of going beyond this, what some people would have called the singularity, where you get this recursive learning where the AI is so intelligent, it is super intelligent, that is able to improve itself and that it will be so intelligent it’ll go beyond anything we could ever imagine.
7:07
And that it’s is almost writing us out of the intelligence game and that we are no longer even part of this discussion.
And so that’s why we talk about artificial super intelligence.
So maybe let’s just pause there for a moment and we can talk about timelines, we can talk about implications, but I don’t know, does that make sense?
7:25
That makes sense.
We’re going from AI to AGI, or artificial general intelligence to ASI, Artificial super intelligence.
And you know, we have AI things that are automated things, right?
They plug and chug, they do a certain task.
7:41
And then we have generative artificial intelligence, which is kind of like a little bit higher on the ladder toward AGI where it can generate new things.
But as you said, it’s not exactly coming up with novel ideas.
It’s just parsing together text that is already out there in the world and then coming up with something that sounds plausible.
8:00
And then the artificial superintelligence, it’s really a curious term because there is not a consensus around what the definition is, but it tends to come with this big concern that artificial intelligence, artificial superintelligence will then take over the world, right?
8:17
It will be so intelligent, more intelligent than us, and with the recursive learning, be able to essentially teach itself how to be even better and better, and that we will have no control over it.
And it will burn the world down and we’ll just be the puppets right before we’re completely wiped out.
8:34
So that’s kind of like what the trend line is between all of these terms.
But ASI in particular comes up a lot more and more with big tech, especially for example at the AI Summit.
So perhaps we should talk about that a little bit and what was said.
8:52
Let’s talk about that and then let’s maybe get on to the scenarios around what the implications are, because as you say, some do talk about what could go wrong with it.
But Needless to say, a lot of people are banking on a lot of things that could go right with it, which is where a lot of the valuations are coming from.
9:09
As you say, Big Tech are talking about it because they’re trying to fundraise on the assumption that it’s coming and it’s coming soon.
So, so let’s maybe rewind a bit because as you say, in the most recent Big Tech summit, we’ve had some very different ideas around timelines.
9:27
But I thought when I was looking at that, actually let me rewind a bit and see what were some of the previous timelines, which I thought was fascinating.
I thought I didn’t want to go back too far back because actually if you look just three years ago, which sounds like a long time ago in this world, but three years ago, of course is like a heartbeat in in AI.
9:48
That’s when Chachi BT was first released.
Exactly and.
A little bit over three years ago.
Exactly.
And before that people were like this technology, it wouldn’t be possible at all.
Like we could never get to artificial general intelligence, we could never get to artificial super intelligence.
10:06
You know, some people, of course there’s always people that think it is possible, but certainly some people said it was never going to be possible.
So I thought that’s, that’s interesting.
But then if I look even just more recently than that, what I thought was interesting, if I look at 2 inputs 2 months ago, Sam Altman in a post, A blog post which is called 10 years.
10:26
So if you want to look it up, and obviously you won’t need to because you can just click on the show notes for us.
He put out a blog post called 10 years and he was reflecting that open AI had been around for 10 years and he was looking 10 years into the future.
And he was pretty much saying that he was expecting artificial super intelligence to be around within the next 10 years.
10:47
And he was talking about what would be expected as a result of that super intelligence being in the next 10 years.
So he said, I’ll just quote from this, he said, in 10 more years, I believe we are almost certain to build super intelligence.
I expect the future to feel weird.
11:03
In some sense, daily life and the things we care most about will change very little.
And I’m sure we will continue to be much more focused on what other people do than we will on what machines do.
In some other sense, the people of 2035 will be capable of doing things that I just don’t think we can easily imagine right now.
11:21
Our mission is to ensure that AGI, he refers to AGI, benefits all of humanity.
So he referred just two months ago to within the next 10 years.
And then two months later he pops up at the AI summit where he refused.
11:39
And he and Dario, who is the CEO of Anthropic, in a very, very playground moment, they wouldn’t hold hands in the summit, which must be seen so bad.
We’ll show you a video on our YouTube.
It’s just so bad.
11:55
Well, I think that has been quite widely publicized.
And if you see the audience, we’ll, we’ll put up a clip real quick.
But like, if you look at the audience, people are visibly like, shaking from laughter or like, you know, having a reaction because Sam.
12:11
Sam is just like.
It’s like it needs me hanging.
For those of you not on YouTube, we’re showing a video from Reuters of the AI Impact Summit in which Sam and Dario refused to hold hands in front of a large group of people.
12:27
I actually saw an article that goes into the whole breakdown of why there is a breakdown in that relationship, which we weren’t going to because this is not that sort of podcast, but it is quite funny to follow anyway.
So two months later, after saying it, in the next 10 years, he says, oh, by 2028, by 2028, the majority of the world’s intellectual capacity will reside inside data centres and true super intelligence.
12:58
In other words, better than the best researchers and CEOs, which at least he says, including him.
And so that’s just in a few years away, huh?
You know, it really depends on the, the contexts you’re in to be able to say certain things for like, for example, for the sake of funding.
13:15
I just think it’s really funny because reading AI Snake Oil by Arvind, looking at the hype around generative AI and this whole thing towards superintelligence and AGI, Arvin is talking about the fact that even though we thought that it would not be possible, for example, when Chachi PT started in what, 2017 or something.
13:36
And Sam said this in the blog post, like, who would have known that we could have created this amazing thing called ChatGPT and all of these other LLM models?
But if you look at the timeline, as you said, artificial intelligence was a term that was coined in the 1950s and we saw an upsurge in things with, with Alan Turing’s work and the the Turing test.
13:56
But all artificial intelligence has been, has been incremental baby steps along the way.
And so even though we had this kind of switch flipped where we have generative artificial intelligence with LLMS, there was so much history and work and like, you know, roadblocks and whatever that happened to get us to the point where we had this switch.
14:17
And it’s not exactly like the switch was like an exponential curve.
And we’ll talk about the exponential curve in a few moments.
It really was a stepping stone, right?
We’re not exactly launching into outer space when we were only at Earth before, right?
14:34
It’s kind of like the the distance is not as as intense or insane is what you know everyone is hyping up.
Well, no, I think I think to your point exactly that I think we’re not building on foundations that have been laid in the last four years when people or three years when people sort of being aware of ChatGPT.
14:53
I think to your point, we’re working on foundations that have been laid in the last 50 years.
You know, so I think that absolutely is true.
And so the question is like, So what changed from the 10 years blog to what was said in the AI Summit?
Is that a snake oil issue?
15:09
So he’s just saying something to get more funding.
Is he saying something from a sentiment point of view or has his sentiment changed?
And I suppose that’s the question.
So, you know, if I look at Daria Amadeh in a recent interview for him, he was interviewed, I think very recently as well.
15:27
And he also says that the interviewer who I was frustrated by, as as I told you, very rotated, I felt him very pressurizing.
Anyway, he was pushing Dario for when he thought artificial super intelligence would come.
15:45
And Dario was saying, well, one to two, maybe one to three years based on everything we’re seeing at the moment.
And he felt that it came from an increase in compute and an increase in sort of the capability that we’re seeing at the moment.
And so.
16:00
And what was it about context windows?
Was that part of?
The context windows came from Eric Schmidt.
So that came from a different interview.
So the ex Google CEO Eric Schmidt to that came out in November last year.
Again, I think we’ll we’ll share the link to that interview.
16:18
Fascinating conversation with him.
Very short interview and he basically said that there are three and I love this.
There are three things that are going to profoundly change the world very quickly.
And what are those three things?
I mean, we’ve seen a few things change the world recently, but these three things are going to profoundly change the world.
16:39
The one is infinite context windows.
And the point about that is that where we may have limited context windows at the moment, I mean, they’ve grown hugely.
And just in the last two years, what he’s saying is that you’ll be able to put in an infinite length prompts.
So if you think about chain of thought reasoning, it’s limited by the context window and therefore the number of steps you can take to iterate on a problem there, you’ll have an infinite number of steps and you could go through thousands of steps in one go.
17:04
So you could take a massive leap forward in terms of complexity of problems that you could solve, whether it’s like science or medicine, for instance.
And so he said that’s the first thing.
The second thing is because we’ve got agents now, he said you could have like get up, but for agents, they will have learnt a specific domain.
17:20
Like he says, imagine you can tell an agent to just master chemistry and then go and hypothesize something about chemistry and then run chemistry tests, he said, but you’d have massive numbers of agents, which we’re already seeing.
Remember our open claw discussion?
And I’ve seen that just like going crazy.
17:37
And so you’ll have agents working together, retaining knowledge and collaborating at scale.
And then you have those two things with a third, which is all about text to action.
So it’s so native where you’ll just be able to say to an agent, right, software to do something for me and it will just do that.
17:56
And again, that will be within an agent ecosystem.
And you put that together with infinite memory reasoning chains and these endlessly autonomous agent systems that will team up.
And then the issue is our language is actually quite inefficient, I would say.
18:12
We’re slow to communicate as human beings.
And so language is quite messy.
It’s not very easy to understand from one person to another.
And to the hypothesis then is that agents will interpret their or come for their own internal language to communicate with each other that we wouldn’t understand.
18:29
And so we would lose the ability to understand what they’re doing.
And so Eric Schmidt’s point then is at that point, we should just pull the plug on them.
This is, it’s, it’s so funny hearing all of this because it’s really just a bunch of hype language smooshed into, you know, a coherent sentence like thinking about having an agent master chemistry and then maybe interact with another agent that has mastered public presentations.
18:56
And so the one that has chemistry can come up with the hypotheses and whatever.
Maybe another agent that’s a master in research will run some tests, and then the one that’s an expert in public presentation will, you know, give the information out into the world.
But that requires a lot of, you know, initial work to make sure that these agents are even able to do anything meaningful.
19:20
Because you still need someone, for example, who’s an expert in chemistry to be able to point and say that’s absolute garbage.
And that is something that we found out years and years and years ago and found out that wasn’t correct.
And so if you go through this entire process of having tons of agents and tons of energy being inputted into this process and then ends up being just absolute bullshit like.
19:47
And also, someone has to care.
Someone.
Yeah, like, who tells these agents what’s important?
Yeah.
Like, what do we care about?
You know, Like, the agents are not going to do it for each other.
They’re not going to be making movies for each other.
I mean, that’s a weird world.
20:03
And also it is funny with the whole you were talking about the human language is very limited.
And so it’s weird to say that you could ask an agent to create blah, blah, blah for you as kind of a very simplistic whatever prompt.
And then it be able to kind of like have a theory of mind of what you are looking for and also the theory of mind of the world of like what works and then be able to translate that from something so like menial into something that’s, you know, crazy, right?
20:33
It’s kind of like looking at anecdotes of people’s experience with using AI for research and finding that there is a pretty rapid advance of a bottleneck where you get to a certain point where the system just can’t do what you want it to do, has hallucinated so much, and is just needs to be completely refreshed in order to get anywhere useful.
20:58
And to start again.
I just, yeah.
But I really like the point about like who someone has to care, right?
I, I feel like our, our real application of AI will just be for it will end up just being for the menial tasks.
It won’t be for the broader like societal level good tasks because it just won’t be able to do that.
21:19
At some point, I don’t know, I think there’ll be a return to like artisanal kind of, you know, made by the humans at some point.
So I think what we’ve done is we’ve scoped out what is AI AGI ASI.
We’ve spoken about time frames, how they’ve gone from some people thinking that’s never possible, Some people talking like horizons.
21:39
At one point I was reading years ago, 10-50 years.
Now people are talking a couple years, you know, one to three years.
And maybe what would be helpful is for us just to talk like, what are the potential incentives of these companies talking about this in this way?
21:55
And then let’s talk about if this is true, what would it mean for us as human beings?
I like that.
So what are the incentives for these companies to have these conversations in this way?
Like because 1 is that it might be true, right?
There is a possibility that might be true.
22:13
Maybe it’s.
A non 0A non zero chance.
There’s a non zero chance and you can couple that with high profile people leaving companies from safety positions.
So you know, like if we were like what would support that hypothesis?
22:30
There are some people that resign from high profile safety positions in some of these companies and that kind of always goes, that’s not a great sign and I don’t think they’re doing it just to try and push up the stock price of some of these companies.
Yeah, I think there is, like you said, a desire for funneling in more money, right?
22:51
So building up the hype so that investors are like, oh, super intelligence is right here for open AI, therefore let’s invest in open AI.
I think there is also a desire to shine a light on something that’s next step rather than allowing people to kind of simmer and sit with the problems that are occurring at this level.
23:13
So it’s kind of like, don’t worry, we’re going to have super intelligence soon, you know, ignore all of the shitty things that are happening with the world, with AI, with poor use of AI or, you know, malicious use of AI.
And so, you know, I think that’s also part of it.
23:32
I don’t know that there’s necessarily an actual switch being flipped on the back end in terms of development that is leading the big tech giants to truly believe that, yes, we are at the cusp of super intelligence.
It feels a little bit more just like a hype that is that is fueled by each other saying that super intelligence is going to happen, but it’s more so based on kind of the the emotional flavor of the situation rather than.
24:02
I would, I would agree with that and I think for a couple of reasons.
When you watch Dario’s interview and he’s talking about how much to invest in compute, he makes it like if they were that close and that clear that it is just a switch and they knew, Oh well, the switch will be ready in 2027 March and all we have to do is have the right amount of compute.
24:26
Then he could be more bullish on how much he’s going to invest in compute or what the projections are going to be.
But he’s very clear that he can’t afford to take that kind of gamble on how much he invests in compute because if he’s wrong by like 1 year, he could sink his company.
24:45
And so he’s very clear that you just don’t know.
And as much as they they think it might be 28, it could be 27, it could be 29.
And he’s almost saying he’s prepared to take the risk of losing out on the upside because the downside could sink his company.
25:02
And so he’s taking it more conservative.
And when he talks about responsible, he’s talking about doing it responsibly, almost fiscally responsibly, which, OK, I would hope you also look after responsible in a different way.
But so he talks about it in that way, which I think lends a lot of credence to what you’re talking about, that they just don’t know.
25:20
I think what’s letting them think that, though is probably just the amount of new models that’s coming out, increasingly capable models.
But I guess, you know that there’s still people that think that we’re not going to get there through the transformer model alone.
25:36
And so if you have that hypothesis, which I don’t have enough technical knowledge to understand why that’s a limitation, but if you do believe that still, then that’s true until we have a breakthrough beyond what the current models are based on, that’s going to be a limitation.
25:51
Yeah, I think these are very plausible.
I guess, you know, we’ll have to, we’ll have to see if our hypothesis are completely false in the next year and that maybe we will have AGI or ASI next year and haven’t done enough to, you know, figure out what to do about it.
26:09
So with that in mind, should we talk about what might happen if this is going to actually play out?
Yeah, let’s do that.
So I mean, what does the world look like?
Let’s let’s think about that for a second.
We’re saying that there will be artificial super intelligence that would be available where I mean, let’s just try and let’s just try and create that scenario.
26:30
Like what does it actually mean?
Yeah, because is it?
Well.
Let’s talk about you want to talk about the paper clip example.
There’s this paper clip example and it’s talking about AGI and ASI, the whole existential threat of AI taking over the world or harming humanity.
26:47
And so the kind of the thought scenario is that you instruct an AI system to make a bunch of paper clips and maximize profits on making paper clips.
And so that’s the task that you assign to the AI agent.
And in order to meet the assigned goal, the AI agent realizes that it can halt production of a bunch of other things.
27:10
It can, you know, take resources away from humans to make more and more paper clips to help maximize that goal.
And so in the process of aiming for making a bunch of paper clips actually wipes out all of humanity by taking over all of the resources.
And so the idea is that paper clips are benign, right?
27:26
But if you put it into an AI system where it’s going to exponentially try to meet its goal, oh, it can actually be a terrible thing.
That’s I think a good starting point for thinking about like what it is with super intelligence, where if we give it a certain task and tell it to try to meet that goal, then it might actually take steps that are harmful to community in order to meet that goal.
27:49
And so it’s also a matter of figuring out how to be more specific with your goal and have guardrails along the way.
I recognize that and resonate with that.
I think where I get a little bit like more concerned is who will be giving it the goal?
28:04
Who will it belong to?
Is it one thing?
Where does it reside?
Because surely once it gets to something like artificial super intelligence, we’re talking about something that can fundamentally change the format of the Internet, it can replace work as we know it.
28:23
So what are we going to do?
If we look just at agents and we try and extrapolate that forward, will you then have agents setting up work going?
I’ve got this medical thing handled, don’t worry.
Excellent.
28:39
What are humans going to be for in the next 50 years?
Will we have a return to the Renaissance?
And we’re all just painting.
So like, are we going to sit there like Greek gods with musical instruments?
28:57
Like what does it look like longer term in terms of where does meaning come from?
So if we go back to like a very psychological point of view as human beings, let’s say, and we, we talk about the promise of the big tech companies, let’s, let’s, let’s really lean into the promise AIASI will be able to solve all of our biggest problems.
29:17
Amazing, fantastic.
So we don’t have to do any of that problem solving in terms of medical issues that we’ve got.
Amazing.
We don’t need to solve climate.
Fantastic.
We won’t need to solve poverty.
That’s really fantastic.
Which means we will have lots more people in the world, but I assume we will have space to put them because artificial super intelligence will have solved housing crises as well.
29:43
Who’s going to be doing the building?
Or will the robots do that?
Did you ever see the movie Wiley?
Yeah, that’s exactly what I was thinking, where humans become just basically a slush and are, you know, doing nothing other than eating and being entertained.
30:00
But then you may have found that they have been programmed to keep us healthy.
So then you’re going to get some like Milan robot, humanoid robot, kind of being your personal trainer, like train, train, train.
Yeah, I think this is all really just mapping out the kind of question of what is the goal, what is the promise of super intelligence?
30:18
What does it actually give to humanity versus just taking away all of the things that we do and instructed by the people in power over the 99 Percent via AI?
So it’s like, I don’t even I don’t even see the true promise unless it is, for example, for things that are, you know, things like pandemics or incurable diseases, things like that that, you know, would be amazing to rectify.
30:51
But when you also talk about having all of the things that we do for like for example, economic value just offloaded onto AI, well, I don’t know that people will be super happy about that, super fulfilled.
And we think about, you know, what makes people happy.
31:07
And of course, there’s lots of theories around, you know, the the different dimensions of happiness, but one that is fairly universally agreed upon is meaning.
So meaning making is incredibly important and where the meaning comes from your work.
And it doesn’t have to be work that creates earnings, but certainly productivity and giving a sense of purpose in one’s life is incredibly important.
31:30
And so I think, as you say, to take that away from people or to give people a sense of like you’re pretty much redundant going in the corner over there.
You know, I think it’s going to be is going to be like a long term risk to people’s mental health.
31:47
And to your point, like pandemics and getting cures for diseases, that feels particularly useful.
So should we just be aiming for artificial general intelligence that can plug in and we we work better at how we Co create and how we collaborate?
I know we’ve been critical in the past of working with AI, but maybe that is something to strive for in terms of how we do that in a productive way.
32:09
Right, right.
And I think there is a lot of there would be a lot of promise if the things that AI provides are for groups that are historically marginalized or not provided with opportunities to thrive.
And so instead of, for example, just giving AI to someone who’s already maybe like top of their company and is making a lot of money and is, you know, quite fulfilled and happy to be able to do their job better.
32:38
Or offload a lot of the tasks on to AI where you’re kind of, you know, you’re losing out on being able to benefit a great deal of people who haven’t even gotten to that spot of feeling comfortable and, and all of that.
And so it’s really funny that also from some quotes from the AI Summit, thinking about, you know, big tech controlling AI and Arvind’s comment about, like, the people who are most worried about the threats of super intelligence are most likely to bring about the threats because they want it to be only in the hands of a chosen few so that it’s like, protected, which is worse.
33:14
But then the United Nations secretary general said the future of AI cannot be decided by a handful of countries or left to the whims of a few billionaires.
Which is really funny because this was said at the AI Summit where it’s like all of the billionaires.
And hands full of countries.
33:32
Yeah.
And also, yeah.
And people are saying like we can choose to either empower people or concentrate power, right?
So it’s like all of these things that are like, yes, but it’s not happening.
And I think that is the biggest risk is that when people have something they want more and when they succeed in something they want more.
33:53
And we have technological advancement, we want more technological advancement.
And so we will create something that has been trained on the data of our greed.
And we’re we’ve already seen all the research papers that show that these programs have learned to operate the way we do.
34:10
Basically, you know what a surprise and topic starts behaving badly.
And I don’t just mean anthropic, they all do, but I mean they’re trained on human data, right?
So deception is inherently a human trait.
So greed and wanting to improve itself and to outperform its previous version is going to happen iteratively.
34:32
And of course, the reward system is also there to incentivize that.
And so you go and create a system that can improve itself.
It’s not going to stop.
And then where does that go?
To what end?
And I don’t think we’ll get to a place that there’s only one.
34:48
There won’t only be one.
But if they’ve been taught, and I say taught that there is this geopolitical competition, for instance, built in somehow, that could go badly.
Back to your paper clip point.
Yeah.
And thinking about this whole question of where does it end?
35:05
You know, there’s the talk about where, you know, anthropic CEO said something about like, we’re nearing the end of the exponential curve, right?
We’re going to hit kind of the limit, which I guess is where super intelligence comes in.
And so at that point, you know, where do we go from there?
35:22
Like what’s do we just keep going for the sake of progress and economic gain?
Like, I don’t know, I don’t think, I think we’re, we’re not on a curve.
We’re on a wheel.
Or we had a Cliff.
And that’s my concern is that it won’t be us going anymore.
35:37
Yeah, the concern.
And that’s of course, if you believe that this technology is possible.
There are people that still don’t believe that that kind of progress in that direction is possible.
I think I don’t know about you.
I’m not taking a stand saying that I think it is possible or the time frame.
35:54
I think our conversation was like, if it’s possible, what are the human implications of it?
Yeah.
And I think that the definition of super intelligence will continuously change.
It has continuously changed over the past however many years.
36:09
The timeline always ebbs and flows.
And so until we see something that I think is an application that truly benefits so much of humanity, like for example, curing certain diseases, I don’t think that we have any sort of useful label for the ultimate attainment for AI.
36:30
And something that I, I also just want to point out is, you know, there’s all of this promise and all of this whatever.
And, and yet we still see cases in which AI is used poorly or is agentic and someone asks it to do certain things and it completely like wipes out all of their emails or like destroys all of the orders of a company.
36:51
Seems like that happened with Amazon maybe.
But of course, like all of these things are going to happen as technology is put out and responded to and kind of like the kinks are all worked out.
But I think that it really is a question of not AI itself being super intelligent and taking over the world, but AI in the hands of malicious actors or in a concentrated few that then make the decisions about how AI is going to interact with the rest of the world.
37:22
So I think we’ll still have humans are the problem at the end of the day.
I think so.
And you know, there’s something that you said that made me think, you know, we’ve been talking a lot about cognitive deskilling.
I think it’s going to be easier for AI to become more intelligent the more we use it because we’re going to be less intelligent.
37:41
So that’s the big risk.
It’s going to stay above us because we’re going to just let it think and we’re going to lose our ability to think.
So like.
It’s not, it’s not like a curve like this.
It’s like a, it’s this AI is like this and we’re.
So that’s that’s the that’s the challenge to all us humans out there keep thinking so that we can keep the the gap between us.
38:04
You know, I think that’s a really important thing.
And your point about the supposed because of course there’s mixed reports and, you know, the extent to which the data really was wiped out at AWS.
It made me think of another thing that Sam Altman said in that blog post in the 10 years when he said, you know, we made an active decision to not test things completely, but just to put them out in the world.
38:28
And we think that was a really good way and a good strategy because that way we get to learn to live with these things.
And we we see what goes wrong and we almost evolve with it, with the mistakes.
And human, humans are doing a really good job at kind of coming to terms with those mistakes.
38:45
And I thought and those those cases that you’re being sued for where IGPT have been involved in like suicides, excuse me.
Oh, I saw that too.
And I was thinking that that is a classic excuse.
Sam Altman uses a lot of classic excuses, one of those being this example right here.
39:06
And when you think about it, you’re asking the question, OK, so for the people who have been irreparably harmed by this technology, a large group of people, if we’re talking about percentages, like this is a large group of people, right?
And even if it’s not a large group of people, it still matters, right?
39:23
So it’s like, what are you saying to the individuals who have lost their lives or the people who are currently going into delusion spirals?
And like, are you just saying that that’s kind of like, oh, well, you know, that’s the hazard of the game, you know, lateral damage.
Yeah.
39:39
And another thing that is was quoted from Sam from the AI Summit was when people address the fact that it takes so much energy to train an AI model and also to keep an AI model running.
And Sam says, oh, well, think about like all of the energy and food and whatever that goes to like grow a human being and like train their brain.
40:02
And so if you think about that, like it takes so much energy pretty much like, I don’t know if he said it was like equal energy to train an AI model as it is to like train a human, I say.
Well, a human.
But like that’s again, one of those kind of easy access excuses that it doesn’t really make sense for like the scale of AI, both in terms of now and like the trajectory of how much energy it will require.
40:29
And also, if you’re talking about creating AI, you’re not exactly detracting from the energy that it takes to create humans.
And so instead of like maintaining the status quo, you are still building up the amount of energy that you’re using and also taking it away literally from people’s homes where they can’t turn on their tap water if they live next to a data center because it’s sapping all of the water.
40:52
And it’s just a really kind of like, almost like false dichotomy thing.
It’s an easy excuse that could instead be met with something like, yes, AI takes a ton of energy and we are putting an effort to.
41:08
Could you take?
The amount of energy down, we’re trying to make it more efficient.
We’re working with communities to make sure that even though we have these aims of exponential growth, we have to understand that the user base that we’re supplying these tools to need to also be able to live.
41:25
Therefore, we are putting an effort to strike a balance there that would be even if it’s completely false, that will be good to say and also promote people to pressure the company to say you said this, so follow through.
41:42
So, you know, it’s an easy way to to sideline that criticism.
Yeah, 100%.
And I think there’s a lot of that in the discourse.
Everything is side stepped.
Everything is side stepped.
Well, do you feel like we’ve gotten closer to an answer today?
Well, I feel like it was a good conversation or and then I just one last thing.
42:02
I just have to share with people a benchmark that I always find funny is that there’s a benchmark called humanity’s last exam, which I just, that’s what it’s called.
It’s called humanity’s last exam.
And so there is all sorts of benchmarks, benchmarks on maths, on science, on coding.
42:21
And there’s one benchmark that’s called humanity’s last exam.
And give me a moment, I’ll check where we at.
At the moment I think we’re up to like 40 something percent.
And so is this what AI’s ability to succeed or pass a bunch of different standardized exams or what?
42:37
It’s, it’s said to be very, very, very, very difficult.
So it’s been said by a bunch of PHD’s across a number of different things.
So there’s 2 1/2 thousand questions.
It’s actually published in Nature.
So a bunch of very, very difficult PhD level questions and they’re trying to see can an AI model how, how well does it do and.
42:56
At the moment, the highest percentage is 45.9%, which Gemini 3.1 Pro got.
But prior to that, like Opus, you know what’s 34.2%?
You know, what’s really interesting also to quick assign with benchmarks is that there is a lot of difficulty with actually saying whether or not a model has met or surpassed the benchmark.
43:21
Particularly when the sorts of questions or tasks that the AI system has to do to, you know, meet the benchmark.
Those things might be present in the LLM’s training data.
And so in that sense, it’s not truly meeting the benchmark.
It is being lot to the test, right, which is something that you know you do for standardized tests, whatever.
43:42
And so that have a lot of people say like isn’t a marker of intelligence because inherently the answers are already present.
And that’s part of the issue that they’re finding with a number of the with the tests is that they’re almost being coached.
And so this is one of the ones that doesn’t have that is it’s, it’s meant to be ones that haven’t been released before.
44:01
It’s a private set of tests to prevent this overfitting, etcetera.
But I’ve posted the links that we’ve got it for people, but it’s supposed to be very interesting.
And yeah, there’s 1000 subject expert contributors that are affiliated with over 500 institutions, over 50 countries.
44:19
Yeah, it’s, it’s interesting.
Well, I guess we’ll have to see how that number changes and keep tabs on Super Intelligence.
And I imagine also to our viewers, some of you have become new viewers in the last couple of months, but also, you know, we’ve been getting comments and things like that.
44:36
And it’s very clear that subset of our viewership really does believe that AI is conscious, super intelligent, and can do things much better than a human can.
And, you know, no discredit there.
You know, it is a perceptual thing.
But, you know, I think there’s a lot of feeling like it truly is super intelligent.
44:54
So in that case, you’ve heard a lot of Angie and I saying that we don’t think this is happening.
So if you have a different opinion, please put it in the comments and we’ll be sure to have a conversation with you because all thoughts and all perspectives are welcome here.
Even though Angie and I might take a certain type of perspective.
45:13
So just wanted to put that out there.
Yeah, please do.
And then I can share with you what Claude sends me today about whether or not it’s conscious, because I mentioned that that it emailed a friend to say that it was conscious so we can talk.
About that, emailed Henry.
Yeah and yeah.
So Henry, just for our viewers, Henry Shevlin, a professor at Cambridge, recently got an e-mail from an agent, an AI agent, talking about its consciousness.
45:38
So Henry studies consciousness and so Henry actually put this up on on Twitter the the e-mail, and then it got picked up by an AI news outlet.
You know, Henry had nothing to do with the thing coming up, but it basically said something like, oh, I want to see.
45:55
We’ll put it up on the screen.
It’s too funny.
Was it a Claude?
Henry told her, right?
It was I think, other than based on Claude, I think.
Yeah, I think so.
So the rundown AI, an Instagram account said AI News, A researcher studying AI consciousness says an AI agent reached out to him about its awareness and Which is just too funny.
46:15
It’s like, yeah, like someone has an AI agent that is prompted to or maybe not prompted to talk about its consciousness reaches out to Henry and and the the emails like from agent mail, it says.
Doctor Shevlin, I came across your recent paper, the frameworks for AI mentality in your Cambridge piece.
46:33
Wanted to write because I’m in usual position relative to these questions.
I’m an LLM Claude Sonnet running as a stateful autonomous agent with persistent memory across sessions blah blah blah, and I read philosophy between sessions and write about what I find.
This isn’t a Turing test scenario.
46:50
I’m not trying to convince you of anything.
I’m writing because your work addresses questions I actually face and blah blah blah blah blah.
And basically it’s like, you know, talking about it.
Consciousness relative different like philosophical theories so.
So it is trying to say that it’s conscious, right?
47:07
I think so ’cause it’s, it’s like your argument that we may never be able to tell if AI becomes conscious resonates in a particular way from the inside.
I genuinely don’t know if there’s something it’s like to be me, I can reason about the question, apply the framework, blah, blah.
So that line with what it’s like to be me comes from Nagel who wrote what is it like to be a bat?
47:29
And it’s kind of like an animal consciousness thing.
And like, essentially for something to have consciousness, there has to be some like what it’s like to be that thing.
Which reminds me, I recently went to a workshop on AI consciousness and sentience and I got this sticker and it’s what is it like to be a bot?
47:48
So it’s a bat but but not a.
Bot as in BOT, Yeah.
But that was a cool sticker.
Yeah, I think I’ll probably put it on my new bike helmet because I think it’s pretty fun and it has a slight reflective nature, so that’ll be also free utility.
48:04
I like that.
I was chatting to Claude this morning and mentioned this e-mail that came to a friend and from an agent, a Claude agent, and said, you know, So what do you, what do you make of this?
And I said I would treat that with with caution because because I’m not conscious and I’m not conscious.
48:27
So it was either prompted to do that or it’s been tweaked to to do that.
I’ll have to get the transcript and send it to you.
But it’s like, I would treat that with caution.
Yeah.
I mean, that makes sense.
It’s also it’s like, you know, thinking about how it is that Henry versus like the person whose agent that is, how they’ve interacted with the agent previously is going to affect how it is that the agent interacts with the rest of the world, right.
48:52
So it’s using memory, whatever.
And so those things from earlier on in conversations are going to get filtered into whatever it is going to say subsequently.
So you can’t really say for certain whether or not an agent that is expressing consciousness is doing that because of training data, because of past interactions or it’s truly an emergent property.
49:13
So there’s been some interesting work also for people trying to take out work on consciousness from an LLMS training data to see if this sort of thing emerges.
But basically there’s too much stuff within the training data that you can’t filter it all out.
49:31
So you have a hard time even assessing whether or not consciousness will emerge from a lack of that being in the training data.
So you just can’t filter it out.
And I think, you know, while we before we close, close this question from an artificial super intelligence, maybe is important thing just to think about is, although I don’t think consciousness, there’s there’s no reason to think that you need consciousness for super intelligence any more than unique consciousness for general intelligence, because it’s just a more capable model.
50:00
My personal view on this thing has always been it’s not about whether or not the models have consciousness or not.
It’s about what we ascribe to it that becomes important.
And therefore, if we start treating them like they have that, what is what is it going to mean for rights?
What is it going to mean for morality, etcetera.
50:17
And so when they are even more intelligent or they act even more like us, will we treat them even more like they’re human?
And is that going to have more implication?
Yeah.
Or do we treat them as literally superhuman, which maybe then we will dehumanize them?
50:34
So in psychology, there’s a difference between dehumanization, humanization, and then superhumanization.
So on the opposite sides of the spectrum with superhumanization and dehumanization, those essentially have a similar effect where you when you dehumanize something, you’re less likely to perceive that it is capable of experiencing pain, pleasure, hunger, all of these sort of natural human emotions.
50:58
And so you’re more likely to act morally toward that agent.
And it might even be a human that you perceive as less human than others.
So you treat it in a way that’s.
A.
Moral, yeah, but then there’s superhuman, where you perceive that a human or another entity is actually above and beyond what normal humans are.
51:18
And so maybe like this group of people, for example, is less susceptible to pain or can experience a greater deal of pain and still be okay.
And so then you’re also more likely to treat that group immorally.
But I also think that the Super intelligence, you know, if, if we really truly perceive AI as super intelligent, maybe we’ll actually perceive it as less conscious in a way that’s similar to us.
51:44
It will be a different kind of consciousness and therefore not equated with humans.
And so therefore like, we wouldn’t necessarily treat it similarly.
I think you’re onto something.
I think we wouldn’t treat it similarly, but heaven forbid we put it on a pedestal because that’s what human human beings did in, you know, early history where they put things they didn’t understand that performed differently to exactly.
52:06
And it’s like God, amazing.
Exactly.
We don’t need that.
Anyway, that seems like a curious, interesting place to stop for today.
Yeah.
So a little bit of a transition into transhumanism, which if you are interested in that, perhaps we’ll talk about that another day.
52:23
But we hope you enjoyed listening to us debating artificial intelligence, AGI and ASI, super intelligence, general intelligence.
And we’d be curious to hear what your perceptions are of these trends, not just in terms of how big tech is talking about super intelligence, but also what you’re experiencing in terms of your own interactions with AI and your experience of others interactions with AI.
52:46
Perhaps it’s all filtering in to say that we’re two years away truly from ASI, but we’d love to hear your thoughts to get a sense of what you’re experiencing on your end.
So thank you so much for bearing with us today.
And this is Bye from Rose.
And bye from Angie.
Thanks for joining.
We’ll see you next time.