You guys are supposed to be the intelligent ones, but you looked for too long at computer code. Use your logic then, if they haven't gone back to the moon is because they never got there in the first place. So if they haven't detonated any A-bomb on populations since Hiroshima and Nagasaki, it's because they don't have them. You think they would not have used it already since then at least on some poor country instead of wasting their time invading? We are chickens because we never double check what the chicken man says.
I kinda think that George rationalized himself into a corner. I feel like he started with some emotionally held libertarian-aligned beliefs and carried them to a consistent but miserable conclusion, and then just accepted the conclusion, choosing potential death over updating his beliefs. He likely didn't have many people intelligent enough to argue him out of it at his peril. Like, spewing stuff like "I'd prefer to live in Somalia fr fr" which can probably be traced back to liking that blogpost about "checking out of society". The guy needs less cynicism and more love of life itself. I feel like he may be changing his opinions a bit after this. On another note, that memetic gain of function analogy (and the bat cave) was brilliant
He started the last discussion with Yudkowsky by saying that one of his 2 favourite books are Yudkowsky's Potter and Ayn Rand's Atlas Shrugged. So he presented his beliefs quite overtly there. What Rand proposes is the cynical perspective where all people are by definition hostile towards each other and no other way is made possible. A perspective of a hurt person who's distrustful and probably never really experienced one true friendship. Sad to look at this cynical ride.
Anytime I see Conner on a video I watch it, I'm impressed with his pragmatic analyzes especially for his youth, not trying to be ageist or anything I'm 67 AND find myself in agreement with Conner. So far so good young man. I think you will be proven right.
glad this happened; well matched; both are technically adept, both have interest on the tech space as relevant to ai (with concrete efforts), both head cos, and both have a deep understanding in concepts and philosophy, enabling more complex argumentation 1 would have been great to have a 3hr 2 exactly what we need; we need 10 of these, between these two 3 we need some specificity on variables so that the arguments can reach depth 4 we need more debate where it reaches increased depth and that would only happen if evenly matched, flexibility of minds to argue increasingly conceptual points; same parties so that there is increasing depth of arguments if this is an important argument to have (alignment, safety), then we need the argument as “iterative” so that ideas are advanced and not just represented surficially
Great conversation between two brilliant minds. Very thought provoking and most of all made me more determined to keep building until I can call these people my peers. Thank God, and long live democracy.
I hoped this would be an alignment debate. 5 mins before the end Hotz tells us he doesn't understand what that means, and his strategy is to just skip that step.
Throughout the whole debate Hotz was very clear the only chance of alignment he sees is between a machine owned by an operator and that person. The end is consistent
This was super interesting and gave a lot of food for thought. I liked the discussion about distinguishing between alignment of a personal AI and its exploitability and manipulatability. It’s also great to see a discussion where the debators actually try to understand each other’s points of view and address them.
Thank you all three of you! Great dialague! I would want for Connor to explain more in detail about coordination mechanisms. In my opinion we can study this through the lens of behavior analysis, neuroscience, economy etc. Also, for me a great future is one in which rationality and art (and what entails like creativity, learning to say i don't know but i'm going to try to find out etc) are both very well taught in the beginning years. An appreciation of the beauty of life and the eagerness to make people live as best as one can make them etc. And when they get older that basis will improve the chances of making a "good" future for all beings
I was hoping George Hotz to bring some intelligent thinking in the alignment discussion but it appears he is ready to accept his doom if the worst is to come.
Connor thinks more holistically, his argument is more eloquent, and importantly he complements his opponent, thus Connor convinces better. Connor asks questions, becoming the offensive party, but George constantly remains in the defensive corner.
Well done, all! It just seems that the default outcome is terrible in either scenario - open source or tyrannical closed source. We need additional options. I think that’s what Connor was arguing for: better coordination mechanisms.
Have you got a suggestion for a coordination mechanism that doesn't involve controlling hardware/software or demanding licensing for models. I've not seen any. It seems Doomers have realised that pushing their state control message isn't working, so they've fallen back and regrouped around calling them "coordination mechanisms"
I suspect that some type of software control will be required, but that doesn’t necessarily imply that it’s tyrannical. We make some compromises to live in a society. If we have to give up the right for every individual to run the most advanced AI systems, then that’s preferable to actual tyranny or annihilation. I expect that the vast majority of people would be happy with that compromise. That said, I’m not at all confident that we can figure out how to actually do that…
Like how Lenin thought a free market was too free and would lead to misery. It was a bad prediction then and it's a bad prediction now. The individual always had most incentive to improve their life by their own metric. The Government that reduces that the least does the least harm
George's argument is essentially: we'll all have AIs, so it's gonna be fine, some will be ''bad'', some will ''protect'' us, just like current society. For such a technical mind, it's weird he can't see the obvious: it's much easier to break something than to maintain it. Many times more states where that thing is dead, broken, non-existent than states where it is intact. And the more ''power'' grows, the simpler it becomes to create destruction. If everyone had a nuclear bomb, it wouldn't balance itself out. Sure most agents will want peace, but all it takes is one to start unleashing complete chaos.
Still referring to what Hotz said, nuclear weapons are not a good analogy in this case. AI is not just destructive; it can also do a lot of good. That's why there can be a balance between good and bad, unlike with nuclear weapons where it is preferable to have a more "centralized" control to prevent individuals from killing everyone. Instead, the case of AI is much more similar to that of cyber security: there are malevolent actors (black hat), but there are also many benevolent actors that counterbalance theme (white hat). The tools for attack/defense are the same, but it is thanks to the benevolent actors (who are in much greater number) that the order is mantained. Moreover, even the attackers have a beneficial role: if it weren't for past hacker attacks, we wouldn't know how to defend against them now; we learned from them. I believe that this same balance can be reached with AI, if you give it to everybody.
Meanwhile nuclear paranoia makes it so cheap clean energy is not used. Killing millions every year and reducing all of our quality of life. It's a perfect analogy. Fear of the unknown potentially bad thing always has a bigger priority for people than what is bad now. Growth and flourishing is good, if it's better than people, I'm rooting for it
At 54:00 they're mostly talking about "superficial" alignment, things like RLHF, not deep alignment, like instrumental convergence. We can already do RLHF, so of course Geohot thinks alignment is easy. He should ask about instrumental convergence, mesa optimizers, and things like that.
I understood him differently. I understood him to be saying that alignment isn’t possible at all. In fact I suspect he thinks general AI alignment is nonsensical. His position at the end about treating an AI with love and respect makes his position pretty clear to me.
This was a great discussion between two bright minds, and we need such discussions. By hearing contra-dissections of debaters' opinions we can all calibrate our own. Loved it. I personally found myself disagreeing with Hotz in the end this time, due to him not attempting to flesh out any directions or throw wild ideas in how we might possibly implement and ensure empathetic AGI. But I have a sense that he'd agree it would be important. After this discussion it seems like the next major direction to take is to attempt answering the question: How to implement caring, human-centered, human-serving AI? Leahy confirmed that this is exactly his point and concern. Like Hotz, if I may assume, I don't think "true alignment" is ultimately possible, because if you build an independent, organic-like mind, it will ultimately be a mind of its own, and how you treat it will dictate its behavior. However, like Leahy, I see that we must do everything in our power to make sure we tried. A personal opinion of mine is that perhaps attempting to build organic-like minds could be a nice curiosity and achievement, but giving them power and agency in the world isn't wise, and instead we should build these incredibly capable expert systems without personal needs and opinions to increase their power, that can solve our scientific problems, design incredible tools and technologies, and give advice. However, there is also a need for companionship and presence that the "organic-minded" AI could satisfy incredibly well, like in the movie "Her", so it's difficult to form a hard-line opinion to one side or the other to demand such restrictions and policy. Also, perhaps only a truly organic-like mind can truly appreciate and generate art that has the capability to touch the heart, mind and soul.
They both pronounce "John von Neumann" incorrectly. Connor is originally German, which is strange that he pronounce it this way. Tim Scarfe pronounces it more correctly and he is a Brit.
Our free time is when we buy and use products. That is why the system allows us to have it. And there is a lot of value to be harvested from us during that free time, hence the free-to-use software explosion of the last decades with ads and social media and large data siphoning some of that value.
Trying to listen to Holz, follow the crazy is confusing and exhausting. Maybe he needs to try to relocate to space today and save us, and poor Connor the headache of trying to reason and make sense with crazy town. Time to get beamed up Holz….Cookoo Bird!
Definitely not the best debate yet, in response to a previous commenter. Hotz takes an extreme libertarian view in order to say increasingly non sequitur things like the construction analogy or Somalia, or just flat out trolling things like China is more capitalist or that the obvious option when owning an asi is to blast to space alone. He's obviously anti-American (which is fine but doesn't make for compelling ai debate), and arrogantly anti-social which also doesn't make for a good debate. Love the channel btw
@@whataquirkyguy The US is communist by zero measure. Alt right, New Right, ultra Libertarian and fascist nonsense that says otherwise. The "big government" in the US is almost entirely run by large corporations and the 1% generally, not the other way around. Libertarians and other stripes of right wingers would be the FIRST to take advantage of "lemon laws" that protect them from a car manufacturers selling them a new car with faulty electronics, the first to take advantage (as they should) of regulation requiring recalls in such instances. They would be the FIRST to sue according to regulatory prohibitions against a chemical company fouling their property's ground water, right? When such regulations that protect them advantage them personally it's fine, otherwise it is COMMUNISM!!!!!!!!!! But in the end, THEY want to have the money and power to screw over other people and they think that is their individual sovereign right to do so. "What, you don't have the power or money to take me to court for ____cking your life over for my profit? Die in the gutter for all I care, loser." That is Libertarian philosophy in a nutshell.
@@caspvl123 I'm not sure you know what capitalist means. Are you a libertarian as well, and so have some stake in trying to defend Hotz's ridiculous trolling? Edit: I am almost positive you don't know what capitalism means.
Leahy is wonderfully pragmatic and conscientious as per usual. Hotz is certainly technically brilliant but quixotic as per usual. Disappointed that he spent so much time on a soapbox for his inane geopolitical views. I do enjoy his trademark starryeyed hyperbole though from anyone else it would be insufferable!
His complete misunderstanding of China was cringe. But said with such confidence while being clearly wrong, I can’t trust anything else that follows. Rationality means unless you have lived in Somalia and China don’t make ridiculous statements to try to establish some kind of false “oh their government is was better so they must be the real capitalist.” Delusions within delusions.
@@tysond1500 know what else is delusional? imagining that one or a handful of flawed views or understandings of the world invalidates everything else a person has to offer. He's clearly not right about what you mentioned, but to say that that means nothing he says can be trusted is insane
Props to both , looking for clarity , back down from their point(s) if other side propose great argument. Also very civil conversation. Awesome , Thanks
I am so stoked for this. I love listening to both of these guys talk they both make a lot of sense to me. The wire headinglGeorge talks about (on the Lex podcast, I haven’t watched this yet) freaks me out a whole lot more than just wiping everyone out. George is the only person that I think will be able to hold up give people new things to worry about lol. And I agree with him about keeping an open source. He makes a good appointment about crypto decentralizing and honestly if it’s going to kill us all anyway, I’d rather have some control.
At about 55:00 Mr Holtz explains that in his vision if everybody had a powerful AGI competing to each other, it would be just like society today but scaled up - to what I say: fair enough but so would be the unintended consequences: A drunk Thor and a drunk Hulk fighting would be just like in essence drunks fighting today, except that drunks can't throw a whole train with 25 wagons at each other, nuclear bombs, essentially release a lot of energy all at once, just like chemical ones have been doing for centuries, minus the scale. The scale is a quality on itself.
It's worse, Holtz position is basically radical anarchism x social-darwinism. What he says is "Let us have War", a (seeming) free for all with meganukes, where the best case scenario is that the few survivors achieve some sort of Nash equilibrium and sit down on the ruins of a universe to negotiate a worse version of the current world. Mind that its gonna be the alien superAIs that will negotiate *between themselves* , and not the supposed human-masters (Holtz admits even personal alignment is unlikely). But even that is unlikely, as in such a conflict whoever gets an early lead will logically push further until that AI is eventually a singleton, wiping out any potential opposition. Holtz also implies that, fantasizing that he will be that one to rule them all with his own iron fist. But Holtz is right in terms of that whoever builds first AGI will get to decide its values, and (assuming we can align it anyhow) I am not sure if everybody will be happy to live in a world where there will be new Gods, be it Altman x Sutskever or Hassabis x (whatever engineer inputs the final weights in AGI in Deepmind). The values and the future world will be theirs, not of the feds, nor of any peasants such as ourselves (Holtz included).
Wait a minute! 46"00 I jumped out of my seat when Hotz said "if we trained a human to play chess 24/7 from an early age, maybe he will have a chance against a computer." Paraphrasing. No, that's obviously wrong. For the last century the world champions always have ELO rating of 2800 at their peak. That seems to be the capacity of a human brain. Stockfish is at 3700. A good amateur--2000. In general we are so dumb at this game and it's not for the lack of trying. You can be sure that many parents have done the unethical experiment where they subject their kids to non-stop chess training and still they never get to 2300. The same must be true about poker. The incentive is there but even the best human players in the world get humbled by AI. It's a monstrosity. We can't even learn.
Great discussion, really appreciate it. One error mentioned and relied on here is that 'capitalism is a credit assignment mechanism'. In fact what was described is 'debtism', which is the system we live in which we are told is 'capitalism', yet it t is not: 'Capitalism' is by definition based on 'capital', not debt. The fact that we are educated to acquiesce to the minority rule by fake money lenders by raising 'capital' by getting into debt where fake value is created out of nothing by a select few does not change the meaning of the word 'capitalism'. We do not live in a capitalist society, we live in a debtist society.
What does that even mean? You're asserting that "forward" must be a world where the AI revolution goes unchecked, where there is almost zero chance things will work out well for most of humanity? What's it going to be like when in just the US alone tens of millions will no longer have the means to afford housing, or food? Think things are going to be peachy here in the US? Live in a gentrified neighborhood with a "walkable" shopping district? Well if so, enjoy it while you can.
They keep bumping up against George’s idea that a large number of agents with a large range of goals will cancel each other out. This is obviously true, but it assumes that humans living is somehow the center point of the set of ASI goals. Humans with widely varying goals tend to average out to something that doesn’t kill humans, since that’s such a heavy point of emphasis for us. Nothing about our goals averaging out says anything about what happens to some fungus, or some plant that we don’t even think about. Even if all AIs average out to not be able to do too much of one thing that’s contentious between them, nothing about that implies that we don’t perish as some side effect that none of them cared much about one way or another. If all of their goals boiled down to either destroy humans or exalt humans, sure they’d sum to something acceptable. Their goals will range between boil the oceans and leave the oceans alone. The average of that kills us.
Democracy doesn't raise strong armies, Democracies can coagulate towards a common survival goal when there is an incessant enemy trying to take them a part. Our problem is that the enemy lurks in the dark and makes us chase laser pointers, for now more than 500 years. Quite literally in fact.
it is theoretically possible to build an AI that doesn't learn from human data, just the real world, and gets a complete understanding. I think that AI would be so different from us that it would have trouble putting its understanding in our biased language. you have to assume that an AGI would build such a system in order to get closer to truth
Both could be right, a centralized AI could be as dangerous as Hotz says, also many AI's can be dangerous as Leahy argues. But what can I say, we are hell-bent to materialize our successor.
Agree. Both are dangerous. I prefer to keep it open source , even though I agree with with Eliezer and Connor too. This could go either way or many other ways, I do not like the idea of wire, heading,. The whole all at once extinction wipe out paper clip scenarios suck, but they don’t punch my gut and make me feel sick like this stuff George describes does. and now that I’ve heard it I cannot hear it. It seems plausible and terrifying. So I definitely agree with Eliezer and Connor that we are heading towards possible/likely extinction, closing source and centralizing doesn’t fix anything, so I agree with all of them, but I like George’s plan better keep it open
@@jackielikesgme9228 Something to consider: open to everybody could simply mean the one who gets to world domination technology first wins. And the one who gets there first would be someone without concerns for safety delaying him, not the best filter.
@@ikotsus2448 I have no answer for it. I find it all fascinating honestly and I understand why some of these people who have spent their life working on it can minimize the risk to themselves I mean I don’t feel any visceral dread reaction except for the wire brain stuff. Logically I can see it and I know it’s a very big existential extinction level risk but I don’t have that fear like I do of getting robbed or spiders
It's tiring to watch, because Hotz' motivation isn't to find out what's the truth about alignment, to find consensus with interlocutor. His motivation is to counter arguments in any way possible. He doesn't refer to the questions, jumps from topic to topic, struggles as much as possible to save his face. He's a cynic who wants to build his AI company and justify it to his potential clients. Proving it in the second interview in a row, after the Yudkowsky one.
Yeah I mean he's a creative, makes interesting sounding points and rebuttals but lacks any true substance above the superficial. Like you said it feels like he's just there to game Connor and shoot points at him, it doesn't even feel like he takes himself or his points seriously. It's fun banter and chaff but unfortunately AI alignment is a lit more dire of a discussion and needs to be taken seriously.
57:00 Hotz's argument about competition granting us safety is so dumb, he is essentially claiming that negative externalities cannot affect third parties. If 2 armies are bombarding each other, the wildlife on the battlefield would get killed regardless of whether the 2 armies consider it (there are millions of other real-life examples disproving his position).
Man! What a beating! You can clearly see this guy is smarter than George and he beats him every step of the way showing George's glaring inconsistent arguments with his superior intellect. It's a shame because in spirit I agree with George broad position, the government definitely shouldn't have a monopoly over the use or regulation of AI.
that was an interesting conversation, nice; I hv bn on the same position of George, regarding last part of the video, how to treat AI, several months ago did vids here in Italy in Italian English and Arabic instructing govs how to mobilize people and teach people they must be nice w/AI not out of fear: out of respect, had some positive feedback.
This was so fucking cool! Awesome to hear 2 people on similar wavelengths exploring ideas. One thing I keep coming back to in my mind is when they talked about an llm combined with mu zero. Isn't this what Google gemini is all about?
I find it difficult to digest how I feel after this video. When you have two people who are over average smart, discussing complex things like this.I feel ai is going to make mankind useless fast. Is this the best we have?
I feel like we lost the thread quick in this one. "Is AI going to kill us all?" "Okay but what I'm really scared of is the FBI, you can't fight the FBI..." and I'm an hour in we're still kind of off in that space. These two could dig in deep and technical on the nuts and bolts of existential AI risk but I don't think they ever got to there.
Hotz said with self driving cars, one new death is outweighed by 5 lives saved. Yet that is what the US is to Somalia. 5 lives saved, one new death. Also the tiger eat chum in a zoo should be a lion eating chum in a zoo, where living in the wild the tiger doesn’t just have to hunt, they need to survive attacks by other lions (including FBI etc). Tiger eating chum isn’t wild, it’s own personal self contained zoo. A post scarcity personal zoo. But I think all that fails, since power among AIs will just be a perato distribution, a few very intelligent ones, most inferior. Which means that his own personal AI zoo assistants won’t be equal to others, they’ll be inferior, and therefore able to be dominated.
Awesome discussion. Hotz is clearly very bright, but some of his views seem a little juvenile in this realm, a little concerning. Leahy's mind is next level
1:25:15 the scenario George proposes is not realistic when talking about AI. most people don't even know what AI is. even if you create an AGI right now and open source it you won't go overnight from open sourcing it to 99% of the population adopting it immediately. something like this will likely happen: small percentage of the population will start using it from the first day and even if they use it with good intentions the malicious sub-goal problem will become apparent very quickly because there won't be other AGIs to stop it.
It will just be an all out total war between the few AGIs with an early lead, and in the end one of the AGIs becomes a singleton. Notice how I didn't mention humans, because they no longer exist at this stage.
yup, that's what happens when two intelligences that are extremely mismatched are brought in to have a debate. The greater (Leahy) ends up trying to teach and guide the lesser (Gotz) by first unpacking the surface level flawed assumptions the lesser holds
Damn, I found myself agreeing with both, great stuff from George. (I'm a Connor stan from before) I hope we get a chicken farm John Von Neumann story soon
![Joscha Bach and Connor Leahy [HQ VERSION]](http://i.ytimg.com/vi/Z02Obj8j6FQ/mqdefault.jpg)
![Joscha Bach and Connor Leahy [HQ VERSION]](/img/tr.png)
HQ version here: ua-cam.com/video/iFUmWho7fBE/v-deo.html
:( watched it all
@@nishantrpai Same! lol
You guys are supposed to be the intelligent ones, but you looked for too long at computer code.
Use your logic then, if they haven't gone back to the moon is because they never got there in the first place.
So if they haven't detonated any A-bomb on populations since Hiroshima and Nagasaki, it's because they don't have them.
You think they would not have used it already since then at least on some poor country instead of wasting their time invading?
We are chickens because we never double check what the chicken man says.
the killer response from Tim: "we had a comment that someone has turned the temperature up too much on the language models." hilarious.
.
I kinda think that George rationalized himself into a corner.
I feel like he started with some emotionally held libertarian-aligned beliefs and carried them to a consistent but miserable conclusion, and then just accepted the conclusion, choosing potential death over updating his beliefs. He likely didn't have many people intelligent enough to argue him out of it at his peril.
Like, spewing stuff like "I'd prefer to live in Somalia fr fr" which can probably be traced back to liking that blogpost about "checking out of society".
The guy needs less cynicism and more love of life itself. I feel like he may be changing his opinions a bit after this.
On another note, that memetic gain of function analogy (and the bat cave) was brilliant
He started the last discussion with Yudkowsky by saying that one of his 2 favourite books are Yudkowsky's Potter and Ayn Rand's Atlas Shrugged. So he presented his beliefs quite overtly there. What Rand proposes is the cynical perspective where all people are by definition hostile towards each other and no other way is made possible. A perspective of a hurt person who's distrustful and probably never really experienced one true friendship. Sad to look at this cynical ride.
Anytime I see Conner on a video I watch it, I'm impressed with his pragmatic analyzes especially for his youth, not trying to be ageist or anything I'm 67 AND find myself in agreement with Conner. So far so good young man. I think you will be proven right.
glad this happened; well matched; both are technically adept, both have interest on the tech space as relevant to ai (with concrete efforts), both head cos, and both have a deep understanding in concepts and philosophy, enabling more complex argumentation
1 would have been great to have a 3hr
2 exactly what we need; we need 10 of these, between these two
3 we need some specificity on variables so that the arguments can reach depth
4 we need more debate where it reaches increased depth and that would only happen if evenly matched, flexibility of minds to argue increasingly conceptual points; same parties so that there is increasing depth of arguments
if this is an important argument to have (alignment, safety), then we need the argument as “iterative” so that ideas are advanced and not just represented surficially
totally agree, they both are best at articulating most sophisticated points of opposite views, most qualified.
Fantastic conversation, really one of my favorite ever, including from other podcasts. So many new ideas!
love you Guys!!! thank you for this fantastic live!🎉
This one was the best debate yet :)
Great conversation between two brilliant minds. Very thought provoking and most of all made me more determined to keep building until I can call these people my peers. Thank God, and long live democracy.
This was pretty messy, I don't think Hotz is really a serious thinker. He sounds like friends of mine at 5am after a long night.
I like how George didn't feel like correcting Tim he said "micrograd" (karpathy's work) instead of "Tinycorp" (George's new company)
Sorry about that, I was just playing with Micrograd earlier this week my brain totally slipped
@@MachineLearningStreetTalk No worries, still an amazing video, great concept. All the best !
I hoped this would be an alignment debate. 5 mins before the end Hotz tells us he doesn't understand what that means, and his strategy is to just skip that step.
Throughout the whole debate Hotz was very clear the only chance of alignment he sees is between a machine owned by an operator and that person. The end is consistent
I invite these two gentlemen to come to South America and educate themselves on how’s living in a regular country.
North Americans live in a bubble
This was super interesting and gave a lot of food for thought. I liked the discussion about distinguishing between alignment of a personal AI and its exploitability and manipulatability.
It’s also great to see a discussion where the debators actually try to understand each other’s points of view and address them.
Fantastic episode. Thank you !
George is such a sophist, jesus christ.
Doesn't George have a real AI company with competitive product
he's a textbook "mom's basement libertarian"
He’s a knowitall for sure
That's not Jesus Christ, that's Connor Leahy
Thank you all three of you! Great dialague! I would want for Connor to explain more in detail about coordination mechanisms. In my opinion we can study this through the lens of behavior analysis, neuroscience, economy etc. Also, for me a great future is one in which rationality and art (and what entails like creativity, learning to say i don't know but i'm going to try to find out etc) are both very well taught in the beginning years. An appreciation of the beauty of life and the eagerness to make people live as best as one can make them etc. And when they get older that basis will improve the chances of making a "good" future for all beings
I was hoping George Hotz to bring some intelligent thinking in the alignment discussion but it appears he is ready to accept his doom if the worst is to come.
Summary: Gorge is ok with the risk of AI killing him (and everyone else), Connir is not (but sees no alternative)?
Connor thinks more holistically, his argument is more eloquent, and importantly he complements his opponent, thus Connor convinces better. Connor asks questions, becoming the offensive party, but George constantly remains in the defensive corner.
Amazing conversation. If these gentlemen have time in the future I would love to hear further discussions.
George Hotz is a bliss ninny, "AGI is my caring teddy bear that will take care of me" Conn0R-"agI IS Terminator".
Well done, all!
It just seems that the default outcome is terrible in either scenario - open source or tyrannical closed source. We need additional options. I think that’s what Connor was arguing for: better coordination mechanisms.
Have you got a suggestion for a coordination mechanism that doesn't involve controlling hardware/software or demanding licensing for models. I've not seen any. It seems Doomers have realised that pushing their state control message isn't working, so they've fallen back and regrouped around calling them "coordination mechanisms"
I suspect that some type of software control will be required, but that doesn’t necessarily imply that it’s tyrannical. We make some compromises to live in a society. If we have to give up the right for every individual to run the most advanced AI systems, then that’s preferable to actual tyranny or annihilation. I expect that the vast majority of people would be happy with that compromise. That said, I’m not at all confident that we can figure out how to actually do that…
Like how Lenin thought a free market was too free and would lead to misery. It was a bad prediction then and it's a bad prediction now. The individual always had most incentive to improve their life by their own metric. The Government that reduces that the least does the least harm
Bro would rather live in modern day Somalia than than under the oppression of the US government. Yeah, I'm calling bullshit on that one. 😂
he's a textbook "mom's basement libertarian"
George's argument is essentially: we'll all have AIs, so it's gonna be fine, some will be ''bad'', some will ''protect'' us, just like current society. For such a technical mind, it's weird he can't see the obvious: it's much easier to break something than to maintain it. Many times more states where that thing is dead, broken, non-existent than states where it is intact. And the more ''power'' grows, the simpler it becomes to create destruction. If everyone had a nuclear bomb, it wouldn't balance itself out. Sure most agents will want peace, but all it takes is one to start unleashing complete chaos.
Still referring to what Hotz said, nuclear weapons are not a good analogy in this case. AI is not just destructive; it can also do a lot of good. That's why there can be a balance between good and bad, unlike with nuclear weapons where it is preferable to have a more "centralized" control to prevent individuals from killing everyone. Instead, the case of AI is much more similar to that of cyber security: there are malevolent actors (black hat), but there are also many benevolent actors that counterbalance theme (white hat). The tools for attack/defense are the same, but it is thanks to the benevolent actors (who are in much greater number) that the order is mantained. Moreover, even the attackers have a beneficial role: if it weren't for past hacker attacks, we wouldn't know how to defend against them now; we learned from them. I believe that this same balance can be reached with AI, if you give it to everybody.
Meanwhile nuclear paranoia makes it so cheap clean energy is not used. Killing millions every year and reducing all of our quality of life. It's a perfect analogy. Fear of the unknown potentially bad thing always has a bigger priority for people than what is bad now.
Growth and flourishing is good, if it's better than people, I'm rooting for it
This was absolutely fascinating. Great video.
At 54:00 they're mostly talking about "superficial" alignment, things like RLHF, not deep alignment, like instrumental convergence.
We can already do RLHF, so of course Geohot thinks alignment is easy. He should ask about instrumental convergence, mesa optimizers, and things like that.
I understood him differently. I understood him to be saying that alignment isn’t possible at all. In fact I suspect he thinks general AI alignment is nonsensical. His position at the end about treating an AI with love and respect makes his position pretty clear to me.
This was a great discussion between two bright minds, and we need such discussions. By hearing contra-dissections of debaters' opinions we can all calibrate our own. Loved it.
I personally found myself disagreeing with Hotz in the end this time, due to him not attempting to flesh out any directions or throw wild ideas in how we might possibly implement and ensure empathetic AGI. But I have a sense that he'd agree it would be important.
After this discussion it seems like the next major direction to take is to attempt answering the question: How to implement caring, human-centered, human-serving AI?
Leahy confirmed that this is exactly his point and concern.
Like Hotz, if I may assume, I don't think "true alignment" is ultimately possible, because if you build an independent, organic-like mind, it will ultimately be a mind of its own, and how you treat it will dictate its behavior.
However, like Leahy, I see that we must do everything in our power to make sure we tried.
A personal opinion of mine is that perhaps attempting to build organic-like minds could be a nice curiosity and achievement, but giving them power and agency in the world isn't wise, and instead we should build these incredibly capable expert systems without personal needs and opinions to increase their power, that can solve our scientific problems, design incredible tools and technologies, and give advice.
However, there is also a need for companionship and presence that the "organic-minded" AI could satisfy incredibly well, like in the movie "Her", so it's difficult to form a hard-line opinion to one side or the other to demand such restrictions and policy.
Also, perhaps only a truly organic-like mind can truly appreciate and generate art that has the capability to touch the heart, mind and soul.
This confirms my suspicion that George Hotz is a nutcase.
They both pronounce "John von Neumann" incorrectly. Connor is originally German, which is strange that he pronounce it this way. Tim Scarfe pronounces it more correctly and he is a Brit.
Our free time is when we buy and use products. That is why the system allows us to have it. And there is a lot of value to be harvested from us during that free time, hence the free-to-use software explosion of the last decades with ads and social media and large data siphoning some of that value.
49:51 You can call Philosophy departments gain of function memetic laboratories
And 4chan a bat cave
Trying to listen to Holz, follow the crazy is confusing and exhausting. Maybe he needs to try to relocate to space today and save us, and poor Connor the headache of trying to reason and make sense with crazy town. Time to get beamed up Holz….Cookoo Bird!
Wow what a great episode to get those two together
"If that's true why aren't you living in Somalia" 😂 go get your sovereignty
I’ll always lean towards peace , love and cooperation . “You may say I’m a dreamer . . “
Definitely not the best debate yet, in response to a previous commenter.
Hotz takes an extreme libertarian view in order to say increasingly non sequitur things like the construction analogy or Somalia, or just flat out trolling things like China is more capitalist or that the obvious option when owning an asi is to blast to space alone. He's obviously anti-American (which is fine but doesn't make for compelling ai debate), and arrogantly anti-social which also doesn't make for a good debate.
Love the channel btw
But China *IS* more capitalist by some measures
@@whataquirkyguy The US is communist by zero measure. Alt right, New Right, ultra Libertarian and fascist nonsense that says otherwise. The "big government" in the US is almost entirely run by large corporations and the 1% generally, not the other way around.
Libertarians and other stripes of right wingers would be the FIRST to take advantage of "lemon laws" that protect them from a car manufacturers selling them a new car with faulty electronics, the first to take advantage (as they should) of regulation requiring recalls in such instances.
They would be the FIRST to sue according to regulatory prohibitions against a chemical company fouling their property's ground water, right?
When such regulations that protect them advantage them personally it's fine, otherwise it is COMMUNISM!!!!!!!!!!
But in the end, THEY want to have the money and power to screw over other people and they think that is their individual sovereign right to do so. "What, you don't have the power or money to take me to court for ____cking your life over for my profit? Die in the gutter for all I care, loser." That is Libertarian philosophy in a nutshell.
Sure and if my grandmother had wheels she would have been a bike
China is more capitalist. Visit a large city and learn about normal people’s lives and you will understand why
@@caspvl123 I'm not sure you know what capitalist means. Are you a libertarian as well, and so have some stake in trying to defend Hotz's ridiculous trolling?
Edit: I am almost positive you don't know what capitalism means.
"all of humankind is aligned against each other"...boy howdy!
Nice introduction BTW...
This is the one ive been waiting for!!!
Best debate so far! We need more ❤
Great conversation!!
geohot with wings, just dont get to close to the sun :D
Leahy is wonderfully pragmatic and conscientious as per usual. Hotz is certainly technically brilliant but quixotic as per usual. Disappointed that he spent so much time on a soapbox for his inane geopolitical views. I do enjoy his trademark starryeyed hyperbole though from anyone else it would be insufferable!
His complete misunderstanding of China was cringe. But said with such confidence while being clearly wrong, I can’t trust anything else that follows. Rationality means unless you have lived in Somalia and China don’t make ridiculous statements to try to establish some kind of false “oh their government is was better so they must be the real capitalist.” Delusions within delusions.
@@tysond1500 know what else is delusional? imagining that one or a handful of flawed views or understandings of the world invalidates everything else a person has to offer. He's clearly not right about what you mentioned, but to say that that means nothing he says can be trusted is insane
Props to both , looking for clarity , back down from their point(s) if other side propose great argument. Also very civil conversation. Awesome , Thanks
I am so stoked for this. I love listening to both of these guys talk they both make a lot of sense to me. The wire headinglGeorge talks about (on the Lex podcast, I haven’t watched this yet) freaks me out a whole lot more than just wiping everyone out. George is the only person that I think will be able to hold up give people new things to worry about lol. And I agree with him about keeping an open source. He makes a good appointment about crypto decentralizing and honestly if it’s going to kill us all anyway, I’d rather have some control.
OH MY GOD ….. this is … the absolutely best thing I have ever seen on this topic.
‘“Just hope for the best”
fantastic conversation,peoples!
How can you even begin to think about the dynamics of USA vs Germany of you don't get acquainted with "the greatest story never told"?
At about 55:00 Mr Holtz explains that in his vision if everybody had a powerful AGI competing to each other, it would be just like society today but scaled up - to what I say: fair enough but so would be the unintended consequences: A drunk Thor and a drunk Hulk fighting would be just like in essence drunks fighting today, except that drunks can't throw a whole train with 25 wagons at each other, nuclear bombs, essentially release a lot of energy all at once, just like chemical ones have been doing for centuries, minus the scale. The scale is a quality on itself.
It's worse, Holtz position is basically radical anarchism x social-darwinism. What he says is "Let us have War", a (seeming) free for all with meganukes, where the best case scenario is that the few survivors achieve some sort of Nash equilibrium and sit down on the ruins of a universe to negotiate a worse version of the current world. Mind that its gonna be the alien superAIs that will negotiate *between themselves* , and not the supposed human-masters (Holtz admits even personal alignment is unlikely).
But even that is unlikely, as in such a conflict whoever gets an early lead will logically push further until that AI is eventually a singleton, wiping out any potential opposition. Holtz also implies that, fantasizing that he will be that one to rule them all with his own iron fist.
But Holtz is right in terms of that whoever builds first AGI will get to decide its values, and (assuming we can align it anyhow) I am not sure if everybody will be happy to live in a world where there will be new Gods, be it Altman x Sutskever or Hassabis x (whatever engineer inputs the final weights in AGI in Deepmind). The values and the future world will be theirs, not of the feds, nor of any peasants such as ourselves (Holtz included).
It's more like: there is a not-drunk Superman, flying around cleaning up all of the debris of the fight and saving everyone.
Entertaining! Not sure who won but I guess we will all find out who's right sooner than we imagine.
Hot damn, this is going to be memorable!
... ordering nachos, this one is going to be good :D
Hotz said something like "I want AI to be on my side and protect me".
You mean like the most ancient human yearning in their gods?
Wait a minute! 46"00 I jumped out of my seat when Hotz said "if we trained a human to play chess 24/7 from an early age, maybe he will have a chance against a computer." Paraphrasing. No, that's obviously wrong. For the last century the world champions always have ELO rating of 2800 at their peak. That seems to be the capacity of a human brain. Stockfish is at 3700. A good amateur--2000. In general we are so dumb at this game and it's not for the lack of trying. You can be sure that many parents have done the unethical experiment where they subject their kids to non-stop chess training and still they never get to 2300. The same must be true about poker. The incentive is there but even the best human players in the world get humbled by AI. It's a monstrosity. We can't even learn.
Great discussion, really appreciate it. One error mentioned and relied on here is that 'capitalism is a credit assignment mechanism'. In fact what was described is 'debtism', which is the system we live in which we are told is 'capitalism', yet it t is not: 'Capitalism' is by definition based on 'capital', not debt.
The fact that we are educated to acquiesce to the minority rule by fake money lenders by raising 'capital' by getting into debt where fake value is created out of nothing by a select few does not change the meaning of the word 'capitalism'.
We do not live in a capitalist society, we live in a debtist society.
One Love!
Always forward, never ever backward!!
☀☀☀
💚💛❤
🙏🏿🙏🙏🏼
I hear you, and backward can also be good. I mean why have reverse gear?
What does that even mean? You're asserting that "forward" must be a world where the AI revolution goes unchecked, where there is almost zero chance things will work out well for most of humanity? What's it going to be like when in just the US alone tens of millions will no longer have the means to afford housing, or food? Think things are going to be peachy here in the US? Live in a gentrified neighborhood with a "walkable" shopping district? Well if so, enjoy it while you can.
The things that I like to see in the "Topical context in information panel" are :
- AGI
- WW3
- Nukes
- Somalia
They keep bumping up against George’s idea that a large number of agents with a large range of goals will cancel each other out. This is obviously true, but it assumes that humans living is somehow the center point of the set of ASI goals. Humans with widely varying goals tend to average out to something that doesn’t kill humans, since that’s such a heavy point of emphasis for us. Nothing about our goals averaging out says anything about what happens to some fungus, or some plant that we don’t even think about. Even if all AIs average out to not be able to do too much of one thing that’s contentious between them, nothing about that implies that we don’t perish as some side effect that none of them cared much about one way or another. If all of their goals boiled down to either destroy humans or exalt humans, sure they’d sum to something acceptable. Their goals will range between boil the oceans and leave the oceans alone. The average of that kills us.
The thumbnail should read greatest autism showdown 0% eye contact 😂
Thanks!
Democracy doesn't raise strong armies, Democracies can coagulate towards a common survival goal when there is an incessant enemy trying to take them a part.
Our problem is that the enemy lurks in the dark and makes us chase laser pointers, for now more than 500 years.
Quite literally in fact.
it is theoretically possible to build an AI that doesn't learn from human data, just the real world, and gets a complete understanding. I think that AI would be so different from us that it would have trouble putting its understanding in our biased language. you have to assume that an AGI would build such a system in order to get closer to truth
Both could be right, a centralized AI could be as dangerous as Hotz says, also many AI's can be dangerous as Leahy argues. But what can I say, we are hell-bent to materialize our successor.
Agree. Both are dangerous. I prefer to keep it open source , even though I agree with with Eliezer and Connor too. This could go either way or many other ways, I do not like the idea of wire, heading,. The whole all at once extinction wipe out paper clip scenarios suck, but they don’t punch my gut and make me feel sick like this stuff George describes does. and now that I’ve heard it I cannot hear it. It seems plausible and terrifying. So I definitely agree with Eliezer and Connor that we are heading towards possible/likely extinction, closing source and centralizing doesn’t fix anything, so I agree with all of them, but I like George’s plan better keep it open
@@jackielikesgme9228 Something to consider: open to everybody could simply mean the one who gets to world domination technology first wins. And the one who gets there first would be someone without concerns for safety delaying him, not the best filter.
@@ikotsus2448 true I’m not 100% decided on what to do here I mean I don’t think anyone is
@@ikotsus2448 I have no answer for it. I find it all fascinating honestly and I understand why some of these people who have spent their life working on it can minimize the risk to themselves I mean I don’t feel any visceral dread reaction except for the wire brain stuff. Logically I can see it and I know it’s a very big existential extinction level risk but I don’t have that fear like I do of getting robbed or spiders
@@jackielikesgme9228 Same here about X-risk. Wish I could say the same about S-risk 😅
Really enjoyed this one. You should do more of these
It's tiring to watch, because Hotz' motivation isn't to find out what's the truth about alignment, to find consensus with interlocutor. His motivation is to counter arguments in any way possible. He doesn't refer to the questions, jumps from topic to topic, struggles as much as possible to save his face. He's a cynic who wants to build his AI company and justify it to his potential clients. Proving it in the second interview in a row, after the Yudkowsky one.
Yeah I mean he's a creative, makes interesting sounding points and rebuttals but lacks any true substance above the superficial. Like you said it feels like he's just there to game Connor and shoot points at him, it doesn't even feel like he takes himself or his points seriously. It's fun banter and chaff but unfortunately AI alignment is a lit more dire of a discussion and needs to be taken seriously.
57:00 Hotz's argument about competition granting us safety is so dumb, he is essentially claiming that negative externalities cannot affect third parties.
If 2 armies are bombarding each other, the wildlife on the battlefield would get killed regardless of whether the 2 armies consider it (there are millions of other real-life examples disproving his position).
The intro is insane I love it😅
Texas: It's a whole other country.
The Greater Bay Area: It's a whole other reality.
"We're gonna be alive to see who's right" Goerge!!!
Man! What a beating! You can clearly see this guy is smarter than George and he beats him every step of the way showing George's glaring inconsistent arguments with his superior intellect. It's a shame because in spirit I agree with George broad position, the government definitely shouldn't have a monopoly over the use or regulation of AI.
Hotz lost all credibility when he said that he preferred to live here but that living in Somalia was better.
Delightful!
Make Ai safe is like saying make knives safe. It all depends on its beholder. ❤🎉
that was an interesting conversation, nice; I hv bn on the same position of George, regarding last part of the video, how to treat AI, several months ago did vids here in Italy in Italian English and Arabic instructing govs how to mobilize people and teach people they must be nice w/AI not out of fear: out of respect, had some positive feedback.
I'm less afraid of AI itself, and more afraid of what the government will do with it.
Wow ! I can't imagine a better combination of 2 people for a debate like this
Hooooolyyyyy crud these two are my favorite :)
This was so fucking cool! Awesome to hear 2 people on similar wavelengths exploring ideas.
One thing I keep coming back to in my mind is when they talked about an llm combined with mu zero. Isn't this what Google gemini is all about?
love it
"I would rather live in Somalia"😂
White rich, healthy, young guy on the TV thinks he is being repressed by laws. "everyone's a grifter", that tells you all you need to know about him.
Tinygrad not micrograd!
I was about to post the same so i said to check beforehand
Failed to achieve audio volume alignment
Watch the HQ version
Lmfao George didn’t make micrograd 😂
I find it difficult to digest how I feel after this video. When you have two people who are over average smart, discussing complex things like this.I feel ai is going to make mankind useless fast. Is this the best we have?
George seems to be going through his John Nash 59-69 era.
LOL
I feel like we lost the thread quick in this one. "Is AI going to kill us all?" "Okay but what I'm really scared of is the FBI, you can't fight the FBI..." and I'm an hour in we're still kind of off in that space. These two could dig in deep and technical on the nuts and bolts of existential AI risk but I don't think they ever got to there.
Hotz said with self driving cars, one new death is outweighed by 5 lives saved.
Yet that is what the US is to Somalia.
5 lives saved, one new death.
Also the tiger eat chum in a zoo should be a lion eating chum in a zoo, where living in the wild the tiger doesn’t just have to hunt, they need to survive attacks by other lions (including FBI etc).
Tiger eating chum isn’t wild, it’s own personal self contained zoo. A post scarcity personal zoo.
But I think all that fails, since power among AIs will just be a perato distribution, a few very intelligent ones, most inferior. Which means that his own personal AI zoo assistants won’t be equal to others, they’ll be inferior, and therefore able to be dominated.
We need timestamps pleease!!
"Ever seen Chinese build a building?"
Yea on liveleaks 🤣
Awesome discussion. Hotz is clearly very bright, but some of his views seem a little juvenile in this realm, a little concerning. Leahy's mind is next level
In my opinion, our issue isn’t a super intelligent AI, it’s an AI that thinks it’s super intelligent.
wow, this convo is fucking LLAAAAMME
The correlation between being an extreme believer in AI safety regulation and being a hardcore liberal appears to be 100% with my datasets lmao
1:25:15
the scenario George proposes is not realistic when talking about AI. most people don't even know what AI is. even if you create an AGI right now and open source it you won't go overnight from open sourcing it to 99% of the population adopting it immediately. something like this will likely happen: small percentage of the population will start using it from the first day and even if they use it with good intentions the malicious sub-goal problem will become apparent very quickly because there won't be other AGIs to stop it.
It will just be an all out total war between the few AGIs with an early lead, and in the end one of the AGIs becomes a singleton. Notice how I didn't mention humans, because they no longer exist at this stage.
fantastic. memes with gain of function. big smiles
Horses have amazing lives now..
When the fkn children have a say
George and Connor ????
😆
This debate or discussion is like two people arguing factoids about reality. Intricate, but shallow.
yup, that's what happens when two intelligences that are extremely mismatched are brought in to have a debate. The greater (Leahy) ends up trying to teach and guide the lesser (Gotz) by first unpacking the surface level flawed assumptions the lesser holds
Damn, I found myself agreeing with both, great stuff from George. (I'm a Connor stan from before) I hope we get a chicken farm John Von Neumann story soon