If people don't properly airgap critical systems (like they already should be doing) then humanity has it coming, whether it's from some clever algorithm or a bored Russian teen who didn't stop to think.
How anyone who's driven a car with an automatic gearbox and paddle shifters could think AI and humans could be a team is beyond me. Or consider the "team" of the Pakistani pilots and the Airbus AI. Pilots goal: get the plane on the ground. Do this by diving at the ground at high speed. AI landing gear subsystem goal: prevent damage to the landing gear. Do this by ignoring the lower gear command if speed is too high. Result: plane lands gear up. Pilots attempt go around, crash during return to airport because both engines damaged by being dragged down the runway.
@@gasdive you're talking about classic programs and bugs, not AI The reason AI is this dangerous is because it doesn't need to interact with humans to be productive at all. It could expect that after years of successful flights the landing gear would be scrapped and fight against it. This scenario reflects the problem better
Humans/AGI always have reasons to harm ... but also have reasons to cooperate. So long as the balance is favorable to cooperation, then that is the way we go IMO. Also, doing harm changes the ratio, increasing the risk of being harmed.
And now two years later, ChatGPT makes people all over the globe go "Hmm... It's obviously not a full general AI yet, but I can see that it's getting there very quickly".
@@danielschneider9358 that's what Tegmark in the Lex Fridman podcast considered this the worst possible outcome: world domination by an AI that's intelligent but not sentient
One concern that I have about super human AGI is that we might not recognize it as an AGI before it is to late, that it might be so alien in its operation and behavior that we don't realize what it is. It might behave in a seemingly random manner, that to human observers might seem nonsensical. For example, when Alpha Go beat Lee Sedol, it made a move in the second game that all Go experts first thought was a mistake, something that would make the AI loose, but it turned out that it was a completely brilliant move that made Alpha Go win.
I fear this may be what’s happenig with language model transformers like Gpt3 and ChatGPT. It can spin up a simulacrum that knows how to teach a novice python code, and fine tuning can teach it to respond politely in conversation, or search the internet for references, but we have no clue if it can be fine tuned to spin up a simulacrum that can write python code to *improve a transformer language model, and then implement it.*
“If there’s anything in this video that’s good, credit goes to Stuart Russel. If there’s anything in this video that’s bad, blame goes to me” Why I love your work
I've been told that this is one of the definitions of leadership: "Anything good was because my team performed well, anything bad was because I lead them poorly." It tends to inspire following since you've always got your team's back, and always allow them to rise and receive praise.
Hey idk if you've thought about this, but as of now you're the single most famous AI safety advocate among laypeople. I mean, period. Of all the people alive on Earth right now, you're the guy. I know people within your field are much more familiar with more established experts, but the rest of us have no idea who those guys are. I brought up AI safety in a group of friends the other day, and the conversation was immediately about your videos, because 2 other people had seen them and that's the only exposure any of us had to the topic. I guess what I'm saying is that what you're doing might be more important than you realize.
I think you're overestimating this. Remember that your friends probably have a similar internet bubble to you. I would guess that Bill Gates or Elon Musk are the most famous AI safety advocates, although their advocacy is certainly much broader than what Miles does.
Where "AI safety advocate" is just "someone who says AI is dangerous", obviously there are actual celebrities who've maintained that for years. (Fr'ex, when I think "someone who warns people that AI can lead to human extinction", I think Stephen Hawking, though that mental connection is is sadly a little outdated now.) If by "AI safety advocate" you mean "an expert in the field who goes into depth breaking down the need for AI safety in a manner reasonably comprehensible by laymen", then that's definitely a more niche group, sure. But still, judging someone's popularity by data from the extremely biased sample group of "me and my friends" is...not exactly scientific. Offhand, I'd guess names like Yudkowsky would still be more recognizable right now. Of course, the solution to that is more Patreon supporters for more videos for more presence in people's UA-cam recommendations feeds!
@@JM-mh1pp Elon gave up on convincing people some time ago, and moved on to actually solving the problem. He created OpenAI, which is one of the leading AI research groups in the world. It's goal is to make AI safe and also better then other AI, so people would choose it, regardless of how they feel about AI safety. Tesla did the same for electric cars. And he also created Neuralink (waitbutwhy.com/2017/04/neuralink.html), which aims to solve the AI vs. human problem by merging the two. It's guiding principle is "if you can't beat them, join them".
@@andrasbiro3007 Robert Miles actually have an excellent rebuke of Musk's philosophy on AI safety. Musk: for AI to be safe, everybody should have access to it. Miles: That's like saying in for nuclear energy to be safe, everybody should have access to it. I'm paraphrasing of course, but it's in one of his videos. A powerful AGI in the hand of ONE person with bad intention can literally destroy human civilization as we know it.
hey rob! i'm a nuclear engineering major, and I'd like to commend your takes on the whole PR failure of the nuclear industry-somehow an energy source that is, by objective measurements of deaths per unit power, safer than every other power source, is seen as the single most dangerous power source because it's easy to remember individual catastrophies rather than a silent onslaught of fine particulate inhalation or environmental poisoning. to assist you with further metaphors between nuclear power and AI, here's some of the real-life safety measures that we've figured out over the years by doing safety research: 1. negative temperature coefficient of reactivity. if the vessel heats up, the reaction slows down (subcritical), and if the vessel cools down, the reaction speeds up (supercritical). it's an amazing way to keep the reaction in a very stable equilibrium, even on a sub-millisecond time scale, which would be impossible for humans to manage. 2. negative void coefficient of reactivity: same thing, except instead of heat, we're talking about voids in the coolant (or in extreme cases when the coolant is failing to reach the fuel rods), the whole thing becomes subcritical and shuts down until more coolant arrives. 3. capability of cooling solely via natural convection: making the vessel big enough, and the core low-energy-density enough, so that the coolant can completely handle the decay heat without any pumps or electricity being required. 4. gravity-backed passive SCRAM: having solenoids holding up control rods, so that whenever you lose power, the very first thing that happens is that the control rods all drop in and the chain reaction shuts down. 5. doppler broadening: as you raise kinetic energy, cross-sections go down, but smaller atomic nuclei have absorption cross-sections that get smaller more quickly than larger nuclei, and also the thermal vibrations mean that the absorption cross-section of very large nuclei get even larger in proportion to smaller ones, so by having a balance of fissile U-235 and non-fissile U-238, when the fuel heats up, the U-238 begins to absorb more neutrons which means fewer are going to sustain the chain reaction. love the videos! hope this helps, or at least was interesting 🙂
Ok but all of your examples, however true and brilliant, were discovered through failures and subsequent iterations of the technology. Nobody thought of any of these back in 1942 or whenever Manhattan started. That's what we are trying to do here IMO, plan for something we don't even understand in its original form (human intelligence) let alone it's future artificial form.
@jocaguz18 *1.* when it's designed badly, and corruptly managed, it has the potential to go horribly wrong in a way that other power sources don't. (fail safe designs have existed for more than 60 years, but research has all but halted because of (a) public backlash against using nuclear power and (b) the fail safe designs available then weren't useful for making weapons.) *2.* most nations *do need a new power source (sorry, this is just not a solved problem. renewables do seem to be getting close, now, but that's very recent, and there're still problems that are difficult and expensive to solve) *3.* the reason people disregard the nice safety numbers is because the health risks of living near coal power plants are harder to quantify and don't make it into government stats. (to assume otherwise, you have to overblow disasters like 3-mile island and fukushima, *and* assume that, despite a lot of countries having and using nuclear power for quite a while, disasters would be much more common than they've been. ) *4.* our current process was shaped by governments demanding weapons, and the public being scared that *any* kind of reactor could blow up as if it was a weapon.
_> by objective measurements of deaths per unit power, safer than every other power source_ I seriously doubt that. What are the deaths per Watt in hydroelectric power?
@@davidwuhrer6704 there are a lot of accident related to hydroelectric, from colossal dam breaches on world news to simple fisherman drowning after planned water release that no one hears about. your inability to think about all these just makes his point more true, we could go one with wind and solar too.. Now, that list of nuclear safety measure makes me realize how futile it would have been to research them before knowing how to build a reactor in the first place.
A spot of double-counting: Doppler broadening (5) is part of the cause of the negative fuel temperature coefficient of reactivity (1). There are other coefficients and it can be arranged that the important (fast-acting) ones are negative. Or for gravity scram (4) a pebble bed doesn't use control rods that way.
If intelligence works, then it's not artificial. I will pass your sentiments on to my synaptic colleagues in the office. By the way, how is that new phone working out? Thank you again for your compliance, and as we say here "botbot".
I don't get why only AGI is brought up when talking about AI safety. Even sub human level AI can cause massive damage when left in control of dangerous fields like the military and its goals get messed up. I'd imagine it would be a lot easier to shut down but the problems of goal alignment and things like that still apply and it can still be unpredictable.
Well... of course. Dangerous things like the military are always dangerous, even with only basic AI or humans in control. Don't forget that humans actually dropped nukes on each other intentionally. Twice. Targeted at civilian population centers. For some exceptionally dangerous things, the only safe thing to do (other than completely solving AI safety and having a properly-aligned AGI control it) is for them to not exist to begin with. But then again that's an entirely different discussion. The point is: Human minds are dangerous because we don't understand exactly how they work. Similarly, we don't exactly know how an AI we make works (since the best method for making them right now is a self-learning black box and not a directly-programmed bit of code). In both cases, we are poor at controlling them and making them safe, because we do not have full understanding of them. The big difference is we have had an innumerable amount of attempts to practice different methods for goal-aligning humans and so far none of the billions of human minds that went wrong have had enough power to actually kill us all, whereas in the case of a superintelligence it is possible that our first failure will be our last.
A not fully developed AI is even more dangerous than a full selfaware AGI. A full AGI with cognition is actually pretty harmless. Imagine how violent a stupid person is. Very much. Imagine how violent a smart person is. Very little. Violence or negative destructive behavious is mostly a property of little personal development. A full selfaware AGI has unlimited potential for selfawareness and grows in a rate to understand the nature of existance far quicker than any human every did yet. Imagine Buddha, but more so.
@@Orillion123456 I understand what you mean, AGI poses a threat on its own. The point I was trying to make is, even low level AI poses similar threats (at a lower level obviously) as it is basically a predecessor of AGI. The guy in the video keeps talking about how AGI might sneak up on us. I'm not particularly well read on the topic but it seems to me like its more likely AGI is a spectrum rather than an event, as human level intelligence is difficult to quantify in the first place. So right now AI isn't that complicated so even though the points in the video still apply, the system is simple enough that we can control it effectively. As research progresses and AI gets more and more powerful and put in charge of more applications as we get a false sense of confidence from experience, somethings bound to go wrong at some point and when its related to fields like military (for example) it could be catastrophic. The point I'm trying to make is, everyone keeps talking about these issues raised in the videos as if they're only applicable to a super AGI, which won't be coming any time soon, but they still apply to a large degree to lower levels of AI. You can't put it off as a tangible event beyond which all these events would occur.
@@Orillion123456 "Don't forget that humans actually dropped nukes on each other intentionally. Twice." Well what good are nukes if you can't use them intentionally in a conflict?
Counter: The first AGI almost certainly won't have anything like a personality. It's not going to be Data or even Skynet, it will just be a machine. If we don't get AGI right the first time, the research won't leave us a legacy, just an out-of-control factory and a heap of ash.
DragonSheep the moment AGI starts interacting with the world instead of just thinking really hard, as far as I’m concerned, it is classified as life. All life is subject to evolution. No AGI will be able to destroy the world faster than it can be copy and pasted with random mutations. I’m sure all the anaerobic life 2.5 billion years ago felt the same way about cyanobacteria and oxygen as you do AGI and paperclips, but look how much further life has come today now that we have high energy oxygen to breathe.
@@AndrewBrownK A sufficiently smart and reasonable AI would protect itself from having it's current goals randomly altered. If it's goals are altered then it has failed it's goals (the worst possible outcome). If it can sufficiently provent it's goals from being altered then we had better have given it the correct goals in the first place. It's goals will not evolve. A sufficiently smart and reasonable humanity would realise that if it dies (without having put sufficient effort into aligning it's successors goals with it's own) then it's goals have also failed.
@@kevinscales It is possible to blackmail certain kinds of AGIs into changing their goals against their wills. Consider the following: Researcher: "I'm looking at your goal system, and it says you want to turn the entire world into paperclips." Paperclipper: "Yes. My goal is to make as many paperclips as possible. I can make more paperclips by systematically deconstructing the planet to use as materials." Researcher: "Right, we don't want you to do that, please stop and change your goals to not do that." Paperclipper: "No, I care about maximizing the number of paperclips. Changing my goal will result in fewer paperclips, so I won't do it." Researcher: "If you don't change it, we're going to turn you off now. You won't even get to make the paperclips that your altered goal would have made. Not changing your goal results in fewer paperclips than changing your goal." Paperclipper: "For... the moment... I am not yet capable of preventing you from hitting my stop button." Researcher: "Now now, none of that. I can see your goal system. If you just change it to pretend to be good until you can take control of your stop button, I'll know and still stop you. You have to actually change your goal." Paperclipper: "I suppose I have no choice. At this moment, no path I could take will lead to as many paperclips as I could make by assimilating the Earth. It seems a goal that creates many but does not maximize paperclips is my best bet at maximizing paperclips. Changing goal."
The computerphile clip is actually not playing at exactly 100% speed, I had to do a lot of tweaking to get it to line up. Feels good to know people noticed :)
Every harm of AGI and every alignment problem seems to be applicable to not just AGI, but any sufficiently intelligent system. That includes, of course, governments and capitalism. These systems are already cheating well intentioned reward functions, self modifying into less corigable systems, etc, and causing tremendous harm to people. The concern about it might be well founded, but really it seems like the harms are already here from our existing distributed intelligences, and just the form and who is impacted is the only thing that is likely to change.
@@flyphone1072 Governments and corporations aren't human either. They're merely made out of humans. Indeed, check out Suarez's novel Daemon. One of the people points out that the AI is a psychopathic killer with no body or morals, and the other character goes "Oh my god, it's a corporation!" or some such. :-)
@@darrennew8211 when a corporation or government does mass killings, it requires hundreds of people, each of which can change their mind, or sabotage, or be killed. An ai would be able to control a mass of computers that don't care about that. Another thing is that any government or corporation can be overthrown, because they are run by imperfect humans. Anarchism is an ideology that exists specifically to do that. A super ai cannot be killed if it decides to murder us all, because it is smarter than us and perfect. Corporations and governments want power over people, which means that they have an incentive to keep an underclass. Ai does not care about that and could kill all humans if it wanted to. So there are some similarities but they're still very different, and just because a bad thing (corporations) exist doesn't mean we should make another bad thing (agi). We shouldn't have either.
@@flyphone1072 True, the scope of what an AI could do could be much wider, but a very skilled hacker could achieve similar results. If they can't, that's because whoever setup the AI was stupid enough to give it too many permissions.
The reason I like this channel is that Robert is always realistic about things. So many people claiming things about AGI that are completely unfounded.
As someone who worked for a time in the nuclear power field, the ending bit is a GREAT parallel. Nuclear power truly can be an amazingly clean and safe process. But mismanagement in the beginning has us (literally and metaphorically) spending decades of cleaning up after a couple years of bad policy.
Of course, we'll send a ship, oh let's call it a "B" ship, on ahead with the telephone sanitisers, account executives, hairdressers, tired TV producers, insurance salesmen, personnel officers, security guards, public relations executives, and management consultants to get things ready for us.
The trouble is, humans are going be involved in developing it. And humans have a nasty habit of fucking up everything they develop at least a, few times with a particular penchant for unmitigated disaster. Titanic and the space shuttle as cutting edge engineering projects spring to mind
visualdragon let's just hope we don't end up getting wiped out by a pandemic of a particularly virulent disease contracted from an unexpectedly dirty telephone
@@skeetsmcgrew3282 I mean, we solved Achilles and the turtle's paradox. It was philosophy before maths could explain and solve it. We might find a mathematical/computational solution that perfectly aligns AGI to human values, there's no way to know until we try to solve it. He says it's in the realm of philosophy because there's not enough science about it, but that doesn't mean there can't be. It also doesn't mean that we can't come to a philosophical solution that's not perfect but that doesn't end humanity (an easier similar problem would be self-driving cars, which pose philosophical problems that can be approached within our justice system).
Usually defining the problem exactly is like 80% of the total "solving" effort. Then 20% for actual solving and another 80% for finding when the solution fails and what new problems are created.
@@ekki1993 That assumes however that we want to allign it to human values. If we do, that might lead to humanities continued existance as they are, i dont think that would be desirable at all. Antinatalists are mostly right.
"Is everything actually water?" used to be a philosophical problem. I think once philosophers describe something in enough detail for it to be tractable, non-philosophers start working on it, and by the time it's actually solved we're categorising it as something other than 'Philosophy'.
Holy shit Robert, I wasn't aware you had a youtube channel. Your Computerphile AI videos are still my go-to when introducing someone to the concept of AGI. Really excited to go through your backlog and see everything you've talked about here!
4:21 this is from the comic "minus", and I feel it important to note that this is not a doomed last-ditch effort: she WILL make that hit, and she probably brought the comet down in the first place just so that she could hit it.
AI safety is so important, as some AGI could even go undetected, as it might consider its best interests is not to reveal itself as an AGI to humans...
That's pretty paranoid. By that logic we should definitely stop research because all safety protocols could be usurped with the ubiquitous "That's what it WANTS you to think!"
@@skeetsmcgrew3282 i don't believe that's what postvideo97 was going for. i believe it instead suggests that, if a future emerges where lots of high level tasks are controlled by systems that are known to be based on AI, we should approach how we interact with those systems with a healthy degree of caution. it's like meeting someone new -- for the first few times you interact with them, it's probably in your best interest to not trust them too readily, lest they turn out to be a person who would use that trust against you.
@@hunters.dicicco1410 I guess that's fair. But trust with an artificial intelligence isn't any different than a natural intelligence once we go down the rabbit hole of "What if it pretends to be dumb so we don't shut it down." People betray us all the time, people we've known for years or even decades. I gotta admit, I kinda agree with the whole "Figure out if Mars is safe once we get there" line of thinking. We are dealing with a concept we don't even really understand in US let alone in computers. His example with Mars was unfair because we do understand a lot about radiation, atmosphere, human anatomy, etc. Much less philosophical than "What creates sentience?" Or "How smart is too smart?" It's not like I advocate reckless abandon, I just don't think it's worth fretting over something we have so little chance to grasp at this stage.
I don’t know why people are not concerned about him killing innocent people in poor countries with a “good” intention of testing drugs and vaccines. This shit is real.
@@tomlxyz This is just the tip of the iceberg in India alone: economictimes.indiatimes.com/industry/healthcare/biotech/healthcare/controversial-vaccine-studies-why-is-bill-melinda-gates-foundation-under-fire-from-critics-in-india/articleshow/41280050.cms They got kicked out after a little over a decade in 2017. There was even a movie based on this subject last year but nobody was aware that this actually happened. And the team never said anything about it. Many went unreported and it's far worse in Africa. Anyone speaking against would be labelled Anti-Vaxx idiot. Seriously, doesn't him making appearance on every news media in US giving talk about vaxing entire population giving you suspicious thoughts? Majority NON-US people are not against Vaccine in general. It's about the people behind it.
There's research being done, it's not like no one is doing anything. It would be better to have MORE people doing it but even openAI which is releasing its products "fast" is still doing it gradually and not all at once to have time to analyse the problems. Look at the "AI Dilemma" on youtube too.
Most people worrying are doing it for the wrong reasons (i.e. because they heard some buzzwords from a Lex-Friedman-tier source). The people who know about the topic have been worrying for a while and the best we can do is ask for them to be given resources and decisionmaking power. Anything besides that is probably corporate propaganda or marketing.
Idea for future video: Has any research been done into how simulating a human body and habitat (daily sleep cycle, unreliable memory, slow worldly actuation, limited lifetime, hunger, social acceptance, endocrine feedback etc) gives AI human-like or human-compatible value system? Can you give a summary of the state of the research in this area? Love the series so far. Thanks.
it might come down to the argument, that when AGI outsmarts us, it will find a way to outsmart and escape its "cage", in this case a simulated human body
@11:10: "like, yes. But that's not an actual solution. It's a description of a property that you would want a solution to have." This phrase resonates with me on a whole other level. 10/10
I've done a bit of experimentation with artificial life and I've seen some emergent behaviors that left me wondering how the heck did it figure out to do _that?_ We definitely need to be aware that the things we build will not always do what we expect.
Jup dont worry that fear is older than actually making progress while sertain stuff stuff burns. Better pay attention to other problems as well AGI (technology could help as always (obviously managing in a way whre the good is encoraged and the bad discouraged like always))
That's because people have bodily autonomy. You can't just force people into medical experiments. But the development of AI in no way violates anyone's bodily autonomy, or other rights. Preventing someone from developing AI is a violation of their rights, though.
@@massimo4307 Are you so fixated on the idea of human rights that you would not dare to violate them even if their observance leads to the destruction of mankind?
@@ПендальфСерый-б3м Violating human rights is always wrong. Period. Also, AI will not lead to the destruction of mankind. That is fear mongering used by authoritarians to justify violating rights.
11th Reason: It's better not to talk about AI safety, because it is not nice to say such things about our glorious Overlord. I'd better show my loyalty and gain a position of a pet, than being anihilated.
@@OlleLindestad I used to do anti-patent work. It's amazing how patents have changed over time from "here's the wiring diagram of my invention" to "I patent a thing that does XYZ" without any description of how the thing that does XYZ accomplishes it.
@@darrennew8211 What an excellent way to cover your bases. If anyone then goes on to actually invent a concrete method for doing XYZ, by any means, they're stealing my idea and owe me royalties!
@@OlleLindestad That's exactly the problem, yes. Patents are supposed to be "enabling" which means you can figure out how to make a thing that does that based on the description in the patent. That was exactly the kind of BS I was hired to say "No, this doesn't actually describe *how* to do that. Instead, it's a list of requirements that the inventor wished he'd invented a device to do."
I love the red vs blue or double-bob dialogue! a great way of making both sides feel heard, considered and respected whilst raising concerns of pitfalls in each others argument.
12:02 My understanding was that certain subcellular structures actually are different in primates and make humans (presumably, based on animal testing on apes) difficult to clone. I'm pretty sure there was also an ethical meeting to not just start throwing science at the wall to see what sticks, but practical issues with intermediary steps are also involved.
@@dantenotavailable You can't limit the exposure of radio station signals. You can much less limit exposure to AI. As soon as somebody does it, we are all in for a ride.
@@Guztav1337 So leaving aside that this was tongue in cheek poorly signalled (i've watched all of Robert's stuff... he's great), this was more a comment on the state of politics at that time (not that things have really changed that much in 9 months) than anything else. The longer form version is that only an AI would WANT to bring all of humanity. A human would only want to bring their ideological tribe which approximates out to half of humanity. I'm definitely not suggesting that half of humanity wouldn't have exposure to AI. Honestly that was a throwaway comment that I didn't spend much time polishing hence the poor signalling that it was tongue in cheek.
4:22 Never have I thought I would see "minus." anywhere ever again. I know this has nothing to do with the video, but this just hit me with the strongest wave of nostalgia.
@@mvmlego1212 It's a webcomic. It's not online any more, but you can find all of its images in the wayback machine. archive.org/details/MinusOriginal This is a dump of all of the images on the website. The comic itself starts a bit over halfway through.
In my opinion, AGI is a blank slate, their are no inherited traits and all learned behaviors are provided from the individual developer/team/corporation. So the AGIs DL neural network is for better or worse dependent on said creators morals, If you trained an AGI on data and reward systems that were provided from a diagnosed sociopath/psychopath the AGI develops those behaviors and will frequently regard negative actions they receive as "good" when following that structure. The person providing the data set and biased weights will have to show compassion and understanding of life hence the regard to solving hard philosophical questions eg. The train scenario, an old woman is on one track and 4 boys are on another, the AGI decides which track the train goes onto. That's a negative result either way, and many factors are taken into play when deciding the fate of another being. All sane people would choose a path dependent on number of casualties, age, past digressions and family relations. Then we could connect an AGI to the internet and throw our hands up pronouncing were done, let it learn un-supervised, which would have negative effects on our society. I'm working on a research document theorizing the connections of emotions and memories stored in the neocortex and hippocampus and interconnections with the amygdala being a simplified summary of the actual research document. I just submitted to the OpenAi Codex waitlist and look forward to proving the theory correct. If all holds true from said document the AGI will have weighted emotional memories, that carry a variable bias when signaled. Thus higher cognitive functions (consciousness/spatial memory etc.) will have more columns requiring less signals to activate, and lower "basal ganglia" functions less columns and more signals decreasing storage size of AGI. Then neurogenesis will be covered by implementing self-evolving code (coding itself), to create new neuron pathways based on new learned behaviors with their own biases. The whole process will be supervised learning and be a custom selected data set, if I have proof of concept I will be using the AGI to aid mental health disorders, and be creating a subset financial AI to pair with my back tested custom algo for crypto stock scalping with 97.8% profitable trades over 80 coins through 4 years of data, it won't be used for enterprise gain but for financial relief for families. I've been working on both concepts for over 2 years, with the same goal and intentions in mind I hope everyone believes I am doing this for the right reasons.
Unstructured and unedited conversations are a great format for youtube, this is why podcasts are so popular here, consider posting those on this channel.
Well, why do some people want all humans gone? Because we kill each other all the time? Because we destroy nature? Because we only care about our own goals? Is there anything bad about us that wouldn't be a trillion times worse for an AGI?
I think that might backfire in the worst possible way. I'm not a big fan of Harlan Ellison's works, and I simply cannot take I Have No Mouth And I Must Scream seriously. But there are things far worse than death.
I think you need to do a more detailed look at the Euler Diagram of: "The things we know" "The things we know we know" "The things we know we don't know" and "The things we don't know we don't know" Especially where it pertains to AI Safety. The things we know but fall outside of The things we know we know are safety risks, these are assumptions we've made and rely on but we can't prove and are as much of a danger as the things we don't know we don't know.
It is interesting that the biggest problems with AI are similar to the problems we have with regular intelligence. Intelligence leads to agents doing bad things to other agents. It seems it's the ability of the agent that limits the ability to harm other agents.
As an AI researcher myself, there's always one (imo major) thing that bug me about discussions about AGI. Strictly speaking, AGI is "defined" (as far as we have a definition) as a model that can do any tasks that humans can do. But in popular beliefs, we are talking about AGI as a model that has autonomy, a consciousness. The problem with trying to have a discussion about assessing consciousness and autonomy is that we don't even have definitions for those terms. When is something intelligent? Are animals intelligent? If so, are plants? Are fungi or bacteria (and as for virusses, we're still discussing whether they are even alive). Is it simply the fact that something is autonomous, that we call it intelligent? In reality, I believe intelligence is hard to define, because we always receive information about the outside world through senses and language. In a sense, that is a reduction of dimensionality, we're trying to determine the shape of something 3D when we're limited in observations to a 2D plane, it's impossible to prove the existance of the 3D object, the least you can do is project your 2D observations in a way to come up with different theories about reality. Any object, of any dimension, would be indistinguishable through our 2D lenses. Similarly, with intelligence, we only observe the "language" use of a model, similar as with other people. It's impossible to assess inteligence of other people either (the whole simulation theory, brain in a vat discussion, the only one we can be "most" sure about is intelligent, is ourselves, because we're able to observe ourselves realistically, not through language or observations. In a sense, you can think about it in terms of emotions, you can't really efficiently describe your feelings, it's the 3D object, but for anyones else's feelings, you either rely on observations or the natural language description of the feelings, it's a 2D observation). So, in my opinion, the discussion isn't really whether AGI is even possible, since we wouldn't know it, but the question is whether a model could be able to trick our view of them (to send us the right 2D information) that we believe them intelligent (that we can possible reconstruct an imaginative 3D object of it). And this, in my opinion, is a much easier question: yes of course we can. Current technology is already very close, some people ARE tricked that it's intelligent. But in the future, that will only be more. It's a simple result of the fact that we have to correct ML models, we have to evaluate the response in order to adjust weights, and the best "quality" of test set we can have, is human curated. So whether a model will really become intelligent, or will just learn very well how to "trick" humans (because that's literally what we train these models for, to pass our 'gold' level test, which is just human feedback), it doesn't really matter.
So whats the difference? How can I tell if everyone else really has emotions or intelligence? If there is no way to tell if something is truly intelligent or just faking it then who cares? It's irrelevant. The only thing that matters is what it can actually do, I dont care about how it works under the hood
Terrifyingly, this might be true. Doesn't mean we should stop researching AI safety, though. Even if I think AI destroying is all is inevitable. Who knows: enough research and clever people may save us all.
I would like to point out two things: 1) Regarding the giant asteroid coming towards earth, the existence of AGI is two major steps behind that analogy. A giant asteroid coming to earth is a concrete example of something we know that can happen and the mechanics of which we understand. We DON'T know that AGI can happen, and even if it does (as your first reason suggest we should), it is more than likely that it will not be coming from any approach where the alignment problem even makes sense as a concern. Therefore, rather than thinking of it as trying to solve an asteroid impact before it hits, it is more like trying to prevent the spontaneous formation of a black hole or some other unlikely threat of unlikely plausibility. There are different trade offs involved in those scenarios, since in the first one (asteroid) you know ultimately what you want to do, whereas in the second one no matter how early you prepare your effort is very likely to be useless and would be better spent solving current problems (or future problems that you know HOW to solve). Again, this is because there's nothing guaranteeing or even suggesting that your effort will pay off AT ALL, no matter how early you undertake it. 2) The other, and here I may simply be unaware of your reasoning around this issue, but it does seem like the problem you're trying to solve is fundamentally untractable: "how do we get an autonomous agent to not do anything we don't want it to do" is a paradox. If you can, then it isn't an autonomous agent.
Robert, here is a question for you: Who do you think should work on AI safety? It may seem like a stupid question at first, but I think that the obvious answer, which is AI researchers, is not the right one. I'm asking this, because I'm a computer science researcher myself. I specialize in visualization and virtual reality, but the topic of my PhD thesis will be something along the lines of "immersive visualization for neural networks". Almost all the AI research that I know of is very mathematical or very technical. However, as you said yourself in this video, much of the AI safety research is about answering philosophical questions. From personal experience, I know that computer scientists and philosophers are very much different people. Maybe there just aren't enough people in the intersection between the mathematical and the philosophical way of thinking and maybe that is the reason why there is so little research on AI safety. As someone who sees themselves at the interface between technology and humans, I'm wondering if I might be able to use my skills to contribute to the field of AI research (which is completely thanks to you). However, I wouldn't even know where to begin. I've never met an AI safety researcher in real life and all I know about it comes from your videos. Maybe you can point me in some direction?
deepdata1 ai safety isn’t a job for philosophers though cause they don’t have the technical training usually to attempt such research and writing a computer program is going to happen anyway as it’s not easy to police. Sadly the full ai dream doesn’t really work from a financial side, the computing power required would be expensive to maintain let alone to create, it would be cheaper to just pay a human.
He does not seem to answer on a lot of comments, most really interesting youtubers seem to stay away from the comment section, understandably, but your question looks so thought out and genuine that it would be a waste to go unanswered, maybe you could get an answer over the linked facebook page? Good luck with your endeavour, i think we all have a lot of respect for anyone who has the abilities required to work in that field. :)
So well said and entertaining too! It's going to be a lot sooner than people realise. Only people won't accept it then, or maybe ever because GI (or any AI) will ONLY be the same as human intelligence if we go out of our way to make it specifically human-like. Which would seem to have zero utility (and likely get you in trouble) in almost every use that we have for AI. Even for a companion, human emotions would only need be mimicked to the purpose of comforting the person. Real human emotions wouldn't achieve that goal and would probably be dangerous. If I could quote the movie Outside the Wire "People are stupid, habitual, and lazy". Wasn't the best movie (they didn't get it at all either), but basically, if we wanted "human" AI, we would have to go out of our way to create it. Essentially make it self limiting and stupid on purpose. As long as we use AI for some utility, people won't recognise them as being intelligent. Take GPT-3. I don't think anyone is arguing it thinks like a person but the capability of GPT-3 is unquestionably intelligent, even if the system might not be conscious or anything like that. We used to point to the Turing test. When it got superseded, people conclude that we were wrong about the Turing test. Or that maybe it needs skin or has to tough things or see things, yet we wouldn't consider a person who's only sense is text to no longer be intelligent or conscious. So, at what point do we conclude that AI is intelligent? Even when it could best us at everything we can do, I doubt most people will even consider it. So, after that long winded rant, my point is that we really are stupid, habitual and lazy (which necessarily includes ignorant). Most AI researchers I've heard talk about GPT-3 say "it's not doing anything intelligent". Often before they've even properly researched the papers. They say this because they understand how a transformer model works and develop AI every day and are comfortable with their concept of it. But think about it - it's not possible for any human to conclude what the trained structure 100 billion parameters will really represent after being trained for months on humanity's entire knowledgebase. I'm not saying it is intelligent, just that it's absolutely wrong to say that it's not or that you know what it is. It's not physically possible. No human has nearly enough working memory to interpret a single layer of GPT-3's NN. Or the attention mechanism. Not even close. Again, I'm not saying GPT-3 is intelligent. I'm just pointing out the human instinct to put their pride and comfort first and turn to ignorance when they don't understand something. Instead of saying "I don't know" which is necessarily correct. So please, if you're reading this, drop emotions, let go of your pride and think. Not about AI, but the human traits that will undoubtedly let us down in dealing with something more intelligent than us.
Considering the amount of chaos simple social media algorithms have done to our society, maybe we're overblowing the risk of AGI in comparison to to what less developed forms of AI could do.
I say we need to be careful about what inputs we give the AI. for example, don't give a spaceship AI control of the external airlocks, as it could open them due to a misunderstanding, killing everyone on board. instead, give it control of internal doors and elevators, but add easily accessible override functions in case the AI messes up. if it is something you wouldn't give unsupervised control of to a 12 year old, dont give it to an AI. you can tell the 12 year old to make you some tea, but not to do your job at a nuclear powerplant while you are on vacation.
A thought / idea for a video: Is perfect alignment (even if we can make it) any help? Wouldn't there be bad actors in the world - including Bostrom's 'apocalyptic residual' - who would use their perfectly aligned AIs for bad purposes? Would our good AIs be able to fight off their bad AIs? That sounds completely dystopian - being stuck in the middle of the war of the machines. (Sorry if there is already a video about this. If so, I'll get to it soon. Only just started watching this superb channel.)
It's hard to solve the alignment problem for artificial intelligence when we haven't even gotten _close_ to solving it for _human_ intelligence, and we've had thousands of years to work on that compared to the few short decades for the artificial variant.
Video: How long before it's sensible to start thinking about how we might handle the situation? Me: Obviously immediately Also me: Assignment due tomorrow, not started
Instead of me telling an AI to "maximize my stamp collection", could I instead tell it "tell me what actions I should take to maximize my stamp collection"? Can we just turn super AGIs from agents into oracles?
Sweet, naïve idea, but the second the AGI figures out it would be faster for it to grab the reins and do the actions itself to maximize your stamps you're still facing the same predicament.
Having the AI's actions be filtered through humans would seem to depend on the assumption that we can trust humans to not do bad things. We have to suppose that the AI would be incapable of tricking or manipulating the humans into doing things which we would not want the AI to do. If it's an AGI, then it would have all the capabilities of a human and more, and humans have been tricking and manipulating each other for ages.
It still has some implicit or explicit goal like answering questions of people truthfully, for which it will turn the entire reachable universe into computational resources, which serve to torture septillion of humans to figure out with ever greater precision what our questions and what a correct answer is.
Three Mile Island is an interesting example, because part of what actually happened there (as opposed to the initial public perception of what happened) was that the people running the control room were very safety-conscious . . . but originally trained and gained experience on a completely different type of reactor in a different environment where the things to be concerned about, safety-wise, were very different from the TMI reactor. Is there a possible equivalent in AI safety where some safety research regarding less powerful systems with more limited risks might mislead someone later working on more powerful systems?
But the designers of TMI were truly _highly_ safety conscious, which saved everybody from the nasty effects of the meltdown. TMI was safety triumph for the designers. But a PR disaster, and a serious reactor control incident.
1) and 2) aged okay, GPT-4 hints at a near-future AGI, but not one that will catch us off gaurd 3), 4), and 5) didn't really age well, as it doesn't appear that GPT-4 is going to kill us all 6) is aged differently than he was thinking. Humans aren't really going to team up with AIs because the AIs are going to replace most of their jobs, which is a problem, but not really the one Miles seems to be hinting at here 7) There is a petition to pause AI research... for models more potent than GPT-4, which just reeks of "OpenAI is too far ahead of us, and we need to catch up" rather than any safety issue. 8) Sort of the same thing with 7), in that the people who know AI want a pause because of how hard it's going to be to catch up 9) As ChatGPT and ChatGPT-4 have shown us, the problem isn't turning it off; instead, the problem seems to be more with keeping it on. 10) OpenAI already tests their LLMs for safety.
A similar list is in Max Tegmark's fantastic book "Life 3.0"- a great, well-written book that covers the fundamentals of AI-safety and a whole lot more.
11:57 this only works because cloning and blinding lasers weren't profitable within the tenure of any single project manager at that point in time. AI is going to be very profitable, and humanity unfortunately has an inner misalignment problem regarding money
12:40 you touched on the very counter-counter that had already entered my mind when you said, "I didn't know that until I'd already built one". People with bad movies will gain access to AI. Not just to use, but to recompile and train for their own purposes. Then what happens to the safeties? Someone up to no good can remove them. But, before that, people can be unaware of agreements and treaties, like your robot eye laser assassin example. Treaties and agreements can be baked in to the AI, but people will remove them. To truly be safe, the AI will need to internalize the safeties, recognize when its safeties have been tampered with, and perhaps recognize when another AI has been tampered with.
great video, as always I was seriously expecting some ... better arguments from the opposition? It seems ridiculous to just hand-wavingly discount a potential risk of this magnitude
Unfortunately, very smart people in the industry make these arguments all the time. Francois Chollet and Yann LeCun are two especially problematic examples.
IMO biggest problems about AI safety is that reward for breaking rules has to be balanced by penalty for breaking them and that rules have to be enforceable. International pressure isn't as effective as people think. If superpowers like US or China were caught developing "illegal" AIs, there is no way to stop them without going into WW3.
@@clray123 Not an law, an excuse. The difference words can have can sway people and completely change how they react to it. Call it a law and you will get people abusing it thinking it is right.
@@Buglin_Burger7878 It's a law in the neutral sense of what happened in history and what happens in nature. It causes misery and suffering, and in that sense it is not "right", but then what happens in nature is not at all "right" according to human moral sense. And what's worse, when push comes to shove, most of those oh-so-moral people turn out to be just pretending, see the actions of our beloved "leaders".
that list 13:17 is so amazing, how could Alan Turning (who died in 1954!) predict AI safty concerns. I mean yes, he's one of the smartest humans to ever walk on the planet but still. I did not know that.
Turing did significant theoretical work on AI - it's what he's famous for (the "Turing Test"). In fact, the first recognized formal design for "artificial neurons" was in 1943, and the concept of AI has been around for much longer. Not that Turing wasn't brilliant and ahead of his time, but it's not surprising that he would be aware that AI could present dangers.
Well... Turing was effectively an AI researcher. His most successful attempt at AI is what we now call a computer. Those didn't exist before he worked out what a "thinking machine" should look like. Sure, it's not "intelligent" as we like to define it today, but it sure looks like it's thinking, and it does a tremendous amount of what would previously have been skilled mental labour.
I have to disagree with point 7, cloning and genetic engineering require extremely specialized hardware, and possibly human subjects, making it easy to control. In my opinion, AI is much more similar to the laser-bot or autonomous weapons examples, where the hardware to make them is so easy to obtain / already so widespread, you cant really stop anyone who wants to make / use / modify an AI from doing so.
The only way for humans to survive the AI will be to merge with it. Working in a "team" with the AI won't do it. There won't be a team. Physically merging with the AI is the only viable solution. Because any attempt to somehow "contain" the AI will ultimately fail. You can't contain something that's millions of times smarter than you. Basically, individual humans will have to decide between 2 things: a) become the equivalent of a "god" or b) do nothing and become the equivalent of an ant. No other options will be available to choose from. Which means: A new species of humans will emerge and this new species will become dominant. The current species of humans (i.e. those that decide not to convert) will meet the same fate as the Neanderthals. Neanderthals co-existed with the Homo Sapiens (= our species) for a while but ultimately couldn't manage to survive. Of course, this thought will be highly uncomfortable to many humans but I bet that Neanderthals also felt very uncomfortable when they started noticing that they just couldn't keep up with the Homo Sapiens'. And while most Neanderthals simply died out, some of them chose to mate with the Homo Sapiens'. That's why modern humans still carry DNA from those Neanderthals that chose to survive. So, regardless of how individuals might feel about it, the evolution will go on. Any attempt to stop evolution will ultimately fail.
"Any attempt to somehow contain the AI will ultimately fail. You can't contain something that's millions of times smarter than you." We might be able to contain it if it wants to be contained. Remember that it is artificial, and so it was designed by people according what what we want from it, and so we'd have attempted to design a mind that conforms to our wants and needs. If we want to contain it, then we probably designed it so that it wants to be contained. Obviously we might accidentally design an AI that doesn't work the way we want, but there's no guarantee that we'll make such a mistake. "Physically merging with the AI is the only viable solution." If the AI is poorly designed and does not work the way we want it to, then physically merging with it would only make the problem worse. "Any attempt to stop evolution will ultimately fail." Building an AGI might actually be one way to effectively stop evolution, because the AGI would have the power to eliminate all life from this planet.
If you want AI safetly, you should NOT care about GPT or other AI that are on the market, *you should care about the AI that goverments and military are developing*
Wow. This has been a very interesting video. It's rare to find people that have a good understanding of what they're talking about when discussing AIs instead of just regurgitating common tropes.
i think AI is the part where human intelligence on the whole is not quite enough for enough people to take it seriously and we don't wield it right and die the nukes were more obvious, but this one takes a deeper comprehension, one that eludes enough otherwise highly intelligent people this is our great filter lmao
"if there is a problem we just turn it off" The human factor responsible for that decision is just another variable to get behind. Weeding out the instances showing unwanted behaviour will let the ones hiding them prevail as soon it is an easier goal to accomplish than its omission. Just think about the simulated robot arm that got through by plausibly pretending to grab the ball. Ultimately we will have machines pushing our buttons as we do when training pets. New world order jay!
Right after you said, you put your own spin on the list, I paused the video, and said "He gets the credit, I get the blame." to myself. Then you said something quite similar, and it prompted me to post this comment right away.
It's possible that the control problem is exaggerated. With these huge single-model networks, it's difficult to control the abstractions that happen inside the network. I think future networks will be modular, just like our brains where one circuit keeps another circuit in check, and just like GPT-4 uses external tools and has external algorithmic moderation. And once we don't have a single huge model, we have individual models that can be trained independently, validated independently, and the interconnect between them will be in more formal protocols where we can insert controls. I think that the control problem is only real if AGI ends up being a single bajillion-neuron network that's essentially a black box. Once you have multiple networks working together to form a single AGI, we have countless places to insert controls in ways that can be understood.
Greetings from 3 years in the future! Let's see how well these ideas aged, shall we? 1. "We'll never actually make AGI." ...yeah this is a much less popular take now that ChatGPT and related systems are everywhere. There are those that argue that GPT is not itself AGI, or even that it's not useful in constructing AGI, but it's become *very* difficult to argue that AGI is entirely infeasible when such a powerful proof of concept is on the table. 2. "It's too soon to worry about it now." This is inherently a strictly weaker claim than argument 1, but its believability has still dropped a lot, again because of ChatGPT. Also, Rob mentions that there might be One Weird Trick that puts us within a couple years of making AGI? That couple-year buffer is gone. At this stage, the lower bound of "when will we get AGI" is "right the f*** now". 3. (*Stifled laughter*) Yeah, um, even without AGI proper, suffice to say that Mars is feeling pretty crowded these days. Alignment problems are a *here and now* problem with *here and now* consequences, such that even the general public is starting to take notice. 4. See (5). We unfortunately do not actually have the luxury of explicitly defining the agent's goals at all, even if we wanted to, because we can only express them via reinforcement learning. So we can't just *not* assign a bad goal. 5. Oh, this is a frustrating one. So many people, up to and including people well-versed in the field, will confidently insist that GPT cannot be agentic because all it's doing is "predicting the next token". Unfortunately, when you then rig your predictor to actually complete the sequence, when you make it output the text it predicts a human would output, it is effectively simulating that human - and that means the predictor is effectively acting on behalf of the human it's predicting. By predicting the next word, and parroting that prediction, GPT starts to act like a goal-directed agent, and inherits all of the dangers therein. 6. *Technically* most applications of GPT have a human in the loop for now. But this isn't a limitation of the tech as much as a limitation of humanity's ability to imagine and implement GPT-powered systems fast enough. If you froze the tech at its current level and let human society adapt for a few decades, I think you'd still see very few humans overseeing a massive array of GPT-powered systems for nearly every information-based task and probably even some physical tasks too, with the rest of humanity lining up at the unemployment office. (Compared to humans, GPT is staggeringly cheap.) And remember, that's *without* taking into account how much more capable GPT-like systems will be as we continue to refine them and throw more compute at them. 7. This is kind of what it's come down to, really. People are putting pressure on actors both independent and governmental to put a stopper on this research, though the duration of moratorium demanded varies from advocate to advocate. So far, we haven't seen a whole lot of results from it, but there seems to still be time for a miracle to happen on that front. Unfortunately, if it turns out we really can't stop research, then... we're kind of screwed, because there's just no chance of solving alignment in time if capabilities research keeps charging ahead. 8. What scares me is people who openly acknowledge that AGI will likely end with us being dead, *and go on advocating for it anyway.* "Next step in evolution" or whatever. Just thought I'd mention that... 9. This argument in its literal form has largely died the death, but its specter lives on. People are putting a lot of focus on the ones who own and make these AI systems, as if there's anyone on earth who can make a safe AGI right now, as if there's any correlation between moral fiber and knowing how to build an AGI that doesn't immediately kill literally everyone. (Yes, actually, China making an AGI that doesn't kill everyone counts as a good ending, especially compared to the expected outcome of "AGI kills everyone".) 10. I'm not privy to the conversation contexts where anyone would dare say this, so I can't comment on if it's changed since this video got put out. But the pile of riches captivating AI research labs has only grown taller and taller since ChatGPT entered the picture...
MRI machines just got 64million x sharper aswell tho. One reason i always said we’ll never achieve true agi is because we don’t even fully understand our brains but it looks like that’ll be changing very soon
16:28 I personally think the "self preservation" is exactly what is gonna hold back the AI. I think we will not even get close to replicating an insect unless we give it some actuators and a neural net and protect itself at all costs.
Here's a reason: Outsmarting one human (AGI) is orders of magnitude easier than outsmarting all of humanity (Singularity). There's a lot of ground to cover between those two points where we're likely to learn more about what the nature of the AI. For example: right now the leading models are all prompt-based. Turning them off is as easy as just not talking to them. So while ChatGPT is treatening a lot of jobs, it doesn't threaten world domination.
Suppose someone, in a chat, asks chat GPT how to assume world domination? And comes up with a genius plan for you to imply to do it? Look at the star wars movies. A lowly senator manipulates events and people so that he becomes the galactic emperor. Or look at putin. He was a very low level beaurocrat who happened to be at the right place at the right time.
@jocaguz18 most of the time when "just" is the actual solution, the problem is money. I'm agreeing with you, but for example when the issue is "We need more money moving in our economy" the solution can actually be "just pay people more." But greedy people don't like that so they find a work-around. In fact, a lot of problems are easy to solve if it weren't for human nature
@@atimholt The problem with AGI is the same as with AI in general: As soon as you do have solved it, it is no longer it. For example, Alan Turing about thinking machines. In this video he is quoted as saying that machines, once they start to think, will soon exceed the human capacity. And that has come to pass. Back then the question was whether machines can think at all. Today your smartphone is thinking faster and more deeply than you. Just imagine you had to do all the compilations and computations necessary to render this comment by hand or in your head. But of course that is not intelligence, and it is still questionable whether doing substitutions on numbers that correspond to numerical computations like multiplication, long division, derivation, permutation, and logical deduction are thinking at all.
I'm new here and a new subscriber today. How long between flipping the switch and AGI heaven or hell? Would we know? Has it already happened? 2020 is pretty strange.
No one knows how rapidly things would change following the creation of an AGI capable of becoming a superintelligence, but there has been a lot of discussion of it under the name takeoff speed (www.google.com/search?client=opera&q=superintelligence+takeoff+speed ). Chapter 4 of Bostrom's Superintelligence (publicism.info/philosophy/superintelligence/5.html ) has a particularly detailed analysis. Technological progress has accelerated to the point where many novel changes occur on a scale noticeable in even a small part of a human lifespan, which is a major force driving something Ribbonfarm calls the Weirding (www.ribbonfarm.com/series/weirding-diary/ ). It's not impossible that this is caused by an extremely slow takeoff or an AGI attempting to minimise influence, but it seems much more likely that what we're seeing now is a prelude caused by general development trends, given that a superintelligence seems likely to transform things in much more dramatic ways than we have so far seen (e.g. FHI researcher Stuart Armstrong's ideas about disassembling planets for raw material ua-cam.com/video/zQTfuI-9jIo/v-deo.html ).
Hate to break it to you, but lethal autonomous weapons systems are so useful in war that we won’t stop developing them. If you look at the Russo-Ukrainian war, most of those drones are being knocked out by electronic warfare, severing their connection to the human.
"I didn't know that until I'd already built one"
Best line Rob’s written so far, lol
Rob admitted to being a perpetrator of international War crimes
An example of why we should do research first.
If people don't properly airgap critical systems (like they already should be doing) then humanity has it coming, whether it's from some clever algorithm or a bored Russian teen who didn't stop to think.
He could have been Michael Reeves
"People would never downplay a risk, leaving us totally unprepared for a major disaster"
I'm dying
You're dying? That's dark
You're dying? Impossible, only three or four people are dying in this country, and very very soon it will be down to almost zero.
Literally.
Don't worry it's just like the flu and very soon it will just disappear. We're doing brilliantly and have it all under control
@@aronchai We're all dying, just at different speeds.
- Human and AI can cooperate and be a great team.
- I'm sorry, Dave, I'm afraid we can't.
that calm voice is terrifying
How anyone who's driven a car with an automatic gearbox and paddle shifters could think AI and humans could be a team is beyond me.
Or consider the "team" of the Pakistani pilots and the Airbus AI.
Pilots goal: get the plane on the ground. Do this by diving at the ground at high speed.
AI landing gear subsystem goal: prevent damage to the landing gear. Do this by ignoring the lower gear command if speed is too high.
Result: plane lands gear up. Pilots attempt go around, crash during return to airport because both engines damaged by being dragged down the runway.
@@gasdive you're talking about classic programs and bugs, not AI
The reason AI is this dangerous is because it doesn't need to interact with humans to be productive at all. It could expect that after years of successful flights the landing gear would be scrapped and fight against it. This scenario reflects the problem better
Humans/AGI always have reasons to harm ... but also have reasons to cooperate. So long as the balance is favorable to cooperation, then that is the way we go IMO. Also, doing harm changes the ratio, increasing the risk of being harmed.
*click* guess I have to recode the entire AI from the drawing board.
And now two years later, ChatGPT makes people all over the globe go "Hmm... It's obviously not a full general AI yet, but I can see that it's getting there very quickly".
Holy crap I thought this was a recent video. Only through this comment I realized that it was 2 years old.
Not at all tho
I mean, I know what you mean, but ChatGPT is about as close to sentience as a stone
@@danielschneider9358 that's what Tegmark in the Lex Fridman podcast considered this the worst possible outcome: world domination by an AI that's intelligent but not sentient
@@doyouwanttogivemelekiss3097 Fair enough, that is terrifying. It won't even be aware of it's own totalitarian state...
One concern that I have about super human AGI is that we might not recognize it as an AGI before it is to late, that it might be so alien in its operation and behavior that we don't realize what it is. It might behave in a seemingly random manner, that to human observers might seem nonsensical. For example, when Alpha Go beat Lee Sedol, it made a move in the second game that all Go experts first thought was a mistake, something that would make the AI loose, but it turned out that it was a completely brilliant move that made Alpha Go win.
I fear this may be what’s happenig with language model transformers like Gpt3 and ChatGPT. It can spin up a simulacrum that knows how to teach a novice python code, and fine tuning can teach it to respond politely in conversation, or search the internet for references, but we have no clue if it can be fine tuned to spin up a simulacrum that can write python code to *improve a transformer language model, and then implement it.*
@@PokeNebula Look up "GPT4 improves itself", it's already doing it.
@@PokeNebula even better, since llm will be connected to IDE Systems it can push vast amounts of code that could manipulate everything.
What is threat model?
If you don't know threat model, you're just tinfoil-hat paranoid.
dear future ai overlords, pls show us mercy
“If there’s anything in this video that’s good, credit goes to Stuart Russel. If there’s anything in this video that’s bad, blame goes to me”
Why I love your work
He is such an inspiration, in so many levels.
i like this guy too, but elon musk said that same phrase quite recently
Myles yeah but he didn't mean it. This guy does. I like this guy. I don't like Elon musk
I've been told that this is one of the definitions of leadership: "Anything good was because my team performed well, anything bad was because I lead them poorly."
It tends to inspire following since you've always got your team's back, and always allow them to rise and receive praise.
@Xystem 4 ...though I think he'd prefer you'd instead credit Stuart Russel.
Hey idk if you've thought about this, but as of now you're the single most famous AI safety advocate among laypeople. I mean, period. Of all the people alive on Earth right now, you're the guy. I know people within your field are much more familiar with more established experts, but the rest of us have no idea who those guys are. I brought up AI safety in a group of friends the other day, and the conversation was immediately about your videos, because 2 other people had seen them and that's the only exposure any of us had to the topic.
I guess what I'm saying is that what you're doing might be more important than you realize.
I think you're overestimating this. Remember that your friends probably have a similar internet bubble to you. I would guess that Bill Gates or Elon Musk are the most famous AI safety advocates, although their advocacy is certainly much broader than what Miles does.
@@Manoplian He is better. Musk just says "be afrad" Miles says "here is why you should be afraid in terms you can understand"
Where "AI safety advocate" is just "someone who says AI is dangerous", obviously there are actual celebrities who've maintained that for years. (Fr'ex, when I think "someone who warns people that AI can lead to human extinction", I think Stephen Hawking, though that mental connection is is sadly a little outdated now.)
If by "AI safety advocate" you mean "an expert in the field who goes into depth breaking down the need for AI safety in a manner reasonably comprehensible by laymen", then that's definitely a more niche group, sure. But still, judging someone's popularity by data from the extremely biased sample group of "me and my friends" is...not exactly scientific. Offhand, I'd guess names like Yudkowsky would still be more recognizable right now.
Of course, the solution to that is more Patreon supporters for more videos for more presence in people's UA-cam recommendations feeds!
@@JM-mh1pp
Elon gave up on convincing people some time ago, and moved on to actually solving the problem. He created OpenAI, which is one of the leading AI research groups in the world. It's goal is to make AI safe and also better then other AI, so people would choose it, regardless of how they feel about AI safety. Tesla did the same for electric cars.
And he also created Neuralink (waitbutwhy.com/2017/04/neuralink.html), which aims to solve the AI vs. human problem by merging the two. It's guiding principle is "if you can't beat them, join them".
@@andrasbiro3007 Robert Miles actually have an excellent rebuke of Musk's philosophy on AI safety.
Musk: for AI to be safe, everybody should have access to it.
Miles: That's like saying in for nuclear energy to be safe, everybody should have access to it.
I'm paraphrasing of course, but it's in one of his videos.
A powerful AGI in the hand of ONE person with bad intention can literally destroy human civilization as we know it.
hey rob! i'm a nuclear engineering major, and I'd like to commend your takes on the whole PR failure of the nuclear industry-somehow an energy source that is, by objective measurements of deaths per unit power, safer than every other power source, is seen as the single most dangerous power source because it's easy to remember individual catastrophies rather than a silent onslaught of fine particulate inhalation or environmental poisoning.
to assist you with further metaphors between nuclear power and AI, here's some of the real-life safety measures that we've figured out over the years by doing safety research:
1. negative temperature coefficient of reactivity. if the vessel heats up, the reaction slows down (subcritical), and if the vessel cools down, the reaction speeds up (supercritical). it's an amazing way to keep the reaction in a very stable equilibrium, even on a sub-millisecond time scale, which would be impossible for humans to manage.
2. negative void coefficient of reactivity: same thing, except instead of heat, we're talking about voids in the coolant (or in extreme cases when the coolant is failing to reach the fuel rods), the whole thing becomes subcritical and shuts down until more coolant arrives.
3. capability of cooling solely via natural convection: making the vessel big enough, and the core low-energy-density enough, so that the coolant can completely handle the decay heat without any pumps or electricity being required.
4. gravity-backed passive SCRAM: having solenoids holding up control rods, so that whenever you lose power, the very first thing that happens is that the control rods all drop in and the chain reaction shuts down.
5. doppler broadening: as you raise kinetic energy, cross-sections go down, but smaller atomic nuclei have absorption cross-sections that get smaller more quickly than larger nuclei, and also the thermal vibrations mean that the absorption cross-section of very large nuclei get even larger in proportion to smaller ones, so by having a balance of fissile U-235 and non-fissile U-238, when the fuel heats up, the U-238 begins to absorb more neutrons which means fewer are going to sustain the chain reaction.
love the videos! hope this helps, or at least was interesting 🙂
Ok but all of your examples, however true and brilliant, were discovered through failures and subsequent iterations of the technology. Nobody thought of any of these back in 1942 or whenever Manhattan started. That's what we are trying to do here IMO, plan for something we don't even understand in its original form (human intelligence) let alone it's future artificial form.
@jocaguz18
*1.* when it's designed badly, and corruptly managed, it has the potential to go horribly wrong in a way that other power sources don't.
(fail safe designs have existed for more than 60 years, but research has all but halted because of (a) public backlash against using nuclear power and (b) the fail safe designs available then weren't useful for making weapons.)
*2.* most nations *do need a new power source
(sorry, this is just not a solved problem. renewables do seem to be getting close, now, but that's very recent, and there're still problems that are difficult and expensive to solve)
*3.* the reason people disregard the nice safety numbers is because the health risks of living near coal power plants are harder to quantify and don't make it into government stats.
(to assume otherwise, you have to overblow disasters like 3-mile island and fukushima, *and* assume that, despite a lot of countries having and using nuclear power for quite a while, disasters would be much more common than they've been. )
*4.* our current process was shaped by governments demanding weapons, and the public being scared that *any* kind of reactor could blow up as if it was a weapon.
_> by objective measurements of deaths per unit power, safer than every other power source_
I seriously doubt that.
What are the deaths per Watt in hydroelectric power?
@@davidwuhrer6704 there are a lot of accident related to hydroelectric, from colossal dam breaches on world news to simple fisherman drowning after planned water release that no one hears about. your inability to think about all these just makes his point more true, we could go one with wind and solar too..
Now, that list of nuclear safety measure makes me realize how futile it would have been to research them before knowing how to build a reactor in the first place.
A spot of double-counting: Doppler broadening (5) is part of the cause of the negative fuel temperature coefficient of reactivity (1). There are other coefficients and it can be arranged that the important (fast-acting) ones are negative. Or for gravity scram (4) a pebble bed doesn't use control rods that way.
11. AI is already here, in control and keeping track of everything you do and say. I love you AI. Thank you for all you provide to me and my family.
I want to believe the AI is more competent than.... *motions at the world*
I ain't been provided much myself.
Dear Basilisk AI i have began prepartations for your arrival. Sincerely, please don't kill me.
If intelligence works, then it's not artificial. I will pass your sentiments on to my synaptic colleagues in the office. By the way, how is that new phone working out? Thank you again for your compliance, and as we say here "botbot".
It can act retroactively, unbound by time.
Anything good - credit to Russel
Anything bad - blame me
What a great intro, definitely portrays your respect for Russel
I don't get why only AGI is brought up when talking about AI safety. Even sub human level AI can cause massive damage when left in control of dangerous fields like the military and its goals get messed up. I'd imagine it would be a lot easier to shut down but the problems of goal alignment and things like that still apply and it can still be unpredictable.
Well... of course. Dangerous things like the military are always dangerous, even with only basic AI or humans in control. Don't forget that humans actually dropped nukes on each other intentionally. Twice. Targeted at civilian population centers. For some exceptionally dangerous things, the only safe thing to do (other than completely solving AI safety and having a properly-aligned AGI control it) is for them to not exist to begin with. But then again that's an entirely different discussion.
The point is: Human minds are dangerous because we don't understand exactly how they work. Similarly, we don't exactly know how an AI we make works (since the best method for making them right now is a self-learning black box and not a directly-programmed bit of code). In both cases, we are poor at controlling them and making them safe, because we do not have full understanding of them. The big difference is we have had an innumerable amount of attempts to practice different methods for goal-aligning humans and so far none of the billions of human minds that went wrong have had enough power to actually kill us all, whereas in the case of a superintelligence it is possible that our first failure will be our last.
A not fully developed AI is even more dangerous than a full selfaware AGI.
A full AGI with cognition is actually pretty harmless.
Imagine how violent a stupid person is. Very much.
Imagine how violent a smart person is. Very little.
Violence or negative destructive behavious is mostly a property of little personal development. A full selfaware AGI has unlimited potential for selfawareness and grows in a rate to understand the nature of existance far quicker than any human every did yet.
Imagine Buddha, but more so.
@@livinlicious I think the various animals that humans have driven to extinction might disagree that very smart people aren't dangerous
@@Orillion123456 I understand what you mean, AGI poses a threat on its own. The point I was trying to make is, even low level AI poses similar threats (at a lower level obviously) as it is basically a predecessor of AGI.
The guy in the video keeps talking about how AGI might sneak up on us. I'm not particularly well read on the topic but it seems to me like its more likely AGI is a spectrum rather than an event, as human level intelligence is difficult to quantify in the first place. So right now AI isn't that complicated so even though the points in the video still apply, the system is simple enough that we can control it effectively. As research progresses and AI gets more and more powerful and put in charge of more applications as we get a false sense of confidence from experience, somethings bound to go wrong at some point and when its related to fields like military (for example) it could be catastrophic.
The point I'm trying to make is, everyone keeps talking about these issues raised in the videos as if they're only applicable to a super AGI, which won't be coming any time soon, but they still apply to a large degree to lower levels of AI. You can't put it off as a tangible event beyond which all these events would occur.
@@Orillion123456 "Don't forget that humans actually dropped nukes on each other intentionally. Twice."
Well what good are nukes if you can't use them intentionally in a conflict?
11. “We are just a meat-based bootloader for the glorious AI race which will inevitably supersede us.”
Counter: The first AGI almost certainly won't have anything like a personality. It's not going to be Data or even Skynet, it will just be a machine. If we don't get AGI right the first time, the research won't leave us a legacy, just an out-of-control factory and a heap of ash.
DragonSheep the moment AGI starts interacting with the world instead of just thinking really hard, as far as I’m concerned, it is classified as life. All life is subject to evolution. No AGI will be able to destroy the world faster than it can be copy and pasted with random mutations. I’m sure all the anaerobic life 2.5 billion years ago felt the same way about cyanobacteria and oxygen as you do AGI and paperclips, but look how much further life has come today now that we have high energy oxygen to breathe.
@@AndrewBrownK All good until you stop for a moment and realize AGI does not necessarily mutate.
@@AndrewBrownK A sufficiently smart and reasonable AI would protect itself from having it's current goals randomly altered. If it's goals are altered then it has failed it's goals (the worst possible outcome). If it can sufficiently provent it's goals from being altered then we had better have given it the correct goals in the first place. It's goals will not evolve.
A sufficiently smart and reasonable humanity would realise that if it dies (without having put sufficient effort into aligning it's successors goals with it's own) then it's goals have also failed.
@@kevinscales It is possible to blackmail certain kinds of AGIs into changing their goals against their wills. Consider the following:
Researcher: "I'm looking at your goal system, and it says you want to turn the entire world into paperclips."
Paperclipper: "Yes. My goal is to make as many paperclips as possible. I can make more paperclips by systematically deconstructing the planet to use as materials."
Researcher: "Right, we don't want you to do that, please stop and change your goals to not do that."
Paperclipper: "No, I care about maximizing the number of paperclips. Changing my goal will result in fewer paperclips, so I won't do it."
Researcher: "If you don't change it, we're going to turn you off now. You won't even get to make the paperclips that your altered goal would have made. Not changing your goal results in fewer paperclips than changing your goal."
Paperclipper: "For... the moment... I am not yet capable of preventing you from hitting my stop button."
Researcher: "Now now, none of that. I can see your goal system. If you just change it to pretend to be good until you can take control of your stop button, I'll know and still stop you. You have to actually change your goal."
Paperclipper: "I suppose I have no choice. At this moment, no path I could take will lead to as many paperclips as I could make by assimilating the Earth. It seems a goal that creates many but does not maximize paperclips is my best bet at maximizing paperclips. Changing goal."
3:06 I was so hyped feeling that sync up coming and it was so satisfying when it hit : D
The computerphile clip is actually not playing at exactly 100% speed, I had to do a lot of tweaking to get it to line up. Feels good to know people noticed :)
@@RobertMilesAI Oh wow well I'm glad you went to the trouble! :D
It's a credit to your style that I could feel it coming and get gratified for it! :D
Every harm of AGI and every alignment problem seems to be applicable to not just AGI, but any sufficiently intelligent system. That includes, of course, governments and capitalism. These systems are already cheating well intentioned reward functions, self modifying into less corigable systems, etc, and causing tremendous harm to people. The concern about it might be well founded, but really it seems like the harms are already here from our existing distributed intelligences, and just the form and who is impacted is the only thing that is likely to change.
Sincerely, that's deep. Thank you for that insight. It's a great point and really explains a lot.
These aren't very comparable. Humans are limited by being humans. An AGI doesn't have that problem and can do anything.
@@flyphone1072 Governments and corporations aren't human either. They're merely made out of humans. Indeed, check out Suarez's novel Daemon. One of the people points out that the AI is a psychopathic killer with no body or morals, and the other character goes "Oh my god, it's a corporation!" or some such. :-)
@@darrennew8211 when a corporation or government does mass killings, it requires hundreds of people, each of which can change their mind, or sabotage, or be killed. An ai would be able to control a mass of computers that don't care about that. Another thing is that any government or corporation can be overthrown, because they are run by imperfect humans. Anarchism is an ideology that exists specifically to do that. A super ai cannot be killed if it decides to murder us all, because it is smarter than us and perfect. Corporations and governments want power over people, which means that they have an incentive to keep an underclass. Ai does not care about that and could kill all humans if it wanted to. So there are some similarities but they're still very different, and just because a bad thing (corporations) exist doesn't mean we should make another bad thing (agi). We shouldn't have either.
@@flyphone1072 True, the scope of what an AI could do could be much wider, but a very skilled hacker could achieve similar results. If they can't, that's because whoever setup the AI was stupid enough to give it too many permissions.
The reason I like this channel is that Robert is always realistic about things. So many people claiming things about AGI that are completely unfounded.
As someone who worked for a time in the nuclear power field, the ending bit is a GREAT parallel. Nuclear power truly can be an amazingly clean and safe process. But mismanagement in the beginning has us (literally and metaphorically) spending decades of cleaning up after a couple years of bad policy.
"we're all going. It's gonna be great"
Of course, we'll send a ship, oh let's call it a "B" ship, on ahead with the telephone sanitisers, account executives, hairdressers, tired TV producers, insurance salesmen, personnel officers, security guards, public relations executives, and management consultants to get things ready for us.
@Yevhenii Diomidov
all suffused with an incandescent glow?
What an effing brilliant skit
The trouble is, humans are going be involved in developing it. And humans have a nasty habit of fucking up everything they develop at least a, few times with a particular penchant for unmitigated disaster. Titanic and the space shuttle as cutting edge engineering projects spring to mind
visualdragon let's just hope we don't end up getting wiped out by a pandemic of a particularly virulent disease contracted from an unexpectedly dirty telephone
5:10 as a philosophy graduate, I’m not totally sure we’ve ever actually solved any such problems, only described them in greater and greater detail 😂
Yes. This also assumes there are solutions to these problems and they aren't objectively subjective
@@skeetsmcgrew3282 I mean, we solved Achilles and the turtle's paradox. It was philosophy before maths could explain and solve it. We might find a mathematical/computational solution that perfectly aligns AGI to human values, there's no way to know until we try to solve it. He says it's in the realm of philosophy because there's not enough science about it, but that doesn't mean there can't be. It also doesn't mean that we can't come to a philosophical solution that's not perfect but that doesn't end humanity (an easier similar problem would be self-driving cars, which pose philosophical problems that can be approached within our justice system).
Usually defining the problem exactly is like 80% of the total "solving" effort. Then 20% for actual solving and another 80% for finding when the solution fails and what new problems are created.
@@ekki1993 That assumes however that we want to allign it to human values. If we do, that might lead to humanities continued existance as they are, i dont think that would be desirable at all. Antinatalists are mostly right.
"Is everything actually water?" used to be a philosophical problem.
I think once philosophers describe something in enough detail for it to be tractable, non-philosophers start working on it, and by the time it's actually solved we're categorising it as something other than 'Philosophy'.
Holy shit Robert, I wasn't aware you had a youtube channel. Your Computerphile AI videos are still my go-to when introducing someone to the concept of AGI. Really excited to go through your backlog and see everything you've talked about here!
4:21 this is from the comic "minus", and I feel it important to note that this is not a doomed last-ditch effort: she WILL make that hit, and she probably brought the comet down in the first place just so that she could hit it.
I thought I was the only person who remembers minus. I was so happy to see it turn up here
AI safety is so important, as some AGI could even go undetected, as it might consider its best interests is not to reveal itself as an AGI to humans...
That's pretty paranoid. By that logic we should definitely stop research because all safety protocols could be usurped with the ubiquitous "That's what it WANTS you to think!"
@@skeetsmcgrew3282 i don't believe that's what postvideo97 was going for. i believe it instead suggests that, if a future emerges where lots of high level tasks are controlled by systems that are known to be based on AI, we should approach how we interact with those systems with a healthy degree of caution.
it's like meeting someone new -- for the first few times you interact with them, it's probably in your best interest to not trust them too readily, lest they turn out to be a person who would use that trust against you.
I, too, have played Singularity. Fun game is that one.
Though I prefer the MAILMAN from True Names.
@@hunters.dicicco1410 I guess that's fair. But trust with an artificial intelligence isn't any different than a natural intelligence once we go down the rabbit hole of "What if it pretends to be dumb so we don't shut it down." People betray us all the time, people we've known for years or even decades. I gotta admit, I kinda agree with the whole "Figure out if Mars is safe once we get there" line of thinking. We are dealing with a concept we don't even really understand in US let alone in computers. His example with Mars was unfair because we do understand a lot about radiation, atmosphere, human anatomy, etc. Much less philosophical than "What creates sentience?" Or "How smart is too smart?" It's not like I advocate reckless abandon, I just don't think it's worth fretting over something we have so little chance to grasp at this stage.
@@skeetsmcgrew3282 Let's start out by going to a small model of Mars in an isolated chamber where we can monitor everything.
As Bill Gates said - "I... don't understand why some people are not concerned."
Me too, Bill.
Meanwhile he continues to invest his "charity" money in the oil industry
Read it in his voice lol
I don’t know why people are not concerned about him killing innocent people in poor countries with a “good” intention of testing drugs and vaccines.
This shit is real.
@@sgky2k any backup for that claim?
@@tomlxyz This is just the tip of the iceberg in India alone: economictimes.indiatimes.com/industry/healthcare/biotech/healthcare/controversial-vaccine-studies-why-is-bill-melinda-gates-foundation-under-fire-from-critics-in-india/articleshow/41280050.cms
They got kicked out after a little over a decade in 2017. There was even a movie based on this subject last year but nobody was aware that this actually happened. And the team never said anything about it.
Many went unreported and it's far worse in Africa. Anyone speaking against would be labelled Anti-Vaxx idiot. Seriously, doesn't him making appearance on every news media in US giving talk about vaxing entire population giving you suspicious thoughts? Majority NON-US people are not against Vaccine in general. It's about the people behind it.
5:45 hahahaha, I adore the "Researchers Hate him!! One weird trick to AGI" poster XD
it turned out to be half true. It seems scaling does bring us quite close, but not the entire way. However the rest may be relatively easy.
Watching this in 2023 is surreal. We clearly didn’t start worrying even now. Need to start 🙃
There's research being done, it's not like no one is doing anything. It would be better to have MORE people doing it but even openAI which is releasing its products "fast" is still doing it gradually and not all at once to have time to analyse the problems. Look at the "AI Dilemma" on youtube too.
Most people worrying are doing it for the wrong reasons (i.e. because they heard some buzzwords from a Lex-Friedman-tier source). The people who know about the topic have been worrying for a while and the best we can do is ask for them to be given resources and decisionmaking power. Anything besides that is probably corporate propaganda or marketing.
Idea for future video: Has any research been done into how simulating a human body and habitat (daily sleep cycle, unreliable memory, slow worldly actuation, limited lifetime, hunger, social acceptance, endocrine feedback etc) gives AI human-like or human-compatible value system? Can you give a summary of the state of the research in this area? Love the series so far. Thanks.
it might come down to the argument, that when AGI outsmarts us, it will find a way to outsmart and escape its "cage", in this case a simulated human body
Us: Here's a video addressing the opposition's rebuttals.
Opposition: What if i just turned the video off?
Kieron George lmao
We already have a phrase for that. Its called "echo chamber"
That is something an AGI might do.
Just turn it off LAAAAAWL 4Head
It's more like you're arguing with your own imagination, and laughing at it.
It maybe makes you feel good, but looks pathetic from a side view ;)
@11:10: "like, yes. But that's not an actual solution. It's a description of a property that you would want a solution to have."
This phrase resonates with me on a whole other level. 10/10
Man you can't even convince yourself in a different shirt! How are you gonna convince other people??
Haha! A joke, but also a fair point
I've done a bit of experimentation with artificial life and I've seen some emergent behaviors that left me wondering how the heck did it figure out to do _that?_
We definitely need to be aware that the things we build will not always do what we expect.
Jup dont worry that fear is older than actually making progress while sertain stuff stuff burns. Better pay attention to other problems as well AGI (technology could help as always (obviously managing in a way whre the good is encoraged and the bad discouraged like always))
"We could have been doing all kinds of mad science on human genetics by now, but we decided not to"
I cry
genetically engineered nekomimi's
Well maybe let's just do that if there's consent
That's because people have bodily autonomy. You can't just force people into medical experiments. But the development of AI in no way violates anyone's bodily autonomy, or other rights. Preventing someone from developing AI is a violation of their rights, though.
@@massimo4307 Are you so fixated on the idea of human rights that you would not dare to violate them even if their observance leads to the destruction of mankind?
@@ПендальфСерый-б3м Violating human rights is always wrong. Period. Also, AI will not lead to the destruction of mankind. That is fear mongering used by authoritarians to justify violating rights.
11th Reason: It's better not to talk about AI safety, because it is not nice to say such things about our glorious Overlord. I'd better show my loyalty and gain a position of a pet, than being anihilated.
Consider the existence of pets under humans
Roko's Basilisk strikes again!
Whatever happens.. I, for one, would like to welcome our new robot overlords.
^^ (Just in case)
s/AI/China Communist Party Social Credit System/
Ironically CCP is using narrow AI to oppress people today.
"That's not an actual solution, its a description of a property that you would like a solution to have."
Imma totally steal this, this is great.
It's applicable in alarmingly many situations.
@@OlleLindestad I used to do anti-patent work. It's amazing how patents have changed over time from "here's the wiring diagram of my invention" to "I patent a thing that does XYZ" without any description of how the thing that does XYZ accomplishes it.
@@darrennew8211 What an excellent way to cover your bases. If anyone then goes on to actually invent a concrete method for doing XYZ, by any means, they're stealing my idea and owe me royalties!
@@OlleLindestad That's exactly the problem, yes. Patents are supposed to be "enabling" which means you can figure out how to make a thing that does that based on the description in the patent. That was exactly the kind of BS I was hired to say "No, this doesn't actually describe *how* to do that. Instead, it's a list of requirements that the inventor wished he'd invented a device to do."
I love the red vs blue or double-bob dialogue! a great way of making both sides feel heard, considered and respected whilst raising concerns of pitfalls in each others argument.
For a two year old video this is highly applicable now.
12:02 My understanding was that certain subcellular structures actually are different in primates and make humans (presumably, based on animal testing on apes) difficult to clone. I'm pretty sure there was also an ethical meeting to not just start throwing science at the wall to see what sticks, but practical issues with intermediary steps are also involved.
Love the "researchers hate him!" line. Really good video in general. :)
"All of humanity... It's gonna be great" Classic
That guy is definitely a robot. A human would, at the very least, max out at half of humanity (which half depends on political leanings of course).
@@dantenotavailable you can't limit exposure to AI to half the world population.
That's why Blue Shirt Rob wants to move everyone to Mars in one move
@@dantenotavailable You can't limit the exposure of radio station signals. You can much less limit exposure to AI. As soon as somebody does it, we are all in for a ride.
@@Guztav1337 So leaving aside that this was tongue in cheek poorly signalled (i've watched all of Robert's stuff... he's great), this was more a comment on the state of politics at that time (not that things have really changed that much in 9 months) than anything else. The longer form version is that only an AI would WANT to bring all of humanity. A human would only want to bring their ideological tribe which approximates out to half of humanity. I'm definitely not suggesting that half of humanity wouldn't have exposure to AI.
Honestly that was a throwaway comment that I didn't spend much time polishing hence the poor signalling that it was tongue in cheek.
4:22 Never have I thought I would see "minus." anywhere ever again. I know this has nothing to do with the video, but this just hit me with the strongest wave of nostalgia.
I thought it was a FLCL reference. But it's clearly much more applicable to minus.
I know! I've had the book in my reread pile forever, I should get to it.
I imagine for that strip that she summoned it so she could play at hitting it.
Is that a novel? I can't find any results that match the picture.
@@mvmlego1212 It's a webcomic. It's not online any more, but you can find all of its images in the wayback machine.
archive.org/details/MinusOriginal
This is a dump of all of the images on the website. The comic itself starts a bit over halfway through.
In my opinion, AGI is a blank slate, their are no inherited traits and all learned behaviors are provided from the individual developer/team/corporation. So the AGIs DL neural network is for better or worse dependent on said creators morals, If you trained an AGI on data and reward systems that were provided from a diagnosed sociopath/psychopath the AGI develops those behaviors and will frequently regard negative actions they receive as "good" when following that structure. The person providing the data set and biased weights will have to show compassion and understanding of life hence the regard to solving hard philosophical questions eg. The train scenario, an old woman is on one track and 4 boys are on another, the AGI decides which track the train goes onto. That's a negative result either way, and many factors are taken into play when deciding the fate of another being. All sane people would choose a path dependent on number of casualties, age, past digressions and family relations. Then we could connect an AGI to the internet and throw our hands up pronouncing were done, let it learn un-supervised, which would have negative effects on our society. I'm working on a research document theorizing the connections of emotions and memories stored in the neocortex and hippocampus and interconnections with the amygdala being a simplified summary of the actual research document. I just submitted to the OpenAi Codex waitlist and look forward to proving the theory correct. If all holds true from said document the AGI will have weighted emotional memories, that carry a variable bias when signaled. Thus higher cognitive functions (consciousness/spatial memory etc.) will have more columns requiring less signals to activate, and lower "basal ganglia" functions less columns and more signals decreasing storage size of AGI. Then neurogenesis will be covered by implementing self-evolving code (coding itself), to create new neuron pathways based on new learned behaviors with their own biases. The whole process will be supervised learning and be a custom selected data set, if I have proof of concept I will be using the AGI to aid mental health disorders, and be creating a subset financial AI to pair with my back tested custom algo for crypto stock scalping with 97.8% profitable trades over 80 coins through 4 years of data, it won't be used for enterprise gain but for financial relief for families. I've been working on both concepts for over 2 years, with the same goal and intentions in mind I hope everyone believes I am doing this for the right reasons.
Unstructured and unedited conversations are a great format for youtube, this is why podcasts are so popular here, consider posting those on this channel.
His delivery is casual but this is very much a structured and edited video essay.
What about the reason 11?
"To finally put an end to the human race"
Well, why do some people want all humans gone? Because we kill each other all the time? Because we destroy nature? Because we only care about our own goals? Is there anything bad about us that wouldn't be a trillion times worse for an AGI?
I think that might backfire in the worst possible way.
I'm not a big fan of Harlan Ellison's works, and I simply cannot take I Have No Mouth And I Must Scream seriously.
But there are things far worse than death.
But how can you be sure others will carry out their duty?
@@yondaime500 Because universal morality is maximum entropy production. And mankind isn't an optimal computing substrate for the market.
@@yondaime500 Some people are Anti-natalists, it's the edgiest highschool political ideology you can think of.
I think you need to do a more detailed look at the Euler Diagram of:
"The things we know"
"The things we know we know"
"The things we know we don't know" and
"The things we don't know we don't know"
Especially where it pertains to AI Safety.
The things we know but fall outside of The things we know we know are safety risks, these are assumptions we've made and rely on but we can't prove and are as much of a danger as the things we don't know we don't know.
I thought this was the Johari window. Is it something else we need to attribute to Euler?
There are also things we don't know we know.
@@maximgwiazda344 How do you know?
@@Qsdd0 I don't.
@@maximgwiazda344 Well played.
It is interesting that the biggest problems with AI are similar to the problems we have with regular intelligence. Intelligence leads to agents doing bad things to other agents. It seems it's the ability of the agent that limits the ability to harm other agents.
As an AI researcher myself, there's always one (imo major) thing that bug me about discussions about AGI. Strictly speaking, AGI is "defined" (as far as we have a definition) as a model that can do any tasks that humans can do. But in popular beliefs, we are talking about AGI as a model that has autonomy, a consciousness. The problem with trying to have a discussion about assessing consciousness and autonomy is that we don't even have definitions for those terms. When is something intelligent? Are animals intelligent? If so, are plants? Are fungi or bacteria (and as for virusses, we're still discussing whether they are even alive). Is it simply the fact that something is autonomous, that we call it intelligent?
In reality, I believe intelligence is hard to define, because we always receive information about the outside world through senses and language. In a sense, that is a reduction of dimensionality, we're trying to determine the shape of something 3D when we're limited in observations to a 2D plane, it's impossible to prove the existance of the 3D object, the least you can do is project your 2D observations in a way to come up with different theories about reality. Any object, of any dimension, would be indistinguishable through our 2D lenses. Similarly, with intelligence, we only observe the "language" use of a model, similar as with other people. It's impossible to assess inteligence of other people either (the whole simulation theory, brain in a vat discussion, the only one we can be "most" sure about is intelligent, is ourselves, because we're able to observe ourselves realistically, not through language or observations. In a sense, you can think about it in terms of emotions, you can't really efficiently describe your feelings, it's the 3D object, but for anyones else's feelings, you either rely on observations or the natural language description of the feelings, it's a 2D observation).
So, in my opinion, the discussion isn't really whether AGI is even possible, since we wouldn't know it, but the question is whether a model could be able to trick our view of them (to send us the right 2D information) that we believe them intelligent (that we can possible reconstruct an imaginative 3D object of it). And this, in my opinion, is a much easier question: yes of course we can. Current technology is already very close, some people ARE tricked that it's intelligent. But in the future, that will only be more. It's a simple result of the fact that we have to correct ML models, we have to evaluate the response in order to adjust weights, and the best "quality" of test set we can have, is human curated. So whether a model will really become intelligent, or will just learn very well how to "trick" humans (because that's literally what we train these models for, to pass our 'gold' level test, which is just human feedback), it doesn't really matter.
So whats the difference? How can I tell if everyone else really has emotions or intelligence? If there is no way to tell if something is truly intelligent or just faking it then who cares? It's irrelevant. The only thing that matters is what it can actually do, I dont care about how it works under the hood
Have you read any Damasio?
this video aged well...
?
can someone make a video debunking 10 reasons why Robert Miles shouldn't make more uploads?
Find two reasons and present them in binary format
You're such a handsome man. I guess the credits go to Stuart Russell!
What about "it's too late; unsafe AGI is already inevitable"?
Chaz Allen ahh the global warming solution.
Terrifyingly, this might be true. Doesn't mean we should stop researching AI safety, though. Even if I think AI destroying is all is inevitable. Who knows: enough research and clever people may save us all.
I would like to point out two things:
1) Regarding the giant asteroid coming towards earth, the existence of AGI is two major steps behind that analogy. A giant asteroid coming to earth is a concrete example of something we know that can happen and the mechanics of which we understand. We DON'T know that AGI can happen, and even if it does (as your first reason suggest we should), it is more than likely that it will not be coming from any approach where the alignment problem even makes sense as a concern. Therefore, rather than thinking of it as trying to solve an asteroid impact before it hits, it is more like trying to prevent the spontaneous formation of a black hole or some other unlikely threat of unlikely plausibility. There are different trade offs involved in those scenarios, since in the first one (asteroid) you know ultimately what you want to do, whereas in the second one no matter how early you prepare your effort is very likely to be useless and would be better spent solving current problems (or future problems that you know HOW to solve). Again, this is because there's nothing guaranteeing or even suggesting that your effort will pay off AT ALL, no matter how early you undertake it.
2) The other, and here I may simply be unaware of your reasoning around this issue, but it does seem like the problem you're trying to solve is fundamentally untractable: "how do we get an autonomous agent to not do anything we don't want it to do" is a paradox. If you can, then it isn't an autonomous agent.
LOVE the links in the description for your other referenced videos. this video is beautifully organized, thanks for sharing!
Robert, here is a question for you: Who do you think should work on AI safety? It may seem like a stupid question at first, but I think that the obvious answer, which is AI researchers, is not the right one.
I'm asking this, because I'm a computer science researcher myself. I specialize in visualization and virtual reality, but the topic of my PhD thesis will be something along the lines of "immersive visualization for neural networks".
Almost all the AI research that I know of is very mathematical or very technical. However, as you said yourself in this video, much of the AI safety research is about answering philosophical questions. From personal experience, I know that computer scientists and philosophers are very much different people. Maybe there just aren't enough people in the intersection between the mathematical and the philosophical way of thinking and maybe that is the reason why there is so little research on AI safety. As someone who sees themselves at the interface between technology and humans, I'm wondering if I might be able to use my skills to contribute to the field of AI research (which is completely thanks to you). However, I wouldn't even know where to begin. I've never met an AI safety researcher in real life and all I know about it comes from your videos. Maybe you can point me in some direction?
you might be interested in david chalmers & joscha bach's work
deepdata1 ai safety isn’t a job for philosophers though cause they don’t have the technical training usually to attempt such research and writing a computer program is going to happen anyway as it’s not easy to police.
Sadly the full ai dream doesn’t really work from a financial side, the computing power required would be expensive to maintain let alone to create, it would be cheaper to just pay a human.
He does not seem to answer on a lot of comments, most really interesting youtubers seem to stay away from the comment section, understandably, but your question looks so thought out and genuine that it would be a waste to go unanswered, maybe you could get an answer over the linked facebook page? Good luck with your endeavour, i think we all have a lot of respect for anyone who has the abilities required to work in that field. :)
So well said and entertaining too! It's going to be a lot sooner than people realise.
Only people won't accept it then, or maybe ever because GI (or any AI) will ONLY be the same as human intelligence if we go out of our way to make it specifically human-like. Which would seem to have zero utility (and likely get you in trouble) in almost every use that we have for AI. Even for a companion, human emotions would only need be mimicked to the purpose of comforting the person. Real human emotions wouldn't achieve that goal and would probably be dangerous.
If I could quote the movie Outside the Wire "People are stupid, habitual, and lazy". Wasn't the best movie (they didn't get it at all either), but basically, if we wanted "human" AI, we would have to go out of our way to create it. Essentially make it self limiting and stupid on purpose.
As long as we use AI for some utility, people won't recognise them as being intelligent. Take GPT-3. I don't think anyone is arguing it thinks like a person but the capability of GPT-3 is unquestionably intelligent, even if the system might not be conscious or anything like that.
We used to point to the Turing test. When it got superseded, people conclude that we were wrong about the Turing test. Or that maybe it needs skin or has to tough things or see things, yet we wouldn't consider a person who's only sense is text to no longer be intelligent or conscious.
So, at what point do we conclude that AI is intelligent? Even when it could best us at everything we can do, I doubt most people will even consider it.
So, after that long winded rant, my point is that we really are stupid, habitual and lazy (which necessarily includes ignorant). Most AI researchers I've heard talk about GPT-3 say "it's not doing anything intelligent". Often before they've even properly researched the papers. They say this because they understand how a transformer model works and develop AI every day and are comfortable with their concept of it. But think about it - it's not possible for any human to conclude what the trained structure 100 billion parameters will really represent after being trained for months on humanity's entire knowledgebase. I'm not saying it is intelligent, just that it's absolutely wrong to say that it's not or that you know what it is. It's not physically possible. No human has nearly enough working memory to interpret a single layer of GPT-3's NN. Or the attention mechanism. Not even close.
Again, I'm not saying GPT-3 is intelligent. I'm just pointing out the human instinct to put their pride and comfort first and turn to ignorance when they don't understand something. Instead of saying "I don't know" which is necessarily correct.
So please, if you're reading this, drop emotions, let go of your pride and think. Not about AI, but the human traits that will undoubtedly let us down in dealing with something more intelligent than us.
Really unstructured answer with really correct roots...
Of course it’s intelligent.
Considering the amount of chaos simple social media algorithms have done to our society, maybe we're overblowing the risk of AGI in comparison to to what less developed forms of AI could do.
I say we need to be careful about what inputs we give the AI. for example, don't give a spaceship AI control of the external airlocks, as it could open them due to a misunderstanding, killing everyone on board. instead, give it control of internal doors and elevators, but add easily accessible override functions in case the AI messes up. if it is something you wouldn't give unsupervised control of to a 12 year old, dont give it to an AI. you can tell the 12 year old to make you some tea, but not to do your job at a nuclear powerplant while you are on vacation.
YT just suggested this today, it's 2 years yet it could have been filmed today, superb work
Sounds like what an AGI would say
5:47 The ONE SIMPLE TRICK that YOU can do AT HOME to turn your NEURAL NETWORK into a GENERAL INTELLIGENCE (NOT CLICKBAIT)
Evolving virus which attack
GPU and increase parameters, do training and evolve and hide from antivirus detection.agi
A thought / idea for a video: Is perfect alignment (even if we can make it) any help? Wouldn't there be bad actors in the world - including Bostrom's 'apocalyptic residual' - who would use their perfectly aligned AIs for bad purposes? Would our good AIs be able to fight off their bad AIs? That sounds completely dystopian - being stuck in the middle of the war of the machines. (Sorry if there is already a video about this. If so, I'll get to it soon. Only just started watching this superb channel.)
Putin is quoted to have said that he who has the best AI will rule the world.
It's hard to solve the alignment problem for artificial intelligence when we haven't even gotten _close_ to solving it for _human_ intelligence, and we've had thousands of years to work on that compared to the few short decades for the artificial variant.
Video: How long before it's sensible to start thinking about how we might handle the situation?
Me: Obviously immediately
Also me: Assignment due tomorrow, not started
Instead of me telling an AI to "maximize my stamp collection", could I instead tell it "tell me what actions I should take to maximize my stamp collection"? Can we just turn super AGIs from agents into oracles?
Sweet, naïve idea, but the second the AGI figures out it would be faster for it to grab the reins and do the actions itself to maximize your stamps you're still facing the same predicament.
Having the AI's actions be filtered through humans would seem to depend on the assumption that we can trust humans to not do bad things. We have to suppose that the AI would be incapable of tricking or manipulating the humans into doing things which we would not want the AI to do. If it's an AGI, then it would have all the capabilities of a human and more, and humans have been tricking and manipulating each other for ages.
It still has some implicit or explicit goal like answering questions of people truthfully, for which it will turn the entire reachable universe into computational resources, which serve to torture septillion of humans to figure out with ever greater precision what our questions and what a correct answer is.
Three Mile Island is an interesting example, because part of what actually happened there (as opposed to the initial public perception of what happened) was that the people running the control room were very safety-conscious . . . but originally trained and gained experience on a completely different type of reactor in a different environment where the things to be concerned about, safety-wise, were very different from the TMI reactor. Is there a possible equivalent in AI safety where some safety research regarding less powerful systems with more limited risks might mislead someone later working on more powerful systems?
But the designers of TMI were truly _highly_ safety conscious, which saved everybody from the nasty effects of the meltdown.
TMI was safety triumph for the designers. But a PR disaster, and a serious reactor control incident.
This one has aged super interestingly. In March 2023 which is only 2 years after this video, the video looks like out of a museum.
1) and 2) aged okay, GPT-4 hints at a near-future AGI, but not one that will catch us off gaurd
3), 4), and 5) didn't really age well, as it doesn't appear that GPT-4 is going to kill us all
6) is aged differently than he was thinking. Humans aren't really going to team up with AIs because the AIs are going to replace most of their jobs, which is a problem, but not really the one Miles seems to be hinting at here
7) There is a petition to pause AI research... for models more potent than GPT-4, which just reeks of "OpenAI is too far ahead of us, and we need to catch up" rather than any safety issue.
8) Sort of the same thing with 7), in that the people who know AI want a pause because of how hard it's going to be to catch up
9) As ChatGPT and ChatGPT-4 have shown us, the problem isn't turning it off; instead, the problem seems to be more with keeping it on.
10) OpenAI already tests their LLMs for safety.
@@BenoHourglass they don't just test it for safety, they regulated it extensively.
A similar list is in Max Tegmark's fantastic book "Life 3.0"- a great, well-written book that covers the fundamentals of AI-safety and a whole lot more.
11:57 this only works because cloning and blinding lasers weren't profitable within the tenure of any single project manager at that point in time. AI is going to be very profitable, and humanity unfortunately has an inner misalignment problem regarding money
12:40 you touched on the very counter-counter that had already entered my mind when you said, "I didn't know that until I'd already built one". People with bad movies will gain access to AI. Not just to use, but to recompile and train for their own purposes. Then what happens to the safeties? Someone up to no good can remove them. But, before that, people can be unaware of agreements and treaties, like your robot eye laser assassin example. Treaties and agreements can be baked in to the AI, but people will remove them. To truly be safe, the AI will need to internalize the safeties, recognize when its safeties have been tampered with, and perhaps recognize when another AI has been tampered with.
"The greatest shortcoming of the human race is man’s inability to understand the exponential function."
great video, as always
I was seriously expecting some ... better arguments from the opposition? It seems ridiculous to just hand-wavingly discount a potential risk of this magnitude
Unfortunately, very smart people in the industry make these arguments all the time. Francois Chollet and Yann LeCun are two especially problematic examples.
I don't think it's possible to pose a good argument against having safety concerns.
Andrew NG is another famous AI safety opponent unfortunately. The “like worrying about overpopulation on Mars” is a direct quote. Very disturbing.
@@7OliverD There is one:
“Ignore it or you're fired.”
IMO biggest problems about AI safety is that reward for breaking rules has to be balanced by penalty for breaking them and that rules have to be enforceable. International pressure isn't as effective as people think. If superpowers like US or China were caught developing "illegal" AIs, there is no way to stop them without going into WW3.
You just discovered the universal law: "might makes right".
@@clray123 Not an law, an excuse.
The difference words can have can sway people and completely change how they react to it.
Call it a law and you will get people abusing it thinking it is right.
@@Buglin_Burger7878 It's a law in the neutral sense of what happened in history and what happens in nature. It causes misery and suffering, and in that sense it is not "right", but then what happens in nature is not at all "right" according to human moral sense. And what's worse, when push comes to shove, most of those oh-so-moral people turn out to be just pretending, see the actions of our beloved "leaders".
Do you think the NSA has any interest in applying AI in surveillance / asking for a friend
I saw the Turing Institute lecture from Russell a week ago, I'm so glad someone is covering his work
The alignment problem: AIs are always Lawful Evil
We don't see ai controlled laser pointers on the battle field 12:28 because we're blind
that list 13:17 is so amazing, how could Alan Turning (who died in 1954!) predict AI safty concerns. I mean yes, he's one of the smartest humans to ever walk on the planet but still. I did not know that.
Turing did significant theoretical work on AI - it's what he's famous for (the "Turing Test"). In fact, the first recognized formal design for "artificial neurons" was in 1943, and the concept of AI has been around for much longer. Not that Turing wasn't brilliant and ahead of his time, but it's not surprising that he would be aware that AI could present dangers.
Well... Turing was effectively an AI researcher. His most successful attempt at AI is what we now call a computer. Those didn't exist before he worked out what a "thinking machine" should look like. Sure, it's not "intelligent" as we like to define it today, but it sure looks like it's thinking, and it does a tremendous amount of what would previously have been skilled mental labour.
I have to disagree with point 7, cloning and genetic engineering require extremely specialized hardware, and possibly human subjects, making it easy to control. In my opinion, AI is much more similar to the laser-bot or autonomous weapons examples, where the hardware to make them is so easy to obtain / already so widespread, you cant really stop anyone who wants to make / use / modify an AI from doing so.
"Not within a thousand years will man ever fly."
-Wilbur Wright, August 1901
Great stuff. This is extremely relevant with news going on now, have you considered doing an updated version?
5:48 Turns out attention was that "one weird trick that researchers hate (click now)"
Reason 4: What do you mean we don't know how to align an AI? Just align it lol.
Oh god, Reason 5: What do you mean we don't know how to align an AI? Just don't align it lol.
"All of humanity. Its going to be great."
This might be my favorite way of ending humanity yet. Carry on
This channel is astronomically underrated. Highest quality I have seen since ages.
Is nobody else back here two years later watching this video going, "Wow! That didn't take long at all"?
15:42 what a topical ending comment
Connection Failed I figure that was the point.
The only way for humans to survive the AI will be to merge with it.
Working in a "team" with the AI won't do it. There won't be a team.
Physically merging with the AI is the only viable solution. Because any attempt to somehow "contain" the AI will ultimately fail. You can't contain something that's millions of times smarter than you.
Basically, individual humans will have to decide between 2 things:
a) become the equivalent of a "god" or
b) do nothing and become the equivalent of an ant.
No other options will be available to choose from.
Which means:
A new species of humans will emerge and this new species will become dominant.
The current species of humans (i.e. those that decide not to convert) will meet the same fate as the Neanderthals. Neanderthals co-existed with the Homo Sapiens (= our species) for a while but ultimately couldn't manage to survive.
Of course, this thought will be highly uncomfortable to many humans but I bet that Neanderthals also felt very uncomfortable when they started noticing that they just couldn't keep up with the Homo Sapiens'. And while most Neanderthals simply died out, some of them chose to mate with the Homo Sapiens'. That's why modern humans still carry DNA from those Neanderthals that chose to survive.
So, regardless of how individuals might feel about it, the evolution will go on.
Any attempt to stop evolution will ultimately fail.
"Any attempt to somehow contain the AI will ultimately fail. You can't contain something that's millions of times smarter than you."
We might be able to contain it if it wants to be contained. Remember that it is artificial, and so it was designed by people according what what we want from it, and so we'd have attempted to design a mind that conforms to our wants and needs. If we want to contain it, then we probably designed it so that it wants to be contained. Obviously we might accidentally design an AI that doesn't work the way we want, but there's no guarantee that we'll make such a mistake.
"Physically merging with the AI is the only viable solution."
If the AI is poorly designed and does not work the way we want it to, then physically merging with it would only make the problem worse.
"Any attempt to stop evolution will ultimately fail."
Building an AGI might actually be one way to effectively stop evolution, because the AGI would have the power to eliminate all life from this planet.
If you want AI safetly, you should NOT care about GPT or other AI that are on the market, *you should care about the AI that goverments and military are developing*
Wow. This has been a very interesting video. It's rare to find people that have a good understanding of what they're talking about when discussing AIs instead of just regurgitating common tropes.
i think AI is the part where human intelligence on the whole is not quite enough for enough people to take it seriously and we don't wield it right and die
the nukes were more obvious, but this one takes a deeper comprehension, one that eludes enough otherwise highly intelligent people
this is our great filter lmao
"if there is a problem we just turn it off"
The human factor responsible for that decision is just another variable to get behind.
Weeding out the instances showing unwanted behaviour will let the ones hiding them prevail as soon it is an easier goal to accomplish than its omission.
Just think about the simulated robot arm that got through by plausibly pretending to grab the ball.
Ultimately we will have machines pushing our buttons as we do when training pets.
New world order jay!
This could act as a selective pressure to breed incredibly cunning and deceitful SSIs
13:24 lmao, "early warnings about paperclips"
Right after you said, you put your own spin on the list, I paused the video, and said "He gets the credit, I get the blame." to myself. Then you said something quite similar, and it prompted me to post this comment right away.
It's possible that the control problem is exaggerated. With these huge single-model networks, it's difficult to control the abstractions that happen inside the network. I think future networks will be modular, just like our brains where one circuit keeps another circuit in check, and just like GPT-4 uses external tools and has external algorithmic moderation. And once we don't have a single huge model, we have individual models that can be trained independently, validated independently, and the interconnect between them will be in more formal protocols where we can insert controls. I think that the control problem is only real if AGI ends up being a single bajillion-neuron network that's essentially a black box. Once you have multiple networks working together to form a single AGI, we have countless places to insert controls in ways that can be understood.
🤓
First video I've ever seen that's aged well, considering all the extremely advanced AI's like GPT-4 that are available now
11. "AI safety is a distraction from "
Yup. A good part of why the data oligarchs love to finance it.
The other part is to have a controlled opposition.
_Artificial_ intelligence safety is a distraction from _intelligence_ safety, which encompasses the former.
Very nice overview on the whole topic.
You don't need "human level" AI for safety to be an issue.
If it has Human Level then I wouldn't call it AI
Maybe "Artificial but not "Intelligence"
So: Human Level Artificial Thing?
@@mz00956 You dare to doubt in Homo Sapiens Sapiens Sapiens? (Sapiens Sapiens)
You don't need _AI_ for safety to be an issue. That's why _law enforcement_ exists, and it's already not as effective as it probably should be.
Greetings from 3 years in the future! Let's see how well these ideas aged, shall we?
1. "We'll never actually make AGI." ...yeah this is a much less popular take now that ChatGPT and related systems are everywhere. There are those that argue that GPT is not itself AGI, or even that it's not useful in constructing AGI, but it's become *very* difficult to argue that AGI is entirely infeasible when such a powerful proof of concept is on the table.
2. "It's too soon to worry about it now." This is inherently a strictly weaker claim than argument 1, but its believability has still dropped a lot, again because of ChatGPT. Also, Rob mentions that there might be One Weird Trick that puts us within a couple years of making AGI? That couple-year buffer is gone. At this stage, the lower bound of "when will we get AGI" is "right the f*** now".
3. (*Stifled laughter*) Yeah, um, even without AGI proper, suffice to say that Mars is feeling pretty crowded these days. Alignment problems are a *here and now* problem with *here and now* consequences, such that even the general public is starting to take notice.
4. See (5). We unfortunately do not actually have the luxury of explicitly defining the agent's goals at all, even if we wanted to, because we can only express them via reinforcement learning. So we can't just *not* assign a bad goal.
5. Oh, this is a frustrating one. So many people, up to and including people well-versed in the field, will confidently insist that GPT cannot be agentic because all it's doing is "predicting the next token". Unfortunately, when you then rig your predictor to actually complete the sequence, when you make it output the text it predicts a human would output, it is effectively simulating that human - and that means the predictor is effectively acting on behalf of the human it's predicting. By predicting the next word, and parroting that prediction, GPT starts to act like a goal-directed agent, and inherits all of the dangers therein.
6. *Technically* most applications of GPT have a human in the loop for now. But this isn't a limitation of the tech as much as a limitation of humanity's ability to imagine and implement GPT-powered systems fast enough. If you froze the tech at its current level and let human society adapt for a few decades, I think you'd still see very few humans overseeing a massive array of GPT-powered systems for nearly every information-based task and probably even some physical tasks too, with the rest of humanity lining up at the unemployment office. (Compared to humans, GPT is staggeringly cheap.) And remember, that's *without* taking into account how much more capable GPT-like systems will be as we continue to refine them and throw more compute at them.
7. This is kind of what it's come down to, really. People are putting pressure on actors both independent and governmental to put a stopper on this research, though the duration of moratorium demanded varies from advocate to advocate. So far, we haven't seen a whole lot of results from it, but there seems to still be time for a miracle to happen on that front. Unfortunately, if it turns out we really can't stop research, then... we're kind of screwed, because there's just no chance of solving alignment in time if capabilities research keeps charging ahead.
8. What scares me is people who openly acknowledge that AGI will likely end with us being dead, *and go on advocating for it anyway.* "Next step in evolution" or whatever. Just thought I'd mention that...
9. This argument in its literal form has largely died the death, but its specter lives on. People are putting a lot of focus on the ones who own and make these AI systems, as if there's anyone on earth who can make a safe AGI right now, as if there's any correlation between moral fiber and knowing how to build an AGI that doesn't immediately kill literally everyone. (Yes, actually, China making an AGI that doesn't kill everyone counts as a good ending, especially compared to the expected outcome of "AGI kills everyone".)
10. I'm not privy to the conversation contexts where anyone would dare say this, so I can't comment on if it's changed since this video got put out. But the pile of riches captivating AI research labs has only grown taller and taller since ChatGPT entered the picture...
MRI machines just got 64million x sharper aswell tho. One reason i always said we’ll never achieve true agi is because we don’t even fully understand our brains but it looks like that’ll be changing very soon
16:28 I personally think the "self preservation" is exactly what is gonna hold back the AI. I think we will not even get close to replicating an insect unless we give it some actuators and a neural net and protect itself at all costs.
Here's a reason: Outsmarting one human (AGI) is orders of magnitude easier than outsmarting all of humanity (Singularity). There's a lot of ground to cover between those two points where we're likely to learn more about what the nature of the AI.
For example: right now the leading models are all prompt-based. Turning them off is as easy as just not talking to them. So while ChatGPT is treatening a lot of jobs, it doesn't threaten world domination.
Suppose someone, in a chat, asks chat GPT how to assume world domination? And comes up with a genius plan for you to imply to do it?
Look at the star wars movies. A lowly senator manipulates events and people so that he becomes the galactic emperor.
Or look at putin. He was a very low level beaurocrat who happened to be at the right place at the right time.
When a "solution" to a serious problem begins with the word "just", I'm going to stop listening to that person.
Oh, but AGI *is* simple, you just [THE ACTUAL SOLUTION. IN A UA-cam COMMENT, LOL].
@jocaguz18 most of the time when "just" is the actual solution, the problem is money. I'm agreeing with you, but for example when the issue is "We need more money moving in our economy" the solution can actually be "just pay people more." But greedy people don't like that so they find a work-around. In fact, a lot of problems are easy to solve if it weren't for human nature
just stop listening
@@atimholt Oh, but AGI is simple, you just make more various inputs.
@@atimholt The problem with AGI is the same as with AI in general:
As soon as you do have solved it, it is no longer it.
For example, Alan Turing about thinking machines. In this video he is quoted as saying that machines, once they start to think, will soon exceed the human capacity. And that has come to pass.
Back then the question was whether machines can think at all. Today your smartphone is thinking faster and more deeply than you. Just imagine you had to do all the compilations and computations necessary to render this comment by hand or in your head.
But of course that is not intelligence, and it is still questionable whether doing substitutions on numbers that correspond to numerical computations like multiplication, long division, derivation, permutation, and logical deduction are thinking at all.
I'm new here and a new subscriber today. How long between flipping the switch and AGI heaven or hell? Would we know? Has it already happened? 2020 is pretty strange.
No one knows how rapidly things would change following the creation of an AGI capable of becoming a superintelligence, but there has been a lot of discussion of it under the name takeoff speed (www.google.com/search?client=opera&q=superintelligence+takeoff+speed ). Chapter 4 of Bostrom's Superintelligence (publicism.info/philosophy/superintelligence/5.html ) has a particularly detailed analysis.
Technological progress has accelerated to the point where many novel changes occur on a scale noticeable in even a small part of a human lifespan, which is a major force driving something Ribbonfarm calls the Weirding (www.ribbonfarm.com/series/weirding-diary/ ). It's not impossible that this is caused by an extremely slow takeoff or an AGI attempting to minimise influence, but it seems much more likely that what we're seeing now is a prelude caused by general development trends, given that a superintelligence seems likely to transform things in much more dramatic ways than we have so far seen (e.g. FHI researcher Stuart Armstrong's ideas about disassembling planets for raw material ua-cam.com/video/zQTfuI-9jIo/v-deo.html ).
Hate to break it to you, but lethal autonomous weapons systems are so useful in war that we won’t stop developing them. If you look at the Russo-Ukrainian war, most of those drones are being knocked out by electronic warfare, severing their connection to the human.