@@jeremywvarietyofviewpoints3104 There are plenty of professional cosplayers that aren't familiar with their characters. But that's mainly because they cosplay many characters.
One can always feel confident in your own intelligence but once you get comfortable with how much you think you know that's just a recipe for disaster...
He's obviously the best at designing CPUs , but Expertise and intelligence are different things. I'm pretty sure experts in other fields would disagree with some of his opinions.
@@kaba_me No. He doesn't merely have knowledge, he is clearly highly intelligent as far as I can tell. Using abstract concepts to solve complex problems is not merely a matter of having knowledge
LMAO! XD Node is pretty fast, the problem is the Golden Hammer law, Node was born as a server-side tool for building non-blocking custom services. Today is pretty much shoved everywhere, transpilers, syntax checkers, desktop apps (electron), game-engines, which is usually what happens when a thing becomes popular.
@@SwaggySolidarity So? Doesn't require eye contact. He's just running a grappling algorithm,.. "I opponent leg leg goes there, put right leg there,..." etc, etc. The chap clearly has a completely "spectrum" typical aversion to eye contact, with forced token engagement with his interlocutors.
Lex was right though, a Convolutional Neural Network (not to be confused with just convolution) is a search. The CNN, like all searches, is trying to to find a given pattern with within the confines of a grouping of data. The only difference between it and CNN search is that it breaks the main search up into small functions that look for traits and features and each search returns a probability of discovery. This result is then used in the next search to hone in on the least inaccurate guess as to whether or not the object was found. A CNN is still a search though.
Man the level of intelligence this man is so high it made him so humble, like what I do might not even matter as much in the grand scheme of things. 👏🏻
Great sign of intelligence on both sides to disagree and then agree again within a few minutes. No need to even finish most of the thoughts, just dance through it in seconds.
May's Law states, in reference to Moore's Law: Software efficiency halves every 18 months, compensating Moore's Law. David May (born 24 February 1951) is a British computer scientist. He is a Professor in the Department of Computer Science at the University of Bristol and founder of XMOS Semiconductor, serving until February 2014 as the chief technology officer. May was lead architect for the transputer. As of 2017, he holds 56 patents, all in microprocessors and multi-processing. When Inmos was formed in 1978, May joined to work on microcomputer architecture, becoming lead architect of the transputer and designer of the associated programming language Occam. This extended his earlier work and was also influenced by Tony Hoare, who was at the time working on CSP and acting as a consultant to Inmos. The prototype of the transputer was called the Simple 42 and was completed in 1982. The first production transputers, the T212 and T414, followed in 1985; the T800 floating point transputer in 1987. May initiated the design of one of the first VLSI packet switches, the C104, together with the communications system of the T9000 transputer. Working closely with Tony Hoare and the Programming Research Group at Oxford University, May introduced formal verification techniques into the design of the T800 floating point unit and the T9000 transputer. These were some of the earliest uses of formal verification in microprocessor design, involving specifications, correctness preserving transformations and model checking, giving rise to the initial version of the FDR checker developed at Oxford. In 1995, May joined the University of Bristol as a professor of computer science. He was head of the computer science department from 1995 to 2006. He continues to be a professor at Bristol while supporting XMOS, a University spin-out he co-founded in 2005. Before XMOS he was involved in Picochip, where he wrote the original instruction set.
I listened to the video, I was quiet impressed. Not just a chip architect. Loved the discussion of levels of abstraction among other things. But I disagree with 2 points. (I'm an algorithms guy, my ph.d. is in operations research). a) as Lex said, there's a lot of complexity in not just human behavior, but time-varying road conditions. We're talking the Curse of Dimensionality here and it's not clear to me that a brute-force camera-data-only training approach converges on an answer. Rain or hail or sleet can change the driving conditions (traction) quantitatively within 10 seconds. Shade or a bank or a tunnel inlet/outlet can have very different traction. Static and dynamic friction. If we're talking L5 or even L4 capabilities, to my mind that doesn't mean idealized Arizona-driving conditions. It means L4 capability everywhere on Earth. And every video I have seen if full of people making wild predictions while ignoring just how dynamic and chaotic driving can be. Like 60 mph blizzard conditions with chains, one of which was accidentally cut in the wrong place. Was that in the data set? Maybe it will be eventually. But we're very far away. (My adviser says we'll never get there, which is an interesting statement. I disagree. Then again he's one of the smartest people I've ever met). n! is nasty if n is the number of possible objects in the world. b) I actually found while optimizing Matlab code that there are many situations in which a Search problem can be formulated as a maximization problem. And many times it's actually faster within such a scripting language. But my point is that there's definitely a relationship. E.g. "Search for an inflection point" equates to "argmin_x d/dt F(x)". It's a min versus argmin or max versus argmax type of distinction. Highly related. And they must needs have the same Big-O. c) There's a massive conservation of power constraint which is presently ignored in how people think about "AI". Intelligence means not being able to solve a particular problem, but potentially any novel problem. When people are assuming that if we can play checkers, chess, backgammon, sorry, connect 4 and a million other games better than human, then they're assuming that every game can be played at better than human capability. Maybe so, but there are a finite number of games. There are not a finite number of ways in which intelligence is applied. Let's talk cardinals of infinity shall we? We have bandwidth limitations on training and finite training data. No finite number of dots gets you every dot between 0 and 1 on the real line. We have object recognition but we have no scene understanding. No posited models / relationships / dynamics. Perhaps by the time a neural network masters language it will be able to find relationships between many dynamic entities in a dynamic world in real-time. Btw, I asked my adviser in grad school what he thought would be the most remarkable / impressive thing to come in the next few decades ahead. He said materials / manufacturing technologies, which surprised me. So this basically agrees with what Jim Keller said. We're clearly entering into a new frontier when we start being able to construct things (like SpaceX rocket nozzles) on an atomic scale via 3D printing. And what Jim said about the potential for reducing feature dimensions is amazing. Last of all, I wanted to make the point that most every chip is predominantly a 2D device with a couple dozen layers from what I understand. At some point we're going to break that mold and start working with truly 3D designs. And the same goes for neural networks. E.g. treating LIDAR data as a 2D image is pretty silly. We're making enormous performance sacrifices for treating a sparse thing data set as approximately dense. Many other interesting points on data quantization, significant figures, determinism, and the number 42 ;). I think people are badly failing, however, to anticipate what disruption novel technologies are going to have on our way of life, however. Elon Musk gets it. But if energy becomes free (or an ax+b situation where a is tiny and b is amortized to 0...), that fundamentally changes the entire civilization. The conversation about the evolving relationship between humans and technology has many, many implications beyond self-driving cars. It relates to how our bodies become more robotic. How we start working alongside robots. How we have advisory services / counselors / mentors / teachers and eventually bosses that are 'AI'. (I hate this term, it's an oxymoron). About telepresence via robot. About agency. About cybercrime and warfare. About ethics. About responsibility. About passive versus active roles in life. About turning the world into a zoo on the first day we decide we need to setup a MIMO control system to govern a climate gone insane.
L4 almost undoubtedly will be achieved somewhere in this universe. The same applies here on earth for humans and automobiles, assuming we keep progressing (even at lower than current trajectory) for the next several millenniums. I'm very confident in this assessment. It may or may not be achievable with solely cameras, hardware and the right algorithms, this, I suppose we'll discover in time. What I am almost certain WOULD work is to successfully synthesize a smarter, more capable "brain" that we could then connect to moving cameras and sensors that it would have full control over, similar to how we have control of where we aim our own eyes when driving. The obvious problem here, is how to achieve such a feat. The thing about it is, we already know for a fact that the human brain and its equivalent(s) is doable, because, well, here we are with said brains. The evolution of a certain type of matter made this brain and there is zero evidence to suggest that we can't synthesize whatever nature has developed through time/ evolution, and plenty of evidence that we CAN, with sufficient knowledge/ capability. So, yeah, we'll probably synthesize a human brain and, through developing a clearer understanding of how it functions, maximize its potential to a point where it's simply better at doing any task that we could set our own brains on doing, including driving a car. Better yet, we can just scrap the whole human brain idea and jump straight into synthesizing a far superior type of brain from the onset.
@@RalphDratman No, this makes us good people, we believe there are ways to communicate, to deal with things without getting confrontational. Though some might argue it is necessary evil.
Keller: "...a change in quantity is a change in kind". Mathematician: "A profound change in system dynamics from a change in scale is called _emergent behavior_"
@David M Do you have any idea what emergent behavior even is? There are behaviours that are only aparent on certaing scales. Phase transitions, Consciousness, the list goes on. It is in that sense that people mean when they say "a change in quantity can be a change in kind".
25:22 So, a neural network is taking a complex data set and extracting a pattern from it. Now this pattern doesn't explain the whole of the data set or maybe even purpose of the data set. The neural network in our brain doing the same thing to the Universe. So, can our brain explain the purpose of the universe rather than just extracting patterns from it and calling them laws of physics.
the purpose of the universe is the same as any object born into it. to be created, to exist, and then to die. no greater point needed. as for humanities purpose, again to be created/ to exist/ and then to die. no greater point is needed. the point of life is to exist, and anything else is our own subjective point of view. as the absurdist camus pointed out, the stance is to not live a good life because life has any inherent meaning, but that it's possible to live a good life in defiance of its ultimate futility.
I loved when he asked Lex if he knows how CNN layers work, how they can detect ears, eyes of a cat. Sometimes you have to know your interviewer as well
I find this fascinating. In 1980, when I was trying to get an overview of how chips were designed and produced, Moore's Law was predicted to end around 1995. A quarter century after that deadline, and with feature sizes now some 100 times smaller and transistors around the size of a smallish virus particle, your guest says Moore's law, perhaps in a somewhat different mode, is continuing apace. And it strikes me that the public need to understand the role played by the basic science in their way of living. As Keller said with respect to semiconductor manufacturing, "There's equipment, there's optics, there's chemistry, there's physics, there's material science, there's metallurgy... literally thousands of technologies involved." Meanwhile some segments of the population claim that science and expertise are unnecessary. It might help them to know that without, for example, the highly mathematical study of relativity and quantum mechanics, as pursued over the past 120 years, the airplane they travel on would never leave the ground. The cell phone they organize their life with could not possibly exist. The internet they get their news from could never have been developed. The MRI machine that helped save their life last year would never have been dreamed of. I see concrete technological achievements as playing a unique role in proving beyond a reasonable doubt that our lives depend on science and mathematics, including (among many others) medical, food and energy science. How skeptical can one remain about science while cruising in a passenger jet 10 km above the surface of the earth?
Because they're idiots. Or intellectually dishonest. A great Carlin quote that still applies today is, paraphrasing, think about how dumb the average person is, and realize that half are dumber than that.
It's interesting to note that general relativity (not special relativity) is technologically almost useless. The only example I know of were it is used to a significant effect is to correct GPS location for the mass of the Earth, which slightly bends spacetime.
TSMC couldn't do it without EUV tho, as 10nm intel would be the same as DUV 7NM. EUV got to the market around 2018-2019, when it got financially viable to use them in mass production.
Lex, I'm sure you get a lot of advice from people so here's mine: you should release your short clips first or along with your long form interview. You can draw people in with the short and get them to listen to the long if interested. I always find that I see your long interviews and not sure I have the time for it but when I do they are always great. Maybe I'm wrong but I know I'd listen more this way.
I really enjoyed this talk. Is there a talk on quantum computing. No matter how much I try to understand quantum and I think "i got it 1 or 0 or anything in between". Then I realize i don't get it.
brush up on quantum mechanics before you try to jump into quantum computing. MIT open courseware has some great lectures on introductory QM ocw.mit.edu/courses/physics/8-04-quantum-physics-i-spring-2016/
I have no idea but if I had to guess I'd say quantum computing is '1' and '0' QP is not something humans can comprehend,...to be both alive & dead, up & down
@@interstellarbeatteller9306 Quantum mechanics is isolated enough from other subjects in physics that you should be able to start from scratch. Classical mechanics and electricity & magnetism are usually taught before quantum but this is mostly a formality. Quantum is self-contained, as long as you're just studying the basics there's no need to develop a holistic picture of the different fields of physics beforehand
Search for a youtube / Ted talk on quantum computing in 5 levels of detail. The people here are being absolutist MIT open course ware is not the place for a newcomer to start. UA-cam and blogs and Ted talks are such places. Also, don't get your hopes up, things are extremely primitive. First for comparison though it might be of use to study the quantum aspect of a transistor and how they switch. Cause that's a simple binary interaction. When you get into quantum mechanics you get into probabilities, waves and information theory. I expect it'll get complex very fast.
@@0dyss3us51I can see that take on things but think it misses the main point.. This for me this boils down to the LIDAR vs. passive photon debate... and the humans can't be modeled vs. x/y movements and accelerations are dampened so it is just a ballistics problem debate. Lots of folks in in ivory towers proclaiming they are correct meanwhile 1.) Tesla's have driven 3,000,000,000 miles under autopilot. 2.) Expect they are statistically provably safer than human drivers (Attention vs. accuracy debate) 3.) FSD is nearly feature complete.... Then it is just moving from 2-3X safer than humans to 10x safer. Etc. Reminds me of the PhD's and Astronauts that were paraded in front of Congress in an attempt to salvage the NASA budget proclaiming 1.) There was no place for private companies in first line positions controlling space flight. 2.) The first stage could never be brought back. 3.) It would have no impact on the $/kg to leo pricing if it did. Mind you many where making these claims AFTER spaceX had already brought back BOTH boosters from the Falcon Heavy. Love Lex and his podcasts. See deep value in his openness of mind and the litany of wonderful guests he interviews. But in this instance he was decidedly shown to be man behind the curtain we are told to ignore. And that is OK. Does not in any way diminish his value in his wheelhouse. Will make his wheelhouse much bigger if he himself will allow it. How as individuals can we ask for more?
@@cclose14111 Hi Chris I am not sure how my comment is missing the point or how all of that you wrote is really relevant, to my comment, but that is fine :)
@@0dyss3us51 Guess my point was it seemed that, ""software guy meets hardware guy" description under sells Jim by an order of magnitude. On about 4-6 different dimensions.
His thoughts on value of abstraction in chip design reminded me of ideas from Yuval Noah Harari's book Sapiens. The ability to abstract may well be one of the most important steps in the evolution from apes brains to the human brain.
He's only the best microarchitecture engineer to have ever walked the planet and has merely had his fingers in every major leap in CPU design for the last 40. No big deal.
@@Fyrwulf that's very nice that you know who he is my man. I do too now, I wrote that 7 months ago. Thanks though for all that publicly available information. If you were 8 months quicker you might have taught me something and earned your internet big boy points xx
Transistors are already not doubling every two years, the "innovations" part Jim is saying is the important thing. 3D stacking, chiplet design, L2 cache distribution, 4-way SMT, these are what pushes the industry forward now. I am excited to see Jim's Royal Core design. That looks to be Intel's last stand before prioritizing foundry
So with modifications and abstract conceptual ideas to extend Moore's Law, Moores Law becomes a self-perpetuating sort of Mandelbrot Set only limited by time and technological capability to manifest it ?
soylentgreenb this is a very ignorant statement. Quantum computing’s processing power expands exponentially with each q-bit added in a calculation. Improving traditional computation relies on increasing the speed which a computer can process bits (1’s and 0’s). Now, a typical computer processes each individual value of 1’s and 0’s; akin to picking through a tub of toys to find the one a child might want to play with. Q-bits change that. A set of q-bits are all possible combinations of 1’s and 0’s simultaneously. Now imagine if a person could randomly pick something out of the box and get the desired toy. Almost every single time. That’s quantum computing. No need to go through each individual toy. It just happens. Quantum computers calculate by just guessing the correct answer. And because of that, it doesn’t have to go through all the hoops our processors usually go through to reach an answer. It can just spontaneously go through all of the potential options in its calculations using q-bits through black magic fuckery. An 8 letter password might have billions of possible combinations. But a quantum processor would probably find the correct one without any trouble, if given enough q-bits to work with. That’s the gist of the processing power of these machines. No matter how deep or wide the toy box is, we’d be able to find the answer within a reasonable amount of time with this kind of power. A traditional processor would be hundreds of magnitudes slower when compared to a quantum computer with the same amount of bits. Work smarter, not harder.
There's a lot of hype about Quantum Computing but no one seems to be able to tell me what it can actually be used for except some new cryptography and modeling quantum behavior. It seems like a whole lot of buzz about nothing much. Any normal algorithm will still run faster on a the fastest common CPU.
Ron Ray There have actually been some pretty big advances in fusion in recent years. ITER is expected to produce net positive energy balance when it goes online and is currently under construction. AI might also help with the fusion problem, as plasma instability is a complex beast that could probably benefit from machine learning.
Jim did not mention about thermal issues associated with further miniaturization. It may be a critical issue as heat gets more concentrated in smaller footprints. Less electrons to do the switching and rising ambient temperature make it more challenging to make reliable devices. Will it be the end of silicon and time for GaN?
The amount of heat release by the transistors happens because of inefficiency in the switching itself. We use both negative an positive transistors to only use power when we send power to it, instead of a constant draw of power to it. The small space between this is the source of heat as the doping is. TSMC wants to solve this with stacking the types of transistors, as it would decrease the resistance and length of both transistors, therefore making chips efficient. Look it up, TSMC announced this at a Semiconductor expo in Amsterdam
Moore's Law was that "Transistor density doubles every 18 months". You can't just redefine it to say Moore's Law isn't dead, under the original definition it has been dead for well over a decade, when Intel stopped going from 386->486->Pentium->P2->P3->P4 and instead switched to core i5 etc was the point when Moore's Law died. Intel had to change it's marketing and branding to obfuscate the fact they could keep up with Moore's Law and keep a differentiated market place.
Moore's law is not strictly about density but was about the number of transistors that can integrated onto a circuit. It does not stipulate the size of the circuits, topology or geometry of integrated circuits. Intel chips went from producing 1 million transistors to producing about 48 billion in about 30 years which is a doubling every 2 years.
@@bighands69 Bullshit. When Moore spoke he was strictly speaking about transistor density. Yes I'm old, yes I was there at the time. That bullshit free definition is exactly why it became the Iron law of digitisation for a time. Moore's law has been dead for a long time, stop making excuses and limiting yourself, it was a useful guide for a while but today it is a crippling constraint on creativity and progress. Stop worshiping a dead paradigm.
Absolutely... I remember first learning about the Sense + Refresh circuit in DRAM back in 1999 as an undergrad. I was pretty horrified at the complexity. There's a Deux Ex Machina effect going on there straight from Heisenburg. You touch it, you change it. Moore Machines. Mealey Machines. Karnaugh Maps. Boot-strapping. 2's complement. Booth's Algorithm. Carry-lookahead adders. That's all undergrad knowledge (which is where I am in this topic). But I can only imagine the horrors of dealing with professional-grade design issues on noise margins, fan-in / fan-out, latencies, energy optimization ~ k*v*f^2, impurities, process deviations, the wonders of all that is DMA, interupts, asynchronous processing, variable frequencies, voltage-switching, time-varying master/slave relationships between cores. I can not imagine. And neither can any of the people that are not in the heart of this domain. If someone isn't at Intel, Nvidia, AMD, Micron, TI, Motorolla, Qualcomm, et al, they can't even begin to comprehend the complexity. They're not off by 100x, more like 1 million x. From my side of things, I think people would be horrified by the statistical information-theoretic algorithms that have been developed as well ;)
Read Bernard Lonergan's Insight: 'Thoroughly understand what it is to understand, and not only will you understand the broad lines of all there is to be understood but also you will possess a fixed base, and invariant pattern, opening upon all further developments of understanding.'
So basically there is the spatial variable along with a time variable for transistor production. It's almost like we live in a time space continuum or something.
@lex Friedman would be great to interview Jim Keller re RISC vs CISC and if RISC will prevail. Also interested which one will be more applicable to the field of AI?
Keller Definition of Moore's Law: "The combined process of shrinking things and manipulating/placing them in a working transistor layout using varying contributions of the innovation stack consisting of equipment, optics, chemistry, physics, material science, and metallurgy contingent on manufacturing repeatability and yield"
Instead of focusing on how to make current transistors smaller, we should focus on how to place individual atoms faster to produce the smallest possible transistors at the fastest speed possible.
Moore’s Law has been dead for over 10 years in terms of fundamental limits on shrinking transistor sizes. [Good news: If you purchased a computer in the past decade with a multi-core processor, multi-channel memory, and SSD then your computer will likely be a good performer indefinitely.] Jim Keller is not lacking for confidence even when he’s wrong. Welcome to Intel.
how does that make moores law dead? everything still has a limit even the computer and ssd you bought. i've been hearing this for 20 years and yet nothing.
Moore's law has absolutely nothing to do with the size of transistors. It is about the number of transistors that are in an integrated circuit. There is a lot more space down at the bottom than people realize.
fascinating discuss. Moore's Law predicted the growth in the number of devices on a computer chip, a successive doubling per unit time. Gordon Moore said once that predicting exponential growth of anything into the future is risky, because after some number of generations you exceed the global capacity to support any construct at that scale. In today's cloud world, the growth of computing power has accelerated beyond what Moore predicted, mostly because the number of servers being installed in cloud data centers is growing so fast.
The actual law is that the time to double is around two years. ie we look at the time to double, not the multiplication factor after two years, even tho there is an equivalence there.
Sure the math may look better to you when t is the variable, but for someone designing the system t is fixed. It's the change in performance when the product ships that's variable.
Nice to know we can keep going. As to the universe being "wastefully large", well I think that's a value judgement. Why not think of it as a lovely large blank canvas?
I keep looking at the screw that is holding the whiteboard to the wall as if it was random debris on my monitor. I tried wiping it off nearly a dozen times, yet, it's still there... Guess, I got screwed...
Maybe the correct question is "Has Moore's Law slowed". Intel has been stuck on 14nm for mass produced CPUs for 6 years now when its normal cadence was every 2 years for a die shrink. So while transister count hasn't frozen, the rate of improvement has certainly slowed dramatically.
@@walterwhite4234 I'm talking about "mass production" and not what's projected to come. Also, TSMC 7nm is more or less equivalent to Intel 10nm. Intel uses a different measurement standard.
@@walterwhite4234 You need to compare transistor density, not the marketing. Intel's 10nm density is higher than TSMC 7nm mobile chips, but lower than TSMC 7nm +. That said, TSMC got ahead of Intel because of Intel's recent stumbles. www.techcenturion.com/7nm-10nm-14nm-fabrication#nbspnbspnbspnbsp7nm_vs_10nm_vs_12nm_vs_14nm_Transistor_Densities
@@GeorgeOu Well Intel claims it is equivalent, could be marketing speech. The TSMC 5nm node is up and running and the road map to 3nm and so on seems robust, 3d stacking is on it's way, if Moore's law slows down its certainly not a catastrophic slowing.
"The number of people predicting the death of Moore's law doubles every two years."
-Peter Lee, Microsoft
I heard that the number of Elvis impersonators is growing exponentially. One day we'll all be Elvis impersonators even if we don't like his music.
@@jeremywvarietyofviewpoints3104 There are plenty of professional cosplayers that aren't familiar with their characters. But that's mainly because they cosplay many characters.
😂😂😂😂👌👌 good one
I am a bit annoyed that a lot of really amazing silicon was wasted on phones that don't at all need it.
Btw. For people who don't know. This is the guy who designed AMD's Ryzen architecture.
Mike Clark?
and?
not really designed, but he was a main lead overseeing a bunch of teams. He himself doesn't like to be credited as a creator of this architecture.
@@gickygackers My god is that not enough?
A bigger FYI, it wasn't just Jim Keller, there was also Mike Clark, Suzanne Plummer and a slew of other people. Jim gets too much recognition imo.
Whenever I begin to get confident in my own intelligence I like to listen to guys like Jim Keller talk so I can be pounded back to reality.
One can always feel confident in your own intelligence but once you get comfortable with how much you think you know that's just a recipe for disaster...
He's obviously the best at designing CPUs , but Expertise and intelligence are different things.
I'm pretty sure experts in other fields would disagree with some of his opinions.
@@kaba_me No. He doesn't merely have knowledge, he is clearly highly intelligent as far as I can tell. Using abstract concepts to solve complex problems is not merely a matter of having knowledge
@@museitup4741 I don't doubt his intelligence... I doubt his expertise in other fields.
@@kaba_me You are correct, there is a huge difference.
This guy does not break eye contact and continues conversation without skipping a beat. I don't think I could do this in a dream.
It's a learned skill. You'll get it eventually
@@xzcsdf9574 Like many things, simply practicing and not getting in your own way by telling yourself you can't do something leads to progress.
He also sounds like he is recovering from a hangover.
@@beachboy_boobybuilder idk if you’re Albert Einstein’s son, if you sound like this recovering from a hangover.
He's used to being the smartest man in the room
Jim is like the Rambo of integrated circuits.
Imagine if Joe Rogan interviewed Jim Keller? Their gap in knowledge would be so great that they would form a black hole in between them.
I thought the same. We need to get Jim on Joe Rogen
I'm imagining Eddie Bravo poking his head out of that hole like a loony toons closing animation saying "space isn't real"
Best
It would probably be a great interview, he is a master interviewer. His forte
Renier Van Rensburg And Lex is a master debater! 😂
Meanwhile, the JavaScript developers are making sure that any hardware progress is nullified by adding more NPM dependencies
Cesar Canassa leave us alone
npm: **feels bad man**
LMAO! XD
Node is pretty fast, the problem is the Golden Hammer law, Node was born as a server-side tool for building non-blocking custom services.
Today is pretty much shoved everywhere, transpilers, syntax checkers, desktop apps (electron), game-engines, which is usually what happens when a thing becomes popular.
LOL!
@@rewrite1239 I tried building rust for Android once, regretted wasting 10 days on it.
Listening to Jim Keller talk is THE most mind blowing experience. Lex you do a great job asking questions and keeping up.
Lol, this got confrontational, but in a totally academic kind of way, then got instantly resolved. Good chat.
I don't know if confrontation is a thing on Lex's part of the spectrum.
@@poneill65 He's a grappler bro.
@@SwaggySolidarity So? Doesn't require eye contact. He's just running a grappling algorithm,.. "I opponent leg leg goes there, put right leg there,..." etc, etc. The chap clearly has a completely "spectrum" typical aversion to eye contact, with forced token engagement with his interlocutors.
Lex was right though, a Convolutional Neural Network (not to be confused with just convolution) is a search. The CNN, like all searches, is trying to to find a given pattern with within the confines of a grouping of data. The only difference between it and CNN search is that it breaks the main search up into small functions that look for traits and features and each search returns a probability of discovery. This result is then used in the next search to hone in on the least inaccurate guess as to whether or not the object was found.
A CNN is still a search though.
I really enjoy when people ask for more, because that means I get to know more and that's a good thing no?
Last part of that interview was actually chilling. Absolutely incredible interview
Jim is such a humble and intelligent guy. He seems like he would make an incredible teacher/mentor if he wanted to go that route someday.
He's a teacher/mentor for hundreds of Intel employees
Listen to what he says he reads and how often. And then listen to what Elon Musk reads and how often. And then compare with the rest of humanity.
Man the level of intelligence this man is so high it made him so humble, like what I do might not even matter as much in the grand scheme of things. 👏🏻
that happens with any knowledge you gain. the more you know, the more you know how much you don't know.
Keller really puts it into perspective when it comes to layering from a simple switch to an infinity of equations. Impressive.
06:50 building a brick building with more Moore in mind
16:38 a difference in quantity is a difference in kind
17:54 the prob with early optimization
Great sign of intelligence on both sides to disagree and then agree again within a few minutes. No need to even finish most of the thoughts, just dance through it in seconds.
I love this! Two extremely intelligent people talking about complex things in a way that even dumbasses like me can follow! Cheers guys!
I mean, probably one *extremely* intelligent person but lex is very valuable in is own way haha
May's Law states, in reference to Moore's Law:
Software efficiency halves every 18 months, compensating Moore's Law.
David May (born 24 February 1951) is a British computer scientist. He is a Professor in the Department of Computer Science at the University of Bristol and founder of XMOS Semiconductor, serving until February 2014 as the chief technology officer.
May was lead architect for the transputer. As of 2017, he holds 56 patents, all in microprocessors and multi-processing.
When Inmos was formed in 1978, May joined to work on microcomputer architecture, becoming lead architect of the transputer and designer of the associated programming language Occam. This extended his earlier work and was also influenced by Tony Hoare, who was at the time working on CSP and acting as a consultant to Inmos.
The prototype of the transputer was called the Simple 42 and was completed in 1982. The first production transputers, the T212 and T414, followed in 1985; the T800 floating point transputer in 1987. May initiated the design of one of the first VLSI packet switches, the C104, together with the communications system of the T9000 transputer.
Working closely with Tony Hoare and the Programming Research Group at Oxford University, May introduced formal verification techniques into the design of the T800 floating point unit and the T9000 transputer. These were some of the earliest uses of formal verification in microprocessor design, involving specifications, correctness preserving transformations and model checking, giving rise to the initial version of the FDR checker developed at Oxford.
In 1995, May joined the University of Bristol as a professor of computer science. He was head of the computer science department from 1995 to 2006. He continues to be a professor at Bristol while supporting XMOS, a University spin-out he co-founded in 2005. Before XMOS he was involved in Picochip, where he wrote the original instruction set.
A Cascade of diminishing return curves: moorception...
"Galavanting through the nether realms of possibility" lol
It's encouraging to hear that Moore's Law is still on track from an actual chip architect. Excellent interview.
this guy is awesome....... wish he had more content out
This was truly remarkable. Thanks for cutting it down to 26mins now I think I might go watch the full one
I listened to the video, I was quiet impressed. Not just a chip architect. Loved the discussion of levels of abstraction among other things. But I disagree with 2 points. (I'm an algorithms guy, my ph.d. is in operations research). a) as Lex said, there's a lot of complexity in not just human behavior, but time-varying road conditions. We're talking the Curse of Dimensionality here and it's not clear to me that a brute-force camera-data-only training approach converges on an answer. Rain or hail or sleet can change the driving conditions (traction) quantitatively within 10 seconds. Shade or a bank or a tunnel inlet/outlet can have very different traction. Static and dynamic friction. If we're talking L5 or even L4 capabilities, to my mind that doesn't mean idealized Arizona-driving conditions. It means L4 capability everywhere on Earth. And every video I have seen if full of people making wild predictions while ignoring just how dynamic and chaotic driving can be. Like 60 mph blizzard conditions with chains, one of which was accidentally cut in the wrong place. Was that in the data set? Maybe it will be eventually. But we're very far away. (My adviser says we'll never get there, which is an interesting statement. I disagree. Then again he's one of the smartest people I've ever met). n! is nasty if n is the number of possible objects in the world. b) I actually found while optimizing Matlab code that there are many situations in which a Search problem can be formulated as a maximization problem. And many times it's actually faster within such a scripting language. But my point is that there's definitely a relationship. E.g. "Search for an inflection point" equates to "argmin_x d/dt F(x)". It's a min versus argmin or max versus argmax type of distinction. Highly related. And they must needs have the same Big-O. c) There's a massive conservation of power constraint which is presently ignored in how people think about "AI". Intelligence means not being able to solve a particular problem, but potentially any novel problem. When people are assuming that if we can play checkers, chess, backgammon, sorry, connect 4 and a million other games better than human, then they're assuming that every game can be played at better than human capability. Maybe so, but there are a finite number of games. There are not a finite number of ways in which intelligence is applied. Let's talk cardinals of infinity shall we? We have bandwidth limitations on training and finite training data. No finite number of dots gets you every dot between 0 and 1 on the real line. We have object recognition but we have no scene understanding. No posited models / relationships / dynamics. Perhaps by the time a neural network masters language it will be able to find relationships between many dynamic entities in a dynamic world in real-time. Btw, I asked my adviser in grad school what he thought would be the most remarkable / impressive thing to come in the next few decades ahead. He said materials / manufacturing technologies, which surprised me. So this basically agrees with what Jim Keller said. We're clearly entering into a new frontier when we start being able to construct things (like SpaceX rocket nozzles) on an atomic scale via 3D printing. And what Jim said about the potential for reducing feature dimensions is amazing. Last of all, I wanted to make the point that most every chip is predominantly a 2D device with a couple dozen layers from what I understand. At some point we're going to break that mold and start working with truly 3D designs. And the same goes for neural networks. E.g. treating LIDAR data as a 2D image is pretty silly. We're making enormous performance sacrifices for treating a sparse thing data set as approximately dense. Many other interesting points on data quantization, significant figures, determinism, and the number 42 ;). I think people are badly failing, however, to anticipate what disruption novel technologies are going to have on our way of life, however. Elon Musk gets it. But if energy becomes free (or an ax+b situation where a is tiny and b is amortized to 0...), that fundamentally changes the entire civilization. The conversation about the evolving relationship between humans and technology has many, many implications beyond self-driving cars. It relates to how our bodies become more robotic. How we start working alongside robots. How we have advisory services / counselors / mentors / teachers and eventually bosses that are 'AI'. (I hate this term, it's an oxymoron). About telepresence via robot. About agency. About cybercrime and warfare. About ethics. About responsibility. About passive versus active roles in life. About turning the world into a zoo on the first day we decide we need to setup a MIMO control system to govern a climate gone insane.
L4 almost undoubtedly will be achieved somewhere in this universe. The same applies here on earth for humans and automobiles, assuming we keep progressing (even at lower than current trajectory) for the next several millenniums. I'm very confident in this assessment.
It may or may not be achievable with solely cameras, hardware and the right algorithms, this, I suppose we'll discover in time. What I am almost certain WOULD work is to successfully synthesize a smarter, more capable "brain" that we could then connect to moving cameras and sensors that it would have full control over, similar to how we have control of where we aim our own eyes when driving. The obvious problem here, is how to achieve such a feat.
The thing about it is, we already know for a fact that the human brain and its equivalent(s) is doable, because, well, here we are with said brains. The evolution of a certain type of matter made this brain and there is zero evidence to suggest that we can't synthesize whatever nature has developed through time/ evolution, and plenty of evidence that we CAN, with sufficient knowledge/ capability.
So, yeah, we'll probably synthesize a human brain and, through developing a clearer understanding of how it functions, maximize its potential to a point where it's simply better at doing any task that we could set our own brains on doing, including driving a car. Better yet, we can just scrap the whole human brain idea and jump straight into synthesizing a far superior type of brain from the onset.
These two are not always speaking the same language. Lex is imperturbable.
You put it in words I couldn't. I feel from watching just a few of Lex's interviews he is there to talk and not to listen.
I couldn't agree more, I cringe whenever I see things get confrontational, but Lex handled it like a pro.
@@wentaoqiu4072 Yes, I cringe too. Does that make us cowards, you and me?
@@RalphDratman No, this makes us good people, we believe there are ways to communicate, to deal with things without getting confrontational. Though some might argue it is necessary evil.
@@wentaoqiu4072 Disagreeing on a subject is not being confrontational. There was no confronting in this interview.
"There are hundreds of billions galaxies...seems repetitive at best" loll
Keller: "...a change in quantity is a change in kind".
Mathematician: "A profound change in system dynamics from a change in scale is called _emergent behavior_"
Keller: a circle
Mathematician: "The locus of all points equidistant from a central point"
@David M Do you have any idea what emergent behavior even is? There are behaviours that are only aparent on certaing scales. Phase transitions, Consciousness, the list goes on. It is in that sense that people mean when they say "a change in quantity can be a change in kind".
@@iurigrang yes. And that all falls exactly into the definition of emergent behaviour.
Spectacular podcast. Period.
25:22 So, a neural network is taking a complex data set and extracting a pattern from it. Now this pattern doesn't explain the whole of the data set or maybe even purpose of the data set. The neural network in our brain doing the same thing to the Universe. So, can our brain explain the purpose of the universe rather than just extracting patterns from it and calling them laws of physics.
parishrut badoni cue Donald Hoffman
We know the answer already. 42.
Damn... never thought of it like that. Wow
the purpose of the universe is the same as any object born into it. to be created, to exist, and then to die. no greater point needed. as for humanities purpose, again to be created/ to exist/ and then to die. no greater point is needed. the point of life is to exist, and anything else is our own subjective point of view. as the absurdist camus pointed out, the stance is to not live a good life because life has any inherent meaning, but that it's possible to live a good life in defiance of its ultimate futility.
I loved when he asked Lex if he knows how CNN layers work, how they can detect ears, eyes of a cat. Sometimes you have to know your interviewer as well
Daler Rahimov My thought as well.
"Angry bird apps might be the whole point". Hopefully not.
16:40 "A difference in quantity is a difference in kind...".
"Quantity has a quality all its own"...
Moore’s Law is always alive, dead, and going to die all at once.
It’s the Schrödinger’s cat of laws.
I find this fascinating. In 1980, when I was trying to get an overview of how chips were designed and produced, Moore's Law was predicted to end around 1995. A quarter century after that deadline, and with feature sizes now some 100 times smaller and transistors around the size of a smallish virus particle, your guest says Moore's law, perhaps in a somewhat different mode, is continuing apace.
And it strikes me that the public need to understand the role played by the basic science in their way of living. As Keller said with respect to semiconductor manufacturing, "There's equipment, there's optics, there's chemistry, there's physics, there's material science, there's metallurgy... literally thousands of technologies involved."
Meanwhile some segments of the population claim that science and expertise are unnecessary. It might help them to know that without, for example, the highly mathematical study of relativity and quantum mechanics, as pursued over the past 120 years, the airplane they travel on would never leave the ground. The cell phone they organize their life with could not possibly exist. The internet they get their news from could never have been developed. The MRI machine that helped save their life last year would never have been dreamed of.
I see concrete technological achievements as playing a unique role in proving beyond a reasonable doubt that our lives depend on science and mathematics, including (among many others) medical, food and energy science. How skeptical can one remain about science while cruising in a passenger jet 10 km above the surface of the earth?
Because they're idiots. Or intellectually dishonest. A great Carlin quote that still applies today is, paraphrasing, think about how dumb the average person is, and realize that half are dumber than that.
everything you just said is horseshit and absurd and clearly not sane news fake mri's bs and the tech useless
@@movingurbanly4346 Bad bait
@David N interestingly enough correct but most people don't know that...
It's interesting to note that general relativity (not special relativity) is technologically almost useless. The only example I know of were it is used to a significant effect is to correct GPS location for the mass of the Earth, which slightly bends spacetime.
This is a great topic to cover. Please post more like this in the future. Excellent!!!
tremendous Questions Lex. I enjoy your channel and will continue to watch and learn.
Spicy discussion. Liked it very much!
Thanks to TSMC. If we were to rely on Intel we would have 14+++++ on 2025.
@Red Dead well this aged well
TSMC are not at Intels level and people that think they are just have no clue.
TSMC couldn't do it without EUV tho, as 10nm intel would be the same as DUV 7NM. EUV got to the market around 2018-2019, when it got financially viable to use them in mass production.
I did like learning from Keller’s perspective ... the heuristics of progress
That last point was incredible. We're developing machines right now that are making calculations that we don't even fully understand.
It's how AI works. CGP Grey has a great video on the topic: ua-cam.com/video/R9OHn5ZF4Uo/v-deo.html
This makes me so excited to be studying electrical engineering
22:05 Yes but they literally weren't in a closet all day. Cellphones are very easy to keep at hand all day.
Best interview ever!
Lex, I'm sure you get a lot of advice from people so here's mine: you should release your short clips first or along with your long form interview. You can draw people in with the short and get them to listen to the long if interested. I always find that I see your long interviews and not sure I have the time for it but when I do they are always great. Maybe I'm wrong but I know I'd listen more this way.
Agreed
True.
I love how he explains himself . What a mind
This is so good.
Love it. The way Jim talks makes Michio sound like he works with crayons.
As important as communication is, jargon literacy is required after a point.
I wish I grew an appreciation for the beauty and art in science earlier in life.
Amazing video! Thank you for making this and the others you have done!
This was a damn good watch.
I really enjoyed this talk. Is there a talk on quantum computing.
No matter how much I try to understand quantum and I think "i got it 1 or 0 or anything in between". Then I realize i don't get it.
brush up on quantum mechanics before you try to jump into quantum computing. MIT open courseware has some great lectures on introductory QM ocw.mit.edu/courses/physics/8-04-quantum-physics-i-spring-2016/
@@naught_ Should probably brush up on all other subjects before tackling the quantum realm!
I have no idea but if I had to guess I'd say quantum computing is '1' and '0'
QP is not something humans can comprehend,...to be both alive & dead, up & down
@@interstellarbeatteller9306 Quantum mechanics is isolated enough from other subjects in physics that you should be able to start from scratch. Classical mechanics and electricity & magnetism are usually taught before quantum but this is mostly a formality. Quantum is self-contained, as long as you're just studying the basics there's no need to develop a holistic picture of the different fields of physics beforehand
Search for a youtube / Ted talk on quantum computing in 5 levels of detail. The people here are being absolutist
MIT open course ware is not the place for a newcomer to start. UA-cam and blogs and Ted talks are such places. Also, don't get your hopes up, things are extremely primitive. First for comparison though it might be of use to study the quantum aspect of a transistor and how they switch. Cause that's a simple binary interaction. When you get into quantum mechanics you get into probabilities, waves and information theory. I expect it'll get complex very fast.
given search space versus found search space ... such an elegant way to describe the distinction
Theorist meet practitioner.
Lex does ML that is pretty hands on, wouls say software guy meets hardware guy would be a more fitting description.
@@0dyss3us51I can see that take on things but think it misses the main point.. This for me this boils down to the LIDAR vs. passive photon debate... and the humans can't be modeled vs. x/y movements and accelerations are dampened so it is just a ballistics problem debate. Lots of folks in in ivory towers proclaiming they are correct meanwhile 1.) Tesla's have driven 3,000,000,000 miles under autopilot. 2.) Expect they are statistically provably safer than human drivers (Attention vs. accuracy debate) 3.) FSD is nearly feature complete.... Then it is just moving from 2-3X safer than humans to 10x safer. Etc. Reminds me of the PhD's and Astronauts that were paraded in front of Congress in an attempt to salvage the NASA budget proclaiming 1.) There was no place for private companies in first line positions controlling space flight. 2.) The first stage could never be brought back. 3.) It would have no impact on the $/kg to leo pricing if it did. Mind you many where making these claims AFTER spaceX had already brought back BOTH boosters from the Falcon Heavy. Love Lex and his podcasts. See deep value in his openness of mind and the litany of wonderful guests he interviews. But in this instance he was decidedly shown to be man behind the curtain we are told to ignore. And that is OK. Does not in any way diminish his value in his wheelhouse. Will make his wheelhouse much bigger if he himself will allow it. How as individuals can we ask for more?
@@cclose14111 Hi Chris I am not sure how my comment is missing the point or how all of that you wrote is really relevant, to my comment, but that is fine :)
@@0dyss3us51 Guess my point was it seemed that, ""software guy meets hardware guy" description under sells Jim by an order of magnitude. On about 4-6 different dimensions.
COSMOLOGIST meet ENGINEER
25:00 hitchhikers guide to the galaxy anyone?
What an interview
He's the Chuck Norris of CPU design
This was a tough interview
But Lex pulled it off and I think he's about the only podcaster that can. I don't imagine Joe Rogan having a conversation on the same level lol.
@@ArmoredNeko Most podcasters are dumb.
@@haliax8149 Most people are dumb.
Really enjoyed this. Thanks.
The prognostications of Moore’s laws death sound an awful lot like the prognostications of “Manhattan will be underwater by 2015”.
His thoughts on value of abstraction in chip design reminded me of ideas from Yuval Noah Harari's book Sapiens. The ability to abstract may well be one of the most important steps in the evolution from apes brains to the human brain.
I have never heard of Jim Keller before he clearly knows his shit though. This is really interesting
He's only the best microarchitecture engineer to have ever walked the planet and has merely had his fingers in every major leap in CPU design for the last 40. No big deal.
@@Fyrwulf that's very nice that you know who he is my man. I do too now, I wrote that 7 months ago. Thanks though for all that publicly available information. If you were 8 months quicker you might have taught me something and earned your internet big boy points xx
10:11 Don't look at me mate, I ain't got a clue
I know a lot about this, but you know what, I'm a fucking moron when it comes to explaining or asking questions. Lex is brilliant!
Transistors are already not doubling every two years, the "innovations" part Jim is saying is the important thing. 3D stacking, chiplet design, L2 cache distribution, 4-way SMT, these are what pushes the industry forward now. I am excited to see Jim's Royal Core design. That looks to be Intel's last stand before prioritizing foundry
What do you mean they are not doubling every two years. That is a very vague statement.
So with modifications and abstract conceptual ideas to extend Moore's Law, Moores Law becomes a self-perpetuating sort of Mandelbrot Set only limited by time and technological capability to manifest it ?
That is nonsense.
Quantum computing is about to make Moore's Law look like a gentlemen's agreement.
It was a "gentlemans agreement" among hardware manufacturers for some time. :)
Wake me when someone discovers a quantum algorithm for something I care about. The main "benefit" so far appears to be making cryptography obsolete.
soylentgreenb this is a very ignorant statement. Quantum computing’s processing power expands exponentially with each q-bit added in a calculation. Improving traditional computation relies on increasing the speed which a computer can process bits (1’s and 0’s). Now, a typical computer processes each individual value of 1’s and 0’s; akin to picking through a tub of toys to find the one a child might want to play with. Q-bits change that. A set of q-bits are all possible combinations of 1’s and 0’s simultaneously. Now imagine if a person could randomly pick something out of the box and get the desired toy. Almost every single time. That’s quantum computing. No need to go through each individual toy. It just happens. Quantum computers calculate by just guessing the correct answer. And because of that, it doesn’t have to go through all the hoops our processors usually go through to reach an answer. It can just spontaneously go through all of the potential options in its calculations using q-bits through black magic fuckery. An 8 letter password might have billions of possible combinations. But a quantum processor would probably find the correct one without any trouble, if given enough q-bits to work with. That’s the gist of the processing power of these machines. No matter how deep or wide the toy box is, we’d be able to find the answer within a reasonable amount of time with this kind of power. A traditional processor would be hundreds of magnitudes slower when compared to a quantum computer with the same amount of bits. Work smarter, not harder.
@@soylentgreenb You mean like finding the global minimum with quantum annealing? I would argue pretty useful.
There's a lot of hype about Quantum Computing but no one seems to be able to tell me what it can actually be used for except some new cryptography and modeling quantum behavior. It seems like a whole lot of buzz about nothing much. Any normal algorithm will still run faster on a the fastest common CPU.
Moore's Law is the opposite of Fusion is will be gone in 30 years and always will be.
Ron Ray There have actually been some pretty big advances in fusion in recent years. ITER is expected to produce net positive energy balance when it goes online and is currently under construction.
AI might also help with the fusion problem, as plasma instability is a complex beast that could probably benefit from machine learning.
Jim did not mention about thermal issues associated with further miniaturization. It may be a critical issue as heat gets more concentrated in smaller footprints. Less electrons to do the switching and rising ambient temperature make it more challenging to make reliable devices. Will it be the end of silicon and time for GaN?
The amount of heat release by the transistors happens because of inefficiency in the switching itself. We use both negative an positive transistors to only use power when we send power to it, instead of a constant draw of power to it. The small space between this is the source of heat as the doping is. TSMC wants to solve this with stacking the types of transistors, as it would decrease the resistance and length of both transistors, therefore making chips efficient. Look it up, TSMC announced this at a Semiconductor expo in Amsterdam
Really great. Gonna check out the audio podcast.
Moore's Law was that "Transistor density doubles every 18 months".
You can't just redefine it to say Moore's Law isn't dead, under the original definition it has been dead for well over a decade, when Intel stopped going from 386->486->Pentium->P2->P3->P4 and instead switched to core i5 etc was the point when Moore's Law died.
Intel had to change it's marketing and branding to obfuscate the fact they could keep up with Moore's Law and keep a differentiated market place.
I'm going to go with Jim Keller on this one.
Moore's law is not strictly about density but was about the number of transistors that can integrated onto a circuit. It does not stipulate the size of the circuits, topology or geometry of integrated circuits.
Intel chips went from producing 1 million transistors to producing about 48 billion in about 30 years which is a doubling every 2 years.
@@bighands69 Bullshit.
When Moore spoke he was strictly speaking about transistor density.
Yes I'm old, yes I was there at the time.
That bullshit free definition is exactly why it became the Iron law of digitisation for a time.
Moore's law has been dead for a long time, stop making excuses and limiting yourself, it was a useful guide for a while but today it is a crippling constraint on creativity and progress.
Stop worshiping a dead paradigm.
Theres so much more to cpu hardware than moore’s law. When it keeps being brought up our eyes roll.
Absolutely... I remember first learning about the Sense + Refresh circuit in DRAM back in 1999 as an undergrad. I was pretty horrified at the complexity. There's a Deux Ex Machina effect going on there straight from Heisenburg. You touch it, you change it. Moore Machines. Mealey Machines. Karnaugh Maps. Boot-strapping. 2's complement. Booth's Algorithm. Carry-lookahead adders. That's all undergrad knowledge (which is where I am in this topic). But I can only imagine the horrors of dealing with professional-grade design issues on noise margins, fan-in / fan-out, latencies, energy optimization ~ k*v*f^2, impurities, process deviations, the wonders of all that is DMA, interupts, asynchronous processing, variable frequencies, voltage-switching, time-varying master/slave relationships between cores. I can not imagine. And neither can any of the people that are not in the heart of this domain. If someone isn't at Intel, Nvidia, AMD, Micron, TI, Motorolla, Qualcomm, et al, they can't even begin to comprehend the complexity. They're not off by 100x, more like 1 million x. From my side of things, I think people would be horrified by the statistical information-theoretic algorithms that have been developed as well ;)
Read Bernard Lonergan's Insight: 'Thoroughly understand what it is to understand, and not only will you understand the broad lines of all there is to be understood but also you will possess a fixed base, and invariant pattern, opening upon all further developments of understanding.'
Super informative guest 🤯
Great insight
So basically there is the spatial variable along with a time variable for transistor production. It's almost like we live in a time space continuum or something.
@lex Friedman would be great to interview Jim Keller re RISC vs CISC and if RISC will prevail. Also interested which one will be more applicable to the field of AI?
Lex schooled on this video 🤣
“Angry Bird apps? Who knows, maybe that’s the whole point of the universe.” -Jim Keller
Nice Video!
Loved listening to Mr. Keller challenge his host.
Seems like a very legit computer scientist. Just look at those bags under his eyes
Keller Definition of Moore's Law: "The combined process of shrinking things and manipulating/placing them in a working transistor layout using varying contributions of the innovation stack consisting of equipment, optics, chemistry, physics, material science, and metallurgy contingent on manufacturing repeatability and yield"
It's not a law it's just a really good prediction and objectively it's dead by definition. You can change the definition and say it's not dead
Moore said that the cost of a transistor dropped 50% every time period.
In terms of the production price.
If you let AI decide how to design a city properly, I think it would look very different from what we know
Instead of focusing on how to make current transistors smaller, we should focus on how to place individual atoms faster to produce the smallest possible transistors at the fastest speed possible.
17:00 Lenin's favorite quote.
I think that was Stalin
@@Michael-it6gb Dialectical materialism. Karl Marx and Friedrich Engels + Lenin. Stalin was much later.
@@Michael-it6gb Perhaps Stalin also liked this quote.
Moore’s Law has been dead for over 10 years in terms of fundamental limits on shrinking transistor sizes. [Good news: If you purchased a computer in the past decade with a multi-core processor, multi-channel memory, and SSD then your computer will likely be a good performer indefinitely.] Jim Keller is not lacking for confidence even when he’s wrong. Welcome to Intel.
how does that make moores law dead? everything still has a limit even the computer and ssd you bought. i've been hearing this for 20 years and yet nothing.
Erm, you can suffer with your 2600K, I'll take the modern stuff, thanks
Moore's law has absolutely nothing to do with the size of transistors. It is about the number of transistors that are in an integrated circuit.
There is a lot more space down at the bottom than people realize.
Hello sir, i saw you with Andrew NG. When that podcast is coming
"I would say that it's absolutely not semantics..." hahaha
fascinating discuss. Moore's Law predicted the growth in the number of devices on a computer chip, a successive doubling per unit time. Gordon Moore said once that predicting exponential growth of anything into the future is risky, because after some number of generations you exceed the global capacity to support any construct at that scale. In today's cloud world, the growth of computing power has accelerated beyond what Moore predicted, mostly because the number of servers being installed in cloud data centers is growing so fast.
Moore never predicted the power of computing. He was talking about the number of transistors on an integrated circuit.
Jim Keller the John Wick of AMD without him AMD won't be where it is today
The actual law is that the time to double is around two years. ie we look at the time to double, not the multiplication factor after two years, even tho there is an equivalence there.
Sure the math may look better to you when t is the variable, but for someone designing the system t is fixed. It's the change in performance when the product ships that's variable.
Nice to know we can keep going. As to the universe being "wastefully large", well I think that's a value judgement. Why not think of it as a lovely large blank canvas?
You're misrepresenting his point.
I had no idea Luke Skywalker knew so much about computing.
I keep looking at the screw that is holding the whiteboard to the wall as if it was random debris on my monitor. I tried wiping it off nearly a dozen times, yet, it's still there... Guess, I got screwed...
this guy is pretty smart, he should code.
To tighten the vice of the microphone to your desk isn't a good idea at least not at this where you are sitting. Thank you for sharing
Maybe the correct question is "Has Moore's Law slowed". Intel has been stuck on 14nm for mass produced CPUs for 6 years now when its normal cadence was every 2 years for a die shrink. So while transister count hasn't frozen, the rate of improvement has certainly slowed dramatically.
Intel was mismanaged, look at TSMC and Samsung they are the leading fabs, they are at 5nm right now.
@@walterwhite4234 I'm talking about "mass production" and not what's projected to come. Also, TSMC 7nm is more or less equivalent to Intel 10nm. Intel uses a different measurement standard.
@@walterwhite4234 You need to compare transistor density, not the marketing. Intel's 10nm density is higher than TSMC 7nm mobile chips, but lower than TSMC 7nm +. That said, TSMC got ahead of Intel because of Intel's recent stumbles. www.techcenturion.com/7nm-10nm-14nm-fabrication#nbspnbspnbspnbsp7nm_vs_10nm_vs_12nm_vs_14nm_Transistor_Densities
@@GeorgeOu Well Intel claims it is equivalent, could be marketing speech. The TSMC 5nm node is up and running and the road map to 3nm and so on seems robust, 3d stacking is on it's way, if Moore's law slows down its certainly not a catastrophic slowing.
Fin fet increased speed without changing node size, moores law is about performance not node size.
If I understood half of what I think I did, that was fascinating!
Please: a link to the Rich Sutton paper?