Or Ian could learn to use some of the voice generation AIs to replicate Keller's voice and use it to "denoise" the audio. Denoise under quotation marks because he would be replacing the signal, and not treating it
@@prgnify I've already been using those tools. I even used them in part of this video for my own voice, and no-one has picked up on it. I've even fooled my own family
I usually don’t make comments under UA-cam clips, but please do feed us more contents like this one… very interesting episode, and would love to learn more!
I absolutely want more!! This is content you can't find ANYWHERE ELSE! Let alone with such fantastic talking points. I get more excited through these videos than a marketing team could ever make me. The practical business applications for this are absolutely bonkers!
+1 for the micro architecture deep dive from me also. Most interesting point: AI is going to write software that’s different from what humans write. That’s a pretty profound insight and was something I hadn’t thought of before. (Also “C has no loss function” 😁)
Since your here, I assume your aware > " Software that is different from what humans write " is the basis behind Elon Musk's open letter to pause development of ChatGPT-4. That AGI, Artificial General Intelligence "proposed" ability to write code that was beyond our grasp. While probable, its a deceptive half-truth based upon the work of technologist and futurist Ray Kurzweil's concept of autonomous computing called the Singularity. Once the Turing Test threshold is passed intelligent systems will create new versions of "themselves" In reality, we could benefit from annotated assembly, or a new machine language that could break speed records, efficient and compact - but it better have solid, concise commenting :)
There is a good chance they can’t do a deep dive. I don’t think sifive cores are open source hence the license may prevent them from talking about it too much and as for the changes they have made or their own designs that are attached then they might not want to talk too much about it in case someone copies it.
Lots of people think like him. Most come up with a "good idea" in theory and find it's not practical in reality. Sometimes people come up with great solutions to questions nobody is asking. There are thousands of "the next big thing" startups that have come and gone, whilst very few succeed and come to full fruition. Even the big names in the industry get it wrong, Intel's Itanium and P4 Netburst architecture for example. Only time will tell if this is the way forward.
Love what their doing, will be interesting to see where they go, but we really need to get more low cost RISC5 SBC's into the DIY and Startup DEV world as this can be a driving force for performance increase and to push the platform into the mainstream, leveraging the opensource community to adapt current software to natively support RISC5 can only help.
What are they doing? You know k8s for AI training/operation/experiment good enough to call out a set of O'Reilly books, say? Is there an OS already? I mean, if that laptop thought about being closer to $35 than it is to $44,000 (dual Epyc 6 screen laptop) I would like that (but one split is at $22k and that's...will I not want housing anymore in 2025?)
I can live without low cost as long as they are performant and smol at the same time. The best we can get on the ARM side of SBCs is A76 which was announced in 2018, making it a full 5 generations of core design out of date. Even just a jump to Cortex X1/Neoverse V1 level performance would be a big leap, and while SiFive's latest P870 core would match this there are no such SoC designs using it on the horizon.
Great job on this video! It's quite the skill you have to pull key insights from people during interviews. I learned more about where AI hardware is going from those two interviews than I did from reading hundreds or articles, videos, and talks/presentations.
Jim is one of the smartest, sad guy I have ever listened to. Thank you so much for this video. I loved Jim’s rebuttal to the AI, black box problem (AI decisions are not auditable).
8-wide RISC-V seems like it means serious business. I remember when Apple started building their own custom cores that they were surprisingly wide -- wider than even the latest desktop chips at the time. That seems to have done well for them. Is there a table somewhere where we can reference/compare the instruction-width of various modern CPU architectures? I feel like an outdated table exists on Anandtech somewhere but I can't seem to find it. I'd love to see how old and new architectures compare in width.
Afaik Alderlake was the first x86 Architecture with a 6-wide decoder and AMD is 4-wide up to and including Zen 4. That doesn‘t mean they are limited to that number of instructions per cycle though, as the instruction microcache may be wider.
@@noergelstein No, all instructions must be decoded first regardless of if they are accessed from the icache or elsewhere. IPC can not exceed decode width
After the tech restrictions with China I expect RISC-V will grow in leaps and bounds especially in China. They have GNU toolchain and can run Linux. From a security point of view RISC-V on Linux from multinationals will be better than using US controlled tech. I personally like the idea of a third instruction set in mainstream not controlled by US. This will improve competition and innovation. The x86 instruction set and extensions have barely changed in 10 years and even ARM is mostly just modifications to old instruction sets apart from the 64 bit extensions. RISC-V ISC already supports 64 bit. But this AI acceleration looks vary exciting I will keep an eye on this company may become a NVidia competitor in the future.
Not many at all. You see ARM put their licence fees up and coupled to the issue with the US banning Western chip technology to China, the country wants to switch to open source RISC-V ASAP. The idea is they can build all the tool chains and compilers from scratch and not be chained to US companies that could be banned from doing business. You see it is like they have their own UA-cam and Google, they want their own home-grown software as well. Chiplets were also something they developed to get around the EUV ban. Huawei built the Harmony OS in record time.
probably something like 5 years, the time for the foundation to finalize the ISA extensions and for designers to refine the different microarchitectures. The issue remains the price though.
@@reddeimon475 ARM history is quite different. A lot had to be paved before the 2010 decade, both from the company itself and other links of the chain. Moreover Risc-V benefits from the community shared effort and experience and a clean design as well as huge interests from non-US companies and countries.
@@PainterVierax The community shared effort is not something that's going to help. Companies like this will come up with proprietary, closed CPU designs based on the RISC V ISA, and have complete control over them, then they can charge what they want. This open ISA is nothing like the open software model. The "community" is very different.
It'll be interesting to see the power envelope that these different core designs run in. Also curious in the performance of the cores - though I doubt we will be running vanilla Linux on it :D
Well, you can run Linux on RISC V today, so it would probably run on this too, even if it probably wouldn't be able to utilize any of the closed source IP in it...
@@Luredreier you sure can. it doesn't look like it has storage on it (past firmware) so you probably won't be running Ubuntu or fedora is more what I meant
The past 10 years have been really fascinating watching how designs change based on compute needs and what tradeoffs to make in the full system and all the optimizations at the instruction level on up.
unauditable software sucks no matter who or what makes it. Jim begs the question of machine-generated software by assuming software is not auditable, then using that as a justification for unauditable machine-generated software. We're already drowning in sh*t implementations; machine learning can only compound that.
It would be neat if they could build a consumer graphics processor with RISC-V. I'm sure it's possible, i just don't know what it would take to make it economical.
Sad to see Keller resorting to voodoo at the end of that interview. Modularity is an engineering principle that allows us to reason about a five million line program. Safety critical systems don't run on Windows.
And the guy cautiously omitted that opensource licensing already allows thousands of people to audit that code in search for bugs, vulnerabilities or ways to optimize.
I have seen plenty of medical equipment still being run on windows at least at the interface level (and that too in xp sometimes - the horrors) The average joe couldn't care less of safety critical systems sadly
Lots of embedded systems run windows or obsolete OS's. It's a major security problem, because your milling machine has a windows NT controller and they don't support it any more, or your switch runs CentOS 7.9 and you can upgrade the OS to Rocky 9 but good luck getting the drivers and switch CLI to run on that because the mfgr doesn't give out source and the drivers are behind a high paywall on the silicon vendor's web site.
@@tristan7216 MS' own EULA says windows is not to be used in safety critical systems. Workers are not supposed to be anywhere within the swing of a mill while it is under numeric control, and CNC machines are user-programmed, it's not windows' place to refuse a command that crashes the milling head into a part
Surely if you buy a machine with a six- or seven-figure price tag, you would also extract some binding commitments about the support of that machine that would last the expected lifetime of its usefulness. Continuing to use something that is unsupported simply because you cannot afford to replace it means you are just one failure (hardware or software) away from bankruptcy.
@@lawrencedoliveiro9104 The problem with your logic is fairly obvious. X86 has a totally different market to either ARM or RISC V. It's like saying JCB has a tiny share of the entire automobile market. Several of the fastest supercomputers ever made are based on the IBM POWER architecture... and most people have never even heard of that. Most of the other top 500 Supercomputers use x86.
@@another3997 Following your same logic, you are trying to imply that RISC-V will not be successful until it takes over the construction-equipment market.
16:53 In the open-source world there are several examples of large code bases being successfully maintained over periods of decades. The largest one is probably the Linux kernel, currently at about 20 million lines. They have a policy that nothing stays in the code base without a maintainer to respond to bug reports. If nobody is able to handle problems with a piece of code, then it gets dropped--as Microsoft discovered the hard way, when it was contributing patches for Hyper-V.
The open source software world is very different to the Risc V open ISA world. It's only the Instruction Set that is open. Chip designers are free to implement whatever features and chip designs they want using that Instruction Set. Think of Risc V as like a programming language, the built in commands are fixed by the creators of the language, but the 3rd party software can be written any way the programmer wants. And the language creators have no control over that.
Dare I hope for a RISC-V chip that is a direct competitor with the Raspberry Pi RP2040? Including something like the RP's PIO state machines and (as long as I am dreaming) the ability to run micro python programs?
Jim Keller and team solve problems and challenges that move the semiconductor, firmware, compiler, software, server systems future forward at an accelerated rate! Fantastic interviews, insights and roadmap. Thank you TechTechPotato for your diligent efforts. Would be interested in Tenstorrent plans on leveraging Co-Packaged Optics. (i.e. Ranovus, Poet Technologies). Something tells me Portable, Mobile, Distributed Cloud Infrastructure is about to become a reality! Distributed microClouds/nanoClouds will be a hoot!! Such a boon for future Automotive EV Ecosystems that have to derive revenue and profit from innovative cloud products and services. Good times are coming! Insanity ensues! Ha!!
Would love to see how "easy" (or hard) it is to port a neural net made for x86 cpu to a tenstorrent compute card. In my case it's written in c++ without any ml specific libraries. Interested to see what kind of structures, algorithms or kind of compute it support for acceleration.
For Tenstorremt I think you need to write your AI in Pytorch. Everyone is already using Pytorch for AI anyway so it is much more efficient for them to focus their efforts on creating tools for Pytorch only.
At this moment in time, a Risc V desktop machine would be very slow in comparison to both x86 and ARM. The ISA may be open, but the architectural designs and implementations of actual CPUs haven't yet reached maturity. If a company comes up with a competitive design, they still have to make it competitive in terms of price, availability and third party support.
@@another3997 Its inevitable our understanding of computers will radically change. Today's Internet of things comprised insecure, inflexible embedded systems must evolve. The future resides in leveraging the power of the network to offload and distribute compute over a wide area. New systems comprised of low power NvLink or Mellanox style interconnects could alter how humans interpret or view what a "computer" is. It's possible, our isolated single purpose built machines will be enhanced then superseded by networks comprised of embedded nodes, where openness and security are paramount. As for today, there are legitimate security concerns within proprietary hardware design. For example, it would great to upgrade or deny "features " within the Intel ME. When it comes to a "drop in" solution for the desktop, low power options focusing on security and scalability would be interesting. New add in cards which promote "low cost" data science would be great! The RTX 8000, A100 are unaffordable, power hungry and overkill for smaller tasks.
I’m more on the side of Casey Moratory, AI is just a good way to find new ways todo normal Algorithms but we first have to bridge the gap between abstract model and reality. It’s also great for very dumb stuff that one wouldn’t want todo themselves anyway. If we don’t find out how to get away with the black box we will be unable to advance as simple as it is.
If a larger company wanted to include some of Tenstorrent’s IP/chips on package would Tenstorrent permit them access? Assuming they become popular and used for a lot of tasks it may be nice to move their package from an add in board to an on package solution.
I suspect that if this were to happen, Tenstorrent would licence it just like Intel and AMD cross licence x86 technology. The uptake and success then depends on price and licensing restrictions. Which is where you start to lose any benefit of the open ISA... you become wholly dependent on one company.
Will Skynet be run by RISC V? This is vital information for the future of humanity! If I get some ambition and can find information I find palatable I will try to go down the Architecture rabbit hole.
I really love all your videos. I'm no where CLOSE to being as smart as you, but the way you talk and present things sure makes me feel smart 😂. Is YT your full time job or do you do things with chip makers?
Thanks for the video. Jim Keller seems to be at the forefront of advancing CPU tech. AMD to Arm back to AMD to Intel now Risc V and Apple is in there as well.
@@aravindpallippara1577 Jim Keller's list of work on CPUs is long and impressive, and my list was just off the top of my head, forgot Apple and had to edit it in.
x86 and ARM are both gated behind "corporate IP", therefore discourages independent developers to contribute. RISC-V is opensource. It will have more room to grow.
I remember reading about Jim Keller in a PC gamazine back in 2012 and how he was going to join AMD, when they still sold FX CPUs at a loss. Never would I have expected that he (with the help of many others of course) could turn AMD around and even surpass Intel on some markets. I hope RISC-V will have a similar sucess with him on board. ^^
So he worked on Apple’s M1? The Apple approach seems to get its performance primarily from tying everything together into a monolithic design which cannot be upgraded. That may work OK for laptops, but it does mean their desktop designs have basically become laptops in desktop cases. They are supposed to be coming up with a replacement for the Mac Pro workstation line soon, and it’s not so clear this monolithic approach will fly there. That market demands upgradeability and adaptability of hardware configurations.
@@lawrencedoliveiro9104 For Apple, that's not specifically a CPU architectural design, that's a physical construction decision. The CPU core itself could be standalone, or as they chose to do, a system on a chip. The integrated SOC solutions have certain benefits for a company like Apple, as they only design build CPUs build for themselves. There's no technical reason the CPU core can't be separated from the peripheral components.
One of most interesting part was why chiplets, and that you don't have choice. Question is, what Nvidia think about it? Because they more and more looks as giant with clay legs.
Dr Cutress, i have a hope you might hear me. The Ryzen 7000 cooking issue is a classic bus driver latchup - a CPU designer like yourself would know what these are unlike just about every other tech on youtube. DDR5 has its own memory power controller on board so we are getting the classic instance of a bus signal being sent higher than/too close to the bus driver MOSFET VCC or below the driver ground. This is the most common cause of latchup. There may have been no need to have input protection OVP on the memory controller bus driver til now. Ofc once latchup occurs there is nothing a thermal system can do (even if it isnt taken out by the lacthup), by the time the thermal system can know something is wrong the damage is done. Im pretty sure teh only way to stop it would be power excursion monitoring. Im not in CS like yourself, im a lowly engineering technician in audio/embedded but this one is pretty obviuos to me
A RISC-V Hardware Developer needs to get with someone like a 'System76' (Linux Desktop/Notebook Manufacturer) that can begin wide spread availability of Linux/RISC-V based Workstations and other Consumer Devices out to the public. That is unless this plans to stay a Server/Micro-Controller Chip and miss out on the 'middle ground' entirely.
@@lawrencedoliveiro9104 I just meant that (LIKE APPLE back in the day) they have a say in the design and component choices onboard) to meet OS requirements. I should remember that this is the Internet and everyone has a PHD along side Dr Cuttress.
I love the concept of Risc-V, but I have yet to see any way to actually take the files and turn them into a working computer that doesn't involve going right back to a centralized market, since the circuitry to go between the peripherals and the CPU is only manufactured by a handful of Chinese companies, and there is *zero* documentation available on DIY-level motherboard design.
@@leeroyjenkins0 that sounds like the natural outcome. But the thing is since GPT, being more productive at existing tasks is how I’m *currently* using it. That’s the starting point, so it might only evolve from there and displace what I’ve worked for. I busted my arse for years labouring and for a while in service too so after pivoting career and really finding happiness in my work it’s a bit of a kick in the guts to not know what’s coming.
People get it all wrong. Huawei can still use ARM instruction set. Other China companies also still use ARM instruction set. It is chip manufacturing. They have problem in getting their designs manufactured by chip manufacturer like TSMC. RISC-V cannot help. RISC-V chips still needs to find chip manufacturers to produce.
Mainframe class IO at last? For so many cores, there's obvious hurdles with cache coherency unless you go NUMA. And then there's the compiler backends..
I don't completely follow Jim Keller's argument about auditable functions and AI. We know that AI is non-deterministic. Maybe that can be changed for software development processes, but doesn't that imply a static training set?
For AI if you have your dataset you can check if the network functions correctly. Yes it is not deterministic but you can performed statistical analysis to see how much the NN model is working correctly. You can do this analysis to any arbitrary precision you want limited only by your dataset. So if your data allows it you can determined if the AI is correct 99.9999% of the time. Handcoded program don't have this powerful tools and debugging is usually an esoteric witchcraft.
Xilinx and I think AMD used chiplets, it works. Xilinx can ship parts with different mix and match capability like HBM. And it probably boosts yield and reduces design time. But the interconnect (NOC) adds complexity. You don't *have* to use chiplets tho, you could go the other way: Cerebras.
What could be a potential use in the consumer space? Could it be an AI add-in card for PCs, analogous to perhaps how many years ago Ageia Physx was intended for physics simulation?
I just had a thought IAN, risc-v is an open source instruction set for CPU, could someone release an open source GPU architecture so both cpu and gpu are open source? Takecare.
Whether the ISA is open or closed, makes very little difference to the end customers. Because the companies designing and building chips using that ISA are still able to make their chip designs proprietary and closed source. Just like a program written using an open source language doesn't have to be free or open.
Do a deep dive on their arch and compare it to what EuroHPC folks are doing with their risc-v design. That is funded with EU money so most of the info should be accessible. Otherwise come to ISC in Hamburg in one month where you'll be able to meet EuroHPC people and learn things firsthand.
How does open-source work for hardware ? Do you have to put the designs made from open source IP into the public domain, just as a GPLv3 software license would enforce ?
I'm not sure what specific license they use, but it's not GPL. It allows proprietary designs. RISC-V is an ISA, i.e. a list of possible instructions and their binary format, including several optional extensions for things like floating point math, vectorization, etc. Designing the circuitry to interpret those instructions and do the actual work, that's up to the manufacturer, and designs will vary widely in how they do that. That part does not need to be open source. Companies don't have to publish their verilog files or litho masks. All it means is that manufacturers don't have to pay royalties to use that instruction set.
@@DFPercush Risc V is pushed by Berkeley, so the ISA is BSD licensed, which is known as a permissive license that allows forks to get proprietary licensed (unlike a copyleft license like the GPL). There are many Risc-V cores designed, some are opensourced but the majority are closed sourced.
Excellent video. Thank for the content and I wish Jim and his company success in ramping risc-v designs and bringing them to market. When you mention they have laptop designs, are they windows laptop designs or some other OS? Finally, Santa Clara, isn’t that where intel is? So Jim is just down the street eh?
@@TechTechPotato I figured that it would be Linux-based as my understanding was that windows didn’t have a risc-v port. I wish them success. Good to see Jim do good things.
![[25] Wei-han Lien, Lead CPU Architect, Tenstorrent](http://i.ytimg.com/vi/DnM3J2lAYd0/mqdefault.jpg)
![[25] Wei-han Lien, Lead CPU Architect, Tenstorrent](/img/tr.png)
The first optimization Tenstorrent should do is to put sound panels in their meeting room to minimize echo.
Or Keller has to learn to put the mike on instead of leaving it on the table.
@@peterfireflylund Yes this lol
Or Ian could learn to use some of the voice generation AIs to replicate Keller's voice and use it to "denoise" the audio.
Denoise under quotation marks because he would be replacing the signal, and not treating it
@@prgnify I've already been using those tools. I even used them in part of this video for my own voice, and no-one has picked up on it. I've even fooled my own family
@@TechTechPotato They truly are amazing!
I would love to see a deep dive into the architecture of the chip, and why the particular decisions were made.
YES
Its a wide chip, like apple silicon
8 wide decoder
Yeah, I'm wondering about that as well.
@@backacheache are you referring to RISC-V as an ISA? afaik the implementation of these cores are not open source themselves
I usually don’t make comments under UA-cam clips, but please do feed us more contents like this one… very interesting episode, and would love to learn more!
Jim Keller is always insightful.
*\(^___^)/* 🐼
I absolutely want more!! This is content you can't find ANYWHERE ELSE! Let alone with such fantastic talking points. I get more excited through these videos than a marketing team could ever make me. The practical business applications for this are absolutely bonkers!
*\(^___^)/* 🐼
*\(^___^)/* 🦒
RISC-V is gaining so much momentum it's crazy! So excited!
+1 for the micro architecture deep dive from me also.
Most interesting point: AI is going to write software that’s different from what humans write. That’s a pretty profound insight and was something I hadn’t thought of before.
(Also “C has no loss function” 😁)
Since your here, I assume your aware > " Software that is different from what humans write " is the basis behind Elon Musk's open letter to pause development of ChatGPT-4. That AGI, Artificial General Intelligence "proposed" ability to write code that was beyond our grasp. While probable, its a deceptive half-truth based upon the work of technologist and futurist Ray Kurzweil's concept of autonomous computing called the Singularity. Once the Turing Test threshold is passed intelligent systems will create new versions of "themselves" In reality, we could benefit from annotated assembly, or a new machine language that could break speed records, efficient and compact - but it better have solid, concise commenting :)
*\(^___^)/* 🐒
*\(^___^)/* 🐴
There is a good chance they can’t do a deep dive. I don’t think sifive cores are open source hence the license may prevent them from talking about it too much and as for the changes they have made or their own designs that are attached then they might not want to talk too much about it in case someone copies it.
Feel free to interview Jim more. If only more people think like him.
Lots of people think like him. Most come up with a "good idea" in theory and find it's not practical in reality. Sometimes people come up with great solutions to questions nobody is asking. There are thousands of "the next big thing" startups that have come and gone, whilst very few succeed and come to full fruition. Even the big names in the industry get it wrong, Intel's Itanium and P4 Netburst architecture for example. Only time will tell if this is the way forward.
*\(^___^)/* :😺
I laughed at the "auditable code" bit. "Most of whom no longer work for your company!" Jim is so right.
Very cool to see more information about Risc-V, I am definitely interested in seeing more about it :)
Love what their doing, will be interesting to see where they go, but we really need to get more low cost RISC5 SBC's into the DIY and Startup DEV world as this can be a driving force for performance increase and to push the platform into the mainstream, leveraging the opensource community to adapt current software to natively support RISC5 can only help.
What are they doing? You know k8s for AI training/operation/experiment good enough to call out a set of O'Reilly books, say? Is there an OS already? I mean, if that laptop thought about being closer to $35 than it is to $44,000 (dual Epyc 6 screen laptop) I would like that (but one split is at $22k and that's...will I not want housing anymore in 2025?)
*\(^___^)/* 😄
I can live without low cost as long as they are performant and smol at the same time.
The best we can get on the ARM side of SBCs is A76 which was announced in 2018, making it a full 5 generations of core design out of date.
Even just a jump to Cortex X1/Neoverse V1 level performance would be a big leap, and while SiFive's latest P870 core would match this there are no such SoC designs using it on the horizon.
Awesome video! I'd love to see more on RISC-V
Best episode so far, the singularity is neigh. Moore please. Giddy up!
Deep dive into architecture of the chip sounds wonderful!
*\(^___^)/* 🐀
This is very exciting, another golden interview, deeper dive please! :)
With all the echo in that office and Jim Keller's tone of voice make the interview segments next to incomprehensible.
Brilliant content. Must feel good to be given some camera time with the CEO. Congrats!
*\(^___^)/* 🐧
So the key to these different RISC-V designs would be a good test suite and a certification process.
10:42 yes please! i would love to see the core explained in detail!
Great job on this video! It's quite the skill you have to pull key insights from people during interviews. I learned more about where AI hardware is going from those two interviews than I did from reading hundreds or articles, videos, and talks/presentations.
definitely would love to see a deeper dive on the uArch, pls
I love the audio quality in the interview
When I close my eyes, I really believe this interview took place in a cave!
yes; info on the microarchitecture - and especially the thinking that made the decisions in the micro arch - would be welcome.
Jim is one of the smartest, sad guy I have ever listened to. Thank you so much for this video.
I loved Jim’s rebuttal to the AI, black box problem (AI decisions are not auditable).
8-wide RISC-V seems like it means serious business. I remember when Apple started building their own custom cores that they were surprisingly wide -- wider than even the latest desktop chips at the time. That seems to have done well for them.
Is there a table somewhere where we can reference/compare the instruction-width of various modern CPU architectures? I feel like an outdated table exists on Anandtech somewhere but I can't seem to find it.
I'd love to see how old and new architectures compare in width.
Afaik Alderlake was the first x86 Architecture with a 6-wide decoder and AMD is 4-wide up to and including Zen 4.
That doesn‘t mean they are limited to that number of instructions per cycle though, as the instruction microcache may be wider.
*\(^___^)/* 🐹
*\(^___^)/* 🐖
@@noergelstein No, all instructions must be decoded first regardless of if they are accessed from the icache or elsewhere. IPC can not exceed decode width
"RISC V is going to change everything." Said some hacker somewhere.
After the tech restrictions with China I expect RISC-V will grow in leaps and bounds especially in China. They have GNU toolchain and can run Linux. From a security point of view RISC-V on Linux from multinationals will be better than using US controlled tech. I personally like the idea of a third instruction set in mainstream not controlled by US. This will improve competition and innovation. The x86 instruction set and extensions have barely changed in 10 years and even ARM is mostly just modifications to old instruction sets apart from the 64 bit extensions. RISC-V ISC already supports 64 bit. But this AI acceleration looks vary exciting I will keep an eye on this company may become a NVidia competitor in the future.
*\(^___^)/* 🐕
*\(^___^)/* 🐒
Ascalon analysis pretty please ❤
How many years do you think it'll take before RISC-V processors can comfortably rival high-power ARM processors of today?
Not many at all. You see ARM put their licence fees up and coupled to the issue with the US banning Western chip technology to China, the country wants to switch to open source RISC-V ASAP. The idea is they can build all the tool chains and compilers from scratch and not be chained to US companies that could be banned from doing business. You see it is like they have their own UA-cam and Google, they want their own home-grown software as well. Chiplets were also something they developed to get around the EUV ban. Huawei built the Harmony OS in record time.
probably something like 5 years, the time for the foundation to finalize the ISA extensions and for designers to refine the different microarchitectures. The issue remains the price though.
10? Which is quite fast if we compare to ARM history.
@@reddeimon475 ARM history is quite different. A lot had to be paved before the 2010 decade, both from the company itself and other links of the chain.
Moreover Risc-V benefits from the community shared effort and experience and a clean design as well as huge interests from non-US companies and countries.
@@PainterVierax The community shared effort is not something that's going to help. Companies like this will come up with proprietary, closed CPU designs based on the RISC V ISA, and have complete control over them, then they can charge what they want. This open ISA is nothing like the open software model. The "community" is very different.
Tents to Rent was what I was reading
It'll be interesting to see the power envelope that these different core designs run in. Also curious in the performance of the cores - though I doubt we will be running vanilla Linux on it :D
Well, you can run Linux on RISC V today, so it would probably run on this too, even if it probably wouldn't be able to utilize any of the closed source IP in it...
@@Luredreier you sure can. it doesn't look like it has storage on it (past firmware) so you probably won't be running Ubuntu or fedora is more what I meant
*\(^___^)/* 😃
Maybe throw Jim´s Mic into the hallway next time. :D
Very good video! I want to know more about RISK-5 and Tenstorrent work.
RISC V, not RISK 5. 😉
*\(^___^)/* 🐴
The past 10 years have been really fascinating watching how designs change based on compute needs and what tradeoffs to make in the full system and all the optimizations at the instruction level on up.
*\(^___^)/* 🦊
Appreciate for introducing this company, i really want to know more about other companies working on the AI Accelerators, thanks for the video.
Absolutely knocking it out of the park. Thank you for your work!
unauditable software sucks no matter who or what makes it. Jim begs the question of machine-generated software by assuming software is not auditable, then using that as a justification for unauditable machine-generated software. We're already drowning in sh*t implementations; machine learning can only compound that.
It would be neat if they could build a consumer graphics processor with RISC-V.
I'm sure it's possible, i just don't know what it would take to make it economical.
yes for deep dive. Also More Jim Keller is like More cores - always good ;)
*\(^___^)/* 😄
More. Yes. Please.
It appears that there is a manufacturer focused on all market segments from IoT, phones, laptop, desktop servers and DPUs. These are exciting times.
*\(^___^)/* 🐕
Sad to see Keller resorting to voodoo at the end of that interview. Modularity is an engineering principle that allows us to reason about a five million line program. Safety critical systems don't run on Windows.
And the guy cautiously omitted that opensource licensing already allows thousands of people to audit that code in search for bugs, vulnerabilities or ways to optimize.
I have seen plenty of medical equipment still being run on windows at least at the interface level (and that too in xp sometimes - the horrors)
The average joe couldn't care less of safety critical systems sadly
Lots of embedded systems run windows or obsolete OS's. It's a major security problem, because your milling machine has a windows NT controller and they don't support it any more, or your switch runs CentOS 7.9 and you can upgrade the OS to Rocky 9 but good luck getting the drivers and switch CLI to run on that because the mfgr doesn't give out source and the drivers are behind a high paywall on the silicon vendor's web site.
@@tristan7216 MS' own EULA says windows is not to be used in safety critical systems. Workers are not supposed to be anywhere within the swing of a mill while it is under numeric control, and CNC machines are user-programmed, it's not windows' place to refuse a command that crashes the milling head into a part
Surely if you buy a machine with a six- or seven-figure price tag, you would also extract some binding commitments about the support of that machine that would last the expected lifetime of its usefulness. Continuing to use something that is unsupported simply because you cannot afford to replace it means you are just one failure (hardware or software) away from bankruptcy.
Awesome
I'd like to see a chat between Jim Keller and Jeff Hawkings 🤯
I want to invest in Tenstorrent right now. In 10 years time they could be something major
Cool gizmo, but I'm putting it along with photonics, germanium substrate and spintronics in the "No it can't run Crysis" bucket.
I'm genuinely excited about RISC-V, I hope it replaces both x86 (32- and 64-bit) as well as ARM (Cortex A and M) in the medium future.
Many companies are gonna move from ARM, as they recently announced extreme price hikes.
Given that RISC-V is already shipping in the billions of cores, that puts x86 at number 3 or number 4 in the market.
@@lawrencedoliveiro9104 The problem with your logic is fairly obvious. X86 has a totally different market to either ARM or RISC V. It's like saying JCB has a tiny share of the entire automobile market. Several of the fastest supercomputers ever made are based on the IBM POWER architecture... and most people have never even heard of that. Most of the other top 500 Supercomputers use x86.
That is unlikely to ever happen, for a variety of reasons.
@@another3997 Following your same logic, you are trying to imply that RISC-V will not be successful until it takes over the construction-equipment market.
16:53 In the open-source world there are several examples of large code bases being successfully maintained over periods of decades. The largest one is probably the Linux kernel, currently at about 20 million lines. They have a policy that nothing stays in the code base without a maintainer to respond to bug reports. If nobody is able to handle problems with a piece of code, then it gets dropped--as Microsoft discovered the hard way, when it was contributing patches for Hyper-V.
The open source software world is very different to the Risc V open ISA world. It's only the Instruction Set that is open. Chip designers are free to implement whatever features and chip designs they want using that Instruction Set. Think of Risc V as like a programming language, the built in commands are fixed by the creators of the language, but the 3rd party software can be written any way the programmer wants. And the language creators have no control over that.
*\(^___^)/* 🐼
Dare I hope for a RISC-V chip that is a direct competitor with the Raspberry Pi RP2040? Including something like the RP's PIO state machines and (as long as I am dreaming) the ability to run micro python programs?
Jim Keller and team solve problems and challenges that move the semiconductor, firmware, compiler, software, server systems future forward at an accelerated rate! Fantastic interviews, insights and roadmap. Thank you TechTechPotato for your diligent efforts.
Would be interested in Tenstorrent plans on leveraging Co-Packaged Optics. (i.e. Ranovus, Poet Technologies).
Something tells me Portable, Mobile, Distributed Cloud Infrastructure is about to become a reality! Distributed microClouds/nanoClouds will be a hoot!!
Such a boon for future Automotive EV Ecosystems that have to derive revenue and profit from innovative cloud products and services.
Good times are coming! Insanity ensues! Ha!!
It would be nice to have a 128 core CPU to replace Intel or AMD offerings in PCs.
Would love to see how "easy" (or hard) it is to port a neural net made for x86 cpu to a tenstorrent compute card. In my case it's written in c++ without any ml specific libraries. Interested to see what kind of structures, algorithms or kind of compute it support for acceleration.
For Tenstorremt I think you need to write your AI in Pytorch. Everyone is already using Pytorch for AI anyway so it is much more efficient for them to focus their efforts on creating tools for Pytorch only.
*\(^___^)/* 😄
Yes please to the deep dive!
I would love a drop in RISC-V desktop processor. Especially dumping the legacy of hidden "features" within the x86 architecture.
At this moment in time, a Risc V desktop machine would be very slow in comparison to both x86 and ARM. The ISA may be open, but the architectural designs and implementations of actual CPUs haven't yet reached maturity. If a company comes up with a competitive design, they still have to make it competitive in terms of price, availability and third party support.
@@another3997 Its inevitable our understanding of computers will radically change. Today's Internet of things comprised insecure, inflexible embedded systems must evolve. The future resides in leveraging the power of the network to offload and distribute compute over a wide area. New systems comprised of low power NvLink or Mellanox style interconnects could alter how humans interpret or view what a "computer" is.
It's possible, our isolated single purpose built machines will be enhanced then superseded by networks comprised of embedded nodes, where openness and security are paramount. As for today, there are legitimate security concerns within proprietary hardware design. For example, it would great to upgrade or deny "features " within the Intel ME.
When it comes to a "drop in" solution for the desktop, low power options focusing on security and scalability would be interesting. New add in cards which promote "low cost" data science would be great! The RTX 8000, A100 are unaffordable, power hungry and overkill for smaller tasks.
LOL, Jim made me lough when talking about the blue screens of Windows! 😊
*\(^___^)/* 😄
I’m more on the side of Casey Moratory, AI is just a good way to find new ways todo normal Algorithms but we first have to bridge the gap between abstract model and reality. It’s also great for very dumb stuff that one wouldn’t want todo themselves anyway. If we don’t find out how to get away with the black box we will be unable to advance as simple as it is.
*\(^___^)/* 😄
*\(^___^)/* 🦔
I’m extremely interested in the architecture of the chip along with more of the design philosophy. I very specific applications in mind.
*\(^___^)/* 🐕
If a larger company wanted to include some of Tenstorrent’s IP/chips on package would Tenstorrent permit them access?
Assuming they become popular and used for a lot of tasks it may be nice to move their package from an add in board to an on package solution.
Yup, that's one business model
I suspect that if this were to happen, Tenstorrent would licence it just like Intel and AMD cross licence x86 technology. The uptake and success then depends on price and licensing restrictions. Which is where you start to lose any benefit of the open ISA... you become wholly dependent on one company.
Will Skynet be run by RISC V? This is vital information for the future of humanity! If I get some ambition and can find information I find palatable I will try to go down the Architecture rabbit hole.
*\(^___^)/* 😄
I want a deep dive into Askelon architecture please 😸🔥
*\(^___^)/* 🐒
I really love all your videos. I'm no where CLOSE to being as smart as you, but the way you talk and present things sure makes me feel smart 😂. Is YT your full time job or do you do things with chip makers?
There's a video for that! ua-cam.com/video/dtG9I3mZlJo/v-deo.html
*\(^___^)/* 🐹
*\(^___^)/* 🐘
@@TechTechPotato haha thanks! 👍
I would also very much like to see a deep dive into how these new chips and how they might be leveraged for robotics.
Thanks for the video.
Jim Keller seems to be at the forefront of advancing CPU tech. AMD to Arm back to AMD to Intel now Risc V and Apple is in there as well.
you forgot intel - upcoming arrow lake is rumoured to use the royal core which had Mr Keller's involvement in development
@@aravindpallippara1577 Jim Keller's list of work on CPUs is long and impressive, and my list was just off the top of my head, forgot Apple and had to edit it in.
x86 and ARM are both gated behind "corporate IP", therefore discourages independent developers to contribute.
RISC-V is opensource. It will have more room to grow.
I remember reading about Jim Keller in a PC gamazine back in 2012 and how he was going to join AMD, when they still sold FX CPUs at a loss.
Never would I have expected that he (with the help of many others of course) could turn AMD around and even surpass Intel on some markets.
I hope RISC-V will have a similar sucess with him on board. ^^
Well, he also went to work for Tesla, and then Intel
@@TechTechPotato Guess that's how Intel managed to finally relase their 10nm nodes xD
So he worked on Apple’s M1? The Apple approach seems to get its performance primarily from tying everything together into a monolithic design which cannot be upgraded. That may work OK for laptops, but it does mean their desktop designs have basically become laptops in desktop cases.
They are supposed to be coming up with a replacement for the Mac Pro workstation line soon, and it’s not so clear this monolithic approach will fly there. That market demands upgradeability and adaptability of hardware configurations.
@@lawrencedoliveiro9104 For Apple, that's not specifically a CPU architectural design, that's a physical construction decision. The CPU core itself could be standalone, or as they chose to do, a system on a chip. The integrated SOC solutions have certain benefits for a company like Apple, as they only design build CPUs build for themselves. There's no technical reason the CPU core can't be separated from the peripheral components.
@@another3997 Except Apple don’t seem capable of producing such designs.
God, so many exiting improvement, but none of them for end users like us.
One of most interesting part was why chiplets, and that you don't have choice. Question is, what Nvidia think about it? Because they more and more looks as giant with clay legs.
*\(^___^)/* 🐒
Dr Cutress, i have a hope you might hear me. The Ryzen 7000 cooking issue is a classic bus driver latchup - a CPU designer like yourself would know what these are unlike just about every other tech on youtube.
DDR5 has its own memory power controller on board so we are getting the classic instance of a bus signal being sent higher than/too close to the bus driver MOSFET VCC or below the driver ground. This is the most common cause of latchup. There may have been no need to have input protection OVP on the memory controller bus driver til now.
Ofc once latchup occurs there is nothing a thermal system can do (even if it isnt taken out by the lacthup), by the time the thermal system can know something is wrong the damage is done. Im pretty sure teh only way to stop it would be power excursion monitoring.
Im not in CS like yourself, im a lowly engineering technician in audio/embedded but this one is pretty obviuos to me
Fwiw, I'm not a CS. I'm a chemist 👌
he right about auditing in projects of that scale
Still need open source synthesis tools and whatever the software is to used to physically modify the actual doped area.
You needed a 2nd mic:/
A RISC-V Hardware Developer needs to get with someone like a 'System76' (Linux Desktop/Notebook Manufacturer) that can begin wide spread availability of Linux/RISC-V based Workstations and other Consumer Devices out to the public. That is unless this plans to stay a Server/Micro-Controller Chip and miss out on the 'middle ground' entirely.
System76 don’t actually build their own laptops, they get them from makers like Clevo.
@@lawrencedoliveiro9104 I just meant that (LIKE APPLE back in the day) they have a say in the design and component choices onboard) to meet OS requirements. I should remember that this is the Internet and everyone has a PHD along side Dr Cuttress.
My PhD must be in knowing who Clevo is.
I heard there would be one open source Core design ... is that enough given the boost RISC-V has given them? $0.02
I love the concept of Risc-V, but I have yet to see any way to actually take the files and turn them into a working computer that doesn't involve going right back to a centralized market, since the circuitry to go between the peripherals and the CPU is only manufactured by a handful of Chinese companies, and there is *zero* documentation available on DIY-level motherboard design.
What files? RISC-V is just free instruction sets for a CPU architecture. Nothing else.
More Uarch details would be much appreciated
is there a roadmap for a tenstorrent ipo?
Not yet
Is there a future in programming? I’m a tired the way through my degree and have a good job *for now* and I’m worried about my future
@@leeroyjenkins0 that sounds like the natural outcome. But the thing is since GPT, being more productive at existing tasks is how I’m *currently* using it. That’s the starting point, so it might only evolve from there and displace what I’ve worked for. I busted my arse for years labouring and for a while in service too so after pivoting career and really finding happiness in my work it’s a bit of a kick in the guts to not know what’s coming.
Risc-V is also interesting for companies that do not want to end up like Huawei.
People get it all wrong. Huawei can still use ARM instruction set. Other China companies also still use ARM instruction set. It is chip manufacturing. They have problem in getting their designs manufactured by chip manufacturer like TSMC. RISC-V cannot help. RISC-V chips still needs to find chip manufacturers to produce.
I’d love to see posits running on RISC V
avoid deep prediction pipelines, make more cores with vectorization instead to amortize instruction latency
Mainframe class IO at last? For so many cores, there's obvious hurdles with cache coherency unless you go NUMA. And then there's the compiler backends..
I don't completely follow Jim Keller's argument about auditable functions and AI. We know that AI is non-deterministic. Maybe that can be changed for software development processes, but doesn't that imply a static training set?
For AI if you have your dataset you can check if the network functions correctly. Yes it is not deterministic but you can performed statistical analysis to see how much the NN model is working correctly. You can do this analysis to any arbitrary precision you want limited only by your dataset. So if your data allows it you can determined if the AI is correct 99.9999% of the time. Handcoded program don't have this powerful tools and debugging is usually an esoteric witchcraft.
Glad to see an alternative to Nvidia for AI acceleration
Do you think we will ever see them on a regular desktop?
If you have a regular desktop, then yes. I have several CPUs sitting on my desktop.
Super interesting content!! Thank you!!!
I'd love to see a microarch deep dive.
Would each chiplet have its own RAM, and its own storage, say 1meg RAM and 1meg storage?
Xilinx and I think AMD used chiplets, it works. Xilinx can ship parts with different mix and match capability like HBM. And it probably boosts yield and reduces design time. But the interconnect (NOC) adds complexity. You don't *have* to use chiplets tho, you could go the other way: Cerebras.
*\(^___^)/* 😄
A defect on a chip is not as critical. Much smaller investment for each piece.
What could be a potential use in the consumer space? Could it be an AI add-in card for PCs, analogous to perhaps how many years ago Ageia Physx was intended for physics simulation?
*\(^___^)/* 🐼
I just had a thought IAN, risc-v is an open source instruction set for CPU, could someone release an open source GPU architecture so both cpu and gpu are open source? Takecare.
Whether the ISA is open or closed, makes very little difference to the end customers. Because the companies designing and building chips using that ISA are still able to make their chip designs proprietary and closed source. Just like a program written using an open source language doesn't have to be free or open.
Do a deep dive on their arch and compare it to what EuroHPC folks are doing with their risc-v design. That is funded with EU money so most of the info should be accessible. Otherwise come to ISC in Hamburg in one month where you'll be able to meet EuroHPC people and learn things firsthand.
would be interesting but sadly this is a sponsored video. Nowadays Ian just presents products from companies hiring him for consulting job.
15:04 Colossus: The Forbin Project
Ian, How does it compares to Cerebras?
How does open-source work for hardware ? Do you have to put the designs made from open source IP into the public domain, just as a GPLv3 software license would enforce ?
I'm not sure what specific license they use, but it's not GPL. It allows proprietary designs. RISC-V is an ISA, i.e. a list of possible instructions and their binary format, including several optional extensions for things like floating point math, vectorization, etc. Designing the circuitry to interpret those instructions and do the actual work, that's up to the manufacturer, and designs will vary widely in how they do that. That part does not need to be open source. Companies don't have to publish their verilog files or litho masks. All it means is that manufacturers don't have to pay royalties to use that instruction set.
@@DFPercush Risc V is pushed by Berkeley, so the ISA is BSD licensed, which is known as a permissive license that allows forks to get proprietary licensed (unlike a copyleft license like the GPL).
There are many Risc-V cores designed, some are opensourced but the majority are closed sourced.
So it has all riscv instruction sets?
Excellent video. Thank for the content and I wish Jim and his company success in ramping risc-v designs and bringing them to market.
When you mention they have laptop designs, are they windows laptop designs or some other OS?
Finally, Santa Clara, isn’t that where intel is? So Jim is just down the street eh?
Almost everyone has a Santa Clara office. But on laptop designs, probably not Windows, but a Chromebook or Linux most likely.
@@TechTechPotato I figured that it would be Linux-based as my understanding was that windows didn’t have a risc-v port. I wish them success. Good to see Jim do good things.
*\(^___^)/* 🐰
*\(^___^)/* 🐁
With venture capital and research money getting tighter, open hardware is more important today.
2:45 if anyone should be able to use Ai-software to clean up soundquality it's you 😀!
did anyone else see the ceo of that company start to get mad when he thought TTP said AI is inferior to humans?
uArch breakdowns please!
I love this content. I watch it when my wife is out the house.
*\(^___^)/* 😄