Second that. Deep knowledge like this used to be available only to those few who worked directly alongside those who accumulated it. Lots of wisdom has been lost and relearned only to be lost again this way. Making it available to a wider audience is invaluable. And it doesn’t hurt that you’re one hell of a teacher. Really glad I found your channel.
You are FANTASIC at explaining concepts to an audience who is unfamiliar. I noticed around 2:20 you started to say "it" but then corrected yourself and said "the computer". I started a contract job in IT security several months ago and the guy who's training me explains everything with pronouns. He will say things like "When it comes in, it forwards it to the right one" and I constantly have to slow him down... and of course, I'm the dummy for not understanding. I can't wait until my contract comes to term.
I sometimes slip and use 'it', but its actually one of my pet peeves, when someone is lecturing and says "just do this to it", i hate that because it derails me from paying attention, trying to figure out what 'it' is. Thank you @50PullUps glad you share the same concern over the use of that term 'it'.
Isn't it amazing that, trough only one lifetime, a person learned to use in production environments a Honeywell 2020 and modern operating systems such as Linux distros? Is really a treat to get to open an app in a mobile device that can do more than any supercomputer of the late 70s and watch a video of you, DJ ware. Thank you!
He knows the entire history of computing, experienced a lot of it firsthand and more impressivelyy seems to remember pretty much everything. DJ Ware's channel is a bit like drinking from a firehose but if I was only allowed to watch one Linux UA-cam channel it would absolutely be his. The changes that happened over the course of DJ Ware's career really are nothing short of amazing. What's also been amazing me recently is the thought that relatively cheap Casio digital watches that are more or less exactly what they were in the 80s or even 70s are still competitive and still loved by people in places like NASA. Anyway, thanks DJ Ware for another great video.
I would stay after work sometimes just to learn more about that old H2020, I once wanted to see what a tape sort was like (I always did mine to disk), but i was curious how bad they were. So I wrote a program to do it, and while I was running the Burroughs sales guy showed up to take my boss out to dinner, and came into the computer room to say 'Hi', he stopped and looked at the tape drives, spinning forward and back and said..."Wow, that looks like something out of TV shows that always showed tape drives doing something". Yeah, he was right that's the view of most had before getting to play with them ourselves.
@@CyberGizmo Hello DJ Ware. Although it's unrelated to this video, I went down a rabbit hole about watches and discovered that there was a Linux smartwatch developed by IBM and later they collaborated with Citizen. They called it Watchpad and it came in several different versions over the course of its development. They started it in 2001 and continued with the project for 3 or 4 years before it got canned. Yesterday I came across an amazing lecture here on UA-cam by one of the developers. Although this pioneering smartwatch was not a commercial product, it seems to have really pushed the boundaries of what was possible at the time and directly led to the development of other technologies and influenced later smartwatches. I wonder if you have ever talked about this or have any interest in this project...
keep going DJ your years of experience put you way ahead of so many other tech channels. you have a soothing voice too. lots of things working in your favor!
I really like the description of the Honeywell software loading. Today, our microcomputers such as the Arduino (and derivatives like the SMT32) run two specific programs when they are turned on. The microcomputers I am referring to, complete with motherboard, can be as small as 20mm x 20mm, but can be large enough to run a 3D Printer, a CNC Machine or an Automobile. The operation is the same. First the bootloader (same as the executive) runs a quick power on check, and then points to a section of memory, where the program exists. The programs, typically written in C or C++, are compiled on a workstation (using, Windows, MacOS, or Linux operating system). The compiled program is then loaded into the flash memory of the Microcomputer. Of course the first line of compiled code is placed where the bootloader points to when it is done the power on sequence. There is no Operating System, and all instructions run in real time.
Absolutely great channel. Content that is highly informative and is easy to follow. Wish instructors were this way, you would learn everything there is to know in the world.
14:46 not having management of resource allocations is not a problem when you have a sea of machines, you just allocate machines to programs in unikernels. The allocation system runs besides the entire machine itself in another machine. That makes much more sense than the "mainframe" approach of multi-user multi-purpose operating systems, that would make sense if computers were expensive, but they are incredibly cheap.
Unix, Minix and Linux. Sweet old memories... I grew up being a MS Windows admin, swaped to Linux a couple of years ago tho... And I'm not going back. You gonna love the "flat text file" philosophy...
I had a similar reaction to you on discovering virtualisation. "isolation and multiplexing is the entire point of a kernel, but modern operating systems fail at both?" FWIW - you mentioned Kata - it's not a unikernel, it's more like a Rust SDK for seL4 at this point, but it does have process isolation.
This sounds sort of like a hardware server. So the first kernel really serves cpu slices and pci devices. It would help if there was a connection standard, then you would get slave OSs that use the standard to get resources. I always wondered how like in a pure VM, the cpu would ideally see the environment as bare metal. Also in the vm all the traditional devices like timers would be there. I don't know these days what ISA features are expected to exist in hardware, or you can just use cpu features and pci devices. I started using a vm in Fedora to run some Windows applications in non 3d acceleration. Also I have an Audigy 2 pci card in my system and I channel the motherboard SPDIF through it so I can use in line 10 band EQs going to my receiver here. I use the KX project drivers.
This is such an interesting project. I was looking for an operating system to add as a "subsystem for windows" just so I can learn how subsystems work and what kind of cool hack like "WSL" can we also make. An Unikernel application, that would be cool.
5:13 It could be Plan 9. Supported Xen (as a guest) possibly earlier than some bigger OSes. Lguest, too (as a guest), though that support is long gone.
I was more interested in the unikernel because of what I had read about EROS, KeyKOS, Coyotos, et al, and what was going on in RedoxOS (which had also started but was not public yet).I tried to get on the kernel developer list that was used to bring up the MirageOS project when that was 1st made public, but got bumped to the MirageOS list. BTW nothing of interest happened there except that it broke to some degree every time the underlying packages were updated (once or twice every couple of months), or for a couple of weeks when someone new came along and was trying to bring up MirageOS on a new system.
I installed it first time this week, and it went smoothly this time, but yeah I see your point, they recently (last update) started moving to dune for the build out perhaps it will help, or perhaps will cause more issues will have to try it and see.
Thank you for this fascinating introduction. I wonder how many notions for these tools come from mach kernel development? The idea of breaking down each kernel component to an object that can be added to the microkernel seems to be in line with these ideas as you present them. From what I understood mach kernels are difficult because of the messaging necessary to make them all work but I guess if you take the hypervisor approach it can have a macro kernel reducing complexity of the unikernal eliminating most of the issues seen in mach.
do you mean to manage them, Bill? Or for them to be a server? The last question is yes, absolutely. The first there is the albatross project which might do what you want here is the github link github.com/roburio/albatross
kata in the context of unikernels? seems like a miss... kata is a runtime to run container images in light VMs, on provided by kata tuned linux kernel. based on similar ground as kata was virtlet, runtime for VMs which are mimicking containers, which in fact was used to run unikernels: ua-cam.com/video/L-QrxDJSZBA/v-deo.html virtlet was made to run any kind of VMs, so we had as an example windows based VM which was providing IIS as a kubernetes service ;)
So they still require an operating system ["hypervisor"] for the hardware. Sounds more like a special subset of containers than an executive kernel. I'm probably missing something though as I'm solidly in the amateur category. Setting aside security, is there an open source library of code for beginning bare metal programming? I mean if a person has a reason to make a bare metal program for a specific performance oriented task like small scale HPC[without network access], but doesn't want to totally reinvent the wheel to interact with a commodity x86 platform. (Maybe a stripped down OS would accomplish the same effect?) I mean super stripped, like no terminal or anything extra, just power on, it runs the calculation, writes the result to disk, then halts/powers off. Reboot with an OS on another drive to read the results.
Hi MyTech, in MirageOS they use the hypervisor to maintain separation from both traditional VMs and other unikernels, and because there are so many different models/makes of hardware it would be pretty heavy weight to put them into each unikernel application. Yes, there are stripped down OS's which do bare-metal apps, are you working on an Edge solution, or Real Time problem?
@@CyberGizmo Ah yeah I was looking at the benefits it from the wrong angle. Unikernals are more like VM's stripped down to run a single application. No real time or edge problem, mostly just curious what efficiency/speed/memory gains I could get on a calculation and data analysis problem, which I expect will take several weeks per run on a single socket workstation. I'm not to the optimization point where I would bother putting in the extra programming effort to go bare metal. I'm just adding to my list of availible options.
Nice teaser. I am thinking .. an application runs perfectly good and starts and stops perfectly safe. But what about, say, daemons, would they not own too much resources forever? Sorry: I am certainly missing a point.
Yes the daemons would, that's why the unikernel apps are single purpose and very small in foot print, it really takes some planning to keep the do one thing and do it well point of view, and no you aren't missing the point at all
Only problem they face now is that , since each kernel is in its own network space it harder for them group together for a efficient network connection..
your channel is a true goldmine 🙂
Wow, thank you!
Yeah, he's not bad.
Ex IBM guy, which explains the
Attitude.
Second that. Deep knowledge like this used to be available only to those few who worked directly alongside those who accumulated it. Lots of wisdom has been lost and relearned only to be lost again this way. Making it available to a wider audience is invaluable. And it doesn’t hurt that you’re one hell of a teacher. Really glad I found your channel.
You are FANTASIC at explaining concepts to an audience who is unfamiliar. I noticed around 2:20 you started to say "it" but then corrected yourself and said "the computer".
I started a contract job in IT security several months ago and the guy who's training me explains everything with pronouns. He will say things like "When it comes in, it forwards it to the right one" and I constantly have to slow him down... and of course, I'm the dummy for not understanding.
I can't wait until my contract comes to term.
I sometimes slip and use 'it', but its actually one of my pet peeves, when someone is lecturing and says "just do this to it", i hate that because it derails me from paying attention, trying to figure out what 'it' is. Thank you @50PullUps glad you share the same concern over the use of that term 'it'.
Isn't it amazing that, trough only one lifetime, a person learned to use in production environments a Honeywell 2020 and modern operating systems such as Linux distros?
Is really a treat to get to open an app in a mobile device that can do more than any supercomputer of the late 70s and watch a video of you, DJ ware. Thank you!
He knows the entire history of computing, experienced a lot of it firsthand and more impressivelyy seems to remember pretty much everything. DJ Ware's channel is a bit like drinking from a firehose but if I was only allowed to watch one Linux UA-cam channel it would absolutely be his. The changes that happened over the course of DJ Ware's career really are nothing short of amazing. What's also been amazing me recently is the thought that relatively cheap Casio digital watches that are more or less exactly what they were in the 80s or even 70s are still competitive and still loved by people in places like NASA.
Anyway, thanks DJ Ware for another great video.
I would stay after work sometimes just to learn more about that old H2020, I once wanted to see what a tape sort was like (I always did mine to disk), but i was curious how bad they were. So I wrote a program to do it, and while I was running the Burroughs sales guy showed up to take my boss out to dinner, and came into the computer room to say 'Hi', he stopped and looked at the tape drives, spinning forward and back and said..."Wow, that looks like something out of TV shows that always showed tape drives doing something". Yeah, he was right that's the view of most had before getting to play with them ourselves.
Hahaha, and you are welcome.
@@CyberGizmo Hello DJ Ware. Although it's unrelated to this video, I went down a rabbit hole about watches and discovered that there was a Linux smartwatch developed by IBM and later they collaborated with Citizen. They called it Watchpad and it came in several different versions over the course of its development. They started it in 2001 and continued with the project for 3 or 4 years before it got canned. Yesterday I came across an amazing lecture here on UA-cam by one of the developers. Although this pioneering smartwatch was not a commercial product, it seems to have really pushed the boundaries of what was possible at the time and directly led to the development of other technologies and influenced later smartwatches.
I wonder if you have ever talked about this or have any interest in this project...
keep going DJ your years of experience put you way ahead of so many other tech channels. you have a soothing voice too. lots of things working in your favor!
Thanks Michael I so glad that you and others find this type of video helpful. It helps me too, pushes me to learn new things and try them out.
I really like the description of the Honeywell software loading. Today, our microcomputers such as the Arduino (and derivatives like the SMT32) run two specific programs when they are turned on. The microcomputers I am referring to, complete with motherboard, can be as small as 20mm x 20mm, but can be large enough to run a 3D Printer, a CNC Machine or an Automobile.
The operation is the same. First the bootloader (same as the executive) runs a quick power on check, and then points to a section of memory, where the program exists. The programs, typically written in C or C++, are compiled on a workstation (using, Windows, MacOS, or Linux operating system). The compiled program is then loaded into the flash memory of the Microcomputer. Of course the first line of compiled code is placed where the bootloader points to when it is done the power on sequence. There is no Operating System, and all instructions run in real time.
Can't wait the next part. Sir, you're doing amazing job explaining all that
Thanks Alx I am just glad people like you find this type of video helpful.
@DJ Your vids are always good and informative and certainly appreciated
Absolutely great channel. Content that is highly informative and is easy to follow. Wish instructors were this way, you would learn everything there is to know in the world.
Hi @DavidGrishko1893 nice to see you on the channel and thank you for the kind words!
14:46 not having management of resource allocations is not a problem when you have a sea of machines, you just allocate machines to programs in unikernels. The allocation system runs besides the entire machine itself in another machine.
That makes much more sense than the "mainframe" approach of multi-user multi-purpose operating systems, that would make sense if computers were expensive, but they are incredibly cheap.
Unix, Minix and Linux. Sweet old memories...
I grew up being a MS Windows admin, swaped to Linux a couple of years ago tho...
And I'm not going back. You gonna love the "flat text file" philosophy...
I had a similar reaction to you on discovering virtualisation. "isolation and multiplexing is the entire point of a kernel, but modern operating systems fail at both?"
FWIW - you mentioned Kata - it's not a unikernel, it's more like a Rust SDK for seL4 at this point, but it does have process isolation.
True about Kata, and yeah was mentioning it from the standpoint of process isolation, thanks William
This is awesome. It has been quite some time that the youtube algorithm actually worked for me, today it did. Interesting topic & well told.
I needed that Bones reference just now... Thanks
My favorite line from that movie
This sounds sort of like a hardware server. So the first kernel really serves cpu slices and pci devices. It would help if there was a connection standard, then you would get slave OSs that use the standard to get resources. I always wondered how like in a pure VM, the cpu would ideally see the environment as bare metal. Also in the vm all the traditional devices like timers would be there. I don't know these days what ISA features are expected to exist in hardware, or you can just use cpu features and pci devices. I started using a vm in Fedora to run some Windows applications in non 3d acceleration. Also I have an Audigy 2 pci card in my system and I channel the motherboard SPDIF through it so I can use in line 10 band EQs going to my receiver here. I use the KX project drivers.
Thanks Mathew is always cool to hear how people are using their machines to get more out of them.
This is such an interesting project. I was looking for an operating system to add as a "subsystem for windows" just so I can learn how subsystems work and what kind of cool hack like "WSL" can we also make.
An Unikernel application, that would be cool.
Excellent and good to hear some ol history
Congrats, we came full circle!
5:13 It could be Plan 9. Supported Xen (as a guest) possibly earlier than some bigger OSes. Lguest, too (as a guest), though that support is long gone.
I was more interested in the unikernel because of what I had read about EROS, KeyKOS, Coyotos, et al, and what was going on in RedoxOS (which had also started but was not public yet).I tried to get on the kernel developer list that was used to bring up the MirageOS project when that was 1st made public, but got bumped to the MirageOS list. BTW nothing of interest happened there except that it broke to some degree every time the underlying packages were updated (once or twice every couple of months), or for a couple of weeks when someone new came along and was trying to bring up MirageOS on a new system.
I installed it first time this week, and it went smoothly this time, but yeah I see your point, they recently (last update) started moving to dune for the build out perhaps it will help, or perhaps will cause more issues will have to try it and see.
I m in love with this guy, but willing to break up if no second part this episode.
LOL, well i guess i best get working on the 2nd part then
Great content.
Have a great Thanksgiving.
Thanks tmendoza and Happy Thanksgiving to you as well!
What an interesting technology, thanks so much DJ!
Welcome Albert
Thank you for this fascinating introduction.
I wonder how many notions for these tools come from mach kernel development? The idea of breaking down each kernel component to an object that can be added to the microkernel seems to be in line with these ideas as you present them. From what I understood mach kernels are difficult because of the messaging necessary to make them all work but I guess if you take the hypervisor approach it can have a macro kernel reducing complexity of the unikernal eliminating most of the issues seen in mach.
Another great vid. Thanks! 👍👍
Thank you Rashie, very kind of you
Is it possible to use a unikernel for running servers? It appears that it would be more secure if you could.
do you mean to manage them, Bill? Or for them to be a server? The last question is yes, absolutely. The first there is the albatross project which might do what you want here is the github link github.com/roburio/albatross
kata in the context of unikernels? seems like a miss...
kata is a runtime to run container images in light VMs, on provided by kata tuned linux kernel.
based on similar ground as kata was virtlet, runtime for VMs which are mimicking containers, which in fact was used to run unikernels: ua-cam.com/video/L-QrxDJSZBA/v-deo.html
virtlet was made to run any kind of VMs, so we had as an example windows based VM which was providing IIS as a kubernetes service ;)
Yes i know, I was not using it as a unikernel, just an example of other types of containers
So they still require an operating system ["hypervisor"] for the hardware. Sounds more like a special subset of containers than an executive kernel. I'm probably missing something though as I'm solidly in the amateur category.
Setting aside security, is there an open source library of code for beginning bare metal programming? I mean if a person has a reason to make a bare metal program for a specific performance oriented task like small scale HPC[without network access], but doesn't want to totally reinvent the wheel to interact with a commodity x86 platform. (Maybe a stripped down OS would accomplish the same effect?)
I mean super stripped, like no terminal or anything extra, just power on, it runs the calculation, writes the result to disk, then halts/powers off. Reboot with an OS on another drive to read the results.
Hi MyTech, in MirageOS they use the hypervisor to maintain separation from both traditional VMs and other unikernels, and because there are so many different models/makes of hardware it would be pretty heavy weight to put them into each unikernel application. Yes, there are stripped down OS's which do bare-metal apps, are you working on an Edge solution, or Real Time problem?
@@CyberGizmo Ah yeah I was looking at the benefits it from the wrong angle. Unikernals are more like VM's stripped down to run a single application.
No real time or edge problem, mostly just curious what efficiency/speed/memory gains I could get on a calculation and data analysis problem, which I expect will take several weeks per run on a single socket workstation.
I'm not to the optimization point where I would bother putting in the extra programming effort to go bare metal. I'm just adding to my list of availible options.
Really nice👍
Thank you
Thanks DJ 👍
Welcome, anytime
Nice teaser. I am thinking .. an application runs perfectly good and starts and stops perfectly safe. But what about, say, daemons, would they not own too much resources forever? Sorry: I am certainly missing a point.
Yes the daemons would, that's why the unikernel apps are single purpose and very small in foot print, it really takes some planning to keep the do one thing and do it well point of view, and no you aren't missing the point at all
Only problem they face now is that , since each kernel is in its own network space it harder for them group together for a efficient network connection..
To me, unikernels seem a lot more like embedded RTOSs, except that all hardware access is mediated by a hypervisor.
well kinda different workloads don't ya think? The hypervisor is there is maintain separation between unikernel apps.
The current unikernels space is Fake they are mostly implemented via vmm virtual machine Monitor Interfaces
*Thank you!*