Once Jon has finished his language, his OS, his chipset, fixed production line manufacturing of components, solved global supply chain inefficiency, invented renewable computing resources, and resolved the problem of labor exploitation, then his next game is really going to run like greased lightning
Oh well. We did this like a decade ago. The L4 is a mirco kernel. It used shared memory to do synchronized IPC in a very efficient manner. It used namespaces and capabilities instead of absolute paths to resources and there are neat tricks like user level page handler. It was a very great time, but it made me a bit depressed since the major players like MS and linux - even now are not able to pick up those advances. If you want to look up those things, I think the L4 project page is still up (although it might be a bit dated now). Another OS to look at is Fuchsia (which I think borrowed at least the user level page handlers from L4)
@@kelownatechkid the kernel is MIT and user space is BSD or Apache. Obviously the reason is that they want to support phone providers developing closed hardware drivers, which isn't great, but that's already the situation in Android for various dumb reasons (the TIVO box situation) So no, not really a step backwards.
The high number of processes is due to the Unix philosophy. Maybes it's less than 100 on stock Ubuntu but you can easily move that number down or up depending on what you use or what daemons or services you enable
But one thing that is missing from that philosophy is a standard reliable way to start small background tasks and have data pre-fetched and cached out side of it in a configurable way. To many processes are just waiting on some event like a connected USB device which might never happen. I don't need all those deamons running (sleeping) and requiring precious boot time. But the OS must have a way for the process to quickly start up when an event happens. Processes need to be really light weight again. The biggest problem there is processor cache invalidation upon context switching. I mean, lots of boot time is spend to load state into memory that is the same every time. It amounts to a couple of MBs of important data which could be pulled from disk in less than a second if the OS (or Init) had some more knowledge of what all those deamons need. It is the same problem with JVM like virtual machines and script languages. They need to compile the same code over and over again. And they also have various solutions bolted on to solve these problems. These might be generalized.
If you think about it, if the operating system is done in this way, and metadata is kept in the binaries, there could be multiple operating systems that all use the same apps. "Drivers" are apps that know machine specific things, and comunicates with mmaped pages, so drivers could still exist to smoth over the differences between kernels.
@@RenamedChannel I grew up using DOS, and I worked for GameTap on the DOS games emulator, so I'm pretty familiar with DOS internals. Yez, it does work this way to some extent; DOS and DPMI work through software interrupts, which can switch contexts to the driver when needed. File system process sharing and isolation isn't something that's done in DOS, unfortunatly. Allocating memory through DPMI and talking to the hardware was nice, but it wasn't technically part of the base OS. DesqView (without X) was one of the best user interfaces that I think I've ever used becides Plan 9's. You open a new window, and whatever command you gave can own the window or pass the window to the next process. You could open another Window with a different command, also.
@@franciscofarias6385 IIRC TempleOS does run HolyC functions from the command line though which seems to be what Jon was mentioning in this clip. TempleOS was never meant to be any kind of mainstream OS, because it's written for fun it doesn't do pointless things like security, you have to see it for what it is.
@@pleggli Tempel OS was never meant to be anything but the delusions of a sick man. Don't get me wrong, Tempel OS is very entertaining, but it's not a serious tool.
If anyone here is interested in OS design of the likes that Jonathan Blow just lined out, I can seriously recommend having a look at the design of the L4 micro kernel, because that's literally a kernel which considers & implements all of these aspects: the kernel is only supposed to be a small piece of code doing basic hardware ressource management (adress space, i/o space, cpu & affinity time allocation while enforcing basic access rights & privilege separation of processes but doing nothing else other then that). The only three problems L4 has are a) hardly anyone ever used it because b) hardly anyone ever knew about it because d) its code base is an outdated & semi-abandoned academic project & e) it's just a (micro)kernel, without either driver or user space libraries or programs, so someone would need to build (modern) driver infrastructure, basic system admin tools & basic ui to even being able use it. However, if anyone's planning on build an entire new OS using the aforementioned concepts, then at least there already IS an exceedingly well designed kernel which you could use, either directly or more likely at least rewrite it in a more modern & up-to-date way (using modern language & tool sets) - which in the specific case of L4 is NOT a big undertaking because the entire original source code of the entire kernel is just ~5000 lines of C code - absolutely tiny compared to other software projects.
i would love to see the complete interrupt cue and cycle for a modern OS, even an old OS like XP. I get the feeling there must be thousands of interrupts going on, before your code even gets a sniff, if the OS gives it time.
ever try to SIGINT a running application and have it take a couple seconds to stop? this is what thread priorities are for almost every modern app you use in practice will sleep, aside from command line pipelines
Some guys criticize userspace, that things like peripharicals should not use userspace. But the fact of the matter is that userspace saved me a lot of times. I don't have to build a driver each time I want to talk to a USB device. And just using libusb makes the code much more portable and easier to compile (and install). Jonathan has a good point, and I'm glad he thinks that way. Only people that never had to communicate with and control USB devices think that everything should be done at the kernel level. It is just ludicrous.
I now want a talk with Jonathan Blow and Linus Torvalds. It seems liek those two just might be able to communicate clearly and have an argumentative but productive discussion.
Terry Davis blows them all away. Dude wrote a whole os, with drivers, file system, kernel, shell, graphics, and apps, and did it while fighting insanity. Torvalds wrote a kernel? Pretty good, but not true greatness.
I'm not sure I want each process talking directly to network card, there needs to be some intermediate system to controll access to the device, tell other processes when it is busy, etc.
But it would be even better if that was done in hardware and we are somewhat getting closer to that in some areas. There already exists ways to share PCIE devices transparently between multiple consumers (SR-IOV). The networking interface simplu appears as separate devices for each consumer. It's just one thing that could be leveraged by a forward looking OS kernel.
You know he's a good programmer because he's always trying to simplify things, and use less resources. A lot of our OSes etc. were not developed from the ground up with good security or even good architecture, they just kind of evolved and things were bolted on. Because of backwards compatibility etc. and hidden OS features, the NTFS filesystem has some crazy amount of attributes for file access control, I think it was close to 30 different ones.
The problem is, starting over from scratch only this time doing things right just results in another pile of code that's been upgraded, maintained etc etc. There's no such thing as "ground up, secure, great architecture" for any serious, big software product. Blow has no clue about shipping products that billions of people use. He's sold a few non-mission critical games. And that small bit of success has gone to his head to the point where he then deludes himself he has all the answers to every computing problem from programming languages through to OS. He doesn't. You know thousands of people have spent thousands and thousands of hours actually developing OSes. Often, even in spite of this, people still don't want it on their desktop. So, you know, it's easy to talk about how an OS should work but much, much harder to get to the stage MS, Android etc are at with an OS that billions are actually using. Similarly for servers etc etc. Thinking you know better is the sign of a moron (or perhaps like the guy who wrote templeOS, mental instability)
@@michael1 They say you can't have genius without a scoop of madness. And I say that as someone 2 years into making the OS described in the video become reality. Why do nearly all the successful tech founders have some form of autism? Because everyone who doesn't gets talked out of their creative and innovative ideas.
@@michael1I mean, touche, but let's not talk about this like it's just 'a little business success'. In terms of surpassing technical challenges, Jai seems to do a lot of incredibly impressive stuff, and part of why it's impressive is Blow's approach to developing it - what he thinks a programming language should or shouldn't do. Now, just because his theories seem to have accomplished something here doesn't mean they're going to accomplish something in the OS space, but I think it would be very easy before Blow made Jai to say "Thousands of people have spent thousands of hours making programming languages, get off your high horse." So while I agree he's getting a bit ahead of himself, he's also demonstrably done some amazing things I that I don't think should be ignored. I think seeing whether Jai gets more widely adopted after this game ships is going to be a very interesting test of its viability as a product, and Blow's ability to sell it.
@@michael1if these people do know better, hopefully that means they'll start fixing the problem where input is only processed after the display updates(which can be a long delay nowadays) resulting in misclicks
I love the small efficient operating systems from the 70s like CP/M, RSX11, early BSD and VMS. They were lean and to the point. Even OS/360 that was considered huge, is small and lean and to the point compared to today’s systems.
He want an OS with fewer processes yet he wants drivers to run in user space which I don't disagree with but it will probably results in more processes. Ok 400 is maybe a bit much, my Linux boots to the window manager below 100 processes and at least 20 of them are development related stuff that isn't really needed for a generic user. When I start chrome though I quickly get 70 new processes.
I wonder if he also had the class of operating design and implementation following Professor Tanenbaum’s book. When I studied EE that part of design and implementation of an OS was the best thing I’ve learned. Making the SBC running a Z80 (which I loathed as a 6502 and 68000 user) was the trivial part. But it was great to get a basic simple OS that could dynamically load binaries and swap between them with cooperative multitasking. But it also taught me something viable. And that is that for most embedded systems you don’t need an OS and it will just make things bigger, bulkier, less efficient and more error prone.
This discussion happened decades ago with the Tanenbaum-Torvalds debate. What he's arguing for is similar to Minix, with elements of powershell mixed in. Neither gets far.
I think you have to go a step further than that to get to something fundamentally better. Linux's advantage wasn't really the more monolithic approach (and Tanenbaum wasn't wrong about the drawbacks) but that it was open for hobbyists and that it stayed with POSIX. As history has shown that was the right call to get successful. That doesn't mean there couldn't be something fundamentally better that throws a bunch of old ideas overboard. Whether or not such a product could successfully be pushed onto the OS market is another question. Although I think there still might be room if you go via mobile devices. A mobile OS that doesn't push you into the hands of Google or Apple, respects both your ownership of the device and your privacy, isn't completely defined by the availablity of commercial software and runs on a device that on average gets replaced every other year anyway might conceivably have a chance right now. And speaking of mobile devices: Some of the things iOS brought to the table, even if just on top of unix, are worth considering on a more fundamental level. Like apps generally running in tight sandboxes with strictly limited communication (be it access to files, certain hardware, networking, other apps or whatever) controlled by the user. It reminds me of C++. It's obvious for anyone willing to look that there is aspects at every level that sucks to a degree that it's constantly flirting with having to be declared plain broken but everybody argues that that's the best it can be and you just have to throw layers and layers of makeup over it and sell those as revolutionary. And then along comes Rust and proves that most of those assumptions were bullshit if you were just willing to throw enough of them out and tackle things from a fundamentally different perspective.
@@juliuszkopczewski5759 It's certainly more challenging to make fast, especially on x86, since you'll have additional context switches. But if L4 can reach 90-95% of the official Linux Kernel's performance while maintaining compatibility it probably wouldn't be unrealistic for something bespoke to be on equal footing. With that being said, my bigger point is exactly that there might be an architecture to be figured out that's fundamentally outside the box of current common wisdom about how things should be done. That's why I made the comparsion to Rust and what it did for systems programming. It turns a couple of basic assumptions from half a century ago on their head, reintroduces a couple concepts that completely fell out of favor, adds a couple of original ideas and allows to approach programming on bare metal with a completely different mindset.
FreeBSD and Mac are really good OSes. But they can't be ideal OSes. We can only come as far as Mac for general usability but that too is hard for some users.
I've played around with Linux From Scratch a few times. It's a lot of fun. I really like LFS and Gentoo as my favored Operating Systems, but sadly I have spent most of my life on Windows systems. It's a pain to have to have multiple operating systems to use certain software or games, but I think that is what I'm going to do sometime soon with my new 1TB M.2 drive when I get around to it, and then just have a separate Windows drive, or maybe see how WINE has come along, for games and other software that relies on Windows.
I can build an exe on one PC, save it to a flash drive, plug that drive into another PC, install it, and it'll run. That's Windows. You can't do that with Linux.
@@robinpage2730 who asked? and yes you can do that with linux, and there are plenty of programs that don't work between different versions of windows lol
CLI and library being the same thing would be quite usefull but I'm afraid it may be a problem becuase it would mean that scripts need to be written in one lnanguage that accepts that library so it raises the question: what programming language should these libraries be designed for? Or maybe we should all agree to a protocol that all programs agree on for "output of programs" so writing CLI and/or wrapping libraries becomes easier (imagine, just for the sake of making example, using JSON for CLI output, representing JSON types in each language and making every library work that JSON type for data that needs to be passed around)
In TempleOS all HolyC functions you write are loaded in global executable memory and you can call any HolyC function from CLI. I don't know for sure, but I assume old Lisp Machines did the same with Lisp functions.
the thing is now JIT compilers are pretty fast and every serious programming language has APIs for filesystem and files and time etc. As for interface an 'object' interface like powershell/nushell is a good modern choice
There are probably hundreds of people that have developed operating systems just as Blow describes. Those operating systems are used exclusively by their creators. When more people start using the operating system and adding things to it to meet their requirements, it will almost inevitably get more bloated. The features Blow describes are great, but I'd be surprised if an operating system used by many has them any time soon.
I don't think he was talking about an OS that should be used for anything else than his game(s). I think it's basically just a new way to launch a platform-independent game that is more performant. I'm not experienced enough to imagine the realistic possibilities, but I'm hoping he has better user experiences in mind than rebooting the computer, manually changing BIOS settings and making a hardware partition, to play the game.
quotes from this vid i though were hard, and showed experience. "To start I would just not support graphics accelerators because they are terrible to deal with, and I would just hope that by the time we had the os running in a reasonable way that something in the graphics adapter space would have changed enough that we would be able to do something better" "thats another way of saying you bet that I will fail because its too hard, I don't fail at this kind of thing anymore, like when I was young i would fail at this kindof thing because I would give up and move onto a different project but i know how to succeed at these kinds of things now and once you know that it's, its not that hard [...] but making an operating system, we can do that, come on bro"
I wonder if he will actually make his own OS after he is done with the compiler... If he does, I think I would at least test it out, and follow the development.
Probably depends on how his next game does financially, he's already had to spend a lot on their current game and the Jai language. Hoping he gets to make that is though
Use Oberon System instead done by Nikolaus Wirth in the 80/90s written in the Oberon language, reducing complexity. The very odd thing about the command shell cmd.exe is the implementation of the shell language, its the worst thing of software i used so far
Anyone tought of the fact that with CLI in library you could easily (unit?) test it diffing output with the expected one without programming, on any machine, using a compiled binary?
Hubris is a definite blocker for collaboration and learning, that's for sure. Especially when folks can't recognize there are experts in these other fields, that could help clarify why things exist and paths to improve.
@@MurtagBY I was curious what was the discussion about after 1 year that you decided to reply. Not too long I needed to wait until he said "why do you have 400 processes running". There is alot to unpack why this statement is nonsense (and probably more after I stopped watching), and I don't want to waste more than 5 minutes on some youtube reply. But the actual number of processes is really not an issue to a modern OS and HW (modern meaning from 2000). From scheduling, memory management, space and even security. The problem is not the number, the problem is bad design and accountability from developers.
@@MurtagBY So I watched until the end, some notes: - At the beginning he describes some very logical and right way of how an OS should work: everything is sandbox, kernel should be thin layer, most in user. - But that contradicts his assumption that there should be very few "processes". - Every IPC should be just memory map and the user is responsible to parsing. That is very hard to do right without breaking basic security assumptions, and would not increase performance as most of serialization logic happens in user anyway and you still need a kind of sync logic that's go through the os. - No drivers or gpu? - No dynamic libs? well how do you expect the user parse your standard ipc? make them static? how would you update all execs with new ipc security fix on the system without installers? you can't update only some, that would break ipc. - No path? home directory? basic app place? he basically described path as he tried to figure what he means. just with some restrictions on path usage (sandbox?) - Every time he tackled with how to solve a real problem, he said he did not really think about it. Here he really confuses "I think I know how it should be" with "That is clearly how it should be". This is a clear case of the dunning kruger effect.
I love John and I love these kind of people, they are those who get shit done. But man, for someone with virtually no experience on OS development he's way too confident in his opinions. As long as his OS design experience is limited to a college class all he's doing is spitballing.
It's not like Jon is breaking any new ground here. He is basically describing a microkernel architecture which is commonly accepted to be a very good idea for a kernel/OS design for the reasons he talks about. Someone just needs to be the first one to implement such a kernel that is feature complete and has good support. Googles Zircon kernel from the Fuchsia OS project might be something to look out for in the future. The linux kernel can already do the memory mapping shared ring buffer stuff he is talking about for communicating between the kernel and programs where read/write/... syscalls are too expensive.
@@pleggli Early in the development of Linux, Linus Torvalds had a flame war with Andrew Tanenbaum (then professor of Computer Science) about wether monolithic kernels or microkernels were preferred. Tanenbaum argued that using a monolithic design would quickly render Linux obsolete due to new microkernels replacing it. That was almost 30 years ago and they are still not used as "end user facing" operations systems. (They are used in quite a few real time operating systems though.)
@@MarcusHast Yes but that doesn't prove that it's a good or bad idea, just that no one has put the resources for it to really happen. It's not like IT security is going to be less of an hot topic in the future and smaller kernel with more restrictive communication between components probably means a kernel will less exploitable bugs.
@@pleggli Sure, but people have been pouring money and development effort into this problem for over 30 years. Case in point, around the same time Linux development started there was a similar project by the GNU team (most of the surrounding tools and libraries in a Linux distro come from GNU) called HURD which was a micro-kernel. But it has yet to really show any advantage compared to the Linux kernel. (Last I checked it was not something you wanted to use for anything other than experimentation or testing.) The point of this is that I think the fundamental fallacy in micro kernel design is believing it makes the problem easier. Usually when you can reduce a programming model to something simpler and cleaner that makes the end product easier to build. (Once the scaffolding is in place.) That's obviously not the case with micro kernel design or we wouldn't still be waiting for them. Basically dynamic behavior on modern hardware at the scale of an operating system is very difficult to manage. I'm not convinced that a micro-kernel would help significantly against modern threats like Spectre or Meltdown which are caused by the interaction between hardware and software. Arguably I could see it being even more difficult to protect against if the user applications had a more direct access to the hardware.
note that on linux chrome/firefox can show up as hundreds of processes for some reason. its stupid becuare it makes taskmanagers like unusable as you have to scroll over 9 pages of just chrome. I have ~500 firefox processes/threads/entrys on Htop and and 15 processes on btop (8 tabs).
Is this not resolved by using tree mode in htop? I'm not super sure but I assumed this one process per tab design was to make it easier to isolate and terminate tabs with errors. I'm sure that leads to a lot of bloat though.
@@ConfuzedFx Hmm, I was unaware of this tree collape setting but it looks like it would have solved the issue. But strangely, either FF has changed how it makes processes, or my system is just bugging out, but htop now shows zero Firefox processes (with tree enabled or not), and btop shows 12.
Chrome does process isolation, so you have many processes for a single purpose. You may have: - Main process - A rederer process - A site process (each tab) And so on. Many resources are shared using IPC mechanisms. This is why you see many chromes in htop.
4:59 CLI & libs shouldn't be separate; some interface fxns extra 5:20 e.g. grep & libgrep ohwwww, ohkayh.... yeah, that makes so sense. i never thought about it why it wasn't the case. one contradiction in small case with previous point: u cant achieve both statically linkes libs & cli executable libs at the same time -- but yeah, thats easily solvable by dynamically linking the common system shipped utils/libs (liko say GNU utils)
App developers would want to develop on top of abstractions. If you build an operating system so simple where applications need to deal with bare metal to get anything useful done, someone will need to build an abstraction that sits on top on this OS and supports these apps, at which point it will not make a huge difference between a complex OS and (a simple OS + abstractions).
not necessarily. your app would just need to have low-level developers on board to design that part of your app, which enables your app to run better than other similar apps without having to rely on a third-party vendor to supply you with the required intermediate layers. I really like Casey Muratori's idea about having games be shipped with small OSes that talk to the hardware directly and don't require you to have any OS on your PC to run games.
Good point. I think that he framed it in the way, that it is OK for apps to talk with hardware without leaving user-space and not entering kernel-mode as it is expensive operation. So this could be solved by "libs/drivers" being available as basically libs in any programming language. The security and shared use of hardware is not clear in this model, right now it is the system who resolves race conditions and keeps mutexes and buffers. All above in my limited understanding of Python dev
@@VirtuousSaint "your app would just need to have low-level developers..." Wich makes your app more expensive and time intensive, nothing is free. OPs point stands.
@@marcossidoruk8033How would it be more expensive? Perhaps you could argue that there's fewer programmers today who know low -level stuff, but a high-level developer can literally become sufficiently qualified in less than a week. And developing software that touches low-level aspects may be categorically faster than traditional programming taught in CS. So I'm curious what your specific reasons are for that claim
@@Muskar2 because you have to write more code that is significantly harder to get right, wich means more man hours and thus more expensive, duh. Also what you just said just shows you have no idea what you are talking about. "Developers could become qualified in a week" not even a chance. Most developers are just straight up bad at their jobs, they couldn't get qualified in 10 years if they tried. Also there is a difference between writing low level code and writing low level code that is actually any good and efficient, the fact that you even suggest that you could learn that in a week shows you have no idea what you are taking about.
at work we use RHEL and it uses way fewer than 400 afaict. I haven't counted but I think it's more like 30 by default It's true that that's for a server where it doesn't need a bunch of 'daemons' for the user, but using a minimal distro won't use that many. isn't one of the big tradeoffs between dynamic vs static partially not having to reload big commonly used libs/functions in memory? Also things like plugins have to be linked at runtime so those seem like they have to be dynamic by definition.. unless you want to call those 'executables' (but that's more of a semantic difference)
I don't think this is entirely true, since many of those files are read-only for non-admin users. It still leaves the user profile accessible though (don't delete or you may lock your user).
micro kernels are as my OS teacher said just horrible to work with. He pointed that most of networking features are from OS development, which made me conclude that microservices are horrible.
1:07 "if you're gonna talk to the network card that should not go through the operating system at all" HELL YEAH!!! Write your own driver for it!!! Write the whole stack of network protocols!!! Write your own firewall!!! Manage sockets in your application!!! By that time your app *IS* the operating system. Show him TempleOS, he'll like it. Better yet, show him TETRIS OS.
I'm an astronomical noob in the subject bur I don't quite get why disallow programs to access the file system?? Don't programs need to access the file system to write files? I'm sure he is meaning something else I can't quite grasp.
Probably allow each program to create and read its own private files, like in mobile apps. And maybe save something into a dedicated "downloads" folder. There are not that many programs that need to have a read and write access to all files on the computer.
Can we get the suckless guys to develop the core utils so that we can have some pompous guys fight over if it should be called Suck/Blow OS or Suck plus Blow OS.
I don't understand the first 3 minutes of this video, a lot of what he says there, is how Linux operates as an OS, so is he agreeing with how Linux works or is he criticizing it? I'm having a hard time understanding because the tone he set when saying he read an article about a Ubuntu user suggests he is criticizing the OS. Could someone clarify for me perhaps? For instance, IPC in Linux goes *way* back to the Unix days of SystemV IPC of message queues, mem-mapping etc. Pipes is essentially that to begin with; it's a pair of file descriptors which in theory can point to anything that operates using file descriptors, a socket, a file, memory handle, etc. Also, is he joking at 5:32 when says "drivers shouldn't be a thing"? I am no expert in operating systems, but it certainly seems to me that Jon is contradicting himself in a lot of ways in this video. His critiques are very fair if they are applied to Windows, because in essence it suffers from a lot of the problems Jon brings up, but those aren't true when it comes to Linux. This whole notion of libraries for instance, anybody who has programmed for Linux/*Nix vs Windows, knows that we are utterly spoiled in the *Nix universe when it comes to these things in comparison to Windows. And that's not all we are spoiled with. Also at 8:28 he is *literally* describing how Linux does the "Path" variable for linux users. Now I am starting to think he is trolling. He is much too smart to say this nonsense. Also when he says after that the "no registry for sure" I totally agree. Windows Registry is a hellish nightmare - and although *Nix does something similar it isn't as conflated and just depression inducing as Windows, so once again, I am doubling down on him trolling, 100% now. Or perhaps he has no clue what he is talking about? ANOTHER addendum: I have come to the conclusion that he knows not of what he speaks of. The 400 processes thing, is windows in a nutshell. Sure, in later years they have managed to pull it down to 180ish processes (I have right now 180, and I have Firefox open, Steam open & task manager). But on Linux that tends to be lower for me, around 100. And again, processes incur context switching which does inflict a performance penalty, but it also sandboxes applications. If we had master processes where multiple smaller things ran in, security would start to break down or become less secure. If one wants to learn about OS, buy the book "The Linux Programming Interface" and the book "Operating Systems - Three Easy Pieces" (it's one of the course books for a Stanford course if I don't remember wrong) for the people interested.
I don't think he's trolling. Honestly, I don't have enough knowledge of operating systems to discuss this. If you want to hear clarification on his views, you can come to his stream and ask him: www.twitch.tv/naysayer88
If by Linux you mean Linux + an OS (from now I'll refer to this as Linux and assume that a typical OS is used), then I would be curious to know what part of what he said describes Linux. Linux doesn't have sandboxing. Programs usually have access to large chunks of the filesystem. It has messy installs with multiple side-effects. Each program relying on a million shared libraries is even worse than installing Windows programs (it's better for programming but not for using programs). I don't see how the Path on Linux is like what he describes. Placing program executables directly in the default Path directories is really bad. And adding directories to the Path is also bad. By bad I mean not ideal. No registry means no registry, not "ok we have a registry that's not as bad as the Windows one". Again, Linux isn't ideal here. Dunno why you're saying he's trolling by saying that. Pretty much 6:54.
@@aenpien and let's not even get started with developing software. Developing software on Linux (in any of the systems programming languages C, Cpp, Rust, D) is a "delight" in comparison to Windows. Just the fact that Linux has ptrace is a game changer entirely. Which is why rr (as far as I last checked which truthfully was quite some time ago) exist on Linux where in a debugging session you can literally save every executed instruction and play/replay it. I'm sure windows has software that has caught up to this, but rr has existed on Linux for quite some time due to aforementioned syscall.
We don't need browser tabs. That's a failing of the window manager. All windows should be "groupable" into tabs, and each browser window should just be a single process and windows. Things could be much more simple.
why would each window be a process? that's a pretty arbitrary and frankly bad grouping. For some programs like shells it makes sense but most of the time multi-window programs need to share data/memory
@@sub-harmonik Browsers already have multiple processes, and 1 process per window, but also 1 process per tab. That means 2 tabs and 1 window is 3 processes, and 1 tab per window, and 2 windows is 4 processes. There's also possibly more processes depending on what's being done. I didn't say anything about the inter process communication at all. What I said was that the Window manager should group windows into tabs. That may not change the architecture of the browser, but it could simplify it by rendering in the same process in which the web engine runs. Crash semantics remain robust, and everything is simpler, more modular (it becomes easier to make the web engine embedable).
@@IshIKoara yeah, I think once his hypothetical os reaches the same level of hardware support it will have the same number of processes, except they're gonna be user space now.
@@IshIKoara well, it's always a trade-off isn't it? Afaik if you run drivers in user space you still eventually have to do a syscall to access the hardware and syscalls are slow. Moving all drivers to user space could be a huge performance hit.
Cool. The first buggy software (there is no other software that does anything interesting past very few pieces like on space probes) used (no install needed, just run and also share the memory with no OS partitioning) will blow up the whole OS and everything with it. There will be a huge user base for that. Why talk security after that?
9 місяців тому+1
It makes me sad how many novice developers are enthralled with Jonathan Blow's ramblings.
5:20 - that's one of the biggest issues I have with Jon's ramblings: he uses words 'simplicity' and 'complexity' a lot, but doesn't give any definitions to them. The only actionable definition of 'simple' and 'complex' in context of SW dev that I know of, are the ones given in "Simple Made Easy" by Rich Hickey. And according to those definitions, making the library and CLI tool the same would be an example of complecting 2 things together.
Your talking sounds like a ancient language for web developers. Inter process comunication, named pipes, user space, kernel space, long gone words I remember from my past 😂😂 Now everything is http request, response, EF, Cloud yada yada. Just trivia stuff I see devs doing over and over again
I think we tried this idea with micro kernel architecture and I dont know why, but it did not work out! I think it is a logical solution but it did not work for us!
I get the impression that Blow doesnt understand the idea of making it easy for application developers to use hardware other than the CPU. That's what drivers are for. Graphics are the obvious example, but there's also cameras, printers, robotics, smart home stuff, etc. You can write to a common API (Vulkan, DirectX, etc) and the driver can translate to the device or you can write different paths in every applicarion for not just each manufacturer, but for every chip.
His perspective of a good OS is probably biased towards game development. Like I do not think I agreed with a single thing he said on OS architecture. No fileystem, memory mapped interprocess communication, and direct hardware access for network card? What's the pattern here? The pipe system and file system in Linux were a huge win for productivity. Stable APIs have proven more useful than stable ISAs over time.
Pipes and other forms of IPC are very costly due to having to do a lot of syscalls that also copy the data multiple times, from the user process into kernel buffer, and then from the kernel buffer to the destination process. There been some attempts at eliminating copies, but it is all messy. All you really need for communicating is a ring buffer on a shared memory region between processes. And all communication is just regular memory read/write, and you don't have to ping the kernel. That is a win-win-win sutuation. Same thing with network card. I have no idea why everyone interprets what he said as "user programs have to know how to talk with newtwork card hardware". What is mean here is that you have a region of memory(a ring buffer), where you put all your packets that you want to send, you ring some kind of doorbell, and the network card just takes packets (or complete frames - that would be a question to debate + security of it) from that region of memory and just sends them. Same with receiving packets. So all the networking communication boils down to memory read/writes and some kind of signalling - to signal the hardware flush the ring buffer. And same with file systems, have a memory-mapped region of storage, layout of which you manage yourself, as a user program. So it just tunrs into memory read/writes. And no need to talk with the kernel. As all programs are isolated there is no need for super-generic complex kernel-orchistrated synchronization. Just as basic as what the program needs, if it wants to run in multiple processes. This point is debatable though, as having a standard way for the user to interact with the program data may be useful, at least for debugging purposes. But you can still have a filesystem for user files, and a way to give programs the access to them, like the way things work nowadays :) etc.. The pattern is here is to make systems much simpler, by having only one very basic communication mechanism - memory read/write, that exploits properties of the paging. And this mechanism is the most optimal that you can get in a computer, so you are picking-up the performance that is lost in expensive communication. Also the security is potentially increased, as much smaller amount of kernel code is running at the request of user programs that also interacts with user program' provided data.
That's one area that I think it has been left behind. We know a lot by looking at the flaws of current ones we have, but to challenge the world with a new solution is to go against a huge stablished market.
i dont understand how he says in one sentence that 400 processes running is too many but then suggest drivers should run in userspace. would this not also result in a huge number of processes?
shared libraries, depend on who is allowed to upload. if f.eks only linux staff is allowed to upload to them then its a great idea but if its open freely to anyone then its a horrible idea.
Once Jon has finished his language, his OS, his chipset, fixed production line manufacturing of components, solved global supply chain inefficiency, invented renewable computing resources, and resolved the problem of labor exploitation, then his next game is really going to run like greased lightning
LOL, greased lightning is quite a picture.
Yes
And it will all be live streamed :)
If people were smart, they would collaborate for it.
I now want and need a Jon Blow musical running on broadway and across the world for 40 years.
After the death of King Terry, is Jon the man chosen by God to rebuild the temple?
Temple OS 2: Electric Bug a lot
SerenityOS
@@_rcs Is SerenityOS even good?
@@jamesevans2507 I am not sure. I don't know anything about it actually.
@@jamesevans2507 it's one of the most well made hobbyist OS out there for sure
"I don't fail at this kind of things anymore"
I love this energy
Which timestamp?
@@furyzenblade3558 13:50
@@matthew314engineering7ah ty
Oh well. We did this like a decade ago. The L4 is a mirco kernel. It used shared memory to do synchronized IPC in a very efficient manner. It used namespaces and capabilities instead of absolute paths to resources and there are neat tricks like user level page handler. It was a very great time, but it made me a bit depressed since the major players like MS and linux - even now are not able to pick up those advances. If you want to look up those things, I think the L4 project page is still up (although it might be a bit dated now). Another OS to look at is Fuchsia (which I think borrowed at least the user level page handlers from L4)
Fuscia isn't GPL. Step backwards.
@@kelownatechkid the kernel is MIT and user space is BSD or Apache. Obviously the reason is that they want to support phone providers developing closed hardware drivers, which isn't great, but that's already the situation in Android for various dumb reasons (the TIVO box situation)
So no, not really a step backwards.
@@SimonBuchanNzything less gpl is still Talmudic
@@SimonBuchanNz so because there is something already badly done that it means is ok to keep tolerating in the alternatives?
@@SimonBuchanNz You can accomplish that with the GPLv2 no?
The high number of processes is due to the Unix philosophy. Maybes it's less than 100 on stock Ubuntu but you can easily move that number down or up depending on what you use or what daemons or services you enable
But one thing that is missing from that philosophy is a standard reliable way to start small background tasks and have data pre-fetched and cached out side of it in a configurable way. To many processes are just waiting on some event like a connected USB device which might never happen. I don't need all those deamons running (sleeping) and requiring precious boot time. But the OS must have a way for the process to quickly start up when an event happens. Processes need to be really light weight again. The biggest problem there is processor cache invalidation upon context switching.
I mean, lots of boot time is spend to load state into memory that is the same every time. It amounts to a couple of MBs of important data which could be pulled from disk in less than a second if the OS (or Init) had some more knowledge of what all those deamons need. It is the same problem with JVM like virtual machines and script languages. They need to compile the same code over and over again. And they also have various solutions bolted on to solve these problems. These might be generalized.
@@mennovanlavieren3885 Why do we care so much about boot time?
If you think about it, if the operating system is done in this way, and metadata is kept in the binaries, there could be multiple operating systems that all use the same apps. "Drivers" are apps that know machine specific things, and comunicates with mmaped pages, so drivers could still exist to smoth over the differences between kernels.
treating the OS like a virtual device is a nice idea, but it would drive him ballistic (which I would like to see)
This has already been implemented in MS-DOS, PC-DOS and other DOS variants.
@@RenamedChannel I grew up using DOS, and I worked for GameTap on the DOS games emulator, so I'm pretty familiar with DOS internals. Yez, it does work this way to some extent; DOS and DPMI work through software interrupts, which can switch contexts to the driver when needed. File system process sharing and isolation isn't something that's done in DOS, unfortunatly. Allocating memory through DPMI and talking to the hardware was nice, but it wasn't technically part of the base OS. DesqView (without X) was one of the best user interfaces that I think I've ever used becides Plan 9's. You open a new window, and whatever command you gave can own the window or pass the window to the next process. You could open another Window with a different command, also.
"Come on bro"
Yes, totally agree. Im terrified with the complexity of OC. Sometimes its seems ridiculous. Tiny Linux requires MB to do stuff!
I love the way this dude thinks.
He is gonna like Temple OS :p
I don't know if you're being sarcastic Temple OS does a lot of things the opposite way from what he's saying.
@@franciscofarias6385 IIRC TempleOS does run HolyC functions from the command line though which seems to be what Jon was mentioning in this clip. TempleOS was never meant to be any kind of mainstream OS, because it's written for fun it doesn't do pointless things like security, you have to see it for what it is.
@@pleggli Tempel OS was never meant to be anything but the delusions of a sick man. Don't get me wrong, Tempel OS is very entertaining, but it's not a serious tool.
@@ViktorHJ Wow we have a serious professional here guys beware. (Literally nobody markets TempleOS as a "serious tool" stop being such a tryhard)
@@Borgilian That_escalated_quickly.gif
Coming soon: Blow OS
Blow-S
Bows, an OS faster than arrows
blOS
I’m not clever
Winblows
If anyone here is interested in OS design of the likes that Jonathan Blow just lined out, I can seriously recommend having a look at the design of the L4 micro kernel, because that's literally a kernel which considers & implements all of these aspects: the kernel is only supposed to be a small piece of code doing basic hardware ressource management (adress space, i/o space, cpu & affinity time allocation while enforcing basic access rights & privilege separation of processes but doing nothing else other then that). The only three problems L4 has are a) hardly anyone ever used it because b) hardly anyone ever knew about it because d) its code base is an outdated & semi-abandoned academic project & e) it's just a (micro)kernel, without either driver or user space libraries or programs, so someone would need to build (modern) driver infrastructure, basic system admin tools & basic ui to even being able use it.
However, if anyone's planning on build an entire new OS using the aforementioned concepts, then at least there already IS an exceedingly well designed kernel which you could use, either directly or more likely at least rewrite it in a more modern & up-to-date way (using modern language & tool sets) - which in the specific case of L4 is NOT a big undertaking because the entire original source code of the entire kernel is just ~5000 lines of C code - absolutely tiny compared to other software projects.
i would love to see the complete interrupt cue and cycle for a modern OS, even an old OS like XP. I get the feeling there must be thousands of interrupts going on, before your code even gets a sniff, if the OS gives it time.
You can see it in a profiler i think? At least i can see other apps in Tracy in my gl engine
ever try to SIGINT a running application and have it take a couple seconds to stop? this is what thread priorities are for
almost every modern app you use in practice will sleep, aside from command line pipelines
This video get better and better with time.
Some guys criticize userspace, that things like peripharicals should not use userspace. But the fact of the matter is that userspace saved me a lot of times. I don't have to build a driver each time I want to talk to a USB device. And just using libusb makes the code much more portable and easier to compile (and install). Jonathan has a good point, and I'm glad he thinks that way. Only people that never had to communicate with and control USB devices think that everything should be done at the kernel level. It is just ludicrous.
Looking forward to the inevitable JBlow vs cat-v showdown.
Thank you for upload, very interesting topic.
Glad you liked it!
I now want a talk with Jonathan Blow and Linus Torvalds. It seems liek those two just might be able to communicate clearly and have an argumentative but productive discussion.
Yeah, all Linus does is develop the kernel, so their experience gap in this field is too big
I believe that a user developer dialogue could be enlightening for both though.
@@malteborgmanm2626 yeah, that would be interesting
@@Sergeeeek right, Jon doesn't have remotely enough experience to produce a decent and qualitative discussion.
Terry Davis blows them all away. Dude wrote a whole os, with drivers, file system, kernel, shell, graphics, and apps, and did it while fighting insanity. Torvalds wrote a kernel? Pretty good, but not true greatness.
Plan 9 is a step in the right direction
I wonder what would've happened if Plan 9 had been open source before Linus started writing his kernel...
I'm not sure I want each process talking directly to network card, there needs to be some intermediate system to controll access to the device, tell other processes when it is busy, etc.
But it would be even better if that was done in hardware and we are somewhat getting closer to that in some areas. There already exists ways to share PCIE devices transparently between multiple consumers (SR-IOV). The networking interface simplu appears as separate devices for each consumer. It's just one thing that could be leveraged by a forward looking OS kernel.
@@pleggli and a moment of silence for intel based systems with shared IOMMU groups killing the dream of virtualized hardware sharing.
You know he's a good programmer because he's always trying to simplify things, and use less resources. A lot of our OSes etc. were not developed from the ground up with good security or even good architecture, they just kind of evolved and things were bolted on. Because of backwards compatibility etc. and hidden OS features, the NTFS filesystem has some crazy amount of attributes for file access control, I think it was close to 30 different ones.
The problem is, starting over from scratch only this time doing things right just results in another pile of code that's been upgraded, maintained etc etc. There's no such thing as "ground up, secure, great architecture" for any serious, big software product.
Blow has no clue about shipping products that billions of people use. He's sold a few non-mission critical games. And that small bit of success has gone to his head to the point where he then deludes himself he has all the answers to every computing problem from programming languages through to OS. He doesn't. You know thousands of people have spent thousands and thousands of hours actually developing OSes. Often, even in spite of this, people still don't want it on their desktop. So, you know, it's easy to talk about how an OS should work but much, much harder to get to the stage MS, Android etc are at with an OS that billions are actually using. Similarly for servers etc etc. Thinking you know better is the sign of a moron (or perhaps like the guy who wrote templeOS, mental instability)
@@michael1 They say you can't have genius without a scoop of madness. And I say that as someone 2 years into making the OS described in the video become reality. Why do nearly all the successful tech founders have some form of autism? Because everyone who doesn't gets talked out of their creative and innovative ideas.
@@michael1I mean, touche, but let's not talk about this like it's just 'a little business success'. In terms of surpassing technical challenges, Jai seems to do a lot of incredibly impressive stuff, and part of why it's impressive is Blow's approach to developing it - what he thinks a programming language should or shouldn't do.
Now, just because his theories seem to have accomplished something here doesn't mean they're going to accomplish something in the OS space, but I think it would be very easy before Blow made Jai to say "Thousands of people have spent thousands of hours making programming languages, get off your high horse."
So while I agree he's getting a bit ahead of himself, he's also demonstrably done some amazing things I that I don't think should be ignored. I think seeing whether Jai gets more widely adopted after this game ships is going to be a very interesting test of its viability as a product, and Blow's ability to sell it.
@@michael1 well, you don't even have the "few non-mission critical games"
@@michael1if these people do know better, hopefully that means they'll start fixing the problem where input is only processed after the display updates(which can be a long delay nowadays) resulting in misclicks
I love the small efficient operating systems from the 70s like CP/M, RSX11, early BSD and VMS. They were lean and to the point. Even OS/360 that was considered huge, is small and lean and to the point compared to today’s systems.
He want an OS with fewer processes yet he wants drivers to run in user space which I don't disagree with but it will probably results in more processes. Ok 400 is maybe a bit much, my Linux boots to the window manager below 100 processes and at least 20 of them are development related stuff that isn't really needed for a generic user. When I start chrome though I quickly get 70 new processes.
# A simple:
ps -A | wc
# gives me:
272 1109 10851
on a Debian 12 system used by only me.
I wonder if he also had the class of operating design and implementation following Professor Tanenbaum’s book. When I studied EE that part of design and implementation of an OS was the best thing I’ve learned. Making the SBC running a Z80 (which I loathed as a 6502 and 68000 user) was the trivial part. But it was great to get a basic simple OS that could dynamically load binaries and swap between them with cooperative multitasking. But it also taught me something viable. And that is that for most embedded systems you don’t need an OS and it will just make things bigger, bulkier, less efficient and more error prone.
This discussion happened decades ago with the Tanenbaum-Torvalds debate. What he's arguing for is similar to Minix, with elements of powershell mixed in. Neither gets far.
I think you have to go a step further than that to get to something fundamentally better. Linux's advantage wasn't really the more monolithic approach (and Tanenbaum wasn't wrong about the drawbacks) but that it was open for hobbyists and that it stayed with POSIX. As history has shown that was the right call to get successful.
That doesn't mean there couldn't be something fundamentally better that throws a bunch of old ideas overboard. Whether or not such a product could successfully be pushed onto the OS market is another question. Although I think there still might be room if you go via mobile devices. A mobile OS that doesn't push you into the hands of Google or Apple, respects both your ownership of the device and your privacy, isn't completely defined by the availablity of commercial software and runs on a device that on average gets replaced every other year anyway might conceivably have a chance right now.
And speaking of mobile devices: Some of the things iOS brought to the table, even if just on top of unix, are worth considering on a more fundamental level. Like apps generally running in tight sandboxes with strictly limited communication (be it access to files, certain hardware, networking, other apps or whatever) controlled by the user.
It reminds me of C++. It's obvious for anyone willing to look that there is aspects at every level that sucks to a degree that it's constantly flirting with having to be declared plain broken but everybody argues that that's the best it can be and you just have to throw layers and layers of makeup over it and sell those as revolutionary. And then along comes Rust and proves that most of those assumptions were bullshit if you were just willing to throw enough of them out and tackle things from a fundamentally different perspective.
@@juliuszkopczewski5759 It's certainly more challenging to make fast, especially on x86, since you'll have additional context switches. But if L4 can reach 90-95% of the official Linux Kernel's performance while maintaining compatibility it probably wouldn't be unrealistic for something bespoke to be on equal footing.
With that being said, my bigger point is exactly that there might be an architecture to be figured out that's fundamentally outside the box of current common wisdom about how things should be done. That's why I made the comparsion to Rust and what it did for systems programming. It turns a couple of basic assumptions from half a century ago on their head, reintroduces a couple concepts that completely fell out of favor, adds a couple of original ideas and allows to approach programming on bare metal with a completely different mindset.
I don't wanna trust my applications with network security. I feel like the network should be OS controlled. Atleast to avoid.... developer tomfoolery
I see, he'll write his OS next, after he's finished with his game engine. And after that, he'll design his own properly working computer
RISC-V looks good for facilitating that
??
BlowOS is the only hope we have for the future
Sounds like an exokernel design.
I often agree with him. And even when I don't he has a very good reason for his idea, and I feel like I could have a productive conversation with him.
"I mean cmon bro!" haha love it!
FreeBSD and Mac are really good OSes. But they can't be ideal OSes. We can only come as far as Mac for general usability but that too is hard for some users.
it always feels to me like "[Jonathan] [blows] [on xyz]" which is super funny. rather than "[Jonathan B.] [on xyz]"
thanks for these videos man
Glad you liked it!
I've played around with Linux From Scratch a few times. It's a lot of fun. I really like LFS and Gentoo as my favored Operating Systems, but sadly I have spent most of my life on Windows systems. It's a pain to have to have multiple operating systems to use certain software or games, but I think that is what I'm going to do sometime soon with my new 1TB M.2 drive when I get around to it, and then just have a separate Windows drive, or maybe see how WINE has come along, for games and other software that relies on Windows.
I can build an exe on one PC, save it to a flash drive, plug that drive into another PC, install it, and it'll run. That's Windows. You can't do that with Linux.
@@robinpage2730 who asked? and yes you can do that with linux, and there are plenty of programs that don't work between different versions of windows lol
@@robinpage2730 Wat. What makes you think that a precompiled binary can't run on different Linux computers? Do you understand how programs work?
@@robinpage2730 Are you joking or are you really this dumb?
@@robinpage2730 Go and educate yourself , then speak.
CLI and library being the same thing would be quite usefull but I'm afraid it may be a problem becuase it would mean that scripts need to be written in one lnanguage that accepts that library so it raises the question: what programming language should these libraries be designed for?
Or maybe we should all agree to a protocol that all programs agree on for "output of programs" so writing CLI and/or wrapping libraries becomes easier (imagine, just for the sake of making example, using JSON for CLI output, representing JSON types in each language and making every library work that JSON type for data that needs to be passed around)
In TempleOS all HolyC functions you write are loaded in global executable memory and you can call any HolyC function from CLI.
I don't know for sure, but I assume old Lisp Machines did the same with Lisp functions.
the thing is now JIT compilers are pretty fast and every serious programming language has APIs for filesystem and files and time etc.
As for interface an 'object' interface like powershell/nushell is a good modern choice
There are probably hundreds of people that have developed operating systems just as Blow describes. Those operating systems are used exclusively by their creators.
When more people start using the operating system and adding things to it to meet their requirements, it will almost inevitably get more bloated.
The features Blow describes are great, but I'd be surprised if an operating system used by many has them any time soon.
I don't think he was talking about an OS that should be used for anything else than his game(s). I think it's basically just a new way to launch a platform-independent game that is more performant. I'm not experienced enough to imagine the realistic possibilities, but I'm hoping he has better user experiences in mind than rebooting the computer, manually changing BIOS settings and making a hardware partition, to play the game.
it's incredible to see what one man can do.
hes just spewing junk
"Come on bro" -Jonathan Blow
Plan9 got the path thing right. Union dirs.
I love this man. And I want his OS.
quotes from this vid i though were hard, and showed experience.
"To start I would just not support graphics accelerators because they are terrible to deal with, and I would just hope that by the time we had the os running in a reasonable way that something in the graphics adapter space would have changed enough that we would be able to do something better"
"thats another way of saying you bet that I will fail because its too hard, I don't fail at this kind of thing anymore, like when I was young i would fail at this kindof thing because I would give up and move onto a different project but i know how to succeed at these kinds of things now and once you know that it's, its not that hard [...] but making an operating system, we can do that, come on bro"
I wonder if he will actually make his own OS after he is done with the compiler...
If he does, I think I would at least test it out, and follow the development.
Probably depends on how his next game does financially, he's already had to spend a lot on their current game and the Jai language. Hoping he gets to make that is though
Use Oberon System instead done by Nikolaus Wirth in the 80/90s written in the Oberon language, reducing complexity.
The very odd thing about the command shell cmd.exe is the implementation of the shell language, its the worst thing of software i used so far
Anyone tought of the fact that with CLI in library you could easily (unit?) test it diffing output with the expected one without programming, on any machine, using a compiled binary?
The moment when someone you really respect start to talk nonsense with a lot of confidence about a subject that you know a lot about: 😣
Hubris is a definite blocker for collaboration and learning, that's for sure. Especially when folks can't recognize there are experts in these other fields, that could help clarify why things exist and paths to improve.
Honestly that's pretty much everything he talks about. He's pretty embarrassing, frankly.
Well, tell me where is the nonsense part to start with?
I doubt your expertise here tbh.
@@MurtagBY I was curious what was the discussion about after 1 year that you decided to reply. Not too long I needed to wait until he said "why do you have 400 processes running".
There is alot to unpack why this statement is nonsense (and probably more after I stopped watching), and I don't want to waste more than 5 minutes on some youtube reply. But the actual number of processes is really not an issue to a modern OS and HW (modern meaning from 2000). From scheduling, memory management, space and even security.
The problem is not the number, the problem is bad design and accountability from developers.
@@MurtagBY So I watched until the end, some notes:
- At the beginning he describes some very logical and right way of how an OS should work: everything is sandbox, kernel should be thin layer, most in user.
- But that contradicts his assumption that there should be very few "processes".
- Every IPC should be just memory map and the user is responsible to parsing. That is very hard to do right without breaking basic security assumptions, and would not increase performance as most of serialization logic happens in user anyway and you still need a kind of sync logic that's go through the os.
- No drivers or gpu?
- No dynamic libs? well how do you expect the user parse your standard ipc? make them static? how would you update all execs with new ipc security fix on the system without installers? you can't update only some, that would break ipc.
- No path? home directory? basic app place? he basically described path as he tried to figure what he means. just with some restrictions on path usage (sandbox?)
- Every time he tackled with how to solve a real problem, he said he did not really think about it.
Here he really confuses "I think I know how it should be" with "That is clearly how it should be". This is a clear case of the dunning kruger effect.
Most of the points I heard him bring up made me think "OK, MS-DOS".
No.
I love John and I love these kind of people, they are those who get shit done. But man, for someone with virtually no experience on OS development he's way too confident in his opinions. As long as his OS design experience is limited to a college class all he's doing is spitballing.
It's not like Jon is breaking any new ground here.
He is basically describing a microkernel architecture which is commonly accepted to be a very good idea for a kernel/OS design for the reasons he talks about. Someone just needs to be the first one to implement such a kernel that is feature complete and has good support. Googles Zircon kernel from the Fuchsia OS project might be something to look out for in the future.
The linux kernel can already do the memory mapping shared ring buffer stuff he is talking about for communicating between the kernel and programs where read/write/... syscalls are too expensive.
@@pleggli Early in the development of Linux, Linus Torvalds had a flame war with Andrew Tanenbaum (then professor of Computer Science) about wether monolithic kernels or microkernels were preferred. Tanenbaum argued that using a monolithic design would quickly render Linux obsolete due to new microkernels replacing it. That was almost 30 years ago and they are still not used as "end user facing" operations systems. (They are used in quite a few real time operating systems though.)
@@MarcusHast Yes but that doesn't prove that it's a good or bad idea, just that no one has put the resources for it to really happen. It's not like IT security is going to be less of an hot topic in the future and smaller kernel with more restrictive communication between components probably means a kernel will less exploitable bugs.
@@pleggli Sure, but people have been pouring money and development effort into this problem for over 30 years. Case in point, around the same time Linux development started there was a similar project by the GNU team (most of the surrounding tools and libraries in a Linux distro come from GNU) called HURD which was a micro-kernel. But it has yet to really show any advantage compared to the Linux kernel. (Last I checked it was not something you wanted to use for anything other than experimentation or testing.)
The point of this is that I think the fundamental fallacy in micro kernel design is believing it makes the problem easier. Usually when you can reduce a programming model to something simpler and cleaner that makes the end product easier to build. (Once the scaffolding is in place.) That's obviously not the case with micro kernel design or we wouldn't still be waiting for them.
Basically dynamic behavior on modern hardware at the scale of an operating system is very difficult to manage. I'm not convinced that a micro-kernel would help significantly against modern threats like Spectre or Meltdown which are caused by the interaction between hardware and software. Arguably I could see it being even more difficult to protect against if the user applications had a more direct access to the hardware.
@@pleggli like redox os?
So basically an immutable OS with containerized apps running in userspace like suse microOS or fedora silverblue?
note that on linux chrome/firefox can show up as hundreds of processes for some reason. its stupid becuare it makes taskmanagers like unusable as you have to scroll over 9 pages of just chrome. I have ~500 firefox processes/threads/entrys on Htop and and 15 processes on btop (8 tabs).
I used to think 32GB would be enough for anything. That damn chrome monster proved me otherwise.
Is this not resolved by using tree mode in htop? I'm not super sure but I assumed this one process per tab design was to make it easier to isolate and terminate tabs with errors. I'm sure that leads to a lot of bloat though.
@@ConfuzedFx Hmm, I was unaware of this tree collape setting but it looks like it would have solved the issue. But strangely, either FF has changed how it makes processes, or my system is just bugging out, but htop now shows zero Firefox processes (with tree enabled or not), and btop shows 12.
Chrome does process isolation, so you have many processes for a single purpose.
You may have:
- Main process
- A rederer process
- A site process (each tab)
And so on. Many resources are shared using IPC mechanisms.
This is why you see many chromes in htop.
on windows is from like the digitalised typewriter era or something, shit sticks!
that man just speaks it out
4:15 link-time agnostic libraries statically linked retaining their info so can be patched with updates.
woahhhh
4:59 CLI & libs shouldn't be separate; some interface fxns extra
5:20 e.g. grep & libgrep
ohwwww, ohkayh.... yeah, that makes so sense. i never thought about it why it wasn't the case.
one contradiction in small case with previous point: u cant achieve both statically linkes libs & cli executable libs at the same time -- but yeah, thats easily solvable by dynamically linking the common system shipped utils/libs (liko say GNU utils)
I’m curious how he feels about the Hurd, it sounds like a lot of his opinions are in line with its design.
App developers would want to develop on top of abstractions. If you build an operating system so simple where applications need to deal with bare metal to get anything useful done, someone will need to build an abstraction that sits on top on this OS and supports these apps, at which point it will not make a huge difference between a complex OS and (a simple OS + abstractions).
not necessarily. your app would just need to have low-level developers on board to design that part of your app, which enables your app to run better than other similar apps without having to rely on a third-party vendor to supply you with the required intermediate layers. I really like Casey Muratori's idea about having games be shipped with small OSes that talk to the hardware directly and don't require you to have any OS on your PC to run games.
Good point.
I think that he framed it in the way, that it is OK for apps to talk with hardware without leaving user-space and not entering kernel-mode as it is expensive operation.
So this could be solved by "libs/drivers" being available as basically libs in any programming language.
The security and shared use of hardware is not clear in this model, right now it is the system who resolves race conditions and keeps mutexes and buffers.
All above in my limited understanding of Python dev
@@VirtuousSaint "your app would just need to have low-level developers..." Wich makes your app more expensive and time intensive, nothing is free.
OPs point stands.
@@marcossidoruk8033How would it be more expensive? Perhaps you could argue that there's fewer programmers today who know low -level stuff, but a high-level developer can literally become sufficiently qualified in less than a week. And developing software that touches low-level aspects may be categorically faster than traditional programming taught in CS. So I'm curious what your specific reasons are for that claim
@@Muskar2 because you have to write more code that is significantly harder to get right, wich means more man hours and thus more expensive, duh.
Also what you just said just shows you have no idea what you are talking about. "Developers could become qualified in a week" not even a chance. Most developers are just straight up bad at their jobs, they couldn't get qualified in 10 years if they tried. Also there is a difference between writing low level code and writing low level code that is actually any good and efficient, the fact that you even suggest that you could learn that in a week shows you have no idea what you are taking about.
Your resemblance to Hunter S. Thompson is striking. Wouldn’t be a bad Halloween costume choice.
This is not me, this is Jon Blow. I'm a different person, I just edited and uploaded this video.
at work we use RHEL and it uses way fewer than 400 afaict. I haven't counted but I think it's more like 30 by default
It's true that that's for a server where it doesn't need a bunch of 'daemons' for the user, but using a minimal distro won't use that many.
isn't one of the big tradeoffs between dynamic vs static partially not having to reload big commonly used libs/functions in memory?
Also things like plugins have to be linked at runtime so those seem like they have to be dynamic by definition.. unless you want to call those 'executables' (but that's more of a semantic difference)
5:23 you can kinda sorta do this in powershell but you need to write the lib in c#
Does anyone have a link to the Hacker News post he's mentioning? I'd be really interested in taking a look at it
I always find it amazing that on Windows, any program can delete valuable system files.
I don't think this is entirely true, since many of those files are read-only for non-admin users.
It still leaves the user profile accessible though (don't delete or you may lock your user).
I feel like he is describing CubesOS (particularly about sandboxing) or micro kernels
micro kernels are as my OS teacher said just horrible to work with. He pointed that most of networking features are from OS development, which made me conclude that microservices are horrible.
Some concepts of this theoretical OS resemble plan9 OS
This guy should leave computing to the experts and get started on the next Smashing Pumpkins album.
1:07 "if you're gonna talk to the network card that should not go through the operating system at all" HELL YEAH!!! Write your own driver for it!!! Write the whole stack of network protocols!!! Write your own firewall!!! Manage sockets in your application!!! By that time your app *IS* the operating system. Show him TempleOS, he'll like it. Better yet, show him TETRIS OS.
And you totally missed the point. You don't need anything of the shit that you listed. ;)
I mean, you'd use something like Java which pretty much layers a minimalist OS on top of the JVM to run the applications at.
You must live by an airport because you didn't even notice the roar of the point flying over your head! ;)
Library Operating systems - so drivers run as libraries
I don’t get how he does access control without someone to control work? And how does he think how nvme and pcie stuff works..
Can we pls get Blow-S or BL-OS
Yes blow jobs ftw
it seems like that gummy bear you were eating at the beginning gave you most of the insight to this
I'm an astronomical noob in the subject bur I don't quite get why disallow programs to access the file system?? Don't programs need to access the file system to write files? I'm sure he is meaning something else I can't quite grasp.
Probably allow each program to create and read its own private files, like in mobile apps. And maybe save something into a dedicated "downloads" folder. There are not that many programs that need to have a read and write access to all files on the computer.
@@swifton same as processes can't look up in memory of another process.
Most of the programs don't need access to your whole computer
An OS needs to do 2 things:
1) Boot up and stay up
2) Have a web browser that keeps up with porn (i.e., the pinnacle of web tech)
He's quite an enthusiast, I see... Jokes aside, I'm really his fun after Braid And The Witness.
Can we get the suckless guys to develop the core utils so that we can have some pompous guys fight over if it should be called Suck/Blow OS or Suck plus Blow OS.
The ideal OS would push the average user away from any type of personal computing fast.
I don't understand the first 3 minutes of this video, a lot of what he says there, is how Linux operates as an OS, so is he agreeing with how Linux works or is he criticizing it? I'm having a hard time understanding because the tone he set when saying he read an article about a Ubuntu user suggests he is criticizing the OS. Could someone clarify for me perhaps? For instance, IPC in Linux goes *way* back to the Unix days of SystemV IPC of message queues, mem-mapping etc. Pipes is essentially that to begin with; it's a pair of file descriptors which in theory can point to anything that operates using file descriptors, a socket, a file, memory handle, etc. Also, is he joking at 5:32 when says "drivers shouldn't be a thing"? I am no expert in operating systems, but it certainly seems to me that Jon is contradicting himself in a lot of ways in this video. His critiques are very fair if they are applied to Windows, because in essence it suffers from a lot of the problems Jon brings up, but those aren't true when it comes to Linux. This whole notion of libraries for instance, anybody who has programmed for Linux/*Nix vs Windows, knows that we are utterly spoiled in the *Nix universe when it comes to these things in comparison to Windows. And that's not all we are spoiled with.
Also at 8:28 he is *literally* describing how Linux does the "Path" variable for linux users. Now I am starting to think he is trolling. He is much too smart to say this nonsense.
Also when he says after that the "no registry for sure" I totally agree. Windows Registry is a hellish nightmare - and although *Nix does something similar it isn't as conflated and just depression inducing as Windows, so once again, I am doubling down on him trolling, 100% now. Or perhaps he has no clue what he is talking about?
ANOTHER addendum: I have come to the conclusion that he knows not of what he speaks of. The 400 processes thing, is windows in a nutshell. Sure, in later years they have managed to pull it down to 180ish processes (I have right now 180, and I have Firefox open, Steam open & task manager). But on Linux that tends to be lower for me, around 100.
And again, processes incur context switching which does inflict a performance penalty, but it also sandboxes applications. If we had master processes where multiple smaller things ran in, security would start to break down or become less secure. If one wants to learn about OS, buy the book "The Linux Programming Interface" and the book "Operating Systems - Three Easy Pieces" (it's one of the course books for a Stanford course if I don't remember wrong) for the people interested.
I don't think he's trolling. Honestly, I don't have enough knowledge of operating systems to discuss this. If you want to hear clarification on his views, you can come to his stream and ask him:
www.twitch.tv/naysayer88
If by Linux you mean Linux + an OS (from now I'll refer to this as Linux and assume that a typical OS is used), then I would be curious to know what part of what he said describes Linux. Linux doesn't have sandboxing. Programs usually have access to large chunks of the filesystem. It has messy installs with multiple side-effects. Each program relying on a million shared libraries is even worse than installing Windows programs (it's better for programming but not for using programs).
I don't see how the Path on Linux is like what he describes. Placing program executables directly in the default Path directories is really bad. And adding directories to the Path is also bad. By bad I mean not ideal.
No registry means no registry, not "ok we have a registry that's not as bad as the Windows one". Again, Linux isn't ideal here. Dunno why you're saying he's trolling by saying that.
Pretty much 6:54.
@@aenpien and let's not even get started with developing software. Developing software on Linux (in any of the systems programming languages C, Cpp, Rust, D) is a "delight" in comparison to Windows.
Just the fact that Linux has ptrace is a game changer entirely. Which is why rr (as far as I last checked which truthfully was quite some time ago) exist on Linux where in a debugging session you can literally save every executed instruction and play/replay it. I'm sure windows has software that has caught up to this, but rr has existed on Linux for quite some time due to aforementioned syscall.
@@simonfarre4907 it has, but it depends:)
7:00 Amen.
We don't need browser tabs. That's a failing of the window manager. All windows should be "groupable" into tabs, and each browser window should just be a single process and windows. Things could be much more simple.
like the suckless team's "surf" browser and "tabbed" utility
this how suckless tools work
that is just a simplistic and cosmetic optimization
why would each window be a process? that's a pretty arbitrary and frankly bad grouping.
For some programs like shells it makes sense but most of the time multi-window programs need to share data/memory
@@sub-harmonik Browsers already have multiple processes, and 1 process per window, but also 1 process per tab. That means 2 tabs and 1 window is 3 processes, and 1 tab per window, and 2 windows is 4 processes. There's also possibly more processes depending on what's being done. I didn't say anything about the inter process communication at all. What I said was that the Window manager should group windows into tabs. That may not change the architecture of the browser, but it could simplify it by rendering in the same process in which the web engine runs. Crash semantics remain robust, and everything is simpler, more modular (it becomes easier to make the web engine embedable).
Most of the kernel is driver code, if not drivers then what?
You can ask him on the stream. I don't know the answer.
I think he wants drivers but running them in Usermode.
@@IshIKoara yeah, I think once his hypothetical os reaches the same level of hardware support it will have the same number of processes, except they're gonna be user space now.
@@Sergeeeek Making more space for exploits is always good ^^
@@IshIKoara well, it's always a trade-off isn't it? Afaik if you run drivers in user space you still eventually have to do a syscall to access the hardware and syscalls are slow.
Moving all drivers to user space could be a huge performance hit.
Cool. The first buggy software (there is no other software that does anything interesting past very few pieces like on space probes) used (no install needed, just run and also share the memory with no OS partitioning) will blow up the whole OS and everything with it. There will be a huge user base for that. Why talk security after that?
It makes me sad how many novice developers are enthralled with Jonathan Blow's ramblings.
Damn Jonathon, do it! Please release an actually useful OS!
Some of what he said, NixOs does.
Interesting musings but I really don't understand what's wrong with 4500 executables on a filesystem.
5:20 - that's one of the biggest issues I have with Jon's ramblings: he uses words 'simplicity' and 'complexity' a lot, but doesn't give any definitions to them.
The only actionable definition of 'simple' and 'complex' in context of SW dev that I know of, are the ones given in "Simple Made Easy" by Rich Hickey. And according to those definitions, making the library and CLI tool the same would be an example of complecting 2 things together.
So should each app ship its own device drivers?
I wonder if he is aware that we tried all this stuff, SINCE MICRO KERNELS were a thing and it didn't take off!
@@mostaposta like your mom.
@@mostaposta AM NOT!
13:27
Dec 18, has he done it yet
He actually said recently that he won't even play with it this year because there are too many projects.
Your talking sounds like a ancient language for web developers. Inter process comunication, named pipes, user space, kernel space, long gone words I remember from my past 😂😂 Now everything is http request, response, EF, Cloud yada yada. Just trivia stuff I see devs doing over and over again
Web Dev caused a whole lot of brain rot for me too lmao
I didn't understand a word this man said. But I'm sure he knows what he's talking about...
average jon blow fan
Winblows
does he know about flatpak?
BlowOS
How much effort is anti piracy vs real os? Should that be an effort?
I think we tried this idea with micro kernel architecture and I dont know why, but it did not work out! I think it is a logical solution but it did not work for us!
I get the impression that Blow doesnt understand the idea of making it easy for application developers to use hardware other than the CPU. That's what drivers are for.
Graphics are the obvious example, but there's also cameras, printers, robotics, smart home stuff, etc. You can write to a common API (Vulkan, DirectX, etc) and the driver can translate to the device or you can write different paths in every applicarion for not just each manufacturer, but for every chip.
His perspective of a good OS is probably biased towards game development. Like I do not think I agreed with a single thing he said on OS architecture. No fileystem, memory mapped interprocess communication, and direct hardware access for network card? What's the pattern here?
The pipe system and file system in Linux were a huge win for productivity. Stable APIs have proven more useful than stable ISAs over time.
Pipes and other forms of IPC are very costly due to having to do a lot of syscalls that also copy the data multiple times, from the user process into kernel buffer, and then from the kernel buffer to the destination process. There been some attempts at eliminating copies, but it is all messy.
All you really need for communicating is a ring buffer on a shared memory region between processes. And all communication is just regular memory read/write, and you don't have to ping the kernel. That is a win-win-win sutuation.
Same thing with network card. I have no idea why everyone interprets what he said as "user programs have to know how to talk with newtwork card hardware".
What is mean here is that you have a region of memory(a ring buffer), where you put all your packets that you want to send, you ring some kind of doorbell, and the network card just takes packets (or complete frames - that would be a question to debate + security of it) from that region of memory and just sends them.
Same with receiving packets. So all the networking communication boils down to memory read/writes and some kind of signalling - to signal the hardware flush the ring buffer.
And same with file systems, have a memory-mapped region of storage, layout of which you manage yourself, as a user program. So it just tunrs into memory read/writes. And no need to talk with the kernel. As all programs are isolated there is no need for super-generic complex kernel-orchistrated synchronization. Just as basic as what the program needs, if it wants to run in multiple processes.
This point is debatable though, as having a standard way for the user to interact with the program data may be useful, at least for debugging purposes.
But you can still have a filesystem for user files, and a way to give programs the access to them, like the way things work nowadays :)
etc..
The pattern is here is to make systems much simpler, by having only one very basic communication mechanism - memory read/write, that exploits properties of the paging. And this mechanism is the most optimal that you can get in a computer, so you are picking-up the performance that is lost in expensive communication.
Also the security is potentially increased, as much smaller amount of kernel code is running at the request of user programs that also interacts with user program' provided data.
That's one area that I think it has been left behind. We know a lot by looking at the flaws of current ones we have, but to challenge the world with a new solution is to go against a huge stablished market.
i dont understand how he says in one sentence that 400 processes running is too many but then suggest drivers should run in userspace. would this not also result in a huge number of processes?
He explicitly said that drivers should not be a thing
What does he even mean by installs?
shared libraries, depend on who is allowed to upload. if f.eks only linux staff is allowed to upload to them then its a great idea but if its open freely to anyone then its a horrible idea.
I hear you, man. Today's OSes are bloated pieces of crap.
AmigaOS was sort of good and simple. Windows look so ugly to program; at least there is Linux though not perfect.
"Every program should be sandboxed" then proceeds to "User space dekstop environment"
Title should be "Jonathan Blow on fever dream"