1. Unused RAM is wasted RAM. 2. Sometimes high CPU usage is a good thing. 3. Inodes need to be tracked, as well as available space. df -i command is importatnt. 4. Large number of distros is good thing, a superpower. It allows you to change up the user interface. 5. Use LVM (logical volume manager) to get more control over your storage, treating multiple things as one. You can resize file systems online. 6. You don't have to memorize Linux commands. Memorize everyday ones; it's okay to look up lesser-used ones. 7. Always have a backup distro. 8. GIT version control is not just for software developers but also for system administration. 9. LVM has a snapshot feature, useful for trying new things. 10. Especially for cloud servers, take all individual volumes when backing up data.
Point 1 applies only if you have high enough ram to begin with. You'd need a decently high amount of ram if your workflow involves sometimes using lightweight apps and occasionally running multiple heavy programs at the same time. No unused memory is really wasted. You're just being prepared for when you do need it.
For years i was obsessing about have "free memory". And getting mad, that my operating system, "ate" all my memory. Untill someone, asked me why i wanted a slow inefficient computer.
The only real thing that needs to have a free excess is hard disk space. Everything else only needs to have a buffer of free. Its not about how much you have free, its about how fast that ram can do its job. That's all.
Around 2002, when I was fed up with the repeated crashes of my system, I went to ask three questions on the usenet forum: - "As a non-IT person, do you think I can install and configure a GNU/Linux system? - Am I obliged to know by heart a myriad of magic formulas? - Will these orders still be valid in a few years? The old wolves reassured me, and told me that if I got into the habit of reading the manuals and getting information before doing any manipulations, things would fall into place quite naturally. Moreover, the commands that I will have to use will be the same in ten years. They were right, and I just jotted down some "cooking recipes". 8-)
In my experience, most GNU utilities tend to keep the options the same over the course of time, just adding new ones. But, if you jump over to BSD the utilities may have the same name, but different options, and sometimes the GNU has things BSD doesn't. I encountered such issues with the "whois" commands, so I had to tweak how I automated checking for expiration dates and name servers when jumping between the two.
Fast forward ten years and you have systemd with totally new commands and concepts, plus a lot of other replacements (ip address show instead of ifconfig for example).
@@illegalsmirf very true, I didn't say that we don't get new software. Like getting a list of open sockets, we now have ss -tpln to see what daemons are listening on which TCP ports. I have also moved from ipchains, to iptables years ago, and I think that we have moved on from there. But I don't manage several firewalls for ISPs anymore, so a bit out of the loop there.
Thanks for being transparent about not having to memorize commands. You just lifted a huge load off of me brother. I have definitely built my own cheat sheet. Which is very helpful. As always, I love your content. Keep up the good work.
The real skill you need is to be able to refresh your memory quickly. So you can look up what you need and understand what you are reading. Only the real world tests this - it applies to every aspect of computing and most other things as well.
I now remember cut because I've used it a few times in BASH scripts, but I don't really remember awk and sed, so I just consult some scripts I've used them in. I throw those scripts up on github and consult them when I need to remember what I did before. Also tab is a real blessing trying to remember or discover commands.
After you’ve used certain commands and options a few dozen or hundred times, you do automatically memorize them. For the rest, there’s man (among other sources). This is why I’ve started writing man pages for some of my own programs. Nitpick: grammatically, it should be “I wish I had known”, not “I wish I knew”.
The biggest handicap I experienced with Linux, which is what I have used for years as my primary OS, was a an absence of well documented applications. That has greatly eased over the last 20 years or so, and generally anything you can do using a Windows (or Mac?) system can be done on a Linux system.
More or less, yes. There's always the learning curve, though depending on the person. A lot of people can pickup Linux pretty quick and others will struggle a lot.
Most things, yes. The difference is more often “how easily” or “how quickly”, rather than “at all”. Frex, I just found out that there’s something very much like Time Machine for Linux that’s been around for a while, but it’s nowhere near as easy to set up or use. And sometimes the deficit in UX is a dealbreaker-for some software, the UX is as much or more important than the functionality. For me, the main software I’m not willing to give up in order to use Linux is Scrivener. Several open-source and commercial imitators now exist for macOS, MSWindows, and Linux , and some people find some of them just as good as Scrivener. I don’t. There’s other software that I’d have trouble replacing (BBEdit, InDesign, 1Password, Things, frex), but I either already know of Linux alternatives that are “good enough”, or they’re more “what I’m used to” than “what I love”, so I’m sure I could adapt to new software and new workflows.
not really true. audio production does not have a workflow on linux that isn't a huge pain in the ass. and most plugins are not supported. its unfortunately the reason why i can never use it as daily driver. also, no photoshop isnt quite a deal breaker but its close. there are plenty of examples of this where you can technically do the same thing but you are forced to using a greatly inferior (much of the time) app to do the job. the fact is that even here in 2024, no one uses linux as their daily driver because its good but because they just hate windows and apple too much to stomach using them anymore.
@@vivsavagex There is an immense difference between can't be done, and "I don't like the work," which is your position. Many computer users confuse having to learn an interface with "difficulty." There are many entries these days on audio production on Linux. Many even recommend Linux as a platform. That said, the first time I ever met someone who did his own processing, he was a black jazz musician who used Linux. I still have a CD of his.
To explain the multitude of distros, I like to use a car analogy. There is one brand of engine, Linux, and a bunch of related models (aka versions). Then there are a couple of popular brands of chassis (Debian and Redhat) and a few other less common ones (Arch, Suse). On these chassis, companies then add the controls (gear stick, steering wheel, pedals) and interface, generally using off-the-shelf families of controls (KDE, Gnome, XFCE), and then add the body and styling and such. So all the cars have the same engine, and there is a small number of choices of chassis, and then each distro adds its own customisations on top of that engine and chassis. Ultimately, like driving a car, once you are comfortable with changing gear and steering, you can easily move from one car to another and only have to re-learn stuff like how to use the radio. Some cars will go fast; some will let you carry more cargo; there are vans and go-carts, and such. By comparison, Apple is like a Rolls Royce that makes three brands of cars, using entirely in-house engines, chassis, and so on, and with a policy that if the electric windows stop working, the solution is to buy a whole new car.
I've heard OSs compared to cars and the makers compared to dealerships before... the analogy I heard involves a busy street intersection. At the corners, there are two dealerships, and also a large paved lot full of various groups of people in tents and makeshift sheds working on tanks. One of the dealerships sells sleek, modern, sports cars. They're difficult to maintain and almost impossible to fix yourself, and overpriced... but they look fancy and slick, and owning one is a status symbol. If it breaks you can bring it to the dealership for servicing, but most likely they'll tell you to buy a new one, or a repair will cost more than a replacement. The other dealership sells family sedans. They're cheaper, but also less slick and stylish. They're technically easier to service and somewhat less tightly integrated and hermetically sealed up, but nobody who buys one tries to service it, beyond maybe a new paint job or an aftermarket horn. They break down, you take it to the dealership and pay the dealership to fix it. The guys across the street, are teams of volunteers. They make tanks. The tanks are sometimes uglier than a standard car, and can be far more confusing if you pop the hood, and the controls aren't always as simple as the dealership sedans or slick sports cars. But they're insanely safe, extremely smooth and fast on the road, and if you know what you're doing, far more accessible to tinker with. They come with extensive manuals and documentation. And the volunteers will literally give them away to you for free. If something breaks - they will give you the manual and tools to fix it if you broke it, or come and repair it for you if they broke it. These people do this because they wanted the tanks, and believe that something of this nature should be made available to anyone who wants to use it, for free and with the blueprints available openly for anyone to build their own or modify the blueprints. Multiple times a day, someone will come to this street corner with a broken or out of date sports car or sedan, or looking to buy their first car. The volunteers will tell them all about the tanks, and try to get them to take one. They'll protest they don't know how to maintain a tank. When told they don't know how to maintain a sedan or a sports car either, they'll say "But I can bring it to the dealership and pay them to fix it when it breaks." The tank people explain that they will fix it when it's their fault, will never charge you again for the newest model, and if you break it they will give you the tools and documentation to fix it, for free. The car shopper will still go into one of the dealerships and buy their overpriced product, because the tanks sound overwhelming. (On the other side of the spectrum, nowadays, a parent will pull up to the tank people in their well maintained tank, their young child in tow, seeking to pick out a good tank for their child to learn to drive in.) Once or twice every so often, someone who didn't come looking for the tanks will actually take a tank. Most of them return it in frustration and buy an overpriced new sedan.
For anyone wondering where the previous analogy came from, it's from "In the Beginning... Was the Command Line", an essay by the fiction author Neal Stephenson.
And when you go from a car that just works and us uniform to a tank and you want to set the chair and inside mirrors, suddenly something breaks and when you bring that up the volunteers say "well mine never did that, it must be a rare issue" and you have to learn everything to find out what the problem is and then the seat is too small but you are caugh by choice paralysis and just go back to the sedan bc it's easy, it works and it doesn't expect you to learn a lot about a topic that doesn't interest you BC you don't care about cars or tanks, you use them as a tool, not a character trait so having em just work is something you just want
1:47 The key thing to identify this situation (in the “top” display, or output from the “free” command) is to look at “available” RAM, not “free” RAM. The former includes cache usage, the latter doesn’t. So don’t worry if the latter is small, so long as the former still shows some useful amount.
I agree with you, Jay, regarding the certification process. It seems that the primary focus of these certifications is more about generating revenue rather than genuinely ensuring quality or skill. Many organizations appear to leverage these certifications as a way to monetize professional development, rather than offering a truly valuable and rigorous credential. This commercial approach can undermine the credibility of the certification and the true expertise it is supposed to represent. It's important for us to be discerning about which certifications we pursue, considering not just the cost but also the tangible benefits and recognition they provide in our field.
This video made me feel much better in regard to memorizing all the commands. I took the Red Hat course through my school and was overwhelmed with all the commands. Thanks.
Yeah, that is something that took me a while to get comfortable with as well. I learned very early on to be very specific in my comments in my code, etc. because, next week, next month, or next year, I might need to do that same exact thing, but I will not remember it. If I wanted to memorize stuff, I guess, I could have been a Lawyer. LOL!
As I love it, to add the costs of the items in my cart, having the exact amount in my hand, when approaching cashier, see his smile cause it went so quickly, so I like to write cmd because stuff is quickly done. To memorize ~30cmds should be no problem for average pple.
@@DrKellieOwczarczak This is exactly me as software engineer. I always add comments and detailed docs to my projects (especially APIs). I do other stuff like flowcharts, system schemas etc. I always write test for my code and have backups for all of these. I even created myself a tool (as a project) that keeps details with tags, topics and description, which I can pull if needed. Also in most cases I prefer code readability than length or syntax, and that helps a lot. In that way I am not stressed and worried about my work.
4:56 It would be easier just to say that inodes limit the maximum _number_ of files you can create on the volume, regardless of the space they occupy. So if you run out of inodes, you cannot create any new files (or directories), you can only extend existing ones.
@@CTimmerman Depends on the config, you can specify a number of inodes per X MB's .... but using XFS they are created dynamically, so you never run out.
8:28 The point about using a different distro if you don't like the user interface - I would take this one step further and point out that for the most part you can install pretty much any desktop environment or window manager on any distribution. You don't have to even switch distros. Ubuntu is famous for it's unique spin on Gnome, but you can easily rip that out and replace it with anything you want. Or, you can install several desktop environments and switch between them as your mood dictates without any real issues.
I was just thinking about this the other night, while trying to sum up why I like Linux to a friend. The extensibility is actually insane when you think about it too much. You can do everything up to and including compiling a custom kernel. Literally every aspect of the way your machine behaves, looks, acts, and reacts is up for modification. But even before that you have desktop environment/window manager choice and extensibility, and before that you have the, by Windows standards, already insane levels of customisation that the vanilla install of something like plasma or gnome offer. People in Linuxland get mad about sysD, people in windowsLand get served ads on their desktop and have to just like it.
I ran into the inode issue a very long time ago, also on a mail server, however, it wasn't the messages themselves, it was a file system based grey listing solution I was using to help with spam. The answer then was to dedicate another drive to that solution, but to make the clusters very small, so very many inodes. As to LLVM, I avoided it on physical servers, because of the physical backup and recovery of a failing drive was much harder with LLVM in the mix. I see it as a great plus in a virtual server system.
Point 1 applies only if you have high enough ram to begin with. You'd need a decently high amount of ram if your workflow involves sometimes using lightweight apps and occasionally running multiple heavy programs at the same time. No unused memory is really wasted. You're just being prepared for when you do need it
I think it depends on the intended use case. If you're running a database, it's a good idea since as the index grows it just occupies more RAM. If it's something that has more sporadic usage, maybe just let it use some of the swap space when it spikes.
When it comes to using Linux commands, I think it’s worthwhile to try to remember the general syntax of commands you use frequently, but don’t worry about remembering all the options / flags / arguments/ etc. So long as you have the name of the command, most commands are a simple “command -h” or “command -help” away from getting a list of available options and ways to use it. Yes you can do “man command”, but manual pages are confusing and terrible! 😀 They’re either way to brief with no additional info or WAY too long and convoluted, so at least see if the command has a -h or -help flag first.
@@brettsjoholm yep. Sometimes -h or -help doesn’t do much better, but often it will give a nicely condensed blurb of the command syntax and what each of the possible flags / arguments do.
I've been using S.u.S.E. Linux since 1994. One thing I never understood is why you would want to install a new distribution just because you don't like the GUI. Not a fan of gnome? just install KDE. Problem solved. Well, if your distribution doesn't make it easy to install another GUI, then yeah, change the distro.
I think it's such an important point that even the most experienced people do not memorise everything, and thank you for saying it because I think it's often a scary thing for us newer and less experienced people often worry about. It's something I learnt with Photography that even the best photogs do not turn up take one shot and it's perfect. they take thousands of pictures to find their best shot to show the world. I've been watching lots of the "into the terminal" series form Red Hat recently and it's so great seeing one of their most experience people there even makes mistakes and forgets things. It's just down to what you use on a daily basis is different from the next person. I'd love to learn more about your career, how you transitioned and what you do now. I think it would also be really great to learn what you use for your note taking. Obsidian is such a widely discussed tool these days. I've taken a look at Logseq which looks really interesting but I cannot wrap my head around how to use it effectively and always find myself comparing it to Obsidian which I probably shouldn't. Do you host things like your notes or do you keep them local on your computer?
2:27 Yes, it most certainly is true. Remember, userland apps like GNOME are not responsible for managing filesystem cache--that is done by the kernel. So any RAM the apps are using is actual regular application RAM, and if they are using a lot more than other comparable apps, then yes, that counts as “bloat”.
Yeah, gonna be directing my students to you. Your calm explanations are sound. Heck, I learn things and I've been around the block a few times. Thanks for putting these out.
I had what I called the big 2 when I started in 2004 - Ubuntu, Fedora and Opensuse. These were what I used either as a primary driver or my backup. Lately I now use Linux Mint. I've learned to use a Home partition to make this easier.
LVM is not a backup solution but many think so because it can include a “cloud” service. Data exists only where it located. In other words if the data in any one place, like an LVM cloud space there is no backup. Unless there a considered plan to make data replicated in more than one place it can be lost. Usually data should exist in three places, the original, a live backup, and an offline backup (one offsite and dead to changes, think of a place that is kept in a safety deposit box.
I use flash cards for Linux Commands & specialty terminology. I wanted to remember ... to memorize certain things because they were new to me when I first learned them, and I planned on using that info regularly. For me, repetition was the most effective memory tool for Linux.
High CPU usage might not be an issue on a server, but if your desktop is idling at, say 80%, you don't really have much overhead for other applications
Dont forget that you dont have to change distros if you want a specific desktop environment. More often than not you have plenty of tutorials on how to swap DE without remaking your whole system or learning the ins and outs of another distro. Distro is more of an idea of an amalgamation of what a gnu/linux system should look and work like as the foundation. I might be wrong but the distro selection usually revolves around what package manager it uses... While newcommers are usually just looking into the looks and popularity (which isnt wrong at all). I use arch (btw) because i didnt mind the rolling releases (tbh you can kind of opt out of this ) and the fact that it comes out of the box with basically just the kernel, bootloader and package manager, from there i just pick my "goey" and some apps. You like apt-get more but you wanna use a specific distro that uses something else? Noone is stopping you and its usually not that painful to swap package managers
The main disadvantage of having 100s of distros is that it makes extremely difficult for a 3rd party vendor to supply support for their products. Imagine having to teach your support team Ubuntu, Fedora, OpenSuse, Arch and then all it's dozens of variants. It's insane. The one thing google has done with linux is take it out of open source and streamline it into ONE OS period.
Yes, that's info that I can definitely use. Now I will use git and LVM in ways that I didn't realize. I see them all of the time and thought that only admins and devs needed them. I had forgotten about inodes because I haven't had problems with those for years. But it's a good idea to know how to fix the problem in case it ever comes about. I think that it happened to me a long time ago but I thought that the distribution was buggy. 😏
Cool thing about lvm, sounds like you were suggesting you can just span your volume across drives. You csn, but people often consider that unwise. What you can do that's cool though is move your data while the server is online. Add new drive to the old one as 'free blocks", then online move the old drive's blocks to the new drive. Then you remove the old drive (easy since it's empty, block-wise) and then expand the partition to use the rest of the new drive's free blocks. Doesn't play nice with btrfs because what does?
For point 7: I don't really care about the distribution, I care about the DE. I'm a desktop user, not a server admin, so the look is more important to me than the underlying stuff. But the point still stands: if KDE goes away, I'll have to have a hard look at which DEs provide the most Windows-like environment.
I personally like GNOME because I was using Windows since 2001 and i'm just bored of seeing a bar with a menu so the way of Gnome to show apps and all always makes me to use it. But I don't like some ideas of GNOME and lack of customization even there's extensions that can break the system in every update... I'm using Debian rn
Couple of handy analogies for gamer types: think of Linux like modding a game, example Skyrim. Linux is your Skyrim and the various bits you add are the mods. Some are content like a new merchant or gear (apps, package manager, games, etc) Others are basic frameworks or dependencies, like XP32 for skeletons and ragdolls. As for the I-nodes that's like inventory systems, everything has weight (kb, mb, gb) while the nodes are your inventory slots, doesn't matter if your carry capacity is 10,000kg if you're out of inventory slots. Yes I know Skyrim doesn't have limited inventory slots but it is quite crash happy if you collect too many items in a single map location, so the concept is close enough.
Number 7: It seems, due to the path Ubuntu is going, Vanilla OS and Pop OS will derivate themselves directly from Debian, no more from Ubuntu. Things are changing.
The TL;DR on inodes is that a directory entry (when you see a file in your file manager, or when running ls, what you're really seeing is a directory entry) points to an inode. An inode then tells Linux where the file's data is on the disk. The number of inodes is (I think) set when you create the filesystem. Each file or directory requires an inode. So no available inodes means you can't create a new file, only allocate more free space to existing files. A nice thing about inodes, and actually something you get with NTFS (and I think it only got put there because MS wanted to have a minimal Posix subsystem in Windows NT so that they could get government contracts) is that you can have multiple directory entries pointing to the same file (hard links). This means you can efficiently create a clone of a directory hierarchy, which I tend to do before doing any destructive power renaming (e.g. applying a naming scheme to a bunch of media files) in case my Python script has an error which messes things up. With NTFS, hard links are hard to make unless you're running cygwin or git-bash.
Hell yes. This was a great video. I wish you'd cover a lot of things you didn't know at the first. Maybe, organize them according to complexity (or career-period) versus just random topics.
I've been in Linux System Operations for about a dozen years but have been using Linux since 2.2. This is a well done list. I think the only thing I'd add to really introductory material like this is this: du and df. They might (probably will) report different usages. This can be frustrating when tracking down a "disk full" issue. This, really, is far more common than the exhausted inodes issue. If a file is deleted (but still actually open by a process), df will report that space as still used, while du will not. If this deleted file is very large, the difference could be huge. lsof (list open files) will help you track down the culprit in this case. Often this is caused by a log rotation that does not restart the application doing the logging.
You spent a bit of time talking about LVM and it's snapshot and backup capabilities, but in my experience, LVM has basically been supplanted by zfs in most instances. for pretty much any system that requires redundancy in file systems (multiple disks / some level of RAID) zfs is significantly superior when it comes to filesystem integrity over stuff like ext4 and btrfs, as well as using hardware raid controllers (which often do the wrong thing in a collision). I would suggest folks learning about linux learn about the zfs tools rather than using the old lvm ways of doing things
Memory from the cache can be provided to any application which needs it. So basically we should have free memory more, instead of Cache memory as cache memory is not that important. If we have cache memory more, we can either reboot the system to increase the memory and eliminate it from cache, or we can use a command -> [echo 3 > proc/sys/vm/drop_caches] Correct me, if I'm wrong 👍🏻👍🏻
I also want to add. Notion has been super useful for creating my own cheat sheets. I basically extract all the useful stuff from the man page for the command, and than embellish that with stuff I’ve learned elsewhere.
@@PoeLemic it’s an app for taking notes. It basically uses an abstracted form of markdown language. You just type backslash followed by what ever your looking to add (ie table, bullet, heading, url, page).
I had a terrible weekend trying to fix an error related to disk. And yes it was inodes. You learn about that really fast if the main website is down...
Thanks so much for your Linux Instructional Videos: When I first jumped into Linux 15 years ago, as an alternative to Windows, I just wanted to be a user ... not a script writer, analyst, or administrator. I soon found out that to be a Linux User, is to wear all of those hats. The ability to self-educate was difficult for me as the availability of quality instructional material was difficult to find ... at least at first. I do have a friend who is an experienced Linux Administrator and that has helped over the years ... especially with the catastrophes that can happen. I am glad I found your channel ... it's been a good source of quality information. I still have a long journey ahead. Thanks once again.
Yes, you are not the only one. I'm moving away from WIndows, because I think LInux is more the future. So, I'm trying to educate myself too. And, it feels like a long road too.
i was trained that on a virtual memory machine CPU utilixation should be 30% ... for responsiveness ... over 50% and the system is thrashing .. moving pages and not working and not responding to requests
I’m getting there with the Linux commands. I think my biggest hurdle right now is learning how to effectively use pipes. Also I feel like I’ve barely scratched the surface of things like grep and awk.
Server CPU can be low if it's waiting for you - like my nextcloud on my raspberry pi ;) But you are right and this was a major driver in virtualization and later containerization - use the resources to the max with minimal overhead and bottlenecks. Yes, backups are only really in existence if you tested them and retest on a schedule!
I think that the CPU usage is widely variable. I manage web servers and email servers for customers these days, so unless there's a lot of activity, CPU usage is very low. But I used to deal with monitoring servers for ISPs, and high loads were common. When pulling data from thousands of devices on a regular basis, the servers were kept very busy.
I agree about not trying to remember commands you don't frequently use. As long as you can remember a search term or part of the actual command your trying to utilize then you can look it up online or in the man pages, etc.
nr 7. unfortunately there is not always easy to have a backup distro at hand. in many environments there are other constraints than the distro itself and the app running on it: availability of official/verified images in public clouds, your security agent may only support a limited number of distros, etc. some distros, even if supported by your apps get support last of all (debian :) in some cases you just have to go wherever the big players are going.
And LVM can do raid , check the doc on how and what kinds of raid . A great way to learn Linux is to read the man pages , as many as you can stand . LVM has great man pages . Also name LVM name conflicts with LLVM which is Low Level Virtual Machine , so some distros are naming LVM , LVM2 now which stands for version 2 but also gives it more of it's own name , since LVM conflicts with Level Virtual Machine.
I've seen in practice that using LVM in a cloud environment is not the best solution. LVM carries the inherent risk of data inconsistency, especially during restoration. For example, we had a case involving 14 hard drives. The restoration process was complicated due to issues in correctly mapping the drives. Consider using cloud provider’s native snapshot and backup tools instead. These tools are designed to handle such complexities and provide seamless volume management and restore capabilities.
If I can remember a few letters of a command I once used, sometimes I'll drag .bash_history file into a text editor so I can use the search function. If you use a shell other than bash, the file name will be different. Although the Up-arrow will eventually get you to that past command, sometimes I used it two or three years ago. That can mean a lot of clicks.. Olden days .bash_history would get truncated to save resources, but on desktop systems the file can stay alive for years.
i agree with Shawn Lewis below after hearing what you said about commands. I've always felt exactly as you do, but didn't think about this idea of certification tests demanding knowledge of commands mostly likely soon to be forgotten post exam. I totally agree that we have documentation at hand, and quickly on the Net, for all Linux commands. And the one's that we use often are the ones we naturally memorize.
Thank you for the effort put into this video. I would try to minimize the redundant sentences however. For instance the section anout inodes really could have been explained in a much more direct fashion.
I have a shell script that displays a text file in the terminal. It's my personal help system. Plus I also keep a lot of notes whenever I perform tasks in the OS. Like say when I compile some software I create a text file with notes about how I built it. I can always refer to those notes when I want to do that task again.
4:24 you could argue that the purpose of a server is not doing work but providing the possibility that work could be done if it is demanded. So even an idling service provides the service of the possibility and so is not without purpose.
Great video, Jay. I'm always learning things from you. It seems that you have an endless supply of Linux knowledge to share. That's OK, though; 'cause I have a pretty healthy appetite for learning those kinds of things. Thanks for all of your videos and books. They've been a great help to me. From the Linux Mint Essentials book that got me started with Linux (back in the day); to the Mastering Ubuntu Server books of recent times... What a long "road" we've traveled since the CP/M days. ...and you've helped to make the trip a great experience for me. Thanks bunches, Jay.
12:58 writing down commands goes double for anything tricky, obscure, or troubleshooting. I’ve got a notebook in evernote of these sorts of things, ranging from my favorite parameters for rsync [actually, I should go add that one] through standard settings I want to change after a new install (or check after a major version upgrade) and aliases I want to add to my terminal config, to the magic incantation to patch around Solus Budgie’s inability to detect my screen resolution or maintain it once set. I started this notebook after a particularly frustrating time configuring openvpn on my server because I (1) hadn’t changed anything, but it stopped working and (2) hadn’t touched it in a couple years and couldn’t remember what I had done. So I wasn’t even sure what the settings _should_ be or where some of the config files needed to be or what responses indicated the vpn was running and funneling all the traffic through it. And when I went back to the resources I had originally consulted, the webpages were either gone or had been updated to give instructions for different software than I was using.
Git is great for everyone - whether youre writing documents, anything from memos, a PhD thesis or novels to hardcore software projects like the Kernel, less formal code projects like data analysis/data science projects or sysadmin scripts. If you write anything in your job, it's worth learning git. Just learn to write mostly text based (like .txt, Markdown, LaTeX, code files) rather than flooding your repo with large binaries like Word docs
number 2: I ran a linux server on a vm for the company I work for. It was always at 100% cpu usage even though it was literally never doing anything. In my case, this just had to do with how the vm was reporting usage to the hypervisor, which is fine and I understand that. I could have allocated 1000 cores to it and it would have reported 100% usage. Shrugging it off is wrong though. The vm or server should be sized to the workload and not waste any extra resources. IF it legitimately needs those resources, then by all means allocate them and explain to the client that they are required for fast workflows. But losses and inaccuracies can occur. You have to stay cognizant of that fact.
Whew!! I’m so glad to know I’m not the only Linux noob using man pages. I learned a few things reading the man ls pages the other day. So much more than I realised. Great video. I’m going get that book too.
Thanks very much Jay, this was very useful advice. I'm particularly interested in the LVM implementation - will definitely be checking out your content for that. 👍
I use the ZFS filesystem as root filesystem with native encryption on /home and direct boot from UEFI (no grub) with the Gentoo distribution. ZFS has some cool features like deduplication, which is very hardware intensive, and snapshots like described in this video on LVM. Gentoo allows me to not use systemd (which I hate), compiles the code specifically for my CPU model, is a cutting-edge rolling release, and has very good tools for compiling/upgrading the kernel. I am giving this as possible alternatives, but not as advice. Use whatever works best for you.
This was another great video! Interesting things you'd learned, practical uses for familiar tools, and an honest look at how much you need to "know" to run a server. Since 1994, the `apropos` command has been a stalwart friend!
I use Manjaro and Linux Mint on my main PC, Fedora on my laptop for work, Ubuntu on my x86-tablet, Tuxedo OS on my private laptop and Proxmox, OpenSUSE, Debian and Arch on my Servers.
Thanks for this sir! Side note, I'm living the plan B distro. Always wanted to run Debian and the Red Hat/IBM shenanigans were sour for me. Bummer, Fedora is so nice. C'est la vie. Hello Debian!
There should be an additional point: Linux plays well along with the other „kids in the yard“ like Windows and MacOS, even when they are not vise-versa that friendly against Linux. You can mount and interact with the others quite well, even that they pretend they are the only one’s that exist. That even goes to the users of those systems! I can‘t count, how many discussions i had in it support like: „Thank god we are using windows and not Linux!“ - „Dude, you know that you wrote that mail on a Linux-terminal, which remote controls a windows system, which is virtualized on a Linux system …. right? And by the way, each system has his advantages and disadvantages just saying“
Personally, I would prefer fewer distributions and better built (visual consistency and more controlled packaging) than the current situation. MacOS is an example of something well finished in every detail, although I do not like some things, the first thing is enjoyed and thanks much more.
I always thought that KDE used about the same amount of RAM as windows with just a browser open. Maybe thats actually not the case? When I was doing my electrical engineering course (seems a lifetime ago). One teacher in particular had not issue with us using our text books during exams. As he put it, you're always going to have access to information in the workplace. Its knowing where to look and interpreting is the test.
@MichaelDustter When you first start it up, maybe. I'm running Fedora ATM and with only a browser with a couple of tabs, it's 4.6GB. Similar to what windows would use. Even in TOP it says about the same.
2 commands 4 locations to memorize. ls, man, /bin, /sbin, /usr/bin, /usr/sbin. ls /bin, etc... shows programs to run. man which provides the manual for the which program.
Man I’m feeling this right now. Bought a Dell Micro 7000 to run my plex/arr set up and it’s sitting at 2% even under “heavy” use. Could have just used the old i5 I had lying around.
Oracle "Unbreakable Linux" was the choice of most of my Centos clients. Red Hat screwed a lot of people by depreciating Centos. I don't really trust Oracle but they will be around for awhile. Like IBM supporting SUSE, and IBM pays for Linux development. (IBM and NCR use SUSE in their POS and ATMs.)
Is it possible, to some extent, to place the distribution you're using in a GIT (a form of backup)? Either, to reinstall or synchronize parts of the FS (with this differential you can know several aspects of your FS)?
Aliases are great, but that’s not designed for memorizing things . aliases are for simplifying commands and creating short cuts . I have several videos that cover aliases.
The number one thing I would tell people who are new to Linux: don't let *anyone*, no matter how experienced they are, tell you that there is one "right way" to do something. Whether you choose floating windows or a tiling window manager, desktop or terminal window, "beginner" distro like Mint or compile your system from the ground up with Gentoo - doesn't matter. What works for you and what you like is what you should go with. Period. Yes, Linux people will have (usually well meaning) opinions about tools and OSes you should check out. Take their advice if you want to, but don't feel obligated to do *anything* because it's your computer. You're the one who will be daily driving it, so do with it what *you* want; not what the Linux Community thinks is right.
Number 3 reminds me of the Unreal Engine UObjects limit. Your RAM and drive might not be out of space, but the garbage collector can only keep track of a certain number of UObjects before acting up.
Instead of memorizing individual commands - is it a good idea to create the series of commands necessary to perform a certain function under a macro, and then just call up the macro as you need it?
I’m a Windows user who used Linux infrequently and is very rusty. I was researching curl and I saw this complex example in Stack Overflow using Lynx, SED and Awk and instead of installing Windows versions decided to install Mint on an old laptop to apply this example I found (which had typos). I was gratified to connect to the internet and then change to the command line with ct-alt-F1 still connected to the internet to play with curl. I do remember running Redhat 2.4 which ran beautifully with sound and I then ran a distro upgrade and my anguish when the upgrade broke the sound driver. I was using a weird-ass motherboard sound chip and I ended up buying a common sound card just to run with Linux. SoundBlaster from memory. The mobo sound chip was Aureal and the company went out of business so the support had stopped. Follow-up: Creative Labs (maker of SoundBlaster) had sued Aureal over patent infringements (multiple lawsuits) and Aureal won but the legal costs bankrupted the company and Creative Labs acquired the IP at the bankruptcy hearings and Redhat withdrawing the drivers would have been the nail in the coffin for Aureal products. Sucky yes, it really is a dog eat dog world.
Honestly when I first started using Linux, what kept drawing me in when I wanted to give up was 1. Most of the time, especially on modern hardware, Linux was faster, and that was with the supposed resource heavy GNOME desktop of Ubuntu 18.04 at the time. 2. Was the fact that there was a bunch of options, so if I got tired of Ubuntu, I could try a different OS, and I love as much customization as possible. I understand the necessity for a universal UI, but the fact that I can't even move the taskbar to the left side of my screen anymore in Windows, that's too restrictive in my opinion, and with the gap for gaming getting smaller, I don't have as much of a reason to go back to Windows anymore.
Great video Jay, as always! I have never really understood the fixation some Linux distro reviewers have with looking at how much RAM the system is using after first boot, or what it's using during normal operations. I'm like, what is this, 1998?
I've just started dabbling into Linux, so far I've tried Debian, Lubuntu and Mint. So far I like Mint the best. What started me down this path was my brother in law gave me 2 old laptops to mess around with, knowing my affinity for working with old computers, but there were 2 issues out of the gate: first one was that he didn't know the current Windows passwords for either them, and the second one was that even if he did, they were both running Windows 8, 'nuff said. I think they would have supported at least 10, and one of them, maybe even 11, but why would I shell out the money for licensed copies of Windows on computers that aren't my "daily drivers"? That said, I decided to step out of my comfort zone and give Linux a shot with old hardware and nothing to lose, much like I did with DOS and Windows 3.1 back in the 90's when I first started building computers, and/or messing around with old systems that were headed for the trash anyway
LVM sounds like it is mostly for servers; not desktops. Am I right? 🤔 Also there should be backup options that are aware of LVM volumes to get it all 👀
1. Unused RAM is wasted RAM.
2. Sometimes high CPU usage is a good thing.
3. Inodes need to be tracked, as well as available space. df -i command is importatnt.
4. Large number of distros is good thing, a superpower. It allows you to change up the user interface.
5. Use LVM (logical volume manager) to get more control over your storage, treating multiple things as one. You can resize file systems online.
6. You don't have to memorize Linux commands. Memorize everyday ones; it's okay to look up lesser-used ones.
7. Always have a backup distro.
8. GIT version control is not just for software developers but also for system administration.
9. LVM has a snapshot feature, useful for trying new things.
10. Especially for cloud servers, take all individual volumes when backing up data.
Good notes :)
Video sounds like an AI hallucination
Point 1 applies only if you have high enough ram to begin with. You'd need a decently high amount of ram if your workflow involves sometimes using lightweight apps and occasionally running multiple heavy programs at the same time. No unused memory is really wasted. You're just being prepared for when you do need it.
tab completion and bash history were the things that I wish I'd known about from day 1.
Thanks for the notes
For years i was obsessing about have "free memory". And getting mad, that my operating system, "ate" all my memory. Untill someone, asked me why i wanted a slow inefficient computer.
Why wouldn't you use what you paid for? Only worry when there is little left or it's frequently close to full.
"Unused memory is wasted memory."
This is the most concise explanation I've heard of this concept.
The only real thing that needs to have a free excess is hard disk space. Everything else only needs to have a buffer of free. Its not about how much you have free, its about how fast that ram can do its job. That's all.
@@scheimong Is your swap file the size of your free space yet? Virtual memory matters!
@@lliamthrumble Chrome gets really slow when the OS starts to swap, so i built an OOM killer to save my SSD and other programs' data.
Around 2002, when I was fed up with the repeated crashes of my system, I went to ask three questions on the usenet forum:
- "As a non-IT person, do you think I can install and configure a GNU/Linux system?
- Am I obliged to know by heart a myriad of magic formulas?
- Will these orders still be valid in a few years?
The old wolves reassured me, and told me that if I got into the habit of reading the manuals and getting information before doing any manipulations, things would fall into place quite naturally.
Moreover, the commands that I will have to use will be the same in ten years.
They were right, and I just jotted down some "cooking recipes". 8-)
In my experience, most GNU utilities tend to keep the options the same over the course of time, just adding new ones. But, if you jump over to BSD the utilities may have the same name, but different options, and sometimes the GNU has things BSD doesn't. I encountered such issues with the "whois" commands, so I had to tweak how I automated checking for expiration dates and name servers when jumping between the two.
@@javabeanz8549
Rigor seems to be in order at GNU.
Thank you for this information about BSD that I personally don't know well.
Fast forward ten years and you have systemd with totally new commands and concepts, plus a lot of other replacements (ip address show instead of ifconfig for example).
@@illegalsmirf That's right.
Luckily we still have distros without this ugly thing.
@@illegalsmirf very true, I didn't say that we don't get new software. Like getting a list of open sockets, we now have ss -tpln to see what daemons are listening on which TCP ports. I have also moved from ipchains, to iptables years ago, and I think that we have moved on from there. But I don't manage several firewalls for ISPs anymore, so a bit out of the loop there.
Thanks for being transparent about not having to memorize commands. You just lifted a huge load off of me brother. I have definitely built my own cheat sheet. Which is very helpful. As always, I love your content. Keep up the good work.
The real skill you need is to be able to refresh your memory quickly. So you can look up what you need and understand what you are reading. Only the real world tests this - it applies to every aspect of computing and most other things as well.
And me!
I have no reason to think you should have to memorize a bunch of commands. I'm interested in why you might feel it necessary?
I now remember cut because I've used it a few times in BASH scripts, but I don't really remember awk and sed, so I just consult some scripts I've used them in. I throw those scripts up on github and consult them when I need to remember what I did before. Also tab is a real blessing trying to remember or discover commands.
After you’ve used certain commands and options a few dozen or hundred times, you do automatically memorize them. For the rest, there’s man (among other sources). This is why I’ve started writing man pages for some of my own programs.
Nitpick: grammatically, it should be “I wish I had known”, not “I wish I knew”.
The biggest handicap I experienced with Linux, which is what I have used for years as my primary OS, was a an absence of well documented applications. That has greatly eased over the last 20 years or so, and generally anything you can do using a Windows (or Mac?) system can be done on a Linux system.
More or less, yes. There's always the learning curve, though depending on the person. A lot of people can pickup Linux pretty quick and others will struggle a lot.
Most things, yes. The difference is more often “how easily” or “how quickly”, rather than “at all”.
Frex, I just found out that there’s something very much like Time Machine for Linux that’s been around for a while, but it’s nowhere near as easy to set up or use.
And sometimes the deficit in UX is a dealbreaker-for some software, the UX is as much or more important than the functionality.
For me, the main software I’m not willing to give up in order to use Linux is Scrivener. Several open-source and commercial imitators now exist for macOS, MSWindows, and Linux , and some people find some of them just as good as Scrivener. I don’t. There’s other software that I’d have trouble replacing (BBEdit, InDesign, 1Password, Things, frex), but I either already know of Linux alternatives that are “good enough”, or they’re more “what I’m used to” than “what I love”, so I’m sure I could adapt to new software and new workflows.
not really true. audio production does not have a workflow on linux that isn't a huge pain in the ass. and most plugins are not supported. its unfortunately the reason why i can never use it as daily driver. also, no photoshop isnt quite a deal breaker but its close. there are plenty of examples of this where you can technically do the same thing but you are forced to using a greatly inferior (much of the time) app to do the job. the fact is that even here in 2024, no one uses linux as their daily driver because its good but because they just hate windows and apple too much to stomach using them anymore.
@@vivsavagex There is an immense difference between can't be done, and "I don't like the work," which is your position. Many computer users confuse having to learn an interface with "difficulty." There are many entries these days on audio production on Linux. Many even recommend Linux as a platform. That said, the first time I ever met someone who did his own processing, he was a black jazz musician who used Linux. I still have a CD of his.
@@vivsavagex same with graphic design tools or even cad programs
To explain the multitude of distros, I like to use a car analogy. There is one brand of engine, Linux, and a bunch of related models (aka versions). Then there are a couple of popular brands of chassis (Debian and Redhat) and a few other less common ones (Arch, Suse). On these chassis, companies then add the controls (gear stick, steering wheel, pedals) and interface, generally using off-the-shelf families of controls (KDE, Gnome, XFCE), and then add the body and styling and such. So all the cars have the same engine, and there is a small number of choices of chassis, and then each distro adds its own customisations on top of that engine and chassis. Ultimately, like driving a car, once you are comfortable with changing gear and steering, you can easily move from one car to another and only have to re-learn stuff like how to use the radio. Some cars will go fast; some will let you carry more cargo; there are vans and go-carts, and such. By comparison, Apple is like a Rolls Royce that makes three brands of cars, using entirely in-house engines, chassis, and so on, and with a policy that if the electric windows stop working, the solution is to buy a whole new car.
I've heard OSs compared to cars and the makers compared to dealerships before... the analogy I heard involves a busy street intersection. At the corners, there are two dealerships, and also a large paved lot full of various groups of people in tents and makeshift sheds working on tanks. One of the dealerships sells sleek, modern, sports cars. They're difficult to maintain and almost impossible to fix yourself, and overpriced... but they look fancy and slick, and owning one is a status symbol. If it breaks you can bring it to the dealership for servicing, but most likely they'll tell you to buy a new one, or a repair will cost more than a replacement. The other dealership sells family sedans. They're cheaper, but also less slick and stylish. They're technically easier to service and somewhat less tightly integrated and hermetically sealed up, but nobody who buys one tries to service it, beyond maybe a new paint job or an aftermarket horn. They break down, you take it to the dealership and pay the dealership to fix it.
The guys across the street, are teams of volunteers. They make tanks. The tanks are sometimes uglier than a standard car, and can be far more confusing if you pop the hood, and the controls aren't always as simple as the dealership sedans or slick sports cars. But they're insanely safe, extremely smooth and fast on the road, and if you know what you're doing, far more accessible to tinker with. They come with extensive manuals and documentation. And the volunteers will literally give them away to you for free. If something breaks - they will give you the manual and tools to fix it if you broke it, or come and repair it for you if they broke it. These people do this because they wanted the tanks, and believe that something of this nature should be made available to anyone who wants to use it, for free and with the blueprints available openly for anyone to build their own or modify the blueprints.
Multiple times a day, someone will come to this street corner with a broken or out of date sports car or sedan, or looking to buy their first car. The volunteers will tell them all about the tanks, and try to get them to take one. They'll protest they don't know how to maintain a tank. When told they don't know how to maintain a sedan or a sports car either, they'll say "But I can bring it to the dealership and pay them to fix it when it breaks." The tank people explain that they will fix it when it's their fault, will never charge you again for the newest model, and if you break it they will give you the tools and documentation to fix it, for free. The car shopper will still go into one of the dealerships and buy their overpriced product, because the tanks sound overwhelming. (On the other side of the spectrum, nowadays, a parent will pull up to the tank people in their well maintained tank, their young child in tow, seeking to pick out a good tank for their child to learn to drive in.) Once or twice every so often, someone who didn't come looking for the tanks will actually take a tank. Most of them return it in frustration and buy an overpriced new sedan.
For anyone wondering where the previous analogy came from, it's from "In the Beginning... Was the Command Line", an essay by the fiction author Neal Stephenson.
And when you go from a car that just works and us uniform to a tank and you want to set the chair and inside mirrors, suddenly something breaks and when you bring that up the volunteers say "well mine never did that, it must be a rare issue" and you have to learn everything to find out what the problem is and then the seat is too small but you are caugh by choice paralysis and just go back to the sedan bc it's easy, it works and it doesn't expect you to learn a lot about a topic that doesn't interest you BC you don't care about cars or tanks, you use them as a tool, not a character trait so having em just work is something you just want
1:47 The key thing to identify this situation (in the “top” display, or output from the “free” command) is to look at “available” RAM, not “free” RAM. The former includes cache usage, the latter doesn’t. So don’t worry if the latter is small, so long as the former still shows some useful amount.
I agree with you, Jay, regarding the certification process. It seems that the primary focus of these certifications is more about generating revenue rather than genuinely ensuring quality or skill. Many organizations appear to leverage these certifications as a way to monetize professional development, rather than offering a truly valuable and rigorous credential. This commercial approach can undermine the credibility of the certification and the true expertise it is supposed to represent. It's important for us to be discerning about which certifications we pursue, considering not just the cost but also the tangible benefits and recognition they provide in our field.
This video made me feel much better in regard to memorizing all the commands. I took the Red Hat course through my school and was overwhelmed with all the commands. Thanks.
Yeah, that is something that took me a while to get comfortable with as well. I learned very early on to be very specific in my comments in my code, etc. because, next week, next month, or next year, I might need to do that same exact thing, but I will not remember it. If I wanted to memorize stuff, I guess, I could have been a Lawyer. LOL!
As I love it, to add the costs of the items in my cart, having the exact amount in my hand, when approaching cashier, see his smile cause it went so quickly, so I like to write cmd because stuff is quickly done. To memorize ~30cmds should be no problem for average pple.
@@DrKellieOwczarczak This is exactly me as software engineer. I always add comments and detailed docs to my projects (especially APIs). I do other stuff like flowcharts, system schemas etc. I always write test for my code and have backups for all of these.
I even created myself a tool (as a project) that keeps details with tags, topics and description, which I can pull if needed.
Also in most cases I prefer code readability than length or syntax, and that helps a lot.
In that way I am not stressed and worried about my work.
4:56 It would be easier just to say that inodes limit the maximum _number_ of files you can create on the volume, regardless of the space they occupy. So if you run out of inodes, you cannot create any new files (or directories), you can only extend existing ones.
How soon would a 1TB drive with Etx4 run out of inodes?
@@CTimmerman Depends on the config, you can specify a number of inodes per X MB's .... but using XFS they are created dynamically, so you never run out.
@@openevents Is XFS better than ZFS? A redundant storage pool of random old drives sounds useful.
8:28 The point about using a different distro if you don't like the user interface - I would take this one step further and point out that for the most part you can install pretty much any desktop environment or window manager on any distribution. You don't have to even switch distros. Ubuntu is famous for it's unique spin on Gnome, but you can easily rip that out and replace it with anything you want. Or, you can install several desktop environments and switch between them as your mood dictates without any real issues.
I was just thinking about this the other night, while trying to sum up why I like Linux to a friend. The extensibility is actually insane when you think about it too much. You can do everything up to and including compiling a custom kernel. Literally every aspect of the way your machine behaves, looks, acts, and reacts is up for modification. But even before that you have desktop environment/window manager choice and extensibility, and before that you have the, by Windows standards, already insane levels of customisation that the vanilla install of something like plasma or gnome offer.
People in Linuxland get mad about sysD, people in windowsLand get served ads on their desktop and have to just like it.
Would have been good to know this was about your career before I clicked on it. I am looking at personal tip for home use.
I ran into the inode issue a very long time ago, also on a mail server, however, it wasn't the messages themselves, it was a file system based grey listing solution I was using to help with spam. The answer then was to dedicate another drive to that solution, but to make the clusters very small, so very many inodes.
As to LLVM, I avoided it on physical servers, because of the physical backup and recovery of a failing drive was much harder with LLVM in the mix. I see it as a great plus in a virtual server system.
Point 1 applies only if you have high enough ram to begin with. You'd need a decently high amount of ram if your workflow involves sometimes using lightweight apps and occasionally running multiple heavy programs at the same time. No unused memory is really wasted. You're just being prepared for when you do need it
I think it depends on the intended use case. If you're running a database, it's a good idea since as the index grows it just occupies more RAM. If it's something that has more sporadic usage, maybe just let it use some of the swap space when it spikes.
@@TheMet4lGod thats not enough. also swap space is for unused apps. not good for multitasking
@@RenderingUser the swap is the equivalent of the Windows page file
@@TheMet4lGod it saves to disk. Which is slow
@@RenderingUser yeah. I am aware. I feel like you missed the point of what I'm saying.
Jay, I think you’re a great teacher and I learned so much from you. Thank you for being on the channel.
This is one of the worst channels ever. He rambles too much without saying anything of substance. There are far better channels out there
When it comes to using Linux commands, I think it’s worthwhile to try to remember the general syntax of commands you use frequently, but don’t worry about remembering all the options / flags / arguments/ etc. So long as you have the name of the command, most commands are a simple “command -h” or “command -help” away from getting a list of available options and ways to use it. Yes you can do “man command”, but manual pages are confusing and terrible! 😀 They’re either way to brief with no additional info or WAY too long and convoluted, so at least see if the command has a -h or -help flag first.
Ohhh. That's awesome. I never knew that... Because yes, man pages are useless for me most of the time.
@@brettsjoholm yep. Sometimes -h or -help doesn’t do much better, but often it will give a nicely condensed blurb of the command syntax and what each of the possible flags / arguments do.
Try to use "tldr" command - it is VERY helpful for me, I actually use it every day to see stuff about commands in very quickly way lol
Make notes of your most used commands. As you said, navigating the man pages is just terrible.
tealdeer is also handy for common programs. It's basically a mini man page with the commands and options you're most likely to want to use.
I've been using S.u.S.E. Linux since 1994. One thing I never understood is why you would want to install a new distribution just because you don't like the GUI. Not a fan of gnome? just install KDE. Problem solved. Well, if your distribution doesn't make it easy to install another GUI, then yeah, change the distro.
I think it's such an important point that even the most experienced people do not memorise everything, and thank you for saying it because I think it's often a scary thing for us newer and less experienced people often worry about. It's something I learnt with Photography that even the best photogs do not turn up take one shot and it's perfect. they take thousands of pictures to find their best shot to show the world. I've been watching lots of the "into the terminal" series form Red Hat recently and it's so great seeing one of their most experience people there even makes mistakes and forgets things. It's just down to what you use on a daily basis is different from the next person.
I'd love to learn more about your career, how you transitioned and what you do now. I think it would also be really great to learn what you use for your note taking. Obsidian is such a widely discussed tool these days. I've taken a look at Logseq which looks really interesting but I cannot wrap my head around how to use it effectively and always find myself comparing it to Obsidian which I probably shouldn't.
Do you host things like your notes or do you keep them local on your computer?
2:27 Yes, it most certainly is true. Remember, userland apps like GNOME are not responsible for managing filesystem cache--that is done by the kernel. So any RAM the apps are using is actual regular application RAM, and if they are using a lot more than other comparable apps, then yes, that counts as “bloat”.
Yeah, gonna be directing my students to you. Your calm explanations are sound. Heck, I learn things and I've been around the block a few times. Thanks for putting these out.
I had what I called the big 2 when I started in 2004 - Ubuntu, Fedora and Opensuse. These were what I used either as a primary driver or my backup. Lately I now use Linux Mint. I've learned to use a Home partition to make this easier.
Did you mean to say "Debian" when you said "Ubuntu" ? I ask because the latter didn't exist yet.
@@szr8 TRUE...✌👍
Ubuntu came out in October 2004.
@@szr8Ubuntu came out in October 2004 and I started to use it in early 2005
I would love a series of videos like these. Tips for current Sys Admins that are still learning.
LVM is not a backup solution but many think so because it can include a “cloud” service. Data exists only where it located. In other words if the data in any one place, like an LVM cloud space there is no backup. Unless there a considered plan to make data replicated in more than one place it can be lost. Usually data should exist in three places, the original, a live backup, and an offline backup (one offsite and dead to changes, think of a place that is kept in a safety deposit box.
regarding commands, installing 'tldr' is a massive help
I use flash cards for Linux Commands & specialty terminology. I wanted to remember ... to memorize certain things because they were new to me when I first learned them, and I planned on using that info regularly. For me, repetition was the most effective memory tool for Linux.
Truth in its purest sense
High CPU usage might not be an issue on a server, but if your desktop is idling at, say 80%, you don't really have much overhead for other applications
Dont forget that you dont have to change distros if you want a specific desktop environment. More often than not you have plenty of tutorials on how to swap DE without remaking your whole system or learning the ins and outs of another distro.
Distro is more of an idea of an amalgamation of what a gnu/linux system should look and work like as the foundation. I might be wrong but the distro selection usually revolves around what package manager it uses... While newcommers are usually just looking into the looks and popularity (which isnt wrong at all).
I use arch (btw) because i didnt mind the rolling releases (tbh you can kind of opt out of this ) and the fact that it comes out of the box with basically just the kernel, bootloader and package manager, from there i just pick my "goey" and some apps. You like apt-get more but you wanna use a specific distro that uses something else? Noone is stopping you and its usually not that painful to swap package managers
6:51 the large amount of Linux distros is what made me shift to FreeBSD as my main server OS, the insane number of choices just paralysed me
The main disadvantage of having 100s of distros is that it makes extremely difficult for a 3rd party vendor to supply support for their products. Imagine having to teach your support team Ubuntu, Fedora, OpenSuse, Arch and then all it's dozens of variants. It's insane. The one thing google has done with linux is take it out of open source and streamline it into ONE OS period.
Yes, that's info that I can definitely use. Now I will use git and LVM in ways that I didn't realize. I see them all of the time and thought that only admins and devs needed them. I had forgotten about inodes because I haven't had problems with those for years. But it's a good idea to know how to fix the problem in case it ever comes about. I think that it happened to me a long time ago but I thought that the distribution was buggy. 😏
Never mentioned exactly what inodes are, nor why they dictate if I can save something or not....
Cool thing about lvm, sounds like you were suggesting you can just span your volume across drives. You csn, but people often consider that unwise. What you can do that's cool though is move your data while the server is online. Add new drive to the old one as 'free blocks", then online move the old drive's blocks to the new drive. Then you remove the old drive (easy since it's empty, block-wise) and then expand the partition to use the rest of the new drive's free blocks. Doesn't play nice with btrfs because what does?
For point 7: I don't really care about the distribution, I care about the DE. I'm a desktop user, not a server admin, so the look is more important to me than the underlying stuff.
But the point still stands: if KDE goes away, I'll have to have a hard look at which DEs provide the most Windows-like environment.
yeah, options are good, the facts built my own personal Desktop experience built around sway says a lot
LXQT :) its very dull & boring. modern, and boring. classic desktop paradigm. its fantastic.
I personally like GNOME because I was using Windows since 2001 and i'm just bored of seeing a bar with a menu so the way of Gnome to show apps and all always makes me to use it.
But I don't like some ideas of GNOME and lack of customization even there's extensions that can break the system in every update... I'm using Debian rn
Couple of handy analogies for gamer types: think of Linux like modding a game, example Skyrim.
Linux is your Skyrim and the various bits you add are the mods.
Some are content like a new merchant or gear (apps, package manager, games, etc)
Others are basic frameworks or dependencies, like XP32 for skeletons and ragdolls.
As for the I-nodes that's like inventory systems, everything has weight (kb, mb, gb) while the nodes are your inventory slots, doesn't matter if your carry capacity is 10,000kg if you're out of inventory slots.
Yes I know Skyrim doesn't have limited inventory slots but it is quite crash happy if you collect too many items in a single map location, so the concept is close enough.
I skipped a lot of those issues by jumping straight in to a Debian net install 😅
Number 7: It seems, due to the path Ubuntu is going, Vanilla OS and Pop OS will derivate themselves directly from Debian, no more from Ubuntu. Things are changing.
The TL;DR on inodes is that a directory entry (when you see a file in your file manager, or when running ls, what you're really seeing is a directory entry) points to an inode. An inode then tells Linux where the file's data is on the disk. The number of inodes is (I think) set when you create the filesystem. Each file or directory requires an inode. So no available inodes means you can't create a new file, only allocate more free space to existing files.
A nice thing about inodes, and actually something you get with NTFS (and I think it only got put there because MS wanted to have a minimal Posix subsystem in Windows NT so that they could get government contracts) is that you can have multiple directory entries pointing to the same file (hard links). This means you can efficiently create a clone of a directory hierarchy, which I tend to do before doing any destructive power renaming (e.g. applying a naming scheme to a bunch of media files) in case my Python script has an error which messes things up. With NTFS, hard links are hard to make unless you're running cygwin or git-bash.
Hell yes. This was a great video. I wish you'd cover a lot of things you didn't know at the first. Maybe, organize them according to complexity (or career-period) versus just random topics.
That book plug was so smooth :D
I've been in Linux System Operations for about a dozen years but have been using Linux since 2.2. This is a well done list. I think the only thing I'd add to really introductory material like this is this: du and df. They might (probably will) report different usages. This can be frustrating when tracking down a "disk full" issue. This, really, is far more common than the exhausted inodes issue.
If a file is deleted (but still actually open by a process), df will report that space as still used, while du will not. If this deleted file is very large, the difference could be huge.
lsof (list open files) will help you track down the culprit in this case. Often this is caused by a log rotation that does not restart the application doing the logging.
You spent a bit of time talking about LVM and it's snapshot and backup capabilities, but in my experience, LVM has basically been supplanted by zfs in most instances. for pretty much any system that requires redundancy in file systems (multiple disks / some level of RAID) zfs is significantly superior when it comes to filesystem integrity over stuff like ext4 and btrfs, as well as using hardware raid controllers (which often do the wrong thing in a collision). I would suggest folks learning about linux learn about the zfs tools rather than using the old lvm ways of doing things
Memory from the cache can be provided to any application which needs it.
So basically we should have free memory more, instead of Cache memory as cache memory is not that important. If we have cache memory more, we can either reboot the system to increase the memory and eliminate it from cache, or we can use a command -> [echo 3 > proc/sys/vm/drop_caches]
Correct me, if I'm wrong 👍🏻👍🏻
I also want to add. Notion has been super useful for creating my own cheat sheets. I basically extract all the useful stuff from the man page for the command, and than embellish that with stuff I’ve learned elsewhere.
What is Notion?
@@PoeLemic it’s an app for taking notes. It basically uses an abstracted form of markdown language. You just type backslash followed by what ever your looking to add (ie table, bullet, heading, url, page).
That is an amazing tip! Thank you, I use Notion a lot but never imagined this
I had a terrible weekend trying to fix an error related to disk. And yes it was inodes. You learn about that really fast if the main website is down...
Thanks so much for your Linux Instructional Videos: When I first jumped into Linux 15 years ago, as an alternative to Windows, I just wanted to be a user ... not a script writer, analyst, or administrator. I soon found out that to be a Linux User, is to wear all of those hats. The ability to self-educate was difficult for me as the availability of quality instructional material was difficult to find ... at least at first. I do have a friend who is an experienced Linux Administrator and that has helped over the years ... especially with the catastrophes that can happen. I am glad I found your channel ... it's been a good source of quality information. I still have a long journey ahead. Thanks once again.
Yes, you are not the only one. I'm moving away from WIndows, because I think LInux is more the future. So, I'm trying to educate myself too. And, it feels like a long road too.
i was trained that on a virtual memory machine CPU utilixation should be 30% ... for responsiveness ... over 50% and the system is thrashing .. moving pages and not working and not responding to requests
I’m getting there with the Linux commands. I think my biggest hurdle right now is learning how to effectively use pipes. Also I feel like I’ve barely scratched the surface of things like grep and awk.
I had a block years ago with pipes and grep. The only way I got past that block was through working with them and other commands to redirect data.
grep is useful, but I never bothered with awk. I learned Perl first, and that does everything that awk can do, just as concisely, and more.
The Linux Command line by William Shotts is a great free online resource.
Server CPU can be low if it's waiting for you - like my nextcloud on my raspberry pi ;)
But you are right and this was a major driver in virtualization and later containerization - use the resources to the max with minimal overhead and bottlenecks.
Yes, backups are only really in existence if you tested them and retest on a schedule!
I think that the CPU usage is widely variable. I manage web servers and email servers for customers these days, so unless there's a lot of activity, CPU usage is very low.
But I used to deal with monitoring servers for ISPs, and high loads were common. When pulling data from thousands of devices on a regular basis, the servers were kept very busy.
Love the videos thanks!
There is no need for lvm on virtual servers. 😉 just enlarge existing virtual disk, use "growpart" to expand partition and then grow file system.
I agree about not trying to remember commands you don't frequently use. As long as you can remember a search term or part of the actual command your trying to utilize then you can look it up online or in the man pages, etc.
I'm pretty sure that the inode issue only affects ext volumes ( ext4.....3? ). Apparently its not an issue on other file systems.
nr 7. unfortunately there is not always easy to have a backup distro at hand. in many environments there are other constraints than the distro itself and the app running on it: availability of official/verified images in public clouds, your security agent may only support a limited number of distros, etc. some distros, even if supported by your apps get support last of all (debian :)
in some cases you just have to go wherever the big players are going.
And LVM can do raid , check the doc on how and what kinds of raid . A great way to learn Linux is to read the man pages , as many as you can stand . LVM has great man pages . Also name LVM name conflicts with LLVM which is Low Level Virtual Machine , so some distros are naming LVM , LVM2 now which stands for version 2 but also gives it more of it's own name , since LVM conflicts with Level Virtual Machine.
I've seen in practice that using LVM in a cloud environment is not the best solution. LVM carries the inherent risk of data inconsistency, especially during restoration. For example, we had a case involving 14 hard drives. The restoration process was complicated due to issues in correctly mapping the drives.
Consider using cloud provider’s native snapshot and backup tools instead. These tools are designed to handle such complexities and provide seamless volume management and restore capabilities.
If I can remember a few letters of a command I once used, sometimes I'll drag .bash_history file into a text editor so I can use the search function. If you use a shell other than bash, the file name will be different. Although the Up-arrow will eventually get you to that past command, sometimes I used it two or three years ago. That can mean a lot of clicks.. Olden days .bash_history would get truncated to save resources, but on desktop systems the file can stay alive for years.
you can search a file on the command line. Just use grep on it.
i did not understand first argument. how is it in favor of gnome? if it constantly eats 1gb of ram - it is bloated
i agree with Shawn Lewis below after hearing what you said about commands. I've always felt exactly as you do, but didn't think about this idea of certification tests demanding knowledge of commands mostly likely soon to be forgotten post exam. I totally agree that we have documentation at hand, and quickly on the Net, for all Linux commands. And the one's that we use often are the ones we naturally memorize.
Thank you for the effort put into this video. I would try to minimize the redundant sentences however. For instance the section anout inodes really could have been explained in a much more direct fashion.
I have a shell script that displays a text file in the terminal. It's my personal help system. Plus I also keep a lot of notes whenever I perform tasks in the OS. Like say when I compile some software I create a text file with notes about how I built it. I can always refer to those notes when I want to do that task again.
4:24 you could argue that the purpose of a server is not doing work but providing the possibility that work could be done if it is demanded. So even an idling service provides the service of the possibility and so is not without purpose.
Great video, Jay. I'm always learning things from you. It seems that you have an endless supply of Linux knowledge to share. That's OK, though; 'cause I have a pretty healthy appetite for learning those kinds of things.
Thanks for all of your videos and books. They've been a great help to me. From the Linux Mint Essentials book that got me started with Linux (back in the day); to the Mastering Ubuntu Server books of recent times...
What a long "road" we've traveled since the CP/M days.
...and you've helped to make the trip a great experience for me.
Thanks bunches, Jay.
12:58 writing down commands goes double for anything tricky, obscure, or troubleshooting.
I’ve got a notebook in evernote of these sorts of things, ranging from my favorite parameters for rsync [actually, I should go add that one] through standard settings I want to change after a new install (or check after a major version upgrade) and aliases I want to add to my terminal config, to the magic incantation to patch around Solus Budgie’s inability to detect my screen resolution or maintain it once set.
I started this notebook after a particularly frustrating time configuring openvpn on my server because I (1) hadn’t changed anything, but it stopped working and (2) hadn’t touched it in a couple years and couldn’t remember what I had done. So I wasn’t even sure what the settings _should_ be or where some of the config files needed to be or what responses indicated the vpn was running and funneling all the traffic through it. And when I went back to the resources I had originally consulted, the webpages were either gone or had been updated to give instructions for different software than I was using.
Git is great for everyone - whether youre writing documents, anything from memos, a PhD thesis or novels to hardcore software projects like the Kernel, less formal code projects like data analysis/data science projects or sysadmin scripts.
If you write anything in your job, it's worth learning git. Just learn to write mostly text based (like .txt, Markdown, LaTeX, code files) rather than flooding your repo with large binaries like Word docs
number 2:
I ran a linux server on a vm for the company I work for.
It was always at 100% cpu usage even though it was literally never doing anything.
In my case, this just had to do with how the vm was reporting usage to the hypervisor, which is fine and I understand that. I could have allocated 1000 cores to it and it would have reported 100% usage.
Shrugging it off is wrong though. The vm or server should be sized to the workload and not waste any extra resources. IF it legitimately needs those resources, then by all means allocate them and explain to the client that they are required for fast workflows.
But losses and inaccuracies can occur. You have to stay cognizant of that fact.
That’s not correct. if a vm is reporting 100% usage, there is something wrong and that’s not normal for any hyper visor
Whew!! I’m so glad to know I’m not the only Linux noob using man pages. I learned a few things reading the man ls pages the other day. So much more than I realised. Great video. I’m going get that book too.
Very,very useful for beginners! Respect for video! Thanks 🙏👍
My pleasure 😊
Thanks very much Jay, this was very useful advice. I'm particularly interested in the LVM implementation - will definitely be checking out your content for that. 👍
I use the ZFS filesystem as root filesystem with native encryption on /home and direct boot from UEFI (no grub) with the Gentoo distribution. ZFS has some cool features like deduplication, which is very hardware intensive, and snapshots like described in this video on LVM. Gentoo allows me to not use systemd (which I hate), compiles the code specifically for my CPU model, is a cutting-edge rolling release, and has very good tools for compiling/upgrading the kernel. I am giving this as possible alternatives, but not as advice. Use whatever works best for you.
This was another great video! Interesting things you'd learned, practical uses for familiar tools, and an honest look at how much you need to "know" to run a server. Since 1994, the `apropos` command has been a stalwart friend!
I just do “man -k”.
@@lawrencedoliveiro9104 That's pretty apropos.
Not apt?
Far more informative than I thought this would be. Great tips.
I use Manjaro and Linux Mint on my main PC, Fedora on my laptop for work, Ubuntu on my x86-tablet, Tuxedo OS on my private laptop and Proxmox, OpenSUSE, Debian and Arch on my Servers.
I love have every Linux video has cmatrix running in the background
Thanks for this sir! Side note, I'm living the plan B distro. Always wanted to run Debian and the Red Hat/IBM shenanigans were sour for me. Bummer, Fedora is so nice. C'est la vie. Hello Debian!
There should be an additional point: Linux plays well along with the other „kids in the yard“ like Windows and MacOS, even when they are not vise-versa that friendly against Linux. You can mount and interact with the others quite well, even that they pretend they are the only one’s that exist.
That even goes to the users of those systems! I can‘t count, how many discussions i had in it support like: „Thank god we are using windows and not Linux!“ - „Dude, you know that you wrote that mail on a Linux-terminal, which remote controls a windows system, which is virtualized on a Linux system …. right? And by the way, each system has his advantages and disadvantages just saying“
Thank you. I have picked up some more sys admin tips and for me today was the 'tippet' on disk space. I appreciate your work muchly.
Personally, I would prefer fewer distributions and better built (visual consistency and more controlled packaging) than the current situation. MacOS is an example of something well finished in every detail, although I do not like some things, the first thing is enjoyed and thanks much more.
As we’ve seen with Ubuntu and RedHat, choice is good, monopoly turns to evil.
I always thought that KDE used about the same amount of RAM as windows with just a browser open. Maybe thats actually not the case?
When I was doing my electrical engineering course (seems a lifetime ago). One teacher in particular had not issue with us using our text books during exams. As he put it, you're always going to have access to information in the workplace. Its knowing where to look and interpreting is the test.
@MichaelDustter When you first start it up, maybe. I'm running Fedora ATM and with only a browser with a couple of tabs, it's 4.6GB. Similar to what windows would use. Even in TOP it says about the same.
You got a little bit of a crackle @14:04. Just thought you might wanna know that lol.
2 commands 4 locations to memorize. ls, man, /bin, /sbin, /usr/bin, /usr/sbin. ls /bin, etc... shows programs to run. man which provides the manual for the which program.
When your Server's CPU is almost always idle, it could also mean that it is way oversized and you are paying way too much
Man I’m feeling this right now. Bought a Dell Micro 7000 to run my plex/arr set up and it’s sitting at 2% even under “heavy” use. Could have just used the old i5 I had lying around.
If your CPU is running at 70% consistently that can be good or bad. If no sods using it, you’re in trouble.
Oracle "Unbreakable Linux" was the choice of most of my Centos clients. Red Hat screwed a lot of people by depreciating Centos. I don't really trust Oracle but they will be around for awhile. Like IBM supporting SUSE, and IBM pays for Linux development. (IBM and NCR use SUSE in their POS and ATMs.)
Awesome content! Thanks so much for always delivering!
Is it possible, to some extent, to place the distribution you're using in a GIT (a form of backup)? Either, to reinstall or synchronize parts of the FS (with this differential you can know several aspects of your FS)?
For fast usage of commands I use: tealdeer, navi. They contain fst essential summary of usage.
I Remember back in the amiga era there was a dynamic ramdisk in workbench
As for memorizing commands, there is this thing called “aliases” that are much better than just text files
Aliases are great, but that’s not designed for memorizing things . aliases are for simplifying commands and creating short cuts . I have several videos that cover aliases.
The number one thing I would tell people who are new to Linux: don't let *anyone*, no matter how experienced they are, tell you that there is one "right way" to do something. Whether you choose floating windows or a tiling window manager, desktop or terminal window, "beginner" distro like Mint or compile your system from the ground up with Gentoo - doesn't matter. What works for you and what you like is what you should go with. Period.
Yes, Linux people will have (usually well meaning) opinions about tools and OSes you should check out. Take their advice if you want to, but don't feel obligated to do *anything* because it's your computer. You're the one who will be daily driving it, so do with it what *you* want; not what the Linux Community thinks is right.
Number 3 reminds me of the Unreal Engine UObjects limit.
Your RAM and drive might not be out of space, but the garbage collector can only keep track of a certain number of UObjects before acting up.
Backups don't matter at all, successful restore tests are what matter.
Well... To have a successful restore test you have to have a successful backup
L@@rockymarquiss8327 Lol, that gotcha moment... 🤣 #Pricless
#BravoZulu 🎖️
The only commands to memorize is "apropos". It searches a list with descriptions of all the command for keywords specified.
Instead of memorizing individual commands - is it a good idea to create the series of commands necessary to perform a certain function under a macro, and then just call up the macro as you need it?
I’m a Windows user who used Linux infrequently and is very rusty. I was researching curl and I saw this complex example in Stack Overflow using Lynx, SED and Awk and instead of installing Windows versions decided to install Mint on an old laptop to apply this example I found (which had typos).
I was gratified to connect to the internet and then change to the command line with ct-alt-F1 still connected to the internet to play with curl.
I do remember running Redhat 2.4 which ran beautifully with sound and I then ran a distro upgrade and my anguish when the upgrade broke the sound driver. I was using a weird-ass motherboard sound chip and I ended up buying a common sound card just to run with Linux. SoundBlaster from memory.
The mobo sound chip was Aureal and the company went out of business so the support had stopped.
Follow-up: Creative Labs (maker of SoundBlaster) had sued Aureal over patent infringements (multiple lawsuits) and Aureal won but the legal costs bankrupted the company and Creative Labs acquired the IP at the bankruptcy hearings and Redhat withdrawing the drivers would have been the nail in the coffin for Aureal products.
Sucky yes, it really is a dog eat dog world.
Honestly when I first started using Linux, what kept drawing me in when I wanted to give up was 1. Most of the time, especially on modern hardware, Linux was faster, and that was with the supposed resource heavy GNOME desktop of Ubuntu 18.04 at the time. 2. Was the fact that there was a bunch of options, so if I got tired of Ubuntu, I could try a different OS, and I love as much customization as possible. I understand the necessity for a universal UI, but the fact that I can't even move the taskbar to the left side of my screen anymore in Windows, that's too restrictive in my opinion, and with the gap for gaming getting smaller, I don't have as much of a reason to go back to Windows anymore.
Why I want to work with a Kali linux; two reasons:
1. Possibly dual boot a android OEM
2. It's quality assurance is better to handle
That's why.
Great video Jay, as always! I have never really understood the fixation some Linux distro reviewers have with looking at how much RAM the system is using after first boot, or what it's using during normal operations. I'm like, what is this, 1998?
I've just started dabbling into Linux, so far I've tried Debian, Lubuntu and Mint. So far I like Mint the best. What started me down this path was my brother in law gave me 2 old laptops to mess around with, knowing my affinity for working with old computers, but there were 2 issues out of the gate: first one was that he didn't know the current Windows passwords for either them, and the second one was that even if he did, they were both running Windows 8, 'nuff said. I think they would have supported at least 10, and one of them, maybe even 11, but why would I shell out the money for licensed copies of Windows on computers that aren't my "daily drivers"? That said, I decided to step out of my comfort zone and give Linux a shot with old hardware and nothing to lose, much like I did with DOS and Windows 3.1 back in the 90's when I first started building computers, and/or messing around with old systems that were headed for the trash anyway
As a software engineer, I agree that any resources not in use are being wasted...
In general a very informative list, thank you :)
Used ram is ram that can't be used by other things. My os should use a tiny amount so my games or what have you can use as much as they want.
LVM sounds like it is mostly for servers; not desktops. Am I right? 🤔
Also there should be backup options that are aware of LVM volumes to get it all 👀
gnome use more ram for because it alocates more ram to os but when your trying to open firefox for example it might be faster
For those who find man pages hard to read, there is tldr that shows important commands better explained.