The Level1 Guide to the ULTIMATE Machine Learning Workstation!
Вставка
- Опубліковано 3 чер 2024
- Wendell uses Windows Subsystem for Linux to set up a fire breathing Stable Diffusion monstrosity! Woo!
0:00 - Intro
2:25 - Windows Terminal setup
4:49 - Some useful utilities to consider
9:36 - Setting up WSL2
11:28 - Some cool tricks for WSL
16:16 - Setting up CUDA
20:32 - Oh My ZSH and other considerations
25:40 - Stable Diffusion setup
27:27 - A few final cool things to check out
31:46 - Outro
Forum thread: forum.level1techs.com/t/the-p...
Our older Stable Diffusion Guide (thanks Taylor!): forum.level1techs.com/t/mi25-...
Check out Falcon Northwest here: www.falcon-nw.com/
**********************************
Check us out online at the following places!
bio.link/level1techs
IMPORTANT Any email lacking “level1techs.com” should be ignored and immediately reported to Queries@level1techs.com.
-------------------------------------------------------------------------------------------------------------
Intro and Outro Music By: Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 3.0 License
creativecommons.org/licenses/b... - Наука та технологія
Sweet lord man, I have a 13900K and a 7900xtx sitting here ready for a ML build. I cannot wait to put it together this weekend and jump into this video. AMD challenge accepted.
Giga Guzzler 5000
AMD? come on I have enough problems getting the multitude of ML projects working with CUDA without going for the headache of ROC...
Diversity of hardware is paramount. Do we really want to be solely dependent on one company in this space? We need all the resilience we can get in the AI/ML ecosystem.
Yea I tried forever to get wsl going with ROC but failed. Good luck!
Please tell us how it went/goes. 😊
The little nuggets shared in this video about Dev-Drive, how the CUDA/WSL Drivers interact between the environments, and so much more….I knew some of it, but not in that detail. I don’t know if it’ll be next week or next year, but I know for a fact that the tips shared here will save me weeks of frustration from an easy mistake these tasty nibbles arcane knowledge nodules.
P.S. For the love of all that is, was and/or ever will be holy, please do more videos just like this that either focuses on, or sidetracks to a complete setup/tips n’ tricks for configuring/using Oh My ZSH.
Wow. Just wow. There are some absolute gems in this guide. This is worth going through just to find those little things you've been missing out on.
4:05 - I believe this, technically, may fall under the MS's Embrace, Extend, Extinguish strategy.
Couple of months ago some called me paranoid for saying the same thing. He told me that was 20 years ago.
It begins. IBM bought RedHat , MS buys|renames Ubuntu.
I think they eventually want to run linux and be a I/P/SaaS company that their Windows Linux runs on. Maintaining Windows code is too expensive.
There are no sheep that safely graze in Microsoft's pastures.
I remember when they obliterated Quarterdeck Systems who developed the QEMM386 memory manager for MS-DOS.
My theory is that Linux becomes the hypervisor and Windows becomes the first-class VM. Look at the work on their own CBL-Mariner distro.
Fantastic video! Over my head but I’m definitely interested and trying to learn more about this. Thank you!
I'm a data scientist that has been going back and forth between Linux and Windows, and WSL is a great medium! Great guide! Here are some things I prefer: fish shell (& starship prompt), mambaforge instead of anaconda, VS code for everything (including notebooks), and finally custom openssh server on WSL to allow connections from any machine. Also keeping my WSL instance on a separate SSD, but looking forward to DevDrive!
1:23 It’s okay, we can handle you just saying current AMD/Radeon GPUs aren’t that great for machine learning.
Wendell has made much more explicit comments about the AMD vs Nvidia situation for ML, which boil down to Nvidia having better software support in terms of the user experience but AMD being far better than most people give them credit for. Notably, how some high end supercomputer projects are using AMD Instinct GPUs. Relevant to the consumer space is that for those willing to put up with AMD's steadily improving ROCm suite the price floor for high VRAM cards is much lower than Nvidia since Nvidia deliberately puts relatively small amounts of VRAM on most of their consumer GPUs
Generally correct, but sucks that AMD currently only plans to bring ROCm to the W7900 and 7900XTX and only on Linux while basically all NVIDIA GPUs can use CUDA.
@@bosstowndynamics5488 Yeah, AMDs hardware is great, but their software sucks. They need to improve by huge leaps and bounds.
I love these vids that tie in with software dev. Thx
Thanks. It reminded me when I installed WSL the first time on a older PC, all the tools and later Cuda - Will do this on my TR 11 machine when I feel ok.
Man you're melting away weight, good fucking job Wendell
Lots of valuable information in here, thanks.
Everything you mentioned works on W10 afaik
If your using WSL and have a non Nvidia GPU (AMD, INTEL or even a older GPU even IGPU) you could still use WindowsML (DirectML) for AI GPU acceleration, my own test shows around 80% of the performance of cuda (same machine using WSL-CUDA, and WSL-DML)
Sentencepiece finally released wheels for 3.11 a while ago, should be fine to finally upgrade from 3.10! Great video as always Wendel.
Need something "lower level" but great content and why I sub. Thank you. What's the min hardware requirements to have fun with macbine learning?
Just the video I needed!
In the recent latent space episode George Hotz loosely spec'd out the $15,000 tiny box ML computer. Server motherboard and 6+ high-end GPUs skipping nvlink because it would cost too much. GPUs underclocked to run on a single 15amp circuit. Would be an interesting project to either buy or try to build. Hotz said it wasn't easy to do well.
A1111 is a great tool (Running on Windows local machine atm), I'm not a linux guy, but I'm definitely going to save this for future use.
Ah so informative and relaxing
You should show people how to Shrink the WLS2 storage image. I had one that sucked up 500GB because it wouldn't reclaim empty space.
First time I've ever heard about WSL and it seems interesting. I've missed having CoLinux ever since I switched to 64bit.
Powertoys isn't installed by default because its a preview / experimental playground for features which may be pulled in to main Windows. Like Fancy Zones inspired the integrated snapping in Windows 11.
If it was installed by default, it would be a lot slower and less experimental.
I think PowerToys are not installed is pretty much why advanced features are not built into windows by default - to leave the field open for third party software vendors and software industry.
1 minute into this and your already over my head.
"Oh cool, an AI/ML workstation vid"
> WSL
"Nevermind"
Thank you!! Hopefully this covers multi-GPU? I though WSL had an issue with that though...
Would love to hear your configs for colors, vim and tmux extensions, etc you find useful for those of us that live all day in a mobaxterm ssh session to a bastion/management server! I’m a fan of true color solarized dark myself…
should make a full power toys guide
I'd watch that
As another fellow MobaXterm user who just really likes Moba, I'm glad you mentioned it! Got my whole team on it now. Who needs WinSCP and Putty??? Not I!
Yes; definitely will enjoy a lower level into this
Funny, just went through setting up my 5800x3d for a dask use-case in wsl 2. Also, didn’t see any reference to docker for windows using wsl going forward. Might be worth mentioning, I think a lot of people would see this as a reason to use wsl instead of dual-booting or VM’s, ig all they require is linux compatabilty, which was my main concern regarding potentially wasting performance going the VM approach or “wasting” storage by maintaining to OS’ in a dual-boot config. Now that I see that performance in wsl 2.0 is bo concern or good as expected I can have a gaming pc and a comfortable shell environment to do my work on.
14:00 what is the googly eyes program ?
Love pengwin wsl for this reason personally
About what kind of videos and topics to cover that I would like to see in the future.
I personally would like to go full on Linux however some of the software I work with are Windows only; Also in my case I have Radeon card and want to develop and also run some AI software like Stable diffusion (ROCm) that doesn't support Windows, so Linux only with ROCm.
Now while WSL sounds like a good middle ground, I would prefer to go Linux as much as possible, having it as the main OS; And have whatever I must have on Windows be translated/emulated on something like Wine or go full virtualized Windows environment on that Linux host.
However that has its own issues with hardware like sharing accelerator resources (GPUs etc). Maybe there is a thing we can do here? Is there some madness that Wendell could bring to light? What is the state of Looking glass and other similar solutions?
So basically I want to migrate over to Linux by starting using it as host OS and *slowly* switch to Linux alternative software one by one as they become available and mature, I bet that some of my software will never be Linux native but hey it's the same case the other way round :)
Wine + Bottles and then [MS Vms on KVM]. I'm doing the same things. The gaming community are making tools to go to linux as the host, so look at that.
No way, I'm running MS as the hypervisor or host on my Epyc Machines. I like freebsd + zfs and other stuff and MS ain't going play well with that.
I just found out about Bottles which is a gui front-end for creating wine environments and moving them around. Look for stuff like that and there competitors or siblings. It seems everybody still prefers [Team Green for the GPUs]|nvidia.
I use podman instead of docker but now I'm looking for other options since IBM closed the RedHat source code; it makes me feel weird about it.
This video is really for MS developers trying to do a little linux. I'm OS guy trying to do alot of development for Linux, Freebsd, MS, RaspberryPi, altera, xilinx, arm, android, ios: pretty much everything. I have to do it that way.
when will the intel xeon max 9480 actually be released?
I am plan to build a compact workstation in October. And for some bad reason, I plan to keep the A750 I won from Intel as my gaming GPU in that build. Doesn't mean I won't put a second accelerator card in there, perhaps I can get the Intel GPU Max 1100 watercooled once that's available. Already been denied Gaudi2 as that won't be a PCIe card.
I am extremely unhappy with commitment to software support like IPEX. They have docs for Windows Native but tell us it's unsupported (I haven't tried building it myself) and they tell you to use WSL instead. Which I can't use as Hyper-V is blocked by the anti cheat software I depend on to play competitive events.
OpenVINO works, for inference. It's only a few more lines and that's fine. One issue is the addition data, but release 2023.0 is trying to skip the onnx steps and go directly from pytroch to openVINO IR. Sadly the files are larger tho.
And inference for CausalLM is currently broken or extremely slow, and that's what I am working on.
How do you achieve memory pooling?
More videos like this plz
welcome to the club...lol the quadro experience is so much better than gforce experience ...I only have 4608 cuda cores for now
The 4090 has 16384 cudo cores... Sure the quadro will be a better experience?
Every Machine Learning Engineer at my company just SSHs into the GPU Machines(Linux based). It's mostly the Data Analysts that prefer to work on WSL or vanilla Windows. WSL has low overhead but it still exists.
WSL is a good option for a home gaming rig hobbyists.
Yeah I agree bare metal is still the best but when you're network admin forces you to use windows you don't have a lot of options. Luckily I've got a standalone NVME with Linux installed on there
@@WhiteError37 nice.
I test things in wsl mostly.
Then resetting it is just a few click in windows.
Or I use OpenFOAM in wsl.
the explorer.exe . command on ubuntu launches my documents (directory on windows) how can that be remapped?
Likewise the visual studio code installed on powershell via winget, when typing "code ." in ubuntu doesn't get recognized.
Pwd from Linux to print working directory, explorer.exe with the trialing dot should launch explorer I. The current folder on wsl. Possibly your registry on windows is a bit messed up
maybe i missed the reason for using windows for this task, and to each their own, but why would i want to choose win for dev/ml instead of linux?
Isn’t cuda 12.x not compatible with PyTorch yet?
Wendell as someone who's worked with alot of software/gaming startups hear in the Raleigh/Durham Triangle I'd love to see what could be done with one of AMD's W7900 workstation gpu's
up on patreon now. I used an mi210 but as long as the gpu is on the rocm list, it'd work the same.
It would be great to have a Windows “Power Toys Edition”, just the basic operating system with everything including RDMA support enabled but without the “Fisher Price” BS on top.
It would be great to have Microsoft actually make a competent non-invasive operating system, but we can't have that, so here we go with MX Linux then.
glad to know im not missing much over here in pycharm land
Got my mind racing thinking about my 7950X-3D / 7900XTX system I just built sitting here just playing PinballFX
I'm dying for a budget dual GPU build. Used 3090s are on firesale.
Another impressive and content-rich effort. Thanks Wendell.
I look at your finished configuration and ask myself: How do you do backup/restore of that mess?
When I need to backup Windows machines, I just boot with Clonezilla Live and point it to my NAS as a backup target. Not sure how the WSL2 file system looks from the Windows side...
Oh yes, window snapping. You mean like several Linux window managers have provided for YEARS. Yeah, that. Good job catching up with the rest of the world, Microsoft - again.
disk2vhd is an easy shotgun blast approach, that's also free :) for full machine imaging. dont need to do that weekly or even monthly is fine, then snapshot the data elsewhere.
@@Level1Techs Thanks Wendell
I'm a an AI dev and my company forces me to use Windows.
And I think that is probably the only reason to use Windows for ML.
The idea behind the setup you show here is the same that I use: Offload as much as possible to Linux (or Docker containers) and let Windows touch as little as possible.
Some ML packages can't even be installed on Windows!
Why go through all the pain of Windows if you end up running everything on Linux anyway. If Windows is your base OS you have to go through WSL, have to deal with powershell and bash, have to deal with Docker Desktop eating your RAM (open issue for over a year. Re-installing might help), etc.
Unless of course the company you work for forces you to use Windows...
Do another with bare Linux
It would be cool if somebody created a choose your own adventure ML program that not only did a Choose your own Adventure story but also created art work such as drawing maps as your characters progressed through the story. Something you could give to elderly in retirement homes to keep themselves entertained while exercising their brains like that CS:GO team of elderly players.
I'm running Linux natively on an Arc A750, I'll have to look at grabbing OpenVino notebooks for my system and see how well that works.
About your saying most Linux guides assume Ubuntu. In my experience, that does not apply to enterprise-leaning software, like some LDAP servers and OAuth server software. A lot of those types of software assume something in the Red Hat family.
I just made a dual CPU workstation using 2* 2687Wv2 128GB ram for a few 100 quid. Fits in a fractal north and is pretty quiet. Just hunting for something better than a 3070 for the GPU.
I got Stable Diffusion A1111 working with Nvidia (duh), AMD, and Intel (A770) using a PopOS host.
Mobaxterm is amaaazzzing.
I have moved from linux to wsl for 2 years and it’s been very good. Moving from dual boot to one os p, really simplify a lot of things, all my personal apps can update while I’m working (steam, seeding, etc). The only hassle is yubikey on wsl is a bit finicky, but development experience on wsl is just as good in my experience.
We are stil waiting on a MegaDesk update! Don't you think we have forgot!
"SysInternals from Mark Russinovich, who Microsoft hired." I have never seen someone so reluctant to give credit to Microsoft. He is the CTO of all of Microsoft Azure.
stolen valor: en.wikipedia.org/wiki/Sysinternals Sysinternals did not originate from microsoft, and was written by mark before microsoft. and microsoft really had an hilarious internal struggle about whether or not to hire him. but sure, microsoft can get credit for all of that. he's cto of azure /now/ but that wasn't the context of my comment ffs
@@wendelltron Respect to you Wendell, really enjoyed the July 4th show allot.
I just spec'ed out a 39k USD machine learning workstation on the Falcon configurator. I need way more moneys
I think I could count on two hands the number of YTers who really know what
they are actually doing technically with modern computer hardware, after L1 I'm down to 7 digits.
Hotz claimed to have one in productions ?
I don't think there is any data scientist out there that does not know about jupyter notebooks yet :D
If you work in pycharm for everything (because it's obviously so much better) why do you go through the steps installing vs code?
Weak bait
@@WhiteError37 no I'm genuinely asking because maybe there are dependencies or something else that gets added that is important. Just like how he mentions you do VS before cuda because cuda detects it and automatically ads certain things, that are a massive pain to put in when you do things in the reverse order.
Need one CFD/ Engineering build
Problem with WSL is that non-official distros won't get HW accel working correctly, like EndeavousOSWSL2.
I'm very commited to use ArchLinux because 99.99% of packages of the world are already pre-built and at their latest version with their respective latest features already added.
Or maybe it was bc i was using the latest dev insider preview for W11...
Or maybe bc i was using an Intel ARC A770 16Gb along with WSABuilds github repo prebuilds all together at the same time...
Or maybe bc i already was virtualizing W11 passingthrough the A770 on a Linux host with nested virtualization enabled and working all together at the same time...
Idk. I'll test that on a baremetal Alder Lake laptop and comment the results.
BTW, i prefer chocolatey for the Win pacman.😊
Arch Linux on WSL is fun. (Although I prefer Linux running on bare metal for freedom, privacy and security.)
After a year of not using Windows at all, I was pleasantly surprised with winget. It even works for applications which were not installed via winget.
Personally I prefer System Informer (previously known as Process Hacker) over Process Explorer, even if Windows Defender doesn't like it too much because the devs are yet to release a signed stable release and Microsoft doesn't like any task managers not sanctioned by them (System Informer is licensed under MIT btw), and out of all keyboard launchers I find Keypirinha to be the fastest, snappiest and most robust of them all.
Reject React, return to monke. (I pretty much made my homepage as simple as possible with Hugo, no javascript whatsoever.)
Package managers have been a phwcking nightmare forever.
When they work, great. When they break, you're phwcked.
And then let me introduce you to a nightmare called.. dependencies.... and breaking them.
I basically only use WSL to ssh into my real Linux machine lol
glorified putty replacement
I still use cygwin
I'm pretty sure Windows 10/11 come with an SSH client preinstalled.
Use Terminal for that…
Wat
What I want to know is if I can toss a 24gb Vram Nvidia Tesla P40 in my dell poweredge sever to host LLMs in.
I’ll take any hand I can get zsh pls
FancyWM look it up for your Windows computer. Auto tiles for Windows.
Whats the config, lol?
Watched this for curiosity, but I am wondering: Why go to Windows in the first place for ML? I have worked on ML projects for over 10 years now, and I have been running Linux exclusively for that. Maybe I am missing out on something?
There are a LOT of people that Just Want Windows. This might help them get to where we are by showing them that linux actually is pretty awesome.
I have tested and benchmarked WSL with tensor flow and found 10%-40% reduction in performance compared to Linux on bare metal.
First thing I thought of when seeing the intro. “Y tho”
Just dual boot from separate drives if you need windows, with this hardware you’ve got the money.
Why not just use Linux directly though?
It surprises me how long it took windows to catch up to 3rd party software like DualMonitorTools (DMT).
This
I would rather run AI, either image generation or text generation (chat bot) on my AMD 7900xtx. We really need that ROCm for windows to release....
O boy. I'll keep my sanity and go bare metal Linux
You should have gotten a Threadripper pro 64 core. Or get the Zen4 Threadripper with 96 cores at the end of the year.
I can’t tell you how many times I’ve solved a problem by finding a relevant GitHub comment while looking for _something else_ hehe
I found OhMyZsh to get unbearably slow in certain Git repos. No idea why, and I didn't feel like spending endless hours figuring out the intricacies of the interactions. Sooo, after a mere handful of hours, I started checking out alternatives, and landed on Zim. Worked out very nicely for me, and maybe for others too, should there be strange interactions with special directories. 👍
Hi L1, I've been noticing a very slight color fluctuation in these video's, Wendell's skin reddens ever so slightly about once a second. not distractingly so, but noticeable seen it in a couple vids now. It could be on my end, but i've not seen it anywhere else this far. Maybe look into it, no rush. And if you can also see it happening? Like or comment on this comment i guess? Haha sounds like "like" bait 🤣 anyway, baii 😙
um....proxmox franken dev kit? 0.o
powertoys is not installed by default because windows is used in corporate settings all across the world, they dont want employees customizing computers because it can be counter productive.
Oooweee did I do some things wrong 😂. Better get hacking
Just embrace Windows
Me: NOOOOOOO!
Hi sir.. I have a problem with AMD card. When I am playing games or when i start the game the black screen appears suddenly. Everytime this happens i have to restart my pc then everytime i also have to reinstall my AMD software... This happens reraly. I don't know what is the problem... Please help me to fix it.. I am using AMD RX560 card. Please tell me how can I fix this problem?
This is a FANTASTIC video.
I don't HAVE a dev platform (yet).
(I don't do ANY sort of development work.)
But my almost 8 year old tiny human #1, likely won't be able to get away with it like I have, so far, been able to get away with it.
So, I figure that if I can at least dip my toes into, at least a little bit now, with an Intel 8th NUC and a 3090 eGPU, it might, potentially set it up where both tiny humans will be able to learn this stuff and do WAYYYY better than me, in regards to dev work.
I have VERY little exposure to this stuff (via work, and the guy that used to train me - he's no longer at the company), so it would be nice to be able to do something other than MATLAB.
re: killing SSDs with bajillion tiny files
Yep!!!
Heck, even if you used a SSD as a swap file, you will be killing your SSD in very little time.
I've already killed 7 SSDs this way.
As such, again, I tend to think of "SSDs are the brake pads of super/hypercars. It's a wear item. The faster they go, the faster they wear out." And it's not IF the SSDs will wear out, but WHEN.
Что с Венделом? Надеюсь это не рак?
Even if they made Windows orders of magnitude better than Linux with WSL, I still just can't bring myself to using it, because I just don't trust it.
The 1 year delay has made these xeon's less compelling with Emerald rapids slated for the end of this year ... however I have little faith they'll release on-time which is a bummer ... I really wanted to like intels workstations
We need AMD to release the new ThreadRippers already
RUNBSD
What's the repo at 22:30? Gggl is fkin useless. So is the video description. Again!
Will Wendell discover Kubernetes before the end of 2023? 🍿🍿🍿
without a doubt Linux is the better development environment. I call Linux a developer-first OS. Mac is (media-)consumer first and Windows 11 can't decide what it is.