Soo.. you really want him to sleep over and do all the work while you cringe with excitement. Just let me know when you're ready to invite me. Bwhaha haha
Hi nice video I have 6 hp workstations I’ve been wanting to build a beowulf cluster (it just sounds so cool). If thus works then I’m wanting to build a huge cluster. How far up can a system like this go??
Hello. We set up cluster using this video. However, you just showed a task working on each computer individually but we need to give a task and seperate it to 10 subtask and give all the nodes then all of them need to work parallel. How can we do that, Sometimes we also need to divide a large tasks that can't dividable, in that case can we spead up using all of the rams together
A cheap way to do this is to use about 50 Raspberry Pie Zeros in a cluster with all of the microprocessors slightly over clocked in a nice safe range and then connect massive heat sinks and multiple cooling fans to keep everything at a nice safe temperature. Once you've got the hardware set up than use the software like this guy is showing us and you will have a decent super computer that takes up a fraction of the space for a regular computer system cluster for a fraction of the cost of desktop computer clusters.
Also what this guy doesn't mention is that you can use many different types of computers in the cluster. If you can get your hands on a bunch of computers that are being thrown out or on a bunch of computers that are damaged merchandise because of the casing being dented or hardware components being broken but the CPU and mother board is intact than you can use them to make a super cluster of very ugly computers. You'll be using a ton of electricity and your electric bill will be through the roof but you can do it!!!
AI based video games, EEG/TMS devices for BCI interface, robotics and cryptocurrency mining operation are all good reasons to have a really good super computer cluster.
Hey, would this technique be good for building a home cluster computer to run neural network processes on? I'm an artist/designer with a hobbyist interest in neural networks, particularly their use in art, and I was tired of using my home desktop, and a friend had a bunch of old bitcoin mining rigs lying around.... so I thought, what the heck?! Might be worth it, just to try. But I'm not a coder or a computer science person, despite being fairly technically competent for an artist. Is this tutorial suitable for what I'm wanting to do?
Dave, is there some way to do this on windows platform ? Or, after creating all of this, adding more noes, can one create on the top, a virtual machine that runs windows, so we can run for instance, MS compatible games with the power of this supercomputer ?
it's part of setting up parallelisation for linux cluster. The pc can be anything as long as it has openssh. Most supercomputers have optimised networking so the cpus work together efficiently.
tutorialzvid84 do you know how to actually go about doing that? Ive been working on doing this at my school and they're willing to dump $5k on it. I just want to know how to have multiple computers operate as an individual (at least for when it comes to computing)
tutorialzvid84 the setup I've got planned is 4 towers each one has 8 4.4ghz cores 16 gb ram 8 gb of vram full liquid cooling 4 tb from 2 hard drives using raid0 (the case has 11 drive bays though for later)
sorry bud, these types of distributed clusters are for specialized applications and behave nothing like a single computer, the closest you will get however is multiple identical computers running citrix xenserver in a group
I'm not looking to make it be like your standard gaming computer, I'm looking to make this into a server for hosting websites and potentially to feed information to a few raspberry Pis things like that, I've seen a few different methods to have a shared desktop image that kind of thing but I'm just hoping to have them work very well together without having to always input a specific program to run and run it through mpich, I don't want to be limited to a Beowulf cluster, right now I'm on the stage of benchmarking the cluster I have and then tuning it to make it as efficient as possible and then also be able to find a standard way of determining what size operation is necessary to make the returns worth it
Hey Dave, how could I use a couple/few old PC's as a PLEX media server? My current PC is old server hardware, but the CPU is waaay too slow to keep up with MCEBuddy (it deletes TV commercials and compresses the MPEG2 into MP4 files). The queue on MCEB is currently like 600+ files and it only does about 6 per night (8 hours of working). The reason I don't run MCEB 24/7 (even though it technically is, but "Paused") is that it takes every ounce of CPU power needed to do what little it gets done each night. I'd like to have a much faster "server" but that would cost even more $ which I don't have. However, I think I may have a couple/few old Intel DuoCore PCs laying around that might be able to do this parallel processing that you're talking about. Is that possible? Also, if it is, then I think I'd be able to actively run MCEB 24/7 thus catching up on all the files in the queue.
clusters can be helpful to even a novice user. For instance adding spare machines to a Boinc Project, or using them for network rendering in 3DS Max projects. Last but not least clusters for crypto currency.
I created a passwordless ssh key. I can access the slave machine without a password, but when I enter the following command mpiexec -n 30 -f /home/cluster/mpich/hosts ./cpi I get this error --> bash: /home/cluster/mpich/bin/hydra_pmi_proxy: No such file or directory but when the command like --> ls -al /home/cluster/mpich/bin/hydra_pmi_proxy is entered. it turned out like this --> -rwxr-xr-x 1 cluster cluster 2494144 Sep 17 14:51 /home/cluster/mpich/bin/hydra_pmi_proxy so the file exists.
I get the same weird urges and have ended up with a million spare computer parts and a few fully built tanks and many"projects". 20 feet to the left and the car version of this story is located right outside the garage.
How do we run normal software like bitcoin miners or video games or just intensive programs? Does the software we run have to be specially coded for this?
You are the most literate tech guy on UA-cam. I watch a ton of videos. You must have been doing this a while and possibly some college classes in IT and Software. lmao. Wow.
i followed this tutorial but im stuck at the ssh-keygen part. i have computers i made worker1 and worker0. when i use ssh-copy-id worker1 it tell me to enter the password for worker1. i do that but then i get a messge saying "permission denied, please try again". i am putting in the proper password but i do not know why it is not working. can anyone please help me?
Thanks, however, this is parallel workers as both workers did the same job, and didn't get better timing or ant exceeding in processing speed, how can you make workers share the load of processing not to do the same process in parallel?
Was thinking of working a bunch cell phones together but seems it really wouldn't be any cheaper to build the ultimate 220 s10 phones together with ethernet cables lol yeah pipe dreams
your expalnation is simple and good , I already have NIS and NFS on my ubuntu clent server model, server is IBM , can you plz tell me how can i convert it to super computer
The default question in this type of tutorial is: can you just run any Linux application with this method, or it has to be one specifically designed for that??. For instance; if I want to run Jack an some audio plugins, should I just install the the apps in both computers, and just let the first one to do the job??. I have not seen anyone explaining this..
What I meant is that on the server side it is a killer application, but on the regular user side there is very little, since there are not many programs that run on Linux. A lot of the hardware is not compatible with Linux, and now even the motherboards are not compatible with Linux when they only run UEFI bios. On the enterprise, most of the work is typed on Word, Excel, or Power Point, but when it comes to movie or picture editing, or when it freezes (it does a lot), there is very little that can be done. At home, every time you need to install an application, apt-get command is a little complicated to use: the so many security steps, passwords, and strange terminology make it hard to use. I love Libre Office, and for the most partt, it does the job quite well, but I would ,like a little more standardization so more hardware and software can be produced for Linux. BTW, the taskbar on the left hand side drives me nuts.
Linux is super powerful. All high end systems run Linux. Windows is just for gaming. Linux is for supercomputing, particle colliders and pretty much every server on the web. You want to learn res computing? use Linux. Want to play shooting games? windows
If I built a ,10 raspberry pi2 cluster,will it works on it too? Sorry but how fast is supercomputer compared to the latest Mac. computer? Sorry I am a computer Newbie and a nerd.
"do you ever just wake up and say, ' I want to make a supercomputer'"
Literally every single flippn day. I kid you not.
Yeah. I thought to myself, "Yes. Yes; sometimes I do."
same, try find a deal at electronic scrap recycling plant on computer hardware maybe get creative for cheap
XD
If your serious, contact me
@@davidprock904 Hi
Now every time I will wake up, I'll say: "What a nice day to play minesweeper at 4k 60 fps in my lovely supercomputer."
a 4000 by 4000 grid with a time limit
Or you could wake up and play crysis on your supercomputer @480p, 30fps.
@@mylesbell2398 why does this not have more likes
I followed halfway through the tutorial, then I asked myself: why am I even doing this?
same here
lol same
too complicated !
You made it longer than me...lol
me either, but I gOt until the end and gave a like, maybe in the future I will make a super mother f computer with raspberries, regards everybody
"Do you ever wake up and say, '"Today I want to build a supercomputer'"
Yes. Why do you think I'm here?
same
Lol same
"do you ever wake up and say 'today i want to build a supercomputer'"
yes.
yes, all the time. i'm not joking.
Damn.. I can't believe my 14 year old self used to watch this years ago and now I'm actually studying Computer Science in uni. It feels so nostalgic.
Things are progressing well I see.
Best of luck and hope you stick with it, because your bills will.
-Shows AMD CPU in start of the vid-
That's not a damn cpu it's a damn Lava Generator
no thats not a lava generator its clearly a plasma generator
Luckaly its better today
Nah, it’s a legit freezer.
UGH DON'T FUCKING HOLD THE CPU LIKE THAT AAAAAAAAAAH MY INNER NERD SCREAMS
+EpicLPer Hahaha, its time is long done.
+EpicLPer I feel your pain, but its an Athlon 64. If it isn't a paperweight already it is time for it to be.
He can easily bend the pins on the back and leave nasty human finger-grease on the pins that go inside the motherboard.
ICs are also generally very static sensitive. "It's as though millions of micro-transistors screamed out...and were suddenly silenced!"
Jack Outside the Box unless it LGA775 or later. It okay to hold like that on Intel
wait, I actually did wake up and think that
That's pretty cool, but I would love to see this done IRL and see benchmarks.
Soo.. you really want him to sleep over and do all the work while you cringe with excitement. Just let me know when you're ready to invite me. Bwhaha haha
If all your videos are at this standard, you got yourself another viewer, well done!
Links have been updated!
looked at the thumbnail and said to myself "well, I'm on the right track"
I really enojoyed how easy you are to follow
I really enjoyed this - Thank You Sir!
Thanks Dave for the introduction. I, now, have time to explore this subject so I am glad to have stumbled onto your video.
Very nice! I won't use this personally, but I enjoy watching you!
Hi nice video I have 6 hp workstations I’ve been wanting to build a beowulf cluster (it just sounds so cool). If thus works then I’m wanting to build a huge cluster. How far up can a system like this go??
I just bought a PC and wrote "super" on the case. Done! lol.
Thanks! Nice introducton to mpich. I have to build a Beowulf cluster at work and this is helpful.
D-did you hold t-the C-CP-P-P-PU-U l-l-like that?
lmao at the piece of thread/hair sticking out the side of the CPU 😂
Oscar Jacobo lol. I had to go back a look at it. Saw it.
Damn threads, they keep going in! No wonder my intel cpu is slower, amd stole my threads!
please help I got an error that said command mpi not found at the last part of the tutorial
Hello. We set up cluster using this video. However, you just showed a task working on each computer individually but we need to give a task and seperate it to 10 subtask and give all the nodes then all of them need to work parallel. How can we do that, Sometimes we also need to divide a large tasks that can't dividable, in that case can we spead up using all of the rams together
A cheap way to do this is to use about 50 Raspberry Pie Zeros in a cluster with all of the microprocessors slightly over clocked in a nice safe range and then connect massive heat sinks and multiple cooling fans to keep everything at a nice safe temperature. Once you've got the hardware set up than use the software like this guy is showing us and you will have a decent super computer that takes up a fraction of the space for a regular computer system cluster for a fraction of the cost of desktop computer clusters.
Also be sure to play this video as you're following along. Like baking a cake!!! Do it as you can get the instructions!!!
Also what this guy doesn't mention is that you can use many different types of computers in the cluster. If you can get your hands on a bunch of computers that are being thrown out or on a bunch of computers that are damaged merchandise because of the casing being dented or hardware components being broken but the CPU and mother board is intact than you can use them to make a super cluster of very ugly computers. You'll be using a ton of electricity and your electric bill will be through the roof but you can do it!!!
AI based video games, EEG/TMS devices for BCI interface, robotics and cryptocurrency mining operation are all good reasons to have a really good super computer cluster.
www.amazon.com/Raspberry-Pi-Zero-Wireless-model/dp/B06XFZC3BX/ref=sr_1_6?ie=UTF8&qid=1537683252&sr=8-6&keywords=raspberry+pi+zero
Why on earth would somebody do that? You're better off just getting the latest Intel cpu
Hey, would this technique be good for building a home cluster computer to run neural network processes on? I'm an artist/designer with a hobbyist interest in neural networks, particularly their use in art, and I was tired of using my home desktop, and a friend had a bunch of old bitcoin mining rigs lying around.... so I thought, what the heck?! Might be worth it, just to try.
But I'm not a coder or a computer science person, despite being fairly technically competent for an artist. Is this tutorial suitable for what I'm wanting to do?
Got a stack of old 6 and 8 core opteron servers at work. That should shave a couple minutes off my handbreak jobs. Thanks!
Wally West teaching us how to build a super computer.
I love that you said you were using "Trusty" but you're not. You're using 15.10 - Willy Werewolf.
I noticed that too.
this video is so cool man, thanks for the video!!!!!
This was very interesting, but copying files to both computers seems like a burden itself. Can you guide us how to do it on shared location?
This massively underrated
this is great work and great representation....keep it up!!
As I imagined, building a supercomputer is SUPERCOMPLICATED, :)))
No... Just no...
Dave, is there some way to do this on windows platform ? Or, after creating all of this, adding more noes, can one create on the top, a virtual machine that runs windows, so we can run for instance, MS compatible games with the power of this supercomputer ?
Is this part of a larger series? Because this has literally nothing to do with building a computer.
it's part of setting up parallelisation for linux cluster. The pc can be anything as long as it has openssh. Most supercomputers have optimised networking so the cpus work together efficiently.
tutorialzvid84 do you know how to actually go about doing that? Ive been working on doing this at my school and they're willing to dump $5k on it. I just want to know how to have multiple computers operate as an individual (at least for when it comes to computing)
tutorialzvid84 the setup I've got planned is 4 towers each one has 8 4.4ghz cores 16 gb ram 8 gb of vram full liquid cooling 4 tb from 2 hard drives using raid0 (the case has 11 drive bays though for later)
sorry bud, these types of distributed clusters are for specialized applications and behave nothing like a single computer, the closest you will get however is multiple identical computers running citrix xenserver in a group
I'm not looking to make it be like your standard gaming computer, I'm looking to make this into a server for hosting websites and potentially to feed information to a few raspberry Pis things like that, I've seen a few different methods to have a shared desktop image that kind of thing but I'm just hoping to have them work very well together without having to always input a specific program to run and run it through mpich, I don't want to be limited to a Beowulf cluster, right now I'm on the stage of benchmarking the cluster I have and then tuning it to make it as efficient as possible and then also be able to find a standard way of determining what size operation is necessary to make the returns worth it
i have my nerdy dreams of making a super computer
Sweet video buddy!
+Denzel Garcia Thanks!
Dave Bennett can this be done with windows?
Yes, but linux is preferred because 1. it's free and 2. There are versions of linux specifically made for it.
Matthew Sorensen ok. do all the pcs need to be the same? or can I have different pc's with different cpu's
They don't need to be the same but it would be better that way.
Hey Dave, how could I use a couple/few old PC's as a PLEX media server? My current PC is old server hardware, but the CPU is waaay too slow to keep up with MCEBuddy (it deletes TV commercials and compresses the MPEG2 into MP4 files). The queue on MCEB is currently like 600+ files and it only does about 6 per night (8 hours of working). The reason I don't run MCEB 24/7 (even though it technically is, but "Paused") is that it takes every ounce of CPU power needed to do what little it gets done each night. I'd like to have a much faster "server" but that would cost even more $ which I don't have. However, I think I may have a couple/few old Intel DuoCore PCs laying around that might be able to do this parallel processing that you're talking about.
Is that possible? Also, if it is, then I think I'd be able to actively run MCEB 24/7 thus catching up on all the files in the queue.
clusters can be helpful to even a novice user. For instance adding spare machines to a Boinc Project, or using them for network rendering in 3DS Max projects. Last but not least clusters for crypto currency.
0:11. Fuck yeah buddy, every god damn day.
is possible to use cluster computers to play heavy games ?
Yes
I created a passwordless ssh key. I can access the slave machine without a password, but when I enter the following command
mpiexec -n 30 -f /home/cluster/mpich/hosts ./cpi
I get this error
--> bash: /home/cluster/mpich/bin/hydra_pmi_proxy: No such file or directory
but when the command like
--> ls -al /home/cluster/mpich/bin/hydra_pmi_proxy is entered.
it turned out like this
--> -rwxr-xr-x 1 cluster cluster 2494144 Sep 17 14:51 /home/cluster/mpich/bin/hydra_pmi_proxy
so the file exists.
would this work on a Chromebook running ubuntu xiwi with ten gigs of storage?
I get the same weird urges and have ended up with a million spare computer parts and a few fully built tanks and many"projects".
20 feet to the left and the car version of this story is located right outside the garage.
How do we run normal software like bitcoin miners or video games or just intensive programs? Does the software we run have to be specially coded for this?
You are the most literate tech guy on UA-cam. I watch a ton of videos. You must have been doing this a while and possibly some college classes in IT and Software. lmao. Wow.
i followed this tutorial but im stuck at the ssh-keygen part. i have computers i made worker1 and worker0. when i use ssh-copy-id worker1 it tell me to enter the password for worker1. i do that but then i get a messge saying "permission denied, please try again". i am putting in the proper password but i do not know why it is not working. can anyone please help me?
Awesome,
If I supplied all materials and shipping, how much would you charge me to make me one?
Probably a lot of money
$25
5:10 so by adding ip's you are creating a botnet?
but the ip has to be from a linux pc?
I think you can make windows 10 linux compatible
Hey Dave,
Will it still work with let's say PS3? I heared they blocked Linux with an update but not sure if they unblocked it later on.
Wait so you can only link them over wifi? You cant physically tether 2 computers offlinem
Nice video, thanks for this
I can’t wait to see 4K porn on my new supercomputer 😍
Man 😂😂 I think you should try 8k now 😂😂
what is a supercomputer
easy and real good tuto...thanks
can it be used to get extra performance in gaming?
I cant seem to connect to my secondary computer (port 22), connection refused
any ideas?
just what I was looking for!!!!!!!
wait so i7 is obsolete right?
NO! Wtf?
+uziel900able No xD why would you think that?
+uziel900able i7 isn't fast enough for some things
This vid makes the i9 obsolete.
1. instead of openning/closing terminal, do 'source .bashrc'
2. extract mpich...tar.gz from the command line: tar -xzvf mpich....tar.gz
Thanks, however, this is parallel workers as both workers did the same job, and didn't get better timing or ant exceeding in processing speed, how can you make workers share the load of processing not to do the same process in parallel?
guys I'm a beginner, some of you have information on how to build a supercomputer with windows 10 or linux, I have 5 computers and I want to join them
have you tried yet?
Was thinking of working a bunch cell phones together but seems it really wouldn't be any cheaper to build the ultimate 220 s10 phones together with ethernet cables lol yeah pipe dreams
ok, but first, let me understand something. If join 2 computers together using this method, will this increase the perfomance of games??
Fortran may be old, but it is still one of the fastest languages (least overhead) to program with.
I lauged at the intro because I have that processor in my old pc
Will this also handle processing from GUI applications such as HandBrake for video rendering?
I am designing a computer architecture radically different, really different, and it is a super computer, its the last stop end all architecture
Hey man can use cluster with Windows (master) and Linux (cluster)
your expalnation is simple and good , I already have NIS and NFS on my ubuntu clent server model, server is IBM , can you plz tell me how can i convert it to super computer
Nice intro!!!
I get the src/pm/hydra configure failed error
both as sudo and not sudo
The default question in this type of tutorial is: can you just run any Linux application with this method, or it has to be one specifically designed for that??.
For instance; if I want to run Jack an some audio plugins, should I just install the the apps in both computers, and just let the first one to do the job??. I have not seen anyone explaining this..
Can I use any other operating systems?
can you combine GPU's this way?
+Tomas Sab No. Not with MPICH. MPICH is made for CPU's and CPU's only.
Thanks for the video! Would this work for games or mprime?
Dose this work with non rasburypi computers
yes I used to have a day wake up and said: I will build a cluster computer
The link for the text tutorial comes up with nothing but adds
What is the music that plays at the beginning
My wallet: heh lol no
Me: why
Gary Coleman explains lol. You did a Pi cluster, I'm no way did you create any kind of supercomputer
Nice, but what can the regular user do with Ubuntu? Type?
What do you mean?
I'd assume you could do this with Ubuntu Server aswell if you install a GUI
What I meant is that on the server side it is a killer application, but on the regular user side there is very little, since there are not many programs that run on Linux. A lot of the hardware is not compatible with Linux, and now even the motherboards are not compatible with Linux when they only run UEFI bios. On the enterprise, most of the work is typed on Word, Excel, or Power Point, but when it comes to movie or picture editing, or when it freezes (it does a lot), there is very little that can be done. At home, every time you need to install an application, apt-get command is a little complicated to use: the so many security steps, passwords, and strange terminology make it hard to use. I love Libre Office, and for the most partt, it does the job quite well, but I would ,like a little more standardization so more hardware and software can be produced for Linux. BTW, the taskbar on the left hand side drives me nuts.
My dad is a regular user, he's been fine with Ubuntu for 2 years. Never needed Windows or OSX for any purpose.
Linux is super powerful. All high end systems run Linux. Windows is just for gaming. Linux is for supercomputing, particle colliders and pretty much every server on the web. You want to learn res computing? use Linux. Want to play shooting games? windows
Dave Bennett i can run firefox in mpich ?
Why is the font on your chromebook all f**ked up?
Today, I want to get two Asrock PV530A mATX boards to work as one somewhat better single computer.
I tried the command:
impich = trump
It didn't work. Do I need to add sudo before the command?
Maybe he would get impeached if Americans could spell............
near the end, it says" No Command 'mpi' found
Do mpirun and remove exec
Are you using the unity interface on trusty?
Does it work on my Windows 7?
If I built a ,10 raspberry pi2 cluster,will it works on it too? Sorry but how fast is supercomputer compared to the latest Mac. computer? Sorry I am a computer Newbie and a nerd.
Macs are not the best computers at all.
I feel like you're trolling
Yeah you can do this with a Raspberry Pi cluster.
Thanks bro
umm..my pc hard is 500 giga and ram is 4gb and does that work with it or only on chromebook ?
Only on Chromebook.
If you have Linux installed, it should.
I did say that yesterday when I woke up. =D
so you are installing the program on a live Linux?
+indykoning this is only for tutorial ;)
Can I do it on Windows 10
OMG I’ve wanted to build a distrusted processing system in OS X like forever
Connect about 500 Parralella boards into a cluster before you do this.
OH MY GOD IS HUGEEEEE!!!! THX MANN!!!!!!!!!!!!!!!!!
Can i do this on Windows?