Raspberry Pi 3 Super Computing Cluster Part 2 - Software Config

Поділитися
Вставка
  • Опубліковано 27 сер 2016
  • If you missed Part 1 (Hardware and Assembly) go watch it here: • Raspberry Pi 3 Super C...
    In this video, I walk you through the steps to configure the first node in the cluster as a DHCP server providing NAT to the remaining nodes in the cluster. We also walk through installing BOINC, getting an account, and attaching the nodes to the Einstein@Home project.
    The adafruit tutorial referenced in the video can be found here: learn.adafruit.com/setting-up...
    www.einsteinathome.org/
    Thread about Pi and Einstein at Home: www.einsteinathome.org/conten...
    Join the Raspberry Pi Einstein@Home Team: einsteinathome.org/community/...
  • Наука та технологія

КОМЕНТАРІ • 202

  • @adavistheravyn573
    @adavistheravyn573 7 років тому +19

    Really cool stuff! I work in the field of high-performance computing and had something similar in mind for my own numerical simulations or BOINC.
    Before spending hundreds of Euros for RPi3 boards, I did some tests with a special version of my nbody code which is written in C, highly optimized and utilizes the OpenMP library for parallel computing. My benchmark focussed on floating-point performance with negligible RAM consumption. What are my results? Well, it's disillusioning. My benchmark task took 65 minutes on a single core of my RPi3, while a single Core i5-6500 solved the problem in 65 seconds! Using four threads, the RPi3 still took more than 18 minutes, while my Intel Core i5-6500 got that job done in 18 seconds.
    Conclusion: Neglecting communication overhead, I would have to come up with more than 60 RPi3 boards to get on par with a decent Core i5-6500 ... ARM might give you more FLOPS per Watt, but when it comes to pure floating-point performance, the architecture is still far behind.

  • @svanhoosen
    @svanhoosen 4 роки тому

    I just finished building a similar system, with 8 Raspberry Pi 4 computers. I mounted them in a rack made of 3D printed rack modules. I followed a lot of your build, and especially liked the idea of the 5v USB power for the switch and fans. It is nice having a unit that only requires a single power cord. I'm currently running Rosetta@home on it, which is looking for a cure for Covid-19, so nice to have it doing something useful. I'll continue to experiment with it, and hope to learn to do some real clustering with it. Thank you for your tutorial!

  • @SanderGroenendijk
    @SanderGroenendijk 7 років тому +3

    Thanks for the video! helped me out a lot.

  • @gristlevonraben
    @gristlevonraben 7 років тому

    amazing. one day I hope to do something similar, but create a video editing and media system, turning one pi into a computer focused only on sound.

  • @PiperUsmc
    @PiperUsmc 7 років тому +1

    Nice Videos, very professionally done sir!

  • @usptact
    @usptact 5 років тому

    Very well made and explained video!

  • @InactivePhoenix
    @InactivePhoenix 8 років тому +2

    Great vids keep it up dude

  • @KarstenJohansson
    @KarstenJohansson 8 років тому +1

    I've been using avahi so I can reference all my pi's (and other home systems) by hostname. Much easier than remembering them all. Especially if you swap an SSD card, and don't want to reconfigure *everything* to have the right IP addresses pointing to it.

    • @KF7IJZ
      @KF7IJZ  8 років тому +1

      Would love some more information on this. I was playing with some of that stuff, but had limited time and my focus was completing work units. Now that I have these clusters, I could be doing a lot more to optimize the setup!

    • @KarstenJohansson
      @KarstenJohansson 8 років тому +1

      You've already done 90% of the work. If you install it with apt-get, it'll be working after your next reboot. If you stick with defaults, you can reach your rpis like ssh KF7IJZ@rpi1.local and it should work. You can of course change the domain to something other than local if you want. I haven't bothered since it is internal only anyway. No DNS needed.

  • @Mobi2525
    @Mobi2525 8 років тому +2

    Even though this isnt what i thought it was, i still like this setup. I am going to create something like this. I have an extra 48 port switch just laying around here might as well put it to use. Lol

    • @KF7IJZ
      @KF7IJZ  8 років тому +2

      You could mount a LOT of stacked Pis on that!

    • @GeekBlogTV
      @GeekBlogTV 8 років тому

      So much room for activities! Great video series, sir. I hope I can learn a few things from you to take home and show my friends :) Keep up the good work! Subbed, and liked.

    • @ultimatenetworkninja3139
      @ultimatenetworkninja3139 7 років тому +1

      my CCNA lab might be handy, 3x48, 3x24 😀 but no rasp ;d

  • @brianstarr
    @brianstarr 8 років тому +3

    Very nice tutorial. The music wasn't too loud and thank god it wasn't techno. :) The only thing I can suggest is to explain the objective first in a few sentences. I think we get the point but some of us are new to distributed computing. Please keep the great work!

    • @KF7IJZ
      @KF7IJZ  8 років тому +1

      Good feedback. I actually had an intro and some other details like that, but opted to trim more content out of the video. I thought I was getting a little preachy :)

    • @brianstarr
      @brianstarr 8 років тому +1

      I'm a special nutcase. Don't be influenced by my opinion. :)

    • @KF7IJZ
      @KF7IJZ  8 років тому +2

      The first step to solving any problem is... :)

    • @brianstarr
      @brianstarr 8 років тому

      Sighting Elvis?

  • @ElDirtyFly
    @ElDirtyFly 7 років тому +77

    Well. this is not really a cluster per se, in the meaning that the only thing clustering is the physical location. The nodes are not working together as one or as an unit. Loved the hardware part though. I've made a 3 raspberry pi2 hadoop cluster just for fun, contact me if you want the details.

    • @pecke86
      @pecke86 7 років тому +1

      ElDirtyFly how to find a manual to do that? I need to create a cluster computer for 3 different screens

    • @xieys3985
      @xieys3985 7 років тому

      would you please give me some instruction for the real cluster?thanks!

    • @dushyantk
      @dushyantk 7 років тому +1

      care to share the details?

    • @gipen
      @gipen 7 років тому +2

      I was watching the video expecting a cluster too, disappointed :(

    • @shanejohns7901
      @shanejohns7901 5 років тому +4

      @@gipen You don't understand what you're seeing. The clustering is for something like Einstein at Home, which he mentions, and the submission of work units. As he says around the 21:00 area -- you have to install boinc individually after the cloning, to get a unique number, not one that gets cloned for each rpi, mostly for proper tracking of submitted work packets. This is basically a distributed work model, where your home machine(s) sits back, accepts work submissions from the Einstein@home program, does math on the package you're meant to work on, and then uploads the answer back to the Einstein@home server. The Einstein @home program is specifically designed to look for gravitational waves and pulsars. And there are other programs that work on this kind of model as well, such as protein folding modeling to help the biological sciences. In fact, there's nothing to stop anyone from creating some other kind of massive number-crunching tasks which can be broken up into piece-work. The limit really is your imagination...
      ``Einstein@Home searches through data collected by the LIGO and GEO600 gravitational wave detectors for evidence of periodic sources (such as spinning neutron stars), which would be the gravitational equivalent of pulsars. Einstein@Home was launched as part of the World Year of Physics 2005.``
      boinc.berkeley.edu/wiki/Einstein@Home

  • @deeeeeds
    @deeeeeds 7 років тому +5

    Perhaps I missed it but why bother with the WLAN configuration when you're using ethernet for connectivity regardless?

  • @MoTown44240
    @MoTown44240 7 років тому

    Great videos, Part 1 and Part 2. 73

  • @cnoyes72
    @cnoyes72 7 років тому

    After enabling IPv4 forwarding in /etc/sysctl.conf and the echo stsement I had to reboot the machine for iptables to be able to accept NAT table modification (before that iptables complained that the nat module didn't exist).

  • @nielst.3891
    @nielst.3891 7 років тому +1

    Just joined with 1 Rpi and 2-3 available cores on EinsteinAtHome and SetiAtHome (teamed up with Rpi team). Other resource is used as a bitcoin full node. Still in doubt about building a cluster/bramble/beowulf of Rpi’s...

  • @TelmoMachadoSistemas
    @TelmoMachadoSistemas 6 років тому

    Good afternoon, I was watching your video, and I would like you to show a video playing a game to check if the processors are being used in a cluster? as well as the memories. could you do that as an example of performace?

  • @chlorine8477
    @chlorine8477 7 років тому +3

    Hi, thanks for this awesome tutorial. However, there are two mistakes at 13:03, when you edit the dhcpd.conf file.
    The correct options are 'routers' and 'domain-name-servers'.
    In general, it's convenient to check if the dhcp daemon is started using 'sudo service isc-dhcp-server status'.

    • @dalamar437
      @dalamar437 7 років тому

      You sir are my hero! Great catch!

  • @AlexNiebla
    @AlexNiebla 7 років тому +9

    This cluster is just working for BOINC or SETI.
    Would it be possible to make a cluster with RPis that behaves as a single more powerful (more cores, more RAM) computer for any day-to-day application? I mean, for daily work, keeping many tabs open of Chrome, several spreadsheets and text documents on Libreoffice, all sharing and behaving as a single computer sharing RAM, CPU Cores, CPU power and accessing one local hard drive on the main RPi?
    Has that been created? (I know the ethernet port on RPi are slow, my question is geared more towards other similar ARM Linux dev boards with gigabit ethernet).

    • @ShaoVideoProduction
      @ShaoVideoProduction 5 років тому

      No that is not realistic, the latency would be way to high.

    • @madezra64
      @madezra64 5 років тому

      @@ShaoVideoProduction Even if you networked them together via the GPIO?

  • @JayTheKing111
    @JayTheKing111 7 років тому

    Would be good to show an alternative to BOINC and einstein at home, i recommend Docker swarm.

  • @oskarpaulsson5885
    @oskarpaulsson5885 4 роки тому

    Hi, I liked your video and I'm curious to get going with my own cluster. I'm wondering what you use it for? My usecase is that I'd like an affordable way to run code like simulations in MATLAB for university tasks to be able to still use my laptop for other tasks while simulations are running. Do you think a cluster like this would be good for something like that?

  • @billyashworth3944
    @billyashworth3944 7 років тому

    Do you have to follow notes for the command line entries or do you just know this stuff?

  • @FlyBy2507
    @FlyBy2507 6 років тому

    How do you deal with the "short" life time of sd cards ?
    Did you disable all log files or send the log to /dev/null ?

  • @videoyou7031
    @videoyou7031 5 років тому +1

    Intresting, but what you can do with that cluster? Why you do this thing? What you do on it?

  • @DontScareTheFish
    @DontScareTheFish 6 років тому

    At 15:15 rather than echoing 1 to ip_forward you can do "sudo sysctl -p"
    At 18:35 you can run egrep 'lease|ethernet' /var/lib/dhcp/dhcpd.leases rather than cat and copy

  • @iDVB2
    @iDVB2 7 років тому

    YIKES! your video blips at 13:04 from two pluralization syntax errors with `domain-name-server` and `routers`. Immediately after the blip your file is magically corrected. Took me about 3hrs of messing around and rewatching to figure this out!

  • @SambitNayakchandan
    @SambitNayakchandan 8 років тому

    wow, great tutorial about pi 3 cluster. can we use this raspberry pi 3 setup for running a spark cluster?

    • @KF7IJZ
      @KF7IJZ  8 років тому

      A quick googling shows that yes, Spark could be made to run on a Raspberry Pi.

  • @DiyintheGhetto
    @DiyintheGhetto 5 років тому

    Hello i have a few of questions. First is when you cluster all the pi's together does it combine all CPU cores together into one as well as memory? Second thing other applications that are CPU and memory intense can this work better then just using a Single PI? The last question is. For the price of 8 pi's plus fans, network stick, Power adapters, heat sinks and what not. You are looking into the $350 maybe or so range. Wouldn't it be better to get a computer motherboard,CPU and computer power supply plus memory that can put out a lot of cores and memory? I know it's teaching us how to cluster a server together and I'm happy learning. But from my stand point the cost I'm worried about. Thanks

  • @BasNunnikhoven
    @BasNunnikhoven 5 років тому

    I am very new in this... But is it like if I have 1 PI wich is not strong enough to run specific software or tools I can make a cluster of 4 or more to have a performance boost as big I expand?

  • @rxmas7234
    @rxmas7234 5 років тому

    So, I'm setting this up and using the current 'Buster' lite, it fails at the dhcp install. I can configure to the point of the reboot at 14:35. At that reboot I don't have any config for eth0. I didn't configure wlan0 because I don't want wireless on this, just eth0. Do I need to start over with Jessie?

  • @BrianTaylor-AlwaysInTao
    @BrianTaylor-AlwaysInTao 7 років тому

    Would this be difficult to do with mixed architectures?

  • @kppavu8809
    @kppavu8809 6 років тому

    Bro can you tell me the advantages of this super computer plz

  • @440diesel9
    @440diesel9 7 років тому +22

    I have sifted through plenty of these sort of "raspberry pi cluster" videos and I have yet to see someone make an actual MPI message parsing cluster that the head node splits work to worker nodes like it's supposed to be. All these setups are the same. Just a bunch of individual PIs sharing a case and power supply but work independently. That's not a true cluster at all. Also, what's the point of running a dhcp server on the head node if you're gonna tie a specific IP for each MAC address in the .conf file. You might as well have set a static one for each pi. Just making it convoluted and running around in circles to end up with the same end result.

    • @KF7IJZ
      @KF7IJZ  7 років тому +2

      I use DHCP to assign specific IP addresses based on the mac address so that when an SD card goes bad, it's one less thing I have to maintain on each node, so there was a specific purpose. And no, there is no MPI messaging, but there in theory could be. You could also run a Docker swarm on this.

    • @bobsagget823
      @bobsagget823 5 років тому +2

      @@KF7IJZ wrong. clueless moron

    • @craigaskew9115
      @craigaskew9115 4 роки тому

      @@bobsagget823 I think that was a bit strong. I would also like to see MPI in action here as I haven't found a good set of instructions for that yet, but I'm not going to go insulting the man.

  • @alejandropenedo6676
    @alejandropenedo6676 6 років тому

    Great video series! Any chance for an updated version for Raspbian Stretch? I managed to port most of the networking stuff to wpa_supplicant.conf and dhcpcd.conf, but I can't get iptables to update on a restart and my dhcp is not assigning ips to the rest of the boards connected.

  • @troyBORG
    @troyBORG 7 років тому +1

    For BOINC I've always called it bionic.

  • @hectorsauvage7511
    @hectorsauvage7511 6 років тому

    Could do with an errata vis a vis '/etc/network/interfaces' which no longer is relevant. 'Boinctui' could be of use for raspbian light sans X. Cheers.

  • @ahmedalahmari
    @ahmedalahmari 6 років тому

    Well by using POE switch to provide pi3 power?

  • @danielkurniadi8805
    @danielkurniadi8805 4 роки тому

    @KF71JZ Hi there, just to be sure, I'm going to do this repeat over all raspberry pi that I have isn't it?

  • @VirtuelleWeltenMitKhan
    @VirtuelleWeltenMitKhan 5 років тому +1

    Could you upload a version without music please? So we can choose which one to watch.
    Thx
    Greetings

  • @broski499
    @broski499 7 років тому

    How many Gflops does this thing register on boinc? I have a pi 2 running seti@home and it shows around 1.5 gflops per core. Pi 2 shows .94 gflops per core and the pi 1 shows .34 gflops for its one core.

  • @vizionthing
    @vizionthing 8 років тому +55

    Nicely done - apart from the very annoying background music, would have been better with none.

    • @saxonlight
      @saxonlight 7 років тому +6

      Yeah the music is so annoying I lost all interesting in the video.

    • @chaniem.5422
      @chaniem.5422 7 років тому +1

      i actually would have left if their wasn't because it would be too boring, maybe it could have been turned down a little so its easier to hear him but that wasn't a problem for me

    • @jokindajoker
      @jokindajoker 7 років тому

      yea put some nicki minaj

    • @AlexNiebla
      @AlexNiebla 7 років тому

      I liked it with the music, better than without music.

    • @Phonomatic
      @Phonomatic 5 років тому +2

      music is just too loud compared to the voice, especially around 7:30 min.

  • @jimwelch4481
    @jimwelch4481 6 років тому

    Great vidio, but the background music could use a couple dB of attenuation.

  • @Jacespace23
    @Jacespace23 5 років тому +12

    I'm a noob , what would this be used for?

    • @ForOdinAndAsgard
      @ForOdinAndAsgard 4 роки тому

      Parallel computing.

    • @ForOdinAndAsgard
      @ForOdinAndAsgard 4 роки тому

      @george What is the fun in that? Why not make a faster computer with a lot of cheap slow ones. Like 64 overclocked Pies in a stack plus one controller unit. That is only 256 cores for the nodes. It will blow a threadripper out of the water.

  • @TheNZJester
    @TheNZJester 7 років тому

    I was wondering if there is a way to tie both the LAN and WiFi together to act in unison with some sort of load sharing to increase the available network speed?

    • @KF7IJZ
      @KF7IJZ  7 років тому

      My guess is that it wouldn't help. For instance, on the original Pi and the Pi 2, the onboard ethernet is tied to the USB hub which is where the speed constraint comes from. In this case, I'm guessing the onboard ethernet and WiFi are both enumerated through USB, thus you would still be fighting against the bottleneck of USB.

  • @agausmann
    @agausmann 7 років тому

    In order to get more performance out of the Pis (and as a general recommendation), I prefer Arch Linux ARM. The Raspbian image supplied by the Raspberry Pi developers has so much bloatware pre-installed.

  • @craigaskew9115
    @craigaskew9115 4 роки тому

    I'm having difficulties with this. The file /etc/network/interfaces in Rasbian Buster Lite doesn't look anything like the one shown here for Jesse. Typing in the missing lines to make it look like the Jesse file shown here seems to render the connection over ssh invalid.

  • @Flip-Flop-Rio
    @Flip-Flop-Rio 8 років тому +1

    Friend, good afternoon !
    I wonder what the processing power of a raspbery pi alone and all together this cruster .
    Comprarar can with a PC i7

    • @KF7IJZ
      @KF7IJZ  8 років тому +4

      I quickly found this: GenuineIntel Intel(R) Core(TM) i7 CPU 975 @ 3.33GHz [Family 6 Model 26 Stepping 5] - Measured floating point speed:
      3679.09 million ops/sec
      Measured integer speed:10314.42 million ops/sec. One RPi3 gives this: Measured floating point speed:
      753.49 million ops/sec
      Measured integer speed:2479.17 million ops/sec - so 8 PI3 for Integer or Floating Point is about twice the instructions total of that particular CPU.

    • @petergillock9533
      @petergillock9533 7 років тому

      That's impressive, given the price difference. And I bet the Pi's use way less power.

    • @capri2wd
      @capri2wd 7 років тому +1

      at £30 for a pi 3 (in the uk) it would probably cost alot more for the pi's than to build a system with 2nd parts with the i7 975. Although the pi's should have more grunt together, the i7 could utilize a GPU for extra crunching power, probably resulting in much more compute power than the pi's. BUT it would not be anywhere near as cool as this and 2nd hand parts can be unreliable.

    • @adavistheravyn573
      @adavistheravyn573 7 років тому +2

      I did some tests with a special version of my nbody code which is written in C, highly optimized and utilizes the OpenMP library for parallel computing. My benchmark focussed on real-life floating-point performance with negligible RAM consumption. What are my results? Well, it's disillusioning. My benchmark task took 65 minutes on a single core of my RPi3, while a single Core i5-6500 solved the problem in 65 seconds! Using four threads, the RPi3 still took more than 18 minutes, while my Intel Core i5-6500 got that job done in 18 seconds.
      Conclusion: Neglecting communication overhead, I would have to come up with more than 60 RPi3 boards to get on par with a decent Core i5-6500 ... ARM might give you more FLOPS per Watt, but when it comes to pure floating-point performance, the architecture is still far behind.

    • @Jadinandrews
      @Jadinandrews 7 років тому

      Adavis Theravyn I think by pure floating point performance you mean IPC, right? I guess another thing to look at is TDP. The i5 6500 has a 65w TDP, while the rPi3 is somewhere around 1 - 1.2w for the whole board. So ignoring the obvious network overhead, and cost of switches, at least 60x RPI3's are somewhat in the ballpark of the TDP of the i5.

  • @austindale3129
    @austindale3129 4 роки тому

    I am at 18:36 of the video, with 2 pi's connected to the switch the first running the control node and the 2nd with the copied disk image, when I enter "cat /var/lib/dhcp/dhcpd.leases" it returns # The format of this file is documented in the dhcpd.leases(5) manual page. What went wrong?

  • @dreamyrhodes
    @dreamyrhodes 7 років тому

    You could also just do an image of the SD-card with the boinc-client then edit the dhcp config and type all MAC addresses in there giving them fixed ip numbers. This way you don't have to set it up on each pi in the cluster, just plug the card in and run. Network config will be done by dhcp then and the software will be pre-installed everywhere.
    A much more advanced way would be of course having the OS in a PXE boot.

    • @KF7IJZ
      @KF7IJZ  7 років тому +1

      I originally did that, but I learned that BOINC generates a GUID when first installed. This GUID was being propagated to all individual PIs in the cluster and thus they would get duplicate work units and their results wouldn't be accepted. I would LOVE to net boot these things from a single node with a USB hard drive - probably will do that next year when the Pi 4s coming out :) I'm guessing net and usb booting will be out of beta with next generation of hardware. Thanks for watching!

    • @dreamyrhodes
      @dreamyrhodes 7 років тому

      KF7IJZ Sounds like this would maybe an acceptable feature request to have the client like creating it's GUID on DHCP configuration or like each time a MAC address changes. The MAC needs to be unique in your setup anyhow, because each device is on the same network, so it sounds reasonable to base the GUID creation on the MAC. Or maybe that you can pre-configure BOINC to create it's GUID on first start and have this pre-configured image on PXE or your USB-image that you distribute.
      I don't know about the development process of BOINC and suppose you're not involved in that, but maybe they take feature requests.

    • @Fritzrgc
      @Fritzrgc 7 років тому

      The GUID is used to identify the "Host". When the GUID changes, the machine is identified as a new Host.

  • @austindale3129
    @austindale3129 5 років тому

    What do I do if sudo ifdown wlan0 returns ""ifdown: interface wlan0 not configured"

  • @georgedicu7397
    @georgedicu7397 7 років тому +6

    So this is just a pi stack, not a real software cluster that can utilize and combine all this power in a controlled way?

    • @AndrewJamison79
      @AndrewJamison79 7 років тому

      he said as much in Part 1 for this particular case its just a stack not a combined resource cluster

    • @georgedicu7397
      @georgedicu7397 7 років тому

      Cool. Tnx pretty cool anyway

  • @austindale3129
    @austindale3129 5 років тому

    Would I run into any problems if I installed the full Raspbian stretch with desktop and followed along with the rest of your setup tutorial?

    • @KF7IJZ
      @KF7IJZ  5 років тому

      I ran light because for the workload I used, it demands every ounce of resources that can be given to it. If you aren't running the specific BOINC/E@H workload that I was running, all the rest of the information still applies whether running light or not.

  • @ronaksuchak
    @ronaksuchak 5 років тому

    Why we can't assign static ip from router ?

  • @ryanwakebradtelle8682
    @ryanwakebradtelle8682 6 років тому

    So what are you supposed to do Minecraft server.

  • @xelionizer
    @xelionizer 7 років тому +5

    What is the point of going through all the hassle setting up dhcp if in the end you end up assigning a fixed ip to every node anyway??? What am I missing here?

    • @KF7IJZ
      @KF7IJZ  7 років тому +2

      I manage the IP addresses of each node using DHCP so that if/when I have to rebuild a node's SD Card, I don't have to manually set an IP address on that specific machine - the static addresses are centrally managed on Node 0.

    • @kdocki
      @kdocki 7 років тому

      You can manage statically assigned IPs through your DHCP instead of having to do that on individual machines. Nice video btw...

    • @user-gr3ow9bw8e
      @user-gr3ow9bw8e 7 років тому +2

      So do you have back ups for Node 0 or is that now your single point of failure? And do these cards actually fail often enough that this is worth the effort? What about just having byte for byte backups for all the nodes? They are only 16 gigs after all.

  • @Mobi2525
    @Mobi2525 8 років тому +1

    woohoo! Thank for the video!
    I think I might be a bit confused.. This seems like each Pi is running individual jobs and not using the individual resources of each Pi combined to complete a single job. Is that true? Essentially, Node 0 created a private network for the other Pis to access the internet to get these jobs. Is that true, there is no distributed computing involved?

    • @RobertFHGroeneveld
      @RobertFHGroeneveld 8 років тому

      You are partly correct, the pi's are running the einstienbinary on every individual node. but the einstein software already gets it workload from a controller somewhere. So instead of using MPI for instance to cluster nodes that are local it uses the boinc software to subscript to a project. einstein@home in this case and einstein@home, tells every node what to do. Its stil clustering only you are not controlling what calculations are going to be run.
      I'm wondering if you would install MPI first and then run boinc on the MPI cluster would it better perform of worse in comparison ? to bad i don't have some pi's laying around to test this.

    • @Mobi2525
      @Mobi2525 8 років тому +1

      So this is essentially like any other @home project.. If I have multiple computers running Seti@home isn't this the same thing?

    • @RobertFHGroeneveld
      @RobertFHGroeneveld 8 років тому

      its is. but as a group they are computing to help and solve a big problem. this is essence is a cluster a group of machine be it pi's or desktops etc. how compute to solve the same problem, only each node solves a part of it.

    • @KF7IJZ
      @KF7IJZ  8 років тому

      No option to MPI BOINC for E@H I'm afraid. and thanks for the great explanation! Couldn't have said it better myself!

    • @KF7IJZ
      @KF7IJZ  8 років тому

      You nailed it. It has been suggested to me however that I could run E@H in a docker container and setup a swarm on the machine.

  • @ultimatenetworkninja3139
    @ultimatenetworkninja3139 8 років тому

    Is that impossible to make it all unattended with some XML? We should think about some deployment system to make cluster config faster.

    • @KF7IJZ
      @KF7IJZ  8 років тому

      Agreed - in the future I would want to play with Ansible, Chef, or possibly even Docker.

  • @kevinramirez7
    @kevinramirez7 3 роки тому

    5years later... But if I have other proyecto that I want to run on the cluster?? Or a script

  • @pengzhou3655
    @pengzhou3655 7 років тому +1

    Have it all set up, been running 24 hours and no credits have showed up on Einsteinathome website yet. Am I doing something wrong ?

    • @KF7IJZ
      @KF7IJZ  7 років тому +2

      Make sure your'e watching the "Account" at einsteinathome.org . It can take a day or two for results to be validated as well. Please consider joining the "Raspberry Pi" team!

    • @pengzhou3655
      @pengzhou3655 7 років тому +1

      I just checked on it and its all good. Thanks for the help.

    • @peterwildbichler6756
      @peterwildbichler6756 6 років тому +1

      Joined you with mi Pi2 ;)

  • @mrdot1126
    @mrdot1126 5 років тому

    aparently on Pi 2 100% meant 1 core working, and 4 cores ment 400% but maybe the software or the newer Pi works otherwise...

  • @sanjeevthomas7582
    @sanjeevthomas7582 6 років тому

    How to install windows on raspberry cluster?

  • @f-ben
    @f-ben 7 років тому +17

    So what you did is a video about 8 pis running boinc in the same network. That´s not even close to a cluster you know that right.

  • @mrguiltyfool
    @mrguiltyfool 7 років тому

    Can you do a tutorial on orange pi cluster since it is cheaper than pi3

    • @KF7IJZ
      @KF7IJZ  7 років тому

      Honestly, most everything should still apply except for the base configuration of things like SSH and WiFi - all of the network and BOINC material should still apply assuming you're running a Debian based OS on the Orange Pi.

  • @brentshaw9723
    @brentshaw9723 7 років тому +24

    great vid but LOOSE THE MUSIC!

    • @aidanmco
      @aidanmco 7 років тому +2

      Brent Shaw
      How does one "loose music?"

    • @layton3503
      @layton3503 6 років тому

      Get in your car with a shovel drive to a remote location, dig a hole and place the music in the hole would be my recommendation for this music. It really is annoying. The video is great, but I've been on the verge of a headache the whole way due to the music.

  • @opless
    @opless 8 років тому +4

    Great vlog, just a small niggle - 'etc' means 'et cetera' so you can just say it the same way as you would read it normally rather than 'et see' :-) Keep up the good work!

    • @cnoyes72
      @cnoyes72 7 років тому +12

      All the Unix people I know pronounce the /etc directory as 'et see'.

    • @juddwest3295
      @juddwest3295 6 років тому +1

      The Linux Foundation training videos say "etsee"

    • @draco5991rep
      @draco5991rep 5 років тому

      Could it be "et" = and
      and "c" for directory C ?
      I don't think it's supposed to mean et cetera

  • @redrockrabbitk6078
    @redrockrabbitk6078 6 років тому

    benefits of doing this? can i play more "hardcore" games (no i don't mean call of duty, maybe farcry 2)

  • @ryanwakebradtelle8682
    @ryanwakebradtelle8682 6 років тому

    The Raspberry Pi 4 should have USB 3.2 type c for charging and data transfer so you could get custom cables made with no excess will get a $150 fancy USB hub cable that can connect 150 to 200 Raspberry Pi 4's. Also the Raspberry Pi 4 should have a thunderbolt port just because. If you want to pay an extra $20 you should be able simple custom that can't fit into the $40 model. $2 after the first 4 features.

  • @sebaaa9371
    @sebaaa9371 7 років тому

    How much is it in total?

  • @epistte
    @epistte 8 років тому +2

    Has anyone tried gaming on a Raspberry Pi cluster? Can you rig a PCI lane on one of this mini computers?

    • @Wolfeydubstep
      @Wolfeydubstep 8 років тому +1

      Would be difficult to take good advantage of each system's resources for that specific task. Overall not really worth the effort.

    • @epistte
      @epistte 8 років тому +2

      Anony Mouse No. I'm a female engineer who like to push the limits of technology. Push it to the limits, fix it and push it to the limits again. I found destructive testing in a lab to be very enjoyable.

    • @epistte
      @epistte 8 років тому +2

      Troll harder. I'm not amused.

    • @user-gr3ow9bw8e
      @user-gr3ow9bw8e 7 років тому +2

      Why did you feel the need to qualify your gender? And no, there really isn't much in terms of gaming you can do with fucking 1.2 GHz ARM and 1GB of RAM. You can push, pull, kick, and scream all you want, but you aren't going to get it to work. Unless you are way into Tux Racing.

    • @anonymouse7290
      @anonymouse7290 7 років тому +2

      epistte I am not here to amuse your dumb ass i'm here to tell you it's dumb.

  • @TheWoodad
    @TheWoodad 7 років тому

    Your videos are very well done and now I'm off to go build me a cool pi cluster! It may be off-topic, so if you don't mind my asking: what is the name of the opening music?

  • @demonrabbit6080
    @demonrabbit6080 7 років тому

    how many BTC I CAN GET?

  • @aldienightstar9033
    @aldienightstar9033 7 років тому

    Wow! Very nice music ^^ Send me here what calls :3

  • @galsherp6173
    @galsherp6173 5 років тому +1

    I dont get it, why the whole Wifi thing when u connect all your pis to a switch!?

    • @jrlx86
      @jrlx86 5 років тому

      Gals Herp because the switch isn't connected to a router. Rpi3-0 is by wifi, providing internet access to the other pis in the cluster. Einstein@home gets its work packets from the internet. Cluster is the wrong word really, this is a stack as the pis aren't sharing workload determined by the controller (rpi3-0)

    • @dntanderson
      @dntanderson 5 років тому

      But the workload for each core is being determined by the Einstein@Home controller ....

  • @johanperdana2108
    @johanperdana2108 8 років тому

    can I use this cluster for editing video ?

    • @KF7IJZ
      @KF7IJZ  7 років тому

      Doubtful as the software you use would have to be optimized for many factors such as ARM and multi processing support.

  • @matthewolleres8149
    @matthewolleres8149 6 років тому +1

    is this setup of raspberry pi can do bitcoin mining?

  • @jarisipilainen3875
    @jarisipilainen3875 4 роки тому

    15:14 its file right? you could just open it nano and change content to 1?. linux again

  • @drkcodeman
    @drkcodeman 7 років тому +1

    I don't see this being cost effective for projects compared to a single x64 even if it is distributed computing. Unless you are compiling projects for ARM architecture.

    • @KF7IJZ
      @KF7IJZ  7 років тому +5

      Part of this was proof of concept. One area where these are very cost effective is if you actually needed to simulate orchestration across a 32 core machine but don't care about the horse power. I had seen a grad student do this with the original Pi B because he didn't need computational power, he needed multiple systems to simulate their working together.

    • @drkcodeman
      @drkcodeman 7 років тому +1

      With Cuda support it kind of blows distributed computing out of the water. What you can do with 1 computer and 4 high end gpu's.

    • @tedvanmatje
      @tedvanmatje 7 років тому +3

      +KF7IJZ - good point. many moons ago, I made a cluster out of ten pentium III based machines running @900Mhz to run a neural network. A colleague pointed out that it wasn't very efficient (due to bandwidth and power consumption), but like you said:"it's proof of concept". I embarked on that project due to curiousity and bordom of using computers just like everyone else does. it's just nice to be able to do things like the 'big guys', without spending a fortune and although it was a huge learning curve (back then, the internet community wasn't even close to what it is now....you had to work things out the old fashioned way), it broaden my horizons and opened more doors than I thought it would.
      nicely explained and presented....it should make this a breeze for newcomers to the sport ;)
      thanks for posting!

  • @return496
    @return496 7 років тому +1

    can we install windows 10 iot in raspberry pi 3 cluster

  • @_yu630
    @_yu630 5 років тому

    スーパーコンピュータで何やってるんだろう

  • @TobiasDettinger
    @TobiasDettinger 7 років тому

    Is it compatible with Folding@home?

  • @eddie-san8913
    @eddie-san8913 6 років тому

    Why not with windows configuration?

  • @Ratteler
    @Ratteler 8 років тому

    The GPU can't be used to speed up any math functions? Something like nVidia CUDA support? That's a shame.

    • @KF7IJZ
      @KF7IJZ  8 років тому

      There was some work by the FFTW folks to make use of the GPU but I haven't seen much about it or found anyone who's had much success. There are folks working on optimizing Einstein@Home processors for the Pi 3 CPU though that cuts processing time in half.

  • @nielst.3891
    @nielst.3891 6 років тому

    Needs rework anyway, following this using Jessie release is not succesfull, also using Stretch not able to get things running.

  • @joseantoniodeoliveira3425
    @joseantoniodeoliveira3425 6 років тому +1

    which coin does she mine? what is the hashrate (MH / S) ?

  • @jarisipilainen3875
    @jarisipilainen3875 4 роки тому

    26:01 you allready installed it. just copy rpi3-1 card to rest and change hostname and ip thing lol.why complicate things. yes if follow some other tutorial

  • @chaniem.5422
    @chaniem.5422 7 років тому

    so they all download and start running a program for a website, but what is it ACTUALLY doing? did i miss something or does Einstein at home just use your computing power at your cost while you get nothing?

    • @markkidwell9716
      @markkidwell9716 7 років тому +1

      I was involved with seti@home for some years using dialup, basically you provide the computing resources and get bragging rights for your efforts. I only had one pentium 133 Linux PC running 24/7 and was getting bested by others with much more horsepower. These 8 raspberries would be a fun way to do this again but I would want to cluster if someone has information on that?

    • @chaniem.5422
      @chaniem.5422 7 років тому +1

      well that doesn't seem worth it to me xD

  • @kimchapen4477
    @kimchapen4477 5 років тому

    i am on windows and cant find jessie lite

    • @petrslansky6659
      @petrslansky6659 5 років тому

      This video is from 2016. Jessie already passed away.. ;-) Use Scretch Lite, it is younger and stronger, it will do the same job.

  • @jarisipilainen3875
    @jarisipilainen3875 4 роки тому

    5:28 this is how linux is very confusing. conf file is in there! put those in conf file! if that conf file have other network settings it would reject what you did. would it? it would? conf file place holder is there to let you know do there change so it will me universal. becouse when you lauch desktop and change WIFI they go that conf lol. now you would have 2 wlan setting top each other lol. thats linux!

  • @snarbywrx
    @snarbywrx 6 років тому

    This is not really a "cluster" as it does not use MPI or Coarrays.

  • @ernstschulz6377
    @ernstschulz6377 6 років тому

    A custer is not faster than ohne rspberry pi because the Network bottleneck

  • @moonlightman38
    @moonlightman38 7 років тому

    If I had 5000 pi cluster how fast and smart will be

    • @cnoyes72
      @cnoyes72 7 років тому

      I doubt that Raspberry Pi clusters can scale that high, considering it's networking is limited to 100 Mbps.

  • @FinlayDaG33k
    @FinlayDaG33k 5 років тому

    This is more of a grid setup than a cluster...

  • @frankschafer1505
    @frankschafer1505 5 років тому +2

    background music made me stop watching

  • @raticide4you
    @raticide4you 6 років тому

    I really can't bear the music any longer. A pity because I really want to know more about your project. Please make a another copy of your video without the music.

  • @cydget644
    @cydget644 7 років тому

    K you smell! The cat will get a sheet of paper.

  • @aidanmco
    @aidanmco 7 років тому

    Eastern time? I wonder how close we are!

  • @TattooedHomesteadLithia
    @TattooedHomesteadLithia 6 років тому

    just got parts in for ours
    we will start setup tomorrow when we get the plexi case and spacers.
    www.rigotech.net/ultimate-starter-pi-cluster/

  • @RobertFHGroeneveld
    @RobertFHGroeneveld 8 років тому +1

    Have you tried tftp booting ? then you wouldn't need a SD card per pi ?
    www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/net_tutorial.md

    • @KF7IJZ
      @KF7IJZ  8 років тому +2

      Haven't tried it yet, but this would be a great 2.0 feature. This article wasn't written when I started doing all of this!

  • @oilczar
    @oilczar 4 роки тому

    Very cool, but way too hard

  • @cityofplant
    @cityofplant 7 років тому

    I honestly dont understand what it does..

    • @KF7IJZ
      @KF7IJZ  7 років тому

      It searches for binary pulsars by analyzing radio astronomy data from the Arecibo radio telescope - einsteinathome.org

  • @consig1iere294
    @consig1iere294 7 років тому

    I have watched bunch of RPi cluster videos. None of them mentioned the purpose of the cluster. What a waste of time and money lol.