@Peter Lustig Can you clarify how this isn't decentralised storage? For those of us that are new to this sort of stuff, what would you consider to be decentralised storage if you don't think that this is considered to be decentralised storage?
This is a nice, simple, clear, 20,000 ft overview of parallel/distributed storage. It would be nice to see a comparison between Ceph, Lustre, pNFS, and Gluster to see what are the differences between each of these different technologies and the relative pros and cons of each.
3:51 -- I love how every time someone talks about Ceph they have a moment like this. A little Freudian slip. "This is the approach that WE favor.." --throat clear-- Sometimes instead you'll catch a little chuckle, as if the speaker is thinking to themselves how awesome Ceph is, as if they can't help themselves. (they're better than I am, they know a better way than I do and they straight up know it... and for once I actually like that. [Because I can learn from it.]) And it's almost like it's secretly a joke if you aren't already fully aware that it's the future.. I was worried about the unknown but I'm starting to gain a comfortable understanding. (I love UA-cam learning.) Enough to deploy a 4 node cluster. Little clues show me that we're still only in the beginning with Ceph but I'm also pretty late to the party. Production ready but the best is still in store. I'm ready to party now! ... this video in particular helped me realize that I have 1 more node that I can add to my cluster. Time to put my resources to REAL use. Thank you.
determinism means that a specific key always generates a specific value no matter how many times the algorithm is rolled. This basically means there is no randomness involved in the hashing algorithm itself.
I've been searching all day for a decent explanation of CEPH and this is by far one of the "BEST" I have seen. Thank you very much. The light bulb just came on and it's burning bright. How can I get more of that type of training and explanation?
@@axiom1650 yeah ... the description is more detailed. I think the website has some collections of buzzwords. Good for business leaders, not really that great for engineers.
I could see some correlation between the Dynamic Subtree Partitioning which CEPH uses and the DNE (Distributed Namespace) used in Lustre. Do they branch out from the same origin?
It remindes me of the 432 chips by Intel and distributed computing and distributed voted i/o processing. Does it allow for offline and non-deterministic osd in the factoring of recovery/deletion/renaming options ? I will look on the sites you reccommended at: ua-cam.com/video/7I9uxoEhUdY/v-deo.html
The only part that seems especially dated now is mention of each OSD running on top of a normal filesystem, which is less common these days for new deployments. Be sure to read up on Bluestore vs. Filestore. Bluestore (OSD directly atop block devices, with optional separate WAL / write-ahead-log) can have much better write performance
From what I gathered on the Openstack documentation, Cinder allows a driver to be provided for how to write to the disks. Cinder will manage control of the disks while the Ceph RBD driver will manage how to communicate with the drives by managing a storage pool. With Cinder you can provide a batch of drives to be the target for the storage pool that Ceph will manage. In the following hyperlink there is an example given for a scenario where you might want to create three tiers of SSDs: "Gold", "Silver", and "Bronze" respectively that would correspond to the speed of the SSDs and number of replications made for backups. superuser.openstack.org/articles/ceph-as-storage-for-openstack/#:~:text=Cinder%20is%20the%20block%20storage,as%20gold%2C%20silver%20or%20bronze.
only issue i see here is as you scale out you scale failure points instead of 1 huge storage appliance with redundancy you have tons of small appliances with no redundancy
Why can’t anyone explain CEPH without the BLOATED explanation??? WTF? Can’t you just go step by step on how to install it and configure it and that’s it??? I’m a web Developer and when I explain PHP or JAVA I don’t explain HOW they work, I explain how to get it going!
THE best explanation of Ceph I have seen so far. Great work!
Simple and clear explanation
@Peter Lustig
Can you clarify how this isn't decentralised storage?
For those of us that are new to this sort of stuff, what would you consider to be decentralised storage if you don't think that this is considered to be decentralised storage?
This is a nice, simple, clear, 20,000 ft overview of parallel/distributed storage.
It would be nice to see a comparison between Ceph, Lustre, pNFS, and Gluster to see what are the differences between each of these different technologies and the relative pros and cons of each.
3:51 -- I love how every time someone talks about Ceph they have a moment like this. A little Freudian slip. "This is the approach that WE favor.." --throat clear-- Sometimes instead you'll catch a little chuckle, as if the speaker is thinking to themselves how awesome Ceph is, as if they can't help themselves. (they're better than I am, they know a better way than I do and they straight up know it... and for once I actually like that. [Because I can learn from it.]) And it's almost like it's secretly a joke if you aren't already fully aware that it's the future.. I was worried about the unknown but I'm starting to gain a comfortable understanding. (I love UA-cam learning.) Enough to deploy a 4 node cluster. Little clues show me that we're still only in the beginning with Ceph but I'm also pretty late to the party. Production ready but the best is still in store. I'm ready to party now! ... this video in particular helped me realize that I have 1 more node that I can add to my cluster. Time to put my resources to REAL use. Thank you.
Best tech presentation I seen in a long time
Great presentation. Exactly what I was looking for.
determinism means that a specific key always generates a specific value no matter how many times the algorithm is rolled. This basically means there is no randomness involved in the hashing algorithm itself.
I've been searching all day for a decent explanation of CEPH and this is by far one of the "BEST" I have seen. Thank you very much. The light bulb just came on and it's burning bright. How can I get more of that type of training and explanation?
lol. how the description text below this video is better than the whole ceph website at explaining what ceph is and does.
Still true 3 years later
@@axiom1650 yeah ... the description is more detailed. I think the website has some collections of buzzwords. Good for business leaders, not really that great for engineers.
The best intro. Thanks
Great talk. Thank you.
Awesome information. Thanks a lot!
Very cool. Will be the future!
Great explanation, easy to understand
Thank you for posting this great talk
Some good info, looking forward to setting up my first CEPH cluster
I could see some correlation between the Dynamic Subtree Partitioning which CEPH uses and the DNE (Distributed Namespace) used in Lustre. Do they branch out from the same origin?
I think the description below the video is a part of science public paper. Have any one help me find that title or link on google scholar?
Huh. File systems should be fully deterministic. The same data in the same file should put it in the same blocks. That would allow reduction again.
fantastic talk
It remindes me of the 432 chips by Intel and distributed computing and distributed voted i/o processing.
Does it allow for offline and non-deterministic osd in the factoring of recovery/deletion/renaming options ?
I will look on the sites you reccommended at: ua-cam.com/video/7I9uxoEhUdY/v-deo.html
yeah in overall thats a pretty comprehensive introduction to ceph, nice
This is still a really good presentation
The only part that seems especially dated now is mention of each OSD running on top of a normal filesystem, which is less common these days for new deployments. Be sure to read up on Bluestore vs. Filestore. Bluestore (OSD directly atop block devices, with optional separate WAL / write-ahead-log) can have much better write performance
where can i check the slides used in this great keynote ?
Great talk...but Now I'm baffled as to how CEPH works with the other Openstack storage solutions like Swift and Cinder...?
From what I gathered on the Openstack documentation, Cinder allows a driver to be provided for how to write to the disks. Cinder will manage control of the disks while the Ceph RBD driver will manage how to communicate with the drives by managing a storage pool. With Cinder you can provide a batch of drives to be the target for the storage pool that Ceph will manage. In the following hyperlink there is an example given for a scenario where you might want to create three tiers of SSDs: "Gold", "Silver", and "Bronze" respectively that would correspond to the speed of the SSDs and number of replications made for backups.
superuser.openstack.org/articles/ceph-as-storage-for-openstack/#:~:text=Cinder%20is%20the%20block%20storage,as%20gold%2C%20silver%20or%20bronze.
really good stuff
This is cool. What is max performance of Ceph? Can it read/write at 50 GB/sec to a clustered fs?
Anyone have the slides for this talk?
www.slideshare.net/buildacloud/ceph-intro-and-architectural-overview-by-ross-turk
Are hard links supported? (not softlinks) Its required for things like Cassandra and other high bandwidth file persisted products.
Since RADOS sits on top of a pre-existing filesystem such as ext4, xfs, btrfs, etc. hard links are supported.
Good one. How the fragmentation is handled
RDOS sits on top of a filesystem;
only issue i see here is as you scale out you scale failure points instead of 1 huge storage appliance with redundancy you have tons of small appliances with no redundancy
Thank you!
Nice talk.
I cannot find ceph.com/qsg
This is the same old 2012 talk.
Human and clay brah
Cool
vox news opt/ceph intro&archtectural overview
Why can’t anyone explain CEPH without the BLOATED explanation??? WTF? Can’t you just go step by step on how to install it and configure it and that’s it??? I’m a web Developer and when I explain PHP or JAVA I don’t explain HOW they work, I explain how to get it going!
Great talk. Thank you.
Thank you!