You have managed to take the written article published by google and explains Google File System and MapReduce in a very good, clear way so anyone can consume it. good job!
Excellent Explanation sir. Loved it. Wanted to learn more on it. I am now big fan of yours. Watched most of your videos and trying to read as much possible from you sir. Please consider to share more videos on this topic.
FWIW the google paper left out some of the details you needed for full resilience. We had to learn those through things going wrong -e.g the infamous facebook cascade failure where, once a critical mass of servers went offline, the recovery work took the rest down
Very nice explanation. Thank you so much for the same. I had a few questions - 1. Which type(s) pf write operation(s) can a client application perform on the files. In other words, what does GFS supports in terms of writing into files? 2. Where in the file the writing can be done? 3. How much data can be written there and by how many applications at the same time and at the same location? 4. In what type of memory is the metadata stored on the master
Bit late but hopefully others find it useful: Regarding (1) & (2), GFS was primarily aimed at applications which involved appending data to a long-running file, with random location writes happening rarely (if at all). For (3), the limits would depend on the actual implementation - this was just a theoretical paper on GFS. But yes, it is meant to support concurrent writes. And regarding (4), the Master keeps a track of all the files on the GFS, a list of chunk handlers that make up each file, and the list of chunk servers that hold a particular chunk (there are multiple servers for redundancy) Additionally the master keeps a log of all the changes that were made (along with checkpoints), so that it can recover from crashes
Thanks for the great content. But I haven't seen any recent videos from you, it'd be great if you start making these videos again, they're just wonderful.
🙏 You are the Best Teacher that I have ever seen.🥇, A classical method of approach = Superb indeed.👌.🎯Please Could you kindly teach us Flutter & Dart & Go lang & Scala too . Pleeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeese Sir..... 🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏
hey, thanks so much for reaching out.. just been busy with new job, and added laziness :( I am researching for NoSQL series, but will take time. Regarding funding, absolutely.. I love teaching and hope to someday do this fulltime. Might start UA-cam subscriptions for earning. Let me know your thoughts, is ~130 INR a month something everyone would go for, or should it be course based fees on a separate website?
@@DefogTech , this channel is my first point of reference for any cs related topic , things that make your videos unique are the awesome slides , quality content & very crisp explanation. I myself being a software professional at Adobe frequently watch your videos, so yes youtube subscription seem to be a viable option. Hoping you to resume soon so that we can make best amidst the lockdown. :)
You have managed to take the written article published by google and explains Google File System and MapReduce in a very good, clear way so anyone can consume it. good job!
This is how one should teach. Loved it!
Best explanation of GFS on UA-cam. This is how one should teach. Wish the channel had posted more such videos.
Now I realsed that my teacher must have watched this video before lecture .
amazing video. the finest tutorial of GFS on youtube.
Thank you for getting into distributed systems realm. Would love to see some videos about the workings of cassandra.
high quality video. your explanations are clear as crystal
This explanation and the voice clarity is too good, thanks a lot for the video
Excellent Explanation sir. Loved it. Wanted to learn more on it. I am now big fan of yours. Watched most of your videos and trying to read as much possible from you sir. Please consider to share more videos on this topic.
Thank you so much for your kind words! Planning to do more videos soon
Excellent video clearing the basics of GFS.
Your explanation is GREAT.
Very well systematically explanation.! Great Work! Thank you. God bless you.
Your videos are crisp and very clear. Thanks. Keep up the good work😊👍
Amazing video, super clear.
So a simple explanation. Amazing.
FWIW the google paper left out some of the details you needed for full resilience. We had to learn those through things going wrong -e.g the infamous facebook cascade failure where, once a critical mass of servers went offline, the recovery work took the rest down
Explained clearly. Thanks.
Very well explained. Thanks for the content and effort
Very nice explanation. Thank you so much for the same. I had a few questions - 1. Which type(s) pf write operation(s) can a client application perform on the files. In other words, what does GFS supports in terms of writing into files? 2. Where in the file the writing can be done? 3. How much data can be written there and by how many applications at the same time and at the same location? 4. In what type of memory is the metadata stored on the master
Bit late but hopefully others find it useful:
Regarding (1) & (2), GFS was primarily aimed at applications which involved appending data to a long-running file, with random location writes happening rarely (if at all).
For (3), the limits would depend on the actual implementation - this was just a theoretical paper on GFS. But yes, it is meant to support concurrent writes.
And regarding (4), the Master keeps a track of all the files on the GFS, a list of chunk handlers that make up each file, and the list of chunk servers that hold a particular chunk (there are multiple servers for redundancy)
Additionally the master keeps a log of all the changes that were made (along with checkpoints), so that it can recover from crashes
Great job!
Such a good explanation.
Very well explained and articulated.
Amazing video. Thank you so much
Very nicely explained. Thank you.
Thanks for the great content. But I haven't seen any recent videos from you, it'd be great if you start making these videos again, they're just wonderful.
Exceptional sir.
Perfect explanation .Thank you for this.
Very good explanation!
Can't wait for the BigTable and DynamoDB episodes!! Also can you do one on pagerank?
Nice! Very easy to follow
great content , thank you
Very nicely explained.Thanks a lot :)
Well explained ,thank you.
Very well explained , thanks
Thank you! a very clear explanation.
Great explanation! Thanks.
If each chunk has 3 replicas, then will there be a total of 4 such chunks ?
its architecture somewhat similar to napster ( I think napster was ahead of its time)
Great video. Thanks for making. There is a typo in the last slide about chunk size being 64kb instead of 64mb :)
Great explanation
I really like your videos and regular follower of your channel. Please publish a video on Cassandra architecture. If possible.
Yes sir! Cassandra is similar to Google Bigtable. Will cover it soon.
Hi , Can you make video related to transaction management ,propagations ,isolations,2-phase commit, deadlocks in transactions..
🙏 You are the Best Teacher that I have ever seen.🥇, A classical method of approach = Superb indeed.👌.🎯Please Could you kindly teach us Flutter & Dart & Go lang & Scala too . Pleeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeese Sir..... 🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏
Thank you so much for the kind words. Definitely plan to continue creating videos, and also covering more topics like Dart and Go
Awesome 👍
Really good video.
Good work
Well done,thank you
Thanks, can you also explain MapReduce please?
Absolutely. That video is coming tomorrow!
Awesome
Can you please do Dynamo paper
Thank you so much..
waw u explained so smilply
Good one buddy👏
Thank you buddy! Btw, Congrats on 40K, that was fast!
Defog Tech thanks dude🙏🏻
I m happy to see 2 good Java content creators at one place. Continue the legacy. :)
Sanjay Kantheti 🤘
Gold !
Very helpful.
Hey , request you to please make new video.
Eagerly waiting for your new videos.
Thank you for your video
Hey , why have you stopped uploading videos....let me know if you need any funding
hey, thanks so much for reaching out.. just been busy with new job, and added laziness :(
I am researching for NoSQL series, but will take time.
Regarding funding, absolutely.. I love teaching and hope to someday do this fulltime. Might start UA-cam subscriptions for earning. Let me know your thoughts, is ~130 INR a month something everyone would go for, or should it be course based fees on a separate website?
@@DefogTech , this channel is my first point of reference for any cs related topic , things that make your videos unique are the awesome slides , quality content & very crisp explanation. I myself being a software professional at Adobe frequently watch your videos, so yes youtube subscription seem to be a viable option. Hoping you to resume soon so that we can make best amidst the lockdown. :)
Epic!
Gr8
Perfect.
The chunk is **64KB** NOT MB
It is 64MB. GFS is used mainly for large files with append-only structure. In the paper there is good trade-off of why they chose 64MB.
MB
is the master something like a zookeeper?
No, it's a single instance backed by a secondary one unlike concensus based one like zookeeper
👍🏿👍🏿👍🏿
💙
Great explanation.
Awesome explanation
Very clear explanation. Thank you!