transformation and action in spark
Вставка
- Опубліковано 5 жов 2024
- In this video I have talked about transformation and action in spark in great details. please follow video entirely and ask doubt in comment section below.
Directly connect with me on:- topmate.io/man...
For more queries reach out to me on my below social media handle.
Follow me on LinkedIn:- / manish-kumar-373b86176
Follow Me On Instagram:- / competitive_gyan1
Follow me on Facebook:- / manish12340
My Second Channel -- / @competitivegyan1
Interview series Playlist:- • Interview Questions an...
My Gear:-
Rode Mic:-- amzn.to/3RekC7a
Boya M1 Mic-- amzn.to/3uW0nnn
Wireless Mic:-- amzn.to/3TqLRhE
Tripod1 -- amzn.to/4avjyF4
Tripod2:-- amzn.to/46Y3QPu
camera1:-- amzn.to/3GIQlsE
camera2:-- amzn.to/46X190P
Pentab (Medium size):-- amzn.to/3RgMszQ (Recommended)
Pentab (Small size):-- amzn.to/3RpmIS0
Mobile:-- amzn.to/47Y8oa4 ( Aapko ye bilkul nahi lena hai)
Laptop -- amzn.to/3Ns5Okj
Mouse+keyboard combo -- amzn.to/3Ro6GYl
21 inch Monitor-- amzn.to/3TvCE7E
27 inch Monitor-- amzn.to/47QzXlA
iPad Pencil:-- amzn.to/4aiJxiG
iPad 9th Generation:-- amzn.to/470I11X
Boom Arm/Swing Arm:-- amzn.to/48eH2we
My PC Components:-
intel i7 Processor:-- amzn.to/47Svdfe
G.Skill RAM:-- amzn.to/47VFffI
Samsung SSD:-- amzn.to/3uVSE8W
WD blue HDD:-- amzn.to/47Y91QY
RTX 3060Ti Graphic card:- amzn.to/3tdLDjn
Gigabyte Motherboard:-- amzn.to/3RFUTGl
O11 Dynamic Cabinet:-- amzn.to/4avkgSK
Liquid cooler:-- amzn.to/472S8mS
Antec Prizm FAN:-- amzn.to/48ey4Pj
Hello Manish,
I am presently working as a Data Engineer at Hyland, and it is your theory and practical playlists that I've been following to get me up and running with regard to learning Spark. Your content is very useful not just to me but to many professionals out there.
Please do keep up the good work-although some may be casually watching, there are plenty of us who are seriously following through in the long term. Your work is making a real difference, and I really value your commitment.
Thank you for all that you do, and please continue the great work!
Best regards, Gaurav.
Literally i am watching it at 3 am in the morning and my eyes are not blinking. That much interesting ur way of teaching is. I hope you will not stop creating the content because of demotivation of any type let’s say less number of subscribers ( i can feel it’s disheartening sometimes when u put that much efforts) . You just increased my crave for learning. Dil se shukriya bhaii ❤
same happens with me bro....these contents are diamonds....
@@tusharkantighosh4641 even with me also i was almost to quite spark but after this video i got much interest in spark
Give this guy some medal for Quality
Because of you I’m confidently attending interviews. Thank you sooo much
Hi Sir, I am very thankful that I came across your playlist . Trust us many of us are implementing whatever things you are explaining so don’t be demotivated. We understand the amount of extra work you are doing for us and we won’t let your effort go on vein. Your playlist is a hidden gem.
Hello Sir, You are doing a great job and there are very few humble and selfless Data Engineers like you who could dedicate their valuable time for sharing their knowledge and experience. Please continue creating these videos on PySpark. I am preparing for Data Engineering Interviews for the the 3rd time i.e. for my 3rd switch of companies and interviews are getting scheduled and looking upto this series for cracking them.
Thank you in advance for creating such informative and detailed videos.
Jai Mata Di! 🙏
to be honest the way is explain the fundamental is amazing, for a beginners the course is really worth to watch instead of buying some random expensive courses.. thank you bother for such amazing and useful content ..
👍👍👍
I am preparing for the interview, and I can say you are playing main part. I do make note and follow all steps. Thanks for the supper.
Please continue this series brother, I just started watching this and found it to be fantastic resource. Keep up the good work.
Bro your content is very good. We need tutors like you. Hats off to you.
Interview k liye bahut benifit ho raha hai ❤
Most underrated Teacher on UA-cam I wish you all the best and one day you will be famous because the way you teach is awesome.
Aap hame jaante hai itna hi kaafi hai😂😂
Thank you manish bro,
Your series is really good,. I started watching it today.. Please keep making it.. really helpful.
sir g dont loose motivations, we are fallowing with all dedication, sometime people with all the implementation knowledge also see your video for revision or to brush up basics, thank you for all the hard work and giving these great content in hindi
Fantastic explanation, understood the concepts very clearly. Thanks for making such amazing content.
Bhai..you are the best teacher for data Engineer... concepts are very clear.....thanks
sach mei aapko bahut ache se aata hai ... one place for all IT courses
Hi Manish, Thanks for your efforts. It's really good.
very helpful series sir thank you for giving us your imp time ❣
Sir I have got you link in the last video, sir i am following you both playlist practical and theory. please don't stop videos
The way you explained everything made it so simple to understand. Earlier it used to be difficult for me to understand the architecture concept and I would forget that in sometime, but now I feel I wont forget it as this video felt like a story and everything was explained in detail and in very simple way. Its such a great content.
you should make another video in which you address whether the spark.read methods are transformation or action.
I do not have words to praise this masterpiece. Keep creating this awesome content.
Sir you are doing really awesome job. Please making all your videos. As its will help to others in future as well. I am working as ETL tester and still i keep watching your videos to get some insight on data engineer roadmap.
Bro you are doing a great job keep going..
I am watching and following from past 1 week
Very informative video. Must watch for someone beginner or intermediate in pyspark.
hi , im learning well through your course . I literally realised after watching this video , about the csv file , because i had to literally make a new one using the data. Thanks for motivation and Please be Motivated !!😊
Great explanation Manish. Thank you very much for such informative videos
Manish You are simply awesome. i'm glad that i have found best video's on UA-cam to learn spark.
Manish bhaiya your videos are very valuable.
I learn new things every time.
Very Informative.
Really appreciate your efforts Manish! You teach really well. I work as a Data Engineer in one of the Big 4 and go through your videos whenever i need understanding of topics in depth.
We got the file and used it as you taught. Didn't feel need to comment this also, after you are giving this much for free. Thanks for awesome content.
bht ache se undrstand horha bhai
Your inputs are really helping in understanding complicated things easily, thank you
Most informative and needful video. thank u so much.
Ab tak bhaut sahi chl rha hai jo aap smjhaye 100 marks aayenge ab hume
Wonderful explanation bro. Read couple of articles about Narrow and Wide transformation but could not grasp it completely but now after watching your video things got clear.
Thanks a lot for your course
Phenomenal lectures loved it✌️👍
No no sir .. I m honestly following your course.
Thank you for sharing your knowledge among us and I am really enjoying learning the concepts. I am sure these informative videos are helping a lot of people. Keep on teaching us!!
I mean to say in my last comment, you have explained very well with simple examples
Sir, Awesome explanation
Thank you so much
i was there almost to quite spark but after this video i got much interest in spark
Your teaching is really good and helpful.thanks alot
You are awesome. I make it simple to understand. thanks
Sir aapke lecture se bahut sare doubt clear ho raha hai
You are a gentle man seriously :)
your kadva sach is very very true
धन्यवाद मनीष भाई। 🙏
Best teacher ever met , thank you sir
Very very helpful content.. Please keep it up
Most informative @ Thanks Bro
I am Following your course, Where you shared spark Documentations, I didn't see in Anywhere
Amazing work. Thanks for lectures...👏
content is very good
Sach bat hai sir
You are doing great job. Your lectures are awesome.
You are outstanding. I love the way you are teaching. Thanks Manish bhai explaining along with flowchart👏
Manish bhai is real 💎!!
Very helpful sir.. Thank you..
Manish Ji....excellent hands-on course.. Keep adding and let your channel grow !
Manish bhai bhot ache se pdha rhe ho.......lge rho....hm bhi apke sath lge hue h....Maine abhi start Kia pr complete zrur krunga
sir in lec 5 there was no mention of error records.
You make each and every topic very simple to understand. Thank you so much and keep up the work. 👍
If possible, please make some videos on Kafka, Kubernetes, Azure, Databricks. Your videos are really very helpful!
Appreciate your hard work.
Sir App Azure ke bhi kuch session loge to bahot acha hoga..kyuki apki padane ki technique bahot achi hai..
Amazing explanation Sir 🤩
This series is phenomenal.❤
Sir your videos are very helpfull
I want to tell you that I am following your videos and also implementing them. Whenever possible I saw links from your video and got the doc or data file. Once, I couldn't find data so I asked you :).
Please don't loose motivation for doing these videos!!
Very well explained! I have a doubt though -
What if for example - One partition can handle only 100 records of data but the original input data consists of 150 records all having the same ID. In that case how will the groupby() function work because it wont be possible to bring all data in to the same partition even after shuffling
What is the answer
Nice Video, got lots of learning
Hi Manish Bhai,
csv file was downloaded by the steps which you guided, you are doing great work, i am following and applying all the concepts which you taught.
Good
Outstanding tutorial....maja aa gaya sir..@manish
Hi Manish, really good information, I am going though one by one, hope you have covered all the databricks topics in your videos and please provide us the PPT as well, thank you for your support. Vineet Goel
this content should be paid
well explained
good morning sir, ur lectures are great
Manish Bhai, I would like to request you, please make a real time end to end project. So, that it can help us during the interview.
Awesome
Sir one question.
At 14:03, when you showed output of question 1 (list of people with age less than 18)
Here, as we can see the output contains redundant rows, so in this case once both the executors have returned their outputs which are kind of appended on top of each other in the output you showed - what happens after this step to remove the redundancy? Is a ‘distinct’ run by spark?
Very nice
Appreciate it truly
last video mein aapne spark ka architecture padhaya tha ...corrupt records k baare mein nai bataya tha
Please continue this series
Sure
great bhai bahut acha concept padha raheho Azure Data Brick ka v lekar aawo aap series
nice content sir
Last video kaha hai Currerpted record wala
Please share link last video
You are just awesome ❤
keep it up manish
Awesome 👏
Kya yeh possible nhi h ki same ids ka data same container me pehle se hi move ho jae taki data suffeling na krni pade aur wide dependency ki zarurat hi na pade
Sach me ab promise to myself karta hu ab practice karunga
Sir, should i finish your theory lecture of spark and then follow practical videos for practice ?
I like your teaching.very nice.i am seriously looking for job
Can u guide me
Good initiative🙂