AWS EMR Cluster Create using AWS Console | Submitting Spark Jobs in AWS EMR Cluster

Create AWS EMR cluster and Submitting Spark Jobs in AWS EMR Cluster using aws management console.
Git link for code
github.com/sau...
In this video, I have covered end to end life cycle of development EMR Cluster and submit Pyspark job using AWS Step
Create Up EMR Cluster
EMR Cluster configuration
Bootstrap action
Spark ETL
AWS Step Functions
IAM ROLE
Run Pyspark Script using EMR
Job Logs
EMR Cluster using Step
Amazon EMR is the cloud big data solution for petabyte-scale data processing, interactive analytics,
and machine learning using open-source frameworks such as Apache Spark, Apache Hive, and Presto.
I have also created videos related to AWS EMR.
create cluster using AWS CLI
• Create AWS EMR Cluster...
AWS EMR theory
• AWS EMR Tutorial | Ama...
DIFF AWS GLUE VS EMR
• DIFF AWS GLUE VS EMR| ...

[@WolfmaninKannada]

Bhai ❤ thank you so much you have explained so nicely even a noobie can understand and work on emr.....

[@shashwatkhare1086]

Sir you are super great, pehle maine python scripts me apne bucket ka naam edit nhi kia to error aa rhe the, lekin ab saari jobs run ho rhi hai, thankyou so much sir for this valuable information!

[@tinkuchanchal1238]

The way you explain every thing in detail is wonderful.

[@mashalkhan7699]

Cheers. I saw a first person that teach honestly.

[@ankushbehere7263]

Thanks Bhai bhut informative video hai aur sare concepts kafi ache se samze muze.

[@harishkanta3711]

Maine poori video pehle baar dekhi, great content and explanation. Thank you for taking concern and explaining things in easier way. Thanks again. Keep good work going on, am still amazed how you do not have more views.

[@TechnoDevs]

Thanks and welcome

[@sudippandit7051]

Great content and explanation!! Really appreciate of posting such kind of knowledge 😊

[@sakshi0070]

correct......easy explanation hai....1 yr se muje samaj nai aya tha job kaise submit hote hai aj all daubts clear hogaye

[@shubhamalsunde3230]

best content ever Sir your videos are so much helpful

[@FarooqKhan-d8c]

Great work, Sir.
Keep up the good work!

[@tejeswararaoankani]

Great content with detailed explanation. Thanks

[@akshaymuktiramrodge3233]

Thank you so much Saurabh. Great content.

[@electricalsir]

thanks man it very very usefull to use from long time serching for video like this who can exlplain evry htin in detail please make content for us i know this rquird more effort but othere hand this is very use full to us this conets was awesome thank you thank you so much

[@rohitgaikwad2266]

Thank you for this fantastic tutorial! It was very helpful.

[@devprasad609]

Well streamlined video and content.!

[@vinodvashistha4260]

Great work, Saurabh!

[@ShubhamPatil-x8r]

Thank You sir🤗🤗 your videos are of great help!!

[@nadianizam6101]

Excellent explanation

[@romi9442]

very informative and easy to learn.

[@TechnoDevs]

Glad to hear that

[@harshalalige9293]

Good one, can you create video on working with complete data pipeline with EMR or lambda !!

[@someshkumartiwari6330]

fantastic video

[@rajsolanki6865]

Waiting for next video

[@EmailacS]

Saurabh brother , you are doing a great job. Simple content easy to learn and grasp the topic. Expect more such videos from you. Can you make more such demo video on ETL pipeline.

[@tejaspise4638]

Really great work

[@surajghatol9040]

Thank you sir... Very clear explanation...
I subscribed to your channel..

[@thesuccessfactor383]

Mast Vdeo

[@udaynayak4788]

Hi Saurabh, this video is filled with knowledge and learning, one query in real time which deploy mode is more effective?

[@sakshi0070]

thank you sir dil se....plz ek bar aur spark submit command se karke batao if you have free time....

[@TechnoDevs]

thanks for comment . refer this video ua-cam.com/video/XsWnW7-8IGQ/v-deo.html

[@rajbir_singh0517]

Really great explanation.
One question ,I am creating an EMR cluster with a private subnet. Getting bootstrap error and description is bootstrap error while application provision.

[@TechnoDevs]

Thanks for watching video. Might be you are getting issue due to Firewall. check your security group rules once.

[@raviyadav-dt1tb]

Do you have any videos for sparks development?

[@ripponmangaraj782]

Amazing tutorial !!

[@prathapn01]

Is it possible to include SSH -i command using ec2 keypair -- to connect to the EMR cluster using bootstrap script ?

[@rajkumarichodari111]

That’s a killer video 🤩

[@harjeetsinghgoldy1]

Can we connect the spark master cluster to python fastapi using Pyspark???

[@ritumeena1623]

Impressive video 🙌🙌

[@YatriSpecial]

If i run emr cluster in free tier account. Will it cost money?

[@NinjaAnkit]

I am facing issue Terminated with Errors and and not able to create EMR cluster. Can anyone guide me?

[@lalitabehari5827]

You are awesome

[@nemaramsharma4437]

Wowwww 🤗

[@sumitkokhar3808]

Fantastic

[@saurav0777]

Thank you bhai

[@Rohit-Hs]

Grt!

[@pawangupta-mp5zp]

Thank you so much.

[@TechnoDevs]

You're welcome!

[@prasad6062]

great content

[@nikhildahatonde-r8n]

Thank you!

[@TechnoDevs]

You're welcome!

[@electricalsir]

thank

[@sumanpawriya702]

😍😍😍😍

[@DeeshuKumarPandit]

Thanks man for your effort, It helped a lot. But I'm stuck at a point where I'm getting an error. I need your help.

[@TechnoDevs]

Thanks for watching it. what help needed. please write in comment box . I will try my best

[@DeeshuKumarPandit]

@@TechnoDevs I have a python script which uses spark and connected to a database. I have also setup all dependencies and followed your steps to run on EMR. The script is running fine while running on driver node , as soon as my partitioned dataset goes to worker node it is giving error as - An error occurred while calling z:org.apache.spark.api.python.PythonRDD.runJob. , assert SparkContext._active_spark_context is not None
AssertionError. I'm trying it to resolve from last week and not able to resolve it yet, getting frustrated. Please help.

[@DeeshuKumarPandit]

Reply please.

[@TechnoDevs]

Send your script into technodevs13@gmail
.Com and also send the snapshot. I will try to replicate your issue in my system. Might be your SparkContext object not being properly initialized on the worker node.

[@RajeshKumar-re8tj]

How to open yarn in emr

[@raviyadav-dt1tb]

We’ll explain

[@aniketbhonde1864]

Can you please share that's file

[@mayankbajiya5977]

💯By💯

[@Prashantkumar-uh5uh]

Great content with detailed explanation. Thanks

[@electricalsir]

thanks man it very very usefull to use from long time serching for video like this who can exlplain evry htin in detail please make content for us i know this rquird more effort but othere hand this is very use full to us this conets was awesome thank you thank you so much

[@electricalsir]

thanks man it very very usefull to use from long time serching for video like this who can exlplain evry htin in detail please make content for us i know this rquird more effort but othere hand this is very use full to us this conets was awesome thank you thank you so much

[@electricalsir]

thanks man it very very usefull to use from long time serching for video like this who can exlplain evry htin in detail please make content for us i know this rquird more effort but othere hand this is very use full to us this conets was awesome thank you thank you so much