Please Subscribe🔔, like, 👍and share 📣the video to unlock the following bonus videos. Let me know which video topic you want me to cover here: forms.gle/UMp4GA3krcSMMWzy9 🎁 Bonus Videos: ▶ Hit 50,000 views to unlock a video about building an end-to-end machine-learning pipeline with PySpark. ▶ Hit 100,000 views to unlock another video video about end-to-end spark streaming.
Simple yet very informative and helpful content..please dont get discouraged by less views.. slowly more and more people will come and watch your videos
Wasn't expecting a 50 min video to be this informative! Covered all the major topics related to PySpark. Truly, a well structured video and quality content. Thanks!
Have to question the AI voice. But honestly, with 99% of tech tutorials on UA-cam being presented by very, very thick Hindi accents, it's probably for the best.
I found this to be an excellent guided tour, but only because I've gone through a lot of tutorials on webpages, books, and videos. PySpark is so vast and diverse that, after seeing all these other tutorials, one needs a video like this one to knit them all together.
this is excellent! could you make one on LakeSail's Sail? supposedly its 4x faster and more cost effective than Spark since its built on rust. would love to learn more.
This is the latest and most beginner friendly tutorial regarding apache spark. As an aspiring data engineering this is absolutely helpful! If you ever have free time, I would like to request creating a tutorial series regarding docker, dbt, and kafka, also if possible could you add more tutorials for dagster. As a student, out of all the tutorials in youtube I was able to follow you tutorial the best. Thank you for the airflow and pyspark playlist. I hope you still continue recording 😊
First, thank you for your video, it is really helpful for people who know nothing about PySpark like me. But I have searched for spark installation, and there are many tutorials that install very complicated above Hadoop Cluster. And it seems like you installed it so easily, I wonder if this is a version of installing Spark standalone, right?
It is standalone. To get started, it is enough. And for the real project, you wouldn't need to know how to set up the cluster yourself. You can use Spark in clouds like databricks instead.
Thanks for the informative video. The pyspark environment is not activating on my windows 10 machine. Can you please help me what might be missing '.pyspark-env' is not recognized as an internal or external command, operable program or batch file.
I am unable to get the jupyter notebooks to work. It keeps complaining about py4j. I am able to run the commands in a terminal tho. Not sure what the setup is supposed to be. I wish this video started with how to install everything.
Please Subscribe🔔, like, 👍and share 📣the video to unlock the following bonus videos. Let me know which video topic you want me to cover here: forms.gle/UMp4GA3krcSMMWzy9
🎁 Bonus Videos:
▶ Hit 50,000 views to unlock a video about building an end-to-end machine-learning pipeline with PySpark.
▶ Hit 100,000 views to unlock another video video about end-to-end spark streaming.
Hands down! The best Pyspark tutorial that I've come across.
Thanks!
Simple yet very informative and helpful content..please dont get discouraged by less views.. slowly more and more people will come and watch your videos
Thank you so much 🙂
Wasn't expecting a 50 min video to be this informative! Covered all the major topics related to PySpark. Truly, a well structured video and quality content. Thanks!
great explanation! thanks!
Brilliant! Invaluable info. Thank you!
Have to question the AI voice. But honestly, with 99% of tech tutorials on UA-cam being presented by very, very thick Hindi accents, it's probably for the best.
I found this to be an excellent guided tour, but only because I've gone through a lot of tutorials on webpages, books, and videos. PySpark is so vast and diverse that, after seeing all these other tutorials, one needs a video like this one to knit them all together.
Glad it helped! ☺️
Thanks! This helped me set this up locally. I was able to run it on Collab and Kaggle but not locally.
Thanks a lot for the clear and structured explanation, so helpful to grasp the basics of the Spark framework!
You are welcome 🤗
Thanks for this wonderful tutorial. Eagerly waiting for the Pyspark for advanced tutorial.
this is excellent! could you make one on LakeSail's Sail? supposedly its 4x faster and more cost effective than Spark since its built on rust. would love to learn more.
Very detailed tutorial, hope to see more PySpark ML and Spark Streaming. Thank the author very much!
Thank you! 🤗
This tutorial was pretty solid. Thanks.
27:34 - "rdd.foreach(lambda x: print(x))" did not print for me - instead suggest: "for x in rdd.collect():
print(x)"
This was super helpful, thanks!! It answered all of my questions & provided me with everything I needed to know!
Excellent work. Thanks so much for the content.
This was good, thanks! The screen was super blurry on that first section in Jupyter. But after that it was fine.
This is the latest and most beginner friendly tutorial regarding apache spark. As an aspiring data engineering this is absolutely helpful! If you ever have free time, I would like to request creating a tutorial series regarding docker, dbt, and kafka, also if possible could you add more tutorials for dagster. As a student, out of all the tutorials in youtube I was able to follow you tutorial the best. Thank you for the airflow and pyspark playlist. I hope you still continue recording 😊
Thank you for your feedback. I will definitely consider that! 😍
Thank you brother, this is the best Spark introduction.
Thank you!
Really nice overall Tutorial! Thanks for creating it
Thank you!
excellent video, please do hesitate to do more like these
13:49 give error for me "spark not defined"
this was super helpful! thank you for making this video!
You are welcome 🤗
First, thank you for your video, it is really helpful for people who know nothing about PySpark like me. But I have searched for spark installation, and there are many tutorials that install very complicated above Hadoop Cluster. And it seems like you installed it so easily, I wonder if this is a version of installing Spark standalone, right?
It is standalone. To get started, it is enough. And for the real project, you wouldn't need to know how to set up the cluster yourself. You can use Spark in clouds like databricks instead.
@@coder2j When we set SPARK_HOME variable, is pyspark library in your .spark-env could recognize that Spark and run above that?
amazing tutorial, Thank you!
Glad it was helpful!
Do i need to create a new env after closing or restarting a session ?
I have the error of : Py4JJavaError: An error occurred while calling o42.showString. How can I solve this ?
Thanks for the informative video.
The pyspark environment is not activating on my windows 10 machine. Can you please help me what might be missing
'.pyspark-env' is not recognized as an internal or external command,
operable program or batch file.
Did you set the spark path correctly?
Thank you for sharing it. Helped me a lot.
You are welcome 🤗
@@coder2j ♥
Thank you a lot, that's super helpful
Amazing video❤
Even though I've paid $20,000 to University still this video holds more value than that of learning in the class. Thank you @coder2j
Thanks for your comments! Glad it helped ☺️
I am unable to get the jupyter notebooks to work. It keeps complaining about py4j. I am able to run the commands in a terminal tho. Not sure what the setup is supposed to be. I wish this video started with how to install everything.
U need to set environment variable in the paths for java. Type it in Google it will give u steps
df.show() is throwing an error
amazing as alaways
Thanks 🙏
Very great tutorial, I hope you can launch the tutorials of Spark ML and Streaming asap. Thanks
Thanks. Stay tuned!
Thank you for you tutorial. It is amazing 👏
You are welcome 🤗
Thank you
Could you please share this PPT.
Hey, are you using any AI for voice modulation or content creation.?
nice content though
I am using the text to speech for the audio. Do you enjoy it or find it annoying?
@@coder2j content is very nice,tts sounds robotic
Thanks for your honest feedback. Would you enjoy it more if I use my own voice?
Absolutely! 💯
so good video
Thank you.
Thank you. :)
windows version bugged or something. Can't run df.show() without weird error codes. Ah well. on to another tutorial I guess. Thanks for trying
Thank you.
Where is the pyspark ML? you promised :'(
👍👍👍
9:51
we need apache nifi
BEST
Gonzalez Carol Gonzalez Melissa Miller Kevin
Mason Terrace
Deepfake, or not, keep responding to me using same word over and over again
Thank you for not to be a indian voice.
first thank you for your video. Just one thing you never say nothing about the file spark-defaults.conf and im having an error that i can fix it