How to Read Spark Query Plans | Rock the JVM

[@gilesbbb]

This material hits the sweet spot for me. Thank you.

[@sumanto_pal]

best video on spark ever

[@kapilsharma20]

@Daniel, an absolutely genuine content and in a very straight forward and step by step explanation, Thank you so much for this.

[@vash47]

Great video. I feel more comfortable now reading query plans. It's still tough!

[@ignaciomonetto6246]

You make us see the things simpler. Great job!

[@atech_guide]

Great Explanation Daniel :)
I have a request for a similar video on Spark UI Please

[@svapneel1486]

Great Video. Explaining complex concepts so simply. Fantastic!!

[@slipcatchsnatcher]

mindblown! great content, and indepth explanation!

[@omerziv8507]

Thanks! Great explanation!

[@doshimihir1]

Awesome videos.. looking forward to the next one..

[@ladakshay]

Superb Video! Query plans looked difficult to me before, but you made it simple. Thanks!!!

[@rockthejvm]

That's my goal - share it with other people that might find it useful!

[@shyamsundarrajasekaran8392]

Subscribed😍 Wishing you all the best nice explanation daniel

[@Jay-m8n7i]

Thank you for clear cut explanation ☺️

[@urxword]

You are so helpful, thank you so much for the explanation!

[@cajaykiran]

Thanks for the quality content, very very helpful 👌

[@prashantdhaundiyal2209]

Well done daniel. It was nice to know explain.

[@dataarchitect]

A very good explaination !

[@rockthejvm]

Glad you liked it!

[@ethanquenum4778]

Just amazing! Keep it up :)

[@chandanprakash15]

very nicely explained, great video

[@marclamblin6211]

@Daniel FYI In the description Link "Written form to keep for later" is broken

[@rockthejvm]

Fixed - there was a / at the end dammit

[@learn_technology_with_panda]

Nicely explained

[@murthyspec2002]

wonderful explanation!!

[@rahulberry4806]

nice and crisp explanation

[@rockthejvm]

That's my goal!

[@mootazmansouri7880]

awesome videos !
keep them going

[@rockthejvm]

More incoming!

[@DevTipsForYou]

Good explanation

[@technologyexplorer9193]

Precise explanation.
Can you also please tell how to make a full outer join in NON-ANSI way.

[@vinitsunita]

Awesome video

[@freetrainingzone7867]

Very Nice explanation Daniel. Is it possible to walk us through a real world use-case end to end using SPARK SQL with mild complexity.

[@rockthejvm]

Will do!

[@ashishlimaye2408]

Great Video!

[@rockthejvm]

Glad it helps!

[@rishigc]

Quick question - For Stage 1 and Stage 4, why is Spark using 6 partitions ? How does Spark derive the number of partitions which it will use to create a dataset or read an existing dataset ?

[@rockthejvm]

Unless previously determined, Spark will take the number of partitions equal to the number of cores on your machine.

[@rishigc]

@@rockthejvm Hi Daniel, how can I find the number of cores my cluster has (thru Spark UI etc) and how many cores are being used by my application. Also, could you pls let me know if I can increase the number of cores from the %sql prompt ?

[@naveenu8150]

Hi, Could you please make video on SPARK UI

[@rockthejvm]

Check out the video on the DAGs! If you're interested in learning about the Spark UI in detail, the Spark Essentials course on the site does exactly that:
rockthejvm.com/p/spark-essentials

[@Capt_X]

Great Video with nice explanation! Is there any list which help us to identify which exchanges performed by spark and by the codebase? Is it available on spark documentation link?

[@rockthejvm]

Spark docs usually mention which transformations involve a shuffle. Major examples include joining, sorting and grouping.

[@VikasVerma-s3b]

Awesome

[@sansanjusingh]

good one