Partitions | map vs mapPartition Vs MapPartitionWithIndex | Spark with Scala

[@technologyexplorer9193]

Nice explanation.

[@sravanalakshmipisupati6533]

Thank you very much.

[@padmsrikanth1672]

Nice explanation, Thanks

[@sravanalakshmipisupati6533]

@Srikanth, Thank you very much for watching the video and providing your feedback.

[@Jay-m8n7i]

very well explained with practical.. thanks

[@sravanalakshmipisupati6533]

Thank you Jayanaresh.

[@pradeepkoduru6890]

Good explanation... Thanks

[@sravanalakshmipisupati6533]

Thank you Pradeep.

[@tejaswinishewale4636]

Thanks

[@technologyexplorer9193]

Can we use mappartition or mappartitionWithIndex on top of dataframe?
If i want to control the number of partitions while reading data from hive table?
Please let me know.

[@sravanalakshmipisupati6533]

You can use these transformations on dataframe as well. If any error, import spark implicits and try.
You can directly use filter conditions on hive partitions, no need to use mappartitions in this case.
For hbase connections, you can use mappartitions.