116. Databricks | Pyspark| Query Dataframe Using Spark SQL
Вставка
- Опубліковано 8 вер 2024
- Azure Databricks Learning: Query Dataframe Using Spark SQL
==========================================================
SQL is one of the most convenient language for most of the data engineers across globe. Though Spark provides the feature of exploring Dataframe using SQL only after converting it to table or view, Spark has introduced new feature in 3.4 onwards through which no need of converting dataframe into table or view.
To understand this feature better, watch this video
• 116. Databricks | Pysp...
#DataframeinSparkSQL, #SparkSQL, #ParameterizedSparkSQL, #SparkDevelopment,#DatabricksDevelopment, #DatabricksPyspark,#PysparkTips, #DatabricksTutorial, #AzureDatabricks, #Databricks, #Databricksforbeginners,#datascientists, #datasciencecommunity,#bigdataengineers,#machinelearningengineers
Hi Raja,pls make a detailed video on spark performance tuning with a usecase n example
Hi Anil, sure will create a video on this requirement soon
with interpolation it is not necessary to create the TempView. I liked!
Glad you liked it !
Thanks for the video Raja . But for me personally , I will still prefer using tempview feature as parametrizing the query seems to be a extra overhead to me . But I think there must be some more usecases for new feature which as of now I am not able to think of.
Agreed. There are multiple options available now. Depending on developer's comfort, anthing can be used
Hi Raja, How to follow your videos for a. beginner who wants to learn Pyspark? It would be great if you can provide datasets for all the above vidoes
Thank you and waiting for your response