96. Databricks | Pyspark | Real Time Scenario | Schema Comparison

[@vishalaaa1]

excellent video Raaj
the scenario on dynamically adding missing columns is awesome

[@rajasdataengineering7585]

Glad you liked it!

[@putuism1]

excellent video Raja , thank you

[@rajasdataengineering7585]

You are welcome! Glad it helps

[@sravankumar1767]

Hi Raja, how can we merge 3 dataframe using union or unionbgname in the user defined function

[@rajasdataengineering7585]

Hi Sravan, yes it possible
Df=df1.union(df2).union(df3)

[@nagulmeerashaik5336]

Very nice👍👏😊.... Lot of content you have

[@rajasdataengineering7585]

Thank you 😊

[@sravankumar1767]

I have one doubt , how can we merge these 2 csv files as a single csv, can we use union by name

[@rajasdataengineering7585]

Yes we can

[@seenme2951]

That's great sir

[@rajasdataengineering7585]

Thank you

[@muvvalabhaskar3948]

Hi I have created one databricks job what ever output comes from it needs to save in s3 as log file how can I do that?

[@mallikarjunad3549]

Hi Raja, could you please make a video on CICD pipeline for Azure databricks and ADF by using Azure Devops. Many thanks in advance

[@rajasdataengineering7585]

Sure I will

[@alluchandrasekhar2992]

Useful content 🔥

[@rajasdataengineering7585]

Thank you

[@dataengineerazure2983]

@@rajasdataengineering7585 Thank you. Have you the data source(github)? thank you in advance

[@playsomeboards]

Thank you for your videos, It's important to point out that you keep calling things a UDF that are not UDFs. For example, addMissingColumns(df1, df2) is not a UDF

[@rajasdataengineering7585]

Addmissingcolumns is a user defined function. What do you call that?

[@David-rm1wn]

HI thanks for very good videos...Is it possible to get free azure databricks service?

[@sabesanj5509]

Raja bro please make a video on frequently asked interview questions in hive and spark.

[@rajasdataengineering7585]

Sure bro, will make

[@sabesanj5509]

@@rajasdataengineering7585 Thanks bro