Data Validation with Pyspark || Real Time Scenario

[@RahulEternalLearner]

Amazing content. Thank you for sharing. This time youtube didn't show repeated ads. Thank you youtube.

[@vamshimerugu6184]

Great explanation ❤.Keep upload more content on pyspark

[@DataSpark45]

Thank you, I will

[@mohitupadhayay1439]

Amazing content. Keep a playlist for Real time scenarios for Industry.

[@ajaykiranchundi9979]

Very helpful! Thank you

[@ArabindaMohapatra]

I just started watching this playlist. I'm hoping to learn how to deal with schema-related issues in real time.Thanks

[@DataSpark45]

Thanks a million bro

[@listentoyourheart45]

Nice

[@ComedyXRoad]

do we apply these techniques for delta tables also

[@Lapookie]

Cool, but is it like this every time ? Like you have a reference df containing all columns and file name / path and you have to iterate over it to see if its matching ?

[@lokeswarreddyvalluru5918]

Yes

[@TelugodiPrapanchaYathra]

how did you define reference_df and control_df

[@DataSpark45]

we defined as a table in any DataBase. As of know i used them as a csv

[@OmkarGurme]

while working with databricks we dont need to start a spark session right ?

[@DataSpark45]

No need brother, we can continue with out defining spark session, i just kept for practice