Crystal Clear Explanation. You have hard coded the version of the table while merging , when you schedule in a pipeline , how to make it dynamic ? is it like get max ( _change_type) and then use it in merge?
At 18:08, 2nd insert of 7 rows, Six rows are on date "4/1/2021", One row on date "3/1/2021" and they are inserted, how come at 18:28, showing 7 rows inserted, but Six of them are on date "3/1/2021" and One row is on "4/1/2021" ?? I'd expect them to be the same dates as of 18:08 ? Am I missing something or I got something wrong ? Can someone explain to me please ?
Hi Thanks for the video on new feature of Data bricks. I have one question , you have hardcoded the version number in code. how can we pass the latest version dynamically in merge part. Thanks in advance
Hi Bhavana, great explanation for CDF. Now when update happens on a table , table_changes contains two records update_preimage and update_postimage. I am more curious about how you handle these cases while taking table into next merge. because i can see you take table name and version only.
Great video and explanation. I have a question - In Cmd22- For Merge stmt we need to keep updating the version number from 2 to 3 and so on for each incremental load right based on the latest version. How do I do that automatically?
Hey thank you so much for creating this! One question - how can i always pick up the latest version of changes?
Crystal Clear Explanation. You have hard coded the version of the table while merging , when you schedule in a pipeline , how to make it dynamic ? is it like get max ( _change_type) and then use it in merge?
At 18:08, 2nd insert of 7 rows, Six rows are on date "4/1/2021", One row on date "3/1/2021" and they are inserted, how come at 18:28, showing 7 rows inserted, but Six of them are on date "3/1/2021" and One row is on "4/1/2021" ?? I'd expect them to be the same dates as of 18:08 ? Am I missing something or I got something wrong ? Can someone explain to me please ?
Hi Thanks for the video on new feature of Data bricks. I have one question , you have hardcoded the version number in code. how can we pass the latest version dynamically in merge part. Thanks in advance
Did you find the answer for that? The only way I can think of, would be to store a checkpoint somewhere for the latest processed _commit_version...
@@felipecastro3710 how do you do that and I am looking for an answer for the same question.
Loved your explanation. Simple and to the point 👌
Hi Bhavana, great explanation for CDF. Now when update happens on a table , table_changes contains two records update_preimage and update_postimage.
I am more curious about how you handle these cases while taking table into next merge. because i can see you take table name and version only.
Crystal clear presentation 👏
Another amazing video, thanks!
Thank You mam for databricks
series.
When you selected from the silver table on the table_changes, you need to pick the postimage record only , correct ?
Great video and explanation. I have a question - In Cmd22- For Merge stmt we need to keep updating the version number from 2 to 3 and so on for each incremental load right based on the latest version. How do I do that automatically?
Great Demo!
Hi
Can you show how to implement this delta lake in AWS?
Superb explanation 👌 👏
Thanks Bhawna for sharing the knowledgeable video..
Is it possible to share this code through sharing Apps like GIT so it would be easy to do hands-on
Where can we get the tutorial codes?
Is medillian architecture and delta live tables are same?
That is my understanding as well.
Great demo RIGHT , RIGHT ?
lol😂
👌👍👍👍
Hi
Can you show how to implement this delta lake in AWS?