Thank you for the explanation! Can please someone confirm that the files can only be read from FileStore and not blob storage by this approach (SQL)? Tried reading it from blob storage and cannot access using cloud_files() function. I guess it requires pyspark to access blob storage and create delta live tables.
Nice demo, but why does the comment for the "cleaned_station_status" table say "partitioned by station_id" when the code actually uses the last_updated_date column? You should update the comment in that notebook. :-)
He mentions the python scripts/notebooks that get the data. Probably using an api and saving results to DBFS. I’m sure you can find how to do that in other videos.
This is great. Can you share notebook for querying the transaction log and presenting it in reDash?
Great video, easy to understand, good overview for the Databricks beginners. Thanks!
Thank you for the explanation! Can please someone confirm that the files can only be read from FileStore and not blob storage by this approach (SQL)? Tried reading it from blob storage and cannot access using cloud_files() function. I guess it requires pyspark to access blob storage and create delta live tables.
Do you have a link to show how the queries work for monitoring the data quality. Thanks
Nice demo, but why does the comment for the "cleaned_station_status" table say "partitioned by station_id" when the code actually uses the last_updated_date column? You should update the comment in that notebook. :-)
Anyone knows how to create a live dashboard like this in databricks?
Thank you for this video. I'm a little confused about what the "data.stations' refers to? Is it an array in the source json?
Well explained. Thank you
Can an API hosted on an App service in anyway fetch Delta live tables data ? thanks
am I missing something or does the video really doesnt show how he got all those files in the data lake in the first place?
He mentions the python scripts/notebooks that get the data. Probably using an api and saving results to DBFS. I’m sure you can find how to do that in other videos.
Hes using autoloader to load the data probably from an s3 bucket, Azure cloud storage or volume
GREAT VIDEO
Good video thank you. Quick question, is the DLT lineage also auotmatically avaialable and visisble in Unity Catalog?
It’s been in gated preview for too long, when will it be made GA?
Today😃
@@adityaprakash6383 is this GA Aditya?
@@noorbashashaik2124 yes