- 2
- 18 916
Melbourne Databricks User Group
Приєднався 4 лис 2021
We’re the meetup for everything Databricks: Apache Spark, Delta Lake, MLflow, the Lakehouse-and all the other goodies that are part of, support or integrate with Databricks.
We post recordings of past events on this channel.
Our events are hybrid or virtual. You’re welcome to come in-person in Melbourne (should that ever be possible again), or you can join virtually from anywhere.
We meet the first Thursday of each month at 6 pm, Melbourne time (AEST or AEDT).
We post recordings of past events on this channel.
Our events are hybrid or virtual. You’re welcome to come in-person in Melbourne (should that ever be possible again), or you can join virtually from anywhere.
We meet the first Thursday of each month at 6 pm, Melbourne time (AEST or AEDT).
Data Governance on top of your Lakehouse with Databricks | Leon Eller | Gurpreet Singh | Amine Slima
From Databricks themselves, we get an overview and demo of implementing various aspects of data governance on a Databricks Lakehouse using Unity Catalog, which was in private preview at the time of recording.
--
This was the talk from the December 2021 event of the Melbourne Databricks User Group.
--
This was the talk from the December 2021 event of the Melbourne Databricks User Group.
Переглядів: 208
Відео
Unit testing with Databricks | Jonathan Neo | November 2021
Переглядів 19 тис.3 роки тому
Just like eating vegetables, no one likes writing tests. However, writing unit tests is good for your programming diet. It helps ensure that data flows from one end of the pipeline to the other without any hitches. In this talk, Jonathan Neo, Senior Data Engineer at Cuusoo, will explain why and how you can write unit tests, and where does unit testing fit in the bigger picture. Jonathan will de...
Hi Jonathan, thanks for this demo, this is fantastic. few things have changed in this 2 years but the basics are same. When are you going to show us the remaining two parts like integration testing and data quality testing? or if you already have those videos can you please upload them to your channel. Thanks.
is there a GitHub link to this project explanation?
Great session very much thankful to you.
nothing is visible on your code screenshots. Very bad presentation.
Why 45 minutes needed for a demo example?
Thanks a ton. Loved it
Unit testing info starts at 10:32
Great video! Since databricks-connect is now deprecated, how should we set up unit testing?
Hi Bjarne, have you managed to get an answer on this somewhere else?
@@yoshitjuh, actually there's an update to the databricks-connect package so that it now supports runtime 10.4 LTS, but databricks apparently recommends not using it and rather use dbx by databrickslab to setup a project, run all unit testing locally and supply convenient command line functions for deployment and running jobs. Still not sure about testing during pull requests and stuff like that.
informative
Good stuff! Liked and subscribed ! :)