Hi , this is an amazing tutorial, thanks for that. need a suggestion. If I want to create three layers - bronze, silver and gold - should these layers be at catalog level or schema level?
I prefer having it at the catalog level and prefixed with the environment. For instance, dev_bronze, test_bronze, prod_bronze. Then underneath the correct environment, you'd have a schema for each data source. So the catalog would be prod_bronze and the schema would be salesforce, dynamics, workday, oracle_erp, etc.
What about updated rows? If I have a transaction or a customer that was updated in the source. What do you suggest? I keep all the historical on the silver layer, to create a SCD in my gold dim_customer, for example?
Great video, I was looking for this!
Great video and explanation. Thank you!
Thank you Ike, this makes sense and it was very helpful .. keep posting!
Hi , this is an amazing tutorial, thanks for that. need a suggestion. If I want to create three layers - bronze, silver and gold - should these layers be at catalog level or schema level?
I prefer having it at the catalog level and prefixed with the environment. For instance, dev_bronze, test_bronze, prod_bronze. Then underneath the correct environment, you'd have a schema for each data source. So the catalog would be prod_bronze and the schema would be salesforce, dynamics, workday, oracle_erp, etc.
great video, very helpful
Interesting video, thanks UA-cam suggestion!
Underdiscussed topic. Could you elaborate on the storage layer? How do you create separation in storage containers?
I have a video on very close to this topic that might help you: ua-cam.com/video/z9L7rNtKjCg/v-deo.html
What about updated rows? If I have a transaction or a customer that was updated in the source. What do you suggest? I keep all the historical on the silver layer, to create a SCD in my gold dim_customer, for example?