Reading Data from Hudi INC & Joining with Delta Tables using HudiStreamer & SQL-Based Transformer

Reading Data from Hudi Tables Incrementally, Joining with Delta Tables using HudiStreamer and SQL-Based Transformer to Build Denormalized Table: An Interior Peek into Hued Iceberg and Delta Lakes (XTable)
Exercise Files
github.com/soumilshah1995/Hud...

[@pavan792reddy]

Hi soumil,
I am not able to access the git lab for the exercise looks like the link wasn't found

[@ashraf7172]

I fetched data from
Postgres as CDC pipeline,2-3 tables
Then with help of deltastreamer I creste a universal data lakehouse this will become my bronze layer.
I had a question now what approach should I use? For creating a silver level data i.e joined and transformed table
A - fetching data again from postgres and joining at the time of hui streamer pyspark job
Or B fetching data from the universal datalakehouse bronze layer and then creating a silver level lakehouse.
Or any other method
I want it like whenever I update data in postgres all my data in bronze and silver lvl both gets updated ?

[@SoumilShah]

Ofcourse bronze why would you want to fetch from postgres again it’s already bronze lagers
Read data from Hudi and join them build your silver

[@ashraf7172]

Great,
I was confused
That when I will create silver from my hudi bronze tables
If I make changes in postgres will it reflect in my silver level or only in bronze
I got clarified