What Table Format Should I Choose For My Data Lake? Hudi | Iceberg | Delta Lake
Вставка
- Опубліковано 22 лип 2024
- LINKS TO FULL BLOG:
ℹ️ AWS Blog:
aws.amazon.com/blogs/big-data...
Using a blog recently posted on AWS I break down and discuss the key considerations when deciding on an open source format for your transactional data lake tables in AWS. We look at the general considerations you should factor into your decision making process before diving into the different options available for streaming, CDC and batch. The video covers the three most popular open source table formats: Apache Hudi, Apache Iceberg, and Delta Lake.
SUPPORT THE CHANNEL:
☕ Buy Me A Coffee: www.buymeacoffee.com/johnnych...
🖥️ My VPN: go.nordvpn.net/aff_c?offer_id...
▬▬▬▬▬▬ T I M E S T A M P S ⏰ ▬▬▬▬▬▬
00:00 - Intro
01:19 - General Considerations
02:52 - Streaming
04:11 - Change Data Capture
05:16 - Batch Loads
06:01 - Outro
OTHER USEFUL LINKS:
ℹ️ My Website: johnnychivers.co.uk
🔗 Linkedin: / johnny-chivers
😎 About me
I have spent the last decade being immersed in the world of big data working as a consultant for some the globe's biggest companies.My journey into the world of data was not the most conventional. I started my career working as performance analyst in professional sport at the top level's of both rugby and football. I then transitioned into a career in data and computing. This journey culminated in the study of a Masters degree in Software
Enjoy 🤘 - Наука та технологія
Another good video from the Chiverse.. :)
Good video. It's worth having a peek at Delta Lakes Liquid Clustering offering, and by default it does clean up the version history. I don't know if that's a good or a bad thing. I think it's bad.
Anyway great work! Might be outdated a bit though. Both techs seem to be arms racing against each other.
Good one..Thank you
Good one. Thank you.
Thank you too!
Is this guy Scottish or Jamaican? Never heard an accent like this before it’s wild
Iceberg > delta lake > hudi