- 40
- 13 026
AfroInfoTech
Приєднався 14 чер 2020
This channel is for sharing knowledge about my career and helping people actualize their IT Data engineering career goals
Відео
Info session DataEngineer BootCamp Fall 2024
Переглядів 3821 день тому
Register for upcoming bootcamp afroinfotech.teachable.com/p/bootcamp-azure-data-engineering
Databricks For Each Task Review
Переглядів 196Місяць тому
For more details on Databricks For Each please refer to docs.databricks.com/en/jobs/for-each.html#:~:text=You can add a For,in the Task name field. For details about my bootcamp, see link below. It starts Sept 24, 2024 afroinfotech.teachable.com/p/bootcamp-azure-data-engineering You can also visit my official Data Engineering coaching website at afroinfotech.teachable.com/. Follow me on other p...
Real World End to End Meta data driven pipeline Azure Data Factory Databricks
Переглядів 1902 місяці тому
Join me as I demo a real-world, metadata-driven, and configurable data pipeline that processes files landing in a data lake into the Databricks Medallion Architecture. This pipeline is a simplified version of a real-world project I built. I developed it using Databricks and Azure Data Factory with PySpark. Watch as I guide you step by step from requirements to production code. For the input fil...
Navigating Databricks Spark UI the EASY way
Переглядів 2044 місяці тому
As a databricks developer navigating the UI is crucial, but tricky. Watch this video to help you get started I am Nnaemezue Obi-eyisi, a Senior Azure Databricks Data Engineer at Capgemini and the founder of AfroInfoTech, an online coaching platform for Azure data engineers specializing in Databricks If you’re interested, join my waitlist for the upcoming Data Engineer bootcamp by signing up wit...
Databricks Admin roles 101
Переглядів 2084 місяці тому
In this video, I go over the various Admin roles someone needs to have to properly administer access and manage Databricks workspaces and objects I am Nnaemezue Obi-eyisi, a Senior Azure Databricks Data Engineer at Capgemini and the founder of AfroInfoTech, an online coaching platform for Azure data engineers specializing in Databricks If you’re interested, join my waitlist for the upcoming Dat...
How to run Databricks workflows using service principal the Easy way!
Переглядів 5625 місяців тому
I demonstrate how to do some basic Databricks Administration to set up an Azure service Principal and use it to run Databricks workflow jobs with UI I am Nnaemezue Obi-eyisi, a Senior Azure Databricks Data Engineer at Capgemini and the founder of AfroInfoTech, an online coaching platform for Azure data engineers specializing in Databricks If you’re interested, join my waitlist for the upcoming ...
Azure DataFactory - How to Design an Incremental Load Pipeline
Переглядів 1075 місяців тому
In this video, we discuss how to design an incremental load pipeline for update and insert use case using ADF and using the Max value of Watermark date approach. I am Nnaemezue Obi-eyisi, a Senior Azure Databricks Data Engineer at Capgemini and the founder of AfroInfoTech, an online coaching platform for Azure data engineers specializing in Databricks If you’re interested, join my waitlist for ...
Interview with Andreas on my Azure Data Engineering Bootcamp
Переглядів 675 місяців тому
In this interview, I talk about myself, my experiences in data engineering, what I teach in my bootcamp and why data engineering. I am Nnaemezue Obi-eyisi, a Senior Azure Databricks Data Engineer at Capgemini and the founder of AfroInfoTech, an online coaching platform for Azure data engineers specializing in Databricks If you’re interested, join my waitlist for the upcoming Data Engineer bootc...
Fundamentals of designing an Incremental ETL Data load
Переглядів 57Рік тому
In this video, I explain how we can design an incremental data pipeline. I explain the basic design and considerations. I hope it is helpful for beginners Connect with me on Linkedin: www.linkedin.com/in/nobieyisi/ Connect on medium: medium.com/@afroinfotech Connect on Instagram: afroinfotech Email: mezulity@gmail.com Phone Text: 435.565.1641
Azure Data Engineer BootCamp info
Переглядів 83Рік тому
Connect with me on Linkedin: www.linkedin.com/in/nobieyisi/ Connect on medium: medium.com/@afroinfotech Connect on Instagram: afroinfotech Email: mezulity@gmail.com Phone Text: 435.565.1641
SQL Merge Upsert with NULL values (Real World use case)
Переглядів 137Рік тому
This video will show you how to handle NULL values that create bugs when you do a conditional Merge Upsert I show 3 different approaches to solve this problem.... watch till the end for the best approach SQL Script used can be found here: github.com/nobieyi00/SQLNULLS/blob/main/MergeUpsert_null.sql Connect with me on Linkedin: www.linkedin.com/in/nobieyisi/ connect on medium: medium.com/@afroin...
Deep Dive in SQL Nulls
Переглядів 346Рік тому
In this video, I go over SQL Nulls thoroughly revealing some of the common nuances we have with SQL Nulls and come mistakes developers make. SQL file used in Demo: github.com/nobieyi00/SQLNULLS/tree/main Connect with me on Linkedin: www.linkedin.com/in/nobieyisi/ connect on medium: medium.com/@afroinfotech #sql #null #data #analytics #database
Diving into Data Engineering 2023 info session UIU
Переглядів 632Рік тому
Data Engineering information session hosted by UIU. In this information session, we discuss what is data engineering, how to become a data engineer, how to prepare resume and job interviews and more... The presentation link can be found here docs.google.com/presentation/d/1KFBmc69381mJW6yC5QYMpmMHdr0hwJ9u/mobilepresent#slide=id.p1 Connect with me on Linkedin: www.linkedin.com/in/nobieyisi/ conn...
Databricks Delta Lake Change Data feed - Real World Use case
Переглядів 6 тис.Рік тому
This video is a real world demonstration of how we can implement databricks change data feed to optimize our ELT pipeline in Delta Lake house. You can use this for practicing your data engineering skills. I go over full load, incremental load, Merge Upsert, Change Data capture, ETL logic. I hope this really helps someone gain experience as a Data engineer. Find the notebook and source files her...
Top 5 reasons why I am not using Delta Live tables
Переглядів 532Рік тому
Top 5 reasons why I am not using Delta Live tables
Learn how to use Spark/Databricks as a SQL Developer
Переглядів 140Рік тому
Learn how to use Spark/Databricks as a SQL Developer
Optimize your Datawarehouse ELT data pipeline NOW
Переглядів 84Рік тому
Optimize your Datawarehouse ELT data pipeline NOW
SQL Window Function Partition by Orderby Deep Dive
Переглядів 50Рік тому
SQL Window Function Partition by Orderby Deep Dive
How to mount an Azure Data Lake to Databricks cluster: Raw Uncut
Переглядів 492 роки тому
How to mount an Azure Data Lake to Databricks cluster: Raw Uncut
Part 2 Using Databricks to ingest incremental CDC data
Переглядів 722 роки тому
Part 2 Using Databricks to ingest incremental CDC data
Part 1 Using Databricks to ingest Incremental CDC data Raw Uncut
Переглядів 1352 роки тому
Part 1 Using Databricks to ingest Incremental CDC data Raw Uncut
Part 10: Raw Uncut Real World Data Engr Project: ETL Change Data Capture Full and Increment load
Переглядів 382 роки тому
Part 10: Raw Uncut Real World Data Engr Project: ETL Change Data Capture Full and Increment load
Part 9: Raw Uncut Real World Data Engr Project: ETL Change Data Capture Full and Increment load
Переглядів 152 роки тому
Part 9: Raw Uncut Real World Data Engr Project: ETL Change Data Capture Full and Increment load
Part 8: Raw Uncut Real World Data Engr Project: ETL Change Data Capture Full and Increment load
Переглядів 112 роки тому
Part 8: Raw Uncut Real World Data Engr Project: ETL Change Data Capture Full and Increment load
Part 7: Raw Uncut Real World Data Engr Project: ETL Change Data Capture Full and Increment load
Переглядів 182 роки тому
Part 7: Raw Uncut Real World Data Engr Project: ETL Change Data Capture Full and Increment load
Part 6: Raw Uncut Real World Data Engr Project: ETL Change Data Capture Full and Increment load
Переглядів 192 роки тому
Part 6: Raw Uncut Real World Data Engr Project: ETL Change Data Capture Full and Increment load
Part 5: Raw Uncut Real World Data Engr Project: ETL Change Data Capture Full and Increment load
Переглядів 232 роки тому
Part 5: Raw Uncut Real World Data Engr Project: ETL Change Data Capture Full and Increment load
Part 4: Raw Uncut Real World Data Engr Project: ETL Change Data Capture Full and Increment load
Переглядів 212 роки тому
Part 4: Raw Uncut Real World Data Engr Project: ETL Change Data Capture Full and Increment load
Nice- this was a much needed feature. thanks for the walkthru!
When navigating the Spark UI, something that helped me is "Jobs are full of Stages are full of Tasks" - its a helpful saying I remember when I start to get lost in the weeds haha. thanks for this walkthrough!
Can you do a video explaining how to solve spark issues like OOM error, data spill , data skew and a lot of others using spark UI?
Coming from using legacy systems like T-SQL, ETL tools etc... I agree I am not a big fan of this Delta tables, it is too much of a learning curve from what we are used to !
Great video brother
Thanks for the vid. So you have to read the entire refined table in order to get the last commit version? Is there any more efficient way to get around this?
"Promo SM" 😭
Great tutorial, I would like to see autoloader as the source -> raw and then go through the medallion layers
Hello Did you found how to implement it? Any ideas? Thanks
Good job Obi 👍
Good one Obi
ᑭᖇOᗰOᔕᗰ
Thank you for the video, really appreciate it!
Learned something new. Thank you for sharing!
What are your thoughts on Data engineering Bootcamp from Springboard??
Send me your link so i can review please
Great video sir , but you kept mentioning only azure and AWS , does that mean GCP isint as good as the cloud tools you mentioned ?
GCP is good but not yet as popular as AWS and Azure. GCP maybe has 10 to 15 percent market share now so that is why I didn't mention it. feel free to learn the equivalent
Also i am only familiar with US market
Great video!!!!
Great content chief
Great information!
Check out my blog to get your started training while i drop the other videos plsqlandssisheadaches.blogspot.com/2020/07/become-data-engineer-in-3-months-with.html
I feel encouraged to delve into this field. Much thanks MEZUlity 💪
9ice one Mezu. Keep it up bro 👍
The starting is very good . Your teaching style is full of passion.. you described everything very well. I hope, you will continue the same passion in the next coming videos.