- 59
- 258 863
Tybul on Azure
Poland
Приєднався 13 лип 2023
This channel is all about free Azure trainings, especially in data-related areas.
DP-203 vs DP-600 vs DP-700
Hey there! Microsoft has introduced a new Fabric exam: DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric. But how does it compare to the existing DP-203 and DP-600 exams? What sets them apart? And which one should you take first?
I dive into all these questions in my latest UA-cam video. Check it out and enjoy!
▬▬▬▬▬▬ IMPORTANT LINKS ▬▬▬▬▬▬
My LinkedIn profile: www.linkedin.com/in/piotr-tybulewicz-81a8793/
My GitHub with drawings and source code: github.com/TybulOnAzure/StandaloneEpisodes
DP-700 exam announcement: techcommunity.microsoft.com/blog/microsoftlearnblog/prove-your-data-engineering-skills-and-join-the-ai-transformation/4240416
DP-203 study guide: learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-203
DP-600 study guide: learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-600
DP-700 study guide: learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-700
▬▬▬▬▬▬ MEMBERSHIP ▬▬▬▬▬▬
Join this channel to get access to perks:
ua-cam.com/channels/LnXq-Fr-6rAsCitq9nYiGg.htmljoin
▬▬▬▬▬▬ CHAPTERS ▬▬▬▬▬▬
00:00 Introduction
00:28 Comparison
16:58 How to choose the exam to take?
27:13 Summary
I dive into all these questions in my latest UA-cam video. Check it out and enjoy!
▬▬▬▬▬▬ IMPORTANT LINKS ▬▬▬▬▬▬
My LinkedIn profile: www.linkedin.com/in/piotr-tybulewicz-81a8793/
My GitHub with drawings and source code: github.com/TybulOnAzure/StandaloneEpisodes
DP-700 exam announcement: techcommunity.microsoft.com/blog/microsoftlearnblog/prove-your-data-engineering-skills-and-join-the-ai-transformation/4240416
DP-203 study guide: learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-203
DP-600 study guide: learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-600
DP-700 study guide: learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/dp-700
▬▬▬▬▬▬ MEMBERSHIP ▬▬▬▬▬▬
Join this channel to get access to perks:
ua-cam.com/channels/LnXq-Fr-6rAsCitq9nYiGg.htmljoin
▬▬▬▬▬▬ CHAPTERS ▬▬▬▬▬▬
00:00 Introduction
00:28 Comparison
16:58 How to choose the exam to take?
27:13 Summary
Переглядів: 472
Відео
Learn KQL with Kusto Detective Agency
Переглядів 47014 днів тому
Hey there! Have you heard of Kusto Query Language (KQL)? It’s a powerful querying language that’s becoming widely used across several Azure services, like Azure Monitor, Azure Data Explorer, and Microsoft Fabric. So, how do you get started with KQL? Sure, you could comb through pages of documentation, but there’s a more engaging way: the Kusto Detective Agency! It’s a clever gamified experience...
Watch out! Security danger with Azure Databricks Delta Tables
Переглядів 387Місяць тому
Hey there! Did you know you could unintentionally give someone read access to your Delta tables in ADLSg2? This can happen when working with ACLs, especially if you're not fully aware of how Databricks manages permissions. I encountered this exact issue in one of my projects. In this video, I'll guide you through the business scenario, set everything up, showcase the unintended access, explain ...
DP-203: 54 - Exam overview and course summary
Переглядів 1,5 тис.Місяць тому
Hey Data Engineers! It's official - my DP-203 course is now complete! After more than a year of hard work, it's finally ready to go. In the final episode, I cover everything you need to know about the exam: how to register, the types of questions to expect, and tips for last-minute prep. I also share some exciting updates about what’s next for my UA-cam channel. But here’s the key takeaway: YOU...
DP-203: 53 - 4th milestone - Streaming and data governance recapped
Переглядів 5682 місяці тому
Hey Data Engineers! We did it! We’ve successfully wrapped up another data engineering area, focusing on streaming and data governance. Catch the 53rd episode of my free DP-203 course, where I’ll recap these key topics and introduce a challenge to sharpen your skills. Enjoy! ▬▬▬▬▬▬ IMPORTANT LINKS ▬▬▬▬▬▬ My LinkedIn profile: www.linkedin.com/in/piotr-tybulewicz-81a8793/ GitHub with my drawings: ...
DP-203: 52 - Data governance with Microsoft Purview
Переглядів 1,5 тис.2 місяці тому
Hey Data Engineers! It's time to take a deep dive into data governance. But what exactly does that mean? Is it simply about controlling data access, or is there more to it? In the 52nd episode of my free DP-203 course, I’m covering data governance in Azure and exploring how Microsoft Purview can help manage it effectively. Tune in and enjoy! ▬▬▬▬▬▬ IMPORTANT LINKS ▬▬▬▬▬▬ My LinkedIn profile: ww...
DP-203: 51 - Handling time & performance in Azure Stream Analytics
Переглядів 9492 місяці тому
Hey data engineers! Did you know that time management is crucial in streaming solutions? Have you ever faced the challenge of dealing with events that arrive out of order or with significant latency? And how can you be sure your solution is performing optimally? It’s better to address these issues proactively before they impact your users. Join me in the 51st episode of my free DP-203 course, w...
DP-203: 50 - Azure Stream Analytics overview
Переглядів 1,5 тис.2 місяці тому
Hey Data Engineers! We've successfully ingested streaming data into Azure using Azure Event Hubs, but it's just sitting there, untapped and underutilized. It's time to unlock its potential and dive into the insights it holds! In the 50th episode of my free DP-203 course, I'll guide you through Azure Stream Analytics-a powerful tool for processing streaming data. We'll explore its windowing func...
DP-203: 49 - Introduction to streaming, Event Hubs
Переглядів 1,4 тис.3 місяці тому
Hey Data Engineers! Tired of the same old batch processing? Let's dive into something more exciting-streaming and real-time data processing! In the 49th episode of my free DP-203 course, I'm exploring streaming, sharing real-world examples, outlining a high-level architecture, and taking a deep dive into the ingestion phase using Azure Event Hubs. Enjoy! ▬▬▬▬▬▬ IMPORTANT LINKS ▬▬▬▬▬▬ My LinkedI...
DP-203: 48 - 3rd milestone - Serve phase recapped
Переглядів 8073 місяці тому
Hey data engineers! We’ve done it! We’ve just completed another phase of our projects: serving data to the end users/data consumers. Join me in the 48th episode of my free DP-203 course, where I'll walk you through the entire data serving process and present a challenge to help you refine your skills. Enjoy! ▬▬▬▬▬▬ IMPORTANT LINKS ▬▬▬▬▬▬ My LinkedIn profile: www.linkedin.com/in/piotr-tybulewicz...
DP-203: 47 - Azure Synapse Analytics - Serverless SQL Pool
Переглядів 1,5 тис.3 місяці тому
Hey data engineers! We've all seen the modern data warehouse architecture that uses Dedicated SQL Pool for storing data and serving it to consumers. But do we really need another copy of the data? What if we could just use the lakehouse approach? Well, we can! The key component for this is the Serverless SQL Pool available in Azure Synapse Analytics. Join me in the 47th episode of my free DP-20...
DP-203: 46 - Dedicated SQL Pool security
Переглядів 1,1 тис.4 місяці тому
Hey data engineers! So you've decided to store your data in Synapse Dedicated SQL Pool. Great! But how do you secure that data? How do you make sure that only selected principals are allowed to query the data? Is there a way to restrict which columns and rows users can see? Join me in the 46th episode of my free DP-203 course, where I cover the following topics: • Firewall in Synapse • Transpar...
DP-203: 45 - Azure Synapse Dedicated SQL Pool - extra features
Переглядів 1,3 тис.4 місяці тому
Hey data engineers! Imagine we’ve designed our Dedicated SQL Pool using a star schema and allowed users to query it. But now we’re getting complaints that it’s incredibly slow, and users have to wait too long for the data. How can we handle concurrent workloads in a Dedicated SQL Pool? How can we prioritize important queries to ensure they are processed first? How can we reserve resources to en...
DP-203: 44 - Loading data to Dedicated SQL Pool
Переглядів 1,6 тис.4 місяці тому
Hey data engineers! If you’ve chosen to use Dedicated SQL Pool for your data storage and serving needs, you might be wondering how to load data into it. What options are available, and what are their pros and cons? Join me for the 44th episode of my free DP-203 course, where I explore various methods for inserting data into Dedicated SQL Pool: • PolyBase • COPY INTO • ADF/Synapse data flows • A...
DP-203: 43 - Azure Synapse Dedicated SQL Pool - Architecture and overview
Переглядів 1,8 тис.4 місяці тому
Hey data engineers! We've finished transforming our data - it's now pure, pristine, and ready to be consumed as facts and dimensions. Great! The only issue is that it's located in a data lake using the delta format, and not every consumer can connect to it. Or maybe we just prefer a good, old-fashioned, traditional relational data warehouse instead of this "data lake thing." What should we do? ...
DP-203: 42 - 2nd milestone - transform phase recapped
Переглядів 1,1 тис.5 місяців тому
DP-203: 42 - 2nd milestone - transform phase recapped
DP-203: 41 - Transforming data with Data Flows
Переглядів 1,5 тис.5 місяців тому
DP-203: 41 - Transforming data with Data Flows
DP-203: 40 - Extra features in Azure Synapse Analytics Spark Pools
Переглядів 1,4 тис.5 місяців тому
DP-203: 40 - Extra features in Azure Synapse Analytics Spark Pools
DP-203: 39 - Azure Synapse Analytics - Spark Pools
Переглядів 2 тис.6 місяців тому
DP-203: 39 - Azure Synapse Analytics - Spark Pools
DP-203: 38 - Transforming data with dbt
Переглядів 1,9 тис.6 місяців тому
DP-203: 38 - Transforming data with dbt
DP-203: 37 - Orchestrating Databricks notebooks
Переглядів 1,8 тис.6 місяців тому
DP-203: 37 - Orchestrating Databricks notebooks
DP-203: 36 - Automating the process with Azure Databricks Autoloader
Переглядів 2,1 тис.6 місяців тому
DP-203: 36 - Automating the process with Azure Databricks Autoloader
DP-203: 35 - Writing data to ADLSg2 from Azure Databricks
Переглядів 1,9 тис.7 місяців тому
DP-203: 35 - Writing data to ADLSg2 from Azure Databricks
DP-203: 34 - Common data transformations
Переглядів 2,3 тис.7 місяців тому
DP-203: 34 - Common data transformations
DP-203: 33 - Connecting to ADLSg2 from Azure Databricks
Переглядів 2,7 тис.7 місяців тому
DP-203: 33 - Connecting to ADLSg2 from Azure Databricks
DP-203: 32 - A closer look at Databricks notebooks
Переглядів 2,9 тис.7 місяців тому
DP-203: 32 - A closer look at Databricks notebooks
DP-203: 31 - Introduction to Azure Databricks
Переглядів 3,8 тис.8 місяців тому
DP-203: 31 - Introduction to Azure Databricks
DP-203: 30 - Slowly changing dimensions, data lake structure
Переглядів 2,5 тис.8 місяців тому
DP-203: 30 - Slowly changing dimensions, data lake structure
DP-203: 29 - Introduction to dimensional modeling
Переглядів 2,8 тис.8 місяців тому
DP-203: 29 - Introduction to dimensional modeling
Hello Tybul… I just passed my DP 203 exam. I would definitely say that you played a big part in it. Your videos are so good and helped me a lot in understanding the concepts. I can’t thank you enough. Thank you very much ✨
Awesome! Congratulations 🎉👏
following your content need some practice question so that i can prepare well
There are challenges prepared for you after each milestone.
I have to admit up until now I wasn't sure if Data Engineering is right for me, but seeing you design that pipeline was fascinating (I know it's a simple pipeline) but man it was awesome to watch. Hats off to you my friend
Thanks Tybul for the video and your efforts. I'm going to try these steps today.
26:28 You are absolutely right! After taking the DP-203 (especially your lessons), I am working on the DP-600, and I can tell that I have much more knowledge than it requires. It takes way less time than expected.
Joining because of your amazing explanations!
Great! Welcome aboard!
Can you please tell how can we install this application. I get the link but what are the next steps ?
Hey i dont understand one thing about SCD Type 3. Do we add 'Previous value' for every atrribute of every dimension if keeping last record is a business need? like anna changed her name, gary could change his cell no and john could change his address.
Great explanation. Glad to find this playlist
Could you please create a video series on DP-700, similar to the one you made for DP-203? Your DP-203 playlist was incredibly helpful and played a significant role in my success with the exam. Thank you for putting together such a valuable resource. I understand that creating such a playlist takes a lot of effort and time, but it would be immensely beneficial and well worth it in the end.
I believe Microsoft is shifting its focus away from Synapse, as it no longer appears to be receiving major updates, and is prioritizing Fabric for data engineering. For someone starting fresh with no prior background, it would be better to pursue certifications like DP-600 or DP-700. Even for those who have already cleared DP-200, focusing on DP-700 would be more relevant since new projects are likely to be developed using Fabric. Synapse will primarily be used only for maintaining or extending existing projects. As other companies are likely to adopt Fabric, DP-203 alone may not be sufficient for data engineering in the current scenario. Fabric knowledge is becoming essential. What do you think?
The question is: is azure going to disappear? Why would Microsoft keep two services which contain mostly the same on data engineering?
Again we need to start .. with new SAAS. Kindly start a new series for DP 700
Sir, I commend your work, and I have all the best wishes in my heart for you. I was working as a junior data engineer at a company here in Canada; your teachings perfectly align with the skills that are required to shine in this field. As a junior, I always have so many questions about data, and your course addresses those above and beyond. Please don't leave making videos; we need you. If you can do some lectures on PySpark, that would be great.
It is rare to find technical people who are also good storytellers, and you are one . Thank you for these comprehensive videos. Great work!
thanks, this is awesome.
Hey! Do these CI/CD pipelines directly related to DP-203? These videos are too much for me right now since i have zero foundation in devops. I'm pursuing dp-203 to land my first job/internship. Do i have to learn devops and CI/CD for this exam or can I pass the exam without these and focus on it after the certification because although i can follow along this video but i dont think it will do me any good until i have a strong conceptual foundation about devops and CI/CD. Thanks a bunch!
Hi, you can skip CI/CD topics completely as they are not a part of the exam scope.
I started the course the last week, i can say that the way how you explain course is to simple to assimilate, i would like to thank you for that ;)
I've spent about 10 minutes trying to think of what to write. All I want to express is my immense gratitude for this amazing playlist. The effort you've invested in each video and topic is evident. I've just finished the playlist, and I'm marking this comment as a personal milestone. I hope to return when I've obtained the certification and landed my first job. Thank you once again.
Not only prepping a person for the cert but asserting best practices and big picture thinking. PHENOMENAL AND SENSATIONAL
Love your way of teaching but getting tired of ' It's still not the best solution' at the end of the video everytime 😆
True:) In real life, a "good enough" solution is often more valuable than a perfect one, as it can be implemented more quickly.
I have passed my DP-203 exam, thanks to your lessons! They were my main source for studying and practicing (along with practicing some free exam questions from ExamTopics). Regardless of the exam, having someone like Piotr, with his experience and expertise in the market, to teach you is a big deal. Additionally, providing milestones with projects to work on is another factor that boosts confidence in your skills and deepens your knowledge. Thanks again, Piotr! Looking forward to your next topics.
Awesome! Congratulations 🎉
Hey there, great explaination ! please upload more such videos. ALso can you let us know if there is any other platform to enhance our KQL skills in an investigative way. Thanks again !
I'm not aware of any other platform other than Kusto Detective Agency. But the good news is that a brand new season is under construction!
Perfectly explained all concepts. Well done Piotr! The course is truly inspiring and the "Brief history of data architectures" should be watched by anyone who is working with modern architectures.
Glad you liked it!
Passed my exam today. Thanks for the awesome videos!
Awesome! Congratulations 🎉
but how can we limit access to our event hub as anyone with the connection string of the share access policy can send event data there? i think I can restrict the access through networking! or there are other methods?
You can also use RBAC (if that's applicable in your case).
i actually wanted to skip this episode because I thought that as synapse will be retired at some point in the future will not be needed to know but actually concepts revolves around data factory concepts that every data engineer must know . great video! thanks Tybul
Thanks!
your just blowing me away with your knowledge..... your a Addiction
never thought you would go so deep in subjects....... your a addiction
your putting your soul to make the topic understandable... hats off to you - your an addiction :)
thanks a lot
nice explanation - your a addiction.:)
If Azure Synapse is (or will soon be) dead (not much used) , why is the DP-203 exam so heavily focused on it?
I guess that when DP-203 was released, Microsoft wanted to promote Synapse a lot.
I have a question for students : How can I get a free Azure account? I had one, but after one month, I couldn't use it anymore because my subscription was disabled.
According to Microsoft documentation: "After you use your credit, wait until the 12 months are over and sign up again if you’re still a student. You'll get $100 credit and free services-just like before." Source: azure.microsoft.com/en-us/free/students
Thanks for your dedication in creating this series of videos! They are really helpful for understanding how to build secure data engineering solutions. I will have my exam in a week and many of the concepts I had doubts about were clarified by watching your videos. Kudos to you! 😁
Thanks! I really appreciate your support.
@@TybulOnAzure Actually, I have changed my exam date to today, and I've just passed! Thanks once again 👏
@menezesnatalia congratulations 🎉
I got my certificate yesterday, thank you a lot for your effort it's very usefull. A nice channel to prepare the dp203 exam
Awesome! Congratulations 🎉
A quick question From your point of view, what is the best method to monitor the performance of my Dedicated SQL pool, track resource utilization, and determine if I am using the SQL pool efficiently?
Hi, take a look at Azure Synapse Analyzer Report: techcommunity.microsoft.com/blog/azuresynapseanalyticsblog/azure-synapse-analyzer-report-to-monitor-and-improve-azure-synapse-sql-dedicated/3276960
Thanks for the video and the course itself! Would you be so kind to share queries you've used, so it would be easier to follow along?
I've just uploaded the json definition of the whole ADF pipeline that contains those queries. You can find it in my GitHub (link is in the video description).
Best session on connection with ADLS. Thank you
My pleasure
do you think we will be forced to data movement somehow!? for example, orders table that has our customer's orders, suppose that if we use our customer ID column in hash distribution (evenly distribute our data) if we run a T-SQL query that groups or filters by product category or date, it will need to retrieve data from other nodes. or am I wrong?
Yes, that could happen. The reality is that your data distribution method won't be able to satisfy every possible query, so it's best to focus on optimizing the most important and frequent ones.
Thank you so much for the time and effort and deatils you put in these vidoes :)
I'm glad you like them!
Your a Addiction
I just stumbled across your channel, lucky are the people who have just started on their data engineering journey.... but one thing is for sure " your a addiction" :)
Hey Piotr , Thank you for this session, You talked about secuirty to access to the datalake but you didn't talk about how to manage security stuff when we work with other storage services like : sql database , is it the same ? or you gonna talk about it in other session ? Thanks
SQL DB security - not in scope of DP-203. Synapse SQL security - in scope and covered later in the course.
@@TybulOnAzureThanks
38:53 that "better for you to get the correct answer" look xD
Did you know the answer? ;)
Excellent content as always, Piotr! I have a question. Instead of using Key Vault, couldn't we simply assign a Reader role to ADF in the Storage account and connect using the System-assigned Managed Identity in Linked Service? Did you choose the Key Vault approach just to cover that option, or do you see any drawbacks with the method I mentioned above? Thanks a ton! ❤
Yup, the ultimate solution is to use system-assigned managed identity and skip the Key Vault at all. That's the simplest and the most elegant solution.
@@TybulOnAzure Thank you! I watched the next videos, and they made it clear. Loving your content.
Your explanation is very informative and clear. I love to watch your training. Thank you so much!
Thanks, I'm glad you like it!
Like first and then watch 😅
I did the same😂😂
I hope it was worth liking it upfront.