Microsoft Fabric: End-to-End Project
Вставка
- Опубліковано 6 сер 2024
- In this video, we go through the end-to-end Power BI project using various components of Microsoft Fabric. Join us to understand the full scope of Microsoft Fabric's functionalities in real-world applications.
📊 What you'll learn:
- Set up a new Lakehouse in Microsoft Fabric.
- Upload tables from multiple sources using different ways to the Lakehouse.
- Create a data model and calculations directly within the Lakehouse.
- Build a Power BI report using DirectLake mode.
⏱️ Timestamps:
00:00 - Introduction
00:50 - Agenda
01:30 - Create a Lakehouse
02:25 - Different ways to upload data into the Lakehouse
04:00 - Create a shortcut to Fabric Warehouse
05:40 - Create a pipeline
09:54 - Turn CSV files to tables using the “Load to Tables” feature
12:30 - Turn Excel files to tables using Dataflows Gen2
22:24 - Create relationships between the tables
26:36 - Create measures
31:57 - Troubleshoot and build a Power BI report with DirectLake mode
🔗 Useful links and resources:
WideWorldImporters sample used in the video: shorturl.at/fsyM1
Lakehouse documentation: learn.microsoft.com/en-us/fab...
Warehouse documentation: learn.microsoft.com/en-us/fab...
DirectLake mode in Fabric: learn.microsoft.com/en-us/fab...
🤝 Let's Connect:
LinkedIn: / centida
Website: centida.com/
Centida GmbH
34 Kernenhofstrasse, Oberkirch
77704, Germany
Email: info@centida.com
#MicrosoftFabric #PowerBI #Training #EndToEndProject
👍 If you found this video helpful, please give it a thumbs up, and consider subscribing to stay updated with our series. Your engagement helps us create more valuable content for the community. - Наука та технологія
Thanks man!
Thanks for sharing.. earned a new follower
great demo
You're welcome. Please let us know if you have any questions.
awesome comprehensive
Thank you. If you have any further questions on how Fabric works, feel free to let us know. Have a good one.
what I have been looking for, this is great!
Thanks, we're glad you liked it!
amazing video. tnks you
Glad you liked it!
Very Well explained, Thank you so much for sharing this! . Can you also advise on the integration of Azure Key vault in Fabric..
It’s really helpful
Thank you. We are glad that it was helpful for you. Is your company thinking about using Fabric in their operations?
Very Good...
Thank you. I am glad you liked.
Thank you! Your teaching style is awesome! This is the first of your videos that I have watched. Can I ask you for some pointers in loading XML files?
Riz
Thanks for your comment. I'm glad you liked it.
As for the XML files, you'd first ingest them into OneLake using a ADF pipeline. Then, parse and transform using PySpark to create a structured Delta table.
I love this so much.
Can I have access to the data used so I could practice it?
Would really love that
Your 'error' on the column data type (price), was actually very instructive...are you sure you didn't do it on purpose (laughs!) Good to see how to problem solve : thanks. OH, and super video. Glad you made it fairly straighforward.
Thanks a lot!
No, it was a real error, honestly :)
Great video. Once the modelling is done, I could also use PBI desktop to connect to the model and build a report, right?
Thank you. Yes, you can also connect to a Power BI dataset in Lakehouse using the standard Power BI Desktop. When building production reports, it is recommended to use the desktop version, as it has more features available than the browser version.
Thanks for sharing this. For create relationship, is it better in SQL Analytics Endpoint or by create new dataset (Semantic model). And may I know the pros and cons between the two ? Thanks again.
Hi, it's Nik here. In my personal opinion, if you have PBI Desktop on your computer, use that to model and build reports. It gives you a full range of capabilities.
The browser version is a good solution for a quick analysis and putting together a basic report; or if you don't have Power BI Desktop on your machine. Now that Fabric is GA, I'm sure Microsoft will improve and change the SQL endpoint view and the report building experience in the browser.
Currently, the two biggest drawbacks of the browser version are:
1. You don't have a full set of options and capacity to manage a model, like you do in the Desktop version. For example, you cannot create new tables and columns.
2. Currently, the browser version does not have a proper version control, e.g. everything you do in the browser is saved immediately. This can be a concern for some developers until Microsoft comes with a solution to fix this.
Hope that helps. Thank you!
Thanks for making this. Fabric seems to be getting traction among enterprises. Do you know when the copilot is out and how can we start using that?
Yeah, companies are taking a closer look at Fabric, now it is Generally Available. As for the copilot, it is out in a preview mode now. Microsoft is aiming to roll out Copilot by the spring 2024.
thanks for the video @centidaBI and I have a question/ How you loaded the factorders.csv file into data warehouse ?
I could see other files as well loaded into warehouse. can you help me with the approach to load the csv files into warehouse ?
instead of using flow for excel we can use notebook to populate data into new tables from excel
How did you increase font size in formula bar at timestamp of 27:20. Please help me with the shortcut.
Hi Amit. It's super simple - hover over the measure line and hold CTRL + scroll your mouse wheel up.
Thank you for the lesson. I have an doubt in the line chart, we can arrange the month in order on powerbi desktop .What we have done in powerbi sortby column that i cannot able to perform in fabric. Can you able to teach me to sort out this problem.
This is a good question, thanks!
As of today, when building Power BI reports in the browser, you have some limitations when editing datasets. One of these limitations is the option to sort one column using another column. For example, when you need to sort the month name (January, February, March) by the column month number (1, 2, 3, …).
One workaround is to use Tabular Editor. Here is the step-by-step process:
1. Open Tabular Editor > go to File > Open from DB.
2.In the field “Server” you need to put your Fabric workspace connection string. This can be found in Workspace Settings > Premium.
3. For the Authentication method choose Windows Integrated or Azure ID login. Once authenticated, connect to your Lakehouse or DWH.
4. In the object explorer, find your table under the folder “Tables”.
5. Select your column (MonthName). Under the Basic group of properties, find the field “Sort by Column”. Choose the name of a column by which you want to sort the selected column (Month Number).
6. In the upper menu, find the button “Save the changes to the connected database”.
7. Go back to the Power BI report and update it. After the update, you should see the changes.
Hope this works for you.
Thank you for this demo. Can you share the Order Details and Fact Orders as a csv? This way we can still complete the training without the shortcut or pipeline data load.
Thanks for feedback. My bad, it was added in the shared folder, the link is in the description below.
Thank you :D@@CentidaBI
My company has some data on Azure SQL that refreshes once every 24 hours.
I can write dataflow to bring selected rows into PowerBI.
However I want to create some reusable summary tables for which I think Python would be great.
Is it possible that in Lakehouse,
1- I create Dataflow that brings data from Azure SQL into a table
2 - Create scheduled pipeline that runs every 24 hours and runs the Dataflow that overwrite my table
3 - Use Pyspark or something similar to create the summary table
4 - Write that table to Lakehouse
I am not sure how step 3 and 4 will be automated on schedule? And I am not sure if above is possible at all?
Can you please help?
Hi, thanks for your comment and apologies for a late reply. With the holiday season got caught up in a lot of things.
Re - Summary tables. If you're working within the Microsoft ecosystem and the data transformations are straightforward, Power Query within Power BI might be a simpler and more integrated option (this is Nik's personal opinion). If the transformations are complex and you're more comfortable with Python, then go with Python.
Answers to your questions:
1. Yes. You can definitely use Microsoft Fabric to import data from Azure SQL Database into your Lakehouse, given you have all the permission.
2. It's possible to configure an ADF pipeline to automate your Dataflow, with schedules to refresh and overwrite data as per your needs every 24 hours.
3. PySpark is a good choice for summarizing data within Microsoft Fabric.
4. You should be able to automate data summarization and updates in Lakehouse notebooks, with scheduling handled by pipelines.
During Power query online get data from one drive is it possible to point it to folder so all files inside gets loaded in one shot ?
Very good question, thank you! I have not tested it myself yet. But here is a video I found that talks about this: ua-cam.com/video/zoH_9L9sPYs/v-deo.html
Is there a way to prevent snakes and mosquitoes? I guess if the water isn't stagnant mosquitoes may not come but snakes can get attracted to water. I wasn't sure if this was accounted for during the construction.
Beautiful home... absolutely phenomenal.
u should provide the data set link that you have used in the project..which will be helpfull for us to practice along with you
Hi, thanks for pointing this out. Just added a SQL script to create the Fact table.
@@CentidaBI thank you very much
@centidabi @nik, Great video on end-to-end fabric. is there way I get the same dataset for trying out.?
Thanks for pointing this out. My bad, just added the SQL script to create a Fact table: centida-my.sharepoint.com/:f:/p/nikolai_pavlov/Elp_2jvMZlFLleo32E1-lN4BS92bJKV-AjNa7ZKBFi4DlQ?e=Z9UBiU
thats so great of you to respond thanks.. @@CentidaBI
Would also share the FactOrders and Order Details@@CentidaBI
@@CentidaBI Please can you share also the FactOrders and Order Details ?
Hello, @5:05 How to get the WWI_FactOrders table? Please help. Thank you in advance.
Hi, thank you for your comment. Yes, it's there now.
in this whole process, were any landing zones used, if yes where , when ? i m beginner ;)
Thanks for your feedback. No, we did not use any landing zones. Data was imported using a Dataflow, a pipeline, and shortcut from a warehouse.
I have lerned powerbi and sql. Should I learn datafabric, azure data factor to get job?
Thank you for your feedback! If you're already proficient in Power BI and SQL, I'd recommend focusing on getting hands-on experience with real-world projects to further strengthen your skills and build your portfolio.
And Python
Dude. The column "Avg Order Value" shouldn't be a percentage value.
Hi. You're right, I made a mistake. Thank you for pointing this out!
For some reason, I was thinking about the profit margin type calculation in my head while recording the video. Apologies for a confusion. But the main idea of the video is to show how to go through a project from end-to-end.
It's a great video and inspired me a lot but I thought you may think of adding a note to that section.