Azure End-To-End Data Engineering Project for Beginners | Real Time Tutorial

I hope you enjoyed, any comments questions or queries lmk.
This project addresses a critical business need by building a comprehensive data pipeline on Azure. The goal is to extract customer and sales data from an on-premises SQL database, transform it in the cloud, and generate actionable insights through a Power BI dashboard. The dashboard will highlight key performance indicators (KPIs) related to gender distribution and product category sales, allowing stakeholders to filter and analyze data by date, product category, and gender.
Tech Stack: On-Prem SQL DB, Data Factory, Data Lake Gen 2, Databricks, Synapse Analytics, Power BI, Entra ID (Active Directory), Key Vault
GitHub: github.com/luk...
Further detail on setting up SSMS and SQL: • Data Analyst Portfolio...
How to set up PowerBI without work email: • How to use Power BI wi...
Mr Ks original vid: • An End to End Azure Da...

[@mr.ktalkstech]

Would've been nice to get some credit for using my video as a reference. Please do it in the future. Good luck with yours, though :)

[@cloudconsultant]

Hi, at 00:20 I give you credit. Not having your vid in description was simply an oversight - in there now. Likewise :)

[@mr.ktalkstech]

@@cloudconsultant Thank you :)

[@MexiqueInc]

I originally developed this project with your video . I came here because all the code is available for reviewing it if you get stuck. Which makes a huge difference. He also make great job attending sql server connections issues, creating the resources and other thing you fast forwaded.Therefore it took lots of hours to figure it out. WIshing success to both.

[@mr.ktalkstech]

@@MexiqueInc That's great to know, happy his video is adding more value :)

[@cloudconsultant]

@@MexiqueInc Thank you 🙏

[@rakshithuk7414]

1st View, 1st Like. All the best. Your video Added to playlist and will be watched soon. Thanks for the project.

[@cloudconsultant]

Thanks Rakshith 🙏 I hope it helps.

[@apdoelepe7937]

Could u please put the github link for the projects in the description because i want to see the documentation

[@cloudconsultant]

In description now mate!

[@AurelieAKA]

Hi Luke,
Can I load my data directly in Azure Synapse without going through Databricks, considering that my data is already structured from SQL Server ?

[@LadyM827]

Hi Luke on 57.13 i can't see the sql script , can you please assist to write it down here on the comments.

[@cloudconsultant]

Hi! Yes, of course:
SELECT
s.name AS SchemaName,
t.Name AS TableName
FROM sys.tables t
INNER JOIN sys.schemas s
ON t.schema_id = s.schema_id
WHERE s.name = 'SalesLT'