What no one tells you about Data Engineering Real time Project? Reality and Facts in 15 minutes
Вставка
- Опубліковано 14 лип 2024
- #DataEngineering #ProjectCreation #RealTimeDataEngineering #DataEngineeringTutorial #DataEngineeringProject #PreImplementationPhase #ImplementationPhase #PostImplementationPhase #DataEngineers #SolutionArchitects #DevOpsEngineers #InfrastructureEngineers #ProjectManagers #ModernDataPlatform #Terraform #CICDPipelines #DataPlatformMaintenance #DataEngineeringCareer #DataEngineeringInterviewTips #AzureDataFactory #Databricks #AWSDataEngineering #GCPDataEngineering #DataPlatformArchitecture #InfrastructureAsCode
Welcome to my channel! In today's video, we’re diving deep into the creation of a real-time Data Engineering project from scratch. This comprehensive guide covers everything you need to know, from the initial discussions to the final implementation and maintenance. Unlike many tutorials you might have seen, we’ll explore every detail of the process, making it a must-watch for anyone looking to break into the Data Engineering field.
What We’ll Cover:
1. Introduction
2. Key Roles Involved
3. Pre-Implementation Phase
4. Implementation Phase
5. Post-Implementation Phase
Throughout the video, we’ll provide detailed insights into each phase, emphasizing the critical role of Data Engineers across all stages. By the end, you’ll have a thorough understanding of the complete lifecycle of a Data Engineering project, equipping you with the knowledge to excel in your career or ace your next interview.
If you find this video helpful, please like, share, and subscribe to my channel for more insightful content. Let's get started on this exciting journey into the world of Data Engineering!
Cheers, and see you in the video!
TimeStamp:
0:00:00 - Intro
0:00:59 - Key Roles Involved
0:02:25 - Pre-Implementation Phase
0:07:17 - Implementation Phase
0:12:27 - Post-Implementation Phase
- - - Book a Private One on One Meeting with me (1 Hour) - - -
www.buymeacoffee.com/mrktalks...
- - - Express your encouragement by brewing up a cup of support for me - - -
www.buymeacoffee.com/mrktalks...
- - - Other useful playlist: - - -
1. Microsoft Fabric Playlist: • Microsoft Fabric Tutor...
2. Azure General Topics Playlist: • Azure Beginner Tutorials
3. Azure Data Factory Playlist: • Azure Data Factory Tut...
4. Databricks CICD Playlist: • CI/CD (Continuous Inte...
5. Azure Databricks Playlist: • Azure Databricks Tutor...
6. Azure End to End Project Playlist: • End to End Azure Data ...
7. End to End Azure Data Engineering Project: • An End to End Azure Da...
- - - Let’s Connect: - - -
Email: mrktalkstech@gmail.com
Instagram: mrk_talkstech
- - - About me: - - -
Mr. K is a passionate teacher created this channel for only one goal "TO HELP PEOPLE LEARN ABOUT THE MODERN DATA PLATFORM SOLUTIONS USING CLOUD TECHNOLOGIES"
I will be creating playlist which covers the below topics (with DEMO)
1. Azure Beginner Tutorials
2. Azure Data Factory
3. Azure Synapse Analytics
4. Azure Databricks
5. Microsoft Power BI
6. Azure Data Lake Gen2
7. Azure DevOps
8. GitHub (and several other topics)
After creating some basic foundational videos, I will be creating some of the videos with the real time scenarios / use case specific to the three common Data Fields,
1. Data Engineer
2. Data Analyst
3. Data Scientist
Can't wait to help people with my videos.
- - - Support me: - - -
Please Subscribe: / @mr.ktalkstech - Наука та технологія
Hi Everyone! Creating this video took a lot of time and effort, and I truly hope you find it helpful. If you did, please like, share, and subscribe to my channel. Your support means the world to me and motivates me to create more valuable content for you. Thank you! 😊
Follow my LinkedIn account for regular updates: www.linkedin.com/in/mrk-talkstech/
Hi, This presentation is very different from your old videos. This presentation points out the data engineer's role in the project implementation part. I appreciate you creating these videos for the younger communities to learn and understand the environment of data science, data engineering, and data analysis. Thank you for your service.🙏🙏🙏
You nailed it again.. just awesome.
Thats so insightful thanks sir🎉
Very useful video and now I get an idea of what really happens and how are project is developed from scratch! Highly appreciated to create this for us. 🙏🏻
Thank you so much :)
Now words to appreciate...this presentation👏
Much appreciated work sir
Such a beautiful content…Thank you very much
Thank you so much :)
Thankyou I got some idea how it's working in industries
thank you very much for this video . its helpful
Glad to hear that :)
Thanks for explaining the process in detail.
Thank you so much :)
Really helpful and must needed video.please make one more video regarding how data engineer daily work activities, day to day tasks, meetings, and how to handle all these. Please make one video.
Great video! This really clarifies how live data engineering environments operate and the various phases involved. Could you please create more real-time complex data engineering projects that we can add to our portfolios and resumes? Your guidance would be immensely valuable!
Very Helpful. Thanks.
Please create more end-to-end projects on Azure Stack.
Also, Please take data which has issues so that the project flow can also show edge cases.
Thank you so much, sure will try :)
Can u pls make one big data engineering couese for aspiring professional please sir?
please make some more insightful videos on DE, it would be really helpful for freshers and less experienced. please also make a video on Tools data engineer used for ducementation and day to day activity tracker etc etc
sir can you please make a SAP data engineering project and a apache data engineering tools project playlist
Can data engineering be an implementation specialist or support specialist of the project like dba does
@mr.k does data engineers need to work on IAC as some job requirements are asking for Terrraform , docker and kubernetes as well . Is it true that for 10 DE only 2 - 3 devops/platform engineers are maintained by organisations??
A suggestion/request : Can we get on a zoom seminar where you start a data engineering project from scratch and build it end-to-end while answering question and explaining reasons behind a certain approach taken and the other ones avoided ?
Sure, will try :)
What is the next step for data engineering
Found gold
Thank you so much :)
Please start training for data engineers
Sure, will do :)
Please let us know as soon as possible the course has been initiated@@mr.ktalkstech
I am in third phase and struggling to go to second phase
You will soon, try to look for some startup :)
@@mr.ktalkstech thank you for your response, I have gone through your end to end project and did practice I got good understand can you make video series focusing on complex pipelines with parametrization and difference between full load and incremental load and creating complex whare house modeling in your up coming real time project which will helpful to understand complexity.
@@chirunjeevichiru8308 Sure, will do in the future :)
Hey I would like to know where you are from
Chennai :)
@@mr.ktalkstech great work man I loved your way of teaching You should be in a great position in future.
@@manibaddireddy5477 Thank you for your kind words :)
@@mr.ktalkstech I hope you had a great experience with various domains so why can't you share domain knowledge because technical knowledge what you are sharing is different but think about domain knowledge sharing.
Can u share ur LinkedIn profile