God Tier Data Engineering Roadmap 2024 with End-To-End Projects

Поділитися
Вставка
  • Опубліковано 21 лис 2024

КОМЕНТАРІ • 527

  • @DarshilParmar
    @DarshilParmar  11 місяців тому +37

    Let me know your thoughts in the comments and don't forget to hit LIKE and SUBSCRIBE :)

    • @sohamshah4666
      @sohamshah4666 10 місяців тому

      why you join Microsoft? you have your freelancing and your own one!?

    • @Coolthoom
      @Coolthoom 10 місяців тому

      @darshilparmar Thank you so much for your videos. I'm trying to purchase your 3 courses through the combo plan link but the page isn't loading for some reason. The individual courses load properly but the combo one doesn't. Kindly advise further.

    • @sivasankardevendla4108
      @sivasankardevendla4108 10 місяців тому

      ​@@CoolthoomDid you purchase Courses?

    • @BShabnam
      @BShabnam 10 місяців тому

      Can you please provide me your discord link, the link which mentioned is showing as expired

    • @logsofzany
      @logsofzany 8 місяців тому

      Bro can you please mention the duration for each of your courses?

  • @samarjeetsinghkheda9460
    @samarjeetsinghkheda9460 8 місяців тому +44

    1) Book:- fundamentals of data engineering
    2) Data warehouse (olap vs oltp, dimension table, ETL, ER-modelling, dimension modelling)
    3) Snowflake (data warehouse)
    4) Book:- The data warehouse ETL toolkit
    5) Data processing (bath processing, real-time data processing)
    6) Apache spark - real time data processing (core architecture, high level API, functions available)
    7) Data spark (Pyspark)
    8) Apache Kafka
    9) Apache Airflow (workflow)
    10) cloud
    11) open table format
    12) data observability tool
    13) docker, Kubernetes

  • @phucluong6341
    @phucluong6341 9 місяців тому +6

    00:04 Data engineering is a fast-growing field with high demand and numerous tools available.
    02:24 Acquire computer science fundamentals and foundational data engineering skills.
    04:41 SQL is the universal data language for communicating with databases.
    06:58 Learning approach: Combine video learning with reading a book for effective learning.
    09:11 Learning about data warehouses includes foundation and tools.
    11:33 Data engineering involves bringing various data formats together and processing them efficiently.
    13:52 Learning Data Engineering and Cloud Platforms
    16:09 Open table format and data observability tools are key trends in data engineering.
    18:10 Stay consistent and learn in public
    20:15 Encouragement to engage with the video content

  • @ahmetalperenyildirim6184
    @ahmetalperenyildirim6184 10 місяців тому +13

    Darshil, thank you! I'm following you for 2 years and now working as Data Analyst. I want be a DE! Great topics quite clear, thank you for sharing!

    • @khushbukeshari5577
      @khushbukeshari5577 8 місяців тому

      Hi
      Can you please help me on this? Can you provide me the roadmap how to move to as a data analyst? What are ths courses are required?

  • @VirajPrabhu-nn9ny
    @VirajPrabhu-nn9ny 4 місяці тому +2

    As a Data Engineer I can say this is a very good roadmap! Very well structured

  • @alansmailov6147
    @alansmailov6147 11 місяців тому +21

    watched till the end and already committed for 4 month, just learned basics of ci/cd in azure devops. hope getting a decent job in new year. massive thanks to u Darshil and all the community u building here! happy new year from Kazakhstan, Almaty

    • @LOFI_WORLD_SONG
      @LOFI_WORLD_SONG 10 місяців тому

      Hello can you tell me what are challenging and hard in data engineering also. I am in devops should I continue or should I switch to data engineer. As I don't want coding !!!

    • @alansmailov6147
      @alansmailov6147 10 місяців тому +6

      guys i did it, landed my first job as Azure Data Engineer, huge thanks to Darshil

    • @alansmailov6147
      @alansmailov6147 10 місяців тому

      @@LOFI_WORLD_SONGhonestly i don’t know😅

    • @MohamedSahith87
      @MohamedSahith87 6 місяців тому

      @@alansmailov6147 where i can learn data engineering?

    • @ChristianeRhelyJoselleA.Bacani
      @ChristianeRhelyJoselleA.Bacani 4 місяці тому +1

      @@LOFI_WORLD_SONG if you don`t want coding as sad I said it but Data Engineering is a hard coding jobs you will automate the task as much as possible and the core concepts is data pipelines and databases so you need to code as much as possible too so yeah if you don`t want coding data engineering is not for you they are software engineers also that specialized in backend

  • @tamarainwang6362
    @tamarainwang6362 10 місяців тому +2

    Getting my first data engineering role in 2024. Thank you!!

  • @hritikapal683
    @hritikapal683 11 місяців тому +12

    Already followed your 2023 roadmap and learnt everything mentioned just building projects these days thankyou Darshil watched this to get the knowledge of modern data engineering tools quality content as always!

    • @DarshilParmar
      @DarshilParmar  11 місяців тому +2

      Thank you

    • @dawateden-yh4yy
      @dawateden-yh4yy 11 місяців тому

      @@DarshilParmar MAKE COMPLETE COURSE AS SHASHANK WEBSITE GROW DATA SKILLS

    • @prof.mangabhai
      @prof.mangabhai 11 місяців тому

      Hey hritika , how did you build your project, any good UA-cam channels or resources you used?

  • @srajangupta2921
    @srajangupta2921 6 місяців тому

    00:04 Data engineering is a fast-growing field with high demand and numerous tools available.
    02:24 Acquire computer science fundamentals and foundational data engineering skills.
    04:41 SQL is the universal data language for communicating with databases.
    06:58 Learning approach: Combine video learning with reading a book for effective learning.
    09:11 Learning about data warehouses includes foundation and tools.
    11:33 Data engineering involves bringing various data formats together and processing them efficiently.
    13:52 Learning Data Engineering and Cloud Platforms
    16:09 Open table format and data observability tools are key trends in data engineering.
    18:10 Stay consistent and learn in public
    20:15 Encouragement to engage with the video content

  • @rupeshphanindrasaiande7993
    @rupeshphanindrasaiande7993 16 днів тому +1

    I am going to be data engineer by December 2024 for sure.... My journey started with Data engineer :)

  • @adib_shaikh
    @adib_shaikh 11 місяців тому +7

    As per your roadmap that you mentioned in 2023, i just completed the advance postgresSQL , now i sharpening my skills with python, thanks for guiding us❤️ hope I'll become Data engineer , with pursuing your roadmap in 2024😊

    • @Greyhat2
      @Greyhat2 11 місяців тому

      Same case here

  • @karthikvibuthi9635
    @karthikvibuthi9635 7 місяців тому

    I will start my data engineering journey today. With many projects I will gain skills from these road map as well as essential technologies outside the course and as they emerge

  • @faruqolasupo2277
    @faruqolasupo2277 10 місяців тому +3

    Thanks for sharing this roadmap - watched to the end with ❤ from Nigeria

    • @peterobumneme3911
      @peterobumneme3911 7 місяців тому

      Good day. I am also a Nigerian pathing on the journey of Data Engineering. can we link up??

  • @joelsarpong1847
    @joelsarpong1847 11 місяців тому +6

    I will become a Data Engineer in the year 2024 due to your road maps and your course provided. Looking forward to the rest of the courses coming up.

  • @njorogekamau3820
    @njorogekamau3820 5 місяців тому +5

    I'm giving myself at least one year to learn data engineering without any expectations. I hope to revisit this comment again next year having landed a data engineering role.

    • @AHSAN88077
      @AHSAN88077 4 місяці тому +1

      Me too on the same path.

    • @Vincent45678
      @Vincent45678 4 місяці тому

      ​@@AHSAN88077 Same

    • @HoopAlchemy
      @HoopAlchemy Місяць тому

      How is the journey so far

  • @payalpokale214
    @payalpokale214 10 місяців тому +3

    Hey Darshil,I am working as data engineer since 9 years almost now, couldn’t stop myself to go till last, good content man, almost have touched based all the tools & technologies are mentioned. Yeah it’s like big sea one should keep learning and growing. Keep up the good work and keep creating good content.

    • @newagene8276
      @newagene8276 10 місяців тому

      what would you suggest a person wanting to switch from swe to data engineering

    • @ankurghosh2387
      @ankurghosh2387 7 місяців тому

      hey can I connect with you for guidance?

  • @vasundras4746
    @vasundras4746 5 місяців тому

    This is the best video on UA-cam right now 💯 we need more of this energy !!

  • @sabyasachichoudhury2717
    @sabyasachichoudhury2717 10 місяців тому +4

    1. Positive Mindset
    2. Stop Distractions
    3. First 5 videos CS50 By havard university
    4. Python
    5. SQL
    6. Raed basics of data engineering
    7. Data warehouse - snowflake
    8. Data processing - Batch processing - Apache spark
    Streaming Data - apache kalfka
    9.Workflow management tool -Apache work flow
    10. AWS

    • @malikmudassarawan
      @malikmudassarawan 10 місяців тому

      Consistency is the key and to have consistency you need proper motivation. Mostly we loose motivation along the way

  • @mrweirdo163
    @mrweirdo163 10 місяців тому +87

    I am gonna become a Data Engineer in 2024 💪

  • @ayandazwane_
    @ayandazwane_ 10 місяців тому +2

    Thank you, this video was very informative, I watched all of it and I will certainly become a data engineer.

  • @navarrodba
    @navarrodba 11 місяців тому +4

    20 years as an Oracle DBA. Finally I’ll move do DE

    • @darshanmr5425
      @darshanmr5425 2 місяці тому

      Actually I am trying to change my domain to data engineering. I am currently working in oracle. Can u please guide me.

  • @fahadbawazir562
    @fahadbawazir562 9 місяців тому

    Data Engineering + DEVOPS (Aws or GCP or Azure) that is Docker + Kubernetes = ULTIMATE Data Engineer

  • @Shahnawaz_Ansari_
    @Shahnawaz_Ansari_ 11 місяців тому +4

    Great video and resources mentioned
    Appreciated 👏

  • @Maverickghst
    @Maverickghst Місяць тому

    Thank you for the detailed roadmap & guide. I will become a data engineer in coming months.

  • @kavyamuralidharan6878
    @kavyamuralidharan6878 11 місяців тому +1

    Very clear and driving roadmap video, thanks for the guidance!😃

  • @moinshaikh8551
    @moinshaikh8551 9 місяців тому

    I’m going to become a Data Engineer!
    Thank you for your useful information!

  • @soumobratamanna6681
    @soumobratamanna6681 11 місяців тому +1

    I alreasy started learning data engineering after watching your past videos , I will be become data engineer . Thanks for the clear roadmap and instructions .

    • @DarshilParmar
      @DarshilParmar  11 місяців тому

      All the best

    • @soumobratamanna6681
      @soumobratamanna6681 11 місяців тому

      @@DarshilParmar Thanks a lot for sharing such insightful content

    • @soumobratamanna6681
      @soumobratamanna6681 11 місяців тому

      @@DarshilParmar Previously I was working as a data engineer in a early growing startup company in Bangalore. I resigned last month only due to some project issues. Now actively looking for a job . Would like to get some suggestions from you regarding job hunt in data science domain

  • @angellavado763
    @angellavado763 11 місяців тому

    I'll become a master in Data Engineering in 2024! 💪🏼🤩 Greetings from Peru 🇵🇪.

  • @Korudo
    @Korudo 9 місяців тому

    I’m going to become a data engineer.
    I do data integration in my day job, and I want to grow and become truly excellent at this.

  • @the_witcherr275
    @the_witcherr275 4 місяці тому

    I watched this this video till the end. I really want to become data engineer. Thanks for this amazing roadmap.

  • @narenbot
    @narenbot 3 місяці тому

    Awesome god tier roadmap.. I will become a data engineer by this year .. thanks bro ❤

  • @teegeorgee
    @teegeorgee Місяць тому

    I watched this video to the end thrice. I enjoyed it. So cleared and i il be buying your course. I am a Data Engineer and there are just too many things t learn to be on top of the game.. This life is a Game. Get in touch.

  • @prof.mangabhai
    @prof.mangabhai 11 місяців тому

    Highly anticipated for spark and aws courses !!

  • @kalyanishende618
    @kalyanishende618 8 місяців тому

    Thank you Darshil... I am on the journey to become DE now watching this video and the roadmap you mentioned in the video is pretty cool. Thanks for sharing such useful content

  • @Cheer4SK
    @Cheer4SK 10 місяців тому +1

    I’m going to become a data engineer! 🥳 You guys have got this, too! 🌟

  • @SatyaMalik-id5vm
    @SatyaMalik-id5vm 7 місяців тому

    Thank you for being this clear and complete roadmap i just started my data engineer journey ❣️❣️

  • @SiempreSunshine3
    @SiempreSunshine3 9 місяців тому

    My goal is to become a data engineer by the end of the year 2025 mastering all the concepts and technical tools related to this field

  • @AYUSH-KHAIRE
    @AYUSH-KHAIRE 2 місяці тому +1

    sat 14 sept 2024 : I am going to become the best data engineer !

  • @haridatta5495
    @haridatta5495 10 місяців тому +1

    Watched till the end!! Great content. Will start learning from today

  • @akhilsrivatsav
    @akhilsrivatsav 10 місяців тому

    I will be data engineer by March. And watched your video completely. ❤

  • @wckd6726
    @wckd6726 10 місяців тому +1

    Watched it till the end and I swear to god I'm going to become a data engineer this year

  • @sibusisokanityi
    @sibusisokanityi 10 місяців тому

    Great road map to data engineering
    I'm currently pushing data science but I also love data engineering....so now I'm pushing both 😅

  • @sabarishv9043
    @sabarishv9043 6 місяців тому

    Excellent Road map, sure I'll become data engineer in two months ❤

  • @grant5941
    @grant5941 10 місяців тому +6

    Thank you, Darshil! I watched until the end! I already started Harvard Cs50 last week so I'm on my way. Thank you for this roadmap.

  • @deepthireddy3480
    @deepthireddy3480 11 місяців тому +2

    I am going to become data engineer..
    I have watched your video till the end..
    And i wanna say you something that you are going to reach 200k subscribers very soon...
    Thank you so much for spreading such a great knowledge..

  • @FarhadYousefiRazin
    @FarhadYousefiRazin 9 місяців тому +1

    Hi. Thanks for the roadmap, everything was clear. I didn't undrestand just one thing. You said we better to stick to just one web service like AWS. I wanted to know its function in the learning path, like what will we do in the web service

    • @DarshilParmar
      @DarshilParmar  9 місяців тому

      Get started, you'll understand it, create account and do my UA-cam data Analytics projects

  • @harishparamathmananda1385
    @harishparamathmananda1385 10 місяців тому

    Watched till the end! Thanks for the tips and tricks, I'll become data Engineer in less than 6 months. 🎉

  • @Reactionz.3402
    @Reactionz.3402 4 місяці тому

    I'm going to become by Data Engineer before the end of 2024. The journey just began.

  • @Aditi99177
    @Aditi99177 10 місяців тому

    Stayed till the end ❤
    A data analyst here, will be a data engineer in 2024 ❤
    Thanks for the video

  • @simpledrawing7284
    @simpledrawing7284 11 місяців тому +7

    Hi Darshil, can you make a video on role of data engineering in Generative AI and LLM models.

  • @sanjanagoud1155
    @sanjanagoud1155 10 місяців тому

    Thank you for the video Darshil..... I am going to start my career in the field Data Engineer. for sure in future i will become an Data Engineer.

  • @dreamlands3970
    @dreamlands3970 10 місяців тому

    I am now watched complete video i will become data engineer in 2024 from basic to strate from now❤

  • @adityaloya161
    @adityaloya161 9 місяців тому

    I am going to become a Data Engineer. I watched this video till the end. Thank you so much for the content.

  • @jigarrthaker2441
    @jigarrthaker2441 6 місяців тому

    Hi Darshil,
    Thank you so much for providing such clarity Data engineering path,
    Currently I am working at organization which handle Splunk stacks and also i handle and work on splunk stacks which are on Snowflakes and also i work on Docker and kubernetes so somewhere this roadmap gives me clarity and open opportunity path to get carrier Data Engineering.
    Thank you once again and keep updating us with such of informative knowledge and also from your experience.

  • @fahadbawazir562
    @fahadbawazir562 9 місяців тому

    please make Apache Spark + Data bricks along with Snowflake complete video with handful of exercises & its application in the real life scenario, much awaited your reply

  • @lily0721
    @lily0721 11 місяців тому

    I’m going to become a Data Engineer!
    Thank you for your useful information!

  • @AryanRaghuvanshi-tm9el
    @AryanRaghuvanshi-tm9el 11 місяців тому +2

    I have watched your end to end project videos previously, really amazing content you have been providing, I will check out your courses.. keep uploading videos like these ..

  • @viveksingh112
    @viveksingh112 10 місяців тому

    Thanks Darshil, I watched this video till end and will keep following you to build my journey to become a Data Engineer

  • @davebatista8636
    @davebatista8636 10 місяців тому

    @18:25 Yes sir, I watched this video till end.

  • @ManiKanturi
    @ManiKanturi 10 місяців тому

    Thank you :)
    I'm going to become the Data Engineer.
    Please make the course for the remaining topics or tools.

  • @vishwasoundararajan6841
    @vishwasoundararajan6841 Місяць тому

    I will became a data engineer and watched till the end🎉...thanks for sharing❤❤

  • @nanayaw7159
    @nanayaw7159 9 місяців тому

    I am going to be a Data Engineer in 2024 and the years beyond

  • @hazarijayesh9807
    @hazarijayesh9807 11 місяців тому

    Intrested , watched video till the end. ready to start journey for data engineer.

  • @KeyBooksInsights
    @KeyBooksInsights 11 місяців тому +2

    Everyone is teaching roadmaps but no one is teaching domain knowledge like retail, hospital,aerospace etc. please do a video.

    • @DarshilParmar
      @DarshilParmar  11 місяців тому +2

      I have many projects, those are enough to get you started with all of these

    • @KeyBooksInsights
      @KeyBooksInsights 11 місяців тому

      ​@@DarshilParmardoes a video regarding your day to day life.

    • @abhinavtiwari4238
      @abhinavtiwari4238 10 місяців тому

      Domain knowledge matters the most with the advent of AI which can be utilized for technical part

  • @hemanthsagar3537
    @hemanthsagar3537 7 місяців тому

    Thank you for your inputs! Completed Databricks certified Data Engineer associate after following your certification path. I am following every video that you upload. Thanks alot ❤

  • @oshebase
    @oshebase 11 місяців тому

    Thanks for this guide. I'll bevome data engineer come 2024

  • @_k_a_m_a_l.
    @_k_a_m_a_l. 11 місяців тому +1

    I am working as an ETL developer and I will become a Data engineer in 2024
    we are using Vertica Data Warehouse
    My doubt is which is good Vertica or Snowflake
    I saw some blogs that Vertica is more powerful in query performance than Snowflake

  • @stevenlomon
    @stevenlomon 7 місяців тому

    I watched til the end and I am going to become a data engineer! Love and good vibes from Sweden

  • @LeftFootOfPodolski
    @LeftFootOfPodolski 9 місяців тому

    Thank you for this video Darshil! I am already a Data Engineer but my foundational knowledge is not the best and there is so much I still do not know, so thank you for explaining so thoroughly 😊

  • @atharvajoshi1003
    @atharvajoshi1003 7 місяців тому

    Watched till the end. Really inspiring video for all data engineer aspirants.

  • @mr.amitdas1541
    @mr.amitdas1541 11 місяців тому +1

    i ve completed this vedio.. This is very insightful.. Thnk you for the guid.

  • @okorochaobinna7701
    @okorochaobinna7701 10 місяців тому

    You're a blessing to an aspiring data engineer

  • @manojreddy2871
    @manojreddy2871 5 місяців тому

    thank you sir, for giving us a helping hand to become a data engineer.

  • @karenemody
    @karenemody 8 місяців тому

    Thanks Darshil. Very informative. Watched till the end.

  • @ShivamKumar-pk1rf
    @ShivamKumar-pk1rf 10 місяців тому

    Got the roadmap Thank you so much

  • @icon5819
    @icon5819 11 місяців тому

    Hi Darshil, can you please drop rest of the courses quickly. I want everthing related to the course at a single place and your course is worthy enough for that

  • @jbh7079
    @jbh7079 10 місяців тому

    Excellent content. Keep the good work up!

  • @amoghshukla2292
    @amoghshukla2292 2 місяці тому +1

    I am going to become a Data Engineer & Learn Every SKills Mentioned here before 18 feb 2025(Thats my birthday)

  • @waynelast1685
    @waynelast1685 2 місяці тому

    I don't say this lightly.... you have excellent content.

  • @Gokulprasanth-gs8zi
    @Gokulprasanth-gs8zi 11 місяців тому

    I am already in the journey, thanks for sharing valuable content

  • @venkatsaiphanindraanagam
    @venkatsaiphanindraanagam 11 місяців тому +1

    thanks for the video Darshil.Pls keep posting such awesome content.

  • @reynardtcloete6959
    @reynardtcloete6959 10 місяців тому

    Great video brother :) thanks man

  • @DataScienceAndEngineering
    @DataScienceAndEngineering 10 місяців тому

    Thanks for the video. Really Great Video.

  • @lamboarun
    @lamboarun 5 місяців тому

    Informative & Well Structured 😊

  • @Aviitr372
    @Aviitr372 2 місяці тому

    Next year till june i will develop my skillset fully to become date engineer and doing intern at end of my third year daily devotion 2.5 to 3 hr

  • @thedataholics
    @thedataholics 11 місяців тому +1

    Thanks for the knowledge, you seem really truthful. Keep helping us sir

  • @Moses.Daryoue
    @Moses.Daryoue 10 місяців тому

    Thanks a lot, you have given me a lot of information and clarity on how to start data science engineering

  • @himanshusikandriya5133
    @himanshusikandriya5133 11 місяців тому

    I will be Data engineer in 2024 and I will definitely make it count 😇

  • @saljeshmaharjan6760
    @saljeshmaharjan6760 4 місяці тому

    Watched till end, this is literally god tier roadmap

  • @guillermocelisprada
    @guillermocelisprada 11 місяців тому

    Thanks for the video man, I found it very informative and well-structured. Keep up the hard work! 👍🏻

  • @vivektyagi5744
    @vivektyagi5744 10 місяців тому

    Great video Darshil. Watched till the end.

  • @mijailsan241
    @mijailsan241 11 місяців тому

    Good Resume of maybe 5 years learning😊

  • @amitjaiswal781
    @amitjaiswal781 11 місяців тому

    Nice Video, Please also make a video on a GCP Data Engineering Roadmap.

  • @brjkumar
    @brjkumar 11 місяців тому

    Thanks Harshil for this video.

  • @namitabehera4954
    @namitabehera4954 11 місяців тому

    Yes i follow you and i completed your video. Thank you for your suggestion.

  • @Ordinary_vlogs11
    @Ordinary_vlogs11 9 місяців тому

    i'm going to become a data engineer for sure.

  • @nagalakshmia1291
    @nagalakshmia1291 11 місяців тому +1

    i have already started my data engineering career that's from now

  • @JAYESHCHAK
    @JAYESHCHAK Місяць тому

    I am definitely going to become a Data Engineer.

  • @ManishSharma-fi2vr
    @ManishSharma-fi2vr 5 місяців тому

    This video is gonna be a game changer for me.

  • @jk_roh3t
    @jk_roh3t 9 місяців тому

    I am following this roadmap