The Ultimate Big Data Engineering Roadmap: A Guide to Master Data Engineering in 2024

Поділитися
Вставка
  • Опубліковано 8 січ 2025

КОМЕНТАРІ •

  • @amarjeetroy5369
    @amarjeetroy5369 6 місяців тому +111

    Summary
    1. Any Programming Language (Python/Scala/Java)
    2. Hadoop Ecosystem
    - Apache Spark ( Preprocessing Framework)
    - HDFS (File Storage)
    - Hadoop YARN (Resource Manager)
    - Apache Mesos
    3. Storage System
    - SQL (RDBMS)
    - NoSQL (MongoDB, Cassandra)
    4. Datawarehousing
    - Apache Hive
    - Snowflake
    5. Real Time Data Preprocessing
    - Apache Kafka (Data Streaming and Preprocessing)
    - Apache Flink
    - Apache Storm
    6. WorkFlow Management/Orchestration - Apache Airflow
    7. Any Cloud Service (AWS, Google Cloud, Microsoft Azure)
    8. Overall Knowledge- Data Modeling, ETL Pipeline
    9. System Design
    10. Sqoop
    11 . Data Visualization - PowerBI, Tableau, Python Data viz
    12. Competative programming/ DSA - Leetcode Practice

  • @mgdesire9255
    @mgdesire9255 7 місяців тому +50

    Yes bhaiya aap series banao..end to end data Engineering ki

  • @DEwithDhairy
    @DEwithDhairy 7 місяців тому

    For Data Engineering Interview Preparation follow my UA-cam channel ;
    www.youtube.com/@DEwithDhairy/playlists

  • @sahilkarekar2875
    @sahilkarekar2875 6 місяців тому

    without time wasting...genuine info in short with depth knowledge :)

  • @the._.pandeyji
    @the._.pandeyji 7 місяців тому +7

    OG bhaiya ❤ .. Bss aap Playlist bna do from scratch Data Engineering ka

  • @brohisarfaraz8750
    @brohisarfaraz8750 6 місяців тому +1

    Dear kindly make videos on these technologies.
    Because the way you are defining things is amazing and a person can easily understand.
    And thank you so much for this ❤❤

  • @zainfarooq9949
    @zainfarooq9949 6 місяців тому +3

    Kindly make a series on Learning Data Engineering

  • @Namangen
    @Namangen 6 місяців тому

    Man this video is a no brainer . I exactly wanted this and my first click landed me to the right place.

  • @imnotfromIT
    @imnotfromIT 7 місяців тому +2

    Yes please make Playlist for Data engineering from scratch

  • @Zeus-th6uu
    @Zeus-th6uu 7 місяців тому

    Thanks alot sir this information was really helpful for me I am starting my journey of becoming a Data Engineering from today!!!!!

  • @SohailKhan-hd8gz
    @SohailKhan-hd8gz 7 місяців тому +4

    Ye sabhi k resources per bhi ek video banayi sir . Free resources or paid jaha sey hum structure way mey padh kar apna time bacha sake ❤ very informative ❣️ Thank you 😊

  • @chinyu5934
    @chinyu5934 2 місяці тому

    i know about mongo db ,linux,shell scipting, aws services , python and their libraries, advance excel ,power bi,mysql,sql server ,pl/sql, machine learning(superwised and unsuperwised), jira(WFM),microsoft fabric..

  • @amitkadam5698
    @amitkadam5698 6 місяців тому +2

    Yes , aap total data engineering ke upar tutorial series le aao

  • @NAYANPATIL-ri5ul
    @NAYANPATIL-ri5ul 4 місяці тому

    Superb bro a perfect roadmap for freshers.

  • @SuryaKiranK-j4c
    @SuryaKiranK-j4c 2 місяці тому

    hello sir thank you so much for making this video , mene bohot logon ki videos dekhi hai koi itna clearly and simple way me bata nhi paya , you helped me a lot to get the whole picture , A new subscriber to your channel , I hope god bless you !❤

  • @charugera7654
    @charugera7654 Місяць тому

    Very useful video. Can you please do a videos on Big data vs data engineer as a career choice, salary, future prospects etc.

  • @zeeshansaud9535
    @zeeshansaud9535 5 місяців тому

    Data Engineer Job Role and AI Contribution: ua-cam.com/video/NR7L0TSFtSE/v-deo.htmlsi=k7kdCGx65-gBvpb8

  • @karnkhande5856
    @karnkhande5856 Місяць тому

    Thank you brother 🙏

  • @ojaswighate2588
    @ojaswighate2588 Місяць тому

    Please provide tutorial from beginning to end with projects.
    Please Provide big data course

  • @chiranjivmansis1415
    @chiranjivmansis1415 6 місяців тому +1

    Please make a tutorial on complete end to end data engineering courses sir

  • @hardikkanak
    @hardikkanak 4 місяці тому

    Superb explanation. could you please create end-to-end project in the single video with practical overview. it would be helpful for us. Thanks❤❤❤

  • @FIROZkhan-ud2pt
    @FIROZkhan-ud2pt 7 місяців тому

    Yes Please start making detailed vedios with each and every topic related to data engineering actually i didnt find ant detailed vedio related to big data on youtube. so please start

  • @mohammadthousif7439
    @mohammadthousif7439 6 місяців тому

    can you please make a report or something of that kind that has all the concepts and tools you mentioned in the video and resources where it can be learnt so it will be easy to follow through.

  • @namohnaman
    @namohnaman 5 місяців тому

    Best video so far

  • @vatsalrathi9466
    @vatsalrathi9466 2 місяці тому +1

    I am a 2nd year college student from tier 3 college i should go for data engineering or fullstack development in this pls suggest me

  • @wagalprasad1869
    @wagalprasad1869 2 місяці тому

    Nice video, make it complete video for cloud

  • @Buozberkien
    @Buozberkien 7 місяців тому

    Please make a playlist on priority 🤞🏻 whatever skills you have mentioned in the video. I have seen a lot of videos on youtube but at the end i always got confused. How to start , what to do … so please make the playlist for every skills which is needed . And , please abhi ke liye at least Hadoop ecosystem pe videos bnado jo jo uske part hai jese Hdfs ,hive, and pyspark , ye sab apache kafka , airflow .

  • @x-6790
    @x-6790 7 місяців тому +2

    Bro i am now in 5 th semester AI & Data science Engineering. From where should i start ?

  • @AkashVerma-ig5qv
    @AkashVerma-ig5qv 7 місяців тому

    So insightful

  • @BIKERIDER09-Z
    @BIKERIDER09-Z 2 місяці тому

    Bhaiya ye bhi puchne ki baat h please lekar aayi inke tutorials ❤❤❤❤

  • @hrushikeshkhonde7079
    @hrushikeshkhonde7079 4 місяці тому

    Thanks ...😊❤

  • @HKRajnish
    @HKRajnish 3 місяці тому

    Bhaiya Code with Harry jaisa aap v ak end to end date engineer ka series bana dijiye. With Live project 🙏

  • @shivapradhan431
    @shivapradhan431 7 місяців тому

    Hi sir, congratulations for Joining the JP Morgan
    Sir could please share or make a video on free platform to learn languages & all the topics you mentioned in the video
    because many of people or student who really wants to learn the data engineer but they can't afford the fee for the particular session or paid platforms
    A humble request from mine side to make video on this topic and
    Thanks for the road map this is very helpful me and for those who chooses their carrier towards the Data Engineer.😊

  • @tanmaydas1146
    @tanmaydas1146 Місяць тому

    Sir I am from humanities background... Having 3yrs experience in BPS sector... But I want to be a data engineer..... CURRENTLY I'M LEARNING YOUR MENTIONED SKILLS... I'M doing post graduation degree in data science from IIIT, BANGALORE
    QUESTION:-
    IS MY PAST EDUCATIONAL BACKGROUND WILL IMPACT IF I'M GOOD TO LEARN THIS DATA ENGINEER TOOLS...
    PLEASE GUIDE ME...

  • @inshorttalks
    @inshorttalks 7 місяців тому

    Yes bro . please bring on the series

  • @DataCoholic
    @DataCoholic 6 місяців тому

    Which language you chosed for giving the DSA rounds?

  • @helloabd123
    @helloabd123 Місяць тому

    Mongodb toh bandh hone wala hey iska best alternative konsa hey phir

  • @heighterses
    @heighterses 6 місяців тому

    this is awesome but how can we practice and where should we learn these things what's your recommendation on this any course or any site you prefer??

  • @VarunRithvik-c5o
    @VarunRithvik-c5o 7 місяців тому

    Please explain your VISA process for working in London ..

  • @parmarsunil8937
    @parmarsunil8937 6 місяців тому

    Hii ... Dear
    You explain very well.
    Just i want to know, is any other field person can become data engineer??
    Is it tough to learn those all thinga and become data engineer as well ad get good job in this field?

  • @nanochip1908
    @nanochip1908 7 місяців тому

    Have you used kafka in java or scala or python ?

  • @lokeshsairam8300
    @lokeshsairam8300 5 місяців тому

    bro can you help me by a video in connecting the pyspark in windows

  • @palnitin25
    @palnitin25 6 місяців тому

    Bhai abhi Mera last year start hua hai BCA me or mene data analyst ke 5 project bna rakhe hai SQL, powerbi and excel use karke or merko data analyst ke tools and technology ati hai or merko abhi data engineer ban na hai batao ab kya karo

  • @RahulKumar-q6h5k
    @RahulKumar-q6h5k 5 місяців тому +1

    It is possible without degree..... Please reply bhaiya 🙏

  • @Shruti_Gore
    @Shruti_Gore 6 місяців тому

    Bhaiyaa kya aap har skills ko kitna time dena hai kya wo explain karoge kya?

  • @xx-pn7it
    @xx-pn7it 6 місяців тому

    Bhaiya 4 month data engineering step by step roadmap bnado please 🙂

  • @AncientWonders-g5q
    @AncientWonders-g5q 7 місяців тому

    Please sir complete course

  • @Aman-lv2ee
    @Aman-lv2ee 7 місяців тому +2

    Arre bhai abhe 3~4 skills aur reh gye h, part 2 please.

  • @PrathamSingh-gr5wq
    @PrathamSingh-gr5wq 6 місяців тому

    Sir S a data science ke course me mujhe dsa kena jaruri hai kya python me

  • @DigiPathak7
    @DigiPathak7 7 місяців тому

    Please complete series banaiye DE ke liye

  • @mgdesire9255
    @mgdesire9255 7 місяців тому +3

    Bhaiya series hold pe hai kya...still waiting for video no.4

    • @theengineerguy_
      @theengineerguy_  7 місяців тому +1

      Jald hi ayega

    • @chakrabarti9634
      @chakrabarti9634 7 місяців тому

      @@theengineerguy_ sir what should be the realistic time line to learn these technologies from scratch tell in hours ( like on an avg 4 hours for 3 months almost 300+ hours like in that way) please reply

  • @priyadebnath4941
    @priyadebnath4941 4 місяці тому

    Bhaiya kya aap mujhe guide kar sakte ho to crack product based company with data engineer and resume v bana lia check karnek liye

  • @infinitysoundmotivation9768
    @infinitysoundmotivation9768 5 місяців тому

    Please tell how much required for freshers role

  • @rugvedpatil5604
    @rugvedpatil5604 7 місяців тому +7

    Bhai ye sab to bhout hai main just data engineer ka interview Diya hai or unhone python or SQL he pucha or basic etl vo bhi easy tha
    Or vo bhi fresher ke liye itna nhai mangte hai

    • @theengineerguy_
      @theengineerguy_  7 місяців тому +3

      Very correct! Freshers ki hiring toh kewal DSA and projects se bhi ho jati hai.
      But you never know what’s coming. And these are the skills you’ll be working on as a DE. So I believe better prepare these skills for a bettter chance 🙌

    • @rugvedpatil5604
      @rugvedpatil5604 7 місяців тому +2

      @@theengineerguy_ muze lekin dsa nhai pucha tha or me DSA focused padhta nhai hoon bhai me thoda project oriented hoon Ban ne me maja ata hai muze

    • @uzaifquadri
      @uzaifquadri 7 місяців тому

      For which company bro

    • @rugvedpatil5604
      @rugvedpatil5604 7 місяців тому

      @@uzaifquadri go digital

    • @devanshdubey4588
      @devanshdubey4588 7 місяців тому +1

      Bhai please mujhe bata do, mera interview he data engineer ka, kya puchega wo, python me kya kya pucha, aur sql me bhi

  • @hsrr136
    @hsrr136 7 місяців тому

    Pls reply which is best course for data science to get job

  • @ashpreetsinghanand7260
    @ashpreetsinghanand7260 7 місяців тому

    Bhaiya project bhi bata do apne jo banye

  • @Adeelkhan2112-h2h
    @Adeelkhan2112-h2h 7 місяців тому +1

    Why you did not mention R language sir ?

  • @tech_Animus
    @tech_Animus 5 місяців тому +1

    Can I completed it in just 6 months

  • @pujagopalpatil7502
    @pujagopalpatil7502 5 місяців тому

    Sir mujhe sikane hai sabhi technology

  • @SATYARATHOD1799
    @SATYARATHOD1799 5 місяців тому

    Yes you can

  • @varunthakkar6349
    @varunthakkar6349 2 місяці тому

    Please tell best resources to learn al of these skills Paid or Free . Atleast tell ?

  • @bhikhamkumar1384
    @bhikhamkumar1384 5 місяців тому

    Bhaiya yar aap ne source nhi btaya kaha kaha se padhna hai

  • @manas9926
    @manas9926 7 місяців тому

    Can you suggest industry ready projects in data engineering to start a career as a fresher??

  • @shandeepraula
    @shandeepraula 7 місяців тому

    Hello Bhai , can you suggest resume project?

  • @Self.Writer
    @Self.Writer 4 місяці тому

    bca ke degree se jaa skte hai skill ke sath

  • @_AmitVlogs
    @_AmitVlogs 7 місяців тому

    why you have not applied resources 😕

  • @SaniyaChamanshaikh-n1g
    @SaniyaChamanshaikh-n1g 7 місяців тому

    Plz sir make videos..

  • @nayankumar5833
    @nayankumar5833 7 місяців тому

    But ye sb hum kaha se study krenge playlist bna se bhai aap

  • @brohisarfaraz8750
    @brohisarfaraz8750 6 місяців тому

    Impressive

  • @shortinfovids2218
    @shortinfovids2218 7 місяців тому +1

    sir, how much DSA is required . for DE interview ?

    • @theengineerguy_
      @theengineerguy_  7 місяців тому

      If you can solve blind 75 set of problems from Leetcode then you’re good to go!

  • @pavanbm2001
    @pavanbm2001 3 місяці тому

    databricks?

  • @DSingh-hv9oo
    @DSingh-hv9oo 21 день тому

    Sir
    Can we learn c++
    For data engineer
    😅

  • @sushmitag528
    @sushmitag528 7 місяців тому

    Intellipaat platform fake h kya achhi h vaha 6month ka course h wo bol rahe job mil jayegi? Plz help kro kuch batao na

  • @sahilabbas5712
    @sahilabbas5712 4 місяці тому +2

    Bhai kuch chorna bhi hai...?

  • @ShivamSingh-kb3ff
    @ShivamSingh-kb3ff 6 місяців тому

    Yes yes yes!!

  • @viveksingh2733
    @viveksingh2733 4 місяці тому

    Bhai BCA student Data engineer ban sakte hai??pls reply sir

  • @application_dev7920
    @application_dev7920 29 днів тому

    Ye sab padna hi hai to job to milegi hi, kuch chora bhi hai bhai tumne, Full stack karo ya AI, ye sab future ke liye promising hain, i have 14+ years of experience

  • @ashwinidhomne4244
    @ashwinidhomne4244 6 місяців тому

    Can freshers get n job as a Data Engineer?

    • @JinnahClaus
      @JinnahClaus 6 місяців тому

      in india its easy
      at abroad its difficult for a fresher

  • @ujjawalanand5216
    @ujjawalanand5216 7 місяців тому

    Bhaiya kaha se course padh le cheap rate mein

  • @ejajahemad157
    @ejajahemad157 6 місяців тому

    bhai Maine btech nhi kiya h kya mai Data engneer ki taiyari kr skta hu nhi

  • @shashikantverma7215
    @shashikantverma7215 6 місяців тому

    Aap bataiye nahi ki aap ytube per bhi ho❤😊

  • @rtk_here
    @rtk_here 6 місяців тому

    Itna jyda tool 3 se 4 mahine me kaise hoga
    Isse achha web dev kr lu 😅

  • @vaibhavshah4898
    @vaibhavshah4898 6 місяців тому

    Bhai referal de do j p morgan me 😅

  • @pudinhara8412
    @pudinhara8412 7 місяців тому +1

    How much DSA is needed and which data structures for cracking DE job role.

  • @AncientWonders-g5q
    @AncientWonders-g5q 7 місяців тому

    Python seekh sakta hooooo

  • @EBKCS_SamarthGoyal
    @EBKCS_SamarthGoyal 6 місяців тому

    Bhaia apka laptop kuch jyada hi purana h

  • @abdullahsiddique7787
    @abdullahsiddique7787 7 місяців тому

    Kaish bhai Salam is data engineering field high paying job ? Compared to data analyst or sata sceince or devops field ?

  • @noob_2377
    @noob_2377 6 місяців тому

    I'm looking for peer person who can learn simultaneously, please reply for this comment.

    • @nyalapallykeerthana
      @nyalapallykeerthana 4 місяці тому

      @@noob_2377 hii.... I too wanna learn data engineering

    • @noob_2377
      @noob_2377 4 місяці тому

      @@nyalapallykeerthana hi

    • @noob_2377
      @noob_2377 4 місяці тому

      @@nyalapallykeerthana hi please share your telegram id

    • @nyalapallykeerthana
      @nyalapallykeerthana 4 місяці тому

      @@noob_2377 hello

  • @sujayakumar2076
    @sujayakumar2076 7 місяців тому

    Vaii jaise boLa deemag se ghus nehi rah hai itna saraaa hai , sab padhne ke Liye bol reh ho toh kiya padhe 😅😅😅😂

  • @shubhamkumar2224
    @shubhamkumar2224 5 місяців тому +1

    Thanks for the insightful info , I am starting learning data engineering and do switch from test engineer to data engineer , Hope i will make it. Thanks a lot again @theengineerguy

  • @vishalmane3139
    @vishalmane3139 7 місяців тому

    Roadmap nhi, resources bataya kr!

    • @hrushikeshkhonde7079
      @hrushikeshkhonde7079 4 місяці тому

      @@vishalmane3139 be respectful man...yhi bat pyar se bhi boli ja skti thi...🙏

  • @alhasanmohammedbinquraish960
    @alhasanmohammedbinquraish960 7 місяців тому

    Apache kafka sir 🥺

  • @nanochip1908
    @nanochip1908 7 місяців тому

    Any good sources to learn kafka ?

  • @application_dev7920
    @application_dev7920 29 днів тому

    AI karo bas

  • @alhasanmohammedbinquraish960
    @alhasanmohammedbinquraish960 7 місяців тому

    Sir apache kafka 🥺

  • @shrutisharma1764
    @shrutisharma1764 6 місяців тому +1

    Issey Acha upsc Kr lo😂

    • @abhinav7536
      @abhinav7536 5 місяців тому

      @@shrutisharma1764 aapne crack kar li kya 🤣🤣🤣🤣🤣

    • @abhinav7536
      @abhinav7536 5 місяців тому

      Aaapne crack kar li kya🤣🤣🤣

  • @alhasanmohammedbinquraish960
    @alhasanmohammedbinquraish960 7 місяців тому

    Apache kafka sir 🥺