What skills do you need as a Data Engineer?

Поділитися
Вставка
  • Опубліковано 9 січ 2025

КОМЕНТАРІ • 41

  • @KahanDataSolutions
    @KahanDataSolutions  2 роки тому +3

    Want to build a reliable, modern data architecture without the mess?
    Here’s a free checklist to help you → bit.ly/kds-checklist

    • @JawadHaiderSyed
      @JawadHaiderSyed Рік тому

      Great content! I appreciate the work you are doing. Well I was expecting a pdf following this link but I get an email (Lesson 1 of 5) which asks me to watch another video. Could you please clarify if there is no single text source as in a pdf?

  • @ostrich97
    @ostrich97 Рік тому +8

    You have no idea how helpful your channel is for young data engineers.

  • @ghilmanfatih9751
    @ghilmanfatih9751 7 місяців тому

    Years spending time on the architecture side of the job, watching your videos help me updates on the current trend of tech stack and expected skills. Nice

  • @Omzodijacky
    @Omzodijacky 2 роки тому +3

    Hey Man
    thanks for the outstanding videos .
    your videos' has been a huge help for me .
    Can't wait for you to do an apache airflow video

  • @oluwafeminasiru8835
    @oluwafeminasiru8835 2 роки тому +2

    This video has been so helpful! Thank you

  • @acb_gamez
    @acb_gamez Рік тому +3

    Man, I don't know what I'm doing wrong then. I am a Python wizard (5 years) applying to 100s of these junior engineer positions. I only know PySQL however, so I wonder if that is the main thing hurting me. I have data analytics and viz experience, and some ETL projects I made to fill out more, but nothing. I don't really know what to do and feel like giving up. sorry if that's a lot lol.

  • @SebastienMonnier
    @SebastienMonnier Рік тому +2

    Interesting video. Though, I think you forgot about privacy. This is as important as security.

  • @Lapookie
    @Lapookie Рік тому

    Learn fundamentals. How to set up a data stack with the right environment, plateform. (Terraform, docker, ainsible etc.) and learn about NETWORKS communication between the stack . Know where data metadata will be store etc.

  • @mhelcarta1411
    @mhelcarta1411 8 місяців тому

    This is so helpful. Thank you so much!

  • @georgejetson9801
    @georgejetson9801 2 роки тому +1

    Nice vid. I love being a lead. Some days I wish I was a jr.

  • @LaxmanKumar-rq7hk
    @LaxmanKumar-rq7hk 2 роки тому +2

    Great video! Very useful information 😊

  • @Dreamer3250-w7t
    @Dreamer3250-w7t 2 роки тому +1

    Very useful information,thanks for sharing 🙏

  • @crymeet
    @crymeet 2 роки тому +1

    Great video! Thanks😊

  • @thedailyepochs338
    @thedailyepochs338 Рік тому +1

    Awesome video

  • @ukaszdugozima816
    @ukaszdugozima816 2 роки тому +1

    Hello! Thank you for your invaluable video! I find it extremely useful for beginners! I would like to ask about one thing regarding this skill. I learnt Pandas in terms of Data Wrangling and Transformation. Therefore, how about Pandas for Data Engineers? Is it useful tool for ETL/ELT transformations? Obviously, the next step will be PySpark, but I would like to start learninig Pandas. It seems it is a good path for the next one. What do you think about it ? I would appreciate it if you could share your views about it.

  • @yangdai8347
    @yangdai8347 2 роки тому +8

    Learn sql, then dbt and snowflake. You will be good to go.

    • @mihirranade815
      @mihirranade815 Рік тому +1

      What is dbt

    • @xx-pn7it
      @xx-pn7it 9 місяців тому

      ​@@mihirranade815 data building tool for it is used for ELT it working with the transformation part

    • @ansar2137
      @ansar2137 9 місяців тому

      Started my data engineering career with exactly these 3... And I couldn't agree more with you, these 3 are best to get started

    • @ansar2137
      @ansar2137 9 місяців тому

      ​@@mihirranade815data build tool, it's the T of ELT, i.e, helps with the transformation part of the data journey. Infact this channel has quite amazing dbt content, good starting point.

  • @Trihawk7
    @Trihawk7 2 роки тому +1

    is there any specific bit of python you'd recommend learning for juniors? Like pandas etc, same with sql I guess. Thanks

    • @soyungamer9434
      @soyungamer9434 2 роки тому +1

      I would recommend pandas, numpy, scipy and at least a SQL database management library (MySQL-Python) and a noSQL one (pymongo)

  • @jamesross5120
    @jamesross5120 Рік тому

    I've got a job interview as a data coordinator and i really don't know what they actually require lol i mean i have some sql skills and database skills like data modelling but hardly any and they want someone who can do data transformations/ collections etc ? what should i learn i have a week till the first phone interview?

  • @poizentv
    @poizentv 11 місяців тому

    Hey Kahan. Is it possible to become only Data Warehouse engineer? I’m actually avoiding having to learn Python😭😭😭

  • @limitless003
    @limitless003 7 місяців тому

    Thanks a lot

  • @football2022.
    @football2022. 2 роки тому

    I am not finished my high school can I become Data Engineering

    • @KahanDataSolutions
      @KahanDataSolutions  2 роки тому +2

      Of course! You have a long career ahead of you. I didn't start learning any of this until I after I already graduated college and worked in a different job for nearly 3 years. You are way ahead

  • @MDARUN-ph1dw
    @MDARUN-ph1dw 2 роки тому +1

    👋👋👋😉

  • @Reinales
    @Reinales 2 роки тому +2

    Nice video bro!
    I have a quick question;
    Why did you didn't talk about the "Mid Engineer" ? (I'm a Junior Engineer close to be moving to the next level in my actual Job, and I hope it be the "Mid Engineer")
    Grettings from Colombia!
    🟡🔵🔴

    • @KahanDataSolutions
      @KahanDataSolutions  2 роки тому +2

      I've always wanted to visit Colombia - thanks for tuning in!
      Regarding your question - these positions are very much on a spectrum and the actual job titles will vary a ton depending on where you work. For that reason I decided to stick to just 3.
      Perhaps a "Mid Engineer" at your company is the same as what I'm referring to "Senior Engineer" (But I'm not sure). Either way, I would see if you can find concepts in the video that you are unfamiliar with and try to learn a bit more about it. That will help you level up in general.
      Good luck!