Must-Know PySpark Interview Question for Data Engineers - Live Demo & Tips!

Поділитися
Вставка
  • Опубліковано 10 гру 2024

КОМЕНТАРІ • 28

  • @billionairemindset7192
    @billionairemindset7192 3 місяці тому +1

    Waiting on your full data engineer tutorial video Mr KT.... Some months ago you said you are working on something, i hope you are still working on it, I really look forward to that video as it will help me a lot.
    Thank you for this too.

    • @mr.ktalkstech
      @mr.ktalkstech  3 місяці тому +1

      Working on an End to End Project, will try to upload soon :)

    • @billionairemindset7192
      @billionairemindset7192 3 місяці тому

      @@mr.ktalkstech looking forward to seeing it.... Thank you for your amazing work.

  • @sharaniyaswaminathan8760
    @sharaniyaswaminathan8760 3 місяці тому

    Good one. Waiting for your next project video :)

  • @Abhinavkumar-kt8gj
    @Abhinavkumar-kt8gj 3 місяці тому

    Very nice explanation..Very good..
    Waiting on your full data engineer tutorial video..

  • @062nanthagopalm6
    @062nanthagopalm6 3 місяці тому

    Great video brother! Looking forward to the upcoming videos . Thanks for the efforts 🎉

  • @jaypandya2714
    @jaypandya2714 3 місяці тому

    Your explanation skill is too good ❤️ hoping for more videos on topics suchs as Projects, airflow , dbt , snowflake :)

  • @TheMapleSight
    @TheMapleSight 13 днів тому

    What is the difference between doing it in PySpark and SQL? I think in SQL it's much easier:
    %sql
    drop table if exists test;
    CREATE TABLE test (
    id int,
    total int
    )
    INSERT INTO test (id, total)
    VALUES
    (1,10),
    (1,20),
    (1,30),
    (1,40),
    (2,20),
    (2,40),
    (2,60),
    (2, 80);
    SELECT *,
    SUM(total) OVER (PARTITION BY id ORDER BY total) as emp_run
    FROM test;

  • @emil0847
    @emil0847 3 місяці тому

    Thank you for the dedication on your vidoes man they are very helpfull for hands on projects and learning

  • @moyeenshaikh4378
    @moyeenshaikh4378 3 місяці тому

    Sir please upload scenario based questions for adf, key vault etc
    It's asked in interviews

  • @DheerajMaddula
    @DheerajMaddula 3 місяці тому

    Even if we don't add rowsBetween, it works the same way right? I mean it's default right?

    • @mr.ktalkstech
      @mr.ktalkstech  3 місяці тому +1

      Yup, the main intention is to explain what's happening behind the hood :)

  • @reachrishav
    @reachrishav 3 місяці тому

    Don't we also need to add order by col(total) in the window spec? That would make the code deterministic

    • @mr.ktalkstech
      @mr.ktalkstech  3 місяці тому

      Good point, it's better to add it, you are absolutely right :)

  • @rahulmittal116
    @rahulmittal116 3 місяці тому

    Unboundedpreceding and current row is default, right?

    • @mr.ktalkstech
      @mr.ktalkstech  3 місяці тому

      Yup, the main intention is to explain what's happening behind the hood :)

    • @rahulmittal116
      @rahulmittal116 3 місяці тому

      Right :)
      You are doing great work..
      I learned Azure projects from your videos only..
      Thank you so much🤙
      God bless you🙏

  • @KeyBooksInsights
    @KeyBooksInsights 3 місяці тому

    @mr.ktalkstech do videos on regular basis your subject is awesome

    • @mr.ktalkstech
      @mr.ktalkstech  3 місяці тому

      Sure, I am trying my best :)

    • @KeyBooksInsights
      @KeyBooksInsights 3 місяці тому

      @@mr.ktalkstech can you please do video on unity catalog

    • @mr.ktalkstech
      @mr.ktalkstech  3 місяці тому

      Sure, will upload that soon :)

  • @arunKrish11-yl4sb
    @arunKrish11-yl4sb 3 місяці тому

    Explain in tamil also