Learn to Efficiently Test ETL Pipelines

Поділитися
Вставка
  • Опубліковано 26 сер 2024
  • This talk is a story, using examples in Python and pySpark, about testing ETL pipelines efficiently. I won’t try to convince you that you need unit tests or automated tests - that’s up to you. If you do have unit tests for your ETL pipelines, or if you want them, it can be useful to make sure you aren’t testing more than you need.
    I’ll be describing how a practical (non-pyramid shaped) heuristic helps me efficiently cover edge cases and unexpected bugs in my code by ensuring I test only the code needed for the feature I’m building.
    Connect with us:
    Website: databricks.com
    Facebook: / databricksinc
    Twitter: / databricks
    LinkedIn: / data. .
    Instagram: / databricksinc

КОМЕНТАРІ • 6

  • @felixa4705
    @felixa4705 7 місяців тому +2

    Great talk! It's hard to be vulnerable about mistakes you make, let alone in front of a crowd of strangers. I am definitely going to look into the Saff Squeeze now! Thanks for the explanation!

  • @lucasdepetris
    @lucasdepetris 8 місяців тому

    Found this video while looking for solutions for a problem like the one you had. I hope it works for me as it did for you. Congrats for the amazing talk!

  • @kaname223
    @kaname223 Рік тому +1

    Great presentation, thanks lot , Automation testing is stressful and painful should be better alternatives.

  • @allthingsdata
    @allthingsdata 2 роки тому +1

    awesome talk, not only did I learn sth, I also had fun.

  • @netanelmalka6191
    @netanelmalka6191 2 роки тому

    Great talk :)

  • @qa_career
    @qa_career 2 роки тому

    Amazing!!!