Bulk Copy from SQL DB to Data Lake Parquet using Azure Data Factory [ADF]

Поділитися
Вставка
  • Опубліковано 29 гру 2024

КОМЕНТАРІ • 24

  • @EmilianoEmanuelSosa
    @EmilianoEmanuelSosa 5 місяців тому +1

    Thankkkks , you helped me a lot! In my first job as a data engineer! Greetings from Mendoza, Arg

  • @thiagojxs
    @thiagojxs Рік тому +1

    Saudações brasileiras!! Very good video my friend. This is the type of content that I'm looking lately. Keep going!!

  • @CurlyHairedKiddHD
    @CurlyHairedKiddHD 11 місяців тому +1

    Excellent tutorial! Made my job much easier.

    • @pytalista
      @pytalista  11 місяців тому

      Glad it helped!

  • @marcosadlercreutz
    @marcosadlercreutz Рік тому +1

    Thank you very match! It was very helpful!

  • @HimenSuthar
    @HimenSuthar 4 місяці тому +1

    This is very helpful. Thank You Very much
    Have you extended this exercise to create partition-based parquet file inside table folder.?

    • @pytalista
      @pytalista  4 місяці тому +1

      Hi, do not intend to go on this topic. This is a way to bring data to the lake form transactional systems. then from there I would ingest in delta tables.

  • @sudhirg8773
    @sudhirg8773 7 місяців тому +1

    Thanks a lot for sharing

    • @pytalista
      @pytalista  7 місяців тому

      Welcome 🙏🏻

  • @SomeshVashistha
    @SomeshVashistha 11 днів тому +1

    How do I continue to sync the data?

    • @pytalista
      @pytalista  7 днів тому

      This example just copy the whole table

  • @sphnyhptrs
    @sphnyhptrs Рік тому +1

    Great tutorial!

  • @KeotshepileMosito
    @KeotshepileMosito Рік тому +1

    Hi,
    Which resources did you create under your resource group?
    Thanks

    • @pytalista
      @pytalista  Рік тому

      I don’t remember here but a resource group is just a logical container to organize resources in Azure. Cheers.

  • @ajayigeorge1356
    @ajayigeorge1356 Рік тому +1

    Can you give me a good reason why you would need to copy data from a sql db to Azure data lake?

    • @ajayigeorge1356
      @ajayigeorge1356 Рік тому +1

      Cool video

    • @pytalista
      @pytalista  Рік тому +6

      A good one. A company uses Databricks as its analytics platform and need operational data for all their analytics workload like data science, machine Learning, business reporting and etc. SQL server is a good OLTP database but not good to do analytics at scale.

    • @ajayigeorge1356
      @ajayigeorge1356 Рік тому +1

      @@pytalista alright, thank you.

  • @leafyfist685
    @leafyfist685 9 місяців тому

    Thank you so much brother

    • @pytalista
      @pytalista  9 місяців тому

      Welcome 🙏🏻