Azure Data Factory - Incremental Copy of Multiple Tables
Вставка
- Опубліковано 12 вер 2024
- Previous Video on Incremental Copy with Parameterization:
• Azure Data Factory - I...
This covers the following:
1. Running the Incremental Pipeline for multiple tables
2. Understanding the Process
3. Master Child Pipeline Concept
4. Foreach Loop Activity, Execute Pipeline Used in this Video
For other videos, please subscribe:
/ @azurebitsandbytes5804
Good job brother. Your way of explanation is nice.
Awesome...!!! Simplicity in Teaching...!!! Great..!!!
Well explained 👍
Great explanation and useful video
Wonderful Delivery...!!!! :)
Great video! Have you ever set up an incremental load between an Oracle source table and an SQL sink table before? I am currently trying to do this but can only get my delta load to work when copying from SQL to SQL as you did in this video, any guidance would be greatly appreciated :)
Hi..I'm not getting invoice table is not executed for me in pipeline...where I went wrong...can you please help me to fix it
Actually why we nee to create the same table in target DB (demobits DB)
How Capture data change works in case of timestamp
Please attach the codes for the vedio that you have been used
Does orders and invoice tables has same schema?
How mapping works if table name and column names are different
Sir make a video on master child pipeline
Good video, thanks for that But one question how about the invoice mapping in copy activity(in child pipeline). Master pipeline failing(ForEach activity) due to not being able to load invoice data. Can you please tell me how this can be fixed Thanks
Clear off the schema of the sql dataset. It will work
@@azurebitsandbytes5804 can you elaborate on this? mine only work on the first table but always get "Column name 'xxxx' cannot be found in either source data or column" but if I map the data then it works but then the 1st table would failed with similar message.
@@SangLeWeb Look Up First Row Only Property needs to be unchecked to allow reading all the records from the config table
@@azurebitsandbytes5804 That's not I was referring to, mine is already unchecked and I can see it loops through 2 records but as I said, it works only the 1st loop, the 2nd threw an error about the ID not avail since my 2nd table does not call ID but it says userID. So it seems like the source still using the previous schema instead of new one which I already put a "Select *..". Just try to run the child by changing the table name wouldn't work neither unless I create a mapping then I can run with different table name. The only diff between your sample and mine is my sourct table name is diff from my sink so I created 3 variable (SourceName, TargetName, WatermarkValue) but I don't think this should be an issue.
I figured it out, it was my SP that I ran causing the issue. BTW, thanks for taking the time to put up the videos, very insightful.