ADF metadata driven copy activity , how to process multiple tables dynamically in ADF copy activity

Incrementally copy multiple tables in adf,
incrementally load from multiple tables,
ADF Transform Multiple Tables in Single Pipeline,
Copy multiple tables in Bulk with Lookup,
Copy multiple tables in bulk by using Azure Data Factory
load database tables from many different files,
Loop through multiple files in ADLS,
Loop through multiple tables from sql to ADLS using adf,
a metadata-driven approach to data repository design
metadata driven etl

[@saqibkhan-ih1uu]

thanks for the detailed explanation. how and where to add audit columns like load timestamp to target tables?

[@TheyCalledMeT]

since most ingestion tasks typically go to a data lake, wouldn't it be intuitive to also use the data lake for the config/metadata table(s)?

[@RodrigoBocanegraCruz]

Hello, is there any way to define wildcard paths in the data lake destination? for example to have a folder structure in year/month/date/*. Thank you!

[@snad256]

Hi,
From where did you get the Sales table?

[@TRRaveendra]

U will get default sample database in azure sql and dedicated sql pool

[@TheyCalledMeT]

19:24 really select max watermark from target table?
not select last watermark from watermark-table filtered for this table name?

[@guptaashok121]

What all sources are supported for this?

[@agbtttt5717]

Where can we give PK name for upsert logic for each table?

[@ketanmehta3058]

@TechLake How we can add filter clause to the query?

[@TRRaveendra]

Use lookup query and add sql query there

[@ketanmehta3058]

@@TRRaveendra I mean this process will create three pipelines (Bottom_Level, Middle_Level, Top_Level) but I do not find any option to apply 'where' clause. I can see only the column details for the selected table. How do I mention my filter criteria to a specific table?