Hi Reza, Thanks for the great video. I have encountered a problem with CopyData activity. I am using Copy Data Activity to move data from a fabric warehosue to another. If the pipeline runs more than once, the whole data get appended. Is there a way to overwrite the data from Warehouse to Warehouse using Copy Data Activity ?
Copy data does not let me copy from on-premise to the warehouse directly. and I really don't get the meaning of "Enable staging". Would you please elaborate the settings of Copy data Activity? Thank you for your Videos.
Hi Reza, Thanks for the great video. I have a question. Is there a way to Filter the source table's data (say I have an On-Prem SQL-Server Table as source) and just ingest a subset of the source table records (and not the entire table), using Copy Data Activity?
Hello Reza, thank you very much for your explanation. Unlike your process, when I check "Enable Staging" I have to select a staging account connection, which you didn't need to do. Is this a new requirement, or did I set something up differently? Thank you.
Yeah ok, but I guess we still miss quite a few very important things for this DW, like how would you implement SCD type 1/2 with just copy activity, there is no mapping data flow that usually was great for that, I Dont think that PQ is good enough for this? not sure what are your thoughs on this, I mean I know that Microsoft provide a guide how we can use PQ tools to mimic mapping data flow activities but is it good enough? From your experience what is user feeback on this?
your way of explaining is really awesome. Thanks for making us more knowledgeable in new tools/features.
Thanks. I am glad it helps
Hi Reza, Thanks for the great video. I have encountered a problem with CopyData activity. I am using Copy Data Activity to move data from a fabric warehosue to another. If the pipeline runs more than once, the whole data get appended. Is there a way to overwrite the data from Warehouse to Warehouse using Copy Data Activity ?
Copy data does not let me copy from on-premise to the warehouse directly. and I really don't get the meaning of "Enable staging". Would you please elaborate the settings of Copy data Activity?
Thank you for your Videos.
Thank you Reza. This video cleared up a number of questions I had around use of the copy activity.
Glad it was helpful!
Hi Reza,
Thanks for the great video. I have a question. Is there a way to Filter the source table's data (say I have an On-Prem SQL-Server Table as source) and just ingest a subset of the source table records (and not the entire table), using Copy Data Activity?
Hi Reza, Great video, it would be great if you can create some end to end project and share as a playlist, Thanks!
Hello Reza, thank you very much for your explanation.
Unlike your process, when I check "Enable Staging" I have to select a staging account connection, which you didn't need to do. Is this a new requirement, or did I set something up differently? Thank you.
Interesting, I never saw account setup requirement for the staging. do you get it all the time? or it was a one-off?
Yeah ok, but I guess we still miss quite a few very important things for this DW, like how would you implement SCD type 1/2 with just copy activity, there is no mapping data flow that usually was great for that, I Dont think that PQ is good enough for this? not sure what are your thoughs on this, I mean I know that Microsoft provide a guide how we can use PQ tools to mimic mapping data flow activities but is it good enough? From your experience what is user feeback on this?
Is an ADF equivalent of the Metadata Copy Task option available in Fabric?
Hi Thanks for this video, Its really helpful.
the data movement cost of the fabric is very high. Anyone having problems with this?