34. Get Metadata Activity in Azure Data Factory
Вставка
- Опубліковано 25 сер 2024
- In this video, I discussed about Get Metadata Activity in Azure Data Factory
Link for Azure Functions Play list:
• 1. Introduction to Azu...
Link for Azure Basics Play list:
• 1. What is Azure and C...
Link for Azure Data factory Play list:
• 1. Introduction to Azu...
#Azure #ADF #AzureDataFactory
The most conplete ADF course I've seen until now!!
Thanks a lot!!
Thank you 🙂
@Juan Sergio is this a UA-cam channel?
Amazing ADF Course ....
Thank you 😊
Hi Maheer, I like your Explanation But instead of Explaining can you Please Execute The pipelines so it could be a better understanding. In Many Videos, You are Explaining How to Do it but you are not Executing the Pipeline. So We can get easily how it is impacting
very true...but good explanation...and may be he is wanting us to agile our fingers on keyboard 😃
Agreed
Hi Maheer,
Can you explain how to develop a pipeline/logic to recursively search for files in a directory regardless of any number of sub folders. It has to get details of all the files in the directory.
Best way is use Azure function for it. But still let me give a thought about it implementing it with adf activites
@@WafaStudies it would be better if I use ADF activities. Could you make a video about the implementation?
@Maheer - I like the way you explain all the basic ADF concepts to real life scenarios. I have one query if its possible . Kindly help me with it.
Is there any way we can know in ADF , who has uploaded file at datalake container.
I have real time scenario where, In case of failure of pipeline, I need to notify respective person who has uploaded the file. ?
what if I have multiple files under the folder and I want alter or derived a column in all those , can you do those using Get Meta data activity
No can do that using derived column transformation in dataflow.
Is there a way to recursively go inside the subfolders and extract the files from those subfolders? Or is there any other activity with which this can be achieved?
is there a way to store the contents of a file into variables? For Example, read a value from a file and store this value in the variable/parameter.
I have a scenario where I am reading a set of same structured files from a folder (each file is a sales data from different stores across country). my pipeline is failing every now and then due to the structure of files from some store are changing, so the mapping is failing. after watching this video I am thinking of applying this Metadata combined with if condition. for each loop to iterate on each file, read the COLUMNCOUNT metadata of current item using @item() , if columncount = Xvalue then Run the ExecutePipeline that actually copies the data , ELSE log the wrong file details into a file for after execution investigation. Will that work ? Thanks
Could please explain metadata attributes as well please
don't understand how you use @equals in activity ,when I put it it shows me wrong
hey, could you please help me to get the data count from mongodb collection. how can we get that
hello brother in adf where we use powerbi plz make one video on that one
8:24 i guess it is not MDS, it is MD5 Message digestive file
Hi Maheer,your series is really hepful.
Can We pass the Activity Name to subsequent Activity ? If we can do so, What are the main Activities we can use ? Just please give me a hint ...
Thanks in advance. :)
I agree! it would be more beneficial if we actually executed the pipelines and saw the results
@Maheer - I like the way you explain all the basic ADF concepts to real life scenarios. I have one query if its possible . Kindly help me with it.
Is there any way we can know in ADF , who has uploaded file at datalake container.
I have real time scenario where, In case of failure of pipeline, I need to notify respective person who has uploaded the file. ?