30. Get Error message of Failed activities in Pipeline in Azure Data Factory

In this video, I discussed about getting error message details of failed activities in pipeline in azure data factory
Please watch below video, before watching this
• 29. Run an activity if...
Link for Azure Synapse Analytics Playlist:
• 1. Introduction to Azu...
Link for Azure Databricks Play list:
• 1. Introduction to Az...
Link for Azure Functions Play list:
• 1. Introduction to Azu...
Link for Azure Basics Play list:
• 1. What is Azure and C...
Link for Azure Data factory Play list:
• 1. Introduction to Azu...
Link for Azure Data Factory Real time Scenarios
• 1. Handle Error Rows i...
Link for Azure LogicApps playlist
• 1. Introduction to Azu...
#Azure #AzureDatafactory #DataFactory

[@vijaybodkhe8379]

Thank you very much for sharing .You are covering really real time scenarios.

[@NeumsFor9]

Ive always been disappointed at how this feature of the expression language is not advertised as much as it should be. I know the challenges of representing certain boundaries in ETL tools but MS is long past the time of doing better with task grouping, event handling WITHIN the tool, and setting of transaction boundaries.
Glad you brought this to life.

[@annukumari9629]

You never fail to surprise your audience 😊 Thanks for amazing video again

[@WafaStudies]

Thank you ☺️

[@volodymyrvintonyak9286]

Wow, that's exactly what I was looking for! Followed this part through the comments of p.29, which is also very helpful. Especially the hint with '?', didn't know that it can work like this.
The only question I have, do you know by any chance, can we somehow workaround the errors like 'Activity failed because an inner activity failed' to have the exact error of that inner activity? I mostly get it from the 'For each' activity. Looking forward to hearing from you.
Anyway, I very appreciate your work on this course.

[@sarthaks]

Very good concept indeed. This will definitely help in designing good data flow in ADF

[@WafaStudies]

Thank you 😊

[@srinubathina7191]

Next level content. Thank You Sir

[@TheDouglasDale]

Thank you sir. This really helped! Keep up the good work!

[@WafaStudies]

Welcome ☺️

[@mazhar0001]

this is pure Gold!!

[@WafaStudies]

Thank you ☺️

[@MaheshReddyPeddaggari]

Wow, The? catch is very helpful, thanks for sharing your knowledge.

[@WafaStudies]

Welcome 🤗

[@surajjeswara]

Thanks for the tutorials

[@mehmetadalar7606]

Thanks for the video. Very well explained. One comment.. If we set on success relation between activities this will also make the pipeline fail on any failed activity. So alternatively a monitor alert would also do the same job that a web activity does in a pipeline. What do you think?

[@sravankumar1767]

Nice explanation 👌

[@WafaStudies]

Thank you ☺️

[@pradeeppatil2805]

Hi, Can we use the same concept in logic apps as well? Suppose I have 10 actions ( set of activities) and want to capture the error for the same..?

[@ameyagodbole4476]

Thanks for the in detail explanation. How to get notified on email if any pipeline fails?
Pls guide me on this one

[@prateekkhanna7366]

How to capture error msg in case of failure for Azure function Activity. When I am using set variable activity to capture , It is giving Error.message not present

[@shashikantbhadre4064]

Thanks a lot !!... one comment - I don't think pipe (|) is required for series execution.

[@nanibabu7059]

Thanks for the video this is the one I requested in the previous video. . One more question if my source is api and it is having multiple nested array will it be possible to handle in the adf directly. . In copy activity we can able to select only one array only right but incase if you have nested arrays how we can solve this. . Highly appreciated your efforts

[@karthikrajr8861]

We can try using flatten activity

[@nanibabu7059]

@@karthikrajr8861 but flatten we can use only in dataflows but without dataflows is there any way we can achieve this

[@snehasiktachandra4357]

Thanks for the video; Suppose a pipeline is having execute pipeline activity, in the child pipeline there a notebook is running in loop. Is there any way I can get the error message of child pipeline - following the way in this video, I am getting the message that "execute pipeline failed because inner activity failed"; I need exact error message

[@shashikantbhadre4064]

Thanks a lot !!... one comment - I don't think pipe (|) is required..

[@dannyroy8571]

To extract the pipeline status for each stage or task in Azure DevOps and forward each pipeline stage status to Google Chat Hangouts by implementing this in a pipeline script - how can we do that ?

[@naveenkandipu750]

How do we pass activity result set of pipeline 1 to pipeline 2?

[@vishaljhaveri7565]

10:00 Failure in series execution.

[@muralidhar7573]

I want to join this course . please tell me how can I contact you

[@ssprb7]

How to capture the error msg when we have both serial and parallel activities together in a pipeline

[@alpeshjain3878]

At 06:10, suppose rather than one two activities got failed, then it's capturing logs only for one variable not the other one.

[@NeumsFor9]

?. is the "optional chaining" functionality of JavaScript. It is NOT unique to ADF.

[@balajiawari9740]

Operation on target Copy data1 failed: Failure happened on 'Source' side. 'Type=System.IO.IOException,Message=Received an unexpected EOF or 0 bytes from the transport stream.,Source=System,'
Getting this error, no clue

[@saumyasingh9620]

It is not working for me, I am getting error for pipeline run itself. Please help.

[@ravishankarkusma]

How to Capture Activity name I get the error message only but want to know this is related to which activity

[@mehmetadalar7606]

Error.target

[@cloudfitness]

😃

[@WafaStudies]

😊

[@Ramakrishna410]

Great. how do we get failed activity name? anyone please?

[@mehmetadalar7606]

Error.target

[@SravanKumarA-bz1py]

using the below expression it is throwing me an error for the last activity
{
"DataFactory_Name": "@{pipeline().DataFactory}",
"PipelineName": "@{pipeline().Pipeline}",
"ErrorMessage": "@{concat(activity('copygosworkplanFF').Error?.message,'|',
activity('DataFlow_LFAPO')?.Error?.message,'|',
activity('CopyflatfilePOLabtoGOS')?.Error?.message)}",
"RunID": "@{pipeline().RunId}"
}