#42. Azure Data Factory - Exception\Error Handling Basics
Вставка
- Опубліковано 30 сер 2020
- Error handling is a very useful technique to capture any error occuring in pipeline activities. Though ADF is not matured to handle it very effectively, we have certain workarounds in which we can achieve error logging and handling. Please check out this video to know more
Amazing...super clear!
Thanks
Thank you so much for these kind of videos. Really help a lot in interview point of view.
Glad to hear about it. Please share it with your friends too.
@@AllAboutBI sure already shared. Actually mam not good materials are there regarding azure data factory. Your videos helped me a lot i am lucky to find your channel
@@saikumar-it1gl thanks a lot for ur feedback
Can we use variable inside web activity body section? Also how to come out of pipeline if a condition is not met in if condition activity without continuing the pipeline
what we have to do if pipeline fails when the pipeline is runing please explain and how will we connect different soureces in a singel pipeline
very useful
Thanks for Sharing 👏👏
You are welcome
Well done.
Thanks Sir!
Hi mam,
Can you please make and provide video tutorial on transformation through databricks??
It would great help from ur side.
Thanks for previous all videos.
Sure Manish. I'll try
Yes mam please create databricks playlist .
@@AllAboutBI Thanks :-)
Good one mam
Thanks a lot mam
Great and very useful video!
How would you handle data conversion errors like in SSIS ?
How to read an error message from DataFlow if failed? I tried to use expression like this "@{activity('Run DataFlow').output.errors[0].Message}" to read error message from my DataFlow activity. But there is no such expression to read the exact error message from DataFlow.
mam have created Stored Procedure in synapse pool like below,
BEGIN
declare
@logvalues varchar(max);
set @logvalues = 'INSERT INTO [CONTROL].[SNPS_PIPELINE_LOGS]
(PIPELINE_NAME
,TABLE_NAME
,LOAD_FREQUENCY
,ERROR_LOG
)
VALUES
(' +@pipeline_name+
',' + @table_name +
',' + @load_frequecy +
',' + @error_log +
')';
select(@logvalues);
EXEC (@logvalues);
END
--->procedure is working fine but when am passing error message as u shown in 43 video same i have passed, but it is showing error in side the error.message @inverted commas, please help me mam please.
. Error Message: Parse error at line: 8, column: 94: Incorrect syntax near 'happened'."
We should have the master pipeline to fire child pipeline and in the master pipeline we should add the error activity for genetic error info
Thanks
Welcome!
How to create a business logic to run a pipeline from point of failure ?
I think we might have to use powers hell script to do that
Madam I did same as you said but error data not saving in SQL database table because error value in setvariable activity is not in ' ' (single codes) so I asked some one to solve this issue he said declared variable.... But I don't know how to declare variable please respond
Can you explain how this stored procedure created
Nice video. very helpful information. I have a question related to ALERT Configuration in ADF V2. Do we have any mechanism to configure at Pipeline level , instead of ADF level ?? Please suggest. Thanks in advance.
Hey. I don't get your question. Can you pls rephrase a bit ?
@@AllAboutBI
Thanks for your quick response. When we create any alert, we receive the alert saying 'Some ADF failed with some Error'. My question is that Could we get the alert as 'Some Pipeline inside this ADF failed with some Error'..
Is it possible?
Yes. You mean the proper error message that made the pipeline fail is it. It's possible. I'll try to post a video shortly on that.
@@AllAboutBI Thank you very much mam ! Appreciate your help and it helps a lot for the beginners.