#72. Azure Data Factory - Reject and Log dirty rows while using "Polybase" option in Copy activity
Вставка
- Опубліковано 5 жов 2024
- Source file will not always be clean. It might have some junk characters or incompatible values in one or many columns. ADF gives us a simple way to handle this in runtime with minimal effort. This video takes us through the steps to load records from CSV file to Synapse DB with Polybase option. Also we discuss how to reject the incompatible records and do a logging in BLOB path or ADL.
what is difference between Copy , Polybase and Bulk Insert , in which scenario we use which method ?
Pretty cool feature. Thanks
Yep. You are welcome
@@AllAboutBI do you conduct training?
found more columns than expected column count (DelimitedTextMoreColumnsThanDefined) how to solve this error
Excellent
Thank you..🙏
Mam, If .csv file is coming as pipe (|) separated and i need to load into table.. if file have extra line comes as hyphen (-) then i need to ignore that row.. and load the file..
Filename-File.csv
Name|salary
------
Ram|5000
Aman|6000
Hi , Is there any way to delete header and footer records from flat file using ADF, if file have header ,footer and detail records in it.i.e we should have only detailed records only. Thanks in advance
No. We can't do any manipulation in file using built in controls.
In data flow we can ignore header n footer n just have data. And if it needs to be written to a different sink we can do
Hi Which dataflow we can use to do this. Thanks!
I'll post a video for it.
Thanks for quick reply. Waiting for it.
Nice Feature mam. I am using Postgres database as Sync, and I am facing a very weird issue.
Can you please provide your email address/ LinkedIn Id where I can reach out to you?
Funlearn0007@gmail.com pls mail to this ID
@@AllAboutBI Thank you. I have mailed you. Thank you in advance for your help.