Power Automate BULK / Batch Create Dataverse records FAST with the Web API
Вставка
- Опубліковано 14 жов 2024
- Instead of an Apply to Each you can create Dataverse records in batches using the Invoke an HTTP action mentioned here!
Free Solutions Architect Training:
www.untethered...
Documentation:
learn.microsof...
Screenshot of entire flow:
untethered365....
Here's a link to even better documentation that uses a different endpoint that's even faster:
learn.microsoft.com/en-us/power-apps/developer/data-platform/bulk-operations?tabs=sdk
I can't force it to work, I'm getting InternalServerError
@@Gregix1337 can you even ping the API? Like do a get request. I'd confirm that's accessible first.
@@Untethered365 I was able to check if CreateMultiple method is available for my table using GET method to this endpoint:
GET [Organization Uri]/api/data/v9.2/sdkmessagefilters?$select=sdkmessagefilterid&$filter=sdkmessageid/name eq @message and primaryobjecttypecode eq @table&@message='CreateMultiple'&@table='my_table_logic_name' HTTP/1.1
Content-Type: application/json
Its mentioned in documentation that you provided under "Availability with standard tables" section, and as a respose I recieve appropiate sdkmessagefilterid GUID which according to docs is good (if my table wasn't capable I would recieve an empty string).
I made reddit and PA community post with screenshots of header & body request however I cant put a link because my comment will get automatically removed.
@@Untethered365 yes, I was able to make GET call to check if my table supports CreateMultiple endpoint (see "Availability with standard tables" section in docs^). As a response I recieved sdkmessagefilterid GUID.
I made a post on PA Reddit and PA Community forum with screenshots but I can't put links in a comment because it gets automatically deleted.
@@Untethered365 Just woke up and it randomly started to work. I don't know what was the problem...
finally found great resources where i can learn more about power automate
@@zackmalaysia5384 welcome!
Nice tutorial! I'm going to use this to integrate zoho crm with dataverse
Thanks for this! Would it be possible to do one for Dataverse bulk delete using POST and asyncoperations? Or a better way if you find one.
@@katielux357 have you tried xrmtoolbox for bulk delete?
Thank you for your video. I would recommend making it couple of minutes longer to detail the body content being transferred and the \" thing, this gave me several issues as it didn't work as indicated when using the new editor. It worked out just fine when using the old flow editor.
Great call out! Now others can see what you wrote in case they're stuck - I added the full code as a pinned comment to help others.
This is epic!.. both the approach and the delivery. Spot on. Thanks for sharing 🙏
Woo! I'm curious how often I'll actually use this :)
I have to ask... looking at all this, it knows my server... HOW does it know the table I'm adding it to? the guid() is a general id... not specific to a particular table... so, were is the table identified? as well, my dataverse submission looks TOTALLY different. It is using pluralized "table names" instead of a long integer/string combination...
The guid tracks that batch and which of your operations are related to that batch. It knows which table based on the URL of the API "/api/data/v9.2/the_name_of_your_table HTTP/1.1"
Wow great video 👏
Does this only create records or can it do delta update too?
I haven't tried with update but I'll make a video on it if I get to it
@@Untethered365 Do you know where to put the existing record Id for a batch update? I can’t find documentation anywhere and the newer UpdateMultiple operation will not work for me. Help 😢
Hello , I am in the process of attempting to utilize excel to create / update records in a SharePoint List. I currently have mu flow functioning the way i want. However, I am stuck at that 5000 limit. My data will be upwards of 200 times that. I know this will move slow but will work for what I would need.
How do I implement this on of these methods into it? I understand that I will have to turn my flow into a solution. Just not sure what that will look like with my needs.
Currently I am utilized a Scheduled Flow--->Get files (properties only)--->Apply to each->Update file properties->List rows present in a table->Apply to each-Get items-Conditions->Delay->Update file Properties
The video I watched to do this is
Reza Dorrani - Add & Update Excel Data to SharePoint List using Power Automate | Excel Import using flow
My flow will be almost Identical to his. The only difference is I did not need to do all the extra steps to select choices. My data is very clean and singular text.
But are you using Dataverse? Because SharePoint batch create is a different API.
@@Untethered365 Sorry SharePoint. When I googled this popped up.
Flow running successfully But not creating record in the table. Can you please help what would be the issue? Thanks in advance.
@@nishankajal1077 please read other comments
I really don't get it... Everything is green in PA, but no records are showing up... How can I see some kind of logging how Dataverse receives the data?
I would create a Compose and copy and paste what's in the documentation and just see if you can make it work. From there look at the difference between your createdOutput and the documentated output.
You won't be able to "debug" because it's handled on the Server side for processing. I think there's an executions or "job" table.
@@Untethered365 thanks for your response. The web api doesn't seem to do anything with the body of the request. Even if I make errors or put a placholder text like 'blabla', all green. In the output I see a "$content": "long text string of different ", but not the body I put in there.
Can you do a similar video for bulk delete using api? Specifically bulk delete for records with matching column value?
@@mohammedfarhan9825 I'll put that on the list!
Can't you use the xrm toolbox for bulk deletes?
If you see the documentation, you can understand that's it a similar process of this video
Hi, I followed your steps and my power automate ran successfully, but the data didn't get uploaded into the table. what could be the reason?
Your JSON is valid but you're missing something that the API requires. Because they're batch requests it can't validate. It just queues it. I'd look at the examples in the documentation using the "Task" entity and make that work. Once that works compare your JSON to that and the issue should appear.
@@Untethered365 I tried the same task entity record creation as in the documentation but still the same problem exists. Flow ran successfully but records didn't get added.
@@NorahAI-h5tunfortunately Dataverse can't validate because it doesn't do the processing in real time. I haven't tried this but you could check the "workflow jobs" in the settings area.
unfortunaly i have this error "The response is not in a JSON format."
Try the example payload in the documentation and compare it to yours. I'm sure there is a simple misplaced comma or something.
@@Untethered365 Thank you for the tutorial, I was able to send a simple batch request with just one row, but as soon as I try to send multiple rows, I get the same error as Vincent mentioned. I also notice in the video that there is one more compose action prior to the http request where I assume you are modifying the string that we append to, but I have scoured the video several times and couldn't find any footage of what formula was used in that compose action. Would you be able to share that here or in the description. That would most likely solve this issue and help anyone new who comes across your video. Thank you for your time.
@@keitho6974 I created a PNG that shows every step of the flow - please let me know if this helps:
untethered365.sharepoint.com/:i:/s/VideoVault/EQY69gfJMMJEqAOmW6amGPwB2UePCjfhNaaaZbxH6qaAIA?e=v0bZTn
@@keitho6974 I'm adding a screenshot of the actual flow:
untethered365.sharepoint.com/:i:/s/VideoVault/EQY69gfJMMJEqAOmW6amGPwB2UePCjfhNaaaZbxH6qaAIA?e=v0bZTn
Keep in mind you could use SELECT and JOIN and do this all without a loop but probably best to keep first version simple.
@@Untethered365 Awesome, thank you for taking the time to get back to me. I really appreciate it.