While planning of the implementation of relevant bulk delete jobs for all integrated D365 organizations, the unchanged limitations made me sad.

Bulk deletion jobs have annoying limitations

Initially, this feature does not support the deletion of activities except you would create the job programmatically using Web API. Next point is, that you cannot change the filter criteria after creation. Furthermore, it is not possible to add it to a solution as a component. In addition, it does not support multithreading. Lastly, an email notification could not be limited to a failure case. In conclusion, I decided to check the possibilities in Power Automate.

Recurring flow is powerful and supports CI/CD

A flow does not have any of the limitations listed above. So I decided to delete data in bulk using a recurring flow. The challenge was, to find a way to delete more than 100k records. The solution is to implement pagination. To do so, I am using @odata.nextLink and the Do until control. The list records action of CDS (current environment) always returns an @odata.nextLink property, when the defined filter criteria leads to more than the defined maximum page size (default = 5,000). The implemented flow is going to retrieve and delete those until the property is not returned. Please note, that this pagination design only works, if the retrieved data is deleted before retrieving the next page. I am looking for a working alternatives for other use cases and will publish an article, after I have one using the CDS current environment actions.

Concurrency control is the key for dynamic performance

The list records action of CDS returns a list of items. The best way to process each item is to use the apply to each action. Per default those items will be processed one by one. It is possible to change this behavior in the action settings. The maximum degree of parallelism is 50 and worked in many cases without reaching any API limits. But it might change after the full support of API request capacity add-on.

This is how the flow looks like

#Dynamics365CE #BulkDeletionJob #PowerAutomate #Pagination #ConcurrencyControl #Parallelism #Multithreading