Database maintenance and deletion of data of more than 30 days

Help Orchestrator
1

In the example maintenance script uipath suggest using only status=3 for the delete of dbo.QueueItems

  1. would there be any reason to leave the other QueueItem statuses in Prod? [0-2,4-7]
    (or was status=3 just an example of a random WHERE clause?)

  2. will the insight database have any need (or knowledge) of the deleted data i the uiPath database? (see Architecture)

1 Like
2

Hi @david.jonsson ,

  1. would there be any reason to leave the other QueueItem statuses in Prod? [0-2,4-7]
    (or was status=3 just an example of a random WHERE clause?)
  • UiPath suggest using only status=3 & which are older than 60 days for the delete of dbo.QueueItems. Since those Transaction items are already Processed successfully and transaction items of other statuses you may be needed in future for troubleshooting or retrying purposes.
    However as per your need you can also clear the other Items from the dbo.QueueItems table using its respective status.

Regards,
Rohith

3

Ok, then I’m left with 4 years of failed and other transactions (and growing).
Suggestion?

I assume uiPath would suggest a staggered delete (Status = 3 after 60 days plus Status <> 3 after 365 days)

But - If I’m troubleshooting I rather see the full picture (successful transactions AND all other statuses). I will keep it simple => Clear all transaction, regardless of status, after one year.

Thx

1 Like
4

What you do and/or need to do comes down to your own policies for Data Retention as well as balancing that with resource limitation for the growth of data and potential performance and requirements for Operational Monitoring/Triaging and of course Reporting.

For instance we push a lot of our data into another database and/or platform like Splunk and Snowflake and do not typically use Orchestrator except for the more recent activity < 30 days. Because of this we configured a SQL Server Job with multiple Steps to cleanup various items.

Our Non-Prod Orchestrator where our Development happens is less aggressive in this trimming compared to Production as it more helpful to us during the development phase. Production is more aggressive for logs due to the order of magnitude higher for activity going on and records growing in size quite quickly for us, but everything else is generally after 30 days.

Example from our Non-Prod Orchestrator
image

Example from Prod Orchestrator
image

There are a few other tables that I’ve been watching and considering if we want to include them but the growth on them is not that noticeable.

Alternatives if you want to keep the data / logs, but perhaps keep your primiary database in check is to replicate it to a secondary database, or ElasticSearch. You could also using UiPath Insights which leverages a second database as well. And other options is platforms like Splunk and Snowflake, etc.