Hi all,
We have started to experience a couple of strange issues that I suspect are related as they both involve the Jobs table in Orchestrator.
Using Orchestrator 2019.10.15 and 15 x attended robots on 20.10.2
5 days ago, we started to experience the following 2 issues.
FIRST ISSUE
The Jobs view on Orchestrator would take a long time to return a list - and now they mostly time out after about 100 seconds with an error (#100 or #101) flashing up briefly. I am assuming it is a timeout.
I can see the Orchestrator server working hard during that 100 seconds, especially CPU (it adds about 50% of the CPU capacity to whatever is already being used)
Orchestrator DB server looks unstressed during the 100 seconds.
All the other views in Orchestrator return their usual lists (including Logs which is a much bigger table behind the scenes).
ISSUE 2
Around the same time, we are seeing a large number of attended Bot runs that report as “Faulted”.
However, the logs for those Jobs show the automation completes OK, and the human agents report that the Bot completes it’s tasks.
The odd thing is the info field in the Jobs table (not visible inside Orchestrator, but can be seen by connecting direct to the SQL Server) reports that it is unable to get the result of the job execution (and hence reports it as faulted as it cant confirm it was successful).
The full text in the jobs.info field every time is:
“Could not retrieve the result of the job execution. This might be because a message was too large to process.
RemoteException wrapping System.Exception: Could not retrieve the result of the job execution. This might be because a message was too large to process.”
We have checked and double-checked, and there is nothing we can find that would produce a large message. Nothing has changed in the automation.
I am not familiar with what “RemoteException wrapping” would be - sounds like Orchestrator has some built-in exception handling for info it is receiving from the robots ie generated remote from the Orchestrator?
In summary, both these issues seem to affect the jobs table in Orchestrator.
Nothing has changed in the automations themselves to introduce a new exception.
I am hoping someone in the community has come acorss this before and may be able to give me some pointers.
Thanks for taking the time to read this.
Graham