Troubleshooting: Unable To See Elasticsearch Logs In Orchestrator / 403 Error When Accessing Elasticsearch

News Knowledge Base
1

How to see Elasticsearch logs in Orchestrator if 403 Error is received when access the Elasticsearch?

Issue Description: Users accessing the https://: where Elasticsearch authentication is required are encountering a 403 error.


Background:

If Elasticsearch is secured with username/password, the user must have sufficient rights to access (read/ write) the indices named . Two scenarios in particular are executed whenever the Logs in the Robots/Jobs section of Orchestrator are accessed.


For the logs viewed from the Robots page (default filters/paging), if following is required,

Orchestrator request to get log count:

  • GET /odata/RobotLogs/UiPath.Server.Configuration.OData.GetTotalCount?$filter=(TimeStamp gt and RobotName eq '')&$orderby=TimeStamp desc&$top=10

Corresponding ES request:

POST /-*/logEvent/_count

with JSON body:

{
"query": {
"bool": {
"must": [{
"match_all": {}
}
],
"filter": [{
"range": {
"timeStamp": {
"gt": ""
}
}
}, {
"term": {
"robotName.keyword": {
"value": ""
}
}
}
]
}
}


Orchestrator request to search for the logs:

  • GET /odata/RobotLogs?$filter=(TimeStamp gt and RobotName eq 'robot name')&$orderby=TimeStamp desc&$top=10

Corresponding ES request:

POST /-%2A/logEvent/_search

with JSON body:

{
"size": 10,
"sort": [{
"timeStamp": {
"order": "desc"
}
}
],
"query": {
"bool": {
"must": [{
"match_all": {}
}
],
"filter": [{
"range": {
"timeStamp": {
"gt": ""
}
}
}, {
"term": {
"robotName.keyword": {
"value": ""
}
}
}
]
}
}
}

For the logs viewed from the Jobs page (default filters/paging), there are two requests to the Orchestrator server as well, each with a different request to the ES server:

Orchestrator request:

  • GET /odata/RobotLogs/UiPath.Server.Configuration.OData.GetTotalCount?$filter=(JobKey eq 'job key')&$orderby=TimeStamp desc&$top=10

Corresponding ES request:

POST /-%2A/logEvent/_count

with JSON body:

{
"query": {
"bool": {
"must": [{
"match_all": {}
}
],
"filter": [{
"term": {
"jobId.keyword": {
"value": ""
}
}
}
]
}
}
}

Orchestrator request:

  • GET /odata/RobotLogs?$filter=(JobKey eq )&$orderby=TimeStamp desc&$top=10

Corresponding ES request:

POST /-%2A/logEvent/_search

with JSON body:

{
"size": 10,
"sort": [{
"timeStamp": {
"order": "desc"
}
}
],
"query": {
"bool": {
"must": [{
"match_all": {}
}
],
"filter": [{
"term": {
"jobId.keyword": {
"value": ""
}
}
}
]
}
}
}

Symptoms

Accessing the https://: in the browser and authenticating produces a 403 error.

Or

Turning on the NLog Debugging generates a 403 in the NLog debug logs.

2019-01-14 13:13:10.3460 Error Failed to send log messages to elasticsearch: status=403, message="One or more errors occurred."
2019-01-14 13:13:10.3460 Error Error while sending log messages to elasticsearch: message="One or more errors occurred."
2019-01-14 13:13:10.9295 Debug Targets for UiPath.Web.Controllers.HomeController by level:


Resolution

The user needs to be granted sufficient access to perform the above queries on the indices by the team managing the ElasticSearch application.