Troubleshooting: Unable to see ElasticSearch logs in Orchestrator / 403 Error when accessing Elastic Search

How to see ElasticSearch logs in Orchestrator if receive 403 Error when access the ElasticSearch?

Problem

Users accessing the https://<ELASTICSEARCH>:<PORT> where ElasticSearch authentication is required are encountering a 403 error.

Background

If ElasticSearch is secured with username/password, the user must have sufficient rights to access (read/write) the indices named <tenant name>. Two scenarios in particular are executed whenever the Logs in the Robots/Jobs section of Orchestrator are accessed.

For the logs viewed from the Robots page (default filters/paging):

Orchestrator request to get log count:
GET 
<Orchestrator server>/odata/RobotLogs/UiPath.Server.Configuration.OData.GetTotalCount?$filter=(TimeStamp gt <some datetime> and RobotName eq '<robot name>')&$orderby=TimeStamp desc&$top=10

Corresponding ES request:
POST <ES server>/<tenant name>-*/logEvent/_count

with JSON body:

 

{
	"query": {
		"bool": {
			"must": [{
					"match_all": {}
				}
			],
			"filter": [{
					"range": {
						"timeStamp": {
							"gt": "<some datetime>"
						}
					}
				}, {
					"term": {
						"robotName.keyword": {
							"value": "<robot name>"
						}
					}
				}
			]
		}
	}

 

Orchestrator request to search for the logs: 
GET <Orchestrator server>/odata/RobotLogs?$filter=(TimeStamp gt <some datime> and RobotName eq 'robot name')&$orderby=TimeStamp desc&$top=10

Corresponding ES request:
POST <ES server>/<tenant name>-%2A/logEvent/_search

with JSON body:

 

{
	"size": 10,
	"sort": [{
			"timeStamp": {
				"order": "desc"
			}
		}
	],
	"query": {
		"bool": {
			"must": [{
					"match_all": {}
				}
			],
			"filter": [{
					"range": {
						"timeStamp": {
							"gt": "<some datetime>"
						}
					}
				}, {
					"term": {
						"robotName.keyword": {
							"value": "<robot name>"
						}
					}
				}
			]
		}
	}
}

 

 

For the logs viewed from the Jobs page (default filters/paging):

There are two requests to the Orchestrator server as well, each with a different request to the ES server:

Orchestrator request: 
GET <Orchestrator server>/odata/RobotLogs/UiPath.Server.Configuration.OData.GetTotalCount?$filter=(JobKey eq 'job key')&$orderby=TimeStamp desc&$top=10

Corresponding ES request:
POST <ES server>/<tenant name>-%2A/logEvent/_count

with JSON body:

 

{
	"query": {
		"bool": {
			"must": [{
					"match_all": {}
				}
			],
			"filter": [{
					"term": {
						"jobId.keyword": {
							"value": "<job key>"
						}
					}
				}
			]
		}
	}
}

 

 

Orchestrator request: 
GET <Orchestrator server>/odata/RobotLogs?$filter=(JobKey eq <job key>)&$orderby=TimeStamp desc&$top=10

Corresponding ES request:
POST <ES server>/<tenant name>-%2A/logEvent/_search

with JSON body:
 

{
	"size": 10,
	"sort": [{
			"timeStamp": {
				"order": "desc"
			}
		}
	],
	"query": {
		"bool": {
			"must": [{
					"match_all": {}
				}
			],
			"filter": [{
					"term": {
						"jobId.keyword": {
							"value": "<job key>"
						}
					}
				}
			]
		}
	}
}

 

 

Symptoms

Accessing the https://<ELASTICSEARCH>:<PORT> in the browser and authenticating produces a 403 error.
OR
Turning on NLog debugging generates a 403 in the NLog debug logs.

 

2019-01-14 13:13:10.3460 Error Failed to send log messages to elasticsearch: status=403, message="One or more errors occurred."
2019-01-14 13:13:10.3460 Error Error while sending log messages to elasticsearch: message="One or more errors occurred."
2019-01-14 13:13:10.9295 Debug Targets for UiPath.Web.Controllers.HomeController by level: 

 


Solution

The user needs to be granted sufficient access to perform the above queries on the <tenant name> indices by the team managing the ElasticSearch application.