Robot logs not getting sent to elasticsearch after the upgrade to Orchestrator 2019.10.14

Help
,
1

Have anyone else experienced problems with robot logs not getting sent to elasticsearch after the upgrade to Orchestrator 2019.10.14?

taken from web.config

  <target name="robotElasticBuffer" xsi:type="BufferingWrapper" flushTimeout="5000">
    <target xsi:type="ElasticSearch" name="robotElastic" index="${event-properties:item=indexName}-${date:format=yyyy.MM}" documentType="logEvent" includeAllProperties="true" layout="${message}" excludedProperties="agentSessionId,tenantId,indexName" uri="secret" requireAuth="true" username="secret" password="secret" />
  </target>
<rules>
  <logger name="BusinessException.*" minlevel="Info" writeTo="businessExceptionEventLog" final="true" />
  <logger name="Robot.*" final="false" writeTo="robotElasticBuffer" minlevel="Info" />
  <logger name="Robot.*" final="true" writeTo="database" minlevel="Warn" />
</rules>

Orchestrator successfully reads old logs, so there is no issue with the elasticsearch uri and credentials.

3 Likes
2

Hi @matsi1

Welcome to our UiPath Forum! :slight_smile:

Could you let us know why is this value set to False and what stops you from sending the logs directly to robotElastic instead of robotElasticBuffer?

<logger name="Robot.*" final="false" writeTo="robotElasticBuffer" minlevel="Info" />
4 Likes
3

Hi, well thank you :slight_smile:

we want info and higher logs to end up on ElasticSearch but only want warning and higher level errors to get sent to database, therefor the final is set to false for the first rule, which we were recommended by UiPath to do :slight_smile:

to write to ElasticSearch it should be set to robotElasticBuffer according to documentation (and what we have been using last years)

2 Likes
4

Hi @matsi1

Could you give us an update on the versions you are using?

It seems like 2019.10 bumped the version requirement for Elasticsearch, see the documentation here:
2.3 -6.x in 2018.4

vs
6.x - 7.x in 2019.10

2 Likes
5

Hi, we use 6.8.2

1 Like
6

Hello,

Could you please provide us the error message received in the Event Viewer logs? This is a generic error and we need to better understand your case.
Also, please tell us from which version did you upgrade the Orchestrator.

Rocs

7

Hi, we have now had a private call with UiPath tech support and we haven’t got it resolved yet, but enabling the debug for NLOG it seems this could be ca cause? We use a on-prem ELK cloud with this cluster setup to 1 node and it shouldn’t try to access / PING using any other then the URL we provide in the nlog settings.

ElasticSearch: Failed to send log messages. status=200 Exception: Elasticsearch.Net.ElasticsearchClientException: Failed to ping the specified node… Call: Status code 200 from: HEAD / —> Elasticsearch.Net.PipelineException: Failed to ping the specified node. —> Elasticsearch.Net.PipelineException: An error occurred trying to read the response from the specified node.
at Elasticsearch.Net.RequestPipeline.Ping(Node node)

8

And we upgraded from 2018.4.X

10

Hello,

I have checked again your case and I wasn’t able to reproduce the error, using the same version of ES.
I have checked your settings related target and logger and they are fine. “robotElasticBuffer” is the right target name.
Could you please check if the URI is like “127.0.0.1:9200” or “http://127.0.0.1:9200” ?

Thanks,
Rocsana

1 Like
11

@matsi1, can you try to set DisablePing="true"on your robotElastic target?

I haven’t tried it, so I’m not sure if it’ll work, but from your messages so far I have a feeling this might be the issue, especially given that others have reported similar problems on 1-node clusters: Elasticsearch.Net NEST connection string when Active Directory turn on (Elasticsearch Authentication) - #10 by sabahshariq - Elasticsearch - Discuss the Elastic Stack

(btw, we don’t encourage the use of 1-node clusters, certainly not for production)

2 Likes
12

Hi, this actually solved the issue and it’s now working! (this is a test environment, hence the setup with only one node :slight_smile: )

but this then means that there was a change in the client version you use in 2018.4 and 2019.10 since same setup is working with 2018.4 without any problems, and what is the actual root cause?

1 Like
13

URI is in this form uri=“https://elasticid.something.something.com”, port is not specified.

14

Of course there was a change in the client version… 2018.4 doesn’t support Elasticsearch clusters with version 7.x, and lots of customers expect us to support newer releases. As to “the actual root cause” it’s the fact that ES7 clients are not officially supported to work well with ES6 clusters (or any other cluster versions). We tested the client that we use in a bunch of scenarios & cluster versions and it looked like for our usage it’s ok, but this situation is one of the things we apparently missed (I understand that it does work with a 6.8.3 one-node cluster in Azure, so maybe it’s only an issue with 6.8.2 & below? Anyway should only be an issue on one-node clusters)

3 Likes
15

@virgilp for testing purposes we added a couple of more nodes, but same issue is still with 3 nodes. Without setting DisablePing to True it fails.

How do you configure the StaticConnectionPool? since we only provide one URI in the config, how do you check which nodes are there to setup the connection to check if they are alive or not?

16

Hi, it seems be tied to that we use a LB for elastic, so actually all failover logic is handled at LB level and therfor we are also confident in using DisablePing set to True for our specific case. Thanks for your expertise and guidenc on this matter @virgilp !

1 Like
17

You can now set multiple nodes (comma-separated) in the ‘uri’; LB is probably better, but for simpler/less critical deployment setups, that works too.

You’re welcome! Thanks for circling back, I think you’re right, the issue is likely caused by the LB not by the fact that cluster is made out of a single node.

2 Likes
18

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.