Orchestrator +Elasticsearch&kibana 7.5.0 with security X-pack

I am playing with possibilities of Elasticsearch and Orchestrator.
I have got working my robots. They are sending logs not only in orchestrator but also in Elastic.

Next step was authentication and security in Elastic+kibana.
I folllowed instructions at the elastic and now i can reach my kibana via “HTTPS” and elastic is “Secured at transport level and http level”
I can see my indexes i made BEFORE security settings.

ALL certificates generated without passwords.
Orchestrator, elastic and kibana are installed at ONE win2012 server .
Elastic has ONE node .
NOthing espesial.
But elastic dont bite orchestrator:
oke
elasticsearch.yml

bootstrap.memory_lock: false
cluster.name: elasticsearch
http.port: 9200
node.data: true
node.ingest: true
node.master: true
node.max_local_storage_nodes: 1
node.name: VM1
path.data: C:\ProgramData\Elastic\Elasticsearch\data
path.logs: C:\ProgramData\Elastic\Elasticsearch\logs
transport.tcp.port: 9300
xpack.license.self_generated.type: basic
xpack.ml.enabled: false
xpack.security.enabled: true

xpack.security.transport.ssl.enabled: true

xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: C:\ProgramData\Elastic\Elasticsearch\config\elastic-stack-ca.p12
xpack.security.transport.ssl.truststore.path: C:\ProgramData\Elastic\Elasticsearch\config\elastic-stack-ca.p12

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: C:\ProgramData\Elastic\Elasticsearch\config\ca.key
xpack.security.http.ssl.certificate: C:\ProgramData\Elastic\Elasticsearch\config\ca.crt

Kibana.Yml : ( part)
#---------------------------------------------------------------

elasticsearch.hosts: [“https://localhost:9200”]

#----------------------------------------------------------------
server.ssl.enabled: true
server.ssl.certificate: C:\kibana\kibana-7.5.0-windows-x86_64\config\KB.crt
server.ssl.key: C:\kibana\kibana-7.5.0-windows-x86_64\config\KB.key
#----------------------------------------------------------------
elasticsearch.ssl.certificate: C:\ProgramData\Elastic\Elasticsearch\config\ca.crt
elasticsearch.ssl.key: C:\ProgramData\Elastic\Elasticsearch\config\ca.key
elasticsearch.ssl.certificateAuthorities: C:\ProgramData\Elastic\Elasticsearch\config\ca.crt
elasticsearch.ssl.verificationMode: certificate

They are working well .
Web.config ( part)





i just changed uri=“http://localhost:9200” -> uri=“https://localhost:9200
and it doesnt enought.

QUESTION: what i do wrong ?
and where can i read about how orchestrator is working with secure clusters of elastic?
Thank you and have nise day

P.S:
and yes? I can not see logs in my orchestrator also . Error is generating.

1 Like

Hi,

I have the same issue. Have you been able to resolve it since ?

Mike

2 Likes

I didnt resolve Issue with HTTPS but you have to add Username and password in web.config anyway ! These passwords are the same as password and username in Kibana.Yml

 <target name="robotElasticBuffer" xsi:type="BufferingWrapper" flushTimeout="5000">
    <target xsi:type="ElasticSearch" name="robotElastic1" requireAuth="false" **username="elastic" password="51myGXeQTb0333INgTiC"** index="${event-properties:item=indexName}-${date:format=yyyy.MM}" documentType="logEvent" includeAllProperties="true" layout="${message}" excludedProperties="agentSessionId,tenantId,indexName" uri="http://localhost:9200" />
  </target>
  <target name="serverElasticBuffer" xsi:type="BufferingWrapper" flushTimeout="5000">
    <target xsi:type="ElasticSearch" name="serverElastic" requireAuth="false" username="elastic" password="51myGXeQTb0333INgTiC" index="serverdiagnostics-${date:format=yyyy.MM}" documentType="logEvent" includeAllProperties="true" layout="${machinename} ${message}" uri="http://localhost:9200" />
  </target>
1 Like

also there is something wrong with my SSL certificates ( i think)

1 Like

I think you’re right regarding the certificate.

I removed the ssl config for Elasticsearch and I’m able to see logs both in Orchestrator and Kibana without any problems. And Elasticsearch’s log looks clean.

Then I re-enabled ssl config for Elasticseach and I can only see logs in Orchastrator. Elasticsearch is not getting logs anymore. And Elasticsearch’s log file displays the following:

[2020-01-23T11:11:15,346][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [SERVERNAME] http client did not trust this server’s certificate, closing connection [id: 0x6c323a92, L:0.0.0.0/0.0.0.0:9200 ! R:/0:0:0:0:0:0:0:1:51900]

2 Likes

Hi,
I managed to configure Elasticsearch and Kibana with HTTPS and have logs both in Orchestrator and Elasticsearch/Kibana.

Recreated a new certificate was indeed the key.

Here are the main steps:

Create keystore whith private key pair

cd C:\Program Files\Java\jre1.8.0_181\bin

keytool -genkey -alias YourServerName -keyalg RSA -keysize 2048  -dname "CN=YourServerName.FQDN.local,OU=yourOU,O=yourO,L=YourLocation,ST=Some-State,C=YourCity" -keystore cacerts -storepass YourPwd -keypass YourPwd -validity 3650 –ext SAN=dns:yourservername.fqdn.local,dns:yourservername_alias.fqdn.local

A warning might appear telling you to convert cacert type

keytool -importkeystore -srckeystore cacerts -destkeystore cacerts -deststoretype pkcs12
2 Likes

Export the root certificate

keytool -export -v -alias YourServerName -file C:\temp\yourservername.cer -keystore cacerts -storepass YourPwd

You can export a PFX file including private key, with the following command

C:\OpenSSL-Win64\bin\openssl pkcs12 -in "C:\Program Files\Java\jre1.8.0_181\bin\cacerts" -nokeys -out C:\temp\yourservername.pem

C:\OpenSSL-Win64\bin\openssl pkcs12 -in "C:\Program Files\Java\jre1.8.0_181\bin\cacerts" -nodes -nocerts -out C:\temp\key.pem

C:\OpenSSL-Win64\bin\openssl pkcs12 -inkey C:\temp\key.pem -in C:\temp\yourservername.pem -export -out C:\temp\yourservername.pfx

Then import the certificate into the local store

Using the mmc cert tool import the certificate to the Local Computer / Personnal / Certificate and Local Computer / Truster Root Certification Authorities stores.

2 Likes

Optional: You can use the certificate for Orchestrator as well

You should then import the pfx file into the Server Certificate feature of IIS and reconfigure the Bindings with the new certificate on the UiPath Orchestrator app pool level.

Then restart IIS

2 Likes

Then adapt your config files

web.conf

	      <target name="robotElasticBuffer" xsi:type="BufferingWrapper" flushTimeout="5000">
        <target xsi:type="ElasticSearch" name="robotElastic1" uri="https://yourservername.FQDN.local:9200" requireAuth="false" username="" password="" index="${event-properties:item=indexName}-${date:format=yyyy.MM}" documentType="logEvent" includeAllProperties="true" layout="${message}" excludedProperties="agentSessionId,tenantId,organizationUnitId,indexName" />
      </target>
      <target name="serverElasticBuffer" xsi:type="BufferingWrapper" flushTimeout="5000">
        <target xsi:type="ElasticSearch" name="serverElastic" uri="https://yourservername.FQDN.local:9200" requireAuth="false" username="" password="" index="serverdiagnostics-${date:format=yyyy.MM}" documentType="logEvent" includeAllProperties="true" layout="${machinename} ${message}" />
      </target>
<rules>
  <logger name="BusinessException.*" minlevel="Info" writeTo="businessExceptionEventLog" final="true" />
  <logger name="Robot.*" writeTo="database,robotElasticBuffer" final="true" />
  <logger name="Monitoring.*" writeTo="monitoring" minlevel="Warn" final="true" />
  <logger name="Quartz.*" minlevel="Warn" writeTo="eventLogQuartz" final="true" />
  <logger name="*" minlevel="Info" writeTo="eventLog" />
</rules>

elasticsearch.yml

bootstrap.memory_lock: false
cluster.name: elasticsearch
network.host: 1.2.3.4 <your ip address>
http.port: 9200
node.data: true
node.ingest: true
node.master: true
node.max_local_storage_nodes: 1
node.name: YOURSERVERNAME
path.data: your path to the data folder
path.logs: your path to the logs folder
transport.tcp.port: 9300
xpack.license.self_generated.type: basic
xpack.security.enabled: true
xpack.ssl.verification_mode: none
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.keystore.path: your path to the cacerts
xpack.security.transport.ssl.keystore.password: YourPwd
xpack.security.transport.ssl.truststore.path: your path to the cacerts
xpack.security.transport.ssl.truststore.password: YourPwd
xpack.security.transport.ssl.verification_mode: none
xpack.security.http.ssl.enabled: true
#xpack.security.http.ssl.client_authentication: none
xpack.security.http.ssl.keystore.path: your path to the cacerts
xpack.security.http.ssl.keystore.password: YourPwd
xpack.security.http.ssl.truststore.path: your path to the cacerts
xpack.security.http.ssl.truststore.password: YourPwd

kibana.yml

server.port: 5601
server.host: "YourServerName.FQDN.local"
elasticsearch.hosts: ["https://YourServerName.FQDN.local:9200"]
server.ssl.enabled: true
server.ssl.certificate: C:\temp\YourServerName.pem
server.ssl.key: C:\temp\key.pem
elasticsearch.ssl.verificationMode: none

Remember that any edit on those files need a service restart in order to be effective.

IE and Chrome can have a slightly different behavior regarding the self-signed certificates, you can add the url to the Local Intranet zone in IE Internet Options.

Sorry for the multiple post, timeouts prevent me to edit old posts.

Hope it helps.

Mike

2 Likes

HI Mike! Tomorrow i will try it ! Thank you very much for very detailed explanation!
Greets, Maryna

1 Like

keytool -genkey -alias YourServerName -keyalg RSA -keysize 2048 -dname “CN=YourServerName.FQDN.local,OU=yourOU,O=yourO,L=YourLocation,ST=Some-State,C=YourCity” -keystore cacerts -storepass YourPwd -keypass YourPwd -validity 3650 –ext SAN=dns:yourservername.fqdn.local,dns:yourservername_alias.fqdn.local

L=YourLocation,ST=Some-State,C=YourCity
Those parameters i should change to? i am Netherland located and we have no states ))))) if i set NL NL NL ?

1 Like

Yes, you can basicaly put whatever you want. Those information are here to reflect your infra/company setup, but as it’s a self-signed, there is no need to pay too much attenion.

Nevertheless, the CN and the SAN value are important.

1 Like

HI! I dont have pure JAVA
cd C:\Program Files\Java\jre1.8.0_181\bin"" instead of it i used
cd C:\Program Files\Elastic\Elasticsearch\7.5.0\jdk
and i didnt get parameter SAN and used
keytool -genkey -alias vm1.elastic -keyalg RSA -keysize 2048 -dname “CN=vm1,OU=blabla BV,O=blabla,L=Roermond,ST=Lim,C=NL” -keystore cacerts -storepass password -keypass password -validity 3650

I still not solved problem :slight_smile:

1 Like

I think that using a keytool.exe from somewhere else than Java won’t matter that much.

I would recomment anyway to use a SAN value, as the certificate will be better interpreted by the browser, especially Chrome.

On my side, after running the first cmd, the old cacerts has been renamed carcerts.old and a new one has been created.

Is there a cacerts file with the current time on your location (C:\Program Files\Elastic\Elasticsearch\7.5.0\jd) ?

1 Like

yes yes and all another files with open SSL i have got too :slight_smile: i am confusing a bit with these parameters in “YOUR” elastic config:
*----------------------------------------------------------
xpack.security.transport.ssl.keystore.path: your path to the cacerts
xpack.security.transport.ssl.keystore.password: YourPwd
xpack.security.transport.ssl.truststore.path: your path to the cacerts
xpack.security.transport.ssl.truststore.password: YourPwd
xpack.security.transport.ssl.verification_mode: none
*---------------------------------------------
i made something like this
xpack.security.transport.ssl.keystore.path: C:\TEMP\cacerts
xpack.security.transport.ssl.keystore.password: blablabla
xpack.security.transport.ssl.truststore.path: C:\TEMP\cacerts
xpack.security.transport.ssl.truststore.password: blablabla
xpack.security.transport.ssl.verification_mode: none
*-----------------
service would not be running
logs says - O !NO! certificates has to be located in elastic config map C:\ProgramData\Elastic\Elasticsearch\config
okey ( said me) i copied them from c:\temp an dchanged config parameters - nope ((((

1 Like

Yes, it seems that the cacerts file need to be located in the config folder of Eleasticseach (C:\ProgramData\Elastic\Elasticsearch\config\cacerts), this is also the case on my side.

The same has to be done for the security.HTTP section

xpack.security.http.ssl.keystore.path: C:\ProgramData\Elastic\Elasticsearch\config\cacerts
xpack.security.http.ssl.keystore.password: blablabla
xpack.security.http.ssl.truststore.path: C:\ProgramData\Elastic\Elasticsearch\config\cacerts
xpack.security.http.ssl.truststore.password: blablabla
2 Likes