Get Modern Folder FullyQualifiedName as Index in Log Target

[rdgagp (Rodrigo Augusto Gonçalves Pinto)]

Hello All,

I’ve been looking for a solution to manage the access in our Reporting in Kibana by having the index patterns based on the modern folder structure.

The idea would be the following:

• Users with rights to see the Index Pattern TestFolder* would see all logs of the TestFolder, TestFolderSon and TestFolderGrandson

• Users with rights to see the Index Pattern TestFolder\TestFolderSon* would see all logs of the TestFolderSon and TestFolderGrandson

• Users with rights to see the Index Pattern TestFolder\TestFolderSon\TestFolderGrandson* would see all logs of the TestFolderGrandson

Example

I have set a Target (UiPath.Orchestrator.dll.config) to log into Kibana but I am still having issue to get the information to put as dynamic Index. How do I get the full name of the modern folder the process is currently running? A solution directly in the config file or in Studio would be great. I found in the Orchestrator Database ([dbo].[OrganizationUnits]) exactly the info I need to use as Index. Does anyone know how to get this info in the config file or into a log message in Studio?

Thanks!

[cornelius.nowald]

Hi Rodrigo,

I tried a few thing but could not find a way to get the folder names into index only the FolderIDs.
But this could also be due to my lack of skills in Nlog configuration

First the way I got folder IDs into the index.
I created a similar folder strcture to yours

With IDs 33,34,45

{
  "Key": "6619ec97-042b-48b6-b5ba-f0061e3c8116",
  "DisplayName": "LogTestFolder",
  "FullyQualifiedName": "LogTestFolder",
  "FullyQualifiedNameOrderable": "LogTestFolder",
  "Description": null,
  "ProvisionType": "Automatic",
  "PermissionModel": "FineGrained",
  "ParentId": null,
  "ParentKey": null,
  "IsActive": true,
  "FeedType": "Processes",
  "Id": 33
},
{
  "Key": "2da17247-1051-4c10-b58d-793b2fd617e9",
  "DisplayName": "LogTestFolderSon",
  "FullyQualifiedName": "LogTestFolder/LogTestFolderSon",
  "FullyQualifiedNameOrderable": "LogTestFolder\u0000LogTestFolderSon",
  "Description": null,
  "ProvisionType": "Automatic",
  "PermissionModel": "FineGrained",
  "ParentId": 33,
  "ParentKey": "6619ec97-042b-48b6-b5ba-f0061e3c8116",
  "IsActive": true,
  "FeedType": "Processes",
  "Id": 34
},
{
  "Key": "f664a013-7022-4bad-be0b-674008527fd8",
  "DisplayName": "LogTestFolgerGrandSon",
  "FullyQualifiedName": "LogTestFolder/LogTestFolderSon/LogTestFolgerGrandSon",
  "FullyQualifiedNameOrderable": "LogTestFolder\u0000LogTestFolderSon\u0000LogTestFolgerGrandSon",
  "Description": null,
  "ProvisionType": "Automatic",
  "PermissionModel": "FineGrained",
  "ParentId": 34,
  "ParentKey": "2da17247-1051-4c10-b58d-793b2fd617e9",
  "IsActive": true,
  "FeedType": "Processes",
  "Id": 35
},
{
  "Key": "c44945f5-267c-4beb-96a3-e01e4abddbfc",
  "DisplayName": "LogTestFolderSibling",
  "FullyQualifiedName": "LogTestFolder/LogTestFolderSibling",
  "FullyQualifiedNameOrderable": "LogTestFolder\u0000LogTestFolderSibling",
  "Description": null,
  "ProvisionType": "Automatic",
  "PermissionModel": "FineGrained",
  "ParentId": 33,
  "ParentKey": "6619ec97-042b-48b6-b5ba-f0061e3c8116",
  "IsActive": true,
  "FeedType": "Processes",
  "Id": 36
}

I modified the robotElasticBuffer target to also include tenantId and organizationUnitId

<target name="robotElasticBuffer" xsi:type="BufferingWrapper" flushTimeout="5000">
    <target xsi:type="ElasticSearch" name="robotElastic" requireAuth="false" username="" password="" index="${event-properties:item=indexName}-${event-properties:item=tenantId}-${event-properties:item=organizationUnitId}-${date:format=yyyy.MM}" documentType="logEvent" includeAllProperties="true" layout="${message}" excludedProperties="agentSessionId,tenantId,indexName" uri="http://127.0.0.1:9200/" />
</target>

Runnig a Job in each folder caused to following indices to be created

My next attempt was/is probably to simple but I guessed lets try it anyway. I created an asset in each Folder which hold the full Orchestrator folderpath e.g.:

Then I added fetching the asset and adding it as a logfield orchestratorFolderName to my test bot

Next comes the issue I cannot solve at the moment. I do not know to to use the added Logfield in the Orchestrators Nlog config. I tried a few ways but did not find a working solution.
Here my curtent Orchestrator Config attempt, maybe someone can helpout of how to fetch the orchestratorFolderName logfield, so it can be used in the Nlog targets.

I will also keep trying!

[cornelius.nowald]

Hi @rdgagp ,
I tried few more things to fetch the folder name (not just folder-ID) and directly apply to the index name, but I think without UiPath including the FQN as event property this will not work.

I also created a very cumbersome workaround, but in comparison to this I think managing access to indices by mapping folder-IDs to users is going to be easier.

I manually created a sperate target for each folder (paying attention to the ES index-naming-convention)

and a separate logger rule for each (sub-)folder

This creates log entries in in a sperate index for each folder where hereditary access to index pattern could be configured.

But even with scripts that fetch the folder structure from Orchestrator via API and automatically creates the log targets, this is going to be quite cumbersome.
And the performance will also take a dive once there are a lot of folders.

[cornelius.nowald]

Hi @rdgagp,
I think I found a way which might work using some srcipts the query the Orchestrator API for the folder FQNs and sanitizes them using ES index aliases .

Based on this folder structure

I create index aliases using the ES API (manually sanitizing the names for this test)

PUT /default-1-33*/_alias/logtestfolder
PUT /default-1-34*/_alias/logtestfolder_logtestfolderson
PUT /default-1-35*/_alias/logtestfolder_logtestfolderson_logtestfoldergrandson

then queried to check inf the aliases was setup corrently using

GET /_alias/logtestfolder*

then I could setup a new index pattern using the sanitzed name

So if the folder names hold for example no underscores one could use this as seperator instead of \ so the index names still adhere to ES naming requirements.

[rdgagp (Rodrigo Augusto Gonçalves Pinto)]

Hi @cornelius.nowald. Thank you very much. I guess this is a good solution given the constraints.

For other coleagues having the same idea:

As the aliases can only be created if there is corresponding data in ES. We are setting up a process to deal with that. We will create a procedure in the Go Live which consist of running a process that logs some data to ES and create automatically the aliases (as explained above).