Pulling logs from DB

[Seetharaman_K]

Hi all ,
we want to pull the logs from orch DB and build some reporting in power bi. has any one done this exercise before ?
Note :
we are having elastic server but we thought we will move away from elastic. for various reasons. so we thought this kind of serial architecture would be good.
robot logs – orch – sql db – reporting DB
rather than robot logs — orch – orch db and elastic ( via web. config simultaneously)

so please let me know your experience in this

[codemonkee (Tim)]

I would look at adding an additional target to your Orchestrator NLog configuration so it writes the logs both to Orchestrator’s SQL Server DB and your Reporting DB. If your Reporting platform can keep track of ingesting the database table, that could be an option, but depending on how far back you want to keep in Orchestrator’s DB, I think NLog might be your best route.

We do something similar but for Orchestator > Cloudwatch > Splunk. A simple diagram can be seen here

@codemonkee

NLog - UiPath relies on NLog for Robots and Orchestrator and configured in Web.config in the webroot. This is a good place to centrally configure your logs to go where you want and in the format that you want. NLog is responsible for routing your logging events to where they need to go, whether that’s the SQL Database, Windows Event Logs, Elastic Search, File System, Splunk or many other NLog Targets . More details can be found in the Logging Configuration documentation.

Splunk connection with Uipath

Hi @shweta_B, Thanks for making a new post! There are a few options on how to forward your Orchestrator logs over to Splunk. It would help to have an idea of what your infrastructure looks like or what your desired state is along with what you’ve attempted so far or anything specifics you are having troubles and can provide. As pointed out in the posted linkes by @loginerror you can, of course, use the Splunk Universal Forwarder as a simple solution. Those posts don’t go into any meaningfu…

[Seetharaman_K]

well , thanks for your answer first of all. we are looking for a serial execution because,
when we give two targets , and one being a flat file then there are problems. we are missing some logs. when contacted uipath support they said that "writing to flat file is not a good choice " . currently we have beats which will harvest the logs from the log file and send to ELK. but because beats also creates problem some times and ( we are using only community edition of beats and elk ) fail to harvest the logs, we are missing the vital logs. so below architecure is better

than ,

[codemonkee (Tim)]

In my example I use a flat file written to disk in a JSON format, while I don’t disagree with a flat file being my top choice, it does have its time and place. (I selected it out of time constraints at the time). That said, in the last two years, I haven’t had any issues with missing log messages (at least not at that stage of the pipeline), if you are in that scenario, I would probably dig into why.

Targeting a flat file to disk is by no means your only choice, you could just as easily direct the logs to another database, or directly into an ELK stack or where have you.

The concern that I would have with another system (as I haven’t watched how Orchestrator interacts with the SQL Server is additional load and locking of Orchestrator’s database which could have a performance impact on your Orchestrator/Robot activities.

Are you looking at a replica using SQL Server functionality, or another solution for reading the data directly out of the original database? (e.g. Splunk’s DB Connect) - Did you have a target solution/platform in mind for your reporting piece?

[Seetharaman_K]

db-splunk
db - power bi

this is what we are thinking for. and do u use and time out and buffer writing while target being a flat file ? also concurrent writing set to be true ? am talking about web.config file