Log message doesn't populated in orchestrator log

Help Orchestrator
,
1

Hello Guys

I am having a licensed orchestrator and the logs are not being updated in the orchestrator from today’s morning. Anyone, please guide me on how to solve this

I have tried

  • Searching for this path C:\Windows\SysWOW64\config\systemprofile\AppData\Local\UiPath\Logs\execution_log_data but the VM doesn’t have *\UiPath\Logs\exeution_log_data folder with it.

  • Checked on Web.config and UiPath.Orchestrator.dll.config Uri under RobotElasticSearch is already empty.

Also, I am having a few other processes in another VM too so if I restart the IIS server it will not affect that VM right?

Kindly help me with this

@Yoichi @postwick @Palaniyappan @kirankumar.mahanthi1 @lakshman @pravin_calvin @jeevith

Regards
Sudharsan

1 Like
2

Hi @Sudharsan_Ka ,

Before dong all the research check if you are hard disk having some space for the C drive. some times if we don’t have space in the C drive it will create these kind of issues. if the disk is fully occupied delete temporary and un wanted files and clear sufficient space in the C drive and try executing your process in orchestrator and see if you are getting logs.

And also make sure that you are using logging type as info so that you will get all the informative logs in orchestrator.

it is just a suggestion or thought from my side. please follow others forum members suggestions too. thanks.

3 Likes
3

Sure @kirankumar.mahanthi1

Will check that

Regards
Sudharsan

4

Check the size of your log table in the Orchestrator database. We remove old records and shrink every week.

5

HI @postwick

If possible can you show me the path or screenshot for where I can check the size of the table and how I can remove old records

Regards
Sudharsan

1 Like
6

HI @kirankumar.mahanthi1

C drive is full in my VM and i had made available 5 GB and tried again but still I can’t able to get the logs in the orchestrator

Is there any space requirement is there like to populate logs c drive should have N Gb space like that

Regards
Sudharsan

7

Hi @Sudharsan_Ka ,

Earlier we faced the same issue and we have resolved by freeing some space in that Bot VM c drive. Not sure on the space constraint. Could you free some more space and try and see for other alternative options too. thanks.

8

Sure @kirankumar.mahanthi1

Regards
Sudharsan

9

The logs you see in Orchestrator for Jobs are stored in the database. Check the size of your log table in the database.

10

I would like to provide some context on how UiPath does logging.

UiPath uses the NLog framework for Robot and Orchestrator. The configuration can be found in the NLog.config file of the installation directory.

  • Robot - If installed to the machine would be %programfiles(x86)%\UiPath\Studio\NLog.config
  • Orchestrator - %programfiles(x86)%\UiPath\Orchestrator\UiPath.Orchestrator.dll.config

Robot

The default targets for the Robot NLog are a File and EventLog

  • File will direct the execution logs to

    • ${WorkflowLoggingDirectory}/${shortdate}_Execution.log
    • ${WorkflowLoggingDirectory} = ${specialfolder:folder=LocalApplicationData}/UiPath/Logs
    • ${specialfolder:folder=LocalApplicationData} = %LocalAppData%\UiPath\Logs

  • EventLog will direct certain log events to the Windows Logs > Application under the scope of UiPath which can be found in the Windows Event Viewer. In addition you’ll also find the UiRobot Service logs in here under the Scope of UiRobotSvc

More information on Robot Logging can be found in the following documentation

I’ll highlight the first section of the document as it is important to how the logs flow from the Robot to Orchestrator

If Orchestrator is unavailable, logs are stored in a local database ( C:\Windows\SysWOW64\config\systemprofile\AppData\Local\UiPath\Logs\execution_log_data ), within the available disk space, until the connection is restored. When the connection is restored, the logs are sent in batches in the order they had been generated.

Note:
The database is not deleted after the logs have been successfully sent to Orchestrator.

The status of a job is stored in the memory of the UiPath Robot service. When Orchestrator becomes available, the information regarding the job status is synced between the two. However, if Orchestrator is not available and you restart the UiPath Robot service, the information is lost. This means that whenever Orchestrator becomes available the job is executed again.

Feedback

  • If you are not finding the local execution_log_data database on the robot C:\Windows\SysWOW64\config\systemprofile\AppData\Local\UiPath\Logs\execution_log_data this would indicate to me that Robot Service has not lost communication with Orchestrator and data is flowing as expected.

Orchestrator

NLog for Orchestrator has a handful of default Rules and Targets for directing the logs depending on what you have originally setup. The Logs could flow into Insights / Elastic Search, Orchestrator SQL Server Database, Event Log, or any other custom targets you might have created.

More information about Logging in Orchestrator can be found in the following document

Feedback

By default the only logs that are written locally are the ones sent to the Windows Event Logs, other targets are directing logs to other services such as Database / SQL Server and Elastic Search / Insights

Database / SQL Server

The logs that you view in the Web UI of Orchestrator is coming from the SQL Server UiPath database. Where the database stores its files will be dependent to how you have configured SQL Server. But similarly if your disk space where the underlining files are stored is filled up you will no longer be able to write to the database. Same goes for if your logical db or temp volumes fill up and don’t expand even if there is disk space will prevent you from writing to the DB.

Feedback

Make sure that you are doing general maintenance on your Database Tables, and SQL Server in general, otherwise you could quickly find that your allocated space is filling up due to data records or temp logs running out of space due to hard constraints or allowed to scale out of control maxing out your Disk Space.

Check your Database Properties for Database files, and what their allocated size is configured for along with their Autogrowth and Maxsize

image

You don’t say which C Drive is full, whether it is on your Robot host, Orchestrator host, or SQL Server depending on that you might need to take additional steps (Restart Services if it isn’t recognizing the additional freed up space, maintenance on your database if temp log if filling up faster that the db can be updated, etc.)

While using SSMS, Right Click on the UiPath Database and open the Properties, then click on Files will present you with the screenshot I provided above.

You can use something like the following to get more in-depth details

sp_helpdb UiPath;
DBCC SQLPERF(LOGSPACE);
exec sp_spaceused;
SELECT
 SUBSTRING(a.FILENAME, 1, 1) Drive,
 [FILE_SIZE_MB] = convert(decimal(12,2),
round(a.size/128.000,2)),
 [SPACE_USED_MB] = convert(decimal(12,2),
round(fileproperty(a.name,'SpaceUsed')/128.000,2)),
 [FREE_SPACE_MB] = convert(decimal(12,2),
round((a.size-fileproperty(a.name,'SpaceUsed'))/128.000,2)) ,
 [FREE_SPACE_%] = convert(decimal(12,2),
(convert(decimal(12,2),round((a.size-fileproperty(a.name,'SpaceUsed'))/128.000,2)) 
/ convert(decimal(12,2),round(a.size/128.000,2)) * 100)),
 a.NAME, a.FILENAME
FROM dbo.sysfiles a
ORDER BY Drive, [Name];

In addition to this I also use similar Queries on the various tables, this is just a small sample to see how each table is growing will will break some down down further such as the Queue Items into their respective state as a union.

SELECT 
(SELECT COUNT(*) FROM dbo.Logs) AS Logs,
(SELECT COUNT(*) FROM dbo.QueueItems) AS QueueItems,
(SELECT COUNT(*) FROM dbo.UserNotifications) AS UserNotification,
(SELECT COUNT(*) FROM dbo.TenantNotifications) AS TenantNotification,
(SELECT COUNT(*) FROM dbo.AuditLogs) AS AuditLogs;

Following the recommendations from the Maintenance document referenced above, I have also implemented a daily job as we generate a lot of data in order to trim specific tables to avoid database growth and keep the indexes in a good state (for performance more than disk usage though I’ve never had any issues going significantly beyond the recommendation record count).

The below screenshot is not a recommendation, but just to show what we do for one of our non-prod databases, we flow our logs into other platforms such as Splunk so beyond limited information for for the Orchestrator Ui benefits, we don’t have a reason to keep the data in Orchestrator longer than 30 days, but we term it further just because of volume. Our Production environment is more aggressive, again because of volume.

image

I would recommend doing a quick search of the forums, most likely you’ll find other posted topics that cover similar challenges that could append to the solutions / feedback you are receiving above such as

4 Likes
11

Thanks @codemonkee for such great post. Very informative.

12

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.