How to Increase Redis Memory Size from ArgoCD

How to increase Redis memory size from ArgoCD?

When checking the Redis logs or the logs of the services using Redis, they might show this error:

'10:59:21.1689|WARN|UiPath.DocumentUnderstanding.Common.Clients.Redis.Executor.RedisExecutor|Redis error message: OOM command not allowed when used memory > 'maxmemory''

This happens because the Redis DB is running out of memory and we need to increase the memory size.

  1. ****These steps will bring down Redis which will lead to some downtime for applications using it. Orchestrator will be affected****

  2. ****Re running the full install can undo these settings.****

  3. Login to Argocd

  4. Go to the redis-cluster application:

  5. In the redis cluster, select 'Details':

  6. Select the 'Parameters' column and then select 'Edit':

  7. Find the setting called "database.memorySize" and set the value to 3GB.

  8. Click the save button (it's where the 'edit' button was)

  9. After the change the parameters section should look like the below:

  10. After this is done, run the step 2 commands in this KB

  11. The change will take about 5 minutes to take effect. To verify its complete run the following commands:

kubectl -n redis-system get rec

kubectl -n redis-system get redb

kubectl -n redis-system get pods

The output should look like the following:

[root@autosuitea ~]# kubectl -n redis-system get rec

NAME NODES VERSION STATE SPEC STATUS LICENSE STATE SHARDS LIMIT LICENSE EXPIRATION DATE AGE

redis-cluster 1 7.4.2-129.focal Running Valid Valid 1 2032-01-30T00:00:00Z 22m

[root@autosuitea ~]# kubectl -n redis-system get redb

NAME VERSION PORT CLUSTER SHARDS STATUS SPEC STATUS AGE

redis-cluster-db 7.2.3 12000 redis-cluster 1 active Valid 20m

[root@autosuitea ~]# kubectl -n redis-system get pods

NAME READY STATUS RESTARTS AGE

redis-cluster-0 2/2 Running 0 22m

redis-cluster-db-job-sd5bp 0/1 Completed 0 21m

redis-cluster-recovery-job-29362615-rwf7q 0/1 Completed 0 11m

redis-cluster-recovery-job-29362620-2k5mc 0/1 Completed 0 6m56s

redis-cluster-recovery-job-29362625-s8sp6 1/1 Running 0 116s

redis-cluster-services-rigger-5564f664fb-qqxmp 1/1 Running 0 22m

redis-enterprise-operator-7cbc9c88c9-d285q 2/2 Running 0 14h

Note that the redis-cluster-db-job completed successfully. This is important because it's what recreates the DB. Also make sure that the REC is in the state running and the REDB shows as valid.

Finally, if the issue persists, do the following:

  1. Generate a redis support bundle:

    1. This should give us a very good overview of what is using the memory. Most likely it's just Keys, but it's possible the redis proxy or some other service is using too much memory.

  2. If the above command fails, try running:

    1. kubectl -n redis-system exec -it redis-cluster-0 -- rladmin status extra all

    2. This will at least show the memory allocation if generating a support bundle fails.

  3. Then share all these with the UiPath Support for further investigation.