AI Center Is Not Available After A Restart

News Knowledge Base
,
1

How to handle when AI Center is not available, after a restart of the host machine?

There could be various causes for this but the starting point to diagnose the issue would be to look at the machine and make sure that services are running correctly.

Diagnosing Steps:

  1. Execute the command: systemctl status kubelet
    • It should be in an active state. If it is not, try the following:
      • systemctl start kubelet
      • systemctl status kubelet
      • If the service is still not running go to the next step
  2. If the service is still not running, check the journal logs: journalctl -r -u kubelet. Checking for the following errors
    • failed to run Kubelet: running with swap on is not supported. Please disable swap”
      • Go to the section: Turning Off Swap
    • failed to run Kubelet: failed to create kubelet: misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs"
      • Go to the section: Cgroup Driver Configuration
  3. If the services is running, then click here and run the AI Center diagnostic tool
    • This diagnoses the health of our application
    • Also generate a support bundle
      • This diagnoses the health of the infrastructure components.
    • After this open a ticket with UiPath and send the support artificates.
    • These tools require the kubelet service to be running.
  4. If the service is still not running, open a ticket with UiPath and include the log messages from the journalctl command.

Turning Off Swap

  1. First disable swap using the following commands.
    • sudo swapoff -a
    • service restart kubelet
  2. After this go through the first step in the Diagnosing to make sure the service is now running.
  3. Once the service is running, do the following to ensure swap does not get re-enabled.
    • sudo su root
    • Through “crontab -e” , edit the crontab and add this entry to crontab,
    • @reboot sudo swapoff -a && sudo systemctl restart kubelet

Cgroup Driver Configuration

  1. One of the deamon configuration files needs to be modified.
    • Copy the current confige file to your working directory
    • sudo cp /etc/docker/daemon.json .
    • Open the file for editing.
      • vi daemon.json
    • Add the following
      • "exec-opts": ["native.cgroupdriver=systemd"]
    • Final product should look like:
      • {
        "default-runtime": "nvidia",
        "exec-opts": ["native.cgroupdriver=systemd"],
        "runtimes": {
        "nvidia": {
        "path": "/usr/bin/nvidia-container-runtime",
        "runtimeArgs": []
        }
        }
        }
  2. After the above steps have been done, reload the service
    • systemctl daemon-reload
    • service restart docker
    • swapoff -a
    • service restart kubelet
  3. Return to the diagnosing section to determine if the service is running and what steps to take if AI Center is still not available.

Read more on