Failed to pull image: device or resource busy: unknown
Warning Failed 11s kubelet Failed to pull image “registry.uipath.com/identity-service-datamigrator:23.9.25-release-k8.4”: rpc error: code = Unknown desc = failed to pull and unpack image “registry.uipath.com/identity-service-datamigrator:23.9.25-release-k8.4”: failed to extract layer sha256:cae70eb0d6d3e0b7275a217e49b53c3728328b8ec3c8bae2e20307eddc36244b: failed to unmount /var/lib/rancher/rke2/agent/containerd/tmpmounts/containerd-mount2723411358: failed to unmount target /var/lib/rancher/rke2/agent/containerd/tmpmounts/containerd-mount2723411358: device or resource busy: unknown
Issue Description:
Failed to pull image: device or resource busy: unknown
Warning Failed 11s kubelet Failed to pull image "registry.uipath.com/identity-service-datamigrator:23.9.25-release-k8.4": rpc error: code = Unknown desc = failed to pull and unpack image "registry.uipath.com/identity-service-datamigrator:23.9.25-release-k8.4": failed to extract layer sha256:cae70eb0d6d3e0b7275a217e49b53c3728328b8ec3c8bae2e20307eddc36244b: failed to unmount /var/lib/rancher/rke2/agent/containerd/tmpmounts/containerd-mount2723411358: failed to unmount target /var/lib/rancher/rke2/agent/containerd/tmpmounts/containerd-mount2723411358: device or resource busy: unknown
Root Cause:
This issue can be tricky to debug, but the errors "failed to extract layer" or "failed to unmount target" or "device or resource busy: unknown" would indicate something external to the kubernetes software is interfering with the kubelet processes ability to download or start a container.Once a container is downloaded, the image files are stored under /var/lib/rancher/rke2/agent
Resolving:
- Check anti-virus. In the past in azure, we have seen this caused by mdatp (Microsoft Defender Endpoint). As a solution the directory /var/lib/rancher/rke2/agent
- If the cause cannot be found, on the node where the issue is occurring, please run the following tool and share the logs with UiPath Support: https://raw.githubusercontent.com/UiPath/automation-suite-support-tools/main/Scripts/GeneralTools/rke2logCollector.sh