Train Pipeline - An error occurred (InvalidAccessKeyId) when calling the ListObjects operation: Unknown

News Knowledge Base
,
1

Train pipeline of any ML model is failing with error An error occurred (InvalidAccessKeyId) when calling the ListObjects operation: Unknown. How to resolve this?

Error:

An error occurred (InvalidAccessKeyId) when calling the ListObjects operation: Unknown. can be seen in pipeline logs of a newly started pipeline

Resolution: Follow steps below,

  1. Log into any AIC server node and execute below commands
    1. export KUBECONFIG=/etc/rancher/rke2/rke2.yaml && export PATH=$PATH:/var/lib/rancher/rke2/bin
    2. kubectl get pods -n uipath
  2. Copy the pod name starting with "ai-trainer-deployment-xxx"
  3. Execute below command to shell into ai-trainer pod. Replace with podname copied from above command
    • kubectl -n uipath exec -it -c ai-trainer-deployment -- bash
  4. Execute below commands to get the accesskey and secretkey used by ai-trainer-deployment
    1. cat /secrets/storage.accesskey
    2. cat /secrets/storage.secretkey
  5. Now exit shell of the pod using below command
    • exit
  6. Execute below command to get the accesskey and storagekey stored in the secret
    • kubectl -n uipath get secrets training-storage-credentials -o jsonpath='{.data.*}' | base64 -d
  7. Compare the accesskey and secretkey in step 4 and step 6. If both are different, proceed to step 8. Else, reach out to UiPath Product Support
  8. Take backup of training-storage-credentials secret using below command
    • kubectl -n uipath get secrets training-storage-credentials -o yaml > training-storage-credentials.yaml
  9. Delete the training-storage-credentials secret using below command
    • kubectl delete secret -n uipath training-storage-credentials
  10. Verify that the secret is deleted by checking if the secret is not present by using below command
    • kubectl get secrets -n uipath
  11. Once verified, login to AIC host tenant using admin credentials
  12. Provision AIC for one of the tenants for which AIC is not provisioned earlier
  13. Now verify that the secret is created again by checking if the secret is present by using below command
    • kubectl get secrets -n uipath
  14. Once verified, cross check if secretkey and accesskey match by performing step 4 and step 6 again
  15. Once the credentials are matching, run pipeline without facing this issue.