How to import / export ML packages when script does not work with Automation Suite?
Issue Description:
For exporting/importing packages in AI Center, there is a script available for v21.10 and v22.4 , and post v22.4 the packages can be downloaded from UI. In case this import/export steps do not download correct package or if there are discrepancies, follow the below methods.
Resolution:
Perform the following steps:
-
Install the s3cmd:
# sudo pip3 install s3cmd
-
Run following commands:
# OBJECT_GATEWAY_INTERNAL_IP="rook-ceph-rgw-rook-ceph" # OBJECT_GATEWAY_INTERNAL_HOST=$(kubectl -n rook-ceph get services/$OBJECT_GATEWAY_INTERNAL_IP -o jsonpath="{.spec.clusterIP}") # OBJECT_GATEWAY_INTERNAL_PORT=$(kubectl -n rook-ceph get services/$OBJECT_GATEWAY_INTERNAL_IP -o jsonpath="{.spec.ports[0].port}") # STORAGE_ACCESS_KEY=$(kubectl -n rook-ceph get secret ceph-object-store-secret -o json | jq '.data.OBJECT_STORAGE_ACCESSKEY' | sed -e 's/^"//' -e 's/"$//' | base64 -d) # STORAGE_SECRET_KEY=$(kubectl -n rook-ceph get secret ceph-object-store-secret -o json | jq '.data.OBJECT_STORAGE_SECRETKEY' | sed -e 's/^"//' -e 's/"$//' | base64 -d) # export AWS_HOST=$OBJECT_GATEWAY_INTERNAL_HOST # export AWS_ENDPOINT=$OBJECT_GATEWAY_INTERNAL_HOST:$OBJECT_GATEWAY_INTERNAL_PORT # export AWS_ACCESS_KEY_ID=$STORAGE_ACCESS_KEY # export AWS_SECRET_ACCESS_KEY=$STORAGE_SECRET_KEY
-
Check if buckets can be accessed:
# s3cmd ls --host=${AWS_HOST} --no-check-certificate --no-ssl
-
To check contents of ml-model-files bucket:
# /usr/local/bin/s3cmd ls -r s3://ml-model-files --host=${AWS_HOST} --host-bucket= s3://ml-model-files --no-check-certificate --no-ssl
-
Update the path of the package required from previous step and update below command in the path so that those files are downloaded: