Automation Suite - Import / Export ML packages script do not work with Automation Suite

News Knowledge Base
, , ,
1

How to import / export ML packages when script do not work with Automation Suite?

Issue Description:

For exporting/importing packages in AI Center, there is a script available for v21.10 and v22.4 , and post v22.4 the packages can be downloaded from UI. In case this import/export steps do not download correct package or if there are discrepancies, follow the below methods.

Resolution: Perform the following steps

  1. Install the s3cmd
# sudo pip3 install s3cmd
  1. Run following commands :
# OBJECT_GATEWAY_INTERNAL_IP="rook-ceph-rgw-rook-ceph"

# OBJECT_GATEWAY_INTERNAL_HOST=$(kubectl -n rook-ceph get services/$OBJECT_GATEWAY_INTERNAL_IP -o jsonpath="{.spec.clusterIP}")

# OBJECT_GATEWAY_INTERNAL_PORT=$(kubectl -n rook-ceph get services/$OBJECT_GATEWAY_INTERNAL_IP -o jsonpath="{.spec.ports[0].port}")

# STORAGE_ACCESS_KEY=$(kubectl -n rook-ceph get secret ceph-object-store-secret -o json | jq '.data.OBJECT_STORAGE_ACCESSKEY' | sed -e 's/^"//' -e 's/"$//' | base64 -d)

# STORAGE_SECRET_KEY=$(kubectl -n rook-ceph get secret ceph-object-store-secret -o json | jq '.data.OBJECT_STORAGE_SECRETKEY' | sed -e 's/^"//' -e 's/"$//' | base64 -d)

# export AWS_HOST=$OBJECT_GATEWAY_INTERNAL_HOST

# export AWS_ENDPOINT=$OBJECT_GATEWAY_INTERNAL_HOST:$OBJECT_GATEWAY_INTERNAL_PORT

 # export AWS_ACCESS_KEY_ID=$STORAGE_ACCESS_KEY

# export AWS_SECRET_ACCESS_KEY=$STORAGE_SECRET_KEY

  1. Check if buckets can be accessed
# s3cmd ls --host=${AWS_HOST} --no-check-certificate --no-ssl
  1. To check contents of ml-model-files bucket
# /usr/local/bin/s3cmd ls -r s3://ml-model-files --host=${AWS_HOST} --host-bucket= s3://ml-model-files --no-check-certificate --no-ssl

  1. Update the path of the package required from previous step and update below command in the path so that those files are downloaded.
# /usr/local/bin/s3cmd get s3://ml-model-files/d893ab53-bc45-4070-94aa-ef3153d46a11/d661031a-c4b7-4976-a47f-096747497855/88b49e67-9616-4f5d-936e-09fffada5bcc/f303a9c8-c042-49e4-b05d-186cb9892d08/152eea45-73d6-4bd7-b5f0-2917821bcbc3/invoices_india_package.zip --no-ssl --host=${AWS_HOST} --host-bucket= s3://ml-model-files/d893ab53-bc45-4070-94aa-ef3153d46a11/d661031a-c4b7-4976-a47f-096747497855/88b49e67-9616-4f5d-936e-09fffada5bcc/f303a9c8-c042-49e4-b05d-186cb9892d08/152eea45-73d6-4bd7-b5f0-2917821bcbc3/invoices_india_package.zip