Reinstalling Insights in Automation Suite v24-10-X Fails Due to Retained Persistent Volumes (PVs)

Why does reinstalling the Insights component in Automation Suite v24.10.x result in scheduling failures, and how can it be resolved?

Issue Description:

When attempting to reinstall the Insights application in Automation Suite version v24.10.x, the deployment fails due to Persistent Volume Claims (PVCs) being unable to bind to available Persistent Volumes (PVs). This is caused by leftover PVs from a previous installation that were not deleted due to their Retain reclaim policy.

During reinstallation, new PVCs remain in a Pending state and associated pods fail to schedule, as Kubernetes cannot bind the new PVCs to the released (but not deleted) PVs. This results in errors like FailedScheduling and waiting for pod to be scheduled.

Resolution:

To ensure a successful reinstallation of Insights run follow the steps below:

  1. Remove Insights
    • kubectl delete application insights -n argocd --force
  2. Run the following script to clear all existing data and delete any previously created PersistentVolumes (PVs).
    • #!/bin/bash
      

      set -e

      HOSTNAME=$(hostname)

      Define PV names and their corresponding size and directory path

      declare -A pv_config=(
      ["insights-looker-datadir-pv-${HOSTNAME}"]="10Gi:/datadisk/insights/looker-datadir"
      ["insights-lookerdir-pv-${HOSTNAME}"]="500Mi:/datadisk/insights/looker-dir"
      )

      STORAGE_CLASS="insights-looker-sc"

      echo "Cleaning data directories under /datadisk/insights…"

      Step 1: Remove contents of both target directories

      for config in "${pv_config[@]}"; do
      dir_path=$(echo "$config" | cut -d':' -f2)
      if [ -d "$dir_path" ]; then
      echo " - Clearing $dir_path"
      rm -rf "${dir_path:?}/"* "${dir_path:?}/".* 2>/dev/null || true
      else
      echo " - Creating missing directory $dir_path"
      mkdir -p "$dir_path"
      fi
      done

      Step 2: Recreate the PVs

      for pv_name in "${!pv_config[@]}"; do
      IFS=":" read -r size path <<< "${pv_config[$pv_name]}"

      Delete PV if it exists

      kubectl get pv "${pv_name}" &>/dev/null && kubectl delete pv "${pv_name}"

      echo "Creating PV: $pv_name with size $size and path $path"

      tmp_dir=$(mktemp -d pv.XXXXXX)
      yaml_file="${tmp_dir}/${pv_name}.yaml"

      Generate PV manifest

      cat > "$yaml_file" <<EOF
      apiVersion: v1
      kind: PersistentVolume
      metadata:
      name: ${pv_name}
      spec:
      capacity:
      storage: ${size}
      accessModes:
      - ReadWriteOnce
      persistentVolumeReclaimPolicy: Retain
      storageClassName: ${STORAGE_CLASS}
      local:
      path: ${path}
      nodeAffinity:
      required:
      nodeSelectorTerms:
      - matchExpressions:
      - key: kubernetes.io/hostname
      operator: In
      values:
      - ${HOSTNAME}
      EOF

      Apply PV manifest

      kubectl apply -f "$yaml_file"
      done

      echo "All PVs cleaned and recreated successfully. You may now reinstall Insights after recreating the database."

  3. Recreate the AutomationSuite_Insights database
    • #Example SQL Query
      DROP DATABASE AutomationSuite_Insights;
      CREATE DATABASE AutomationSuite_Insights;
  4. Rerun the installation
    • uipathctl manifest apply input.json --versions versions.json --only insights