Loading Certificate Authority ( /etc/gateway-cert/ca.crt) Failed

News Knowledge Base
,
1

Auth-proxy pod failing due to cert error during Fabric Upgrade in Version 2023.4.2.

Issue Description

An issue occurs while upgrading Automation Suite from 2022.10.3 to 2023.4.2. Upgrade setup stuck on creating auth proxy pod via HELM command.

  • auth-oauth2-proxy pod was failing during Fabric Upgrade.

Symptoms:

  1. auth -oauth2-proxy Pod fails with cert error:

  1. Run the helm command manually to check the exact issue /replicate the issue:

  1. This Helm command creates auth-oauth2-proxy pod, simultaneously check the respective pod logs.
  2. The logs will show the following error:

loading certificate authority ( /etc/gateway-cert/ca.crt) failed.

Resolution

  1. This issue can occur due to a mismatch in the ca.crt of secrets of Istio-system and uipath-auth namespace.
  2. This issue can occur if the cert has been copied from a Windows machine to a Linux machine.
  3. To check for any special characters in the certificate, run the below command:

cat -e /etc/pki/ca-trust/source/anchors/ca.crt

  1. Check certificate of secret istio-ingressgateway-certs under istio-system namespace

kubectl get secrets istio-ingressgateway-certs -n istio-system -o json | jq -r '.data."ca.crt"' | base64 -d

  1. Check certificate of secret istio-ingressgateway-certs under uipath-auth namespace and compare both certs value for any mismatch

kubectl get secrets istio-ingressgateway-certs -n uipath-auth -o json | jq -r '.data."ca.crt"' | base64 -d

  1. Find certificate found under uipath-auth namespace secret does not match with istio-system namespace secret
  2. To unblock it follow below mentioned steps:
  1. Generate yaml file of istio-ingressgateway-certs secret under istio-system namespace

kubectl get secret istio-ingressgateway-certs -n istio-system -o yaml > istio-secret-backup.yaml

  1. Generate yaml file of istio-ingressgateway-certs secret under uipath-atuh namespace for backup

kubectl get secret istio-ingressgateway-certs -n uipath-auth -o yaml > uipath-auth-secret-backup.yaml

  1. Delete secret istio-ingressgateway-certs from uipath-auth namespace
  2. Edit istio-secret-backup.yaml and change namespace from istio-system to uipath-auth. Also remove uuid, creationTimestamp, resourceVersion
  3. Create a new secret with the help of edited istio-secret-backup.yaml

kubectl apply -f istio-secret-backup.yaml

  1. It will create a new secret with the correct value of the certificate under uipath-auth namespace
  2. Edit the fabric installer script so this secret won't be regenerated during the fabric upgrade
  3. Fabric Installer script path: /opt/UiPathAutomationSuite/2023.4.2/installer/Fabric_installer/fabric-installer.sh
  4. Edit fabric-installer.sh file and comment out line number1044

#create_auth_secret_from_tls_cert_secret "${auth_namespace}"

  1. Rerun fabrics and service upgrade script to continue the upgrade.