DU app not syncing because of cert-trustor failing.
Issue Description:
-du-digitizer pod remain in error state and DU app can't be sync.
Investigation Procedure:
- Post describing the pod, Init Container , cert-trustor fails,
Init Containers:
cert-trustor:
Container ID: containerd://0f30f56dce499d4bc0628c2a8e7c4f44ecd7731fff6f62a2c002a18b3014e8c8
Image: localhost:30071/certificate-trustor-debian:22.9.30-develop.1
Image ID: localhost:30071/certificate-trustor-debian@sha256:2d15d6866a6c09722ade00a024c6744830f9882d0de728d2ec1d8aef416246d6
Port: <none>
Host Port: <none>
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 10
- Logs of the Pod show,
[root@czcholstc003402 ~]# ku logs du-digitizer-6c99cb7869-g7zkv -c cert-trustor
coping /etc/webhook/certs/* to /usr/local/share/ca-certificates/
cp: -r not specified; omitting directory '/etc/webhook/certs/additionalca.crt'
updating certificate trust store
update-ca-certificates
Updating certificates in /etc/ssl/certs...
rehash: warning: skipping ca.pem,it does not contain exactly one certificate or CRL
rehash: warning: skipping duplicate certificate in cert-1.pem
rehash: warning: skipping duplicate certificate in cert-3.pem
rehash: warning: skipping duplicate certificate in cert-4.pem
rehash: warning: skipping duplicate certificate in cert-5.pem
rehash: warning: skipping duplicate certificate in cert-7.pem
rehash: warning: skipping tls.pem,it does not contain exactly one certificate or CRL
rehash: warning: skipping duplicate certificate in cert-10.pem
rehash: warning: skipping duplicate certificate in cert-11.pem
rehash: warning: skipping duplicate certificate in cert-12.pem
rehash: warning: skipping duplicate certificate in cert-13.pem
rehash: warning: skipping duplicate certificate in cert-14.pem
- In cert.pem file that is tls certificate, it is found that there is a block that starts with -----BEGIN RSA PRIVATE KEY----- . ideally tls certificate should not contain any private key. This can be verified by running,
kubectl get secret istio-ingressgateway-certs -n istio-system -o json | jq '.data."tls.crt"' | sed 's/"//g' | base64 -d
Root Cause: In cert.pem file that is TLS certificate, we found that there is a block that starts with -----BEGIN RSA PRIVATE KEY----- . Ideally TLS certificate should not contain any private key.
Resolution:
- Remove private key and update certificate again using contigureUiPathAs script .