Certificate is expiring
Symptom
A CustomerCertificateExpiring
alert is firing.
CustomerCertificateExpiring alerts are are sent to subscribers via email. |
Observations
Here are the details on the alerts:
Alert: CustomerCertificateExpiring
Expression |
|
Delay |
|
Labels |
|
Annotations |
|
Determine the ingress(es) using the certificate
The alert notes the host certificate that is expiring (or has expired) and the cluster where the ingress is running. Find the ingress using the certificate on the cluster noted in the alert.
Here’s how to find the ingress using kubectl
:
kubectl config use-context <cluster in alert>
kubectl get ingress -A | grep "<host in alert>"
Results
Note the namespace of the ingress in the first column and the ingress name in the second column.
$ kubectl get ingress -A | grep "www.example.com"
prod bl-example <none> www.example.com a3052a43267a91491a437.......a6503628baf.elb.eu-central-1.amazonaws.com 80, 443 255d
prod cm-acme-example nginx www.example.com a3052a43267a9491491a43.......58a6503628baf.elb.eu-central-1.amazonaws.com 80 8d
There are two ingresses using the certificate.
-
prod/bl-example
is the nginx ingress actually using the certificate. This ingress actually uses the expiring certificate. -
prod/cm-acme-example
- is a temporary ingress used for an HTTP acme challenge (more on this later).
Determine certificate issuer used by the ingress
The ingress you found in the previous observation will show the certificate issuer in an annotation of the ingress.
You can display the annotations of the ingress as follows:
kubectl -n <namespace of the ingress> describe ingress <ingress>
or:
kubectl -n <namespace of the ingress> get ingress <ingress> -o yaml
cert-manager.io/cluster-issuer: letsencrypt-prod (1)
1 | The certificate issuer - cert-manager.io/cluster-issuer . |
Determine the renewal method of the cluster issuer
The "solver" configuration of the cluster issuer determines how the certificate is renewed.
You can display the solver configuration with:
kubectl -n cert-manager describe clusterissuer <clusterissuer>
or
kubectl -n cert-manager get clusterissuer <cluster issuer> -o yaml
In either case, look for the solver section in the spec:
solvers:
- http01:
or:
solvers:
- dns01:
Solutions
This section provides solutions that should help resolve the issue in most cases.
Renew certificate (dns01)
You can renew certificates directly from the Cockpit.
dns01
certificates can also be renewed with a DNS acme challenge. To renew the certificate using DNS, a TXT record for the domain _acme-challenge.<YOUR_DOMAIN>
must be added to the DNS record.