🔐 What cert-manager Does

cert-manager is a Kubernetes controller that automates the entire TLS certificate lifecycle: request → provision → store → renew → rotate. It supports multiple certificate authorities and integrates natively with Ingress, Gateway API, and Istio.

cert-manager controller · webhook · cainjector watches Certificate resources Let's Encrypt ACME (free) Internal CA self-signed / Vault Venafi / AWS ACM enterprise CAs kubernetes.io/tls Secret (auto-rotated) stores cert in Secret

Core CRDs

Issuer

Namespace-scoped CA configuration. Issues certificates only within its namespace.

ClusterIssuer

Cluster-wide CA configuration. Can issue certificates across all namespaces. Use for shared CAs like Let's Encrypt.

Certificate

A request for a TLS certificate. Specifies the issuer, DNS names, duration, and the Secret name where the cert will be stored.

CertificateRequest

Low-level request created by cert-manager internally. Usually managed automatically — you rarely interact with this directly.

Installing cert-manager

# Via Helm (recommended)
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set crds.enabled=true \
  --version v1.14.0

# Verify all components are running
kubectl get pods -n cert-manager
# NAME                                      READY   STATUS
# cert-manager-xxxxx                        1/1     Running  ← controller
# cert-manager-webhook-xxxxx                1/1     Running  ← validates CRDs
# cert-manager-cainjector-xxxxx             1/1     Running  ← injects CA bundles

# Test: self-signed cert to verify installation works
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.0/cmctl-linux-amd64.tar.gz
cmctl check api

📜 The Certificate Resource

A Certificate is your declarative request for a TLS certificate. cert-manager reconciles it continuously — creating the cert, storing it in a Secret, and renewing it automatically before expiry.

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name:      api-example-com
  namespace: production
spec:
  # Where the cert+key will be stored
  secretName: api-example-com-tls

  # Certificate lifetime and renewal window
  duration:    2160h   # 90 days (Let's Encrypt maximum)
  renewBefore: 360h    # renew 15 days before expiry

  # Subject
  commonName: api.example.com
  dnsNames:
  - api.example.com
  - www.api.example.com    # SAN — multiple hostnames on one cert

  # Wildcard cert (requires DNS-01 solver)
  # dnsNames:
  # - "*.example.com"
  # - example.com

  privateKey:
    algorithm:   RSA
    encoding:    PKCS1
    size:        2048
    rotationPolicy: Always   # rotate private key on every renewal

  # Which CA to use
  issuerRef:
    name:  letsencrypt-prod
    kind:  ClusterIssuer
    group: cert-manager.io

# Check certificate status
kubectl describe certificate api-example-com -n production
# Status:
#   Conditions:
#     Message:  Certificate is up to date and has not expired
#     Type:     Ready    Status: True
#   Not After:  2024-04-15T10:30:00Z
#   Not Before: 2024-01-15T10:30:00Z
#   Renewal Time: 2024-03-31T10:30:00Z   ← auto-renewed 15 days before expiry

🔄 Automatic Rotation — Zero-Downtime Cert Renewal

cert-manager renews certificates before expiry (controlled by renewBefore). When a cert is renewed:

  1. cert-manager creates a new CertificateRequest
  2. The ACME challenge is solved (HTTP-01 or DNS-01)
  3. The new cert is written to the Secret — in-place update
  4. Kubernetes Secret watches propagate to mounted volumes within ~1 minute
  5. Applications using projected volumes pick up the new cert automatically
💡 Make your app reload certs from disk If your application caches the TLS cert in memory at startup, cert rotation won't take effect until a restart. Use inotify file watchers or periodic cert reload (e.g. nginx's ssl_certificate with ssl_stapling). Or use cert-manager's rotationPolicy: Always with a rolling restart triggered by a Secret hash in pod annotations.

📋 Ingress Integration — Annotation-Driven

The simplest integration: annotate an Ingress with the issuer and a TLS section. cert-manager automatically creates and manages the Certificate resource:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name:      api-ingress
  namespace: production
  annotations:
    # Tell cert-manager which ClusterIssuer to use
    cert-manager.io/cluster-issuer: letsencrypt-prod
    # For namespace-scoped Issuer:
    # cert-manager.io/issuer: my-issuer
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - api.example.com
    secretName: api-example-com-tls   # cert-manager creates this Secret
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: checkout-api
            port: { number: 8080 }

Troubleshooting certificate issuance

# Check the Certificate resource status
kubectl describe certificate api-example-com -n production

# Check the CertificateRequest (created automatically by cert-manager)
kubectl get certificaterequest -n production
kubectl describe certificaterequest api-example-com-xxxxx -n production

# Check the ACME Challenge (for HTTP-01)
kubectl get challenge -n production
kubectl describe challenge api-example-com-xxxxx -n production
# Status should show: Presented: true, Processing: true → Reason: Waiting for HTTP-01 challenge

# Check cert-manager controller logs for errors
kubectl logs -n cert-manager -l app=cert-manager --tail=50

# Common issues:
# - Challenge 404: Ingress class wrong, or nginx not forwarding /.well-known/ paths
# - DNS-01 timeout: IAM role missing route53:ChangeResourceRecordSets permission
# - Rate limit: too many certs requested — use staging issuer first

🏭 Issuers — Configuring Your CA

1. Let's Encrypt ClusterIssuer (HTTP-01 challenge)

HTTP-01 is the simplest ACME challenge — Let's Encrypt verifies domain ownership by checking a token at http://yourdomain/.well-known/acme-challenge/TOKEN. Works for public-facing services with HTTP access:

# Staging issuer (always test here first — no rate limits)
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    email:  admin@example.com
    privateKeySecretRef:
      name: letsencrypt-staging-key
    solvers:
    - http01:
        ingress:
          class: nginx   # or 'traefik', 'haproxy', etc.

---
# Production issuer (after staging works)
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email:  admin@example.com
    privateKeySecretRef:
      name: letsencrypt-prod-key
    solvers:
    - http01:
        ingress:
          class: nginx
⚠️ Always test with staging first Let's Encrypt production rate limits: 50 certs per registered domain per week. Staging has much higher limits and issues untrusted certs. Test your entire setup (DNS, Ingress, challenge) with staging before switching to production.

2. DNS-01 Challenge — for wildcards and private clusters

DNS-01 proves domain ownership by creating a TXT record in your DNS zone. Required for wildcard certificates (*.example.com) and clusters without public HTTP access (private VPCs, internal services):

# ClusterIssuer using DNS-01 with Route53 (IRSA for credentials)
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-dns
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email:  admin@example.com
    privateKeySecretRef:
      name: letsencrypt-dns-key
    solvers:
    - dns01:
        route53:
          region:            us-east-1
          hostedZoneID:      Z1234567890ABC
          role:              arn:aws:iam::123456789:role/cert-manager-route53
      # Use selector to apply this solver only to *.example.com requests
      selector:
        dnsZones:
        - example.com

3. Internal CA (self-signed or Vault)

# Self-signed root CA for internal services (no public CA needed)
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: internal-ca
spec:
  ca:
    secretName: internal-ca-key-pair   # Secret containing the CA cert+key

# Create the root CA cert first
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name:      internal-ca
  namespace: cert-manager
spec:
  isCA:          true
  commonName:    internal-ca
  secretName:    internal-ca-key-pair
  duration:      87600h   # 10 years for the root CA
  privateKey:
    algorithm:   ECDSA
    size:        256
  issuerRef:
    name:  selfsigned-issuer
    kind:  ClusterIssuer

🧠 Knowledge Check

Q1. You need a wildcard certificate *.example.com. Which ACME challenge type must you use and why?

A) HTTP-01 — cert-manager handles wildcard verification via Ingress
B) DNS-01 — wildcard certs require DNS zone control proof; HTTP-01 only verifies specific hostnames
C) Either — cert-manager automatically chooses the right challenge type for wildcards
D) TLS-ALPN-01 — this is the only challenge that supports wildcards

Q2. What is the difference between Issuer and ClusterIssuer?

A) ClusterIssuer can only issue wildcard certificates; Issuer handles single-domain certs
B) Issuer uses HTTP-01; ClusterIssuer uses DNS-01
C) Issuer is namespace-scoped; ClusterIssuer is cluster-wide and can issue certs for any namespace
D) ClusterIssuer is deprecated — all new deployments should use Issuer

Q3. cert-manager has renewed a certificate and updated the Secret. Your nginx pod is still serving the old certificate. What is the most likely cause?

A) cert-manager has a bug — it updated the Secret but didn\'t notify the Ingress
B> The Secret is in a different namespace from the Ingress — cert-manager cannot cross namespaces
C) nginx cached the cert in memory at startup — it needs a reload or rolling restart to pick up the rotated cert
D) Kubernetes doesn\'t update volume-mounted Secrets automatically

Q4. An HTTP-01 ACME challenge is failing with "404 Not Found" for /.well-known/acme-challenge/TOKEN. What should you check first?

A> The Let\'s Encrypt servers are down — retry after 5 minutes
B> Switch to DNS-01 — HTTP-01 is deprecated for new certificates
C) The Ingress class in the ClusterIssuer doesn\'t match your controller, or the controller is blocking /.well-known/ paths
D> You need to manually create the challenge token file on the web server