✅ Validating Webhook — Policy Enforcement

A validating webhook can only accept or reject — it cannot modify the object. It runs after all mutating webhooks, so it sees the final mutated form. This is the right place for policy enforcement:

// Validating webhook: reject pods requesting privileged mode
func (v *PodValidator) Handle(ctx context.Context, req admission.Request) admission.Response {
    pod := &corev1.Pod{}
    v.decoder.Decode(req, pod)

    for _, c := range pod.Spec.Containers {
        if c.SecurityContext != nil &&
           c.SecurityContext.Privileged != nil &&
           *c.SecurityContext.Privileged {
            return admission.Denied(fmt.Sprintf(
                "container %q requests privileged mode — policy violation",
                c.Name,
            ))
        }
    }

    for _, c := range pod.Spec.InitContainers {
        if c.SecurityContext != nil &&
           c.SecurityContext.RunAsUser != nil &&
           *c.SecurityContext.RunAsUser == 0 {
            return admission.Denied(fmt.Sprintf(
                "init container %q runs as root — policy violation",
                c.Name,
            ))
        }
    }

    // Emit a warning without blocking (K8s 1.19+)
    resp := admission.Allowed("pod accepted")
    if pod.Spec.SecurityContext == nil {
        resp.Warnings = []string{"no pod-level securityContext set — consider adding it"}
    }
    return resp
}

🔐 TLS — Webhooks Require HTTPS

The API server only calls webhook endpoints over TLS. You must serve your webhook with a valid certificate. The recommended production approach is cert-manager with a self-signed CA:

# cert-manager Certificate for the webhook server
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: sidecar-injector-cert
  namespace: webhook-system
spec:
  secretName: sidecar-injector-tls
  dnsNames:
  - sidecar-injector-svc.webhook-system.svc
  - sidecar-injector-svc.webhook-system.svc.cluster.local
  issuerRef:
    name: selfsigned-issuer
    kind: ClusterIssuer

# The webhook Deployment mounts the TLS secret
volumes:
- name: tls
  secret:
    secretName: sidecar-injector-tls
volumeMounts:
- name: tls
  mountPath: /tmp/k8s-webhook-server/serving-certs
  readOnly: true

⚠️ Failure Policies — The Critical Safety Setting

Key configuration best practices

💉 Writing the Sidecar Injector

The webhook receives an AdmissionReview request, inspects the Pod, and returns a JSON Patch that adds the sidecar container. Here's a complete Go implementation using controller-runtime's webhook handler:

Webhook handler (using controller-runtime)

// internal/webhook/pod_sidecar_injector.go
package webhook

import (
    "context"
    "encoding/json"
    "net/http"

    corev1 "k8s.io/api/core/v1"
    "sigs.k8s.io/controller-runtime/pkg/webhook/admission"
)

type SidecarInjector struct {
    decoder *admission.Decoder
}

// Handle is called for every matching Pod CREATE
func (s *SidecarInjector) Handle(ctx context.Context, req admission.Request) admission.Response {
    pod := &corev1.Pod{}
    if err := s.decoder.Decode(req, pod); err != nil {
        return admission.Errored(http.StatusBadRequest, err)
    }

    // Skip if sidecar already present (idempotency)
    for _, c := range pod.Spec.Containers {
        if c.Name == "fluent-bit-sidecar" {
            return admission.Allowed("sidecar already present")
        }
    }

    // Build the sidecar container spec
    sidecar := corev1.Container{
        Name:  "fluent-bit-sidecar",
        Image: "fluent/fluent-bit:2.2",
        Resources: corev1.ResourceRequirements{
            Requests: corev1.ResourceList{
                corev1.ResourceCPU:    resource.MustParse("50m"),
                corev1.ResourceMemory: resource.MustParse("64Mi"),
            },
            Limits: corev1.ResourceList{
                corev1.ResourceMemory: resource.MustParse("128Mi"),
            },
        },
        VolumeMounts: []corev1.VolumeMount{{
            Name:      "varlog",
            MountPath: "/var/log",
        }},
    }

    // Inject the sidecar and a shared volume
    pod.Spec.Containers = append(pod.Spec.Containers, sidecar)
    pod.Spec.Volumes = append(pod.Spec.Volumes, corev1.Volume{
        Name: "varlog",
        VolumeSource: corev1.VolumeSource{
            EmptyDir: &corev1.EmptyDirVolumeSource{},
        },
    })

    // Annotate so we know injection happened
    if pod.Annotations == nil {
        pod.Annotations = map[string]string{}
    }
    pod.Annotations["sidecar.myorg.example.com/injected"] = "true"

    // Marshal the modified pod and return a JSON Patch
    marshaledPod, err := json.Marshal(pod)
    if err != nil {
        return admission.Errored(http.StatusInternalServerError, err)
    }
    return admission.PatchResponseFromRaw(req.Object.Raw, marshaledPod)
}

Registering with the manager (main.go)

// Register the webhook handler at a path
mgr.GetWebhookServer().Register("/mutate-pods", &webhook.Admission{
    Handler: &SidecarInjector{
        decoder: admission.NewDecoder(mgr.GetScheme()),
    },
})

// Or using Kubebuilder markers in the handler struct:
// +kubebuilder:webhook:path=/mutate-pods,mutating=true,failurePolicy=ignore,
//   sideEffects=None,groups="",resources=pods,verbs=create,versions=v1,
//   name=inject.sidecar.myorg.example.com,admissionReviewVersions=v1

Understanding JSON Patch

The webhook returns a RFC 6902 JSON Patch — a list of operations that describe the diff between the original and modified object. controller-runtime's PatchResponseFromRaw generates this automatically, but it's useful to understand the format:

// What the API server receives back from the webhook:
{
  "apiVersion": "admission.k8s.io/v1",
  "kind": "AdmissionReview",
  "response": {
    "uid": "req-uid-abc123",
    "allowed": true,
    "patchType": "JSONPatch",
    "patch": "W3sib3AiOiJhZGQiLCJwYXRoIjoiL3NwZWMvY29udGFpbmVycy8tIiwidmFsdWUiOnt..."
    // base64-encoded JSON Patch — decoded it looks like:
    // [
    //   {"op":"add","path":"/spec/containers/-","value":{"name":"fluent-bit-sidecar",...}},
    //   {"op":"add","path":"/spec/volumes/-","value":{"name":"varlog","emptyDir":{}}},
    //   {"op":"add","path":"/metadata/annotations/sidecar.myorg.example.com~1injected","value":"true"}
    // ]
  }
}

🧠 Knowledge Check