🎯 The Problem with CPU-Only Autoscaling

The built-in HPA scales on CPU and memory. This works well for HTTP services where load directly drives CPU, but fails for event-driven workloads:

  • A Kafka consumer with 10,000 messages in the queue and 0% CPU (waiting for I/O)
  • An SQS worker that's idle because the queue is empty — but you need to scale to zero
  • A batch processor that should run at 3 AM and stop at 5 AM on a schedule
  • A webhook processor that needs to scale instantly when request rate spikes

KEDA (Kubernetes Event-Driven Autoscaling, CNCF graduated) extends Kubernetes with 60+ scalers that watch external event sources and drive HPA or scale to zero directly.

Kafka Topic AWS SQS Prometheus KEDA Operator ScaledObject watcher drives HPA metric scale-to-zero logic HPA adjusts replicas Deployment 0 → N replicas

Installing KEDA

# Via Helm (recommended)
helm repo add kedacore https://kedacore.github.io/charts
helm repo update
helm install keda kedacore/keda \
  --namespace keda --create-namespace \
  --version 2.13.0

# Verify KEDA is running
kubectl get pods -n keda
# NAME                                      READY   STATUS
# keda-operator-xxxxx                       1/1     Running
# keda-operator-metrics-apiserver-xxxxx    1/1     Running
# keda-admission-webhooks-xxxxx            1/1     Running

# KEDA installs two CRDs you'll use:
kubectl get crd | grep keda
# scaledobjects.keda.sh           ← scales Deployments/StatefulSets
# scaledjobs.keda.sh              ← scales Kubernetes Jobs
# triggerauthentications.keda.sh  ← stores credentials for scalers
# clustertriggerauthentications.keda.sh

How KEDA works with HPA

KEDA doesn't replace the HPA — it drives it. When you create a ScaledObject, KEDA automatically creates an HPA that uses KEDA's external metrics API as its data source. You manage the ScaledObject; KEDA manages the HPA lifecycle.

⏰ Cron Scaler — Predictive Scheduling

Scale workloads on a time schedule — useful for predictable load patterns like business hours traffic or nightly batch jobs. Multiple cron rules can be stacked for complex schedules:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: api-business-hours-scaler
spec:
  scaleTargetRef:
    name: checkout-api
  minReplicaCount: 2
  maxReplicaCount: 20
  triggers:
  # Scale up to 10 at 8 AM UTC (business hours start)
  - type: cron
    metadata:
      timezone:        Europe/London
      start:           0 8 * * 1-5      # Mon-Fri 8:00
      end:             0 18 * * 1-5     # Mon-Fri 18:00
      desiredReplicas: "10"
  # Additional Kafka trigger — cron sets a floor, Kafka can scale higher
  - type: kafka
    metadata:
      bootstrapServers: kafka.prod:9092
      consumerGroup:   checkout-api
      topic:            checkout-events
      lagThreshold:    "20"
🔵 Multiple triggers = OR logic, take the MAX When a ScaledObject has multiple triggers, KEDA evaluates each independently and uses the maximum desired replica count. In the example above: if the cron trigger says 10 replicas and the Kafka lag trigger says 15, KEDA targets 15. This lets you combine predictive (cron) and reactive (event) scaling.

🌐 HTTP Scaler — Scale on Request Rate

The keda-add-ons-http project adds an HTTP scaler that intercepts traffic and scales based on in-flight requests — including scale-to-zero for HTTP services:

# Install HTTP add-on
helm install http-add-on kedacore/keda-add-ons-http \
  --namespace keda

apiVersion: http.keda.sh/v1alpha1
kind: HTTPScaledObject
metadata:
  name: checkout-api-http
spec:
  hosts:
  - checkout-api.example.com
  pathPrefixes:
  - /api
  scaledownPeriod: 300
  replicas:
    min: 0
    max: 30
  targetPendingRequests: 100   # 1 replica per 100 concurrent requests
  scaleTargetRef:
    apiVersion: apps/v1
    kind:       Deployment
    name:       checkout-api
    service:    checkout-api
    port:       8080

📊 Prometheus Scaler — Custom Metrics

The most flexible scaler — scale on any Prometheus query result. Ideal for custom business metrics like order queue depth, active user sessions, or processing latency:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: report-generator-scaler
spec:
  scaleTargetRef:
    name: report-generator
  minReplicaCount: 0
  maxReplicaCount: 10
  triggers:
  - type: prometheus
    metadata:
      serverAddress:  http://prometheus.monitoring:9090
      metricName:     pending_reports_total
      threshold:      "5"   # 1 replica per 5 pending reports
      query: sum(pending_reports_total{namespace="production"})
      ignoreNullValues: "false"

🔧 ScaledJob — Scale Kubernetes Jobs

ScaledJob creates a new Kubernetes Job for each batch of events — perfect for one-shot processing where you want isolation and guaranteed completion per item:

apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
  name: video-transcoder
spec:
  jobTargetRef:
    parallelism:    1
    completions:    1
    backoffLimit:   3
    template:
      spec:
        containers:
        - name: transcoder
          image: my-transcoder:v1
          env:
          - name: SQS_QUEUE_URL
            value: https://sqs.us-east-1.amazonaws.com/123/video-jobs
        restartPolicy: Never
  maxReplicaCount: 50   # max 50 concurrent jobs
  scalingStrategy:
    strategy: accurate   # or "default" (faster but may create extra jobs)
  triggers:
  - type: aws-sqs-queue
    authenticationRef:
      name: sqs-trigger-auth
    metadata:
      queueURL:    https://sqs.us-east-1.amazonaws.com/123/video-jobs
      queueLength: "1"   # 1 job per message
      awsRegion:   us-east-1

Debugging KEDA scaling decisions

# Check ScaledObject status and current metric value
kubectl describe scaledobject payment-worker-scaler -n production

# Key fields in status:
# externalMetricNames: [s0-kafka-payment-events]
# hpaName: keda-hpa-payment-worker-scaler
# lastActiveTime: 2024-01-15T10:30:00Z
# conditions:
#   - type: Active    status: "True"   → triggers are firing
#   - type: Ready     status: "True"   → KEDA is healthy

# See the HPA KEDA created
kubectl get hpa -n production
kubectl describe hpa keda-hpa-payment-worker-scaler -n production

# Watch scaling in real time
kubectl get pods -n production -w -l app=payment-worker

📋 ScaledObject — The Core Resource

A ScaledObject links a target workload (Deployment, StatefulSet, etc.) to one or more external triggers. KEDA polls each trigger and computes the desired replica count.

# ScaledObject anatomy — all key fields explained
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name:      payment-worker-scaler
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind:       Deployment
    name:       payment-worker

  # Scaling bounds
  minReplicaCount: 0    # 0 = scale to zero when no events
  maxReplicaCount: 50   # hard cap

  # How fast to scale up/down
  advanced:
    restoreToOriginalReplicaCount: false
    horizontalPodAutoscalerConfig:
      behavior:
        scaleUp:
          stabilizationWindowSeconds: 0     # scale up immediately
          policies:
          - type: Percent
            value: 100
            periodSeconds: 15   # double replicas every 15s
        scaleDown:
          stabilizationWindowSeconds: 300   # wait 5m before scaling down

  # One or more event source triggers
  triggers:
  - type: kafka
    metadata:
      bootstrapServers: kafka.kafka-system:9092
      consumerGroup:   payment-workers
      topic:            payment-events
      lagThreshold:    "100"   # 1 replica per 100 messages lag
      offsetResetPolicy: latest

☕ Kafka Scaler — Scale on Consumer Lag

The Kafka scaler is the most common production use case: scale consumers based on how far behind they are. Target replicas = ceil(lag / lagThreshold).

# TriggerAuthentication for SASL/TLS Kafka (production)
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: kafka-trigger-auth
  namespace: production
spec:
  secretTargetRef:
  - parameter: sasl
    name: kafka-credentials
    key:  sasl
  - parameter: username
    name: kafka-credentials
    key:  username
  - parameter: password
    name: kafka-credentials
    key:  password
  - parameter: tls
    name: kafka-credentials
    key:  tls

---
# Full ScaledObject with auth + multiple topics
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-processor-scaler
spec:
  scaleTargetRef:
    name: order-processor
  minReplicaCount: 1    # keep 1 warm replica for low latency
  maxReplicaCount: 100
  triggers:
  - type: kafka
    authenticationRef:
      name: kafka-trigger-auth
    metadata:
      bootstrapServers: kafka.prod:9092
      consumerGroup:   order-processors
      topic:            orders-v2
      lagThreshold:    "50"
      saslType:         plaintext
      tls:              enable
💡 lagThreshold math If lag = 500 messages and lagThreshold = "50", KEDA targets 10 replicas. If lag drops to 0, KEDA scales to minReplicaCount (or 0 if set). The scaleDown stabilizationWindowSeconds prevents thrashing when lag briefly drops — set to 2–5 minutes for batch workloads.

☁️ AWS SQS Scaler — Scale on Queue Depth

# IAM: KEDA needs sqs:GetQueueAttributes permission
# Use IRSA (IAM Roles for Service Accounts) for credentials
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: sqs-trigger-auth
spec:
  podIdentity:
    provider: aws   # uses IRSA — no credentials in Secret

---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: email-worker-scaler
spec:
  scaleTargetRef:
    name: email-worker
  minReplicaCount: 0     # true scale-to-zero when queue empty
  maxReplicaCount: 20
  pollingInterval: 10    # check queue every 10s
  cooldownPeriod:  300   # 5m after last message before scaling to 0
  triggers:
  - type: aws-sqs-queue
    authenticationRef:
      name: sqs-trigger-auth
    metadata:
      queueURL:          https://sqs.us-east-1.amazonaws.com/123456789/email-queue
      queueLength:       "10"   # 1 replica per 10 messages
      awsRegion:         us-east-1
      identityOwner:     pod

🧠 Knowledge Check

Q1. A Kafka consumer Deployment has 0% CPU but 50,000 messages of lag. The built-in HPA won't scale it. Why does KEDA solve this?

A) KEDA bypasses the Kubernetes scheduler to place pods faster
B) KEDA uses a faster HPA polling interval than the default 15s
C) KEDA reads consumer lag directly from Kafka — scaling is based on queue depth, not CPU
D) KEDA replaces the Kafka broker with a Kubernetes-native queue

Q2. A ScaledObject has two triggers: a cron trigger targeting 10 replicas and a Kafka lag trigger currently targeting 25 replicas. How many replicas does KEDA create?

A) 10 — the cron trigger takes priority as it is defined first
B) 35 — KEDA sums all trigger outputs
C) 25 — KEDA uses the maximum across all triggers
D) 17 — KEDA averages all trigger outputs

Q3. When should you use ScaledJob instead of ScaledObject?

A) When you want faster scaling response times than ScaledObject provides
B) When each event needs isolated processing in a separate Job with completion guarantees
C) ScaledJob is only for CronJob workloads, not queue-based processing
D) ScaledJob is required when minReplicaCount is 0

Q4. A ScaledObject with minReplicaCount: 0 has been scaled to zero. A new SQS message arrives. What happens?

A) The message is lost — KEDA cannot scale from zero once reached
B) Kubernetes automatically restarts the last pod that processed a message
C) KEDA polls the queue, detects the message, scales from 0 → 1 — with cold-start latency of pollingInterval + pod startup
D) SQS pushes a webhook to KEDA, which instantly creates a pod