Not every workload runs forever. Batch processing, migrations, reports, backups — these run to completion and then stop. Jobs and CronJobs are Kubernetes' answer to run-to-completion and scheduled workloads.
1. Jobs — Run to Completion
A Job creates one or more Pods and ensures they run successfully to completion (exit code 0). Unlike Deployments (which restart Pods forever), Jobs have a finite goal.
apiVersion: batch/v1
kind: Job
metadata:
name: db-migration
spec:
completions: 1 # How many successful completions needed
parallelism: 1 # How many Pods run simultaneously
backoffLimit: 3 # Max retries before marking failed
activeDeadlineSeconds: 300 # Max total runtime (kills Pod if exceeded)
ttlSecondsAfterFinished: 600 # Auto-delete Job 10min after completion
template:
spec:
restartPolicy: Never # ← REQUIRED: Never or OnFailure
containers:
- name: migrate
image: myapp:latest
command: ["./migrate", "--up"]
Key Fields
| Field | Purpose | Default |
|---|---|---|
completions | Total successful Pod completions needed | 1 |
parallelism | Max Pods running at the same time | 1 |
backoffLimit | Max retries before Job is marked Failed | 6 |
activeDeadlineSeconds | Hard time limit for entire Job (kills running Pods) | None |
ttlSecondsAfterFinished | Auto-cleanup completed/failed Jobs | None (kept forever) |
restartPolicy Requirement
| Policy | On Failure | Effect |
|---|---|---|
Never | Pod stays Failed, Job creates a new Pod | You can see logs of all attempts |
OnFailure | Container is restarted in same Pod | Cleaner (fewer Pods), but loses logs of previous attempts |
Always | ❌ Invalid for Jobs | Would run forever — contradicts run-to-completion |
Never when you need to inspect logs from failed attempts (each retry is a new Pod with preserved logs). Use OnFailure when failures are transient and you don't need per-attempt debugging — it keeps the namespace cleaner.
2. Parallelism Patterns
The combination of completions and parallelism creates different execution patterns:
Pattern 1: Single Completion (default)
completions: 1 parallelism: 1 # One Pod runs, must succeed once. Simplest case. # Use: migrations, one-off scripts, backups
Pattern 2: Fixed Completion Count
completions: 10 parallelism: 3 # 10 items to process, 3 at a time # Job creates Pods until 10 have succeeded # Use: processing a known list of items (encode 10 videos)
Pattern 3: Work Queue (Indexed Jobs)
completions: 10 parallelism: 3 completionMode: Indexed # Each Pod gets JOB_COMPLETION_INDEX env var # Pod 0 gets index=0, Pod 1 gets index=1, etc. # Each Pod processes its assigned chunk (e.g., rows 0-999, 1000-1999, ...) # Use: parallel data processing with partitioned work
Pattern 4: Work Queue (External Queue)
completions: null # ← unset! (no fixed count) parallelism: 5 # Pods pull work from an external queue (SQS, RabbitMQ, Redis) # Job completes when a Pod exits successfully with no more work # Use: variable-length work queues
JOB_COMPLETION_INDEX environment variable. This eliminates the need for an external queue for embarrassingly parallel workloads — each Pod knows exactly which chunk to process.
3. Failure Handling
backoffLimit
Each time a Pod fails (exits non-zero with restartPolicy: Never, or container restart count exceeds limit with OnFailure), it counts toward the backoff limit.
# backoffLimit: 3 means: # Attempt 1: fails → retry # Attempt 2: fails → retry # Attempt 3: fails → retry # Attempt 4: fails → Job marked Failed (exceeded limit) # Backoff timing between retries: 10s, 20s, 40s... (exponential, max 6min)
activeDeadlineSeconds
A hard wall-clock time limit. Once exceeded, all running Pods are terminated and the Job is marked Failed — regardless of how many completions have succeeded.
spec: activeDeadlineSeconds: 600 # Kill everything after 10 minutes # Use for: jobs that must finish within SLA, prevent runaway Pods
activeDeadlineSeconds on production Jobs. A bug causing a Pod to hang indefinitely will consume resources forever if there's no deadline. Common pattern: set it to 2-3x the expected runtime.
Pod Failure Policy (K8s 1.26+)
Fine-grained control over which failures count toward backoffLimit:
spec:
podFailurePolicy:
rules:
- action: Ignore # Don't count this failure
onPodConditions:
- type: DisruptionTarget # Node preemption — not our fault
- action: FailJob # Immediately fail the entire Job
onExitCodes:
containerName: migrate
operator: In
values: [42] # Exit code 42 = unrecoverable error
- action: Count # Default: count toward backoffLimit
onExitCodes:
operator: NotIn
values: [0]
FailJob lets you fast-fail on known-unrecoverable errors.
4. CronJobs — Scheduled Jobs
A CronJob creates Jobs on a time-based schedule (cron syntax). Think of it as crontab for Kubernetes.
apiVersion: batch/v1
kind: CronJob
metadata:
name: nightly-backup
spec:
schedule: "0 2 * * *" # 2:00 AM daily
timeZone: "America/New_York" # (K8s 1.27+) explicit timezone
concurrencyPolicy: Forbid # Don't overlap runs
startingDeadlineSeconds: 200 # Skip if missed by >200s
successfulJobsHistoryLimit: 3 # Keep last 3 successful Jobs
failedJobsHistoryLimit: 1 # Keep last 1 failed Job
suspend: false # Set true to pause scheduling
jobTemplate:
spec:
activeDeadlineSeconds: 3600
backoffLimit: 2
template:
spec:
restartPolicy: OnFailure
containers:
- name: backup
image: backup-tool:latest
command: ["./backup.sh"]
Cron Schedule Syntax
┌───────────── minute (0-59) │ ┌───────────── hour (0-23) │ │ ┌───────────── day of month (1-31) │ │ │ ┌───────────── month (1-12) │ │ │ │ ┌───────────── day of week (0-6, Sun=0) │ │ │ │ │ * * * * * Examples: "*/5 * * * *" Every 5 minutes "0 * * * *" Every hour "0 2 * * *" Daily at 2AM "0 2 * * 1" Every Monday at 2AM "0 0 1 * *" First day of month at midnight
concurrencyPolicy
| Policy | Behavior | Use Case |
|---|---|---|
Allow (default) | Multiple Jobs can run concurrently | Independent tasks (each run processes different data) |
Forbid | Skip new Job if previous is still running | Tasks that must not overlap (backup, migration) |
Replace | Kill the running Job, start a new one | Latest data matters more than completing old run |
Forbid. If a backup takes longer than expected and overlaps the next scheduled run, you don't want two backup processes competing for locks. The missed run is skipped, and you get alerted via monitoring.
startingDeadlineSeconds
If the CronJob controller misses a scheduled time (e.g., controller was down), it counts missed schedules. If the miss exceeds startingDeadlineSeconds, it skips that run entirely.
# If startingDeadlineSeconds: 200 # Schedule: every hour at :00 # Controller was down from 2:00 to 2:05 # At 2:05, it sees it missed the 2:00 run (5min ago < 200s deadline) # → Creates the Job (late but within deadline) # If controller was down from 2:00 to 3:00 # At 3:00, the 2:00 run was missed by 3600s > 200s # → Skips the 2:00 run, only creates the 3:00 run
5. Cleanup & History
TTL After Finished (Jobs)
spec: ttlSecondsAfterFinished: 3600 # Delete Job + Pods 1 hour after completion # Without this, finished Jobs and their Pods linger forever # Clutters etcd and namespace
History Limits (CronJobs)
spec: successfulJobsHistoryLimit: 3 # Keep last 3 successful Job objects failedJobsHistoryLimit: 1 # Keep last 1 failed Job object # Older Jobs (and their Pods) are automatically deleted # Keeps namespace clean while allowing debugging of recent failures
ttlSecondsAfterFinished (on Jobs) or history limits (on CronJobs). A common pattern: successfulJobsHistoryLimit: 3, failedJobsHistoryLimit: 5 (keep more failures for debugging).
Manual Job Trigger
# Create a Job from a CronJob (run immediately, outside schedule): kubectl create job backup-manual --from=cronjob/nightly-backup # Useful for: testing, incident response, ad-hoc runs
kubectl create job --from=cronjob/name is tested in CKAD. It creates a one-off Job using the CronJob's template. The Job runs immediately regardless of the schedule.
Summary
| Concept | Key Point |
|---|---|
| Job | Run Pods to completion (exit 0), retries on failure |
| completions | How many successful Pods needed (total work items) |
| parallelism | How many Pods run simultaneously |
| backoffLimit | Max retries before marking Job as Failed |
| activeDeadlineSeconds | Hard time limit — kills everything if exceeded |
| Indexed Jobs | Each Pod gets JOB_COMPLETION_INDEX — parallel processing without external queue |
| CronJob | Creates Jobs on a cron schedule |
| concurrencyPolicy | Allow / Forbid / Replace — controls overlap |
| startingDeadlineSeconds | How late a missed schedule can still be triggered |
| restartPolicy | Must be Never or OnFailure (not Always) |
📝 Quiz: Jobs & CronJobs
Q1: A Job has completions: 5, parallelism: 2, backoffLimit: 3. Pod 1 and 2 start. Pod 1 succeeds, Pod 2 fails. What happens next?
Q2: What's the difference between restartPolicy: Never and restartPolicy: OnFailure for a Job?
OnFailure: On failure, the kubelet restarts the container in the same Pod. The Pod stays, but container restart count increases. Cleaner namespace but loses per-attempt logs.
Q3: A CronJob runs every hour. The current Job takes 90 minutes. concurrencyPolicy: Forbid is set. What happens at the next scheduled time?
Forbid, if the previous Job is still running when the next schedule triggers, the new Job is not created. Once the current Job finishes, the following scheduled time will create a new Job normally.Q4: You have an Indexed Job with completions: 100, parallelism: 10. How does Pod #47 know what work to do?
JOB_COMPLETION_INDEX=47. The application uses this index to determine its chunk of work (e.g., process database rows 47000-47999, or file shard 47). No external coordination needed.Q5: A Job has activeDeadlineSeconds: 300. After 300 seconds, 4 of 5 completions have succeeded. What happens?
activeDeadlineSeconds is a hard wall-clock limit that overrides everything else — it doesn't matter that 4/5 were already done. The partial progress is lost (for that Job instance).Q6: How do you manually trigger a CronJob to run immediately (outside its schedule)?
kubectl create job my-manual-run --from=cronjob/my-cronjobThis creates a one-off Job using the CronJob's jobTemplate. It runs immediately and independently of the schedule.