🧱 Why Namespace Isolation Is Not Enough
Linux namespaces provide visibility isolation — a container cannot see other containers' processes or files. But all containers on a node share the same host kernel. A kernel vulnerability (e.g. a container-escape CVE) can potentially give an attacker root on the node.
The Sandbox Threat Model
Container sandboxing addresses the scenario where: an attacker fully compromises a container (e.g. via an app RCE) and then attempts to break out to the host. Sandboxed runtimes interpose an additional kernel-level boundary:
- gVisor — a user-space kernel written in Go that intercepts syscalls before they reach the host kernel
- Kata Containers — runs each pod inside a lightweight VM with its own dedicated kernel via KVM/QEMU
⚙️ RuntimeClass — Selecting a Sandbox per Pod
RuntimeClass is a stable Kubernetes API (GA since 1.20) that lets you select which container runtime handler a pod uses. This is the mechanism that connects a pod spec to gVisor, Kata, or any other CRI-compatible runtime.
How RuntimeClass Works
- A cluster admin creates a
RuntimeClassobject referencing a handler name. - The handler name maps to a configuration in containerd's or CRI-O's config file on the node.
- A pod sets
spec.runtimeClassNameto use that class. - The scheduler ensures the pod lands on a node that supports the handler (via
scheduling.nodeSelector).
# RuntimeClass for gVisor (runsc handler)
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
name: gvisor
handler: runsc
scheduling:
nodeSelector:
sandbox.gke.io/runtime: gvisor # only schedule on nodes with gVisor installed
---
# RuntimeClass for Kata Containers
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
name: kata-qemu
handler: kata-qemu
scheduling:
nodeSelector:
katacontainers.io/kata-runtime: "true"
overhead:
podFixed:
memory: "160Mi" # VM overhead accounted for in scheduling
cpu: "250m"
# Pod using gVisor sandbox
apiVersion: v1
kind: Pod
metadata:
name: sandboxed-workload
spec:
runtimeClassName: gvisor # ← the only change needed in the pod spec
containers:
- name: app
image: myapp:v1
overhead.podFixed field tells the scheduler and kubelet to account for VM/sandbox memory and CPU overhead when computing resource budgets. Without it, a Kata pod could be scheduled on a node that appears to have headroom but actually doesn't once the VM spins up.
🖥️ Kata Containers — VM-Based Isolation
Kata Containers runs each Kubernetes Pod inside a lightweight virtual machine with its own dedicated Linux kernel. The hypervisor (QEMU, AWS Firecracker, or Cloud Hypervisor) provides hardware-enforced isolation. From the application's perspective it's a normal container; from the host's perspective it's a VM.
Kata Architecture
🔌 Kata Shim
containerd-shim-kata-v2 — the CRI shim that interfaces containerd with the Kata runtime. One shim per Pod.
🖥️ Hypervisor
QEMU (default), AWS Firecracker (lightweight, fast boot), or Intel Cloud Hypervisor. Provides the hardware VM boundary.
🐧 Guest Kernel
A minimal Linux kernel (stripped down, read-only) runs inside the VM. The container process uses this guest kernel — not the host kernel.
🔗 Kata Agent
A gRPC agent running inside the VM that manages container lifecycle, mounts volumes, and sets up networking on behalf of the shim.
Kata vs gVisor — Key Differences
| gVisor (runsc) | Kata Containers | |
|---|---|---|
| Isolation mechanism | User-space kernel (Go Sentry) | Hardware VM (KVM/QEMU) |
| Kernel boundary | Sentry intercepts syscalls | Dedicated guest kernel per pod |
| Syscall compatibility | Partial (~240/350 syscalls) | Full (guest kernel is real Linux) |
| I/O performance | Gofer proxy overhead | virtio drivers — near-native |
| Memory overhead | ~10–30 MB per sandbox | ~130–160 MB per pod (VM) |
| Boot time | ~100ms | ~100–500ms (Firecracker: ~125ms) |
| Nested virt needed? | No (ptrace mode) / Yes (KVM mode) | Yes — host must support KVM |
| Best for | Multi-tenant, untrusted code, functions | Strict compliance, privileged workloads, legacy apps |
Installing Kata on a Node
# Using the Kata Containers installer (Ubuntu example)
bash -c "$(curl -fsSL https://raw.githubusercontent.com/kata-containers/kata-containers/main/utils/kata-manager.sh) install-packages"
# Add Kata handler to containerd config
cat >> /etc/containerd/config.toml <<EOF
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata-qemu]
runtime_type = "io.containerd.kata-qemu.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata-fc]
runtime_type = "io.containerd.kata-fc.v2" # Firecracker variant
EOF
systemctl restart containerd
AWS Firecracker — Ultra-Fast MicroVMs
AWS Firecracker is the hypervisor powering AWS Lambda and Fargate. It boots a microVM in ~125ms with ~5MB memory overhead per VM (vs ~130MB for QEMU). Kata supports Firecracker as an alternative hypervisor, making it suitable for serverless-style workloads:
# RuntimeClass using Kata + Firecracker
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
name: kata-fc
handler: kata-fc
overhead:
podFixed:
memory: "60Mi"
cpu: "100m"
Choosing the Right Sandbox
- Untrusted / user-submitted code (e.g. online judge, CI runners) → gVisor or Kata
- Multi-tenant SaaS where tenants share nodes → gVisor (lower overhead)
- Privileged workloads needing full kernel (e.g. eBPF, kernel modules) → Kata (guest kernel)
- Compliance requiring VM-level isolation (PCI-DSS, HIPAA) → Kata
- GPU / HPC workloads → standard runtime (neither sandbox supports GPU well)
- Everything else → standard runtime + seccomp + AppArmor is sufficient
🔍 Verifying Your Sandbox
After deploying a sandboxed pod, confirm it's using the expected runtime:
# Check which runtime a pod used
kubectl get pod sandboxed-workload -o jsonpath='{.spec.runtimeClassName}'
# → gvisor
# From inside the pod — confirm it sees gVisor's kernel
kubectl exec sandboxed-workload -- uname -r
# → 4.4.0 (gVisor reports a static kernel version)
# Kata: the guest kernel version differs from the host
kubectl exec kata-pod -- uname -r
# → 5.15.0-kata (Kata guest kernel)
# On the node — see the shim processes
ps aux | grep -E 'runsc|kata-shim|containerd-shim-kata'
🔬 gVisor — User-Space Kernel
gVisor (open-sourced by Google, powers GKE Sandbox) implements a user-space kernel called runsc (run sandbox). Instead of container processes calling the host kernel directly, they call gVisor's Sentry — a Go process that intercepts syscalls and implements a large subset of the Linux kernel API in user space.
gVisor Architecture: Sentry + Gofer
- Sentry — the user-space kernel. Handles all system calls from the container. Runs in user space so a kernel exploit in Sentry can't directly compromise the host.
- Gofer — a file-proxy process that mediates filesystem access between the Sentry and the host. Sentry never directly opens host files.
- Two platforms: ptrace (portable, slower) and KVM (hardware virtualization, faster, requires nested virt or bare metal).
Installing gVisor on a Node
# Install runsc binary
curl -fsSL https://gvisor.dev/archive.key | gpg --dearmor -o /usr/share/keyrings/gvisor.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/gvisor.gpg] https://storage.googleapis.com/gvisor/releases release main" \
| tee /etc/apt/sources.list.d/gvisor.list
apt-get update && apt-get install -y runsc
# Configure containerd to use runsc handler
cat >> /etc/containerd/config.toml <<EOF
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
runtime_type = "io.containerd.runsc.v1"
EOF
systemctl restart containerd
What gVisor Cannot Sandbox
| Limitation | Detail |
|---|---|
| Syscall coverage | Not all ~350 Linux syscalls are implemented. Some apps (e.g. those using io_uring) may fail. |
| I/O performance | File and network I/O go through the Gofer proxy — significant overhead for I/O-heavy workloads (databases). |
| Privileged containers | privileged: true is not compatible with gVisor. |
| GPU workloads | GPU passthrough is not supported in standard gVisor. |
GKE Sandbox — Managed gVisor
On GKE, gVisor is production-ready via GKE Sandbox. Enable it by creating a node pool with --sandbox type=gvisor. Google runs gVisor in KVM mode for performance and it automatically configures the RuntimeClass.
gcloud container node-pools create sandbox-pool \
--cluster my-cluster \
--sandbox type=gvisor \
--machine-type n2-standard-4 \
--num-nodes 3