🧱 Why Namespace Isolation Is Not Enough

Linux namespaces provide visibility isolation — a container cannot see other containers' processes or files. But all containers on a node share the same host kernel. A kernel vulnerability (e.g. a container-escape CVE) can potentially give an attacker root on the node.

Traditional Containers Container A app process Container B app process Container C app process Shared Host Kernel ⚠️ kernel escape → node compromise Sandboxed Containers Container A app process sandbox kernel Container B app process sandbox kernel Host Kernel (minimal attack surface)

The Sandbox Threat Model

Container sandboxing addresses the scenario where: an attacker fully compromises a container (e.g. via an app RCE) and then attempts to break out to the host. Sandboxed runtimes interpose an additional kernel-level boundary:

  • gVisor — a user-space kernel written in Go that intercepts syscalls before they reach the host kernel
  • Kata Containers — runs each pod inside a lightweight VM with its own dedicated kernel via KVM/QEMU
⚠️ Performance trade-off Both approaches add latency. gVisor adds ~5–10% overhead for CPU-bound workloads but much more for syscall-heavy I/O. Kata adds VM startup overhead (~100–200ms). Use sandboxing selectively for high-risk workloads (untrusted code, multi-tenant SaaS) rather than cluster-wide.

⚙️ RuntimeClass — Selecting a Sandbox per Pod

RuntimeClass is a stable Kubernetes API (GA since 1.20) that lets you select which container runtime handler a pod uses. This is the mechanism that connects a pod spec to gVisor, Kata, or any other CRI-compatible runtime.

How RuntimeClass Works

  1. A cluster admin creates a RuntimeClass object referencing a handler name.
  2. The handler name maps to a configuration in containerd's or CRI-O's config file on the node.
  3. A pod sets spec.runtimeClassName to use that class.
  4. The scheduler ensures the pod lands on a node that supports the handler (via scheduling.nodeSelector).
# RuntimeClass for gVisor (runsc handler)
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: gvisor
handler: runsc
scheduling:
  nodeSelector:
    sandbox.gke.io/runtime: gvisor   # only schedule on nodes with gVisor installed

---
# RuntimeClass for Kata Containers
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: kata-qemu
handler: kata-qemu
scheduling:
  nodeSelector:
    katacontainers.io/kata-runtime: "true"
overhead:
  podFixed:
    memory: "160Mi"    # VM overhead accounted for in scheduling
    cpu: "250m"
# Pod using gVisor sandbox
apiVersion: v1
kind: Pod
metadata:
  name: sandboxed-workload
spec:
  runtimeClassName: gvisor   # ← the only change needed in the pod spec
  containers:
  - name: app
    image: myapp:v1
💡 overhead field matters for scheduling The overhead.podFixed field tells the scheduler and kubelet to account for VM/sandbox memory and CPU overhead when computing resource budgets. Without it, a Kata pod could be scheduled on a node that appears to have headroom but actually doesn't once the VM spins up.

🖥️ Kata Containers — VM-Based Isolation

Kata Containers runs each Kubernetes Pod inside a lightweight virtual machine with its own dedicated Linux kernel. The hypervisor (QEMU, AWS Firecracker, or Cloud Hypervisor) provides hardware-enforced isolation. From the application's perspective it's a normal container; from the host's perspective it's a VM.

Kata Architecture

🔌 Kata Shim

containerd-shim-kata-v2 — the CRI shim that interfaces containerd with the Kata runtime. One shim per Pod.

🖥️ Hypervisor

QEMU (default), AWS Firecracker (lightweight, fast boot), or Intel Cloud Hypervisor. Provides the hardware VM boundary.

🐧 Guest Kernel

A minimal Linux kernel (stripped down, read-only) runs inside the VM. The container process uses this guest kernel — not the host kernel.

🔗 Kata Agent

A gRPC agent running inside the VM that manages container lifecycle, mounts volumes, and sets up networking on behalf of the shim.

Kata vs gVisor — Key Differences

gVisor (runsc)Kata Containers
Isolation mechanismUser-space kernel (Go Sentry)Hardware VM (KVM/QEMU)
Kernel boundarySentry intercepts syscallsDedicated guest kernel per pod
Syscall compatibilityPartial (~240/350 syscalls)Full (guest kernel is real Linux)
I/O performanceGofer proxy overheadvirtio drivers — near-native
Memory overhead~10–30 MB per sandbox~130–160 MB per pod (VM)
Boot time~100ms~100–500ms (Firecracker: ~125ms)
Nested virt needed?No (ptrace mode) / Yes (KVM mode)Yes — host must support KVM
Best forMulti-tenant, untrusted code, functionsStrict compliance, privileged workloads, legacy apps

Installing Kata on a Node

# Using the Kata Containers installer (Ubuntu example)
bash -c "$(curl -fsSL https://raw.githubusercontent.com/kata-containers/kata-containers/main/utils/kata-manager.sh) install-packages"

# Add Kata handler to containerd config
cat >> /etc/containerd/config.toml <<EOF
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata-qemu]
  runtime_type = "io.containerd.kata-qemu.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata-fc]
  runtime_type = "io.containerd.kata-fc.v2"   # Firecracker variant
EOF
systemctl restart containerd

AWS Firecracker — Ultra-Fast MicroVMs

AWS Firecracker is the hypervisor powering AWS Lambda and Fargate. It boots a microVM in ~125ms with ~5MB memory overhead per VM (vs ~130MB for QEMU). Kata supports Firecracker as an alternative hypervisor, making it suitable for serverless-style workloads:

# RuntimeClass using Kata + Firecracker
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: kata-fc
handler: kata-fc
overhead:
  podFixed:
    memory: "60Mi"
    cpu: "100m"

Choosing the Right Sandbox

🔵 Decision guide
  • Untrusted / user-submitted code (e.g. online judge, CI runners) → gVisor or Kata
  • Multi-tenant SaaS where tenants share nodes → gVisor (lower overhead)
  • Privileged workloads needing full kernel (e.g. eBPF, kernel modules) → Kata (guest kernel)
  • Compliance requiring VM-level isolation (PCI-DSS, HIPAA) → Kata
  • GPU / HPC workloads → standard runtime (neither sandbox supports GPU well)
  • Everything else → standard runtime + seccomp + AppArmor is sufficient

🔍 Verifying Your Sandbox

After deploying a sandboxed pod, confirm it's using the expected runtime:

# Check which runtime a pod used
kubectl get pod sandboxed-workload -o jsonpath='{.spec.runtimeClassName}'
# → gvisor

# From inside the pod — confirm it sees gVisor's kernel
kubectl exec sandboxed-workload -- uname -r
# → 4.4.0  (gVisor reports a static kernel version)

# Kata: the guest kernel version differs from the host
kubectl exec kata-pod -- uname -r
# → 5.15.0-kata  (Kata guest kernel)

# On the node — see the shim processes
ps aux | grep -E 'runsc|kata-shim|containerd-shim-kata'

🔬 gVisor — User-Space Kernel

gVisor (open-sourced by Google, powers GKE Sandbox) implements a user-space kernel called runsc (run sandbox). Instead of container processes calling the host kernel directly, they call gVisor's Sentry — a Go process that intercepts syscalls and implements a large subset of the Linux kernel API in user space.

Without gVisor App Process direct syscall ↓ Host Kernel (full attack surface) Hardware With gVisor App Process syscall ↓ (intercepted) Sentry (user-space kernel, Go) minimal syscalls ↓ (ptrace or KVM) Host Kernel (minimal surface)

gVisor Architecture: Sentry + Gofer

  • Sentry — the user-space kernel. Handles all system calls from the container. Runs in user space so a kernel exploit in Sentry can't directly compromise the host.
  • Gofer — a file-proxy process that mediates filesystem access between the Sentry and the host. Sentry never directly opens host files.
  • Two platforms: ptrace (portable, slower) and KVM (hardware virtualization, faster, requires nested virt or bare metal).

Installing gVisor on a Node

# Install runsc binary
curl -fsSL https://gvisor.dev/archive.key | gpg --dearmor -o /usr/share/keyrings/gvisor.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/gvisor.gpg] https://storage.googleapis.com/gvisor/releases release main" \
  | tee /etc/apt/sources.list.d/gvisor.list
apt-get update && apt-get install -y runsc

# Configure containerd to use runsc handler
cat >> /etc/containerd/config.toml <<EOF
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"
EOF
systemctl restart containerd

What gVisor Cannot Sandbox

LimitationDetail
Syscall coverageNot all ~350 Linux syscalls are implemented. Some apps (e.g. those using io_uring) may fail.
I/O performanceFile and network I/O go through the Gofer proxy — significant overhead for I/O-heavy workloads (databases).
Privileged containersprivileged: true is not compatible with gVisor.
GPU workloadsGPU passthrough is not supported in standard gVisor.

GKE Sandbox — Managed gVisor

On GKE, gVisor is production-ready via GKE Sandbox. Enable it by creating a node pool with --sandbox type=gvisor. Google runs gVisor in KVM mode for performance and it automatically configures the RuntimeClass.

gcloud container node-pools create sandbox-pool \
  --cluster my-cluster \
  --sandbox type=gvisor \
  --machine-type n2-standard-4 \
  --num-nodes 3

🧠 Knowledge Check

Q1. What does the handler field in a RuntimeClass object refer to?

A) The name of the Pod that will use this RuntimeClass
B) The Kubernetes version required to use this runtime
C) The runtime binary/config name on the node (e.g. runsc, kata-qemu) that containerd maps to
D) A label selector for choosing which pods get this runtime

Q2. A workload needs to run an eBPF program that loads kernel modules. Should you use gVisor, Kata Containers, or the standard runtime?

A) gVisor — it intercepts syscalls and supports eBPF
B) Kata Containers — each pod gets a real guest kernel that supports full Linux syscalls
C) Standard runtime only — sandboxing always prevents kernel module usage
D) Neither — Kubernetes blocks kernel module access entirely

Q3. What is the purpose of overhead.podFixed in a RuntimeClass?

A) It limits the maximum resources a sandboxed pod can request
B) It sets the default resource requests for pods using this RuntimeClass
C) It adds sandbox/VM overhead to the pod's resource accounting so the scheduler can make accurate placement decisions
D) It defines the memory limit for the hypervisor process on the host

Q4. You run kubectl exec mypod -- uname -r and see 4.4.0 even though the host kernel is 6.1. What does this tell you?

A) The pod is running a very old base image
B) The pod is running inside a gVisor sandbox — gVisor always reports 4.4.0 as its static kernel version
C) The pod is running inside a Kata Containers VM
D) The node kernel was downgraded during a failed upgrade