CSI is the standard plugin interface for exposing storage systems to Kubernetes. Before CSI, storage drivers were compiled into Kubernetes itself — adding a new storage backend meant changing K8s core code. CSI decouples storage plugins from the Kubernetes release cycle, allowing vendors to ship and update drivers independently.
1. CSI Architecture
A CSI driver consists of two components deployed in the cluster:
Controller Plugin (Deployment)
Handles cluster-level operations — things that don't need to run on a specific node:
| Operation | What It Does | Triggered By |
|---|---|---|
CreateVolume | Provision a new disk (e.g., create EBS volume) | PVC created |
DeleteVolume | Delete the disk | PV reclaimed with Delete policy |
ControllerPublishVolume | Attach disk to a node (e.g., attach EBS to EC2) | Pod scheduled to node |
ControllerUnpublishVolume | Detach disk from node | Pod removed from node |
CreateSnapshot | Create a volume snapshot | VolumeSnapshot created |
ControllerExpandVolume | Resize the underlying disk | PVC size increased |
Node Plugin (DaemonSet)
Handles node-level operations — mounting the volume into the Pod's filesystem:
| Operation | What It Does |
|---|---|
NodeStageVolume | Format the disk and mount it to a staging directory on the node |
NodePublishVolume | Bind-mount from staging into the Pod's specific mount path |
NodeUnstageVolume | Unmount from staging |
NodeUnpublishVolume | Remove bind-mount from Pod |
NodeExpandVolume | Resize the filesystem (online expansion) |
2. CSI Sidecar Containers
The controller Pod runs the CSI driver alongside Kubernetes-maintained sidecar containers that bridge K8s API objects to CSI gRPC calls:
| Sidecar | Watches | Calls CSI Driver |
|---|---|---|
csi-provisioner | PVCs (new claims) | CreateVolume / DeleteVolume |
csi-attacher | VolumeAttachment objects | ControllerPublishVolume / ControllerUnpublishVolume |
csi-snapshotter | VolumeSnapshot CRDs | CreateSnapshot / DeleteSnapshot |
csi-resizer | PVC size changes | ControllerExpandVolume |
livenessprobe | Health of driver process | Reports health to kubelet |
csi-node-driver-registrar | (Node plugin only) | Registers node plugin with kubelet |
# Typical CSI controller Deployment has 4-5 containers:
spec:
containers:
- name: ebs-plugin # The actual driver
image: public.ecr.aws/ebs-csi-driver/aws-ebs-csi-driver:v1.25
- name: csi-provisioner # Watches PVCs
image: registry.k8s.io/sig-storage/csi-provisioner:v3.6
- name: csi-attacher # Watches VolumeAttachments
image: registry.k8s.io/sig-storage/csi-attacher:v4.4
- name: csi-snapshotter # Watches VolumeSnapshots
image: registry.k8s.io/sig-storage/csi-snapshotter:v6.3
- name: csi-resizer # Watches PVC size changes
image: registry.k8s.io/sig-storage/csi-resizer:v1.9
CSIDriver Object
# Registered when the driver is installed — tells K8s about driver capabilities:
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
name: ebs.csi.aws.com
spec:
attachRequired: true # Volume needs attach/detach (block storage: yes)
podInfoOnMount: false # Pass Pod info to NodePublish? (for audit)
fsGroupPolicy: File # How fsGroup is applied (File, ReadWriteOnceWithFSType, None)
volumeLifecycleModes:
- Persistent # Supports PVCs
- Ephemeral # Supports inline ephemeral volumes
3. Volume Snapshots
Snapshots capture the state of a volume at a point in time. They're managed via CRDs (not core K8s API):
VolumeSnapshotClass — "How to snapshot"
apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotClass metadata: name: ebs-snapclass driver: ebs.csi.aws.com deletionPolicy: Delete # Delete or Retain the snapshot when VolumeSnapshot is deleted
VolumeSnapshot — "Take a snapshot now"
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
name: db-snap-20240115
spec:
volumeSnapshotClassName: ebs-snapclass
source:
persistentVolumeClaimName: db-data # PVC to snapshot
# Check status: kubectl get volumesnapshot db-snap-20240115 # NAME READYTOUSE SOURCEPVC RESTORESIZE AGE # db-snap-20240115 true db-data 50Gi 2m
Restore from Snapshot — Create PVC from Snapshot
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: db-data-restored
spec:
storageClassName: gp3-encrypted
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 50Gi # Must be ≥ snapshot's restoreSize
dataSource:
name: db-snap-20240115 # ← Reference the snapshot
kind: VolumeSnapshot
apiGroup: snapshot.storage.k8s.io
4. Volume Cloning
Create a new PVC that's a copy of an existing PVC (without going through a snapshot):
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: db-data-clone
spec:
storageClassName: gp3-encrypted
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 50Gi
dataSource:
name: db-data # ← Source PVC (not snapshot)
kind: PersistentVolumeClaim # ← Clone from PVC directly
| Snapshot + Restore | Clone | |
|---|---|---|
| Intermediate object | VolumeSnapshot (can be stored long-term) | None (direct copy) |
| Cross-namespace | Yes (snapshot can be in different ns) | No (source PVC must be in same namespace) |
| Speed | Two operations (snap + restore) | One operation (faster) |
| Use case | Backups, disaster recovery, historical restore | Dev copy of prod data, testing |
5. Common CSI Drivers
| Driver | Storage Backend | Features |
|---|---|---|
ebs.csi.aws.com | AWS EBS | Snapshots, resize, encryption, gp3/io2 |
pd.csi.storage.gke.io | GCP Persistent Disk | Snapshots, resize, regional PDs (multi-zone) |
disk.csi.azure.com | Azure Managed Disk | Snapshots, resize, premium/standard |
efs.csi.aws.com | AWS EFS (NFS) | ReadWriteMany, elastic size |
nfs.csi.k8s.io | NFS server | ReadWriteMany, any NFS server |
ceph.rbd.csi.ceph.com | Ceph RBD | Snapshots, clones, multi-attach |
local.csi.k8s.io | Local node disk | High performance, no portability |
6. Troubleshooting CSI Issues
# Check CSI driver Pods: kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-ebs-csi-driver # Controller logs (provisioning failures): kubectl logs -n kube-system deploy/ebs-csi-controller -c csi-provisioner --tail=50 # Node plugin logs (mount failures): kubectl logs -n kube-system ds/ebs-csi-node -c ebs-plugin --tail=50 # Check VolumeAttachment (is volume attached to node?): kubectl get volumeattachment # NAME ATTACHER PV NODE ATTACHED AGE # csi-abc.. ebs.csi.aws.com pvc-xyz.. worker-1 true 5m # Check CSIDriver registration: kubectl get csidriver # NAME ATTACHREQUIRED PODINFOONMOUNT MODES # ebs.csi.aws.com true false Persistent
Common Issues
| Symptom | Cause | Fix |
|---|---|---|
| PVC stuck Pending | Controller plugin not running or can't reach cloud API | Check controller Pod logs, IAM permissions |
| Pod stuck ContainerCreating | Volume can't attach or mount (wrong zone, IAM, driver crash) | Check VolumeAttachment, node plugin logs |
| "Multi-Attach error" | RWO volume still attached to old node (slow detach) | Wait for detach timeout, or force-detach in cloud console |
| Snapshot stuck "not ready" | Snapshotter sidecar not running or driver doesn't support snapshots | Check snapshotter logs, verify VolumeSnapshotClass |
| Resize doesn't take effect | Filesystem resize pending — needs Pod restart (or driver supports online) | Check PVC conditions for FileSystemResizePending |
kubectl describe pod events. These four steps solve most CSI problems.
Summary
| Concept | Key Point |
|---|---|
| CSI | Standard gRPC plugin interface for storage drivers |
| Controller Plugin | Deployment — handles CreateVolume, Attach, Snapshot, Resize |
| Node Plugin | DaemonSet — handles Stage (format+mount) and Publish (bind-mount to Pod) |
| Sidecars | K8s-maintained containers that bridge API objects → CSI gRPC calls |
| Two-stage mount | Stage (to node) → Publish (to Pod). Enables RWO multi-Pod on same node. |
| VolumeSnapshot | Point-in-time snapshot — create from PVC, restore by creating new PVC from snapshot |
| Cloning | Create PVC from existing PVC (same namespace, copy-on-write) |
| VolumeAttachment | Tracks which PV is attached to which node |
| CSIDriver object | Declares driver capabilities (attach, fsGroup, ephemeral support) |
📝 Quiz: CSI
Q1: What's the difference between the CSI Controller Plugin and the Node Plugin?
Node Plugin (DaemonSet): Handles node-level operations — formatting the disk, mounting to a staging path, bind-mounting into the Pod. Runs on every node because it needs local access to the filesystem.
Q2: What is NodeStageVolume and why is it separate from NodePublishVolume?
NodePublishVolume: Creates a bind-mount from staging into the specific Pod's mount path. Done per Pod.
Why separate? Multiple Pods on the same node can share one RWO volume. The volume is staged once, then published (bind-mounted) into each Pod separately.
Q3: You create a VolumeSnapshot from a 100Gi PVC. How much additional storage does the snapshot consume?
Q4: A Pod is stuck in ContainerCreating with event "Multi-Attach error for volume." What's happening?
Q5: What's the role of the csi-provisioner sidecar container?
CreateVolume gRPC method, then creates a PV object in Kubernetes and binds it to the PVC. It bridges the gap between Kubernetes API (PVCs) and the CSI driver (gRPC). The driver author doesn't need to implement any K8s API logic.Q6: How do you restore a database from a VolumeSnapshot?
dataSource pointing to the VolumeSnapshot:spec:
dataSource:
name: db-snap-20240115
kind: VolumeSnapshot
apiGroup: snapshot.storage.k8s.io
resources:
requests:
storage: 50Gi # ≥ snapshot's restoreSizeThis provisions a new volume pre-populated with the snapshot's data. Point your database Pod at the new PVC. The original volume is unchanged — you can compare or validate before switching.