CSI is the standard plugin interface for exposing storage systems to Kubernetes. Before CSI, storage drivers were compiled into Kubernetes itself — adding a new storage backend meant changing K8s core code. CSI decouples storage plugins from the Kubernetes release cycle, allowing vendors to ship and update drivers independently.

1. CSI Architecture

A CSI driver consists of two components deployed in the cluster:

Controller Plugin (Deployment) Runs as a Deployment (1-3 replicas) CSI Driver CreateVolume Sidecar containers provisioner, attacher, snapshotter Node Plugin (DaemonSet) Runs on every node NodeStageVolume NodePublishVolume API Server kubelet Cloud API (EBS, PD, etc.) watch PVCs kubelet calls create/delete volume

Controller Plugin (Deployment)

Handles cluster-level operations — things that don't need to run on a specific node:

OperationWhat It DoesTriggered By
CreateVolumeProvision a new disk (e.g., create EBS volume)PVC created
DeleteVolumeDelete the diskPV reclaimed with Delete policy
ControllerPublishVolumeAttach disk to a node (e.g., attach EBS to EC2)Pod scheduled to node
ControllerUnpublishVolumeDetach disk from nodePod removed from node
CreateSnapshotCreate a volume snapshotVolumeSnapshot created
ControllerExpandVolumeResize the underlying diskPVC size increased

Node Plugin (DaemonSet)

Handles node-level operations — mounting the volume into the Pod's filesystem:

OperationWhat It Does
NodeStageVolumeFormat the disk and mount it to a staging directory on the node
NodePublishVolumeBind-mount from staging into the Pod's specific mount path
NodeUnstageVolumeUnmount from staging
NodeUnpublishVolumeRemove bind-mount from Pod
NodeExpandVolumeResize the filesystem (online expansion)
Two-stage mount: CSI uses a two-step process: Stage (format + mount to node) → Publish (bind-mount into Pod). This allows multiple Pods on the same node to share one staged volume (for ReadWriteOnce — multiple Pods on same node can access it). Unstaging only happens when the last Pod on that node releases the volume.

2. CSI Sidecar Containers

The controller Pod runs the CSI driver alongside Kubernetes-maintained sidecar containers that bridge K8s API objects to CSI gRPC calls:

SidecarWatchesCalls CSI Driver
csi-provisionerPVCs (new claims)CreateVolume / DeleteVolume
csi-attacherVolumeAttachment objectsControllerPublishVolume / ControllerUnpublishVolume
csi-snapshotterVolumeSnapshot CRDsCreateSnapshot / DeleteSnapshot
csi-resizerPVC size changesControllerExpandVolume
livenessprobeHealth of driver processReports health to kubelet
csi-node-driver-registrar(Node plugin only)Registers node plugin with kubelet
# Typical CSI controller Deployment has 4-5 containers:
spec:
  containers:
    - name: ebs-plugin             # The actual driver
      image: public.ecr.aws/ebs-csi-driver/aws-ebs-csi-driver:v1.25
    - name: csi-provisioner        # Watches PVCs
      image: registry.k8s.io/sig-storage/csi-provisioner:v3.6
    - name: csi-attacher           # Watches VolumeAttachments
      image: registry.k8s.io/sig-storage/csi-attacher:v4.4
    - name: csi-snapshotter        # Watches VolumeSnapshots
      image: registry.k8s.io/sig-storage/csi-snapshotter:v6.3
    - name: csi-resizer            # Watches PVC size changes
      image: registry.k8s.io/sig-storage/csi-resizer:v1.9
Driver authors only implement the gRPC interface. The sidecars handle all Kubernetes API interaction (watching objects, creating PVs, updating statuses). This separation means every CSI driver gets consistent K8s integration for free — the vendor only writes the storage-specific logic.

CSIDriver Object

# Registered when the driver is installed — tells K8s about driver capabilities:
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
  name: ebs.csi.aws.com
spec:
  attachRequired: true          # Volume needs attach/detach (block storage: yes)
  podInfoOnMount: false         # Pass Pod info to NodePublish? (for audit)
  fsGroupPolicy: File           # How fsGroup is applied (File, ReadWriteOnceWithFSType, None)
  volumeLifecycleModes:
    - Persistent                # Supports PVCs
    - Ephemeral                 # Supports inline ephemeral volumes

3. Volume Snapshots

Snapshots capture the state of a volume at a point in time. They're managed via CRDs (not core K8s API):

VolumeSnapshotClass — "How to snapshot"

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
  name: ebs-snapclass
driver: ebs.csi.aws.com
deletionPolicy: Delete            # Delete or Retain the snapshot when VolumeSnapshot is deleted

VolumeSnapshot — "Take a snapshot now"

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: db-snap-20240115
spec:
  volumeSnapshotClassName: ebs-snapclass
  source:
    persistentVolumeClaimName: db-data     # PVC to snapshot
# Check status:
kubectl get volumesnapshot db-snap-20240115
# NAME               READYTOUSE   SOURCEPVC   RESTORESIZE   AGE
# db-snap-20240115   true         db-data     50Gi          2m

Restore from Snapshot — Create PVC from Snapshot

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: db-data-restored
spec:
  storageClassName: gp3-encrypted
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 50Gi                   # Must be ≥ snapshot's restoreSize
  dataSource:
    name: db-snap-20240115            # ← Reference the snapshot
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io
Snapshot-based restore is the standard disaster recovery pattern for databases on K8s. Schedule periodic snapshots (CronJob + VolumeSnapshot), and restore by creating a new PVC from the snapshot. The restore creates a new volume — it doesn't overwrite the original. This lets you test recovery without risking the production volume.

4. Volume Cloning

Create a new PVC that's a copy of an existing PVC (without going through a snapshot):

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: db-data-clone
spec:
  storageClassName: gp3-encrypted
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 50Gi
  dataSource:
    name: db-data                     # ← Source PVC (not snapshot)
    kind: PersistentVolumeClaim       # ← Clone from PVC directly
Snapshot + RestoreClone
Intermediate objectVolumeSnapshot (can be stored long-term)None (direct copy)
Cross-namespaceYes (snapshot can be in different ns)No (source PVC must be in same namespace)
SpeedTwo operations (snap + restore)One operation (faster)
Use caseBackups, disaster recovery, historical restoreDev copy of prod data, testing
Both snapshots and clones use copy-on-write under the hood (on most cloud providers). The clone appears instantly even for terabyte volumes — actual data is only copied when blocks diverge. This makes both operations fast and cost-efficient.

5. Common CSI Drivers

DriverStorage BackendFeatures
ebs.csi.aws.comAWS EBSSnapshots, resize, encryption, gp3/io2
pd.csi.storage.gke.ioGCP Persistent DiskSnapshots, resize, regional PDs (multi-zone)
disk.csi.azure.comAzure Managed DiskSnapshots, resize, premium/standard
efs.csi.aws.comAWS EFS (NFS)ReadWriteMany, elastic size
nfs.csi.k8s.ioNFS serverReadWriteMany, any NFS server
ceph.rbd.csi.ceph.comCeph RBDSnapshots, clones, multi-attach
local.csi.k8s.ioLocal node diskHigh performance, no portability

6. Troubleshooting CSI Issues

# Check CSI driver Pods:
kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-ebs-csi-driver

# Controller logs (provisioning failures):
kubectl logs -n kube-system deploy/ebs-csi-controller -c csi-provisioner --tail=50

# Node plugin logs (mount failures):
kubectl logs -n kube-system ds/ebs-csi-node -c ebs-plugin --tail=50

# Check VolumeAttachment (is volume attached to node?):
kubectl get volumeattachment
# NAME        ATTACHER            PV          NODE       ATTACHED   AGE
# csi-abc..   ebs.csi.aws.com     pvc-xyz..   worker-1   true       5m

# Check CSIDriver registration:
kubectl get csidriver
# NAME                  ATTACHREQUIRED   PODINFOONMOUNT   MODES
# ebs.csi.aws.com       true             false            Persistent

Common Issues

SymptomCauseFix
PVC stuck PendingController plugin not running or can't reach cloud APICheck controller Pod logs, IAM permissions
Pod stuck ContainerCreatingVolume can't attach or mount (wrong zone, IAM, driver crash)Check VolumeAttachment, node plugin logs
"Multi-Attach error"RWO volume still attached to old node (slow detach)Wait for detach timeout, or force-detach in cloud console
Snapshot stuck "not ready"Snapshotter sidecar not running or driver doesn't support snapshotsCheck snapshotter logs, verify VolumeSnapshotClass
Resize doesn't take effectFilesystem resize pending — needs Pod restart (or driver supports online)Check PVC conditions for FileSystemResizePending
CKA storage troubleshooting: if a Pod won't start with a volume issue, check in order: (1) PVC status (Bound?). (2) VolumeAttachment status (Attached?). (3) Node plugin logs (mount errors). (4) kubectl describe pod events. These four steps solve most CSI problems.

Summary

ConceptKey Point
CSIStandard gRPC plugin interface for storage drivers
Controller PluginDeployment — handles CreateVolume, Attach, Snapshot, Resize
Node PluginDaemonSet — handles Stage (format+mount) and Publish (bind-mount to Pod)
SidecarsK8s-maintained containers that bridge API objects → CSI gRPC calls
Two-stage mountStage (to node) → Publish (to Pod). Enables RWO multi-Pod on same node.
VolumeSnapshotPoint-in-time snapshot — create from PVC, restore by creating new PVC from snapshot
CloningCreate PVC from existing PVC (same namespace, copy-on-write)
VolumeAttachmentTracks which PV is attached to which node
CSIDriver objectDeclares driver capabilities (attach, fsGroup, ephemeral support)

📝 Quiz: CSI

Q1: What's the difference between the CSI Controller Plugin and the Node Plugin?

Controller Plugin (Deployment): Handles cluster-level operations — creating/deleting volumes, attaching to nodes, snapshots, resize. Talks to the cloud API. Doesn't need to run on every node.
Node Plugin (DaemonSet): Handles node-level operations — formatting the disk, mounting to a staging path, bind-mounting into the Pod. Runs on every node because it needs local access to the filesystem.

Q2: What is NodeStageVolume and why is it separate from NodePublishVolume?

NodeStageVolume: Formats the disk (if needed) and mounts it to a staging directory on the node. This is done once per volume per node.
NodePublishVolume: Creates a bind-mount from staging into the specific Pod's mount path. Done per Pod.
Why separate? Multiple Pods on the same node can share one RWO volume. The volume is staged once, then published (bind-mounted) into each Pod separately.

Q3: You create a VolumeSnapshot from a 100Gi PVC. How much additional storage does the snapshot consume?

Initially: zero or near-zero. Most cloud providers use copy-on-write for snapshots. The snapshot only stores blocks that differ from the source. At creation time, nothing has diverged so storage cost is minimal. Over time, as the source volume changes, the snapshot stores the original blocks — cost grows incrementally. This is why snapshots are cheap to create.

Q4: A Pod is stuck in ContainerCreating with event "Multi-Attach error for volume." What's happening?

The RWO volume is still attached to the previous node (the old Pod was on a different node and hasn't fully released it). Block storage can only attach to one node at a time. This happens during node failures or fast rescheduling. Fix: wait for the attach timeout (default 6 minutes) for auto-detach, or manually force-detach in the cloud console.

Q5: What's the role of the csi-provisioner sidecar container?

It watches PVC objects in the Kubernetes API. When a new PVC appears that matches its StorageClass, it calls the CSI driver's CreateVolume gRPC method, then creates a PV object in Kubernetes and binds it to the PVC. It bridges the gap between Kubernetes API (PVCs) and the CSI driver (gRPC). The driver author doesn't need to implement any K8s API logic.

Q6: How do you restore a database from a VolumeSnapshot?

Create a new PVC with dataSource pointing to the VolumeSnapshot:
spec:
  dataSource:
    name: db-snap-20240115
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io
  resources:
    requests:
      storage: 50Gi  # ≥ snapshot's restoreSize
This provisions a new volume pre-populated with the snapshot's data. Point your database Pod at the new PVC. The original volume is unchanged — you can compare or validate before switching.