When you need storage that survives Pod restarts, rescheduling, and even deletion — you need PersistentVolumes. The PV/PVC system decouples storage provisioning (admin concern) from storage consumption (developer concern), just like Nodes decouple compute provisioning from Pod scheduling.
1. The PV/PVC Model
| Resource | Scope | Created By | Purpose |
|---|---|---|---|
| PersistentVolume (PV) | Cluster | Admin (or dynamic provisioner) | Represents a piece of real storage |
| PersistentVolumeClaim (PVC) | Namespace | Developer | Request for storage (size, access mode, class) |
How Binding Works
- Developer creates a PVC with requirements (size, access mode, StorageClass)
- The PV controller finds a matching PV (or dynamic provisioner creates one)
- PVC and PV are bound (1:1 relationship — a PV can only bind to one PVC)
- Pod references the PVC by name in its volume spec
- kubelet mounts the underlying storage into the Pod
2. PersistentVolume Spec
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-ebs-01 # Cluster-scoped (no namespace)
spec:
capacity:
storage: 50Gi # Size of the volume
accessModes:
- ReadWriteOnce # How many nodes can mount it
persistentVolumeReclaimPolicy: Retain # What happens when released
storageClassName: gp3 # Links to a StorageClass (for binding)
csi: # Backend-specific (CSI driver)
driver: ebs.csi.aws.com
volumeHandle: vol-0abc123def456
fsType: ext4
nodeAffinity: # Which nodes can access this volume
required:
nodeSelectorTerms:
- matchExpressions:
- key: topology.kubernetes.io/zone
operator: In
values: [us-east-1a]
Access Modes
| Mode | Abbreviation | Meaning | Supported By |
|---|---|---|---|
ReadWriteOnce | RWO | Mounted read-write by one node | Most block storage (EBS, GCE PD, Azure Disk) |
ReadOnlyMany | ROX | Mounted read-only by many nodes | NFS, CephFS, cloud file shares |
ReadWriteMany | RWX | Mounted read-write by many nodes | NFS, CephFS, EFS, Azure Files |
ReadWriteOncePod | RWOP | Mounted read-write by one Pod (K8s 1.27+) | CSI drivers that support it |
ReadWriteOncePod (RWOP). This is a common exam trap.
Reclaim Policies
| Policy | When PVC is Deleted | Use Case |
|---|---|---|
Retain | PV becomes Released — data preserved, admin must manually clean up | Production data you can't afford to lose |
Delete | PV AND underlying storage are deleted | Dynamic provisioning (default for most StorageClasses) |
Recycle | ❌ Deprecated — performs rm -rf and makes PV available again | Don't use |
Retain. If someone accidentally deletes the PVC, the underlying disk survives. You can manually rebind it to a new PVC. With Delete, the disk is gone forever — there's no undo. Dynamic provisioners default to Delete, so override for critical workloads.
3. PersistentVolumeClaim Spec
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: db-data
namespace: production
spec:
accessModes:
- ReadWriteOnce
storageClassName: gp3 # Must match PV's storageClassName
resources:
requests:
storage: 50Gi # Minimum size needed
Binding Rules
A PVC binds to a PV when all these match:
- StorageClass — PVC's
storageClassNamematches PV's (or both empty) - Access mode — PV supports at least the modes requested
- Capacity — PV's capacity ≥ PVC's request
- Selector — if PVC has a
selector, PV must match its labels
# PVC with label selector (static provisioning):
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 20Gi
selector:
matchLabels:
environment: production
tier: database
Using a PVC in a Pod
apiVersion: v1
kind: Pod
metadata:
name: postgres
spec:
volumes:
- name: data
persistentVolumeClaim:
claimName: db-data # Reference the PVC by name
containers:
- name: postgres
image: postgres:16
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
4. PV/PVC Lifecycle
The "Released" Problem
When a PVC is deleted and the PV has Retain, the PV enters Released state. It still contains data but cannot be bound to a new PVC automatically. To reuse it:
# Option 1: Remove the claimRef to make it Available again:
kubectl patch pv pv-ebs-01 -p '{"spec":{"claimRef": null}}'
# Now it can bind to a new PVC
# Option 2: Create a PVC that matches exactly (same name, same spec)
# This is how you recover a StatefulSet's data after accidental PVC deletion
spec.claimRef. This clears the old PVC reference and the PV becomes Available.
5. Static vs Dynamic Provisioning
| Static Provisioning | Dynamic Provisioning | |
|---|---|---|
| PV creation | Admin pre-creates PVs manually | Provisioner creates PV automatically when PVC is created |
| Workflow | Admin creates PV → Dev creates PVC → bind | Dev creates PVC → provisioner creates PV → bind |
| Scale | Doesn't scale (admin bottleneck) | Self-service (developers get storage on demand) |
| Use case | Pre-existing storage, specific hardware, compliance | Cloud environments, self-service platforms |
# Static: Admin creates PV pointing to existing EBS volume # Dynamic: Developer creates PVC with StorageClass → provisioner creates EBS + PV # Dynamic provisioning is the norm in production. # StorageClasses (next lesson) enable it.
6. Volume Expansion (Resize)
# PVC can be expanded (if StorageClass allows it):
kubectl edit pvc db-data
# Change: resources.requests.storage: 50Gi → 100Gi
# Or patch:
kubectl patch pvc db-data -p '{"spec":{"resources":{"requests":{"storage":"100Gi"}}}}'
Requirements for Expansion
- StorageClass must have
allowVolumeExpansion: true - Only growing is supported — you cannot shrink a PVC
- For file-system based volumes: the Pod may need to restart for the filesystem to resize (depends on CSI driver — many support online expansion)
# Check if expansion is supported: kubectl get sc gp3 -o yaml | grep allowVolumeExpansion # allowVolumeExpansion: true # After patching, check PVC conditions: kubectl get pvc db-data -o yaml | grep -A5 conditions # - type: FileSystemResizePending # status: "True" # → Pod restart needed to resize filesystem # (or it happens online if the CSI driver supports it)
allowVolumeExpansion: true on production StorageClasses. Databases grow — and re-provisioning a larger volume means data migration. With expansion, you just patch the PVC size. Most CSI drivers (EBS, GCE PD, Azure Disk) support online expansion without Pod restart since K8s 1.24+.
Summary
| Concept | Key Point |
|---|---|
| PV | Cluster-scoped, represents actual storage. Created by admin or provisioner. |
| PVC | Namespaced request for storage. Created by developer. |
| Binding | 1:1 match by StorageClass + access mode + capacity. PV can't bind to two PVCs. |
| Access modes | RWO (one node), ROX (many read), RWX (many read-write), RWOP (one Pod) |
| RWO | One NODE, not one Pod — multiple Pods on same node can share it |
| Reclaim: Retain | PV preserved after PVC deletion — manual cleanup required |
| Reclaim: Delete | PV + underlying disk deleted when PVC is deleted |
| Released state | PV still has data but can't auto-bind. Patch out claimRef to reuse. |
| Expansion | Increase PVC size if StorageClass allows. Cannot shrink. |
| Dynamic provisioning | PVC + StorageClass → provisioner auto-creates PV + disk |
📝 Quiz: PersistentVolumes & PersistentVolumeClaims
Q1: A PVC requests 20Gi with RWO. Two PVs exist: PV-A (50Gi, RWO) and PV-B (10Gi, RWO). Which does it bind to?
Q2: What's the difference between ReadWriteOnce and ReadWriteOncePod?
RWOP: The volume can be mounted read-write by exactly one Pod. If another Pod (even on the same node) tries to mount it, it's denied. RWOP provides true single-writer semantics.
Q3: A PVC is deleted. The PV had persistentVolumeReclaimPolicy: Retain. What's the state of the PV and underlying data?
claimRef. To reuse: patch out the claimRef or manually delete and recreate the PV.Q4: You want to expand a PVC from 50Gi to 100Gi. What must be true for this to work?
allowVolumeExpansion: true. Also: (1) The underlying CSI driver must support expansion. (2) You can only grow, never shrink. (3) The filesystem may need to be expanded (some drivers do this online, others require a Pod restart).Q5: A developer creates a PVC but it stays in "Pending" status forever. The cluster has no StorageClass. What's wrong?
Q6: In a StatefulSet with volumeClaimTemplates, what happens to PVCs when you scale down from 5 to 3 replicas?