🏢 Tenant Models

Multi-tenancy in Kubernetes spans a spectrum from loose namespace separation to fully isolated virtual clusters. Choose your model based on the trust level between tenants.

🔐 Namespace Isolation & RBAC Patterns

Namespace Bootstrap Template

Automate namespace creation with a consistent set of objects: RBAC, ResourceQuota, LimitRange, and default NetworkPolicy. Use a Helm chart or Kyverno policy to ensure every namespace gets the same baseline.

# Namespace + standard labels
apiVersion: v1
kind: Namespace
metadata:
  name: team-alpha
  labels:
    team: alpha
    environment: production
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/warn: restricted
---
# ResourceQuota — hard cap on resource consumption
apiVersion: v1
kind: ResourceQuota
metadata:
  name: default-quota
  namespace: team-alpha
spec:
  hard:
    requests.cpu: "16"
    requests.memory: "32Gi"
    limits.cpu: "32"
    limits.memory: "64Gi"
    count/pods: "100"
    count/services: "20"
    count/services.loadbalancers: "2"
    persistentvolumeclaims: "20"
    requests.storage: "500Gi"
---
# LimitRange — default requests/limits for pods with no spec
apiVersion: v1
kind: LimitRange
metadata:
  name: default-limits
  namespace: team-alpha
spec:
  limits:
    - type: Container
      default:
        cpu: "500m"
        memory: "256Mi"
      defaultRequest:
        cpu: "100m"
        memory: "128Mi"
      max:
        cpu: "4"
        memory: "8Gi"

RBAC Roles for Tenants

# Developer role — scoped to one namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: developer
  namespace: team-alpha
rules:
  - apiGroups: ["", "apps", "batch", "autoscaling"]
    resources:
      - pods
      - deployments
      - replicasets
      - services
      - configmaps
      - horizontalpodautoscalers
      - jobs
      - cronjobs
    verbs: ["get","list","watch","create","update","patch","delete"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get","list","watch"]   # read secrets but not create/delete
  - apiGroups: [""]
    resources: ["pods/log", "pods/exec", "pods/portforward"]
    verbs: ["get","create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: team-alpha-developers
  namespace: team-alpha
subjects:
  - kind: Group
    name: team-alpha               # OIDC group claim
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: developer
  apiGroup: rbac.authorization.k8s.io

🌐 Network, Admission & Node Isolation

Default-Deny NetworkPolicy per Namespace

Apply this to every tenant namespace immediately after creation. Tenants must explicitly declare what traffic they allow.

# Deny all ingress and egress by default
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: team-alpha
spec:
  podSelector: {}       # matches all pods in namespace
  policyTypes:
    - Ingress
    - Egress
---
# Allow DNS egress (CoreDNS) — required for service discovery
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns-egress
  namespace: team-alpha
spec:
  podSelector: {}
  policyTypes: [Egress]
  egress:
    - ports:
        - port: 53
          protocol: UDP
        - port: 53
          protocol: TCP
---
# Allow intra-namespace communication
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-same-namespace
  namespace: team-alpha
spec:
  podSelector: {}
  policyTypes: [Ingress, Egress]
  ingress:
    - from:
        - podSelector: {}    # any pod in same namespace
  egress:
    - to:
        - podSelector: {}

PodSecurity Standards Enforcement

# Enforce 'restricted' profile on namespace via label
kubectl label namespace team-alpha \
  pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/warn=restricted \
  pod-security.kubernetes.io/audit=restricted

# Restricted profile blocks:
# - privileged containers
# - hostNetwork, hostPID, hostIPC
# - hostPath volumes
# - runAsRoot
# - allowPrivilegeEscalation
# - dropping less than ALL capabilities

Kyverno Policies for Tenant Guardrails

# Require all pods to have resource requests
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-requests
spec:
  validationFailureAction: Enforce
  rules:
    - name: check-requests
      match:
        resources: { kinds: [Pod] }
      validate:
        message: "CPU and memory requests are required"
        pattern:
          spec:
            containers:
              - resources:
                  requests:
                    cpu: "?*"
                    memory: "?*"
---
# Disallow latest image tag in production namespaces
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: disallow-latest-tag
spec:
  validationFailureAction: Enforce
  rules:
    - name: check-image-tag
      match:
        resources:
          kinds: [Pod]
          namespaceSelector:
            matchLabels:
              environment: production
      validate:
        message: "Image tag ':latest' is not allowed in production"
        pattern:
          spec:
            containers:
              - image: "!*:latest"

Dedicated Node Pools per Tenant

# Taint dedicated nodes for a tenant
kubectl taint nodes -l tenant=alpha \
  tenant=alpha:NoSchedule

# Team Alpha pods add toleration to use their nodes
spec:
  tolerations:
    - key: tenant
      operator: Equal
      value: alpha
      effect: NoSchedule
  nodeSelector:
    tenant: alpha      # or use nodeAffinity for required/preferred

🏗️ Hard Multi-tenancy with vcluster

vcluster runs a full Kubernetes control plane (apiserver + etcd + scheduler + controller-manager) as a StatefulSet inside a namespace of the host cluster. From the tenant's perspective they have a real cluster with full admin access. From the host's perspective it's just pods in a namespace.

Create a vcluster

# Install vcluster CLI
curl -L -o vcluster https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-linux-amd64
chmod +x vcluster && mv vcluster /usr/local/bin/

# Create a virtual cluster for customer-c
vcluster create customer-c \
  --namespace vc-customer-c \
  --set "sync.ingresses.enabled=true" \
  --set "isolation.enabled=true" \
  --set "isolation.resourceQuota.enabled=true" \
  --set "isolation.resourceQuota.requests.cpu=8" \
  --set "isolation.resourceQuota.requests.memory=16Gi"

# Connect to the virtual cluster
vcluster connect customer-c --namespace vc-customer-c
# Switches kubeconfig to point at the virtual cluster's apiserver

kubectl get nodes     # sees virtual "fake" nodes
kubectl get pods -A   # only sees pods in this vcluster

vcluster Architecture

# What vcluster creates in the host cluster (vc-customer-c namespace):
kubectl get pods -n vc-customer-c
# NAME                                    READY   STATUS
# customer-c-0                            2/2     Running   ← virtual apiserver + etcd
# customer-c-coredns-abc                  1/1     Running   ← virtual CoreDNS
# customer-c-vcluster-workload-abc        1/1     Running   ← synced pod (tenant workload)

# Tenant pods appear in host namespace as:
# -x--x-
# e.g. nginx-x-default-x-customer-c

Isolation Model Comparison

📝 Knowledge Check