GitOps says "everything in Git." But database passwords can't be in Git as plaintext. This is the most common objection to GitOps — and there are elegant solutions. This lesson compares them and shows you how to implement the best one for Azure.

The Problem

The GitOps Paradox GitOps: "Everything must be in Git" + Security: "Secrets must NEVER be in Git" How do you reconcile these two rules?

Three Solutions Compared

Sealed Secrets (Bitnami) Encrypt secrets client-side Store ciphertext in Git Controller decrypts in-cluster ✅ Simple to set up ✅ Truly GitOps (all in Git) ❌ Key rotation is manual ❌ Lose key = lose secrets ❌ Not integrated with KV Best for: Small teams, simple setup External Secrets ⭐ RECOMMENDED Reference secrets from vault Git has reference, not value Controller fetches at runtime ✅ Azure Key Vault native ✅ Auto-rotation ✅ Central secret management ✅ GitOps-safe (ref in Git) ❌ More components to manage Best for: Azure shops, production SOPS + Age (Mozilla) Encrypt values in YAML Keys stay readable ArgoCD plugin decrypts ✅ Diff-friendly (keys visible) ✅ Can use Azure KMS ❌ ArgoCD plugin required ❌ More complex setup ❌ No auto-rotation Best for: Multi-cloud, advanced

🏋️ External Secrets Operator + Azure Key Vault

How it works:

Git (safe) ExternalSecret YAML "fetch key: db-password" ESO Controller Reads ExternalSecret Fetches from vault Azure Key Vault Actual secret values db-password = "s3cr3t" Creates K8s Secret automatically

Install External Secrets Operator

helm repo add external-secrets https://charts.external-secrets.io
helm install external-secrets external-secrets/external-secrets \
  -n external-secrets --create-namespace

Create Azure Key Vault + store a secret

KV_NAME="kv-cicd-mastery-$(openssl rand -hex 3)"

az keyvault create --name $KV_NAME --resource-group rg-cicd-mastery --location eastus

# Store a secret
az keyvault secret set --vault-name $KV_NAME --name "db-password" --value "super-secret-123"

# Grant access to the managed identity (or workload identity)

Create SecretStore (connection to Key Vault)

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: azure-kv
  namespace: staging
spec:
  provider:
    azurekv:
      vaultUrl: "https://kv-cicd-mastery-XXX.vault.azure.net/"
      authType: WorkloadIdentity
      serviceAccountRef:
        name: default

Create ExternalSecret (this goes in Git!)

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: app-secrets
  namespace: staging
spec:
  refreshInterval: 1h    # Re-fetch every hour (auto-rotation!)
  secretStoreRef:
    name: azure-kv
  target:
    name: app-secrets    # K8s Secret that gets created
  data:
    - secretKey: DB_PASSWORD
      remoteRef:
        key: db-password  # Key Vault secret name

This YAML is safe for Git — it says WHICH secret to fetch, not the value itself.

The pattern: Git stores the reference ("fetch db-password from Key Vault"). External Secrets Controller fetches the value at runtime and creates a real K8s Secret. ArgoCD syncs the ExternalSecret resource; ESO handles the actual secret.

Using the Secret in Your Deployment

spec:
  containers:
    - name: api
      env:
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: app-secrets    # Created by External Secrets
              key: DB_PASSWORD

🧠 Recall Check

  1. Why can't you put a plain Kubernetes Secret YAML in Git?
  2. What does an ExternalSecret resource contain? (Is it sensitive?)
  3. What creates the actual K8s Secret that pods consume?
  4. How does auto-rotation work with External Secrets?
Reveal answers
  1. K8s Secrets are base64-encoded (NOT encrypted). Anyone with Git access can decode them. Git history is permanent — even deleting later doesn't remove it from history.
  2. It contains a reference: which vault, which key name, which namespace. The actual secret value is NOT in it. Completely safe for Git.
  3. The External Secrets Operator controller. It reads the ExternalSecret, calls Azure Key Vault, and creates/updates the K8s Secret.
  4. refreshInterval: 1h — ESO re-fetches from Key Vault every hour. If you rotate the secret in Key Vault, pods get the new value within the refresh interval (no redeploy needed).

Next lesson: Multi-Environment Promotion — PR-based workflows for promoting changes from staging to production.