📋 Software Bill of Materials (SBOM)

An SBOM is a machine-readable inventory of every component (package, library, OS layer) inside a container image. Think of it like a nutritional label for software. When a new CVE drops, you can query your SBOMs to instantly know which images are affected.

SBOM Formats

FormatOriginTypical use
SPDXLinux Foundation / ISO 5962Compliance, government, OSS
CycloneDXOWASPSecurity tooling, vuln scanning
Syft JSONAnchore / Syft toolInternal tooling, Grype integration

Generating an SBOM with Syft

# Install syft
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin

# Generate CycloneDX SBOM for an image
syft ghcr.io/myorg/myapp:v1.2.3 -o cyclonedx-json > sbom.cyclonedx.json

# Generate SPDX SBOM
syft ghcr.io/myorg/myapp:v1.2.3 -o spdx-json > sbom.spdx.json

# Attach SBOM as a cosign attestation (now part of the image in registry)
cosign attest \
  --predicate sbom.cyclonedx.json \
  --type cyclonedx \
  ghcr.io/myorg/myapp@sha256:abc123...

Vulnerability Scanning with Grype

# Scan an image directly
grype ghcr.io/myorg/myapp:v1.2.3

# Scan using a pre-generated SBOM (faster, offline capable)
grype sbom:./sbom.cyclonedx.json

# Only show HIGH and CRITICAL CVEs, fail on CRITICAL
grype ghcr.io/myorg/myapp:v1.2.3 --fail-on critical \
  --only-fixed
🔵 SBOM in CI Gate The recommended pattern: generate SBOM → scan with Grype → fail the pipeline on unacceptable CVEs → attest both SBOM and scan results with cosign → enforce attestation presence via admission webhook.

Image Pinning — Never Use Mutable Tags

Even after signing, a mutable tag (e.g. :latest) is dangerous if imagePullPolicy: Always — a new push replaces the image that was verified. Always pin to a digest in production manifests:

# Get the digest after signing
IMAGE_DIGEST=$(cosign triangulate ghcr.io/myorg/myapp:v1.2.3)
# Or:
docker inspect --format='{{index .RepoDigests 0}}' ghcr.io/myorg/myapp:v1.2.3

# Use in manifest
image: ghcr.io/myorg/myapp@sha256:3b4c5d6e...abcd
# Not:  image: ghcr.io/myorg/myapp:v1.2.3  (mutable tag)

🚫 Enforcing Signatures at Admission Time

Signing images is pointless unless something blocks unsigned images from running. Two tools dominate: Kyverno and Cosign's Sigstore Policy Controller.

Option A — Kyverno Image Verification Policy

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: verify-image-signatures
spec:
  validationFailureAction: Enforce
  rules:
  - name: check-image-signature
    match:
      any:
      - resources:
          kinds: [Pod]
    verifyImages:
    - imageReferences:
      - "ghcr.io/myorg/*"
      attestors:
      - entries:
        - keyless:
            subject: "https://github.com/myorg/myapp/.github/workflows/build.yml@refs/heads/main"
            issuer: "https://token.actions.githubusercontent.com"
      # Optionally require SBOM attestation too
      attestations:
      - predicateType: https://cyclonedx.org/bom
        conditions:
        - all:
          - key: "{{ bomFormat }}"
            operator: Equals
            value: "CycloneDX"

Option B — Sigstore Policy Controller

The Policy Controller (formerly cosign policy) is a purpose-built admission webhook from the Sigstore project, installed via Helm:

# Install Policy Controller
helm repo add sigstore https://sigstore.github.io/helm-charts
helm install policy-controller sigstore/policy-controller -n cosign-system --create-namespace

# Label a namespace to enforce verification
kubectl label namespace production policy.sigstore.dev/include=true

# Define a ClusterImagePolicy
apiVersion: policy.sigstore.dev/v1beta1
kind: ClusterImagePolicy
metadata:
  name: myorg-images
spec:
  images:
  - glob: "ghcr.io/myorg/**"
  authorities:
  - keyless:
      url: https://fulcio.sigstore.dev
      identities:
      - issuer: https://token.actions.githubusercontent.com
        subjectRegExp: "https://github.com/myorg/.*"
🔴 Failure mode to know Both tools have a validationFailureAction / failure policy. Set it to Enforce/Fail in production and Audit while rolling out. An accidental Ignore failure policy means the webhook fails open — all images pass through if the webhook is unreachable.

Private Registry Allow-listing (Defence-in-Depth)

Even with signature verification, a belt-and-suspenders approach is to block images from untrusted registries entirely. A Kyverno policy for this:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: restrict-image-registries
spec:
  validationFailureAction: Enforce
  rules:
  - name: allowed-registries
    match:
      any:
      - resources: { kinds: [Pod] }
    validate:
      message: "Image must be from ghcr.io/myorg or registry.mycompany.internal"
      pattern:
        spec:
          containers:
          - image: "ghcr.io/myorg/* | registry.mycompany.internal/*"

🔏 cosign & the Sigstore Ecosystem

Sigstore is a Linux Foundation project that makes signing software artifacts as easy as HTTPS — no key management nightmares, no separate PKI. Its three components:

cosign

CLI & Go library for signing and verifying container images and other OCI artifacts. Supports keyless and key-based signing.

Fulcio

Free code-signing CA. Issues short-lived certificates (10 min TTL) tied to an OIDC identity (GitHub Actions, Google, etc.).

Rekor

Immutable transparency log (like Certificate Transparency). Every signature is recorded; you can verify inclusion proofs.

Keyless Signing (preferred)

With keyless signing, you never manage a long-lived private key. Instead, your CI identity (e.g. GitHub Actions OIDC token) is used to get a short-lived cert from Fulcio, sign the image, and record everything in Rekor.

CI Job (GitHub Actions) Fulcio CA short-lived cert Registry image + signature Rekor transparency log Admission verify OIDC sign & push record check

Signing an Image with cosign

# Build and push first
docker build -t ghcr.io/myorg/myapp:v1.2.3 .
docker push ghcr.io/myorg/myapp:v1.2.3

# Keyless sign (requires OIDC — runs in CI, no key needed)
COSIGN_EXPERIMENTAL=1 cosign sign ghcr.io/myorg/myapp:v1.2.3

# Key-based sign (long-lived key alternative)
cosign generate-key-pair            # produces cosign.key + cosign.pub
cosign sign --key cosign.key ghcr.io/myorg/myapp:v1.2.3

# Verify (keyless — checks Rekor + Fulcio cert chain)
COSIGN_EXPERIMENTAL=1 cosign verify \
  --certificate-identity-regexp="https://github.com/myorg/myapp" \
  --certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
  ghcr.io/myorg/myapp:v1.2.3

# Verify (key-based)
cosign verify --key cosign.pub ghcr.io/myorg/myapp:v1.2.3
💡 Sign by digest, not by tag cosign pins the signature to the image's SHA-256 digest, not the tag. This means a tag re-push can't substitute a different image without breaking verification — digest immutability is your guarantee.

Attestations — Going Beyond Signatures

A signature says "I built this." An attestation carries a verifiable claim about the artifact — SLSA provenance, vulnerability scan results, SBOM, test results.

# Attach a SLSA provenance attestation (generated by slsa-github-generator)
cosign attest \
  --predicate provenance.json \
  --type slsaprovenance \
  ghcr.io/myorg/myapp@sha256:abc123...

# Verify the attestation
cosign verify-attestation \
  --type slsaprovenance \
  --certificate-identity-regexp="..." \
  --certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
  ghcr.io/myorg/myapp@sha256:abc123...
Chapter 8 · Lesson 4 of 7

Supply Chain Security

Sign, verify, and attest every artifact — images, SBOMs, and provenance — so only trusted software runs in your cluster.

Cert: CKS
Difficulty: Advanced
Read time: ~25 min
Tools: cosign · Rekor · Kyverno · OPA

🔗 Why Supply Chain Attacks Are Devastating

The SolarWinds and XZ Utils compromises taught the industry a hard lesson: the weakest link is the build and distribution pipeline, not the running system. In Kubernetes, every container image that lands on a node is trusted by default — unless you build defences in.

🏗️ Build-time

Malicious code injected into source, dependencies, or build systems (e.g. compromised npm package).

📦 Distribution-time

Tampered layers in a registry, MITM during pull, or pushing a new "latest" tag over an existing image.

🚀 Runtime

Unverified images running as root, privileged containers, or images with CVEs that are never patched.

⚠️ The Kubernetes default Any imagePullPolicy: Always pod will pull whatever is at a tag — there is zero signature verification unless you add it. latest is especially dangerous.

The SLSA Framework

SLSA (Supply chain Levels for Software Artifacts, pronounced "salsa") is a Google-originated framework that grades supply chain hardening across four levels:

LevelRequirementK8s relevance
SLSA 1Scripted build, provenance generatedBasic CI pipeline
SLSA 2Hosted build, signed provenanceGitHub Actions + OIDC
SLSA 3Hardened build, auditableEphemeral builders, hermetic builds
SLSA 4Two-party review, hermetic + reproducibleFull reproducible builds

Kubernetes admission controls typically enforce SLSA 2–3: signed images and verified provenance attestations.

🧠 Knowledge Check

Q1. What does Rekor provide in the Sigstore ecosystem?

A) A free code-signing CA that issues short-lived certificates
B) An immutable transparency log recording all signatures and attestations
C) A vulnerability scanner for container images
D) An admission webhook that blocks unsigned images

Q2. Why should production manifests use image digests instead of tags?

A) Tags are mutable; a new image push can replace the tag without breaking the signature check
B) Digests are shorter and save bandwidth
C) Kubernetes can only pull images by digest
D) Digests are required by the CRI

Q3. What is the difference between a cosign signature and a cosign attestation?

A) Signatures are stored in Rekor; attestations are stored in Fulcio
B) Signatures use keyless signing; attestations require a key pair
C) A signature proves identity/integrity; an attestation is a verifiable claim about the artifact (SBOM, provenance, scan results)
D) There is no difference — they are the same thing

Q4. A Kyverno ClusterPolicy has validationFailureAction: Audit. What happens when an unsigned image is deployed?

A) The Pod is blocked and the user sees an admission error
B) The Pod is allowed but a PolicyReport violation is created
C) The webhook crashes and the Pod is blocked by default
D) Kyverno deletes the Pod after it starts