🔗 Owner References — Automatic Garbage Collection

When your operator creates child resources (StatefulSet, Service, ConfigMap), it should set an owner reference pointing back to the parent CR. Kubernetes garbage collection then automatically deletes the child when the parent is deleted — no cleanup code needed.

// Set the Database CR as the owner of the StatefulSet
func (r *DatabaseReconciler) buildStatefulSet(db *myorgv1.Database) *appsv1.StatefulSet {
    sts := &appsv1.StatefulSet{
        ObjectMeta: metav1.ObjectMeta{
            Name:      db.Name,
            Namespace: db.Namespace,
            Labels:    map[string]string{"app": db.Name},
        },
        // ... spec
    }

    // SetControllerReference adds ownerReferences with controller=true, blockOwnerDeletion=true
    if err := ctrl.SetControllerReference(db, sts, r.Scheme); err != nil {
        return nil
    }
    return sts
}

// In the object metadata this produces:
// ownerReferences:
// - apiVersion: myorg.example.com/v1
//   kind: Database
//   name: prod-postgres
//   uid: abc-123
//   controller: true
//   blockOwnerDeletion: true

🔒 Finalizers — Custom Cleanup Logic

Finalizers let you run arbitrary cleanup before a resource is actually deleted from etcd. When a resource with finalizers is deleted, Kubernetes sets deletionTimestamp but does NOT remove it until all finalizers are removed.

const dbFinalizer = "myorg.example.com/cleanup"

func (r *DatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    db := &myorgv1.Database{}
    r.Get(ctx, req.NamespacedName, db)

    // Add finalizer on creation
    if !controllerutil.ContainsFinalizer(db, dbFinalizer) {
        controllerutil.AddFinalizer(db, dbFinalizer)
        return ctrl.Result{}, r.Update(ctx, db)
    }

    // Handle deletion
    if !db.DeletionTimestamp.IsZero() {
        // Run cleanup: delete external cloud resources, revoke secrets, etc.
        if err := r.cleanupExternalResources(ctx, db); err != nil {
            return ctrl.Result{}, err
        }
        // Remove finalizer → triggers actual etcd deletion
        controllerutil.RemoveFinalizer(db, dbFinalizer)
        return ctrl.Result{}, r.Update(ctx, db)
    }

    // Normal reconcile path...
    return r.reconcileDatabase(ctx, db)
}

⚙️ Wiring the Controller (main.go)

The cmd/main.go entrypoint bootstraps the controller-runtime manager — the component that runs all controllers, handles leader election, serves webhook endpoints, and exposes health/metrics endpoints:

func main() {
    mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
        Scheme:                 scheme,
        Metrics: server.Options{BindAddress: ":8080"},
        HealthProbeBindAddress: ":8081",
        LeaderElection:         true,
        LeaderElectionID:       "database-operator-leader",
    })

    // Register the reconciler with the manager
    if err = (&controller.DatabaseReconciler{
        Client: mgr.GetClient(),
        Scheme: mgr.GetScheme(),
    }).SetupWithManager(mgr); err != nil {
        log.Fatal(err)
    }

    // Health probes (used by readiness/liveness probes in the Deployment)
    mgr.AddHealthzCheck("healthz", healthz.Ping)
    mgr.AddReadyzCheck("readyz", healthz.Ping)

    mgr.Start(ctrl.SetupSignalHandler())
}

SetupWithManager — declaring watches

The SetupWithManager method declares which resources trigger reconciliation and how:

func (r *DatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error {
    return ctrl.NewControllerManagedBy(mgr).
        For(&myorgv1.Database{}).             // primary resource to watch
        Owns(&appsv1.StatefulSet{}).          // reconcile when owned StatefulSet changes
        Owns(&corev1.Service{}).              // reconcile when owned Service changes
        Watches(                              // watch a Secret and map to owning Database
            &corev1.Secret{},
            handler.EnqueueRequestForOwner(
                mgr.GetScheme(), mgr.GetRESTMapper(),
                &myorgv1.Database{},
            ),
        ).
        WithOptions(controller.Options{
            MaxConcurrentReconciles: 3,       // parallel reconcile goroutines
        }).
        Complete(r)
}

RBAC markers — generating ClusterRole

Place // +kubebuilder:rbac markers directly above the Reconcile function. make manifests generates the ClusterRole YAML:

// +kubebuilder:rbac:groups=myorg.example.com,resources=databases,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups=myorg.example.com,resources=databases/status,verbs=get;update;patch
// +kubebuilder:rbac:groups=myorg.example.com,resources=databases/finalizers,verbs=update
// +kubebuilder:rbac:groups=apps,resources=statefulsets,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups=core,resources=services;secrets;configmaps,verbs=get;list;watch;create;update;patch;delete
func (r *DatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    // ... reconcile logic
}

Testing with envtest

Kubebuilder scaffolds envtest integration tests that spin up a real API server binary (no cluster needed) — much more reliable than mocking:

// suite_test.go — bootstrapped by kubebuilder
var _ = BeforeSuite(func() {
    testEnv = &envtest.Environment{
        CRDDirectoryPaths: []string{filepath.Join("..", "..", "config", "crd", "bases")},
    }
    cfg, _ = testEnv.Start()
    k8sClient, _ = client.New(cfg, client.Options{Scheme: scheme})
})

// database_controller_test.go — test reconciliation
It("should create a StatefulSet for a Database", func() {
    db := &myorgv1.Database{
        ObjectMeta: metav1.ObjectMeta{Name: "test-db", Namespace: "default"},
        Spec: myorgv1.DatabaseSpec{Engine: "postgres", Storage: "10Gi"},
    }
    Expect(k8sClient.Create(ctx, db)).To(Succeed())

    sts := &appsv1.StatefulSet{}
    Eventually(func() error {
        return k8sClient.Get(ctx, types.NamespacedName{
            Name: "test-db", Namespace: "default",
        }, sts)
    }, timeout, interval).Should(Succeed())
    Expect(sts.Spec.Replicas).To(Equal(ptr(int32(1))))
})

Build and deploy

# Generate CRD + RBAC manifests from markers
make generate manifests

# Build and push the operator image
make docker-build docker-push IMG=ghcr.io/myorg/database-operator:v0.1.0

# Deploy to cluster (installs CRD + controller Deployment)
make deploy IMG=ghcr.io/myorg/database-operator:v0.1.0

# Or generate plain YAML and apply manually
make build-installer IMG=ghcr.io/myorg/database-operator:v0.1.0
kubectl apply -f dist/install.yaml

🧠 Knowledge Check