🎯 What You'll Learn
- Understand what clustering is and how it differs from supervised classification
- Walk through the K-Means algorithm step by step: centroid initialization, assignment, and update
- Minimize the Within-Cluster Sum of Squares (WCSS/inertia) objective and understand why K alone doesn't minimize it
- Choose the optimal K using the elbow method and silhouette score
- Implement K-Means++ with scikit-learn and apply it to a real customer segmentation problem
1 What Is Clustering?
Every algorithm we've seen so far in Phases 1 and 2 — linear regression, decision trees, SVMs, gradient boosting — requires labeled training data. A human expert provided the answers (house prices, disease diagnoses, customer churn outcomes) and the algorithm learned to replicate them. This is supervised learning.
Clustering is unsupervised: you have a dataset with no labels at all, and your goal is to discover the hidden structure — the natural groupings — that exist within the data itself. No one tells the algorithm what a "group" is; the algorithm infers it purely from patterns in the feature space.
Clustering appears everywhere in practice:
- Customer segmentation: group customers by purchase behavior to personalize marketing campaigns
- Document clustering: group news articles or research papers by topic automatically
- Image compression: replace each pixel's color with the nearest cluster centroid, reducing color palette size
- Gene expression analysis: find groups of genes with similar expression patterns across conditions
- Anomaly detection: points that belong to no cluster (or very small clusters) are likely anomalies
Classification predicts which pre-defined category a new point belongs to — the categories were defined by the labeling process. Clustering discovers the categories themselves from unlabelled data. The clusters have no names until a human inspects them and decides what they represent. This is both the power and the challenge of unsupervised learning.
import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_blobs
# Simulate unlabelled data — three natural clusters
X, _ = make_blobs(n_samples=300, centers=3, cluster_std=0.8, random_state=42)
print(f"Dataset shape: {X.shape}") # (300, 2)
print(f"No labels — purely X")
# Visualize the raw (unlabelled) data
plt.figure(figsize=(7, 5))
plt.scatter(X[:, 0], X[:, 1], alpha=0.6, color='steelblue', edgecolors='white', linewidth=0.3)
plt.title("Unlabelled Data — Can You See the Groups?")
plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.tight_layout()
plt.savefig('raw_data.png', dpi=150)2 The K-Means Algorithm
K-Means is the most widely used clustering algorithm. Given a dataset of n points and a chosen number of clusters K, it finds K "centroids" (cluster centers) such that each data point is assigned to its nearest centroid.
The algorithm is beautifully simple — just four repeating steps:
- Choose K: Decide how many clusters you want. This is your only required input.
- Initialize centroids: Randomly place K centroid positions in the feature space (or use K-Means++, covered in Section 6).
- Assign each point to the nearest centroid: Compute the Euclidean distance from each data point to every centroid; assign the point to the centroid it is closest to.
- Update centroids: Move each centroid to the mean (average position) of all points currently assigned to it.
- Repeat steps 3–4 until no point changes its cluster assignment (convergence).
import numpy as np
def euclidean_distance(a, b):
return np.sqrt(np.sum((a - b) ** 2))
def kmeans_from_scratch(X, K, max_iters=100, random_state=42):
"""Minimal K-Means implementation to illustrate the algorithm."""
np.random.seed(random_state)
n_samples, n_features = X.shape
# Step 2: randomly choose K data points as initial centroids
idx = np.random.choice(n_samples, K, replace=False)
centroids = X[idx].copy()
labels = np.zeros(n_samples, dtype=int)
for iteration in range(max_iters):
# Step 3: assign each point to nearest centroid
new_labels = np.array([
np.argmin([euclidean_distance(x, c) for c in centroids])
for x in X
])
# Check convergence
if np.all(new_labels == labels):
print(f"Converged after {iteration} iterations")
break
labels = new_labels
# Step 4: update centroids to cluster mean
for k in range(K):
mask = labels == k
if mask.sum() > 0:
centroids[k] = X[mask].mean(axis=0)
return labels, centroids
# Test on synthetic data
from sklearn.datasets import make_blobs
X, y_true = make_blobs(n_samples=300, centers=3, cluster_std=0.8, random_state=42)
labels, centroids = kmeans_from_scratch(X, K=3)
print(f"Cluster sizes: {np.bincount(labels)}")
print(f"Centroids:\n{centroids.round(3)}")
# Converged after 5 iterations
# Cluster sizes: [100 100 100] (approximately)K-Means always converges because WCSS (see Section 3) decreases or stays the same at every step and is bounded below by 0. However, it may converge to a local minimum — a suboptimal assignment that depends on the random initialization. This is why sklearn runs K-Means multiple times with different seeds (n_init=10) and keeps the best result.
Watch It Run: K-Means Step by Step
K-Means is not a one-shot calculation — it's an iterative process that alternates between two steps until the assignments stop changing. The simulator below runs the exact algorithm from the code above on a small synthetic dataset with natural-looking groups. Choose a number of clusters K, then click Step repeatedly to watch centroids (the large ✕ markers) jump to the mean of their assigned points, and watch points switch color as they get reassigned to a closer centroid. Click Reset to re-randomize the starting centroids, or Run to Convergence to fast-forward to the final result.
Iteration 0 — initial centroids placed (K-Means++ style seeding). Click Step to assign points.
3 The Objective Function: WCSS
K-Means is not just a heuristic — it is formally minimizing a mathematical objective called the Within-Cluster Sum of Squares (WCSS), also called inertia:
WCSS = Σₖ Σₓ∈Cₖ ‖x − μₖ‖²
Where Cₖ is the set of points in cluster k and μₖ is the centroid (mean) of cluster k. In plain English: for every point in every cluster, compute the squared distance to its cluster's centroid, then sum them all up. Lower WCSS means tighter, more compact clusters — a better solution.
import numpy as np
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
X, _ = make_blobs(n_samples=300, centers=3, cluster_std=0.8, random_state=42)
# Fit K-Means with K=3
model = KMeans(n_clusters=3, init='k-means++', n_init=10, random_state=42)
model.fit(X)
print(f"Cluster labels (first 10): {model.labels_[:10]}")
print(f"Cluster centroids:\n{model.cluster_centers_.round(3)}")
print(f"WCSS (inertia): {model.inertia_:.2f}")
# Manually verify inertia
wcss = 0
for k in range(3):
mask = model.labels_ == k
cluster_points = X[mask]
centroid = model.cluster_centers_[k]
wcss += np.sum((cluster_points - centroid) ** 2)
print(f"Manual WCSS: {wcss:.2f}") # should match model.inertia_
# WCSS decreases as K increases — always!
print("\nWCSS vs K:")
for k in range(1, 8):
km = KMeans(n_clusters=k, n_init=10, random_state=42)
km.fit(X)
print(f" K={k}: WCSS={km.inertia_:.1f}")K=n (one cluster per point) gives WCSS=0 — perfect, but useless. You cannot choose K simply by minimizing WCSS. You need an external criterion — the elbow method or silhouette score — to find a meaningful K that balances compactness with simplicity.
4 Choosing K: The Elbow Method
The elbow method is the most intuitive approach for choosing K. The idea: plot WCSS against K for a range of values (say 1 to 15). As K increases, WCSS drops steeply at first — each new cluster genuinely helps. Once K exceeds the true number of clusters, adding more clusters gives diminishing returns. The plot "elbows" at the optimal K.
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
# Dataset with 4 natural clusters
X, _ = make_blobs(n_samples=400, centers=4, cluster_std=0.9, random_state=42)
# Compute WCSS for K = 1 to 15
wcss_values = []
K_range = range(1, 16)
for k in K_range:
km = KMeans(n_clusters=k, init='k-means++', n_init=10, random_state=42)
km.fit(X)
wcss_values.append(km.inertia_)
print(f"K={k:2d}: WCSS={km.inertia_:8.1f}")
# Plot the elbow curve
plt.figure(figsize=(9, 5))
plt.plot(K_range, wcss_values, 'bo-', markersize=8, linewidth=2)
plt.axvline(x=4, color='red', linestyle='--', alpha=0.7, label='True K=4')
plt.xlabel('Number of Clusters (K)', fontsize=12)
plt.ylabel('WCSS (Inertia)', fontsize=12)
plt.title('Elbow Method — Choosing Optimal K', fontsize=14)
plt.xticks(K_range)
plt.legend()
plt.grid(alpha=0.3)
plt.tight_layout()
plt.savefig('elbow_method.png', dpi=150)
# The "elbow" at K=4 — biggest rate of change drops off after K=4
wcss_array = np.array(wcss_values)
second_derivative = np.diff(np.diff(wcss_array))
elbow_k = np.argmax(second_derivative) + 2 # +2 for offset
print(f"\nElbow detected at K = {elbow_k}")Real-world data rarely has a sharp elbow. If the elbow plot looks like a smooth curve, use silhouette score (Section 5) as a second opinion. In practice, try 2–3 candidate K values from the elbow region, run silhouette analysis on each, and pick the one with the highest average silhouette score.
Here is the actual elbow curve computed on the same synthetic dataset used in the simulator above (run K-Means to convergence for each K from 1 to 8, recording the final WCSS). Notice how steeply WCSS drops for the first few values of K, then flattens out — the "elbow" marks the point of diminishing returns:
WCSS (inertia) vs K for the lesson's synthetic dataset. The elbow is marked in red.
5 Choosing K: Silhouette Score
The silhouette score is a more principled metric that measures how well each point fits its assigned cluster compared to the next-best cluster. For a single point i:
- a(i) = mean distance from point i to all other points in the same cluster (intra-cluster cohesion)
- b(i) = mean distance from point i to all points in the nearest different cluster (inter-cluster separation)
- s(i) = (b(i) − a(i)) / max(a(i), b(i))
Silhouette scores range from -1 to +1: +1 means the point is perfectly placed in its cluster; 0 means it's on the boundary; -1 means it's likely in the wrong cluster. The overall silhouette score is the average over all points.
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score, silhouette_samples
from sklearn.datasets import make_blobs
X, _ = make_blobs(n_samples=400, centers=4, cluster_std=0.9, random_state=42)
# Silhouette score for K = 2 to 10
print(f"{'K':>4} {'Silhouette Score':>18} {'WCSS':>12}")
print("-" * 40)
silhouette_scores = []
for k in range(2, 11):
km = KMeans(n_clusters=k, init='k-means++', n_init=10, random_state=42)
labels = km.fit_predict(X)
sil_score = silhouette_score(X, labels)
silhouette_scores.append(sil_score)
print(f"K={k:2d} {sil_score:>18.4f} {km.inertia_:>12.1f}")
# K=4 should show the highest silhouette score
best_k = np.argmax(silhouette_scores) + 2
print(f"\nBest K by silhouette: {best_k}")
# Plot silhouette scores
plt.figure(figsize=(9, 5))
plt.plot(range(2, 11), silhouette_scores, 'go-', markersize=8, linewidth=2)
plt.axvline(x=best_k, color='red', linestyle='--', alpha=0.7, label=f'Best K={best_k}')
plt.xlabel('Number of Clusters (K)', fontsize=12)
plt.ylabel('Average Silhouette Score', fontsize=12)
plt.title('Silhouette Analysis for Choosing K', fontsize=14)
plt.xticks(range(2, 11))
plt.legend()
plt.grid(alpha=0.3)
plt.tight_layout()
plt.savefig('silhouette_scores.png', dpi=150)
# Per-sample silhouette values to diagnose cluster quality
km_best = KMeans(n_clusters=best_k, init='k-means++', n_init=10, random_state=42)
labels_best = km_best.fit_predict(X)
sample_silhouette = silhouette_samples(X, labels_best)
for k in range(best_k):
cluster_sil = sample_silhouette[labels_best == k]
print(f"Cluster {k}: mean silhouette = {cluster_sil.mean():.4f}, "
f"n_points = {(labels_best == k).sum()}")6 K-Means++ Initialization
Standard K-Means chooses initial centroids uniformly at random from the data points. This can lead to poor local minima — for example, if two initial centroids land in the same natural cluster, the algorithm may never recover the true structure.
K-Means++ (Arthur & Vassilvitskii, 2007) uses a smarter initialization: each new centroid is chosen with probability proportional to its squared distance from the nearest already-chosen centroid. Points far from existing centroids are more likely to be chosen. This spreads the initial centroids across the data, dramatically reducing the chance of bad local minima.
import numpy as np
import time
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
# Difficult dataset: many clusters, some tight, some loose
X, _ = make_blobs(n_samples=2000, centers=8, cluster_std=1.2, random_state=42)
results = []
for init_method in ['random', 'k-means++']:
wcss_list = []
times = []
for seed in range(20):
t0 = time.time()
km = KMeans(
n_clusters=8,
init=init_method,
n_init=1, # single init to measure variance
max_iter=300,
random_state=seed
)
km.fit(X)
wcss_list.append(km.inertia_)
times.append(time.time() - t0)
results.append({
'init': init_method,
'mean_wcss': np.mean(wcss_list),
'std_wcss': np.std(wcss_list),
'min_wcss': np.min(wcss_list),
'mean_time': np.mean(times)
})
print(f"\n{init_method}:")
print(f" Mean WCSS: {np.mean(wcss_list):.1f} ± {np.std(wcss_list):.1f}")
print(f" Best WCSS: {np.min(wcss_list):.1f}")
print(f" Worst WCSS: {np.max(wcss_list):.1f}")
# K-Means++ production usage (recommended defaults)
km_best = KMeans(
n_clusters=8,
init='k-means++', # smarter initialization (default in sklearn)
n_init=10, # run 10 times, keep best result
max_iter=300, # maximum iterations per run
tol=1e-4, # convergence tolerance
random_state=42
)
km_best.fit(X)
print(f"\nProduction KMeans++ (n_init=10): WCSS = {km_best.inertia_:.1f}")Since sklearn 0.24, init='k-means++' is the default. Unless you have a specific reason to use random initialization, always keep this default. The theoretical guarantee of K-Means++ is that its expected WCSS is at most O(log K) times the optimal WCSS — much better than random.
7 Full Implementation with Scikit-learn
Here is a complete, production-quality K-Means workflow: preprocessing, fitting, evaluating, and visualizing clusters.
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import silhouette_score
from sklearn.datasets import make_blobs
# ── Step 1: Prepare data ──
X, y_true = make_blobs(n_samples=500, centers=4, cluster_std=0.8, random_state=42)
# Step 2: Scale features (CRITICAL — K-Means uses Euclidean distance)
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
# ── Step 3: Fit K-Means ──
km = KMeans(
n_clusters=4,
init='k-means++',
n_init=10,
random_state=42
)
km.fit(X_scaled)
# ── Step 4: Inspect results ──
print("=== K-Means Results ===")
print(f"Labels (first 10): {km.labels_[:10]}")
print(f"Inertia (WCSS): {km.inertia_:.4f}")
print(f"Iterations to conv: {km.n_iter_}")
print(f"Cluster sizes: {np.bincount(km.labels_)}")
print(f"Silhouette score: {silhouette_score(X_scaled, km.labels_):.4f}")
# ── Step 5: Centroids in original feature space ──
centroids_original = scaler.inverse_transform(km.cluster_centers_)
print(f"\nCentroids (original scale):\n{centroids_original.round(3)}")
# ── Step 6: Predict cluster for new data ──
X_new = np.array([[0.5, 1.2], [-3.1, 2.8]])
X_new_scaled = scaler.transform(X_new)
new_labels = km.predict(X_new_scaled)
print(f"\nNew point clusters: {new_labels}")
# ── Step 7: Visualize ──
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
colors = ['#e41a1c', '#377eb8', '#4daf4a', '#984ea3']
# True clusters
for k in range(4):
mask = y_true == k
axes[0].scatter(X[mask, 0], X[mask, 1], c=colors[k], alpha=0.6, label=f'True {k}')
axes[0].set_title('True Clusters (for reference)')
axes[0].legend()
# K-Means clusters
for k in range(4):
mask = km.labels_ == k
axes[1].scatter(X_scaled[mask, 0], X_scaled[mask, 1], c=colors[k], alpha=0.6, label=f'Cluster {k}')
axes[1].scatter(km.cluster_centers_[:, 0], km.cluster_centers_[:, 1],
c='black', marker='X', s=200, zorder=5, label='Centroids')
axes[1].set_title('K-Means Clusters (scaled space)')
axes[1].legend()
plt.tight_layout()
plt.savefig('kmeans_result.png', dpi=150)8 Limitations of K-Means
K-Means works beautifully on globular, well-separated clusters of similar size. It struggles — and fails badly — in several common real-world situations:
| Limitation | Why it fails | Solution |
|---|---|---|
| Non-convex shapes (crescents, rings) | Assigns by Euclidean distance to centroid — can't follow curved manifolds | DBSCAN (Lesson 29) |
| Different cluster sizes | Centroid-based assignment favours equal-sized clusters | Gaussian Mixture Models (Lesson 30) |
| Different cluster densities | Dense small cluster may be split to balance another large cluster | DBSCAN, HDBSCAN (Lesson 29) |
| Unscaled features | Large-scale features dominate Euclidean distance | Always use StandardScaler first |
| Outliers | Outliers get absorbed into clusters and pull centroids away from true centers | Remove outliers first; or use K-Medoids |
| K must be specified | No automatic K discovery | DBSCAN, HDBSCAN (no K needed) |
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.datasets import make_moons, make_circles
fig, axes = plt.subplots(2, 2, figsize=(12, 10))
datasets = [
('Moons (non-convex)', make_moons(n_samples=300, noise=0.05, random_state=42)[0]),
('Circles (concentric)', make_circles(n_samples=300, noise=0.05, factor=0.5, random_state=42)[0]),
]
for row, (name, X_fail) in enumerate(datasets):
# K-Means result
km = KMeans(n_clusters=2, n_init=10, random_state=42)
labels = km.fit_predict(X_fail)
colors = ['#e41a1c' if l == 0 else '#377eb8' for l in labels]
axes[row, 0].scatter(X_fail[:, 0], X_fail[:, 1], c=colors, alpha=0.7)
axes[row, 0].scatter(km.cluster_centers_[:, 0], km.cluster_centers_[:, 1],
c='black', marker='X', s=200, zorder=5)
axes[row, 0].set_title(f'K-Means on {name} — FAILS')
# Show what K-Means "should" find
# (correct assignment using true labels for illustration)
from sklearn.datasets import make_moons, make_circles
if row == 0:
_, y_true = make_moons(n_samples=300, noise=0.05, random_state=42)
else:
_, y_true = make_circles(n_samples=300, noise=0.05, factor=0.5, random_state=42)
colors_true = ['#e41a1c' if l == 0 else '#377eb8' for l in y_true]
axes[row, 1].scatter(X_fail[:, 0], X_fail[:, 1], c=colors_true, alpha=0.7)
axes[row, 1].set_title(f'True Structure — what we want')
plt.suptitle("K-Means Fails on Non-Convex Shapes → Use DBSCAN Instead", fontsize=13)
plt.tight_layout()
plt.savefig('kmeans_limitations.png', dpi=150)Real-World Spotlight: Customer Segmentation with RFM Analysis
One of the most impactful applications of K-Means in business is customer segmentation using RFM analysis: Recency (days since last purchase), Frequency (number of purchases in a period), and Monetary value (total spend). These three features capture a customer's engagement and value in a compact, interpretable way.
import numpy as np
import pandas as pd
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import silhouette_score
import matplotlib.pyplot as plt
np.random.seed(42)
n = 2000
# Simulate RFM data for 2000 customers
df = pd.DataFrame({
'customer_id': range(n),
'recency': np.concatenate([
np.random.randint(1, 15, 500), # Champions: bought very recently
np.random.randint(60, 120, 500), # At Risk: haven't bought in a while
np.random.randint(1, 30, 500), # New: just joined
np.random.randint(120, 365, 500), # Hibernating: not seen in months
]),
'frequency': np.concatenate([
np.random.randint(20, 50, 500), # Champions: buy often
np.random.randint(5, 15, 500), # At Risk: used to buy often
np.random.randint(1, 5, 500), # New: few purchases
np.random.randint(1, 5, 500), # Hibernating: rarely bought
]),
'monetary': np.concatenate([
np.random.uniform(500, 2000, 500), # Champions: high spenders
np.random.uniform(200, 800, 500), # At Risk: medium
np.random.uniform(50, 300, 500), # New: low (just starting)
np.random.uniform(10, 200, 500), # Hibernating: low
]).round(2)
})
print("RFM Dataset:")
print(df[['recency', 'frequency', 'monetary']].describe().round(2))
# ── Scale features ──
features = ['recency', 'frequency', 'monetary']
scaler = StandardScaler()
X_rfm = scaler.fit_transform(df[features])
# ── Choose K with elbow + silhouette ──
wcss, sil_scores = [], []
K_range = range(2, 9)
for k in K_range:
km = KMeans(n_clusters=k, init='k-means++', n_init=10, random_state=42)
labels = km.fit_predict(X_rfm)
wcss.append(km.inertia_)
sil_scores.append(silhouette_score(X_rfm, labels))
print("\nK WCSS Silhouette")
for k, w, s in zip(K_range, wcss, sil_scores):
marker = " ← best" if s == max(sil_scores) else ""
print(f"K={k}: {w:8.1f} {s:.4f}{marker}")
# ── Fit optimal K=4 ──
km_final = KMeans(n_clusters=4, init='k-means++', n_init=10, random_state=42)
df['segment'] = km_final.fit_predict(X_rfm)
# ── Interpret segments by computing cluster means ──
segment_profiles = df.groupby('segment')[features].mean().round(1)
segment_profiles['count'] = df.groupby('segment').size()
print("\nSegment Profiles (original scale):")
print(segment_profiles.sort_values('recency'))
# ── Name the segments ──
# Low recency + high frequency + high monetary → Champions
# High recency + medium everything → At Risk
# Low recency + low frequency → New Customers
# Very high recency + low everything → Hibernating
segment_names = {
segment_profiles['recency'].idxmin(): 'Champions',
segment_profiles['monetary'].idxmax(): 'High Value',
segment_profiles['recency'].idxmax(): 'Hibernating',
}
# Fill remaining
for s in range(4):
if s not in segment_names:
segment_names[s] = 'New / Growing'
df['segment_name'] = df['segment'].map(segment_names)
print("\nSegment distribution:")
print(df['segment_name'].value_counts())
# ── Visualize: Recency vs Monetary, colored by segment ──
colors = {'Champions': '#2ecc71', 'High Value': '#3498db',
'New / Growing': '#f39c12', 'Hibernating': '#e74c3c'}
plt.figure(figsize=(9, 6))
for seg, grp in df.groupby('segment_name'):
plt.scatter(grp['recency'], grp['monetary'], label=seg,
alpha=0.4, c=colors.get(seg, 'gray'), s=20)
plt.xlabel('Recency (days since last purchase)')
plt.ylabel('Monetary Value ($)')
plt.title('Customer Segments — RFM K-Means (K=4)')
plt.legend()
plt.tight_layout()
plt.savefig('rfm_segments.png', dpi=150)
# ── Business actions per segment ──
actions = {
'Champions': 'Reward with loyalty program; ask for reviews',
'High Value': 'Upsell premium products; priority support',
'New / Growing': 'Onboarding emails; first-purchase incentives',
'Hibernating': 'Win-back campaign: "We miss you" 30% discount'
}
print("\nMarketing Actions:")
for seg, action in actions.items():
print(f" {seg}: {action}")This pipeline transforms raw transaction logs into actionable customer intelligence. The "Champions" segment typically represents 15–20% of customers but 50–60% of revenue — making it critical to retain them. The "Hibernating" segment is the prime win-back opportunity. Without clustering, these segments would require manual rule-creation by a business analyst; K-Means discovers them automatically from the data.
✍️ Practice Exercises
- Load the Iris dataset (4 features, 3 known classes). Apply K-Means with K=3 after scaling. Compare cluster assignments to the true species labels using
adjusted_rand_scorefromsklearn.metrics. How well did K-Means recover the true structure? - Run the elbow method on the Iris dataset (K=1 to 10). Does the elbow clearly indicate K=3? Then run silhouette analysis — does it agree?
- Use
make_moons(n_samples=300, noise=0.05)to create a crescent dataset. Apply K-Means with K=2 and visualize the result. Explain in one paragraph why K-Means fails here. - Implement mini-batch K-Means using
sklearn.cluster.MiniBatchKMeanson a dataset of 100,000 points. Compare WCSS and runtime against standardKMeans. What trade-off are you making?
▶ Hints
from sklearn.datasets import load_iris
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import adjusted_rand_score, silhouette_score
iris = load_iris()
X_iris = StandardScaler().fit_transform(iris.data)
km = KMeans(n_clusters=3, init='k-means++', n_init=10, random_state=42)
labels = km.fit_predict(X_iris)
print(f"ARI: {adjusted_rand_score(iris.target, labels):.4f}")
# ARI close to 1.0 = perfect recovery; 0 = random; negative = worse than random📚 Primary Source for This Lesson
scikit-learn: K-Means Clustering
The official guide covers the algorithm, initialization methods, mini-batch variant, and complexity analysis. For the K-Means++ initialization paper, see Arthur & Vassilvitskii (2007) "k-means++: The Advantages of Careful Seeding" (SODA 2007). For silhouette analysis, see Rousseeuw (1987) "Silhouettes: A graphical aid to the interpretation and validation of cluster analysis."