Section 3.8: Structure Without a Target

Every model so far learned from a column it was asked to predict. This section's two methods take no target. k-means finds groups among the listings, and PCA (principal component analysis, a rotation of the centered columns onto the directions they vary along most) finds directions that several columns move along together, on standardized columns each an eigenvector (a direction a matrix only stretches) of their correlation matrix. With no held-out price to score them, the section ends on judging what they found: whether the groups survive another seed, how cleanly they separate, and what a direction's weights mean. Three lessons, no install.

Unsupervised methods fail as quietly as anything earlier in the module. k-means on unscaled columns splits the listings by price alone, inertia (the sum of every listing's squared distance to its own center) falls at every k and so its smallest value never picks one, and a single run at the default tolerance can stop before its centers settle, on a grouping another seed does not return. A silhouette says how cleanly groups separate and never whether they are useful: here two clusters, mostly apartments against houses and townhouses, score 0.347, and every k from three to eight scores between 0.219 and 0.251. PCA on unscaled columns returns the price as its first component, a component's sign means nothing on its own, and a share of the variance is not a share of what predicts a price.

Lessons

  1. k-means Clustering - learning without a target, the assign-and-move loop by hand, KMeans in a pipeline behind an imputer and a scaler, inertia and why its smallest value cannot choose k, and centers read back in listing units.
  2. Eigenvectors, and What PCA Is Doing - directions a matrix only stretches, the correlation matrix of standardized columns, np.linalg.eigh, PCA as a rotation onto the eigenvectors, and what keeping fewer components keeps.
  3. Interpreting Unsupervised Structure and Dimensionality Reduction - whether clusters hold across seeds, what a silhouette says, what a component's loadings mean, PCA inside a price model, and what structure without a target cannot tell you.

By the end you will be able to:

  • Fit k-means in a pipeline, run its loop by hand, and read its centers in listing units.
  • Compute the eigenvectors of a correlation matrix with NumPy, and show that PCA rotates the standardized columns onto them.
  • Check a clustering across seeds and with a silhouette before trusting it, and read a component through its loadings.
  • Fit PCA inside a pipeline before a model, and say why the directions of most variance are not the ones that predict best.