Section 2.7: Describing and Visualizing Data

A cleaned table still has to be understood before a model is fitted to it. This section covers describing a column in numbers, its center, its spread, its shape and how it moves with another column, and then in charts: distributions, relationships between two columns, and views of many columns at once.

Section 2.6 built features and summaries, and a summary is only as honest as the statistic chosen for it: a mean pulled by two extreme listings, a correlation driven by one outlier, a histogram whose bins were picked to tell a story. This section checks the numbers and the pictures against each other, on the kind of table Module 3 fits its first models to.

Lessons

  1. Center, Spread, Shape and Correlation - the statistics that describe a column and a pair of columns, and what each one hides.
  2. Matplotlib and Seaborn - Statistical and Multivariate Views - installing both libraries, then histograms, box plots, scatter plots, heatmaps and pairplots, each proved by what the figure holds.

By the end you will be able to:

  • Choose the center and the spread that describe a skewed column honestly, and explain why NumPy and pandas disagree on a standard deviation.
  • Measure skew and correlation, and tell Pearson from Spearman by what one outlier does to each.
  • Build and save charts with matplotlib's Figure and Axes, and keep each chart on the figure you meant.
  • Compare distributions, relationships and many columns at once with seaborn's statistical views.