Matplotlib and Seaborn - Statistical and Multivariate Views

Context

matplotlib draws a chart as a Figure, the whole image, holding one or more Axes, each a plotting area with its own scales, and seaborn draws statistical charts onto those Axes straight from a DataFrame's columns. A script saves the finished Figure as an image file, which you open like any other file.

Charts answer what a summary number cannot: whether a target has one hump or two, which listings sit outside the box, whether a relationship bends, and whether a category shifts it. You draw the target's distribution before choosing a transform, each feature against the target before trusting its coefficient, and the correlation matrix before dropping a column, and Modules 3 and 4 draw residuals, learning curves (a model's score plotted against how much data it trained on) and loss curves on the same Figure and Axes.

A chart fails without a sound. A histogram drops missing values without a word, so the count you report is not the count drawn, a bin count can hide a second hump, a color scale left to float paints a coefficient near 0 in the strongest color the map has, a category changes color between two charts meant to be compared, and a call through plt draws on whichever figure was created last. Lesson 2.7.1 measured the table in numbers, and this lesson draws it, counting what its charts drew.

Goal

• Install matplotlib and seaborn into (bootcamp), and save every chart to a file with savefig.
• Build a chart on an explicit Figure and Axes, label it, give it a legend, and predict which figure a plt call draws on.
• Draw histograms and box plots that report how many values they drew, and read a box plot's whiskers and points against 2.5.4's fences.
• Draw scatter plots with hue, alpha and a log scale, and keep each category's color fixed across charts with hue_order.
• Draw a correlation heatmap on a fixed diverging scale, a pairplot, and a grid of Axes from plt.subplots.

Prerequisites

Optional. The lesson stands on its own, and these show the parts of a chart before you build one.

• YouTube - "Matplotlib Tutorial (Part 1): Creating and Customizing Our First Plots" by Corey Schafer. A line chart built and customized from scratch, a first look at what the plotting calls produce.
• Official docs - matplotlib, "Quick start guide", "Parts of a Figure". One labeled picture of the Figure, the Axes and every part drawn on them.

Table of Contents

• Installing matplotlib and seaborn
• Figure and Axes
• Distribution Views - Histogram and Box Plot
• Relationship Views - Scatter and hue
• Multivariate Views - Heatmap and Pairplot

Theory

After the install, every block from Figure and Axes on adds to one script, charts.py in ~/dev/bootcamp, run with python3 charts.py, except wrong_figure.py, which stands on its own as the install's show_check.py does.

Installing matplotlib and seaborn

What. matplotlib is the plotting library Python's data tools draw with, and seaborn is a library built on it that draws statistical charts from a DataFrame's columns. Neither ships with Python. Why. pandas and NumPy compute, and draw nothing without matplotlib. The charts your scripts draw from here on, Module 3's residual plots and Module 4's loss curves included, are matplotlib figures. When. Once, with (bootcamp) active, before the first chart.

With (bootcamp) active, this installs both pinned versions and what they need, through the uv pip that Module 2's setup page explained:

uv pip install matplotlib==3.11.1 seaborn==0.13.2

uv prints a few progress lines and then one line per library it added:

 + contourpy==1.4.0
 + cycler==0.12.1
 + fonttools==4.67.0
 + kiwisolver==1.5.1
 + matplotlib==3.11.1
 + packaging==26.3
 + pillow==12.3.0
 + pyparsing==3.3.3
 + seaborn==0.13.2

matplotlib and seaborn arrive at exactly the pinned versions. The other seven are libraries matplotlib needs, at whichever release is newest on the day you install, so their numbers can differ on your machine. numpy and pandas are already in (bootcamp) from Lessons 2.2.1 and 2.4.1, so uv leaves them as they are.

# 3.11.1 0.13.2
python3 -c "import matplotlib, seaborn; print(matplotlib.__version__, seaborn.__version__)"

Scripts import them under their conventional names: import matplotlib.pyplot as plt for matplotlib's plotting interface, and import seaborn as sns.

A chart lives in memory until it is saved. fig.savefig("name.png") writes it to a PNG file beside the script, which you open from VS Code's file explorer as Module 2's setup page described, on any machine. plt.show() is meant to show a figure in a window, and what it does depends on the machine, so run it once as show_check.py:

import matplotlib.pyplot as plt

fig, ax = plt.subplots()
ax.plot([1, 2, 3, 4], [0.9, 0.6, 0.45, 0.4])
plt.show()
print("after show")  # after show

In a terminal with no display the script prints after show and nothing else: plt.show() returned at once, showed nothing and warned about nothing. On WSL or Linux with a display, what happens depends on whether matplotlib finds a window toolkit (the library a program draws its windows with) it can load in (bootcamp): finding none, as on the machine that ran this lesson, plt.show() prints UserWarning: FigureCanvasAgg is non-interactive, and thus cannot be shown and returns at once, and finding one, it opens a window and holds the script until you close it. Saving behaves the same everywhere, so this module saves its charts with savefig, and no script after this one calls plt.show().

DON'T:

• DON'T end a script that has to run unattended with plt.show(). Without a display, or without a window toolkit it can load, it shows nothing, and wherever it does open a window it holds the script until someone closes it.
• DON'T call plt.savefig after a plt.show() that opened a window. Closing the window closes the figure, and help(plt.show) says the save that follows writes a new, empty figure.

Figure and Axes

What. A Figure is the whole image, its size and everything on it, and an Axes is one plotting area inside it, with its own x and y scales, ticks, labels and the data drawn against them. plt.subplots() creates one Figure holding one Axes and returns both. Why. Every setting belongs to one of the two: a line, a label, a scale or a legend to an Axes, the size and the saved file to the Figure. Naming both is what keeps each line and label on the chart you meant once a script holds several. When. Start every chart with fig, ax = plt.subplots(...), draw and label through ax, save through fig, and close it once it is saved, except the pairplot of the last subtitle, which builds its own figure.

The first block of charts.py loads the table without its repeated rows, as Lesson 2.7.1 did, and parses the listing dates with 2.5.3's to_datetime:

import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns

df = pd.read_csv("housing.csv").drop_duplicates()
df["listing_date"] = pd.to_datetime(df["listing_date"])

# width and height in inches
fig, ax = plt.subplots(figsize=(8, 5))
print(type(fig).__name__)     # Figure
print(type(ax).__name__)      # Axes
print(fig.get_size_inches())  # [8. 5.]

The chart is the number of listings per month, one line per year:

for year in [2023, 2024]:
    dates = df.loc[df["listing_date"].dt.year == year, "listing_date"]
    # one count per month, sort_index() puts the months in order
    per_month = dates.dt.month.value_counts().sort_index()
    # label= names the line for the legend and draws nothing itself
    ax.plot(per_month.index, per_month.to_numpy(), marker="o", label=str(year))

ax.set_xlabel("Month")
ax.set_ylabel("Listings")
ax.set_title("Listings per month")
ax.legend()
fig.savefig("listings_per_month.png", dpi=150, bbox_inches="tight")

labels = [t.get_text() for t in ax.get_legend().get_texts()]
# a line's y values are its monthly counts
drawn = sum([int(line.get_ydata().sum()) for line in ax.lines])
print(len(ax.lines))    # 2
print(labels)           # ['2023', '2024']
print(ax.get_xlabel())  # Month
print(drawn, len(df))   # 201 204
plt.close(fig)

ax.plot draws a line through the points it is given, and ax.lines holds every line drawn on that Axes, which is how a script proves what a chart contains: the two lines count 201 of the 204 listings, because the three with no listing date belong to no year and dropped out without a word. ax.legend() collects every line that has a label= into the key. dpi=150 sets the saved image's resolution in dots per inch, and bbox_inches="tight" trims the empty margin around the chart so nothing at its edge is cut off. plt.close(fig) releases the figure, because pyplot keeps every figure it creates open until it is closed.

matplotlib also offers plt.plot, plt.title and plt.xlabel, which act on whichever Axes is current, the last one pyplot created, and drawing on another Axes through its own methods does not change which one that is. Saved as wrong_figure.py, two loss curves meant for one chart:

import matplotlib.pyplot as plt

epochs = [1, 2, 3, 4]
fig_a, ax_a = plt.subplots()
ax_a.plot(epochs, [0.9, 0.6, 0.45, 0.4], label="train loss")
fig_b, ax_b = plt.subplots()
# meant for fig_a, it lands on the current Axes
plt.plot(epochs, [1.0, 0.8, 0.7, 0.72], label="validation loss")

print(len(ax_a.lines), len(ax_b.lines))  # 1 1
print(plt.gca() is ax_b)                 # True
fig_a.savefig("loss_curves.png")
plt.close("all")

plt.gca() returns the current Axes, and after the second plt.subplots it is ax_b. The validation curve landed on a figure nobody saved, loss_curves.png holds one line instead of two, and nothing warned.

DON'T:

• DON'T mix plt. drawing calls with ax calls in a script that holds more than one figure. plt.plot drew the validation curve on the newest figure, and the saved chart lost it without an error.
• DON'T call ax.legend() on lines drawn without label=. It draws an empty legend and warns UserWarning: No artists with labels found to put in legend. Note that artists whose label start with an underscore are ignored when legend() is called with no argument., and the script carries on.

Distribution Views - Histogram and Box Plot

What. A histogram cuts one numeric column's range into equal-width bins and draws a bar per bin, its height the number of values that fall in it. A box plot draws a box from Q1 to Q3 with a line at the median, whiskers out to the last values inside 2.5.4's 1.5 x IQR fences, and every value beyond them as a point of its own. Why. They show what 2.7.1's numbers compressed: where the values pile up, whether there is a second hump, and which listings sit outside the fences, group by group. When. A histogram for the shape of one numeric column, and a box plot to compare one column's spread across the groups of a category, such as price per neighborhood.

price = df["price"]
fig, ax = plt.subplots(figsize=(8, 5))
counts, edges, bars = ax.hist(price, bins=30)
plt.close(fig)

print(int(counts.sum()), len(price))  # 185 204
print(int((counts > 0).sum()))        # 4
print(int(counts[0]))                 # 175
print(round(edges[1] - edges[0]))     # 325333

ax.hist returns the bar heights, the bin edges and the drawn bars. Handed the price column with its 19 gaps, it drew 185 values and said nothing about the rest, so the count to report is what the bars add up to, not the row count. The two listings above a million also stretch each of the 30 bins to 325,333 wide, which puts 175 prices in the first bar and leaves four bars with any height.

On the 183 typical prices, the bin count decides what the chart says:

typical = price[price < 1_000_000]
for bins in [4, 12]:
    fig, ax = plt.subplots(figsize=(8, 5))
    counts, edges, bars = ax.hist(typical, bins=bins)
    ax.set_xlabel("Price")
    ax.set_ylabel("Listings")
    fig.savefig(f"price_hist_{bins}.png", dpi=150, bbox_inches="tight")
    plt.close(fig)
    print(bins, counts.astype(int).tolist())

# the edges of the 12-bin chart's ninth bar
print(round(edges[8]), round(edges[9]))

4 [84, 42, 38, 19]
12 [13, 37, 34, 18, 14, 10, 8, 11, 19, 9, 5, 5]
387333 418250

Four bins fall steadily from 84 to 19, one hump with a tail. Twelve bins show a second rise, 19 listings between 387,333 and 418,250, where Lakeview's and Eastgate's prices sit in the box plot below, and four bins averaged it into their wide bars. Neither count is neutral, so choose one on purpose and say which.

seaborn draws the same chart from a DataFrame and a column name:

fig, ax = plt.subplots(figsize=(8, 5))
returned = sns.histplot(data=df, x="price", ax=ax)
heights = [p.get_height() for p in ax.patches]
print(returned is ax)                   # True
print(len(heights), int(sum(heights)))  # 28 185
plt.close(fig)

sns.histplot draws on the Axes passed as ax= and returns that same Axes, so labeling and saving stay matplotlib's job. ax.patches holds its bars. It drops the 19 gaps without a word too, and picks its own bin count from the data, 28 here.

A box plot of price per neighborhood, drawn on the listings under a million because the two above would stretch the axis past 9,900,000:

typical_rows = df[df["price"] < 1_000_000]
# the distinct neighborhood names, in alphabetical order
hoods = sorted(typical_rows["neighborhood"].unique())
fig, ax = plt.subplots(figsize=(10, 5))
sns.boxplot(data=typical_rows, x="neighborhood", y="price", order=hoods, ax=ax)
fig.savefig("price_by_neighborhood.png", dpi=150, bbox_inches="tight")

# seaborn keeps each box's parts in ax.containers
box = ax.containers[0]
fliers = [len(f.get_ydata()) for f in box.fliers]
print(fliers)                                # [2, 0, 0, 1, 1, 2, 0]
print(box.fliers[0].get_ydata().tolist())    # [146000.0, 172000.0]
print(box.whiskers[0].get_ydata().tolist())  # [238250.0, 194000.0]
print(box.whiskers[1].get_ydata().tolist())  # [268250.0, 295000.0]
plt.close(fig)

dt = typical_rows.loc[typical_rows["neighborhood"] == "Downtown", "price"]
q1, q3 = dt.quantile(0.25), dt.quantile(0.75)
low_fence, high_fence = q1 - 1.5 * (q3 - q1), q3 + 1.5 * (q3 - q1)
print(low_fence, high_fence)                 # 193250.0 313250.0

order= names the order of the boxes. Without it they come in the order each neighborhood first appears in the file, which says nothing about the neighborhoods. Downtown is the first box: its whiskers run from the box at 238,250 down to 194,000 and from 268,250 up to 295,000, the lowest and highest Downtown prices inside the fences of 193,250 and 313,250, and the two prices below the lower fence, 146,000 and 172,000, are drawn one by one. A whisker ends at a real listing, the last one inside the fence.

DON'T:

• DON'T report the row count as the number of values a chart drew. ax.hist and sns.histplot both drew 185 of 204 prices and said nothing about the 19 gaps.
• DON'T accept a bin count without trying another. Four bins hid the second hump that twelve showed.
• DON'T read a whisker's end as the fence. Downtown's upper whisker stops at 295,000, and its fence is at 313,250.

Relationship Views - Scatter and hue

What. A scatter plot draws one point per row at the coordinates of two numeric columns. seaborn's scatterplot takes a DataFrame and column names, and hue= colors each point by a third column. Why. It shows the shape 2.7.1's coefficients compress into one number, a line, a curve, a cloud, or one point far from the rest, and with hue, whether a category moves the cloud. When. For every feature you are about to describe with a coefficient, against the target first.

pairs = df[["area_sqm", "price"]].dropna()
fig, ax = plt.subplots(figsize=(8, 5))
points = ax.scatter(pairs["area_sqm"], pairs["price"], alpha=0.4)
ax.set_yscale("log")
ax.set_xlabel("Area (square meters)")
ax.set_ylabel("Price")
fig.savefig("area_price.png", dpi=150, bbox_inches="tight")
plt.close(fig)

print(len(points.get_offsets()))  # 170
print(ax.get_yscale())            # log

ax.scatter returns the drawn points, and get_offsets() holds one position per point, 170 here, the listings with both values. alpha=0.4 makes each point 40 percent opaque, so a spot where many listings overlap draws darker than a lone point and density stays visible. ax.set_yscale("log") spaces the price axis by ratios, so 200,000 to 400,000 takes the same height as 2,000,000 to 4,000,000, and the ordinary prices spread out instead of hugging the bottom of an axis that has to reach 9,900,000.

fig, ax = plt.subplots(figsize=(8, 5))
sns.scatterplot(data=df, x="area_sqm", y="price", hue="condition", alpha=0.6, ax=ax)
drawn = ax.collections[0].get_offsets()
labels = [t.get_text() for t in ax.get_legend().get_texts()]
print(len(drawn))  # 169
print(labels)      # ['fair', 'good', 'excellent', 'poor']
plt.close(fig)

seaborn drops every row missing any column it uses, so hue cost one more listing, the one with no condition, and 169 points were drawn. The legend lists the conditions in the order they first appear in the file, because nothing told it otherwise, and the colors follow the same order:

from matplotlib.colors import to_hex


# each legend label with its color as a #rrggbb string
def legend_colors(ax):
    legend = ax.get_legend()
    entries = zip(legend.get_texts(), legend.legend_handles)
    return {t.get_text(): to_hex(h.get_color()) for t, h in entries}


order = ["poor", "fair", "good", "excellent"]
downtown = df[df["neighborhood"] == "Downtown"]
for frame in [df, downtown]:
    for hue_order in [None, order]:
        fig, ax = plt.subplots()
        sns.scatterplot(data=frame, x="area_sqm", y="price",
                        hue="condition", hue_order=hue_order, ax=ax)
        print(legend_colors(ax))
        plt.close(fig)

{'fair': '#1f77b4', 'good': '#ff7f0e', 'excellent': '#2ca02c', 'poor': '#d62728'}
{'poor': '#1f77b4', 'fair': '#ff7f0e', 'good': '#2ca02c', 'excellent': '#d62728'}
{'good': '#1f77b4', 'fair': '#ff7f0e', 'poor': '#2ca02c'}
{'poor': '#1f77b4', 'fair': '#ff7f0e', 'good': '#2ca02c', 'excellent': '#d62728'}

Without hue_order, seaborn hands out its colors in order of appearance in the rows it was given. Downtown's rows start with a good listing and hold no excellent one, so its chart paints good in the blue the full table used for fair, and two charts meant to be compared disagree on what blue means. With hue_order, both charts give poor blue, fair orange, good green and excellent red, and Downtown's legend lists all four, excellent included.

DON'T:

• DON'T let seaborn order a category you color by. Downtown's chart painted good in the full table's color for fair.
• DON'T read a scatter's point count off the table. Adding hue="condition" dropped one more listing, 169 points from 204 rows.

Multivariate Views - Heatmap and Pairplot

What. A heatmap draws a matrix as a grid of colored cells, here 2.7.1's correlation matrix, with annot=True writing each value into its cell and a colormap turning values into colors. A pairplot draws a scatter for every pair of numeric columns and each column's histogram on the diagonal. plt.subplots(2, 2) builds one Figure holding a 2 by 2 grid of Axes. Why. They look at every column at once: the heatmap reads a whole correlation matrix at a glance, and the pairplot shows the scatter behind each of its cells, the check 2.7.1 ran one pair at a time. When. A heatmap once a matrix has more cells than you can read as numbers, a pairplot on a handful of numeric columns, and a grid whenever several charts belong on one page.

num = ["price", "area_sqm", "rooms", "year_built"]
print("coolwarm" in plt.colormaps())  # True

corr = df[num].corr()
fig, ax = plt.subplots(figsize=(6, 5))
sns.heatmap(corr, annot=True, fmt=".2f", vmin=-1, vmax=1,
            cmap="coolwarm", ax=ax)
fig.savefig("corr_heatmap.png", dpi=150, bbox_inches="tight")
print(len(ax.texts))                  # 16
print(ax.texts[1].get_text())         # 0.69
print(ax.collections[0].get_clim())   # (-1.0, 1.0)
print(len(fig.axes))                  # 2
plt.close(fig)

coolwarm is a diverging colormap, one that runs from one color through a neutral middle to another, the right kind for a quantity whose 0 means something. plt.colormaps() lists every name matplotlib knows, so the name is checked before it is used. vmin=-1, vmax=1 pins the scale to the whole range a correlation can take, so 0 sits on the neutral middle and each color means the same coefficient in every heatmap drawn this way, and get_clim() reads the limits back. fmt=".2f" formats the numbers written in the cells. The color bar beside the grid is an Axes of its own, which is why the figure holds two.

Left to itself, the scale runs from the smallest value in the matrix to the largest:

loose = typical_rows[num].corr()
fig, ax = plt.subplots(figsize=(6, 5))
sns.heatmap(loose, annot=True, fmt=".2f", cmap="coolwarm", ax=ax)
low, high = ax.collections[0].get_clim()
print(round(low, 2), high)  # -0.08 1.0
plt.close(fig)

On the typical listings the smallest coefficient is -0.08, price against rooms, and the floating scale gives it the deepest blue the colormap has, the color the fixed scale keeps for -1.

sns.pairplot builds a figure of its own and returns a PairGrid, which holds the grid of Axes and saves itself:

grid = sns.pairplot(df[num])
panel = grid.axes[0, 1]
drawn = panel.collections[0].get_offsets()
top = drawn[drawn[:, 1].argmax()]
print(type(grid).__name__, grid.axes.shape)  # PairGrid (4, 4)
print(panel.get_ylabel())                    # price
print(panel.get_xlabel())                    # area_sqm
print(len(drawn), top.tolist())              # 170 [2400.0, 9900000.0]
grid.savefig("pairplot.png")
plt.close(grid.figure)

Row 0 holds price on the y axis and column 1 holds area on the x axis. That panel drew 170 points, the rows with both values, exactly as 2.7.1's matrix cell counted them, and its highest point is the listing of 2,400 square meters at 9,900,000 that 2.7.1 traced the coefficient to.

Several charts on one page come from plt.subplots(rows, columns), which returns the Figure and, for two or more rows and columns, a NumPy array of Axes shaped like the grid, axes[row, column] picking one:

fig, axes = plt.subplots(2, 2, figsize=(16, 10))
print(type(axes).__name__, axes.shape)  # ndarray (2, 2)

axes[0, 0].hist(typical, bins=12)
sns.boxplot(data=typical_rows, x="neighborhood", y="price",
            order=hoods, ax=axes[0, 1])
sns.scatterplot(data=df, x="area_sqm", y="price", hue="condition",
                hue_order=order, alpha=0.6, ax=axes[1, 0])
axes[1, 0].set_yscale("log")
sns.heatmap(corr, annot=True, fmt=".2f", vmin=-1, vmax=1,
            cmap="coolwarm", ax=axes[1, 1])
fig.savefig("overview.png", dpi=150, bbox_inches="tight")
print(len(fig.axes))                    # 5
plt.close(fig)

Every seaborn call takes its panel through ax=, and the figure holds five Axes, four panels and the heatmap's color bar.

DON'T:

• DON'T leave a correlation heatmap's scale to float. The typical listings' -0.08 took the deepest blue, the color a fixed scale keeps for -1.
• DON'T type a colormap name from memory. cmap="coolwarmm" raises KeyError: "'coolwarmm' is not a valid value for colormap. Did you mean one of: 'coolwarm', 'coolwarm_r', 'cool_r'?".
• DON'T index a one-row grid with two numbers. plt.subplots(1, 2) returns a 1-D array of shape (2,), and axes[0, 1] raises IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed.

Lab Exercises

Lab Exercise 1: Draw and Count the Floor Area

Work in area_charts.py in ~/dev/bootcamp, loading the table without its repeated rows.

Build an 8 by 5 inch Figure and Axes, draw a histogram of area_sqm with its gaps left in and 20 bins, label both axes, give it a title, and save it as area_hist.png at 150 dots per inch with the margin trimmed. Print how many areas the bars add up to, how many rows the table has, and how many of the 20 bars have any height.

Draw a 20-bin histogram of the areas under 1,000 square meters, save it as area_hist_bulk.png, and print the bar heights as a list of whole numbers.

Draw a seaborn box plot of area_sqm per condition, ordered poor, fair, good, excellent, save it as area_by_condition.png, and print how many points each box drew beyond its whiskers, then the points of the fair box.

Lab Exercise 2: Year Built, Price and the Spearman Matrix

Work in price_views.py in ~/dev/bootcamp, loading the table without its repeated rows.

Build one figure of two Axes side by side with plt.subplots(1, 2, figsize=(16, 6)) and print the shape of the array of Axes. On the left, draw a seaborn scatter with year_built on the x axis and price on the y axis, colored by neighborhood, the neighborhoods in alphabetical order, at alpha=0.6, on a log price axis. On the right, check that coolwarm is a colormap name, then draw the Spearman matrix of the four numeric columns as an annotated heatmap on the fixed scale. Save the figure as price_views.png.

Print the number of points the scatter drew, its legend labels and its y scale, then the heatmap's annotations and its color limits, and finally how many Axes the figure holds, with a comment on why that is not two.

import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns

df = pd.read_csv("housing.csv").drop_duplicates()

fig, ax = plt.subplots(figsize=(8, 5))
counts, edges, bars = ax.hist(df["area_sqm"], bins=20)
ax.set_xlabel("Area (square meters)")
ax.set_ylabel("Listings")
ax.set_title("Floor area")
fig.savefig("area_hist.png", dpi=150, bbox_inches="tight")
plt.close(fig)

print(int(counts.sum()), len(df))  # 189 204
print(int((counts > 0).sum()))     # 4

bulk = df.loc[df["area_sqm"] < 1000, "area_sqm"]
fig, ax = plt.subplots(figsize=(8, 5))
counts, edges, bars = ax.hist(bulk, bins=20)
fig.savefig("area_hist_bulk.png", dpi=150, bbox_inches="tight")
plt.close(fig)
print(counts.astype(int).tolist())

[2, 1, 4, 4, 8, 10, 14, 19, 20, 23, 19, 12, 18, 11, 10, 5, 2, 2, 1, 2]

order = ["poor", "fair", "good", "excellent"]
fig, ax = plt.subplots(figsize=(8, 5))
sns.boxplot(data=df, x="condition", y="area_sqm", order=order, ax=ax)
fig.savefig("area_by_condition.png", dpi=150, bbox_inches="tight")
box = ax.containers[0]
fliers = [len(f.get_ydata()) for f in box.fliers]
fair_points = sorted(box.fliers[1].get_ydata().tolist())
print(fliers)       # [0, 4, 2, 1]
print(fair_points)  # [26.4, 31.1, 165.2, 1850.0]
plt.close(fig)

import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns

df = pd.read_csv("housing.csv").drop_duplicates()
num = ["price", "area_sqm", "rooms", "year_built"]
hoods = sorted(df["neighborhood"].unique())

fig, axes = plt.subplots(1, 2, figsize=(16, 6))
print(axes.shape)

left, right = axes
sns.scatterplot(data=df, x="year_built", y="price", hue="neighborhood",
                hue_order=hoods, alpha=0.6, ax=left)
left.set_yscale("log")

print("coolwarm" in plt.colormaps())
ranks = df[num].corr(method="spearman")
sns.heatmap(ranks, annot=True, fmt=".2f", vmin=-1, vmax=1,
            cmap="coolwarm", ax=right)
fig.savefig("price_views.png", dpi=150, bbox_inches="tight")

print(len(left.collections[0].get_offsets()))
print([t.get_text() for t in left.get_legend().get_texts()])
print(left.get_yscale())
print([t.get_text() for t in right.texts])
print(right.collections[0].get_clim())
# three: the scatter, the heatmap and the heatmap's color bar
print(len(fig.axes))
plt.close(fig)

(2,)
True
185
['Downtown', 'Eastgate', 'Hillcrest', 'Lakeview', 'Northend', 'Riverside', 'Southgate']
log
['1.00', '-0.05', '-0.05', '-0.03', '-0.05', '1.00', '0.02', '0.10', '-0.05', '0.02', '1.00', '-0.00', '-0.03', '0.10', '-0.00', '1.00']
(-1.0, 1.0)
3

Theory Assessment

Q1: Where a Chart Goes

A script ends with plt.show(). Say what that line does on WSL in a terminal with no display and in one with a display, and why this module saves its charts with fig.savefig instead. Then say which object owns each of these: an axis label, the legend, the image size, the saved file.

Hint: Neither terminal shows the figure, and only one of them says so. For the second half, ask which settings belong to one plotting area and which to the whole image.

Q2: What the Bars Did Not Say

ax.hist(df["price"], bins=30) drew 185 values, put 175 of them in its first bar and gave four bars any height. Explain both facts. Then explain why four bins on the 183 typical prices show one hump where twelve show two, and which chart to trust. Finally, Downtown's upper whisker ends at 295,000 while its upper fence is 313,250: explain the gap.

Hint: Ask what ax.hist does with the 19 missing prices, and how wide 30 bins are when the column reaches 9,900,000. A bar's height is a count over its whole width.

Q3: Two Charts That Disagree About Color

One analyst's scatter of the Downtown listings colors good blue, while another's scatter of the whole table colors good orange. One heatmap of the typical listings paints price against rooms, -0.08, in the deepest blue, while another paints the same cell almost neutral. Explain each disagreement, and the argument that removes it.

Hint: In both pairs, one chart was left to decide something from the data it happened to be given: an order in one case, a range in the other.

Q4: The Title That Moved

Read this script without running it.

import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns

df = pd.read_csv("housing.csv").drop_duplicates()
fig1, ax1 = plt.subplots()
ax1.hist(df["price"], bins=12)
fig2, ax2 = plt.subplots()
sns.scatterplot(data=df, x="area_sqm", y="price", hue="condition", ax=ax2)
plt.title("Price distribution")
plt.xlabel("Price")
fig1.savefig("dist.png")

print(repr(ax1.get_title()), repr(ax2.get_title()))
print(ax2.get_xlabel())
print(int(sum([p.get_height() for p in ax1.patches])))
print(len(ax2.collections[0].get_offsets()))

Say what each line prints and what dist.png contains, and explain why nothing raised or warned.

Hint: Work out which Axes is current at the moment plt.title runs. Then remember what each kind of chart does with a row it cannot place.

'' 'Price distribution'
Price
185
169

Assignment

Assignment: A One-Page Report on the Price Target

Before a model is fitted to price, a reviewer wants one image that answers four questions, with a printed check proving what is in it. Write price_report.py in ~/dev/bootcamp that builds a 2 by 2 figure, 16 by 10 inches, and saves it once as price_report.png.

The top-left panel is a histogram of np.log1p of the priced listings, all 185 of them, in 30 bins, with a dashed vertical line at the log of the median price and a dotted one at the log of the mean price, labeled median and mean in a legend. A vertical line across a whole Axes comes from ax.axvline(x, linestyle="--", label="median"): it draws a line at position x from the bottom of the Axes to the top, whatever the y range is, and it is a line like any other, so it joins ax.lines and the legend.

The top-right panel is a box plot of price per neighborhood for the listings under a million, the boxes ordered from the lowest median price to the highest, an order your script computes rather than types. The bottom-left panel is a scatter with area_sqm on the x axis and price on the y axis, colored by condition, poor to excellent, at alpha=0.6, on a log price axis. The bottom-right panel is the Spearman matrix of the four numeric columns on a fixed coolwarm scale.

Then print the check: how many Axes the figure holds, how many lines the histogram panel has and its legend labels, the x positions of the two vertical lines rounded to three places, the order of the boxes, how many points the scatter drew, and the heatmap's color limits.

Hints:

• groupby from Lesson 2.6.2, a median per neighborhood and sort_values, gives the box order as an index you can turn into a list.
• ax.axvline returns the line it drew, and each line's x position is in its get_xdata().

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns

df = pd.read_csv("housing.csv").drop_duplicates()
price = df["price"].dropna()
typical_rows = df[df["price"] < 1_000_000]
num = ["price", "area_sqm", "rooms", "year_built"]
conditions = ["poor", "fair", "good", "excellent"]

medians = typical_rows.groupby("neighborhood")["price"].median()
box_order = medians.sort_values().index.tolist()

fig, axes = plt.subplots(2, 2, figsize=(16, 10))
hist_ax, box_ax = axes[0]
scatter_ax, heat_ax = axes[1]

hist_ax.hist(np.log1p(price), bins=30)
hist_ax.axvline(np.log1p(price.median()), linestyle="--",
                color="black", label="median")
hist_ax.axvline(np.log1p(price.mean()), linestyle=":",
                color="red", label="mean")
hist_ax.set_xlabel("log of price")
hist_ax.legend()

sns.boxplot(data=typical_rows, x="neighborhood", y="price",
            order=box_order, ax=box_ax)
sns.scatterplot(data=df, x="area_sqm", y="price", hue="condition",
                hue_order=conditions, alpha=0.6, ax=scatter_ax)
scatter_ax.set_yscale("log")
sns.heatmap(df[num].corr(method="spearman"), annot=True, fmt=".2f",
            vmin=-1, vmax=1, cmap="coolwarm", ax=heat_ax)
fig.savefig("price_report.png", dpi=150, bbox_inches="tight")

legend = [t.get_text() for t in hist_ax.get_legend().get_texts()]
x_lines = [round(float(line.get_xdata()[0]), 3) for line in hist_ax.lines]
print(len(fig.axes), len(hist_ax.lines), legend)
print(x_lines)
print([t.get_text() for t in box_ax.get_xticklabels()])
print(len(scatter_ax.collections[0].get_offsets()))
print(heat_ax.collections[0].get_clim())
plt.close(fig)

5 2 ['median', 'mean']
[12.417, 12.785]
['Hillcrest', 'Northend', 'Southgate', 'Downtown', 'Riverside', 'Lakeview', 'Eastgate']
169
(-1.0, 1.0)

Additional Resources

Optional, and nothing later depends on either. Worth opening for the histogram's smoothed cousin, or for which kind of colormap fits which data.

• Official docs - seaborn, "Visualizing distributions of data", "Kernel density estimation". The smoothed curve seaborn draws in place of a histogram's bars, and its pitfalls.
• Official docs - matplotlib, "Choosing Colormaps in Matplotlib", "Classes of colormaps". The four classes of colormap, and why data with a critical middle value needs a diverging one.

Summary

A chart is a claim, and like every statistic in Lesson 2.7.1 it rests on a count. The histogram that drew 185 of 204 prices, the scatter that drew 169 and the pairplot panel that drew 170 each said nothing about the rows they left out, and the habit this lesson builds is to report that count beside every picture. What a summary forgets, a chart can show, but only if the chart is honest about what it was given.

The other half of honesty is consistency. A color means something only when it means the same thing in every chart of a set, which is why hue_order and a fixed vmin and vmax are not decoration: they are what lets two charts be read side by side. The bin count belongs to the same family, a choice the reader cannot see unless you state it, and one that can make a second hump appear or disappear.

Under all of it sits the Figure and its Axes. Naming them is what would have kept the validation curve on the chart it was meant for, and it is the structure every chart your scripts draw from here on is built on. Module 3's residual plots and learning curves and Module 4's loss curves are this lesson's plt.subplots figures with different data, saved the same way and checked the same way, by printing what they hold.

Cheat Sheet

Part A - Reference Table

Part B - Compact Code Block

import matplotlib.pyplot as plt
import seaborn as sns

fig, ax = plt.subplots(figsize=(8, 5))     # Figure + one Axes
counts, edges, bars = ax.hist(s, bins=12)  # gaps dropped
drawn = int(counts.sum())                  # report it with the chart
ax.set_xlabel("Price")
fig.savefig("hist.png", dpi=150, bbox_inches="tight")
plt.close(fig)                             # release it

order = ["poor", "fair", "good", "excellent"]
fig, axes = plt.subplots(1, 2)             # 1-D array of 2
sns.scatterplot(data=df, x="area_sqm", y="price",
                hue="condition", hue_order=order, ax=axes[0])
axes[0].set_yscale("log")                  # skewed target
sns.heatmap(corr, annot=True, vmin=-1, vmax=1,
            cmap="coolwarm", ax=axes[1])   # fixed scale
fig.savefig("views.png", bbox_inches="tight")
plt.close(fig)