Section 5.2: Transfer Learning, Vision

A pretrained model is a network whose weights were trained on far more images than you have, and transfer learning adapts those weights to a new task instead of training from a seeded draw. This section loads ResNet-18 with its ImageNet weights through torchvision's weights API, reads what the file holds, from its 11,689,512 parameters to the preprocessing its first layer expects, and adapts it to EuroSAT, 27,000 satellite tiles of land in ten classes that ImageNet's photos never held. A frozen backbone with a new head reaches 0.9226 on the validation tiles from features computed once, fine-tuning every weight at 1e-4 reaches 0.9567, and after every careful variant has been measured, the chosen network scores the test tiles once, at 0.9544, with its errors read class by class and tile by tile.

Section 5.1 trained a small network from nothing on 50,000 grayscale garments. Here the network arrives trained, and the work is deciding how much of it to change and at what rate, on 2,700 labeled tiles. The failures here are quiet. A model built without its weights trains from random filters with no warning, and a frozen backbone left in training mode rewrites its own BatchNorm statistics and takes its head from 0.9226 to 0.5563 without moving a weight. A head trained on normalized tiles and served raw ones scores 0.4281, and a fine-tune at the rate its new head trained at moves the last stage's last weight by a third of itself in its first 20 steps. None of them stops the program.

Lessons

  1. Pretrained Models and the Weights API - loading ResNet-18 and its weights, reading the network, the residual connection, the weights' own preprocessing, and EuroSAT, tiles ImageNet never held.
  2. Freeze and Fine-Tune on EuroSAT - freezing the backbone, features computed once, the frozen backbone in training mode, the normalization contract, and fine-tuning everything.
  3. Unfreezing, Careful Learning Rates and Error Analysis - the rate that undoes pretraining, a learning rate per layer group, unfreezing in stages, training the BatchNorms alone, and choosing, testing once and reading the errors.

By the end you will be able to:

  • Load a pretrained network with its weights, read its metadata, stages and residual blocks, and build the preprocessing its weights expect.
  • Freeze a backbone, train a new head on features computed once, and keep a frozen network frozen, its BatchNorms included.
  • Fine-tune every weight, or each layer group at its own rate, or stage by stage, and measure how far each strategy moved the pretrained weights.
  • Choose among strategies on validation tiles, score the test tiles once, and read the chosen model's errors by class, by pair of classes and by tile.