Module 5
Module 5 - Modern Architectures and Transfer Learning
You will be able to build a convolutional network and a transformer block from their parts, and adapt pretrained vision and text models to your own data with a training loop you wrote and can read.
An architecture is the shape of what sits inside the training loop, and this module changes that shape while the loop stays the one Module 4 wrote. A convolution reads an image through small filters slid over every position, so its parameters belong to the filter and not to the image. Attention lets every position of a sequence weigh every other position by a score computed from both, and the transformer stacks it into blocks. A pretrained model is a network of one of these shapes whose weights were trained on far more data than you have, and transfer learning (adapting those weights to a new task) is how most real vision and text work begins. The module covers the convolution, pooling and BatchNorm, a CNN trained on FashionMNIST and data augmentation, then a pretrained vision model frozen, fine-tuned and unfrozen on EuroSAT's satellite photos of land, then text as tokens and embeddings, self-attention built from matrix products and the transformer block, and last a pretrained text encoder loaded through Hugging Face and fine-tuned with your own loop.
This is where Module 4's network meets the structure of its data. Section 4.5 ended with a fully connected network that scored 0.8677 on clothing and 0.4554 on the same clothing two pixels to the right, and Section 5.1 opens on those two numbers with a network built from filters, trained by the same loop on the same images and measured the same way, down to where its own tolerance for a move ends. Every architecture after it is assembled from operations you have already written, matrix products, sums and the softmax, before a library hands it to you whole.
Without it, a pretrained model is a black box you can only call. You cannot tell why a fine-tune at the learning rate of a fresh layer wrecked the weights it started from, why a network built from filters still fails when a photo is framed off center, or why a model left in training mode gives one image two answers. This module's boundary, stated openly: architecture is in, application is out. You will build and understand the transformer, and prompting, retrieval and agent systems belong to a separate AI Engineering product and are absent here.
What you will build
One project, the capstone, at the end of the module.
The Fine-Tune Ladder (PAID, $49, or $24 at the half price), one pretrained model adapted several ways, from a frozen backbone to a carefully unfrozen one, closing in a written recommendation that sets each strategy's cost beside its quality, where the recommendation is not the highest score.
The sections
- Convolutional Networks, Keeping the Structure - 4 lessons. The convolution, pooling, BatchNorm (batch normalization) and the conv block, a CNN (a convolutional network) built to beat Module 4's network, then data augmentation.
- Transfer Learning, Vision - 3 lessons. Pretrained models and torchvision's weights API, freezing and fine-tuning on EuroSAT, 27,000 satellite photos of land in ten classes such as forest, river and highway, then unfreezing, careful learning rates and error analysis.
- Attention and the Transformer - 3 lessons. Text as tensors, tokens (the pieces a text is cut into) and embeddings (a learned vector for each token), self-attention built from matrix products, then the transformer block.
- Text Transfer Learning and the Architecture Map - 3 lessons. Hugging Face (a library and hub of pretrained models) and loading a pretrained encoder, fine-tuning the encoder with your own loop, then the architecture map and the boundary.
Prerequisites
Module 4. You are assumed to write and read the training loop without notes: the five steps of an update, nn.Module and optim.Adam, nn.CrossEntropyLoss on logits and integer labels, a DataLoader with a seeded generator, a validation accuracy read every epoch and the test set scored once, early stopping with the best state kept by copy.deepcopy, a state_dict saved and loaded, and evaluation mode switched on before a model is scored. Section 4.5's FashionMNIST files, its loaders and its two saved models are where Section 5.1 starts.
What the exam proves
PAID, $49, or $24 at the half price. That you can compute a convolution's output shape, trace attention through its steps, and name what a described fine-tuning failure did wrong. The pass covers the capstone and the exam with every attempt, for the life of the product.
Set up your machine first
Module 5 runs on the machine Module 4 left: (bootcamp) at ~/dev/bootcamp holding PyTorch and torchvision at the processor-only builds Module 4 installed, FashionMNIST in the data folder Lesson 4.5.1 downloaded into, and Section 4.5's fashion.py, image_loop.py and trained.py beside the two models Lesson 4.5.2 saved, which Section 5.1 imports and loads. Section 5.1 installs nothing, and Lesson 5.4.1 installs the library that loads pretrained text models. Module 5's setup page checks that machine, rebuilds Section 4.5's files and models if you are starting here, and ends with a checklist that must pass before Lesson 5.1.1. Read it even if your prompt already shows (bootcamp).