Installing PyTorch and the Tensor

Context

A tensor is PyTorch's array: a grid of numbers of one dtype that knows which device holds it and can record the operations that made it. PyTorch is the library this module builds its networks in, and the tensor is the one container everything in it is made of, a listing's features, its price, a weight the fit chooses and the loss that scores the fit.

Every column that leaves pandas for a model in this module becomes a tensor first, the way the areas and prices of listings.csv become the two tensors Lesson 4.2.1 fits a line through. A network's weights are tensors drawn at random under a seed, a loss is a tensor of one number read out as a Python float, and Lesson 4.4.3's saved model is a file of tensors. A listings site that moves its pricing model from scikit-learn to PyTorch starts here.

What breaks is quiet. A column wrapped without a copy writes back into the frame under a warning that scrolls past, 125 missing areas ride into a mean and turn it into nan, a float32 statistic is reported to sixteen digits of which seven are data, and a weight drawn before the seed can never be drawn again. Lesson 3.1.3 said that a scan works for one parameter or two and is hopeless for thousands, which is why Module 4 searches without a grid, and the search runs on tensors, so this lesson installs the library and builds the container before Lesson 4.1.3 computes the first derivative.

Goal

• Install torch 2.14.1's processor-only build into (bootcamp) with uv pip install and PyTorch's own index, confirm the version your scripts import, and explain what the same line without the index would have fetched.
• Build tensors from lists, from NumPy arrays and from the columns of listings.csv, and distinguish torch.tensor, which copies, from torch.from_numpy, which shares the array's memory.
• Diagnose the warning a frame's column draws from from_numpy, the TypeError a text column raises and the nan a missing value puts in a mean.
• Read a tensor's dtype, shape, ndim and numel(), turn a scalar tensor into a Python number with .item(), and print a float32 statistic to the digits the data has.
• Seed PyTorch's random stream with torch.manual_seed(42), reproduce a draw, and create tensors with zeros, ones, full, eye, arange and linspace at the dtype each returns.

Prerequisites

None of this is required, the lesson stands on its own. Both are here if you want to see tensors made, read and handed back to NumPy before you build your own.

• YouTube - "Introduction to PyTorch Tensors" by PyTorch. Creating tensors, random tensors and seeding, shapes and dtypes, then arithmetic on them, copying, moving to an accelerator, reshaping and the NumPy bridge, the ground this lesson and the next one cover at the keyboard.
• Official docs - PyTorch Tutorials, "Learn the Basics", the page "Tensors". A tensor initialized from data, from a NumPy array and from another tensor, its shape, dtype and device, the operations on it, and the section "Bridge with NumPy", where a tensor and an array share one block of memory.

Table of Contents

• Installing PyTorch
• What a Tensor Is
• Tensors From Lists, From NumPy and From the Listings
• dtype, Shape and the Scalar Tensor
• Random Tensors and the Seed

Theory

This lesson is the first to import PyTorch, and it keeps opening listings.csv, Module 3's dataset of 3,000 listings, which sits in ~/dev/bootcamp from Module 3's setup page, while Module 4's setup page says which machines this module runs on. A Python block that opens with its imports is a script of its own, and a block that does not continues the script above it.

Installing PyTorch

What. PyTorch is the third-party library this module builds its networks in: tensors, the record of operations that computes derivatives, and the layers, losses and optimizers built on both. It is installed and imported under one name, torch, and torch==2.14.1 is the exact release every result in this module was printed by. Why. (bootcamp) holds Module 2's libraries and Module 3's, and none of them differentiates the code you write or runs it on an accelerator. PyTorch ships in several builds, and the one PyPI hands out on Linux and WSL carries NVIDIA's CUDA libraries for a GPU this module never requires, so the install line names PyTorch's own index and takes the processor-only build from it. When. Once, now, with (bootcamp) active. Every lesson of this module imports what this line installs, and Lesson 4.5.1 adds torchvision beside it from the same index.

Before the install, (bootcamp) has no torch to import:

(bootcamp) goku@machine:~/dev/bootcamp$ python3 -c "import torch"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'torch'

In ~/dev/bootcamp, with (bootcamp) active, install the pinned release from PyTorch's processor index:

uv pip install torch==2.14.1 --index-url https://download.pytorch.org/whl/cpu

Among the lines uv prints, the ones that start with + name what it added:

 + filelock==3.32.3
 + fsspec==2026.7.0
 + jinja2==3.1.6
 + markupsafe==3.0.3
 + mpmath==1.3.0
 + networkx==3.6.1
 + setuptools==78.1.0
 + sympy==1.14.0
 + torch==2.14.1+cpu
 + typing-extensions==4.16.0

One name went in and ten libraries came out. torch lists seven as libraries it needs: sympy (symbolic mathematics, which PyTorch uses to reason about shapes it has not seen yet), networkx (graphs of nodes and edges), jinja2 (text templates), fsspec (one interface over local and remote file systems), filelock (a lock that keeps two processes from writing one file at once), setuptools (the tool that builds Python packages) and typing-extensions (type hints newer than the interpreter's own). mpmath (arithmetic at any precision) arrives because sympy needs it, and markupsafe (escaping for the text that goes into a template) because jinja2 does. Only torch is pinned, so the other nine lines can read differently on your machine, in their versions and in which of them appear. NumPy has no + line: torch does not list it, and from_numpy and .numpy() use the numpy 2.5.2 already in (bootcamp).

The +cpu on torch's own line is the build. On 2026-10-25 the processor-only wheel for Linux and WSL was 196 MB. The same torch==2.14.1 asked from PyPI, with no --index-url, is a 555 MB wheel, and a dry run of that line, uv pip install --dry-run torch==2.14.1, resolved 29 packages: torch, the same nine libraries as above, 18 packages of NVIDIA's CUDA libraries and triton (OpenAI's compiler for GPU code), the GPU side of PyTorch, which no lesson in this module uses because every lab here runs on the processor in minutes. A Mac with Apple's chip, on macOS 14 or later, has one build, 127 MB, and the same line installs it, because uv picks the wheel for the machine it runs on from the same index.

Confirm the install from a script, check_torch.py:

import torch

print(torch.__version__)          # 2.14.1+cpu
print(torch.cuda.is_available())  # False

On WSL and Linux the version carries its build, 2.14.1+cpu, and a Mac prints 2.14.1, because the Mac has one build and nothing to tell apart. On WSL and Linux a bare 2.14.1 means the default build is installed, from an earlier uv pip install torch without the index, and the install line above does not replace it, because torch==2.14.1 is satisfied by either build: uv prints Checked 1 package, adds nothing and leaves the NVIDIA build in place. Remove it first with uv pip uninstall torch, which prints - torch==2.14.1, and then run the install line. torch.cuda.is_available() asks whether PyTorch can reach an NVIDIA GPU through CUDA, and False is the processor build's answer, because it holds no CUDA code. Lesson 4.1.2 turns that answer into the one line that names a device.

DON'T:

• DON'T leave --index-url off. On 2026-10-25, uv pip install torch==2.14.1 against the default index resolved a 555 MB wheel, 18 packages of NVIDIA's CUDA libraries and a GPU compiler beside it on Linux and WSL, where the 196 MB processor build runs every lab in this module.
• DON'T read +cpu as a lesser library, or False from torch.cuda.is_available() as a fault. The processor build is the whole library without the GPU kernels, every tensor, gradient and network in this module runs on it, and False is that build saying what it is. On WSL and Linux 2.14.1+cpu is the only right answer, and a bare 2.14.1 is the default build, which uv pip uninstall torch removes before the install line goes in.

What a Tensor Is

What. A tensor (PyTorch's array: a grid of numbers of one dtype that knows which device holds it and can record the operations that made it) is Section 2.2's ndarray with two additions. The device (the hardware that holds a tensor's numbers and runs its arithmetic, the processor here, or an accelerator such as a GPU) is one, and the record of operations is the other. Why. That record is what the module exists for: a loss computed from recorded operations can hand back the derivative of itself with respect to every weight, which Lesson 4.1.4 shows, and the device lets the same script run on an accelerator by changing one line, which Lesson 4.1.2 shows. Neither is something an ndarray can do, which is why a second array library sits beside NumPy. When. Every number a model in this module touches is a tensor: the features, the target, the weights, the loss. NumPy and pandas stay for loading and reshaping the data, and a tensor is what the data becomes on its way into a model.

first_tensor.py builds one from the first two areas in the file:

import numpy as np
import torch

areas = torch.tensor([175.7, 30.5])
print(areas)         # tensor([175.7000,  30.5000])
print(type(areas))   # <class 'torch.Tensor'>
print(areas.dtype)   # torch.float32
print(areas.shape)   # torch.Size([2])
print(areas.device)  # cpu

arr = np.array([175.7, 30.5])
print(arr)           # [175.7  30.5]
print(arr.dtype)     # float64

The printed form names its type, tensor, and shows every float to four decimals, where NumPy shows the shortest digits that read back exactly. Two of the three attributes are Section 2.2's, dtype, the one type every element has, and shape, a torch.Size that reads like NumPy's tuple, and the third, device, no array has. The dtype is the first difference between the two libraries, float32 against float64 for the same two literals, and the next subtitle takes it apart.

A nested list gives a two-dimensional tensor, a table with one row per listing:

# area and rooms for two listings, one row each
table = torch.tensor([[175.7, 6], [30.5, 1]])
print(table)
print(table.shape)  # torch.Size([2, 2])
print(table.dtype)  # torch.float32

tensor([[175.7000,   6.0000],
        [ 30.5000,   1.0000]])

The 6 and the 1 went in as integers and came out as 6.0000 and 1.0000: one dtype per tensor, and a list that mixes floats and integers is float. The print names the dtype only when it is not the default of its kind, float32 for floats and int64 for integers, so a float64 tensor carries a dtype= at its end and an int64 one carries nothing:

# tensor([175.7000,  30.5000], dtype=torch.float64)
print(torch.tensor([175.7, 30.5], dtype=torch.float64))
print(torch.tensor([6, 1]))  # tensor([6, 1])
# tensor([6, 1], dtype=torch.int32)
print(torch.tensor([6, 1], dtype=torch.int32))

Recording is off until asked for:

print(areas.requires_grad)  # False
print(areas.grad_fn)        # None

requires_grad is the switch and grad_fn the record of the operation that produced a tensor, and both stay this way for every tensor in this section. Lesson 4.1.4 turns the switch on and reads the record.

DON'T:

• DON'T hand a tensor to NumPy's functions. np.mean(areas) raises TypeError: mean() got an unexpected keyword argument 'axis', because NumPy calls the tensor's own mean with an argument name the tensor does not take. A tensor has areas.mean(), and .numpy() hands its numbers to NumPy when NumPy is the tool you want.
• DON'T build a tensor with the capital-T torch.Tensor([...]). It is an older constructor that ignores what the list holds and always returns float32, so torch.Tensor([509000, 169000]) prints tensor([509000., 169000.]) where torch.tensor keeps the two prices as int64.

Tensors From Lists, From NumPy and From the Listings

What. torch.tensor(data) builds a tensor from a list, a number or a NumPy array and copies the data into it, with the dtype read from the data unless dtype= says otherwise. torch.from_numpy(arr) wraps an array's memory in a tensor without copying, and .numpy() hands a tensor's memory back to NumPy the same way. Why. The listings live in pandas and leave it as NumPy arrays, so every column reaches a model through one of these two calls, and which one decides whether the tensor owns its numbers or shares them with the frame. Sharing is a feature when a large array must not be held twice, and a trap when the array belongs to a DataFrame. When. Once per script, where the columns leave the frame. The module's line is torch.tensor(frame["col"].to_numpy(), dtype=torch.float32), and this subtitle is why each piece of it is there.

The dtype a list gives is PyTorch's default for its kind, from_lists.py:

import numpy as np
import torch

print(torch.tensor([1.5, 2.0]).dtype)     # torch.float32
print(torch.tensor([1, 2]).dtype)         # torch.int64
print(torch.tensor([True, False]).dtype)  # torch.bool
print(torch.tensor([1.5, 2]).dtype)       # torch.float32
print(np.array([1.5, 2.0]).dtype)         # float64

Floats are float32, integers int64, booleans torch.bool, and a mixed list is float. The same two floats give NumPy float64, so a number meant to be checked against NumPy is built with dtype=torch.float64 on purpose, and this module's models and the data they compute with are float32, PyTorch's default. From a NumPy array the two calls part ways:

arr = np.array([175.7, 30.5, 143.6])
copied = torch.tensor(arr)
shared = torch.from_numpy(arr)
print(copied.dtype)  # torch.float64
print(shared.dtype)  # torch.float64

arr[0] = 9.0
print(copied)        # tensor([175.7000,  30.5000, 143.6000], dtype=torch.float64)
print(shared)        # tensor([  9.0000,  30.5000, 143.6000], dtype=torch.float64)
shared[1] = 0.0
print(arr)           # [  9.    0.  143.6]

Both tensors took NumPy's float64, because torch.tensor keeps an array's dtype where it reads a list's. Writing 9.0 into the array changed shared and left copied alone, and writing 0.0 into shared changed the array, because from_numpy built a tensor over the array's own memory and torch.tensor built one over a copy. .numpy() goes the other way with the same sharing:

back = shared.numpy()
back[2] = 1.0
fresh = copied.numpy()
print(arr)                           # [9. 0. 1.]
print(np.shares_memory(back, arr))   # True
print(np.shares_memory(fresh, arr))  # False

Three names, one block of memory: arr, shared and back all read [9. 0. 1.], while copied still holds the three areas it was built from. np.shares_memory is the question to ask when a tensor's origin is unclear.

The trap this module meets at once is a column taken straight from the frame read_csv built. pandas 3 hands to_numpy() out as a read-only view of the column, and shared_column.py wraps it:

import pandas as pd
import torch

listings = pd.read_csv("listings.csv")
col = listings["area_sqm"].to_numpy()
print(col.flags.writeable)
area = torch.from_numpy(col)
area[0] = 1.0
print(listings["area_sqm"].iloc[0])

False
/home/goku/dev/bootcamp/shared_column.py:7: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /__w/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:213.)
  area = torch.from_numpy(col)
1.0

The warning's first line names your script and the line in it, 7, and its second line shows that line. It says what then happened: PyTorch has no read-only tensor, so from_numpy wrapped the array anyway, the write at line 8 went through, and listings["area_sqm"].iloc[0] reads 1.0 where the file says 175.7. The frame changed without a single pandas call, and every later line of the script that reads the area column reads the wrong one. "Suppressed for the rest of this program" is the second half of the trap: a second from_numpy on a second read-only column, later in the same script, warns about nothing.

torch.tensor copies and warns about nothing, and dtype= converts on the way, module_tensors.py:

import pandas as pd
import torch

listings = pd.read_csv("listings.csv")
has_area = listings.dropna(subset=["area_sqm"])
# False, dropna's columns are read-only too
print(has_area["area_sqm"].to_numpy().flags.writeable)

area = torch.tensor(has_area["area_sqm"].to_numpy(), dtype=torch.float32)
price = torch.tensor(has_area["price"].to_numpy(), dtype=torch.float32)
print(area[:4])                      # tensor([175.7000,  30.5000, 143.6000,  42.0000])
print(price[:4])                     # tensor([509000., 169000., 582000.,  93000.])
area[0] = 1.0
print(has_area["area_sqm"].iloc[0])  # 175.7

This is the module's line. dropna built has_area as a frame of its own, and its columns are new arrays, which pandas hands out read-only as well, so from_numpy draws the same warning on them. torch.tensor copies the 2,875 values into float32 as it goes, area[:4] holds the four areas Lesson 3.1.1 printed, and writing into area leaves the frame at 175.7. The prices arrive as 509000. with a trailing point, float32 for whole numbers, because Section 4.2's loss subtracts them from float32 predictions.

A text column is refused, with the list of what a tensor may hold:

try:
    torch.tensor(listings["neighborhood"].to_numpy())
except TypeError as err:
    print(err)

can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, complex64, complex128, int64, int32, int16, int8, uint64, uint32, uint16, uint8, and bool.

A tensor holds numbers, and the seven neighborhoods reach one as Lesson 3.2.2's encoded columns, never as text. A missing value is a number, nan, and travels:

every_area = torch.tensor(listings["area_sqm"].to_numpy())
print(every_area.dtype)               # torch.float64
print(torch.isnan(every_area).sum())  # tensor(125)
print(every_area.mean())              # tensor(nan, dtype=torch.float64)
print(every_area.nanmean())           # tensor(99.0200, dtype=torch.float64)

The 125 listings without an area went in as nan, one of them is enough to make .mean() return nan, and .nanmean() skips them. Nothing refused the column, which is why the module drops those listings with dropna before the first tensor is built rather than discovering them in a loss. Two columns at once make a table, and the module's line does not change:

X = torch.tensor(has_area[["area_sqm", "rooms"]].to_numpy(), dtype=torch.float32)
print(X.shape)  # torch.Size([2875, 2])
print(X[:2])

tensor([[175.7000,   6.0000],
        [ 30.5000,   1.0000]])

DON'T:

• DON'T build a tensor from a frame's column with torch.from_numpy. The array to_numpy() hands out is read-only, the call warns once and wraps it anyway, and every write through the tensor lands in the frame, listings["area_sqm"].iloc[0] read back as 1.0.
• DON'T take torch.tensor for sharing. It copies: arr[0] = 9.0 left copied at 175.7 and moved shared to 9.0, and only from_numpy and .numpy() share.
• DON'T expect NumPy's float64 from a list of floats. torch.tensor([1.5]) is float32 where np.array([1.5]) is float64, so a number meant to be checked against NumPy is built with dtype=torch.float64 by hand.
• DON'T pass a column with missing values through .mean(). One nan makes the mean nan, .nanmean() skips them, and the module drops the 125 listings with dropna before the tensor exists.

dtype, Shape and the Scalar Tensor

What. dtype is the one type every element of a tensor has, shape is its size along each dimension, and a scalar tensor (a tensor of shape torch.Size([]), one number with no dimension around it) is what every reduction returns. .item() turns a scalar tensor into a Python number. Why. A float32 holds about seven significant digits, so a statistic read through .item() shows digits that are the nearest float32 and not the data, and a float32 sum over 2,875 listings lands on a slightly different number depending on the order the processor adds in. How a number is printed decides whether its digits mean anything, and every number this module prints follows the rules this subtitle sets. When. Every time you print, round or compare a statistic, a loss or a weight, and every time two tensors meet in an operation, where Lesson 4.1.2's rules for shapes and dtypes start from what shape and dtype say.

shape_of_area.py builds the module's tensors and measures them:

import pandas as pd
import torch

listings = pd.read_csv("listings.csv")
has_area = listings.dropna(subset=["area_sqm"])
area = torch.tensor(has_area["area_sqm"].to_numpy(), dtype=torch.float32)
price = torch.tensor(has_area["price"].to_numpy(), dtype=torch.float32)
X = torch.tensor(has_area[["area_sqm", "rooms"]].to_numpy(), dtype=torch.float32)

print(area.shape)             # torch.Size([2875])
print(area.size())            # torch.Size([2875])
print(area.size(0))           # 2875
print(area.shape == (2875,))  # True
print(area.ndim)              # 1
print(area.numel())           # 2875
print(len(area))              # 2875
print(X.shape)                # torch.Size([2875, 2])
print(X.size(1))              # 2
print(X.ndim)                 # 2
print(X.numel())              # 5750
print(len(X))                 # 2875

shape and size() are the same torch.Size, which compares equal to the tuple (2875,), and size(0) is the length of one dimension. ndim counts the dimensions, numel() counts every element, NumPy's .size, and len counts along the first dimension only, so for the two-column table it is the rows, 2,875, while numel() is 5,750.

A reduction returns a scalar tensor:

mean_area = area.mean()
print(mean_area)                   # tensor(99.0200)
print(mean_area.shape)             # torch.Size([])
print(mean_area.ndim)              # 0
print(mean_area.item())            # 99.02003479003906
print(round(mean_area.item(), 2))  # 99.02
try:
    area.item()
except RuntimeError as err:
    print(err)                     # a Tensor with 2875 elements cannot be converted to Scalar

mean_area prints as tensor(99.0200) with no brackets, has an empty shape and zero dimensions, and is still a tensor, so .item() is how it becomes a number Python can round, format or compare. .item() works on exactly one element: on the 2,875 areas it raises. The number it returned, 99.02003479003906, is the float32 nearest to the mean written out as a Python float, and float32 holds about seven significant digits, so 99.02003 is the mean and the rest is representation, which on a processor that adds the 2,875 areas in another order can come out differently. The lesson's rule follows: a float32 statistic is printed rounded, round(mean_area.item(), 2), to the digits the data can carry. Comparing floats follows the same rule:

print(mean_area == 99.02)  # tensor(False)
# tensor(True)
print(torch.isclose(mean_area, torch.tensor(99.02)))

torch.isclose compares within a tolerance, one part in a hundred thousand of the value by default, and torch.allclose does the same for a whole tensor and returns one True or False, so a float32 number is never compared with ==. Float32's resolution is the same on every machine, and two literals show it:

print(torch.tensor(16777217.0))    # tensor(16777216.)
print(torch.tensor(175.7).item())  # 175.6999969482422
# 175.7, the same literal in float64
print(torch.tensor(175.7, dtype=torch.float64).item())

Past 16,777,216, float32 cannot hold every whole number, so 16,777,217 lands on its neighbor, and 175.7 has no exact float32, so the nearest one is stored and .item() writes it out in full, where the float64 nearest to 175.7 reads back as 175.7. A sum over a column meets both limits at once:

print(has_area["area_sqm"].sum())         # 284682.6
print(area.double().sum())                # tensor(284682.6000, dtype=torch.float64)
print(area.sum() == area.double().sum())  # tensor(False)
# tensor(True)
print(torch.isclose(area.sum().double(), area.double().sum()))
print(round(area.sum().item()))           # 284683

pandas sums the column in float64 and gets 284,682.6, and so does the tensor once .double() makes it float64. The float32 sum is not that number: at 284,682 a float32 can only move in steps of about 0.03, each of the 2,875 additions rounds to the nearest step, and PyTorch adds in an order chosen by the processor's instruction set, so the last digits of a float32 sum can differ between machines, and this lesson prints none of them. What holds on every machine is the comparison within a tolerance and the rounded sum, 284683. dtype= at creation decides the type, and four methods convert an existing tensor:

print(area.double().dtype)   # torch.float64
print(area.long()[:4])       # tensor([175,  30, 143,  42])
# tensor([175.7000,  30.5000, 143.6000,  42.0000], dtype=torch.float64)
print(area.to(torch.float64)[:4])
# tensor([1., 2.])
print(torch.tensor([1, 2], dtype=torch.float32))
print(area.float() is area)  # True

.float(), .double() and .long() return a new tensor in float32, float64 and int64, .to(torch.float64) takes the dtype as an argument, and .long() cuts the decimals off rather than rounding, 175.7 to 175. A conversion to the dtype a tensor already has returns the tensor itself, which is why is printed True. Indexing and masks are Section 2.2's:

print(area[-1])                           # tensor(156.3000)
print(area[1:3])                          # tensor([ 30.5000, 143.6000])
print(area[[0, 2]])                       # tensor([175.7000, 143.6000])
large = area > 200
print(large.dtype)                        # torch.bool
print(large.sum())                        # tensor(76)
print(round(price[large].mean().item()))  # 793145

A single index returns a scalar tensor, a slice or a list of positions a smaller tensor, and a comparison a torch.bool tensor whose .sum() counts its Trues: 76 listings have more than 200 square meters, and the mask selects their prices, whose mean is 793,145 to the unit.

DON'T:

• DON'T read .item()'s trailing digits as data. 99.02003479003906 is the float32 nearest to the mean written out in full, seven of its digits are the mean, and round(mean_area.item(), 2) prints what the data supports.
• DON'T print a float32 sum, mean or loss as its bare .item(). Its last digits depend on the order the processor added in, so a sum prints round(area.sum().item()), and a number to check against NumPy is computed in float64, area.double().sum().
• DON'T compare floats with ==. mean_area == 99.02 is tensor(False) for a mean that prints tensor(99.0200), and torch.isclose and torch.allclose are the comparison.
• DON'T read len(X) as a count of numbers. For the (2875, 2) table len(X) is 2875, the rows, and X.numel() is 5750.

Random Tensors and the Seed

What. torch.manual_seed(42) fixes the start of PyTorch's random stream, so torch.rand (uniform draws between 0 and 1), torch.randn (draws from the standard normal), torch.randint (integers in a range) and torch.randperm (a shuffled range) draw the same numbers on every run, each draw continuing the stream from where the last one left it. torch.zeros, torch.ones, torch.full, torch.eye, torch.arange and torch.linspace build tensors with no randomness in them. Why. Lesson 4.2.2's nn.Linear draws its weight and bias at random, and a run that cannot be repeated cannot be debugged: the same script with a different first weight gives a different loss curve, and no one can tell the change they made from the change the draw made. The seed is set beside NumPy's np.random.default_rng(42) and scikit-learn's random_state=42, because three libraries hold three streams. When. At the top of every script that draws anything, before the first draw, and once.

seeded.py:

import torch

torch.manual_seed(42)
print(torch.rand(3))  # tensor([0.8823, 0.9150, 0.3829])
print(torch.rand(3))  # tensor([0.9593, 0.3904, 0.6009])
torch.manual_seed(42)
print(torch.rand(3))  # tensor([0.8823, 0.9150, 0.3829])

The second rand(3) is a different draw, because the stream moved on, and the second manual_seed(42) sets it back to the start, so the third draw repeats the first. These are the numbers torch 2.14.1 draws on a processor from seed 42, and every seeded draw in this module is quoted from a run of its own script, because a draw depends on every draw before it:

torch.manual_seed(42)
print(torch.randn(3))              # tensor([0.3367, 0.1288, 0.2345])
print(torch.randint(0, 10, (5,)))  # tensor([0, 3, 8, 4, 0])
print(torch.randperm(6))           # tensor([0, 2, 1, 5, 3, 4])
torch.manual_seed(42)
torch.rand(3)                      # a draw that moves the stream
print(torch.randn(3))              # tensor([-0.8293, -1.6137, -0.2147])

randn draws from the standard normal, randint(0, 10, (5,)) five integers from 0 up to but not including 10, and randperm(6) the numbers 0 to 5 in a shuffled order, the draw Lesson 4.3.3 shuffles its rows with. All four take from one stream: the randn(3) that follows a rand(3) is not the randn(3) that follows the seed, so moving one line of a script moves every draw below it. A generator (an object that holds a random stream of its own) keeps a draw apart from the rest of the script:

g = torch.Generator().manual_seed(42)
print(torch.rand(3, generator=g))  # tensor([0.8823, 0.9150, 0.3829])

Seeded with 42 it draws the numbers the global stream drew, and only a call that names generator=g moves it, so draws elsewhere in the script leave it where it is. Why the seed matters is one line away from Lesson 4.2.2: a weight tensor starts as a draw:

torch.manual_seed(42)
w = torch.randn(2)  # a first weight for area and for rooms
print(w)            # tensor([0.3367, 0.1288])
print(w.dtype)      # torch.float32

Tensors with no randomness in them:

print(torch.zeros(3))                # tensor([0., 0., 0.])
print(torch.full((2,), 7.0))         # tensor([7., 7.])
print(torch.full((2,), 7))           # tensor([7, 7])
print(torch.arange(5))               # tensor([0, 1, 2, 3, 4])
print(torch.arange(5).dtype)         # torch.int64
print(torch.arange(0.0, 1.0, 0.25))  # tensor([0.0000, 0.2500, 0.5000, 0.7500])
print(torch.linspace(0, 1, 5))       # tensor([0.0000, 0.2500, 0.5000, 0.7500, 1.0000])
print(torch.zeros_like(w))           # tensor([0., 0.])
print(torch.ones(2, 3))
print(torch.eye(3))

tensor([[1., 1., 1.],
        [1., 1., 1.]])
tensor([[1., 0., 0.],
        [0., 1., 0.],
        [0., 0., 1.]])

zeros, ones, eye and linspace return float32, full takes the dtype of its fill value, 7.0 against 7, arange with integer arguments returns int64 and with float arguments float32, and all of them take dtype= when the default is not the one wanted. arange stops before its end value, so arange(0.0, 1.0, 0.25) has four entries where linspace(0, 1, 5) includes both ends and has five. torch.zeros_like(w) builds zeros in w's shape and dtype.

DON'T:

• DON'T draw before the seed. A torch.rand above torch.manual_seed(42) is a different draw on every run, and nothing in the script can reproduce it.
• DON'T re-seed before every draw. torch.manual_seed(42) followed by torch.randn(2) gives tensor([0.3367, 0.1288]) every time, so two weight tensors each drawn after their own manual_seed(42) start identical, where one seed at the top and the stream running on gives each its own numbers.
• DON'T expect NumPy's seed to reach PyTorch. np.random.default_rng(42).random(3) is [0.77395605 0.43887844 0.85859792] and torch.manual_seed(42) then torch.rand(3) is tensor([0.8823, 0.9150, 0.3829]), two libraries, two streams, each seeded on its own line.

Lab Exercises

Lab Exercise 1: The Bridge From the Listings

With (bootcamp) active, confirm with a one-line check that torch imports at 2.14.1 and that CUDA is not available.

Then, in tensor_bridge.py, build a tensor from the list of the first four areas, 175.7, 30.5, 143.6 and 42.0, and print it, its dtype, its shape and its device. Build a NumPy array of the first four prices, 509000, 169000, 582000 and 93000, and from it two tensors, one with torch.tensor and one with torch.from_numpy. Print the first one's dtype, change the array's last price to 95000, and print both tensors.

Then load listings.csv, take distance_km with to_numpy() and print whether the array is writable. Build the float32 tensor the module's way and print its dtype, its shape and its first four values. Hand property_type to torch.tensor inside a try and print the message. Finally build a tensor from year_built with no dtype=, and print its dtype, how many of its values are nan, what .mean() returns, and the mean that skips them rounded to one decimal.

Lab Exercise 2: Rooms, Counted and Shuffled

In rooms_stats.py, drop the listings without an area and build three float32 tensors: rooms, price, and a two-column X of rooms and bathrooms. Print the shape and ndim of rooms and of X, and X's numel() beside its len.

Take the mean of rooms and print the scalar tensor, its shape, its .item() and that value rounded to two decimals, then the sum of the rooms as a whole number. Build a mask of the listings with five rooms or more, and print its dtype, how many listings it selects and their mean price to the unit. Convert rooms to int64 and print the dtype and the sum.

Then seed PyTorch with 42, draw a permutation of the row positions with torch.randperm, and print its first five entries and its dtype. Draw a weight pair with torch.randn(2) and print it. Seed again with 42 and print the first five entries of a new permutation, to show that they repeat.

# 2.14.1+cpu False
python3 -c "import torch; print(torch.__version__, torch.cuda.is_available())"

import numpy as np
import pandas as pd
import torch

first_areas = torch.tensor([175.7, 30.5, 143.6, 42.0])
print(first_areas)         # tensor([175.7000,  30.5000, 143.6000,  42.0000])
print(first_areas.dtype)   # torch.float32
print(first_areas.shape)   # torch.Size([4])
print(first_areas.device)  # cpu

prices = np.array([509000, 169000, 582000, 93000])
copied = torch.tensor(prices)
shared = torch.from_numpy(prices)
print(copied.dtype)        # torch.int64
prices[3] = 95000
print(copied)              # tensor([509000, 169000, 582000,  93000])
print(shared)              # tensor([509000, 169000, 582000,  95000])

listings = pd.read_csv("listings.csv")
col = listings["distance_km"].to_numpy()
print(col.flags.writeable)  # False
distance = torch.tensor(col, dtype=torch.float32)
print(distance.dtype)       # torch.float32
print(distance.shape)       # torch.Size([3000])
print(distance[:4])         # tensor([9.6000, 5.7000, 3.1000, 9.6000])
try:
    torch.tensor(listings["property_type"].to_numpy())
except TypeError as err:
    print(err)

can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, complex64, complex128, int64, int32, int16, int8, uint64, uint32, uint16, uint8, and bool.

built = torch.tensor(listings["year_built"].to_numpy())
print(built.dtype)                       # torch.float64
print(torch.isnan(built).sum())          # tensor(140)
print(built.mean())                      # tensor(nan, dtype=torch.float64)
print(round(built.nanmean().item(), 1))  # 1990.2

import pandas as pd
import torch

listings = pd.read_csv("listings.csv")
has_area = listings.dropna(subset=["area_sqm"])
rooms = torch.tensor(has_area["rooms"].to_numpy(), dtype=torch.float32)
price = torch.tensor(has_area["price"].to_numpy(), dtype=torch.float32)
X = torch.tensor(has_area[["rooms", "bathrooms"]].to_numpy(), dtype=torch.float32)

print(rooms.shape, rooms.ndim)      # torch.Size([2875]) 1
print(X.shape, X.ndim)              # torch.Size([2875, 2]) 2
print(X.numel(), len(X))            # 5750 2875

mean_rooms = rooms.mean()
print(mean_rooms)                   # tensor(3.5621)
print(mean_rooms.shape)             # torch.Size([])
print(mean_rooms.item())            # 3.562087059020996
print(round(mean_rooms.item(), 2))  # 3.56
print(round(rooms.sum().item()))    # 10241

many = rooms >= 5
print(many.dtype)                        # torch.bool
print(many.sum())                        # tensor(766)
print(round(price[many].mean().item()))  # 559466

as_counts = rooms.long()
print(as_counts.dtype)                   # torch.int64
print(as_counts.sum())                   # tensor(10241)

torch.manual_seed(42)
order = torch.randperm(len(rooms))
print(order[:5])                         # tensor([ 292, 2688, 1152, 2249, 1480])
print(order.dtype)                       # torch.int64
w = torch.randn(2)
print(w)                                 # tensor([0.3918, 0.9243])
torch.manual_seed(42)
print(torch.randperm(len(rooms))[:5])    # tensor([ 292, 2688, 1152, 2249, 1480])

Theory Assessment

Q1: One Line, Ten Libraries, One Pin

The install line names one library and uv printed ten + lines. Explain why only one of the ten versions is promised to match yours, why NumPy had no line of its own, and what --index-url changed about what was fetched. Then say what torch.cuda.is_available() printing False tells you about the install, and what a tensor has that the ndarray behind it does not.

Hint: Look at which name carries == in the install line, at what the other nine have in common, and at what PyPI's default build of torch carries beside it. For the last part, count the attributes first_tensor.py printed that NumPy's array also has.

Q2: One Warning, Two Columns, One Frame

A script builds area = torch.from_numpy(listings["area_sqm"].to_numpy()) and sees the warning, then builds rooms = torch.from_numpy(listings["rooms"].to_numpy()) a few lines later and sees nothing. A teammate concludes that the rooms column was safe to wrap, and later centers the areas in place with area -= 99.02. Explain what the warning said, why the second call was silent, what the in-place subtraction did to listings, and what the module's line does differently.

Hint: Reread the warning's last sentence, and ask whether to_numpy() on the second column hands out a different kind of array. For the subtraction, ask whose memory area is a view of.

Q3: Seven Digits

area.mean().item() printed 99.02003479003906, torch.tensor(175.7).item() printed 175.6999969482422, and area.sum() == area.double().sum() printed tensor(False) for two sums of the same 2,875 areas. Say which digits of the first number are data, why the second number is the same on every machine while the last digits of area.sum().item() are not, and how this lesson prints a float32 mean, a float32 sum and a float32 comparison.

Hint: Ask how many significant digits a float32 carries, whether a single literal involves any addition, and what 2,875 additions in steps of about 0.03 do to the last digits. The printing rules are the ones every output comment in this lesson followed.

Q4: Four Tensors, Two Streams

Read this script without running it, then say what each print shows.

import numpy as np
import torch

arr = np.array([175.7, 30.5, 143.6, 42.0])
a = torch.tensor(arr)
b = torch.from_numpy(arr)
c = torch.tensor([175.7, 30.5, 143.6, 42.0])
arr[1] = 0.0

print(a[1].item(), b[1].item())
print(a.dtype == c.dtype)
print(a == c)
torch.manual_seed(42)
first = torch.rand(2)
torch.manual_seed(42)
second = torch.rand(2)
print(first)
print(second)

A teammate reads the two Falses in the third line as proof that torch.tensor(arr) and torch.tensor on the list stored different areas at positions 0 and 2, and the True at position 1 as a sign that arr[1] = 0.0 reached a as well. Explain where each reading goes wrong.

Hint: Ask which of the four literals have an exact binary form (a number does when it is a whole number or a whole number plus halves, quarters and so on). Then ask what a[1].item() on the first line already told you.

30.5 0.0
False
tensor([False,  True, False,  True])
tensor([0.8823, 0.9150])
tensor([0.8823, 0.9150])

Assignment

Assignment: A Tensor Bundle for the Loops Ahead

Section 4.2 fits a price model through a loop you write by hand, and a project that grows past one script needs every script to load the same tensors and the same split. Rebuilding them in each script from the CSV is where drift starts: one script forgets the dropna, another draws its permutation after a different number of random calls, and two loss curves stop being comparable without anyone noticing. Your team wants the tensors built once, checked once and written to a file that every later script loads.

Write tensor_bundle.py. From the listings that have an area, build X, a float32 tensor of area_sqm, rooms and distance_km in that order, and y, a float32 tensor of price. Print the shape and dtype of each, the number of nans in X, and X's first row. Show with np.shares_memory that X does not share memory with the frame's array, then print each feature's name beside its mean rounded to two decimals, taking column j of X as X[:, j].

Seed PyTorch with 42 and draw a permutation of the 2,875 row positions. Print its first five entries, index X and y with its first 2,300 positions as the training rows and with the rest as the test rows, print the shape of the training features and of the test prices, and print the first three training prices.

Save X, y, the permutation and the list of feature names in one dict as listings_tensors.pt, then load the file back and show that what came back is what went in. Two calls this lesson did not teach do the file work: torch.save(obj, path) writes a dict, a list or a tensor to a file, and torch.load(path) reads it back as the same kind of object, with new tensors in it. torch.equal(x, y), which the lesson did not use either, returns True only when two tensors have the same shape and every element equal, so it is the check for a saved tensor against its original. Print the keys of the loaded dict, the dtype of its X, the result of torch.equal for X and for the permutation, and whether the loaded X is the same object as the one you saved.

Hints:

• has_area[features].to_numpy() hands torch.tensor a (2875, 3) float64 array and dtype=torch.float32 converts all three columns at once, and enumerate(features) pairs each name with the column X[:, j] whose mean it labels.
• The permutation indexes X and y as Section 2.2 indexed with an integer array, X[perm[:2300]], and torch.save takes the dict and the file name while torch.load takes the file name alone.

import numpy as np
import pandas as pd
import torch

listings = pd.read_csv("listings.csv")
has_area = listings.dropna(subset=["area_sqm"])
features = ["area_sqm", "rooms", "distance_km"]

X = torch.tensor(has_area[features].to_numpy(), dtype=torch.float32)
y = torch.tensor(has_area["price"].to_numpy(), dtype=torch.float32)
print(X.shape, X.dtype)      # torch.Size([2875, 3]) torch.float32
print(y.shape, y.dtype)      # torch.Size([2875]) torch.float32
print(torch.isnan(X).sum())  # tensor(0)
print(X[0])                  # tensor([175.7000,   6.0000,   9.6000])
# False, torch.tensor copied the frame's numbers
print(np.shares_memory(X.numpy(), has_area[features].to_numpy()))
for j, name in enumerate(features):
    print(name, round(X[:, j].mean().item(), 2))

area_sqm 99.02
rooms 3.56
distance_km 5.5

torch.manual_seed(42)
perm = torch.randperm(len(X))
print(perm[:5])                           # tensor([ 292, 2688, 1152, 2249, 1480])
train_rows = perm[:2300]
test_rows = perm[2300:]
print(X[train_rows].shape)                # torch.Size([2300, 3])
print(y[test_rows].shape)                 # torch.Size([575])
print(y[train_rows][:3])                  # tensor([405000., 754000., 343000.])

bundle = {"X": X, "y": y, "perm": perm, "features": features}
torch.save(bundle, "listings_tensors.pt")
loaded = torch.load("listings_tensors.pt")
print(list(loaded))                       # ['X', 'y', 'perm', 'features']
print(loaded["X"].dtype)                  # torch.float32
print(torch.equal(loaded["X"], X))        # True
print(torch.equal(loaded["perm"], perm))  # True
print(loaded["X"] is X)                   # False

Additional Resources

Optional, and nothing later depends on either. Worth opening if you want the full list of dtypes a tensor can take, or the rules that decide when a seeded run repeats and when it cannot.

• Official docs - PyTorch 2.14, "Tensor Attributes", the section "torch.dtype". The two tables of every dtype a tensor can hold, float32 and float64 beside the 16-bit floats, the integer widths and torch.bool, with the aliases some carry, torch.float for torch.float32, torch.double for torch.float64 and torch.long for torch.int64.
• Official docs - PyTorch 2.14, "Reproducibility". What torch.manual_seed promises and what it does not, the note that results are not guaranteed to repeat across releases or platforms, the seeds Python's random and NumPy need of their own, and torch.use_deterministic_algorithms, which switches every operation that has a deterministic algorithm to it and raises an error on the ones known to have none.

Summary

A tensor turned out to be an array with two promises attached: it knows where it lives, and it can remember how it was made. Neither promise was used in this lesson, and both are the reason the lesson exists, because everything from the next derivative to the last network runs on the container this one built. What the lesson did use was Section 2.2's habits, carried across unchanged: check the dtype, check the shape, ask whether a selection shares memory with its source.

The traps were all about where a number came from and where it was going. A column wrapped without a copy pointed back into the frame, a list of floats arrived in float32 where NumPy would have given float64, a missing value rode into a mean, and a statistic read to sixteen digits carried seven of data and nine of representation. None of them stopped a program. Each was caught by printing something small, a flag, a dtype, a count of nans, a rounded number, which is the habit this module asks for at every boundary a tensor crosses.

The seed closed the lesson because it opens the ones to come. From Lesson 4.2.2 on, every model starts from a random draw, and a result that cannot be drawn again cannot be debugged, compared or trusted. One line at the top of a script, before the first draw, is the whole discipline, and the rest of the module assumes it is there.

Cheat Sheet

Part A - Reference Table

Part B - Compact Code Block

import numpy as np
import pandas as pd
import torch

listings = pd.read_csv("listings.csv")
rows = listings.dropna(subset=["area_sqm"])  # one nan poisons a mean
area = torch.tensor(rows["area_sqm"].to_numpy(), dtype=torch.float32)
area.dtype, area.shape, area.device          # float32, Size([2875]), cpu
round(area.mean().item(), 2)                 # 99.02, the digits data has
round(area.sum().item())                     # a float32 sum prints whole
area.double().sum()                          # float64, to check NumPy
# never ==
torch.isclose(area.mean(), torch.tensor(99.02))

arr = np.array([175.7, 30.5])
torch.tensor(arr)             # a copy, float64 like the array
torch.from_numpy(arr)         # shares arr's memory
torch.tensor([175.7, 30.5])   # float32, PyTorch's default

torch.manual_seed(42)         # once, before the first draw
w = torch.randn(2)            # tensor([0.3367, 0.1288])
perm = torch.randperm(2875)   # a shuffled range, int64
torch.zeros_like(w)           # zeros in w's shape and dtype
torch.save({"w": w}, "w.pt")  # a dict of tensors to a file
torch.load("w.pt")["w"]       # back, a new tensor