Packages, pip, and Virtual Environments

Context

Code written by other people is distributed as packages and installed with a package installer. A virtual environment is a private directory of installed packages belonging to one project, and the exact versions in use can be written to a file so the set can be recreated later.

Every library the rest of this bootcamp installs arrives this way. Isolation is what keeps one project's version of a library from deciding another project's behavior, and the recorded version list is what makes a result reproducible, because the same code against a different library version is a different experiment. This is also where the rule Module 0's setup page set, never install into the system interpreter, is put to work.

Skipping the environment produces failures that are hard to even describe. A script that worked last month breaks after an unrelated project upgraded a shared library, and the code has not changed at all. Without a recorded version list, a colleague running your pipeline gets different numbers and neither of you can explain why. Installing into the wrong active environment is the everyday version, the package installs successfully and the import still fails. The previous lesson organized your own files, this lesson brings in everyone else's.

Goal

• Organize related modules into a package directory and import across it with dotted paths.
• Distinguish a third-party package from a standard-library module.
• Create and activate an isolated virtual environment for a project.
• Install a third-party package with pip install into that environment.
• Pin a project's dependencies in requirements.txt and recreate the environment from it.

Prerequisites

Optional, neither is required. The lesson stands on its own, they are here if you want to watch an environment being made before you make one.

• YouTube - "Python Tutorial: VENV (Mac & Linux) - How to Use Virtual Environments with the Built-In venv Module" by Corey Schafer. The create-and-activate portion, for the mental model of isolation.
• Official docs - venv, "Creating virtual environments". The exact creation command and what the folder it builds contains.

Table of Contents

• Package
• Third-Party Package
• Virtual Environment
• pip install
• requirements.txt

Theory

Package

What. A package is a directory of related modules treated as a single import unit, addressed with dotted paths like data.loader. Why. As modules multiply, a flat pile of files becomes unmanageable, grouping them into packages gives a project a navigable shape, data code here, model code there. When. You create a package the moment a single folder of loose modules would be clearer as a named group, typically once you have several modules that belong together.

A package is just a folder, with the modules as .py files inside it. Historically a file named __init__.py (often empty) inside the folder marks it as a package, and including it remains the clear, safe convention. You then import using the dotted path: the package name, a dot, then the module name, and once the module is imported, a name inside it is one more dot away.

bootcamp/
├── data/
│   ├── __init__.py      # marks 'data' as a package (can be empty)
│   ├── loader.py        # module reachable as 'data.loader'
│   └── splitter.py      # module reachable as 'data.splitter'
└── main.py              # the entry-point script

The folder name becomes the package name and each file inside becomes a dotted sub-name, so the tree is already the import path written down.

The package lives in ~/dev/bootcamp like every Module 1 script before this lesson, with an empty data/__init__.py and these two modules.

# === file: data/loader.py ===
# 'load' returns the raw records, a fixed list standing in for a file
def load():
    return [0.30, 0.25, 0.20, 0.22]  # per-epoch training loss

# === file: data/splitter.py ===
# 'split' cuts the records into a first half and a second half
def split(records):
    cut = len(records) // 2
    return records[:cut], records[cut:]

# === file: main.py ===
# dotted path: reach 'load' inside module 'loader' inside package 'data'
from data.loader import load
# or bring in the whole module and use its prefix
import data.splitter

records = load()                      # call the from-imported function
parts = data.splitter.split(records)  # reach through the full dotted path
print(parts)                          # ([0.3, 0.25], [0.2, 0.22])

Both import forms used here, from Lesson 1.7.1, survive intact, they simply gain a folder layer in front of the module name.

DON'T:

• DON'T put a hyphen in a package directory name like data-utils, dotted imports such as import data-utils are not valid Python.
• DON'T assume an empty __init__.py is useless, its presence is what reliably and unambiguously marks the folder as an importable package.

Third-Party Package

What. A third-party package is code written by someone outside the Python core team, published online, and installed into your project rather than shipped with Python. Why. The standard library cannot cover everything, the big data and ML libraries, numpy, pandas, scikit-learn, live outside it, and you install what each project needs. When. You reach for a third-party package whenever the standard library does not solve your problem and a well-maintained external library does.

Third-party packages are fetched from the Python Package Index, the public repository commonly called PyPI. They are not present in a fresh Python, so importing one before installing it raises ModuleNotFoundError. The numerical library numpy (a third-party package for fast numeric arrays, which you will use heavily in the coming ML modules) is the example this lesson installs.

# 'numpy' is a THIRD-PARTY package, NOT part of the standard library
# this import only works AFTER 'numpy' has been installed
import numpy as np                  # 'np' is the conventional, widely-used alias

# 'np.array' builds a fast numeric array from a list
arr = np.array([0.91, 0.88, 0.94])  # accuracy scores
# numpy's own mean, far faster than a Python loop on big data
print(arr.mean())                   # 0.91

Before numpy is installed this very import fails with ModuleNotFoundError: No module named 'numpy', because nothing third-party ships with Python. The as np alias is a near-universal convention, and following it is what keeps your code readable to everyone else.

DON'T:

• DON'T assume a third-party package is automatically available, every machine must install it before the import works.
• DON'T install third-party packages into your system-wide Python, install them into a per-project virtual environment instead, covered next.

Virtual Environment

What. A virtual environment is an isolated, per-project set of installed packages, kept in a folder so it does not mix with the system Python or other projects. Why. Two projects can need different versions of the same package, and installing globally makes them collide, isolation gives each project its own clean, independent set. When. You create a virtual environment at the very start of every project that will use a third-party package, before installing a single one.

(bootcamp) is a virtual environment already, built with uv venv on Module 0's setup page, and this subtitle builds one with the standard-library venv tool instead, a self-contained folder conventionally named .venv. You then activate it, which switches your current shell to use that folder's Python and pip. Activation works by changing your PATH (the ordered list of folders your shell searches for commands, from Module 0): it puts the environment's bin folder first, so typing python3 or pip now finds the environment's copies. Deactivating undoes that change.

# a folder of its own, '~/dev/bootcamp/.venv' is (bootcamp) itself
mkdir ~/myproject && cd ~/myproject
# 'python3 -m venv' creates an isolated environment in a folder '.venv'
python3 -m venv .venv

The new environment is not active yet, it has only been created, and (bootcamp) is still the active one. The -m venv form runs the standard-library venv module as a tool, so it needs no installation of its own on a Python that can build an environment. (bootcamp)'s Python can, and stock Ubuntu's cannot without its python3-venv package.

# 'source .../activate' switches THIS shell to use the venv
source .venv/bin/activate
# the prompt now shows (.venv), prove the switch worked:
which python3  # /home/goku/myproject/.venv/bin/python3
# 'deactivate' leaves the environment when finished
deactivate

source is doing something specific and easy to miss: it runs the script in your current shell rather than in a child of it, which is the only reason a PATH change can outlive the command. Running it as bash .venv/bin/activate instead would change the PATH of a child process that then exits, leaving your shell untouched. This is the same environment-variable mechanism from Module 0, and deactivate puts back the PATH this activation saved, which no longer holds (bootcamp): activating an environment first switches off whichever one is already active, so after deactivate no environment is active at all and python3 is the system one, which on stock Ubuntu cannot build an environment. Run source ~/dev/bootcamp/.venv/bin/activate to return to (bootcamp).

DON'T:

• DON'T commit the .venv/ folder to version control, it is large and machine-specific, recreate it from requirements.txt instead.
• DON'T install project packages before activating, an inactive environment means the install either fails outright or lands in the wrong Python.

pip install

What. pip install is the command that downloads a package from PyPI and installs it into the active environment, when that environment has a pip of its own. Why. It is how third-party code arrives in an environment python3 -m venv built, turning a failing import numpy into a working one. When. You run it right after activating a project's virtual environment, every time you add a new dependency.

pip is Python's package installer, and python3 -m venv puts a copy of it in every environment it builds by default, while (bootcamp), built by uv venv, has none. The crucial detail is where it installs: into whichever environment is active. Activate the virtual environment first, and the install is isolated to that project. You can install the latest version, or pin an exact one with ==.

# activate FIRST so the install is isolated to this project
source .venv/bin/activate
# 'pip install numpy' downloads the newest numpy into the venv
pip install numpy
# pin an EXACT version with '==' for reproducibility
pip install numpy==2.5.2
# 'pip list' shows what is installed in THIS environment
pip list

Adding ==2.5.2 requests one specific release rather than "the newest", which is how an install stays predictable months later. pip list is the confirmation step worth taking, because it reports what this environment contains and nothing about any other.

# 'pip show numpy' prints details: version, location, dependencies
pip show numpy

The location it reports should sit inside your .venv. If it points anywhere else, or pip is not found at all, you forgot to activate, and that one line is the fastest way to find out.

DON'T:

• DON'T run sudo pip install. Ubuntu 24.04 has no system pip until one is installed, and once it has one it refuses with error: externally-managed-environment, the guard Module 0's setup page described, because the system Python belongs to the operating system's own tools.
• DON'T pip install for a project without its environment active, the install either fails outright or lands in a Python the project never runs.

requirements.txt

What. requirements.txt is a plain text file listing a project's dependencies with pinned exact versions, one per line. Why. It makes a project reproducible, anyone, including future you on another machine, can recreate the same packages, at the same versions, from this one file. When. You write it as soon as a project has dependencies, and update it whenever you add or change one.

You generate the file with pip freeze, which prints every installed package except pip itself, each with its exact version, and you redirect that output into the file. To rebuild the environment elsewhere, you create a fresh virtual environment and install from the file with pip install -r. The pinned versions guarantee everyone runs the identical set.

# 'pip freeze' prints every installed package but pip, with its EXACT version
pip freeze
# the '>' redirects that listing INTO the file 'requirements.txt'
pip freeze > requirements.txt

The > overwrites rather than appends, so each run replaces the whole file rather than adding to an accumulating list.

# === file: requirements.txt ===
numpy==2.5.2

Pinning is the whole point. numpy==2.5.2 means everyone gets that exact release, so a teammate runs your script against the numpy it worked with, with no "it broke on my machine" surprise from a silently newer version.

# on a FRESH machine or clone, with a Python that can build environments, reinstall the pinned packages
python3 -m venv .venv      # create the isolated environment
source .venv/bin/activate  # activate it
# '-r' reads the file and installs every pinned line
pip install -r requirements.txt

That is the payoff of the whole lesson. Most of the data and ML libraries the rest of this bootcamp relies on, numpy and pandas among them, live outside the standard library, and this file plus a virtual environment, built on the same Python version, is what makes a project run the same way on a machine it has never seen.

DON'T:

• DON'T leave dependencies unpinned (numpy instead of numpy==2.5.2) in a project meant to be reproducible, an unpinned name silently installs whatever is newest.
• DON'T run pip freeze > requirements.txt without the project's environment active. With (bootcamp) active, or no environment at all, pip is not found, and the > empties requirements.txt anyway, so the pinned record is gone.

Lab Exercises

Lab Exercise 1: Build a Package, Then Try to Import Something You Do Not Have

Work in ~/dev/bootcamp with (bootcamp) active, like every Module 1 script before this lesson. Create a package mathkit/ containing an empty __init__.py and a module basics.py with a function double(x) returning x * 2. From a main.py outside the package, import and call double two ways, through the full dotted path and through a from-import, and print both results.

Then, from the terminal and without installing anything, run two one-line checks: one that imports your package and prints a confirmation, and one that imports numpy. Record the exact output of each, including the error, and say why Python found the one and not the other.

Lab Exercise 2: Isolate, Install, Pin, and Rebuild From the File Alone

In a new, empty folder ~/project, made with mkdir ~/project && cd ~/project, create a virtual environment named .venv and activate it. Before installing anything, run which python3 and record what it prints, then compare it against what the same command printed before activation.

Install numpy pinned to the exact version 2.5.2, then verify it with a one-line check that prints numpy.__version__. Write the environment to requirements.txt with pip freeze.

Now simulate a fresh machine. Deactivate, delete .venv entirely, then return to (bootcamp), recreate the environment, activate it, and install from requirements.txt alone. Run the same verification line and add a comment stating what was needed to rebuild the environment and what was not.

bootcamp/
├── mathkit/
│   ├── __init__.py   # empty, and its presence marks 'mathkit' a package
│   └── basics.py     # module 'mathkit.basics'
└── main.py

# === file: mathkit/basics.py ===
def double(x):
    return x * 2

# === file: main.py ===
import mathkit.basics              # binds the MODULE, reached through the dotted prefix
from mathkit.basics import double  # binds the bare function name

print(mathkit.basics.double(5))    # 10
print(double(7))                   # 14

$ python3 -c "import mathkit.basics; print('local package: fine')"
local package: fine

$ python3 -c "import numpy"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'

# A new, empty folder for this lab.
mkdir ~/project && cd ~/project

# Before anything: 'python3' still resolves to (bootcamp), outside this project.
which python3              # /home/goku/dev/bootcamp/.venv/bin/python3

python3 -m venv .venv      # build the isolated environment, no install needed
source .venv/bin/activate  # switch THIS shell to the venv's copies

which python3              # /home/goku/project/.venv/bin/python3

pip install numpy==2.5.2       # the '==' asks for one exact release
# 2.5.2
python3 -c "import numpy; print(numpy.__version__)"
pip freeze > requirements.txt  # write the pinned manifest

# Simulate a fresh machine: throw the environment away entirely.
deactivate                                # leave BEFORE deleting, or the shell points into nothing
rm -rf .venv
source ~/dev/bootcamp/.venv/bin/activate  # back to (bootcamp), which can build a venv

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt           # '-r' reads and installs every pinned line

python3 -c "import numpy; print(numpy.__version__)"
# 2.5.2
# Rebuilding needed 'requirements.txt', a Python that can build a venv,
# and three commands. It did not need
# the old '.venv', which is why that folder is never committed and never
# missed, the file is the artifact and the folder is disposable.

Theory Assessment

Q1: What Makes a Folder a Package, and What Makes a Package Third-Party

Explain what turns an ordinary folder of .py files into an importable package, and how you would then import a module inside it. Then state the difference between a standard-library module and a third-party package, and the one extra step the third-party one requires before an import of it can work.

Hint: For the first half, think about the conventional marker file and the dotted path. For the second, ask which of the two is already on a fresh machine.

Q2: What Activation Actually Changes, and Where an Install Lands

Explain, in terms of PATH, what happens when you activate a virtual environment and why which python3 is enough to prove it worked. Then explain why pip install pandas can report success while an import of pandas fails in a different terminal a minute later.

Hint: Activation does not copy or move anything. For the second half, ask which environment was active at the moment each of the two commands ran.

Q3: Why the Version Is Written Down, and What Gets Committed

Explain why numpy==2.5.2 in requirements.txt is preferable to a bare numpy for a project other people will run. Then say which commands a teammate runs after cloning a project that has a requirements.txt and no .venv, and why .venv was not in the clone in the first place.

Hint: For the first half, ask what "newest available" means to somebody installing six months from now. For the second, given a Python that can build an environment, the answer is three commands and always the same three.

Q4: The Install Worked and the Import Still Fails

Read the following terminal session, then say exactly what the last command prints and why.

(bootcamp) goku@machine:~/project$ python3 -m venv .venv
(bootcamp) goku@machine:~/project$ source .venv/bin/activate
(.venv) goku@machine:~/project$ pip install numpy==2.5.2
Collecting numpy==2.5.2
  Downloading numpy-2.5.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (6.6 kB)
Downloading numpy-2.5.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.7/16.7 MB 6.0 MB/s eta 0:00:00
Installing collected packages: numpy
Successfully installed numpy-2.5.2

[notice] A new release of pip is available: 24.3.1 -> 26.2.1
[notice] To update, run: pip install --upgrade pip
(.venv) goku@machine:~/project$ deactivate
goku@machine:~/project$ source ~/dev/bootcamp/.venv/bin/activate
(bootcamp) goku@machine:~/project$ python3 -c "import numpy; print(numpy.__version__)"

Nothing in that session failed, and pip said so in writing. Say what the last line does, name the one command that would have exposed the mistake before the import ran, and say what the fix is.

Hint: Two different python3 executables appear in this session and only one of them is ever told about numpy. Work out which interpreter each command was talking to, in the order they ran.

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'

source .venv/bin/activate
# 2.5.2
python3 -c "import numpy; print(numpy.__version__)"

Assignment

Assignment: A Fully Reproducible Package Project

Build a complete small project in this order, in a new folder ~/pipeline-project of its own.

A package pipeline/ holding __init__.py, a load.py exposing load_values() that returns a list, and a score.py exposing score(values) that returns their numpy mean. Keep load.py free of any third-party import so the dependency has exactly one home.

A main.py entry point that imports both package functions and runs the two steps behind a main guard.

Then the environment: create and activate a .venv, install numpy, and pin it with pip freeze into requirements.txt. Add a .gitignore that excludes the environment folder from version control.

Finally, make the pinning trade-off concrete rather than describing it. Write a second file requirements-compatible.txt that asks for numpy with the flexible ~= specifier instead of ==, and state in a comment what each of the two files would install if a numpy 2.5.9 and a numpy 2.6.0 were both released tomorrow.

Hints:

• Confine import numpy to score.py alone, so the rest of the package can be imported on a machine that has never installed it.
• ~=2.5.0 means "this minor series only", so ask which of 2.5.9 and 2.6.0 still counts as 2.5.*.

pipeline-project/
├── pipeline/
│   ├── __init__.py               # marks 'pipeline' a package
│   ├── load.py                   # dependency-free
│   └── score.py                  # the ONE module that imports numpy
├── main.py                       # entry point behind a main guard
├── requirements.txt              # pinned with '==', reproducible
├── requirements-compatible.txt   # flexible with '~=', compatible updates
└── .gitignore                    # excludes '.venv/'

# === file: pipeline/load.py ===
def load_values():
    return [10, 20, 30, 40]  # sample data, no third-party code here

# === file: pipeline/score.py ===
import numpy as np  # the single home of this project's dependency

def score(values):
    return np.array(values).mean()

# === file: main.py ===
from pipeline.load import load_values
from pipeline.score import score

def main():
    values = load_values()
    print("mean score:", score(values))  # mean score: 25.0

if __name__ == "__main__":
    main()

cd ~/pipeline-project
python3 -m venv .venv
source .venv/bin/activate  # activate BEFORE installing, always
pip install numpy==2.5.2
pip freeze > requirements.txt
python3 main.py
# mean score: 25.0

# === file: requirements.txt ===
numpy==2.5.2

# === file: requirements-compatible.txt ===
numpy~=2.5.0

# === file: .gitignore ===
.venv/

# If numpy 2.5.9 and numpy 2.6.0 were both released tomorrow:
#   requirements.txt             installs 2.5.2, exactly as it did today.
#   requirements-compatible.txt  installs 2.5.9, because '~=2.5.0'
#                                accepts any 2.5.* and refuses 2.6.0,
#                                which is a new series.

Additional Resources

Optional, and nothing later depends on either. Worth opening if you want the pinning practice in depth or a look at where a modern project declares its dependencies.

• Official docs - pip, "Repeatable Installs", the "Pinning the package versions" section. Pinning and reproducible installs in more detail than one lesson can carry.
• Official docs - Python Packaging, the pyproject.toml specification, "Dependencies". A forward look at the modern project manifest and the key where a project declares the packages it depends on.

Summary

This lesson is where a pile of scripts finally becomes a project. A package gives your modules a shape, so that "the data code" and "the scoring code" are not loose files but named groups you import through a path that reads like a sentence. That structure is not decoration, it is what lets you find anything in a codebase you have not opened in a month, and it is the form every serious Python project takes. The dotted import you learned is the same import from earlier lessons, only now it carries a folder of organization in front of it.

The deeper shift is the realization that your project does not live alone on a perfect machine. Most of the libraries that make data and machine-learning work possible live outside Python, and the moment you depend on one, you have made a promise that the next machine can reproduce what yours does. The virtual environment is how you keep that promise honest, an isolated, throwaway folder that holds this project's dependencies apart from every other project's. Activation is a small trick with the PATH you met in Module 0, but its consequence is large: two projects on one computer stop fighting over versions, because neither one is global anymore.

The pinned requirements.txt is the quiet hero of all of this. It is a few lines of text, yet it is the difference between a project that runs only where it was born and one that runs anywhere someone is willing to recreate the environment. From here on, every project that needs a third-party package starts the same way, make the environment, install what you need, pin it, ignore the environment folder in version control, and that small ritual is what separates code that happens to work from code that is genuinely shippable. You now have the full arc of this section: from one script, to many files, to the standard library, to a packaged, reproducible project.

Cheat Sheet

# the full reproducible-project ritual, start to finish
python3 -m venv .venv            # 1. isolated environment
source .venv/bin/activate        # 2. activate (changes PATH)
pip install numpy                # 3. install a third-party package
pip freeze > requirements.txt    # 4. pin exact versions
deactivate                       # 5. leave when done

# on another machine, from the project files only,
# with a Python that can build environments:
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt  # reinstall the pinned packages

# importing across a package + a third-party library
from data.loader import load  # dotted path into a local package
import numpy as np            # third-party, installed in the venv