Skip to content
  • Home
  • How To
  • M0
  • M1
  • Agents
  • M2
  • M3
  • M4
  • M5
  • Module 6 - MLOps
  • Progress
  • Contact
  • Pricing
  • Sign in
  • Future
  1. Module 6M6
  2. ›
    S3Containers, Databases and Pipelines
  3. ›
    Lesson 4 of 4Automated Pipelines
  4. ›
    S3›Lesson 4 of 4
  1. Module 6
  2. ›
  3. Section 3
  4. ›
  5. Lesson 4

Automated Pipelines

FREE·~1.5 h·2 labs·4 questions·1 assignment


Context

A pipeline here is a file in the repository that tells GitHub what to run on every push to main and every pull request: check the code out on a fresh machine, install the locked environment, run the tests against a real PostgreSQL, then train the model, build the image and ask the running container for one price. GitHub Actions, GitHub's service for running such files, reads it from .github/workflows/, runs it on a rented Linux machine, and marks the commit with a green check or a red cross.

This is the machine a team trusts instead of each other's memory. A pull request from a teammate is tested before anyone reads it, a Monday morning push of a new pin either builds or says exactly why not, and the image that will later be deployed is built from the commit, not from whatever happened to sit in someone's project folder. It is the same check you have run by hand since Lesson 6.1.2, moved to a machine that never skips it.

Without it, every green result lives on one laptop. The database test skips wherever DATABASE_URL is unset, model/ and mlflow.db exist only where someone created them, and a lock that uv run quietly rewrote can pass every local test while the commit carries a stale one. Lesson 6.3.3 left an image that builds and a compose stack that runs, both on your machine, and this lesson is the first time the project runs somewhere you have never touched.


Goal

  • Write a GitHub Actions workflow with a test job that runs pytest against a PostgreSQL service container and an image job that trains, builds and smoke tests the service.
  • Check a workflow with actionlint before pushing, and name which mistakes it catches and which it lets through.
  • Push the project to a private GitHub repository over SSH and read a run on the Actions tab, job by job and step by step.
  • Explain why the image job trains its own model, and trace a missing model/ or a container that cannot start to the step that fails and the exit code it reports.
  • Reproduce a stale lock in a fresh clone, distinguish uv sync --locked, --frozen and a plain uv sync, and fix the commit that caused it.

Prerequisites

Neither is required, and the lesson stands on its own without them. They are here if you want the picture of GitHub Actions before you build one.

  • YouTube - "GitHub Actions Tutorial - Basic Concepts and CI/CD Pipeline with Docker" by TechWorld with Nana. Thirty-two minutes on what GitHub Actions automates, the event, workflow, job and action vocabulary, and a pipeline that builds a Docker image.
  • GitHub documentation - "Understanding GitHub Actions". Workflows, events, jobs, actions and runners, each defined in a paragraph, the steps of a job described under jobs, and each job running on its own virtual machine or in a container.

Table of Contents

  • A Workflow File
  • The Test Job and Its Database
  • The Image Job and Its Smoke Test
  • Pushing to GitHub and Reading the Run
  • A Stale Lock Stops the Run

Theory

Everything runs in ~/dev/listings-service at the state Lesson 6.3.3 left it, with (bootcamp) deactivated: the layered Dockerfile, compose.yaml, model/ exported from the registry's champion, and the tests that pass with or without a database. The lesson adds one project file, .github/workflows/ci.yml, and one commit. The fresh clone it works in is ~/dev/ci-check, beside the project, and it is deleted at the end.

A Workflow File

What. A workflow is a YAML file, the indented key-and-value text format compose.yaml is written in, saved under .github/workflows/ in the repository. Its on: block names the events that start it, a push or a pull request. Its jobs: block names one or more jobs, and each job runs on a runner, a fresh virtual machine GitHub starts for that job alone and discards afterwards. A job is a list of steps run in order, and each step either runs a published action with uses:, a packaged piece of setup code someone else maintains, or runs shell commands with run:. Why. The file lives in the repository, so the check travels with the code it checks: a commit that changes the tests or the Dockerfile changes the workflow in the same history, and a commit is always checked by the version of the workflow it carries. The runner starts empty every time, so a pass means the commit alone was enough. When. In every project more than one person or more than one machine touches, from the first push on.

Two things are needed that the project does not install, and neither goes into pyproject.toml. The first is a GitHub account with the SSH key Lesson 0.3.4 added, which Module 6's setup page checked with ssh -T git@github.com. The second is actionlint, a checker for workflow files, which uvx runs: uvx is uv's command for running a tool from the package index in a throwaway environment of its own, outside the project's .venv, so it downloads the tool on first use and changes neither pyproject.toml nor uv.lock.

Create the folder and save .github/workflows/ci.yml:

mkdir -p .github/workflows
name: ci

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-24.04
    services:
      postgres:
        image: postgres:18.6
        env:
          POSTGRES_USER: listings
          POSTGRES_PASSWORD: listings
          POSTGRES_DB: listings
        ports:
          - 5432:5432
        options: >-
          --health-cmd "pg_isready -h localhost -U listings -d listings"
          --health-interval 2s
          --health-retries 15
    env:
      DATABASE_URL: postgresql://listings:listings@localhost:5432/listings
    steps:
      - uses: actions/checkout@v7.0.1
      - uses: astral-sh/setup-uv@v10.3.0
        with:
          version: "0.13.0"
      - run: uv sync --locked
      - run: uv run pytest

  image:
    needs: test
    runs-on: ubuntu-24.04
    steps:
      - uses: actions/checkout@v7.0.1
      - uses: astral-sh/setup-uv@v10.3.0
        with:
          version: "0.13.0"
      - run: uv sync --locked
      - name: Train, promote and export the model
        run: |
          uv run python -m listings.train
          uv run python -m listings.promote 1
          uv run mlflow artifacts download -u models:/listings-price@champion -d model
      - run: docker build -t listings-service .
      - name: Smoke test the image
        run: |
          docker run -d --name listings-service -p 8000:8000 listings-service
          for i in $(seq 1 60); do
            curl -sf http://127.0.0.1:8000/health && break
            sleep 1
          done
          curl -sf -X POST http://127.0.0.1:8000/predict \
            -H "Content-Type: application/json" \
            -d '{"area_sqm": 85.0, "year_built": 1998, "distance_km": 3.0, "rooms": 3, "bathrooms": 1, "neighborhood": "Riverside", "property_type": "apartment", "energy_rating": "C", "condition": "good"}'

The file is the project's own, so it carries no comments, and the top half reads as follows. name: ci is what the Actions tab calls the workflow. on: starts it on a push to main and on every pull request, so a branch is checked when someone proposes merging it and main is checked again once it lands. test and image are job ids, names you choose. runs-on: ubuntu-24.04 asks for GitHub's Ubuntu 24.04 machine, an exact image where the label ubuntu-latest moves to a newer Ubuntu when GitHub decides to. actions/checkout@v7.0.1 copies the commit being tested onto the runner, and astral-sh/setup-uv@v10.3.0 installs uv itself, with version: "0.13.0" pinning the same uv the Dockerfile copies from ghcr.io/astral-sh/uv:0.13.0, so the runner and the image resolve the lock with one uv. The text after @ is a release tag of the action's repository, which pins the action's code the way == pins a library. A run: step's commands run under bash -e, so the step stops at the first command that exits with anything but 0 and reports that exit code, and the job stops with it. run: | takes the lines below it as one script, and name: labels a step on the Actions tab, where a step without one is shown as its command. The services, the second job and the smoke test belong to the next two subtitles.

A workflow with a mistake in it is only found out after a push, on GitHub, so check it first. From the project folder:

uvx --from actionlint-py actionlint
echo $?
   Building actionlint-py==1.7.12.25
      Built actionlint-py==1.7.12.25
Installed 1 package in 1ms
0

--from actionlint-py names the package that provides the actionlint command, and with no arguments actionlint checks every file in .github/workflows/. The first three lines are uvx installing it, printed on the first run only, and actionlint itself prints nothing for a clean file and exits with 0. Now change needs: test to needs: tests and run it again:

.github/workflows/ci.yml:34:3: job "image" needs job "tests" which does not exist in this workflow [job-needs]
   |
34 |   image:
   |   ^~~~~~
1

The error names the file, the line and column, the problem, and the rule in brackets, and the exit code is 1. A step written runs: instead of run: fails the same way, with step must run script with "run" section or run action with "uses" section [syntax-check]. A misspelled input does not: versoin: "0.13.0" under setup-uv, and fetch-deep: 1 under checkout, both passed with exit code 0, because actionlint knows the structure of every workflow and the inputs of only the actions it has a list for, and these two releases are not on it. The same fetch-deep: 1 under actions/checkout@v4, an older release on its list, was caught with input "fetch-deep" is not defined in action "actions/checkout@v4" and the inputs it does take. A misspelled with: key leaves the action on its default, the newest uv for setup-uv, and nothing on your machine says so. Put needs: test back and check again, which prints nothing.

DON'T:

  • DON'T pin runs-on to ubuntu-latest. The label points at a newer Ubuntu whenever GitHub moves it, and a run that passed yesterday then fails on a machine the commit never asked for.
  • DON'T reference an action by a branch, actions/checkout@main. A branch moves with every commit its owners make, and the next run executes code nobody in your project has read.
  • DON'T take actionlint's silence as proof that every with: key is spelled right. versoin and fetch-deep both passed it.

The Test Job and Its Database

What. services: starts containers beside a job, here one called postgres from the same postgres:18.6 image the compose stack runs. Its env: sets the variables the image reads at its first start, ports: maps the container's 5432 to the runner's 5432, and options: hands extra flags to the docker create command GitHub runs, the health check here. GitHub waits for the health check to pass before the job's first step. The job-level env: sets DATABASE_URL for every step of the job. Why. tests/test_db.py skips without DATABASE_URL, so on a laptop with no database the suite passes without ever writing a row. In the test job the variable is always set and PostgreSQL is always up, so the one test that proves a prediction lands in the table runs on every push to main and every pull request, against the server version the service runs with. When. Whenever a test needs a server the job can start from an image: a database, a cache, a message queue.

options: >- is YAML for folding the three indented lines into one line, which becomes --health-cmd "pg_isready -h localhost -U listings -d listings" --health-interval 2s --health-retries 15 on the docker create command. The check is the one compose.yaml runs, pg_isready over TCP with -h localhost, which passes only once the real server is up and not during the image's first initialization. Every two seconds, fifteen tries.

The steps run on the runner itself, not in a container, which is why localhost works in DATABASE_URL. In Lesson 6.3.2 the service inside its container was refused at localhost, because a container's localhost is the container. Here the tests are an ordinary process on the runner, and ports: - 5432:5432 publishes the database on the runner's own port 5432, exactly as -p 5432:5432 publishes it on your machine. The password is a literal listings, written in the file, because this database exists for the length of one job and holds nothing.

uv sync --locked installs the environment from uv.lock, refusing a stale lock with exit code 1, as it does in Lesson 6.3.1's Dockerfile. The step's log on the runner, without the 19 pairs of Downloading and Downloaded lines of the larger packages and the list of what it installed:

Downloading cpython-3.12.8-linux-x86_64-gnu (download) (20.3MiB)
 Downloaded cpython-3.12.8-linux-x86_64-gnu (download)
Using CPython 3.12.8
Creating virtual environment at: .venv
Resolved 100 packages in 1ms
   Building listings @ file:///home/runner/work/listings-service/listings-service
      Built listings @ file:///home/runner/work/listings-service/listings-service
Prepared 96 packages in 2.41s
Installed 96 packages in 858ms

100 is every package the lock holds, the project among them, for every platform it was resolved for, and 96 is what Linux installs: the service's packages and the dev group with pytest, httpx and the full MLflow the tests save a model with, without colorama, pywin32, tzdata and waitress, which the lock marks for Windows. The runner starts with no Python 3.12.8, no .venv and no uv cache, so the step downloads the interpreter .python-version names and prepares all 96 packages, and the checkout lives in /home/runner/work/listings-service/listings-service, the repository's name twice. The image job's sync, on its own runner, installed the same 96 in 353ms from the cache the test job's Post step had saved. uv run pytest then runs the suite, and the runner's report, without its warnings summary, reads:

============================= test session starts ==============================
platform linux -- Python 3.12.8, pytest-9.1.1, pluggy-1.7.0
rootdir: /home/runner/work/listings-service/listings-service
configfile: pyproject.toml
testpaths: tests
plugins: anyio-4.15.1
collected 27 items

tests/test_app.py ........                                               [ 29%]
tests/test_data.py ....                                                  [ 44%]
tests/test_db.py .                                                       [ 48%]
tests/test_features.py .....                                             [ 66%]
tests/test_model.py .........                                            [100%]

======================== 27 passed, 2 warnings in 2.89s ========================

tests/test_db.py . is the line your laptop's run prints as tests/test_db.py s. The same suite with DATABASE_URL unset ends 26 passed, 1 skipped, 2 warnings, the skip being that one test, and both runs exit with 0, so both would give a green check. The difference is only visible in the log. The two warnings are the Starlette notice about httpx2 and MLflow's hint about integer columns that Lesson 6.2.4 explained.

DON'T:

  • DON'T read the check mark as "the database test passed". Without the job's env: block the run is just as green, with 1 skipped in a log nobody opens.
  • DON'T write a password that matters into a workflow. The file is committed, and everyone who can read the repository reads it, where listings here guards a database that is deleted with the runner.
  • DON'T point DATABASE_URL at the service name, postgres:5432, in a job that runs on the runner. A container's name resolves only inside its Docker network: on this machine getent hosts found a container's name from a second container on its network and nothing from the host, and these steps run on the host.

The Image Job and Its Smoke Test

What. The image job builds the Docker image from the commit and runs it. needs: test makes it wait for the test job and skip if that job fails. Its runner is a second fresh machine, so it checks out and syncs again, trains a model, points champion at it and exports it into model/, builds the image, starts a container, waits for /health and asks for the reference listing's price. That last step is a smoke test, a quick check that the built thing runs at all and answers one known question correctly. Why. Every test in tests/ runs against code imported on the runner, and none of them proves that the image starts, that the model inside it loads with mlflow-skinny alone, or that port 8000 answers. A missing model/ and a container that cannot load its model both pass the test job and fail here, as this subtitle and Lab Exercise 2 show. When. For every project that ships as an image, after the tests and before anything uses the image.

Nothing passes from one job to the next, and the clone the runner checks out holds only what git keeps. mlflow.db, mlruns/ and model/ are all in .gitignore, model/ since Lesson 6.3.1, so the runner has no registry and no model, and the image's COPY model ./model needs one. data/listings.csv is committed, which Lesson 6.1.1 decided, so the runner can train. The named step trains into a brand-new mlflow.db on the runner, where the first registered version is always version 1, so promote 1 is the right version on every run, and the export writes model/ exactly as it was written in Section 6.3. Among the step's log lines:

Successfully registered model 'listings-price'.
Created version '1' of model 'listings-price'.
version 1 56,562 0.766
listings-price version 1 is now champion

56,562 and 0.766 are the MAE and R² every default fit of this pipeline on this split has printed since Lesson 6.1.1, because the split is fixed and a linear regression on the same rows lands on the same coefficients. The registry on your laptop is at version 3 by now and the runner's is at 1, and both models price the same.

Leave the export out and the run stops at the build. A fresh clone of the project is exactly what a runner checks out, so make one beside the project and build there:

cd ~/dev
git clone listings-service ci-check
cd ci-check
docker build -t listings-service .
echo $?

The last lines:

Dockerfile:13
--------------------
  11 |     RUN --mount=type=cache,target=/root/.cache/uv \
  12 |         uv sync --locked --no-dev
  13 | >>> COPY model ./model
  14 |     
  15 |     RUN useradd --create-home --uid 1001 app
--------------------
ERROR: failed to build: failed to solve: failed to compute cache key: failed to calculate checksum of ref yj8ib3vs6biuyj4q8vl30ag0d::pm9cykyved0di6kgqqgkcmhrz: "/model": not found
1

Every layer above it came from the cache Lesson 6.3.3's builds left, and COPY model ./model failed because the clone has no model/, which BuildKit reports as "/model": not found. A clone whose lock was stale as well reported this same error and not the lock's, so a build with both mistakes names the missing folder first. The long ref names differ on every machine. Keep ~/dev/ci-check for the last subtitle and go back to the project with cd ~/dev/listings-service.

The smoke test's script reads as follows. docker run -d starts the container, and the loop has the shape of Lesson 6.3.3's ready.sh: each of its 60 passes asks /health once and sleeps a second, so it waits up to a minute, and && break leaves it on the first poll that succeeds. -s is the quiet curl of Lesson 6.3.1, and -f makes curl fail with exit code 22 when the server answers with a status of 400 or above, where a plain curl exits with 0 for any answer at all. The last command is the request of Lesson 6.2.4 with the reference listing, and as the last command of the step its exit code is the step's. The step's log:

4b85ae5e639c9b497952a25b8ae4c6f422901ee686c6d6592cc85478253d3027
{"status":"ok"}{"price":384235}

The container id, then the two bodies on one line, because neither ends with a newline. 384235 is the price the service has given the reference listing since Lesson 6.2.4. The container runs without DATABASE_URL, so it logs nothing to a database, and the service starts without one by design since Lesson 6.3.2.

The -f flag is the whole check. With the compose stack down, so port 8000 is free, build the project's image, start it with the step's first line, give the model the few seconds it takes to load, and send a listing with zero rooms:

docker build -t listings-service .
docker run -d --name listings-service -p 8000:8000 listings-service
sleep 10
curl -s -X POST http://127.0.0.1:8000/predict -H "Content-Type: application/json" -d '{"rooms": 0}' -o /dev/null -w "%{http_code}\n"
echo $?
curl -sf -X POST http://127.0.0.1:8000/predict -H "Content-Type: application/json" -d '{"rooms": 0}'
echo $?
422
0
22

After the build's lines and the container id, the three lines are the curls' and the echoes'. Without -f curl reported the 422 and exited with 0, a passing step. With -f it exited with 22 and printed nothing, because -s hides the error message too, and the step fails. Remove the container with docker rm -f listings-service.

DON'T:

  • DON'T smoke test with /health alone. /health answers {"status":"ok"} from a route that never touches the model, and only the price proves the model inside the image loaded and predicts.
  • DON'T drop -f to get the response body into the log. A 422 then exits with 0, and a service that refuses every listing passes its smoke test.
  • DON'T promote a version number the runner did not create. A fresh registry starts at version 1, and promote 3 in a fresh clone stopped with Model Version (name=listings-price, version=3) not found and exit code 1.

Pushing to GitHub and Reading the Run

What. GitHub runs the workflow when the commit that carries it reaches a GitHub repository. Pushing the project there is Lesson 0.3.4's remote and first push, to a new empty repository. The Actions tab of the repository lists every run, each run shows its jobs, each job its steps, and each step its log, the same lines the commands print in a terminal. Why. The run's log is the only view of a machine you will never log into. Reading it top down, job, step, last lines, is how a red cross becomes a cause. When. After every push to main and every pull request, and above all when a check turns red.

Commit the workflow first:

git add .github/workflows/ci.yml
git commit -m "ci: test with PostgreSQL, then build and smoke test the image"
[main 77ff044] ci: test with PostgreSQL, then build and smoke test the image
 1 file changed, 58 insertions(+)
 create mode 100644 .github/workflows/ci.yml

The hash is your own commit's. On the GitHub website, use New repository, name it listings-service, set it to Private, and leave it completely empty, no README, no .gitignore and no license, for the reason Lesson 0.3.4 gave: a pre-filled repository holds a commit yours does not and refuses the first push. Then attach it and push:

git remote add origin git@github.com:goku/listings-service.git
git push -u origin main

The last three lines:

To github.com:goku/listings-service.git
 * [new branch]      main -> main
branch 'main' set up to track 'origin/main'.

Every commit since Lesson 6.1.1 went up, the workflow among them, and the push itself started the first run. Private keeps the code, the data and the run logs to you. On GitHub Free, a private repository's runs draw on the account's included compute, $12 a month, at $0.006 a minute for the standard Linux runner, which is 2,000 minutes, and standard runners in a public repository are free.

Open the repository's Actions tab. It lists the run of the workflow ci on main, marked with a check once it passes or a cross once it fails. Open the run and it shows its two jobs with how long each took, test in 46 seconds and image in 1 minute 18 seconds in the run behind this lesson, and image started only once test had finished. Open test and its steps are listed in the order the runner ran them:

Set up job
Initialize containers
Run actions/checkout@v7.0.1
Run astral-sh/setup-uv@v10.3.0
Run uv sync --locked
Run uv run pytest
Post Run astral-sh/setup-uv@v10.3.0
Post Run actions/checkout@v7.0.1
Stop containers
Complete job

Set up job reports the runner image, Initialize containers pulls postgres:18.6, starts it and waits for its health check, and the four steps of the file follow. The Post steps are the actions' cleanup, setup-uv saving uv's download cache for the next run, and Stop containers removes the database. Click Run uv run pytest and the log opens on the report of the previous subtitle, 27 passed on its last line. The image job lists its own steps the same way, Set up job, the checkout, setup-uv and Run uv sync --locked, then Train, promote and export the model, Run docker build -t listings-service . and Smoke test the image, then its Post steps and Complete job. When both jobs pass, the run and the commit each carry a green check, the commit's two checks named test and image.

DON'T:

  • DON'T create the repository with a README or a license. The push of the project's history is refused, as Lesson 0.3.4 showed.
  • DON'T push to a public repository to save the minutes without deciding to publish the code. Public makes the code, data/listings.csv and every run log readable by anyone.
  • DON'T stop reading at the color. Open the failed step and read its last lines first, where the command printed why it stopped.

A Stale Lock Stops the Run

What. A stale lock is a commit whose uv.lock no longer answers its pyproject.toml. uv sync --locked, the first run: of both jobs, refuses it with exit code 1 before installing anything. uv run and a plain uv sync do the opposite on a laptop: they re-resolve and rewrite uv.lock without a line about it, so the folder where the change was made passes every test. Why. A lock exists so that every machine installs the same packages, and a runner that re-resolved on its own would test a lock no commit holds. The refusal sends the fix back to where it belongs, a commit with both files. When. Every time pyproject.toml changes, by uv add or by hand, and every time a run fails at uv sync --locked.

A teammate decides the service should declare SciPy, which scikit-learn already installs, the way Lesson 6.2.4 declared pydantic. They type the line into pyproject.toml by hand instead of running uv add, run the tests, which pass, and commit pyproject.toml alone. Reproduce their commit in the clone the image job's subtitle made: cd ~/dev/ci-check, and add one line to its pyproject.toml, after scikit-learn:

    "scikit-learn==1.9.1",
    "scipy==1.18.1",
    "skops==0.16.0",

Then run the workflow's first command:

uv sync --locked
echo $?
Using CPython 3.12.8
Creating virtual environment at: .venv
Resolved 100 packages in 29ms
error: The lockfile at `uv.lock` needs to be updated, but `--locked` was provided.

hint: To update the lockfile, run `uv lock`.
1

Pushed, the same commit stops the run at the same command. The test job's Run uv sync --locked step ends:

Using CPython 3.12.8
Creating virtual environment at: .venv
Resolved 100 packages in 616ms
error: The lockfile at `uv.lock` needs to be updated, but `--locked` was provided.

hint: To update the lockfile, run `uv lock`.
##[error]Process completed with exit code 1.

The last line is the runner's own. The step carries a cross, Run uv run pytest and setup-uv's Post step are skipped, so the failed run saves no cache, and image, which needs test, is skipped, so nothing is built from the commit and its checks read test failed and image skipped. The error is the one Lesson 6.3.1 quoted for --locked, and its question is Lesson 6.1.1's uv lock --check: resolving pyproject.toml again gives a lock that differs from the committed one, here by the new requirement on SciPy, even though the version it resolves, 1.18.1, is the one already locked.

Now the teammate's side. uv sync --frozen installs whatever uv.lock says without asking that question:

uv sync --frozen
echo $?
git status --short

The first lines, without the 96 + lines, and the rest:

   Building listings @ file:///home/goku/dev/ci-check
      Built listings @ file:///home/goku/dev/ci-check
Prepared 1 package in 8ms
Installed 96 packages in 57ms
0
 M pyproject.toml

It exits with 0 and leaves uv.lock as it was, stale. Then the command the teammate ran:

uv run pytest -q
git status --short

The report's last line, then the status:

26 passed, 1 skipped, 2 warnings in 1.74s
 M pyproject.toml
 M uv.lock

The tests pass, the one database test skipped, and uv run rewrote uv.lock before running them and printed nothing about it, as Lesson 6.1.1 said it writes the lock. On the teammate's machine both files changed, and the commit of pyproject.toml alone left the rewritten lock behind. The fix is that lock, committed beside the line that caused it:

git diff --stat
uv sync --locked
echo $?
 pyproject.toml | 1 +
 uv.lock        | 2 ++
 2 files changed, 3 insertions(+)
Resolved 100 packages in 1ms
Checked 96 packages in 0.82ms
0

The lock gained two lines, SciPy among the project's dependencies and among its requirements, and with it uv sync --locked passes. In the teammate's project the fix is uv lock, which writes the same two lines when uv run has not already, then one commit with both files and a push, and the next run goes green. The project you push keeps no SciPy line, so delete the clone:

cd ~/dev
rm -rf ci-check

DON'T:

  • DON'T change the workflow to a plain uv sync to make the red go away. In the clone it rewrote the stale lock and exited with 0, and on a runner every later step would then test a lock that exists in no commit.
  • DON'T use uv sync --frozen in CI. It installed the stale lock, exited with 0, and the requirement in pyproject.toml was never checked against it.
  • DON'T git add pyproject.toml alone after a dependency change. uv run had already rewritten uv.lock in the folder, and the commit left that change out.

Lab Exercises

Lab Exercise 1: The Test Job by Hand

Rehearse the test job on your machine before any push depends on it. Clone the project into ~/dev/ci-check and work only in the clone. Start the service container of the test job yourself with docker run, as ci-postgres, with the job's three variables, its port and its three health flags written out, and wait for it with a loop of the smoke test's shape that asks docker inspect -f '{{.State.Health.Status}}' ci-postgres once a second, leaves on healthy and prints how many tries it took. Then run the job's two steps, uv sync --locked and uv run pytest, first without DATABASE_URL and then with the job's value, and print each run's last line. Remove the container and its volumes with docker rm -f -v ci-postgres and delete the clone. If the compose stack from Lesson 6.3.3 is up, run docker compose down in the project first, which frees port 5432 and keeps its volume. Say in a comment what the two last lines show and what GitHub does that your loop did.

Lab Exercise 2: A Container That Cannot Start

The smoke test has only seen an image that works. In the project folder, build the image as listings-service, then run the smoke step's lines by hand in the terminal, with the docker run line given one more flag, -e MODEL_URI=models:/listings-price@champion, so the service loads the registry's champion instead of the folder the image carries. Print the exit code of the last command, the container's status from docker ps -a --filter name=listings-service --format '{{.Status}}', and the last three lines of docker logs listings-service 2>&1. Remove the container, run the step again without the extra flag, and remove it once more. Say in a comment how long the failing step took, what it printed about the cause, what the cause is, and why the workflow exports model/ instead.


Solution 1: The Test Job by Hand
cd ~/dev
git clone listings-service ci-check
cd ci-check
docker run -d --name ci-postgres -p 5432:5432 -e POSTGRES_USER=listings -e POSTGRES_PASSWORD=listings -e POSTGRES_DB=listings --health-cmd "pg_isready -h localhost -U listings -d listings" --health-interval 2s --health-retries 15 postgres:18.6
for i in $(seq 1 30); do
  [ "$(docker inspect -f '{{.State.Health.Status}}' ci-postgres)" = healthy ] && break
  sleep 1
done
echo "healthy after $i tries"
uv sync --locked
uv run pytest | tail -1
DATABASE_URL=postgresql://listings:listings@localhost:5432/listings uv run pytest | tail -1
docker rm -f -v ci-postgres
cd ~/dev
rm -rf ci-check
# No DATABASE_URL: 26 passed and the database test skipped. With the
# job's value: 27 passed, so the row was written and read back. GitHub
# waits for the health check itself before the job's first step.

The lines after the container id and the sync, and the last:

healthy after 3 tries
================== 26 passed, 1 skipped, 2 warnings in 1.92s ===================
======================== 27 passed, 2 warnings in 1.59s ========================
ci-postgres

[ "$(...)" = healthy ] passes once the health status docker inspect prints reads healthy. The three options: flags became --health-cmd, --health-interval and --health-retries on docker run unchanged, because options: is passed to docker create as written, and the job's env: and ports: became -e and -p. The first two tries found the container not yet healthy, because Docker runs the first check only once the two-second interval has passed, and the third found it healthy. The two last lines are the same 27 tests with and without the variable, and only the second ran test_db.py. docker rm -f -v takes the database's anonymous volume with the container, as in Lesson 6.3.2, so nothing of the rehearsal is left.

DON'T:

  • DON'T start the tests the moment docker run returns. The container exists at once and the server answers only after its health check passes, which is what Initialize containers waits for on the runner.
  • DON'T rehearse in the project folder. The clone holds only what git keeps, which is what the runner checks out, and the project holds mlflow.db, model/ and .env besides.
Solution 2: A Container That Cannot Start
docker build -t listings-service .
docker run -d --name listings-service -p 8000:8000 -e MODEL_URI=models:/listings-price@champion listings-service
for i in $(seq 1 60); do
  curl -sf http://127.0.0.1:8000/health && break
  sleep 1
done
curl -sf -X POST http://127.0.0.1:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"area_sqm": 85.0, "year_built": 1998, "distance_km": 3.0, "rooms": 3, "bathrooms": 1, "neighborhood": "Riverside", "property_type": "apartment", "energy_rating": "C", "condition": "good"}'
echo $?
docker ps -a --filter name=listings-service --format '{{.Status}}'
docker logs listings-service 2>&1 | tail -3
docker rm -f listings-service
# The failing run took about a minute and printed the container id and
# exit code 7, nothing else. The skinny image has no tracking store for
# sqlite, so the lifespan cannot read a registry at all, and the image
# carries the exported model folder, which needs none.

After the container id:

7
Exited (3) 56 seconds ago
mlflow.tracking.registry.UnsupportedModelRegistryStoreURIException:  Model registry functionality is unavailable; got unsupported URI 'sqlite:////app/mlflow.db' for model registry data storage. Supported URI schemes are: ['', 'file', 'databricks', 'databricks-uc', 'uc', 'http', 'https']. See https://www.mlflow.org/docs/latest/tracking.html#storage for how to run an MLflow server against one of the supported backend storage locations.

ERROR:    Application startup failed. Exiting.
listings-service

The container exited with code 3 a few seconds after it started, Lesson 6.2.4's startup failure, and took its port mapping with it. Every poll then failed to connect, silently because of -s, the loop ran its full sixty seconds, and the POST failed to connect too, with exit code 7, curl's code for a connection that could not be made. On the runner the step would fail with that 7 after a minute, and its log would hold one container id. The cause is only in the container's own log: models:/listings-price@champion asks the registry in sqlite:///mlflow.db, which is /app/mlflow.db inside the image, and the image's MLflow is mlflow-skinny, Lesson 6.3.3's choice, which cannot read a sqlite registry at all. The workflow exports the champion into model/ on the runner for exactly that reason, and the default MODEL_URI of "model" loads it with no registry. Run without the extra flag:

docker run -d --name listings-service -p 8000:8000 listings-service
for i in $(seq 1 60); do
  curl -sf http://127.0.0.1:8000/health && break
  sleep 1
done
curl -sf -X POST http://127.0.0.1:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"area_sqm": 85.0, "year_built": 1998, "distance_km": 3.0, "rooms": 3, "bathrooms": 1, "neighborhood": "Riverside", "property_type": "apartment", "energy_rating": "C", "condition": "good"}'
echo
docker rm -f listings-service

After the container id:

{"status":"ok"}{"price":384235}
listings-service

The project's model/ holds version 3 of your registry, and it prices the reference listing at 384235 like the runner's version 1.

DON'T:

  • DON'T read the smoke test's exit code 7 as a network problem. It meant nothing was listening, and the container's log said why.
  • DON'T point the image at the registry to save the export step. The skinny image refused the sqlite registry at startup.

Theory Assessment

Q1: localhost Twice

The test job's DATABASE_URL names localhost:5432, and it works. In Lesson 6.3.2, the service in its container was refused at localhost. Say why the same host name works in one place and not the other, what ports: does here, and what the run shows if the job's env: block is deleted.

Hint: Ask where the tests run, on the runner or in a container, and what tests/test_db.py does when the variable is missing.

Q2: Delete needs: test

A teammate deletes needs: test from the image job "so the run finishes sooner". Say when each job then starts, what happens to the image job when a test fails, and why the image job still has to check out, sync and train on its own when needs: is there.

Hint: Jobs run in parallel unless one needs another, and each job gets a fresh runner.

Q3: Two Curls Without -f

A teammate removes -f from both curls in the smoke test, so the log shows the body of a failed request. Say what the step does now with a service whose /predict answers 422 to every listing, and with a container that never starts, and which of the two the change hides.

Hint: Without -f, curl's exit code says whether an answer came back, not what it said.

Q4: Green by Removing a Flag

The teammate's stale commit of the last subtitle turned the run red. Someone changes both uv sync --locked lines in the workflow to uv sync, pushes, and the run goes green, image built and smoke test passed. Say what the test job tested, which lock the image was built from, given that the Dockerfile itself runs uv sync --locked --no-dev, and what another teammate meets who clones that commit and runs uv sync --locked, the command the workflow ran before the change.

Hint: A plain uv sync rewrites uv.lock in the folder it runs in, and docker build . copies the files of that folder.


Answer 1:

The tests in the test job are an ordinary process on the runner, because the job has no container of its own, so their localhost is the runner. ports: - 5432:5432 publishes the database container's port on the runner's port 5432, as -p 5432:5432 does on your machine, and a connection to localhost:5432 reaches it. In Lesson 6.3.2 the client was the service inside its own container, whose localhost is that container, where nothing listens on 5432, so the connection was refused and the service had to name the database by its container name on a shared network. With the job's env: deleted, DATABASE_URL is unset on the runner, tests/test_db.py skips, and the run ends 26 passed, 1 skipped with exit code 0, a green check that no longer means a prediction reached PostgreSQL.

DON'T judge a database test by the run's color. The skip and the pass both end in exit code 0.

Answer 2:

Without needs:, both jobs start together on two runners, because jobs run in parallel by default. The image job then builds and smoke tests every commit whatever the tests say, so a commit whose tests fail still ends with a built image and a passing smoke test beside a red test job. With needs: test, a failed test job skips the image job. Either way the image job starts on a fresh runner that holds none of the test job's files, its .venv, its downloads or anything it made, so it checks out the commit, syncs and trains again, and its registry is new, which is why its version is always 1.

DON'T read needs: as sharing. It orders two jobs, and the second still starts from nothing.

Answer 3:

Without -f, curl exits with 0 whenever the server answers, whatever the status. A service whose /predict answers 422 to everything answers /health with 200, so the loop leaves at once, and the POST gets its 422, prints the body and exits with 0, as the zero-rooms request exited with 0 without -f: the step passes and the check turns green. A container that never starts still fails, because no answer comes back at all and curl exits with 7 with or without -f, after the loop's sixty seconds. The change hides exactly the service that runs and gives wrong answers, the more dangerous of the two, and keeps catching only the one that is already obvious.

DON'T trade the exit code for a readable log. Put the body in the log with a separate command if it is needed, and keep -f on the check.

Answer 4:

The test job's plain uv sync found the stale lock, re-resolved, rewrote uv.lock on the runner, installed from the rewritten lock and exited with 0, so the tests passed against a lock that exists only on that runner. The image job's uv sync rewrote the lock on its own runner in the same way, and docker build . copied that rewritten uv.lock into the image, so the Dockerfile's uv sync --locked --no-dev found a lock that matched pyproject.toml and passed, and the image built with exit code 0 from a lock no commit holds. In a clone where a plain uv sync had rewritten the lock, docker build did exactly that, and with the stale lock left in place the same build stopped at the first uv sync --locked --no-dev --no-install-project layer with exit code 1. The commit itself still holds the stale lock, so the teammate's uv sync --locked prints the same needs to be updated error and exits with 1. The green run proved a lock nobody can check out.

DON'T fix a red run by loosening the check that turned it red. The lock was stale before the change and still is after it.


Assignment

Assignment: A Smoke Test That Says Why

The smoke step proves the reference price and nothing else, and when the container never starts it waits a full minute and logs a container id. Write ~/dev/listings-probes/smoke.py, a stricter smoke test kept beside the project until it has earned a place in the workflow, and run it from the project folder with uv run python ../listings-probes/smoke.py against a container named listings-service on port 8000. It waits up to 30 seconds for /health to answer 200, then checks that the reference listing is priced at 384235, the listing with the area, the year and the rating left out at 434348, and the reference listing with zero rooms refused with 422. On the first check that fails it prints the container's last four log lines and exits with a message and a non-zero code. On success it prints how long /health took and the three results. Run it against the working image and against the container of Lab Exercise 2 that cannot start, and say in a comment what each run printed and how long the failing one took.

Three pieces of Python the lesson did not use do the work. httpx.get(url) and httpx.post(url, json=...) send one request each and return a response with .status_code and .json(), from the library TestClient is built on, which the project's dev group already installs, and when nothing listens they raise httpx.ConnectError, one kind of httpx.TransportError, the exception for a request that got no answer. subprocess.run([...], capture_output=True, text=True) runs a command and returns its output as .stdout and .stderr strings. sys.exit("message") prints the message to standard error and exits with code 1.

Hints:

  • time.monotonic() read before the loop and on each pass gives the seconds waited, and a try around httpx.get that passes on httpx.TransportError keeps the loop alive while the port is closed.
  • uvicorn writes its startup and error lines to standard error and its access lines to standard output, so the container's lines arrive split between .stderr and .stdout of docker logs.

Solution:
import subprocess
import sys
import time

import httpx

BASE = "http://127.0.0.1:8000"
CONTAINER = "listings-service"
LISTING = {
    "area_sqm": 85.0, "year_built": 1998, "distance_km": 3.0, "rooms": 3,
    "bathrooms": 1, "neighborhood": "Riverside", "property_type": "apartment",
    "energy_rating": "C", "condition": "good",
}
BLANK = {k: v for k, v in LISTING.items()
         if k not in ("area_sqm", "year_built", "energy_rating")}


def fail(reason):
    logs = subprocess.run(["docker", "logs", "--tail", "4", CONTAINER],
                          capture_output=True, text=True)
    print(logs.stdout + logs.stderr, end="")
    sys.exit(f"smoke test failed: {reason}")


def wait_for_health(seconds):
    start = time.monotonic()
    while time.monotonic() - start < seconds:
        try:
            if httpx.get(f"{BASE}/health").status_code == 200:
                return time.monotonic() - start
        except httpx.TransportError:
            pass
        time.sleep(1)
    fail(f"/health did not answer 200 within {seconds} s")


waited = wait_for_health(30)
for listing, price in ((LISTING, 384235), (BLANK, 434348)):
    reply = httpx.post(f"{BASE}/predict", json=listing)
    if reply.json() != {"price": price}:
        fail(f"expected {price}, got {reply.status_code} {reply.text}")
refused = httpx.post(f"{BASE}/predict", json=LISTING | {"rooms": 0})
if refused.status_code != 422:
    fail(f"zero rooms got {refused.status_code}, not 422")
print(f"healthy after {waited:.1f} s, 384235, 434348 and a 422")
# Working image: one line, healthy after about 4 s. The container that
# cannot start: the last log lines, the registry error among them, then
# the failure message and exit code 1, after the 30 s deadline.

Against the working image, started with the smoke step's docker run line:

docker run -d --name listings-service -p 8000:8000 listings-service
uv run python ../listings-probes/smoke.py
echo $?
docker rm -f listings-service

After the container id, and four lines of uv rebuilding the project, Building listings, Built listings, Uninstalled 1 package and Installed 1 package, which this first uv run in the project folder since Lesson 6.3.3's Lab Exercise 1 edited and restored pyproject.toml prints and every later one skips:

healthy after 4.1 s, 384235, 434348 and a 422
0
listings-service

Against the container that cannot start:

docker run -d --name listings-service -p 8000:8000 -e MODEL_URI=models:/listings-price@champion listings-service
time uv run python ../listings-probes/smoke.py
echo $?
docker rm -f listings-service

After the container id:

    raise UnsupportedModelRegistryStoreURIException(
mlflow.tracking.registry.UnsupportedModelRegistryStoreURIException:  Model registry functionality is unavailable; got unsupported URI 'sqlite:////app/mlflow.db' for model registry data storage. Supported URI schemes are: ['', 'file', 'databricks', 'databricks-uc', 'uc', 'http', 'https']. See https://www.mlflow.org/docs/latest/tracking.html#storage for how to run an MLflow server against one of the supported backend storage locations.

ERROR:    Application startup failed. Exiting.
smoke test failed: /health did not answer 200 within 30 s

real	0m30.536s
user	0m0.466s
sys	0m0.066s
1
listings-service

The working run waited 4.1 seconds for the model to load and checked three answers where the step checked one. The failing run printed the four lines of the container's log that name the cause, then its own message, and exited with 1 after 30.5 seconds, half the step's minute. fail runs docker logs only once something has failed, so a passing run never pays for it, and logs.stdout + logs.stderr keeps both streams because uvicorn's startup and error lines arrive on standard error and its access lines on standard output. The 30 seconds still pass in full on a container that exited after three, because the loop only learns that nothing listens and not why. The workflow could run the same file with uv run python once it lives in the repository, in place of the step's two curls.

DON'T let a smoke test fail without the evidence. A red step that holds only an exit code sends someone to reproduce the failure by hand, where the four log lines here named it.


Additional Resources

Optional, and nothing later in the module depends on them. Worth a look for the uv side of CI and for the other way to reach a service container.

  • uv documentation - "Using uv in GitHub Actions". Installing uv with setup-uv at a pinned version, installing Python from the project's own pin, syncing with uv sync --locked before the tests, and caching.
  • GitHub documentation - "Creating PostgreSQL service containers". The same service configured for a job that runs in a container, which reaches PostgreSQL by its service name, beside the job on the runner machine, which reaches it at localhost and a mapped port.

Summary

The workflow is fifty-eight lines and almost all of it is commands the project already knew. uv sync --locked is Lesson 6.1.1's lock, uv run pytest is Section 6.1's tests, the train, promote and export lines are Section 6.2's registry and Section 6.3's export, and docker build is the image Lesson 6.3.3 made small. What the file adds is a machine with no memory: a runner that has never seen your .venv, your mlflow.db, your model/ or the lock uv run rewrote last Tuesday, and that therefore asks of the commit alone whether it is enough.

Each part of the run failed in this lesson in the way that part fails. A lock that was stale stopped it at the first command, a missing export stopped it at the build, a container that could not start stopped it at the last curl after a minute of silence, and a missing env: block would not have stopped it at all, only turned a pass into a skip. The red ones were easy. The ones worth remembering are the greens that meant less than they looked, the skip that exits with 0, the curl without -f, the plain uv sync that rewrote the lock it was meant to check.

From here on, every change to the service lands as a commit that a machine builds and asks for a price before anyone else uses it. That is the floor Section 6.4 stands on: once the image is known to start and answer, the question becomes whether its answers stay right as the listings it sees drift away from the ones it learned on.


Cheat Sheet

Part A - Reference Table

Idea Code Notes
Workflow file .github/workflows/ci.yml committed, run on every push to main and every pull request
Triggers on: push: branches: [main], pull_request: main and every proposed merge
Runner runs-on: ubuntu-24.04 fresh machine per job, an exact image
Action uses: actions/checkout@v7.0.1 a release tag pins its code
Command run:, run: | for several lines bash -e, first non-zero exit fails it
uv on the runner astral-sh/setup-uv@v10.3.0, version: "0.13.0" same uv as the Dockerfile
Lint first uvx --from actionlint-py actionlint structure yes, unknown inputs no
Database services: postgres: with ports: and options: GitHub waits for the health check
Reach it localhost:5432 steps run on the runner
Order jobs needs: test skipped if test fails, nothing shared
Fresh registry promote 1 first version on a new runner is 1
Smoke test curl -sf -f exits 22 on 400 and above
Nothing listening curl exit 7 the cause is in docker logs
Locked install uv sync --locked exit 1 on a stale lock
Unchecked install uv sync --frozen installs a stale lock, exit 0
Silent rewrite uv run, plain uv sync rewrite uv.lock, print nothing
Fix a stale lock uv lock, commit both files then push
First push git push -u origin main starts the first run

Part B - Compact Code Block

jobs:
  test:
    runs-on: ubuntu-24.04
    services:
      postgres:
        image: postgres:18.6
        ports:
          - 5432:5432
        options: >-
          --health-cmd "pg_isready -h localhost -U listings -d listings"
    env:
      DATABASE_URL: postgresql://listings:listings@localhost:5432/listings
    steps:
      - uses: actions/checkout@v7.0.1
      - uses: astral-sh/setup-uv@v10.3.0
      - run: uv sync --locked
      - run: uv run pytest
  image:
    needs: test
uvx --from actionlint-py actionlint    # lint every workflow
git remote add origin git@github.com:goku/listings-service.git
git push -u origin main                # starts the first run
curl -sf http://127.0.0.1:8000/health  # exit 22 on 400 and up, 7 if down
uv sync --locked                       # exit 1 on a stale lock
←Previous: Lesson 3
Next: Section 4, Monitoring and the Closing Loop→

Questions about this lesson: #module-6 on Discord

Found an error in this lesson? Report it

Your progress
Current streak
🔥 0
Lessons marked today
0 / 3
Longest streak
0

Sign in

The bootcamp

  • Curriculum
  • How To
  • Pricing
  • Changelog
  • Future

The community

  • Discord
  • Reviews
  • Contact

Follow

  • Instagram
  • X
  • TikTok
  • Reddit

Legal

  • Terms
  • Privacy
  • Refunds

Coming soon on November 16, 2026

Theme