Memory vs Storage - Where Your Data Lives and What a Crash Destroys

Context

Data your program is holding lives in one of two places. Memory holds it only while the process is alive, disk holds it after the process is gone. Which of the two a given value sits in decides whether an interruption costs you a second or an afternoon.

A training loop keeps its model weights, its gradients, and its epoch counter in memory because that is the only place fast enough to touch them thousands of times per second. None of it survives on its own. This is the entire reason checkpointing exists, writing the model to disk every few epochs so a run that dies at epoch 40 of 50 restarts at 40 instead of at zero. The same split decides whether you load a dataset into memory all at once or stream it from disk in pieces.

Get this wrong and you lose work you already paid for in compute time. The previous lesson established that a process is a live thing running in memory, this one is about what happens to everything that process was holding the instant it stops.

Goal

• Distinguish volatile memory (RAM) from persistent storage (disk) and predict, for any piece of data, whether it survives the program ending.
• Demonstrate volatility by building an in-memory counter that resets every run, and persistence by building an on-disk counter that keeps climbing across runs.
• Identify a process's state, the live data it holds in RAM, and explain precisely why that state vanishes the moment the process terminates.
• Reproduce a crash (abnormal termination) with an unhandled exception, observe the lost in-memory progress, and fix it by checkpointing state to disk.
• Read a crashed child process's non-zero exit code with subprocess to confirm it terminated abnormally rather than finishing cleanly.

Prerequisites

This lesson continues directly from Lesson 0.1.1 and assumes program versus process, the process lifecycle, and os/subprocess.

The two below are optional. The lesson teaches every call it uses from zero, so skipping both costs you nothing.

1. (YouTube) Memory & Storage: Crash Course Computer Science #19 - CrashCourse channel. Twelve minutes on the storage that keeps its data when the power dies, from punch cards and magnetic cores to hard disks and SSDs, and why a computer still pairs it with faster, volatile memory.
2. (YouTube) Files & File Systems: Crash Course Computer Science #20 - CrashCourse channel. How persistent storage is organized into the files you will read and write below.

Table of Contents

• Volatile Memory (RAM)
• Persistent Storage (Disk)
• Memory vs Storage Distinction
• Process State
• Crash / Abnormal Termination

Theory

Volatile Memory (RAM)

What. Volatile memory, RAM (Random Access Memory), is the fast, temporary workspace where a running process keeps the data it is actively using: your variables, your loaded dataset, the half-finished result of a loop. "Volatile" means its contents do not last on their own: RAM holds them only while it has power, and the OS lets a process keep its share only while the process lives, so end the process (or cut the power) and that memory is wiped clean.

Why. Every variable you have ever created lived in RAM. Beginners imagine a variable as something that "stays" after the program ends, it does not. The instant a process reaches the Terminated state (Lesson 0.1.1), the OS reclaims all the RAM it held and hands those bytes to the next program. Internalizing this kills a whole class of confusion: "why did my counter reset?", "why is my data gone after the script finished?", "where did my trained model go?".

When. This matters every single time a program holds data it wants to keep. During active computation, RAM is exactly where you want your data, it is fast. The danger appears at the boundaries: when the process ends, when it crashes, when the machine reboots. Anything still only in RAM at that moment is lost.

RAM is volatile, so a value created during one run carries nothing into the next run. Save this as volatile_counter.py:

# volatile_counter.py - a value that lives only in RAM, only while this runs
runs = 0                                  # created fresh in memory every time the program starts
runs = runs + 1                           # update it in RAM
print("this session's run count:", runs)  # always prints 1, RAM remembers nothing from before

Run it, then run it again:

# same on every OS (Windows users: inside WSL)
python3 volatile_counter.py
python3 volatile_counter.py

Both runs print this session's run count: 1. The variable runs did not "remember" the first run, because the first process's RAM was wiped when it terminated and the second process started with brand-new, empty memory. That reset is volatility.

You can also see that data physically occupies RAM while the process is alive:

import sys                 # built-in (used in Lesson 0.1.1): sys.getsizeof reports RAM footprint
dataset = [0] * 1_000_000  # allocated entirely in RAM right now
# how much memory this one object occupies
print("bytes held in RAM:", sys.getsizeof(dataset))

This prints the number of bytes the list occupies while the process runs. That memory is real and reserved for your process, but the moment the script ends, the OS frees every one of those bytes. The dataset was never anywhere except RAM, so it leaves no trace behind.

DON'T:

• DON'T assume a variable "saves itself" between runs, RAM is wiped when the process terminates, nothing in it survives on its own.
• DON'T blame your code when a counter or accumulator "resets to zero" on the next run, that is RAM behaving exactly as designed, not a bug.
• DON'T confuse "the program is using a lot of memory" with "the data is saved", high RAM usage is the opposite of saved, it is purely temporary.

Persistent Storage (Disk)

What. Persistent storage, the disk (an SSD or hard drive), is the slower, larger place where data is written as files that remain after the process ends, after the program closes, and even after the power goes out. "Persistent" (also called non-volatile) means the bytes stay put until something explicitly changes or deletes them.

Why. Disk is the only way to make data outlive the process that created it. A trained model, a downloaded dataset, your source code itself, all of these are valuable precisely because they sit on disk and survive. If RAM is the desk you work on, disk is the filing cabinet you put things in so they are still there tomorrow. Without it, every program would start from absolute zero every time, having forgotten everything.

When. You reach for disk whenever data must survive a boundary: the end of a run, a reboot, or a crash. The cost is speed, reading and writing disk is far slower than RAM, so you don't keep working data on disk, you save finished or important data to it at chosen moments (this deliberate save is what later sections call a checkpoint).

Writing to disk is just opening a file and writing text into it. Save this as save.py:

# save.py - write a value to disk, it will outlive this process
checkpoint = 42                         # a value we want to keep after this program ends
with open("checkpoint.txt", "w") as f:  # open a file on disk for writing ("w"), auto-closes after
    f.write(str(checkpoint))            # store the value as text on disk
print("saved", checkpoint, "to checkpoint.txt")

open("checkpoint.txt", "w") asks the OS to create (or overwrite) a file on disk, the with block guarantees the file is properly closed afterward, even if an exception is raised inside it. f.write(...) records the bytes. After this process exits, checkpoint.txt is still sitting on disk, the data outlived the process that made it.

Now a completely separate process can read those bytes back. Save this as load.py and run it after save.py:

# load.py - a brand-new process reads what a previous, already-finished process left on disk
with open("checkpoint.txt") as f:   # open the same file for reading (default mode is "r")
    loaded = f.read()               # pull the stored text back from disk into RAM
print("loaded from disk:", loaded)  # prints 42, it survived the first process ending

load.py knows nothing about save.py and never shared any memory with it, yet it prints 42, because the value was on disk, not in the (now-gone) RAM of the first process. That is persistence: data handed forward across the boundary between two separate processes.

You can confirm the file is real, independent of any running program:

# the file is real bytes on disk, sitting there between runs:
cat checkpoint.txt  # 'cat' prints a file's contents straight from disk (same everywhere, WSL included)

This prints 42 straight from the file, no Python involved. save.py wrote no newline after the 42, so on WSL and Linux your next prompt starts on that same line, right after it, while macOS's zsh marks the missing newline with a highlighted % and starts the prompt on the next line. The data exists on disk on its own, which is the entire point of persistent storage.

DON'T:

• DON'T treat open(..., "w") as harmless, "w" overwrites the whole file, use "a" (append) when you mean to add to it, not replace it.
• DON'T assume writing a file is as fast as setting a variable, disk is orders of magnitude slower than RAM, so save at chosen moments, not in your tightest loops without thought.
• DON'T forget to close files, relying on with open(...) does it for you, skipping it risks data not being fully written.

Memory vs Storage Distinction

What. RAM and disk are not two flavors of the same thing, they trade off on opposite axes. RAM is fast, small, and volatile, disk is slow, large, and persistent. The CPU computes directly on data in RAM, disk is where data rests when it must be kept. Every program is a constant traffic of data being loaded from disk into RAM to work on, and saved from RAM back to disk to keep.

Why. Almost every real decision about data comes down to this distinction. Where should the dataset live while I process it? RAM, it's fast. Where should the finished model go so it exists tomorrow? Disk, it's persistent. Get the distinction wrong and you either grind to a halt (working off disk) or lose everything (never saving to disk). This one trade-off quietly shapes how every serious program, and every ML training run, is structured.

When. You apply it continuously: load input from disk → RAM at the start, compute in RAM, write results RAM → disk at the end (and at safe points in between). The art is choosing when to cross the boundary, because each crossing costs speed but each save buys survival.

Here is the contrast in one row-per-axis table:

The cleanest way to feel the difference is to run one script several times and watch one counter forget while the other remembers. Save this as distinction.py:

# distinction.py - run this 3+ times in a row and compare the two numbers each time
ram_runs = 0                           # lives in RAM: reset to 0 on every single run
ram_runs = ram_runs + 1                # so it can never climb past 1

with open("disk_runs.txt", "a") as f:  # "a" = append, creates the file if missing
    f.write("x")                       # add one mark to the file each run (stays on disk)
with open("disk_runs.txt") as f:       # reopen for reading (and auto-close when done)
    disk_runs = len(f.read())          # count all marks accumulated on disk over time

# always 1
print("RAM counter (volatile):   ", ram_runs)
# 1, then 2, then 3, ... climbing across runs
print("DISK counter (persistent):", disk_runs)

Run it three times:

# same on every OS (Windows users: inside WSL)
python3 distinction.py
python3 distinction.py
python3 distinction.py

The RAM counter prints 1 every time, each new process starts with empty memory. The disk counter prints 1, then 2, then 3, because each run appended a mark to a file that persists. Same script, same logic, two opposite behaviors: that gap between the two numbers is the entire memory-vs-storage distinction, made visible.

DON'T:

• DON'T keep working data on disk to "be safe", reading/writing disk in a hot loop is painfully slow, compute in RAM, save to disk at deliberate points.
• DON'T keep important results only in RAM, if it isn't written to disk, it does not survive the process, full stop.
• DON'T assume bigger RAM removes the need for disk, no amount of RAM is persistent, capacity and persistence are different axes.

Process State

What. A process's state is the complete set of live data it currently holds in RAM: every variable's value, the dataset it loaded, the running totals it has accumulated, and how far through its work it has gotten (e.g., "epoch 3 of 5"). It is the running process from Lesson 0.1.1 viewed as a bundle of in-memory data rather than as a PID. Because that data lives in volatile RAM, the process state is tied to the life of the process, it exists only while the process does.

Why. "State" is the word for everything you would lose if the process stopped right now. When you say a training run is "at epoch 80 with a loss of 0.3," that sentence describes its state, and that state is sitting in RAM. Understanding that progress is in-memory state is what makes the next section (crashes) click: a crash doesn't just stop a program, it annihilates its entire state. Tying this back to the lifecycle: a process in Running or Waiting owns a rich state, the instant it hits Terminated, the OS frees that memory and the state is gone.

When. You think about process state whenever progress accumulates over time, a long loop, a multi-step computation, a training run. The key question becomes: if this process died right now, what would I lose, and have I saved any of it to disk? If the answer is "all of it" and "none of it," you have a fragile program.

Every variable in a loop is part of the process's state, held entirely in RAM. Save this as train_state.py:

# train_state.py - the process's STATE is all the live data it holds in RAM while running
dataset = [3, 1, 4, 1, 5]  # loaded into RAM, part of the process state
epoch = 0                  # how far we've gotten, part of the process state
running_total = 0          # accumulated result so far, part of the process state

for value in dataset:      # each pass updates the in-memory state
    epoch = epoch + 1      # advance progress
    running_total = running_total + value
    # the state evolving in RAM
    print(f"epoch {epoch}: running_total={running_total}")

This prints the state advancing epoch by epoch (running_total ends at 14). At every line, epoch, running_total, and dataset together are the process state, and all three live in RAM, owned by this one process. When the loop ends and the process terminates, the OS frees that memory and every bit of that state disappears. Run it again and it starts from epoch = 0 once more, because process state does not carry between processes.

To make state survive the process, you must copy it from RAM to disk. Add a save at the end:

# persist the final state so a future process can pick up where this one left off:
with open("state.txt", "w") as f:        # open a file to hold a copy of the state that outlives the process
    f.write(f"{epoch},{running_total}")  # store epoch and total as text, e.g. "5,14"
print("state saved to disk")

Now the in-RAM state has a twin on disk that outlives the process: the file state.txt holds 5,14. A later, completely separate process can read that line and resume from epoch 5 with total 14 instead of starting over. This deliberate "copy the state to disk" act is exactly what a checkpoint is, and it is the only thing standing between a long run and a crash.

DON'T:

• DON'T think of "state" as only one variable, it is all the live data that defines where the process is, losing any of it can mean losing your progress.
• DON'T assume two runs of the same script share state, each process gets its own isolated RAM (Lesson 0.1.1), so the second run knows nothing of the first.
• DON'T treat in-RAM state as safe just because the program is running fine right now, it is one termination away from being gone unless it has been written to disk.

Crash / Abnormal Termination

What. A process can reach Terminated two ways. Normal termination: it finishes its instructions and exits cleanly, conventionally with exit code 0 (Lesson 0.1.1). Abnormal termination, a crash: it stops before finishing because of an unhandled error (an exception), an external kill, or sudden power loss, exiting with a non-zero code (or none at all). Either way the process ends, but a crash ends it mid-work, before the code that would have saved its results ever runs, so the entire in-RAM process state is destroyed.

Why. This is where the whole lesson pays off. Because process state lives in volatile RAM, an abnormal termination at epoch 80 of 100 throws away all 80 epochs of progress unless they were already written to disk. This single fact is the reason real ML training writes checkpoints to disk, hours of GPU work must not depend on a process surviving to the very end. A crash is not an exotic event, it is the default thing you must design against.

When. Assume any long-running process can crash at any point, a bug on a rare input, the OS killing it for using too much memory, the laptop's battery dying. The defensive posture is always the same: write progress to disk often enough that a crash costs you a little, not everything.

Watch it happen. BUILD a loop that accumulates state in RAM, then BREAK it with a crash at epoch 3. Save this as crash_demo.py:

# crash_demo.py - BUILD then BREAK: state in RAM, destroyed by a crash
total = 0
for epoch in range(1, 6):                   # "train" for 5 epochs
    total = total + epoch                   # update the in-memory state
    print(f"epoch {epoch}: total={total}")
    if epoch == 3:                          # BREAK: something goes wrong mid-run
        # unhandled error -> abnormal termination
        raise RuntimeError("simulated crash!")
print("training finished, total =", total)  # NEVER reached, the crash killed the process

OBSERVE: run it and you see epochs 1, 2, 3 print, then a RuntimeError traceback, and the final line never runs. The variable total was only ever in RAM, so it is gone, nothing was saved. Three epochs of "work" evaporated because the process terminated abnormally before it could finish.

A crash leaves a fingerprint: a non-zero exit code. We can launch the crashing script as a child process and read that code, exactly as in Lesson 0.1.1. Save this as observe_crash.py:

# observe_crash.py
import subprocess, sys                      # subprocess + sys: from Lesson 0.1.1
# run the crashing script as a child
proc = subprocess.run([sys.executable, "crash_demo.py"])
print("child exit code:", proc.returncode)  # NOT 0 -> it terminated abnormally (a crash)

A clean finish would give returncode of 0, here it prints a non-zero value (typically 1) because the child died on an unhandled exception. That non-zero code is precisely how a parent detects "the child crashed" rather than "the child succeeded", the exit-code idea from Lesson 0.1.1, now meaning abnormal termination.

FIX: write the state to disk every epoch, so a crash can destroy at most the current step. Save this as crash_safe.py:

# crash_safe.py - FIX: checkpoint state to disk each epoch so a crash can't erase progress
total = 0
for epoch in range(1, 6):
    total = total + epoch
    with open("checkpoint.txt", "w") as f:  # overwrite the checkpoint every epoch (latest wins)
        f.write(f"{epoch},{total}")         # the state outlives the process BEFORE any crash
    print(f"epoch {epoch}: checkpointed total={total}")
    if epoch == 3:
        # crash again, but the checkpoint already exists
        raise RuntimeError("simulated crash!")

EXPLAIN: this crashes at epoch 3 just like before, but this time checkpoint.txt holds 3,6 on disk, written before the crash. A restart can read that file and resume from epoch 3 instead of starting at zero. The rule the whole lesson has been building toward: the only state that survives a crash is the state already written to persistent storage before the crash happened. RAM state dies with the process, disk state remains. Checkpointing trades a little disk I/O for the difference between "lost an epoch" and "lost everything."

DON'T:

• DON'T assume a crash is rare enough to ignore, for any long run, design as if it will crash partway through, because eventually one does.
• DON'T treat a non-zero exit code as your script being broken when it's a child you launched, it is the child reporting its abnormal termination, which is data for you to handle (Lesson 0.1.1).
• DON'T believe in-RAM progress is "basically saved" because the loop is almost done, until it is on disk, a crash one line before the end loses all of it.

Additional - crashes that are not exceptions

Not every crash is a Python exception. A force-kill (closing the program from Task Manager / kill on the process, or a sudden power loss) terminates the process even more abruptly, it doesn't even get to run cleanup code or finish a half-written file. That is why durability means "already on disk," never "about to be saved": you cannot assume a crash will give you a polite moment to save. (Two calls take a write all the way onto the physical disk: f.flush() hands the bytes Python is still holding inside your process to the OS, then os.fsync() makes the OS write them to the disk. You will meet both in the assessment and the assignment. For now, just know that "saved" is something you must do ahead of the crash, not during it.)

Lab Exercises

Both problems first, then both solutions.

Lab Exercise 1: Forgets vs Remembers

Covers the first half (Volatile Memory, Persistent Storage). (a) Write volatile.py that sets score = 0, increments it once, and prints it. Run it twice and confirm it prints 1 both times, explain in a comment why RAM does not remember the first run. (b) Write persist.py that writes the text model-v1 into a file tag.txt. Then write a separate script read_tag.py that opens tag.txt and prints its contents, proving disk hands data from one finished process to a brand-new one.

Lab Exercise 2: Saving Process State

Covers the second half (Memory vs Storage Distinction, Process State, Crash). Write progress.py that holds in-RAM state (epoch and total) over the dataset [5, 10, 15, 20], prints the state at each epoch, and after the loop writes the final epoch,total to progress.txt. End the script by printing one sentence stating which part of the run survives the process ending and which part does not.

# volatile.py - part (a): RAM is volatile, so each run starts from scratch
score = 0
score = score + 1
print("score this run:", score)

# persist.py - part (b): write a tag to disk so it outlives this process
with open("tag.txt", "w") as f:
    f.write("model-v1")
print("wrote model-v1 to tag.txt")

# read_tag.py - a separate process reads what persist.py left on disk
with open("tag.txt") as f:
    tag = f.read()
print("tag on disk:", tag)

# progress.py
dataset = [5, 10, 15, 20]
epoch = 0
total = 0

for value in dataset:
    epoch = epoch + 1
    total = total + value
    print(f"epoch {epoch}: total={total}")

with open("progress.txt", "w") as f:
    f.write(f"{epoch},{total}")

print("the file on disk survives, the in-RAM variables vanish when this process ends")

Theory Assessment

Conceptual reinforcement, not a coding section. All four questions with their hints first, then all four answers.

Q1: One forgets, one remembers

Two halves of the same fact. First, why does a variable created during a program's run no longer exist after that program ends? Second, what makes disk "persistent", and give one everyday example where you depend on that persistence without thinking about it.

Hint: For the first, recall what happened to the counter when you ran volatile_counter.py a second time. For the second, a recipe written in a notebook is still there tomorrow even after you close it and leave the kitchen.

Q2: Two opposite tools

List three concrete differences between RAM and disk. Then say which you would use to hold a dataset while actively processing it, and which to keep a trained model for tomorrow, and why.

Hint: The two axes that matter most are speed and whether the data survives the process ending.

Q3: What a crash destroys

Define a process's "state", then explain precisely why an abnormal termination at epoch 80 of 100 can lose all 80 epochs of progress.

Hint: Ask where epoch and the accumulated result physically live while the loop is still running.

Q4: The file that is empty even though the write ran

A program opens a file, writes to it, and crashes a millisecond later. Afterwards the file is empty. Give a plausible reason this happens even though the write() call definitely ran, and name what closes that gap. Then the part that catches people: explain why a force-kill or a power loss is less recoverable than an unhandled Python exception, even though both end the process.

Hint: "I called write" and "the bytes are physically on the disk platter" are not the same instant, and something sits between them. For the second part, ask which of the two endings still gives your program a final moment to act.

Assignment

One assignment. The problem and its hints first, then the solution.

Assignment: Atomic Checkpoint Trainer

Build atomic_trainer.py that "trains" over a dataset while keeping epoch and total as in-RAM state, checkpoints that state to disk every epoch, resumes from the checkpoint on startup if one is there, and is safe even if it crashes in the middle of writing the checkpoint. Save the state as structured JSON (a plain-text format for named values, such as {"epoch": 3, "total": 6}, which Python's built-in json module writes and reads), but never write directly over the real checkpoint file: write to a temporary file first, then atomically rename it over the real one, so the real checkpoint is always either the complete old state or the complete new state, never a half-written mess. After each save, reload the checkpoint to verify it parses. Train over dataset = [1, 2, 3, 4, 5], resume if a checkpoint exists, and crash once at epoch 3 to prove the saved checkpoint is still valid afterward.

Hints:

• Write the JSON to ckpt.tmp, then call os.replace("ckpt.tmp", "ckpt.json"), os.replace swaps the file in one atomic step, so a crash either leaves the old complete file or the new complete file, never a torn one.
• To verify a save, immediately json.load the checkpoint back and check it has the expected keys, resume by loading that same file at startup if os.path.exists says it is there.

# atomic_trainer.py
import os
import json

CKPT, TMP = "ckpt.json", "ckpt.tmp"
dataset = [1, 2, 3, 4, 5]

def save_atomic(state):        # write the new state without ever corrupting the old
    with open(TMP, "w") as f:  # 1) write the full new state to a TEMP file
        json.dump(state, f)    # dict -> JSON text
        f.flush()              # push out of Python's buffer...
        os.fsync(f.fileno())   # ...and force it onto the physical disk
    os.replace(TMP, CKPT)      # 2) atomically swap temp over real
    # at no instant is CKPT half-written: it is the old complete file until os.replace moves TMP onto its name in one step, then the new complete file

# resume if a valid checkpoint exists, else start fresh:
if os.path.exists(CKPT):       # is a checkpoint already on disk?
    with open(CKPT) as f:
        state = json.load(f)   # JSON text -> dict
    print("resumed:", state)
else:
    state = {"epoch": 0, "total": 0}
    print("starting fresh")

while state["epoch"] < len(dataset):
    state["epoch"] += 1
    state["total"] += dataset[state["epoch"] - 1]
    save_atomic(state)         # crash-safe checkpoint each epoch

    with open(CKPT) as f:      # verify the save actually parses back
        check = json.load(f)
    # confirm the checkpoint is well-formed
    assert "epoch" in check and "total" in check
    print(f"epoch {state['epoch']}: total={state['total']} (verified checkpoint)")

    # crash once at epoch 3
    if state["epoch"] == 3 and not os.path.exists("passed3.flag"):
        open("passed3.flag", "w").close()
        raise RuntimeError("crash mid-training at epoch 3!")

print("training complete:", state)

Additional Resources

Optional, and nothing later depends on either. Worth a look only if the lesson left you curious about the layer underneath it.

1. (YouTube) How computer memory works - Kanawat Senanan, TED-Ed. Five minutes on how memory physically holds bits, the leaking charge in a RAM cell first and then the magnetic, optical and flash storage that keeps data once the power is off, which is the hardware reason RAM is fast and volatile.
2. (Official docs) Python - os, sections "File Descriptor Operations" (os.fsync) and "Files and Directories" (os.replace). The calls behind an atomic, durable save that survives a crash mid-write.

Summary

This lesson drew the line every program lives by: the line between what is happening now and what is kept. On one side sits RAM, fast, roomy enough for the work in front of you, and utterly forgetful. Everything a running process knows, every variable and half-summed total and loaded dataset, lives there, and lives there only. On the other side sits the disk, slower, but with the one property RAM can never have: it remembers. The whole craft of writing durable programs is learning to move the things you care about across that line, from the workspace that forgets into the cabinet that keeps, at the right moments and for the right reasons.

The reason this matters is that a process is so much more fragile than it feels while it is running smoothly. Its state, all of it, is a structure built in volatile memory, and that structure exists at the mercy of the process staying alive. A clean exit takes it away gently, a crash takes it away violently, but both take it away, and neither cares how close to done you were. The only progress that survives either ending is the progress you had already written to disk. That is why "checkpoint" stopped being a piece of jargon partway through this lesson and became the central idea: a checkpoint is simply the act of copying live state across the line to safety, before you need it there.

You will feel the weight of this far beyond these toy scripts. When a model trains for ten hours and the machine reboots at hour nine, the difference between catastrophe and a shrug is whether someone wrote a checkpoint to disk. When a data job dies on the last file of a thousand, the difference between redoing everything and resuming is the same. The vocabulary you built here, volatile and persistent, state and termination, normal exit and crash, is the vocabulary you will reach for every time a long-running thing must be trusted to survive the world being unreliable. Everything ahead, all the way to training real models, is more elaborate state living in the same forgetful memory, made safe by the same deliberate act of saving it to the same kind of disk.

Cheat Sheet

Part A - Reference Table

Part B - Compact Code Block

# RAM is volatile: a fresh run forgets everything
runs = 0; runs += 1               # always 1 on the next run

# Disk is persistent: write text that outlives the process
with open("ckpt.txt", "w") as f:  # "w" overwrites, "a" appends
    f.write("epoch=3,total=6")    # bytes that outlive the process
loaded = open("ckpt.txt").read()  # a later process reads it back

# Checkpoint = copy in-RAM state to disk BEFORE a crash can hit
state = {"epoch": 3, "total": 6}  # process state (lives in RAM)
import json, os
with open("ckpt.json", "w") as f:
    # force onto disk
    json.dump(state, f); f.flush(); os.fsync(f.fileno())

# A crash terminates abnormally -> non-zero exit code
import subprocess, sys
# != 0
print(subprocess.run([sys.executable, "crash_demo.py"]).returncode)

# inspect a saved file straight from disk (no Python, same on every OS)
cat ckpt.txt  # print the file's bytes as text