Decomposition and Composition - Building With Functions

Context

Decomposition is splitting one large task into several small functions that each do one thing. Composition is feeding the output of one into the next, so the small pieces together perform the original task.

A pipeline is exactly this. Read the rows, drop the malformed ones, convert the numeric fields, compute the summary, write the report, five functions chained in order rather than one function doing all of it. Built that way, a change to how missing values are handled touches one function, and the stage that produced a wrong number can be identified by checking each stage's output in turn instead of reading the whole script.

A single function doing everything cannot be tested in pieces or reused in part, so when the next dataset needs the same cleaning with a different summary, the only options are copying it or adding another flag to it, and both make the next change harder than the last. Ordering matters just as much, because composing the same stages in the wrong sequence, summarizing before dropping bad rows, gives an answer that is confidently wrong. The previous lesson defined what a good single function looks like, this lesson assembles them.

Goal

• Split a multi-step task into single-responsibility functions, each with one clear job.
• Name each function for the one thing it does, so its purpose is obvious from the call.
• Compose functions by feeding one's returned value straight into the next.
• Trace data flowing through a load → clean → transform → summarize pipeline.
• Judge where to draw the boundary between two functions instead of one.

Prerequisites

Skip both if you like, nothing here depends on them. They are offered for the picture, not as a gate.

• YouTube - "Programming Terms: First-Class Functions" by Corey Schafer. Watch for the picture of a function handed to another function as a value.
• Official docs - Functional Programming HOWTO, "Introduction", for the broader mindset of building from small pieces.

Table of Contents

• Decomposition
• Single Responsibility
• Function Composition

Theory

Nothing needs installing for this lesson. Decomposition, single responsibility, and composition are design ideas built on the plain functions you already know, with nothing to import.

Decomposition

What. Decomposition is splitting a problem into smaller, separately solvable parts, each handled by its own function. Why. A single large function that does everything is hard to read, hard to test, and hard to change without breaking something far away. Breaking the task into named pieces lets you reason about, test, and fix each part on its own. When. Decompose whenever a function is doing more than one nameable thing, or whenever it has grown long enough that you must scroll to understand it.

The move is to look at one big job and ask "what are the distinct steps?" Each distinct step becomes its own function with a name. The original task then becomes a short sequence of calls, which reads like a description of the steps.

# One big function doing everything at once, hard to follow or test.
def process(raw):                   # raw is a single comma-free numeric string like "88"
    cleaned = raw.strip()           # remove surrounding spaces
    number = float(cleaned)         # text -> number
    scaled = (number - 50) / 10     # center and scale
    return "result=" + str(scaled)  # format for display

Here .strip() is a string method that removes surrounding whitespace, a small forward reference covered fully in Lesson 1.5.1, for now read it simply as the cleaning step. Four unrelated concerns are jammed into one function, so to test the normalizing alone you must run the whole thing, and a change to formatting risks the parsing.

# Decomposed: each step is a separate, testable function.
def clean(raw):            # step 1: tidy the text
    return raw.strip()

def parse(text):           # step 2: text -> number
    return float(text)

def normalize(number):     # step 3: center and scale
    return (number - 50) / 10

def format_result(value):  # step 4: number -> display string
    return "result=" + str(value)

You can now test normalize with a number and never touch the parsing, and changing format_result cannot break clean. The next subtitle sharpens how small each piece should be.

DON'T:

• DON'T decompose into pieces so tiny they add noise, a wrapper used once whose name says no more than the call inside it rarely earns its place.
• DON'T leave several unrelated concerns in one function just because it runs, untangling them later is harder.

Single Responsibility

What. Single responsibility means each function does exactly one well-named thing, one reason to exist, one reason to change. Why. A function with one job is easy to name, easy to test, and safe to modify, because its single concern is isolated. When a function does several things, a change to one concern can silently damage another. When. Aim for single responsibility every time you write or split a function, if you cannot name it without using "and," it is probably doing too much.

The naming test is the quickest check. If an honest name would be clean_and_parse_and_normalize, the function has three responsibilities and should be three functions. A good name like normalize describes one thing, and the body should do only that thing.

# VIOLATION: this function cleans AND parses AND normalizes. The name lies.
def prepare(raw):                # really does three jobs
    number = float(raw.strip())  # cleaning + parsing tangled together
    return (number - 50) / 10    # plus normalizing

print(prepare("  88 "))          # 3.8, but you cannot test the steps separately

It runs fine, and that is the problem, a bug in cleaning and a bug in normalizing both hide behind the same name and neither can be checked without exercising the other.

# Single responsibility: one job per function, each name honest.
def clean(raw):         # ONLY cleans
    return raw.strip()

def parse(text):        # ONLY parses
    return float(text)

def normalize(number):  # ONLY normalizes
    return (number - 50) / 10

print(normalize(88))    # 3.8 - tested in complete isolation

The judgment of where one responsibility ends and the next begins is the real skill here, too coarse and you get prepare, too fine and you drown in trivial wrappers.

DON'T:

• DON'T give a function a name with "and" in it, that is the signal it has more than one responsibility.
• DON'T split a single coherent step into multiple functions just to chase the rule, one responsibility can be a few lines.

Function Composition

What. Function composition is feeding one function's returned value directly into another function, so the output of each step becomes the input of the next. Why. Composition is how small single-purpose functions become a complete task. It is only possible because functions return values, and it is cleanest when the pieces are pure, since each step then depends only on the value handed to it. When. Compose functions whenever a task is a sequence of transformations, load, then clean, then transform, then summarize.

The mechanics are simply nesting calls or chaining them through variables. Because a call evaluates to its return value, parse(clean(raw)) runs clean first, then feeds its result into parse. The same flow can be written step by step with intermediate variables, which is usually more readable.

# Reuse the single-purpose functions from before.
def clean(raw): return raw.strip()
def parse(text): return float(text)
def normalize(number): return (number - 50) / 10
def format_result(value): return "result=" + str(value)

# COMPOSE by nesting: inner call runs first, its result feeds outward.
# clean -> parse -> normalize -> format
out = format_result(normalize(parse(clean("  88 "))))
print(out)  # result=3.8

A nested composition is read from the inside out, which is the opposite of the direction you read the line, and that is the single most common way people mis-trace one.

Deeply nested calls get hard to read, so the same pipeline is usually written as a sequence of steps, which also makes each intermediate value inspectable.

# Same pipeline, written step by step, clearer and debuggable.
def run_pipeline(raw):            # composes four single-purpose functions
    cleaned = clean(raw)          # step 1 output
    number = parse(cleaned)       # step 2 takes step 1's output
    scaled = normalize(number)    # step 3 takes step 2's output
    return format_result(scaled)  # step 4 takes step 3's output

print(run_pipeline("  88 "))      # result=3.8

run_pipeline is itself a single-responsibility function, its one job is to orchestrate the four steps in order. Every intermediate value now has a name, which means every one of them can be printed while you are hunting for the stage that went wrong. Load → clean → transform → summarize is exactly this shape, and it is the blueprint for every data pipeline you will build later in the program.

DON'T:

• DON'T nest so many calls that the order becomes unreadable, switch to step-by-step variables past two or three levels.
• DON'T compose functions that have hidden side effects, impure steps can make the pipeline's result depend on more than the values flowing through it.

Lab Exercises

Lab Exercise 1: Break It Apart and Name It Honestly

You are given one function process(raw) whose body strips a string, converts it to a float, and doubles it. Decompose it into three single-responsibility functions, clean, to_number and double, call them in sequence on " 21 ", and print the final result.

Then write a second function prep(raw) that deliberately does all three jobs in one body, and add a comment giving it the honest name it would need if the name had to describe everything it does. State in that comment what the honest name reveals about the design.

Lab Exercise 2: Compose It Two Ways, Then Build a Pipeline

Using add_ten(x) returning x + 10 and halve(x) returning x / 2, compute the result for 30 twice, once by nesting the two calls on one line and once step by step with an intermediate variable. Print both and confirm in a comment that they match, and that the nested form runs the inner call first.

Then write four single-purpose functions, clean(raw), parse(text), normalize(number) returning (number - 50) / 10, and label(value) returning "high" when value >= 0 and "low" otherwise. Write pipeline(raw) that composes all four step by step and returns the label, then print pipeline(" 73 ") and pipeline(" 40 ").

# Each function now has ONE responsibility.
def clean(raw):              # only strips whitespace
    return raw.strip()

def to_number(text):         # only converts text to float
    return float(text)

def double(number):          # only doubles
    return number * 2

# Call them in sequence: each takes the previous result.
cleaned = clean("  21 ")     # "21"
number = to_number(cleaned)  # 21.0
result = double(number)      # 42.0
print(result)                # 42.0

# All three jobs fused back into one body, for contrast.
def prep(raw):        # runs correctly, and that is what hides the problem
    return float(raw.strip()) * 2

print(prep("  21 "))  # 42.0
# An honest name would be 'clean_and_convert_and_double'. Needing two 'and's to
# describe it is the design problem, the name is only the symptom that reports it.

def add_ten(x):              # one job
    return x + 10

def halve(x):                # one job
    return x / 2

nested = halve(add_ten(30))  # 'add_ten' is the INNER call, so it runs first
added = add_ten(30)          # the same step, given a name
stepwise = halve(added)      # and fed forward explicitly

print(nested)                # 20.0
print(stepwise)              # 20.0
# Identical results. The step form is not slower or different, it is only visible.

# Four single-purpose functions.
def clean(raw):                 # strips
    return raw.strip()

def parse(text):                # text -> number
    return float(text)

def normalize(number):          # center and scale
    return (number - 50) / 10

def label(value):               # number -> category
    if value >= 0:
        return "high"
    return "low"

# Orchestrator: its one job is to run the steps in order.
def pipeline(raw):              # composes the four
    cleaned = clean(raw)        # step 1
    number = parse(cleaned)     # step 2
    scaled = normalize(number)  # step 3
    return label(scaled)        # step 4

print(pipeline("  73 "))        # high
print(pipeline("  40 "))        # low

Theory Assessment

Q1: Count the Jobs, Then Read the Name

A function prep(raw) strips a string, converts it to a float, and rounds it. How many responsibilities does it have and how many functions should it become? Then explain why a function honestly named clean_and_parse is a sign of a design problem rather than merely a naming one.

Hint: Count the distinct nameable jobs. Then ask what the word "and" in an honest name is actually reporting.

Q2: Which Runs First, and Where Does the Value Go

For halve(add_ten(30)), where add_ten adds 10 and halve divides by 2, which function runs first and what is the final value? Then trace pipeline(" 40 ") through clean, parse, normalize (computing (n - 50) / 10) and label ("high" when >= 0, else "low"), naming every intermediate value on the way to the final label.

Hint: An argument must be fully evaluated before the call it belongs to can happen, which fixes the order of a nested composition regardless of how it reads left to right.

Q3: Why Purity Composes, and When to Stop Splitting

Why is composing pure functions safer than composing functions that print or write files? Then judge this: a teammate writes def strip_it(s): return s.strip() and uses it in exactly one place. Is that good decomposition? Argue it either way.

Hint: For the first half, ask what each stage depends on and what it leaves behind. For the second, remember that decomposition has a cost as well as a benefit, and that the cost is paid in names and indirection.

Q4: Same Functions, Same Input, Different Answer

Read the following without running it, then state what each print shows.

def center(number):  # pure, one job
    return number - 50

def scale(number):   # pure, one job
    return number / 10

def label(value):    # pure, one job
    if value >= 0:
        return "high"
    return "low"

print(label(scale(center(70))))
print(label(center(scale(70))))

Every function here is pure and every one has a single responsibility, and the two lines are built from exactly the same three functions on exactly the same input. Explain why the answers differ, and say what that tells you about what purity does and does not protect you from.

Hint: Work out the intermediate value each composition produces before label sees it. Neither line raises an error, so the thing that differs is not correctness in any sense Python can check.

Assignment

Assignment: A Pluggable Pipeline, and the Two Ways It Breaks

Build the following in one file, in this order.

Write the fixed stages clean(raw) and parse(text), then two pure transforms, normalize(n) returning (n - 50) / 10 and square(n) returning n * n, and two pure deciders, high_low(v) returning "high" when v >= 0 and "low" otherwise, and pos_neg(v) returning "positive" when v > 0 and "non-positive" otherwise.

Now write pipeline(raw, transform, decide) that cleans and parses the input, applies the function transform, then applies the function decide, both passed in as arguments rather than named inside the body. Print the result of at least three different combinations on the same input, so the varying stages are visible against a fixed one.

Then demonstrate both failure modes rather than describing them. First, call the pipeline passing normalize() with parentheses instead of normalize, inside a try, and print the error Python raises. Second, build two pipelines from the same two pure stages composed in opposite orders, print both results, and add a comment stating which one is wrong and why no error was raised for it.

Hints:

• A function name without parentheses is the function itself, a value you can pass around, while adding parentheses calls it immediately and passes whatever it returned, so transform and decide are handed over bare and called inside as transform(number) and decide(value).
• For the second failure, remember every stage involved is pure and correct on its own, so nothing in the individual functions can be blamed, and work out the intermediate value each ordering hands to the decider.

# Fixed stages.
def clean(raw):                             # only strips
    return raw.strip()

def parse(text):                            # only converts
    return float(text)

# Two transforms, both pure.
def normalize(n):                           # center and scale
    return (n - 50) / 10

def square(n):                              # square it
    return n * n

# Two deciders, both pure.
def high_low(v):                            # threshold at zero
    if v >= 0:
        return "high"
    return "low"

def pos_neg(v):                             # sign label
    if v > 0:
        return "positive"
    return "non-positive"

# 'transform' and 'decide' arrive as FUNCTIONS, not as results.
def pipeline(raw, transform, decide):       # two pluggable stages
    number = parse(clean(raw))              # the fixed part
    changed = transform(number)             # call whichever transform was handed in
    return decide(changed)                  # call whichever decider was handed in

# high
print(pipeline("  73 ", normalize, high_low))
print(pipeline("  73 ", square, high_low))  # high
# positive
print(pipeline("  73 ", normalize, pos_neg))

# FAILURE 1: parentheses call the function now instead of handing it over.
try:
    # 'normalize()' runs immediately, with no argument
    pipeline("  73 ", normalize(), high_low)
except TypeError as e:              # bind the error object to 'e' so it can be printed
    print("passing normalize() failed:", e)

# FAILURE 2: same two pure stages, composed in opposite orders.
def center(n):                      # pure, one job
    return n - 50

def scale(n):                       # pure, one job
    return n / 10

print(high_low(scale(center(70))))  # high
print(high_low(center(scale(70))))  # low
# The second is wrong. 'center' must run before 'scale' for the result to mean
# anything, but both stages are pure and correct, so nothing raised and nothing
# is None. Order is a property of the arrangement, not of any function in it.

passing normalize() failed: normalize() missing 1 required positional argument: 'n'
high
low

Additional Resources

Optional, and nothing downstream needs them. Worth opening only if you want to read more about building from small pieces.

• YouTube - "Python Full Course for Beginners" by Programming with Mosh, the functions segment. A second pass on defining functions and passing them arguments, keyword and default ones included.
• Official docs - The Python Tutorial, "Function Annotations". A look at documenting the types flowing between composed stages.

Summary

This lesson is where functions stop being isolated tricks and become a way of building. Decomposition is the act of looking at one tangled job and seeing the distinct steps inside it, then giving each step its own function. Single responsibility is the discipline that keeps those pieces honest, one nameable job per function, so that the name tells the truth and a change to one concern is far less likely to quietly break another. The naming test is the whole rule in miniature: if an honest name for a function needs the word "and", it is probably doing too much. None of this changes what your code computes, it changes how easy your code is to read, test, and survive contact with the next change.

Composition is the reward for that discipline. Because functions return values, the output of one becomes the input of the next, and a sequence of small single-purpose functions snaps together into a complete task. Written as nested calls it reads inside-out, written as step-by-step variables it reads top-to-bottom and exposes every intermediate value for debugging, both are the same flow of data through a chain of transformations. The cleanest pipelines are built from pure functions, because then each stage depends only on the value handed to it and leaves nothing behind, so the whole pipeline is as predictable as its parts.

The real skill, the one that takes practice rather than memorization, is judgment about where to draw the boundaries, coarse enough to avoid a clutter of trivial wrappers, fine enough that each piece has one clear reason to exist. Load → clean → transform → summarize is the shape you will return to again and again, and it is no accident that it is a composition of single-purpose functions. Every data pipeline in the rest of this program is this lesson scaled up, which is why the habits you build here, name one job, keep it pure, pass the result forward, are the quiet foundation of everything that follows.

Cheat Sheet

def clean(raw): return raw.strip()      # one job
def parse(text): return float(text)     # one job
def normalize(n): return (n - 50) / 10  # one job
def label(v):                           # one job (explicit branch)
    if v >= 0: return "high"            # threshold at zero
    return "low"

def pipeline(raw):                      # orchestrator: only sequences
    n = parse(clean(raw))               # step 1 -> 2
    return label(normalize(n))          # step 3 -> 4

print(pipeline("  73 "))                # high

def pipeline2(raw, transform):          # takes a function: step is pluggable
    # pass 'transform' without ()
    return label(transform(parse(clean(raw))))