Dictionaries and Sets - Lookup, Uniqueness, and Membership
Context
A dictionary stores values under keys you choose rather than under numeric positions, so you hand it a key and it hands back the matching value. A set holds the keys and nothing else, unordered and with no value allowed to appear twice.
Named lookup and uniqueness are the two questions data work asks constantly. One row of a dataset is far clearer as a record whose fields are reached by column name than as a list where the price happens to sit at position four, counting rows per category is a dictionary from category to count, and a model's settings are a dictionary from setting name to value. Which categories actually appear in a column, which identifiers repeat, and which tokens (the individual words of a text) the test split (the data held back from training to score the model) carries that training never saw are all set questions, answered in one expression each.
Asking a dictionary with [] for a key that is not there raises KeyError, which matters because real data is full of absent fields and unexpected categories. Duplicate keys are the quiet version of the same problem, the second write replaces the first and nothing is raised. Sets carry their own silent cost, because order and duplicate counts are destroyed on the way in and cannot be recovered. The previous lessons reached values by position, this one reaches them by name and by presence.
Goal
- Build a dictionary literal and an empty dictionary, and add or overwrite entries by key.
- Retrieve values with
[]and.get(), and distinguish their failure behavior precisely. - Test key membership with
in, and iterate keys, values and pairs with.keys(),.values()and.items(). - Build a count-per-key tally by widening the accumulator pattern across keys.
- Deduplicate any collection with a set, and apply union, intersection and difference to answer shared, missing and combined questions.
Prerequisites
Neither is required, the lesson stands without them. Open one if you would rather meet these containers somewhere else first.
- YouTube - "Python Tutorial for Beginners 5: Dictionaries - Working with Key-Value Pairs" by Corey Schafer. Watch the build, access, and iterate sections.
- Official docs - The Python Tutorial, "Sets". Read it once for the literal syntax and the operations.
Table of Contents
- Dictionary, Keys and Values
- Dict Access & Methods
- Membership and Iteration
- Set
- Set Operations
Theory
A label-to-count table, a column-name-to-values mapping, a config of hyperparameters, these are all dictionaries, the workhorse container of data work. A set is the same machinery with the values thrown away, keeping only the question "is this in here, and how many distinct things are". Both build on tuples, because both file their keys by a value computed from them, which is exactly the property a tuple that is immutable all the way down guarantees and a list does not.
Dictionary, Keys and Values
What. A dictionary is a mapping from unique keys to values. Each key appears at most once and is associated with exactly one value. The key is the label you look up by, usually an immutable value such as a string, a number or a tuple of them, the value is whatever data is stored under it. Insertion order is preserved, guaranteed by the language since Python 3.7.
Why. Lookup by position is wrong for most data. You rarely want "the value at index 7", you want "the accuracy for model 'rf'" or "the count for label 'spam'". Keys are restricted and values are not, because the dictionary uses the key to locate the entry and never inspects the value at all.
When. Reach for a dictionary whenever data is naturally a set of named values: configurations, lookups, counts-per-category, any "X maps to Y" relationship. Use a list instead when order and position are what matter.
A dictionary literal uses curly braces with key: value pairs separated by commas. An empty dictionary is {}, and that expression is worth marking now, because the empty braces belong to the dictionary and an empty set has to be built another way.
# A config mapping hyperparameter names to values.
config = {"epochs": 10, "lr": 0.01, "batch_size": 32}
# An empty dictionary, ready to be filled.
counts = {}
# Add or overwrite by assigning to a key.
counts["spam"] = 1 # adds the key "spam" with value 1
counts["spam"] = 2 # overwrites: "spam" now maps to 2
config["epochs"] = 20 # overwrites an existing keyAssignment to a key both adds and updates, with nothing in the line to say which, and which one happened depends entirely on whether the key was already there. That is also why a mistyped key silently gains you an entry instead of an error.
# Duplicate key "a": the LAST value wins, no error.
d = {"a": 1, "b": 2, "a": 3} # 'd' is {"a": 3, "b": 2}
# Strings, numbers and TUPLES of them can be keys, a list cannot.
grid = {(0, 0): "origin", (1, 2): "cell"} # a fixed coordinate as a key
bad = {[1, 2]: "x"} # raises TypeError: unhashable type: 'list'
# Values carry NO such restriction, and may repeat.
columns = {"sepal_length": [5.1, 4.9, 6.3], "label": ["a", "b", "c"]}
model = {"name": "rf", "params": {"depth": 5, "trees": 100}}Uniqueness constrains keys only, and what may BE a key ties straight back to the previous lesson. A dictionary files an entry by computing something from the key (that computed value is the key's hash, the word the error message uses) and finds it again by computing that same value, so a key that could change after insertion would leave its own entry unreachable, and Python refuses a list, a dictionary or a set as a key rather than let that happen. The freedom on the value side is what lets columns be a tiny table and model nest a dictionary inside a dictionary.
DON'T:
- DON'T expect
{}to be a set.{}is an empty dictionary, and an empty set is writtenset(). - DON'T rely on duplicate keys raising an error, the last value silently wins and can hide a typo.
- DON'T use a list as a key, reach for a tuple when you need a compound key.
Dict Access & Methods
What. Access means retrieving a value, and the core tools are [] for direct lookup, .get() for safe lookup, and the view methods .keys(), .values() and .items().
Why. The difference between [] and .get() is a difference in failure behavior, and choosing wrong is a real bug. [] raises on a missing key, which is right when absence is an error. .get() returns a default, which is right when absence is normal.
When. Use [] when the key must exist and its absence signals a bug. Use .get() when the key may be missing and you want a sensible fallback instead of a crash. Use the view methods to iterate.
Square brackets retrieve the value for a key and raise KeyError if it is absent. When a missing key genuinely means something went wrong, that crash is the feature.
config = {"epochs": 10, "lr": 0.01}
# [] returns the value for a present key.
e = config["epochs"] # 'e' is 10
# [] on a MISSING key raises KeyError, halting the program.
b = config["batch_size"] # raises KeyError: 'batch_size'.get() is the safe alternative, returning the value if present or a default if absent, never raising KeyError.
config = {"epochs": 10, "lr": 0.01}
print(config.get("epochs")) # 10
print(config.get("batch_size")) # None, no error, no default supplied
print(config.get("batch_size", 32)) # 32, the fallback
print(list(config.keys())) # ['epochs', 'lr']
print(list(config.values())) # [10, 0.01]
print(list(config.items())) # [('epochs', 10), ('lr', 0.01)]Read the second print carefully. A .get() with no default hands back None, which is not 0 and not "", and which raises TypeError the moment you do arithmetic with it, well away from the line that produced it. The last three calls return view objects rather than lists, which is why list() wraps them just to print. A view stays connected to the dictionary and reflects later changes rather than being a snapshot.
DON'T:
- DON'T use
[]when a key might be missing and nothing handles theKeyError. Use.get()with a default. - DON'T assume
.get()of a missing key is0or"". With no default it isNone, andNonein arithmetic raisesTypeError.
Membership and Iteration
What. in tests whether a key exists, returning True or False. Looping the dictionary directly visits keys, while .items() visits (key, value) pairs you unpack together.
Why. Before you act on a key, incrementing a count or reading with [], you often need to know whether it is there, and in answers that cheaply. Almost every use of a dictionary then ends in a loop, to print every entry, sum every value, or transform the data.
When. Use in to guard a [] access or to separate "first time seen" from "seen before". Loop directly or over .keys() for keys alone, over .values() for values alone, and over .items() with unpacking when you need both, which is the most common case.
in checks keys, not values. Testing a value requires pointing it at .values() instead.
counts = {"spam": 3, "ham": 5}
print("spam" in counts) # True
print("eggs" in counts) # False
print(3 in counts) # False, 3 is a value not a key
print(3 in counts.values()) # True, 3 is among the valuesThe line worth staring at is 3 in counts, False even though 3 is plainly in the dictionary. It is in there as a value, and in does not look at values, so the answer is a correct response to a different question than the one you meant to ask.
config = {"epochs": 10, "lr": 0.01, "batch_size": 32}
# Looping the dict directly yields KEYS.
for key in config:
print(key) # epochs, then lr, then batch_size
# .items() yields (key, value) tuples, unpacked into two names.
for key, value in config.items():
print(key, value) # epochs 10, then lr 0.01, then batch_size 32
# .values() when only the values matter.
total = 0
for v in config.values():
total += v # every value in this config is numericThe middle loop is the loop-with-unpacking idiom from the tuples lesson, because .items() yields ordinary two-item tuples. The payoff is the count-per-key tally, which widens the accumulator you already know so one dictionary holds many running totals at once, one bucket per key.
labels = ["spam", "ham", "spam", "spam", "ham"]
# The accumulator pattern, but a dict gives ONE bucket per distinct key.
counts = {}
for label in labels: # walk every label
# default 0 on first sight, then +1
counts[label] = counts.get(label, 0) + 1
print(counts) # {'spam': 3, 'ham': 2}The assignment at the front of that line is doing as much work as the .get at the back. .get(label, 0) produces a number, + 1 produces the next one, and counts[label] = is what stores it. Drop the assignment and the computed value is discarded, a failure with nothing to see.
DON'T:
- DON'T use
into test for a value and expect a hit, usevalue in d.values()for that. - DON'T loop
for k, v in config:over the bare dictionary, it yields keys alone, so the unpacking takes each key apart instead of pairing it with its value, which raisesValueErroron a key like"epochs"and silently splits a two-letter key like"lr". - DON'T add or remove keys while iterating the same dictionary. A change in its size raises
RuntimeErrorat the loop's next step, and an add matched by a remove in the same step can raise nothing and skip a key.
Set
What. A set is an unordered collection of unique values. Unordered means there is no index and no guaranteed position. Unique means it cannot contain the same value twice. Why. Some questions are not about order or position at all, they are about membership and distinctness. A set answers these directly and quickly, where a list would force you to scan and a dictionary would carry unused values. When. Reach for a set when you care about uniqueness or fast membership and do not care about order. Use a list when order or duplicates matter, and a dictionary when each value needs an associated payload.
A set literal uses curly braces with comma-separated values, like a dictionary but with no colons. Duplicates in the literal are silently collapsed.
labels = {"setosa", "versicolor", "setosa", "virginica"}
print(len(labels)) # 3, the duplicate "setosa" was dropped
not_a_set = {} # WRONG for a set, this builds an empty DICTIONARY
empty = set() # the constructor builds an empty set
empty.add("audio")
empty.add("audio") # already present, so this changes nothing and does not raise
print(empty) # {'audio'}
raw = ["a", "b", "a", "c", "b", "a"]
print(len(set(raw))) # 3, the distinct count, no intermediate list
print(sorted(set(raw))) # ['a', 'b', 'c'], dedup then a stable orderNever index a set and never read its printed order as meaningful. Its elements carry the same requirement dictionary keys do, and for the same reason, {1, 2, (3, 4)} is fine and {[1, 2]} raises TypeError: unhashable type: 'list'. Deduplication is the everyday use, and list(set(xs)) gives back an indexable sequence while sorted(set(xs)) gives back a stable one.
Membership is where the set earns its place a second time. x in some_list compares x against the elements one at a time until it matches or runs out, so a miss on a long list touches every element. x in some_set computes a value from x and goes more or less straight to where that value would be stored, so the work does not grow as the set grows. A membership test inside a loop is the single most common reason to build a set first.
DON'T:
- DON'T use
{}to make an empty set, the dictionary already took that expression. Useset(). - DON'T expect
set(my_list)to preserve order, sort it, or uselist(dict.fromkeys(my_list)), which builds a dictionary keyed by the items in first-appearance order and lists its keys. - DON'T deduplicate with a set when duplicates carry meaning, a frequency dictionary keeps the counts.
Set Operations
What. Set operations compare two sets: union (everything in either), intersection (only what is in both), difference (what is in the first but not the second), and symmetric difference (what is in exactly one). Each has an operator form and a method form. Why. These answer the recurring membership questions of data validation in one expression each. Which categories appear in both splits, which appear in training but not test, what is the full set across both. Written by hand with loops they are verbose and easy to get wrong. When. Use intersection to find shared values, difference to find missing or extra values, and union to merge distinct values.
train_labels = {"cat", "dog", "bird"}
test_labels = {"dog", "bird", "fish"}
all_labels = train_labels | test_labels # union: {'cat', 'dog', 'bird', 'fish'}
shared = train_labels & test_labels # intersection: {'dog', 'bird'}
only_train = train_labels - test_labels # in training, NOT in test: {'cat'}
only_test = test_labels - train_labels # the OTHER direction: {'fish'}
exclusive = train_labels ^ test_labels # in exactly one: {'cat', 'fish'}
# Method forms do the same thing.
shared_m = train_labels.intersection(test_labels)Difference is the one operation where operand order changes the answer, and the two directions ask genuinely different questions, "what did training have that test lacks" against "what did test bring that training never saw". The second is the out-of-vocabulary question and usually the one that matters, because those are the values a model has no basis for handling. One practical distinction separates the two forms: an operator with a set on one side needs a set, or a set-like view such as a dictionary's .keys(), on the other, while the methods accept any iterable on the right.
labels = {"cat", "dog"}
incoming = ["dog", "fish"] # a LIST, not a set
# shared = labels & incoming
# TypeError: unsupported operand type(s) for &: 'set' and 'list'
print(labels.intersection(incoming)) # {'dog'}, the method converts for youDON'T:
- DON'T assume
a - bequalsb - a. Difference is directional, pick the order that matches your question. - DON'T use the operator form with a non-set on the right, use the method form or convert first.
Lab Exercises
Lab Exercise 1: A Config, Two Ways to Look Up, and a Tally
Create a dictionary config holding "epochs": 10, "lr": 0.01 and "optimizer": "sgd". Overwrite "epochs" to 25, add "batch_size": 64, then print the dictionary, the value of "lr" and the number of entries.
Then, given metrics = {"accuracy": 0.92, "f1": 0.88}, read "accuracy" with [] and "precision" with .get() defaulting to 0.0, and print both. Add a commented-out [] read of "precision" naming the exact error it would raise, then print .get("precision") with no default and comment on why that value is dangerous in arithmetic.
Finally, given labels = ["spam", "ham", "spam", "eggs", "ham", "spam"], build a counts dictionary of frequencies with the .get(key, 0) + 1 idiom and print it. Print 3 in counts and 3 in counts.values() with a comment on why they differ, then scan counts.items() to print the most frequent label with its count, without sorting anything.
Lab Exercise 2: Dedup, Then Shared, Missing and Combined
Given tags = ["nlp", "cv", "nlp", "rl", "cv", "nlp"], print the number of distinct tags and a sorted list of them, with a comment saying why the sorted list, not the raw set, belongs in a report. Then build an empty set the correct way, add "audio" to it twice, and print it with its length, including a commented line showing the wrong way and what that expression actually builds.
Then, given train_tokens = ["the", "cat", "sat", "the", "mat"] and test_tokens = ["the", "dog", "ran", "fast", "the"], build each vocabulary as a set and print the shared tokens, the test tokens never seen in training, the training tokens the test split never exercises, and the size of the combined vocabulary, every set through sorted() so the output is stable. Finally show the operator form refusing a list on the right as a commented line with the error it raises, then do the same operation with the method form.
Solution 1: A Config, Two Ways to Look Up, and a Tally
config = {"epochs": 10, "lr": 0.01, "optimizer": "sgd"}
config["epochs"] = 25 # key exists, so this OVERWRITES
config["batch_size"] = 64 # key is new, so this ADDS
print(config) # {'epochs': 25, 'lr': 0.01, 'optimizer': 'sgd', 'batch_size': 64}
print(config["lr"]) # 0.01
print(len(config)) # 4metrics = {"accuracy": 0.92, "f1": 0.88}
acc = metrics["accuracy"] # key must exist, and it does
prec = metrics.get("precision", 0.0) # key may be missing, fall back to 0.0
print(acc, prec) # 0.92 0.0
# metrics["precision"] # KeyError: 'precision'
print(metrics.get("precision")) # None
# 'None' is not 0.0 and not "". A '.get' with no default hands back None, and
# 'None + 1' raises TypeError, so an omitted default moves the crash later.labels = ["spam", "ham", "spam", "eggs", "ham", "spam"]
# One running total per distinct key, the accumulator widened across keys.
counts = {}
for label in labels:
# 0 on first sight, then +1
counts[label] = counts.get(label, 0) + 1
print(counts) # {'spam': 3, 'ham': 2, 'eggs': 1}
print(3 in counts) # False
print(3 in counts.values()) # True
# 'in' on a dictionary searches KEYS. 3 is in there as a value, so the first
# line answers a question about keys correctly and it is not the question asked.
# Scan the pairs, keeping the best seen so far.
top_label = None
top_count = -1 # below any real count, so the first item always wins
for label, c in counts.items():
if c > top_count:
top_count = c
top_label = label
print(top_label, top_count) # spam 3DON'T:
- DON'T reach for a separate "add" method, assignment to a key both adds and updates and you cannot tell which from the line itself.
- DON'T omit the default from
.get()when the result feeds arithmetic,Nonewill not error where you wrote it, it errors somewhere else. - DON'T carry a fixed seed such as
0or-1to a scan whose values can go negative, seed from the first item, asmin_max_meandid in Lesson 1.4.2.
Theory Assessment
Q1: What Assignment Does, and Which Container Owns the Braces
Given d = {"a": 1}, what does d["a"] = 5 do, and what does d["b"] = 2 do, and why does one syntax cover both? Then say what {} builds, how you build the other brace-shaped container empty, and why a key must be immutable while a value need not be.
Hint: Whether an assignment adds or replaces depends on the dictionary, not on how the line is written. For the last part, ask how the dictionary would find an entry again if the key it filed it under had since changed.
Q2: Three Lookups, Three Outcomes
For config = {"epochs": 10}, state exactly what config["lr"], config.get("lr") and config.get("lr", 0) each produce and when you would choose each. Then say what "epochs" in config and 10 in config return, and how you would ask the question 10 in config fails to answer.
Hint: One lookup stops the program, one hands back a value that looks harmless and is not. For the second half, one operator and one bare loop both default to the same half of the entry.
Q3: What a Set Throws Away, and the One Operation With a Direction
Given {"a", "b", "a"}, what does the set contain, what is its length, and which two properties of the original collection are destroyed? Then for train = {"a", "b", "c"} and test = {"b", "c", "d"}, give train & test, train | test, train - test and test - train, say which changes its answer when the operands swap, and name the validation question each difference direction answers.
Hint: A set keeps one of each value, so ask what that destroys. Two of the three operations treat both sides identically and one does not.
Q4: Six Values, Two Buckets
Read the following without running it, then state what each print shows.
flags = [1, True, 1.0, 0, False, 0.0]
counts = {}
for f in flags:
counts[f] = counts.get(f, 0) + 1
print(set(flags))
print(len(set(flags)))
print(counts)Six values went in and the set holds two. Explain why, and explain why the dictionary shows the counts it does and why its keys print the way they do rather than as True and False.
Hint: Uniqueness is decided by equality, not by type, and 1 == True is True in Python. Then ask, for a key that is already present, whether an assignment replaces the key or only the value.
Answer 1:
d["a"] = 5 replaces the value under the existing key, so d becomes {"a": 5}. d["b"] = 2 adds a new entry, so d becomes {"a": 5, "b": 2}. One syntax covers both because the operation is "make this key map to this value", and whether that meant creating or replacing is a fact about the dictionary at that moment, not about the line you wrote. {} builds an empty dictionary, so an empty set is written set().
A key must be immutable because the dictionary files each entry by computing a value from the key and finds it later by computing that value again, so a key that changed after insertion would leave its entry filed under something that no longer exists. A value never locates anything, so nothing restricts it, which is why {[1, 2]: "x"} raises TypeError: unhashable type: 'list' while {"a": [1, 2]} is ordinary.
DON'T rely on a duplicate key raising an error, the later value silently wins and a typo can cost you an entry with nothing on screen.
Answer 2:
config["lr"] raises KeyError: 'lr' and stops the program, config.get("lr") returns None with no error, and config.get("lr", 0) returns 0. Use [] when the key must exist and its absence is a bug you want reported immediately, and .get() with an explicit default when absence is normal. The middle form is the one to watch, because None is not an error and not a number, so it travels quietly until something does arithmetic with it and raises TypeError somewhere else.
"epochs" in config is True and 10 in config is False, because in searches keys and 10 is a value. The question it fails to answer is asked as 10 in config.values(), which is True. Both in and a bare for loop default to keys, and the view methods redirect them to the other half.
DON'T treat .get() without a default as a safe version of [], it only moves the failure.
Answer 3:
{"a", "b", "a"} contains "a" and "b" and its length is 2. It destroys multiplicity, so it can no longer report that "a" appeared twice, and order, so it cannot report which came first. Both are gone for good, so a set is the wrong tool the moment either is part of the question.
train & test is {"b", "c"}, train | test is {"a", "b", "c", "d"}, train - test is {"a"} and test - train is {"d"}. Only the difference changes its answer when the operands swap. test - train is the out-of-vocabulary question, which values the test split carries that training never saw, and it is usually the one that matters. train - test reports what training covered that the test split never exercises, a coverage gap in the evaluation rather than a defect in the model.
DON'T write the difference in whichever order reads more naturally, decide which of the two questions you are asking first.
Answer 4:
set(flags) prints {0, 1}. len(set(flags)) prints 2. counts prints {1: 3, 0: 3}.
A set decides uniqueness by equality, not by type, and in Python True == 1 and False == 0 are both true, as are 1.0 == 1 and 0.0 == 0. The six values are two distinct values as far as a set is concerned, 1 and 0, each arriving three times in three spellings. The set keeps the one that arrived first, which is why they print as the integers rather than as True and False.
The dictionary tells the same story from the other side. counts[f] = ... looks the key up by equality, so counts[True] finds the entry counts[1] created rather than making a second one, leaving two entries with three counted into each. The keys print as 1 and 0 because assigning to an existing key replaces the value and leaves the key alone, so the stored key is the one from the first insertion. Deduplicating a column that mixes booleans with the integers 0 and 1 therefore merges them silently, and counting it reports two categories where you may have believed there were four, with nothing raised.
DON'T assume a set or a dictionary distinguishes values that merely have different types, if they compare equal they occupy one slot.
Assignment
Assignment: A Frequency Table, a Vocabulary Check, and the Bucket That Never Filled
Build the following in one file, in this order.
Given tokens = ["the", "cat", "the", "dog", "the", "cat", "bird"], build a frequency dictionary with the .get accumulator idiom and print it. Then produce the two most frequent tokens as a list of (token, count) tuples ordered by count descending, and print that.
Then, given data = [("a", 1), ("b", 2), ("a", 3), ("c", 4), ("b", 5)], build a dictionary mapping each category to the list of its values, so "a" maps to [1, 3]. Do it once with an explicit in guard that creates the empty list before appending and once with setdefault, and print both to show they agree. Then write a third version using .get(cat, []) in place of setdefault, print what it produces with a comment stating what each iteration actually did and why nothing raised, and print the length of all three side by side.
Finally, treat tokens as a training split and test = ["the", "cat", "fish", "owl", "the"] as a test split. Print the sorted out-of-vocabulary tokens, the out-of-vocabulary rate as the fraction of distinct test tokens that are unseen, and whether the training vocabulary covers every distinct test token, where a <= b on two sets asks whether every member of a is also in b.
Hints:
sorted(counts.items(), key=lambda kv: kv[1], reverse=True)orders the pairs by their second field, and slicing[:2]takes the first two.setdefault(key, [])differs from.get(key, [])in exactly one respect, and that respect is whether the default it hands back has been put into the dictionary first.
Solution:
tokens = ["the", "cat", "the", "dog", "the", "cat", "bird"]
# One running count per distinct token.
counts = {}
for t in tokens:
counts[t] = counts.get(t, 0) + 1
print(counts) # {'the': 3, 'cat': 2, 'dog': 1, 'bird': 1}
# 'sorted' orders the (token, count) pairs by the second field, highest first.
ranked = sorted(counts.items(), key=lambda kv: kv[1], reverse=True)
print(ranked[:2]) # [('the', 3), ('cat', 2)]data = [("a", 1), ("b", 2), ("a", 3), ("c", 4), ("b", 5)]
# Version 1: explicit guard, create the list before appending into it.
guarded = {}
for cat, val in data:
if cat not in guarded: # first sighting of this category
guarded[cat] = [] # store an empty list under the key
guarded[cat].append(val) # now the key exists, so this reaches the stored list
print(guarded) # {'a': [1, 3], 'b': [2, 5], 'c': [4]}
# Version 2: 'setdefault' stores the default under a missing key, THEN returns it.
via_setdefault = {}
for cat, val in data:
via_setdefault.setdefault(cat, []).append(val)
print(via_setdefault) # {'a': [1, 3], 'b': [2, 5], 'c': [4]}
# Version 3: '.get' returns a default WITHOUT storing it.
via_get = {}
for cat, val in data:
via_get.get(cat, []).append(val)
print(via_get) # {}
# Each iteration: '[]' built a fresh empty list, '.get' found nothing and handed it
# back, '.append' put the value in it, the line ended, and the list was discarded
# with the value inside. Nothing raised because every call did what it promises.
# 3 3 0
print(len(guarded), len(via_setdefault), len(via_get))setdefault stores the default under a missing key first and only then returns whatever is now at that key, so the list it hands back is the dictionary's own list and appending to it appends into the dictionary. .get skips the storing step. That makes it right for the counting loop, where the value handed back is a number assigned back explicitly, and silently useless for the grouping loop, where the value handed back is a container nobody assigns anywhere.
test = ["the", "cat", "fish", "owl", "the"]
train_vocab = set(tokens)
oov = set(test) - train_vocab # in test, never seen in training
oov_rate = len(oov) / len(set(test)) # a fraction of DISTINCT test tokens
print(sorted(oov)) # ['fish', 'owl']
print(oov_rate) # 0.5
print(set(test) <= train_vocab) # False, 'fish' and 'owl' are not in trainingThe word "rate" hides a choice. len(oov) / len(set(test)) is the share of distinct test tokens that are unseen, 0.5 here because two of the four are new. The share of test token occurrences that are unseen is a different number, 2 out of 5, because "the" appears twice and is not out of vocabulary. Neither is more correct, but reporting one and calling it the other is how a coverage number ends up misleading whoever reads it.
DON'T use .get(key, []) as the setup step of a grouping loop, it hands you a list that is not in the dictionary, and don't take the absence of an error as evidence a line did what you meant.
Additional Resources
Optional, and nothing later needs them. Worth opening if you want to see the ready-made tools you reached past today.
- YouTube - "Learn Python - Full Course for Beginners [Tutorial]" by freeCodeCamp.org, the "Dictionaries" chapter.
- Official docs -
collections, theCounteranddefaultdictentries, the ready-made tally and grouping tools.
Summary
The dictionary is the moment data work starts to feel like data work. A list answers questions about position, which is fine when position is what you have, but most real questions are about names: the accuracy of this model, the count of that label, the setting for this hyperparameter. The set is the same idea with the values removed, answering the two questions that are left, is this here and what are the distinct things. Once you have both you start seeing them everywhere, a config, a column lookup, a frequency table, a vocabulary, a leakage check between splits (whether any test row also appears in training).
Two ideas are worth keeping and both are quieter than the syntax. The first is that [] and .get() differ in what they do about absence, which is really a question about what absence means in your program. If a missing key is a bug, let [] raise and tell you. If it is the normal "not yet, or not applicable", let .get() hand back a default and carry on. Choosing wrong does not produce wrong syntax, it produces a program that crashes when it should cope, or copes when it should crash. The second is the multi-bucket accumulator, d[k] = d.get(k, 0) + 1, widened so one dictionary holds a separate running total for every key it meets. That line will appear in your code more times than almost anything else in this section.
Notice too what the previous lessons paid for. Keys and set elements must not change in a way that changes their computed value, and now you know what that sentence costs, a tuple that is immutable all the way down can be a key precisely because it can never change and a list cannot be one for exactly the opposite reason. What you have after three lessons is a small vocabulary of shapes. A list when order and position matter, a tuple when the grouping must not change, a dictionary when each value needs a name, a set when only presence and uniqueness matter. The final lesson of the section puts that judgment front and center, nesting these containers inside one another, compressing the loops that build them into comprehensions, and practicing the one skill that ties the section together, looking at a problem and knowing which container it wants before writing a line.
Cheat Sheet
Part A - Reference Table
| Operation | Syntax | Notes |
|---|---|---|
| Build dict | {"k": v} / {} |
{} is a dict, never a set |
| Read strict | d[k] |
raises KeyError when absent |
| Read safe | d.get(k) / d.get(k, default) |
None / the default when absent |
| Add or update | d[k] = v |
one syntax for both |
| Key test | k in d |
tests keys, never values |
| Value test | v in d.values() |
searches values |
| Pairs | d.items() |
(key, value) tuples to unpack |
| Tally | d[k] = d.get(k, 0) + 1 |
multi-bucket accumulator |
| Group | d.setdefault(k, []).append(v) |
stores under a missing key, then returns, .get would not |
| Build set | {1, 2, 3} / set() |
an empty set is set(), never {} |
| Dedup | set(xs) / sorted(set(xs)) |
unordered, then stable |
| Union | a | b / a.union(b) |
in either |
| Intersection | a & b / a.intersection(b) |
in both |
| Difference | a - b / a.difference(b) |
in a, not b, directional |
| Symmetric diff | a ^ b |
in exactly one |
Part B - Compact Code Block
d = {"a": 1, "b": 2} # key -> value, keys unique
v = d["a"] # strict read, raises if missing
v = d.get("z", 0) # safe read with default
d["c"] = 3 # add or overwrite
for k, val in d.items(): # iterate pairs, unpack
print(k, val)
counts = {} # multi-bucket accumulator
for x in ["a", "b", "a"]:
counts[x] = counts.get(x, 0) + 1
groups = {}
for k, v in [("a", 1), ("a", 2)]:
groups.setdefault(k, []).append(v) # '.get' here would store nothing
s = set([3, 1, 3, 2]) # {1, 2, 3}, NOT {} which is a dict
print(2 in s) # fast membership, no scan
a, b = {1, 2, 3}, {2, 3, 4}
print(a | b, a & b) # {1, 2, 3, 4} {2, 3}
print(a - b, b - a) # {1} {4}, directional
print(a.intersection([2, 9])) # method form takes any iterable