Lesson 4. Dicts and Sets

Mission link: The dictionary is where most Python programs keep their state, and choosing the wrong lookup idiom is how a program acquires silent defaults it was never meant to have.
Primary source: The Python Tutorial, Dictionaries
Prerequisites: Lesson 2, Lesson 3

Warm-up

1. ▢ a = [1]; b = a; a += [2]; print(b). What prints, and what would a = a + [2] have printed instead?

2. ▢ Why can a tuple be a dict key while a list cannot?

Know this

A dict maps keys to values. A set holds unique members. Both are built on hashing, which is why both impose the same requirement on what you put in them.

Keys are hashable, and order is insertion order

counts = {"a": 1, "b": 2}
counts["c"] = 3
print(list(counts))         # ['a', 'b', 'c']

Since Python 3.7 the language guarantees that a dict preserves insertion order. Sets do not, and never have: a set's iteration order is an artefact of hashing, so any code that depends on it is broken even though it will appear to work.

Assigning to an existing key replaces the value and keeps the key's original position.

Four ways to read a key, and they mean different things

config = {"retries": 3}

config["timeout"]                  # KeyError, and that is often correct
config.get("timeout")              # None
config.get("timeout", 10)          # 10, and config is unchanged
config.setdefault("timeout", 10)   # 10, and config now HAS the key

Pick by intention, not by habit:

The trap is get with a default that hides a real error. If a missing key means a misconfigured program, KeyError is a feature: it names the key and stops immediately, which is far better than a None that surfaces three functions later.

Never use config.keys() for a membership test. "k" in config already tests keys, and it is both clearer and faster.

A mutable default needs care

Counting and grouping is the most common dictionary task, and the naive version reads badly:

groups = {}
for word in words:
    if word[0] not in groups:
        groups[word[0]] = []
    groups[word[0]].append(word)

Two better spellings:

from collections import defaultdict

groups = defaultdict(list)
for word in words:
    groups[word[0]].append(word)     # missing keys create a list on access

groups = {}
for word in words:
    groups.setdefault(word[0], []).append(word)

defaultdict is clearer when the whole dictionary works that way. Be aware of its one edge: merely reading a missing key creates it, so a defaultdict grows when you only meant to look.

Views are live, not snapshots

keys(), values() and items() return views onto the dictionary:

config = {"a": 1}
keys = config.keys()
config["b"] = 2
print(list(keys))       # ['a', 'b'], the view updated

A view is a window, not a copy. It also means you cannot add or remove keys while iterating the dictionary, and Python raises RuntimeError: dictionary changed size during iteration rather than producing nonsense. To delete while iterating, iterate a copy of what you need:

for key in list(config):        # list() takes the snapshot
    if config[key] is None:
        del config[key]

Sets, briefly

a = {1, 2, 3}
b = {3, 4}
a | b       # {1, 2, 3, 4}   union
a & b       # {3}            intersection
a - b       # {1, 2}         difference
a ^ b       # {1, 2, 4}      symmetric difference

Membership in a set is fast and does not depend on size, which is the reason to build one: turning a list into a set before a loop of in tests is the single most common honest optimisation in Python. {} is an empty dict, and set() is an empty set.

Practice

1. ▢ Predict each of the four lines, and say whether config changed.

   config = {"retries": 3}
   print(config.get("timeout"))
   print(config.get("timeout", 10))
   print(config.setdefault("timeout", 10))
   print(config)
   

2. ▢ This loop is meant to drop empty values. Predict what happens.

   data = {"a": 1, "b": None, "c": None}
   for key in data:
       if data[key] is None:
           del data[key]
   

3. ▢ Which lookup would you write for each intention? Match the four.

   • Reading a required setting from a parsed config file
   • Reading an optional setting with a sensible fallback
   • Building up a dictionary of lists while grouping records
   • Deciding whether to log "created" or "updated"

4. ▢ Both of these count words. One has a bug that only appears with certain input. Which, and what input?

   # A
   counts = defaultdict(int)
   for w in words:
       counts[w] += 1
   
   # B
   counts = {}
   for w in words:
       counts[w] = counts[w] + 1
   

5. ▢ A function takes a list of user IDs and, for each one, checks membership against a list of 50,000 banned IDs. Name the one-line change that matters, and say what makes it correct rather than merely faster.

Real-world reps

• [ ] Take a counting or grouping loop you have written and rewrite it three ways: with if key not in, with setdefault, and with defaultdict. Decide which you would defend in review, and why.
• [ ] Trigger the RuntimeError deliberately, then fix it with list(). Getting the error once makes it recognisable forever.
• [ ] Tomorrow: find one .get( in code you know where a KeyError would have been the better outcome. They are common, and each one is a bug that has been converted into a mystery.

Going further

• Dictionaries: the tutorial's treatment, including items() in loops
• collections: defaultdict, Counter and the rest, which cover most of what people hand-roll
• Mapping types: every method, and the view objects in detail
• Resources

Not landing? Reread the primary source at the top, since this lesson compresses it and compression is where understanding leaks. Check the glossary for any term that felt slippery.

If the lesson itself is unclear rather than the material, that is a defect: open an issue.