Lesson 10. Exceptions

Mission link: Python expects exceptions to carry ordinary control flow, not just disasters. Code written as though they are disasters ends up checking everything twice and swallowing the one failure that mattered.
Primary source: The Python Language Reference, The try statement
Prerequisites: Lesson 4, Lesson 9

Warm-up

1. ▢ Lesson 4 gave four ways to read a dict key. Which one raises, and which one hides a missing key behind a value that might also be stored?

2. ▢ What does except Exception fail to catch?

Know this

The idiom has a name in the Python documentation: EAFP, easier to ask forgiveness than permission. Try the operation, handle the failure.

# EAFP: one lookup, and the failure is handled where it happens
try:
    port = config["port"]
except KeyError:
    port = 8080

The alternative, LBYL, looks before it leaps:

if "port" in config:                 # two lookups
    port = config["port"]
else:
    port = 8080

LBYL is not wrong, and here config.get("port", 8080) beats both. LBYL becomes wrong when the check and the action can disagree: a file that exists when you check and is gone when you open it, a key another thread removes in between. The check buys nothing and reads as though it bought safety.

try costs almost nothing when no exception is raised. That is a design decision in the language, and it is what makes EAFP affordable.

The four clauses, and the one people skip

try:
    data = fetch(url)
except TimeoutError:
    data = cached()
else:
    cache.store(data)                # ran only if fetch succeeded
finally:
    connection.close()               # runs no matter what

• except handles one class of failure. Several may be listed, and the first matching one wins.
• else runs only when the try block completed without raising. It exists so that code which must not be protected by the except stays out of the try. Without it, a KeyError raised by cache.store would be caught by an except KeyError meant for fetch.
• finally always runs: after success, after a handled exception, after an unhandled one on its way out, and after a return or break in the try.

One sharp edge: a return inside finally discards whatever the rest of the function was doing, including an exception in flight. It is legal and it is almost always a bug.

Catch what you can handle, and nothing else

except Exception:                    # too wide: hides typos and logic errors
except:                              # worse: also swallows Ctrl-C
except (KeyError, ValueError):       # a claim you can defend

except Exception: pass is the single most expensive line in Python code, because the failure it hides is exactly the one nobody expected. Two places where a wide catch is legitimate:

1. A top-level loop that must survive one bad item, and logs the exception with its traceback before continuing.
2. A boundary that converts anything into a domain failure, using raise MyError(...) from exc.

Both keep the information. The bare pass destroys it.

logging.exception("failed to process %s", item_id) inside an except block records the traceback. logging.error(str(exc)) records a sentence, and str on many exceptions is empty.

Re-raising, and chaining

try:
    parse(raw)
except ValueError as exc:
    log.warning("bad payload")
    raise                            # same exception, original traceback

A bare raise inside a handler re-raises what is being handled. raise exc also works and resets nothing important, but the bare form states the intent.

When translating one failure into another, say where it came from:

raise ConfigError(f"bad port in {path}") from exc

from exc sets the cause, and the traceback prints both with "The above exception was the direct cause". Omit it and Python still chains implicitly, with "During handling of the above exception, another exception occurred", which reads as an accident. from None suppresses the chain, for when the inner exception is noise.

Since Python 3.11, exc.add_note("port came from the environment") attaches context to an exception without wrapping it.

Your own exceptions

class StoreError(Exception):
    """Base for everything this module raises."""

class ItemMissing(StoreError):
    pass

One base class per library or package is the whole pattern. It lets a caller write except StoreError and be complete, without knowing the internals, and it lets you add a subclass later without breaking them. Inherit from Exception, not BaseException. Reuse a built-in when it genuinely fits: ValueError for a bad value, TypeError for a wrong type, KeyError and LookupError for a missing entry.

assert is not error handling. Assertions are removed when Python runs with -O, so validation written as assert user.is_admin disappears in exactly the deployment where it mattered. Assert internal invariants; raise for anything a caller or a user can cause.

contextlib.suppress

from contextlib import suppress

with suppress(FileNotFoundError):
    path.unlink()

Exactly equivalent to try/except FileNotFoundError: pass, and better, because the suppression is named and cannot accidentally cover three more statements.

Several failures at once

Since Python 3.11, an ExceptionGroup carries multiple exceptions, and except* handles them by type without discarding the rest:

try:
    run_all_tasks()                  # raises ExceptionGroup
except* TimeoutError as eg:
    ...                              # eg holds only the timeouts
except* ValueError as eg:
    ...

Where this actually shows up is concurrency, and stage 6 uses it. Know now that a plain except TimeoutError does not catch a TimeoutError inside a group, which is the surprise.

Practice

1. ▢ Predict the output.

   def f():
       try:
           return "try"
       finally:
           print("finally")
   
   print(f())
   

finally
try

2. ▢ What is wrong with this, given that cache.store can raise KeyError?

   try:
       data = fetch(url)
       cache.store(data)
   except KeyError:
       data = {}
   

try:
    data = fetch(url)
except KeyError:
    data = {}
else:
    cache.store(data)

3. ▢ Rank these four handlers from most to least defensible, and say what each hides.

   • a) except Exception: pass
   • b) except Exception as exc: log.exception("item %s failed", item.id)
   • c) except (KeyError, ValueError) as exc: raise ItemInvalid(item.id) from exc
   • d) except: pass

4. ▢ Rewrite as EAFP, and say what the original version's real defect is.

   if os.path.exists(path):
       with open(path) as f:
           return f.read()
   return None
   

try:
    with open(path) as f:
        return f.read()
except FileNotFoundError:
    return None

5. ▢ This library raises KeyError from its public API. Why is that a design problem, and what should it raise?

   def get_user(user_id):
       return _users[user_id]
   

class UserMissing(StoreError):
    pass

def get_user(user_id):
    try:
        return _users[user_id]
    except KeyError as exc:
        raise UserMissing(user_id) from exc

6. ▢ A service validates input with assert amount > 0, "amount must be positive". Give the failure mode.

if amount <= 0:
    raise ValueError(f"amount must be positive, got {amount}")

Real-world reps

• [ ] Search code you own for except Exception and bare except. For each, write one sentence naming what it is meant to catch. Any handler you cannot write that sentence for is a defect, and narrowing it is the fix.
• [ ] Find a try block with more than one statement in it, and check whether every statement should be covered by the handler. Move the ones that should not into an else.
• [ ] Add a single base exception class to a module you own and make everything it raises inherit from it. Then simplify one caller.
• [ ] Tomorrow: replace one try/except X: pass with contextlib.suppress(X) and notice how much narrower the suppressed region became.

Going further

• The try statement: the exact rules for else, finally, and return inside them
• Errors and Exceptions, in the tutorial: the guided version, including chaining
• Built-in Exceptions: the hierarchy, and which built-in to reuse instead of inventing one
• PEP 654, Exception Groups and except*: the multiple-failure model, needed in stage 6
• contextlib.suppress: the named form of a narrow catch
• Resources

Not landing? Reread the primary source at the top, since this lesson compresses it and compression is where understanding leaks. Check the glossary for any term that felt slippery.

If the lesson itself is unclear rather than the material, that is a defect: open an issue.