Lesson 40. Designing an API

Mission link: Everything in the arc so far was about making code correct. This stage is about the part you cannot change later: what callers can see, what they will depend on, and what you have promised without noticing.
Primary source: PEP 8, Designing for Inheritance and Public Interfaces
Prerequisites: Lesson 15, Lesson 20, Lesson 24

Warm-up

1. ▢ Lesson 15 gave a rule about parameter and return types. State it.

2. ▢ Lesson 12 asked which shape a bundle of data deserves. Which shape can gain a field without breaking callers?

Know this

The signature is the contract

def send(
    message: str,
    *,
    retries: int = 3,
    timeout: float = 5.0,
    on_failure: Callable[[Exception], None] | None = None,
) -> Receipt:

Five decisions are visible there, and each one is a promise.

Keyword-only after the first one or two. * in the signature forces the rest to be named at the call site, which makes send(msg, 3, 5.0) impossible to write and impossible to misread. It also means you can reorder, rename with a shim, or insert a parameter without breaking anyone. Positional parameters are a promise about order, and order is the hardest thing to change.

Positional-only where the name is noise. def get(self, key, /, default=None) says the first parameter's name is not part of the API, so you may rename it. Use it for the one or two arguments whose meaning is obvious from position.

Defaults that are values, not behaviour. timeout: float = 5.0 is a documented default. timeout: float | None = None meaning "use the global config" is a second code path hidden in a default.

An annotated return. -> Receipt tells callers what they can do next, and it is verified, which a docstring is not.

No boolean parameters that select behaviour. send(msg, validate=True) is two functions in one signature; two functions with names are clearer, and the checker can then type each properly.

Everything observable becomes an interface

The uncomfortable rule, usually attributed to Hyrum Wright: with enough users, every observable behaviour of your system will be depended on by somebody, regardless of what you documented.

Things people depend on that are not in the signature:

You cannot prevent all of it. You can decide deliberately about the top four, because they are cheap to get right and expensive to change:

• Exceptions are part of the API. Document what you raise, give your package one base exception from lesson 10, and translate anything from a dependency, or the dependency's exception type becomes yours.
• Order is part of the API if you return a list. If it is not meant to be, say so in the docstring, and consider returning a set or frozenset so the promise is visible in the type.
• Private means underscore, and means it. A published attribute without one is public whatever the docs say.
• A stable repr is worth writing once, per lesson 24, because it lands in other people's logs.

What to expose

# shop/__init__.py
from shop.orders import Order, OrderError, place_order

__all__ = ["Order", "OrderError", "place_order"]

The package's public surface is the smallest set that lets callers do the job. Everything else is an underscore, an internal module, or absent. Two arguments for keeping it small: every public name is a thing you cannot rename, and a small surface is documentable in one screen, which is the difference between a library people use correctly and one they cargo-cult.

__all__ states the intention for import * and for tooling. It is not enforcement, and combined with no_implicit_reexport from the strict checker settings in lesson 16, it becomes something a tool can check.

The shapes that age well

The last one is the largest in practice. A module-level client, connection, or config object cannot be configured twice, cannot be replaced in a test without patching, and ties its lifetime to the process. Handing the caller a constructor costs one line and removes all three problems, which is lesson 31's argument arriving as a design rule.

Documenting the contract

def place_order(items: Sequence[Item], *, customer: CustomerId) -> Order:
    """Create and persist an order.

    Args:
        items: at least one item; quantities must be positive.
        customer: must exist, or CustomerMissing is raised.

    Returns:
        The persisted order, with `id` assigned.

    Raises:
        EmptyOrder: if `items` is empty.
        CustomerMissing: if `customer` does not exist.
    """

The types are in the annotations, so the docstring carries what annotations cannot: preconditions, what is raised and when, side effects, and whether the result is ordered. A docstring that repeats the types is maintenance with no benefit.

doctest is worth knowing here: an example in a docstring that is actually executed cannot drift, and it doubles as the first test.

Practice

1. ▢ Rewrite this signature, and name each change.

   def export(data, True, "csv", None, 100):
       ...
   # def export(data, headers, format, path, chunk):
   

def export(
    data: Iterable[Row],
    *,
    include_headers: bool = True,
    format: Literal["csv", "json"] = "csv",
    path: Path | None = None,
    chunk_size: int = 100,
) -> int:

2. ▢ Which of these can you change in a patch release without breaking callers?

   • a) The name of the first positional parameter
   • b) Adding a keyword-only parameter with a default
   • c) The exception type raised on invalid input
   • d) The order of a returned list
   • e) Adding a field to a returned frozen dataclass

3. ▢ Why is a module-level client worse than a constructor?

   # shop/api.py
   client = HttpClient(base_url=os.environ["API_URL"], timeout=5)
   
   def fetch(order_id): return client.get(f"/orders/{order_id}")
   

class ShopApi:
    def __init__(self, client: HttpClient) -> None:
        self._client = client

    def fetch(self, order_id: OrderId) -> Order: ...

4. ▢ A function returns None when the order is not found. Give two better designs and when each applies.

5. ▢ A library raises KeyError from its public function, which internally is a dict lookup. Two years later the storage becomes a database. What breaks, and what should have been done?

class StoreError(Exception): ...
class OrderMissing(StoreError): ...

def get_order(order_id):
    try:
        return _orders[order_id]
    except KeyError as exc:
        raise OrderMissing(order_id) from exc

6. ▢ A colleague argues that keyword-only parameters are noise, since the team knows the argument order. Answer them.

Real-world reps

• [ ] Take the widest signature in code you own and count how many parameters could be keyword-only without breaking a caller. Change those.
• [ ] List every exception your package's public functions can raise, including from dependencies. Anything not one of yours is a leak.
• [ ] Write out your package's public surface from __all__ and ask whether each name has to be public.
• [ ] Find a module-level client, connection or config object and try turning it into a constructor argument. Note how many tests stop patching.
• [ ] Tomorrow: write the docstring for one public function stating what it raises, and see whether you can answer that question without reading the body.

Going further

• PEP 8, Designing for Inheritance and Public Interfaces: the naming and privacy conventions Python reviewers cite
• PEP 570, Positional-Only Parameters: why / exists and when to use it
• PEP 3102, Keyword-Only Arguments: the rationale for * in a signature
• PEP 257, Docstring Conventions: the shape of a docstring
• doctest: examples in a docstring that cannot drift
• Resources

Not landing? Reread the primary source at the top, since this lesson compresses it and compression is where understanding leaks. Check the glossary for any term that felt slippery.

If the lesson itself is unclear rather than the material, that is a defect: open an issue.