Lesson 18. Contract Testing

Mission link: Lesson 16 made the contract a machine-readable artifact; lesson 17 checked whether a change to that artifact was backward-compatible. Neither one runs the provider's actual code against a consumer's actual expectations. Contract testing is the check that does: not "is the schema well-formed" or "did the schema change safely," but "does the provider genuinely behave the way this specific consumer needs it to."
Primary source: Docs: "Consumer Driven Contracts", Pact
Prerequisites: Lesson 17, Contract

Warm-up

1. ▢ What question does a style linter like Spectral answer, and what question does a breaking-change detector like buf breaking or oasdiff breaking answer instead?

2. ▢ Why can't a client SDK generated from a shared .proto file drift out of sync with the server the way a hand-written client library can?

Know this

A schema can be well-formed, backward-compatible, and still wrong for a real consumer

Neither an OpenAPI document's structural validity nor a clean buf breaking run says anything about whether the provider's actual runtime behavior matches what a specific consumer needs. A field can be present, typed correctly, and unchanged between versions, and the provider can still return the wrong value, omit a case a consumer's code depends on, or behave differently from what the consumer's tests assume. Both of the previous lessons' checks operate on the description of the contract; contract testing is the check that runs against the provider's actual behavior.

A consumer-driven contract is generated from real usage, not written down separately

Pact's model inverts the usual order: instead of a provider publishing a schema and hoping consumers use it correctly, each consumer writes tests against a mock of the provider, and those tests themselves generate a pact file, a record of the exact interactions (requests and expected responses) that consumer actually relies on. The provider then verifies its real implementation against every consumer's published pact. Because the contract is derived from actual consumer usage rather than declared independently, only the interactions a consumer genuinely depends on are tested; provider behavior no consumer uses can change freely without breaking any contract test, since no contract claims it in the first place.

The check that decides whether it's safe to deploy, without needing a shared staging environment

A pact broker is where consumers publish their pacts and providers publish their verification results. The can-i-deploy check queries the broker for a specific pacticipant version against a target environment, and answers a concrete question: has every consumer's contract already been verified against this exact provider version (or vice versa)? A "yes" means the two sides are known-compatible from recorded verification results, without spinning up a full end-to-end environment with every service running simultaneously to find out. This is the same shift lesson 16 and 17 made for the contract's shape, applied to actual behavior: a question that used to require a live, synchronized staging environment becomes a query against results already on record.

Contract by example, not contract by declaration

A schema-based artifact (lesson 16) declares a shape once, in the abstract; a contract test instead accumulates a set of concrete examples, this exact request produced that exact response, and verifies the provider against every one of them. This is why Pact's own documentation calls it "contract by example": the contract isn't one static declaration checked for internal consistency, it's a growing set of real interactions checked for continued truth. The tradeoff is coverage: a contract test only knows about interactions some consumer's tests actually exercised, while a schema declares every field whether or not anything uses it yet.

Practice

1. ▢ A provider's OpenAPI document is valid, and buf breaking reports no incompatibility against the previous version. A consumer's integration still breaks in production. What kind of check would this lesson say could have caught it that the previous two lessons' checks wouldn't?

2. ▢ In Pact's model, who writes the contract, and what does that determine about which interactions get tested?

3. ▢ A team wants to know whether it's safe to deploy a new provider version without spinning up every consumer service in a shared staging environment. What does can-i-deploy check instead?

4. ▢ Why does Pact's documentation describe its model as "contract by example" rather than "contract by declaration"?

5. ▢ Which claim correctly distinguishes contract testing from the schema-based checks in the previous two lessons?

   • a) Contract testing is just another name for running a breaking-change detector against an OpenAPI document
   • b) Contract testing verifies a provider's actual runtime behavior against a specific consumer's recorded expectations, a check neither a schema's validity nor its backward-compatibility with a previous version can perform
   • c) A pact file must be written by the provider before any consumer can use the API
   • d) can-i-deploy requires a full shared staging environment to determine deployment safety

Real-world reps

• [ ] For a service you work on that has multiple consumers, check whether compatibility between it and its consumers is verified by contract tests, end-to-end tests in a shared environment, or neither.
• [ ] If contract tests exist, find one pact file or equivalent and note whether it covers every field your team assumed was part of the "real" contract, or only what a consumer's own tests happened to exercise.
• [ ] Tomorrow: read the primary source's section on provider states, and note what problem they solve for a provider that needs specific data set up before it can honor a consumer's recorded interaction.

Going further

• Docs: "Consumer Driven Contracts", Pact
• Docs: "5 Minute Getting Started Guide", Pact
• Resources

Not landing? Reread the primary source at the top, since this lesson compresses it and compression is where understanding leaks. Check the glossary for any term that felt slippery.

If the lesson itself is unclear rather than the material, that is a defect: open an issue.