Lesson 13. LINQ

Mission link: The mission names querying and transforming data fluently with LINQ. This lesson is LINQ itself: where the operators come from, what the two syntaxes have in common, and when the query actually runs.
Primary source: Docs: "Language Integrated Query (LINQ)", Microsoft Learn
Prerequisites: Lesson 12, Lesson 5, Lesson 4

Warm-up

▢ A type declares Process(int) and an extension method with the same signature exists. Which is called?

Check

The type's own member, always. Extension members have lower priority than declared members, the compiler searches the type first, and an extension with a matching signature is never called.

▢ A Span<int> can be used in a foreach but LINQ methods do not appear on it. Why?

Check

foreach needs only a shape, a public parameterless GetEnumerator returning a type with Current and MoveNext. LINQ needs the interface, because its operators are defined over IEnumerable<T>, which Span<T> does not implement. Iteration is duck-typed; querying is interface-bound.

▢ Why is IEnumerable<T> covariant while IList<T> is not?

Check

A covariant parameter may appear only in output positions. IEnumerable<T> only produces a T; IList<T> has Add(T), which is an input position, so it must stay invariant.

Know this

Start with where Where comes from, because it answers three earlier lessons at once. IEnumerable<T> does not have a Where method. The standard query operators, Where, Select, SelectMany, Join, GroupBy, Max, Average and the rest, are extension members over it (Write LINQ queries). So lesson 4's rule that any IEnumerable<T> is a queryable type, and lesson 12's mechanism for attaching members to a type you do not own, are the same fact seen from two sides, and this is where they meet at a call site.

Two syntaxes, one meaning. A query can be written declaratively as a query expression or as a chain of method calls, and the relationship between them is not a matter of taste at the language level:

At compile time the compiler converts query expressions into standard query operator method calls, following rules in the C# specification.
Any query expressible in query syntax is expressible in method syntax.
There is no semantic or performance difference between the two forms (LINQ).

What differs is readability, in both directions, and reach. Some operations have no query expression clause at all and must be written as method calls, Count and Max among them, and the two syntaxes can be mixed in one query.

The shape of a query expression is fixed at both ends: it must begin with a from clause and must end with a select or a group clause. Between them it may carry where, orderby, join, let, and further from clauses. The into keyword makes the result of a join or group the source for more clauses in the same expression, which is how you filter or sort the groups after grouping (Query expression basics).

A query expression is a first-class language construct, usable anywhere a C# expression is valid, and its variables are all strongly typed. That is worth noticing rather than skimming: this is not a string handed to a library, it is code the compiler checks.

Nothing runs until you iterate. The rule, in the documentation's words: a query is not executed until you iterate over the query variable, for example in a foreach. Three consequences follow, and the second is the one that bites:

The query variable holds the query, not the results. Building it is nearly free.
Iterating it twice executes it twice, so a source that changed in between yields different results the second time, and any work in the query happens again.
The operators that cannot be written in query syntax, Count, Max, ToList and their relatives, are exactly the ones that force execution, because they must produce a single value or a materialised collection. When you want the results fixed, that is how you fix them.

And now the part that makes LINQ more than a collections API. A query expression can be compiled to a delegate or to an expression tree, depending on the type being queried. Against an in-memory sequence it becomes delegates: your lambda is compiled code that runs per element. Against a queryable data source it becomes an expression tree, a data structure describing the query, which a provider can inspect and translate, into SQL for a database, for instance. That is why a single query can retrieve data from a SQL database and produce an XML stream as output, and why the same Where you write over a List can also be sent to a server.

flowchart TD A["a LINQ lambda,
e.g. Where(x => ...)"] --> B{"queried source?"} B -- "in-memory sequence" --> C["compiled to a delegate:
ordinary code, runs per element"] B -- "queryable data source" --> D["compiled to an expression tree:
a data structure a provider translates"] D --> E["e.g. translated into SQL"]

The practical rule that follows is worth holding now and will matter again in stage 6: what you may put in a lambda depends on which of the two it becomes. Code that runs locally can do anything; code that has to be translated can only do what the provider understands.

On Java, briefly. The mission puts the LINQ-against-Stream-API comparison in stage 7, so it is not this lesson's job. The one structural difference worth naming now is that LINQ has syntax the compiler translates, and a form that compiles into a data structure rather than into code, which is what allows a provider to rewrite the query. A stream pipeline is always compiled code.

Practice

▢ The same query is written both ways: from num in numbers where num % 2 == 0 orderby num select num, and numbers.Where(num => num % 2 == 0).OrderBy(n => n). What is the type of each query variable, and is either faster?

Check

Both are IEnumerable<int>, and neither is faster. The compiler converts the query expression into exactly those standard query operator calls, so the two forms are semantically identical with no performance difference. Choosing between them is a readability decision, and the documentation is explicit that either can be the more readable one depending on the query.

Worth noticing what this rules out: there is no runtime translation step to pay for, and no "query syntax is the slow declarative one" trade to reason about. The translation happens at compile time.

▢ var evens = from n in numbers where n % 2 == 0 select n; runs. Then numbers.Add(42);. Then foreach (var n in evens). Is 42 in the output?

Hint

Ask what the first line actually did, and when the where predicate has run.

Check

Yes, 42 is included. The first line built a query and executed nothing; the predicate first runs during the foreach, by which time the source contains 42. A query is not executed until you iterate over the query variable.

The general shape to internalise: the query variable is a question, not an answer. Iterate it twice and it is asked twice, so a query over a changing source is not a snapshot, and any expensive work inside it is repeated. When you need an answer rather than a question, force execution with ToList or ToArray and pass that around instead.

▢ Why can you not write Count as a query expression clause, and what does that tell you about the two syntaxes?

Check

Because no such clause exists: a query expression must end with select or group, and both of those produce a sequence, while Count produces a single number. Operations of that shape, Count and Max among them, have no query expression equivalent and must be method calls.

What it tells you is that method syntax is the complete surface and query syntax is a convenient subset of it. Anything you can write as a query expression you can write as method calls; the reverse is not true. Mixing them is normal and supported, which is why (from ... select ...).Count() is idiomatic rather than a compromise.

▢ The same Where call appears in two places: over a List<Order> in memory, and over a database-backed queryable source. What is different about what the compiler produces, and why does it matter to what you write in the lambda?

Check

Over the list, the lambda is compiled to a delegate: ordinary code that the operator invokes once per element, locally. Over the queryable source it is compiled to an expression tree, a data structure describing the query, which the provider inspects and translates, typically into SQL.

That is why the lambda's contents are constrained in the second case and not the first. Local code may call anything you like; a translated query can only contain what the provider knows how to express, and anything else either fails or silently pulls the data down to be filtered in memory. The habit worth forming: know which of the two a given source gives you before writing anything clever in a predicate.

▢ Which claim about query syntax and method syntax is correct?
- a) Method syntax runs faster, because query syntax is translated at run time
- b) The two syntaxes are semantically identical, with no performance difference between them
- c) Query syntax is strictly more expressive, since every operator has a clause
- d) The two differ in timing: query syntax defers and method syntax does not

Check

b) The documentation states it directly: no semantic or performance difference, because the conversion happens at compile time. (a) puts the translation at run time, where it is not. (c) inverts the containment: method syntax is the full surface, and operators such as Count have no clause at all. (d) invents a difference, since deferred execution is a property of the query operators themselves and applies identically to both forms.

Real-world reps

[ ] Find a LINQ query in C# you have access to whose result is stored in a variable and used more than once. Work out how many times it executes.
[ ] Find a query written in the syntax you use less often, and rewrite it in the other. Decide which one you would rather read in six months, and whether the answer depends on the query.
[ ] Tomorrow: find a query against a database-backed source and check every lambda in it for a call the provider would have to translate. Note what you would do if one of them could not be.

Going further

Not landing? Reread the primary source at the top, since this lesson compresses it and compression is where understanding leaks. Check the glossary for any term that felt slippery.

If the lesson itself is unclear rather than the material, that is a defect: open an issue.