chapter 23 - introduction to linq

Introduction to LINQ 1C# 30

C# 3.0C# 3.0

Chapter 23 – Introduction to LINQ

© Copyright SELA Software & Education Labs Ltd. 14-18 Baruch Hirsch St. Bnei Brak 51202 Israel

Why LINQ?Introduction to LINQ 2C# 30

Why LINQ?

Data ! Objects• Data != Objects

• State-of-the-art tooling has perfected OO methodologies

• The next challenge:• The next challenge:

– Accessing and integrating non-OO ccess g a d teg at g o OOinformation, while abstracting away its sourceits source


LINQIntroduction to LINQ 3C# 30

LINQ

LINQ is po ered b lang age• LINQ is powered by language enhancements– Some of which we have already seen (C# 3.0)

– Some others we will examine in this chapter– Some others we will examine in this chapter

• LINQ is powered by a framework– LINQ to Objects – the Enumerable extension

methods

– LINQ to XML

– LINQ to SQL LINQ to EntitiesLINQ to SQL, LINQ to Entities

– Parallel LINQ (PLINQ) and other future technologies

W d th LINQ i C# 3 0


– We recommend the LINQ via C# 3.0 course

Implicit TypingIntroduction to LINQ 4C# 30

Implicit Typing

Local ariables can ha e an implicit t pe if• Local variables can have an implicit type if initialized within the declaration statement

var i = 5;var s = “Hello”;var e = new Employee();var dict = new Dictionary<string,List<string>>();var dict = new Dictionary<string,List<string>>();

var a; //Illegal!


Implicit Typing – NotesIntroduction to LINQ 5C# 30

Implicit Typing – Notes

This is not VB6 Dim• This is not VB6 Dim

• This is not VB6 VariantThis is not VB6 Variant

• This is not System.Object

Th i bl i t l t d• The variable is strongly typed– But you don’t have to specify the typey p y yp

The variable must be initialized• The variable must be initialized– null is not a valid initializer (what type?)


Anonymous TypesIntroduction to LINQ 6C# 30

Anonymous Types

Remo es the b rden of declaring “data• Removes the burden of declaring “data-holder” typesyp– Narrows the Data Objects gap

i {var pi = new {Name = “John”,Pay = CalcSalary()Pay CalcSalary()

};


Anonymous From ExistingIntroduction to LINQ 7C# 30

Anonymous From Existing

Anon mo s t pes can be generated sing• Anonymous types can be generated using existing types’ properties:g yp p p

Person person = ...;var partialDetails new {var partialDetails = new {person.Name, person.Age

};};//Equivalent to:var partialDetails = new {Name person Name Age person AgeName = person.Name, Age = person.Age

};


Anonymous Types – NotesIntroduction to LINQ 8C# 30

Anonymous Types – Notes

The generated t pe is imm table• The generated type is immutable– Typical for data-holders, and thread-safe

• The generated type is genericTh i t b d– The same generic type can be reused:

var x = new { Name = “A”, Pay = 15};

var y = new { Name = 5, Pay = 20};

• Provides value semantics• Provides value semantics– Equals compares contents (like value types)

– GetHashCode depends on contents (like value types)


Expression TreesIntroduction to LINQ 9C# 30

Expression Trees

An e pression lambda can be capt red as• An expression lambda can be captured as an expression treep– Code = Data

• The expression tree can be analyzed and executed at runtimeE i F i t i t *Expression<Func<int,int>> square = x => x*x;

Func<int,int> compiled = square.Compile();, p q p ();Console.WriteLine(compiled(5));


The Expression TreeIntroduction to LINQ 10C# 30

The Expression Tree

At r ntime the tree appears as follo s• At runtime, the tree appears as follows:

Expression<Func<int,int,int>>

BinaryExpressionBinaryExpressionExpressionType=Multiply

Type=Int32Type=Int32

ParameterExpressionName=x, Type=Int32

ParameterExpressionName=y, Type=Int32


Expression Trees – NotesIntroduction to LINQ 11C# 30

Expression Trees – Notes

E pression trees t rn code into data• Expression trees turn code into data

• They can be used to dynamically analyze expressions and provide different behavior– LINQ Providers use this extensivelyLINQ Providers use this extensively

E i T Vi li (MSDN• Expression Tree Visualizer (MSDN sample)sample)– http://tinyurl.com/ExpTreeVis


Review So FarIntroduction to LINQ 12C# 30

Review So Far

Implicit t ping• Implicit typing

• Anonymous types

E i t• Expression trees


What Are Query Operators?Introduction to LINQ 13C# 30

What Are Query Operators?

Q er operators are t picall e tension• Query operators are typically extension methods that operate on a sequence of p qelements

• Some query operators return a sequenceSome query operators return a sequence of elements, some return a single value


Why Query Operators?Introduction to LINQ 14C# 30

Why Query Operators?

Q er operators facilitate declarati e• Query operators facilitate declarative programming instead of imperative p g g pprogramming

List<int> sortedPrimes = new List<int>();foreach (int i in numbers) {if (I P i (i)) t dP i Add(i)if (IsPrime(i)) sortedPrimes.Add(i);

}sortedPrimes.Sort();

Becomes

sortedPrimes.Sort();sortedPrimes.ForEach(Console.WriteLine);//Becomes:

b Wh (i I P i (i)) O d B (i i)numbers.Where(i => IsPrime(i)).OrderBy(i => i).ToList().ForEach(Console.WriteLine);


Developing Query OperatorsIntroduction to LINQ 15C# 30

Developing Query Operators

Most q er operators are er• Most query operators are very straightforwardg

• Let’s develop some query operators that operate on IEnumerable<T>operate on IEnumerable<T>– Where (filtering a sequence)

d ( d i )– Order (ordering a sequence)

– Select (projecting a sequence to another sequence)

– Count (aggregating a sequence)

– First (first element)


First (first element)

Implementing Where


Implementing Wherestatic class EnumerableExtensions {static class EnumerableExtensions {public static IEnumerable<T> Where<T>(

this IEnumerable<T> source,Predicate<T> filter)

{foreach (T elem in source)foreach (T elem in source)

if (filter(elem))yield return elem;

ar here is}}

var here is IEnumerable<int>

var numbers = Range(0, 100);var evens = numbers.Where(i => i%2==0);


OrderIntroduction to LINQ 17C# 30

Order

To order a seq ence e m st iterate it• To order a sequence, we must iterate it first… (And rely on a sorting container)( y g )

public static IEnumerable<T> Order<T>(this IEnumerable<T> source) {

SortedDictionary<T, bool> elements =new SortedDictionary<T, bool>();new SortedDictionary<T, bool>();

foreach (T elem in source)elements.Add(elem, default(bool));

foreach (T key in elements.Keys)yield return key;

}}


SelectIntroduction to LINQ 18C# 30

Select

Projection req ires a con erter• Projection requires a converter:

//Defined in System Core dll//Defined in System.Core.dllpublic delegate TResult Func<T,TResult>(T arg);

public static IEnumerable<S> Select<T,S>(this IEnumerable<T> source,Func<T S> projection) {Func<T,S> projection) {

foreach (T elem in source)yield return projection(elem);

}


Count FirstIntroduction to LINQ 19C# 30

Count, Firstpublic static long Count<T>(p g (

this IEnumerable<T> source) {long count = 0;ar en merator so r e GetEn merator()var enumerator = source.GetEnumerator();

while (enumerator.MoveNext())++count;;

return count;}

bli t ti T Fi t<T>(public static T First<T>(this IEnumerable<T> source) {

var enumerator = source.GetEnumerator();();if (enumerator.MoveNext())

return enumerator.Current;//Th E t //Throw: Empty source

}


Our Operators In ActionIntroduction to LINQ 20C# 30

Our Operators In Action

This gi es s some room for• This gives us some room for experimentation:p

var numbers = Range(0, 100);var query numbers Where(i > i % 2 0);var query = numbers.Where(i => i % 2 == 0);

Console.WriteLine("{0} numbers, first: {1}",( { } , { } ,query.Count(), query.First());

foreach (int i in query)foreach (int i in query)Console.WriteLine(i);

Employee[] employees = ...;var sortedEmpIds = employees.Select(e => e.Id).Order();


Deferred ExecutionIntroduction to LINQ 21C# 30

Deferred Execution

Statements s ch as• Statements such asvar numbers = Range(0, 100);

provide deferred execution semantics

var evens = numbers.Where(i => i%2==0);

provide deferred execution semantics

• The enumeration is not executed yet!– foreach forces execution

– Our Order operator forces execution

– Our Count operator forces execution


Deferred ExecutionIntroduction to LINQ 22C# 30

Deferred Execution

This infers interesting conseq ences• This infers interesting consequences:– We can produce infinite streams:

IEnumerable<int> Primes(int start) {for (; ; ++start)

if (IsPrime(start))yield return start;

}

– We can extract M out of N elements without touching th t

}

the rest

– We can throw the query away without executing it


Review So FarIntroduction to LINQ 23C# 30

Review So Far

Q er operators as declarati e• Query operators as declarative programmingp g g

• Our hand-rolled query operators

• Deferred executione e ed e ecu o


Standard LINQ Query OperatorsIntroduction to LINQ 24C# 30

Standard LINQ Query Operators

LINQ (in S stem Core dll) pro ides• LINQ (in System.Core.dll) provides standard query operators for q y pIEnumerable<T>– Filtering: Where Skip Take Distinct OfType– Filtering: Where, Skip, Take, Distinct, OfType

– Aggregation: Sum, Max, Min, Count, First, FirstOrDefault, Single, SingleOrDefault, Last, LastOrDefault, All, Any, g , g , , , , y,Aggregate, Contains

– Ordering: OrderBy, OrderByDescending, ThenBy, ThenByDescending

– Grouping: GroupBy

P j ti S l t R t S l tM T A T Li t– Projection: Select, Repeat, SelectMany, ToArray, ToList, ToLookup, ToDictionary, Cast

– Miscellaneous: Reverse Union Intersect Join Range Concat


– Miscellaneous: Reverse, Union, Intersect, Join, Range, Concat

Standard LINQ Query OperatorsIntroduction to LINQ 25C# 30

Standard LINQ Query Operators

LINQ q er operators are e tension• LINQ query operators are extension methods in the static Enumerable class

• Can be combined with our own!– As long as there’s no name collisionAs long as there s no name collision

N S f h di d LINQ• Note: So far we have discussed LINQ to Objects query operatorsObjects query operators– Other LINQ provides do not work with IEnumerable


Examples Of LINQ OperatorsIntroduction to LINQ 26C# 30

Examples Of LINQ Operators

var q = Enumerable.Range(0, 100).Select(i => new { Id=i, Name=i.ToString() }).Where(e => e.Name.StartsWith("1")).Where(e => e.Name.StartsWith( 1 )).OrderByDescending(e => e.Id).GroupBy(e => e.Name.Substring(e.Name.Length‐1)).Select(grp => grp);

foreach (var group in q) {foreach (var group in q) {Console.WriteLine("Key: " + group.Key);Console.WriteLine("Elements: " +group.Aggregate("",(a,e) => a+=e.Name+" "));

}


This Looks FamiliarIntroduction to LINQ 27C# 30

This Looks Familiar

The SQL resemblance is ob io s• The SQL resemblance is obvious:SELECT product name, COUNT(*)S C p oduct_ a e, COU ( )FROM ordersWHERE orders.date > ...GROUP BY d tGROUP BY product_name

ordersorders.Where(o => o.Date > ...).GroupBy(o => o.ProductName)S l t( {.Select(grp => new {ProductName = grp.Key,Count = grp.Count()Count grp.Count()

});


Language Query OperatorsIntroduction to LINQ 28C# 30

Language Query Operators

Lang age q er operators are lang age• Language query operators are language keywords that are mapped to query y pp q yoperators

Not all query operators are accessible via keywords– Not all query operators are accessible via keywords

• C# integrated query operators:g q y p– Projection: select

– Filtering: whereFiltering: where

– Grouping: group by

O d i d b– Ordering: orderby

– Miscellaneous: join, let


Language Query OperatorsIntroduction to LINQ 29C# 30

Language Query Operatorsvar q2 =var q2 =from i in Enumerable.Range(0, 100)select new { Id=i, Name=i.ToString() };

var q3 =from e in q2where e Name StartsWith("1")where e.Name.StartsWith( 1 )orderby e.Id descendinggroup e by e.Name.Substring(e.Name.Length‐1)

into gselect g;

foreach (var group in q3) ...var here is

bl

here is

IEnumerable<Anon>


var here isIGrouping<string,Anon>

Syntactic SugarIntroduction to LINQ 30C# 30

Syntactic Sugar

Lang age q er operators translate• Language query operators translate directly to LINQ query operatorsy q y p

• Query operators can be mixed:

int cnt =(from i in Enumerable.Range(0, 100)where i % 2 == 0where i % 2 == 0select i).Count(i => i > 25);


Advanced Example: JoinsIntroduction to LINQ 31C# 30

Advanced Example: Joins

Most SQL like joins are nnecessar• Most SQL-like joins are unnecessary because objects are often denormalizedj

var largeOrders =from cust in customersfrom order in cust.Orderswhere order.Amount > THRESHOLDwhere order.Amount > THRESHOLDselect new { cust.Name, order.Amount };

var largeOrders =from order in orderswhere order.Amount > THRESHOLDwhere order.Amount > THRESHOLDjoin cust in customers

on order.CustomerId equals cust.Id


select new { cust.Name, order.Amount };

Advanced Example: LetIntroduction to LINQ 32C# 30

Advanced Example: Let

let is a helper keyword for complex• let is a helper keyword for complex queries:q

var query =from c in customerswhere c.Joined.AddYears(1) < DateTime.Nowlet orders = c.Orders.ToList()let orders = c.Orders.ToList()select new {Name = c.Name,OrderQuantity = orders.Sum(o=>o.Amount)OrderCount = orders.Count

};};


LINQ to ObjectsIntroduction to LINQ 33C# 30

LINQ to Objects

LINQ to Objects relies on the En merable• LINQ to Objects relies on the Enumerableclass, which contains query operators as q y pextension methods

• Any IEnumerable<T> can be queriedAny IEnumerable<T> can be queried

• Additional types can be queried if they implement the query patternimplement the query pattern


Query PatternIntroduction to LINQ 34C# 30

Query Pattern

If the type is not IEn merable there are• If the type is not IEnumerable, there are two ways to make it queryable:y q y

– Add instance methods for query operators: Select<T>, Where, OrderBy, etc.

– Add extension methods in a separate class for query p q yoperators: Select<T>, Where, OrderBy, etc.


Query Pattern: SelectIntroduction to LINQ 35C# 30

Query Pattern: Select

class PayrollSystem {public IEnumerable<T> Select<T>(

Func<Employee,T> selector) {Func<Employee,T> selector) {return employees.Select(selector);

}internal List<Employee> employees = ...;//More implementation omitted

}}

PayrollSystem payroll = ...;var names = from employee in payroll

select employee.Name;


Query Pattern: Select ExtensionIntroduction to LINQ 36C# 30

Query Pattern: Select Extension

Alternati el an e tension method can be• Alternatively, an extension method can be used:class PayrollSystem {internal List<Employee> employees = ...;//More implementation omitted

}static class PayrollExtensions {static class PayrollExtensions {public static IEnumerable<T> Select<T>(

this PayrollSystem payroll,Func<Employee,T> selector) {

return payroll.employees.Select(selector);}}

}PayrollSystem payroll = ...;


var names = from employee in payrollselect employee.Name;

Query Pattern: WhereIntroduction to LINQ 37C# 30

Query Pattern: Where

class PayrollSystem {public IEnumerable<Employee> Where(

Func<Employee,bool> filter) {Func<Employee,bool> filter) {return employees.Where(filter);

}internal List<Employee> employees = ...;//Select<T> omitted

}}

PayrollSystem payroll = ...;var names = from employee in payroll

where employee.Salary > 220select employee.Name;select employee.Name;


Query Pattern – NotesIntroduction to LINQ 38C# 30

Query Pattern – Notes

The q er pattern implementation can also• The query pattern implementation can also be used to customize query behaviorq y– For example, specify additional filters for the Where

query operator (even if the object is IEnumerable)que y ope ato (e e t e object s u e ab e)

static class PayrollExtensions {public static IEnumerable<Employee> Where(public static IEnumerable<Employee> Where(

this PayrollSystem payroll,Func<Employee,bool> filter) {

return payroll.employees.Where(e => e.City==“London” && filter(e));

}}}


LINQ to Objects ApplicabilityIntroduction to LINQ 39C# 30

LINQ to Objects Applicability

An object model is a potential data• Any object model is a potential data source for LINQ queriesq– The requirements are not strict – either IEnumerable

or the query patternor the query pattern

M k bj t d l LINQ f i dl !• Make your object models LINQ-friendly!


LINQ to StringsIntroduction to LINQ 40C# 30

LINQ to Strings

string s = Guid.NewGuid().ToString();string digitsOnly =new string((from c in snew string((from c in s

where Char.IsDigit(c)select c))

.ToArray());Console.WriteLine(s + “\n” + digitsOnly);

//Output:8faf837d‐f0a6‐4e69‐8ac5‐23635452f6f9883706469852363545269


LINQ to ReflectionIntroduction to LINQ 41C# 30

LINQ to Reflection//Find all collection‐initializable types//Find all collection initializable types//(Rough sketch)var queryableTypes =from asm in AppDomain.CurrentDomain.GetAssemblies()from t in asm.GetExportedTypes()where t GetInterfaces() Any(itf =>where t.GetInterfaces().Any(itf =>

itf.IsGenericType &&((itf.IsGenericTypeDefinition

||&& itf == typeof(ICollection<>)) ||(!itf.IsGenericTypeDefinition &&itf GetGenericTypeDefinition() ==itf.GetGenericTypeDefinition() ==

typeof(ICollection<>)))) ||typeof(IEnumerable).IsAssignableFrom(t) &&

t.GetMethods().Any(m => m.Name == "Add")select t;


select t;

LINQ to File SystemIntroduction to LINQ 42C# 30

LINQ to File System

//Find all large DLLs in System32var largeDllFiles =from file in Directory GetFiles(from file in Directory.GetFiles(

Environment.GetFolderPath(Environment.SpecialFolder.System))p y ))

let info = new FileInfo(file)where info.Extension == ".dll" &&

info Length > 5000000info.Length > 5000000select info.FullName;


LINQ to Objects PerformanceIntroduction to LINQ 43C# 30

LINQ to Objects Performance

1 Don’t turn every f and foreach loop1. Don’t turn every for and foreach loop into a LINQ queryq y

2. yield return is often slower than h d ll d t f lhand-rolled enumerators or for loops

3 Invoking lambdas is often more3. Invoking lambdas is often more expensive than an inline loop body

4. Measure to ensure sane results and sane number of calls (run-time complexity)number of calls (run-time complexity)



Chapter 23 Exercises

C# 3.0, QUERY OPERATORS AND LINQ TO OBJECTSAND LINQ TO OBJECTS


Chapter SummaryIntroduction to LINQ 45C# 30

Chapter Summary

• More C# 3 0 language features• More C# 3.0 language features– Anonymous types, implicit typing, expression trees

• C# 3 0 meets LINQ• C# 3.0 meets LINQ– Query operators, query pattern

• LINQ to ObjectsLINQ to Objects


chapter 23 - introduction to linq

Documents