CS 121: Lecture 2Math Review

Madhu Sudan

https://madhu.seas.Harvard.edu/courses/Fall2020

Book: https://introtcs.org

Only the course heads (slower): cs121.fall2020.course.heads@gmail.com{How to contact usThe whole staff (faster response): CS 121 Piazza

Administrative:• My office hours: Tu, Th: After lectures. Drop in (for now). • All office hours + sections: See calendar.• Hw0 graded. Solutions to be posted soon.• Patel fellows: cs121.fall2020.patel.fellows@gmail.com

• 121.5: Thursday 4:30pm. Boaz Barak on “Coding, Compression, Entropy”

Serena Davis Nari Johnson

Mathematical approach• Take concept we want to model (number, triangle, graph, algorithm,..)

and give precise definition.• Use definition to prove properties of object• Use properties to build higher level objects from it.

Today: Math Review• Basic Blocks (12 min)• Exercise break 1 (7+5 min)• Functions (7 min) • Exercise break 2 (7+5 min) • Graphs (10 min) • Exercise break 3 (7+5 min)• Asymptotics (10 min)• Take home exercises (0 Min)

Math is like…Lego bricks, constructing a building, Russian dolls, …

Building blocks• Sets: 1,2,7 , Harvard , 1 , 1,2 , 1,2,7 , ,Գ,Ժ,Թ

• Products of sets ܣ × ܤ × ܥ ; tuples (ordered sets): 1,2,7 , 7,1,2 ,…• Relations: ك ܣ × ܤ ; Functions: :ܣ ՜ ܤ

• Binary Strings: Tuples of 0’s and 1’s: 011101, 101110, 0,””, 0001110111011• Sets of tuples/functions: : {0,1}ଶ ՜ {0,1} }

• Quantifiers: ܣ ܤ s.t. one to one function :ܣ ՜ ܤ

• Kleene star: כܣ = Գא ܣ = A Aଵ Aଶ ڮ (e.g., binary strings = 0,1 (כ• Conventions/Notation:

Ժ = … ,െ2,െ1,0,1,2, … ;Գ = 0,1,2, … ; = 0,… , െ 1

Indexing of Strings: ݔ = …ଵݔݔ ;ଵݔ ݔ =

Logic Exercise (Break – 7 min)• Consider the expression: For : ՜ Ժ

߶ : א െ 1 > s. t. > ר .ݏ .ݐ < < , ()

0. Is the meaning of every symbol clear? Ask now if not!1. Understand what the expression says.2. Does ߶ hold for = 7, = 21?

3. Does ߶ hold for = ଶ, = 21?

4. Simplify the expression- In English- In Logic

such that

V-ieffh-Dflikflnlane-flils.fi#

- f is a non - decreasing function .

- f can 't be constant .

- fin -D needs to be target . "niggas . .

Logic Exercise (Break – 7 min)• Consider the expression: For : ՜ Ժ

߶ : א െ 1 > s. t. > ר .ݏ .ݐ < < , ()

0. Is the meaning of every symbol clear? Ask now if not!1. Understand what the expression says.2. Does ߶ hold for = 7, = 21?

3. Does ߶ hold for = ଶ, = 21?

4. Simplify the expression- In English- In Logic

[Solutions]

→ no!

YES : flo)- ft) -- O , - .

. f- (8)=fH9) -- 9f- 1201=10 .

→ flat) is larger than everything & f is non - decreasing .

→ Hi Efn-D ffli) s flit) ) e ( fli) cfln -D)

Functions• Terminology:• Relations: ك ܣ × .ܤ (Graphs, directed graphs are “relations”).• Function: Relation ك ܣ × ܤ s.t for every א ܣ there is exactly one א ܤ s.t.

, א . Denoted :ܣ ՜ ܣ .ܤ “domain”, ܤ “codomain/range”• Partial function: Relation ك ܣ × ܤ s.t for every א ܣ there is at most one א ܤ

s.t. , א . Sometimes denoted :ܣ ՜ ܤ ٣. = ٣ “No s.t. , א • Types of functions:

• Injective/one-to-one: • Surjective/onto:• Bijective/one-to-one & onto/one-to-one correspondence/invertible:

AB Fonte.io. one

① I notonto-

Why are functions important to us?• Almost everything we want to compute are functions. • Rarely: It is a relation.• Even more rarely: it is something else.

• Even basic object such as function gives tools that are central to analyzing computation.

FunctionsProve: If ܣ and ܤ finite sets and ܣ |ܤ| then exists onto function from ܤ to ܣ

Prove in class

÷÷÷÷±÷A's A - a

7- Onto function from B'to A

'

FunctionsProve: If ܣ and ܤ finite sets and ܣ |ܤ| then exists onto function from ܤ to ܣ

ܣ is empty or

Defn of function :

vi. bets ,"therefore " B

A'.

- A - la}

÷:*.no#.l::i:÷÷÷÷.€ BEB ffb)=a .

Ase 2 : as on previous .

Functions Exercise (Break – 7 min)Prove: If ܣ and ܤ sets and exists onto function from ܤ to ܣ then exists one to one function from ܣ to ܤ

finite

af.¥oa8%im'''

⇒

"

II::* .- --

co

"

i÷n. .P. ( Sal = T element

yea) -- bone-to-one ⇐ { . yea , = manyelements}

Functions Exercise (Break – 7 min)Prove: If ܣ and ܤ sets and exists onto function from ܤ to ܣ then exists one to one function from ܣ to ܤ

[Solutions]finite

^

• Let f :B -7A be onto

• for ac- A let Sa -- { beBlflbka }

• Claim 't : Sato since f is onto

a Claim 2 : Sansa' = & fat a' EA .

Because f is a function .

c- Putting together : . for every a-Abt gla) be arbitrary element Asa .

- this is a one - to - one function . Details Omitted) .

Graphs:• Directed graphs: ܩ = ܧ, ,where ܧ ك × is a relation.• Undirected graphs: ܧ is a symmetric relation: ݑ, ݒ א ܧ ݑ,ݒ א ܧ

• Why do we study graphs:• Most common (part of) input to problems we wish to solve …

• We study road networks, social networks, communication networks, gene networks …

• Also used to describe our computational models!• Circuits, Finite automata, Turing machines !!

Graphs Exercise (Break – 7 min): Prove: if ܩ = (ܧ,) is a directed graph s.t. ݑ א with indeg ݑ <outdeg(ݑ) then ݒ א s.t. indeg ݒ < outdeg(ݒ)

Def: indeg(ݑ) = # of edges pointing into ݑ= ݓ א | ݑ,ݓ א ܧ

outdeg(ݑ) = # of edges pointing out of ݑ= ݓ א | ݓ,ݑ א ܧ

→

A- .

.-

- Ei-

indeg(a) =3Qntdeylu) = 4

Graphs Exercise (Break – 7 min): Prove: if ܩ = (ܧ,) is a directed graph s.t. ݑ א with indeg ݑ <outdeg(ݑ) then ݒ א s.t. indeg ݒ < outdeg(ݒ)

Def: indeg(ݑ) = # of edges pointing into ݑ= ݓ א | ݑ,ݓ א ܧ

outdeg(ݑ) = # of edges pointing out of ݑ= ݓ א | ݓ,ݑ א ܧ

[Solution]

- s.

.

-

T

.

-

Claim : u.cqindeglul-fzuoutdeg.lu)

Bogdani: By induction on # edges .

Base if # edges -

- O then indegla) -- outdegla ) -

- o fu .

a ⇒ {milgln) = {ouldglu )-

- O

indutiresiop : .it?n--w.fft&Y!n.dgg.tu)=uEouldesgw). indegglvt -

-indgg.tw/tl.0nldegglvl--outdeggi/v)tl. . . hypothesis continues to hold . .

.

Claim ⇒ Exercise-

:

Assume for contradiction that indeglv) E ①htdegcv) tr EV

then £ indeedv) = E indeed) t indeglu )

VEV VEV - Siu }

< ¥7,ugindiglv) t outdegcu)

⇐ E out deglv) t oudtdegla)

VEV - fu}'

= § outdeglv ) .

But this contradicts claim T DX

item 's" "'

am, }eimn:tIi'if fcm.co/glnl) if Fosco ,

c. ,no set .

for all n> no.li?inffgfIY3Ico.gln) s fin) E C, Gln)

f- In) = n'

gin) -- ÷ +lo"

. nEt→:

Asymptotics:• Need to know:• . (Big-Oh) notation: What it means, how to prove, and shortcuts!• . (little-oh) :• Related notions:

• ȳ . ,ȣ . , . . (But no ߠ . (little-theta)?)• Short cuts:

• Sum = Max! + = max ,()• Can ignore constants

• 30ଶ + భబ

ଵ+ 273.425 = ଶ + ଵ + = ଵ

• When input = number A, then its length (in bits/digits) = ȣ logܣ

OC ) - s not growing faster than

e. ( ) -g not growing slower than

①D= Onr

-- growing exactly

as fast as .

°- -

①f- = Olg)

- -

I⇒ g-- SHH

AH

-

tag

Exercises (take home):

• Let = 2 ୪୭ ! and = ଶ + ͳͲ: • Is = ( )? • Is = ) )? • Is = ȳ( )? • Is = ( )? • Is = ȣ( )?

• Let = .• If = ଶ and = (ଷ) then is = ?

Exercises (take home):

• Let = 2 ୪୭ ! and = ଶ + ͳͲ: • Is = ( )? • Is = ) )? • Is = ȳ( )? • Is = ( )? • Is = ȣ( )?

• Let = .• If = ଶ and = (ଷ) then is = ?

[solution :]

Party .

.

We use m! = gotmlosm)

① Hnl :(2%7 ! = 20128in )

② gun) : ⑦ Cri ) = 20168N)

since login -- o ( Struan ) we get

lil fink algin)) & gin) . o( find

←so Lin ) f- Ink right) & gin) = Olgin))

(iii ) fin) # Olgin)

Parts : C)flint. dm) ⇒ fco> o Iho sit . th>

nofln) ⇐ co - n' WARNING

(lil gin) : 01ns ) ⇒ Fc, ,n ,sit . An > n ,

Part 2 was

harder thanciiistosnowncnhocng.mn?ed4on3snow:bj!nn;doHoz Zha Hh> nz for completenesshlnlfczinb

Given Gso we need to show how to

pick Nz .

• Idea : if n is large enough e fin ) is

large enough we are fine since

f-Ch) s co - NZ & gcflnD.ec, fin)'

⇒ GAIN) s (c:c , )n6

. But trickiness comes up if fln) .sn ,- . . .

i. . this requires more work but not really . . .

since now fin) is a" constant

" (doesn't dependon n ) s for large enough n , gffln) )E gin ,)

. .. roughly .

.We now turn to the formal proof . really we

do part I first a fill in part I later to

fit part II .

Part : . Let G.n.be from the condition on g .

o Let G = Max

Kien ,

{ gli) }

•Let co = (Czk , )

's.

Let no be from property of fo

o Let nz= Max {no , a)"b) }

We will show gffln)) E Cz n 6 ,for n > Nz o

Parti : Let n> nz .

Asai : fan) > n , .

then we have

fin) s com & gffln)) s c , fin)3

fsnia n> hand E C ,.co?nb

E Cz . Nb Dx

cased : fln)En . .

Let i-th)

we have gain)) s gli) E G [definition DG]& OE i EN ,

← can! [ definition of ha]

E Ghb [ Mhz) DX

Example:

-

-

Rest of the course:Part I: Circuits: Finite computation, quantitative study

Part II: Automata: Infinite restricted computation, quantitative study

Part III: Turing Machines: Infinite computation, qualitative study

Part IV: Efficient Computation: Infinite computation, quantitative study

Part V: Randomized computation: Extending studies to non-classical algorithms

,:

Rest of the course:Part I: Circuits: Finite computation, quantitative study

Part II: Automata: Infinite restricted computation, quantitative study

Part III: Turing Machines: Infinite computation, qualitative study

Part IV: Efficient Computation: Infinite computation, quantitative study

Part V: Randomized computation: Extending studies to non-classical algorithms

Rest of the course:Part I: Circuits: Finite computation, quantitative study

Part II: Automata: Infinite restricted computation, quantitative study

Part III: Turing Machines: Infinite computation, qualitative study

Part IV: Efficient Computation: Infinite computation, quantitative study

Part V: Randomized computation: Extending studies to non-classical algorithms