Computation and

Discrete Mathematics

Klaus Sutner

Carnegie Mellon University

Fall 2019

1 Administrivia

� Course Material

� Computation and Math

� Proofs

Dramatis Personae 3

Prof:Klaus Sutner, sutner@cs.cmu.edu

TA:Chris Grossack cgrossac@andrew.cmu.edu

Vaidehi Srinivas vaidehis@andrew.cmu.edu

Course secretary:Rosie Battenfelder, rosemary@cs.cmu.edu

Email 4

Some of you may still remember email (a medieval communication toolinvolving pigeons, predating social networks by thousands of years). Ifyou decide to get in touch with me via email, use

Subject line:

[CDM] will miss midterm

or some such. I have hacked emacs vm and filter rather aggressively,make sure to have the [CDM] tag.

Bureaucracy 5

The usual testing:

• homeworks 50%

• midterm (in-house) 20%

• final 30%

There are no makeups; if you miss some assessment you can sit foran oral exam (I am not a great supporter of the currently popularhigh school approach to university education).

HW is critical, for your grade (summative) and for feedback andunderstanding (formative). Make sure to get it right.

Bureaucracy II 6

Midterm is in-house, 80 minutes on Oct 16.

Final will be the usual 3 hour gig; ugly, but inevitable.

Homework should take around 8 hours; we’ll monitor as we go along.

Bureaucracy III 7

You have a total of 6 (six) late days at your disposal; use prudently.

A late day is a discrete atom, with no smaller parts.

Mention lateness in the header of your HW.

I reserve the right to give you no credit at all for work submittedbeyond the allotted time.

Preserving TA Sanity 8

Typeset your solutions to the homework and submit pdf onGradescope (more on this on Piazza).

If you have extensive conversations with other students about a HW,mention them as “collaborators” in your submission.

If you use a computer program in your homework, make sure toreference it properly (but do not hand in 50 pages of code).

Math Typesetting 9

My preferred environment for math typesetting is emacs and (acustomized version of) AUCTEX.

For HW this is admittedly a fairly big gun, but if you ever want topublish a paper, write a thesis or do any other scholarly work, there isreally no choice—get used to it now (and start working on your own stylefiles, a macro file, a bib file, learn a drawing tool . . . )

If you prefer some IDE type environment, pick whichever one you likebest.

Don’t use Word or some such garbage. Many conferences/journals willnot accept this stuff, and if they do, the likelihood of acceptance dropssignificantly.

Cooperation 10

Lectures will be warm and friendly. Make sure to be an activeparticipant – CDM is not a spectator sport.

You are strongly encouraged to talk about the course material toeach other, the course staff and other students.

This includes discussions of homework problems.

Limits to Cooperation 11

However, even after ample consultation, the work you submit mustbe written entirely by yourself.

List all your “consultants” on the first page of your homework.

To avoid problems with originality, do not take notes whendiscussing homework problems.

If you write on a board, erase everything in the end.

Yet More on Limits 12

And, of course, all the official university policies apply.

http://www.cmu.edu/academic-integrity/index.html

� Administrivia

2 Course Material

� Computation and Math

� Proofs

Web and Communication 14

http://www.cs.cmu.edu/∼cdm/

https://piazza.com ⇒ 15-354

Piazza 15

. . . is great

It’s a good way to share information and get answers withoutterrible delays.

. . . sucks

It can be used to avoid work by asking lots of silly questions, andrelying on others to do all the heavy lifting.

This is an upper level class, don’t play games.

Also, don’t be rude and repost the same question over and over.

Asking Questions 16

Always use descriptive titles:

HW 3, Q 2: why is f primitive recursive ?

Lect 5, sld 15: gap in proof of theorem

Provide all the necessary information, include links whenever appropriate.

If the questions is ill-posed the likelihood of a useful answer diminishes.

More Piazza 17

Our Piazza has a label category Notes.

Use this tag to point out problems with lecture slides and lecture notes.

This is not just about a typos or plain errors–I also want to hear if a slideis misleading, or difficult to read and understand, another example isneeded, . . . Ideally, propose an improvement. But at the least let meknow that there is a problem.

Course Material 18

There is no text book.

There are lots of slides and/or lecture notes at CDM.

Often you will be required to read ahead of lecture so we have timefor discussions and problem solving.

Sometimes we will not cover all the material in lecture, you areexpected to read the rest on your own.

Additional material will be posted on the web.

Learning Style 19

The topics covered in this course translate into quite a bit ofmaterial.

Read the notes, check out CDM, search the net, go to the library,talk to each other, talk to us.

One of the desired outcomes of this course is that you know whereto find more information if and when you need it.

And: Turn up at office hours regularly, not just in times of majorcrisis.

Sources 20

At this point, there is a large amount of high-quality material on the web.

Make sure you know where to look.

Google.

Google scholar.

Course notes, online courses, blogs.

DBLP, ECCC, arXiv.

Conferences (FOCS,STOC,SODA,CCC,ICALP).

Tools 21

Computer AlgebraMAPLE, Mathematica, SAGE (algorithmically roughly equivalent,Mta has best user interface).

CAS Packages

OEIS, Inverse Symbolic Calculator

Exotic ToolsE.g., Budnik’s Ordinal Calculator

Again, the point here is to know about these resources ahead of time,and to be reasonably familiar with them. Frontload the process—it canbe a live-saver.

� Administrivia

� Course Material

3 Computation and Math

� Proofs

Central Axiom 23

The most important development in mathematics in the20th century is the development of the digital computer.

Yet we approach math much in the way of the 19th century (look at anycalculus course). This amounts to a huge opportunity cost.

Jobs’ Genius (1988) 24

Uses of Computing in Math 25

Number CrunchingSolving complicated differential equations; optimization problems.Historically the first major application; real arithmetic is difficult ondigital machines.

Symbolic ComputationDirectly manipulate symbolically presented entities (computeralgebra, SAT solvers, model checkers).

• large, complex computations• example/counterexample generation• integrated computational environments

Bleeding Edge 26

Knowledge ManagementA global mathematical library would be some 108 pages. Some smallbut growing part of this is available in digital form on the web.Some even smaller part is indexed and searchable (semanticmarkup). Some yet smaller part is validated.

Proof Checking, Theorem ProvingOn rare occasions, proof assistants and automatic theorem proversare helpful in finding (parts of) a proof. Better at verification (proofcheckers) than search, require some amount of skill on the side ofthe user.

The Magic Spiral 27

The

Creativity

Spiral

Compute/Experiment

Specify/Formalize

Prove

Visualize

Conjecture

Main Work Loop 28

Specify/FormalizeWe start with a question, often vague and imprecise. With someeffort the question is turned into a concise and precise problem in(discrete) mathematics.

Experiment/ComputeTo help develop basic understanding we use computation togenerate data (examples and counterexamples). Setting up theprograms may well require a bit of work.

Analyze/VisualizeAnalyze and interpret the (tons of) data, find patterns and structure.

Loop, Part 2 29

Specify/Formalize

Experiment/Compute

Analyze/Visualize

Conjecture/ProveUltimately formulate a conjecture and proceed to prove it. Wrongconjectures and dead-ends in proof attempts can sometimes beeliminated by more computation.

Apply/GeneralizeUse the new and proven theorem to improve the power ofcomputation; discover new questions.

Aside: Visualization 30

Julia and Fatou had basic mathematical ideas 100 years ago, Hausdorffknew about non-integral dimensions. Mandelbrot had IBM computers.Bourbaki completely blundered this.

� Administrivia

� Course Material

� Computation and Math

4 Proofs

Math and Proofs 32

Applications of digital computers to numerical and symbolic computationas well as mathematical knowledge management are not controversial(well, at least not very).

But the greatest impact of digital computers on math in the long run isprobably going to be in connection with proofs.

Status Quo 33

Standard math proofs are directed at smart, highly trained experts, whospent years and even decades on becoming familiar with a particular styleof exposition. In particular, they have learned to fill in gaps and interpretambiguous assertions.

A prover/proof checker is a completely mindless and purely mechanicaldevice, a “persistent plodder” (Hao Wang). It will do exactly what it isprogrammed to do, no more and no less. If there is any flaw in thealleged proof, the verification attempt will fail.

Rigor 34

It is a cherished myth that mathematical proofs are objects with absoluteprecision and rigor. Here is an interesting 19th century quote:

If Gauss says he has proved something, it seems very prob-able to me; if Cauchy says so, it is about as likely as not; ifDirichlet says so, it is certain.

C.G.J. Jacobi, in a letter to A. von Humboldt

Jacobi was no slouch; if he criticizes a proof it is because there are real,serious and difficult-to-fix problems, not just some superficial lack ofunderstanding.

Ancient Mathematical History 35

For our purposes, that’s everything up to the middle of the 19th century.

Up to this point, mathematics was doing just fine (at least in the eyes ofthe beholder): there were generally accepted modes of mathematicalreasoning that were considered to be perfectly reliable–assuming thepractitioner did not blunder, the results were good for perpetuity.

E.g., Euclid’s old argument for the infinitude of primes is still perfectlygood (unless you are a dyed in the wool constructivist).

High Sophistication 36

The level of development in the first half of the 19th century was alreadyquite astounding.

Take, for example, Fourier’s theory of heat flow (1822), leading to hisfamous representation of periodic real functions:

f(x) =∑n≥0

an cosnx+ bn sinnx

But: Fourier had only physics-based plausibility arguments for thecorrectness of his representation, nothing resembling modern proof. Infact, he did not even have a good definition of a function (Weierstrass,Dirichlet, Lipschitz and Riemann).

Foundations 37

How does one establish a solid theory of Fourier expansion? What arethe underlying mathematical foundations? Up until 1850 or so, no onecared.

If anything, people would have followed in Kant’s footsteps andsubscribed to a theory that

spatial intuition is a given of human cognition, and gives rise togeometry,

intuition of time is similarly given, and leads to arithmetic.

Hamilton (as in Hamiltonian systems and quaternions) tried to formulatesuch a model in 1853.

Abstraction and Complexity 38

Why did these difficulties appear a century and a half ago and notsooner? To first-order approximation, one can guess that the steady andrelentless increase in abstraction took its toll: intuition becomes less andless reliable the further away one moves from every-day notions.

Is should be noted that even today not everyone participates in the questfor absolute precision. For example, physics super-star Steven Weinbergwrites in a book on quantum field theory

. . . there are parts of this book that will bring tears to theeyes of the mathematically inclined reader.

In physics, this is probably a good thing that helps the field along. In CS,it would more likely be a disaster.

So What’s The Problem? 39

Following Godel, proofs are usually defined as sequences of formulae insome formal language:

ϕ0, ϕ1, ϕ2, . . . , ϕn

where each formula is an axiom or follows from previous ones by a simplelogical rule of inference (like modus ponens).

A very simple, clear formalization that allowed Godel to construct hisinfamous incompleteness theorem (much to the dismay of Hilbert).

Unfortunately, it has just about no connection to what counts as a proofin the RealWorldTM. In practice, proofs live in the Wild Wild West.

Euler, an Example 40

Euler had the amazing ability to concoct arguments that were eminentlyplausible, and led to correct results, but were exceedingly difficult tojustify in the modern sense. Here is an example:

Problem: Find a way to calculate ex for positive reals x.

We know that for x > 0 reasonably small we can write

ex = 1 + x+ error

with the error term being small. Alas, we have no idea what exactly theerror is.

Infinitesimals 41

Euler considers an infinitesimal δ > 0. Then

eδ = 1 + δ

This is justified by Leibniz’s lex homogeneorum transcendentalis, thetranscendental law of homogeneity (proof by higher authority).

Then, by the laws of exponentiation,

ex = (eδ)x/δ

Using the binomial theorem we get

ex = (eδ)x/δ

= (1 + δ)x/δ

= 1 +

(x/δ

1

)δ +

(x/δ

2

)δ2 + . . .

= 1 + x+ 1/2x(x− δ) + . . .

= 1 + x+ 1/2x2 + . . .

=∑i≥0

xi/i!

Headache? 43

The result is perfectly correct. But the argument . . . oy vey.

Incidentally, Euler also had an ingenious argument to show that

1 + 2 + 3 + . . .+ n+ . . . = − 1

12

No, that’s not nuts.

Look up ζ functions and analytic continuations.

Food for Thought 44

ExerciseFind all the places where Euler’s reasoning is dubious from a modern dayperspective.

Exercise (Very Hard)Fix all the problems with Euler’s argument.

ExerciseFigure out why it makes sense to claim that 1 + 2 + 3 + . . . = −1/12.Similarly 1 + 1 + 1 + . . . = −1/2.

The Antidote 45

Apparently, the only reliable (and somewhat unpalatable) solution to thisproblem of rigor is to be exceedingly formal and precise in all arguments.At least four frameworks emerged that appear to be helpful in thisenterprise (perhaps in combination):

Logic and Formalization (Boole, Frege, Peano)

Axiomatization (Peano, Dedekind, Hilbert)

Set theory (Dedekind, Cantor, Frege, Zermelo, Fraenkel)

Type theory (Russell, Howard-Curry, Church, Per-Lof, Voevodsky)

The first two merged more or less into the notion of a formal system.Under Bourbaki, set theory developed into the reference implementation,for the last half century the gold standard. Alas, type theory is nowclearly more important in CS, and perhaps at some point also again inmath.

Russell and Whitehead 46

As it turns out, it is easy to design a formal system that is inconsistent.For example, Frege had an extremely elegant and concise system that fellprey to a classical paradox discovered by Russell (and before by Zermelo).In modern notation

S = {x | x /∈ x }

To construct a formal system that avoids inconsistencies and that is alsopowerful enough to cover all of mathematics is rather difficult. Thetype-theoretic approach taken by Russell and Whitehead in their PrincipiaMathematica is horribly technical (and somewhat self-defeating) and wasnever appreciated by “ordinary” mathematicians.

Godel 47

To add insult to injury, Godel showed some 20 years later that anysystem like Principia is necessarily incomplete: some true statementscannot be proven in the system.

In particular consistency (i.e. lack of internal contradictions) is mostelusive: e.g., one cannot prove consistency of arithmetic in arithmetic.

Interestingly, Godel’s argument is based on a version of the oldEpimenides paradox, exploiting self-reference:

This sentence is false.

Note that this type of paradox is quite different from the purely logicalparadox of Russell.

Zermelo-Fraenkel Set Theory 48

A technically less daunting approach than Russell-Whitehead is toaxiomatize set theory.

Zermelo proposed a system in 1908, augmented in 1922 by Fraenkel (todeal with functions). The standard version of ZF set theory has only 9simple axioms:

extensionality, empty set, unordered pair, union, power set,separation, replacement, foundation, infinity.

This axiomatization is arguably the most successful in all of mathematics(and, by implication, theoretical CS), it is still the de facto referencestandard. Bourbaki’s groundbreaking work in the middle of the 19thcentury contributed majorly to this state of affairs.

Hilbert’s Program 49

Partially in response to intuitionistic lunacy, in the 1920s Hilbertproposed a program to salvage all of mathematics. In a nutshell:

Formalize mathematics and concoct a finite set of axiomsthat are strong enough to prove all theorems of mathematics(completeness) and show that the system is consistent; bystrictly finitary means. Also show that statements about“ideal objects” can be proven in the system, without usingideal objects.

Initially some good progress (completeness of propositional logic, then ofpredicate logic).

But then, in 1931, Godel drops a bombshell: any formal mathematicalsystem built on predicate logic is necessarily incomplete (or inconsistent).

Computability 50

Though some parts of Hilbert’s program were irreparably damaged byGodel’s result, in many ways things just started to become reallyinteresting.

Thus the notion “computable” is in a certain sense “abso-lute,” while almost all metamathematical notions otherwiseknown (for example, provable, definable, and so on) quiteessentially depend upon the system adopted.

K. Godel, 1936

There is a clear connection between incompleteness and unsolvability, socomputability is a rather central notion in mathematics.

Entscheidungsproblem 51

The Entscheidungsproblem is solved when one knows a pro-cedure by which one can decide in a finite number of oper-ations whether a given logical expression is generally validor is satisfiable. The solution of the Entscheidungsproblemis of fundamental importance for the theory of all fields, thetheorems of which are at all capable of logical developmentfrom finitely many axioms.

D. Hilbert, W. AckermannGrundzuge der theoretischen Logik, 1928

In modern terminology: find a decision algorithm for statements ofmathematics (or at least some part like arithmetic, group theory, . . . ).

And Computers . . . 52

In modern terminology, Hilbert is looking for a decision algorithm for allof math.

Similarly the various formalization attempts all nicely translate intocomputation: all formal systems come with associated algorithms.

So it is quite natural to use computation as a universal lever to tacklemath.

Naysayers 53

Lastly, and most dastardly, there is the much ballyhooed exten-sion of the notion of mathematical proof, far beyond the classicalGreek paradigm of of axioms and rules of inference, to include so-called computer-proofs, whose non-surveyability by human beingsdemands appeals to faith inimical to the enterprise of science,and yet allegedly yields results which are otherwise of necessityfar beyond the powers of mortal man to obtain.

A mediocre mathematician with a computer might be able tosimulate the creative powers of a top notch mathematician withpencil and paper.

This was written in 1991 by a propellerhead who shall go unnamed andunmentioned.

Naysayers II 54

OK, one more quote from the same guy.

Admitting the computer shenanigans of Appel and Haken to theranks of mathematics would only leave us intellectually unsatis-fied.

As my old high school philosophy teacher would say: That you will haveto take into your own hands.

Mechanization versus Intuition 55

Just to be clear: a formal proof achieves exceedingly high levels ofreliability. Alas, the process of formalization does not help at all when itcomes to discovering the basic structure of a proof.

Finding the right “proof idea” is really quite a mysterious process, wedon’t understand well enough how human cognition works to explain it.Often, the critical insight pops up, sometimes after a long and protractedstruggle, that may well seem to go nowhere.

Zagier’s Proof 56

There is an old theorem by Fermat that every prime of the form 4k + 1 isthe sum of two squares.

Here is an astounding proof of this fact. Batten down the hatches.

Pick such a prime p and consider the finite set

S = { (x, y, z) ∈ N3 | x2 + 4yz = p }

Note that (1, 1, k) ∈ S, so S 6= ∅.

We would like to identify some (x, y, y) ∈ S.

An Involution 57

Now, out of nowhere, define an involution π on S:

π(x, y, z) =

(x+ 2z, z, y − x− z) if x < y − z,

(2y − x, y, x− y + z) if y − z < x < 2y,

(x− 2y, x− y + z, y) if 2y < x,

The missing cases x = y − z and x = 2y contradict our assumptions.

It is straightforward but tedious to check that π really is an involution.

Let S = S1 ∪ S2 be the partition into fixed points and 2-cycles inducedby π.

Example: p = 61 58

In this case,

S = (1, 1, 15), (1, 3, 5), (1, 5, 3), (1, 15, 1), (3, 1, 13),

(3, 13, 1), (5, 1, 9), (5, 3, 3), (5, 9, 1), (7, 1, 3), (7, 3, 1)

and π induces the permutation (1, 8, 11, 5, 4, 7, 6, 2, 10, 9, 3), whichconsists of 4 2-cycles and 1 fixed point.

ExerciseProve that this is true in general: there is one fixed point, and a numberof 2-cycles.

One Fixed Point 59

It is easy to check that (1, 1, k) is the only fixed point of π. Hence Smust have odd cardinality.

It follows that any involution ρ on S must have an odd number of fixedpoints, and in particular at least one. So consider

ρ(x, y, z) = (x, z, y)

Done! 2

The Catch? 60

So where is the catch? In 2019, why isn’t it standard to approach(certain parts of) of math as a trip into the computational universe?

In fact, why is there quite so much resistance to consider the obviousconnections?

Alan Turing 61

I expect that digital computing machines will eventually stimu-late a considerable interest in symbolic logic . . . The language inwhich one communicates with these machines . . . forms a sort ofsymbolic logic.

Bernard Chazelle 62

The Algorithm’s coming-of-age as the new language of sciencepromises to be the most disruptive scientific development sincequantum mechanics.

Algorithmic Thinking 63

This may actually be an understatement. Thinking about scientificproblems from the perspective of someone who understands computationdeeply may be the paradigm shift (apologies to Kuhn).

https://www.cs.princeton.edu/∼chazelle/pubs/algorithm.html

Harsh Reality 64

In theory there is no difference between theory and practice.

In practice there is.

Yogi Berra

It’s Hard . . . 65

Everybody who has worked in formal logic will confirm that itis one of the technically most refractory parts of mathematics.The reason for this is that it deals with rigid, all-or-none con-cepts, and has very little contact with the continuous conceptof the real or of the complex number, that is, with mathemati-cal analysis. Yet analysis is the technically most successful andbest-elaborated part of mathematics. Thus formal logic is, by thenature of its approach, cut off from the best cultivated portionsof mathematics, and forced onto the most difficult part of themathematical terrain, into combinatorics.

John von Neumann, 1948

And more . . . 66

The theory of automata, the digital, all-or-none type as discussedup to now, is certainly a chapter in formal logic. It would, there-fore, seem that it will have to share this unattractive property offormal logic. It will have to be, from the mathematical point ofview, combinatorial rather than analytical.

Smile 67

Sadly, von Neumann is somewhat right.

But things turned out really well, after all.

For example, automata theory is now one of the huge success stories inlogic/cs, in theory as well as applications. It is very different fromclassical, 19th century type mathematics. And it is difficult, but in theend, really no more difficult than the traditional material (try partialdifferential equations if you disagree).

The future is golden . . . 68

Generally, computer science, that no-nonsensechild of logic, will exert growing influence on ourthinking about the languages by which we expressour vision of mathematics.

Yuri Manin

Reading 69

Take a look at the notes on primitive recursive functions onthe web.

Don’t worry about technical details, just get a general idea.

Ask questions on Thursday (or on Piazza if time-critical).