Automata and Formal Languages - CM0081Introduction

Andrés Sicard-Ramírez

Universidad EAFIT

Semester 2018-2

Administrative Information

Course CoordinatorAndrés Sicard Ramírez

Head of the Department of Mathematical SciencesCarlos Mario Vélez Sánchez

Course web pagehttp://www1.eafit.edu.co/asr/courses/automata-CM0081/

Exams, programming labs, etc.See course web page.

ConventionThe numbers assigned to examples, exercises or theorems correspond to thenumbers in the textbook (Hopcroft, Motwani and Ullman 2007).

Introduction 2/29

Computability (Decidability)

QuestionWhat can a computer do at all?

DefinitionA computable (or decidable) problem is a problem than can be solved bya computer (informal definition).

Introduction 3/29

Computability (Decidability)

QuestionWhat can a computer do at all?

DefinitionA computable (or decidable) problem is a problem than can be solved bya computer (informal definition).

Introduction 4/29

Computability (Decidability)

Example (The halting problem: An undecidable problem)Given an program P and an input I, to decide if the program will halt orwill run forever.

The halting algorithm

P Yes

I No

Introduction 5/29

Algorithmic Complexity (Tractability)

Comparison of several time complexity functions

𝑓(𝑛) 10 50 100log 𝑛 2.3 sec 3.9 sec 4.6 sec

𝑛 10.0 sec 50.0 sec 1.7 min

𝑛2 1.7 min 41.7 min 2.8 h  

2𝑛 17.1 min 3.6 × 105 c 4.0 × 1020  c

3𝑛 16.4 h   2.3 × 1014 c 1.6 × 1038  c

𝑛! 42.0 d   9.7 × 1054 c 3.0 × 10148 c

Introduction 6/29

Algorithmic Complexity (Tractability)

QuestionWhat can a computer do efficiently?

DefinitionA tractable problem is a problem than can be solved by a computer al-gorithm that run in polynomial time.

Introduction 7/29

Algorithmic Complexity (Tractability)

QuestionWhat can a computer do efficiently?

DefinitionA tractable problem is a problem than can be solved by a computer al-gorithm that run in polynomial time.

Introduction 8/29

Algorithmic Complexity (Tractability)

DefinitionA literal is an atomic formula (propositional variable) or the negation of anatomic formula.

DefinitionA (propositional logic) formula 𝐹 is said to be in conjunctive normal form,if and only if,

𝐹 has the form 𝐹1 ∧ ⋯ ∧ 𝐹𝑛, 𝑛 ≥ 1,where each 𝐹1, … , 𝐹𝑛 is a disjunction of literals.

Introduction 9/29

Algorithmic Complexity (Tractability)

Example (3-SAT: An intractable problem)To determine the satisfiability of a propositional formula in conjunctivenormal form where each disjunction of literals is limited to at most threeliterals.The problem was proposed in Karp’s 21 NP-complete problems (Karp1972).

Introduction 10/29

Algorithmic Complexity (Tractability)

Improvements on 3-SAT deterministic algorithmic complexity†

𝑂(1.3303𝑛) Makino, Tamaki and Yamamoto (2011, 2013)𝑂(1.3334𝑛) Moser and Scheder (2011)𝑂(1.439𝑛) Kutzkov and Scheder (2010)𝑂(1.465𝑛) Scheder (2008)𝑂(1.473𝑛) Brueggemann and Kern (2004)𝑂(1.481𝑛) Dantsin, Goerdt, Hirsch, Kannan, Kleinberg, Papadimitriou,

Raghavan and Schöning (2002)𝑂(1.497𝑛) Schiermeyer (1996)𝑂(1.505𝑛) Kullmann (1999)𝑂(1.6181𝑛) Monien and Speckenmeyer (1979, 1985)𝑂(2𝑛) Brute-force search

†Main sources: Hertli (2011, 2015). Last updated: July 2018.Introduction 11/29

Algorithmic Complexity (Tractability)

SupercomputersMachines from: www.top500.org†

Petaflops (PFs): 1015 floating-point operations per second

Date Machine PFs2018-06 Summit 122.302016-06 Sunway TaihuLight 93.012013-06 Tianhe-2 33.862012-06 Blue Gene/Q 16.322011-06 K computer 8.16

†Last updated: TOP500 List - June 2018.Introduction 12/29

Algorithmic Complexity (Tractability)

Simulation: Running 3-SAT times on different supercomputers using thefaster deterministic algorithm, i.e., 𝑂(1.3303𝑛)

Machine PFs 150 200 400Summit 122.30 32.1 sec 1.6 y 9.9 × 1022 c

Sunway TaihuLight 93.01 42.1 sec 2.1 y 1.3 × 1023 c

Tianhe-2 33.86 1.9 min 5.8 y 3.6 × 1023 c

Blue Gene/Q 16.32 4.0 min 12.0 y 7.4 × 1023 c

K computer 8.16 8.0 min 24.0 y 1.5 × 1024 c

Introduction 13/29

Algorithmic Complexity (Tractability)

Simulation: Running 3-SAT times for different deterministic algorithmsusing the faster supercomputer, i.e., 122.30 PFs

Complexity 150 200 400𝑂(1.3303𝑛) 32.1 sec 1.6 y 9.9 × 1022 c

𝑂(1.3334𝑛) 45.5 sec 2.6 y 2.5 × 1023 c

𝑂(1.439𝑛) 48.6 sec 1.1 × 105 c 4.4 × 1036 c

𝑂(1.465𝑛) 1.9 y   3.8 × 106 c 5.6 × 1039 c

𝑂(2𝑛) 3.7 y   4.2 × 1033 c 6.7 × 1093 c

Introduction 14/29

Computability and Algorithmic Complexity

Problem(TM-Computability)

⎧{{{{{{{{{⎨{{{{{{{{{⎩

Computable(decidable)problem(AlgorithmComplexity)

⎧{{{{⎨{{{{⎩

Tractableproblem

Intractableproblem

Non-computable(undecidable)problem(Hypercomputation)

Introduction 15/29

Course Outline

Language Machine Other modelsRegular DFA • Regular expressions

• NFA• 𝜀-NFA

Context-free Pushdownautomata

Recursive Halting TMs • 𝜆-calculus• Total recursive functions

Recursivelyenumerable

TMs • 𝜆-calculus• Partial recursive functions

DFA: Deterministic finite automataNFA: Non-deterministic finite automata𝜀-NFA: Non-deterministic finite automata with 𝜀-transitionsTM: Turing machine

Introduction 16/29

Paradigms of Programming

Some paradigmsImperative: Describe computation in terms of state-transformingoperations such as assignment. Programming is done with statements.

Logic: Predicate calculus as a programming language. Programming isdone with sentences.

Functional: Describe computation in terms of (mathematical)functions. Programming is done with expressions.

Examples

Imperative⎧{⎨{⎩

CC++Java

Logic {CLP(R)Prolog

Functional⎧{⎨{⎩

Standard MLErlangHaskell

Introduction 17/29

Pure Functional Programming

Side effects“A side effect introduces a dependency between the global state of thesystem and the behaviour of a function... Side effects are essentially invisibleinputs to, or outputs from, functions.” (O’Sullivan, Goerzen and Stewart2008, p. 27)

Introduction 18/29

Pure Functional Programming

Pure functions“A function may be described as pure if both these statements aboutthe function hold:”†

1. “The function always evaluates the same result value given thesame argument value(s). The function result value cannotdepend on any... state that may change as program executionproceeds or between different executions of the program, nor canit depend on any external input from I/O devices.”

2. “Evaluation of the result does not cause any semanticallyobservable side effect or output, such as mutation of mutableobjects or output to I/O devices.”

†Wikipedia: Pure function (July 28, 2014).Introduction 19/29

Pure Functional Programming

Pure functions“A function may be described as pure if both these statements aboutthe function hold:”†

1. “The function always evaluates the same result value given thesame argument value(s). The function result value cannotdepend on any... state that may change as program executionproceeds or between different executions of the program, nor canit depend on any external input from I/O devices.”

2. “Evaluation of the result does not cause any semanticallyobservable side effect or output, such as mutation of mutableobjects or output to I/O devices.”

†Wikipedia: Pure function (July 28, 2014).Introduction 20/29

Pure Functional Programming

Pure functions“...take all their input as explicit arguments, and produce all theiroutput as explicit results.” (Hutton 2016, § 10.1)

Introduction 21/29

Pure Functional Programming

Referential transparencyEquals can be replaced by equals

“By definition, a function in Haskell defines a fixed relation betweeninputs and output: whenever a function 𝑓 is applied to the argumentvalue 𝑎𝑟𝑔 it will produce the same output no matter what the overallstate of the computation is. Haskell, like any other pure functionallanguage, is said to be ‘referentially transparent’ or ‘side-effect free’.This property does not hold for imperative languages...” (Grune, Bal,Jacobs and Langendoen 2003, pp. 544–545)

Introduction 22/29

Pure Functional Programming

Referential transparencyEquals can be replaced by equals“By definition, a function in Haskell defines a fixed relation betweeninputs and output: whenever a function 𝑓 is applied to the argumentvalue 𝑎𝑟𝑔 it will produce the same output no matter what the overallstate of the computation is. Haskell, like any other pure functionallanguage, is said to be ‘referentially transparent’ or ‘side-effect free’.This property does not hold for imperative languages...” (Grune, Bal,Jacobs and Langendoen 2003, pp. 544–545)

Introduction 23/29

Pure Functional Programming

Reasoning about (pure) functional programsEquational reasoning + induction + co-induction + …

Introduction 24/29

Reading

HomeworkTo read from the textbook the following sections:§ 1.1. Why Study Automata Theory?§ 1.2. Introduction to Formal Proofs§ 1.3. Additional Forms of Proofs

Introduction 25/29

References

Brueggemann, T. and Kern, W. (2004). ‘An Improved Deterministic LocalSearch Algorithm for 3-SAT’. In: Theoretical Computer Science 329.1–3,pp. 303–313. doi: 10.1016/j.tcs.2004.08.002 (cit. on p. 11).Dantsin, E. et al. (2002). ‘A Deterministic (2 − 2/(𝑘 + 1))𝑛 Algorithm for𝑘-SAT Based on Local Search’. In: Theoretical Computer Science 289.1,pp. 69–83. doi: 10.1016/S0304-3975(01)00174-8 (cit. on p. 11).Grune, D., Bal, H. E., Jacobs, C. J. H. and Langendoen, K. G. (2003).Modern Compiler Desing. John Wiley & Sons (cit. on pp. 22, 23).Hertli, T. (2011). ‘3-SAT Faster and Simpler - Unique-SAT Bounds forPPSZ Hold in General’. In: Proceedings of the 52nd Annual Symposium onFoundations of Computer Science (FOCS 2011). IEEE, pp. 277–284. doi:10.1109/FOCS.2011.22 (cit. on p. 11).— (2015). ‘Improved Exponential Algorithms for SAT and ClSP’.PhD thesis. ETH Zurich. doi: 10.3929/ethz-a-010512781 (cit. on p. 11).

Introduction 26/29

References

Hopcroft, J. E., Motwani, R. and Ullman, J. D. (2007). Introduction toAutomata theory, Languages, and Computation. 3rd ed. Pearson Education(cit. on p. 2).Hutton, G. (2016). Programming in Haskell. 2nd ed. Cambridge UniversityPress (cit. on p. 21).Karp, R. M. (1972). ‘Reducibility Among Combinatorial Problems’. In:Complexity of Computer Computations. Ed. by Miller, R. E. andThatcher, J. W. Plenum Press, pp. 85–103. doi:10.1007/978-1-4684-2001-2_9 (cit. on p. 10).Kullmann, O. (1999). ‘New Methods for 3-SAT Decision and Worst-CaseAnalysis’. In: Theoretical Computer Science 223.1–2, pp. 1–72. doi:10.1016/S0304-3975(98)00017-6 (cit. on p. 11).Kutzkov, K. and Scheder, D. (2010). ‘Using CSP to Improve Deterministic3-SAT’. CoRR abs/1007.1166. url: https://arxiv.org/abs/1007.1166(cit. on p. 11).

Introduction 27/29

References

Makino, K., Tamaki, S. and Yamamoto, M. (2011). ‘Derandomizing HSSWAlgorithm for 3-SAT’. In: Computing and Combinatorics (COCOON 2011).Ed. by Fu, B. and Du, D.-Z. Vol. 6842. Lecture Notes in Computer Science.Springer, pp. 1–12. doi: 10.1007/978-3-642-22685-4_1 (cit. on p. 11).— (2013). ‘Derandomizing HSSW Algorithm for 3-SAT’. In: Algorithmica67.2, pp. 112–124. doi: 10.1007/s00453-012-9741-4 (cit. on p. 11).Monien, B. and Speckenmeyer, E. (1979). 3-Satisfiability is Testable in𝑂(1.62𝑟) Steps. Tech. rep. 3/1979. Reihe Theoretische Informatik,Universität-Gesamthochschule-Paderborn (cit. on p. 11).— (1985). ‘Solving Satisfiability in less than 2𝑛 Steps’. In: DiscreteApplied Mathematics 10.3, pp. 287–295. doi:10.1016/0166-218X(85)90050-2 (cit. on p. 11).Moser, R. A. and Scheder, D. (2011). ‘A Full Derandomization ofSchöning’s 𝑘-SAT Algorithm’. In: Proceedings of the Forty-third AnnualACM Symposium on Theory of Computing (STOC 2011), pp. 245–252.doi: 10.1145/1993636.1993670 (cit. on p. 11).

Introduction 28/29

References

O’Sullivan, B., Goerzen, J. and Stewart, D. (2008). Real World Haskell.O’REILLY (cit. on p. 18).Scheder, D. (2008). ‘Guided Search and a Faster Deterministic Algorithmfor 3-SAT’. In: Proc. of the 8th Latin American Symposium on TheoreticalInformatic (LATIN 2008). Ed. by Laber, E. S., Bornstein, C.,Nogueira, T. L. and Faria, L. Vol. 4957. Lecture Notes in Computer Science.Springer, pp. 60–71. doi: 10.1007/978-3-540-78773-0_6 (cit. on p. 11).Schiermeyer, I. (1996). ‘Pure Literal Look Ahead: An 𝑂(1.497𝑛)3-Satisfability Algorithm (Extended Abstract)’. Workshop on theSatisfability Problem, Siena 1996. url:http://gauss.ececs.uc.edu/franco_files/SAT96/sat-workshop-abstracts.html (cit. on p. 11).

Introduction 29/29