CHAOTIC COMPUTATION

By

ABRAHAM MILIOTIS

A DISSERTATION PRESENTED TO THE GRADUATE SCHOOLOF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT

OF THE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA

2009

1

c© 2009 Abraham Miliotis

2

TABLE OF CONTENTS

page

LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

CHAPTER

1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.1 Dissertation Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.2 Chaos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1.2.1 Logistic Map, Topological Transitivity and Period Three . . . . . . 151.2.2 The Tent Map, Topological Conjugacy and Universality . . . . . . . 381.2.3 Threshold Control and Excess Overflow Propagation . . . . . . . . . 43

1.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

2 INTRODUCTION TO CHAOTIC COMPUTATION . . . . . . . . . . . . . . . 49

2.1 Number Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492.1.1 Excess Overflow as a Number . . . . . . . . . . . . . . . . . . . . . 492.1.2 Periodic Orbits for Number Representation . . . . . . . . . . . . . . 502.1.3 Representation of Numbers in Binary . . . . . . . . . . . . . . . . . 52

2.2 Arithmetic Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532.2.1 Decimal Addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532.2.2 Binary Addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562.2.3 Decimal Multiplication and Least Common Multiple . . . . . . . . . 59

2.3 Binary Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 602.3.1 Logic Gates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612.3.2 Parallel Logic and the Half Adder . . . . . . . . . . . . . . . . . . . 642.3.3 The Deutsch-Jozsa Problem . . . . . . . . . . . . . . . . . . . . . . 67

2.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3 SEARCHING AN UNSORTED DATABASE . . . . . . . . . . . . . . . . . . . . 76

3.1 Encoding and Storing Information . . . . . . . . . . . . . . . . . . . . . . 773.2 Searching for Information . . . . . . . . . . . . . . . . . . . . . . . . . . . 803.3 Encoding, Storing and Searching: An Example . . . . . . . . . . . . . . . . 833.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

4 A SIMPLE ELECTRONIC IMPLEMENTATION OF CHAOTIC COMPUTATION 90

4.1 An Iterated Nonlinear Map . . . . . . . . . . . . . . . . . . . . . . . . . . 904.2 Threshold Control Chaos into Different Periods . . . . . . . . . . . . . . . 904.3 Electronic Analog Circuit: Experimental Results . . . . . . . . . . . . . . . 92

3

4.4 Fundamental Logic Gates with a Chaotic Circuit . . . . . . . . . . . . . . 974.5 Encoding and Searching a Database Using Chaotic Elements . . . . . . . . 1004.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

5 LOGIC OPERATIONS FROM EVOLUTION OF DYNAMICAL SYSTEMS . . 106

5.1 Generation of a Sequence of (2-input) Logic Gate Operations . . . . . . . . 1065.2 The Full Adder and 3-Input XOR and NXOR . . . . . . . . . . . . . . . . 1105.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

6 MANIPULATING TIME FOR COMPUTATION . . . . . . . . . . . . . . . . . 115

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1156.1.1 Flexible Logic Gates . . . . . . . . . . . . . . . . . . . . . . . . . . 1156.1.2 Search Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

6.2 Neural Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1226.2.1 Neural Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1226.2.2 Algorithm Implementations . . . . . . . . . . . . . . . . . . . . . . . 127

6.3 Electronic Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . 1326.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

7 CONCLUSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

BIOGRAPHICAL SKETCH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

4

LIST OF TABLES

Table page

1-1 Summary of the transitions in behaviour of the different intervals described inSection 1.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

1-2 Topological conjugacy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

1-3 Experimental measurements of Feigenbaum’s constant (δ) in different systemsbased on their period doubling. . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

2-1 Truth-table for AND, OR, XOR, NOR, NAND, NOT, and WIRE. . . . . . . . . 63

2-2 Necessary and sufficient conditions for a chaotic element to satisfy AND, OR,XOR, NOR, NAND, NOT, WIRE. . . . . . . . . . . . . . . . . . . . . . . . . . 63

2-3 Initial values, xprog, and threshold values, x∗, required to implement the logicgates AND, OR, XOR, NOR, NAND, NOT, and the identity operation (WIRE),with δ = 0.25. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

2-4 Truth table for XOR and AND logic gates on the same set of inputs. . . . . . . 65

2-5 Truth table for two AND gates operating on independent inputs. . . . . . . . . 66

2-6 Required conditions to satisfied parallel implementation of the XOR and ANDgate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

2-7 Required conditions for implementing two AND gates on independent sets ofinputs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

2-8 Examples of initial values, xprog, yprog, and thresholds x∗, y∗, yielding the paralleloperation of XOR and AND gates. . . . . . . . . . . . . . . . . . . . . . . . . . 67

2-9 Examples of initial values xprog, yprog, and thresholds , yielding operation of twoAND gates on independent inputs. . . . . . . . . . . . . . . . . . . . . . . . . . 67

4-1 Truth-table for the five fundamental logic gates NOR, NAND, AND, OR andXOR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

4-2 Necessary and sufficient conditions to be satisfied by a chaotic element in orderto implement the logical operations NOR, NAND, AND, OR and XOR. . . . . . 99

4-3 Numerical values of xprog for implementing logical operations NOR, NAND, AND,OR and XOR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

4-4 Updated state values, x1 = f(x0), of a chaotic element in order to implementthe logical operations NOR, NAND, AND, OR and XOR. . . . . . . . . . . . . 99

5-1 The truth table of the five basic logic operations NAND, AND, NOR, XOR, OR. 107

5

5-2 Necessary and sufficient conditions to be satisfied by a chaotic element in orderto implement NAND, AND, NOR, XOR and OR on subsequent iterations. . . . 108

5-3 The truth table of full adder, and necessary conditions to be satisfied. . . . . . . 112

5-4 The truth table of the 3-input XOR and NXOR logic operations, necessary andsufficient conditions to be satisfied by the map. . . . . . . . . . . . . . . . . . . 114

6-1 The truth table of each of the five fundamental logic gates, AND, NAND, OR,NOR, XOR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

6-2 Time values, in arbitrary time “units” for each of the five gates considered . . . 119

6-3 Appropriate time sample instances, based on simulation points, to perform eachof the five gates considered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

6-4 Appropriate R values so time shift an action potential in order to perform eachof the five gates considered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

6-5 Delay times for Vs to implement each of the five gates considered . . . . . . . . 134

6

LIST OF FIGURES

Figure page

1-1 Bifurcation diagram for the Logistic map. . . . . . . . . . . . . . . . . . . . . . 18

1-2 Forward and Backward evolution of F4. . . . . . . . . . . . . . . . . . . . . . . . 19

1-3 Indicative behaviour of I under multiple applications of Fμ for, (a) μ < 1 and(b) 1 < μ. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

1-4 Plots of F2 and F2.5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

1-5 Exhibition of the “trapping” of two points in two different configurations of Fμ. 28

1-6 Renormalization of two cases of F 2μ . . . . . . . . . . . . . . . . . . . . . . . . . . 29

1-7 Demonstration of birth of odd periodicity fixed points. . . . . . . . . . . . . . . 33

1-8 Plot of the function F : I→ I. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

1-9 Plot of the function T : I→ I. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

1-10 Bifurcation diagram for the Tent map. . . . . . . . . . . . . . . . . . . . . . . . 40

1-11 Topological conjugacy between evolved states, up to n = 5. . . . . . . . . . . . . 42

1-12 Logistic map bifurcation diagram for some values within 3 < μ < μ∞. . . . . . . 44

1-13 Threshold Control Mechanism. . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

1-14 Threshold values for confining the logistic map on orbits of periodicity 2 to 50. . 47

2-1 Emitted excess by thresholding the logistic map in the interval [0, 0.75]. . . . . . 51

2-2 Encoding the set of integers {0, 1, ..., 100}. . . . . . . . . . . . . . . . . . . . . . 52

2-3 Number encoding in binary format. . . . . . . . . . . . . . . . . . . . . . . . . . 54

2-4 Serial Addition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2-5 Decimal parallel addition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

2-6 The branching algorithm can be extended to a larger treelike structure. . . . . . 57

2-7 Schematic representation of the serial addition method for binary numbers. . . . 58

2-8 Schematic representation of the parallel addition method for binary numbers. . . 59

2-9 Schematic representation of the method for the Least Common Multiple of fournumbers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

2-10 Basis function T - Tent Map. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

7

2-11 Basis function T - Inverted Tent Map. . . . . . . . . . . . . . . . . . . . . . . . 72

2-12 Four realizations of the chaotic Deutsch-Jozsa algorithm for the case k = 3. . . . 74

2-13 The total excess emitted from each of the 72 functions. . . . . . . . . . . . . . . 75

3-1 The Tent map under the threshold mechanism. . . . . . . . . . . . . . . . . . . 78

3-2 Schematic representation of the changes in the state of different elements. . . . . 82

3-3 Searching for “l”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

3-4 Searching for “e”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

3-5 Searching for “x”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

4-1 Bifurcation diagram of the iterated map for various values of α and β. . . . . . 91

4-2 Graphical form of the map to be implemented by an electronic circuit . . . . . . 92

4-3 Effect of threshold value x∗ on the dynamics of the system. . . . . . . . . . . . . 93

4-4 Circuit diagram of the nonlinear device of Equation 4–3. . . . . . . . . . . . . . 94

4-5 Voltage response characteristics of the nonlinear device. . . . . . . . . . . . . . . 95

4-6 Schematic diagram for implementing the threshold controlled nonlinear map. . . 96

4-7 Circuit diagram of the threshold controller. . . . . . . . . . . . . . . . . . . . . . 96

4-8 PSPICE simulation results of the experimental circuit. . . . . . . . . . . . . . . 97

4-9 Searching for “b”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

4-10 Searching for “o”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

4-11 Searching for “d”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

5-1 Graphical representation of five iterations of the Logistic map. . . . . . . . . . . 109

5-2 Patterns of binary two input symmetric operations. . . . . . . . . . . . . . . . . 111

6-1 Schematic representation of a flexible 2-input logic gate . . . . . . . . . . . . . . 116

6-2 Construction of a “generic” signal . . . . . . . . . . . . . . . . . . . . . . . . . . 119

6-3 Schematic representation of the time based search method . . . . . . . . . . . . 122

6-4 Time Delay Unit (TDU) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

6-5 Demonstration of operating a NOR gate, with a single neuron using differentsampling times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

8

6-6 Demonstration of operating a NOR gate, with a neural circuit using differentdelay times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

6-7 Schematic representation of an electronic circuit for logic using time . . . . . . . 133

6-8 Demonstration of operating a NOR gate using an electronic circuit utilizing timedependant computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

6-9 Schematic representation of an electronic circuit for the time dependant searchmethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

6-10 Demonstration of performing a search using an electronic circuit utilizing timedependant computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

9

Abstract of Dissertation Presented to the Graduate Schoolof the University of Florida in Partial Fulfillment of theRequirements for the Degree of Doctor of Philosophy

CHAOTIC COMPUTATION

By

Abraham Miliotis

May 2009

Chair: William L. DittoMajor: Biomedical Engineering

Chaotic Computation is the exploitation of chaotic systems to perform computational

tasks. The abundance of uncountable distinct behaviours by chaotic systems, along with

their embedded determinism, position such systems as perfect candidates for developing a

new computational environment. The present dissertation focuses on algorithms developed

over the past decade within the realm of Chaotic Computation. After a brief exposition

of general Chaos Theory, we proceed to give detailed instructions for performing such

algorithms, as well as specific examples of implementations.

We begin with multiple methods for number representation and basic arithmetic

manipulations, providing from the start evidence of the flexibility of Chaotic Computation.

The compatibility with Turing machines is subsequently shown through an algorithm

for logic operations whose general form is a recurrent theme. We soon though, proceed

further than Turing machines and present a solution to the Deutsch-Jozsa problem, of

arbitrary binary functions. Even more, a practical issue is also handled by showing how

chaotic systems have a natural way for selecting matches of a searched item from within

an unsorted database.

Finally we present our latest results in handling “prolonged” evolution of chaotic

systems. Specifically we demonstrate the dominance of selecting the appropriate

behaviour, for a computational task, over being exact with specific state values, or

even confined to specific physical quantities.

10

CHAPTER 1INTRODUCTION

Over the last five decades the advancement of computational machines has been

an invaluable achievement by our society and extremely beneficial. Moore‘s law [1]

either as a self-fulfilling prophecy, or as a simple description of this progress, has been

obeyed up to now closely. The immense importance of preserving this progress has driven

many to investigate possible reasons that could hinder further increase in computational

performance.

Obviously the ultimate limits imposed by the physical nature of our world were

the first to be identified. The three most important include the speed of light limit on

transmission of information, the limit on the amount of information a finite system

can store, and the thermodynamic limit of thermal energy dissipation by the erasure of

information [2], the latter being the most relevant to current semiconductor technology.

Perhaps most important though, at this time, are the limitations inherent in

semiconductor technology, which is the basis of contemporary computers. Regularly

over the last decade the International Technology Roadmap for Semiconductors (ITRS)

has been published with the aim to indicate the tread of specific features of semiconductor

technology [3], for example lithography resolution, transistor size and connecting wire

diameter. Current technology is “relatively” far from fundamental limits for these

attributes (molecular sizes and thermal noise), but approaching them at an alarming

rate; for example Austin et. al. report a method for creating memory half-pitch size

equivalent to the size of an insulin molecule(6 nm) [4].

As the need for alternative computational paradigms is becoming progressively

evident many physical systems have been proposed as alternative basis for computational

machines. The most prominent being quantum computing [5, 6] and DNA computing

[7–10]. Even after many years of intense research though, both quantum and DNA

computing are still facing some fundamental problems. The main problem with both

11

those paradigms is that they are confined, by their nature, to a single physical realization.

In each case this confinement presents different problems that are difficult to overcome.

For quantum computing the dominant issue is that of thermal noise, which randomizes

individual qubits and leads to “decoherence” of the system. DNA computing is mainly

restricted by the time-scale with which the necessary chemical reactions take place,

making any computational task very slow in producing a result.

Regardless of the physical system used for computation it is clear that there will be

ultimate limitations in effectiveness for accomplishing any individual task. The common

theme in all alternative computational paradigms is flexibility and parallelization; this

is also pursued by conventional computer science research. The common goal is the

construction of a methodology that will provide a computational model that will be able

to solve each computational problem the most efficient way and, if possible, multiple

problems simultaneously. The main driving force to achieve such a model is the fact that

the most powerful computational machine we have, the human brain, is able to both, solve

problems in multiple ways, and handle multiple problems simultaneously.

Chaotic systems over the last few decades have attracted the attention of a large

portion of the scientific community. The main reason is the abundance of such systems in

nature and the extensive repertoire of behaviours they exhibit [11, 12]; compounded by the

fact that they are deterministic systems and can be described by a small set of equations.

More recently there has been intense research in methods to control chaotic systems,

inspired primarily by the work of Ott, Grebogi and Yorke [13]. Control mechanisms have

provided the means to confine a chaotic system to a specific behaviour. Such techniques

have been taken advantage by many fields of contemporary research including control,

synchronization, communications and information encoding [14, 15].

Many attempts have been made since before the 90‘s, and many are still active

fields of research, for bridging dynamics and computation [16–23]. Probably though

the clearest suggestion that a chaotic system, specifically, can be used for computation

12

can be attributed to C. Moore [24]. It wasn’t until late 90‘s though, that the use of a

non-feedback control mechanism on a chaotic system to perform specific computational

tasks was demonstrated [25, 26]. Over the last nine years non-feedback control has been

utilized to show how chaotic systems can perform arithmetic and binary logic operations,

and even further to solve more complex problems like the Deutsch-Jozsa problem and

searching an unsorted database [27, 28]. Non-feedback control typically is achieved either

by the use of a threshold on a state variable or by selecting specific values for the system

parameters, in either case confining a system to a specific subset of all available points.

The key idea is that from the wide variety of behaviours embedded in a chaotic

system one can find a specific pattern of behaviour that can accomplish a specific task.

Control can confine a system to the required specific behavioural pattern, without loss

of the ability to switch to a different behaviour and perform a different task. This is

accomplished by rigorous investigation of a chaotic system to identify the required

behaviours that can reliably represent computational tasks; and the state variable

thresholds, or parameter values, that will confine the system to evolve in the required

manner. This allows us to envision a computational machine which has as its building

block a chaotic system; each element being identical to all others, providing redundancy

and reliability; each element able to perform a multitude of functions, providing flexibility;

and each element independent of all others, providing parallelism of tasks.

The reason we are specific about which chaotic system to use is the Universality of

chaos [29]. Chaotic systems, even if their governing equations differ, behaviourally are

the same. This not only allows us to choose the most convenient system for theoretical

development of algorithms, but more importantly does not confine the physical realization

to a specific technology. As mentioned above, chaotic systems exist in abundance in nature

including high-speed electronic circuits, lasers, and even neurons; any of which can be used

as the building block of a chaos based computer.

13

1.1 Dissertation Overview

This dissertation can be considered to be of four parts. The first part is the remaining

of this introductory chapter, the second part chapters two and three, while the fourth

chapter can be considered to be on its own. Finally chapters five and six comprise the

final part of our exposition of Chaotic Computation.

In the remaining of this introductory chapter we provide necessary parts of Chaos

Theory so that to relate it with computation. Specifically we expose the reader to a view

of Chaos Theory from a “loose” Set Theory approach, with the intension to imply to the

reader the connections from Set Theory, to Chaos Theory, to Computation and general

Mathematical Logic. As this dissertation is concerned solely with Chaotic Computation

we do not delve in depth on the peripheral issues and direct the reader to appropriate

references [30, 31].

Chapters two and three can be considered to be the main part of this dissertation;

they are an exposition of established algorithms of Chaotic Computation [25–28, 32, 33].

We begin in chapter two with the earliest algorithms of Chaotic Computation developed

over the last nine years by Ditto et. al., algorithms for number representation, arithmetic

operations and a solution to the Deutsch-Jozsa problem. While the third chapter presents

exclusively the recent algorithm for searching an unsorted database [28].

In the relatively small third part that is chapter four, we present a recent implementation

of most abilities of Chaotic Computation with an extremely simple electronic circuit [34].

Despite its length, the aims of this part are twofold; first and foremost to give substance

of realization to the whole concept of Chaotic Computation through a physical system

implementation; and beyond, to present a concrete demonstration of the Universality of

chaos, and the ease with which we can translate our results to any chaotic system.

The final part is our most recent results, the “cutting edge” developments in Chaotic

Computation [33]. Chapter five, while focuses solely on binary operations, introduces

the importance of what can be considered to be “the time dimension”; in addition it

14

expands on our treatment of “state values”, by showing more concretely the importance

of specific behaviours over exact state values. Finally chapter six expands further the

idea of manipulating time. Through demonstrations, with electronic circuits, and mainly

neural circuits we exhibit implementations of Chaotic Computation algorithms using “time

instances” as the medium for computational commands.

We wish to the reader the encounter with this exposition to be both informative and

enjoyable.

1.2 Chaos

Chaos Theory is the third major “revolution” of Physics of the 20th Century. Even

though its “birth date” can be placed as contemporary to Relativity and Quantum

Mechanics, circa 1900, it wasn’t until much later, in the early 1960s, that the field actually

attracted enough attention to attain “critical mass”, a slightly American biased popular

science recount of the history of Chaos Theory can be found in [35].

This dissertation is not about Chaos Theory as such, but more about some of the

“features” of Chaos Theory. We utilize much of what is defined, predicted, and expected

from Chaos Theory to propose a whole new realm for computation. The problem is that

Chaos Theory is a very closely packed theory, with each part relating to some other part

and the whole to the details (a concept which is a result of the theory as well). We will

present the “features” of Chaos Theory that are necessary for projecting to the reader our

results and direct the reader to: Devaney (1982) [36] for a solid mathematical exploration

of Chaos and the origin of one of the most accepted definitions of Chaos; Peitgen, Jugens

and Saupe (1992) [37] for a more hands on demonstration with heavy emphasis on fractals,

and Ott (1993) [38] for a midpoint approach to Chaos Theory, with the addition of an

exposition of Quantum Chaos.

1.2.1 Logistic Map, Topological Transitivity and Period Three

In this introductory chapter we provide a somewhat “non-traditional” view of chaos.

The approach we take is one closer to a mathematician‘s rather than a physicist‘s, since

15

computation, our main concern, is more of a mathematical concept than physical. We

present chaos through in a “loose” set theory context, that is we treat topological spaces,

intervals and even state variables as sets of points, we try to avoid any “linearizing” tool,

like differentiation, and any representation that involves discretization of our set of points,

like statistical manipulations. At the same time our exposition is “loose”, since we do

not go into extreme formal mathematical precision, that is we omit providing proofs and

extensive definitions of terms used. We direct the reader to two seminal sources, Principia

Mathematica of Whitehead and Russell [30] and Kurt Godel‘s ever important paper,

“On formally undecidable propositions of principia mathematica and related systems”

(1931) [31], for both more background information of our approach, especially the missing

mathematical details, and more importantly for justification for our approach and how it

relates to the concept of computation.

Our main “tool” for presenting our necessary parts of Chaos Theory is the discrete

time Logistic map:

F n+1μ (xn) = xn+1 = μ ∙ xn ∙ (1− xn), (1–1)

where x is the state of the system, n1 is the time step and 0 < μ can be thought of as a

growth rate. Actually the origins of this quadratic equation is from population dynamics,

specifically it is modeling the behaviour of a population with limited resources. In that

context x represents the current fraction of the population with respect to the maximum

possible sustainable population (x = 1). In its continuous-time form, the model can be

solved analytically and gives rise to the most common sigmoid function, a “well behaved”

function of continuous growth and eventual saturation2 . When we consider discrete time

1 We shall drop the superscript for the cases of n → n + 1, unless we wish to emphasisethe use of a single application of Fμ.

2 A theorem by Poincare and Bendixson [39] guarantees that continuous time twodimensional planar systems can not exhibit chaos.

16

though, for example n denoting the current generation of beings in our population, we

have Equation 1–1 and the behaviour of x is no longer simple continuous growth leading

to eventual saturation; in fact in this discrete case if the population does manage to reach

saturation it collapses to zero.

For our purposes we need a closer look at Equation 1–1. With a first glance it is

clear the function is governed by the three variables x, μ, n, each in principle unbounded.

There is no need though to go to such lengths, as we can confine ourselves to the domains

0 ≤ x ≤ 1, 0 < μ ≤ 4, 0 < n. This is truly a very small portion of the whole available

space, so briefly lets justify this confinement, and we will provide more justification as we

focus more in the above domains. Starting with 0 < n, we can set the “beginning of time”

at any “moment in time”, we will actually see below that this is totally arbitrary; actually

for some results of Chaos Theory reversing the arrow of time, n→ n− 1, is very useful. In

the case of μ things are a bit more complicated, except for the case μ = 0; for μ < 0 we

can show the same behaviours as for 0 < μ; for 4 < μ most x values escape to infinity and

the ones that are “trapped” form a Cantor set which is the prototypical fractal and, again,

shares many of the “features” that are present in 0 < μ ≤ 4, subsequently we will confine

μ further. Finally x, with the other two variables confined to the above domains, we have

x < 0 and 1 < x escaping to infinity or eventually collapsing to zero.

Therefore our concern is what happens to a state x ∈ [0, 1] for values of μ ∈ (0, 4], in

the “forward” direction of time n → n + 1. An immediate view of the behaviours in this

domain can be obtained through a bifurcation diagram, see Figure 1-1. Such a diagram is

obtained by evolving an initial x0 state,123 in this case, for a large number of time steps

and plotting the state at the final few time steps, in this case we evolved the map 500 time

steps and plotted the last 200 states. From this empirical approach we can already see the

3 Using the critical point of the map we are guaranteed to evolve into an attractingstate, if one exists.

17

richness that this simple map provides, at least three behaviours are evident: collapse to

zero, attraction to a non-zero fixed point and attraction to a periodic orbit.

Figure 1-1. Bifurcation diagram for the Logistic map. Eventual behaviour of an initialstate x0 under repeated applications of Fμ.

This static picture of what happens, even though extremely rich, is a generalization

of the details of the behaviours of x. For example it should be clear, but it is not clearly

shown, that this picture is “almost” completely invariant to the initial x0, that is what

happens to a state x ∈ [0, 1] happens to “almost” any other initial state, the map has

a local effect that is global, and visa versa. It is not clear though what happens at the

onset of chaos, when “almost” all points behave the same way and at the same time all

the “rest” of the points behave in a different way!

To see the details inside the bifurcation diagram we take a more “dynamic” approach

with respect to the three available variables. Starting with n we will consider the question

“Where do states come from?”, we are not going to address the issue of “time reversal”,

18

but rather take a step “back in time” and consider:

x0 = μ ∙ x−1 ∙ (1− x−1) = Fμ(x−1), (1–2)

where x0 ∈ [0, 1] and 0 < μ ≤ 4. Figure 1-2 shows two plots of the function F 14 each

showing how the function evolves in either “direction of time”.

Figure 1-2. Forward and Backward evolution of F4. (a) Forward evolution - Empty circle,#, marks an arbitrary initial point x0; we can track its forward evolution bymoving “up” (or “down”) to meet the function F4 and then “right” (or “left”)back to the diagonal, each full cycle of these two steps is equivalent to a singleapplication of F4; thus the full circle, , marks the point F 54 (x0) = x5. (b)Backward evolution - The steps of the forward evolution can also be reversed;i.e. from an arbitrary initial point x0, empty circle #, we can move “left” and“right” (or “left” twice) to the two points on F4 and then “up” and “down”(or “down” twice) to meet the diagonal on the points of F−14 (x0) = {x

+−1, x

−−1},

the two empty squares, o; and of course further back to the four points ofF−24 , the four full squares, n, and so on.

By solving Equation 1–2 we observe the following: by setting x0 = x−1 we find the

fixed points at x∞ = 0 and x∞ = 1− 1μ ; now setting x0 = 0 and x0 = 1−1μwe find for each

fixed point its two pre-images, specifically x−1 ∈ {0, 1} → 0 and x−1 ∈ { 1μ , 1−1μ} → 1− 1

μ;

19

and finally by solving the quadratic for any x0 we reach x−1 =12±√μ2−4∙μ∙x02∙μ , which leads

to μ4< xn ⇒ xn−1 ∈ C, that is ∀x >

μ4, F−1μ (x) ∈ C; the pre-images of any x value greater

than a fourth of the “growth factor” (μ), of the applied map, are in the complex plane.

This is an interesting result and the start of another long story, it shows how the dynamics

of the logistic map extend in the complex plane. This is beyond our current scope, but in

passing we note that the logistic map is a reduction of the quadratic maps Qc(z) = z2 + c

where z, c ∈ C, the source of the famous Mandelbrot and Julia fractals [40], this is an

initial hint to the concept of Universality, which we will address in Section 1.2.2.

We are now ready to begin to investigate how μ changes the behaviour of x ∈ [0, 1].

We can not explain what happens at every value of μ so we consider the change before

and after a point of transition. Clearly at μ = 1 we have a major change, the fixed point

x∞ = 1 − 1μenters the domain [0, 1] and becomes an attractive fixed point, actually the

“big event” at this μ-value is that the two fixed points cross paths. At the same time 0

becomes a repelling fixed point; these conclusions can be drawn by taking the absolute

value of the derivative of the map at the fixed points,∣∣∣dF (x∞)dx

∣∣∣, a standard technique for

characterization of dynamical systems. This is though a local linearization of the map, we

prefer to show this, and subsequent transitions, in terms of global effects, thus prepare the

ground for what happens at higher values of μ.

We need to deviate for a while to single out some special points on the I = [0, 1]

interval. We already seen the four points that map to the fixed points, {0, 1, 1μ, 1 − 1

μ},

and we add to this list {14, 12, 34}. In general 1

2is the most important point of Fμ, not

because of its value, but since it is the critical point of the map and its backward and

forward evolution can actually characterize the map extensively; the theory behind the

evolution of the critical point is called kneading theory [36]. The importance of the

points 14and 3

4is topological, following Fμ(

14)�Fμ(12) =

34and of course Fμ(

14) = Fμ(

34),

in addition to Fμ(12) = sup Fμ(I), visually this means what will happen to the interval

[14, 34] → F n+2μ ([1

4, 34]) is what happened previously to the interval [ 3

4, 1] → F n+1μ ([3

4, 1]) and

20

what is going to happen to [ 34, 1] → F n+1μ ([3

4, 1]) is what happened to [0, 1

4] → F n+1μ ([0, 1

4]),

in a normalized sense. Consider the interval [0, 1] separated in three parts I1 = [0,14),

I2 = [14, 34], I3 = (

34, 1]. No matter the μ-value we know the following: Fμ(0) = 0,

Fμ(14) = Fμ(

34) = 3∙μ

16, Fμ(

12) = 1∙μ

4and Fμ(1) = 0 so Fμ({0, 14 ,

12, 34, 1})→ {0, 3

4∙(14∙μ), (1

4∙μ)},

divide out by a factor of ( 14∙ μ) and its like nothing happened! We need to emphasise that

this picture is only for visualization purposes and it applies only to a single application of

the map, a “snapshot” if you like that we can keep track of while iterating n → ∞, below

we will discuss the proper way to view evolution of intervals, the renormalization operator.

Returning to the question “What happens as μ < 1 → 1 < μ?”. There are multiple

ways we can analyse the situation, we look at it with respect to changes to the behaviour

of the aforementioned intervals. Globally before the point of transition (μ < 1) we simply

have Fμ(I) < I so its clear that in the limit n → ∞ the whole interval will collapse to

zero. To “see” the change at the transition consider Fμ(I1) → In+11 ∪ In+12 , that is I1 will

become the union of what can be considered to be the I1 and I2 of the next application

(n + 1) of the map, at which F n+1μ (In+12 ) → In+23 and F n+2μ (In+23 ) → In+31 ∪ In+32

completing the “circle”. Actually it is a double spiral with a moving pivot, the sequence

of post-images of 14and 3

4denoted as ∗x. So we have at μ < 1, Fμ(

12) < 1

4, which implies

that F nμ (I2) ∩ I2 = �, this is enough in some sense to characterize the evolution since we

have seen that in some sense both I3 and I1 end up, at least partly, in an interval that can

be called Im2 , another way to describe this evolution is Fnμ (∗x) = 0 as n → ∞. With this

description of the evolution of Fμ for μ < 1 we can see what changes at μ = 1, specifically

Fμ(I2) ∩ I2 6= � and at the single point magnification F1(12) =14. The consequence of this

change can be expressed in multiple ways: since the post-images of In1 and In3 are in I

n+12

it means that the relationship Fμ(I2) ∩ I2 6= � can be extended to F nμ (I2) ∩ In−12 6= �

even as n→∞ so I∞2 contains a single point, which is the eventual post-image of all these

points that get “trapped” in the sequence of Im2 ; or simply as n → ∞, Fnμ (∗x) → 1 − 1

μ

(= 0, for μ = 1); and even though the “pivot” point does converge to zero eventually, there

21

will always be at least one more “point” between its value and 0, these “points” we will

see are cental to what is chaos. So now an increase in μ toppling it over 1 separates 0 and

1 − 1μby a more “concrete” amount, giving substance to the fixed point at 1 − 1

μ, and to

the points inside the interval (0, 1 − 1μ), for a view of the evolution of points in either of

these two regions (μ < 1 or 1 < μ) see Figure 1-3. We will come across transitions like the

one we just described an infinite number of times going from μ = 1 to μ = 4.

Figure 1-3. Indicative behaviour of I under multiple applications of Fμ for, (a) μ < 1 and(b) 1 < μ. (a) (μ = 1

2) The two fixed points are one at 0, and the other outside

I. (b) (μ = 112) 0 remains a fixed point and now 1 − 1

μ∈ I; with one pre-image

being itself at 1− 1μ= 13and the other at (μ

4<) 1μ= 23. Empty circles, #, mark

initial points and full circles, , final points (not the fixed point).

Now we are in the 1 < μ region and, even though the bifurcation diagram doesn’t

show it, the next point of transition is at μ = 2. We have seen how the behaviour of

the whole interval I changed at μ = 1, and with it we should change our view of its

subintervals. We could continue the discussion with I1, I2 and I3, and we will return to

them when necessary. Our aim though is to make the behaviour of the whole interval I as

22

visual as possible, and so given that now 1 − 1μexists in I (as well as both its pre-images

{1 − 1μ, 1μ}) we have a better choice of subintervals to use, specifically J1 = [0, 1 − 1

μ),

J2 = [1 − 1μ, 1μ], J3 = (

1μ, 1]. As we have done for the previous transition, first we will

see how the interval I behaves with 1 < μ < 2. A quick side-note, taking the derivative

at the fixed points we find the “nature” of 0 to be repelling and 1 − 1μattracting. The

details though make all the difference, as we have defined the intervals Ji we see that

F 2μ(J03 ) = J

11 = F

1μ(J

01 ). The interval J3 gets mapped onto J1 and J1 onto itself, so it seems

that J1 is not changing and even more it seems that J1 and J3 are “disconnected”4 from

J2. So it seems we have three types of points, the set of four points {0, 1− 1μ ,1μ, 1}, and the

two intervals (1 − 1μ, 1μ) and (0, 1 − 1

μ) ∪ ( 1

μ, 1), the easy part is the fixed points and their

pre-images Fμ({0, 1}) = 0 and Fμ({1 − 1μ, 1μ}) = 1 − 1

μand in addition, since μ

4< { 1

μ, 1},

we know all other pre-images of the fixed points are in the complex plane. For (1 − 1μ, 1μ)

since Fμ : (1 − 1μ, 1μ) → (1 − 1

μ, 14∙ μ) and 1 − 1

μ< 14∙ μ < 1

μwe see that all points in the

limit n → ∞ are “squeezed” closer and closer to 1 − 1μ. Finally for (0, 1 − 1

μ) ∪ ( 1

μ, 1), the

simple picture is that ∀x ∈ (0, 1 − 1μ) and ∀y ∈ ( 1

μ, 1), F nμ (x) > x so as n → ∞, x → 1 −

1μ

and as we have seen ∀x : Fμ(y) = x, so F n+1μ (y) → 1 − 1μ, therefore every point moves

away from zero (or one) and towards 1 − 1μ. We have though seen that Fμ(J1) = J1, note

that ∀x ∈ J1, x <μ4so every point in J1 has two real pre-images, one of them is of course

in J3 and it has no pre-images of its own, but the other one is in J1 and x−1 < x0 so the

pre-image also has two pre-images, and so on. So the complete picture of the evolution of

J1 is that each x0 does evolve to x1(> x0) and eventually, at n → ∞, to 1 − 1μ, but on

every application of the map it is “replaced” by two other points; in other words, points

from the neighbourhood of 0, and of course 1, eventually escape this neighbourhood and

evolve into the neighbourhood of 1 − 1μ, 0 is a repelling fixed point and 1 − 1

μattracting.

4 “Disconnected” here is used as in every day language, in fact since F nμ (J3) → 1 −1μas

n→∞ the intervals are of course still connected.

23

As soon as we cross from μ < 2 into 2 < μ we shall see more clearly the importance in

considering this behaviour.

At μ = 2 we reach the second transition point, 1 − 1μ= 1μ= 12, in some sense as

at μ = 1 we had 1 − 1μ= 0, two of the points that lead to a fixed point cross paths;

the difference in this case is two points that lead to the same fixed point (1 − 1μand 1

μ).

Before we go on to the exciting changes that happen in 2 < μ, we note in passing that at

μ = 2 the derivative at the fixed point is 0 and so 12is called “super-attractive” point; 1

2

has no pre-images real or imaginary, in a sense all points (even from the complex plane)

converge to 12; more importantly though, it is the only point in J2 as a result we now have,

in addition to F2(J3) = F2(J1) = J1, F2(J2) = J2, that is all our intervals are “eventually

invariant”5 , see Figure 1-4(a) for a view of the behaviour at μ = 2.

We are now in the 2 < μ region; there are two important general changes in

behaviour, first of the fixed point acquires now an infinite number of pre-images, since

1μ< μ

4, and as a consequence, topological “mixing” is now possible, that is any closed

interval contains points that have pre-images, or even post-images, outside the interval,

the set of points that lead to the fixed point has more than one point; more visually notice

that J1 ∩ J2 ∩ J3 6= �, see Figure 1-4(b). This leads us to the first concept that is a

requirement for chaos, topological transitivity.

Definition 1.1. Topological Transitivity.

Given a map f : J → J , f is said to be topologically transitive on J , if for any two

open subintervals U , V ⊂ J there exists a x ∈ U and a y ∈ V such that fn(x) = y, for

some 0 < n.

In other words, given an interval any point in it can, eventually, be mapped on any

other point in the interval. This is not active for values 2 < μ < 3.5699456..., but the fact

5 We shall see that the transition from “eventually invariant” to invariant makes all thedifference.

24

that we have intervals “mixing” is the first step, it is the difference between “eventually

invariant” and invariant. Before we proceed let us redefine our intervals so that they are

not “mixed” before we even apply the map and to keep the picture of how the intervals

evolve as it has been up to now. So we re-define J1 = [0,1μ), J2 = [

1μ, 1− 1

μ], J3 = (1− 1

μ, 1],

basically acknowledging the fact that the pre-images of the fixed point (1 − 1μ) have

switched sides around 12, see Figure 1-4(b) for the “mixing” of the Ji intervals and their

redefinition as Ji, where i ∈ {1, 2, 3}. The behaviour of these intervals is straightforward,

we still have Fμ(J3) = Fμ(J1), but now Fμ(J1) = J1 ∪ J2 and Fμ(J2) = J6 ⊂ {1 − 1μ} ∪ J3.

Visually we have J3 stretched and rotated around 1− 1μ onto J1 ∪ J2, J1 is simply stretched

onto J1∪J2 and J2 folded inside itself, with 12 as the pivot of the folding, and again rotated

around 1 − 1μto land inside J3; overall all points are spiralling around and inwards 1 − 1

μ,

which is the situation we had at μ < 1, so it should come as no surprise what happens at

μ = 3.

Figure 1-4. Plots of F2 and F2.5. (a) Display of the “super-attractive” case when12is the

fixed point. (b) Display of the six intervals {J1, J2, J3} and {J1, J2, J3}, theoverlap of Ji shows the initiation of topological “mixing”.

25

As the points are spiralling inwards the fixed point, and μ is gradually raised we have

in J3 what was happening to I, the points collapsing towards the fixed point, until there is

a point left out, specifically when Fμ(12) ≥ (1 − 1

μ) + 1

4∙[1− (1− 1

μ)]6 ; or more visually

when the stretching of J3 (and J1) “creates” more points than the “available” points in

J1 ∪ J2 and so “a” point is left out that will never spiral on 1 − 1μ7 . All this happens at

μ = 3 and, as at μ = 1, the point left out is a post-image of 12only this time we have, for

0 < n, F 2n−13 (12) ∈ J3 and F 2n3 (

12) ∈ J2 as n → ∞, so we have a fixed point created in J3

and one in J2, thus initiating the famous period-doubling route to chaos. Before we follow

this route, we need to note what happens to 1 − 1μour “current” fixed point. Just as at

μ = 1, the fixed point at 0 changed form from attracting to repelling, in the same manner

the nature of 1 − 1μchanges, only with an important difference as we have already noted,

unlike for 0 the number of pre-images for 1 − 1μis infinite, but at least countable, for now.

It is time to provide the promised further justification of neglecting the regions 4 < μ

and μ < 0. The creation of the two fixed points by “entrapment” is exactly what happens

to points on I for 4 < μ, with the slight difference that the trapped points for 4 < μ

collapse to 0, basically the top part of the parabola is protruding over some value on

which the two points that happen to be on, are captured, Figure 1-5 displays this process.

For μ < 0 consider F 23 (J2), it is an inverted parabola, differing from F10 for 0 < μ only

by a shift of 23; and a small curvature deviation caused by the higher power terms of x,

see Figure 1-6(b). In fact we can take F 23 , or for any μ for that matter, and using the

renormalization operator translate it to a form for I, as follows:

Define:

x∞ = 1− 1μand x∞ =

1μ.

6 Compare with F1(0.5) ≥ 0 + 14∙ (1− 0).

7 This is really a simplistic picture as it can be argued that there are always enoughpoints in an interval, but its visual aid is of great value.

26

Lμ(y) =(y−x∞)x∞−x∞

and L−1μ (y) = x∞ + (x∞ − x∞) ∙ y.

We have:

RF (x) = Lμ(F 2μ(L−1μ (x)

))= Lμ ◦ F 2μ ◦ L

−1μ (x).

Where RFμ is the renormalized function of F2μ that translates J2 = [

1μ, 1 − 1

μ] onto

I = [0, 1], both by reflecting it upwards and rotating around so that 1 − 1μ= 0 and 1

μ= 1,

see Figure 1-6 for two explicit examples of renormalization, relating to the issues of μ < 0

and 4 < μ. This should make it clear how everything that happened to F 1μ will happen

to F 2μ ; of course appropriate Lμ and L−1μ can be found for all n, but this is beyond our

current scope, see the work of Kenneth G. Wilson [41, 42]. This sidetrack8 was intended

to show how the actual value of μ is very “relative”, as well as the actual interval of x; or

to be more precise the actual set of x values.

Returning to the 3 < μ region and how the complexity of behaviours increases

exponentially with increasing μ. At the moment we have two attracting fixed points, each

with its two pre-images, one of them being the other fixed point and the other pre-image

has its own two pre-images and so on. The process we have explored up to now will

repeat, in fact infinite times, as μ is raised further. We will have a “super-attractive” case,

when F 2μ(12) = 1

2, at μ ≈ 3.23..., and of course the trapping of yet another two points for

each one of the “current” two cycle fixed points, and thus the creation of the four fixed

point cycle, at μ = 1 +√6 ≈ 3.44... and of course the change of the two cycle fixed points

into repelling points. The four cycle will become eight cycle and so on for all powers of 2.

To our visual picture, we still have the action of the primordial fixed point, 1 − 1μ, even

though now, at 3 < μ it is repelling, its pivot action in the spiraling-in still exists. In

addition we have more “local” effects from the newborn fixed points acting as pivots for a

spiralling-in of their local surrounding points. In effect we have vortices within vortices.

8 Another reason for this diversion is to provide a small glimpse, without going intodetails, to the picture of F 2

k

μ∞with k →∞, which we will meet soon.

27

Figure 1-5. Exhibition of the “trapping” of two points in two different configurations ofFμ. (a) Plot of both F

13.2, solid line, and F

23.2, dashed line showing the eventual

evolution of different initial points, marked by #, to the 2-cycle fixed pointsmarked by . The solid triangles, K, mark the sequence between the two fixedpoints. Note that F 2 and the diagonal can be used like F 1 in Figure 1-2 totrack the 2∙n-step evolution of points. (b) Plot of F 1

2+√5as well as the

evolutions of some different initial points, marked by #, under multipleapplications of F2+

√5. Note that the two points marked by , are “trapped”

by F 12+√5, while their four pre-images, marked by n, are “trapped” by F 2

2+√5,

and so on to F∞2+√5, and even further to F2+

√52∞ .

As more and more fixed points are created the gaps between them for all this swirling

to take place are getting smaller and smaller, leading to smaller and smaller increments

of μ to create the subsequent doubling, we will return to this evolution in Section 1.2.2

when we talk about Universality. For now we are interested in what happens at the

accumulation point, also know as Feigenbaum point, named after its discoverer Mitchell

Feigenbaum, μ∞ ≈ 3.569945671.... Let us attempt to account all the points in I and their

behaviour starting with the easiest part {0, 1} the two points that collapse to 0. We have

the fixed point of F 1μ , that is 1−1μ, with it we still have its pre-image 1

μand “going further

28

Figure 1-6. Renormalization of two cases of F 2μ . (a) Case of F23.8, for which

L3.8 ◦ F 23.8 ◦ L−13.8(x) translates into (b) A plot of RFμ(x), showing how

representative points, marked by symbols, are translated onto points of similartopology. It should also make clear how cases of 4 < μ are “contained” in someform in 0 < μ ≤ 4. (c) Case of F 23 , for which L3 ◦ F

23 ◦ L

−13 (x) translates into

(d) A plot of RFμ(x), showing how representative points, marked by symbols,are translated onto points of similar topology. It should also make clear howcases of μ < 0 are “contained” in some form in 0 < μ ≤ 4.

back time” all the points that lead to it, as we mentioned infinite number of them. From

F 2μ we have the two fixed points, of course different from 1 −1μand 1

μ, each of these two

fixed points has the other one as one of its pre-images, but in addition each point has one

more pre-image, which of course has its own pre-images, once again in infinite number.

Continuing this for each F nμ with n = 2k as k → ∞ we can account for all fixed points of

periodicity a power of 2, as already explained above through the period-doubling route,

and all their pre-images. With all these points accounted for, it might seem we have run

out of points for F 2k

μ∞at k → ∞, to have any fixed points, in fact though we still have just

as many points as we accounted for, we have a Cantor set once again and technically our

fist instance of chaos. The details of the behaviour at the Feigenbaum point (μ∞) of F2∞

μ∞,

29

and for that matter of F 1μ∞ , are complicated to describe, both visually and mathematically,

without the introduction of symbolic dynamics, further exposition of Cantor sets and

renormalization group operator, concepts which are beyond our current scope. Instead

we will proceed in increasing μ further and show how F 1μ becomes chaotic on the whole

interval I = [0, 1] at μ = 4, at which point the behaviours are the same as for F 2∞

μ∞at μ∞,

but easier to describe and visualize.

We utilize the period doubling route to the Feigenbaum point to introduce the second

requirement for chaos, fixed points of periodicity three. The original definition a chaotic

system by Devaney [36] required of a system: (i) Topological Transitivity, (ii) the set of

periodic points to be dense in the space of the system, and (iii) sensitive dependance on

initial conditions. Banks et. al. [43] though, showed how transitivity and density of the

set of periodic points imply sensitive dependance on initial conditions, while Li and Yorke

[44] proved that existence of period three fixed point implies the set of periodic points is

dense; in fact they also show how the set of “non-periodic” points is also dense. Before Li

and Yorke though, it was Sharkovsky [45, 46] that introduced his famous theorem9 that

actually claims more than just period three implies all other periodicities, it actually

provides an ordering of the natural10 numbers that will guide us to find fixed points of

periodicities other than a power of 2.

Theorem 1.1. Sharkovsky’s Theorem.

Given a continuous map f : R → R, the ordering of the natural numbers: 1 C 2 C

22 C 23 C ... C 2n C ... C 7 ∙ 2n C 5 ∙ 2n C 3 ∙ 2n C ...

... C 7 ∙ 2 C 5 ∙ 2 C 3 ∙ 2... C 9 C 7 C 5 C 3,

9 Even though the theorem applies only for maps on the real line, its strong conclusionmakes it important to note in any exposition of chaos.

10 Actually the ordering is for more than just the Natural (N) numbers, it covers asmany periodicities as all the Reals (R) [47].

30

and if the map has a fixed point of periodicity q, where p C q, then the map also has

a fixed point of periodicity p. If the map does not have a fixed point of periodicity p, where

p C q, then the map does not have a fixed point of periodicity q. Further, for any p a

continuous map does exist which has fixed points of periodicity p, but no fixed points of any

periodicity q.

The most obvious implication of Sharkovsky’s theorem, which is analogue to the

Li-Yorke theorem, is that if a map has a fixed point of period three then it has fixed

points of all number periodicities. Of course the specification of the ordering allows us to

say more, if a map has a finite number of periodic points then they are all a power of 2,

and the converse if a map has a periodic point that is not a power of 2 then it has infinite

number of periodic points. The second part of the theorem allows for the existence of

maps with an infinite number of points of distinct periodicity and yet without all natural

number periodicities. Actually the existence of such maps can be utilized in the proof of

the theorem to establish this number ordering, and perhaps more importantly to guarantee

the success of the threshold control mechanism for chaotic systems, see Section 1.2.3.

Furthermore there is a corollary to Sharkovsky’s theorem, which states that if the

map f is dependant on a varying parameter the ordering of the “birth” of new period

fixed points is given by the Sharkovsky ordering, which brings us back to Fμ and the

further changes as we increase μ. We have already seen the birth of all periods of power

of 2 up to the Feigenbaum point, our next step is to show the birth of all other fixed

points of even periodicity by increasing μ over the Feigenbaum point to μ2 ≈ 3.67...11 and

showing how this region of μ allows only even periodicities.

Readjusting the intervals under observation we define J4 = [F2μ(12), Fμ(

12)], or more

clearly using Fμ ≡ h and h(12) =μ4, J4 = [h(

μ4), μ4]; in addition we define J5 = [h(

μ4), 1 − 1

μ]

11 The algebraic expression for μ2 =23[ 4u+u+1], where u = (19+3

√33)

13 ; obtained either

by F 2μ(12) = 1− 1

μor sup {F−1μ (

1μ)} ≥ μ

4.

31

and J6 = [1 − 1μ, μ4], therefore we have Fμ(J5) ⊂ J6 and Fμ(J6) ⊂ J5, for μ∞ < μ < μ2.

Before focusing on these intervals let us consider the rest of the points in I, of course

Fμ ({0, 1}) = 0 nothing new there; Fμ : (μ4, 1) →

(0, h(μ

4))and F nμ :

(0, h(μ

4))→ J4

as n → ∞. So it is clear that all points in I eventually are mapped inside J4, once

there we have ...J5 → J6 → J5... so it is impossible to have a point that is mapped

back on itself after an odd number of applications of the map, furthermore points of all

even periodicities are possible and exist. At μ2 we can see what happens to F2μ and it

is precisely what has happened to each other even periodicity while increasing μ from

the Feigenbaum point, with a small but important difference in the behaviour of 12.

Specifically F 2μ2(12) = 1

μthat is 1

2is mapped to the pre-image of the primordial fixed

point, so 12is a pre-image of a fixed point, but not itself fixed as in the “super-attractive”

case. Visually this means that there is at least one interval that is completely wrapped

around onto itself, in contrast to previously when 12was “eventually” mapped onto a fixed

point. We can also now see the difference between “eventually invariant” and invariant,

F 2μ2(J5) = J5 and F2μ2(J6) = J6, note the equality, and see Figure 1-7(a) for a visual of

the intervals. Perhaps most importantly both pre-images of 1μare now less than μ

4, which

implies that both have two real pre-images, and at least two of those pre-images each have

two pre-images of their own, and so on; the fixed point 1 − 1μhas now a set of pre-images

which form a Cantor set. When we put all these implications together, along with the fact

that the Sharkovsky ordering for F 2μ is the same as the ordering for F1μ without the last leg

of the odd integers (so all number periodicities exist for F 2μ2), it should be clear that F2μ2is

chaotic. It is important to note that F 2μ2 is chaotic on J4 not the whole I.12

Therefore now we are left with the last leg of the Sharkovsky ordering, the odd

number periodicities. Obviously the most important being period three and the easiest

one to visualize since it is relatively “short”. But before we focus on period three lets see

12 Actually F 2μ is independently chaotic on J5 and J6.

32

how odd periodicities become possible in the first place. So at μ2 < μ we have seen that

F 2μ2(12) < 1

μ, so F 2μ(J2) ∩ J1 6= ∅ or to relate two sets from different views J4 ∩ J1 6= ∅, lets

call this intersection J7 ⊂ J5, most importantly J7 < 1μ⇒ Fμ(J7) < 1 − 1

μ⇒ Fμ(J7) ⊂ J5

so we have J7 → J5 →...→ J6 → J5 → ...︸ ︷︷ ︸

infinite9 even times

→ J6 → J7, hence the “highest” odd

periodicity orbit is “born”, see Figure 1-7(b).

Figure 1-7. Demonstration of birth of odd periodicity fixed points. (a) Plot of Fμ2, solidline, and Fμ

22, dashed line. We also mark the intervals J4, J5 and J6, it should

be clear that Fμ2 : J5 → J6 and Fμ2 : J6 → J5; Fμ22 is chaotic on both these

intervals. Note also that Fμ2 : I �{0, 1}�J4 → J4. (b) Plot of Fμ, solid line,and F 2μ , dashed line, at μ = μ3 + 0.02, a value for which the period three fixedpoints are attractive. Note how J5 ∩ J1 = J7 6= ∅. Within J7 we have one ofthe period three trapping regions, see inset. (c) Inset, showing a plot of F 3μ(x),for μ = μ3 + 0.02 and x ∈ [0.1358, 0.1722], topologically the same as someFμ(x), for x ∈ I.

9 Or none in the case of period three.

33

We return now to the issue of the birth of the fixed points of periodicity three for F 1μ .

At μ3 = 1 + 2√2 14 we have the birth of the period three orbit, the simplest way to view

it is that the solutions to the equation F 3μ − x = 0, become real from complex. In terms

of our visual picture, we have a growth factor which is large enough for 12to fall outside

the “trapping” region. The situation is very similar to what happened at μ = 1 when we

had the introduction of a “new” fixed point in I by F1(I2) ∩ I2 6= �, only this time the

set of this intersection does not contain a post-image of 12, F1(

12) /∈ F1(I2) ∩ I2, so this set

contains some other point or more correctly points. Since now it is not the maximal point

that gets captured it means that the intersection can not be collapsed to a single point,

but there will always be at least two points, marking the boundaries of this intersection

interval, and since at least both these points are “trapped” we have two new fixed points,

for each trapping region. Once we consider that we are dealing with F 3μ , there are three

“trapping” regions, so a total of six new fixed points, along with the two primordial,

{0, 1 − 1μ}, eight in total agreeing with the fact that F 3μ is an eighth degree polynomial

15 .

The nature of these points is one of them attracting and the other repelling, since we are

talking about an interval being “trapped” the points that actually exist between the two

fixed points are also “trapped” by eventually collapsing to the attracting fixed point. So

the overall situation in a trapping region is “identical” to the situation we had in I for

1 < μ, see inset (c) of Figure 1-7. Therefore it should be of no surprise that everything

that we have talked about up to now for I happens within each “trapping” region over and

over again as we increase μ even further; in some sense this is the reason why period three

implies chaos and not simply the fact that it allows all other periodicities; even though one

argument implies the other.

14 Derivation of this value can be found in [48–50].

15 In fact since Fμ is a second order polynomial any Fkμ will be a polynomial of order 2

k,so solutions to any case of F kμ − x = 0, will have an even number of real roots.

34

Finally μ3 < μ ≤ 4, with μ = 4 the most important part, given on Figure 1-8. In

the region between μ3 and 4 everything we have seen repeats infinite number of times.

In some subset of I or other, we have the creation of new other fixed points that turn

from stable to unstable giving birth to more and more fixed points in the same way, all

the way to the creation of the “period three orbit”, for the subset in question, when, of

course, we start over again! At μ = 4 we have the final brick on the wall, F4(12) = 1!16

We have seen the consequences of this before, at μ2,12is mapped to 0, the other fixed

point, and as before, more importantly 1 ≤ μ4the pre-images of 0 form a Cantor set; and

after this long journey we finally have F : I → I, F is chaotic on the whole interval I.

We have seen all that has been created as we increased μ, and everything is still there, so

we will not repeat ourselves. Instead, and in conclusion to this section, we will provide

a holistic picture of the points in I. We have a set of points which are fixed, of some

period or other, this set is dense in I, there is a difficult issue here, we can exclude from

this set the fixed points of F 2∞

μ17 , we still have the fixed points of all periodicities, even

infinity, but we are only considering the enumerable number periodicities, and this set is

still dense in I. The set of fixed points of F 2∞

μ we can now put on their own and it is in

itself a Cantor set, and of course dense in I, but even more, uncountable. We are not done

though, there are still “many” points left and specifically the pre-images of all fixed points

of enumerable periodicity, which as we have seen for each fixed point form a Cantor set on

their own. Hence we can consider the following two sets, the fixed points of F 2∞

μ (and their

pre-images in some sense), and the set of all other fixed points and their pre-images; both

sets are an infinite collection of Cantor sets, both dense in I, both uncountable. For its

16 Subsequently we shall drop the subscript on F4 and refer to the function of thisparameter simply as F .

17 Note that μ here is not the Feigenbaum point value of μ∞, instead we are consideringall points of all F 2

∞-functions, even the ones created after period three restarts the

process.

35

immense importance, our final point in this section is to emphasise the following, we have

seen how a function on its own is not chaotic, as a set on its own is not chaotic, it is the

combination of a many-one function on a specific set, under specific parameters that create

chaos.

Figure 1-8. Plot of the function F : I→ I. Also marked are the intervals I1, I3 and I2, thelast broken up into I2L and I2R , which are the two intervals that are mappedcompletely onto each other.

36

Table 1-1. Summary of the transitions in behaviour of the different intervals described inSection 1.2. Each arrow represents a single application of Fμ. x is used toindicate that more than one point is trapped in the interval, while y is used toindicate that a single point is trapped (typically an endpoint of the interval).

μ I = I1 ∪ I2 ∪ I3 I = J1 ∪ J3 I = J1 ∪ J2 ∪ J3I = (J1�J5) ∪J5∪J6∪(J3�J6)

0→ 1xI1←I2↑I3

1→ 2y xI1→I2↑↗I3

∗

x xJ1→J2↑J3

2→ 3y xI1→I2↑↗I3

x xJ1� J3

J2 = J1 ∩ J3

y yJ1→ J2↑↗↙J3

y yJ5�J6

3→μ∞y xI1→ I2↑↗↙I3

x xJ1� J3

J2 = J1 ∩ J3

y yJ1→ J2↑↗↙J3

y yJ5�J6

μ∞→μ2y xI1→ I2↑↗↙I3

x xJ1� J3

J2 = J1 ∩ J3

y yJ1→ J2↑↗↙J3

y yJ5�J6

μ2→μ3

y xI1→ I2↑↗↙I3

x xJ1� J3

J2 = J1 ∩ J3

y yJ1→ J2↑↗↙J3

y yJ5� J6↑↙J7 xJ5=J5∪J7

μ3→4

y yI1→ I2↑↗↙I3

x xJ1� J3

J2 = J1 ∩ J3

y yJ1→ J2↑↗↙J3

y yJ5� J6↑↙yJ7 xJ5=J5∪J7

∗The transition shown actually occurs at μ = 113.

37

1.2.2 The Tent Map, Topological Conjugacy and Universality

This section will briefly introduce the Tent map, which along with the Logistic map

are the two main systems we use for chaos based algorithm development. The Tent map

is a piecewise linear map that also can exhibit chaos. Instead of presenting in detail the

chaos in the Tent map, as we have done in the previous section for the Logistic map, we

will introduce the concept of Topological Conjugacy, which relates one map to the other,

guaranteeing the properties and behaviours in one map to exist in the other. In closing the

section we will move further than Topological Conjugacy, into the promised property of

chaos known as Universality. Universality plays a very important role in computation with

chaotic systems since it releases us from confinement to any single physical realization; as

long as a system is chaotic the algorithms we develop can be implemented by the system,

regardless of whether the nature of the system is electrical, optical, chemical, or even a

realization in some other physical realm.

The Tent map. Just as the Logistic map, the Tent map is discrete time, but instead

of a “continuous” polynomial, it is a piecewise linear “discontinuous” mapping of the form:

T n+1μ (xn) = xn+1 =

μ ∙ xn, for xn <12,

μ ∙ (1− xn), for 12 ≤ xn,

(1–3)

where x ∈ [0, 1](I) , 0 < n, and for our purposes we set μ = 218 . In this domain and

μ value, the Tent map, except for the curvature from the polynomial terms, looks and

behaves like the Logistic map, we can even mark sub-intervals, I1 = [0,13], I2 = [

13, 23]

and I3 = [23, 1] with similar behaviour as the Ii of F , see Figure 1-9. In addition, the

bifurcation diagram for the Tent map is practically the same as for the Logistic map, see

18 As we have done for the Logistic map, for the Tent map in the case of n → n + 1and μ = 2 we will not use the super- and sub-scripts and refer to the function simply as T ,unless emphasis is required.

38

Figure 1-10, even though it looks more “compressed” and the bifurcations for the period

doubling are closely packed, basically everything that we showed for the Logistic map

applies to the Tent map as well. We can actually show this “identity” mathematically

through the Topological conjugacy of the two maps.

Figure 1-9. Plot of the function T : I→ I. From the graph we can see howT : I1 → I1 ∪ I2, T : I3 → I1 ∪ I2, and T : I2L → I3 and T : I2R → I3; and ofcourse how 1

2is a pre-image of 0.

Topological Conjugacy. Topological conjugacy has its roots in set theory,

consider two distinct sets, X and Y, and for each set there is a “relation”, say P and

Q respectively, between its individual elements. For each set consider the “larger” set of

the union of the elements that are in the domain and converse domain of the “relation”,

39

Figure 1-10. Bifurcation diagram for the Tent map. Eventual evolution of an arbitraryinitial state x0. Inset: Magnification of the region μ ∈ [1, 1.1] andx ∈ [0.495, 0.5], showing more clearly the period-doubling.

the “field” of the set, for example given x ∈ X : P(x) = x, then x and x are in the “field”

of X and P , analogously for Y and Q. If there is a “relation” S, which is one-to-one and

has as domain the field {X,P : X} and converse domain {Y,Q : Y}, then the “relations”

P and Q are “similar”, that means P and Q have identical properties, in other words

their effects on their respective sets are indistinguishable. When the sets considered are

topological spaces, the case of the Logistic and Tent map on I, and the “relations” actual

functions, then the functions are said to be Topologically conjugate to each other.

To make this concept more concrete, and show as promised that there is no difference

between the Logistic and Tent maps, consider the two functions F : I → I and T : I → I;

the fact that I acts as the domain and converse domain for both functions should not

worry us for two reasons, first of the sets (or spaces) considered are actually arbitrary

40

Table 1-2. Topological conjugacy. Given that the Tent map takes every x ∈ I on a x ∈ I,and the Logistic map takes every y ∈ I on a y ∈ I, they are Topologicallyconjugate when a function, G, exists that takes every x ∈ I on a y ∈ I andevery x ∈ I on a y ∈ I; so that G ◦ T (x) = F ◦G(x) = y. SpecificallyG(x) = sin2

(π2∙ x).

IT−→ I

(x) (x)

G ↓ ↓G

IF−→ I

(y) (y)

and in reality unchanged19 it is the “relations” (or functions) that we will actually

manipulate, and second, the actual set of points in I for each function is different,

both functions are applied on the same “interval”, which contains the same points,

but the points themselves are different for each function, except {0, 12, 1}. So following

Table 1-2 we have the following relationships: T : I → I, F : I → I, G : I → I

and of course for any x, y, x, y ∈ I, T (x) = x, F (y) = y, G(x) = y and G(x) = y,

which imply G ◦ T (x) = G(x) = y or F ◦ G(x) = F (y) = y. For the Logistic

and Tent map the conjugating function is given by G(x) = sin2(π2∙ x), specifically:

y1 = F ◦G(x0)

y1 = 4 ∙ y0 ∙ (1− y0)

y1 = 4 ∙ sin2(π2∙ x0)∙((1− sin2

(π2∙ x0))

y1 = 4 ∙ sin2(π2∙ x0)∙ cos2

(π2∙ x0)

y1 = sin2 (π ∙ x0)

y1 = G ◦ T (x0)

x0 <12

12≤ x0

x1 = 2 ∙ x0 x1 = 2 ∙ (1− x0)

y1 = sin2(π2∙ x1)y1 = sin

2(π − π ∙ x0)

y1 = sin2 (π ∙ x0) y1 = sin

2(π ∙ x0)

19 You can actually consider each domain and converse domain as fixed points in a setand the “relations” as just a “virtual” link between these points in each set, not actuallyaffecting the points.

41

Of course by induction, if not by simply considering function composition, this result

applies for any, and at any, n, F n ◦G = G ◦ T n, see Figure 1-11 for an illustration.

When we consider the “spatial” correlation between functions given by Topological

conjugacy, in addition to the “temporal” correlation between a function and its future

iterates (e.g. F and F n) given by renormalization, we have a universal correlation between

any “functions” that exhibit similar behaviours.

Figure 1-11. Topological conjugacy between evolved states, up to n = 5. The dashed linefollows the evolution of G(x0) under the Logistic map (F ), while thedash-dotted line follows the evolution of T n(x0). The action of G(x) both as

the first step (x0G→y0) and last step (x5

G→y5) is shown along the triangular

arrows. (f(x) is {F (parabola), T (piecewise linear), G (sigmoid), I(identity)}).

42

Universality. The theory of Universality is, as the name implies, extremely wide and

intricate. It correlates many branches of Physics, from the theory of critical phenomena

to Hamiltonian mechanics, and many branches of Mathematics, from statistics to vector

spaces. The aim of this section is to present some important qualitative results, both

as a hint of proof of Universality, and its main consequence, the independence between

“behaviours” and actual physical system or mathematical interpretation.

The most formal origins of Universality come from Mitchell Feigenbaum [29, 51, 52],

circa 1975 when he discovered the universal constant δ = 4.669.... Originally δ was

observed within a system as the “rate” of onset of period doubling as:

μ2n+1 − μ2nμ2n+2 − μ2n+1

≡ δ, as n→∞, (1–4)

where μ2k is the μ value of onset of the 2kth cycle, in terms of Figure 1-12 it is the limit

of the sequence b1b2, b2b3, b3b4, ... δ. Almost immediately though with this initial observation,

came both the theoretical and experimental confirmation (see Table 1-3) that extended the

existence of δ from within a single system to every system that undergoes period doubling.

More importantly though, and the actual basis of Universality, out of the theoretical

treatment emerges a “convergent function” that encapsulates all such systems. A very

rudimentary approach to describe this function is the following, consider any function that

relates “points” in a set in the manner we have been concerned with in this chapter.

Further, consider each such function as itself a “point” in a set of functions, these

“function-points” converge to a single “function-point”, in some sense just as the bibi+1

converge to δ. We have seen in the previous section how x, n, μ of F , can all be varied in

some way to create a combination of F and an interval from which chaos emerges, now we

see that the functional form of F itself can also vary.

1.2.3 Threshold Control and Excess Overflow Propagation

In this short section we will introduce two processes which have extended the

breadth of influence of chaos theory, and are widely used in algorithms of chaos based

43

Figure 1-12. Logistic map bifurcation diagram for some values within 3 < μ < μ∞. “//”on the axis indicate discontinuity in the displayed points, b1 and b2 areactually longer than shown. For the Logistic map the sequence of the ratiosb1b2, b2b3, b3b4... converges to δ. In other systems the bi can be different from the

Logistic map, but the convergent value of the ratios is the same. Inset:Figure 1-1.

Table 1-3. Experimental measurements of Feigenbaum‘s constant (δ) in different systemsbased on their period doubling. (Adapted from [53].)

SystemsHydrodynamic Electronic Optic AcousticWater Mercury Diode Transistor Laser Helium

Observed δ 4.3± 0.8 4.4± 0.1 4.3± 0.1 4.7± 0.3 4.3± 0.3 4.8± 0.6

computation. We will not go in details for either process, as both are now quite extensive

in their own right.

Control and synchronisation of chaotic systems [13–15, 54–64] have been studied

for almost 20 years now with numerous results; we simply demonstrate the threshold

control mechanism, as already mentioned in relation to the second part of the Sharkovsky

theorem. The other process we introduce, the excess overflow propagation [65–68], also

44

has a long history and connections to a wide variety of other fields as critical phenomena,

phase transitions, cooperative behaviours, and more.

Threshold Control Mechanism. Following Sharkovsky’s theorem and specifically

the second part, we know that a map exists for every periodicity and more, that once

periodicity q is established periodicity p is guaranteed, where p C q. Therefore if we start

with a map which has period 3, the Logistic map F4 for example, this same map can be

of some other “maximal”20 period simply by confining the domain of the map, even more

this period will seem attractive21 . Basically instead of looking for a whole new different

map of “maximal” periodicity q, we take F4 : I → I and consider the map which is the

part of F4 : J → J, where J = (0, x∗] and x∗ is the maximal point value of the sequence of

points that are the desired orbit, hence the name of the method “threshold control”. The

mechanism can be defined as:

F (x) =

F (x), for F (x) ≤ x∗,

x∗, for x∗ < F (x),

(1–5)

the map, F (x), is evolved as usual unless it exceeds x∗, in which case it is limited back to

x∗ and normal evolution continues, F (x) is the resulting part of F (x) which is the map

of desired periodicity, see Figure 1-13 for a specific example of period 4 selection, and a

demonstration of the process.

Specifically for the logistic map, the interval ( 34, 1) provides thresholds for periodic

orbits of all orders greater than 2, for example 34< x∗ ≤ 0.905 forces the map in a cycle of

order 2, 0.9505 ≤ x∗ ≤ 0.9698 of order 3, 0.905 < x∗ < 0.925 of order 4 and so forth, see

Figure 1-14 for a more extensive list. Obviously thresholds from [0, 34] produce fixed points

(period 1).

20 Maximal in the sense of the Sharkovsky ordering.

21 Only because all other periods are repelling.

45

Figure 1-13. Threshold Control Mechanism. F4 the dotted parabola, is confined on theinterval J = (0, x∗] and as a result the map F4 is produced, the solidparabola. In this case x∗ = 0.915. Empty circles, #, mark three differentinitial conditions and their paths, in dotted lines, showing how they are“pushed” onto the period 4 sequence of points x1, x2, x3, x4, marked with fullcircles, .

Excess Overflow Propagation. Up to now we have been considering, in some

form or other, just a single chaotic system, and even thought the potential of even a

single chaotic system is immense, two is always more than one. Since we are considering

more than one system we need to define some way in which the two systems to interact.

Currently the method used in chaotic computation is the excess overflow propagation

method. We define a “monitoring value”, ∗x, for the state of the “emitting” system, f(1x);

46

Figure 1-14. Threshold values for confining the logistic map on orbits of periodicity 2 to50.

once this “monitoring value” is crossed over by the actual state of the system, f(1x) >∗ x,

the difference between the actual state of the system and the “monitoring value”, the

excess overflow (f(1x) −∗ x = E), is propagated to the receiving system(s) f(2x + E).

Note that the “emitting” system is not affected in any way, unlike with the threshold

control method where we confine the state of the system, also at the “receiving” system

the incorporation of the excess overflow can happen both before and after an iteration of

the system f(2x) + E. The method is very similar to the threshold control method, but

independent, in the sense that the “monitoring value” can act as a threshold control, but

is not necessary, the two values can be different, providing us more flexibility.

1.3 Conclusion

We have presented, as briefly as possible, parts of Chaos Theory that are the basis

for Chaotic Computation. Even though this exposition was made in the spirit of Set

Theory and Logic, the exact connections were not actually laid out since our forthcoming

47

presentation of algorithms, developed for Chaotic Computation, do not actually reach such

lengths. The possibility is there through, and we plan to explore further this view in our

future work.

48

CHAPTER 2INTRODUCTION TO CHAOTIC COMPUTATION

Chaotic computation is the exploitation of the richness in behaviours of chaotic

systems to perform computational tasks. This chapter deals with different manipulations

of state variables that lead to selection of a specific behaviour, without relinquishing

access to other behaviours, demonstrating the inherent versatility. Natural parallelism

emerges both through cooperative processes of independent chaotic systems and through

exploitation of multi-dimensional systems. A very good abstraction of chaotic computation

is the translation of functions and data operators from the software realm onto the

hardware; direct implementation of the objective task in hardware. Here specifically we

almost exclusively present, the theoretical development of algorithms and methods of

chaotic computaion and use the Logistic map to illustrate specific examples. For actual

physical realizations, and verifications, of these results we direct the reader to results of

electronic implementations, using Chua‘s circuit [69], and Logistic map circuit [70, 71],

and to simulation results [26] of a IR NH3 laser using the Lorentz system [72, 73]. In

addition we can direct the reader to the very recent physical realization results for chaotic

computation using synchronization [74] and stochastic resonance [75].

2.1 Number Encoding

The primary requirement for any computational system is representation of data.

There need to be methods through which data can be recognized, stored, and reproduced,

obviously only these three processes do not make a very capable computer. The absolute

universal language is mathematics and so numbers is the most generic form of data

representation. Therefore we begin with the different methods available in chaotic

computation for number representation.

2.1.1 Excess Overflow as a Number

This is the simplest method for encoding numbers using a chaotic computer and

follows from the excess overflow propagation mechanism. For any given threshold (x∗), the

49

amount by which the system variable monitored exceeds the threshold, E = fn(x∗) − x∗,

for some nth iteration such that fn(x∗) > x∗, is named “excess overflow” and is used to

represent an integer. More specifically, we utilize an interval, K1, for which the dynamical

system that drives our chaotic computer contains fixed points for a single iteration, n = 1,

under threshold control, i.e. f(x∗) > x∗, ∀x ∈ K1. Given a requirement as of representing

the set of integer numbers {1, N}, we can find from the interval K1 the x∗ which produces

the largest value of excess Emax and “equate” this with the largest integer we wish to

encode, N . As a result, we can define 1 ≡ Emax/N ≡ δ, where δ is called “unit overflow”

and every integer in the set {1, N} can be represented by an excess overflow of z ∙ δ, where

z ∈ {1, N}. Obviously the integer 0 is represented by zero amount of excess, obtained by

setting the system at a natural fixed point, i.e. fn(x) = x, ∀n.

As an illustrative example we use the Logistic map, Equation 1–1. For the Logistic

map the interval [0, 0.75] produces fixed points under threshold control, note that

0 and 0.75 are the “natural” fixed points of this system and either can be used to

represent the integer 0. By taking the derivative of F (x∗) − x∗ we find the threshold

that produces the maximum excess overflow at x∗ = 3/8, with an emitted excess of

Emax = 9/16, see Figure 2-1. Given the set of integers [0, 100] we can encode them

using threshold controlled logistic map elements following the steps described above, i.e.

100 ≡ Emax ⇒ δ = (9/16)/100 and x∗ =−3+√32−4∙(−4)∙(−z∙δ)

2∙(−4) , ∀z ∈ [0, 100], see Figure 2-2.

The extensive versatility of this method will be demonstrated in following sections.

We will show implementations of this method not only for other number representation

methods, but also in algorithms for decimal and binary arithmetic, as well as for boolean

logic operations.

2.1.2 Periodic Orbits for Number Representation

The most immediate extension to the excess overflow encoding method is to consider

the behaviour of the dynamical system outside the region already being utilized, i.e.

x /∈ K1. We can utilize the effect of the threshold control mechanism on this interval, K2,

50

Figure 2-1. Emitted excess by thresholding the logistic map in the interval [0, 0.75]. Thethreshold x∗ = 3/8 produces the largest excess of Emax = 9/16.

to stabilize the system to a periodic orbit. Following Sharkovsky’s theorem, see Theorem

1.1 on page 30, since we have a period three orbit, orbits with periods of all other integer

values, and more, are guaranteed to exist. Therefore it is an obvious extension to utilize

the appropriate orbit to represent its respective integer number. More specifically, for each

n > 1 we find a x∗ ∈ K2, for which fn(x∗) > x∗ and fm(x∗) < x∗, ∀m < n. So for every

integer we have a x∗ that forces the system to emit excess at periodicity n ∈ {2,∞}, in

some sense the converse of what is shown in Figure 1-14. Clearly any of the thresholds

for n = 1 fixed points found in the previous method can be utilized for representation of

integer 1 and as before a natural fixed point can be used for integer 0.

We will show implementations of this method for integer multiplication, calculating

the least common multiple of any set of integers, and in conjunction with the excess

51

Figure 2-2. Encoding the set of integers {0, 1, ..., 100}. We use 100 ≡ Emax = 9/16 toobtain each excess z ∙ δ and the threshold x∗ that produces it.

overflow method in a third method for representing numbers more in the spirit of binary

representation.

2.1.3 Representation of Numbers in Binary

In this final example of number encoding methods we will combine the previous two

methods, we use both periodic orbits and excess overflow. We can represent a number

in binary format by coupling together elements which have their periodicity determined

by their position away from the radix point as 2#digits−position. The farther away from

the radix point an element is the shorter periodicity we give it, with the bit farthest

away being on period one. The elements are joined together serially, so that the “generic

overflow” generated by each element cascades through the array until it reaches the open

end at the most significant digit, where we have the readout, see Figure 2-3. To be more

specific a binary number aN . . . a1 will be represented by N elements. Each element, j,

as in the encoding based on periodicity, will be set at a threshold so that to produce a

52

“generic overflow” at periodicity of 2N−j, if the binary digit aj = 1, and at a threshold

of 0, if aj = 0, thus generating no overflow. The resulting array is updated 2N−1 times

resulting in a multiple of the unit overflow∑Nj=1 aj2

j−1 at the readout.

An illustrative example using the Logistic map is shown in Figure 2-3, where we are

using four elements, N = 4, to represent the binary number 1111. Following the example,

a4, the farthest element from the radix point, is set to a threshold that emits excess on

every update, i.e. periodicity one (x∗ ∈ (0, 0.75)); a3 is given a threshold x∗ ∈ (0.75, 0.905),

to produce excess on every second update; a2 from x∗ ∈ (0.905, 0.925) for an excess every

fourth update; and a1 from x∗ ∈ (0.925, 0.926) for an excess every eighth update. As a

result we will collect emitted excess 8 + 4 + 2 + 1 = 15, respectively from each element, as

is desired for encoding 1111 in binary.

This method allows us to encode large numbers without the need of thresholds that

produce proportionally large periodicities, or of thresholds that produce excess in steps

of very small δ. This is a good example of the flexibility of chaotic computation. In the

specific example we “sacrificed” efficiency, in the number of elements per encoded integer,

to allow short periodicities to encode larger numbers. Obviously, the method can be easily

modified to encode numbers in any other base representation.

2.2 Arithmetic Operations

We have established not one, but three methods for representing numbers, we should

see them in action. The most obvious starting point is the simplest arithmetic operation:

addition, which we extend into multiplication, which we extend to the least common

multiple problem.

2.2.1 Decimal Addition

There are multiple chaotic computation algorithms for decimal addition. We will

present three such algorithms, each one building on its predecessor, from serial addition to

parallel to branching.

53

Figure 2-3. Number encoding in binary format. The excess from each element cascades tothe one “above” it until the readout is reached. The element closest to thereadout, a4, emits on every update, and as we move “down” the chain theelements emit in increasing powers of 2. The overall result is after 8 updates,we have 8 units of excess from a4, 4 from a3, 2 from a2 and 1 from a1, givingus a total of 15 units. Any binary number can be represented with thismethod. (Adapted from [26].)

Serial. The most straightforward algorithm for addition utilizes the excess overflow

encoding method in a serial manner. In reality it is a natural extension of the encoding

method, since each number is encoded as a proportional excess we can chain-link the

elements and cascade the emitted excess from each element into its neighbour all the

way to the end of the chain. Each excess “builds up” on the one after it, all “naturally”

summing up at the edge of the chain, see Figure 2-4. As we have seen in Section 2.1.1

the “unit overflow” (δ) works as the proportionality constant between integers and

emitted excess, therefore the addition of integers i, j, k, l is simply replicated by

54

iδ + jδ + kδ + lδ = (i + j + k + l) ∙ δ, using the avalanching of the excesses. For

this algorithm the computational time is dependent on the adaptive process and the

number of terms in the sum. Specifically after a single chaotic update of all the elements,

it takes as many adaptation steps as there are terms in the sum to complete the operation.

Figure 2-4. Serial Addition. We recruit as many elements as terms in a given sum. Eachelement is assigned a number from the sum, encoded using the excess overflowmethod. The elements are coupled together in a chain such that the excess canflow down the chain. The result is at the open end we collect the sum of allthe emitted excesses as a multiple of δ. (Adapted from [25].)

Parallel and Branching. Following from the previous example, consider the sum

of four terms, i, j, k, l using the serial addition algorithm. If we take a closer look at

the dynamics of the “last” element in the chain (in this case the one encoding l), we see

that this element will receive the excess of all preceding elements “simultaneously”. To

visualize this consider the “local” dynamics of three elements, once the chaotic update

is complete. The excess of the first element is avalanched to the second, where it “builds

up” with the local excess at the second element, so the third element will receive the

combined excess of the two previous to it as a single excess. This can also be achieved by

introducing the two excesses independently of each other, but simultaneously, i.e. arrive

55

at the same time and “build up” on each other locally at the third element instead of at

the second. Of course this can be extended to any number of elements preceding the “last”

element in the chain. Turning back to our example of adding i, j, k, l, the topology of the

connectivity instead of being a chain is now a tree diagram, see Figure 2-5. By turning

the chain into a tree we have collapsed the serial addition algorithm to a two step serial

addition, regardless of the number of terms in the sum. The first step is to sum all, but

one of the terms in parallel and then serially combine the parallel sum to the remaining

term, before reading the result at the open end. We need to note that we can not fully

parallelize the operation by connecting all terms directly to the open end, since we are not

attributing any dynamical properties to the open end, i.e. if the open end had the ability

to correctly “build up” excesses it would be identical to all the other dynamical systems

used in the sum, making it the “last” element1 .

Obviously for a sum of N terms the shortest computational time for the operation is

to connect N − 1 terms in parallel to the “last” term and perform the operation in two

steps, this would be analogous to increasing the number of “branches” in the network. In

case though of connectivity and/or spatial restrictions we could also extend the algorithm

by increasing the number of “trunks” in the network, as is shown in Figure 2-6; note that

now our computational time is dependant on the number of “trunks”. This is another case

of chaotic computation exhibiting its flexibility, we can “sacrifice” temporal performance

to satisfy spatial constraints.

2.2.2 Binary Addition

Extending the above addition algorithms for the binary number encoding method

is straightforward. The serial addition of binary numbers is realized by connecting the

“end bit” of one number with the “end bit” of the following number in a chain manner

all the way to the last number and then to the open end, see Figure 2-7. Similarly for

1 This could actually be overcome by inserting to any sum, 0 as the last term.

56

Figure 2-5. Decimal parallel addition. The excess from the elements encoding i, j, k issimultaneously propagated to the element encoding l on a first avalanchingstep, and on the second avalanching step the collective sum is read at the openedge. (Adapted from [26].)

Figure 2-6. The branching algorithm can be extended to a larger treelike structure. Thecomputational time in this case is proportional to the number of branches inthe longest path that terminates at the “last” element. The above network willsum 15 terms in four avalanching steps. (Adapted from [26].)

57

implementing the parallel addition method for binary numbers we connect the “end

bit” of all terms in the sum, but one, to the “end bit” of the single term chosen to act

as the collection hub for all the excesses, before sending the result to the open end, see

Figure 2-8.

Figure 2-7. Schematic representation of the serial addition method for binary numbers.The numbers 7, 5, 2, 1 are encoded as explained above in Section 2.1.2 and theelements are serially connected. The excess overflow builds up as it movestrough the network and 15 units of excess are collected at the OUTPUT.(Adapted from [26].)

58

Figure 2-8. Schematic representation of the parallel addition method for binary numbers.In this case a branching topology is used for the network, where one of thesystems acts as a collection hub for simultaneous build up of excess. (Adaptedfrom [26].)

2.2.3 Decimal Multiplication and Least Common Multiple

We can extend any addition method to perform multiplication in the usual way,

the product of two numbers m × n is a sum of n terms of the number m (and of course

visa versa m terms of the number n). With our two above addition methods we have

two obvious ways of implementing multiplication the serial and parallel summation of

the terms. Furthermore using the excess overflow encoding of numbers we have a third

method; we can use a single dynamical system that emits the appropriate excess on every

update, specifically the amount that represents one of the numbers, m ∙ δ for instance, and

we update the system n times collecting a total of excess equal to (m× n) ∙ δ, the product

59

of the multiplication. This third method utilizes time as a computational quantity, and

leads to a fourth multiplication method, which we will also use as a stepping stone for the

method of least common multiple of many numbers.

Following the periodic orbit encoding method, we can represent each number, of

a two number product, using for each number a chaotic element set to emit excess at

the appropriate periodicity. Therefore given a product m × n we utilize two dynamical

systems, one set to emit every m updates and the other to emit every n updates, the

product of the two numbers is given by the number of updates on which the two elements

emit simultaneously; more specifically the element emitting every m updates will have its

nth emission on the (m × n)th update and the same applies for the other element. The

simple extension of this method to a larger number of terms results in an algorithm for the

least common multiple of all the terms in consideration2 .

2.3 Binary Operations

The power of conventional computers lies in their ability to perform boolean algebra.

Even so their building blocks are restricted by manufacture to one of the two fundamental

gates, NOR or NAND, which with suitable combinations, of either, the other logic gates

can be reproduced, for example AND(X,Y)≡ NOR(NOR(X,X),NOR(Y,Y)). This clearly

is not the most efficient method for boolean algebra; a much more efficient alternative

is for each gate to require only one building block, conversely each building block to be

able to perform all gates. We show how a single chaotic element can represent each of all

logic gates through simple state manipulations, removing the need for combining elements

[32, 76]. Furthermore we show how multi-dimensionality can lead to natural parallelism,

and we go even further, exiting the capabilities of conventional algorithms and addressing

a problem designed for quantum computation.

2 The case of three numbers in a product is handled serially, find the product of two ofthe terms, and multiply it by the third term.

60

Figure 2-9. Schematic representation of the method for computing the Least CommonMultiple of four numbers. Each element emits excess to the collecting elementat the appropriate periodicity for its encoded number. The result at theOUTPUT is excess of the collecting element with magnitude equal to thenumber of term elements that emitted on the current update. The number ofupdates that causes the collecting element to emit excess equal to the numberof terms (i.e. all term elements emitting simultaneously) is the Least CommonMultiple of the terms. (Adapted from [26].)

2.3.1 Logic Gates

As we have shown an important characteristic of chaotic computation is the

versatility we have in implementing the same algorithm with different methods. This

of course extends to implementations for representation of logic gates, i.e. there are

multiple ways we can achieve this representation. In this section we focus on the most

61

straightforward method developed and in order to present the method more clearly we

specify it to the logistic map3 , Equation 1–1.

The method we use consists of three steps: (a) initialization, (b) chaotic update, and

(c) threshold control and excess overflow. Compared to the methods presented in previous

sections, the new concept for this method is initialization; i.e. the setting of the initial

state of the system, (X0) just before the first chaotic update, based on specific rules. This

initial condition of an element is used to define which logic operation it performs and on

what set of inputs. Specifically, we initialize a logistic map element by setting its initial

value x0 according to:

• x0 = xprog + xI1 + xI2 , for gates that operate on two inputs,

• x0 = xprog + xI , for gates that operate on one input,

xprog can be thought of as “programming” the gate and xIi as the input values. For an

input of logical 1, xIi = δ, and for an input of logical 0, xIi = 0. As before a chaotic

update implies the application of the logistic map: x0 → F (x0). The control and overflow

mechanism remains the same as well: E = 0 if F (x0) ≤ x∗, and E = F (x0) − x∗

if F (x0) > x∗. Where x∗ is the threshold for the element and E the excess overflow

generated. Here in the context of binary algebra where the set of integers contains only 0

and 1, E and δ are actually equivalent4 .

Turning to the specific logic gates to be implemented, the following Table 2-1,

summarizes the input - output relationships we are to represent.

The task is to identify initial conditions, xprog + xIi , and threshold values, x∗, for

which a chaotic update will result in F (x0) ≤ x∗ for where the outputs in the above table

3 Universality of chaotic systems allows us to assume demonstrations on the logisticmap can be carried over to any other chaotic system.

4 Equivalence of inputs and outputs is actually a “soft” requirement.

62

Table 2-1. Truth-table for the logic operations AND, OR, XOR, NOR, NAND, NOT, andthe identity operation (WIRE).

I1 I2 AND OR XOR NOR NAND I NOT WIRE0 0 0 0 0 1 1 0 1 00 1 0 1 1 0 1 1 0 11 0 0 1 1 0 11 1 1 1 0 0 0

are 0, and F (x0) > x∗ where the output is 1. As a specific example, for the OR gate we

have the following requirements:

1. I1 = I2 = 0, which implies xI1 = xI2 = 0, i.e. x0 = x

prog.The required output is 0,which implies F (xprog) ≤ x∗.

2. I1 = 0 and I2 = 1, which implies xI1 = 0 and xI2 = δ, i.e. x0 = x

prog+δ. The requiredoutput is 1, which implies F (xprog + δ) − x∗ = δ. This requirement is symmetric toI1 = 1 and I2 = 0, so the conditions for satisfying both requirements are identical.

3. I1 = I2 = 1, which implies xI1 = xI2 = δ, i.e. x0 = xprog + 2 ∙ δ. The required output

is 1, which implies F (xprog + 2δ)− x∗ = δ.

All of the above requirements need to be satisfied by the same values for xprog and x∗,

such that all three conditions hold true regardless of inputs. In a similar manner we can

provide required conditions for a chaotic element to represent every gate, see Table 2-2.

Table 2-2. Necessary and sufficient conditions for a chaotic element to satisfy the logicoperations AND, OR, XOR, NOR, NAND, NOT, and the identity operation(WIRE).

InputxI1 + xI2 AND OR XOR0 F (xprog) ≤ x∗ F (xprog) ≤ x∗ F (xprog) ≤ x∗

δ F (xprog + δ) ≤ x∗ F (xprog + δ)− x∗ ' δ F (xprog + δ)− x∗ ' δ2δ F (xprog + 2δ)− x∗ ' δ F (xprog + 2δ)− x∗ ' δ F (xprog + 2δ) ≤ x∗

InputxI1 + xI2 NOR NAND0 F (xprog)− x∗ ' δ F (xprog)− x∗ ' δδ F (xprog + δ) ≤ x∗ F (xprog + δ)− x∗ ' δ2δ F (xprog + 2δ) ≤ x∗ F (xprog + 2δ) ≤ x∗

InputxI NOT WIRE0 F (xprog)− x∗ ' δ F (xprog) ≤ x∗

δ F (xprog + δ) ≤ x∗ F (xprog + δ)− x∗ ' δ

63

Values that simultaneously satisfy the above conditions are easily found. Specifically

by choosing δ = 0.25, the OR gate can be realized by choosing values for xprog = 1/8 and

for x∗ = 11/16:

1. F (xprog) = F (1/8) = 7/16 ≤ x∗(= 11/16),

2. F (xprog + δ) = F (3/8)− 11/16 = 15/16− 11/16 = 1/4(= δ),

3. F (xprog + 2δ) = F (5/8)− 11/16 = 15/16− 11/16 = 1/4(= δ).

In fact values that satisfy the conditions for all the gates and δ = 0.25, have been

identified and are summarized in Table 2-3.

Table 2-3. Initial values, xprog, and threshold values, x∗, required to implement the logicgates AND, OR, XOR, NOR, NAND, NOT, and the identity operation(WIRE), with δ = 0.25.

Value AND OR XOR NOR NAND NOT WIRExprog 0 1/8 1/4 1/2 3/8 1/2 1/4x∗ 3/4 11/16 3/4 3/4 11/16 3/4 3/4

The fact that we have a method to implement all logic gates is not impressive, what is

impressive is the fact that we can switch from one gate to another from one computational

step to the next, and even more, switch very easily and “fast’,’ in a relative timescale.

This leads to the concept of on-the-fly hardware re-programming; an architecture based on

conventional computation, so all other components can be easily imported, but with the

efficiency chaotic computation offers.

2.3.2 Parallel Logic and the Half Adder

In this section we present what is probably the most important extension of

performing boolean logic with chaotic systems [32]. The same procedure we illustrated

above for the logistic map is implemented on the 2-dimensional neuron model [77]:

xn = (xn−1)2 ∙ exp(yn−1 − xn−1) + k (2–1)

yn = a ∙ yn−1 − b ∙ xn−1 + c (2–2)

64

where a = 0.89, b = 0.18, c = 0.28, k = 0.03; these parameter values keep the model

dynamics completely chaotic.

Two distinct cases are investigated:

1. The possibility of performing XOR gate and AND gate in parallel (Half Adder)

2. Performing two AND gates independently.

The first case specifically, is the application of the two gates on the same set of inputs,

each gate being performed in a different dimension. The second case involves operating the

two AND gates on different sets of inputs, again each dimension performing one operation.

As with the case of the logistic map, the first step is to define the necessary conditions

needed to satisfy the truth table of each case, specifically following the truth tables

(Tables 2-4, 2-5) we convert them to the conditional Tables 2-6, 2-7. Following the

process of the previous section we determine values for the “programming” state shift and

threshold value that satisfy these conditions, see Tables 2-8, 2-9.

Table 2-4. Truth table for XOR and AND logic gates on the same set of inputs. (Case 1)

I1 I2 XOR AND0 0 0 00 1 1 01 0 1 01 1 0 1

We should mention that each case is investigated independently, i.e. there is no

requirement that the number of iterations (n) is identical for both cases, or that the value

representing a logical 1 (δ) is the same.

We have already seen some type of parallel operations with chaotic computing

in Section 2.2.1 with the parallel addition, here though we see a clear demonstration

of parallelism. The dynamical system performs two completely different operations

simultaneously. In fact the next section builds further on the parallel capabilities of

chaotic computation, by tackling the complex Deutsch-Josza problem.

65

Table 2-5. Truth table for two AND gates operating on independent inputs. (Case 2)

I11 I12 I21 I22 AND(I11 , I12 ) AND(I

21 , I

22 )

0 0 0 0 0 00 0 0 1 0 00 0 1 0 0 00 0 1 1 0 10 1 0 0 0 00 1 0 1 0 00 1 1 0 0 00 1 1 1 0 11 0 0 0 0 01 0 0 1 0 01 0 1 0 0 01 0 1 1 0 11 1 0 0 1 01 1 0 1 1 01 1 1 0 1 01 1 1 1 1 1

Table 2-6. Required conditions to satisfied parallel implementation of the XOR and ANDgate. (Case 1)

Initial conditions XOR ANDxprog, yprog xn ≤ x∗ yn ≤ y∗

xprog + δ1, yprog + δ2 xn − x∗ ' δ1 yn ≤ y∗

xprog + 2 ∙ δ1, yprog + 2 ∙ δ2 xn ≤ x∗ yn − y∗ ' δ2

Table 2-7. Required conditions for implementing two AND gates on independent sets ofinputs. (Case 2)

Initial conditions AND(I11 , I12 ) AND(I

21 , I

22 )

xprog, yprog xn ≤ x∗ yn ≤ y∗

xprog, yprog + δ2 xn ≤ x∗ yn ≤ y∗

xprog, yprog + 2 ∙ δ2 xn ≤ x∗ yn − y∗ ' δ2xprog + δ1, y

prog xn ≤ x∗ yn ≤ y∗

xprog + δ1, yprog + δ2 xn ≤ x∗ yn ≤ y∗

xprog + δ1, yprog + 2 ∙ δ2 xn ≤ x∗ yn − y∗ ' δ2

xprog + 2 ∙ δ1, yprog xn − x∗ ' δ1 yn ≤ y∗

xprog + 2 ∙ δ1, yprog + δ2 xn − x∗ ' δ1 yn ≤ y∗

xprog + 2 ∙ δ1, yprog + 2 ∙ δ2 xn − x∗ ' δ1 yn − y∗ ' δ2

66

Table 2-8. Examples of initial values, xprog, yprog, and thresholds x∗, y∗, that satisfy theconditions presented in Table 2-6, yielding the parallel operation of XOR andAND gates. In this range δ ' 0.7, and the number of required iterations toyield the correct result is n = 10. (Case 1)

xprog yprog x∗ y∗

1.8 1.7 0.44 1.111.4 1.65 0.38 1.07

Table 2-9. Examples of initial values xprog, yprog, and thresholds , that satisfy theconditions presented in Table 2-7, yielding operation of two AND gates onindependent inputs. For these values, δ ' 0.0115 and the number of iterationsrequired n = 20; note that the high number of required iterations and the needfor more decimal precision are a direct consequence of the large number ofrequired conditions to be satisfied simultaneously. (Case 2)

xprog yprog x∗ y∗

0.5375 1.0835 0.231 1.454

An important difference from the previous section on single binary gates is that in

this method the number of iterations that are necessary for the conditions to be satisfied

is greater than 1, actually in Chapter 5 we report extensive progress in involving the time

dimension with chaotic computation.

2.3.3 The Deutsch-Jozsa Problem

The Deutsch-Jozsa problem and its solution algorithm [78] is a celebrated result of

quantum computing, since it is the first example of quantum computing outperforming

classical algorithms. In addition it has been the stepping stone for the other two

major results of quantum computing, Shor’s factoring algorithm [79] and Grover’s

search algorithm [80] (details of the chaotic computation search algorithm are given in

Chapter 3).

In this section we will demonstrate the chaotic computing algorithm for solving the

Deutsch-Jozsa problem, which is just as efficient as its quantum counterpart, provides

the answer in a single computational step, but in contrast is much more realizable and

provides a more apparent result.

67

The problem can be stated as follows: Given a binary domain, i.e. a discrete domain

of 2k states, and given an arbitrary binary function f : {0, 1}k → {0, 1}, i.e. a function

that maps every point in the binary domain to a 0 or a 1, determine whether the function

is constant, maps the whole domain on either 0 or 1 exclusively, or whether the function is

balanced, maps the whole domain in equal terms on 0 and 15 .

In other words, evaluate f for every given point6 , and count the number of resulting

0s and 1s for all points, which basically is the conventional approach in solving the

problem. As a result the conventional algorithm in the best case requires evaluating

the function for only 2 points, i.e. the function results in a 0 (or 1) for the first point

considered, and the opposite for the second point thus the function is balanced. In the

worst case though, i.e. when the first N/2 points give the same result, it would take

2k−1 + 1 evaluations to conclude whether the function is constant or balanced, i.e. the

number of computational steps needed grows exponentially with the number of bits that

define the domain.

In the context of chaotic computation the situation is much simpler since we can

perform the given function on all domain points simultaneously, furthermore we can read

the result in one step. To make the demonstration of the algorithm clearer we explicitly

use the tent map, Equation 1–3 and work in a state space that applies to this map. Also

we partition the explanation of the algorithm in three steps: (a) definition of the domain,

(b) definition of the function space, and (c) implementation of the function along with

reading the result. As is typical of chaotic computation there are at least three different

5 We are guaranteed by the problem that the function will be either constant orbalanced, exclusively.

6 The number of points in a given domain is given as: N = 2k, where k is the number ofbinary digits considered.

68

implementations of the algorithm [26], below we present, with some modifications from

[27], the most elaborate, but clear realization.

(a) Consider a binary domain space (Bk) defined by points of length k binary digits,

therefore the number of points in Bk is equal to N = 2k. Since we are to work with the

tent map we will translate every point in Bk, onto a point in the domain of the tent map

[0, 1]. Take any of the 2k points as a1a2a3...ak, where each ai ∈ {0, 1}, we map each such

point onto [0, 1] using7 :

x = 2−(k+1) +

k∑

i=1

ai2−i (2–3)

We encode the whole domain on an array of N elements, denoted by XN , each

element(j) having a state x(j) given by Equation 2–3. To illustrate this, consider the

case where k = 3, the binary domain (B3) has eight points {000, 001, 010, ..., 111} which

translate onto the tent map domain as { 116, 316, 516, 716, 916, 1116, 1316, 1516}, each of these points

is used as the state value of a dynamical element in an eight element array. So the whole

domain in consideration is encoded by a single array as:

X8 = {x(1) = 116 x(2) =

316 x(3) =

516 x(4) =

716 x(5) =

916 x(6) =

1116 x(7) =

1316 x(8) =

1516} .

(b) The function space of the functions: f : Bk → {0, 1} consists of 22kfunctions,

that is for each of the N = 2k points there are two output possibilities. Regardless

of the total number of possible functions only two are constant, the function that has

outputs all 1 and the function that has outputs all 0, for all input points. The number

of balanced functions though, is dependant on k and is given by simple combinatorics as

CLM =L!

(L−M)!M ! , where for a set of L items, made up of only two distinct objects, C is the

number of combinations where there are M items of one of the two objects. In the case

of balanced functions for a binary domain of N points the above relationship becomes

7 The 2−(k+1) term is added so that points 0 and 0.5 are not used for encoding.

69

CNN/2 =N !

(N−(N/2))!(N/2)! . More specifically for our example of k = 3, we have a total number

of possible functions 223= 256, out of which two are constant and (C84 =)70 are balanced.

In our case, of the function domain being represented by an array of tent maps as

described above, the function space will be populated with functions which are constructed

out of combinations of the following two basis functions:

xn+1 = T (xn) = 1− 2 ∙

∣∣∣∣xn −1

2

∣∣∣∣ (2–4)

xn+1 = T (xn) = 1− T (xn) = 2 ∙

∣∣∣∣xn −1

2

∣∣∣∣ (2–5)

where T is the tent map and T the inverted tent map, and n is used to denote the time

step. Each one of the 22kfunctions is constructed as one of the different combinations of

the two basis functions of length 2k, specifically the function space looks like:

{T (1)T (2)...T (j), T (1)T (2)...T (j), ...

..., T (1)T (2)...T (j − 1)T (j), ...

..., T (1)T (2)...T (j)},

where j = 2k, and each sequence of length j of the two basis functions, is one of the

possible functions (F) to be applied on the domain space8 .

Referring back to our concrete example of k = 3, we have 256 combinations of T and

T in the function space, ranging from the single sequence of eight consecutive T , to 70

combinations of four f : {0, 1}k → {0} and four f : {0, 1}k → {1}, to the single case of

eight consecutive T ; and of course the other 184 combinations9 .

8 In our notation here we use j instead of N to indicate the relation between thefunctions and the array of dynamical elements encoding the domain space.

9 Care not to confuse the 70 functions of four T and four T with the 70 balancedfunctions! A function that is balanced in T and T is not necessarily balanced in itsoutput.

70

Figure 2-10. Basis function T - Tent Map.

(c) Turning to the specific processes involved in solving the problem, we will focus

on the example of k = 3 to make this section more illustrative. As we have seen for

the 3-bit case there are 256 functions, in the context of the problem, we are given one

of these functions (F) (i.e. a specific sequence of 8 T and/or T ) and are assured that it

is either constant or balanced, the task is to find which one it is. In a straightforward

manner we setup the array of dynamical elements, X8, to encode the binary domain

as explained in (a) and then we apply the given function on the whole array F(Xn8 ) =

Xn+18 , where n denotes the time step. Down to the scale of the individual elements

we have T (xn(j)) = xn+1(j) and T (xn(j)) = xn+1(j), where j = 1, ..., 8, whether

71

Figure 2-11. Basis function T - Inverted Tent Map.

it is T or T applied on the jth element is based on which function (F) we are given.

Once the function is applied we threshold the elements at x∗ = 0.5 and collect the

excess the usual way. Basically we are using 0.5 as a separatrix of the state space, i.e.

xn+1(j) > 0.5 ≡ 1 ⇒ xn+1(j) − 0.5 = Ej and xn+1(j) ≤ 0.5 ≡ 0, Ej = 0 (where E is

emitted excess), this is a standard application of symbolic dynamics.

Now we have the collected excess in a single step we also have the answer: If the

collected excess is 0, then the function we have is the constant function F : Bk → {0}k,

72

i.e. the function that has output all 0; If the collected excess10 is 2k/4 = 8/4 = 2, which is

the maximum possible excess for B3, then we have the constant function F : Bk → {1}k,

i.e. the function that has output all 1. Obviously if our collected excess is some other

value we have a balanced function. This is as much as we are required by the original

problem, but we can do even better, using the excess collected we can determine in which

of five classes the given balanced function belongs to.

The importance of this algorithm is not in the actual task it accomplishes, which

is of little practical use. The primary importance is, like for the quantum analogue, in

demonstrating an extremely higher efficiency than conventional computation in solving

problems of this class. Further though for chaotic computation it is a milestone against

quantum computation as well. It shows how chaotic computation can handle problems as

well as quantum computation, if not better.

2.4 Conclusion

This chapter almost exclusively dealt with theoretical developments during the

first four years of Chaotic Computation (1998-2002), while we relegated the experimental

realizations to the references, so that to maintain a uniform global tone to this dissertation.

In closing and without wishing to undermine the other results, once again we draw

attention to the important result of the solution to the Deutsch-Jozsa problem, and ask

the reader to consider the connections between this problem, Set Theory, Logic, and to be

more specific to the structure of a chaotic function at the Feigenbaum point.

10 The factor of 14comes from the fact that the average excess emitted 〈Ej〉 is 0.25.

73

Figure 2-12. Four realizations of the chaotic Deutsch-Jozsa algorithm for the case k = 3.There are eight binary inputs 000, 001, 010, 011, 100, 101, 110, 111 andthrough Equation 2–3 we obtain the state values 1

16, 316, 516, 716, 916, 1116, 1316, 1516

each given to an element of an array. The vertical lines mark the state valueof each element x(j). The horizontal line is the separatrix at 0.5. (a) Given

the constant function F : Bk → {1}k, this is the function: T T TTTT T T , theapplication of the function brings every element over 0.5, the maximumexcess of 2 is emitted from the array. (b) Given the constant function

F : Bk → {0}k, TT T T T T TT , all array elements remain under 0.5 resulting inzero excess. (c) Given the balanced function of eight T functions, produces anexcess of 1. (d) A randomly chosen balanced function, four elements are over0.5 and four under, the excess is 1.25.

74

Figure 2-13. The total excess emitted from each of the 72 functions. The two squaresindicate the two constant functions, referring back to Figures 2-12(a,b). Asyou can see the balanced functions (circles) separate in five groups. Thepoints (c) and (d) refer to the respective graphs of Figure 2-12.

75

CHAPTER 3SEARCHING AN UNSORTED DATABASE

In this chapter we present a chaotic computation algorithm for searching an unsorted

database for a match between a queried for item and the contents of the database [28].

The dynamical system we use for the demonstration of the algorithm is the Tent map,

given in Equation 1–3.

In general most commonly used devices for storing and processing information are

based on the binary encoding of information, i.e. upon bits. Larger chunks of information

are encoded by combining consecutive bits into bytes and words. Here we show a different

approach for information encoding and storage, based on the wide variety of patterns that

can be extracted from nonlinear dynamical systems.

We specifically demonstrate the use of arrays of nonlinear dynamical systems

(or elements) to stably encode and store information (such as patterns and strings).

Furthermore we demonstrate how this storage method enables the efficient and rapid

search for specified items of information in the data store. It is the nonlinear dynamics

of each array element that provides flexible capacity storage, as well as the means to

preprocess data for exact and inexact pattern matching. In particular, we choose chaotic

systems to store and process data through the natural evolution of their dynamics. More

importantly perhaps we note that, our method involves just a single procedural step, it is

naturally set up for parallel implementation and can be realized with hardware currently

employed for chaos-based computing architectures.

We first show a slightly modified storing and encoding scheme, based on the Excess

overflow scheme of Section 2.1.1, in which we use the actual fixed point state of the

system, instead of the excess generated. Specifically we demonstrate this scheme as

applied to the Tent map for storing, and associate the storage with more than just

numbers, different encodings. We then show the actual “search process” and how the

results are a direct consequence of the non-linear nature of the Tent map. Finally with

76

specific examples we show implementations of the method for not only exact matches, but

for also “inexact”, approximate matches, to a given target.

3.1 Encoding and Storing Information

Consider a list of N data storage elements (labeled as j = 1, 2, ..., N) in an array,

where each element stores and encodes of one of M distinct items. N can be arbitrarily

large and M is determined by the kind of data being stored. For instance when storing

English text we can consider the letters of the alphabet to be each a naturally distinct

item, so M = 26. For the case of data stored in decimal representation M = 10, and for

work in bioinformatics (manipulating the symbols A, T, C, and G) we have M = 4. We

can also consider strings and patterns as the items. For instance for manipulating English

text we can use a large set of keywords as the basis, necessitating very large M. We store

this list of N elements by N dynamically evolving chaotic elements. The state of the

elements at discrete time n is given by Xjn[m], where (j = 1, 2, ..., N) indexes each element

of our list and (m = 1, 2, ...,M) indexes an item in our ”alphabet” (namely one of the M

distinct items). To reliably store information one must confine each dynamical system to

a fixed point behaviour, i.e. a state that is stable and constant throughout the dynamical

evolution of the system over time n, basically so that the list remains unchanged. We

therefore employ a threshold control mechanism, see Section 1.2.3, to flexibly control the

dynamical elements onto the large set of period 1 fixed points.

Specifically for the tent map, thresholds in the range [0, 23] yield fixed points, namely

Xn = T, for all time, where T is a threshold from 0 ≤ T ≤ 23. See Figure 3-1 for a

schematic of the tent map under the threshold mechanism, which is effectively described

by a “be-headed map”. It is clear from Figure 3-1 that in the range [0, 23] the value of Xn

lies above Xn implying that the system with state Xn at threshold T will be mapped to a

state higher than T in the subsequent iterate and thus will be clipped back to T. Another

way of graphically rationalizing this is to note that fixed point solutions are obtained

where the Xn+1 = Xn line intersects the “beheaded” tent map. The value of X at the

77

intersection yields the value of the fixed point, and the slope at the intersection naturally

gives the stability of the fixed point. It is clear from Figure 3-1 that in the range [0, 23] this

intersection is on the “plateau”, namely the fixed point solution is equal to the threshold

value. Further the solution of the fixed point for this map is super stable as the slope is

exactly zero on the “plateau”. This makes the thresholded state very robust and quite

insensitive to noise.

Figure 3-1. The Tent map under the threshold mechanism. Shown two cases of thresholdcontrol at 1

4and 1

2. Effectively each threshold value is on a “plateau” yielding

a fixed point. The symbol M indicates the action of the Tent map on thethresholded value Xn < Xn+1, while H indicates the effect of the controlXn+1 → T, for X values in the range [0, 23 ].

78

Returning to our data of a given “alphabet” we can take a large set of thresholds

{T[1],T[2], ...,T[M ]} from the fixed point range, setting up a one-to-one correspondence

of these M thresholds with the M distinct items of our data. This allows each item m to

be uniquely encoded by a specific threshold T[m], with (m = 1, 2, ...M). So the number

of distinct items that can be stored in a single dynamical element is typically large, as the

size of M is limited only by the precision and resolution of the threshold setting and the

noise characteristics of the physical system being employed.

Therefore given an unsorted list of N data, chosen out of the “alphabet” of M items,

as described above, we setup an array of N elements (labeled as j = 1, 2, ..., N), each a

Tent map, each with a threshold Tj[m] reliably storing and encoding the appropriate item

of the list. That is, if element j holds item m in the unsorted list, the threshold value

of element j is set to Tj[m], without changing, or in any way affecting any parameter

of the list. So by denoting the threshold of element j by Tj[m] we have the following:

if the state of element j of the system, Xjn[m], exceeds its prescribed threshold Tj[m]

(i.e. when Xjn[m] > Tj[m]) the state Xjn[m] is reset to T

j[m]. Since the thresholds lie in

the range yielding fixed points of period 1, this enables each element to hold its state at

value Xjn[m] = Tj[m] for all times n.

In our encoding for a reason that will become apparent in the next section, the

thresholds are chosen from the interval (0, 12), namely a subset of the fixed point window

[0, 23]. For specific illustration, without loss of generality consider each item to be

represented by an integer m, in the range [1,M ]. Defining a resolution r between each

integer as:

r =1

2∙

1

(M + 1), (3–1)

gives us a lookup table, mapping the encoded item to the threshold, specifically relating

the integers m in the range [1,M ] to the thresholds Tj[m] in the range [r, 12− r] by:

79

Tj[m] = m ∙ r. (3–2)

Therefore we obtain a direct correspondence between the set of integers 1 to M ,

where each integer can represent any item, and a set of M threshold values of a dynamical

system. Even more we can store N list elements by setting appropriate thresholds, via

Equation 3–2, on N dynamical elements. As mentioned before, the thresholded states

encoding different items are very robust to noise since they are superstable fixed points.

Finally this correspondence, or representation, is important for the process of encoding

information in an M -level representation and, as we shall see below, it is primarily

important for the process of searching the list for certain bits of information, by utilizing a

specific property of the system.

3.2 Searching for Information

Once we have a given list stored by setting appropriate thresholds on N dynamical

elements, we can query for the existence of a specific item in the list. Here we show

how the manner in which the information is encoded helps us preprocess the data such

that the effort required in the pattern matching searches is reduced. Specifically we will

demonstrate how we can use one global operational step to map the state of elements with

the matching item to an unique maximal state that can be easily detected. Note that

such an operation enables us to detect matches to strings/patterns (of length equivalent

to log2M binary bits) in one step. It would take typically log2M steps to do the same for

the case of binary encoded data.

When searching for a specific item in the list, we globally shift the state of all

elements of the list up by the amount that represents the queried item. Specifically the

state Xjn[m] of all the elements (j = 1, ..., N) is raised to Xjn[m] + Q[k], where Q[k] is a

search key given by:

Q[k] =1

2−T[k], (3–3)

80

where k is the item being searched for, and T[k] its respective threshold. This addition

shifts the interval that the list elements can span, from [r, 12− r] to [r +Q[k], 1

2− r +Q[k]],

where Q[k] is the globally applied shift. Note that what we are searching for is the

representation of the item, not the item itself. For example, we can encode each letter of

the alphabet by a number, such that the lowest threshold Tj[1] represents the letter A, the

next highest Tj[2] represents B, etc. When we search for A, we are really searching for the

element with a state with threshold Tj[1].

The item being searched for is encoded in a manner “complimentary” to the encoding

of the items in the list (much like a key that fits a particular lock); i.e. Q[k]+T[k] adds up

to 12. This guarantees that only the element(s) matching the item being searched for will

have its state shifted to 12. The value of 1

2is special in that it is the only state value that

on the subsequent update of the system will reach the value of 1.0, which is the maximum

state value for the Tent map. So only the elements holding an item matching the queried

item will reach the extremal value 1.0 on the dynamical update following a search query.

Note that the important feature here is the nonlinear dynamics mapping uniquely the

state 12to 1, while all other states (both higher and lower than 1

2) get mapped to values

lower than 1. See Figure 3-2 for a schematic of this process.

The salient characteristic of the point 12is the fact that it is the unique critical point,

and so it acts as “pivot” point for the nonlinear dynamical folding that will occur on the

interval [r + Q[k], 12− r + Q[k]] during the next update. This provides us with a single

global monitoring operation to push the state of all the elements matching the queried

item to the unique maximal point in parallel. The crucial ingredient is the use of the

existing critical point in the dynamical mapping to implement selection. Chaos is not

strictly necessary here. It is evident that for unimodal maps higher nonlinearities allow

larger operational ranges for the search operation and also enhance the resolution of the

encoding. For the Tent map specifically, it can be shown that the minimal nonlinearity

necessary for the above search operation to work is operation in the chaotic region.

81

Figure 3-2. Schematic representation of the changes in the state of an element for (i) amatching queried item, (ii) an item “higher” than the queried item, and (iii)an item “lower” than the queried item. The “key” value used is Q = 1

4, so the

matched item as a value of 0.25. The behaviour of three values is shown 0.1,0.25 and 0.3. It is clear that the application of the “key” does not seem torelatively affect the three values, a simple linear translation. The applicationof the map though, indicated by M, clearly maps both 0.1 and 0.3 to lowerstates than the maximal state, acquired solely by 0.25.

82

Another specific feature of the tent map is that its piecewise linearity allows the encoding

and search operation to be very simple indeed.

Of course to complete the search we must now detect the maximal state located at

1. This can be accomplished in a variety of ways. For example, one can simply employ a

level detector to register all elements at the maximal state. This will directly give the total

number of matches, if any. So the total search process is rendered simpler as the state

with the matching pattern is selected out and mapped to the maximal value, allowing

easy detection. Even more, by relaxing the detection level by a prescribed “tolerance”,

we can check for the existence within our list of numbers or patterns that are close to the

item or pattern being searched for. In this case “close to” means “having a representation

that is close to the representation of the item for which we are searching for”. Using the

earlier example of English letters of the alphabet encoded using the lowest threshold

Tj[1] for A, the next higher threshold for B, etc., relaxing the detection threshold a small

amount allows us to find mistyped words, where L or N were substituted for M or where

X or Z were substituted for Y. However, if we had chosen our representation such that

the ordering put T and U before and after Y (as is the case on a standard QWERTY

keyboard), then our relaxed search would find spellings of “bot” or “bou” when “boy” was

intended. Thus “nearness” is defined by the choice of the representation and can be chosen

advantageously depending on the intended use. Figure 3-5 gives an illustrative example of

detecting such inexact matches.

So nonlinear dynamics works as a powerful “preprocessing” tool, reducing the

determination of matching patterns to the detection of maximal states, an operation that

can conceivably be accomplished by simple addition and in parallel.

3.3 Encoding, Storing and Searching: An Example

Consider the case where our data is English language text, encoded as described

above by an array of tent maps. In this case the distinct items are the letters of the

English alphabet. As a result M = 26 and we obtain r = 154= 0.0185185... from Equation

83

3–1, and the appropriate threshold level for each item is obtained from Equation 3–2.

More specifically, consider as our list the sentence “strawberry fields”. Each letter1 in

this sentence is an element of the list with a value selected from our 26 possible values and

can be encoded using the appropriate threshold, as in Figure 3-3(a).

Now the list, as encoded above, can be searched for specific items. Figure 3-3 presents

the example of searching for the letter “l”. To do so the search key value corresponding

to letter “l” (from Equation 3–3, Q[l] = 1554) is added globally to the state of all elements.

Then through their natural evolution, at the next time step the state of the element(s)

containing the letter “l” is maximized. In Figure 3-4 we performed an analogous query for

the letter “e”, which is present twice in our list, to show that multiple occurrences of the

same item can be detected. Finally in Figure 3-5 we search for an item that is not part of

our given list, the letter “x”. As expected Figure 3-5(c) shows that none of the elements

are maximized. By lowering the detection level to the value 1 − (2 ∙ r), we have detected

whether adjacent items to the queried one are present. Specifically we have detected that

the letters “w” and “y” are contained in the list. This demonstrates that inexact matches

can also be found by this scheme.

3.4 Discussion

A significant feature of the presented search method is that it employs a single

simple global shift operation and does not entail accessing each item separately at any

stage. It achieves this through the use of nonlinear folding to select out the matched

item, and this nonlinear operation is the result of the natural dynamical evolution of

the elements. So the search effort is considerably simplified because it uses the native

responses of the nonlinear dynamical elements. We can then think of this as a natural

application, at the machine level, in a computing machine consisting of chaotic modules

[25–27, 32, 69, 70, 76, 81–85]. It is also equally potent as a special-applications “search

1 The space between the words is ignored.

84

Figure 3-3. Searching for “l”. (a) Threshold levels encoding the sentence“strawberry fields”, bars marked as �; (b) the search key value for letter “l”is added to all elements, bars marked as � ; (c) the elements update to thenext time step, bars marked as �. For clarity we marked solid black anyelements that reach the detection level.

chip”, which can be added on to regular circuitry and should prove especially useful in

machines, which are repeatedly employed for selection/search operations.

In terms of the processor timescale, the search operation requires one dynamical

step, namely one unit of the processor‘s intrinsic update time. The principal point here

is the scope for parallelism that exists in our scheme. This is due to the selection process

occurring through one global shift, which implies that there is no scale-up (in principle)

with size N . Additionally conventional search algorithms work with ordered lists, and the

time required for ordering generically scales with N as O(N logN). Here in contrast, there

is no need for ordering, and this further reduces the search time.

Regarding information storage capacity, note that we employ an M -state encoding,

where M can be very large in principle. This offers much gain in encoding capacity. As

85

Figure 3-4. Searching for “e”. (a) Threshold levels encoding the sentence“strawberry fields”, bars marked as �; (b) the search key value for letter “e”is added to all elements, bars marked as � ; (c) the elements update to thenext time step, bars marked as �. For clarity we marked solid black anyelements that reach the detection level.

in the example we present above, the letters of the alphabet are encoded by one element

each; binary coding would require much more hardware to do the same. Specifically,

consider the illustrative example of encoding a list of names, and then searching the list

for the existence of a certain name. In the current ASCII encoding technique, each ASCII

letter is encoded into two hexadecimal numbers or 8 bits. Assuming a maximum name

length of k letters, this implies that one has to use 8 ∙ k binary bits per name. So typically

the search operation scales as O(8kN). Consider in comparison what our scheme offers: if

base 26 (“alphabetical” representation) is used, each letter is encoded into one dynamical

system (an “alphabit”). As mentioned before, the system is capable of this dense encoding

as it can be controlled on to 26 distinct fixed points, each corresponding to a letter. Again

assuming a maximum length of k letters per name, one needs to use k “alphabits” per

86

Figure 3-5. Searching for “x”. (a) Threshold levels encoding the sentence“strawberry fields”, bars marked as �; (b) the search key value for letter “x”is added to all elements, bars marked as � ; (c) the elements update to thenext time step, bars marked as �. It is clear that no elements reach thedetection level at 1.0; (d) By lowering the detection level we can detectwhether items “adjacent” to “x” are present. For clarity we marked solid blackany elements that reach the detection level (“w” and “y”).

name. So the search effort scales as kN . Namely, the storage is 8 times more efficient

and the search can be done roughly 8 times faster as well! In general if base S encoding

is employed, for example where S is the set of all possible names (size(S) ≤ N), then

each name is encoded into one dynamical system with S fixed points (a “superbit”). So

one needs to use just 1 “superbit” per name, implying that the search effort scales simply

as N , i.e. 8k times faster than the binary encoded case. Even more ,in practice the final

step of detecting the maximal values can conceivably be performed in parallel. This would

reduce the search effort to two time steps (one to map the matching item to the maximal

value and another step to detect the maximal value simultaneously). In that case the

search effort would be 8kN times faster than the binary benchmark.

87

Alternate ideas to implement the increasingly important problem of search have

included the use of quantum computers [80]. However, our nonlinear dynamical scheme

has the distinct advantage that the enabling technology for practical implementation

need not be very different from conventional silicon devices. Namely, the physical design

of a dynamical search chip should be realizable through conventional CMOS circuitry.

Implemented at the machine level, this scheme can perform unsorted searches efficiently.

CMOS circuit realizations of chaotic systems, like the tent map, already operate beyond

the region of 1 MHz [86, 87]. Thus a complete search for an item comprising of search key

addition, update, threshold detection, and list restoration can be performed at 250 kHz,

regardless of the length of the list. Even more though, nonlinear systems are abundant in

nature, and so embodiments of this concept can be conceived in many different physical

systems ranging from fluids to electronics to optics. Potentially good candidates for

physical realization of the scheme include nonlinear electronic circuits and optical devices

[88]. Also systems such as single electron tunneling junctions [89], which are naturally

piecewise linear maps, can conceivably be employed to make such search devices.

In summary we have presented a method to efficiently and flexibly store information

using nonlinear dynamical elements. We demonstrate how a single element can store M

distinct items, where M can be large and can vary to best suit the nature of the data

being stored and the application at hand. Namely, we have information storage elements

of flexible capacity, capable of naturally storing data in different bases or in different

alphabets or with multilevel logic. This cuts down space requirements by log2M in

relation to elements storing via binary bits. Further we have shown how this method of

storing information can be naturally exploited for searching of information. In particular,

we demonstrated a method to determine the existence of an item in an unsorted list. The

method involves a single global shift operation applied simultaneously to all the elements

comprising the list, such that the next dynamical step “pushes” the element(s) storing the

matching item (and only those) to a unique, maximal state. This extremal state can then

88

be detected by a simple level detector, directly giving the number of matches. Even more

the maxima state can be treated as a maximal range, in which case approximate matches

are identified as well.

89

CHAPTER 4A SIMPLE ELECTRONIC IMPLEMENTATION OF CHAOTIC COMPUTATION

This chapter is a short exposition of the results of our publication [34] concerning

an iterated map with a very simple (i.e. minimal) electronic implementation. We first

propose and characterize the map and then provide the circuit to implement the map. We

proceed to determine control thresholds for flexibly representing the five fundamental logic

gates and demonstrate how this map (and circuit) can be used to implement the search

algorithm introduced in Chapter 3.

4.1 An Iterated Nonlinear Map

We begin by considering an iterated map governed by the following equation:

xn+1 =α ∙ xn1 + xβn

, (4–1)

where α and β are system parameters. Figure 4-1 shows the bifurcation diagrams for

different values of α and β. It is evident that this map yields dynamics ranging from fixed

points through chaos. It is also clear that the map follows the period-doubling route to

chaos with respect to α, and it does so as well with respect to β. In the following sections

we will consider the map in the chaotic regime, with α = 2 and β = 10, namely the chaotic

map given by:

xn+1 =2 ∙ xn1 + x10n

, (4–2)

This operating point is indicated by the dotted line in the bottom right panel of

Figure 4-1 and the graphical form of this map is presented in Figure 4-2.

4.2 Threshold Control Chaos into Different Periods

Using the map given by Equation 4–2, we wish to construct a system that can

represent M distinct states, where M can be large. The size of M will be limited only by

our ability to distinguish one state from the next in the presence of noise. To do this we

90

Figure 4-1. Bifurcation diagram of the iterated map in Equation 4–1 for various values ofα and β. The dotted line in the bottom right panel, indicates the chosenoperating point as prescribed by Equation 4–2. Here x∞ is the value taken bythe map after initial transients have died out.

use the simple and easily implementable threshold control mechanism described in Section

1.2.3. Specifically we place under control the state variable x as:

xn+1 =

2∙xn1+x10n

, for xn+1 ≤ x∗,

x∗, for x∗ < xn+1,

(4–3)

where x∗ is the imposed threshold value. The effect of this control is to limit the available

phase space by clipping the state variable. In this method no parameters are adjusted,

and only one state variable is occasionally reset. Note that this scheme is computationally

simple and requires no costly run-time computations.

Figure 4-3 illustrates the behaviour of the system under varying thresholds.

Depending on the value of the threshold x∗, this control method produces a wide variety of

nonlinear dynamical behaviours ranging from fixed points to periodic behaviour of various

91

Figure 4-2. Graphical form of the map to be implemented by an electronic circuit. Theparameters for this form are set at α = 2 and β = 10.

periodicities to chaos. As indicated in the figure, the system is controlled to fixed points

for thresholds for x∗ < 1. When the threshold is above unity, many different periodic (as

well as chaotic) orbits become available.

4.3 Electronic Analog Circuit: Experimental Results

The realization of the discrete map of Equation 4–2 in circuitry is depicted in

Figure 4-4. In the circuit Vin and Vo denote input and output voltages and in terms of the

equation xn and xn+1, respectively. A simple nonlinear device is constructed by coupling

two complementary (n-channel and p-channel) (Q1, Q2) junction field-effect transistors

(JFETs) [90] mimicking the nonlinear characteristic curve f(x) = 2x1+x10

. The voltage

across resistor R1 is amplified by a factor of 5 using the operational amplifier U1 in order

to scale the output voltage back into the range of the input voltage, a necessary condition

for a circuit based on a map. The resulting voltage characteristics of the nonlinear device

are depicted in Figure 4-5, compare with the mathematical form of the map in Figure 4-2.

92

Figure 4-3. Effect of threshold value x∗ on the dynamics of the system given by Equation4–3.

In order though to realize the map of Equation 4–3, we require two more sample and

hold circuits, in addition to a threshold controller circuit, see Figure 4-6. The first sample

and hold (S/H) circuit holds the input signal (xn) in response to a clock signal CK1. The

output from this sample and hold circuit is fed as input to the nonlinear device for the

subsequent mapping, that is Equation 4–2. A second sample and hold (S/H) circuit takes

the output from the nonlinear device in response to a clock signal CK2. In lieu of control,

the output from the 2nd sample-and-hold circuit (xn+1) closes the loop as the input to 1st

sample-and-hold circuit, through the threshold control circuit. The main purpose of the

two sample-and-hold circuits is to introduce discreteness into the system and additionally

93

Figure 4-4. Circuit diagram of the nonlinear device of Equation 4–3. (Left) Intrinsic(resistorless), complementary device made of two (n-type and p-type) JFETs.Q1: 2N5457, Q2: 2N5460. (Right) Amplifier circuitry to scale the outputvoltage back into the range of the input voltage. R1: 535 Ω, U1: AD712op-amp, R2: 100 kΩ and R3: 450 kΩ. Here Vin = xn and Vo = xn+1.

to set the iteration speed. To implement the control for nonlinear dynamical computing,

the output from the 2nd sample and hold circuit is input to the threshold controller, as

that described by Equation 4–3. The output from this threshold controller then becomes

the input to the 1st sample-and-hold circuit.

In Figure 4-6, the sample and hold circuits are realized with National Semiconductors

sample and hold IC LF398, triggered by delayed timing clock pulses CK1 and CK2 [70].

Here a clock rate of either 10 kHz or 20 kHz may be used. The threshold controller circuit

is shown in Figure 4-7 is realized with an AD712 operational amplifier, a 1N4148 diode, a

1 kΩ series resistor and the threshold control voltage, x∗(= Vcon).

The Figure 4-8(a) displays the uncontrolled chaotic waveform and the Figures 4-8(b-d)

show representative results of the chaotic system under different threshold values

x∗(= Vcon). It is clear that adjusting the threshold yields cycles of varying periodicities.

Also, note that simply setting the threshold beyond the bounds of the attractor (5 V)

94

Figure 4-5. Voltage response characteristics of the nonlinear device, based on Equation4–2 and circuit of Figure 4-4.

gives back the original dynamics, and so the controller is easily switched on and off. A

detailed comparison shows complete agreement between experimental observations and

analytical results. For instance, the threshold that needs to be set in order to obtain a

certain periodicity and the trajectory of the controlled orbit can be worked out exactly

through symbolic dynamics techniques. Further, the control transience is very short here

(typically of the order of 10−3 times the controlled cycle length) and the perturbation

involved in threshold control is usually small. This method is then especially useful in

the situation where one wishes to design controllable components that can switch flexibly

between different behaviours. Calibrating the systems characteristics at the outset with

95

Figure 4-6. Schematic diagram for implementing the threshold controlled nonlinear map.CK1 and CK2 are clock timing signals, while the modules designated S/H aresample-and-hold circuits.

Figure 4-7. Circuit diagram of the threshold controller. Vin and Vo are the input andoutput, D is a 1N4148 diode, R = 1 kΩ, and U2 is an AD712 op-amp.Vcon = x

∗ (controller input voltage).

96

respect to threshold gives one a look-up table directly and simply to extract widely

varying temporal patterns.

Figure 4-8. PSPICE simulation results of the experimental circuit. The ordinate is xn andthe abscissa is the discrete time n measured in ms. (a) Uncontrolled chaos:x∗ = 6 V, (b) period 5 cycle: x∗ = 4 V, (c) period 2 cycle: x∗ = 3.7 V and (d)period 1 cycle: x∗ = 3.5 V.

4.4 Fundamental Logic Gates with a Chaotic Circuit

Here we explicitly show how, by using the threshold controlled map of Equation 4–3,

we obtains the clearly defined logic gate operations NOR, NAND, AND, OR, and

XOR. The state of the system is represented by the state value of x. The initial

state of the system is represented as x0. In our method all five basic gate operations

involve the following steps: specification of x0 based on the operation and the inputs

through threshold control, nonlinear update (evolution of the circuit dynamics), and

output interpretation through threshold “monitoring” in the spirit of Excess Overflow

Propagation, from Section 1.2.3. Specifically:

1. Inputs and Programming; x0 = xprog + xI1 + xI2 . Here xprog is a programming

shift that fixes the initial state x0 of the system, based on the gate to be operated.Letting a finite voltage δ denote a logical 1, we set xIi = δ for an input of logical 1and xIi = 0 for an input of logical 0.

97

2. Nonlinear update; i.e., x0 → f(x0), where f(x) is the nonlinear function, given byEquation 4–2.

3. Thresholding to obtain the output Z defined as:

Z =

{0, for f(x) ≤ x∗,

f(x)− x∗, for x∗ < f(x),where x∗ = 1 is the threshold reference signal; which is set the same for all gates unlike

previously in the case of the Logistic map in Section 2.3.1. The output is interpreted as

logic output 0 if Z = 0 and logic output 1 if 12∙ δ < Z. Since the system is nonlinear (and

may be chaotic), in order to specify the initial x0 accurately in hardware experiments,

one needs a controlling mechanism. Here we will employ a threshold controller to set the

initial x0. So in this example we will use the clipping action of the threshold controller to

achieve the initialization. A comparator is used to recover the output from the state of Z.

The logical operations are defined by the input to output mappings depicted in the truth

table of Table 4-1.

Table 4-1. Truth-table for the five fundamental logic gates NOR, NAND, AND, OR andXOR.

I1 I2 NOR NAND AND OR XOR0 0 1 1 0 0 00 1 0 1 0 1 11 0 0 1 0 1 11 1 0 0 1 1 0

From the definition of f(x), x∗ and the above truth table, we obtain a set of

inequality conditions that need to be satisfied simultaneously, shown in Table 4-2. Note

that the symmetry with respect to the inputs reduces the four conditions in the truth

table of Table 4-1 to three distinct conditions, with rows 2 and 3 of Table 4-1 leading to

the single condition of row 2 in Table 4-2.

The above inequalities have many possible solutions depending on the size of δ. By

setting δ = 0.3 we can easily solve the equations for the different programming shifts that

each gate requires. The specific xprog values for the five different logical operations are

listed in Table 4-3.

98

Table 4-2. Necessary and sufficient conditions to be satisfied by a chaotic element in orderto implement the logical operations NOR, NAND, AND, OR and XOR. Wheref(x) is given by Equation 4–2 and 1 is the monitoring threshold (x∗) ofinterpretation of the logic output.

I1 I2 NOR NAND AND0 0 1 < f(xprog) 1 < f(xprog) f(xprog) ≤ 10/1 1/0 f(xprog + δ) ≤ 1 1 < f(xprog + δ) f(xprog + δ) ≤ 11 1 f(xprog + 2 ∙ δ) ≤ 1 f(xprog + 2 ∙ δ) ≤ 1 1 < f(xprog + 2 ∙ δ)

I1 I2 OR XOR0 0 f(xprog) ≤ 1 f(xprog) ≤ 10/1 1/0 1 < f(xprog + δ) 1 < f(xprog + δ)1 1 1 < f(xprog + 2 ∙ δ) f(xprog + 2 ∙ δ) ≤ 1

Table 4-3. Numerical values of xprog for implementing logical operations NOR, NAND,AND, OR and XOR, based on Equation 4–2.

Operation NOR NAND AND OR XORxprog 0.9138 0.6602 0.0602 0.3602 0.45

By setting up a map the with initial condition x0, as defined above, we allow the

map to update to a new value x1 = f(x0) and compare this value to the monitoring

threshold x∗ = 1, if the new state of the map is greater than the threshold a logical 1

is the output, if the new state is less than the threshold a logical 0 is the output. The

updated states of a chaotic element, following Equation 4–2, satisfying the conditions

of Table 4-2, with xprog values given in Table 4-3 are shown in Table 4-4. The circuitry

described in the section above has been tested to do the logic operations described here

and shows complete agreement with the simulation results.

Table 4-4. Updated state values, x1 = f(x0), of a chaotic element satisfying the conditionsin Table 4-2 in order to implement the logical operations NOR, NAND, AND,OR and XOR.

Operation NOR NAND AND OR XORf(xprog) 1.3 1.3 0.1204 0.7204 0.8997f(xprog + δ) 0.3057 1.1525 0.7204 1.3 1.42f(xprog + 2 ∙ δ) 0.0472 0.2270 1.3 1.1525 0.7988

We have presented a proof of principle device that demonstrates the capability of this

nonlinear map to implement the five fundamental logic gates; it does this by exploiting the

nonlinear response of the system. The main benefit is its ability to exploit a single chaotic

99

element to reconfigure into different logic gates through a threshold based morphing

mechanism. Contrast this to a conventional field programmable gate array element, where

reconfiguration is achieved through switching between multiple single purpose gates.

4.5 Encoding and Searching a Database Using Chaotic Elements

In the spirit of Chapter 3 we apply that method to the map, and circuit, given by

Equation 4–3. Specifically we show how this map can be utilized to stably encode and

store various items of information (such as patterns and strings) to create a “database”.

Further we demonstrate how this storage method allows to efficiently determine the

number of matches (if any) to some specified item [28]. Consider an array of elements each

of which evolves according to Equation 4–3. The nonlinear dynamics of the array elements

will be utilized for flexible capacity storage, as well as for pre-processing data for exact

(and inexact) pattern matching tasks.

Encoding information. We consider a “database” of length N and each member

of the database is encoded in an element obeying Equation 4–3, we index these elements

with j = {1, 2, 3, ...N}, so the state of the whole array, at a time n, can be represented

as Xjn. At the same time the database is made up of items from an “alphabet” of total

number of unique items M , indexed with m = {1, 2, 3, ...M}. We correlate each item m

with a threshold, x∗, for Equation 4–3, such that the element is confined on a fixed point

of period 1, we define T j[m] as the threshold for the jth element encoding the mth item.

For this map, thresholds ranging from 0 to 1 yield fixed points, as depicted in Figure 4–3.

Namely Xjn = Tj[m], for all time n, when the threshold is chosen as 0 < T j[m] < 1.

This can be obtained exactly from the fact that x < f(x), ∀x ∈ (0, 1), implying that the

subsequent iteration of a state at T j[m] will always exceed T j[m] and thus be reset to

T j[m].

100

In our encoding, the thresholds are chosen from the interval (0, 12), namely a sub-set

of the fixed-point window (0, 1)1 . Without loss of generality, consider each item to be

represented by an integer z from the range [1,M ]. Defining a resolution r between each

threshold as:

r =1

2∙1

M, (4–4)

gives a lookup map from the encoded integer to the threshold, relating the integers z in

the set [1,M ] to thresholds T j[m] in the range [r, 12], by:

T j[m] = z ∙ r. (4–5)

Therefore we obtain a direct correspondence between a set of integers ranging from

1 to M , where each integer represents an item, and a set of M threshold values. So we

can store N database elements by setting appropriate thresholds (via Equation 4–5) on

N dynamical elements. Clearly from Equation 4–5, if the threshold setting has better

resolution (smaller r), then a larger range of values can be encoded. Note however that

precision is not a restrictive issue here, as different data representations can always be

chosen in order to suit a given precision of the threshold mechanism.

Processing Information. Once we have a given database stored by setting

appropriate thresholds on N dynamical elements, we can query for the existence of a

specific item in the database in one global operational step. This is achieved by globally

shifting the state variable of all elements of the database up by an amount that represents

the item being searched for.

1 Actually we can use as much as the interval (0, 0.8027), since 0.8027 is the pre-imageof the maximum.

101

Noting that the maximal state variable value for this system is 1.4449, one raises the

state Xjn of each element j to Xjn +Q[k], where Q[k] is a search key given by:

Q[k] = 0.8027− T [k], (4–6)

where k is the index of the integer(item) being queried for, k ∈ z. 0.8027 is the unique

value of this system that evolves to the maximal value 1.4449 on an iteration of the

system. So the value of the search key is simply 0.8027 (the pre-image of the maximal

state variable value) minus the threshold value corresponding to the item being searched

for, given k ∈ z we have T [k] = k ∙ r.

The addition of the “search key”, Q[k] shifts the interval that the database elements

can span, from [r, 12] to [r + Q[k], 1

2+ Q[k]]. Since Q[k] + T [k] adds up to 0.8027, it

is guaranteed that only the element(s) matching the item being queried for will have

its(their) state shifted to 0.8027, which is the only state which after the subsequent

iteration will maximize to the value of 1.44492 . So the total search process is rendered

simple as the state with the matching pattern is selected out and mapped to the maximal

value, allowing easy detection. Further, by relaxing the detection level by a prescribed

“tolerance”, we can check for the existence within our database of numbers or patterns

that are “close to”3

Representative Example. Consider the case where our data is English language

text, encoded as described above on a letter by letter basis by an array of maps, following

Equation 4–5. In this case the distinct items are the letters of the English alphabet and

we have M = 26. We obtain r = 152≈ 0.0192 from Equation 4–4 and the appropriate

threshold level for each item is obtained via Equation 4–5. More concretely, consider as

2 Note all other states (both higher and lower than 0.8027) get mapped to values lowerthan 1.4449.

3 Where “close to” is defined by the designer of the database.

102

our database the phrase “the quick brown fox”; each letter in this phrase is an element of

the database and can be encoded using the appropriate threshold, as in Figure 4-9(a).

Figure 4-9. Searching for “b”. (a) Threshold levels encoding the phrase“the quick brown fox”, bars marked as �; (b) the search key value for theletter “b” is added to all elements, bars marked as � ; (c) the elements updateto the next time step, bars marked as �. For clarity we mark black theelements that reached the detection level.

Now we query the database regarding the existence of specific items. Figure 4-9

presents the example of querying for the letter “b”. To do so the search key value

corresponding to letter “b” ( 252) is added globally to the states of all elements, Figure

4-9(b). Then through their natural evolution, upon the next time step, the state(s) of the

element(s) containing the letter “b” is(are) maximized, Figure 4-9(c). In Figure 4-10 we

perform an analogous query for the letter “o”, which happens to be present twice in our

database to show that multiple occurrences of the same item can be detected. Finally in

Figure 4-11 we query for an item that is not part of our given database, the letter “d”. As

expected Figure 4-9(c) shows that none of the elements are maximized. By lowering the

103

Figure 4-10. Searching for “o”. (a) Threshold levels encoding the phrase“the quick brown fox”, bars marked as �; (b) the search key value for theletter “o” is added to all elements, bars marked as � ; (c) the elementsupdate to the next time step, bars marked as �. For clarity we mark blackany elements that reached the detection level.

detection level to the value 1.4449− f(0.8027− r) = 1.4411, just “one step” down from the

maximal, we detect whether items “adjacent” to the desired one are present. Specifically

we detect that the letters “c” and “e” are contained in our database. This demonstrates

that inexact matches can also be found, just as easily.

4.6 Conclusion

In summary, we introduced a simple map having rich nonlinear dynamics, and

a simple electronic circuit realization. Then we demonstrated the direct and flexible

implementation of the five basic logic gates using this simple nonlinear map (circuit).

Further, we showed how the dynamics of this map can be utilized to provide an efficient

database search method.We have experimentally implemented the electronic circuit analog

104

Figure 4-11. Searching for “d”. (a) Threshold levels encoding the phrase“the quick brown fox”, bars marked as �; (b) the search key value for theletter “d” is added to all elements, bars marked as � ; (c) the elementsupdate to the next time step, bars marked as �. It is clear that no elementsreach the detection level at 1.4449; (d) By lowering the detection level we candetect whether items “adjacent” to “d” are present (“c” and “e”).

of this nonlinear map and have demonstrated the efficacy of the threshold controller in

yielding different controlled responses from this map circuit.

105

CHAPTER 5LOGIC OPERATIONS FROM EVOLUTION OF DYNAMICAL SYSTEMS

In this chapter we propose the direct and flexible implementation of logic operations

using the dynamical evolution of a nonlinear system [33]. The concept involves the

observation of the state of the system at different time instances to obtain different logic

outputs. We explicitly implement the basic NAND, AND, NOR, OR and XOR logic gates,

as well as multiple-input XOR and XNOR logic gates. Further we demonstrate how the

single dynamical system can do more complex operations such as bit by bit addition in

just two iterations. The concept uses the nonlinear characteristics of the time dependence

of the state of the dynamical system to extract different responses from the system. The

highlight of this method is that a single nonlinear system is capable of yielding a time

sequence of different logic operations. Further we explicitly demonstrate, through the three

examples, how results from this method can be obtained by varying any of the “defining

variables” (x0, xinit, δ, n).

5.1 Generation of a Sequence of (2-input) Logic Gate Operations

We outline a method for obtaining the five basic logic gates using different dynamical

iterates of a single nonlinear system. In particular consider a chaotic system whose state

is represented by a value x. The state of the system evolves according to some dynamical

rule. For instance, the updates of the state of the element from time n to n + 1 may be

well described by a map, i.e., xn+1 = f(xn), where f(x) is a nonlinear function. Now this

element receives inputs before the first iteration (i.e., at n = 0) and outputs a “signal”

after evolving for a (short) specified time or number of iterations. The method can be

applied for any sequence of the gates, for illustrative purposes we chose the sequence

NAND, AND, NOR, XOR and OR (see Table 5-1 for the truth table).

In general the method involves the following steps:

1. Input definition (for a 2 input operation): x0 = xinit + xI1 + xI2 , where xinit is the

initial state of the system. (Comparable to xprog in previous chapters, but now itis not defining a single gate but a sequence of gates, or more general, operations).

106

Table 5-1. The truth table of the five basic logic operations NAND, AND, NOR, XOR,OR.

I1 I2 NAND AND NOR XOR OR0 0 1 0 1 0 00 1 1 0 0 1 11 0 1 0 0 1 11 1 0 1 0 0 1

before data inputs are introduced, x0 is the actual initial state of the system thatincludes the data to be operated on. As previously we set xIi = δ for the logicalinput Ii = 1, and x

Ii = 0 for the logical input Ii = 0. So we need to consider thefollowing three cases:

(a) Both I1, and I2 are 0 (row 1 in Table I), i.e. the initial state of the system is:x0 = x

init + 0 + 0 = xinit.

(b) Either I1 = 0 and I2 = 1, or I1 = 1 and I2 = 0 (row 2 or 3 in Table I), i.e. theinitial state is: x0 = x

init + 0 + δ.

(c) Both I1 and I2 are 1 (row 4 in Table I), i.e. the initial state is:x0 = x

init + δ + δ = xinit + 2 ∙ δ.

2. Chaotic evolution over some prescribed number of steps, i.e. fn(x0)→ xn, for 1 < n.

3. The evolved states fn(x0) yield the logic output, at each n, as follows:Logic Output = 0, if fn(x0) ≤ x∗n, Logic Output = 1, if fn(x0) > x

∗n, where x

∗n is a

reference monitoring value, at time instance n.

Since the system is chaotic, in order to specify the initial x0 accurately we employ

the threshold control mechanism, see Section 1.2.3, we note that this mechanism can be

invoked at any subsequent iteration as well. For logic recovery, the updated or evolved

value of f(x) is compared with x∗n value using a comparator action, in the spirit of Excess

Overflow Propagation, again as in Section 1.2.3.

In order to obtain all the desired input-output responses of the different gates, as

displayed in Table 5-1, we need to satisfy the conditions enumerated in Table 5-2. Note

that the symmetry of inputs to outputs reduces the four conditions in the truth Table

5-1 to three distinct conditions, with 2textnd and 3rd row of Table 5-1 leading to the 2nd

condition of Table 5-2.

107

Table 5-2. Necessary and sufficient conditions to be satisfied by a chaotic element in orderto implement the logic operations NAND, AND, NOR, XOR and OR onsubsequent iterations. Here xinit = 0.325 and δ = 0.25, for all n. Whilex∗n = 0.75, for n = {1, 2, 3, 4}, that is for NAND, AND, NOR, XOR logicoperations, and x∗5 = 0.4 for OR logic operation.

Logic gate NAND AND NOR XOR ORIteration (n) 1 2 3 4 5Condition 1:

x1=f(x0) > x∗1 f(x1) ≤ x∗2 f(x2) > x

∗3 f(x3) ≤ x

∗4 f(x4) ≤ x

∗5Logic input

(0,0)x1 = 0.88 x2 = 0.43 x3 = 0.98 x4 = 0.08 x5 = 0.28x0 = 0.325

Condition 2:x1=f(x0) > x

∗1 f(x1) ≤ x∗2 f(x2) ≤ x∗3 f(x3) > x

∗4 f(x4) > x

∗5Logic input

(0,1) or (1,0)x1 = 0.9775 x2 = 0.088 x3 = 0.33 x4 = 0.872 x5 = 0.45x0 = 0.325 + δ

x0 = 0.575Condition 3:

x1=f(x0) ≤ x∗1 f(x1) > x∗2

textf(x2) ≤ x∗3 f(x3) ≤ x

∗4 f(x4) > x

∗5Logic input

(1,1)x1 = 0.58 x2 = 0.98 x3 = 0.1 x4 = 0.34 x5 = 0.9x0 = 0.325 + 2∙δ

x0 = 0.825

So given dynamics f(x), we must find values of threshold(s) x∗n, initial state(s) xinit

and δ satisfying the conditions derived from the specific truth table to be implemented.

Using, as usual, the Logistic map, Equation 1–1, we incorporated in Table 5-2 actual

values of xinit and x∗n for n = 1, 2, 3, 4, 5, which satisfy the conditions imposed by the truth

table considered. For illustrative purposes, the graphical representation of five iterations of

the Logistic map is shown in Figure 5-1, displaying the results in Table 5-2.

In summary, the inputs setup the initial state xinit+xI1+xI2 . Then the system evolves

over n iterative time steps to each updated state xn. The evolved state is compared to a

monitoring threshold x∗n, at every n. If the state at iteration n, is greater than the

threshold a logical 1 is the output and if the state is less than the threshold a logical 0 is

the output. This process is repeated for each subsequent iteration.

Note that in the above example we present results for which xinit is not varied and

x∗n is varied; the emphasis is on the specific behaviour that represents a computational

108

Figure 5-1. Graphical representation of five iterations of the Logistic map. Three differentinitial conditions are considered, each representing one of the three cases oftwo logic inputs, (0,0) by N, (0,1)/(1,0) by N, (1,1) by M. At each iterationcomparison with a monitoring threshold is performed, 0.75 for n = {1, 2, 3, 4},0.4 for n = 5. The results from two gates are also shown, the circles markn = 1 (the NAND gate) compare with 0.75 and the colouring corresponds tothe appropriate case of data; the squares mark n = 5 (the OR gate) comparewith 0.4, and again the colouring is appropriate to the data.

109

task and not the “actual” values as such. The case of δ is slightly more complicated since

it actually represents a constant “entity”, nevertheless varying its “actual” value is also

possible, but more care is needed, hence in this example we simply set δ = 0.25.

Therefore in a more general context, we are relating inputs and required outputs

with specific behavioural patterns, at the same time we can say the reverse, we find the

behavioural pattern that can represent a needed operation. Hence the “actual” state

values are not of great importance, and we can generate “templates” like Figure 5-2.

To generate this specific template, we set x∗n = 0.65 and varied x0, in contrast to the

example above; once again δ was kept constant at 0.25. Basically the behaviour of each

combination of variables is then interpreted as one of the eight possible symmetric binary

operations. It is clear from this figure that we are not confined to n < 5, so the “length”

of the sequence of operations can be extended, and also the actual order of the sequence

of operations can be changed. Theoretically all operations and all sequences of operations

exist, under some combination of “actual” values of variables.

We should note that, as is clear from Figure 5-2, the range of each operation is

decreasing in size with increasing iterations, and since the dynamics are chaotic at

some point we will lose definition, as we mentioned above though the threshold control

mechanism can be invoked to “re-initialize” the system.

5.2 The Full Adder and 3-Input XOR and NXOR

This section is a direct extension of the previous section, hence it is comprised of

simply two demonstrations. We extend the above method for sequential logic operations

in two ways, first to more than just two data inputs, and second to more than just logic

gate operations. Specifically we show the implementation of the binary full adder and the

implementations of 3-input XOR and NXOR gates.

For these implementations we employ the usual three steps, but modified for 3 inputs

as follows:

110

Figure 5-2. Patterns of binary two input symmetric operations. We use the Logistic mapwith δ = 0.25, x∗n = 0.65, ∀n and vary x

init.

1. Input definition (for a 3 input operation): x0 = xinit + xI1 + xI2 + xI3 , as usual with

the addition of one “extra” input. In the context of the full adder, I1 correspondsto the input binary number A, I2 corresponds to the input binary number B, and I3corresponds to the carry input Cin (the carry from the “previous” positional digitaddition), as in Table 5-3. So we need to consider the following four cases:

(a) If all inputs are 0 (1st row in Table 5-3), i.e. the initial state of the system is:x0 = x

init + 0 + 0 + 0 = x0.

(b) If any one of the input equals 1 (2nd, 3rd and 5th row in Table 5-3), i.e. theinitial state is: x0 = x

init+0+0+δ = xinit+0+δ+0 = xinit+δ+0+0 = xinit+δ.

(c) If any two inputs equal to 1 (4th, 6th and 7th row in Table 5-3), i.e. the initialstate is: x0 = x

init +0+ δ+ δ = xinit + δ+0+ δ = xinit + δ+ δ+0 = xinit +2 ∙ δ.

111

(d) If all inputs equal to 1 (8th row in Table 5-3), i.e. the initial state is:x0 = x

init + δ + δ + δ = xinit + 3 ∙ δ.

2. Chaotic evolution (for two time steps), of the initial state given from the dataf2(x0)→ x2. We confine the time steps to only two to make clear the example of thefull adder. In general of course, further time steps can be also considered providingresults for other 3 input operations, as in Figure 5-2.

3. The evolved state fn(x0) yields the logic output as follows:Logic Output = 0, if fn(x0) ≤ x∗n,Logic Output = 1, if fn(x0) > x

∗n,

where x∗n is a monitoring threshold, with n = 1, 2 in the full adder example.

Table 5-3. The truth table of full adder, and necessary conditions to be satisfied. Usingthe Logistic map, state values x1 (iteration n = 1) and x2 (iteration n = 2) areused to obtain Cout and S respectively. Here x

∗1 = 0.8, x

∗2 = 0.4, x

init = 0.0 andδ = 0.23.

Input bit of Input bit of Input bit of Output Condition for OutputNumber (A) Number (B) Carry (Cin) Cout S Cout (x1) S (x2)

0 0 0 0 0 f(x0) ≤ x∗1 f(x1) ≤ x∗20 0 1 0 1 f(x0 +δ) ≤ x∗1 f(x1) > x

∗2

0 1 0 0 1 f(x0 +δ) ≤ x∗1 f(x1) > x∗2

0 1 1 1 0 f(x0 + 2∙δ) > x∗1 f(x1) ≤ x∗2

1 0 0 0 1 f(x0 +δ) ≤ x∗1 f(x1) > x∗2

1 0 1 1 0 f(x0 + 2∙δ) > x∗1 f(x1) ≤ x∗2

1 1 0 1 0 f(x0 + 2∙δ) > x∗1 f(x1) ≤ x∗2

1 1 1 1 1 f(x0 + 3∙δ) > x∗1 f(x1) > x∗2

The Full Adder. We now demonstrate how one can obtain the ubiquitous bit

by bit arithmetic addition, involving three logic inputs and two outputs, in consecutive

iterations, with a single one-dimensional chaotic element. In general the simple 1-bit

binary arithmetic addition requires a full adder logic which adds three individual bits

together (two bits being the digit inputs and the third bit assumed to be carry from

the addition of the next least-significant bit addition operation, known as “Cin”). A

typical full-adder requires two half-adder circuits and an extra XOR gate. In total, the

implementation of a full-adder requires five different gates (three XOR gates and two

AND gates). However in the present direct implementation by utilizing the dynamical

evolution of a single map, the Logistic map, we need only two iterations to implement a

112

full adder. The truth table and necessary conditions to be satisfied for the parallelized

full adder operation are given in Table 5-3. The Carry bit output Cout and the Sum bit

output S are respectively recovered from the first and second iterations of a Logistic map,

with δ = 0.23, xinit = 0.0 x∗1 = 0.8 and x∗2 = 0.4. Note in this example we have kept x

init

constant and varied the x∗n.

So basically each system takes the three inputs, the value of the two bits to be added

and the carry from the previous bit addition, and produces the carry for the next addition

on the very first update. This new carry can of course be immediately supplied to the next

system ready to perform the addition of the next bit, while the sum (S) is “calculated” on

the second update.

3-Input XOR and NXOR. Three or more input logic gates are advantageous

because they require less complexity in actual experimental circuit realization than that of

coupling conventional 2-input logic gates [91, 92]. Specifically the truth table for 3-input

XOR and XNOR logic gate operations, and the necessary conditions to be satisfied by

a map to perform these operations, are shown in Table 5-4. In this representative case,

we consider a different scenario, for both operations we use the 2nd iteration of the map,

so both operations can not be accomplished by the same system. We set the threshold

(x∗2) at the fixed value of 0.5 and so we accomplish the XOR gate with xinit = 0.0 and

the NXOR gate with xinit = 0.25, most importanly though we have an exact fixed valued

δ ≈ 0.25.

5.3 Conclusion

Our previous results have shown that a single nonlinear dynamical system, with

proper tuning of parameters and control inputs, can flexibly become any logic gate. We

did confine ourselves though, to gates of only two inputs and to patterns created by a

single iteration of a system.

The main result of this chapter is that the various temporal patterns embedded in

the dynamical evolution of nonlinear systems are capable of performing sequences of

113

Table 5-4. The truth table of the 3-input XOR and NXOR logic operations, necessaryand sufficient conditions to be satisfied by the map. State value x2 (iterationn = 2) is used for logic operation recovery. Here x∗ = 0.5 and δ ≈ 0.25.

I1 I2 I3 XOR(I1,I2,I3) XNOR(I1,I2,I3) XOR (xinit = 0.0) NXOR (xinit = 0.25)

0 0 0 0 1 x2 ≤ x∗ x2 − x∗ ≈ δ0 0 1 1 0 x2 − x∗ ≈ δ x2 ≤ x∗

0 1 0 1 0 x2 − x∗ ≈ δ x2 ≤ x∗

1 0 0 1 0 x2 − x∗ ≈ δ x2 ≤ x∗

0 1 1 0 1 x2 ≤ x∗ x2 − x∗ ≈ δ1 0 1 0 1 x2 ≤ x∗ x2 − x∗ ≈ δ1 1 0 0 1 x2 ≤ x∗ x2 − x∗ ≈ δ1 1 1 1 0 x2 − x∗ ≈ δ x2 ≤ x∗

logic operations in time (or iterates), and minimize the control that is needed, we only

invoke control mechanism on initialization, from there on we just monitor the state. The

implementation of a sequence of logic functions in time, as described above, is another

mechanism through which computer architectures based upon the chaos computing

approach can be optimized for better performance. Even further in this chapter we

introduced mechanisms for multi-input logic, and showed how in the context of chaotic

computation behaviours are more important than exact and specific values of variables,

providing even more flexibility in a physical realization.

With these fundamental ingredients in hand it becomes clear that exploiting not just

the pattern formation of nonlinear dynamical systems, but the formation of sequences of

such patterns, produced naturally by such systems, may prove a key ingredient towards

making nonlinear dynamical computational architectures a real alternative to conventional

static logic computer architectures.

114

CHAPTER 6MANIPULATING TIME FOR COMPUTATION

We have shown in Chapter 5 how time can be involved in the computational process,

specifically we have shown how specific “time instances” can be used for programming

a system to perform a specific computational operation. In the present chapter, in some

sense we extend the utility of time; we show how the dimension of time can be used for

a complete definition of a computational operation, that is we use time instances not

only for programming a system, but also for introducing to the system the “data” to

be manipulated. To achieve this we predefine the behaviour of the state of the system

within a “clock cycle” and work with the fact that this behaviour is not constant within

the given cycle and so at different time instances different results can be obtained. In

a simplified mathematical context we are just interchanging x with t in the already

presented algorithms.

6.1 Introduction

We first present, in a general context, the time based algorithm for performing

the five fundamental 2-input logic gates and then the algorithm for performing the

search of an unsorted database. Further we support the method with specific illustrative

examples using neural systems [93, 94], and electronic circuits. We should note that in

both examples the general method is adapted to best suit the context of each the actual

implementation.

6.1.1 Flexible Logic Gates

For any computational system to be able to perform flexible (2-input) logic three

inputs need to be given to the system. The first input is the “Programming Instruction”

that defines which logic operation will be performed. The other two inputs are the two

logic variables, “Input1” (I1) and “Input2” (I2), see Figure 6-1. In the present method

we utilize a time instance, ttotal, as a single adjustable variable that can be used for

the representation of all three inputs. We define one time instance to represent the

115

Figure 6-1. Schematic representation of a flexible 2-input logic gate. For an Output threeparameters need to be defined, the two data streams, Input1 and Input2, andone Programming Instruction, which is to manipulate the data.

programming, tprog, and two more time instances to represent the two inputs, tI1 , tI2

respectively, with the relationship for a total time defined as: ttotal = tprog + tI1 + tI2 . We

represent a logical 1 by a finite time length, tin (relate to δ in the case of algorithms based

on state manipulation), and a logical 0 by a time length of zero length (tin = 0).

It is important to note that:

• Since a logical 0 at the inputs is set to be of zero time length, the set of inputs (0,0)is the same time interval as programming the gate (tprog).

• The time interval for representing an input of logical 1 must be the same irrespectiveof whether the logical 1 is at Input1 or at Input2, as the two are indistinguishable bya logic gate; the sets (1,0) and (0,1) are thus represented by the same time length;

• The total time interval for the set of inputs (1,1) is twice that of the sets (1,0) and(0,1).

116

The way we use the total time, ttotal, is to define a time instance within the clock

cycle of a “generic signal” constructed in such a way so as to encompass the results of

different logic operations at different time instances. Of course because of the “nature”

of time this time instance needs to be in reference to a fixed “zeroth” time instance,

which can be defined by the initialization of the clock cycle. In fact in the general context

demonstration we consider a single cycle initiated at the zeroth instance and use ttotal to

define a “moment of observation”, at which the result is obtained. There is though the

converse scenario as well, which we show in a following illustrative example in Section 6.2,

we can consider the “final” time instance as fixed and use ttotal as a “negative” shift in

instances from this final instance, basically changing the “moment of generation” of the

generic signal.

To construct the “generic signal” we consider the behaviour of the five fundamental

logic gates (AND, NAND, OR, NOR, XOR). From the truth table for these logic gates

(Table 6-1) we can create for each gate a graph of the input-output relationships for the

three pair sets of logical inputs, {(0, 0)}, {(1, 0)/(0, 1)}, {(1, 1)}, Figure 6-2(a)-(e). Since

we are to consider time as the input quantity, we can encapsulate all five graphs into a

single graph where the input dimension is identified with time instances, as explained in

Figure 6-2(f).

The combination of the truth tables into this single non-linear function results in the

form of : f(u, t) > 0, for t−Δt > t > t +Δ t, else f(u, t) = 0; which provides the required

behaviour for a system to be utilized as a flexible logic gate. This function is very similar

to an action potential generated by a neuron; resting at a low voltage (f(u, t) = 0), and

for a brief length of time rising very rapidly to a higher voltage (f(u, t) > 0), and then

dropping very rapidly back to its original level. We are considering this function in neural

context, not only because of the form of the function, but also because it is clear that

in some way neurons are capable of logic operations [95]. Note though, that we do not

117

actually propose that the methods we present here are the method neurons in fact use; we

use neurons primarily for demonstration purposes.

If we consider the inputs (and programming) to be represented as a time instance we

can see from Figure 6-2(f) that at different moments in time this signal is “high” or “low”,

we can interpret these two states as a logical 1 or 0 at the output. Therefore it is only a

matter of “when” we sample such a signal to obtain a different response, i.e. depending on

which time instance we chose to sample the signal we obtain a 1 or 0.

Therefore the method utilizes this behaviour to provide a flexible logic gate system

by varying the exact “moment of observation” (ttotal) of the signal depending on which

gate will be performed on what data. The “flexible” part should be evident, as we can

easily change the tprog from one cycle to the next, enabling the system to perform any two

different computational operations.

Table 6-1. The truth table of each of the five fundamental logic gates, AND, NAND, OR,NOR, XOR.

I1 I2 AND NAND OR NOR XOR0 0 0 1 0 1 01/0 0/1 0 1 1 0 11 1 1 0 1 0 0

Purely as a reference example, based on the signal of Figure 6-2(f) we give in Table

6-2 values for the different times to perform each of the five logic gates, on each of the

three pair sets of inputs. In each of the three cases of inputs, the time length representing

a logical 1 is two time “units” long (tin = 2). We utilize a signal as given in Figure 6-2(f),

with clock cycle defined to be 9 time units long. Depending on which logic operation we

wish to perform and also on what the inputs are, a sampling instance within the clock

cycle is defined, as explained above and given by ttotal. This sampling instance is in time

units with reference to the initialization from the clock cycle. We take the NOR gate as an

illustrative example. For the NOR gate we have tprog = 5 time units. For the set of inputs

(0,0) we sample the signal at the 5th (tprog + 0 + 0) time unit obtaining a 1 at the output,

see Figure 6-2(f). In the case where our inputs are (1,0) or (0,1) we have a sampling

118

Figure 6-2. (a) Input-Output relationship of AND gate; (b) Input-Output relationship ofNAND gate; (c) Input-Output relationship of OR gate; (d) Input-Outputrelationship of NOR gate; (d) Input-Output relationship of XOR gate;(f) Combined figure of the Input-Output relationships of the five fundamentalgates. Different sections of this graph represent different instances ofInput-Output operations. The 15 different cases, of the gates and inputsconsidered, are marked; with marking an Output of 1 and # marking anOutput 0.

Table 6-2. Time values, in arbitrary time “units” that produce each of the five gatesconsidered, based on the signal of Figure 6-2(f).

I1 I2 AND NAND OR NOR XOR0 0 0 4 2 5 31/0 0/1 2 6 4 7 51 1 4 8 6 9 7

instance of 7 time units, given by (tprog + tin + 0), obtaining a logical 0 at the output.

Similarly for the case NOR(1,1) we have (tprog + tin + tin) giving a sampling time at the 9th

time unit again obtaining a logical 0 at the output. Thus we have a correct representation

of the NOR gate. Using a similar scheme, and by simply changing the programming time

length, tprog we can switch to performing a different gate.

119

6.1.2 Search Algorithm

Promoting this idea further, one can now consider each moment in time as an

opportunity to represent any specific computational instruction or data. In order to

demonstrate more practical applications of using time for computational commands (data

and programming), we will demonstrate a variation on our algorithm for performing a

search on a given unsorted database [28].

The construction of a search engine requires the following steps:

1. A representation of data in the database;

2. The matching of data with searched for items;

3. Localization of the matched data.

Any given database is constructed out of M unique items, we already referred to

this set of items as the “alphabet”, and further each item can be represented by a unique

natural number, see Section 3.1. We have shown above that a finite time interval can

represent a logical 1, we can extend this idea to time lengths of multiple sizes, each one

representing a different number. Thus we utilize different time lengths to represent each

one of the unique items that constitute a given alphabet. These time lengths can be

marked by a delta function (a spike in neural context) occurring at a specific moment in

time with respect to the initialization of a clock cycle; the interval between initialization

and the delta function is the time length for a specific item. So given any database we can

represent it by a series of delta function generators, Rx, all receiving initialization from the

clock cycle, and each providing a spike at the specific delayed moment in time providing a

time interval which represents the data item it stores, see Figure 6-3. As shown in Figure

6-3 we can connect these function generators, Rx, each independently to a Coincidence

Detector Unit (CDU). The function of the CDUs is to produce a “confirmation”, a spike,

at the moment when it receives two simultaneous spikes. The “search element” is another

delta function generator, Rsearch,k set to produce a spike at the moment that represents

the data item we wish to search for. As a result the coincidence detector(s) produce a

120

spike only if the searched for item is within the given database since only then will a CDU

receive two simultaneous spikes (from a database element and the searched for item). Note

that the terminology we use is such as to relate this abstract presentation of the method

directly to the neural implementation to follow, specifically R will be a neural synapse

strength coefficient, while the TDU and CDU will be specific neural circuits.

For the final step of localization of positive results of a search we connect all the

coincidence detectors to an accumulator, Integrating Unit (IU), each weighted by a

different weight following Godel numbering [31], i.e. the first CDU weighted by 20, the

second by 21, the third by 22 and so on, providing us a way to uniquely identify the

positions of all matched items, by measuring the “height” of the weighted sum at each

moment in time.

In addition we can perform multiple searches within the same clock cycle; i.e. we

can connect to the CDUs multiple search elements, each encoding a different item and

search for all the items, technically, at the “same time”. Performing multiple searches does

not create a problem at the accumulator since the confirmation spikes from the CDUs

will arrive at different times, for different items, and thus spikes of the same item will be

accumulated separately from spikes of a different item.

Given that our implementation of logic and application to database searching is based

on sampling signals at an appropriate time the main limitation of our methods is: “how

finely can we ‘slice’ time?” For the implementation of logic gates, sampling rate will define

how fast we can perform a logic operation, with a higher sampling rate the smaller the

width of the square pulse can be and thus the shorter the time it takes to complete a

clock cycle (perform an operation). In practice we are limited by the technology used

to implement our method. For the search algorithm, the higher the sampling rate the

more items we can store within a given clock cycle. There is one more limitation to the

search algorithm, the fact that with increasing size of the database the weights for each

121

Figure 6-3. Schematic representation of the time based search method.

location increase exponentially, this would create a problem based on the technology used

to implement the method, but can be addressed by a multi-stage addressing system.

6.2 Neural Implementation

Our demonstrations with neural circuits are mainly inspired by the work of Abarbanel

et. al. [93, 94]. Specifically we take a neural circuit created by Abarbanel et. al. designed

for interspike interval recognition, and we extract from this circuit two compartments, the

Time Delay Unit (TDU) and the Detection Unit (DU) (which we rename to Coincidence

Detectino Unit (CDU)). We utilize these sub-circuits to manipulate the timing of action

potentials, both for their creation and detection.

6.2.1 Neural Models

This short section is a slight deviation from our main theme to very briefly introduce

two classic neuron models and three models of synaptic connections, which we use for

our algorithm implementations. This is purely done to place the reader in, at least, some

122

context of neuroscience; for further expositions we direct to work by Izhikevich [96–98] and

Latham et. al. [99, 100], among many others.

Type I neuron. This is a model in the framework of Hodgkin Huxley, a single

compartment model with fast sodium channel, a delayed rectifier potassium channel and a

leak channel [96, 98, 101] defined by the following equation:

CdV (t)

dt= IDC + gNam

3(t)h(t)(ENa − V (t)) + gKn4(t)(EK − V (t))

+gL(EL − V (t)) + IS(t), (6–1)

where V (t) is the membrane potential, IDC the external DC current, IS(t) the synaptic

current, and by defining r = Na, K, L to refer to the sodium, potassium and leak

channels, we have Er as the reversal potentials and gr as the conductance of each channel.

In our simulations we have used the following parameter values: C = 1 μFcm−2,

ENa = 50 mV, EK = -95 mV, EL = -64 mV, gNa = 215 mScm−2, gK = 43 mScm

−2,

gL = 0.813 mScm−2. Finally the gating variables X = {m,h, n} satisfy the following

equation: dX(t)dt= αX(V (t))(1−X(t))− βX(V (t))X(t), where αX and βX are given by:

αm =.32(13−(V (t)−V th))

e(13−(V (t)−V th))

4.0 −1, βm =

0.28((V (t)−V th)−40)

e((V (t)−V th)−40)

5 −1,

αh = .128e17−(V (t)−V th)

18 , βh =4

e40−(V (t)−V th)

5 +1,

αn =0.032(15−(V (t)−V th))

e(15−(V (t)−V th))

5 −1, βn =

0.5

e(V (t)−V th)−10

40

,

where Vth = −65 mV. In this model the frequency of spiking as function, which in the

context of our algorithm defines a clock cycle, is given by: f = C√IDC − I0, where I0 is

the threshold for spiking and C is a constant based on the model parameters.

Bistable neuron. Again this model derives from Hodgkin Huxley as a single

compartment model with a leak current, but the sodium channel is persistent and the

potassium channel fast, the defining equation is given by:

CdV (t)

dt= IDC + gNam∞(V (t))(ENa − V (t)) + gKn(t)(EK − V (t))

+gL(EL − V (t)) + +IS(t), (6–2)

123

where the gating variable n(t) is given by: dn(t)dt= n∞(V (t))−n(t)

τnand {m∞(V ), n∞(V )} = X∞(V )

by the activation function: X∞(V ) = 1/(1 + exp((VX − V )/kX)); the remaining variables

are the same as in the type I neuron model, see Equation 6–1. In our simulations we

have used the following parameter values: C = 1 μFcm−2, ENa = 60 mV, EK = -90 mV,

EL = -80 mV, gNa =20 mScm−2, gK = 10 mScm

−2, gL = 8 mScm−2, Vm = -20 mV,

Vn = -25 mV, km = 15 mV, kn = 5 mV, τn = 0.16 mV. This model mainly differs from

the type I neuron, as its resting state and oscillating state can both be stable and as a

result the system can switch from one state to the other simply by a variation in its state

variable (V (t)), unlike type I for which, for behavioural changes a system parameter needs

to be varied (IDC) [96, 98]; actually we utilize this difference in behaviour to construct the

TDU.

Synapses. We utilize in our neural circuits three types of synaptic connections and

their currents are given by:

• Excitatory AMPA: IS(t) = gASE(t)(EE − V (t)), where gA is the synaptic

conductance, EE = 0 mV is the reversal potential and SE(t) the gating variable, that

is biologically related to the fraction of bound glutamate receptors;

• Excitatory NMDA: IS(t) = gNB(V (t))SE(t)(EE − V (t)), where gN is the synaptic

conductance and B(V ) = 1.0/(1 + 0.288 exp(−0.062V )). EE and SE(t) are the samequantities as for the AMPA synapse;

• Inhibitory GABAA: IS(t) = gGSI(t)(EI − V (t)), where gG is the synaptic

conductance, EI = −75 mV is the reversal potential and SI(t) the gating variable,related to the fraction of bound gaba receptors.

The gating variables are given by: SX(t) = S0(θ(t))−SX(t)τ(S1−S0(θ(t)))

, where SX(t) = {SE(t), SI(t)},

S0(φ) = 0.5(1+tanh(120(φ−0.1))) is a sigmoidal function, and given the heaviside function

Θ(X) = 1 if X > 0, else Θ(X) = 0 we have θ(t) =∑iΘ(t− ti) ∙Θ((ti+ τR)− t) ∙Θ(X), here

ti is the time of the ith presynaptic neuronal spike. Finally we have two time constants

τR = τ(S1 − 1) and τD = τS1, in biological sense the time for the neurotransmitter to bind

and release, respectively; for the AMPA synapse τR = 0.1 ms and τD = 1.5 ms, for NMDA

τR = 2.5 ms and τD = 70 ms, and for GABAA τR = 1.12 ms and τD = 5.5 ms.

124

Time Delay Unit (TDU) and Coincidence Detection Unit (CDU). The Time

Delay Unit is constructed with one of each of the two neuron models we presented, a

type I neuron and a bistable neuron, our construction follows directions given in [93]. A

schematic of the circuit is given in the inset (a) of Figure 6-4. Neuron α is type I neuron

and neuron β is a bistable neuron and both neurons are initiated at a resting potential

state. The input connections to the neurons are a AMPA synapse to neuron β and a

NMDA synapse to neuron α. The neurons are also connected to each other through

a GABAA synapse, α −− β, and a NMDA synapse, β Ý α, it is this synapse that we

modulate to create a delay. As we have the circuit configured an incoming action potential

arriving at t0 is directed into both neurons. For neuron α, since it is a type I neuron,

a single incoming action potential has no effect other than to increase the membrane

potential slightly, but there will be no behavioural change. For neuron β though the

situation is different, as it is a bistable neuron an incoming action potential “pushes”

the neuron to its oscillating state, so neuron β starts generating action potentials. These

action potentials drive, through the connecting NMDA synapse, neuron α and eventually

will “push” the membrane potential of neuron α enough for it to fire its own “output”

action potential. The timing of this output potential depends on how long it takes for

neuron β to push neuron α over its firing threshold, and of course this time delay can be

modulated by a “stronger” or “weaker” connection between the two neurons. Specifically

the strength of the aforementioned NMDA synapse, β Ý α, can be varied using a weight

parameter R as gNR = R ∙ gN , with gN = 1 mScm−2. Therefore an incoming action

potential at time t0 will be technically “absorbed” by the system and an output action

potential will be generated at a later time t0 + τ(R); in essence though, since there is

no difference between the two action potentials, in effect what we achieve is the delay in

the propagation of a single action potential. Through simulations we have quantified the

effect of R on τ(R) and their relationship is presented in Figure 6-4. Of course we need

to make sure that only a single output potential is created, this is achieved through the

125

Figure 6-4. Time Delay Unit (TDU). The relationship between the parameter R, whichdefines synaptic strength, and the time delay created τ . Inset (a): A schematicrepresentation of the TDU neural circuit. Inset (b): Domain and range whereR vs. τ(R) is linear, and hence the range of R values used for flexible logicgate implementation. (Adapted from [93].)

inhibitory GABAA synapse between the two neurons, α −− β, which takes the “output”

spike of neuron α and directs it, besides to the output, back into neuron β inhibiting it

and pushing it away from its oscillating state back to its resting state, ceasing further

generation of action potentials from neuron β, and as a direct consequence prevents the

generation of further action potentials from neuron α itself.

The Coincidence Detection Unit is a much simpler case, it is simply a single type I

neuron that acts as a classic integrator receiving inputs from two sources through two

AMPA synapses. The strength of the conductance of the two synapses is equated so

that both input sources have equal effect on the neuron. In addition the strength is set

high enough for the neuron not to cross its firing threshold, unless it receives two spikes

126

simultaneously (or within tolerance of a small time interval, defined by the conductance of

the synapses).

6.2.2 Algorithm Implementations

The first neural based implementation of logic emulation is simply based on the idea

of sampling the membrane voltage signal of a single neuron at different moments in time.

The second method utilizes a circuit of two bi-directionally coupled Hodgkin Huxley

neurons, the TDU circuit. Logic gates are emulated by varying the synaptic strength R

that determines the time delay τ(R) and then observing the output at a predetermined

fixed time instant. The search algorithm is straightforwardly implemented along the

directions given in the more general context presentation of Section 6.1.2.

Varying the “moment of observation”. This is truly a very simple implementation.

We just consider a single neuron producing an action potential, see inset of Figure 6-5.

Since this signal is a product of simulation we consider each point generated by the

simulation as an instance in time and with the zeroth simulation point as a reference point

we define ttotal according to the computational operation to be performed, exactly as in

Section 6.1.1.

Take the NOR gate as a specific example. In Figure 6-5 we show the results of

sampling our signal at the time instances of 2555 for the operation NOR(0,0), 2605 for

NOR(0,1)/(1,0) and at 2655 for NOR(1,1). In terms of programming (tprog) and input

time lengths (tin) we have tprog = 2555 time steps, specific to the NOR gate, and tin = 50

time steps. A small deviation from the general case is that the action potential actually

fluctuates between 48 mV and -92 mV, it is not simply 0 or 1. Therefore we set a voltage

threshold, at -45 mV, over which we define the logical 1 and under which the logical 0, at

the output.

As is clear from Figure 6-5 the appropriate Output values are obtained, that is the

voltage is over -45 mV for the (0,0) case and below for the other two cases. The whole

procedure can be interpreted as follows: to perform the logic gate NOR we require to

127

Figure 6-5. Demonstration of operating a NOR gate, with a single neuron using differentsampling times. Depending on the inputs a different sampling moment isdefined beyond the 2555th moment that defines the NOR gate. Each possiblecase is shown with a circle, black circle ( ) indicating a 1 at the Output,empty circle (#) a 0. The Output is defined by a voltage threshold at -45 mV,shown. Inset: The whole action potential considered.

Table 6-3. Appropriate time sample instances, based on simulation points, to performeach of the five gates considered. In brackets the voltage of the action potential,at each point, is given.

I1 I2 AND NAND OR NOR XOR0 0 2405(-57 mV) 2505 (-9 mV) 2455(-56 mV) 2555 (1 mV) 2485(-52 mV)1/0 0/1 2455(-56 mV) 2555 (1 mV) 2505 (-9 mV) 2605(-86 mV) 2535 (34 mV)1 1 2505 (-9 mV) 2605(-86 mV) 2555 (1 mV) 2655(-86 mV) 2585(-50 mV)

wait 2555 time steps as a programming time length and then another 50 time steps for

each occurrence of a single logical input of 1. Using the same system, that is a single

Hodgkin Huxley neuron producing an action potential at the same rate, time instances

for reading an Output, with respect to the zeroth simulation point, along with their

resulting respective voltage value, for accomplishing the five fundamental gates are given

in Table 6-3. For confirmation we refer back to the gate truth table, Table 6-1.

128

Varying the moment of generation. Converse to the previous implementation,

in this implementation we fix the moment of observation, in the specific example at

t∗ = 425 ms from an initiating action potential at t0, and we vary the moment of actual

generation of the signal. We use “actual time” instead of simulation points to emphasise

the independence between the moment of observation and the computational operation.

To have the ability to generate an action potential at different times, with respect to

t∗, we use a TDU as part of our computational system. Following Figure 6-4, the TDU

receives an input action potential at t0 and produces a delayed output action potential

at t0 + τ(R), it is the moment of generation of this output action potential we use for

computation. Exactly as explained before, with respect to time instances, we utilize R

values taken from the relationship τ(R), and with Rtotal = Rprog + RI1 + RI2 we represent

a desired computational operation. Note that we use the domain 1.68 < R < 1.72 where

the curve of Figure 6-4 is linear, see inset (b) of same figure, so that the changes in R are

linearly proportional to the changes in τ(R) (actual time) and hence changes for a logical

1 at the input are independent of whether it is at Input1 or Input2; also note that R and

τ(R) are inversely proportional, that is higher R means earlier action potential generation.

Finally again as in the above implementation we define a voltage threshold at -45 mV for

distinction of the Output between 1 and 0.

Obtaining the result at the “moment of observation”, and actually implementing

neurally a sampling scheme, can be realized with a Coincidence Detection Unit, that is a

neuron that itself will fire when two neurons connected to it fire simultaneously, so one of

these two neurons can be the one “acting” as our gate, having its settings (R) changed

according to the programming and inputs, and the other neuron be fixed to fire solely at a

specific moment, “acting” as the “reader”.

As an explicit example, see Figure 6-6, consider the universal gate NOR with

Rprog = 1.705. For the NOR gate and the case of inputs (0,0) Rtotal = 1.705 + 0 + 0; with

this R value the action potential is generated with its peak at 424.5 ms after the initiating

129

Figure 6-6. Demonstration of operating a NOR gate, with a neural circuit using differentdelay times. All three cases of inputs are shown, in actuality only one actionpotential is generated by the system. The time of generation of the actionpotential is inversely proportional to R. Given the “moment of observation” at425 ms, the Output is obtained using the voltage threshold at -45 mV. Onceagain is 1 at the Output and # 0.

action potential, which results in a voltage of -16 mV at the moment of observation of

425 ms, which is over the threshold of -45 mV, thus the output is 1. For the cases (1,0)

and (0,1), which are degenerate, we have Rtotal = 1.705 + 0.005 + 0. The action potential is

generated with a peak at 424 ms, so by the time of the moment of observation the voltage

drops, below the threshold, to -89 mV, producing a 0 at the output. We have a similar

result for the third and final case of (1,1) at the inputs, Rtotal = 1.705 + 0.005 + 0.005,

the peak of the action potential happens at 423.5 ms, so again the voltage drops below

the threshold by the time of the moment of observation. It is clear that our system works

perfectly to reproduce the logic gate NOR, similar results have been obtained for the other

of the four fundamental gates, the R values, and the respective voltages at the moment of

observation, are given in Table 6-4; which once again can be compared with Table 6-1.

130

Table 6-4. Appropriate R values so time shift an action potential in order to perform eachof the five gates considered. The action potential is sampled always at the samemoment in time, 425 ms after an initiating action potential. In brackets thevoltage of the action potential, at 425 ms, is given.

I1 I2 AND NAND OR NOR XOR0 0 1.690(-58 mV) 1.700 (48 mV) 1.695(-56 mV) 1.705(-16 mV) 1.697(-53 mV)1/0 1/0 1.695(-56 mV) 1.705(-16 mV) 1.700 (48 mV) 1.710(-89 mV) 1.702 (35 mV)1 1 1.700 (48 mV) 1.710(-89 mV) 1.705(-16 mV) 1.715(-85 mV) 1.707(-53 mV)

Neural search algorithm. The search method can also be implemented with the

same neuronal circuits, TDU and CDU along with an Integrating Unit (IU), simply

a single neuron with its threshold for firing an action potential set very high. The

first component of our search engine is encoding by each unique moment in time a

database element. As we have shown above, items from any given alphabet can be

represented by time intervals. We do the same, but instead of using actual time intervals

for representation of items we use unique R values. So a database of size N , regardless of

size, can be constructed using N TDUs each one representing one of the M distinct items

of the alphabet associated with a unique R value and as a result a unique instance in time

for generating an action potential.

The second component, just as in the abstracted explanation above, is a Coincidence

Detector Unit (CDU), this is a single neuron at resting potential. The property of this

neuron that makes our search engine work is that it requires two action potentials to

arrive at its membrane simultaneously for the neuron to produce a response action

potential. If and when, two action potentials coincide the CDU will generate its own

action potential, otherwise it will not fire, providing us a way to confirm the existence or

not of a searched for item in the database.

So now we can incorporate in our model the Search Engine elements, TDUs set at the

R values for the items we wish to search for. We couple each CDU with each TDU of the

database and to all the Search Engine elements, referring back to Figure 6-3, so that when

the Search Engine elements fire, the CDUs that are attached to the searched for items will

131

receive two simultaneous action potentials and fire themselves. If a searched for item does

not exist in the database no CDU will receive two simultaneous spikes so there will be no

action potential from the CDU.

Finally we connect all CDUs to an Intergrading Unit (IU) to combine all search

results. Each CDU is connected to the IU with a synapse of different conductance weight

(g), following Godel numbering; the first CDU is connected with a weight 20g the second

with weight 21g, third unit with 22g and so on. This provides us with a way to uniquely

mark each position of each item in the database. At the output we read the voltage of the

IU and interpret the searched for results.

There are two parameters to consider at the reading of the result, first is the time

instance at which a voltage increase occurs, which represents the searched for item, and

second is the amount by which the voltage of the IU is increased. Basically, each moment

in time represents an information item, as defined from the different R values we use to

encode them, and each increase in voltage represents a position in the database given by

the Godel numbering.

6.3 Electronic Implementation

As it turns out the electronic implementation of this method can be performed with

simple multipliers instead of a complicated sampling circuit of variable delays. Basically

we generate two signals, one representing the “generic signal”, as in Section 6.1.1, and

the other signal is generated at the time instance that represents the computational

operation to be performed, whether logic operation or search operation. The multiplier

simply multiplies the two signals and the product of course will be 1 only if the two signals

coincide in time, in essence we are emulating the neural Coincidence Detection Unit.

Logic gates algorithm. As shown in Figure 6-7 we connect the reference signal

square pulse generator (Vr) to a multiplier (U1) along with the varying “sampling

signal” generator (Vs). The varying sampling signal generator emulates the moment

of observation by producing a short square pulse signal with varying delay compared to

132

Figure 6-7. Schematic representation of an electronic circuit for logic using time. Vrprovides the reference signal while Vs the “moment of observation”. The twosignals switch from 1 V to 0 V, therefore U1 through simple multiplicationprovides the Output, given straightforwardly by the voltage of resistor R1 (1 Vlogical 1, 0 V logical 0).

the reference square pulse of Vr. The function of the multiplier U1 is to simply multiply

the two signals and therefore the resulting signal is the same as if sampling the reference

square pulse Vr at the moment of observation.

For our implementation we used a reference signal of frequency 10 Hz and pulse width

30 ms, shown in Figure 6-8(a), as a result our time length representing a logical 1 is of

30 ms(=tin). The “sampling signal” from Vs is again a pulse of frequency 10 Hz, but its

generation instance can be varied creating a variable delay signal, see Figures 6-8(b)(d)(f).

The time delay of generation of the “sampling signal” is our programming time length plus

the appropriate delay for each of the three pairs of inputs, values for programming the five

fundamental gates and each case of inputs are given in Table 6-5.

133

Figure 6-8. Demonstration of operating a NOR gate using an electronic circuit utilizingtime dependant computation. (a) Reference signal, to be sampled at differenttimes. Refer back to Figure 6-2(f). (b) Sampling signal for the case of inputs(0,0) to a NOR gate, equivalent to tprog + 0 + 0 = 60 ms. (c) Output signal forperforming NOR(0,0), output is a logical 1, signal is “high”. (d) Samplingsignal for the cases of inputs (1,0)/(0,1) to a NOR gate, equivalent totprog + tin + 0 = 90 ms. (e) Output signal for performing NOR(1,0) orNOR(0,1), output is a logical 0, signal is “low”. (f) Sampling signal for thecase of inputs (1,1) to a NOR gate, equivalent to tprog + tin + tin = 120 ms.(g) Output signal for performing NOR(1,1), output is a logical 0.

Table 6-5. Delay times for Vs, the moment of the “sampling signal” with respect to thestart of the clock cycle, to implement each of the five gates considered.

I1 I2 AND NAND OR NOR XOR0 0 0 ms 40 ms 10 ms 60 ms 30 ms1/0 0/1 30 ms 70 ms 40 ms 90 ms 60 ms1 1 60 ms 100 ms 70 ms 120 ms 90 ms

134

As a specific illustrative example take the NOR gate. As indicated in Table 6-5

the programming time length for the NOR gate is 60 ms, and as stated above the time

length representing a logical 1 at the inputs is 30 ms. Therefore for the set of inputs

(0,0), tprog + 0 + 0, we generate a “sampling signal” with simply a delay of 60 ms, see

Figure 6-8(b). For the sets of inputs (1,0) and (0,1), tprog+ tin+0, we generate a “sampling

signal” with delay 90 ms, see Figure 6-8(d); and finally for the set (1,1), tprog + tin + tin, we

generate a “sampling signal” with delay 120 ms, see Figure 6-8(f). The output is read at

the resistor R1 and is shown in Figures 6-8(c)(e)(g), as VOut. From Figures 6-8(c)(e)(g)

we can see that the correct Output is generated for each case of inputs for reproducing the

truth table of a NOR gate.

Search algorithm. For the search method, again our electronic implementation is

greatly simplified with the use of multipliers, U1, U2, U3, simply in place of Coincidence

Detection Units, see Figure 6-9 for the specific schematic. As a specific example consider

encoding the numbers {1, 2, 3} in three database elements Ri, Rj, Rk, using m ∙Δt, where

Δt = 10 ms and m ∈ N, representing any abstract item. In the search elements RS1,

RS2, RS3, we have encoded our search for numbers, specifically all three {1, 2, 3}. The

the accumulated signal from the search elements is fed independently to each multiplier,

along with the signal of each multiplier‘s associated database element. The result from

each multiplier is a signal of a square pulse if the searched for item exists or a zero signal

if it does not exist. For the final step each multiplier is connected to an accumulator

sub-circuit through resistors, R7, R8, R9 whose actual resistance follows Godel numbering.

The readout is set at the output of the accumulator that collects the signals from the

multipliers, at U5.

As seen in Figure 6-10(b) all three items are identified as existing in the given

database, the timely spikes at the output, and their location in the database is given by

the voltage height of their respective spike. Specifically we see in Figure 6-10(b) three

spikes, the first spike at 10 ms represents the occurrence of item given the number 1 in

135

Figure 6-9. Schematic representation of an electronic circuit for the time dependantsearch method. The R-units are signal generators, Rx being the generatorsused for encoding the database, RSi being the generators used for encodingsearched for items. The U-units are multipliers, replacing the CDUs of Figure6-3. R7, R8, R9 are the resistors associated with each database elementdefining its location through their resistance value. The dash-boxedsub-circuits are accumulator circuits (relating to IUs) “putting togther” both a“search signal”, in the case of multiple simultaneous searches, and the resultsignal from the multipliers.

the database and the height of 1 V represents the fact that this item is stored in the

first location of the database. Analogously for the other two spikes representing items

numbered 2 and 3, at time instances of 20 ms and 30 ms respectively, again the height

of each spike signifies the location of the item; 2 V for 2nd position and 4 V for the 3rd

position in the database.

6.4 Discussion

We have presented two algorithms that use time intervals for computation; the first

for performing flexible logic and the second for searching a unsorted database. The central

136

Figure 6-10. Demonstration of performing a search using an electronic circuit utilizingtime dependant computation. Refereing to the schematic of Figure 6-9: (a)The output of the search key accumulator at U6, this is the “search signal”from all the search elements that is fed to all multipliers; (b) The output ofthe result accumulator U5, confirming the existence of items items {1, 2, 3}with pulses at 10 ms, 20 ms, and 30 ms, respectively; also providing thelocation of each item in the database, through the voltage of each pulse, 1 V,2 V, 4 V, respectively.

idea of the proposed algorithms is that each, and any, moment in time can be utilized

to represent a computational operation and/or data. With the use of a defined clock

cycle length we segment time into manageable pieces, within which we define moments in

time that will produce different responses based on a variable quantity. In our proposed

algorithms the responses are of two types, either the crossing of a threshold or the

coincidence of events, both of which can be interpreted as computational operations.

With careful construction of a signal, which encapsulates the five fundamental logic

gates, we have shown how time intervals can be used to represent the programming of a

universal flexible logic gate. In addition we have shown how a zero length time interval

can represent a logical 0 and a finite length time interval a logical 1. A straightforward

137

combination (summation) of the programming time length and the logical inputs time

length can provide a moment in time at which sampling the signal will produce the desired

output. We can thus visualize an array of many computational elements each receiving a

common signal. Each of these elements is programmed to sample the signal at different

times producing, in parallel, the results of different logical operations. Actually there is

no reason to confine ourselves to the “simple” signal presented here, Figure 6-2(f). The

method can be generalized to sampling more “complicated” signals, reproducing even

more complicated operations at different times; in the spirit of Chapter 5.

Furthermore, we have shown how many different time intervals can be utilized in an

algorithm to perform searches within a given database. We have shown how each natural

number can be represented by a time interval, defined between the “tick” of a universal

clock (clock cycle) and a delta function. Within one clock cycle we search the database by

looking for coincidences between our search for elements and the elements encoded in the

database. It is significant to note that our method completes a search within one single

clock cycle regardless of the size of the database; and it is not confined to searching for

single items sequentially, but multiple searches can be performed within the same clock

cycle.

For both algorithms presented we provided realizations, both with neural and

electronic circuits. As we have shown, the implementations are straightforward, especially

in the case of electronic circuits. The presented methods are a direct consequence of

considering delayed time responses as computational commands. For both implementations

the main limitation is the smallest timescale of the system; a consequence of considering

time as computational data and programming, the smaller the time interval we can utilize,

the more data and operations can be represented.

138

CHAPTER 7CONCLUSION

Chaotic Computation has come a long way since its first step in 1998 [25]. Both

within the “original” group of Ditto et. al. [102, 103], and even now days within the

more the more extended group of collaborators [104, 105], theoretical and experimental

advancements have established Chaotic Computation as a realistic possibility for the

future of computation. We are glad to report that the field has also attracted the

attention of researchers beyond our group [106, 107].

Even further, the aim of bringing research out of the laboratory and into the public

domain is actively pursued; Chaologix Inc. (www.chaologix.com) is an active company

with the sole aim to realize the technology and present the solution to the public. In

accomplishing this goal the main attempts circle around conventional CMOS based VLSI

circuitry [108], but as we emphasised Universality allows us to consider even more esoteric

approaches such as magneto-based circuitry [109], or even high speed chaotic photonic

integrated circuits [110].

The aims of this dissertation were threefold. Obviously foremost, expose the reader

to the whole field of Chaotic Computation; in addition to present the material with its

natural multidisciplinary nature, from the underlining mathematics, to the physics that

governs our systems, the inspiration from biology and the engineered implementations;

note our attempt was to expose the reader to details from each field, but at the same

time without deviating from the main aim. Finally and of equal importance, we aimed at

establishing where the “cutting edge” of research of the field is, prepare the groundwork

for our future work.

In closing we wish to point out projections of the future paths emanating from

each chapter. Developing further the connections implied in chapter one is essential

in determining the ultimate limitations of our research and “bootstraps” directly

to where chapter two left of; specifically the further extension of the Deutsch-Jozsa

139

algorithm, in the spirit of how quantum computing progressed, into algorithms for more

specific functions, like Fourier Transform and integer factorization, and even further.

The importance of searching, matching, pattern recognition in our current general

technological aims, drives us to consider promoting further our method for searching;

a sorting algorithm should probably be the most immediate target, with issues of

multidimensional data, and perhaps extrapolated action, or decision, taking not the

too far of future. While all the aforementioned could be considered mainly theoretical

milestones, the implementation side can not be ignored. Faster, more efficient, smaller,

we share all goals that current established computer hardware technology has, simpler

electronic, or even alternative, realizations will be pursued. Finally, and in some sense

coming full circle back to the fundamental mathematics, through chapters five and six we

have setup the field for investigating further how to obtain independence from actual exact

state values, and/or even the utilization of only specific physical quantities, an attempt

perhaps to make more tangible the abstract concept of “computation”.

140

REFERENCES

[1] G. E. Moore, Electronics 38, 114 (1965).

[2] R. Landauer, IBM Journal of Research and Development 5, 183 (1961).

[3] W. Arden, et al., International technology roadmap for semiconductors, 2008 update,overview, Tech. rep., International Roadmap Committee (2008).

[4] M. D. Austin, et al., Nanotechnology 16, 1058 (2005).

[5] R. P. Feynman, R. W. Allen, T. Hey, Feynman Lectures on Computation (PerseusBooks Group, 2000).

[6] M. A. Nielsen, I. L. Chuang, Quantum Computation and Quantum Information(Cambridge University Press, 2000).

[7] R. Edwards, H. T. Siegelmann, K. Aziza, L. Glass, Chaos: An InterdisciplinaryJournal of Nonlinear Science 11, 160 (2001).

[8] M. N. Stojanovic, D. Stefanovic, Nature Biotechnology 21, 1069 (2003).

[9] M. N. Stojanovic, D. Stefanovic, Journal of American Chemical Society 125, 6673(2003).

[10] J. Hasty, F. Isaacs, M. Dolnik, D. McMillen, J. J. J. Collins, Chaos: An Interdisci-plinary Journal of Nonlinear Science 11, 207 (2001).

[11] S. H. Strogatz, Nonlinear Dynamics and Chaos: With Applications to Physics,Biology, Chemistry and Engineering (Perseus Books Group, 2001).

[12] W. Ditto, T. Munakata, Communications of the ACM 38, 96 (1995).

[13] E. Ott, C. Grebogi, J. A. Yorke, Physical Review Letters 64, 1196 (1990).

[14] M. Ding, et al., Physical Review E (Statistical Physics, Plasmas, Fluids, and RelatedInterdisciplinary Topics) 53, 4334 (1996).

[15] M. Ding, et al., Chaos: An Interdisciplinary Journal of Nonlinear Science 7, 644(1997).

[16] N. Margolus, Physica D: Nonlinear Phenomena 10, 81 (1984).

[17] T. Toffoli, N. Margolus, Cellular automata machines: a new environment formodeling (MIT press, 1987).

[18] J. P. Crutchfield, K. Young, Physical Review Letters 63, 105 (1989).

[19] T. Toffoli, N. Margolus, Physica D: Nonlinear Phenomena 47, 263 (1991).

[20] A. V. Holden, J. V. Tucker, H. Zhang, M. J. Poole, Chaos 2, p367 (1992 (Jul)).

141

[21] J. P. Crutchfield, Physica D: Nonlinear Phenomena 75, 11 (1994).

[22] A. Krikelis, C. Weems, Computer 27, 12 (1994).

[23] Agota Toth, K. Showalter, The Journal of Chemical Physics 103, 2058 (1995).

[24] C. Moore, Physical Review Letters 64, 2354 (1990).

[25] S. Sinha, W. L. Ditto, Physical Review Letters 81, 2156 (1998).

[26] S. Sinha, W. L. Ditto, Physical Review E (Statistical Physics, Plasmas, Fluids, andRelated Interdisciplinary Topics) 60, 363 (1999).

[27] S. Sinha, T. Munakata, W. L. Ditto, Physical Review E (Statistical, Nonlinear, andSoft Matter Physics) 65, 036214 (2002).

[28] A. Miliotis, S. Sinha, W. L. Ditto, International Journal of Bifurcation and Chaos18, 1551 (2008).

[29] M. J. Feigenbaum, Los Alamos Theoretical Division Annual Report pp. 98–102 (July1975 - 1977 Sep.).

[30] A. N. Whitehead, B. Russell, Principia Mathematica to *56 (1910-1913) (CambridgeUniversity Press, 1962).

[31] K. Godel, On formally undecidable propositions of principia mathematica and relatedsystems (1931) (Courier Dover Publications, 1992).

[32] S. Sinha, T. Munakata, W. L. Ditto, Physical Review E (Statistical, Nonlinear, andSoft Matter Physics) 65, 036216 (2002).

[33] K. Murali, A. Miliotis, W. L. Ditto, S. Sinha, Physics Letters A In Press, Cor-rected Proof, (2009).

[34] A. Miliotis, K. Murali, S. Sinha, W. L. Ditto, M. L. Spano, A simple nonlineardynamical computing device (2009). To appear (Chaos, Solitons and Fractals, 2009).

[35] J. Gleick, Chaos: Making a New Science (Penguin (Non-Classics), 1988).

[36] R. L. Devaney, An Introduction to Chaotic Dynamical Systems, 2nd Edition(Westview Pr (Short Disc), 2003).

[37] H. O. Peitgen, H. Jurgens, D. Saupe, Chaos and Fractals: New Frontiers of Science(Springer, 1993).

[38] E. Ott, Chaos in Dynamical Systems (Cambridge University Press, Cambridge,2002).

[39] J. Moehlis, K. Josic, E. T. Shea-Brown, Scholarpedia 1, 1358 (2006).

[40] B. B. Mandelbrot, The Fractal Geometry of Nature (W. H. Freeman, 1982).

142

[41] K. G. Wilson, Physical Review B 4, 3174+ (1971).

[42] K. G. Wilson, Scientific American (1979).

[43] J. Banks, J. Brooks, G. Cairns, G. Davis, P. Stacey, American Mathematical Monthly99, 332 (1992).

[44] T. Li, J. A. Yorke, American Mathematical Monthly 82, 985 (1975).

[45] N. A. Sharkovsky, Scholarpedia 3, 1680 (2008).

[46] N. A. Sharkovsky, International Journal of Bifurcation and Chaos 5, 1263 (1995).

[47] P. Stefan, Communications in Mathematical Physics 54, 237 (1977).

[48] P. Saha, S. H. Strogatz, American Mathematical Monthly 68, 42 (1995).

[49] J. Bechhoefer, American Mathematical Monthly 69, 115 (1996).

[50] W. B. Gordon, American Mathematical Monthly 69, 118 (1996).

[51] M. J. Feigenbaum, Journal of Statistical Physics 19, 25 (1978).

[52] M. J. Feigenbaum, Physica D: Nonlinear Phenomena 7, 16 (1983).

[53] P. Cvitanovic, Universality in chaos (Bristol, Adam Hilger, Ltd., 1984).

[54] W. L. Ditto, S. N. Rauseo, M. L. Spano, Physical Review Letters 65, 3211 (1990).

[55] W. L. Ditto, K. Showalter, Chaos: An Interdisciplinary Journal of Nonlinear Science7, 509 (1997).

[56] V. In, W. L. Ditto, M. L. Spano, Physical Review E (Statistical Physics, Plasmas,Fluids, and Related Interdisciplinary Topics) 51, R2689 (1995).

[57] K. Murali, S. Sinha, Physical Review E (Statistical, Nonlinear, and Soft MatterPhysics) 68, 016210 (2003).

[58] T. Shinbrot, et al., Physical Review Letters 68, 2863 (1992).

[59] L. Glass, W. Zeng, International Journal of Bifurcation and Chaos 4, 1061 (1994).

[60] D. J. Christini, V. In, M. L. Spano, W. L. Ditto, J. J. Collins, Physical Review E(Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics) 56,R3749 (1997).

[61] V. In, et al., Chaos: An Interdisciplinary Journal of Nonlinear Science 7, 605(1997).

[62] C. Wagner, R. Stoop, Physical Review E (Statistical, Nonlinear, and Soft MatterPhysics) 63, 017201 (2000).

143

[63] S. Sinha, W. L. Ditto, Physical Review E (Statistical, Nonlinear, and Soft MatterPhysics) 63, 056209 (2001).

[64] S. Sinha, Physical Review E (Statistical, Nonlinear, and Soft Matter Physics) 63,036212 (2001).

[65] S. Sinha, Physical Review E (Statistical, Nonlinear, and Soft Matter Physics) 66,016209 (2002).

[66] S. Sinha, Physical Review E (Statistical Physics, Plasmas, Fluids, and RelatedInterdisciplinary Topics) 49, 4832 (1994).

[67] P. Bak, C. Tang, K. Wiesenfeld, Physical Review A 38, 364 (1988).

[68] P. Bak, C. Tang, K. Wiesenfeld, Physical Review Letters 59, 381 (1987).

[69] K. Murali, S. Sinha, W. L. Ditto, International Journal of Bifurcation and Chaos13, 2669 (2002).

[70] K. Murali, S. Sinha, W. L. Ditto, Physical Review E (Statistical, Nonlinear, and SoftMatter Physics) 68, 016205 (2003).

[71] K. Murali, S. Sinha, W. L. Ditto, Pramana 64, 433 (2005).

[72] U. Hubner, N. B. Abraham, C. O. Weiss, Physical Review A 40, 6354 (1989).

[73] C. O. Weiss, et al., Applied Physics B: Lasers and Optics 61, 223 (1995).

[74] K. Murali, S. Sinha, Physical Review E (Rapid Communications) 75, 025201 (4)(2007).

[75] K. Murali, S. Sinha, W. L. Ditto, A. R. Bulsara, Reliable logic circuit elements thatexploit nonlinearity in the presence of a noise floor (2009). To appear (PhysicalReview Letters, 2009).

[76] T. Munakata, S. Sinha, W. L. Ditto, IEEE Transactions in Circuits and Systems 49,1629 (2002).

[77] D. R. Chialvo, A. V. Apkarian, Journal of Statistical Physics 79, 375 (1993).

[78] D. Deutsch, R. Jozsa, Proceedings: Mathematical and Physical Sciences 439, 553(1992).

[79] A. Ekert, R. Jozsa, Reviews of Modern Physics 68, 733 (1996).

[80] L. K. Grover, Physical Review Letters 79, 325 (1997).

[81] B. S. Prusha, J. F. Lindner, Physics Letters A 263, 105 (1999).

[82] N. J. Corron, S. D. Pethel, B. A. Hopper, Physical Review Letters 84, 3835 (2000).

144

[83] E. R. Hunt, Physical Review Letters 67, 1953 (1991).

[84] K. Myneni, T. Barr, N. Corron, S. Pethel, Physical Review Letters 83, 2175 (1999).

[85] K. Chlouverakis, M. Adams, Electronics Letters 41, 359 (2005).

[86] T. Morie, S. Sakabayashi, M. Nagata, A. Iwata, Circuits and Systems I: Fundamen-tal Theory and Applications, IEEE Transactions on 47, 1652 (2000).

[87] K. Murakoshi, T. Morie, M. Nagata, A. Iwata, Design Automation Conference, 2000.Proceedings of the ASP-DAC 2000. Asia and South Pacific pp. 23–24 (2000).

[88] J. Garcia-Ojalvo, R. Roy, IEEE Transactions on Circuits and Systems I: Fundamen-tal Theory and Applications 48, 1491 (2001).

[89] T. Yang, L. O. Chua, International Journal of Bifurcation and Chaos 10, p1091(20000501).

[90] L. Chua, J. Yu, Y. Yu, Circuits and Systems, IEEE Transactions on 32, 46 (1985).

[91] M. M. Mano, Computer system architecture (Prentice-Hall, Inc. Upper Saddle River,NJ, USA, 1993).

[92] T. C. Bartee, Computer Architecture and Logic Design (McGraw-Hill, 1991).

[93] S. S. Talathi, H. D. I. Abarbanel, W. L. Ditto, Physical Review E (Statistical,Nonlinear, and Soft Matter Physics) 78, 031918 (17) (2008).

[94] H. Abarbanel, S. Talathi, Physical Review Letters 96, 148104 (4) (2006).

[95] C. N. Zhang, M. Zhao, M. Wang, IEEE Transactions on neural networks 11, 739(2000).

[96] E. M. Izhikevich, International Journal of Bifurcation and Chaos 10, 1171 (2000).

[97] E. Izhikevich, Neural Networks, IEEE Transactions on 15, 1063 (2004).

[98] E. Izhikevich, Dynamical systems in neuroscience: The geometry of excitability andbursting (MIT press, 2007).

[99] P. E. Latham, B. J. Richmond, P. G. Nelson, S. Nirenberg, Journal of Neurophysiol-ogy 83, 808 (2000).

[100] P. E. Latham, B. J. Richmond, S. Nirenberg, P. G. Nelson, Journal of Neurophysiol-ogy 83, 828 (2000).

[101] B. Ermentrout, Neural Computation 8, 979 (1996).

[102] K. Murali, S. Sinha, I. R. Mohamed, Physics Letters A 339, 39 (2005).

145

[103] S. Sinha, W. L. Ditto, Philosophical Transactions of the Royal Society A: Mathemat-ical, Physical and Engineering Sciences 364, 2483 (2006).

[104] M. R. Jahed-Motlagh, B. Kia, W. Ditto, S. Sinha, International Journal of Bifurca-tion and Chaos 17, 1955 (2007).

[105] H. R. Pourshaghaghi, B. Kia, W. Ditto, M. R. Jahed-Motlagh, Chaos, Solitons andFractals In Press, Corrected Proof, (2008).

[106] H. Leung, H. Yu, K. Murali, Physical Review E (Statistical, Nonlinear, and SoftMatter Physics) 66, 036203 (2002).

[107] D. Cafagna, G. Grassi, International Journal of Bifurcation & Chaos 16, p1521(2006 (May)).

[108] W. L. Ditto, K. Murali, S. Sinha, Philosophical Transactions of the Royal Society A:Mathematical, Physical and Engineering Sciences 366, 653 (2008).

[109] R. Koch, Scientific American 293, p56 (2005 (Aug)).

[110] M. Yousefi, et al., Physical Review Letters 98, 044101 (2007).

146

BIOGRAPHICAL SKETCH

Abraham Miliotis was born in Annapolis, Maryland, USA, from parents of Greek-Cypriot

descent. He was raised in Cyprus where he attended The English School and gained

admission to Imperial College, London in 1997. In 2000 he graduated from Imperial

College with a Bachelor of Science in Physics with Theoretical Physics. Under the

supervision of professor Dr. Dimitri Vvedensky, he completed his final year thesis,

“Stock market crashes treated as critical phenomena”. Dr. D. Vvedensky also introduced

Abraham to the work of professor Dr. William Ditto on using dynamical systems for

computation, whom he joined at Georgia Institute of Technology, Atlanta, Georgia in

2000, as a graduate student in the Physics department. Between 2001 and 2003 though,

Abraham paused his studies to pursue some entrepreneurial opportunities. He moved

back to Cyprus where, among other projects, he was involved in the creation of Megaland

Computers LTD, an IT retail sales and small-office/home-office support company, and also

the setting up of Game-Spot Internet/Gaming Cafes.

In 2003 he rejoined Dr. William Ditto at the Biomedical Engineering department

of University of Florida, where he completed his Ph.D. in May of 2009 on “Chaotic

Computation”.

147