CHAOTIC COMPUTATION
By
ABRAHAM MILIOTIS
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOLOF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
2009
1
c© 2009 Abraham Miliotis
2
TABLE OF CONTENTS
page
LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
CHAPTER
1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.1 Dissertation Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.2 Chaos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.2.1 Logistic Map, Topological Transitivity and Period Three . . . . . . 151.2.2 The Tent Map, Topological Conjugacy and Universality . . . . . . . 381.2.3 Threshold Control and Excess Overflow Propagation . . . . . . . . . 43
1.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2 INTRODUCTION TO CHAOTIC COMPUTATION . . . . . . . . . . . . . . . 49
2.1 Number Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492.1.1 Excess Overflow as a Number . . . . . . . . . . . . . . . . . . . . . 492.1.2 Periodic Orbits for Number Representation . . . . . . . . . . . . . . 502.1.3 Representation of Numbers in Binary . . . . . . . . . . . . . . . . . 52
2.2 Arithmetic Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532.2.1 Decimal Addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532.2.2 Binary Addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562.2.3 Decimal Multiplication and Least Common Multiple . . . . . . . . . 59
2.3 Binary Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 602.3.1 Logic Gates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612.3.2 Parallel Logic and the Half Adder . . . . . . . . . . . . . . . . . . . 642.3.3 The Deutsch-Jozsa Problem . . . . . . . . . . . . . . . . . . . . . . 67
2.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3 SEARCHING AN UNSORTED DATABASE . . . . . . . . . . . . . . . . . . . . 76
3.1 Encoding and Storing Information . . . . . . . . . . . . . . . . . . . . . . 773.2 Searching for Information . . . . . . . . . . . . . . . . . . . . . . . . . . . 803.3 Encoding, Storing and Searching: An Example . . . . . . . . . . . . . . . . 833.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4 A SIMPLE ELECTRONIC IMPLEMENTATION OF CHAOTIC COMPUTATION 90
4.1 An Iterated Nonlinear Map . . . . . . . . . . . . . . . . . . . . . . . . . . 904.2 Threshold Control Chaos into Different Periods . . . . . . . . . . . . . . . 904.3 Electronic Analog Circuit: Experimental Results . . . . . . . . . . . . . . . 92
3
4.4 Fundamental Logic Gates with a Chaotic Circuit . . . . . . . . . . . . . . 974.5 Encoding and Searching a Database Using Chaotic Elements . . . . . . . . 1004.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
5 LOGIC OPERATIONS FROM EVOLUTION OF DYNAMICAL SYSTEMS . . 106
5.1 Generation of a Sequence of (2-input) Logic Gate Operations . . . . . . . . 1065.2 The Full Adder and 3-Input XOR and NXOR . . . . . . . . . . . . . . . . 1105.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
6 MANIPULATING TIME FOR COMPUTATION . . . . . . . . . . . . . . . . . 115
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1156.1.1 Flexible Logic Gates . . . . . . . . . . . . . . . . . . . . . . . . . . 1156.1.2 Search Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
6.2 Neural Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1226.2.1 Neural Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1226.2.2 Algorithm Implementations . . . . . . . . . . . . . . . . . . . . . . . 127
6.3 Electronic Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . 1326.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
7 CONCLUSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
BIOGRAPHICAL SKETCH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
4
LIST OF TABLES
Table page
1-1 Summary of the transitions in behaviour of the different intervals described inSection 1.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
1-2 Topological conjugacy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
1-3 Experimental measurements of Feigenbaum’s constant (δ) in different systemsbased on their period doubling. . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2-1 Truth-table for AND, OR, XOR, NOR, NAND, NOT, and WIRE. . . . . . . . . 63
2-2 Necessary and sufficient conditions for a chaotic element to satisfy AND, OR,XOR, NOR, NAND, NOT, WIRE. . . . . . . . . . . . . . . . . . . . . . . . . . 63
2-3 Initial values, xprog, and threshold values, x∗, required to implement the logicgates AND, OR, XOR, NOR, NAND, NOT, and the identity operation (WIRE),with δ = 0.25. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
2-4 Truth table for XOR and AND logic gates on the same set of inputs. . . . . . . 65
2-5 Truth table for two AND gates operating on independent inputs. . . . . . . . . 66
2-6 Required conditions to satisfied parallel implementation of the XOR and ANDgate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
2-7 Required conditions for implementing two AND gates on independent sets ofinputs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
2-8 Examples of initial values, xprog, yprog, and thresholds x∗, y∗, yielding the paralleloperation of XOR and AND gates. . . . . . . . . . . . . . . . . . . . . . . . . . 67
2-9 Examples of initial values xprog, yprog, and thresholds , yielding operation of twoAND gates on independent inputs. . . . . . . . . . . . . . . . . . . . . . . . . . 67
4-1 Truth-table for the five fundamental logic gates NOR, NAND, AND, OR andXOR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
4-2 Necessary and sufficient conditions to be satisfied by a chaotic element in orderto implement the logical operations NOR, NAND, AND, OR and XOR. . . . . . 99
4-3 Numerical values of xprog for implementing logical operations NOR, NAND, AND,OR and XOR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4-4 Updated state values, x1 = f(x0), of a chaotic element in order to implementthe logical operations NOR, NAND, AND, OR and XOR. . . . . . . . . . . . . 99
5-1 The truth table of the five basic logic operations NAND, AND, NOR, XOR, OR. 107
5
5-2 Necessary and sufficient conditions to be satisfied by a chaotic element in orderto implement NAND, AND, NOR, XOR and OR on subsequent iterations. . . . 108
5-3 The truth table of full adder, and necessary conditions to be satisfied. . . . . . . 112
5-4 The truth table of the 3-input XOR and NXOR logic operations, necessary andsufficient conditions to be satisfied by the map. . . . . . . . . . . . . . . . . . . 114
6-1 The truth table of each of the five fundamental logic gates, AND, NAND, OR,NOR, XOR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
6-2 Time values, in arbitrary time “units” for each of the five gates considered . . . 119
6-3 Appropriate time sample instances, based on simulation points, to perform eachof the five gates considered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
6-4 Appropriate R values so time shift an action potential in order to perform eachof the five gates considered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
6-5 Delay times for Vs to implement each of the five gates considered . . . . . . . . 134
6
LIST OF FIGURES
Figure page
1-1 Bifurcation diagram for the Logistic map. . . . . . . . . . . . . . . . . . . . . . 18
1-2 Forward and Backward evolution of F4. . . . . . . . . . . . . . . . . . . . . . . . 19
1-3 Indicative behaviour of I under multiple applications of Fμ for, (a) μ < 1 and(b) 1 < μ. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1-4 Plots of F2 and F2.5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
1-5 Exhibition of the “trapping” of two points in two different configurations of Fμ. 28
1-6 Renormalization of two cases of F 2μ . . . . . . . . . . . . . . . . . . . . . . . . . . 29
1-7 Demonstration of birth of odd periodicity fixed points. . . . . . . . . . . . . . . 33
1-8 Plot of the function F : I→ I. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
1-9 Plot of the function T : I→ I. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
1-10 Bifurcation diagram for the Tent map. . . . . . . . . . . . . . . . . . . . . . . . 40
1-11 Topological conjugacy between evolved states, up to n = 5. . . . . . . . . . . . . 42
1-12 Logistic map bifurcation diagram for some values within 3 < μ < μ∞. . . . . . . 44
1-13 Threshold Control Mechanism. . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
1-14 Threshold values for confining the logistic map on orbits of periodicity 2 to 50. . 47
2-1 Emitted excess by thresholding the logistic map in the interval [0, 0.75]. . . . . . 51
2-2 Encoding the set of integers {0, 1, ..., 100}. . . . . . . . . . . . . . . . . . . . . . 52
2-3 Number encoding in binary format. . . . . . . . . . . . . . . . . . . . . . . . . . 54
2-4 Serial Addition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2-5 Decimal parallel addition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
2-6 The branching algorithm can be extended to a larger treelike structure. . . . . . 57
2-7 Schematic representation of the serial addition method for binary numbers. . . . 58
2-8 Schematic representation of the parallel addition method for binary numbers. . . 59
2-9 Schematic representation of the method for the Least Common Multiple of fournumbers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
2-10 Basis function T - Tent Map. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
7
2-11 Basis function T - Inverted Tent Map. . . . . . . . . . . . . . . . . . . . . . . . 72
2-12 Four realizations of the chaotic Deutsch-Jozsa algorithm for the case k = 3. . . . 74
2-13 The total excess emitted from each of the 72 functions. . . . . . . . . . . . . . . 75
3-1 The Tent map under the threshold mechanism. . . . . . . . . . . . . . . . . . . 78
3-2 Schematic representation of the changes in the state of different elements. . . . . 82
3-3 Searching for “l”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
3-4 Searching for “e”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3-5 Searching for “x”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4-1 Bifurcation diagram of the iterated map for various values of α and β. . . . . . 91
4-2 Graphical form of the map to be implemented by an electronic circuit . . . . . . 92
4-3 Effect of threshold value x∗ on the dynamics of the system. . . . . . . . . . . . . 93
4-4 Circuit diagram of the nonlinear device of Equation 4–3. . . . . . . . . . . . . . 94
4-5 Voltage response characteristics of the nonlinear device. . . . . . . . . . . . . . . 95
4-6 Schematic diagram for implementing the threshold controlled nonlinear map. . . 96
4-7 Circuit diagram of the threshold controller. . . . . . . . . . . . . . . . . . . . . . 96
4-8 PSPICE simulation results of the experimental circuit. . . . . . . . . . . . . . . 97
4-9 Searching for “b”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4-10 Searching for “o”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
4-11 Searching for “d”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
5-1 Graphical representation of five iterations of the Logistic map. . . . . . . . . . . 109
5-2 Patterns of binary two input symmetric operations. . . . . . . . . . . . . . . . . 111
6-1 Schematic representation of a flexible 2-input logic gate . . . . . . . . . . . . . . 116
6-2 Construction of a “generic” signal . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6-3 Schematic representation of the time based search method . . . . . . . . . . . . 122
6-4 Time Delay Unit (TDU) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
6-5 Demonstration of operating a NOR gate, with a single neuron using differentsampling times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
8
6-6 Demonstration of operating a NOR gate, with a neural circuit using differentdelay times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
6-7 Schematic representation of an electronic circuit for logic using time . . . . . . . 133
6-8 Demonstration of operating a NOR gate using an electronic circuit utilizing timedependant computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
6-9 Schematic representation of an electronic circuit for the time dependant searchmethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
6-10 Demonstration of performing a search using an electronic circuit utilizing timedependant computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
9
Abstract of Dissertation Presented to the Graduate Schoolof the University of Florida in Partial Fulfillment of theRequirements for the Degree of Doctor of Philosophy
CHAOTIC COMPUTATION
By
Abraham Miliotis
May 2009
Chair: William L. DittoMajor: Biomedical Engineering
Chaotic Computation is the exploitation of chaotic systems to perform computational
tasks. The abundance of uncountable distinct behaviours by chaotic systems, along with
their embedded determinism, position such systems as perfect candidates for developing a
new computational environment. The present dissertation focuses on algorithms developed
over the past decade within the realm of Chaotic Computation. After a brief exposition
of general Chaos Theory, we proceed to give detailed instructions for performing such
algorithms, as well as specific examples of implementations.
We begin with multiple methods for number representation and basic arithmetic
manipulations, providing from the start evidence of the flexibility of Chaotic Computation.
The compatibility with Turing machines is subsequently shown through an algorithm
for logic operations whose general form is a recurrent theme. We soon though, proceed
further than Turing machines and present a solution to the Deutsch-Jozsa problem, of
arbitrary binary functions. Even more, a practical issue is also handled by showing how
chaotic systems have a natural way for selecting matches of a searched item from within
an unsorted database.
Finally we present our latest results in handling “prolonged” evolution of chaotic
systems. Specifically we demonstrate the dominance of selecting the appropriate
behaviour, for a computational task, over being exact with specific state values, or
even confined to specific physical quantities.
10
CHAPTER 1INTRODUCTION
Over the last five decades the advancement of computational machines has been
an invaluable achievement by our society and extremely beneficial. Moore‘s law [1]
either as a self-fulfilling prophecy, or as a simple description of this progress, has been
obeyed up to now closely. The immense importance of preserving this progress has driven
many to investigate possible reasons that could hinder further increase in computational
performance.
Obviously the ultimate limits imposed by the physical nature of our world were
the first to be identified. The three most important include the speed of light limit on
transmission of information, the limit on the amount of information a finite system
can store, and the thermodynamic limit of thermal energy dissipation by the erasure of
information [2], the latter being the most relevant to current semiconductor technology.
Perhaps most important though, at this time, are the limitations inherent in
semiconductor technology, which is the basis of contemporary computers. Regularly
over the last decade the International Technology Roadmap for Semiconductors (ITRS)
has been published with the aim to indicate the tread of specific features of semiconductor
technology [3], for example lithography resolution, transistor size and connecting wire
diameter. Current technology is “relatively” far from fundamental limits for these
attributes (molecular sizes and thermal noise), but approaching them at an alarming
rate; for example Austin et. al. report a method for creating memory half-pitch size
equivalent to the size of an insulin molecule(6 nm) [4].
As the need for alternative computational paradigms is becoming progressively
evident many physical systems have been proposed as alternative basis for computational
machines. The most prominent being quantum computing [5, 6] and DNA computing
[7–10]. Even after many years of intense research though, both quantum and DNA
computing are still facing some fundamental problems. The main problem with both
11
those paradigms is that they are confined, by their nature, to a single physical realization.
In each case this confinement presents different problems that are difficult to overcome.
For quantum computing the dominant issue is that of thermal noise, which randomizes
individual qubits and leads to “decoherence” of the system. DNA computing is mainly
restricted by the time-scale with which the necessary chemical reactions take place,
making any computational task very slow in producing a result.
Regardless of the physical system used for computation it is clear that there will be
ultimate limitations in effectiveness for accomplishing any individual task. The common
theme in all alternative computational paradigms is flexibility and parallelization; this
is also pursued by conventional computer science research. The common goal is the
construction of a methodology that will provide a computational model that will be able
to solve each computational problem the most efficient way and, if possible, multiple
problems simultaneously. The main driving force to achieve such a model is the fact that
the most powerful computational machine we have, the human brain, is able to both, solve
problems in multiple ways, and handle multiple problems simultaneously.
Chaotic systems over the last few decades have attracted the attention of a large
portion of the scientific community. The main reason is the abundance of such systems in
nature and the extensive repertoire of behaviours they exhibit [11, 12]; compounded by the
fact that they are deterministic systems and can be described by a small set of equations.
More recently there has been intense research in methods to control chaotic systems,
inspired primarily by the work of Ott, Grebogi and Yorke [13]. Control mechanisms have
provided the means to confine a chaotic system to a specific behaviour. Such techniques
have been taken advantage by many fields of contemporary research including control,
synchronization, communications and information encoding [14, 15].
Many attempts have been made since before the 90‘s, and many are still active
fields of research, for bridging dynamics and computation [16–23]. Probably though
the clearest suggestion that a chaotic system, specifically, can be used for computation
12
can be attributed to C. Moore [24]. It wasn’t until late 90‘s though, that the use of a
non-feedback control mechanism on a chaotic system to perform specific computational
tasks was demonstrated [25, 26]. Over the last nine years non-feedback control has been
utilized to show how chaotic systems can perform arithmetic and binary logic operations,
and even further to solve more complex problems like the Deutsch-Jozsa problem and
searching an unsorted database [27, 28]. Non-feedback control typically is achieved either
by the use of a threshold on a state variable or by selecting specific values for the system
parameters, in either case confining a system to a specific subset of all available points.
The key idea is that from the wide variety of behaviours embedded in a chaotic
system one can find a specific pattern of behaviour that can accomplish a specific task.
Control can confine a system to the required specific behavioural pattern, without loss
of the ability to switch to a different behaviour and perform a different task. This is
accomplished by rigorous investigation of a chaotic system to identify the required
behaviours that can reliably represent computational tasks; and the state variable
thresholds, or parameter values, that will confine the system to evolve in the required
manner. This allows us to envision a computational machine which has as its building
block a chaotic system; each element being identical to all others, providing redundancy
and reliability; each element able to perform a multitude of functions, providing flexibility;
and each element independent of all others, providing parallelism of tasks.
The reason we are specific about which chaotic system to use is the Universality of
chaos [29]. Chaotic systems, even if their governing equations differ, behaviourally are
the same. This not only allows us to choose the most convenient system for theoretical
development of algorithms, but more importantly does not confine the physical realization
to a specific technology. As mentioned above, chaotic systems exist in abundance in nature
including high-speed electronic circuits, lasers, and even neurons; any of which can be used
as the building block of a chaos based computer.
13
1.1 Dissertation Overview
This dissertation can be considered to be of four parts. The first part is the remaining
of this introductory chapter, the second part chapters two and three, while the fourth
chapter can be considered to be on its own. Finally chapters five and six comprise the
final part of our exposition of Chaotic Computation.
In the remaining of this introductory chapter we provide necessary parts of Chaos
Theory so that to relate it with computation. Specifically we expose the reader to a view
of Chaos Theory from a “loose” Set Theory approach, with the intension to imply to the
reader the connections from Set Theory, to Chaos Theory, to Computation and general
Mathematical Logic. As this dissertation is concerned solely with Chaotic Computation
we do not delve in depth on the peripheral issues and direct the reader to appropriate
references [30, 31].
Chapters two and three can be considered to be the main part of this dissertation;
they are an exposition of established algorithms of Chaotic Computation [25–28, 32, 33].
We begin in chapter two with the earliest algorithms of Chaotic Computation developed
over the last nine years by Ditto et. al., algorithms for number representation, arithmetic
operations and a solution to the Deutsch-Jozsa problem. While the third chapter presents
exclusively the recent algorithm for searching an unsorted database [28].
In the relatively small third part that is chapter four, we present a recent implementation
of most abilities of Chaotic Computation with an extremely simple electronic circuit [34].
Despite its length, the aims of this part are twofold; first and foremost to give substance
of realization to the whole concept of Chaotic Computation through a physical system
implementation; and beyond, to present a concrete demonstration of the Universality of
chaos, and the ease with which we can translate our results to any chaotic system.
The final part is our most recent results, the “cutting edge” developments in Chaotic
Computation [33]. Chapter five, while focuses solely on binary operations, introduces
the importance of what can be considered to be “the time dimension”; in addition it
14
expands on our treatment of “state values”, by showing more concretely the importance
of specific behaviours over exact state values. Finally chapter six expands further the
idea of manipulating time. Through demonstrations, with electronic circuits, and mainly
neural circuits we exhibit implementations of Chaotic Computation algorithms using “time
instances” as the medium for computational commands.
We wish to the reader the encounter with this exposition to be both informative and
enjoyable.
1.2 Chaos
Chaos Theory is the third major “revolution” of Physics of the 20th Century. Even
though its “birth date” can be placed as contemporary to Relativity and Quantum
Mechanics, circa 1900, it wasn’t until much later, in the early 1960s, that the field actually
attracted enough attention to attain “critical mass”, a slightly American biased popular
science recount of the history of Chaos Theory can be found in [35].
This dissertation is not about Chaos Theory as such, but more about some of the
“features” of Chaos Theory. We utilize much of what is defined, predicted, and expected
from Chaos Theory to propose a whole new realm for computation. The problem is that
Chaos Theory is a very closely packed theory, with each part relating to some other part
and the whole to the details (a concept which is a result of the theory as well). We will
present the “features” of Chaos Theory that are necessary for projecting to the reader our
results and direct the reader to: Devaney (1982) [36] for a solid mathematical exploration
of Chaos and the origin of one of the most accepted definitions of Chaos; Peitgen, Jugens
and Saupe (1992) [37] for a more hands on demonstration with heavy emphasis on fractals,
and Ott (1993) [38] for a midpoint approach to Chaos Theory, with the addition of an
exposition of Quantum Chaos.
1.2.1 Logistic Map, Topological Transitivity and Period Three
In this introductory chapter we provide a somewhat “non-traditional” view of chaos.
The approach we take is one closer to a mathematician‘s rather than a physicist‘s, since
15
computation, our main concern, is more of a mathematical concept than physical. We
present chaos through in a “loose” set theory context, that is we treat topological spaces,
intervals and even state variables as sets of points, we try to avoid any “linearizing” tool,
like differentiation, and any representation that involves discretization of our set of points,
like statistical manipulations. At the same time our exposition is “loose”, since we do
not go into extreme formal mathematical precision, that is we omit providing proofs and
extensive definitions of terms used. We direct the reader to two seminal sources, Principia
Mathematica of Whitehead and Russell [30] and Kurt Godel‘s ever important paper,
“On formally undecidable propositions of principia mathematica and related systems”
(1931) [31], for both more background information of our approach, especially the missing
mathematical details, and more importantly for justification for our approach and how it
relates to the concept of computation.
Our main “tool” for presenting our necessary parts of Chaos Theory is the discrete
time Logistic map:
F n+1μ (xn) = xn+1 = μ ∙ xn ∙ (1− xn), (1–1)
where x is the state of the system, n1 is the time step and 0 < μ can be thought of as a
growth rate. Actually the origins of this quadratic equation is from population dynamics,
specifically it is modeling the behaviour of a population with limited resources. In that
context x represents the current fraction of the population with respect to the maximum
possible sustainable population (x = 1). In its continuous-time form, the model can be
solved analytically and gives rise to the most common sigmoid function, a “well behaved”
function of continuous growth and eventual saturation2 . When we consider discrete time
1 We shall drop the superscript for the cases of n → n + 1, unless we wish to emphasisethe use of a single application of Fμ.
2 A theorem by Poincare and Bendixson [39] guarantees that continuous time twodimensional planar systems can not exhibit chaos.
16
though, for example n denoting the current generation of beings in our population, we
have Equation 1–1 and the behaviour of x is no longer simple continuous growth leading
to eventual saturation; in fact in this discrete case if the population does manage to reach
saturation it collapses to zero.
For our purposes we need a closer look at Equation 1–1. With a first glance it is
clear the function is governed by the three variables x, μ, n, each in principle unbounded.
There is no need though to go to such lengths, as we can confine ourselves to the domains
0 ≤ x ≤ 1, 0 < μ ≤ 4, 0 < n. This is truly a very small portion of the whole available
space, so briefly lets justify this confinement, and we will provide more justification as we
focus more in the above domains. Starting with 0 < n, we can set the “beginning of time”
at any “moment in time”, we will actually see below that this is totally arbitrary; actually
for some results of Chaos Theory reversing the arrow of time, n→ n− 1, is very useful. In
the case of μ things are a bit more complicated, except for the case μ = 0; for μ < 0 we
can show the same behaviours as for 0 < μ; for 4 < μ most x values escape to infinity and
the ones that are “trapped” form a Cantor set which is the prototypical fractal and, again,
shares many of the “features” that are present in 0 < μ ≤ 4, subsequently we will confine
μ further. Finally x, with the other two variables confined to the above domains, we have
x < 0 and 1 < x escaping to infinity or eventually collapsing to zero.
Therefore our concern is what happens to a state x ∈ [0, 1] for values of μ ∈ (0, 4], in
the “forward” direction of time n → n + 1. An immediate view of the behaviours in this
domain can be obtained through a bifurcation diagram, see Figure 1-1. Such a diagram is
obtained by evolving an initial x0 state,123 in this case, for a large number of time steps
and plotting the state at the final few time steps, in this case we evolved the map 500 time
steps and plotted the last 200 states. From this empirical approach we can already see the
3 Using the critical point of the map we are guaranteed to evolve into an attractingstate, if one exists.
17
richness that this simple map provides, at least three behaviours are evident: collapse to
zero, attraction to a non-zero fixed point and attraction to a periodic orbit.
Figure 1-1. Bifurcation diagram for the Logistic map. Eventual behaviour of an initialstate x0 under repeated applications of Fμ.
This static picture of what happens, even though extremely rich, is a generalization
of the details of the behaviours of x. For example it should be clear, but it is not clearly
shown, that this picture is “almost” completely invariant to the initial x0, that is what
happens to a state x ∈ [0, 1] happens to “almost” any other initial state, the map has
a local effect that is global, and visa versa. It is not clear though what happens at the
onset of chaos, when “almost” all points behave the same way and at the same time all
the “rest” of the points behave in a different way!
To see the details inside the bifurcation diagram we take a more “dynamic” approach
with respect to the three available variables. Starting with n we will consider the question
“Where do states come from?”, we are not going to address the issue of “time reversal”,
18
but rather take a step “back in time” and consider:
x0 = μ ∙ x−1 ∙ (1− x−1) = Fμ(x−1), (1–2)
where x0 ∈ [0, 1] and 0 < μ ≤ 4. Figure 1-2 shows two plots of the function F 14 each
showing how the function evolves in either “direction of time”.
Figure 1-2. Forward and Backward evolution of F4. (a) Forward evolution - Empty circle,#, marks an arbitrary initial point x0; we can track its forward evolution bymoving “up” (or “down”) to meet the function F4 and then “right” (or “left”)back to the diagonal, each full cycle of these two steps is equivalent to a singleapplication of F4; thus the full circle, , marks the point F 54 (x0) = x5. (b)Backward evolution - The steps of the forward evolution can also be reversed;i.e. from an arbitrary initial point x0, empty circle #, we can move “left” and“right” (or “left” twice) to the two points on F4 and then “up” and “down”(or “down” twice) to meet the diagonal on the points of F−14 (x0) = {x
+−1, x
−−1},
the two empty squares, o; and of course further back to the four points ofF−24 , the four full squares, n, and so on.
By solving Equation 1–2 we observe the following: by setting x0 = x−1 we find the
fixed points at x∞ = 0 and x∞ = 1− 1μ ; now setting x0 = 0 and x0 = 1−1μwe find for each
fixed point its two pre-images, specifically x−1 ∈ {0, 1} → 0 and x−1 ∈ { 1μ , 1−1μ} → 1− 1
μ;
19
and finally by solving the quadratic for any x0 we reach x−1 =12±√μ2−4∙μ∙x02∙μ , which leads
to μ4< xn ⇒ xn−1 ∈ C, that is ∀x >
μ4, F−1μ (x) ∈ C; the pre-images of any x value greater
than a fourth of the “growth factor” (μ), of the applied map, are in the complex plane.
This is an interesting result and the start of another long story, it shows how the dynamics
of the logistic map extend in the complex plane. This is beyond our current scope, but in
passing we note that the logistic map is a reduction of the quadratic maps Qc(z) = z2 + c
where z, c ∈ C, the source of the famous Mandelbrot and Julia fractals [40], this is an
initial hint to the concept of Universality, which we will address in Section 1.2.2.
We are now ready to begin to investigate how μ changes the behaviour of x ∈ [0, 1].
We can not explain what happens at every value of μ so we consider the change before
and after a point of transition. Clearly at μ = 1 we have a major change, the fixed point
x∞ = 1 − 1μenters the domain [0, 1] and becomes an attractive fixed point, actually the
“big event” at this μ-value is that the two fixed points cross paths. At the same time 0
becomes a repelling fixed point; these conclusions can be drawn by taking the absolute
value of the derivative of the map at the fixed points,∣∣∣dF (x∞)dx
∣∣∣, a standard technique for
characterization of dynamical systems. This is though a local linearization of the map, we
prefer to show this, and subsequent transitions, in terms of global effects, thus prepare the
ground for what happens at higher values of μ.
We need to deviate for a while to single out some special points on the I = [0, 1]
interval. We already seen the four points that map to the fixed points, {0, 1, 1μ, 1 − 1
μ},
and we add to this list {14, 12, 34}. In general 1
2is the most important point of Fμ, not
because of its value, but since it is the critical point of the map and its backward and
forward evolution can actually characterize the map extensively; the theory behind the
evolution of the critical point is called kneading theory [36]. The importance of the
points 14and 3
4is topological, following Fμ(
14)�Fμ(12) =
34and of course Fμ(
14) = Fμ(
34),
in addition to Fμ(12) = sup Fμ(I), visually this means what will happen to the interval
[14, 34] → F n+2μ ([1
4, 34]) is what happened previously to the interval [ 3
4, 1] → F n+1μ ([3
4, 1]) and
20
what is going to happen to [ 34, 1] → F n+1μ ([3
4, 1]) is what happened to [0, 1
4] → F n+1μ ([0, 1
4]),
in a normalized sense. Consider the interval [0, 1] separated in three parts I1 = [0,14),
I2 = [14, 34], I3 = (
34, 1]. No matter the μ-value we know the following: Fμ(0) = 0,
Fμ(14) = Fμ(
34) = 3∙μ
16, Fμ(
12) = 1∙μ
4and Fμ(1) = 0 so Fμ({0, 14 ,
12, 34, 1})→ {0, 3
4∙(14∙μ), (1
4∙μ)},
divide out by a factor of ( 14∙ μ) and its like nothing happened! We need to emphasise that
this picture is only for visualization purposes and it applies only to a single application of
the map, a “snapshot” if you like that we can keep track of while iterating n → ∞, below
we will discuss the proper way to view evolution of intervals, the renormalization operator.
Returning to the question “What happens as μ < 1 → 1 < μ?”. There are multiple
ways we can analyse the situation, we look at it with respect to changes to the behaviour
of the aforementioned intervals. Globally before the point of transition (μ < 1) we simply
have Fμ(I) < I so its clear that in the limit n → ∞ the whole interval will collapse to
zero. To “see” the change at the transition consider Fμ(I1) → In+11 ∪ In+12 , that is I1 will
become the union of what can be considered to be the I1 and I2 of the next application
(n + 1) of the map, at which F n+1μ (In+12 ) → In+23 and F n+2μ (In+23 ) → In+31 ∪ In+32
completing the “circle”. Actually it is a double spiral with a moving pivot, the sequence
of post-images of 14and 3
4denoted as ∗x. So we have at μ < 1, Fμ(
12) < 1
4, which implies
that F nμ (I2) ∩ I2 = �, this is enough in some sense to characterize the evolution since we
have seen that in some sense both I3 and I1 end up, at least partly, in an interval that can
be called Im2 , another way to describe this evolution is Fnμ (∗x) = 0 as n → ∞. With this
description of the evolution of Fμ for μ < 1 we can see what changes at μ = 1, specifically
Fμ(I2) ∩ I2 6= � and at the single point magnification F1(12) =14. The consequence of this
change can be expressed in multiple ways: since the post-images of In1 and In3 are in I
n+12
it means that the relationship Fμ(I2) ∩ I2 6= � can be extended to F nμ (I2) ∩ In−12 6= �
even as n→∞ so I∞2 contains a single point, which is the eventual post-image of all these
points that get “trapped” in the sequence of Im2 ; or simply as n → ∞, Fnμ (∗x) → 1 − 1
μ
(= 0, for μ = 1); and even though the “pivot” point does converge to zero eventually, there
21
will always be at least one more “point” between its value and 0, these “points” we will
see are cental to what is chaos. So now an increase in μ toppling it over 1 separates 0 and
1 − 1μby a more “concrete” amount, giving substance to the fixed point at 1 − 1
μ, and to
the points inside the interval (0, 1 − 1μ), for a view of the evolution of points in either of
these two regions (μ < 1 or 1 < μ) see Figure 1-3. We will come across transitions like the
one we just described an infinite number of times going from μ = 1 to μ = 4.
Figure 1-3. Indicative behaviour of I under multiple applications of Fμ for, (a) μ < 1 and(b) 1 < μ. (a) (μ = 1
2) The two fixed points are one at 0, and the other outside
I. (b) (μ = 112) 0 remains a fixed point and now 1 − 1
μ∈ I; with one pre-image
being itself at 1− 1μ= 13and the other at (μ
4<) 1μ= 23. Empty circles, #, mark
initial points and full circles, , final points (not the fixed point).
Now we are in the 1 < μ region and, even though the bifurcation diagram doesn’t
show it, the next point of transition is at μ = 2. We have seen how the behaviour of
the whole interval I changed at μ = 1, and with it we should change our view of its
subintervals. We could continue the discussion with I1, I2 and I3, and we will return to
them when necessary. Our aim though is to make the behaviour of the whole interval I as
22
visual as possible, and so given that now 1 − 1μexists in I (as well as both its pre-images
{1 − 1μ, 1μ}) we have a better choice of subintervals to use, specifically J1 = [0, 1 − 1
μ),
J2 = [1 − 1μ, 1μ], J3 = (
1μ, 1]. As we have done for the previous transition, first we will
see how the interval I behaves with 1 < μ < 2. A quick side-note, taking the derivative
at the fixed points we find the “nature” of 0 to be repelling and 1 − 1μattracting. The
details though make all the difference, as we have defined the intervals Ji we see that
F 2μ(J03 ) = J
11 = F
1μ(J
01 ). The interval J3 gets mapped onto J1 and J1 onto itself, so it seems
that J1 is not changing and even more it seems that J1 and J3 are “disconnected”4 from
J2. So it seems we have three types of points, the set of four points {0, 1− 1μ ,1μ, 1}, and the
two intervals (1 − 1μ, 1μ) and (0, 1 − 1
μ) ∪ ( 1
μ, 1), the easy part is the fixed points and their
pre-images Fμ({0, 1}) = 0 and Fμ({1 − 1μ, 1μ}) = 1 − 1
μand in addition, since μ
4< { 1
μ, 1},
we know all other pre-images of the fixed points are in the complex plane. For (1 − 1μ, 1μ)
since Fμ : (1 − 1μ, 1μ) → (1 − 1
μ, 14∙ μ) and 1 − 1
μ< 14∙ μ < 1
μwe see that all points in the
limit n → ∞ are “squeezed” closer and closer to 1 − 1μ. Finally for (0, 1 − 1
μ) ∪ ( 1
μ, 1), the
simple picture is that ∀x ∈ (0, 1 − 1μ) and ∀y ∈ ( 1
μ, 1), F nμ (x) > x so as n → ∞, x → 1 −
1μ
and as we have seen ∀x : Fμ(y) = x, so F n+1μ (y) → 1 − 1μ, therefore every point moves
away from zero (or one) and towards 1 − 1μ. We have though seen that Fμ(J1) = J1, note
that ∀x ∈ J1, x <μ4so every point in J1 has two real pre-images, one of them is of course
in J3 and it has no pre-images of its own, but the other one is in J1 and x−1 < x0 so the
pre-image also has two pre-images, and so on. So the complete picture of the evolution of
J1 is that each x0 does evolve to x1(> x0) and eventually, at n → ∞, to 1 − 1μ, but on
every application of the map it is “replaced” by two other points; in other words, points
from the neighbourhood of 0, and of course 1, eventually escape this neighbourhood and
evolve into the neighbourhood of 1 − 1μ, 0 is a repelling fixed point and 1 − 1
μattracting.
4 “Disconnected” here is used as in every day language, in fact since F nμ (J3) → 1 −1μas
n→∞ the intervals are of course still connected.
23
As soon as we cross from μ < 2 into 2 < μ we shall see more clearly the importance in
considering this behaviour.
At μ = 2 we reach the second transition point, 1 − 1μ= 1μ= 12, in some sense as
at μ = 1 we had 1 − 1μ= 0, two of the points that lead to a fixed point cross paths;
the difference in this case is two points that lead to the same fixed point (1 − 1μand 1
μ).
Before we go on to the exciting changes that happen in 2 < μ, we note in passing that at
μ = 2 the derivative at the fixed point is 0 and so 12is called “super-attractive” point; 1
2
has no pre-images real or imaginary, in a sense all points (even from the complex plane)
converge to 12; more importantly though, it is the only point in J2 as a result we now have,
in addition to F2(J3) = F2(J1) = J1, F2(J2) = J2, that is all our intervals are “eventually
invariant”5 , see Figure 1-4(a) for a view of the behaviour at μ = 2.
We are now in the 2 < μ region; there are two important general changes in
behaviour, first of the fixed point acquires now an infinite number of pre-images, since
1μ< μ
4, and as a consequence, topological “mixing” is now possible, that is any closed
interval contains points that have pre-images, or even post-images, outside the interval,
the set of points that lead to the fixed point has more than one point; more visually notice
that J1 ∩ J2 ∩ J3 6= �, see Figure 1-4(b). This leads us to the first concept that is a
requirement for chaos, topological transitivity.
Definition 1.1. Topological Transitivity.
Given a map f : J → J , f is said to be topologically transitive on J , if for any two
open subintervals U , V ⊂ J there exists a x ∈ U and a y ∈ V such that fn(x) = y, for
some 0 < n.
In other words, given an interval any point in it can, eventually, be mapped on any
other point in the interval. This is not active for values 2 < μ < 3.5699456..., but the fact
5 We shall see that the transition from “eventually invariant” to invariant makes all thedifference.
24
that we have intervals “mixing” is the first step, it is the difference between “eventually
invariant” and invariant. Before we proceed let us redefine our intervals so that they are
not “mixed” before we even apply the map and to keep the picture of how the intervals
evolve as it has been up to now. So we re-define J1 = [0,1μ), J2 = [
1μ, 1− 1
μ], J3 = (1− 1
μ, 1],
basically acknowledging the fact that the pre-images of the fixed point (1 − 1μ) have
switched sides around 12, see Figure 1-4(b) for the “mixing” of the Ji intervals and their
redefinition as Ji, where i ∈ {1, 2, 3}. The behaviour of these intervals is straightforward,
we still have Fμ(J3) = Fμ(J1), but now Fμ(J1) = J1 ∪ J2 and Fμ(J2) = J6 ⊂ {1 − 1μ} ∪ J3.
Visually we have J3 stretched and rotated around 1− 1μ onto J1 ∪ J2, J1 is simply stretched
onto J1∪J2 and J2 folded inside itself, with 12 as the pivot of the folding, and again rotated
around 1 − 1μto land inside J3; overall all points are spiralling around and inwards 1 − 1
μ,
which is the situation we had at μ < 1, so it should come as no surprise what happens at
μ = 3.
Figure 1-4. Plots of F2 and F2.5. (a) Display of the “super-attractive” case when12is the
fixed point. (b) Display of the six intervals {J1, J2, J3} and {J1, J2, J3}, theoverlap of Ji shows the initiation of topological “mixing”.
25
As the points are spiralling inwards the fixed point, and μ is gradually raised we have
in J3 what was happening to I, the points collapsing towards the fixed point, until there is
a point left out, specifically when Fμ(12) ≥ (1 − 1
μ) + 1
4∙[1− (1− 1
μ)]6 ; or more visually
when the stretching of J3 (and J1) “creates” more points than the “available” points in
J1 ∪ J2 and so “a” point is left out that will never spiral on 1 − 1μ7 . All this happens at
μ = 3 and, as at μ = 1, the point left out is a post-image of 12only this time we have, for
0 < n, F 2n−13 (12) ∈ J3 and F 2n3 (
12) ∈ J2 as n → ∞, so we have a fixed point created in J3
and one in J2, thus initiating the famous period-doubling route to chaos. Before we follow
this route, we need to note what happens to 1 − 1μour “current” fixed point. Just as at
μ = 1, the fixed point at 0 changed form from attracting to repelling, in the same manner
the nature of 1 − 1μchanges, only with an important difference as we have already noted,
unlike for 0 the number of pre-images for 1 − 1μis infinite, but at least countable, for now.
It is time to provide the promised further justification of neglecting the regions 4 < μ
and μ < 0. The creation of the two fixed points by “entrapment” is exactly what happens
to points on I for 4 < μ, with the slight difference that the trapped points for 4 < μ
collapse to 0, basically the top part of the parabola is protruding over some value on
which the two points that happen to be on, are captured, Figure 1-5 displays this process.
For μ < 0 consider F 23 (J2), it is an inverted parabola, differing from F10 for 0 < μ only
by a shift of 23; and a small curvature deviation caused by the higher power terms of x,
see Figure 1-6(b). In fact we can take F 23 , or for any μ for that matter, and using the
renormalization operator translate it to a form for I, as follows:
Define:
x∞ = 1− 1μand x∞ =
1μ.
6 Compare with F1(0.5) ≥ 0 + 14∙ (1− 0).
7 This is really a simplistic picture as it can be argued that there are always enoughpoints in an interval, but its visual aid is of great value.
26
Lμ(y) =(y−x∞)x∞−x∞
and L−1μ (y) = x∞ + (x∞ − x∞) ∙ y.
We have:
RF (x) = Lμ(F 2μ(L−1μ (x)
))= Lμ ◦ F 2μ ◦ L
−1μ (x).
Where RFμ is the renormalized function of F2μ that translates J2 = [
1μ, 1 − 1
μ] onto
I = [0, 1], both by reflecting it upwards and rotating around so that 1 − 1μ= 0 and 1
μ= 1,
see Figure 1-6 for two explicit examples of renormalization, relating to the issues of μ < 0
and 4 < μ. This should make it clear how everything that happened to F 1μ will happen
to F 2μ ; of course appropriate Lμ and L−1μ can be found for all n, but this is beyond our
current scope, see the work of Kenneth G. Wilson [41, 42]. This sidetrack8 was intended
to show how the actual value of μ is very “relative”, as well as the actual interval of x; or
to be more precise the actual set of x values.
Returning to the 3 < μ region and how the complexity of behaviours increases
exponentially with increasing μ. At the moment we have two attracting fixed points, each
with its two pre-images, one of them being the other fixed point and the other pre-image
has its own two pre-images and so on. The process we have explored up to now will
repeat, in fact infinite times, as μ is raised further. We will have a “super-attractive” case,
when F 2μ(12) = 1
2, at μ ≈ 3.23..., and of course the trapping of yet another two points for
each one of the “current” two cycle fixed points, and thus the creation of the four fixed
point cycle, at μ = 1 +√6 ≈ 3.44... and of course the change of the two cycle fixed points
into repelling points. The four cycle will become eight cycle and so on for all powers of 2.
To our visual picture, we still have the action of the primordial fixed point, 1 − 1μ, even
though now, at 3 < μ it is repelling, its pivot action in the spiraling-in still exists. In
addition we have more “local” effects from the newborn fixed points acting as pivots for a
spiralling-in of their local surrounding points. In effect we have vortices within vortices.
8 Another reason for this diversion is to provide a small glimpse, without going intodetails, to the picture of F 2
k
μ∞with k →∞, which we will meet soon.
27
Figure 1-5. Exhibition of the “trapping” of two points in two different configurations ofFμ. (a) Plot of both F
13.2, solid line, and F
23.2, dashed line showing the eventual
evolution of different initial points, marked by #, to the 2-cycle fixed pointsmarked by . The solid triangles, K, mark the sequence between the two fixedpoints. Note that F 2 and the diagonal can be used like F 1 in Figure 1-2 totrack the 2∙n-step evolution of points. (b) Plot of F 1
2+√5as well as the
evolutions of some different initial points, marked by #, under multipleapplications of F2+
√5. Note that the two points marked by , are “trapped”
by F 12+√5, while their four pre-images, marked by n, are “trapped” by F 2
2+√5,
and so on to F∞2+√5, and even further to F2+
√52∞ .
As more and more fixed points are created the gaps between them for all this swirling
to take place are getting smaller and smaller, leading to smaller and smaller increments
of μ to create the subsequent doubling, we will return to this evolution in Section 1.2.2
when we talk about Universality. For now we are interested in what happens at the
accumulation point, also know as Feigenbaum point, named after its discoverer Mitchell
Feigenbaum, μ∞ ≈ 3.569945671.... Let us attempt to account all the points in I and their
behaviour starting with the easiest part {0, 1} the two points that collapse to 0. We have
the fixed point of F 1μ , that is 1−1μ, with it we still have its pre-image 1
μand “going further
28
Figure 1-6. Renormalization of two cases of F 2μ . (a) Case of F23.8, for which
L3.8 ◦ F 23.8 ◦ L−13.8(x) translates into (b) A plot of RFμ(x), showing how
representative points, marked by symbols, are translated onto points of similartopology. It should also make clear how cases of 4 < μ are “contained” in someform in 0 < μ ≤ 4. (c) Case of F 23 , for which L3 ◦ F
23 ◦ L
−13 (x) translates into
(d) A plot of RFμ(x), showing how representative points, marked by symbols,are translated onto points of similar topology. It should also make clear howcases of μ < 0 are “contained” in some form in 0 < μ ≤ 4.
back time” all the points that lead to it, as we mentioned infinite number of them. From
F 2μ we have the two fixed points, of course different from 1 −1μand 1
μ, each of these two
fixed points has the other one as one of its pre-images, but in addition each point has one
more pre-image, which of course has its own pre-images, once again in infinite number.
Continuing this for each F nμ with n = 2k as k → ∞ we can account for all fixed points of
periodicity a power of 2, as already explained above through the period-doubling route,
and all their pre-images. With all these points accounted for, it might seem we have run
out of points for F 2k
μ∞at k → ∞, to have any fixed points, in fact though we still have just
as many points as we accounted for, we have a Cantor set once again and technically our
fist instance of chaos. The details of the behaviour at the Feigenbaum point (μ∞) of F2∞
μ∞,
29
and for that matter of F 1μ∞ , are complicated to describe, both visually and mathematically,
without the introduction of symbolic dynamics, further exposition of Cantor sets and
renormalization group operator, concepts which are beyond our current scope. Instead
we will proceed in increasing μ further and show how F 1μ becomes chaotic on the whole
interval I = [0, 1] at μ = 4, at which point the behaviours are the same as for F 2∞
μ∞at μ∞,
but easier to describe and visualize.
We utilize the period doubling route to the Feigenbaum point to introduce the second
requirement for chaos, fixed points of periodicity three. The original definition a chaotic
system by Devaney [36] required of a system: (i) Topological Transitivity, (ii) the set of
periodic points to be dense in the space of the system, and (iii) sensitive dependance on
initial conditions. Banks et. al. [43] though, showed how transitivity and density of the
set of periodic points imply sensitive dependance on initial conditions, while Li and Yorke
[44] proved that existence of period three fixed point implies the set of periodic points is
dense; in fact they also show how the set of “non-periodic” points is also dense. Before Li
and Yorke though, it was Sharkovsky [45, 46] that introduced his famous theorem9 that
actually claims more than just period three implies all other periodicities, it actually
provides an ordering of the natural10 numbers that will guide us to find fixed points of
periodicities other than a power of 2.
Theorem 1.1. Sharkovsky’s Theorem.
Given a continuous map f : R → R, the ordering of the natural numbers: 1 C 2 C
22 C 23 C ... C 2n C ... C 7 ∙ 2n C 5 ∙ 2n C 3 ∙ 2n C ...
... C 7 ∙ 2 C 5 ∙ 2 C 3 ∙ 2... C 9 C 7 C 5 C 3,
9 Even though the theorem applies only for maps on the real line, its strong conclusionmakes it important to note in any exposition of chaos.
10 Actually the ordering is for more than just the Natural (N) numbers, it covers asmany periodicities as all the Reals (R) [47].
30
and if the map has a fixed point of periodicity q, where p C q, then the map also has
a fixed point of periodicity p. If the map does not have a fixed point of periodicity p, where
p C q, then the map does not have a fixed point of periodicity q. Further, for any p a
continuous map does exist which has fixed points of periodicity p, but no fixed points of any
periodicity q.
The most obvious implication of Sharkovsky’s theorem, which is analogue to the
Li-Yorke theorem, is that if a map has a fixed point of period three then it has fixed
points of all number periodicities. Of course the specification of the ordering allows us to
say more, if a map has a finite number of periodic points then they are all a power of 2,
and the converse if a map has a periodic point that is not a power of 2 then it has infinite
number of periodic points. The second part of the theorem allows for the existence of
maps with an infinite number of points of distinct periodicity and yet without all natural
number periodicities. Actually the existence of such maps can be utilized in the proof of
the theorem to establish this number ordering, and perhaps more importantly to guarantee
the success of the threshold control mechanism for chaotic systems, see Section 1.2.3.
Furthermore there is a corollary to Sharkovsky’s theorem, which states that if the
map f is dependant on a varying parameter the ordering of the “birth” of new period
fixed points is given by the Sharkovsky ordering, which brings us back to Fμ and the
further changes as we increase μ. We have already seen the birth of all periods of power
of 2 up to the Feigenbaum point, our next step is to show the birth of all other fixed
points of even periodicity by increasing μ over the Feigenbaum point to μ2 ≈ 3.67...11 and
showing how this region of μ allows only even periodicities.
Readjusting the intervals under observation we define J4 = [F2μ(12), Fμ(
12)], or more
clearly using Fμ ≡ h and h(12) =μ4, J4 = [h(
μ4), μ4]; in addition we define J5 = [h(
μ4), 1 − 1
μ]
11 The algebraic expression for μ2 =23[ 4u+u+1], where u = (19+3
√33)
13 ; obtained either
by F 2μ(12) = 1− 1
μor sup {F−1μ (
1μ)} ≥ μ
4.
31
and J6 = [1 − 1μ, μ4], therefore we have Fμ(J5) ⊂ J6 and Fμ(J6) ⊂ J5, for μ∞ < μ < μ2.
Before focusing on these intervals let us consider the rest of the points in I, of course
Fμ ({0, 1}) = 0 nothing new there; Fμ : (μ4, 1) →
(0, h(μ
4))and F nμ :
(0, h(μ
4))→ J4
as n → ∞. So it is clear that all points in I eventually are mapped inside J4, once
there we have ...J5 → J6 → J5... so it is impossible to have a point that is mapped
back on itself after an odd number of applications of the map, furthermore points of all
even periodicities are possible and exist. At μ2 we can see what happens to F2μ and it
is precisely what has happened to each other even periodicity while increasing μ from
the Feigenbaum point, with a small but important difference in the behaviour of 12.
Specifically F 2μ2(12) = 1
μthat is 1
2is mapped to the pre-image of the primordial fixed
point, so 12is a pre-image of a fixed point, but not itself fixed as in the “super-attractive”
case. Visually this means that there is at least one interval that is completely wrapped
around onto itself, in contrast to previously when 12was “eventually” mapped onto a fixed
point. We can also now see the difference between “eventually invariant” and invariant,
F 2μ2(J5) = J5 and F2μ2(J6) = J6, note the equality, and see Figure 1-7(a) for a visual of
the intervals. Perhaps most importantly both pre-images of 1μare now less than μ
4, which
implies that both have two real pre-images, and at least two of those pre-images each have
two pre-images of their own, and so on; the fixed point 1 − 1μhas now a set of pre-images
which form a Cantor set. When we put all these implications together, along with the fact
that the Sharkovsky ordering for F 2μ is the same as the ordering for F1μ without the last leg
of the odd integers (so all number periodicities exist for F 2μ2), it should be clear that F2μ2is
chaotic. It is important to note that F 2μ2 is chaotic on J4 not the whole I.12
Therefore now we are left with the last leg of the Sharkovsky ordering, the odd
number periodicities. Obviously the most important being period three and the easiest
one to visualize since it is relatively “short”. But before we focus on period three lets see
12 Actually F 2μ is independently chaotic on J5 and J6.
32
how odd periodicities become possible in the first place. So at μ2 < μ we have seen that
F 2μ2(12) < 1
μ, so F 2μ(J2) ∩ J1 6= ∅ or to relate two sets from different views J4 ∩ J1 6= ∅, lets
call this intersection J7 ⊂ J5, most importantly J7 < 1μ⇒ Fμ(J7) < 1 − 1
μ⇒ Fμ(J7) ⊂ J5
so we have J7 → J5 →...→ J6 → J5 → ...︸ ︷︷ ︸
infinite9 even times
→ J6 → J7, hence the “highest” odd
periodicity orbit is “born”, see Figure 1-7(b).
Figure 1-7. Demonstration of birth of odd periodicity fixed points. (a) Plot of Fμ2, solidline, and Fμ
22, dashed line. We also mark the intervals J4, J5 and J6, it should
be clear that Fμ2 : J5 → J6 and Fμ2 : J6 → J5; Fμ22 is chaotic on both these
intervals. Note also that Fμ2 : I �{0, 1}�J4 → J4. (b) Plot of Fμ, solid line,and F 2μ , dashed line, at μ = μ3 + 0.02, a value for which the period three fixedpoints are attractive. Note how J5 ∩ J1 = J7 6= ∅. Within J7 we have one ofthe period three trapping regions, see inset. (c) Inset, showing a plot of F 3μ(x),for μ = μ3 + 0.02 and x ∈ [0.1358, 0.1722], topologically the same as someFμ(x), for x ∈ I.
9 Or none in the case of period three.
33
We return now to the issue of the birth of the fixed points of periodicity three for F 1μ .
At μ3 = 1 + 2√2 14 we have the birth of the period three orbit, the simplest way to view
it is that the solutions to the equation F 3μ − x = 0, become real from complex. In terms
of our visual picture, we have a growth factor which is large enough for 12to fall outside
the “trapping” region. The situation is very similar to what happened at μ = 1 when we
had the introduction of a “new” fixed point in I by F1(I2) ∩ I2 6= �, only this time the
set of this intersection does not contain a post-image of 12, F1(
12) /∈ F1(I2) ∩ I2, so this set
contains some other point or more correctly points. Since now it is not the maximal point
that gets captured it means that the intersection can not be collapsed to a single point,
but there will always be at least two points, marking the boundaries of this intersection
interval, and since at least both these points are “trapped” we have two new fixed points,
for each trapping region. Once we consider that we are dealing with F 3μ , there are three
“trapping” regions, so a total of six new fixed points, along with the two primordial,
{0, 1 − 1μ}, eight in total agreeing with the fact that F 3μ is an eighth degree polynomial
15 .
The nature of these points is one of them attracting and the other repelling, since we are
talking about an interval being “trapped” the points that actually exist between the two
fixed points are also “trapped” by eventually collapsing to the attracting fixed point. So
the overall situation in a trapping region is “identical” to the situation we had in I for
1 < μ, see inset (c) of Figure 1-7. Therefore it should be of no surprise that everything
that we have talked about up to now for I happens within each “trapping” region over and
over again as we increase μ even further; in some sense this is the reason why period three
implies chaos and not simply the fact that it allows all other periodicities; even though one
argument implies the other.
14 Derivation of this value can be found in [48–50].
15 In fact since Fμ is a second order polynomial any Fkμ will be a polynomial of order 2
k,so solutions to any case of F kμ − x = 0, will have an even number of real roots.
34
Finally μ3 < μ ≤ 4, with μ = 4 the most important part, given on Figure 1-8. In
the region between μ3 and 4 everything we have seen repeats infinite number of times.
In some subset of I or other, we have the creation of new other fixed points that turn
from stable to unstable giving birth to more and more fixed points in the same way, all
the way to the creation of the “period three orbit”, for the subset in question, when, of
course, we start over again! At μ = 4 we have the final brick on the wall, F4(12) = 1!16
We have seen the consequences of this before, at μ2,12is mapped to 0, the other fixed
point, and as before, more importantly 1 ≤ μ4the pre-images of 0 form a Cantor set; and
after this long journey we finally have F : I → I, F is chaotic on the whole interval I.
We have seen all that has been created as we increased μ, and everything is still there, so
we will not repeat ourselves. Instead, and in conclusion to this section, we will provide
a holistic picture of the points in I. We have a set of points which are fixed, of some
period or other, this set is dense in I, there is a difficult issue here, we can exclude from
this set the fixed points of F 2∞
μ17 , we still have the fixed points of all periodicities, even
infinity, but we are only considering the enumerable number periodicities, and this set is
still dense in I. The set of fixed points of F 2∞
μ we can now put on their own and it is in
itself a Cantor set, and of course dense in I, but even more, uncountable. We are not done
though, there are still “many” points left and specifically the pre-images of all fixed points
of enumerable periodicity, which as we have seen for each fixed point form a Cantor set on
their own. Hence we can consider the following two sets, the fixed points of F 2∞
μ (and their
pre-images in some sense), and the set of all other fixed points and their pre-images; both
sets are an infinite collection of Cantor sets, both dense in I, both uncountable. For its
16 Subsequently we shall drop the subscript on F4 and refer to the function of thisparameter simply as F .
17 Note that μ here is not the Feigenbaum point value of μ∞, instead we are consideringall points of all F 2
∞-functions, even the ones created after period three restarts the
process.
35
immense importance, our final point in this section is to emphasise the following, we have
seen how a function on its own is not chaotic, as a set on its own is not chaotic, it is the
combination of a many-one function on a specific set, under specific parameters that create
chaos.
Figure 1-8. Plot of the function F : I→ I. Also marked are the intervals I1, I3 and I2, thelast broken up into I2L and I2R , which are the two intervals that are mappedcompletely onto each other.
36
Table 1-1. Summary of the transitions in behaviour of the different intervals described inSection 1.2. Each arrow represents a single application of Fμ. x is used toindicate that more than one point is trapped in the interval, while y is used toindicate that a single point is trapped (typically an endpoint of the interval).
μ I = I1 ∪ I2 ∪ I3 I = J1 ∪ J3 I = J1 ∪ J2 ∪ J3I = (J1�J5) ∪J5∪J6∪(J3�J6)
0→ 1xI1←I2↑I3
1→ 2y xI1→I2↑↗I3
∗
x xJ1→J2↑J3
2→ 3y xI1→I2↑↗I3
x xJ1� J3
J2 = J1 ∩ J3
y yJ1→ J2↑↗↙J3
y yJ5�J6
3→μ∞y xI1→ I2↑↗↙I3
x xJ1� J3
J2 = J1 ∩ J3
y yJ1→ J2↑↗↙J3
y yJ5�J6
μ∞→μ2y xI1→ I2↑↗↙I3
x xJ1� J3
J2 = J1 ∩ J3
y yJ1→ J2↑↗↙J3
y yJ5�J6
μ2→μ3
y xI1→ I2↑↗↙I3
x xJ1� J3
J2 = J1 ∩ J3
y yJ1→ J2↑↗↙J3
y yJ5� J6↑↙J7 xJ5=J5∪J7
μ3→4
y yI1→ I2↑↗↙I3
x xJ1� J3
J2 = J1 ∩ J3
y yJ1→ J2↑↗↙J3
y yJ5� J6↑↙yJ7 xJ5=J5∪J7
∗The transition shown actually occurs at μ = 113.
37
1.2.2 The Tent Map, Topological Conjugacy and Universality
This section will briefly introduce the Tent map, which along with the Logistic map
are the two main systems we use for chaos based algorithm development. The Tent map
is a piecewise linear map that also can exhibit chaos. Instead of presenting in detail the
chaos in the Tent map, as we have done in the previous section for the Logistic map, we
will introduce the concept of Topological Conjugacy, which relates one map to the other,
guaranteeing the properties and behaviours in one map to exist in the other. In closing the
section we will move further than Topological Conjugacy, into the promised property of
chaos known as Universality. Universality plays a very important role in computation with
chaotic systems since it releases us from confinement to any single physical realization; as
long as a system is chaotic the algorithms we develop can be implemented by the system,
regardless of whether the nature of the system is electrical, optical, chemical, or even a
realization in some other physical realm.
The Tent map. Just as the Logistic map, the Tent map is discrete time, but instead
of a “continuous” polynomial, it is a piecewise linear “discontinuous” mapping of the form:
T n+1μ (xn) = xn+1 =
μ ∙ xn, for xn <12,
μ ∙ (1− xn), for 12 ≤ xn,
(1–3)
where x ∈ [0, 1](I) , 0 < n, and for our purposes we set μ = 218 . In this domain and
μ value, the Tent map, except for the curvature from the polynomial terms, looks and
behaves like the Logistic map, we can even mark sub-intervals, I1 = [0,13], I2 = [
13, 23]
and I3 = [23, 1] with similar behaviour as the Ii of F , see Figure 1-9. In addition, the
bifurcation diagram for the Tent map is practically the same as for the Logistic map, see
18 As we have done for the Logistic map, for the Tent map in the case of n → n + 1and μ = 2 we will not use the super- and sub-scripts and refer to the function simply as T ,unless emphasis is required.
38
Figure 1-10, even though it looks more “compressed” and the bifurcations for the period
doubling are closely packed, basically everything that we showed for the Logistic map
applies to the Tent map as well. We can actually show this “identity” mathematically
through the Topological conjugacy of the two maps.
Figure 1-9. Plot of the function T : I→ I. From the graph we can see howT : I1 → I1 ∪ I2, T : I3 → I1 ∪ I2, and T : I2L → I3 and T : I2R → I3; and ofcourse how 1
2is a pre-image of 0.
Topological Conjugacy. Topological conjugacy has its roots in set theory,
consider two distinct sets, X and Y, and for each set there is a “relation”, say P and
Q respectively, between its individual elements. For each set consider the “larger” set of
the union of the elements that are in the domain and converse domain of the “relation”,
39
Figure 1-10. Bifurcation diagram for the Tent map. Eventual evolution of an arbitraryinitial state x0. Inset: Magnification of the region μ ∈ [1, 1.1] andx ∈ [0.495, 0.5], showing more clearly the period-doubling.
the “field” of the set, for example given x ∈ X : P(x) = x, then x and x are in the “field”
of X and P , analogously for Y and Q. If there is a “relation” S, which is one-to-one and
has as domain the field {X,P : X} and converse domain {Y,Q : Y}, then the “relations”
P and Q are “similar”, that means P and Q have identical properties, in other words
their effects on their respective sets are indistinguishable. When the sets considered are
topological spaces, the case of the Logistic and Tent map on I, and the “relations” actual
functions, then the functions are said to be Topologically conjugate to each other.
To make this concept more concrete, and show as promised that there is no difference
between the Logistic and Tent maps, consider the two functions F : I → I and T : I → I;
the fact that I acts as the domain and converse domain for both functions should not
worry us for two reasons, first of the sets (or spaces) considered are actually arbitrary
40
Table 1-2. Topological conjugacy. Given that the Tent map takes every x ∈ I on a x ∈ I,and the Logistic map takes every y ∈ I on a y ∈ I, they are Topologicallyconjugate when a function, G, exists that takes every x ∈ I on a y ∈ I andevery x ∈ I on a y ∈ I; so that G ◦ T (x) = F ◦G(x) = y. SpecificallyG(x) = sin2
(π2∙ x).
IT−→ I
(x) (x)
G ↓ ↓G
IF−→ I
(y) (y)
and in reality unchanged19 it is the “relations” (or functions) that we will actually
manipulate, and second, the actual set of points in I for each function is different,
both functions are applied on the same “interval”, which contains the same points,
but the points themselves are different for each function, except {0, 12, 1}. So following
Table 1-2 we have the following relationships: T : I → I, F : I → I, G : I → I
and of course for any x, y, x, y ∈ I, T (x) = x, F (y) = y, G(x) = y and G(x) = y,
which imply G ◦ T (x) = G(x) = y or F ◦ G(x) = F (y) = y. For the Logistic
and Tent map the conjugating function is given by G(x) = sin2(π2∙ x), specifically:
y1 = F ◦G(x0)
y1 = 4 ∙ y0 ∙ (1− y0)
y1 = 4 ∙ sin2(π2∙ x0)∙((1− sin2
(π2∙ x0))
y1 = 4 ∙ sin2(π2∙ x0)∙ cos2
(π2∙ x0)
y1 = sin2 (π ∙ x0)
y1 = G ◦ T (x0)
x0 <12
12≤ x0
x1 = 2 ∙ x0 x1 = 2 ∙ (1− x0)
y1 = sin2(π2∙ x1)y1 = sin
2(π − π ∙ x0)
y1 = sin2 (π ∙ x0) y1 = sin
2(π ∙ x0)
19 You can actually consider each domain and converse domain as fixed points in a setand the “relations” as just a “virtual” link between these points in each set, not actuallyaffecting the points.
41
Of course by induction, if not by simply considering function composition, this result
applies for any, and at any, n, F n ◦G = G ◦ T n, see Figure 1-11 for an illustration.
When we consider the “spatial” correlation between functions given by Topological
conjugacy, in addition to the “temporal” correlation between a function and its future
iterates (e.g. F and F n) given by renormalization, we have a universal correlation between
any “functions” that exhibit similar behaviours.
Figure 1-11. Topological conjugacy between evolved states, up to n = 5. The dashed linefollows the evolution of G(x0) under the Logistic map (F ), while thedash-dotted line follows the evolution of T n(x0). The action of G(x) both as
the first step (x0G→y0) and last step (x5
G→y5) is shown along the triangular
arrows. (f(x) is {F (parabola), T (piecewise linear), G (sigmoid), I(identity)}).
42
Universality. The theory of Universality is, as the name implies, extremely wide and
intricate. It correlates many branches of Physics, from the theory of critical phenomena
to Hamiltonian mechanics, and many branches of Mathematics, from statistics to vector
spaces. The aim of this section is to present some important qualitative results, both
as a hint of proof of Universality, and its main consequence, the independence between
“behaviours” and actual physical system or mathematical interpretation.
The most formal origins of Universality come from Mitchell Feigenbaum [29, 51, 52],
circa 1975 when he discovered the universal constant δ = 4.669.... Originally δ was
observed within a system as the “rate” of onset of period doubling as:
μ2n+1 − μ2nμ2n+2 − μ2n+1
≡ δ, as n→∞, (1–4)
where μ2k is the μ value of onset of the 2kth cycle, in terms of Figure 1-12 it is the limit
of the sequence b1b2, b2b3, b3b4, ... δ. Almost immediately though with this initial observation,
came both the theoretical and experimental confirmation (see Table 1-3) that extended the
existence of δ from within a single system to every system that undergoes period doubling.
More importantly though, and the actual basis of Universality, out of the theoretical
treatment emerges a “convergent function” that encapsulates all such systems. A very
rudimentary approach to describe this function is the following, consider any function that
relates “points” in a set in the manner we have been concerned with in this chapter.
Further, consider each such function as itself a “point” in a set of functions, these
“function-points” converge to a single “function-point”, in some sense just as the bibi+1
converge to δ. We have seen in the previous section how x, n, μ of F , can all be varied in
some way to create a combination of F and an interval from which chaos emerges, now we
see that the functional form of F itself can also vary.
1.2.3 Threshold Control and Excess Overflow Propagation
In this short section we will introduce two processes which have extended the
breadth of influence of chaos theory, and are widely used in algorithms of chaos based
43
Figure 1-12. Logistic map bifurcation diagram for some values within 3 < μ < μ∞. “//”on the axis indicate discontinuity in the displayed points, b1 and b2 areactually longer than shown. For the Logistic map the sequence of the ratiosb1b2, b2b3, b3b4... converges to δ. In other systems the bi can be different from the
Logistic map, but the convergent value of the ratios is the same. Inset:Figure 1-1.
Table 1-3. Experimental measurements of Feigenbaum‘s constant (δ) in different systemsbased on their period doubling. (Adapted from [53].)
SystemsHydrodynamic Electronic Optic AcousticWater Mercury Diode Transistor Laser Helium
Observed δ 4.3± 0.8 4.4± 0.1 4.3± 0.1 4.7± 0.3 4.3± 0.3 4.8± 0.6
computation. We will not go in details for either process, as both are now quite extensive
in their own right.
Control and synchronisation of chaotic systems [13–15, 54–64] have been studied
for almost 20 years now with numerous results; we simply demonstrate the threshold
control mechanism, as already mentioned in relation to the second part of the Sharkovsky
theorem. The other process we introduce, the excess overflow propagation [65–68], also
44
has a long history and connections to a wide variety of other fields as critical phenomena,
phase transitions, cooperative behaviours, and more.
Threshold Control Mechanism. Following Sharkovsky’s theorem and specifically
the second part, we know that a map exists for every periodicity and more, that once
periodicity q is established periodicity p is guaranteed, where p C q. Therefore if we start
with a map which has period 3, the Logistic map F4 for example, this same map can be
of some other “maximal”20 period simply by confining the domain of the map, even more
this period will seem attractive21 . Basically instead of looking for a whole new different
map of “maximal” periodicity q, we take F4 : I → I and consider the map which is the
part of F4 : J → J, where J = (0, x∗] and x∗ is the maximal point value of the sequence of
points that are the desired orbit, hence the name of the method “threshold control”. The
mechanism can be defined as:
F (x) =
F (x), for F (x) ≤ x∗,
x∗, for x∗ < F (x),
(1–5)
the map, F (x), is evolved as usual unless it exceeds x∗, in which case it is limited back to
x∗ and normal evolution continues, F (x) is the resulting part of F (x) which is the map
of desired periodicity, see Figure 1-13 for a specific example of period 4 selection, and a
demonstration of the process.
Specifically for the logistic map, the interval ( 34, 1) provides thresholds for periodic
orbits of all orders greater than 2, for example 34< x∗ ≤ 0.905 forces the map in a cycle of
order 2, 0.9505 ≤ x∗ ≤ 0.9698 of order 3, 0.905 < x∗ < 0.925 of order 4 and so forth, see
Figure 1-14 for a more extensive list. Obviously thresholds from [0, 34] produce fixed points
(period 1).
20 Maximal in the sense of the Sharkovsky ordering.
21 Only because all other periods are repelling.
45
Figure 1-13. Threshold Control Mechanism. F4 the dotted parabola, is confined on theinterval J = (0, x∗] and as a result the map F4 is produced, the solidparabola. In this case x∗ = 0.915. Empty circles, #, mark three differentinitial conditions and their paths, in dotted lines, showing how they are“pushed” onto the period 4 sequence of points x1, x2, x3, x4, marked with fullcircles, .
Excess Overflow Propagation. Up to now we have been considering, in some
form or other, just a single chaotic system, and even thought the potential of even a
single chaotic system is immense, two is always more than one. Since we are considering
more than one system we need to define some way in which the two systems to interact.
Currently the method used in chaotic computation is the excess overflow propagation
method. We define a “monitoring value”, ∗x, for the state of the “emitting” system, f(1x);
46
Figure 1-14. Threshold values for confining the logistic map on orbits of periodicity 2 to50.
once this “monitoring value” is crossed over by the actual state of the system, f(1x) >∗ x,
the difference between the actual state of the system and the “monitoring value”, the
excess overflow (f(1x) −∗ x = E), is propagated to the receiving system(s) f(2x + E).
Note that the “emitting” system is not affected in any way, unlike with the threshold
control method where we confine the state of the system, also at the “receiving” system
the incorporation of the excess overflow can happen both before and after an iteration of
the system f(2x) + E. The method is very similar to the threshold control method, but
independent, in the sense that the “monitoring value” can act as a threshold control, but
is not necessary, the two values can be different, providing us more flexibility.
1.3 Conclusion
We have presented, as briefly as possible, parts of Chaos Theory that are the basis
for Chaotic Computation. Even though this exposition was made in the spirit of Set
Theory and Logic, the exact connections were not actually laid out since our forthcoming
47
presentation of algorithms, developed for Chaotic Computation, do not actually reach such
lengths. The possibility is there through, and we plan to explore further this view in our
future work.
48
CHAPTER 2INTRODUCTION TO CHAOTIC COMPUTATION
Chaotic computation is the exploitation of the richness in behaviours of chaotic
systems to perform computational tasks. This chapter deals with different manipulations
of state variables that lead to selection of a specific behaviour, without relinquishing
access to other behaviours, demonstrating the inherent versatility. Natural parallelism
emerges both through cooperative processes of independent chaotic systems and through
exploitation of multi-dimensional systems. A very good abstraction of chaotic computation
is the translation of functions and data operators from the software realm onto the
hardware; direct implementation of the objective task in hardware. Here specifically we
almost exclusively present, the theoretical development of algorithms and methods of
chaotic computaion and use the Logistic map to illustrate specific examples. For actual
physical realizations, and verifications, of these results we direct the reader to results of
electronic implementations, using Chua‘s circuit [69], and Logistic map circuit [70, 71],
and to simulation results [26] of a IR NH3 laser using the Lorentz system [72, 73]. In
addition we can direct the reader to the very recent physical realization results for chaotic
computation using synchronization [74] and stochastic resonance [75].
2.1 Number Encoding
The primary requirement for any computational system is representation of data.
There need to be methods through which data can be recognized, stored, and reproduced,
obviously only these three processes do not make a very capable computer. The absolute
universal language is mathematics and so numbers is the most generic form of data
representation. Therefore we begin with the different methods available in chaotic
computation for number representation.
2.1.1 Excess Overflow as a Number
This is the simplest method for encoding numbers using a chaotic computer and
follows from the excess overflow propagation mechanism. For any given threshold (x∗), the
49
amount by which the system variable monitored exceeds the threshold, E = fn(x∗) − x∗,
for some nth iteration such that fn(x∗) > x∗, is named “excess overflow” and is used to
represent an integer. More specifically, we utilize an interval, K1, for which the dynamical
system that drives our chaotic computer contains fixed points for a single iteration, n = 1,
under threshold control, i.e. f(x∗) > x∗, ∀x ∈ K1. Given a requirement as of representing
the set of integer numbers {1, N}, we can find from the interval K1 the x∗ which produces
the largest value of excess Emax and “equate” this with the largest integer we wish to
encode, N . As a result, we can define 1 ≡ Emax/N ≡ δ, where δ is called “unit overflow”
and every integer in the set {1, N} can be represented by an excess overflow of z ∙ δ, where
z ∈ {1, N}. Obviously the integer 0 is represented by zero amount of excess, obtained by
setting the system at a natural fixed point, i.e. fn(x) = x, ∀n.
As an illustrative example we use the Logistic map, Equation 1–1. For the Logistic
map the interval [0, 0.75] produces fixed points under threshold control, note that
0 and 0.75 are the “natural” fixed points of this system and either can be used to
represent the integer 0. By taking the derivative of F (x∗) − x∗ we find the threshold
that produces the maximum excess overflow at x∗ = 3/8, with an emitted excess of
Emax = 9/16, see Figure 2-1. Given the set of integers [0, 100] we can encode them
using threshold controlled logistic map elements following the steps described above, i.e.
100 ≡ Emax ⇒ δ = (9/16)/100 and x∗ =−3+√32−4∙(−4)∙(−z∙δ)
2∙(−4) , ∀z ∈ [0, 100], see Figure 2-2.
The extensive versatility of this method will be demonstrated in following sections.
We will show implementations of this method not only for other number representation
methods, but also in algorithms for decimal and binary arithmetic, as well as for boolean
logic operations.
2.1.2 Periodic Orbits for Number Representation
The most immediate extension to the excess overflow encoding method is to consider
the behaviour of the dynamical system outside the region already being utilized, i.e.
x /∈ K1. We can utilize the effect of the threshold control mechanism on this interval, K2,
50
Figure 2-1. Emitted excess by thresholding the logistic map in the interval [0, 0.75]. Thethreshold x∗ = 3/8 produces the largest excess of Emax = 9/16.
to stabilize the system to a periodic orbit. Following Sharkovsky’s theorem, see Theorem
1.1 on page 30, since we have a period three orbit, orbits with periods of all other integer
values, and more, are guaranteed to exist. Therefore it is an obvious extension to utilize
the appropriate orbit to represent its respective integer number. More specifically, for each
n > 1 we find a x∗ ∈ K2, for which fn(x∗) > x∗ and fm(x∗) < x∗, ∀m < n. So for every
integer we have a x∗ that forces the system to emit excess at periodicity n ∈ {2,∞}, in
some sense the converse of what is shown in Figure 1-14. Clearly any of the thresholds
for n = 1 fixed points found in the previous method can be utilized for representation of
integer 1 and as before a natural fixed point can be used for integer 0.
We will show implementations of this method for integer multiplication, calculating
the least common multiple of any set of integers, and in conjunction with the excess
51
Figure 2-2. Encoding the set of integers {0, 1, ..., 100}. We use 100 ≡ Emax = 9/16 toobtain each excess z ∙ δ and the threshold x∗ that produces it.
overflow method in a third method for representing numbers more in the spirit of binary
representation.
2.1.3 Representation of Numbers in Binary
In this final example of number encoding methods we will combine the previous two
methods, we use both periodic orbits and excess overflow. We can represent a number
in binary format by coupling together elements which have their periodicity determined
by their position away from the radix point as 2#digits−position. The farther away from
the radix point an element is the shorter periodicity we give it, with the bit farthest
away being on period one. The elements are joined together serially, so that the “generic
overflow” generated by each element cascades through the array until it reaches the open
end at the most significant digit, where we have the readout, see Figure 2-3. To be more
specific a binary number aN . . . a1 will be represented by N elements. Each element, j,
as in the encoding based on periodicity, will be set at a threshold so that to produce a
52
“generic overflow” at periodicity of 2N−j, if the binary digit aj = 1, and at a threshold
of 0, if aj = 0, thus generating no overflow. The resulting array is updated 2N−1 times
resulting in a multiple of the unit overflow∑Nj=1 aj2
j−1 at the readout.
An illustrative example using the Logistic map is shown in Figure 2-3, where we are
using four elements, N = 4, to represent the binary number 1111. Following the example,
a4, the farthest element from the radix point, is set to a threshold that emits excess on
every update, i.e. periodicity one (x∗ ∈ (0, 0.75)); a3 is given a threshold x∗ ∈ (0.75, 0.905),
to produce excess on every second update; a2 from x∗ ∈ (0.905, 0.925) for an excess every
fourth update; and a1 from x∗ ∈ (0.925, 0.926) for an excess every eighth update. As a
result we will collect emitted excess 8 + 4 + 2 + 1 = 15, respectively from each element, as
is desired for encoding 1111 in binary.
This method allows us to encode large numbers without the need of thresholds that
produce proportionally large periodicities, or of thresholds that produce excess in steps
of very small δ. This is a good example of the flexibility of chaotic computation. In the
specific example we “sacrificed” efficiency, in the number of elements per encoded integer,
to allow short periodicities to encode larger numbers. Obviously, the method can be easily
modified to encode numbers in any other base representation.
2.2 Arithmetic Operations
We have established not one, but three methods for representing numbers, we should
see them in action. The most obvious starting point is the simplest arithmetic operation:
addition, which we extend into multiplication, which we extend to the least common
multiple problem.
2.2.1 Decimal Addition
There are multiple chaotic computation algorithms for decimal addition. We will
present three such algorithms, each one building on its predecessor, from serial addition to
parallel to branching.
53
Figure 2-3. Number encoding in binary format. The excess from each element cascades tothe one “above” it until the readout is reached. The element closest to thereadout, a4, emits on every update, and as we move “down” the chain theelements emit in increasing powers of 2. The overall result is after 8 updates,we have 8 units of excess from a4, 4 from a3, 2 from a2 and 1 from a1, givingus a total of 15 units. Any binary number can be represented with thismethod. (Adapted from [26].)
Serial. The most straightforward algorithm for addition utilizes the excess overflow
encoding method in a serial manner. In reality it is a natural extension of the encoding
method, since each number is encoded as a proportional excess we can chain-link the
elements and cascade the emitted excess from each element into its neighbour all the
way to the end of the chain. Each excess “builds up” on the one after it, all “naturally”
summing up at the edge of the chain, see Figure 2-4. As we have seen in Section 2.1.1
the “unit overflow” (δ) works as the proportionality constant between integers and
emitted excess, therefore the addition of integers i, j, k, l is simply replicated by
54
iδ + jδ + kδ + lδ = (i + j + k + l) ∙ δ, using the avalanching of the excesses. For
this algorithm the computational time is dependent on the adaptive process and the
number of terms in the sum. Specifically after a single chaotic update of all the elements,
it takes as many adaptation steps as there are terms in the sum to complete the operation.
Figure 2-4. Serial Addition. We recruit as many elements as terms in a given sum. Eachelement is assigned a number from the sum, encoded using the excess overflowmethod. The elements are coupled together in a chain such that the excess canflow down the chain. The result is at the open end we collect the sum of allthe emitted excesses as a multiple of δ. (Adapted from [25].)
Parallel and Branching. Following from the previous example, consider the sum
of four terms, i, j, k, l using the serial addition algorithm. If we take a closer look at
the dynamics of the “last” element in the chain (in this case the one encoding l), we see
that this element will receive the excess of all preceding elements “simultaneously”. To
visualize this consider the “local” dynamics of three elements, once the chaotic update
is complete. The excess of the first element is avalanched to the second, where it “builds
up” with the local excess at the second element, so the third element will receive the
combined excess of the two previous to it as a single excess. This can also be achieved by
introducing the two excesses independently of each other, but simultaneously, i.e. arrive
55
at the same time and “build up” on each other locally at the third element instead of at
the second. Of course this can be extended to any number of elements preceding the “last”
element in the chain. Turning back to our example of adding i, j, k, l, the topology of the
connectivity instead of being a chain is now a tree diagram, see Figure 2-5. By turning
the chain into a tree we have collapsed the serial addition algorithm to a two step serial
addition, regardless of the number of terms in the sum. The first step is to sum all, but
one of the terms in parallel and then serially combine the parallel sum to the remaining
term, before reading the result at the open end. We need to note that we can not fully
parallelize the operation by connecting all terms directly to the open end, since we are not
attributing any dynamical properties to the open end, i.e. if the open end had the ability
to correctly “build up” excesses it would be identical to all the other dynamical systems
used in the sum, making it the “last” element1 .
Obviously for a sum of N terms the shortest computational time for the operation is
to connect N − 1 terms in parallel to the “last” term and perform the operation in two
steps, this would be analogous to increasing the number of “branches” in the network. In
case though of connectivity and/or spatial restrictions we could also extend the algorithm
by increasing the number of “trunks” in the network, as is shown in Figure 2-6; note that
now our computational time is dependant on the number of “trunks”. This is another case
of chaotic computation exhibiting its flexibility, we can “sacrifice” temporal performance
to satisfy spatial constraints.
2.2.2 Binary Addition
Extending the above addition algorithms for the binary number encoding method
is straightforward. The serial addition of binary numbers is realized by connecting the
“end bit” of one number with the “end bit” of the following number in a chain manner
all the way to the last number and then to the open end, see Figure 2-7. Similarly for
1 This could actually be overcome by inserting to any sum, 0 as the last term.
56
Figure 2-5. Decimal parallel addition. The excess from the elements encoding i, j, k issimultaneously propagated to the element encoding l on a first avalanchingstep, and on the second avalanching step the collective sum is read at the openedge. (Adapted from [26].)
Figure 2-6. The branching algorithm can be extended to a larger treelike structure. Thecomputational time in this case is proportional to the number of branches inthe longest path that terminates at the “last” element. The above network willsum 15 terms in four avalanching steps. (Adapted from [26].)
57
implementing the parallel addition method for binary numbers we connect the “end
bit” of all terms in the sum, but one, to the “end bit” of the single term chosen to act
as the collection hub for all the excesses, before sending the result to the open end, see
Figure 2-8.
Figure 2-7. Schematic representation of the serial addition method for binary numbers.The numbers 7, 5, 2, 1 are encoded as explained above in Section 2.1.2 and theelements are serially connected. The excess overflow builds up as it movestrough the network and 15 units of excess are collected at the OUTPUT.(Adapted from [26].)
58
Figure 2-8. Schematic representation of the parallel addition method for binary numbers.In this case a branching topology is used for the network, where one of thesystems acts as a collection hub for simultaneous build up of excess. (Adaptedfrom [26].)
2.2.3 Decimal Multiplication and Least Common Multiple
We can extend any addition method to perform multiplication in the usual way,
the product of two numbers m × n is a sum of n terms of the number m (and of course
visa versa m terms of the number n). With our two above addition methods we have
two obvious ways of implementing multiplication the serial and parallel summation of
the terms. Furthermore using the excess overflow encoding of numbers we have a third
method; we can use a single dynamical system that emits the appropriate excess on every
update, specifically the amount that represents one of the numbers, m ∙ δ for instance, and
we update the system n times collecting a total of excess equal to (m× n) ∙ δ, the product
59
of the multiplication. This third method utilizes time as a computational quantity, and
leads to a fourth multiplication method, which we will also use as a stepping stone for the
method of least common multiple of many numbers.
Following the periodic orbit encoding method, we can represent each number, of
a two number product, using for each number a chaotic element set to emit excess at
the appropriate periodicity. Therefore given a product m × n we utilize two dynamical
systems, one set to emit every m updates and the other to emit every n updates, the
product of the two numbers is given by the number of updates on which the two elements
emit simultaneously; more specifically the element emitting every m updates will have its
nth emission on the (m × n)th update and the same applies for the other element. The
simple extension of this method to a larger number of terms results in an algorithm for the
least common multiple of all the terms in consideration2 .
2.3 Binary Operations
The power of conventional computers lies in their ability to perform boolean algebra.
Even so their building blocks are restricted by manufacture to one of the two fundamental
gates, NOR or NAND, which with suitable combinations, of either, the other logic gates
can be reproduced, for example AND(X,Y)≡ NOR(NOR(X,X),NOR(Y,Y)). This clearly
is not the most efficient method for boolean algebra; a much more efficient alternative
is for each gate to require only one building block, conversely each building block to be
able to perform all gates. We show how a single chaotic element can represent each of all
logic gates through simple state manipulations, removing the need for combining elements
[32, 76]. Furthermore we show how multi-dimensionality can lead to natural parallelism,
and we go even further, exiting the capabilities of conventional algorithms and addressing
a problem designed for quantum computation.
2 The case of three numbers in a product is handled serially, find the product of two ofthe terms, and multiply it by the third term.
60
Figure 2-9. Schematic representation of the method for computing the Least CommonMultiple of four numbers. Each element emits excess to the collecting elementat the appropriate periodicity for its encoded number. The result at theOUTPUT is excess of the collecting element with magnitude equal to thenumber of term elements that emitted on the current update. The number ofupdates that causes the collecting element to emit excess equal to the numberof terms (i.e. all term elements emitting simultaneously) is the Least CommonMultiple of the terms. (Adapted from [26].)
2.3.1 Logic Gates
As we have shown an important characteristic of chaotic computation is the
versatility we have in implementing the same algorithm with different methods. This
of course extends to implementations for representation of logic gates, i.e. there are
multiple ways we can achieve this representation. In this section we focus on the most
61
straightforward method developed and in order to present the method more clearly we
specify it to the logistic map3 , Equation 1–1.
The method we use consists of three steps: (a) initialization, (b) chaotic update, and
(c) threshold control and excess overflow. Compared to the methods presented in previous
sections, the new concept for this method is initialization; i.e. the setting of the initial
state of the system, (X0) just before the first chaotic update, based on specific rules. This
initial condition of an element is used to define which logic operation it performs and on
what set of inputs. Specifically, we initialize a logistic map element by setting its initial
value x0 according to:
• x0 = xprog + xI1 + xI2 , for gates that operate on two inputs,
• x0 = xprog + xI , for gates that operate on one input,
xprog can be thought of as “programming” the gate and xIi as the input values. For an
input of logical 1, xIi = δ, and for an input of logical 0, xIi = 0. As before a chaotic
update implies the application of the logistic map: x0 → F (x0). The control and overflow
mechanism remains the same as well: E = 0 if F (x0) ≤ x∗, and E = F (x0) − x∗
if F (x0) > x∗. Where x∗ is the threshold for the element and E the excess overflow
generated. Here in the context of binary algebra where the set of integers contains only 0
and 1, E and δ are actually equivalent4 .
Turning to the specific logic gates to be implemented, the following Table 2-1,
summarizes the input - output relationships we are to represent.
The task is to identify initial conditions, xprog + xIi , and threshold values, x∗, for
which a chaotic update will result in F (x0) ≤ x∗ for where the outputs in the above table
3 Universality of chaotic systems allows us to assume demonstrations on the logisticmap can be carried over to any other chaotic system.
4 Equivalence of inputs and outputs is actually a “soft” requirement.
62
Table 2-1. Truth-table for the logic operations AND, OR, XOR, NOR, NAND, NOT, andthe identity operation (WIRE).
I1 I2 AND OR XOR NOR NAND I NOT WIRE0 0 0 0 0 1 1 0 1 00 1 0 1 1 0 1 1 0 11 0 0 1 1 0 11 1 1 1 0 0 0
are 0, and F (x0) > x∗ where the output is 1. As a specific example, for the OR gate we
have the following requirements:
1. I1 = I2 = 0, which implies xI1 = xI2 = 0, i.e. x0 = x
prog.The required output is 0,which implies F (xprog) ≤ x∗.
2. I1 = 0 and I2 = 1, which implies xI1 = 0 and xI2 = δ, i.e. x0 = x
prog+δ. The requiredoutput is 1, which implies F (xprog + δ) − x∗ = δ. This requirement is symmetric toI1 = 1 and I2 = 0, so the conditions for satisfying both requirements are identical.
3. I1 = I2 = 1, which implies xI1 = xI2 = δ, i.e. x0 = xprog + 2 ∙ δ. The required output
is 1, which implies F (xprog + 2δ)− x∗ = δ.
All of the above requirements need to be satisfied by the same values for xprog and x∗,
such that all three conditions hold true regardless of inputs. In a similar manner we can
provide required conditions for a chaotic element to represent every gate, see Table 2-2.
Table 2-2. Necessary and sufficient conditions for a chaotic element to satisfy the logicoperations AND, OR, XOR, NOR, NAND, NOT, and the identity operation(WIRE).
InputxI1 + xI2 AND OR XOR0 F (xprog) ≤ x∗ F (xprog) ≤ x∗ F (xprog) ≤ x∗
δ F (xprog + δ) ≤ x∗ F (xprog + δ)− x∗ ' δ F (xprog + δ)− x∗ ' δ2δ F (xprog + 2δ)− x∗ ' δ F (xprog + 2δ)− x∗ ' δ F (xprog + 2δ) ≤ x∗
InputxI1 + xI2 NOR NAND0 F (xprog)− x∗ ' δ F (xprog)− x∗ ' δδ F (xprog + δ) ≤ x∗ F (xprog + δ)− x∗ ' δ2δ F (xprog + 2δ) ≤ x∗ F (xprog + 2δ) ≤ x∗
InputxI NOT WIRE0 F (xprog)− x∗ ' δ F (xprog) ≤ x∗
δ F (xprog + δ) ≤ x∗ F (xprog + δ)− x∗ ' δ
63
Values that simultaneously satisfy the above conditions are easily found. Specifically
by choosing δ = 0.25, the OR gate can be realized by choosing values for xprog = 1/8 and
for x∗ = 11/16:
1. F (xprog) = F (1/8) = 7/16 ≤ x∗(= 11/16),
2. F (xprog + δ) = F (3/8)− 11/16 = 15/16− 11/16 = 1/4(= δ),
3. F (xprog + 2δ) = F (5/8)− 11/16 = 15/16− 11/16 = 1/4(= δ).
In fact values that satisfy the conditions for all the gates and δ = 0.25, have been
identified and are summarized in Table 2-3.
Table 2-3. Initial values, xprog, and threshold values, x∗, required to implement the logicgates AND, OR, XOR, NOR, NAND, NOT, and the identity operation(WIRE), with δ = 0.25.
Value AND OR XOR NOR NAND NOT WIRExprog 0 1/8 1/4 1/2 3/8 1/2 1/4x∗ 3/4 11/16 3/4 3/4 11/16 3/4 3/4
The fact that we have a method to implement all logic gates is not impressive, what is
impressive is the fact that we can switch from one gate to another from one computational
step to the next, and even more, switch very easily and “fast’,’ in a relative timescale.
This leads to the concept of on-the-fly hardware re-programming; an architecture based on
conventional computation, so all other components can be easily imported, but with the
efficiency chaotic computation offers.
2.3.2 Parallel Logic and the Half Adder
In this section we present what is probably the most important extension of
performing boolean logic with chaotic systems [32]. The same procedure we illustrated
above for the logistic map is implemented on the 2-dimensional neuron model [77]:
xn = (xn−1)2 ∙ exp(yn−1 − xn−1) + k (2–1)
yn = a ∙ yn−1 − b ∙ xn−1 + c (2–2)
64
where a = 0.89, b = 0.18, c = 0.28, k = 0.03; these parameter values keep the model
dynamics completely chaotic.
Two distinct cases are investigated:
1. The possibility of performing XOR gate and AND gate in parallel (Half Adder)
2. Performing two AND gates independently.
The first case specifically, is the application of the two gates on the same set of inputs,
each gate being performed in a different dimension. The second case involves operating the
two AND gates on different sets of inputs, again each dimension performing one operation.
As with the case of the logistic map, the first step is to define the necessary conditions
needed to satisfy the truth table of each case, specifically following the truth tables
(Tables 2-4, 2-5) we convert them to the conditional Tables 2-6, 2-7. Following the
process of the previous section we determine values for the “programming” state shift and
threshold value that satisfy these conditions, see Tables 2-8, 2-9.
Table 2-4. Truth table for XOR and AND logic gates on the same set of inputs. (Case 1)
I1 I2 XOR AND0 0 0 00 1 1 01 0 1 01 1 0 1
We should mention that each case is investigated independently, i.e. there is no
requirement that the number of iterations (n) is identical for both cases, or that the value
representing a logical 1 (δ) is the same.
We have already seen some type of parallel operations with chaotic computing
in Section 2.2.1 with the parallel addition, here though we see a clear demonstration
of parallelism. The dynamical system performs two completely different operations
simultaneously. In fact the next section builds further on the parallel capabilities of
chaotic computation, by tackling the complex Deutsch-Josza problem.
65
Table 2-5. Truth table for two AND gates operating on independent inputs. (Case 2)
I11 I12 I21 I22 AND(I11 , I12 ) AND(I
21 , I
22 )
0 0 0 0 0 00 0 0 1 0 00 0 1 0 0 00 0 1 1 0 10 1 0 0 0 00 1 0 1 0 00 1 1 0 0 00 1 1 1 0 11 0 0 0 0 01 0 0 1 0 01 0 1 0 0 01 0 1 1 0 11 1 0 0 1 01 1 0 1 1 01 1 1 0 1 01 1 1 1 1 1
Table 2-6. Required conditions to satisfied parallel implementation of the XOR and ANDgate. (Case 1)
Initial conditions XOR ANDxprog, yprog xn ≤ x∗ yn ≤ y∗
xprog + δ1, yprog + δ2 xn − x∗ ' δ1 yn ≤ y∗
xprog + 2 ∙ δ1, yprog + 2 ∙ δ2 xn ≤ x∗ yn − y∗ ' δ2
Table 2-7. Required conditions for implementing two AND gates on independent sets ofinputs. (Case 2)
Initial conditions AND(I11 , I12 ) AND(I
21 , I
22 )
xprog, yprog xn ≤ x∗ yn ≤ y∗
xprog, yprog + δ2 xn ≤ x∗ yn ≤ y∗
xprog, yprog + 2 ∙ δ2 xn ≤ x∗ yn − y∗ ' δ2xprog + δ1, y
prog xn ≤ x∗ yn ≤ y∗
xprog + δ1, yprog + δ2 xn ≤ x∗ yn ≤ y∗
xprog + δ1, yprog + 2 ∙ δ2 xn ≤ x∗ yn − y∗ ' δ2
xprog + 2 ∙ δ1, yprog xn − x∗ ' δ1 yn ≤ y∗
xprog + 2 ∙ δ1, yprog + δ2 xn − x∗ ' δ1 yn ≤ y∗
xprog + 2 ∙ δ1, yprog + 2 ∙ δ2 xn − x∗ ' δ1 yn − y∗ ' δ2
66
Table 2-8. Examples of initial values, xprog, yprog, and thresholds x∗, y∗, that satisfy theconditions presented in Table 2-6, yielding the parallel operation of XOR andAND gates. In this range δ ' 0.7, and the number of required iterations toyield the correct result is n = 10. (Case 1)
xprog yprog x∗ y∗
1.8 1.7 0.44 1.111.4 1.65 0.38 1.07
Table 2-9. Examples of initial values xprog, yprog, and thresholds , that satisfy theconditions presented in Table 2-7, yielding operation of two AND gates onindependent inputs. For these values, δ ' 0.0115 and the number of iterationsrequired n = 20; note that the high number of required iterations and the needfor more decimal precision are a direct consequence of the large number ofrequired conditions to be satisfied simultaneously. (Case 2)
xprog yprog x∗ y∗
0.5375 1.0835 0.231 1.454
An important difference from the previous section on single binary gates is that in
this method the number of iterations that are necessary for the conditions to be satisfied
is greater than 1, actually in Chapter 5 we report extensive progress in involving the time
dimension with chaotic computation.
2.3.3 The Deutsch-Jozsa Problem
The Deutsch-Jozsa problem and its solution algorithm [78] is a celebrated result of
quantum computing, since it is the first example of quantum computing outperforming
classical algorithms. In addition it has been the stepping stone for the other two
major results of quantum computing, Shor’s factoring algorithm [79] and Grover’s
search algorithm [80] (details of the chaotic computation search algorithm are given in
Chapter 3).
In this section we will demonstrate the chaotic computing algorithm for solving the
Deutsch-Jozsa problem, which is just as efficient as its quantum counterpart, provides
the answer in a single computational step, but in contrast is much more realizable and
provides a more apparent result.
67
The problem can be stated as follows: Given a binary domain, i.e. a discrete domain
of 2k states, and given an arbitrary binary function f : {0, 1}k → {0, 1}, i.e. a function
that maps every point in the binary domain to a 0 or a 1, determine whether the function
is constant, maps the whole domain on either 0 or 1 exclusively, or whether the function is
balanced, maps the whole domain in equal terms on 0 and 15 .
In other words, evaluate f for every given point6 , and count the number of resulting
0s and 1s for all points, which basically is the conventional approach in solving the
problem. As a result the conventional algorithm in the best case requires evaluating
the function for only 2 points, i.e. the function results in a 0 (or 1) for the first point
considered, and the opposite for the second point thus the function is balanced. In the
worst case though, i.e. when the first N/2 points give the same result, it would take
2k−1 + 1 evaluations to conclude whether the function is constant or balanced, i.e. the
number of computational steps needed grows exponentially with the number of bits that
define the domain.
In the context of chaotic computation the situation is much simpler since we can
perform the given function on all domain points simultaneously, furthermore we can read
the result in one step. To make the demonstration of the algorithm clearer we explicitly
use the tent map, Equation 1–3 and work in a state space that applies to this map. Also
we partition the explanation of the algorithm in three steps: (a) definition of the domain,
(b) definition of the function space, and (c) implementation of the function along with
reading the result. As is typical of chaotic computation there are at least three different
5 We are guaranteed by the problem that the function will be either constant orbalanced, exclusively.
6 The number of points in a given domain is given as: N = 2k, where k is the number ofbinary digits considered.
68
implementations of the algorithm [26], below we present, with some modifications from
[27], the most elaborate, but clear realization.
(a) Consider a binary domain space (Bk) defined by points of length k binary digits,
therefore the number of points in Bk is equal to N = 2k. Since we are to work with the
tent map we will translate every point in Bk, onto a point in the domain of the tent map
[0, 1]. Take any of the 2k points as a1a2a3...ak, where each ai ∈ {0, 1}, we map each such
point onto [0, 1] using7 :
x = 2−(k+1) +
k∑
i=1
ai2−i (2–3)
We encode the whole domain on an array of N elements, denoted by XN , each
element(j) having a state x(j) given by Equation 2–3. To illustrate this, consider the
case where k = 3, the binary domain (B3) has eight points {000, 001, 010, ..., 111} which
translate onto the tent map domain as { 116, 316, 516, 716, 916, 1116, 1316, 1516}, each of these points
is used as the state value of a dynamical element in an eight element array. So the whole
domain in consideration is encoded by a single array as:
X8 = {x(1) = 116 x(2) =
316 x(3) =
516 x(4) =
716 x(5) =
916 x(6) =
1116 x(7) =
1316 x(8) =
1516} .
(b) The function space of the functions: f : Bk → {0, 1} consists of 22kfunctions,
that is for each of the N = 2k points there are two output possibilities. Regardless
of the total number of possible functions only two are constant, the function that has
outputs all 1 and the function that has outputs all 0, for all input points. The number
of balanced functions though, is dependant on k and is given by simple combinatorics as
CLM =L!
(L−M)!M ! , where for a set of L items, made up of only two distinct objects, C is the
number of combinations where there are M items of one of the two objects. In the case
of balanced functions for a binary domain of N points the above relationship becomes
7 The 2−(k+1) term is added so that points 0 and 0.5 are not used for encoding.
69
CNN/2 =N !
(N−(N/2))!(N/2)! . More specifically for our example of k = 3, we have a total number
of possible functions 223= 256, out of which two are constant and (C84 =)70 are balanced.
In our case, of the function domain being represented by an array of tent maps as
described above, the function space will be populated with functions which are constructed
out of combinations of the following two basis functions:
xn+1 = T (xn) = 1− 2 ∙
∣∣∣∣xn −1
2
∣∣∣∣ (2–4)
xn+1 = T (xn) = 1− T (xn) = 2 ∙
∣∣∣∣xn −1
2
∣∣∣∣ (2–5)
where T is the tent map and T the inverted tent map, and n is used to denote the time
step. Each one of the 22kfunctions is constructed as one of the different combinations of
the two basis functions of length 2k, specifically the function space looks like:
{T (1)T (2)...T (j), T (1)T (2)...T (j), ...
..., T (1)T (2)...T (j − 1)T (j), ...
..., T (1)T (2)...T (j)},
where j = 2k, and each sequence of length j of the two basis functions, is one of the
possible functions (F) to be applied on the domain space8 .
Referring back to our concrete example of k = 3, we have 256 combinations of T and
T in the function space, ranging from the single sequence of eight consecutive T , to 70
combinations of four f : {0, 1}k → {0} and four f : {0, 1}k → {1}, to the single case of
eight consecutive T ; and of course the other 184 combinations9 .
8 In our notation here we use j instead of N to indicate the relation between thefunctions and the array of dynamical elements encoding the domain space.
9 Care not to confuse the 70 functions of four T and four T with the 70 balancedfunctions! A function that is balanced in T and T is not necessarily balanced in itsoutput.
70
Figure 2-10. Basis function T - Tent Map.
(c) Turning to the specific processes involved in solving the problem, we will focus
on the example of k = 3 to make this section more illustrative. As we have seen for
the 3-bit case there are 256 functions, in the context of the problem, we are given one
of these functions (F) (i.e. a specific sequence of 8 T and/or T ) and are assured that it
is either constant or balanced, the task is to find which one it is. In a straightforward
manner we setup the array of dynamical elements, X8, to encode the binary domain
as explained in (a) and then we apply the given function on the whole array F(Xn8 ) =
Xn+18 , where n denotes the time step. Down to the scale of the individual elements
we have T (xn(j)) = xn+1(j) and T (xn(j)) = xn+1(j), where j = 1, ..., 8, whether
71
Figure 2-11. Basis function T - Inverted Tent Map.
it is T or T applied on the jth element is based on which function (F) we are given.
Once the function is applied we threshold the elements at x∗ = 0.5 and collect the
excess the usual way. Basically we are using 0.5 as a separatrix of the state space, i.e.
xn+1(j) > 0.5 ≡ 1 ⇒ xn+1(j) − 0.5 = Ej and xn+1(j) ≤ 0.5 ≡ 0, Ej = 0 (where E is
emitted excess), this is a standard application of symbolic dynamics.
Now we have the collected excess in a single step we also have the answer: If the
collected excess is 0, then the function we have is the constant function F : Bk → {0}k,
72
i.e. the function that has output all 0; If the collected excess10 is 2k/4 = 8/4 = 2, which is
the maximum possible excess for B3, then we have the constant function F : Bk → {1}k,
i.e. the function that has output all 1. Obviously if our collected excess is some other
value we have a balanced function. This is as much as we are required by the original
problem, but we can do even better, using the excess collected we can determine in which
of five classes the given balanced function belongs to.
The importance of this algorithm is not in the actual task it accomplishes, which
is of little practical use. The primary importance is, like for the quantum analogue, in
demonstrating an extremely higher efficiency than conventional computation in solving
problems of this class. Further though for chaotic computation it is a milestone against
quantum computation as well. It shows how chaotic computation can handle problems as
well as quantum computation, if not better.
2.4 Conclusion
This chapter almost exclusively dealt with theoretical developments during the
first four years of Chaotic Computation (1998-2002), while we relegated the experimental
realizations to the references, so that to maintain a uniform global tone to this dissertation.
In closing and without wishing to undermine the other results, once again we draw
attention to the important result of the solution to the Deutsch-Jozsa problem, and ask
the reader to consider the connections between this problem, Set Theory, Logic, and to be
more specific to the structure of a chaotic function at the Feigenbaum point.
10 The factor of 14comes from the fact that the average excess emitted 〈Ej〉 is 0.25.
73
Figure 2-12. Four realizations of the chaotic Deutsch-Jozsa algorithm for the case k = 3.There are eight binary inputs 000, 001, 010, 011, 100, 101, 110, 111 andthrough Equation 2–3 we obtain the state values 1
16, 316, 516, 716, 916, 1116, 1316, 1516
each given to an element of an array. The vertical lines mark the state valueof each element x(j). The horizontal line is the separatrix at 0.5. (a) Given
the constant function F : Bk → {1}k, this is the function: T T TTTT T T , theapplication of the function brings every element over 0.5, the maximumexcess of 2 is emitted from the array. (b) Given the constant function
F : Bk → {0}k, TT T T T T TT , all array elements remain under 0.5 resulting inzero excess. (c) Given the balanced function of eight T functions, produces anexcess of 1. (d) A randomly chosen balanced function, four elements are over0.5 and four under, the excess is 1.25.
74
Figure 2-13. The total excess emitted from each of the 72 functions. The two squaresindicate the two constant functions, referring back to Figures 2-12(a,b). Asyou can see the balanced functions (circles) separate in five groups. Thepoints (c) and (d) refer to the respective graphs of Figure 2-12.
75
CHAPTER 3SEARCHING AN UNSORTED DATABASE
In this chapter we present a chaotic computation algorithm for searching an unsorted
database for a match between a queried for item and the contents of the database [28].
The dynamical system we use for the demonstration of the algorithm is the Tent map,
given in Equation 1–3.
In general most commonly used devices for storing and processing information are
based on the binary encoding of information, i.e. upon bits. Larger chunks of information
are encoded by combining consecutive bits into bytes and words. Here we show a different
approach for information encoding and storage, based on the wide variety of patterns that
can be extracted from nonlinear dynamical systems.
We specifically demonstrate the use of arrays of nonlinear dynamical systems
(or elements) to stably encode and store information (such as patterns and strings).
Furthermore we demonstrate how this storage method enables the efficient and rapid
search for specified items of information in the data store. It is the nonlinear dynamics
of each array element that provides flexible capacity storage, as well as the means to
preprocess data for exact and inexact pattern matching. In particular, we choose chaotic
systems to store and process data through the natural evolution of their dynamics. More
importantly perhaps we note that, our method involves just a single procedural step, it is
naturally set up for parallel implementation and can be realized with hardware currently
employed for chaos-based computing architectures.
We first show a slightly modified storing and encoding scheme, based on the Excess
overflow scheme of Section 2.1.1, in which we use the actual fixed point state of the
system, instead of the excess generated. Specifically we demonstrate this scheme as
applied to the Tent map for storing, and associate the storage with more than just
numbers, different encodings. We then show the actual “search process” and how the
results are a direct consequence of the non-linear nature of the Tent map. Finally with
76
specific examples we show implementations of the method for not only exact matches, but
for also “inexact”, approximate matches, to a given target.
3.1 Encoding and Storing Information
Consider a list of N data storage elements (labeled as j = 1, 2, ..., N) in an array,
where each element stores and encodes of one of M distinct items. N can be arbitrarily
large and M is determined by the kind of data being stored. For instance when storing
English text we can consider the letters of the alphabet to be each a naturally distinct
item, so M = 26. For the case of data stored in decimal representation M = 10, and for
work in bioinformatics (manipulating the symbols A, T, C, and G) we have M = 4. We
can also consider strings and patterns as the items. For instance for manipulating English
text we can use a large set of keywords as the basis, necessitating very large M. We store
this list of N elements by N dynamically evolving chaotic elements. The state of the
elements at discrete time n is given by Xjn[m], where (j = 1, 2, ..., N) indexes each element
of our list and (m = 1, 2, ...,M) indexes an item in our ”alphabet” (namely one of the M
distinct items). To reliably store information one must confine each dynamical system to
a fixed point behaviour, i.e. a state that is stable and constant throughout the dynamical
evolution of the system over time n, basically so that the list remains unchanged. We
therefore employ a threshold control mechanism, see Section 1.2.3, to flexibly control the
dynamical elements onto the large set of period 1 fixed points.
Specifically for the tent map, thresholds in the range [0, 23] yield fixed points, namely
Xn = T, for all time, where T is a threshold from 0 ≤ T ≤ 23. See Figure 3-1 for a
schematic of the tent map under the threshold mechanism, which is effectively described
by a “be-headed map”. It is clear from Figure 3-1 that in the range [0, 23] the value of Xn
lies above Xn implying that the system with state Xn at threshold T will be mapped to a
state higher than T in the subsequent iterate and thus will be clipped back to T. Another
way of graphically rationalizing this is to note that fixed point solutions are obtained
where the Xn+1 = Xn line intersects the “beheaded” tent map. The value of X at the
77
intersection yields the value of the fixed point, and the slope at the intersection naturally
gives the stability of the fixed point. It is clear from Figure 3-1 that in the range [0, 23] this
intersection is on the “plateau”, namely the fixed point solution is equal to the threshold
value. Further the solution of the fixed point for this map is super stable as the slope is
exactly zero on the “plateau”. This makes the thresholded state very robust and quite
insensitive to noise.
Figure 3-1. The Tent map under the threshold mechanism. Shown two cases of thresholdcontrol at 1
4and 1
2. Effectively each threshold value is on a “plateau” yielding
a fixed point. The symbol M indicates the action of the Tent map on thethresholded value Xn < Xn+1, while H indicates the effect of the controlXn+1 → T, for X values in the range [0, 23 ].
78
Returning to our data of a given “alphabet” we can take a large set of thresholds
{T[1],T[2], ...,T[M ]} from the fixed point range, setting up a one-to-one correspondence
of these M thresholds with the M distinct items of our data. This allows each item m to
be uniquely encoded by a specific threshold T[m], with (m = 1, 2, ...M). So the number
of distinct items that can be stored in a single dynamical element is typically large, as the
size of M is limited only by the precision and resolution of the threshold setting and the
noise characteristics of the physical system being employed.
Therefore given an unsorted list of N data, chosen out of the “alphabet” of M items,
as described above, we setup an array of N elements (labeled as j = 1, 2, ..., N), each a
Tent map, each with a threshold Tj[m] reliably storing and encoding the appropriate item
of the list. That is, if element j holds item m in the unsorted list, the threshold value
of element j is set to Tj[m], without changing, or in any way affecting any parameter
of the list. So by denoting the threshold of element j by Tj[m] we have the following:
if the state of element j of the system, Xjn[m], exceeds its prescribed threshold Tj[m]
(i.e. when Xjn[m] > Tj[m]) the state Xjn[m] is reset to T
j[m]. Since the thresholds lie in
the range yielding fixed points of period 1, this enables each element to hold its state at
value Xjn[m] = Tj[m] for all times n.
In our encoding for a reason that will become apparent in the next section, the
thresholds are chosen from the interval (0, 12), namely a subset of the fixed point window
[0, 23]. For specific illustration, without loss of generality consider each item to be
represented by an integer m, in the range [1,M ]. Defining a resolution r between each
integer as:
r =1
2∙
1
(M + 1), (3–1)
gives us a lookup table, mapping the encoded item to the threshold, specifically relating
the integers m in the range [1,M ] to the thresholds Tj[m] in the range [r, 12− r] by:
79
Tj[m] = m ∙ r. (3–2)
Therefore we obtain a direct correspondence between the set of integers 1 to M ,
where each integer can represent any item, and a set of M threshold values of a dynamical
system. Even more we can store N list elements by setting appropriate thresholds, via
Equation 3–2, on N dynamical elements. As mentioned before, the thresholded states
encoding different items are very robust to noise since they are superstable fixed points.
Finally this correspondence, or representation, is important for the process of encoding
information in an M -level representation and, as we shall see below, it is primarily
important for the process of searching the list for certain bits of information, by utilizing a
specific property of the system.
3.2 Searching for Information
Once we have a given list stored by setting appropriate thresholds on N dynamical
elements, we can query for the existence of a specific item in the list. Here we show
how the manner in which the information is encoded helps us preprocess the data such
that the effort required in the pattern matching searches is reduced. Specifically we will
demonstrate how we can use one global operational step to map the state of elements with
the matching item to an unique maximal state that can be easily detected. Note that
such an operation enables us to detect matches to strings/patterns (of length equivalent
to log2M binary bits) in one step. It would take typically log2M steps to do the same for
the case of binary encoded data.
When searching for a specific item in the list, we globally shift the state of all
elements of the list up by the amount that represents the queried item. Specifically the
state Xjn[m] of all the elements (j = 1, ..., N) is raised to Xjn[m] + Q[k], where Q[k] is a
search key given by:
Q[k] =1
2−T[k], (3–3)
80
where k is the item being searched for, and T[k] its respective threshold. This addition
shifts the interval that the list elements can span, from [r, 12− r] to [r +Q[k], 1
2− r +Q[k]],
where Q[k] is the globally applied shift. Note that what we are searching for is the
representation of the item, not the item itself. For example, we can encode each letter of
the alphabet by a number, such that the lowest threshold Tj[1] represents the letter A, the
next highest Tj[2] represents B, etc. When we search for A, we are really searching for the
element with a state with threshold Tj[1].
The item being searched for is encoded in a manner “complimentary” to the encoding
of the items in the list (much like a key that fits a particular lock); i.e. Q[k]+T[k] adds up
to 12. This guarantees that only the element(s) matching the item being searched for will
have its state shifted to 12. The value of 1
2is special in that it is the only state value that
on the subsequent update of the system will reach the value of 1.0, which is the maximum
state value for the Tent map. So only the elements holding an item matching the queried
item will reach the extremal value 1.0 on the dynamical update following a search query.
Note that the important feature here is the nonlinear dynamics mapping uniquely the
state 12to 1, while all other states (both higher and lower than 1
2) get mapped to values
lower than 1. See Figure 3-2 for a schematic of this process.
The salient characteristic of the point 12is the fact that it is the unique critical point,
and so it acts as “pivot” point for the nonlinear dynamical folding that will occur on the
interval [r + Q[k], 12− r + Q[k]] during the next update. This provides us with a single
global monitoring operation to push the state of all the elements matching the queried
item to the unique maximal point in parallel. The crucial ingredient is the use of the
existing critical point in the dynamical mapping to implement selection. Chaos is not
strictly necessary here. It is evident that for unimodal maps higher nonlinearities allow
larger operational ranges for the search operation and also enhance the resolution of the
encoding. For the Tent map specifically, it can be shown that the minimal nonlinearity
necessary for the above search operation to work is operation in the chaotic region.
81
Figure 3-2. Schematic representation of the changes in the state of an element for (i) amatching queried item, (ii) an item “higher” than the queried item, and (iii)an item “lower” than the queried item. The “key” value used is Q = 1
4, so the
matched item as a value of 0.25. The behaviour of three values is shown 0.1,0.25 and 0.3. It is clear that the application of the “key” does not seem torelatively affect the three values, a simple linear translation. The applicationof the map though, indicated by M, clearly maps both 0.1 and 0.3 to lowerstates than the maximal state, acquired solely by 0.25.
82
Another specific feature of the tent map is that its piecewise linearity allows the encoding
and search operation to be very simple indeed.
Of course to complete the search we must now detect the maximal state located at
1. This can be accomplished in a variety of ways. For example, one can simply employ a
level detector to register all elements at the maximal state. This will directly give the total
number of matches, if any. So the total search process is rendered simpler as the state
with the matching pattern is selected out and mapped to the maximal value, allowing
easy detection. Even more, by relaxing the detection level by a prescribed “tolerance”,
we can check for the existence within our list of numbers or patterns that are close to the
item or pattern being searched for. In this case “close to” means “having a representation
that is close to the representation of the item for which we are searching for”. Using the
earlier example of English letters of the alphabet encoded using the lowest threshold
Tj[1] for A, the next higher threshold for B, etc., relaxing the detection threshold a small
amount allows us to find mistyped words, where L or N were substituted for M or where
X or Z were substituted for Y. However, if we had chosen our representation such that
the ordering put T and U before and after Y (as is the case on a standard QWERTY
keyboard), then our relaxed search would find spellings of “bot” or “bou” when “boy” was
intended. Thus “nearness” is defined by the choice of the representation and can be chosen
advantageously depending on the intended use. Figure 3-5 gives an illustrative example of
detecting such inexact matches.
So nonlinear dynamics works as a powerful “preprocessing” tool, reducing the
determination of matching patterns to the detection of maximal states, an operation that
can conceivably be accomplished by simple addition and in parallel.
3.3 Encoding, Storing and Searching: An Example
Consider the case where our data is English language text, encoded as described
above by an array of tent maps. In this case the distinct items are the letters of the
English alphabet. As a result M = 26 and we obtain r = 154= 0.0185185... from Equation
83
3–1, and the appropriate threshold level for each item is obtained from Equation 3–2.
More specifically, consider as our list the sentence “strawberry fields”. Each letter1 in
this sentence is an element of the list with a value selected from our 26 possible values and
can be encoded using the appropriate threshold, as in Figure 3-3(a).
Now the list, as encoded above, can be searched for specific items. Figure 3-3 presents
the example of searching for the letter “l”. To do so the search key value corresponding
to letter “l” (from Equation 3–3, Q[l] = 1554) is added globally to the state of all elements.
Then through their natural evolution, at the next time step the state of the element(s)
containing the letter “l” is maximized. In Figure 3-4 we performed an analogous query for
the letter “e”, which is present twice in our list, to show that multiple occurrences of the
same item can be detected. Finally in Figure 3-5 we search for an item that is not part of
our given list, the letter “x”. As expected Figure 3-5(c) shows that none of the elements
are maximized. By lowering the detection level to the value 1 − (2 ∙ r), we have detected
whether adjacent items to the queried one are present. Specifically we have detected that
the letters “w” and “y” are contained in the list. This demonstrates that inexact matches
can also be found by this scheme.
3.4 Discussion
A significant feature of the presented search method is that it employs a single
simple global shift operation and does not entail accessing each item separately at any
stage. It achieves this through the use of nonlinear folding to select out the matched
item, and this nonlinear operation is the result of the natural dynamical evolution of
the elements. So the search effort is considerably simplified because it uses the native
responses of the nonlinear dynamical elements. We can then think of this as a natural
application, at the machine level, in a computing machine consisting of chaotic modules
[25–27, 32, 69, 70, 76, 81–85]. It is also equally potent as a special-applications “search
1 The space between the words is ignored.
84
Figure 3-3. Searching for “l”. (a) Threshold levels encoding the sentence“strawberry fields”, bars marked as �; (b) the search key value for letter “l”is added to all elements, bars marked as � ; (c) the elements update to thenext time step, bars marked as �. For clarity we marked solid black anyelements that reach the detection level.
chip”, which can be added on to regular circuitry and should prove especially useful in
machines, which are repeatedly employed for selection/search operations.
In terms of the processor timescale, the search operation requires one dynamical
step, namely one unit of the processor‘s intrinsic update time. The principal point here
is the scope for parallelism that exists in our scheme. This is due to the selection process
occurring through one global shift, which implies that there is no scale-up (in principle)
with size N . Additionally conventional search algorithms work with ordered lists, and the
time required for ordering generically scales with N as O(N logN). Here in contrast, there
is no need for ordering, and this further reduces the search time.
Regarding information storage capacity, note that we employ an M -state encoding,
where M can be very large in principle. This offers much gain in encoding capacity. As
85
Figure 3-4. Searching for “e”. (a) Threshold levels encoding the sentence“strawberry fields”, bars marked as �; (b) the search key value for letter “e”is added to all elements, bars marked as � ; (c) the elements update to thenext time step, bars marked as �. For clarity we marked solid black anyelements that reach the detection level.
in the example we present above, the letters of the alphabet are encoded by one element
each; binary coding would require much more hardware to do the same. Specifically,
consider the illustrative example of encoding a list of names, and then searching the list
for the existence of a certain name. In the current ASCII encoding technique, each ASCII
letter is encoded into two hexadecimal numbers or 8 bits. Assuming a maximum name
length of k letters, this implies that one has to use 8 ∙ k binary bits per name. So typically
the search operation scales as O(8kN). Consider in comparison what our scheme offers: if
base 26 (“alphabetical” representation) is used, each letter is encoded into one dynamical
system (an “alphabit”). As mentioned before, the system is capable of this dense encoding
as it can be controlled on to 26 distinct fixed points, each corresponding to a letter. Again
assuming a maximum length of k letters per name, one needs to use k “alphabits” per
86
Figure 3-5. Searching for “x”. (a) Threshold levels encoding the sentence“strawberry fields”, bars marked as �; (b) the search key value for letter “x”is added to all elements, bars marked as � ; (c) the elements update to thenext time step, bars marked as �. It is clear that no elements reach thedetection level at 1.0; (d) By lowering the detection level we can detectwhether items “adjacent” to “x” are present. For clarity we marked solid blackany elements that reach the detection level (“w” and “y”).
name. So the search effort scales as kN . Namely, the storage is 8 times more efficient
and the search can be done roughly 8 times faster as well! In general if base S encoding
is employed, for example where S is the set of all possible names (size(S) ≤ N), then
each name is encoded into one dynamical system with S fixed points (a “superbit”). So
one needs to use just 1 “superbit” per name, implying that the search effort scales simply
as N , i.e. 8k times faster than the binary encoded case. Even more ,in practice the final
step of detecting the maximal values can conceivably be performed in parallel. This would
reduce the search effort to two time steps (one to map the matching item to the maximal
value and another step to detect the maximal value simultaneously). In that case the
search effort would be 8kN times faster than the binary benchmark.
87
Alternate ideas to implement the increasingly important problem of search have
included the use of quantum computers [80]. However, our nonlinear dynamical scheme
has the distinct advantage that the enabling technology for practical implementation
need not be very different from conventional silicon devices. Namely, the physical design
of a dynamical search chip should be realizable through conventional CMOS circuitry.
Implemented at the machine level, this scheme can perform unsorted searches efficiently.
CMOS circuit realizations of chaotic systems, like the tent map, already operate beyond
the region of 1 MHz [86, 87]. Thus a complete search for an item comprising of search key
addition, update, threshold detection, and list restoration can be performed at 250 kHz,
regardless of the length of the list. Even more though, nonlinear systems are abundant in
nature, and so embodiments of this concept can be conceived in many different physical
systems ranging from fluids to electronics to optics. Potentially good candidates for
physical realization of the scheme include nonlinear electronic circuits and optical devices
[88]. Also systems such as single electron tunneling junctions [89], which are naturally
piecewise linear maps, can conceivably be employed to make such search devices.
In summary we have presented a method to efficiently and flexibly store information
using nonlinear dynamical elements. We demonstrate how a single element can store M
distinct items, where M can be large and can vary to best suit the nature of the data
being stored and the application at hand. Namely, we have information storage elements
of flexible capacity, capable of naturally storing data in different bases or in different
alphabets or with multilevel logic. This cuts down space requirements by log2M in
relation to elements storing via binary bits. Further we have shown how this method of
storing information can be naturally exploited for searching of information. In particular,
we demonstrated a method to determine the existence of an item in an unsorted list. The
method involves a single global shift operation applied simultaneously to all the elements
comprising the list, such that the next dynamical step “pushes” the element(s) storing the
matching item (and only those) to a unique, maximal state. This extremal state can then
88
be detected by a simple level detector, directly giving the number of matches. Even more
the maxima state can be treated as a maximal range, in which case approximate matches
are identified as well.
89
CHAPTER 4A SIMPLE ELECTRONIC IMPLEMENTATION OF CHAOTIC COMPUTATION
This chapter is a short exposition of the results of our publication [34] concerning
an iterated map with a very simple (i.e. minimal) electronic implementation. We first
propose and characterize the map and then provide the circuit to implement the map. We
proceed to determine control thresholds for flexibly representing the five fundamental logic
gates and demonstrate how this map (and circuit) can be used to implement the search
algorithm introduced in Chapter 3.
4.1 An Iterated Nonlinear Map
We begin by considering an iterated map governed by the following equation:
xn+1 =α ∙ xn1 + xβn
, (4–1)
where α and β are system parameters. Figure 4-1 shows the bifurcation diagrams for
different values of α and β. It is evident that this map yields dynamics ranging from fixed
points through chaos. It is also clear that the map follows the period-doubling route to
chaos with respect to α, and it does so as well with respect to β. In the following sections
we will consider the map in the chaotic regime, with α = 2 and β = 10, namely the chaotic
map given by:
xn+1 =2 ∙ xn1 + x10n
, (4–2)
This operating point is indicated by the dotted line in the bottom right panel of
Figure 4-1 and the graphical form of this map is presented in Figure 4-2.
4.2 Threshold Control Chaos into Different Periods
Using the map given by Equation 4–2, we wish to construct a system that can
represent M distinct states, where M can be large. The size of M will be limited only by
our ability to distinguish one state from the next in the presence of noise. To do this we
90
Figure 4-1. Bifurcation diagram of the iterated map in Equation 4–1 for various values ofα and β. The dotted line in the bottom right panel, indicates the chosenoperating point as prescribed by Equation 4–2. Here x∞ is the value taken bythe map after initial transients have died out.
use the simple and easily implementable threshold control mechanism described in Section
1.2.3. Specifically we place under control the state variable x as:
xn+1 =
2∙xn1+x10n
, for xn+1 ≤ x∗,
x∗, for x∗ < xn+1,
(4–3)
where x∗ is the imposed threshold value. The effect of this control is to limit the available
phase space by clipping the state variable. In this method no parameters are adjusted,
and only one state variable is occasionally reset. Note that this scheme is computationally
simple and requires no costly run-time computations.
Figure 4-3 illustrates the behaviour of the system under varying thresholds.
Depending on the value of the threshold x∗, this control method produces a wide variety of
nonlinear dynamical behaviours ranging from fixed points to periodic behaviour of various
91
Figure 4-2. Graphical form of the map to be implemented by an electronic circuit. Theparameters for this form are set at α = 2 and β = 10.
periodicities to chaos. As indicated in the figure, the system is controlled to fixed points
for thresholds for x∗ < 1. When the threshold is above unity, many different periodic (as
well as chaotic) orbits become available.
4.3 Electronic Analog Circuit: Experimental Results
The realization of the discrete map of Equation 4–2 in circuitry is depicted in
Figure 4-4. In the circuit Vin and Vo denote input and output voltages and in terms of the
equation xn and xn+1, respectively. A simple nonlinear device is constructed by coupling
two complementary (n-channel and p-channel) (Q1, Q2) junction field-effect transistors
(JFETs) [90] mimicking the nonlinear characteristic curve f(x) = 2x1+x10
. The voltage
across resistor R1 is amplified by a factor of 5 using the operational amplifier U1 in order
to scale the output voltage back into the range of the input voltage, a necessary condition
for a circuit based on a map. The resulting voltage characteristics of the nonlinear device
are depicted in Figure 4-5, compare with the mathematical form of the map in Figure 4-2.
92
Figure 4-3. Effect of threshold value x∗ on the dynamics of the system given by Equation4–3.
In order though to realize the map of Equation 4–3, we require two more sample and
hold circuits, in addition to a threshold controller circuit, see Figure 4-6. The first sample
and hold (S/H) circuit holds the input signal (xn) in response to a clock signal CK1. The
output from this sample and hold circuit is fed as input to the nonlinear device for the
subsequent mapping, that is Equation 4–2. A second sample and hold (S/H) circuit takes
the output from the nonlinear device in response to a clock signal CK2. In lieu of control,
the output from the 2nd sample-and-hold circuit (xn+1) closes the loop as the input to 1st
sample-and-hold circuit, through the threshold control circuit. The main purpose of the
two sample-and-hold circuits is to introduce discreteness into the system and additionally
93
Figure 4-4. Circuit diagram of the nonlinear device of Equation 4–3. (Left) Intrinsic(resistorless), complementary device made of two (n-type and p-type) JFETs.Q1: 2N5457, Q2: 2N5460. (Right) Amplifier circuitry to scale the outputvoltage back into the range of the input voltage. R1: 535 Ω, U1: AD712op-amp, R2: 100 kΩ and R3: 450 kΩ. Here Vin = xn and Vo = xn+1.
to set the iteration speed. To implement the control for nonlinear dynamical computing,
the output from the 2nd sample and hold circuit is input to the threshold controller, as
that described by Equation 4–3. The output from this threshold controller then becomes
the input to the 1st sample-and-hold circuit.
In Figure 4-6, the sample and hold circuits are realized with National Semiconductors
sample and hold IC LF398, triggered by delayed timing clock pulses CK1 and CK2 [70].
Here a clock rate of either 10 kHz or 20 kHz may be used. The threshold controller circuit
is shown in Figure 4-7 is realized with an AD712 operational amplifier, a 1N4148 diode, a
1 kΩ series resistor and the threshold control voltage, x∗(= Vcon).
The Figure 4-8(a) displays the uncontrolled chaotic waveform and the Figures 4-8(b-d)
show representative results of the chaotic system under different threshold values
x∗(= Vcon). It is clear that adjusting the threshold yields cycles of varying periodicities.
Also, note that simply setting the threshold beyond the bounds of the attractor (5 V)
94
Figure 4-5. Voltage response characteristics of the nonlinear device, based on Equation4–2 and circuit of Figure 4-4.
gives back the original dynamics, and so the controller is easily switched on and off. A
detailed comparison shows complete agreement between experimental observations and
analytical results. For instance, the threshold that needs to be set in order to obtain a
certain periodicity and the trajectory of the controlled orbit can be worked out exactly
through symbolic dynamics techniques. Further, the control transience is very short here
(typically of the order of 10−3 times the controlled cycle length) and the perturbation
involved in threshold control is usually small. This method is then especially useful in
the situation where one wishes to design controllable components that can switch flexibly
between different behaviours. Calibrating the systems characteristics at the outset with
95
Figure 4-6. Schematic diagram for implementing the threshold controlled nonlinear map.CK1 and CK2 are clock timing signals, while the modules designated S/H aresample-and-hold circuits.
Figure 4-7. Circuit diagram of the threshold controller. Vin and Vo are the input andoutput, D is a 1N4148 diode, R = 1 kΩ, and U2 is an AD712 op-amp.Vcon = x
∗ (controller input voltage).
96
respect to threshold gives one a look-up table directly and simply to extract widely
varying temporal patterns.
Figure 4-8. PSPICE simulation results of the experimental circuit. The ordinate is xn andthe abscissa is the discrete time n measured in ms. (a) Uncontrolled chaos:x∗ = 6 V, (b) period 5 cycle: x∗ = 4 V, (c) period 2 cycle: x∗ = 3.7 V and (d)period 1 cycle: x∗ = 3.5 V.
4.4 Fundamental Logic Gates with a Chaotic Circuit
Here we explicitly show how, by using the threshold controlled map of Equation 4–3,
we obtains the clearly defined logic gate operations NOR, NAND, AND, OR, and
XOR. The state of the system is represented by the state value of x. The initial
state of the system is represented as x0. In our method all five basic gate operations
involve the following steps: specification of x0 based on the operation and the inputs
through threshold control, nonlinear update (evolution of the circuit dynamics), and
output interpretation through threshold “monitoring” in the spirit of Excess Overflow
Propagation, from Section 1.2.3. Specifically:
1. Inputs and Programming; x0 = xprog + xI1 + xI2 . Here xprog is a programming
shift that fixes the initial state x0 of the system, based on the gate to be operated.Letting a finite voltage δ denote a logical 1, we set xIi = δ for an input of logical 1and xIi = 0 for an input of logical 0.
97
2. Nonlinear update; i.e., x0 → f(x0), where f(x) is the nonlinear function, given byEquation 4–2.
3. Thresholding to obtain the output Z defined as:
Z =
{0, for f(x) ≤ x∗,
f(x)− x∗, for x∗ < f(x),where x∗ = 1 is the threshold reference signal; which is set the same for all gates unlike
previously in the case of the Logistic map in Section 2.3.1. The output is interpreted as
logic output 0 if Z = 0 and logic output 1 if 12∙ δ < Z. Since the system is nonlinear (and
may be chaotic), in order to specify the initial x0 accurately in hardware experiments,
one needs a controlling mechanism. Here we will employ a threshold controller to set the
initial x0. So in this example we will use the clipping action of the threshold controller to
achieve the initialization. A comparator is used to recover the output from the state of Z.
The logical operations are defined by the input to output mappings depicted in the truth
table of Table 4-1.
Table 4-1. Truth-table for the five fundamental logic gates NOR, NAND, AND, OR andXOR.
I1 I2 NOR NAND AND OR XOR0 0 1 1 0 0 00 1 0 1 0 1 11 0 0 1 0 1 11 1 0 0 1 1 0
From the definition of f(x), x∗ and the above truth table, we obtain a set of
inequality conditions that need to be satisfied simultaneously, shown in Table 4-2. Note
that the symmetry with respect to the inputs reduces the four conditions in the truth
table of Table 4-1 to three distinct conditions, with rows 2 and 3 of Table 4-1 leading to
the single condition of row 2 in Table 4-2.
The above inequalities have many possible solutions depending on the size of δ. By
setting δ = 0.3 we can easily solve the equations for the different programming shifts that
each gate requires. The specific xprog values for the five different logical operations are
listed in Table 4-3.
98
Table 4-2. Necessary and sufficient conditions to be satisfied by a chaotic element in orderto implement the logical operations NOR, NAND, AND, OR and XOR. Wheref(x) is given by Equation 4–2 and 1 is the monitoring threshold (x∗) ofinterpretation of the logic output.
I1 I2 NOR NAND AND0 0 1 < f(xprog) 1 < f(xprog) f(xprog) ≤ 10/1 1/0 f(xprog + δ) ≤ 1 1 < f(xprog + δ) f(xprog + δ) ≤ 11 1 f(xprog + 2 ∙ δ) ≤ 1 f(xprog + 2 ∙ δ) ≤ 1 1 < f(xprog + 2 ∙ δ)
I1 I2 OR XOR0 0 f(xprog) ≤ 1 f(xprog) ≤ 10/1 1/0 1 < f(xprog + δ) 1 < f(xprog + δ)1 1 1 < f(xprog + 2 ∙ δ) f(xprog + 2 ∙ δ) ≤ 1
Table 4-3. Numerical values of xprog for implementing logical operations NOR, NAND,AND, OR and XOR, based on Equation 4–2.
Operation NOR NAND AND OR XORxprog 0.9138 0.6602 0.0602 0.3602 0.45
By setting up a map the with initial condition x0, as defined above, we allow the
map to update to a new value x1 = f(x0) and compare this value to the monitoring
threshold x∗ = 1, if the new state of the map is greater than the threshold a logical 1
is the output, if the new state is less than the threshold a logical 0 is the output. The
updated states of a chaotic element, following Equation 4–2, satisfying the conditions
of Table 4-2, with xprog values given in Table 4-3 are shown in Table 4-4. The circuitry
described in the section above has been tested to do the logic operations described here
and shows complete agreement with the simulation results.
Table 4-4. Updated state values, x1 = f(x0), of a chaotic element satisfying the conditionsin Table 4-2 in order to implement the logical operations NOR, NAND, AND,OR and XOR.
Operation NOR NAND AND OR XORf(xprog) 1.3 1.3 0.1204 0.7204 0.8997f(xprog + δ) 0.3057 1.1525 0.7204 1.3 1.42f(xprog + 2 ∙ δ) 0.0472 0.2270 1.3 1.1525 0.7988
We have presented a proof of principle device that demonstrates the capability of this
nonlinear map to implement the five fundamental logic gates; it does this by exploiting the
nonlinear response of the system. The main benefit is its ability to exploit a single chaotic
99
element to reconfigure into different logic gates through a threshold based morphing
mechanism. Contrast this to a conventional field programmable gate array element, where
reconfiguration is achieved through switching between multiple single purpose gates.
4.5 Encoding and Searching a Database Using Chaotic Elements
In the spirit of Chapter 3 we apply that method to the map, and circuit, given by
Equation 4–3. Specifically we show how this map can be utilized to stably encode and
store various items of information (such as patterns and strings) to create a “database”.
Further we demonstrate how this storage method allows to efficiently determine the
number of matches (if any) to some specified item [28]. Consider an array of elements each
of which evolves according to Equation 4–3. The nonlinear dynamics of the array elements
will be utilized for flexible capacity storage, as well as for pre-processing data for exact
(and inexact) pattern matching tasks.
Encoding information. We consider a “database” of length N and each member
of the database is encoded in an element obeying Equation 4–3, we index these elements
with j = {1, 2, 3, ...N}, so the state of the whole array, at a time n, can be represented
as Xjn. At the same time the database is made up of items from an “alphabet” of total
number of unique items M , indexed with m = {1, 2, 3, ...M}. We correlate each item m
with a threshold, x∗, for Equation 4–3, such that the element is confined on a fixed point
of period 1, we define T j[m] as the threshold for the jth element encoding the mth item.
For this map, thresholds ranging from 0 to 1 yield fixed points, as depicted in Figure 4–3.
Namely Xjn = Tj[m], for all time n, when the threshold is chosen as 0 < T j[m] < 1.
This can be obtained exactly from the fact that x < f(x), ∀x ∈ (0, 1), implying that the
subsequent iteration of a state at T j[m] will always exceed T j[m] and thus be reset to
T j[m].
100
In our encoding, the thresholds are chosen from the interval (0, 12), namely a sub-set
of the fixed-point window (0, 1)1 . Without loss of generality, consider each item to be
represented by an integer z from the range [1,M ]. Defining a resolution r between each
threshold as:
r =1
2∙1
M, (4–4)
gives a lookup map from the encoded integer to the threshold, relating the integers z in
the set [1,M ] to thresholds T j[m] in the range [r, 12], by:
T j[m] = z ∙ r. (4–5)
Therefore we obtain a direct correspondence between a set of integers ranging from
1 to M , where each integer represents an item, and a set of M threshold values. So we
can store N database elements by setting appropriate thresholds (via Equation 4–5) on
N dynamical elements. Clearly from Equation 4–5, if the threshold setting has better
resolution (smaller r), then a larger range of values can be encoded. Note however that
precision is not a restrictive issue here, as different data representations can always be
chosen in order to suit a given precision of the threshold mechanism.
Processing Information. Once we have a given database stored by setting
appropriate thresholds on N dynamical elements, we can query for the existence of a
specific item in the database in one global operational step. This is achieved by globally
shifting the state variable of all elements of the database up by an amount that represents
the item being searched for.
1 Actually we can use as much as the interval (0, 0.8027), since 0.8027 is the pre-imageof the maximum.
101
Noting that the maximal state variable value for this system is 1.4449, one raises the
state Xjn of each element j to Xjn +Q[k], where Q[k] is a search key given by:
Q[k] = 0.8027− T [k], (4–6)
where k is the index of the integer(item) being queried for, k ∈ z. 0.8027 is the unique
value of this system that evolves to the maximal value 1.4449 on an iteration of the
system. So the value of the search key is simply 0.8027 (the pre-image of the maximal
state variable value) minus the threshold value corresponding to the item being searched
for, given k ∈ z we have T [k] = k ∙ r.
The addition of the “search key”, Q[k] shifts the interval that the database elements
can span, from [r, 12] to [r + Q[k], 1
2+ Q[k]]. Since Q[k] + T [k] adds up to 0.8027, it
is guaranteed that only the element(s) matching the item being queried for will have
its(their) state shifted to 0.8027, which is the only state which after the subsequent
iteration will maximize to the value of 1.44492 . So the total search process is rendered
simple as the state with the matching pattern is selected out and mapped to the maximal
value, allowing easy detection. Further, by relaxing the detection level by a prescribed
“tolerance”, we can check for the existence within our database of numbers or patterns
that are “close to”3
Representative Example. Consider the case where our data is English language
text, encoded as described above on a letter by letter basis by an array of maps, following
Equation 4–5. In this case the distinct items are the letters of the English alphabet and
we have M = 26. We obtain r = 152≈ 0.0192 from Equation 4–4 and the appropriate
threshold level for each item is obtained via Equation 4–5. More concretely, consider as
2 Note all other states (both higher and lower than 0.8027) get mapped to values lowerthan 1.4449.
3 Where “close to” is defined by the designer of the database.
102
our database the phrase “the quick brown fox”; each letter in this phrase is an element of
the database and can be encoded using the appropriate threshold, as in Figure 4-9(a).
Figure 4-9. Searching for “b”. (a) Threshold levels encoding the phrase“the quick brown fox”, bars marked as �; (b) the search key value for theletter “b” is added to all elements, bars marked as � ; (c) the elements updateto the next time step, bars marked as �. For clarity we mark black theelements that reached the detection level.
Now we query the database regarding the existence of specific items. Figure 4-9
presents the example of querying for the letter “b”. To do so the search key value
corresponding to letter “b” ( 252) is added globally to the states of all elements, Figure
4-9(b). Then through their natural evolution, upon the next time step, the state(s) of the
element(s) containing the letter “b” is(are) maximized, Figure 4-9(c). In Figure 4-10 we
perform an analogous query for the letter “o”, which happens to be present twice in our
database to show that multiple occurrences of the same item can be detected. Finally in
Figure 4-11 we query for an item that is not part of our given database, the letter “d”. As
expected Figure 4-9(c) shows that none of the elements are maximized. By lowering the
103
Figure 4-10. Searching for “o”. (a) Threshold levels encoding the phrase“the quick brown fox”, bars marked as �; (b) the search key value for theletter “o” is added to all elements, bars marked as � ; (c) the elementsupdate to the next time step, bars marked as �. For clarity we mark blackany elements that reached the detection level.
detection level to the value 1.4449− f(0.8027− r) = 1.4411, just “one step” down from the
maximal, we detect whether items “adjacent” to the desired one are present. Specifically
we detect that the letters “c” and “e” are contained in our database. This demonstrates
that inexact matches can also be found, just as easily.
4.6 Conclusion
In summary, we introduced a simple map having rich nonlinear dynamics, and
a simple electronic circuit realization. Then we demonstrated the direct and flexible
implementation of the five basic logic gates using this simple nonlinear map (circuit).
Further, we showed how the dynamics of this map can be utilized to provide an efficient
database search method.We have experimentally implemented the electronic circuit analog
104
Figure 4-11. Searching for “d”. (a) Threshold levels encoding the phrase“the quick brown fox”, bars marked as �; (b) the search key value for theletter “d” is added to all elements, bars marked as � ; (c) the elementsupdate to the next time step, bars marked as �. It is clear that no elementsreach the detection level at 1.4449; (d) By lowering the detection level we candetect whether items “adjacent” to “d” are present (“c” and “e”).
of this nonlinear map and have demonstrated the efficacy of the threshold controller in
yielding different controlled responses from this map circuit.
105
CHAPTER 5LOGIC OPERATIONS FROM EVOLUTION OF DYNAMICAL SYSTEMS
In this chapter we propose the direct and flexible implementation of logic operations
using the dynamical evolution of a nonlinear system [33]. The concept involves the
observation of the state of the system at different time instances to obtain different logic
outputs. We explicitly implement the basic NAND, AND, NOR, OR and XOR logic gates,
as well as multiple-input XOR and XNOR logic gates. Further we demonstrate how the
single dynamical system can do more complex operations such as bit by bit addition in
just two iterations. The concept uses the nonlinear characteristics of the time dependence
of the state of the dynamical system to extract different responses from the system. The
highlight of this method is that a single nonlinear system is capable of yielding a time
sequence of different logic operations. Further we explicitly demonstrate, through the three
examples, how results from this method can be obtained by varying any of the “defining
variables” (x0, xinit, δ, n).
5.1 Generation of a Sequence of (2-input) Logic Gate Operations
We outline a method for obtaining the five basic logic gates using different dynamical
iterates of a single nonlinear system. In particular consider a chaotic system whose state
is represented by a value x. The state of the system evolves according to some dynamical
rule. For instance, the updates of the state of the element from time n to n + 1 may be
well described by a map, i.e., xn+1 = f(xn), where f(x) is a nonlinear function. Now this
element receives inputs before the first iteration (i.e., at n = 0) and outputs a “signal”
after evolving for a (short) specified time or number of iterations. The method can be
applied for any sequence of the gates, for illustrative purposes we chose the sequence
NAND, AND, NOR, XOR and OR (see Table 5-1 for the truth table).
In general the method involves the following steps:
1. Input definition (for a 2 input operation): x0 = xinit + xI1 + xI2 , where xinit is the
initial state of the system. (Comparable to xprog in previous chapters, but now itis not defining a single gate but a sequence of gates, or more general, operations).
106
Table 5-1. The truth table of the five basic logic operations NAND, AND, NOR, XOR,OR.
I1 I2 NAND AND NOR XOR OR0 0 1 0 1 0 00 1 1 0 0 1 11 0 1 0 0 1 11 1 0 1 0 0 1
before data inputs are introduced, x0 is the actual initial state of the system thatincludes the data to be operated on. As previously we set xIi = δ for the logicalinput Ii = 1, and x
Ii = 0 for the logical input Ii = 0. So we need to consider thefollowing three cases:
(a) Both I1, and I2 are 0 (row 1 in Table I), i.e. the initial state of the system is:x0 = x
init + 0 + 0 = xinit.
(b) Either I1 = 0 and I2 = 1, or I1 = 1 and I2 = 0 (row 2 or 3 in Table I), i.e. theinitial state is: x0 = x
init + 0 + δ.
(c) Both I1 and I2 are 1 (row 4 in Table I), i.e. the initial state is:x0 = x
init + δ + δ = xinit + 2 ∙ δ.
2. Chaotic evolution over some prescribed number of steps, i.e. fn(x0)→ xn, for 1 < n.
3. The evolved states fn(x0) yield the logic output, at each n, as follows:Logic Output = 0, if fn(x0) ≤ x∗n, Logic Output = 1, if fn(x0) > x
∗n, where x
∗n is a
reference monitoring value, at time instance n.
Since the system is chaotic, in order to specify the initial x0 accurately we employ
the threshold control mechanism, see Section 1.2.3, we note that this mechanism can be
invoked at any subsequent iteration as well. For logic recovery, the updated or evolved
value of f(x) is compared with x∗n value using a comparator action, in the spirit of Excess
Overflow Propagation, again as in Section 1.2.3.
In order to obtain all the desired input-output responses of the different gates, as
displayed in Table 5-1, we need to satisfy the conditions enumerated in Table 5-2. Note
that the symmetry of inputs to outputs reduces the four conditions in the truth Table
5-1 to three distinct conditions, with 2textnd and 3rd row of Table 5-1 leading to the 2nd
condition of Table 5-2.
107
Table 5-2. Necessary and sufficient conditions to be satisfied by a chaotic element in orderto implement the logic operations NAND, AND, NOR, XOR and OR onsubsequent iterations. Here xinit = 0.325 and δ = 0.25, for all n. Whilex∗n = 0.75, for n = {1, 2, 3, 4}, that is for NAND, AND, NOR, XOR logicoperations, and x∗5 = 0.4 for OR logic operation.
Logic gate NAND AND NOR XOR ORIteration (n) 1 2 3 4 5Condition 1:
x1=f(x0) > x∗1 f(x1) ≤ x∗2 f(x2) > x
∗3 f(x3) ≤ x
∗4 f(x4) ≤ x
∗5Logic input
(0,0)x1 = 0.88 x2 = 0.43 x3 = 0.98 x4 = 0.08 x5 = 0.28x0 = 0.325
Condition 2:x1=f(x0) > x
∗1 f(x1) ≤ x∗2 f(x2) ≤ x∗3 f(x3) > x
∗4 f(x4) > x
∗5Logic input
(0,1) or (1,0)x1 = 0.9775 x2 = 0.088 x3 = 0.33 x4 = 0.872 x5 = 0.45x0 = 0.325 + δ
x0 = 0.575Condition 3:
x1=f(x0) ≤ x∗1 f(x1) > x∗2
textf(x2) ≤ x∗3 f(x3) ≤ x
∗4 f(x4) > x
∗5Logic input
(1,1)x1 = 0.58 x2 = 0.98 x3 = 0.1 x4 = 0.34 x5 = 0.9x0 = 0.325 + 2∙δ
x0 = 0.825
So given dynamics f(x), we must find values of threshold(s) x∗n, initial state(s) xinit
and δ satisfying the conditions derived from the specific truth table to be implemented.
Using, as usual, the Logistic map, Equation 1–1, we incorporated in Table 5-2 actual
values of xinit and x∗n for n = 1, 2, 3, 4, 5, which satisfy the conditions imposed by the truth
table considered. For illustrative purposes, the graphical representation of five iterations of
the Logistic map is shown in Figure 5-1, displaying the results in Table 5-2.
In summary, the inputs setup the initial state xinit+xI1+xI2 . Then the system evolves
over n iterative time steps to each updated state xn. The evolved state is compared to a
monitoring threshold x∗n, at every n. If the state at iteration n, is greater than the
threshold a logical 1 is the output and if the state is less than the threshold a logical 0 is
the output. This process is repeated for each subsequent iteration.
Note that in the above example we present results for which xinit is not varied and
x∗n is varied; the emphasis is on the specific behaviour that represents a computational
108
Figure 5-1. Graphical representation of five iterations of the Logistic map. Three differentinitial conditions are considered, each representing one of the three cases oftwo logic inputs, (0,0) by N, (0,1)/(1,0) by N, (1,1) by M. At each iterationcomparison with a monitoring threshold is performed, 0.75 for n = {1, 2, 3, 4},0.4 for n = 5. The results from two gates are also shown, the circles markn = 1 (the NAND gate) compare with 0.75 and the colouring corresponds tothe appropriate case of data; the squares mark n = 5 (the OR gate) comparewith 0.4, and again the colouring is appropriate to the data.
109
task and not the “actual” values as such. The case of δ is slightly more complicated since
it actually represents a constant “entity”, nevertheless varying its “actual” value is also
possible, but more care is needed, hence in this example we simply set δ = 0.25.
Therefore in a more general context, we are relating inputs and required outputs
with specific behavioural patterns, at the same time we can say the reverse, we find the
behavioural pattern that can represent a needed operation. Hence the “actual” state
values are not of great importance, and we can generate “templates” like Figure 5-2.
To generate this specific template, we set x∗n = 0.65 and varied x0, in contrast to the
example above; once again δ was kept constant at 0.25. Basically the behaviour of each
combination of variables is then interpreted as one of the eight possible symmetric binary
operations. It is clear from this figure that we are not confined to n < 5, so the “length”
of the sequence of operations can be extended, and also the actual order of the sequence
of operations can be changed. Theoretically all operations and all sequences of operations
exist, under some combination of “actual” values of variables.
We should note that, as is clear from Figure 5-2, the range of each operation is
decreasing in size with increasing iterations, and since the dynamics are chaotic at
some point we will lose definition, as we mentioned above though the threshold control
mechanism can be invoked to “re-initialize” the system.
5.2 The Full Adder and 3-Input XOR and NXOR
This section is a direct extension of the previous section, hence it is comprised of
simply two demonstrations. We extend the above method for sequential logic operations
in two ways, first to more than just two data inputs, and second to more than just logic
gate operations. Specifically we show the implementation of the binary full adder and the
implementations of 3-input XOR and NXOR gates.
For these implementations we employ the usual three steps, but modified for 3 inputs
as follows:
110
Figure 5-2. Patterns of binary two input symmetric operations. We use the Logistic mapwith δ = 0.25, x∗n = 0.65, ∀n and vary x
init.
1. Input definition (for a 3 input operation): x0 = xinit + xI1 + xI2 + xI3 , as usual with
the addition of one “extra” input. In the context of the full adder, I1 correspondsto the input binary number A, I2 corresponds to the input binary number B, and I3corresponds to the carry input Cin (the carry from the “previous” positional digitaddition), as in Table 5-3. So we need to consider the following four cases:
(a) If all inputs are 0 (1st row in Table 5-3), i.e. the initial state of the system is:x0 = x
init + 0 + 0 + 0 = x0.
(b) If any one of the input equals 1 (2nd, 3rd and 5th row in Table 5-3), i.e. theinitial state is: x0 = x
init+0+0+δ = xinit+0+δ+0 = xinit+δ+0+0 = xinit+δ.
(c) If any two inputs equal to 1 (4th, 6th and 7th row in Table 5-3), i.e. the initialstate is: x0 = x
init +0+ δ+ δ = xinit + δ+0+ δ = xinit + δ+ δ+0 = xinit +2 ∙ δ.
111
(d) If all inputs equal to 1 (8th row in Table 5-3), i.e. the initial state is:x0 = x
init + δ + δ + δ = xinit + 3 ∙ δ.
2. Chaotic evolution (for two time steps), of the initial state given from the dataf2(x0)→ x2. We confine the time steps to only two to make clear the example of thefull adder. In general of course, further time steps can be also considered providingresults for other 3 input operations, as in Figure 5-2.
3. The evolved state fn(x0) yields the logic output as follows:Logic Output = 0, if fn(x0) ≤ x∗n,Logic Output = 1, if fn(x0) > x
∗n,
where x∗n is a monitoring threshold, with n = 1, 2 in the full adder example.
Table 5-3. The truth table of full adder, and necessary conditions to be satisfied. Usingthe Logistic map, state values x1 (iteration n = 1) and x2 (iteration n = 2) areused to obtain Cout and S respectively. Here x
∗1 = 0.8, x
∗2 = 0.4, x
init = 0.0 andδ = 0.23.
Input bit of Input bit of Input bit of Output Condition for OutputNumber (A) Number (B) Carry (Cin) Cout S Cout (x1) S (x2)
0 0 0 0 0 f(x0) ≤ x∗1 f(x1) ≤ x∗20 0 1 0 1 f(x0 +δ) ≤ x∗1 f(x1) > x
∗2
0 1 0 0 1 f(x0 +δ) ≤ x∗1 f(x1) > x∗2
0 1 1 1 0 f(x0 + 2∙δ) > x∗1 f(x1) ≤ x∗2
1 0 0 0 1 f(x0 +δ) ≤ x∗1 f(x1) > x∗2
1 0 1 1 0 f(x0 + 2∙δ) > x∗1 f(x1) ≤ x∗2
1 1 0 1 0 f(x0 + 2∙δ) > x∗1 f(x1) ≤ x∗2
1 1 1 1 1 f(x0 + 3∙δ) > x∗1 f(x1) > x∗2
The Full Adder. We now demonstrate how one can obtain the ubiquitous bit
by bit arithmetic addition, involving three logic inputs and two outputs, in consecutive
iterations, with a single one-dimensional chaotic element. In general the simple 1-bit
binary arithmetic addition requires a full adder logic which adds three individual bits
together (two bits being the digit inputs and the third bit assumed to be carry from
the addition of the next least-significant bit addition operation, known as “Cin”). A
typical full-adder requires two half-adder circuits and an extra XOR gate. In total, the
implementation of a full-adder requires five different gates (three XOR gates and two
AND gates). However in the present direct implementation by utilizing the dynamical
evolution of a single map, the Logistic map, we need only two iterations to implement a
112
full adder. The truth table and necessary conditions to be satisfied for the parallelized
full adder operation are given in Table 5-3. The Carry bit output Cout and the Sum bit
output S are respectively recovered from the first and second iterations of a Logistic map,
with δ = 0.23, xinit = 0.0 x∗1 = 0.8 and x∗2 = 0.4. Note in this example we have kept x
init
constant and varied the x∗n.
So basically each system takes the three inputs, the value of the two bits to be added
and the carry from the previous bit addition, and produces the carry for the next addition
on the very first update. This new carry can of course be immediately supplied to the next
system ready to perform the addition of the next bit, while the sum (S) is “calculated” on
the second update.
3-Input XOR and NXOR. Three or more input logic gates are advantageous
because they require less complexity in actual experimental circuit realization than that of
coupling conventional 2-input logic gates [91, 92]. Specifically the truth table for 3-input
XOR and XNOR logic gate operations, and the necessary conditions to be satisfied by
a map to perform these operations, are shown in Table 5-4. In this representative case,
we consider a different scenario, for both operations we use the 2nd iteration of the map,
so both operations can not be accomplished by the same system. We set the threshold
(x∗2) at the fixed value of 0.5 and so we accomplish the XOR gate with xinit = 0.0 and
the NXOR gate with xinit = 0.25, most importanly though we have an exact fixed valued
δ ≈ 0.25.
5.3 Conclusion
Our previous results have shown that a single nonlinear dynamical system, with
proper tuning of parameters and control inputs, can flexibly become any logic gate. We
did confine ourselves though, to gates of only two inputs and to patterns created by a
single iteration of a system.
The main result of this chapter is that the various temporal patterns embedded in
the dynamical evolution of nonlinear systems are capable of performing sequences of
113
Table 5-4. The truth table of the 3-input XOR and NXOR logic operations, necessaryand sufficient conditions to be satisfied by the map. State value x2 (iterationn = 2) is used for logic operation recovery. Here x∗ = 0.5 and δ ≈ 0.25.
I1 I2 I3 XOR(I1,I2,I3) XNOR(I1,I2,I3) XOR (xinit = 0.0) NXOR (xinit = 0.25)
0 0 0 0 1 x2 ≤ x∗ x2 − x∗ ≈ δ0 0 1 1 0 x2 − x∗ ≈ δ x2 ≤ x∗
0 1 0 1 0 x2 − x∗ ≈ δ x2 ≤ x∗
1 0 0 1 0 x2 − x∗ ≈ δ x2 ≤ x∗
0 1 1 0 1 x2 ≤ x∗ x2 − x∗ ≈ δ1 0 1 0 1 x2 ≤ x∗ x2 − x∗ ≈ δ1 1 0 0 1 x2 ≤ x∗ x2 − x∗ ≈ δ1 1 1 1 0 x2 − x∗ ≈ δ x2 ≤ x∗
logic operations in time (or iterates), and minimize the control that is needed, we only
invoke control mechanism on initialization, from there on we just monitor the state. The
implementation of a sequence of logic functions in time, as described above, is another
mechanism through which computer architectures based upon the chaos computing
approach can be optimized for better performance. Even further in this chapter we
introduced mechanisms for multi-input logic, and showed how in the context of chaotic
computation behaviours are more important than exact and specific values of variables,
providing even more flexibility in a physical realization.
With these fundamental ingredients in hand it becomes clear that exploiting not just
the pattern formation of nonlinear dynamical systems, but the formation of sequences of
such patterns, produced naturally by such systems, may prove a key ingredient towards
making nonlinear dynamical computational architectures a real alternative to conventional
static logic computer architectures.
114
CHAPTER 6MANIPULATING TIME FOR COMPUTATION
We have shown in Chapter 5 how time can be involved in the computational process,
specifically we have shown how specific “time instances” can be used for programming
a system to perform a specific computational operation. In the present chapter, in some
sense we extend the utility of time; we show how the dimension of time can be used for
a complete definition of a computational operation, that is we use time instances not
only for programming a system, but also for introducing to the system the “data” to
be manipulated. To achieve this we predefine the behaviour of the state of the system
within a “clock cycle” and work with the fact that this behaviour is not constant within
the given cycle and so at different time instances different results can be obtained. In
a simplified mathematical context we are just interchanging x with t in the already
presented algorithms.
6.1 Introduction
We first present, in a general context, the time based algorithm for performing
the five fundamental 2-input logic gates and then the algorithm for performing the
search of an unsorted database. Further we support the method with specific illustrative
examples using neural systems [93, 94], and electronic circuits. We should note that in
both examples the general method is adapted to best suit the context of each the actual
implementation.
6.1.1 Flexible Logic Gates
For any computational system to be able to perform flexible (2-input) logic three
inputs need to be given to the system. The first input is the “Programming Instruction”
that defines which logic operation will be performed. The other two inputs are the two
logic variables, “Input1” (I1) and “Input2” (I2), see Figure 6-1. In the present method
we utilize a time instance, ttotal, as a single adjustable variable that can be used for
the representation of all three inputs. We define one time instance to represent the
115
Figure 6-1. Schematic representation of a flexible 2-input logic gate. For an Output threeparameters need to be defined, the two data streams, Input1 and Input2, andone Programming Instruction, which is to manipulate the data.
programming, tprog, and two more time instances to represent the two inputs, tI1 , tI2
respectively, with the relationship for a total time defined as: ttotal = tprog + tI1 + tI2 . We
represent a logical 1 by a finite time length, tin (relate to δ in the case of algorithms based
on state manipulation), and a logical 0 by a time length of zero length (tin = 0).
It is important to note that:
• Since a logical 0 at the inputs is set to be of zero time length, the set of inputs (0,0)is the same time interval as programming the gate (tprog).
• The time interval for representing an input of logical 1 must be the same irrespectiveof whether the logical 1 is at Input1 or at Input2, as the two are indistinguishable bya logic gate; the sets (1,0) and (0,1) are thus represented by the same time length;
• The total time interval for the set of inputs (1,1) is twice that of the sets (1,0) and(0,1).
116
The way we use the total time, ttotal, is to define a time instance within the clock
cycle of a “generic signal” constructed in such a way so as to encompass the results of
different logic operations at different time instances. Of course because of the “nature”
of time this time instance needs to be in reference to a fixed “zeroth” time instance,
which can be defined by the initialization of the clock cycle. In fact in the general context
demonstration we consider a single cycle initiated at the zeroth instance and use ttotal to
define a “moment of observation”, at which the result is obtained. There is though the
converse scenario as well, which we show in a following illustrative example in Section 6.2,
we can consider the “final” time instance as fixed and use ttotal as a “negative” shift in
instances from this final instance, basically changing the “moment of generation” of the
generic signal.
To construct the “generic signal” we consider the behaviour of the five fundamental
logic gates (AND, NAND, OR, NOR, XOR). From the truth table for these logic gates
(Table 6-1) we can create for each gate a graph of the input-output relationships for the
three pair sets of logical inputs, {(0, 0)}, {(1, 0)/(0, 1)}, {(1, 1)}, Figure 6-2(a)-(e). Since
we are to consider time as the input quantity, we can encapsulate all five graphs into a
single graph where the input dimension is identified with time instances, as explained in
Figure 6-2(f).
The combination of the truth tables into this single non-linear function results in the
form of : f(u, t) > 0, for t−Δt > t > t +Δ t, else f(u, t) = 0; which provides the required
behaviour for a system to be utilized as a flexible logic gate. This function is very similar
to an action potential generated by a neuron; resting at a low voltage (f(u, t) = 0), and
for a brief length of time rising very rapidly to a higher voltage (f(u, t) > 0), and then
dropping very rapidly back to its original level. We are considering this function in neural
context, not only because of the form of the function, but also because it is clear that
in some way neurons are capable of logic operations [95]. Note though, that we do not
117
actually propose that the methods we present here are the method neurons in fact use; we
use neurons primarily for demonstration purposes.
If we consider the inputs (and programming) to be represented as a time instance we
can see from Figure 6-2(f) that at different moments in time this signal is “high” or “low”,
we can interpret these two states as a logical 1 or 0 at the output. Therefore it is only a
matter of “when” we sample such a signal to obtain a different response, i.e. depending on
which time instance we chose to sample the signal we obtain a 1 or 0.
Therefore the method utilizes this behaviour to provide a flexible logic gate system
by varying the exact “moment of observation” (ttotal) of the signal depending on which
gate will be performed on what data. The “flexible” part should be evident, as we can
easily change the tprog from one cycle to the next, enabling the system to perform any two
different computational operations.
Table 6-1. The truth table of each of the five fundamental logic gates, AND, NAND, OR,NOR, XOR.
I1 I2 AND NAND OR NOR XOR0 0 0 1 0 1 01/0 0/1 0 1 1 0 11 1 1 0 1 0 0
Purely as a reference example, based on the signal of Figure 6-2(f) we give in Table
6-2 values for the different times to perform each of the five logic gates, on each of the
three pair sets of inputs. In each of the three cases of inputs, the time length representing
a logical 1 is two time “units” long (tin = 2). We utilize a signal as given in Figure 6-2(f),
with clock cycle defined to be 9 time units long. Depending on which logic operation we
wish to perform and also on what the inputs are, a sampling instance within the clock
cycle is defined, as explained above and given by ttotal. This sampling instance is in time
units with reference to the initialization from the clock cycle. We take the NOR gate as an
illustrative example. For the NOR gate we have tprog = 5 time units. For the set of inputs
(0,0) we sample the signal at the 5th (tprog + 0 + 0) time unit obtaining a 1 at the output,
see Figure 6-2(f). In the case where our inputs are (1,0) or (0,1) we have a sampling
118
Figure 6-2. (a) Input-Output relationship of AND gate; (b) Input-Output relationship ofNAND gate; (c) Input-Output relationship of OR gate; (d) Input-Outputrelationship of NOR gate; (d) Input-Output relationship of XOR gate;(f) Combined figure of the Input-Output relationships of the five fundamentalgates. Different sections of this graph represent different instances ofInput-Output operations. The 15 different cases, of the gates and inputsconsidered, are marked; with marking an Output of 1 and # marking anOutput 0.
Table 6-2. Time values, in arbitrary time “units” that produce each of the five gatesconsidered, based on the signal of Figure 6-2(f).
I1 I2 AND NAND OR NOR XOR0 0 0 4 2 5 31/0 0/1 2 6 4 7 51 1 4 8 6 9 7
instance of 7 time units, given by (tprog + tin + 0), obtaining a logical 0 at the output.
Similarly for the case NOR(1,1) we have (tprog + tin + tin) giving a sampling time at the 9th
time unit again obtaining a logical 0 at the output. Thus we have a correct representation
of the NOR gate. Using a similar scheme, and by simply changing the programming time
length, tprog we can switch to performing a different gate.
119
6.1.2 Search Algorithm
Promoting this idea further, one can now consider each moment in time as an
opportunity to represent any specific computational instruction or data. In order to
demonstrate more practical applications of using time for computational commands (data
and programming), we will demonstrate a variation on our algorithm for performing a
search on a given unsorted database [28].
The construction of a search engine requires the following steps:
1. A representation of data in the database;
2. The matching of data with searched for items;
3. Localization of the matched data.
Any given database is constructed out of M unique items, we already referred to
this set of items as the “alphabet”, and further each item can be represented by a unique
natural number, see Section 3.1. We have shown above that a finite time interval can
represent a logical 1, we can extend this idea to time lengths of multiple sizes, each one
representing a different number. Thus we utilize different time lengths to represent each
one of the unique items that constitute a given alphabet. These time lengths can be
marked by a delta function (a spike in neural context) occurring at a specific moment in
time with respect to the initialization of a clock cycle; the interval between initialization
and the delta function is the time length for a specific item. So given any database we can
represent it by a series of delta function generators, Rx, all receiving initialization from the
clock cycle, and each providing a spike at the specific delayed moment in time providing a
time interval which represents the data item it stores, see Figure 6-3. As shown in Figure
6-3 we can connect these function generators, Rx, each independently to a Coincidence
Detector Unit (CDU). The function of the CDUs is to produce a “confirmation”, a spike,
at the moment when it receives two simultaneous spikes. The “search element” is another
delta function generator, Rsearch,k set to produce a spike at the moment that represents
the data item we wish to search for. As a result the coincidence detector(s) produce a
120
spike only if the searched for item is within the given database since only then will a CDU
receive two simultaneous spikes (from a database element and the searched for item). Note
that the terminology we use is such as to relate this abstract presentation of the method
directly to the neural implementation to follow, specifically R will be a neural synapse
strength coefficient, while the TDU and CDU will be specific neural circuits.
For the final step of localization of positive results of a search we connect all the
coincidence detectors to an accumulator, Integrating Unit (IU), each weighted by a
different weight following Godel numbering [31], i.e. the first CDU weighted by 20, the
second by 21, the third by 22 and so on, providing us a way to uniquely identify the
positions of all matched items, by measuring the “height” of the weighted sum at each
moment in time.
In addition we can perform multiple searches within the same clock cycle; i.e. we
can connect to the CDUs multiple search elements, each encoding a different item and
search for all the items, technically, at the “same time”. Performing multiple searches does
not create a problem at the accumulator since the confirmation spikes from the CDUs
will arrive at different times, for different items, and thus spikes of the same item will be
accumulated separately from spikes of a different item.
Given that our implementation of logic and application to database searching is based
on sampling signals at an appropriate time the main limitation of our methods is: “how
finely can we ‘slice’ time?” For the implementation of logic gates, sampling rate will define
how fast we can perform a logic operation, with a higher sampling rate the smaller the
width of the square pulse can be and thus the shorter the time it takes to complete a
clock cycle (perform an operation). In practice we are limited by the technology used
to implement our method. For the search algorithm, the higher the sampling rate the
more items we can store within a given clock cycle. There is one more limitation to the
search algorithm, the fact that with increasing size of the database the weights for each
121
Figure 6-3. Schematic representation of the time based search method.
location increase exponentially, this would create a problem based on the technology used
to implement the method, but can be addressed by a multi-stage addressing system.
6.2 Neural Implementation
Our demonstrations with neural circuits are mainly inspired by the work of Abarbanel
et. al. [93, 94]. Specifically we take a neural circuit created by Abarbanel et. al. designed
for interspike interval recognition, and we extract from this circuit two compartments, the
Time Delay Unit (TDU) and the Detection Unit (DU) (which we rename to Coincidence
Detectino Unit (CDU)). We utilize these sub-circuits to manipulate the timing of action
potentials, both for their creation and detection.
6.2.1 Neural Models
This short section is a slight deviation from our main theme to very briefly introduce
two classic neuron models and three models of synaptic connections, which we use for
our algorithm implementations. This is purely done to place the reader in, at least, some
122
context of neuroscience; for further expositions we direct to work by Izhikevich [96–98] and
Latham et. al. [99, 100], among many others.
Type I neuron. This is a model in the framework of Hodgkin Huxley, a single
compartment model with fast sodium channel, a delayed rectifier potassium channel and a
leak channel [96, 98, 101] defined by the following equation:
CdV (t)
dt= IDC + gNam
3(t)h(t)(ENa − V (t)) + gKn4(t)(EK − V (t))
+gL(EL − V (t)) + IS(t), (6–1)
where V (t) is the membrane potential, IDC the external DC current, IS(t) the synaptic
current, and by defining r = Na, K, L to refer to the sodium, potassium and leak
channels, we have Er as the reversal potentials and gr as the conductance of each channel.
In our simulations we have used the following parameter values: C = 1 μFcm−2,
ENa = 50 mV, EK = -95 mV, EL = -64 mV, gNa = 215 mScm−2, gK = 43 mScm
−2,
gL = 0.813 mScm−2. Finally the gating variables X = {m,h, n} satisfy the following
equation: dX(t)dt= αX(V (t))(1−X(t))− βX(V (t))X(t), where αX and βX are given by:
αm =.32(13−(V (t)−V th))
e(13−(V (t)−V th))
4.0 −1, βm =
0.28((V (t)−V th)−40)
e((V (t)−V th)−40)
5 −1,
αh = .128e17−(V (t)−V th)
18 , βh =4
e40−(V (t)−V th)
5 +1,
αn =0.032(15−(V (t)−V th))
e(15−(V (t)−V th))
5 −1, βn =
0.5
e(V (t)−V th)−10
40
,
where Vth = −65 mV. In this model the frequency of spiking as function, which in the
context of our algorithm defines a clock cycle, is given by: f = C√IDC − I0, where I0 is
the threshold for spiking and C is a constant based on the model parameters.
Bistable neuron. Again this model derives from Hodgkin Huxley as a single
compartment model with a leak current, but the sodium channel is persistent and the
potassium channel fast, the defining equation is given by:
CdV (t)
dt= IDC + gNam∞(V (t))(ENa − V (t)) + gKn(t)(EK − V (t))
+gL(EL − V (t)) + +IS(t), (6–2)
123
where the gating variable n(t) is given by: dn(t)dt= n∞(V (t))−n(t)
τnand {m∞(V ), n∞(V )} = X∞(V )
by the activation function: X∞(V ) = 1/(1 + exp((VX − V )/kX)); the remaining variables
are the same as in the type I neuron model, see Equation 6–1. In our simulations we
have used the following parameter values: C = 1 μFcm−2, ENa = 60 mV, EK = -90 mV,
EL = -80 mV, gNa =20 mScm−2, gK = 10 mScm
−2, gL = 8 mScm−2, Vm = -20 mV,
Vn = -25 mV, km = 15 mV, kn = 5 mV, τn = 0.16 mV. This model mainly differs from
the type I neuron, as its resting state and oscillating state can both be stable and as a
result the system can switch from one state to the other simply by a variation in its state
variable (V (t)), unlike type I for which, for behavioural changes a system parameter needs
to be varied (IDC) [96, 98]; actually we utilize this difference in behaviour to construct the
TDU.
Synapses. We utilize in our neural circuits three types of synaptic connections and
their currents are given by:
• Excitatory AMPA: IS(t) = gASE(t)(EE − V (t)), where gA is the synaptic
conductance, EE = 0 mV is the reversal potential and SE(t) the gating variable, that
is biologically related to the fraction of bound glutamate receptors;
• Excitatory NMDA: IS(t) = gNB(V (t))SE(t)(EE − V (t)), where gN is the synaptic
conductance and B(V ) = 1.0/(1 + 0.288 exp(−0.062V )). EE and SE(t) are the samequantities as for the AMPA synapse;
• Inhibitory GABAA: IS(t) = gGSI(t)(EI − V (t)), where gG is the synaptic
conductance, EI = −75 mV is the reversal potential and SI(t) the gating variable,related to the fraction of bound gaba receptors.
The gating variables are given by: SX(t) = S0(θ(t))−SX(t)τ(S1−S0(θ(t)))
, where SX(t) = {SE(t), SI(t)},
S0(φ) = 0.5(1+tanh(120(φ−0.1))) is a sigmoidal function, and given the heaviside function
Θ(X) = 1 if X > 0, else Θ(X) = 0 we have θ(t) =∑iΘ(t− ti) ∙Θ((ti+ τR)− t) ∙Θ(X), here
ti is the time of the ith presynaptic neuronal spike. Finally we have two time constants
τR = τ(S1 − 1) and τD = τS1, in biological sense the time for the neurotransmitter to bind
and release, respectively; for the AMPA synapse τR = 0.1 ms and τD = 1.5 ms, for NMDA
τR = 2.5 ms and τD = 70 ms, and for GABAA τR = 1.12 ms and τD = 5.5 ms.
124
Time Delay Unit (TDU) and Coincidence Detection Unit (CDU). The Time
Delay Unit is constructed with one of each of the two neuron models we presented, a
type I neuron and a bistable neuron, our construction follows directions given in [93]. A
schematic of the circuit is given in the inset (a) of Figure 6-4. Neuron α is type I neuron
and neuron β is a bistable neuron and both neurons are initiated at a resting potential
state. The input connections to the neurons are a AMPA synapse to neuron β and a
NMDA synapse to neuron α. The neurons are also connected to each other through
a GABAA synapse, α −− β, and a NMDA synapse, β Ý α, it is this synapse that we
modulate to create a delay. As we have the circuit configured an incoming action potential
arriving at t0 is directed into both neurons. For neuron α, since it is a type I neuron,
a single incoming action potential has no effect other than to increase the membrane
potential slightly, but there will be no behavioural change. For neuron β though the
situation is different, as it is a bistable neuron an incoming action potential “pushes”
the neuron to its oscillating state, so neuron β starts generating action potentials. These
action potentials drive, through the connecting NMDA synapse, neuron α and eventually
will “push” the membrane potential of neuron α enough for it to fire its own “output”
action potential. The timing of this output potential depends on how long it takes for
neuron β to push neuron α over its firing threshold, and of course this time delay can be
modulated by a “stronger” or “weaker” connection between the two neurons. Specifically
the strength of the aforementioned NMDA synapse, β Ý α, can be varied using a weight
parameter R as gNR = R ∙ gN , with gN = 1 mScm−2. Therefore an incoming action
potential at time t0 will be technically “absorbed” by the system and an output action
potential will be generated at a later time t0 + τ(R); in essence though, since there is
no difference between the two action potentials, in effect what we achieve is the delay in
the propagation of a single action potential. Through simulations we have quantified the
effect of R on τ(R) and their relationship is presented in Figure 6-4. Of course we need
to make sure that only a single output potential is created, this is achieved through the
125
Figure 6-4. Time Delay Unit (TDU). The relationship between the parameter R, whichdefines synaptic strength, and the time delay created τ . Inset (a): A schematicrepresentation of the TDU neural circuit. Inset (b): Domain and range whereR vs. τ(R) is linear, and hence the range of R values used for flexible logicgate implementation. (Adapted from [93].)
inhibitory GABAA synapse between the two neurons, α −− β, which takes the “output”
spike of neuron α and directs it, besides to the output, back into neuron β inhibiting it
and pushing it away from its oscillating state back to its resting state, ceasing further
generation of action potentials from neuron β, and as a direct consequence prevents the
generation of further action potentials from neuron α itself.
The Coincidence Detection Unit is a much simpler case, it is simply a single type I
neuron that acts as a classic integrator receiving inputs from two sources through two
AMPA synapses. The strength of the conductance of the two synapses is equated so
that both input sources have equal effect on the neuron. In addition the strength is set
high enough for the neuron not to cross its firing threshold, unless it receives two spikes
126
simultaneously (or within tolerance of a small time interval, defined by the conductance of
the synapses).
6.2.2 Algorithm Implementations
The first neural based implementation of logic emulation is simply based on the idea
of sampling the membrane voltage signal of a single neuron at different moments in time.
The second method utilizes a circuit of two bi-directionally coupled Hodgkin Huxley
neurons, the TDU circuit. Logic gates are emulated by varying the synaptic strength R
that determines the time delay τ(R) and then observing the output at a predetermined
fixed time instant. The search algorithm is straightforwardly implemented along the
directions given in the more general context presentation of Section 6.1.2.
Varying the “moment of observation”. This is truly a very simple implementation.
We just consider a single neuron producing an action potential, see inset of Figure 6-5.
Since this signal is a product of simulation we consider each point generated by the
simulation as an instance in time and with the zeroth simulation point as a reference point
we define ttotal according to the computational operation to be performed, exactly as in
Section 6.1.1.
Take the NOR gate as a specific example. In Figure 6-5 we show the results of
sampling our signal at the time instances of 2555 for the operation NOR(0,0), 2605 for
NOR(0,1)/(1,0) and at 2655 for NOR(1,1). In terms of programming (tprog) and input
time lengths (tin) we have tprog = 2555 time steps, specific to the NOR gate, and tin = 50
time steps. A small deviation from the general case is that the action potential actually
fluctuates between 48 mV and -92 mV, it is not simply 0 or 1. Therefore we set a voltage
threshold, at -45 mV, over which we define the logical 1 and under which the logical 0, at
the output.
As is clear from Figure 6-5 the appropriate Output values are obtained, that is the
voltage is over -45 mV for the (0,0) case and below for the other two cases. The whole
procedure can be interpreted as follows: to perform the logic gate NOR we require to
127
Figure 6-5. Demonstration of operating a NOR gate, with a single neuron using differentsampling times. Depending on the inputs a different sampling moment isdefined beyond the 2555th moment that defines the NOR gate. Each possiblecase is shown with a circle, black circle ( ) indicating a 1 at the Output,empty circle (#) a 0. The Output is defined by a voltage threshold at -45 mV,shown. Inset: The whole action potential considered.
Table 6-3. Appropriate time sample instances, based on simulation points, to performeach of the five gates considered. In brackets the voltage of the action potential,at each point, is given.
I1 I2 AND NAND OR NOR XOR0 0 2405(-57 mV) 2505 (-9 mV) 2455(-56 mV) 2555 (1 mV) 2485(-52 mV)1/0 0/1 2455(-56 mV) 2555 (1 mV) 2505 (-9 mV) 2605(-86 mV) 2535 (34 mV)1 1 2505 (-9 mV) 2605(-86 mV) 2555 (1 mV) 2655(-86 mV) 2585(-50 mV)
wait 2555 time steps as a programming time length and then another 50 time steps for
each occurrence of a single logical input of 1. Using the same system, that is a single
Hodgkin Huxley neuron producing an action potential at the same rate, time instances
for reading an Output, with respect to the zeroth simulation point, along with their
resulting respective voltage value, for accomplishing the five fundamental gates are given
in Table 6-3. For confirmation we refer back to the gate truth table, Table 6-1.
128
Varying the moment of generation. Converse to the previous implementation,
in this implementation we fix the moment of observation, in the specific example at
t∗ = 425 ms from an initiating action potential at t0, and we vary the moment of actual
generation of the signal. We use “actual time” instead of simulation points to emphasise
the independence between the moment of observation and the computational operation.
To have the ability to generate an action potential at different times, with respect to
t∗, we use a TDU as part of our computational system. Following Figure 6-4, the TDU
receives an input action potential at t0 and produces a delayed output action potential
at t0 + τ(R), it is the moment of generation of this output action potential we use for
computation. Exactly as explained before, with respect to time instances, we utilize R
values taken from the relationship τ(R), and with Rtotal = Rprog + RI1 + RI2 we represent
a desired computational operation. Note that we use the domain 1.68 < R < 1.72 where
the curve of Figure 6-4 is linear, see inset (b) of same figure, so that the changes in R are
linearly proportional to the changes in τ(R) (actual time) and hence changes for a logical
1 at the input are independent of whether it is at Input1 or Input2; also note that R and
τ(R) are inversely proportional, that is higher R means earlier action potential generation.
Finally again as in the above implementation we define a voltage threshold at -45 mV for
distinction of the Output between 1 and 0.
Obtaining the result at the “moment of observation”, and actually implementing
neurally a sampling scheme, can be realized with a Coincidence Detection Unit, that is a
neuron that itself will fire when two neurons connected to it fire simultaneously, so one of
these two neurons can be the one “acting” as our gate, having its settings (R) changed
according to the programming and inputs, and the other neuron be fixed to fire solely at a
specific moment, “acting” as the “reader”.
As an explicit example, see Figure 6-6, consider the universal gate NOR with
Rprog = 1.705. For the NOR gate and the case of inputs (0,0) Rtotal = 1.705 + 0 + 0; with
this R value the action potential is generated with its peak at 424.5 ms after the initiating
129
Figure 6-6. Demonstration of operating a NOR gate, with a neural circuit using differentdelay times. All three cases of inputs are shown, in actuality only one actionpotential is generated by the system. The time of generation of the actionpotential is inversely proportional to R. Given the “moment of observation” at425 ms, the Output is obtained using the voltage threshold at -45 mV. Onceagain is 1 at the Output and # 0.
action potential, which results in a voltage of -16 mV at the moment of observation of
425 ms, which is over the threshold of -45 mV, thus the output is 1. For the cases (1,0)
and (0,1), which are degenerate, we have Rtotal = 1.705 + 0.005 + 0. The action potential is
generated with a peak at 424 ms, so by the time of the moment of observation the voltage
drops, below the threshold, to -89 mV, producing a 0 at the output. We have a similar
result for the third and final case of (1,1) at the inputs, Rtotal = 1.705 + 0.005 + 0.005,
the peak of the action potential happens at 423.5 ms, so again the voltage drops below
the threshold by the time of the moment of observation. It is clear that our system works
perfectly to reproduce the logic gate NOR, similar results have been obtained for the other
of the four fundamental gates, the R values, and the respective voltages at the moment of
observation, are given in Table 6-4; which once again can be compared with Table 6-1.
130
Table 6-4. Appropriate R values so time shift an action potential in order to perform eachof the five gates considered. The action potential is sampled always at the samemoment in time, 425 ms after an initiating action potential. In brackets thevoltage of the action potential, at 425 ms, is given.
I1 I2 AND NAND OR NOR XOR0 0 1.690(-58 mV) 1.700 (48 mV) 1.695(-56 mV) 1.705(-16 mV) 1.697(-53 mV)1/0 1/0 1.695(-56 mV) 1.705(-16 mV) 1.700 (48 mV) 1.710(-89 mV) 1.702 (35 mV)1 1 1.700 (48 mV) 1.710(-89 mV) 1.705(-16 mV) 1.715(-85 mV) 1.707(-53 mV)
Neural search algorithm. The search method can also be implemented with the
same neuronal circuits, TDU and CDU along with an Integrating Unit (IU), simply
a single neuron with its threshold for firing an action potential set very high. The
first component of our search engine is encoding by each unique moment in time a
database element. As we have shown above, items from any given alphabet can be
represented by time intervals. We do the same, but instead of using actual time intervals
for representation of items we use unique R values. So a database of size N , regardless of
size, can be constructed using N TDUs each one representing one of the M distinct items
of the alphabet associated with a unique R value and as a result a unique instance in time
for generating an action potential.
The second component, just as in the abstracted explanation above, is a Coincidence
Detector Unit (CDU), this is a single neuron at resting potential. The property of this
neuron that makes our search engine work is that it requires two action potentials to
arrive at its membrane simultaneously for the neuron to produce a response action
potential. If and when, two action potentials coincide the CDU will generate its own
action potential, otherwise it will not fire, providing us a way to confirm the existence or
not of a searched for item in the database.
So now we can incorporate in our model the Search Engine elements, TDUs set at the
R values for the items we wish to search for. We couple each CDU with each TDU of the
database and to all the Search Engine elements, referring back to Figure 6-3, so that when
the Search Engine elements fire, the CDUs that are attached to the searched for items will
131
receive two simultaneous action potentials and fire themselves. If a searched for item does
not exist in the database no CDU will receive two simultaneous spikes so there will be no
action potential from the CDU.
Finally we connect all CDUs to an Intergrading Unit (IU) to combine all search
results. Each CDU is connected to the IU with a synapse of different conductance weight
(g), following Godel numbering; the first CDU is connected with a weight 20g the second
with weight 21g, third unit with 22g and so on. This provides us with a way to uniquely
mark each position of each item in the database. At the output we read the voltage of the
IU and interpret the searched for results.
There are two parameters to consider at the reading of the result, first is the time
instance at which a voltage increase occurs, which represents the searched for item, and
second is the amount by which the voltage of the IU is increased. Basically, each moment
in time represents an information item, as defined from the different R values we use to
encode them, and each increase in voltage represents a position in the database given by
the Godel numbering.
6.3 Electronic Implementation
As it turns out the electronic implementation of this method can be performed with
simple multipliers instead of a complicated sampling circuit of variable delays. Basically
we generate two signals, one representing the “generic signal”, as in Section 6.1.1, and
the other signal is generated at the time instance that represents the computational
operation to be performed, whether logic operation or search operation. The multiplier
simply multiplies the two signals and the product of course will be 1 only if the two signals
coincide in time, in essence we are emulating the neural Coincidence Detection Unit.
Logic gates algorithm. As shown in Figure 6-7 we connect the reference signal
square pulse generator (Vr) to a multiplier (U1) along with the varying “sampling
signal” generator (Vs). The varying sampling signal generator emulates the moment
of observation by producing a short square pulse signal with varying delay compared to
132
Figure 6-7. Schematic representation of an electronic circuit for logic using time. Vrprovides the reference signal while Vs the “moment of observation”. The twosignals switch from 1 V to 0 V, therefore U1 through simple multiplicationprovides the Output, given straightforwardly by the voltage of resistor R1 (1 Vlogical 1, 0 V logical 0).
the reference square pulse of Vr. The function of the multiplier U1 is to simply multiply
the two signals and therefore the resulting signal is the same as if sampling the reference
square pulse Vr at the moment of observation.
For our implementation we used a reference signal of frequency 10 Hz and pulse width
30 ms, shown in Figure 6-8(a), as a result our time length representing a logical 1 is of
30 ms(=tin). The “sampling signal” from Vs is again a pulse of frequency 10 Hz, but its
generation instance can be varied creating a variable delay signal, see Figures 6-8(b)(d)(f).
The time delay of generation of the “sampling signal” is our programming time length plus
the appropriate delay for each of the three pairs of inputs, values for programming the five
fundamental gates and each case of inputs are given in Table 6-5.
133
Figure 6-8. Demonstration of operating a NOR gate using an electronic circuit utilizingtime dependant computation. (a) Reference signal, to be sampled at differenttimes. Refer back to Figure 6-2(f). (b) Sampling signal for the case of inputs(0,0) to a NOR gate, equivalent to tprog + 0 + 0 = 60 ms. (c) Output signal forperforming NOR(0,0), output is a logical 1, signal is “high”. (d) Samplingsignal for the cases of inputs (1,0)/(0,1) to a NOR gate, equivalent totprog + tin + 0 = 90 ms. (e) Output signal for performing NOR(1,0) orNOR(0,1), output is a logical 0, signal is “low”. (f) Sampling signal for thecase of inputs (1,1) to a NOR gate, equivalent to tprog + tin + tin = 120 ms.(g) Output signal for performing NOR(1,1), output is a logical 0.
Table 6-5. Delay times for Vs, the moment of the “sampling signal” with respect to thestart of the clock cycle, to implement each of the five gates considered.
I1 I2 AND NAND OR NOR XOR0 0 0 ms 40 ms 10 ms 60 ms 30 ms1/0 0/1 30 ms 70 ms 40 ms 90 ms 60 ms1 1 60 ms 100 ms 70 ms 120 ms 90 ms
134
As a specific illustrative example take the NOR gate. As indicated in Table 6-5
the programming time length for the NOR gate is 60 ms, and as stated above the time
length representing a logical 1 at the inputs is 30 ms. Therefore for the set of inputs
(0,0), tprog + 0 + 0, we generate a “sampling signal” with simply a delay of 60 ms, see
Figure 6-8(b). For the sets of inputs (1,0) and (0,1), tprog+ tin+0, we generate a “sampling
signal” with delay 90 ms, see Figure 6-8(d); and finally for the set (1,1), tprog + tin + tin, we
generate a “sampling signal” with delay 120 ms, see Figure 6-8(f). The output is read at
the resistor R1 and is shown in Figures 6-8(c)(e)(g), as VOut. From Figures 6-8(c)(e)(g)
we can see that the correct Output is generated for each case of inputs for reproducing the
truth table of a NOR gate.
Search algorithm. For the search method, again our electronic implementation is
greatly simplified with the use of multipliers, U1, U2, U3, simply in place of Coincidence
Detection Units, see Figure 6-9 for the specific schematic. As a specific example consider
encoding the numbers {1, 2, 3} in three database elements Ri, Rj, Rk, using m ∙Δt, where
Δt = 10 ms and m ∈ N, representing any abstract item. In the search elements RS1,
RS2, RS3, we have encoded our search for numbers, specifically all three {1, 2, 3}. The
the accumulated signal from the search elements is fed independently to each multiplier,
along with the signal of each multiplier‘s associated database element. The result from
each multiplier is a signal of a square pulse if the searched for item exists or a zero signal
if it does not exist. For the final step each multiplier is connected to an accumulator
sub-circuit through resistors, R7, R8, R9 whose actual resistance follows Godel numbering.
The readout is set at the output of the accumulator that collects the signals from the
multipliers, at U5.
As seen in Figure 6-10(b) all three items are identified as existing in the given
database, the timely spikes at the output, and their location in the database is given by
the voltage height of their respective spike. Specifically we see in Figure 6-10(b) three
spikes, the first spike at 10 ms represents the occurrence of item given the number 1 in
135
Figure 6-9. Schematic representation of an electronic circuit for the time dependantsearch method. The R-units are signal generators, Rx being the generatorsused for encoding the database, RSi being the generators used for encodingsearched for items. The U-units are multipliers, replacing the CDUs of Figure6-3. R7, R8, R9 are the resistors associated with each database elementdefining its location through their resistance value. The dash-boxedsub-circuits are accumulator circuits (relating to IUs) “putting togther” both a“search signal”, in the case of multiple simultaneous searches, and the resultsignal from the multipliers.
the database and the height of 1 V represents the fact that this item is stored in the
first location of the database. Analogously for the other two spikes representing items
numbered 2 and 3, at time instances of 20 ms and 30 ms respectively, again the height
of each spike signifies the location of the item; 2 V for 2nd position and 4 V for the 3rd
position in the database.
6.4 Discussion
We have presented two algorithms that use time intervals for computation; the first
for performing flexible logic and the second for searching a unsorted database. The central
136
Figure 6-10. Demonstration of performing a search using an electronic circuit utilizingtime dependant computation. Refereing to the schematic of Figure 6-9: (a)The output of the search key accumulator at U6, this is the “search signal”from all the search elements that is fed to all multipliers; (b) The output ofthe result accumulator U5, confirming the existence of items items {1, 2, 3}with pulses at 10 ms, 20 ms, and 30 ms, respectively; also providing thelocation of each item in the database, through the voltage of each pulse, 1 V,2 V, 4 V, respectively.
idea of the proposed algorithms is that each, and any, moment in time can be utilized
to represent a computational operation and/or data. With the use of a defined clock
cycle length we segment time into manageable pieces, within which we define moments in
time that will produce different responses based on a variable quantity. In our proposed
algorithms the responses are of two types, either the crossing of a threshold or the
coincidence of events, both of which can be interpreted as computational operations.
With careful construction of a signal, which encapsulates the five fundamental logic
gates, we have shown how time intervals can be used to represent the programming of a
universal flexible logic gate. In addition we have shown how a zero length time interval
can represent a logical 0 and a finite length time interval a logical 1. A straightforward
137
combination (summation) of the programming time length and the logical inputs time
length can provide a moment in time at which sampling the signal will produce the desired
output. We can thus visualize an array of many computational elements each receiving a
common signal. Each of these elements is programmed to sample the signal at different
times producing, in parallel, the results of different logical operations. Actually there is
no reason to confine ourselves to the “simple” signal presented here, Figure 6-2(f). The
method can be generalized to sampling more “complicated” signals, reproducing even
more complicated operations at different times; in the spirit of Chapter 5.
Furthermore, we have shown how many different time intervals can be utilized in an
algorithm to perform searches within a given database. We have shown how each natural
number can be represented by a time interval, defined between the “tick” of a universal
clock (clock cycle) and a delta function. Within one clock cycle we search the database by
looking for coincidences between our search for elements and the elements encoded in the
database. It is significant to note that our method completes a search within one single
clock cycle regardless of the size of the database; and it is not confined to searching for
single items sequentially, but multiple searches can be performed within the same clock
cycle.
For both algorithms presented we provided realizations, both with neural and
electronic circuits. As we have shown, the implementations are straightforward, especially
in the case of electronic circuits. The presented methods are a direct consequence of
considering delayed time responses as computational commands. For both implementations
the main limitation is the smallest timescale of the system; a consequence of considering
time as computational data and programming, the smaller the time interval we can utilize,
the more data and operations can be represented.
138
CHAPTER 7CONCLUSION
Chaotic Computation has come a long way since its first step in 1998 [25]. Both
within the “original” group of Ditto et. al. [102, 103], and even now days within the
more the more extended group of collaborators [104, 105], theoretical and experimental
advancements have established Chaotic Computation as a realistic possibility for the
future of computation. We are glad to report that the field has also attracted the
attention of researchers beyond our group [106, 107].
Even further, the aim of bringing research out of the laboratory and into the public
domain is actively pursued; Chaologix Inc. (www.chaologix.com) is an active company
with the sole aim to realize the technology and present the solution to the public. In
accomplishing this goal the main attempts circle around conventional CMOS based VLSI
circuitry [108], but as we emphasised Universality allows us to consider even more esoteric
approaches such as magneto-based circuitry [109], or even high speed chaotic photonic
integrated circuits [110].
The aims of this dissertation were threefold. Obviously foremost, expose the reader
to the whole field of Chaotic Computation; in addition to present the material with its
natural multidisciplinary nature, from the underlining mathematics, to the physics that
governs our systems, the inspiration from biology and the engineered implementations;
note our attempt was to expose the reader to details from each field, but at the same
time without deviating from the main aim. Finally and of equal importance, we aimed at
establishing where the “cutting edge” of research of the field is, prepare the groundwork
for our future work.
In closing we wish to point out projections of the future paths emanating from
each chapter. Developing further the connections implied in chapter one is essential
in determining the ultimate limitations of our research and “bootstraps” directly
to where chapter two left of; specifically the further extension of the Deutsch-Jozsa
139
algorithm, in the spirit of how quantum computing progressed, into algorithms for more
specific functions, like Fourier Transform and integer factorization, and even further.
The importance of searching, matching, pattern recognition in our current general
technological aims, drives us to consider promoting further our method for searching;
a sorting algorithm should probably be the most immediate target, with issues of
multidimensional data, and perhaps extrapolated action, or decision, taking not the
too far of future. While all the aforementioned could be considered mainly theoretical
milestones, the implementation side can not be ignored. Faster, more efficient, smaller,
we share all goals that current established computer hardware technology has, simpler
electronic, or even alternative, realizations will be pursued. Finally, and in some sense
coming full circle back to the fundamental mathematics, through chapters five and six we
have setup the field for investigating further how to obtain independence from actual exact
state values, and/or even the utilization of only specific physical quantities, an attempt
perhaps to make more tangible the abstract concept of “computation”.
140
REFERENCES
[1] G. E. Moore, Electronics 38, 114 (1965).
[2] R. Landauer, IBM Journal of Research and Development 5, 183 (1961).
[3] W. Arden, et al., International technology roadmap for semiconductors, 2008 update,overview, Tech. rep., International Roadmap Committee (2008).
[4] M. D. Austin, et al., Nanotechnology 16, 1058 (2005).
[5] R. P. Feynman, R. W. Allen, T. Hey, Feynman Lectures on Computation (PerseusBooks Group, 2000).
[6] M. A. Nielsen, I. L. Chuang, Quantum Computation and Quantum Information(Cambridge University Press, 2000).
[7] R. Edwards, H. T. Siegelmann, K. Aziza, L. Glass, Chaos: An InterdisciplinaryJournal of Nonlinear Science 11, 160 (2001).
[8] M. N. Stojanovic, D. Stefanovic, Nature Biotechnology 21, 1069 (2003).
[9] M. N. Stojanovic, D. Stefanovic, Journal of American Chemical Society 125, 6673(2003).
[10] J. Hasty, F. Isaacs, M. Dolnik, D. McMillen, J. J. J. Collins, Chaos: An Interdisci-plinary Journal of Nonlinear Science 11, 207 (2001).
[11] S. H. Strogatz, Nonlinear Dynamics and Chaos: With Applications to Physics,Biology, Chemistry and Engineering (Perseus Books Group, 2001).
[12] W. Ditto, T. Munakata, Communications of the ACM 38, 96 (1995).
[13] E. Ott, C. Grebogi, J. A. Yorke, Physical Review Letters 64, 1196 (1990).
[14] M. Ding, et al., Physical Review E (Statistical Physics, Plasmas, Fluids, and RelatedInterdisciplinary Topics) 53, 4334 (1996).
[15] M. Ding, et al., Chaos: An Interdisciplinary Journal of Nonlinear Science 7, 644(1997).
[16] N. Margolus, Physica D: Nonlinear Phenomena 10, 81 (1984).
[17] T. Toffoli, N. Margolus, Cellular automata machines: a new environment formodeling (MIT press, 1987).
[18] J. P. Crutchfield, K. Young, Physical Review Letters 63, 105 (1989).
[19] T. Toffoli, N. Margolus, Physica D: Nonlinear Phenomena 47, 263 (1991).
[20] A. V. Holden, J. V. Tucker, H. Zhang, M. J. Poole, Chaos 2, p367 (1992 (Jul)).
141
[21] J. P. Crutchfield, Physica D: Nonlinear Phenomena 75, 11 (1994).
[22] A. Krikelis, C. Weems, Computer 27, 12 (1994).
[23] Agota Toth, K. Showalter, The Journal of Chemical Physics 103, 2058 (1995).
[24] C. Moore, Physical Review Letters 64, 2354 (1990).
[25] S. Sinha, W. L. Ditto, Physical Review Letters 81, 2156 (1998).
[26] S. Sinha, W. L. Ditto, Physical Review E (Statistical Physics, Plasmas, Fluids, andRelated Interdisciplinary Topics) 60, 363 (1999).
[27] S. Sinha, T. Munakata, W. L. Ditto, Physical Review E (Statistical, Nonlinear, andSoft Matter Physics) 65, 036214 (2002).
[28] A. Miliotis, S. Sinha, W. L. Ditto, International Journal of Bifurcation and Chaos18, 1551 (2008).
[29] M. J. Feigenbaum, Los Alamos Theoretical Division Annual Report pp. 98–102 (July1975 - 1977 Sep.).
[30] A. N. Whitehead, B. Russell, Principia Mathematica to *56 (1910-1913) (CambridgeUniversity Press, 1962).
[31] K. Godel, On formally undecidable propositions of principia mathematica and relatedsystems (1931) (Courier Dover Publications, 1992).
[32] S. Sinha, T. Munakata, W. L. Ditto, Physical Review E (Statistical, Nonlinear, andSoft Matter Physics) 65, 036216 (2002).
[33] K. Murali, A. Miliotis, W. L. Ditto, S. Sinha, Physics Letters A In Press, Cor-rected Proof, (2009).
[34] A. Miliotis, K. Murali, S. Sinha, W. L. Ditto, M. L. Spano, A simple nonlineardynamical computing device (2009). To appear (Chaos, Solitons and Fractals, 2009).
[35] J. Gleick, Chaos: Making a New Science (Penguin (Non-Classics), 1988).
[36] R. L. Devaney, An Introduction to Chaotic Dynamical Systems, 2nd Edition(Westview Pr (Short Disc), 2003).
[37] H. O. Peitgen, H. Jurgens, D. Saupe, Chaos and Fractals: New Frontiers of Science(Springer, 1993).
[38] E. Ott, Chaos in Dynamical Systems (Cambridge University Press, Cambridge,2002).
[39] J. Moehlis, K. Josic, E. T. Shea-Brown, Scholarpedia 1, 1358 (2006).
[40] B. B. Mandelbrot, The Fractal Geometry of Nature (W. H. Freeman, 1982).
142
[41] K. G. Wilson, Physical Review B 4, 3174+ (1971).
[42] K. G. Wilson, Scientific American (1979).
[43] J. Banks, J. Brooks, G. Cairns, G. Davis, P. Stacey, American Mathematical Monthly99, 332 (1992).
[44] T. Li, J. A. Yorke, American Mathematical Monthly 82, 985 (1975).
[45] N. A. Sharkovsky, Scholarpedia 3, 1680 (2008).
[46] N. A. Sharkovsky, International Journal of Bifurcation and Chaos 5, 1263 (1995).
[47] P. Stefan, Communications in Mathematical Physics 54, 237 (1977).
[48] P. Saha, S. H. Strogatz, American Mathematical Monthly 68, 42 (1995).
[49] J. Bechhoefer, American Mathematical Monthly 69, 115 (1996).
[50] W. B. Gordon, American Mathematical Monthly 69, 118 (1996).
[51] M. J. Feigenbaum, Journal of Statistical Physics 19, 25 (1978).
[52] M. J. Feigenbaum, Physica D: Nonlinear Phenomena 7, 16 (1983).
[53] P. Cvitanovic, Universality in chaos (Bristol, Adam Hilger, Ltd., 1984).
[54] W. L. Ditto, S. N. Rauseo, M. L. Spano, Physical Review Letters 65, 3211 (1990).
[55] W. L. Ditto, K. Showalter, Chaos: An Interdisciplinary Journal of Nonlinear Science7, 509 (1997).
[56] V. In, W. L. Ditto, M. L. Spano, Physical Review E (Statistical Physics, Plasmas,Fluids, and Related Interdisciplinary Topics) 51, R2689 (1995).
[57] K. Murali, S. Sinha, Physical Review E (Statistical, Nonlinear, and Soft MatterPhysics) 68, 016210 (2003).
[58] T. Shinbrot, et al., Physical Review Letters 68, 2863 (1992).
[59] L. Glass, W. Zeng, International Journal of Bifurcation and Chaos 4, 1061 (1994).
[60] D. J. Christini, V. In, M. L. Spano, W. L. Ditto, J. J. Collins, Physical Review E(Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics) 56,R3749 (1997).
[61] V. In, et al., Chaos: An Interdisciplinary Journal of Nonlinear Science 7, 605(1997).
[62] C. Wagner, R. Stoop, Physical Review E (Statistical, Nonlinear, and Soft MatterPhysics) 63, 017201 (2000).
143
[63] S. Sinha, W. L. Ditto, Physical Review E (Statistical, Nonlinear, and Soft MatterPhysics) 63, 056209 (2001).
[64] S. Sinha, Physical Review E (Statistical, Nonlinear, and Soft Matter Physics) 63,036212 (2001).
[65] S. Sinha, Physical Review E (Statistical, Nonlinear, and Soft Matter Physics) 66,016209 (2002).
[66] S. Sinha, Physical Review E (Statistical Physics, Plasmas, Fluids, and RelatedInterdisciplinary Topics) 49, 4832 (1994).
[67] P. Bak, C. Tang, K. Wiesenfeld, Physical Review A 38, 364 (1988).
[68] P. Bak, C. Tang, K. Wiesenfeld, Physical Review Letters 59, 381 (1987).
[69] K. Murali, S. Sinha, W. L. Ditto, International Journal of Bifurcation and Chaos13, 2669 (2002).
[70] K. Murali, S. Sinha, W. L. Ditto, Physical Review E (Statistical, Nonlinear, and SoftMatter Physics) 68, 016205 (2003).
[71] K. Murali, S. Sinha, W. L. Ditto, Pramana 64, 433 (2005).
[72] U. Hubner, N. B. Abraham, C. O. Weiss, Physical Review A 40, 6354 (1989).
[73] C. O. Weiss, et al., Applied Physics B: Lasers and Optics 61, 223 (1995).
[74] K. Murali, S. Sinha, Physical Review E (Rapid Communications) 75, 025201 (4)(2007).
[75] K. Murali, S. Sinha, W. L. Ditto, A. R. Bulsara, Reliable logic circuit elements thatexploit nonlinearity in the presence of a noise floor (2009). To appear (PhysicalReview Letters, 2009).
[76] T. Munakata, S. Sinha, W. L. Ditto, IEEE Transactions in Circuits and Systems 49,1629 (2002).
[77] D. R. Chialvo, A. V. Apkarian, Journal of Statistical Physics 79, 375 (1993).
[78] D. Deutsch, R. Jozsa, Proceedings: Mathematical and Physical Sciences 439, 553(1992).
[79] A. Ekert, R. Jozsa, Reviews of Modern Physics 68, 733 (1996).
[80] L. K. Grover, Physical Review Letters 79, 325 (1997).
[81] B. S. Prusha, J. F. Lindner, Physics Letters A 263, 105 (1999).
[82] N. J. Corron, S. D. Pethel, B. A. Hopper, Physical Review Letters 84, 3835 (2000).
144
[83] E. R. Hunt, Physical Review Letters 67, 1953 (1991).
[84] K. Myneni, T. Barr, N. Corron, S. Pethel, Physical Review Letters 83, 2175 (1999).
[85] K. Chlouverakis, M. Adams, Electronics Letters 41, 359 (2005).
[86] T. Morie, S. Sakabayashi, M. Nagata, A. Iwata, Circuits and Systems I: Fundamen-tal Theory and Applications, IEEE Transactions on 47, 1652 (2000).
[87] K. Murakoshi, T. Morie, M. Nagata, A. Iwata, Design Automation Conference, 2000.Proceedings of the ASP-DAC 2000. Asia and South Pacific pp. 23–24 (2000).
[88] J. Garcia-Ojalvo, R. Roy, IEEE Transactions on Circuits and Systems I: Fundamen-tal Theory and Applications 48, 1491 (2001).
[89] T. Yang, L. O. Chua, International Journal of Bifurcation and Chaos 10, p1091(20000501).
[90] L. Chua, J. Yu, Y. Yu, Circuits and Systems, IEEE Transactions on 32, 46 (1985).
[91] M. M. Mano, Computer system architecture (Prentice-Hall, Inc. Upper Saddle River,NJ, USA, 1993).
[92] T. C. Bartee, Computer Architecture and Logic Design (McGraw-Hill, 1991).
[93] S. S. Talathi, H. D. I. Abarbanel, W. L. Ditto, Physical Review E (Statistical,Nonlinear, and Soft Matter Physics) 78, 031918 (17) (2008).
[94] H. Abarbanel, S. Talathi, Physical Review Letters 96, 148104 (4) (2006).
[95] C. N. Zhang, M. Zhao, M. Wang, IEEE Transactions on neural networks 11, 739(2000).
[96] E. M. Izhikevich, International Journal of Bifurcation and Chaos 10, 1171 (2000).
[97] E. Izhikevich, Neural Networks, IEEE Transactions on 15, 1063 (2004).
[98] E. Izhikevich, Dynamical systems in neuroscience: The geometry of excitability andbursting (MIT press, 2007).
[99] P. E. Latham, B. J. Richmond, P. G. Nelson, S. Nirenberg, Journal of Neurophysiol-ogy 83, 808 (2000).
[100] P. E. Latham, B. J. Richmond, S. Nirenberg, P. G. Nelson, Journal of Neurophysiol-ogy 83, 828 (2000).
[101] B. Ermentrout, Neural Computation 8, 979 (1996).
[102] K. Murali, S. Sinha, I. R. Mohamed, Physics Letters A 339, 39 (2005).
145
[103] S. Sinha, W. L. Ditto, Philosophical Transactions of the Royal Society A: Mathemat-ical, Physical and Engineering Sciences 364, 2483 (2006).
[104] M. R. Jahed-Motlagh, B. Kia, W. Ditto, S. Sinha, International Journal of Bifurca-tion and Chaos 17, 1955 (2007).
[105] H. R. Pourshaghaghi, B. Kia, W. Ditto, M. R. Jahed-Motlagh, Chaos, Solitons andFractals In Press, Corrected Proof, (2008).
[106] H. Leung, H. Yu, K. Murali, Physical Review E (Statistical, Nonlinear, and SoftMatter Physics) 66, 036203 (2002).
[107] D. Cafagna, G. Grassi, International Journal of Bifurcation & Chaos 16, p1521(2006 (May)).
[108] W. L. Ditto, K. Murali, S. Sinha, Philosophical Transactions of the Royal Society A:Mathematical, Physical and Engineering Sciences 366, 653 (2008).
[109] R. Koch, Scientific American 293, p56 (2005 (Aug)).
[110] M. Yousefi, et al., Physical Review Letters 98, 044101 (2007).
146
BIOGRAPHICAL SKETCH
Abraham Miliotis was born in Annapolis, Maryland, USA, from parents of Greek-Cypriot
descent. He was raised in Cyprus where he attended The English School and gained
admission to Imperial College, London in 1997. In 2000 he graduated from Imperial
College with a Bachelor of Science in Physics with Theoretical Physics. Under the
supervision of professor Dr. Dimitri Vvedensky, he completed his final year thesis,
“Stock market crashes treated as critical phenomena”. Dr. D. Vvedensky also introduced
Abraham to the work of professor Dr. William Ditto on using dynamical systems for
computation, whom he joined at Georgia Institute of Technology, Atlanta, Georgia in
2000, as a graduate student in the Physics department. Between 2001 and 2003 though,
Abraham paused his studies to pursue some entrepreneurial opportunities. He moved
back to Cyprus where, among other projects, he was involved in the creation of Megaland
Computers LTD, an IT retail sales and small-office/home-office support company, and also
the setting up of Game-Spot Internet/Gaming Cafes.
In 2003 he rejoined Dr. William Ditto at the Biomedical Engineering department
of University of Florida, where he completed his Ph.D. in May of 2009 on “Chaotic
Computation”.
147