dynamic problems and learning systems to solve them

Dynamic Problems and Learning Systems toSolve Them

Pablo [email protected]

College of Engineering, University of the Andes, Chile

2008

Zegers Dynamic Problems and Learning Systems to Solve Them

IntroductionDefinition

I Many dynamical problems can be represented by the followingsystem of linear differential equations

z = Az + Bx

y = Cz + Bx

I Many more can be expressed by the nonlinear system

z = f (z, x)

y = g (z, x)


IntroductionThe Problem

I Given the richness of the nonlinear differential equations family, wewould like to be able to train neural networks capable of displayingthe same amount of richness.

I The differential equation model is interesting, among other reasons,because it has the states z that summarize the past (triggerproblem).


IntroductionSubproblems

I Emphasis on the input

I Memorize xI Compress xI Reconstruct x

I Emphasis on the output

I Forecast next future values of x. In this case y should resemblefuture of the input.

I Classification x into one of many classes. in this case yconverges to one of many attractor basins.

I Inverse problems such as inverse kinematics.I Map x into y or as it is normally called: system identification.

I Emphasis on the objective function

I Produce y that minimizes a functional such as in controlproblems.


IntroductionSubproblems (cont.)

I Unlimited trajectories.

I Limited Trajectories.


IntroductionSolution Approaches

I First we will try evolutionary approaches that do not require anyprevious knowledge.

I Then we will use everything we know about static neural networks,i.e. neural networks that do not have feedback, because they areeasier to train.

I We will finally use fully connected recurrent neural networks whosebasic unit is a neuron capable of implementing a differentialequation.

1

τixi (t) = −xi (t) +

n∑j=1

wij fj (xj (t))

with i ∈ [1, . . . , n]


IntroductionSolution Approaches (cont.)

RNNs are interesting because they produce extremely compact solution.However, they are difficult to train.

Taken from Siegelmann, 1997.


Evolutionary ApproachesRandom Recurrent Neural Networks Produce Limit Cycles

I Randomly generated recurrent neural networks spontaneouslyproduce limit cycles.

I It may be possible that to procude interesting dynamicbehavior is not that difficult!

Dayhoff, Palmadesso, Richards, and Lin, “Patterns of Dynamic Activity and Timing in Neural NetworkProcessing”, Neural Networks and Pattern Recognition, edited by Omidvar and Dayhoff, Academic Press, 1998.


Evolutionary ApproachesNEAT

I Start small,

I Optimize in this manageable space,

I Use different exisiting solutions to generate new ones,

I Systematically elaborate on existing structures by adding newdimensions as necessary, and

I Make them compete.Stanley and Miikkulainen, “Competitive Coevolution through Evolutionary Complexification”, Journal of ArtificialIntelligence Research, vol. 21, pp. 63-100, 2004.


Evolutionary ApproachesNEAT (cont.)

Taken from Stanley and Miikkulainen, 2004.



I Innovation numbers tell wichgenes match up withouttopological analysis. Thismechanism allows to preserveexisting functionality.

I Only individuals within aniche (species) compete.This allows new architecturessurvive even if they had notexpressed their potential.




The robot Duel:

I Robots start facing away.

I Each robots has sensors to detectfood and the opponent position.

I They also have sensors that tellenergy difference and where the wallsare.

I Sandwiches represent energy.

I If the robots collide, the one withmore energy wins.

I Complex interaction betweenforaging, pursuit, and evasionbehaviors.




Best complexifying network:

I 11 hidden units, 202connections.

I Lateral and recurrentconnections.

I Very complicated to findthrough fixed-topologyevolution or simplification.



Static Neural Networks for Dynamical ProblemsDelays (Sejnowski and Rosenberg, 1987)


Static Neural Networks for Dynamical ProblemsTime Delay Neural Network (Lang and Hinton, 1988)


Static Neural Networks for Dynamical ProblemsTime Delay Neural Network (Lang and Hinton, 1988) (cont.)


Static Neural Networks for Dynamical ProblemsNonlinear Autoregressive with Exogenous Variables (Siegelmann et al, 1987)

I The underlying process is model according

yt = f (yt−1, yt−2, . . . , xt , xt−1, xt2 , . . .) + ε

where f is implemented by a static neural network.

I NARX networks are as capable as a Turing Universal Machine!(Siegelmann, 1997)


Static Neural Networks for Dynamical ProblemsNonlinear Autoregressive with Exogenous Variables (Siegelmann et al, 1997) (cont.)

Taken from Siegelmann et al, 1997.


Arbitrary Recurrent Neural NetworksDirect Design

The nonlinear system

1


n∑j=1

wij fj (xj (t))

with i ∈ [1, . . . , n] can be approximated to

1


n∑j=1

wijxj (t)

if the neurons operate in the linear region.

0

20

40

60

80

100 1

0. 5

0

0.5

11

0. 5

0

0.5

1

F irst DimensionS econds

Se

con

d D

ime

nsi

on

Taken from Zegers and Sundareshan, 2003.Zegers Dynamic Problems and Learning Systems to Solve Them

Partially Recurrent SystemsDynamic Neural Network

I A specific type of partially recurrent neural network (page 180 ofHertz et al).

I In these cases the system is mostly feedforward and the recurrentconnections are used only in some specific places.

I Zegers and Sundareshan (2001, 2003).

I Zegers (USP 6751601).

Desired TrajectorySimple Trajectory


Partially Recurrent SystemsDynamic Neural Network (cont.)

DesiredTrajectory

.( )h .( )

yN

(t)o (t)

gRNN NRNN

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

First Dimension

Sec

ond

Dim

ensi

on



DesiredTrajectory

yN

(t)

.( )h .( )g

o (t)

e (t)

Input

RNN NRNN

Modulating

−1 0 1−1

−0.5

0

0.5

1−0.5000

−1 0 1−1

−0.5

0

0.5

1−0.3333

−1 0 1−1

−0.5

0

0.5

1−0.1667

−1 0 1−1

−0.5

0

0.5

1+0.0000

−1 0 1−1

−0.5

0

0.5

1+0.1667

−1 0 1−1

−0.5

0

0.5

1+0.3333

−1 0 1−1

−0.5

0

0.5

1+0.5000

−1 0 1−1

−0.5

0

0.5

1−0.4167

−1 0 1−1

−0.5

0

0.5

1−0.2500

−1 0 1−1

−0.5

0

0.5

1−0.0833

−1 0 1−1

−0.5

0

0.5

1+0.0833

−1 0 1−1

−0.5

0

0.5

1+0.2500

−1 0 1−1

−0.5

0

0.5

1+0.4167

−1 0 1−1

−0.5

0

0.5

1+0.5833


Fully Connected Recurrent Neural NetworksVariational Approaches

I Barak Pearlmutter (1990, 1995).


Fully Connected Recurrent Neural NetworksReinforcement Learning

I Tedrake, 2005. Tomada por Andrew Hetherington.


Fully Connected Recurrent Neural NetworksReinforcement Learning

I Check Andrew Ng’s research (see Ng’s helicopter video, 2007).


Universal Approximation Capacity of RNNsFunahashi and Nakamura’s Theorem

I In 1993 Funahashi and Nakamura proved that fully connectedrecurrent neural networks of the type

1


n∑j=1

wij fj (xj (t))

with i ∈ [1, . . . , n] are universal approximators.


Training OptimizationNice Tricks!

I Teacher forcing.

I Complexifying objective function.


Other Dynamic ProblemsIt Can Be Very Complicated!

I Other neuron models.

I Connection delays.

I Complex spaces.

I Noisy trajectories.

I Stochastic processes.

I High dimensionalities.


Analog ProblemsThere Are Many Other Dynamic Problems

I Compressor systems.

I Finding density functions.

I Training problems.

I Solving partial differential equations.

I Mapping surfaces and volumes.


ConclusionQuestions!

Taken from Carlson, 2004.


dynamic problems and learning systems to solve them

Documents