remarks on direct system identification using … › publication › imecs2017 ›...

5
Remarks on Direct System Identification Using Hypercomplex–Valued Neural Network with Application to Time–Series Estimation Kazuhiko Takahashi, Junpei Ozaki, Yasunao Nakahara, Yuka Wakaume, Takahiro Hirayama, and Masafumi Hashimoto Abstract—This study discusses a design method for system identification using multilayer hypercomplex–valued neural net- works, such as complex, hyperbolic, bicomplex and quaternion neural networks, and investigates its characteristics. The di- rect transfer function identifier, which employs a multilayer hypercomplex–valued neural network, is used to estimate the output of the target system. A computational experiment involv- ing estimation of a discrete–time dynamic plant is conducted to evaluate the feasibility of the hypercomplex–valued neural network–based direct transfer function identifier. As a practical application of dynamic system identification, the direct transfer function identifier is used to estimate historical foreign exchange data and motion data of Taijiquan in addition to investigating its characteristics. The experimental results show the effectiveness of the proposed direct transfer function identifier. Index Terms—Hypercomplex–valued neural network, System identification, Dynamic systems, Historical data, Motion data. I. I NTRODUCTION T HE use of hypercomplex numbers for neural networks has attracted increasing attention from engineering re- search fields because it helps to learn to handle a wide variety of geometric objects and their transformation in the form of hypercomplex numbers. To overcome classically hard–to– treat intractable problems with real–number neural networks, high–dimensional neural networks based on hypercomplex numbers, such as complex numbers and quaternions, have been investigated [1] and their effectiveness in solving mul- tidimensional problems has been demonstrated [2]. Complex neural networks have been used successfully in various engineering applications, for example, filtering, time–series signal processing, communications, image processing and video processing [3]. As an extension of complex numbers, quaternions decrease computational complexity significantly in three– or four–dimensional problems compared to real numbers. Therefore, quaternion neural networks can cope with multidimensional issues more efficiently by directly employing quaternions. Several studies have used quaternion neural networks in applications requiring multidimensional signal processing, for instance, rigid body attitude control [4], colour image processing [5], signal processing [6][7], filtering [8], pattern classification [9] and inverse problems [10]. In previous studies, we have presented control schemes for robot manipulators [11] and servo–controllers [12][13]. Manuscript received December 8, 2016; revised December 20, 2016. K. Takahashi, J. Ozaki, Y. Nakahara, Y. Wakaume and T. Hi- rayama are with Information Systems Design, Doshisha University, Ky- oto, Japan e-mail: {katakaha@mail, bun1063@mail4, bum1040@mail4, bun1084@mail4, bun1016@mail4}.doshisha.ac.jp. M. Hashimoto is with Intelligent Information Engineering and Science, Doshisha University, Kyoto, Japan e-mail: [email protected]. In this study, a method of system identification based on a hypercomplex–valued neural network is discussed from the viewpoint of its applicability to time–series signal processing of dynamic systems. A method of designing a direct transfer function identifier in which the neural network output serves as an estimate of the output of a target system is presented. Computational experiments for estimating the output of a discrete–time dynamic plant, historical foreign exchange data and motion data of Taijiquan movements are conducted to evaluate the feasibility of the proposed neural network–based direct transfer function identifier. II. DIRECT TRANSFER FUNCTION I DENTIFIER BASED ON HYPERCOMPLEXVALUED NEURAL NETWORKS Figure 1 shows a schematic of a direct transfer function identifier using a neural network, where the output of the neural network converges with the plant output after the neural network is trained and the plant transfer functions are composed in the neural network. Here y R q is the plant output, u R p is the input to the plant and ˆ y R q is the output of the identifier that consists of the output from the hypercomplex–valued neural network. Assuming a multilayer network topology that has one input layer, one output layer and N hidden layers (N 0), the input–output relationship of the hypercomplex–valued neural network is defined. Because all signals, weights and thresholds of a hypercomplex–valued neural network are hypercomplex numbers, the algebra of hypercomplex numbers should be followed to derive a training algorithm for hypercomplex– valued neural networks. In this study, complex, hyperbolic, bicomplex and quaternion numbers are considered so as to consist of the neural networks. Complex numbers consist of two real numbers and an imaginary unit: C := {z = z 0 + z 1 i|z 0 ,z 1 R, i 2 = -1}. The sum of two complex numbers z 1 and z 2 is given as z 1 ± z 2 =(z 10 ± z 20 )+(z 11 ± z 21 )i, Plant y Neural network y + - u ε ^ Fig. 1. Schematic of neural network–based direct transfer function identifier. Proceedings of the International MultiConference of Engineers and Computer Scientists 2017 Vol I, IMECS 2017, March 15 - 17, 2017, Hong Kong ISBN: 978-988-14047-3-2 ISSN: 2078-0958 (Print); ISSN: 2078-0966 (Online) IMECS 2017

Upload: others

Post on 05-Jul-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Remarks on Direct System Identification Using … › publication › IMECS2017 › IMECS2017_pp213-217.pdfhigh–dimensional neural networks based on hypercomplex numbers, such as

Remarks on Direct System Identification UsingHypercomplex–Valued Neural Network with

Application to Time–Series EstimationKazuhiko Takahashi, Junpei Ozaki, Yasunao Nakahara, Yuka Wakaume,

Takahiro Hirayama, and Masafumi Hashimoto

Abstract—This study discusses a design method for systemidentification using multilayer hypercomplex–valued neural net-works, such as complex, hyperbolic, bicomplex and quaternionneural networks, and investigates its characteristics. The di-rect transfer function identifier, which employs a multilayerhypercomplex–valued neural network, is used to estimate theoutput of the target system. A computational experiment involv-ing estimation of a discrete–time dynamic plant is conductedto evaluate the feasibility of the hypercomplex–valued neuralnetwork–based direct transfer function identifier. As a practicalapplication of dynamic system identification, the direct transferfunction identifier is used to estimate historical foreign exchangedata and motion data of Taijiquan in addition to investigating itscharacteristics. The experimental results show the effectivenessof the proposed direct transfer function identifier.

Index Terms—Hypercomplex–valued neural network, Systemidentification, Dynamic systems, Historical data, Motion data.

I. INTRODUCTION

THE use of hypercomplex numbers for neural networkshas attracted increasing attention from engineering re-

search fields because it helps to learn to handle a wide varietyof geometric objects and their transformation in the formof hypercomplex numbers. To overcome classically hard–to–treat intractable problems with real–number neural networks,high–dimensional neural networks based on hypercomplexnumbers, such as complex numbers and quaternions, havebeen investigated [1] and their effectiveness in solving mul-tidimensional problems has been demonstrated [2]. Complexneural networks have been used successfully in variousengineering applications, for example, filtering, time–seriessignal processing, communications, image processing andvideo processing [3]. As an extension of complex numbers,quaternions decrease computational complexity significantlyin three– or four–dimensional problems compared to realnumbers. Therefore, quaternion neural networks can copewith multidimensional issues more efficiently by directlyemploying quaternions. Several studies have used quaternionneural networks in applications requiring multidimensionalsignal processing, for instance, rigid body attitude control[4], colour image processing [5], signal processing [6][7],filtering [8], pattern classification [9] and inverse problems[10]. In previous studies, we have presented control schemesfor robot manipulators [11] and servo–controllers [12][13].

Manuscript received December 8, 2016; revised December 20, 2016.K. Takahashi, J. Ozaki, Y. Nakahara, Y. Wakaume and T. Hi-

rayama are with Information Systems Design, Doshisha University, Ky-oto, Japan e-mail: {katakaha@mail, bun1063@mail4, bum1040@mail4,bun1084@mail4, bun1016@mail4}.doshisha.ac.jp.

M. Hashimoto is with Intelligent Information Engineering and Science,Doshisha University, Kyoto, Japan e-mail: [email protected].

In this study, a method of system identification based on ahypercomplex–valued neural network is discussed from theviewpoint of its applicability to time–series signal processingof dynamic systems. A method of designing a direct transferfunction identifier in which the neural network output servesas an estimate of the output of a target system is presented.Computational experiments for estimating the output of adiscrete–time dynamic plant, historical foreign exchange dataand motion data of Taijiquan movements are conducted toevaluate the feasibility of the proposed neural network–baseddirect transfer function identifier.

II. DIRECT TRANSFER FUNCTION IDENTIFIER BASED ONHYPERCOMPLEX–VALUED NEURAL NETWORKS

Figure 1 shows a schematic of a direct transfer functionidentifier using a neural network, where the output of theneural network converges with the plant output after theneural network is trained and the plant transfer functionsare composed in the neural network. Here y ∈ Rq is theplant output, u ∈ Rp is the input to the plant and y ∈ Rq

is the output of the identifier that consists of the outputfrom the hypercomplex–valued neural network. Assuming amultilayer network topology that has one input layer, oneoutput layer and N hidden layers (N ≥ 0), the input–outputrelationship of the hypercomplex–valued neural networkis defined. Because all signals, weights and thresholds ofa hypercomplex–valued neural network are hypercomplexnumbers, the algebra of hypercomplex numbers should befollowed to derive a training algorithm for hypercomplex–valued neural networks. In this study, complex, hyperbolic,bicomplex and quaternion numbers are considered so as toconsist of the neural networks.

Complex numbers consist of two real numbers and animaginary unit: C := {z = z0 + z1i|z0, z1 ∈ R, i2 = −1}.The sum of two complex numbers z1 and z2 is given as

z1 ± z2 = (z10 ± z20) + (z11 ± z21)i,

Plant

yNeural

network

y+ -u

ε

^

Fig. 1. Schematic of neural network–based direct transfer functionidentifier.

Proceedings of the International MultiConference of Engineers and Computer Scientists 2017 Vol I, IMECS 2017, March 15 - 17, 2017, Hong Kong

ISBN: 978-988-14047-3-2 ISSN: 2078-0958 (Print); ISSN: 2078-0966 (Online)

IMECS 2017

Page 2: Remarks on Direct System Identification Using … › publication › IMECS2017 › IMECS2017_pp213-217.pdfhigh–dimensional neural networks based on hypercomplex numbers, such as

while the product of two complex numbers is given as

z1z2 = (z10z20 − z11z21) + (z10z21 + z11z20)i.

The norm of a complex number is defined as

|z| =√

zz∗ =√

z20 + z2

1 ,

where z∗ = z0−z1i is the conjugate of the complex numberz.

Hyperbolic numbers consist of two complex numbers andan imaginary unit: D := {z = z0 + z1h|z0, z1 ∈ R, h2 = 1}.The sum of two hyperbolic numbers z1 and z2 is given as

z1 ± z2 = (z10 ± z20) + (z11 ± z21)h,

while the product of two hyperbolic numbers is given as

z1z2 = (z10z20 + z11z21) + (z10z21 + z11z20)h.

The norm of a hyperbolic number is defined as

|z| =√

zz∗ =√

|z20 − z2

1 |,

where z∗ = z0 − z1h is the conjugate of the hyperbolicnumber z.

Bicomplex numbers consist of two complex numbers andan imaginary unit, that is four real numbers and threeimaginary units: T := {z = z1 + z2i2|z1,z2 ∈ C, i2

2 =−1} = {z = z0 + z1i1 + z2i2 + z3h|z0, z1, z2, z3 ∈ R, i2

1 =i22 = −1, i1i2 = h, h2 = 1}. The sum of two bicomplex

numbers z1 and z2 is given as

z1 ± z2 = (z10 ± z20) + (z11 ± z21)i1

+(z12 ± z22)i2 + (z13 ± z23)h,

while the product of two bicomplex numbers is given as

z1z2 = (z10z20 − z11z21 − z12z22 + z13s2h)+(z11z20 + z10z21 − z13z22 − z12z23)i1

+(z12z20 − z13z21 + z10z22 − z11z23)i2

+(z13z20 + z12z21 + z11z22 + z10z23)h.

The norm of a bicomplex number is defined as

|z| =

√zz† + z?(z†)?

2=

√z20 + z2

1 + z22 + z2

3 ,

where z? = z0−z1i1+z2i2−z3h, z∗ = z0+z1i1−z2i2−z3h and z† = z0 − z1i1 − z2i2 + z3h are the conjugates ofthe bicomplex number z.

Quaternion numbers consist of four real numbers andthree imaginary units: H := {z = z0 + z1i + z2j +z3k|z0, z1, z2, z3 ∈ R, i2 = j2 = k2 = ijk = −1, ij =−ji = k, jk = −kj = i, ki = −ik = j}. The sum of twoquaternion numbers z1 and z2 is given as

z1 ± z2 = (z10 ± z20) + (z11 ± z21)i+(z12 ± z22)j + (z13 ± z23)k,

while the product of two quaternion numbers is given as

z1z2 = (z10z20 − z11z21 − z12z22 − z13z23)+(z11z20 + z10z21 − z13z22 + z12z23)i+(z12z20 + z13z21 + z10z22 − z11z23)j

+(z13z20 − z12z21 + z11z22 + z10z23)k.

The norm of a quaternion number is defined as

|z| =√

zz∗ =√

z20 + z2

1 + z22 + z2

3 ,

where z∗ = z0 − z1i − z2j − z3k is the conjugate of thequaternion number z.

The output of the m-th neuron unit in the n-th layer x(n)m

is defined as follows:x

(n)m = fm(v(n)

m )

v(n)m =

∑l

w(n)ml x

(n−1)l + φ(n)

m, (1)

where w(n)ml is the weight between the l-th neuron unit in

the (n − 1)-th layer and the m-th neuron unit in the n-thlayer, φ(n)

m is the threshold of the m-th neuron unit in then-th layer, and f(·) is an activation function of the neuronunit. Here the activation function requires analyticity of thefunction in the hypercomplex number domain [14].

The hypercomplex–valued neural network is trained tominimize the cost function J defined by the norm of theoutput error ε(t) as follows:

J(t) =12

∑σ

∑k

|εk(t)|2, (2)

where εk(t) = dk − x(o)k (t), dk is the desired output of the

k-th neuron unit in the output layer and the superscript n ofo indicates the output layer and t is the iteration number.Assuming increment of the network parameters in the t-th iteration ∆ω(t) = ω(t + 1) − ω(t) where the vectorω(t) is composed of network parameters such as weightsand thresholds, variation of the cost function ∆J can becalculated as follows:

∆J = J(t + 1) − J(t)

≈(

∂J(t)∂ω(t)

)T

∆ω(t) +(

∂J(t)∂ω∗(t)

)T

∆ω∗(t).

Setting ∆ω(t) = −η∂J(t)∂ω∗(t)

and ∆ω∗(t) = −η∂J(t)∂ω(t)

yields

∆J = −2η

∣∣∣∣ ∂J(t)∂ω(t)

∣∣∣∣2 ≤ 0,

where the factor η is positive. Increment of the parametersin the output layer is given as

∆w(o)kj (t) = η

∑σ

δk(t)x∗(n−1)j (t)

∆φ(o)k (t) = η

∑σ

δk(t), (3)

where δk(t) = εk(t)f ′k(v(o)

k (t)), and the superscript (n− 1)indicates the hidden layer links to the output layer, whileincrement of the parameters in the hidden layer is given by

∆w(n)ji (t) = η

∑σ

δj(t)x∗(n−1)i (t)

∆φ(n)j (t) = η

∑σ

δj(t), (4)

where δj(t) =∑

k

δ(n+1)k (t)w(n)

kj (t)f ′j(v

(n)j (t)).

Proceedings of the International MultiConference of Engineers and Computer Scientists 2017 Vol I, IMECS 2017, March 15 - 17, 2017, Hong Kong

ISBN: 978-988-14047-3-2 ISSN: 2078-0958 (Print); ISSN: 2078-0966 (Online)

IMECS 2017

Page 3: Remarks on Direct System Identification Using … › publication › IMECS2017 › IMECS2017_pp213-217.pdfhigh–dimensional neural networks based on hypercomplex numbers, such as

To define the input of the identifier simply, we assume thatthe plant is represented by a linear combination of single–input single–output discrete–time linear systems as follows:

y(t + d) =n∑

s=1

Aysy(t − s + 1) +m+d−1∑

s=0

Ausu(t − s) (5)

where Ays =[αys

ll′

]and Aus =

[αus

ll′

](l =

1, 2, · · · , q, l′ = 1, 2, · · · , p) are plant parameter matri-ces, n and m are plant orders, d is plant dead timeand y(t) =

[y1(t) y2(t) · · · yq(t)

]T, u(t) =[

u1(t) u2(t) · · · up(t)]T

. Considering the conditionlim

t→∞{y(t + d) − y(t + d)} = 0 yields

y(t + d) = ΛI(t), (6)

where

Λ =[

Ay1 Ay2 · · · Ayn

Au0 Au1 · · · Aum+d−1

],

and

I(t) =[

y(t) y(t − 1) · · · y(t − n + 1)

u(t) u(t − 1) · · · u(t − m − d + 1)]T

.

Representing the mapping function of the hypercomplex–valued neural network as F nn(·) yields

x(o)(t) = F nn (ω(t), ξ(t)) , (7)

where ξ(t) is the input to the hypercomplex–valued neuralnetwork. By comparing Eqs. (6) and (7), we choose thevector I(t − d) as input to the hypercomplex–valued neuralnetwork ξ(t).

III. COMPUTATIONAL EXPERIMENTS

To investigate the feasibility of the direct transfer functionidentifier using the hypercomplex–valued neural network,we conducted several computational experiments. In theexperiments, we employed a split function as the activationfunction [15] [16], for example, f(z) = f0(z0) + f1(z1)i+f2(z2)j + f3(z3)k in the quaternion neural network, wherethe function fl(x) = 1/(1 + e−x) (l = 0, 1, 2, 3) is a real–valued function. The split function is not analytic in eachhypercomplex number domain; however, we use it for thesake of computational convenience.

First, the quaternion neural network was applied to thedirect transfer function identifier. The following discrete–time nonlinear plant was considered the target system:

y(t) = 1.3y(t − 1) − 0.3y(t − 2) + u(t − 1)+0.2u(t − 2) + 0.03y(t − 3) + 0.2y2(t − 1),

where the second–order system is dominant. In the exper-iment, the plant input u(t) was synthesized by a digitalproportional controller to make the plant output y(t) followa reference model ym(t). Here, the reference model was afirst–order linear system ym(t) = 0.6ym(t−1)+0.4r(t), andthe reference input r(t) was a rectangular wave. The numberof samples within one cycle of the rectangular wave was 100,and the amplitude of the wave was ±0.5. Considering thedominant part of the plant for designing the direct transferfunction identifier, the input of the hypercomplex–valued

0 100 200 300 400 500

9500 9600 9700 9800 9900 10000sampling number

sampling number

0

-1

1

ou

tpu

t

0

-1

1

ou

tpu

t

y

y

y

y

^

^

Fig. 2. Experimental results of system identification using quaternion neuralnetwork with 1–3–1 (H) network topology, where the grey line indicatesplant output and the black line indicates output of the quaternion neuralnetwork (Top: initial adaptation stage of quaternion neural network; bottom:final adaptation stage of quaternion neural network).

norm

ali

zed c

ost

functi

on

CNN HNN BNN QNN

0.1

0.01

0.001

Fig. 3. Comparison of normalized cost function obtained by thehypercomplex–valued neural network–based direct transfer function iden-tifiers (CNN: complex neural network with 2–5–1 (C) network topology;HNN: hyperbolic neural network with 2–5–1 (D) network topology; BNN:bicomplex neural network with 1–3–1 (T) network topology; QNN: quater-nion neural network with 1–3–1 (H) network topology).

neural network was defined as ξ(t) = y(t − 1) + y(t −2)i + u(t − 1)j + u(t − 2)k. The estimated plant outputcomprised one of the imaginary parts of the quaternion neuralnetwork’s output, that is, y(t) = Imk[x(o)(t)], where Imkdenotes the imaginary part of the imaginary unit k. Figure2 shows an example of the system response when the directtransfer function identifier consisted of the quaternion neuralnetwork with a 1–3–1 network topology. As shown in Fig. 2,the identifier output converges with the plant output. Figure3 shows the normalized cost function averaged within oneperiod of the reference input. Here, the normalized costfunction is averaged using 100 results of each neural networkwithin the range [9000, 10000]. The topology of each neuralnetwork was defined so that the number of parameters, suchas weights and thresholds, corresponded to the number inthe quantum neural network. The normalized cost functionof the quaternion neural network is smaller than that of theother neural network. These results indicate the feasibilityof the proposed direct transfer function identifier using thehypercomplex–valued neural network.

Second, the characteristics of the direct transfer functionidentifier were investigated with historical foreign exchangedata for currency pairs of the Japanese Yen and four majorcurrencies (US Dollar, British Pound, Euro and Swiss

Proceedings of the International MultiConference of Engineers and Computer Scientists 2017 Vol I, IMECS 2017, March 15 - 17, 2017, Hong Kong

ISBN: 978-988-14047-3-2 ISSN: 2078-0958 (Print); ISSN: 2078-0966 (Online)

IMECS 2017

Page 4: Remarks on Direct System Identification Using … › publication › IMECS2017 › IMECS2017_pp213-217.pdfhigh–dimensional neural networks based on hypercomplex numbers, such as

M = 1 M = 2 M = 3 M = 4 M = 5

1 2 3 4 5order of input

-13000

-11000

-12000

AIC

Fig. 4. Experimental results of quaternion neural network for findingoptimal network topology based on AIC, where M indicates the number ofneurons in the hidden layer.

0

0.006

0.002

0.004

norm

aliz

ed c

ost

funct

ion

closed test

open test

RNN CNN HNN BNN QNN

8-2-4 8-4-4 4-2-2 4-2-2 2-1-1 2-1-1

Fig. 5. Comparison of estimation performance obtained by thehypercomplex–valued neural network–based direct transfer function iden-tifiers (RNN: real number neural network; CNN: complex neural network;HNN: hyperbolic neural network; BNN: bicomplex neural network; QNN:quaternion neural network).

Franc) as the target system. Here, the total number of datapoints was 3523 (4/Jan./2002 ∼ 5/Aug./2016). We splitthe data into two sets, namely, training set (4/Jan./2002∼ 2/Jun./2009) and test set (3/Jun./2009 ∼ 5/Aug./2016).The training set was used in closed tests for evaluating thetraining accuracy, while the test set was used in open testsfor evaluating the estimation accuracy of the proposed directtransfer function identifiers. Assuming that the dynamicsof historical data can be represented by the autoregressive

(AR) model yl(t) =m∑

s=1

alsyl(t − s) + ul(t) (l = 1, 2, 3, 4),

the input vector of the hypercomplex–valued neural networkin the direct transfer function identifier was defined asξ(t) = [ y(t − 1) y(t − 2) · · · y(t − m) u(t) ]T. Inthe experiments, y1, y2, y3 and y4 denoted the currency pairsUSD/JPY, GBP/JPY, EUR/JPY and CHF/JPY, respectively,and ul(t) was random noise generated in the interval[−0.02, 0.02]. Figure 4 shows the result of networktopology optimization of the quaternion neural networkby using the Akaike Information Criterion (AIC) in theclosed test. Here the components of the input vector werey = y1 + y2i + y3j + y4k and u = u1 + u2i + u3j + u4k,while the estimation results were given by y(t) =[ Re[x(o)(t)] Imi[x(o)(t)] Imj[x(o)(t)] Imk[x(o)(t)] ]T.

1 2

3 4

5

Fig. 6. Representation of movement ‘Push down and stand on one leg’ in24–form Taijiquan with CG model based on captured motion data.

As shown in Fig. 4, the optimal order of the input mis one and the optimal number of neurons in the hiddenlayer M is one because of the lowest number of the AIC.Figure 5 shows the normalized cost function averagedwithin the range of each test. Here, the normalized costfunction is averaged using 20 results of each neural network.Although the normalized cost functions in the closed testare almost identical, the normalized cost function of eachhypercomplex–valued neural network in the open test issmaller than that of the real number neural network. Theseresults indicate the effectiveness of the hypercomplex–valuedneural network in applications involving multiple–inputmultiple–output system identification.

Next, the direct transfer function identifier was applied toestimate human movements acquired by an optical motioncapture system. Here, the basic routine of 24–form Taijiquanwas considered as the movement and the motion data ofthe movement ‘Push down and stand on one leg’, shown inFig. 6, in which one arm is raised and the other is loweredwhile simultaneously standing on one leg – a crouching stepchanges the high position to a low one and then one legis raised with a weight supported by the other one – wasused as the target system. A subject (Japanese male, age:22, height: 177 cm) who learned Taijiquan for six monthsfrom a Taijiquan instructor performed the movement twice,and the 3D motion data of 36 body parts were capturedat a sampling rate of 30 fps [17]. In the motion capturesystem, the x– and the z–axes comprise the horizontalplane, and the y–axis is perpendicular to the aforementionedplane. In the computational experiment, 3D motion dataof the head (xh(t), yh(t), zh(t)) and centre of the waist(xw, yw(t), zw(t)) were considered as the target data. Thetotal number of data points was 2646, and the data was splitup into two sets – training set covering the range [0, 1322]and test set covering the range [1323, 2646]. Given thatthe motion had a constant acceleration characteristic, thesecond–order AR model was assumed to define the inputvector of the hypercomplex–valued neural network ξ(t) =[ yh(t − 1) yw(t − 1) yh(t − 2) yw(t − 2) uh(t)uw(t) ]T, where yl = xli + ylj + zlk and

ul = ul1i + ul2j + ul3k (l = h,w) representrandom noise. The x–, y– and z– positions of the

Proceedings of the International MultiConference of Engineers and Computer Scientists 2017 Vol I, IMECS 2017, March 15 - 17, 2017, Hong Kong

ISBN: 978-988-14047-3-2 ISSN: 2078-0958 (Print); ISSN: 2078-0966 (Online)

IMECS 2017

Page 5: Remarks on Direct System Identification Using … › publication › IMECS2017 › IMECS2017_pp213-217.pdfhigh–dimensional neural networks based on hypercomplex numbers, such as

0 1000 2000 30000

0.5

norm

aliz

ed p

ositi

on z

0.6

1.0

norm

aliz

ed p

ositi

on y

0

0.5no

rmal

ized

pos

ition

x

x

x

y

y

z

z

closed test open test

closed test open test

closed test open test

sampling number

0 1000 2000 30000

0.5

norm

aliz

ed p

ositi

on z

0

0.5

norm

aliz

ed p

ositi

on y

0

0.5

norm

aliz

ed p

ositi

on x

x

x

y

y

z

z

closed test open test

closed test open test

closed test open test

sampling number

(a) head position

(b) waist position

h

h

h

h

h

h

w

w

w

w

w

w

Fig. 7. Experimental results of motion data estimation using quaternionneural network with 6–2–2 (H) network topology; the black line indicatesthe measured data and the grey line indicates the output of the quaternionneural network (Top: 3D position of head, bottom: 3D position of waist).

head and waist were estimated from the imaginaryparts of the quaternion neural network’s output:y(t) = [ Imi[x(o)(t)] Imj[x(o)(t)] Imk[x(o)(t)] ]T.Figure 7 shows the system response for the case inwhich the direct transfer function identifier consisted of aquaternion neural network with a 6–2–2 network topology.As shown in the figure, the output of the quaternion neuralnetwork could approximate the motion data in the closedand the open tests. This result indicates the usefulness ofthe proposed direct transfer function identifier using thehypercomplex–valued neural network.

IV. CONCLUSIONS

In this study, the capability of hypercomplex–valued neuralnetworks was investigated and their application to system

identification was explored. A direct transfer function iden-tifier in which the output of a target system is estimated bythe multilayer hypercomplex–valued neural network was de-signed and its characteristics were evaluated. Computationalexperiments using a complex neural network, hyperbolicneural network, bicomplex neural network and quaternionneural network were conducted to estimate the output of adiscrete–time dynamic plant, historical foreign exchange dataand the motion data of Taijiquan movements. The experimen-tal results confirmed the feasibility and effectiveness of theproposed hypercomplex–valued neural network–based directtransfer function identifier.

REFERENCES

[1] B. K. Tripathi, High Dimensional Neurocomputing - Growth, Ap-praisal and Applications -, Springer India, 2015.

[2] T. Nitta (ed.), Complex–Valued Neural Networks - Utilizing High–Dimensional Parameters -, Information Science Reference, 2009.

[3] A. Hirose (ed.), Complex–Valued Neural Networks - Advances andApplications -, IEEE Press, WILEY, 2013.

[4] L. Fortuna, G. Muscato and M. G. Xibilia, “A Comparison BetweenHMLP and HRBF for Attitude Control”, IEEE Transactions on NeuralNetworks, Vol. 12, No. 2, pp. 318–328, 2001.

[5] H. Kusamichi, T. Isokawa, N. Matsui, Y. Ogawa and K. Maeda, “ANew Scheme Color Night Vision by Quaternion Neural Network”,in Proceedings of the 2nd International Conference on AutonomousRobots and Agents, pp. 101–106, 2004.

[6] P. Arena, R. Caponetto, L. Fortuna, G. Muscato and M. G. Xibilia,“Quaternionic Multilayer Perceptrons for Chaotic Time Series Predic-tion”, IEICE Transactions Fundamentals, Vol. E79–A, No. 10, pp.1682–1688, 1996.

[7] S. Buchholz and N. Le Bihan, “Optimal Separation of PolarizedSignals by Quaternionic Neural Networks”, in Proceedings of 14thEuropean Signal Processing Conference, pp. 1–5, 2006.

[8] B. C. Ujang, C. C. Took and D. P. Mandic, “Quaternion–Valued Non-linear Adaptive Filtering”, IEEE Transactions on Neural Networks,Vol. 22, No. 8, pp. 1193–1206, 2011.

[9] F. Shang and A. Hirose, “Quaternion Neural–Network–Based PolSARLand Classification in Poincare–Sphere–Space”, IEEE Transactions onGeoscience and Remote Sensing, Vol. 52, No. 9, pp. 5693–5703, 2014.

[10] T. Ogawa, “Neural Network Inversion for Multilayer QuaternionNeural Networks”, Computer Technology and Applications, Vol. 7,pp. 73–82, 2016.

[11] Y. Cui, K. Takahashi and M. Hashimoto, “Design of Control SystemsUsing Quaternion Neural Network and Its Application to InverseKinematics of Robot Manipulator”, in Proceedings of 2013 IEEE/SICEInternational Symposium on System Integration, pp.527–532, 2013.

[12] K. Takahashi, Y. Hasegawa and M. Hashimoto, “Remarks on Quater-nion Neural Network-based Self–tuning PID Controller”, in Proceed-ings of the 3rd International Symposium on Computer, Consumer andControl, pp.231–234, 2016.

[13] K. Takahashi, Y. Wakaume, T. Hirayama and M. Hashimoto, “Remarkson Hypercomplex–valued Neural Network–based Direct AdaptiveController”, in Proceedings of the 2nd International Conference onRobotics and Automation Sciences / International Conference onMultimedia Systems and Signal Processing, S18, 2016.

[14] T. Isokawa, H. Nishimura and N. Matsui, “Quaternionic MultilayerPerceptron with Local Analyticity”, Information, Vol. 3. pp. 756–770,2012.

[15] B. C. Ujang, C. C. Took and D. P. Mandic, “Split Quaternion NonlinearAdaptive Filtering”, Neural Networks, Vol. 23, No. 3, pp. 426–434,2010.

[16] E. Hitzer, “Algebraic Foundations of Split Hypercomplex NonlinearAdaptive Filtering”, Mathematical Methods in the Applied Sciences,Vol. 36, No. 9, pp. 1042–1055, 2013.

[17] K. Takahashi, T. Koroyasu, Y. Kameda and M. Hashimoto, “Evaluationof Taijiquan Skill Acquisition Process using Motion Capture Systemand Simplified NIRS”, WIT Transactions on Information and Com-munication Technologies - Advances in Sport Science and ComputerScience, Vol. 57, pp.787–794, 2013.

Proceedings of the International MultiConference of Engineers and Computer Scientists 2017 Vol I, IMECS 2017, March 15 - 17, 2017, Hong Kong

ISBN: 978-988-14047-3-2 ISSN: 2078-0958 (Print); ISSN: 2078-0966 (Online)

IMECS 2017