This article was downloaded by: [Temple University Libraries]On: 17 November 2014, At: 21:28Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: MortimerHouse, 37-41 Mortimer Street, London W1T 3JH, UK

Cybernetics and Systems: An International JournalPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/ucbs20

Optimal Control of Uncertain Stochastic Systems withMarkovian Switching and Its Applications to PortfolioDecisionsWEIYIN FEI aa School of Mathematics and Physics , Anhui Polytechnic University , Wuhu , Anhui , P.R.ChinaPublished online: 02 Jan 2014.

To cite this article: WEIYIN FEI (2014) Optimal Control of Uncertain Stochastic Systems with Markovian Switchingand Its Applications to Portfolio Decisions, Cybernetics and Systems: An International Journal, 45:1, 69-88, DOI:10.1080/01969722.2014.862445

To link to this article: http://dx.doi.org/10.1080/01969722.2014.862445

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose ofthe Content. Any opinions and views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be reliedupon and should be independently verified with primary sources of information. Taylor and Francis shallnot be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and otherliabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Cybernetics and Systems: An International Journal, 45:69–88Copyright © 2014 Taylor & Francis Group, LLCISSN: 0196-9722 print/1087-6553 onlineDOI: 10.1080/01969722.2014.862445

Optimal Control of Uncertain Stochastic Systems withMarkovian Switching and Its Applications to Portfolio

Decisions

WEIYIN FEISchool of Mathematics and Physics, Anhui Polytechnic University, Wuhu, Anhui, P.R. China

This article first describes a class of uncertain stochastic control systems withMarkovian switching, and derives an Itô-Liu formula for Markov-modulatedprocesses. We characterize an optimal control law, that satisfies the generalizedHamilton-Jacobi-Bellman (HJB) equation with Markovian switching. Then,by using the generalized HJB equation, we deduce the optimal consumptionand portfolio policies under uncertain stochastic financial markets withMarkovian switching. Finally, for constant relative risk-aversion (CRRA)felicity functions, we explicitly obtain the optimal consumption and portfoliopolicies. Moreover, we make an economic analysis through numerical examples.

KEYWORDS generalized Itô-Liu formula, HJB equations, Markovianswitching, optimal consumption and portfolio, optimal control of uncertainstochastic systems, uncertain random variables

INTRODUCTION

Uncertainty traditionally contains randomness, fuzziness, and uncertainty resultingfrom their actions together. In many cases, fuzziness and randomness simultaneouslyappear in a system. To describe this phenomenon, the concept of fuzzy randomvariables was introduced by Kwakernaak (1978) as a random variable taking fuzzyvariable values. According to different requirements of measurability, different studiesfor fuzzy random variables were made by Puri and Ralescu (1986) and Liu and Liu(2003). The fuzzy random differential equations were studied by Fei (2007a, 2007b),and the references therein.

For stochastic dynamic systems, stochastic differential equations were studiedby researchers such as Karatzas and Shreve (1991). When a fuzzy dynamic system

Address correspondence to Weiyin Fei, School of Mathematics and Physics, Anhui PolytechnicUniversity, Wuhu, 241000, Anhui, P.R. China. E-mail: wyfei@dhu.edu.cn

Color versions of one or more of the figures in the article can be found online atwww.tandfonline.com/ucbs.

69

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

70 W. Fei

is considered, fuzzy differential equations were investigated by Kaleva (1987), Fei(2009b), and Fei et al. (2010).

In the real world, some information or knowledge is usually represented byhuman linguistic expressions like ‘‘about 100 km,’’ ‘‘approximately 80 kg,’’ ‘‘warm,’’or ‘‘young.’’ Perhaps they are often considered to be subjective probability or fuzzyconcepts. However, much research has shown that those ‘‘unknown constants’’ and‘‘unsharp concepts’’ behave neither like randomness nor like fuzziness. In order todistinguish this phenomenon from randomness and fuzziness, Liu (2011) called ituncertainty. After further observations, we find that randomness and Liu’s uncertaintyoften appear simultaneously in a system. To describe this phenomenon, the conceptof uncertain random variable was introduced by Liu (2013) as a random variabletaking ‘‘uncertain variable’’ values.

Under the stochastic context with Markovian switching, control systems havebeen used to model many practical systems where they may experience abrupt changesin their structure and parameters caused by such phenomena as component failures orrepairs, changing subsystem interconnections, abrupt environmental disturbances inan economic system, etc. The control systems combine a part of the state that is drivenby the canonical process and Brownian motion and another part of the state that takesdiscrete values. Mariton (1990) explained that hybrid systems had been emerging as aconvenient mathematical framework for the formulation of various design problemsin different fields like target tracking (evasive target tracking problem), fault tolerancecontrol, and manufacturing processes.

One of the important classes of hybrid systems is stochastic differentialequations with Markovian switching. In operation, the system will switch from onemode to another in a random way, and the switching between the modes is governedby a Markovian chain. The optimal regulator, controllability, observability, stability,stabilization, etc., need to be studied. For more information about hybrid systems,the reader can refer to Feng et al. (2010) and Huang and Mao (2009), and referencestherein.

Hamilton (1989) originally proposed the regime switching models of stockreturns, which proved to be a better representation of financial reality than theusual models with deterministic coefficients. Thereafter, regime switching modelshave been studied in different contexts. Option pricing in financial marketswith regime switching has been studied, for instance, by Buffington and Elliott(2002) and Guo and Zhang (2004). In financial markets with regime switching,the maximization of expected utility from consumption and/or terminal wealthhas also been studied by Sotomayor and Cadenillas (2009), and Zhang andYin (2004). Under a mean-variance criterion, Zhou and Yin (2003) proposeda continuous-time Markowitz’s mean-variance portfolio selection model withregime switching that obtains the efficient portfolio that minimizes the risk ofterminal wealth given a fixed expected terminal wealth. In Fei (2010, 2013),the optimal control of Markovian switching systems with applications to optimalconsumption and portfolio under inflation and Markovian switching was discussed.

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

Optimal Control of Uncertain Stochastic Systems 71

The optimal portfolio selection with random fuzzy returns was explored inHuang (2007).

Because randomness and Liu’s uncertainty simultaneously appear in thefinancial market, we begin to consider the uncertain stochastic systems withMarkovian switching. The uncertain stochastic system driven by the canonical processand Brownian motion will switch from one mode to another in a random way, andthe switching between the modes is governed by a Markovian chain. After introducingthe concept of uncertain random processes, which is different from that of You(2007), we will generalize the Itô-Liu formula, from which we deduce the Hamilton-Jacobi-Bellman (HJB) equation or the optimality equation for the optimal controllaw. Next, by using the optimality equation for uncertain stochastic controls, we solvethe optimal portfolio and consumption decision problem of an investor, which is itsfinancial application. We obtain the explicit expression for the optimal consumptionand portfolio. Finally, we make an economic analysis of the particular case of theconstant relative risk-aversion (CRRA) utilities.

In fact, the initial contribution to consumption/investment problems incontinuous time was done by Merton (1969, 1971). Other models of consumption-investment can be found in Fei (2007c, 2009a), Fei and Wu (2002), and Karatzasand Shreve (1998). Traditional financial modeling only deals with the continuouslystochastic changing dynamics of a financial market. However, the market behavioris also affected by uncertain stochastic processes and a finite-state continuous-timeMarkovian chain that represents the uncertainty generated by the steadier marketconditions. Based on the uncertain random calculus, the optimal consumption andportfolio strategies are obtained under the uncertain random context. Our modeldiffers from the above papers and also answers a different economic question: What isthe effect of both Markovian switching and uncertainty on the optimal consumptionand portfolio?

The rest of the article is organized as follows. The following section providesthe general framework of the uncertain stochastic optimal control problem withMarkovian switching. Generalizing an Itô-Liu formula, we deduce an HJB equationfor later research in the subsequent section. The policy of optimal consumption andportfolio under the uncertain random environment with Markovian switching isderived next. Then, for the case of the CRRA utility, the optimal policies are explicitlygiven, and the numerical results of optimal policies together with an economic analysisof the investor’s behavior are made. Finally, our concluding remarks are presented.

GENERALIZED ITÔ-LIU FORMULA AND EQUATIONOF OPTIMALITY

For convenience, we give some useful concepts at first. Let � be a nonempty setand � a �-algebra over � . Each element �∈� is called an event. A set function �defined on the �-algebra over �, which satisfies (i) normality, (ii) monotonicity, (iii)self-duality, and (iv) countable subadditivity, is called an uncertain measure accordingto Liu (2011). Let a probability space be (�,�,�).

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

72 W. Fei

Definition 1. (Liu 2011) Let � be an index set and let (� ,�,�) be anuncertainty space. An uncertain process X(t, �) is a measurable function from� × (� ,�,�) to the set of real numbers, that is, for each t ∈� and any Borelset B of real numbers, the set �X(t) ∈ B�= �� ∈ � |X(t, �) ∈ B� is an event.

For concepts and properties of the canonical process and other uncertainprocesses refer to chapter 9 of Liu (2011). In what follows, we give the notation ofuncertain random variables.

Definition 2. (Liu and Liu 2003) An uncertain random variable is a function� from a probability space (�,�,�) to the set of uncertain variablessuch that ���() ∈ B� is a measurable function of for any Borel setB of �.

We now give the concept of expected value of an uncertain random variable.

Definition 3. Let � be an uncertain random variable. Then its expected value isdefined by

E[�] = EP[EU[�]]provided that the right-hand operations are well defined. Here, the operators EP

and EU stand for probability expectation and uncertain expectation, respectively.

Obviously, if both a and b are constant, then E[aCt + bBt] = 0, where Ct and Bt

are a scalar canonical process and a Wiener process (Brownian motion), respectively.

Definition 4. A hybrid process X(t, �,) is called an uncertain stochasticprocess if for each t ∈ � , X(t) is an uncertain random variable. An uncertainstochastic process X(t) is called continuous if the sample paths of X(t) are allcontinuous functions of t for almost all (�,)∈ (� ,�).

Definition 5. (Itô-Liu integral) Let X(t)= (Y(t), Z(t)) be an uncertainstochastic process. For any partition of closed interval [a, b] with a = t1 <t2 < · · · < tN+1 = b, the mesh is written as = max

1<i<N|ti+1 − ti|� Then the Itô-Liu

integral of X(t) with respect to (Bt, Ct) is defined as follows,∫ b

aX(s)d(Bs, Cs) = lim

→0

N∑i=1

(Y(ti)(Bti+1 − Bti) + Z(ti)(Cti+1 − Cti))

provided that it exists in mean square and is an uncertain random variable,where Ct and Bt are a one-dimensional canonical process and a one-dimensionalWiener process, respectively. In this case, X(t) is called Itô-Liu integrable. Inparticular, when Y(t) ≡ 0, X(t) is called Liu integrable.

Next, we deduce, the Itô-Liu formula for the case of multi-dimensionaluncertain stochastic processes.

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

Optimal Control of Uncertain Stochastic Systems 73

Theorem 1. Let B = (Bt)0≤t≤T = (B1t , � � � , Bm

t )�0≤t≤T and C = (Ct)0≤t≤T =

(C1t , � � � , Cn

t )�0≤t≤T be an m-dimensional standard Wiener process and an

n-dimensional canonical process, respectively. Assume that uncertain stochasticprocesses X1(t), X2(t), � � � , Xp(t) are given by

dXk(t) = uk(t)dt +m∑

l=1

vkl(t)dBlt +

n∑l=1

wkl(t)dClt, k = 1, � � � , p,

where uk(t) are all absolute integrable uncertain stochastic processes, vkl(t)are all square integrable uncertain stochastic processes, and wkl(t) areall Liu integrable uncertain stochastic processes. For k, l = 1, � � � , p, let�G�t (t, x1, � � � , xp), �G

�xk(t, x1, � � � , xp) and �2G

�xkxl(t, x1, � � � , xp) be continuously functions.

Then we have

dG(t, X1(t), � � � , Xp(t))

= �G�t(t, X1(t), � � � , Xp(t))dt +

p∑k=1

�G�xk

(t, X1(t), � � � , Xp(t))dXk(t)

+ 1

2

p∑k=1

p∑l=1

�2G�xk�xl

(t, X1(t), � � � , Xp(t))dXk(t)dXl(t),

where dBkt dBl

t = kldt, dBkt dt=dCı

tdCt = dCı

tdt=dBkt dCı

t = 0, for k, l = 1, � � � , m,ı, = 1, � � � , n. Here

kl ={

0, if k �= l1, otherwise�

Proof. Similar to the discussion in You (2007), we easily derive our Theorem 1 foruncertain stochastic processes. Thus, the proof is complete. �

Remark 1. Similar to the discussions in You (2007), we can obtain the counterpartsof the Itô-Liu formula for multidimensional uncertain stochastic processes. Here,the further discussion is omitted.

In reality, uncertain stochastic systems may experience abrupt changes intheir structure and parameters, so we characterize this phenomenon by anuncertain stochastic context with Markovian switching. To this end, we make somepreliminaries.

Now we consider m-dimensional standard Brownian motion (Bt)0≤t≤T and ann-dimensional canonical process (Ct)0≤t≤T� We observe a continuous-time, stationary,finite-state Markov chain �= (�(t))0≤t≤T and denote by � the state-space of thisMarkov chain: that is, for every t ∈ [0, T], �(t) ∈ � = �1, 2, � � � , S�, where S ∈�1, 2, 3, � � � �. Here, the stochastic process �(t,) represents the regime (mode) ofthe random environment at time t and depends only on ∈�. S is the numberof regimes. Assume that the Markov chain has a strongly irreducible generator

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

74 W. Fei

Q = (qij)S×S, where qii�= −�i < 0 and

∑j∈� qij = 0 for every regime i ∈ � . Denote

by � = ��t, t ∈ [0, T]� the �-augmentation of filtration ��B,�t � generated by the

stochastic processes B, �, where �B,�t

�= ��Bs, �(s), 0 ≤ s ≤ t� for every t ∈ [0, T]. Weassume that the Markov chain �(·) is independent of the Brownian motion B.

Let be a separable metric space, for a control variable D, we consider thefollowing uncertain stochastic controlled system with Markovian switching, for t ∈[0, T], {

dX(t) = f(t, X(t), D(t), �(t))dt + g(t, X(t), D(t), �(t))dBt

+h(t, X(t), D(t), �(t))dCt, X(0) = x,(1)

where

f = (f1, � � � , fp)� : [0, T] × �p × × � → �p,

g = (gkl)p×m : [0, T] × �p × × � → �p×m,

h = (hkl)p×n : [0, T] × �p × × � → �p×n�

Suppose that the cost functional of our control problem is as follows.

J(x, i; D) = E{∫ T

0

�1(t, X(t), D(t), �(t))dt + �2(T, X(t), �(T))}� (2)

Set

�=

{D : [0, T] × � × � → | D is measurable and for each

� ∈ � , D(�) is �-adapted and satisfies

E[∫ T

0

�+1 (t, X(t), D(t), �(t))dt

]< +∞,

E[∫ T

0

�+2 (T, X(t), �(t))dt

]< +∞,

E[∫ T

0

|D(t)|2dt]< +∞

}�

Here, a+ �= max(a, 0), a− �= max(−a, 0) for every a ∈ [−∞, ∞], and | · | denotesthe norm of a vector or matrix.

Under certain assumptions (which will be specified below), for any D(·)∈,Eq. (1) admits a unique solution, and the cost (2) is well defined. The optimalcontrol problem can be stated as follows.

Problem 1. Select an admissible control D ∈ that minimizes J(x, y, i; D) and find

a value function V defined by V(x, i)�= infD∈ J(x, i; D)� The control D is called an

optimal control. �

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

Optimal Control of Uncertain Stochastic Systems 75

Next, we need to consider a family of optimal control problems with differentinitial times and states along a given state trajectory in order to apply the optimalprinciple for uncertain stochastic optimal control. For this, we set up the followingframework. For any (t, x) ∈ [0, T] × �m, consider the state equation, s ∈ [t, T],{

dX(s) = f(s, X(s), D(s), �(s))ds + g(s, X(s), D(s), �(s))dBs

+h(s, X(s), D(s), �(s))dCs, X(t) = x�(3)

The corresponding cost functional is

J(t, x, i; D) = E[∫ T

t�1(s, X(s), D(s), �(s))ds + �2(T, X(T), �(T))

]� (4)

Define the value function V(t, x, i) by V(t, x, i)�= infD∈ J(t, x, i; D).

For our aim, we make the following hypothesis.

Assumption 1. (1) Assume that coefficients f, g, and h are locally Lipschitziancontinuous; that is, there exists one nonnegative function HN(D), where HN(D)is possibly dependent on the control variable D, such that the following inequalityholds for h = f, g, h,

|h(t, x, D, i) − h(t, y, D, i)| < HN(D)|x − y|,for all (t, D, i) ∈ [0, T] × × � and those x, y ∈ �p with |x| ∨ |y| ≤ N.Moreover, for g h, the special growth conditions hold; that is, there is a constantLN such that

|g(t, x, D, i)| ∨ |h(t, x, D, i)| < LN(1 + |x| + H(x)|D|),for all (t, D, i) ∈ [0, T] × × � and |x| ≤ N, where the nonnegative function His locally bounded.

(2) For the cost functions of the control problem �l, l = 1, 2, there exists aconstant K > 0, for all i ∈ � ,

�1(t, x, D, i) ≥ −K(1 + �1(x)|D|2), �2(t, x, i) ≥ −K(1 + �2(x)),

where the nonnegative functions �l, l = 1, 2, are continuous in x.

Similar to the discussion for fuzzy differential equations in Fei (2009b), itis known that under Assumption 1 (1), Eq. (1) has a unique continuous solutionX(t)= X(t, D(t), �(t)) on t ∈ [0, T] for each D ∈ .

Remark 2. We will see that Assumption 1 holds for the linear system and specialcost functions �1 and �2 in the next section. For more general systems, we will studythe existence and uniqueness of solutions to uncertain stochastic controlled systemsand the properties of value functions of our control problems in the future.

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

76 W. Fei

In order to solve our problem, now we generalize the Itô-Liu lemma for anuncertain stochastic process with Markovian swithcing as follows. For the sake ofsimplicity, we drop the arguments whenever convenient and denote

f(t) = f(t, X(t), D(t), �(t)), g(t) = g(t, X(t), D(t), �(t)), h(t)

= h(t, X(t), D(t), �(t))�

Lemma 1. Suppose that the state X(t) = (X1(t), � � � , Xp(t))� of an uncertainstochastic system satisfies Eq. (1). For ∀ i ∈ � , k, l = 1, � � � , p, let �F

�t (t, x, i),�F�xk(t, x, i) and �2F

�xkxl(t, x, i) be continuously functions, where x = (x1, � � � , xp)

�. Thenwe have

dF(t, X(t), �(t)) = �F�t(t, X(t), �(t))dt +

p∑k=1

�F�xk

(t, X(t), �(t))dXk(t)

+ 1

2

p∑k=1

p∑l=1

�2F�xk�xl

(t, X(t), �(t))dXk(t)dXl(t)

+S∑

j=1

q�(t)jF(t, X(t), j)dt + dMFt ,

where the uncertain stochastic process (MFt )0≤t≤T is a real-valued martingale

relative to the filtered probability space (�,�,�,�), dBkt dBl

t = kldt, dBkt dt =

dCıtdC

t = dCıtdt = dBk

t dCıt = 0, for k, l = 1, � � � , m, i, j = 1, � � � , n� Here, kl is

defined in Theorem 1.

Proof. It is well known that almost every sample path of �(t) is a right-continuousstep function. It is useful to recall that a continuous-time Markov chain �(t) canbe represented as a stochastic integral with respect to a Poisson random measure (cf.Lemma 3 in Skorohod 1989):

d�(t) =∫�

h(X(t), �(t), z)�(dz × dt),

where �(dz × dt) is a Poisson random measure with intensity m(dz) × dt in whichm is a Lebesgue measure on �.

Let 0 < �1 < · · · < �n < t be all the times when �(t) has a jump. FromTheorem 1, we get

F(�1, X(�1), �) − F(0, X(0), �)

=∫ �1

0

�F�t(s, X(s), �(s))ds +

∫ �1

0

p∑k=1

�F�xk

(s, X(s), �(s))dXk(s)

+ 1

2

∫ �1

0

p∑k=1

p∑l=1

�2F�xk�xl

(s, X(s), �(s))dXk(s)dXl(s),

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

Optimal Control of Uncertain Stochastic Systems 77

F(�k+1, X(�k+1), �) − F(�k, X(�k), �)

=∫ �k+1

�k

�F�t(s, X(s), �(s))ds +

∫ �k+1

�k

p∑k=1

�F�xk

(s, X(s), �(s))dXk(s)

+ 1

2

∫ �k+1

�k

p∑k=1

p∑l=1

�2F�xk�xl

(s, X(s), �(s))dXk(s)dXl(s),

F(t, X(t), �) − F(�n, X(�n), �)

=∫ t

�n

�F�t(s, X(s), �(s))ds +

∫ t

�n

p∑k=1

�F�xk

(s, X(s), �(s))dXk(s)

+ 1

2

∫ t

�n

p∑k=1

p∑l=1

�2F�xk�xl

(s, X(s), �(s))dXk(s)dXl(s)

Substituting � = i in the first equation, � = ��k in the second, and �= ��n inthe third and adding them over k from 1 to n + 1, we obtain

F(t, X(t), �(t)) − F(0, X(0), i)

=∫ t

0

�F�t(s, X(s), �(s))ds +

∫ t

0

p∑k=1

�F�xk

(s, X(s), �(s))dXk(s)

+ 1

2

∫ t

0

p∑k=1

p∑l=1

�2F�xk�xl

(s, X(s), �(s))dXk(s)dXl(s)

+n∑

k=1

[F(�k, X(�k), �(�k)) − F(�k, X(�k), �(�k−))]

=∫ t

0

�F�t(s, X(s), �(s))ds +

∫ t

0

p∑k=1

�F�xk

(s, X(s), �(s))dXk(s)

+ 1

2

∫ t

0

p∑k=1

p∑l=1

�2F�xk�xl

(s, X(s), �(s))dXk(s)dXl(s)

+∫ t

0

∫�[F(s, X(s), �(s) + h(X(s), �(s), z)) − F(s, X(s), �(s))]m(dz)ds

+∫ t

0

∫�[F(s, X(s), �(s) + h(X(s), �(s), z)) − F(s, X(s), �(s))]�(dz × ds),

where �(dz × dt) = �(dz × dt) − m(dz) × dt is a martingale measure relative to thefiltered probability space (�,�,�,�). Similar to the discussion of Lemma 3 (p.104) in Skorohod (1989), noting that∫ t

0

∫�[F(s, X(s), �(s) + h(X(s), �(s), z)) − F(s, X(s), �(s))]m(dz)ds

=S∑

j=1q�(s)jF(s, X(s), j),

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

78 W. Fei

we get the claim by differentiating the above equation. Thus, the proof iscomplete. �

Yong and Zhou (1999), discussed the principle of optimality and the HJBequation for stochastic optimal control. Next, we similarly explore those for uncertainstochastic optimal control. First, we derive the principle of optimality for uncertainstochastic optimal control (4) with the constraint (3).

Theorem 2. (Principle of optimality) Let Assumption 1 hold. For any (t, x) ∈[0, T] × �p, we have

V(t, x, i) = infD∈

E

[∫ t

t�1(s, X(s), D(s), �(s))ds + V(t, X(t), �(t))

],

0 ≤ t ≤ t ≤ T� (5)

Proof. From Assumption 1 (2), for u ∈ , 0 ≤ t ≤ t ≤ T, we get

0 ≥ −E

[∫ t

t�−

1 (s, X(s), D(s), �(s))ds

]

≥ −E

[∫ t

t(1 + �1(X(s))|D(s)|2)ds

]> −∞,

and

0 ≤ E

[∫ t

t�+

1 (s, X(s), D(s), �(s))ds

]< +∞,

which shows ∣∣∣∣∣E[∫ t

t�1(s, X(s), D(s), �(s))ds

]∣∣∣∣∣ < +∞�

Likewise, we can obtain

|E[�2(T, X(t), �(T))]| < +∞�

Thus, we show that the right-hand side of the identity (5) is well defined.Next, we denote the right side of (5) by V(t, x, i). It follows from the definition

of V(t, x, i) that, for any D ∈ ,

V(t, x, i) ≤ E[∫ t

t �1(s, X(s), D(s), �(s))ds

+ ∫ Tt �1(s, X(s), D(s), �(s))ds + �2(T, X(T), �(T))

]�

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

Optimal Control of Uncertain Stochastic Systems 79

Because the uncertain processes dCs(s ∈ [t, t)) and dCs(s ∈ [t, T]) are independent,we know that∫ t

t�1(s, X(s), D(s), �(s))ds and

∫ T

t�1(s, X(s), D(s), �(s))ds

are independent relative to the uncertain space (� ,�,�). Thus, by Theorem 1 inZhu (2010), we get

V(t, x, i) ≤ E{∫ t

t �1(s, X(s), D(s), �(s))ds

+E[∫ T

t �1(s, X(s), D(s), �(s))ds + �2(T, X(T), �(T))]}

�

Taking the infimum for above inequality with respect to D ∈ , we have V(t, x, i) ≤V(t, x, i)� On the other hand, for ∀D ∈ , we derive

E[∫ T

t�1(s, X(s), D(s), �(s))ds + �2(T, X(T), �(T))

]

= E

{∫ t

t�1(s, X(s), D(s), �(s))ds

+ E[∫ T

t�1(s, X(s), D(s), �(s))ds + �2(T, X(T), �(T))

]}

≥ E

[∫ t

t�1(s, X(s), D(s), �(s))ds + V(t, X(t), �(t))

]≥ V(t, x, i),

which shows V(t, x, i) ≥ V(t, x, i)� Consequently, V(t, x, i) = V(t, x, i)� Thus, theproof of the theorem is complete. �

Let C1,2([0, T] × �p;�) denote all functions V(t, x, i) on [0, T] × �p that arecontinuously differentiable in t, continuously twice differentiable in x for each i ∈ � .If V(·, i) ∈ C1,2([0, T] × �p;�), define operators Li(D)V, for each i ∈ � , by

Li(D)V(t, x, i)�= �V

�t(t, x, i) +

p∑k=1

�V�xk

(t, x, i)fk(t, x, D, i)

+ 1

2

p∑k=1

p∑l=1

�2Vxkxl

(t, x, i)m∑ı=1

gkı(t, x, D, i)glı(t, x, D, i)�

In what follows, we give the optimal equation of the optimal control problem.

Theorem 3. Let Assumption 1 hold. Then V is a solution of the followingterminal problem of a HJB equation

infD∈

�Li(D)V(t, x, i) + �1(t, x, D, i)� = �iV(t, x, i) −∑

j∈�\�i�qijV(t, x, j) (6)

with the terminal condition V(T, x, i) = �2(T, x, i).

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

80 W. Fei

Proof. Let a and b satisfy 0 < a < |X(t)| < b < + ∞. We define the first hitting

times �a�= inf�t ≥ 0 : |X(t)| = a�, �b

�= inf�t ≥ 0 : |X(t)| = b�, and ��= �a ∧ �b.

For ∀t, t ∈ [0, T], from Lemma 1 we obtain

V(t ∧ �, X(t ∧ �), �(t ∧ �)) − V(t, x, i)

=∫ t∧�

t[L�(s)V(s, X(s), �(s)) +

S∑j=1

q�(s)jV(s, X(s), j)]ds

+∫ t∧�

t

p∑k=1

m∑l=1

gkl(s, X(s), D(s), �(s))�V�xk

(s, X(s), �(s))dBls

+∫ t∧�

t

p∑k=1

n∑l=1

hkl(s, X(s), D(s), �(s))�V�xk

(s, X(s), �(s))dCls

+ (MVt∨� − MV

t )� (7)

Because the uncertain stochastic process (MVt )0≤t≤T is a real-valued martingale

relative to the filtered probability space (�,�,�,�), we easily know E[MVt∨�] =

E[MVt ]. Note that for all i ∈ � , the functions Vx(t, x, i) are bounded when |x| is

bounded. Therefore, by Assumption 1 (1) and E[∫ T0 |D(t)|2dt] < +∞, we easily

deduce

E

∣∣∣∣∫ t∧�t

p∑k=1

m∑l=1

gkl(s, X(s), D(s), �(s)) �V�xk(s, X(s), �(s))dBl

s

∣∣∣∣2

< +∞�

Hence, we get

E[∫ t∧�

t

p∑k=1

m∑l=1

gkl(s, X(s), D(s), �(s)) �V�xk(s, X(s), �(s))dBl

s

]= 0� (8)

On the other hand, similar to the discussion of Theorem 12.17 in Liu (2011), it iseasy to check that

E[∫ t∧�

t

p∑k=1

n∑l=1

hkl(s, X(s), D(s), �(s)) �V�xk(s, X(s), �(s))dCl

s

]= o(t − t)� (9)

Taking the expectation with respect to (7), together with (8) and (9), we have

E[V(t, X(t), �(t))]

= V(t, x, i) + E

∫ t∧�

t(L�(s)V(s, X(s), �(s)) +

S∑j=1

q�(s)jV(s, X(s), j))ds

+ o(t − t)� (10)

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

Optimal Control of Uncertain Stochastic Systems 81

By using the principle of optimality (Theorem 1) and (10), we get

0 = infD∈

E

[∫ t∧�

t�1(s, X(s), D(s), �(s))ds

+∫ t∧�

t(L�(s)V(s, X(s), �(s)) +

S∑j=1

q�(s)jV(s, X(s), j))ds + o(t − t)

�

Now letting a ↓ 0 and b ↑ ∞, we obtain � → T� Therefore, applying the Monotoneconvergence theorem, we have

0 = infD∈

E

[∫ t

t�1(s, X(s), D(s), �(s))ds

+∫ t

t(L�(s)V(s, X(s), �(s)) +

S∑j=1

q�(s)jV(s, X(s), j))ds + o(t − t)

� (11)

Dividing Eq. (11) by t − t and letting t → t, we get Eq. (6). The terminal conditionof Eq. (6) holds obviously. Thus, we complete the proof. �

UNCERTAIN RANDOM FINANCIAL MARKET SETUP

Applying the above results, we introduce a financial market with regime switchingin this section. Let a filtered probability space (�,�,�,�) hosting m-dimensionalWiener process (Bt)0≤t≤T and an uncertain space (� ,�,�) hosting n-dimensionalcanonical process (Ct)0≤t≤T, respectively. The Wiener process (Bt)0≤t≤T and thecanonical process (Ct)0≤t≤T model risky uncertainties and behavior uncertainties infinancial assets, respectively. The regime switching is a Markov chain that has astrongly irreducible generator Q as in the previous section.

Assume that uncertain stochastic processes P0(t) and Pk(t), k = 1, � � � , m, on[0, T] represent the prices of the riskless asset and the m risky assets, respectively.They satisfy the Markovian switching uncertain stochastic differential equations

dP0(t) = r�(t)(t)P0(t)dt,

dPk(t) = �k�(t)(t)Pk(t)dt + �k

�(t)(t)Pk(t)dBt + �k�(t)(t)Pk(t)dCt,

with initial prices P0(0) = 1 and Pk(0) = pk > 0 and initial state �(0) = �0.Here, r�(t)(t), ��(t)(t) = (�1

�(t)(t), � � � , �m�(t)(t))

�, �k�(t)(t) = (�k1

�(t)(t), � � � , �km�(t)(t)),

and �k�(t)(t) = (�k1

�(t)(t), � � � , �kn�(t)(t)) (k = 1, � � � , m) are deterministic and bounded

interest rate, expected returns, random volatility functions and uncertain volatilityfunctions, respectively. Denote

��(t)(t) = (�1�(t)(t), � � � , �

m�(t)(t))

�, ��(t)(t) = (�1�(t)(t), � � � , �

m�(t)(t))

��

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

82 W. Fei

We suppose that �i(t)�= �i(t)�i(t)�, i ∈ � , are positive definite, and the

coefficients of the market (i.e., r, �, �, �) depend on the regime of an economy.We easily know that �i(t) is deterministic and bounded. The agent chooses aportfolio � = ��(t) = (�1(t), � � � , �m(t))�, t ∈ [0, T]�, representing the fraction ofwealth invested in each risky asset. We need a technical condition to be satisfied.A portfolio vector process is an uncertain stochastic vector process � such that

E[∫ t

0 ��(s)��(t)�(s)ds

]< +∞ for all t ∈ [0, T]. The fraction of wealth invested in

the riskless asset at time t ∈ [0, T] is then 1 − ∑mk=1 �

k(t). The investor also choosesa consumption rate process c = �c(t), t ∈ [0, T]�: a nonnegative uncertain stochastic

process such that E[∫ t

0 c2(s)ds]< ∞ for all t ∈ [0, T].

Let the market price of risk of the market be ��(t)(t)�= �−1

�(t)(t)(��(t)(t) −r�(t)1), where 1 = (1, � � � , 1)�. Now the evolution of the nominal wealth W(t) attime t can be written as

dW(t) = r�(t)(t)W(t)dt + W(t)��(t)��(t)(t)d[Bt + ��(t)(t)]+ W(t)��(t)��(t)(t)dCt − c(t)dt�

(12)

We introduce the utility functions U1(·) and U2(·) of consumption and wealth,respectively, which are assumed to be twice differentiable, strictly increasing, andconcave. Moreover, U′

l(0) = ∞, U′l(∞) = 0, l = 1, 2. It is easy to see that there

exists a constant K such that Ul(y) ≤ K(1 + y2), l = 1, 2�Given t ∈ [0, T], we now define the objective function, which depends, on the

the market regimes as follows:

�(t, x, i; �, c) = E[∫ T

te−�(s−t)U1 (c(s)) ds + e−�(T−t)U2 (W(T))

],

where for s ∈ [t, T], the wealth W(s) with W(t) = x follows (12), � isthe utility discount rate, which may be different from the risk-free rate.

The control process D�= (�, c) that satisfies

E[∫ T

0

e−�tU−1 (c(t)) dt + e−�TU−

2 (W(T))]< +∞

will be called an admissible control process. The set of all admissible controls will bedenoted by �. We define the value function by

�(t, x, i) = sup(�,c)∈�

�(t, x, i; �, c), (13)

which shows that the agent selects consumption and investment processes in orderto maximize the sum of his expected discounted utilities from consumption andterminal wealth.

In what follows, we deduce the HJB equation satisfied by optimal consumptionand portfolio policies. For this, denote operators i(D)�, for each i ∈ � , by

i(D)�(t, x, i) = (�, c)�(t, x, i)

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

Optimal Control of Uncertain Stochastic Systems 83

�= 1

2x2���i(t)��xx + x���i(t)�i(t)�x(t, x, i)

+ ri(t)x�x − c�x − �� + �t(t, x, i)�

We now give the equation of optimality for the optimal policy as the main theoremin this section.

Theorem 4. (Equation of optimality) For each i ∈ � , �(·, i) is a solution ofthe following terminal problem of the HJB equation

sup(�,c)∈�

� i(�, c)�(t, x, i) + U1(c)� = �i�(t, x, i) −∑

j∈�\�i�qij(t, x, j)�(t, x, i) (14)

with the terminal condition �(T, x, i) = U2(x)�

Proof. In terms of the state equation (12), we assume, in Theorem 3,

�1(t, x, �, c, i) = −e−�tU1(c),

�2(T, x, i) = −e−�TU2(x),

f(t, x, �, c, i) = ri(t)x + x���i(t)�i(t) − c,

g(t, x, �, c, i) = x���i(t),

h(t, x, �, c, i) = x���i(t),

which show that the value function �(t, x, i) in (13) equals −e�tV(t, x, i), where Vsolves Eq. (6). By the definitions of utility functions U1(·) and U2(·), we can take�1(x) = 1 and �2(x) = x in Assumption 1 (2). Hence, the conditions of Theorem 3are fulfilled; moreover, from the HJB equation (6), we derive that �(t, x, i) satisfiesEq. (14). Thus, the proof is complete. �

Corollary 1. The optimal consumption and portfolio problem (13) has theoptimal policy as follows:

c(t, x, �(t)) = �1(�x(t, x, �(t))),

�(t, x, �(t)) = − �x(t, x, �(t))x�xx(t, x, �(t))

(���(t)(t)

)−1��(t)(t)�

(15)

where �1(·) is the unique inverse function of U′1(·).

Proof. Because the consumption policy can be solved by the concave maximizationin (14), we get the optimal feedback consumption policy. In order to get the portfoliopolicy, we make a quadratic maximization problem in (14). �

OPTIMAL STRATEGIES FOR CRRA UTILITY

In this section, we discuss a special case of an investor with CRRA utility. Let theutility functions of an investor be Ul(z) = z1−

1− , > 0, �= 1, l = 1, 2� Thus,

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

84 W. Fei

we obtain �l(y) = y−1/ , where �l(·) are the inverse functions of U′l(·), l = 1, 2.

From (14) of Theorem 4 and optimal policy (�, c) of (15), we easily deduce that thevalue function � satisfies the HJB equation as follows:

�t(t, x, i) − ��(t, x, i) + ri(t)x�x(t, x, i) +

1 − �x(t, x, i)

−1

= �i�(t, x, i) −∑

j∈�\�i�qij�(t, x, j) (16)

with the terminal condition �(T, x, i) = U2(x) = 11−

x1− .We guess the form of solution to Eq. (16) as follows

�(t, x, i) = 1

1 − A

i (t)x1− ,

which shows, for i = 1, � � � , S,

A −1i (t)A′

i(t) − !i(t)A i (t) + A −1

i (t) +∑

j∈�\�i�qijA

j (t) = 0, Ai(T) = 1, (17)

where

!i(t)�= � + �i − (1 − )ri(t) − 1 −

2 |�i(t)|2�

By the existence and uniqueness theorem (cf. Hirsch et al. 2006, p. 144), the systemof equations (17) has the unique solution �Ai(t), i = 1, � � � , S�. From (15), we get

c(t, x, i) = xAi(t)

,

�(t, x, i) = 1

(��

i (t))−1

�i(t)�(18)

The formula (18) shows that an investor’s consumption rate is higher as hebecomes wealthier. In addition, the investor invests more in risky assets with, thehigher market price of risk and the lower the risk of the risky assets.

From now on, for the sake of convenience, we only discuss the followingfirst-order nonlinear differential dynamic system (i.e., S = 2):

A −11 A′

1 − !1A 1 + A −1

1 + �1A 2 = 0, A1(T) = 1,

A −12 A′

2 − !2A 2 + A −1

2 + �2A 1 = 0, A2(T) = 1�

(19)

By the numerical solution to Eqs. (19), we give the economic analysis of therelated results.

Now let us consider the following numerical example. There are two financialassets: a risk-free asset and a risky asset. The market coefficients follow: �1 = 0�15,�2 = 0�25, �1 = 0�25, �2 = 0�6, r1 = 0�05, r2 = 0�01, �1 = 1�2, �2 = 2�5, T = 1.

First, we consider a highly risk-averse investor with = 10 and �= 0�07. Usingour numerical parameter values in (19), from (18) we see how the optimal nominalconsumption to wealth ratio depends on time t. The results are shown in Figure 1.Similarly, we consider a highly risk-tolerant investor with = 0�7 and �= 0�8. Using

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

Optimal Control of Uncertain Stochastic Systems 85

FIGURE 1 An investor with high risk aversion ( = 10 and �= 0�07).

our numerical parameter values in (4.4), from (4.3) we see how the optimal nominalconsumption-to-wealth ratio depends on time t. The results are shown in Figure 2.From Figures 1 and 2 we see that an investor with high risk aversion increases hisoptimal nominal consumption rate as time goes on, and his optimal consumptionrate is higher during the state of a good economy than during the state of a badeconomy; conversely, an investor with high risk tolerance decreases his optimalnominal consumption rate as time goes on, and while his optimal consumptionrate is lower during the state of a good economy than during the state of a badeconomy.

FIGURE 2 An investor with high risk tolerance ( = 0�7 and �= 0�8).

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

86 W. Fei

CONCLUSIONS AND COMMENTS

In this article, we first describe an uncertain stochastic control system with Markovmodulation. In order to build an optimality equation for control laws, we generalizean Itô-Liu formula for uncertain stochastic processes with Markovian switching.From this, the HJB equation of the value function for the uncertain stochasticMarkov-modulated control system is provided, from which the optimal control lawcan be deduced. This is one of the main contributions of this article.

By the application of optimality equation for optimal control law, we studythe optimal consumption and portfolio problem under an uncertain stochasticenvironment with regime switching, in which the prices of the risky assets aredriven by risky uncertainty sources and Liu’s uncertainty sources. An investor’s utilitycomes from both consumption and wealth. Employing the results on a Markovian-modulated uncertain stochastic control, we derive the HJB equation satisfied by theoptimal control law (consumption and portfolio), from which we get the optimalconsumption and portfolio polices. This is another contribution of our article. As faras we know, Zhou and Yin (2003) obtained explicit solutions to classical stochasticcontrol problems with regime switching. Sotomayor and Cadenillas (2009) presentedversions of the HJB equation for classical stochastic control with regime switchingin infinite horizon. However, the framework of our model is different from theirs.

Finally, for an investor with CRRA utility functions, by using Corollary 3, wegive an explicit formula for optimal policies. Through the numerical simulation, aneconomic analysis of optimal policies is made. We find that the qualitative behaviorof the consumption-to-wealth ratio depends not only on the regime but also on thelevel of risk aversion.

In addition, a stochastic financial market with inflation described in Bensoussanet al. (2009) can be extended to an uncertain stochastic market with both regimeswitching and inflation, which will be further studied in the future.

FUNDING

This project was supported by the National Natural Science Foundation ofChina (71171003) and the Anhui Natural Science Foundation of Universities(KJ2012B019, KJ2013B023).

REFERENCES

Bensoussan, A., J. Keppo, and S. P. Sethi. ‘‘Optimal Consumption and PortfolioDecisions with Partially Observed Real Prices.’’ Mathematical Finance 19,no. 2 (2009): 251–79.

Buffington, J. and R. J. Elliott. ‘‘American Options with Regime Switching.’’ InternationalJournal of Theory Applied Finance 5, no. 5 (2002): 497–514.

Fei, W. Y. ‘‘Existence and Uniqueness of Solution for Fuzzy Random Differential Equationswith Non-Lipschitz Coefficients.’’ Information Sciences 177, (2007a): 4329–37.

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

Optimal Control of Uncertain Stochastic Systems 87

Fei, W. Y. ‘‘A Generalization of Bihari’s Inequality and Fuzzy Random Differential Equationswith Non-Lipschitz Coefficients.’’ International Journal of Uncertainty, Fuzziness andKnowledge-Based Systems 15, no. 4 (2007b): 425–39.

Fei, W. Y. ‘‘Optimal Consumption and Portfolio Choice with Ambiguity and Anticipation.’’Information Sciences 177, (2007c): 5178–90.

Fei, W. Y. ‘‘Optimal Portfolio Choice Based on �-MEU under Ambiguity.’’ Stochastic Models25, (2009a): 455–82.

Fei, W. Y. ‘‘Uniqueness of Solutions to Fuzzy Differential Equations Driven by Liu’s Processwith Non-Lipschitz Coefficients.’’ Proceedings of the Sixth International Conference onFuzzy Systems and Knowledge Discovery (FSKD’ 09), 2009, Vol. 6, 565–9, 2009b.

Fei, W. Y. ‘‘Optimal Control of Markovian Switching Systems with Applications to PortfolioDecisions under Inflation.’’ IEEE Transactions on Automatic Control, (2010; in press).

Fei, W. Y. ‘‘Optimal Consumption and Portfolio under Inflation and Markovian Switching.’’Stochastics 85, no. 2 (2013): 272–85.

Fei, W. Y., H. J. Liu, and D. R. Ding. ‘‘On LaSalle-Type Theorems for Fuzzy DifferentialDelay Equations.’’ Proceedings of 2010 International Conference on Computer Applicationand System Modeling (ICCASM 2010), Vol. 2, 81–5, 2010.

Fei, W. Y. and R. Q. Wu. ‘‘Anticipative Portfolio Optimization under Constraints and aHigher Interest Rate for Borrowing.’’ Stochastic Analysis and Applications 20, (2002):311–45.

Feng, J.-E., J. Lam, and Z. Shu. ‘‘Stabilization of Markovian Systems via Probability RateSynthesis and Output Feedback.’’ IEEE Transactions Automatic Control 55, no. 3 (2010):773–77.

Guo, X. and Q. Zhang. ‘‘Closed-Form Solutions for Perpetual American put Options withRegime Switching.’’ SIAM J. Contl. Opt. 64, no. 6 (2004): 2034–49.

Hamilton, J. ‘‘A New Approach to the Economic Analysis of Nonstationary Time Series andthe Business Cycle.’’ Econometrica 57, (1989): 357–84.

Hirsch, M. W., S. Smale, and R. L. Devaney. Differential Equations, Dynamical Systems andan Introduction to Chaos. Singapore: Elsevier 2006.

Huang, L. R. and X. R. Mao. ‘‘On Input-to-State Stability of Stochastic Retarded Systems withMarkovian Switching.’’ IEEE Trans. Automat. Control 54, no. 8 (2009): 1898–1902.

Huang, X. ‘‘A New Perspective for Optimal Portfolio Selection with Random Fuzzy Returns.’’Information Sciences 177, (2007): 5404–14.

Kaleva, O. ‘‘Fuzzy Differential Equations.’’ Fuzzy Sets and Systems 24, (1987): 301–17.Karatzas, I. and S. Shreve. Brownian Motion and Stochastic Calculas. 2nd ed. New York:

Springer, 1991.Karatzas, I. and S. Shreve. Methods of Mathematical Finance. New York: Springer, 1998.Kwakernaak, H. ‘‘Fuzzy Random Variables-I: Definition and Theorem.’’ Information Science

15, (1978): 1–29.Liu, B. D. Uncertainty Theory, 2011. Available at: http://orsc.edu.cn/liu.Liu, Y. H. ‘‘Uncertain Random Variables: A Mixture of Uncertainty and Randomness.‘‘ Soft

Computing Journal 17 (2013): 625–34.Liu, Y. K. and B. Liu. ‘‘Fuzzy Random Variables: A Scalar Expected Value Operator.’’ Fuzzy

Optimization and Decision Making 1, (2003): 143–60.Mariton, M. Jump Linear Systems in Automatic Control. New York: Marcel Dekker, 1990.Merton, R. ‘‘Lifetime Portfolio Selection Under Uncertainty: The Continuous-time Case.’’

Rev. Econ. Stat. 51, (1969): 247–57.Merton, R. ‘‘Optimum Consumption and Portfolio Rules in a Continuous Time Model.’’ J.

Econ. Theory 3, (1971): 373–413.Puri, M. L. and D. Ralescu. ‘‘Fuzzy Random Variable.’’ Journal of Mathematical Analysis and

Applications 114, (1986): 409–22.

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014

88 W. Fei

Skorohod, A. V. Asymptotic Methods in the Theory of Stochastic Differential Equations.Providence: American Mathematical Society, 1989.

Sotomayor, L. R. and A. Cadenillas. ‘‘Explicit Solutions of Consumption-InvestmentProblems in Financial Markets with Regime Switching.’’ Mathematical Finance 19,no. 2 (2009): 251–79.

You, C. L. Some Extensions of Wiener-Liu Process and Ito-Liu Integral, 2007. Available at:orsc.edu.cn/process/071019.pdf

Zhang, Q. and G. Yin. ‘‘Nearly-Optimal Asset Allocation in Hybrid Stock InvestmentModels.’’ Journal of Optimization Theory and Applications 121, no. 2 (2004): 197–222.

Yong, J. M. and X. Y. Zhou. Stochastic Controls: Hamiltonian Systems and HJB Equations.New York: Springer, 1999.

Zhou, X. Y. and G. Yin. ‘‘Markowitz’s Mean-Variance Portfolio Selection with RegimeSwitching: A Continuous Time Model.’’ SIAM Journal on Control Optimization 42,no. 4 (2003): 1466–82.

Zhu, Y. G. ‘‘Uncertain Optimal Control with Application to a Portfolio Selection Model.’’Cybernetics and Systems 41, (2010): 535–47.

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

21:

28 1

7 N

ovem

ber

2014