1

Upper-Lower Solution Method for Differential

Riccati Equations from Stochastic LQR Problems

Libin Mou

Department of Mathematics

Bradley University

Peoria, IL 61625

Abstract

We use upper and lower solutions to study the existence and properties of solutions to differential

Riccati equations arising from stochastic linear quadratic regulator (LQR) problems. The main results

include an interpretation of upper and lower solutions, comparison theorems, an upper-lower solution

theorem, necessary and sufficient conditions for existence of solutions, an estimation of maximal

existence intervals of solutions and an approximation of solutions. Many of the results are new, while

others are generalizations of some known results.

1. Introduction, Notations and Definition

In this paper we study the following matrix differential equation

ÚÝÛÝÜa b

a b a b a bT � E T � T E � G T G � � T

� F T � H T G � W V � H T H F T � H T G � W œ !

w

"

X X

X X X X XX

K

ß> R

C�"

T œa b (1)

of or , where is the transpose of , , Riccati type on a fixed interval M œ > ß Ð�_ß > Ó E E T œc d! ".T.>> R"

wX

is a symmetric matrix, C is a linear map of symmetric matrices, and EßFßGßHß ß V ß WK are matrix

functions in satisfying the M conditions stated in (4).

As a motivation for equation (1), we consider a stochastic LQR)linear quadratic regulator (

problem with noise depending on both of the state and control. For let be a standard Brownian= − Mß [

motion with almost surely and [ = œ !a b hc d=ß >" be the set of all square integrable control processes

defined on and adapted to the -field generated by . For c d=ß [> >" "5 D − ? − =ß‘ h8 and , consider thec dfollowing state equation and cost function :N ?a b

� a b a b a ba b š ›a b a b a b'.B œ EB � F? .> � GB � H? .[ß > − Mà B = œ D

N ? œ I B B � B B � #? W B � ? V? .>

, (2.1)

, (2.2)(2)X X X X> R > K" "

>"

=

where Ief represents the expectation of the enclosed variable. The problem is to

2

maximize/ (3)minimize for N ? ? − =ß Þa b c dh >"

See [26, Ch. 6] for a detailed description of this problem. This problem leads to equation (1) with .C œ !

For a derivation, see [3], [5], [7], [26] or Theorem 1 below. The inclusion of the term is important forC

application of (1) to stochastic control problems with Markovian jumping noises and differential game

problems with state-dependent noises; see [22] and [15], for example. Although the term may beG T GX

considered as a part of , we will keep them separated for generality. Readers who are interested inCa bT

the equation associated with problem (3) may assume that .C œ !

Riccati equations (differential, difference and algebraic) appear in various control and min-max

problems. The classical differential Riccati equation is (1) with . For this case, theG œ H œ W œ œ !C

existence, comparison, approximation and other properties of solutions have been extensively studied; see

[2], [4], [6], [13], [14], [20], [23], [27] and many references therein. Many results for classical Riccati

equations have been extended to in equation (1) with [25], [10], [11] and [9]. Existence andH œ W œ !

approximation of solutions to (1) with have been obtained in [ .G œ œ !C [7], 8] and [26]

We will prove comparison theorems, an upper-lower solution theorem, necessary and sufficient

conditions for existence of solutions, an estimation of maximal existence intervals of solutions and an

approximation of solutions for equation (1) under general settings. In particular, are allowed toK R and

be indefinite and V � H T HX may be either positive or negative semidefinite.

The method of upper and lower solutions has been introduced for differential equations as early as

1940's. Although Riccati inequalities have been studied or used in some literature of Riccati equations

(e.g., [23] and [21]), this paper, together with [17], [16] and [15], appear to be the first systematic

application of the method of upper and lower solutions to Riccati equations. It turns out this method has

many desired merits. For example, it naturally links equation (1) with with the LQR problem (3).C œ !

It derives the main results under general assumptions. It gives verifiable necessary and sufficient

conditions for the existence of solutions. It also gives algorithms for approximating solutions. It can be

used to estimate the maximal existence intervals of solutions to differential Riccati equations without

solving the equations. It applies to Riccati equations of different types.

This paper is organized as follows. Notations and assumptions are introduced in this section

followed by the definitions of upper and lower solutions. In Section 2, we prove a relationship between

upper and lower solutions and the well-posedness of the LQR problem (3). In addition, some intrinsic

structural properties of equation (1) are proved in Section 2. In Section 3 we prove some general

comparison theorems and an upper-lower solution theorem for equation (1). As an application, we obtain

some necessary and sufficient conditions for the existence of solutions to (1). Furthermore, we show that

3

the solution can be approximated by a sequence of solutions to linear equations. In Section 4, we apply

the upper-lower solution theorem to estimate the maximal existence intervals of solutions to (1). Finally,

in Section 5 we mention a generalization of equation (1).

The author would like to thank Stan Liberty and Mike McAsey for stimulating discussions.

Notations. Denote by ’8 the set of all real symmetric matrices. We write ( ) if ,8 ‚ 8 Q   R Q � R Q

R Q � R À Ä   !− ’ C ’ ’ C8 8 8 and positive semidefinite (definite). For a map we write ifis a

Ca bQ   Q   !! > for each . For a Hilbert space and an interval or , is— —M œ > ß Ð�_ß > Ó P Mßc d a b! "_

"

the space of all bounded and measurable functions from to . Furthermore, we define M P Mß œ— —"ß_a bÖT − P Mß ß T − P Mß ×Þ_ w _a b a b— —

Additional notations will be introduced later. For reader's convenience, we list below the

frequently used notations in this paper.

Q , which is the left-hand side of (1)Xa bT is a short hand for T � T � Tw L a b a bC .

LQ L Q , where L , Q are defined in (5). a b a b a b a b a bT œ K � T � T T T Here is "constant" term,K

L is the linear term, while Qa bT a bT is the "quadratic" term.

and are defined in (7). The facts and explain the choices of e f e f ea b a b a b a bT T ! œ V ! œ W

and .f

is a generic notation for a feedback matrix. is the set of all feedback matrices associatedO TŠa bwith ; ; is the unique feedback matrix defined in (7) and (8) in terms of theT O − T T − Ts Š ^ Ša b a b a binverse or psuedoinverse of .e e ea b a b a bT T T�" �

is defined in (19). The fact explain the choice.Z Za b a bO ! œ K

is defined in (22). Note that L_ _a b a b a bO à T !à T œ T Þ

is the boundary value of (1); often represent a solution, an upperR − T ß ] ß ^ −’8 P Mß"ß_a b’8

solution and a lower solution to (1)Þ EßFßGßHß ß V WK and are coefficient matrix functions of (1).

Assumption. The basic assumption for EßFßGßHß ßVßWßK R and in (1) isC

ÚÛÜ a b a b

C C−  P ß_a b’ ’

‘ ‘

’ ’ ’

8 8

8‚8 8‚5

8 8

is linear and (4)

!

K R

,

, .Eß G − P Mß FßHß W − P ÐMß Ñ

V − P ÐMß Ñ − P Mß ß −

_ _

_ 5 _

, ,

X

To write equation (1) concisely, we denote, for T − P Mß_a b’8 ,

ÚÛÜ a b a b

LQ ,

LQ Q(5)

a b a b a b a ba b a bT œ E T � T E � G T G

F T � H T G � W V � H T H F T � H T G � WT œ K � T �

X X

X X X X XX

,

LT œ

T

�"

4

with parameters . Thus (1) becomesEßFßGßHß ß V WK and satisfying (4)

T � T � T œ !ß T œw"L (1)Q . a b a b a bC > R

To indicate the parameters, we may say sayLQ with parameters a b† EßFßGßHß ß V WK and , and

equation (1) with parameters EßFßGßHß ßVßWßK RC and Þ

We remark that Q and LQ may be well-defined even if . Ia b a bT T V � H T HX is singular f

V � H T H TX is nonsingular, then Q can be written asa bQ a b a b a bT œ T T œf fe ^ e ^a b a b a b a bT T T T�" X , (6)

where , aree ^a b a bT Tfa bT and

e

f

^ e f

a ba ba b a b a bT

T F T � H T G � W

T œ T T

œ V � H T H

œ

X

X X

, (7),

.�"

Here can be considered as perturbations of and with respect to . The termea bT and fa bT V W T

^ e fa b a b a bT œ T T�" appears frequently as the optimal feedback matrix for problem (3) when is aT

solution to (1) with ; see Theorem 1 below.C œ !

If is singular, then we defineea bT

^ e fa b a b a bT œ T T� , (8)

where is the pseudoinverse of . Recall that any matrix has a unique pseudoinverse e ea b a bT T Q Q� �

with the following properties (see [18] and [1]).

QQ Q œ Qß Q QQ œ Q

Q − Q − QQ œ Q Q

Q   ! Q   !

� � � �

8 � 8 � �

�

. (9)If , then , and .

if and only if .’ ’

Note that always exists, but it may not satisfy . This^ e f f e ^a b a b a b a b a b a bT œ T T T œ T T�

leads to the following definition.

Definition 1. If satisfies , then is said to be .T − T œ T T TP Mß_a b’ f e ^8 a b a b a b feasible

Denote . Ša b e fT œ −O P Mß ß T œ T O_ 5‚8a b a b a b‘ f e Obviously, if is feasible, thenT

^a bT − Á gŠ Ša b a bT T and so . The converse is also true. We have

Proposition 1. If is feasible and for each Š Ša b a bT TÁ g, then T O − ,

Q (10)a bT œ ^ e ^ ea b a b a b a bT T T œ O T OX X ,

5

where ^ e fa b a b a bT œ T T� .

Proof. If O − Ša bT Á g T œ T O T œ T T T, then , we havef e e e e ea b a b a b a b a b a b. Since �

e ^ e e f e e e e fa b a b a b a b a b a b a b a b a b a bT T œ T T T œ T T T œ T œ T T� � O O . This shows that is

feasible. Furthermore, from that ^a bT œ Oe f e ea b a b a b a bT T œ T T� � and property (9), we have

^ e ^ e e e e e ea b a b a b a b a b a b a b a b a bT T T œ O T T T T T TX X X� � O œ O O .

This shows (10). ¨

Proposition 1 shows that Q is well-defined by (10). Each if thenŠ Ša b a b a bT T O − TsÁ g, will

be called a feedback matrix associated with T − P Mß"ß_ 8a b’ . This term is motivated by the fact (see

Theorem 1 below) if is a feasible solution to (1) with , then each is anT − P Mß"ß_ 8a b’ C Šœ ! O − Ts a boptimal feedback matrix.

Definition 2. Suppose , is feasibleT − MP"ß_a b’8 .

T is an upper solution to (1) if

T � � T Ÿ !àw"LQ a b a bT T  C a b> R .

T is a lower solution to (1) if

T � � T   !à Þw"LQa b a bT T ŸC a b> R

T is a is both an upper solution and a lower solution. An upper or lower solution is calledsolution if it

strict if at least one of the inequalities in the definition is strict.

All differential equations and differential inequalities in this paper are considered pointwise for

almost every in an interval indicated by context. For brevity, > we will write, for example, "K   !" or

" in " to mean that " for almost every ." K K  ! M >   ! > − Ma b The variable is often suppressed. The>

basic results are still of interest if it is assumed that and EßFßGßHß ß V WK are all continuous (or

piecewise continuous) and bounded in , and it is assumed that is (piecewise) continuouslyM T

differentiable in . The equations and inequalities will be then pointwise (except at finite points).M

Note that is a lower solution to (1) if and only if , and that is! − � W V W   !   ! !’8 �"K RX

an upper solution to (1) if and only if , . These are the common cases studied inK R� W V W Ÿ ! Ÿ !X �"

the classical literature of Riccati equations; see [7] and [8] for remarks. Also, is a strict upper or lower!

solution if one of the inequalities is strict. See Proposition 6 for an equivalent description.

§2. Interpretation of Solutions and Structure of LQa bT

6

We start with an interpretation of upper and lower solutions to (1) with , that is,C œ !

T � T œ !à T œw"L (11)Q .a b a b> R

Using the notation of LQRthe problem (2), we have

Theorem 2. Suppose T − P =ß > ß"ß_ 8"a bc d ’ is feasible.

(i) If is a lower solution to (11) with , then .T   ! N ?   ? − =ß >ea b a bT T = Da b c dDX for all h "

(ii) If is an upper solution to (11) with , then .T Ÿ N ? Ÿ ? − =ß >ea b a bT ! T = Da b c dDX for all h "

(iii) ) then is the minimum (maximum,If is a solution to (11) with (T   !e ea b a b a bT T ! T = DŸ D, X

respectively) value of J over occurs at , where and satisfiesa b a b? = > ? œ �OB O − T Bs sh Šc dß " , which

.B œ E � FO B.> � G � HO B.[ ß B = œ DÞs s� ‘ � ‘ a b (12)

Proof. Suppose T − P =ß > ß ? − = >"ß_ 8" "a bc d ’ and is the solution to equation (2.1) with . B ßhc d By the

Fundamental Theorem of calculus and Ito's formula, applied to , we obtainB > T > B >X a b a b a bI B > T > B > � D T = D œ I B > T > B > .>

.

.>

œ I B T � T B � #? F T � H T G B � ? H TH? .>

e f a b a b a b a ba b a b a b œ �(( e fa b a ba b

X X X

X X X X X X

" " "

>

=>

=

w

"

"

L ,

(13)

where L as defined in (5). Adding (13) to and using the notationsa b a bT œ E T � T E � G T G N ?X X

e fa b a bT T F T � H T G � Wœ V � H T H œX X X and in (7), we obtain

N ? � D T = D � I B > T > � R B >

œ I B T � T � K B � #? T B � ? T ? .>Þ

a b a b e fa ba b a ba b( e fa b a b a ba b

X X

X X X

" " ">

=

w"

L

(14)

f e

Since is feasible, for each . By completing the squares and using thatT T œ T O O − Ts sf e Ša b a b a bO T O œ T Ts sX

ea b a b a bQ as in (10) and L in (5), we haveLQ Qa b a bT œ K � T �

N ? � D T = D � I B > T > � R B >

œ I B T � B � ? � OB T ? � OB .>s s

a b a b e fa ba b a ba b( š ›a b a bˆ ‰ ˆ ‰

X X

X X

" " ">

=

w"

LQa bT e .

(15)

In case (i), we have , and . So T > � R Ÿ ! T � ! T !a b a b"w LQ (15) implies thata bT   e  

N ?   N ? Ÿa b a bD T = D ? − =ß > D T = DX Xa b c d a b for every in case (ii), (15) implies that forh " . Similarly,

every . In case (iii), (15) implies that for every ,? − =ß > ? − =ß >h hc d c d" "

N ? � D T = D œ I ? � OB T ? � OB .>Þs sa b a b š ›( ˆ ‰ ˆ ‰a bX X>

=

"

e

7

It follows that has a minimum (maximum) at if .N ? D T = D ? œ �OB T   ! Ð T Ÿ !Ñsa b a b a b a bX e e

Equation (12) is precisely the state equation with . ? œ �OBs ¨

Setting ? œ ! in (2), we obtain

� .B œ EB.> � GB.[ ß > − à B = œ

I B B � B B.>

c d=ß >

> R > K

"

" ">

a bš ›a b a b ' D

N ´

, (16.1)

(16.2)(16)

!X X"

= .

By (14) with , we obtain a representation for by each :? œ ! N! T − P =ß > ß"ß_ 8"a bc d ’ with T > œa b" R

N œ D T = D � I B T � K � T B.>Þ!=

X Xa b a b( a b>w

"

L (17)

In particular, if is the solution to the linear equation LT − P =ß > ß T � T � K œ !"ß_ 8"a b a bc d ’ w with

T œ N œ N !a b> R" , which exists by Proposition 7 below, then Therefore, if for every! !D T = DX a b . œa b=ß D − M ‚ V T ´ ! M œ ! K ´ ! MÞ8, then in , which implies that and in This leads to the followingR

proposition, which will be used in the proof of Proposition 4.

Proposition 3. Suppose and for theK K K K" #_ß − P Mßa b’8 I B B.> œ I B B.>š › š ›' '> >" "

= =" #X X

solution to equation (16.1) with each B =ß D − M ‚ Va b 8. Then in K K" #´ MÞ

Proposition 3 follows from the case of (16) with and . The assumption ofR œ ! K œ K � K" #

Proposition 3 implies that for all N œ !! a b=ß D − M ‚ V8. Therefore , or equivalently, K ´ ! K ´ K" # in

MÞ

Note that equation (16.1) is independent of and . Proposition 3 shows that in ifK K K ´ K M" # " #

I B B.> œ I B B.> Bš › š ›' '> >" "

= =" #X XK K for the solution to certain linear stochastic differential equation

like (16.1) with each ; see the proof of Proposition 4.a b=ß D − M ‚ V8

Suppose O P Mß ? œ �OB− _ 5‚8a b‘ and let . Then (2) reduces to

� a b a ba b

.B œ B.> � B.[ ß B = œ

I B B � B B.> ß

E � FO G � HO

> R > O

a bš ›a b a b ' Dß

N ´

(18.1)

(18.2)(18)

OX X

" ">"

= Z

where are defined asZa bO

Za bO œ O VO � O W � W O � KX X X . (19)

Note that (18) is (16) under the following replacement:

8

a b a ba bEß G ß K Ä E � FOß G � HO O, . (20)Z

Therefore, a representation for (17) under replacement (20); that is,NO follows from

N D T = D � I B T � O � O à T B.>O=

œ X Xa b a b( a b a b>w

"

Z _ , (21)

where (5) under replacement (20), namely,_a bO à T is L defined a bT

_a b a b a b a b a bO à T œ E � FO T � T E � FO � G � HO T G � HO ÞX X (22)

Propositions 4 and 5 below reveal some important structural properties of LQa bT .

Proposition 4. Suppose is feasible and , andT − P Mß − T O"ß_a b a b a b’ ‘8 _ 5‚8O P Mßa b. Let LQ Z

_a bO à T be defined in (5), (19) and (22), respectively. Then

(i) ,LQ (23)a b a b a bT � œ O � O à Ta b a ba b^ ^a b a bT T� O T � OX e Z _

(ii)if (24.1)if (24.2)

.

ÚÛÜ

a b a b a b a ba b a b a b a ba b a b a bLQLQLQ

T Ÿ O � O à T T   !àT   O � O à T T Ÿ !àT œ � à T

Z _ eZ _ eZ _^ ^a b a bT T (24.3)

(24)

Remark 1. From the proof below, it follows that identities (23) and (24.3) hold with ^a bT replaced by

any O − Ts Ša b. Identity (24.3) has been verified directly in [26, Proof of Thm 7.2, Ch. 6]. Here it

follows from (23) with . O œ ^a bT Inequalities (24.1) and (24.2) follow directly from (23) and the

assumptions . So we only need to verify (23), which could be done directly.e ea b a bT   ! T Ÿ ! and

Our derivation of (23) is based the fact that both sides of (23) represent the cost N ?a b in (2) with

R œ T > ? œ �OBa b" and .

Proof of (23). For and a given , ca b=ß D − M ‚ V8 T − P ß N ?"ß_ 8a b a bM ’ onsider the cost in (2) with

R œ T > ? œ �OB N ? Na b a b" O and . Then is precisely in (18), which is represented in (21). On the other

hand, by (15) with , and , we haveR œ T > O œ T ? œ �OBsa b a b" ^

N œ D T = D � I B T � � O T � O B .>ÞO

>

=

wX X Xa b a b a ba b( ˜ ™ˆ ‰"

LQa bT � ^ ^a b a bT Te (25)

So the two integrals in (25) and (21) are identical for the solution to equation (18.1) for eachBa b=ß D − M ‚ ‘8. By Proposition 3, (23) must hold. ¨

9

Proposition 5. Suppose . Denote ,] ^ O P Mß T œ ] � ^ß − P Mß −"ß_a b’ ‘8 _ 5‚8 are feasible, a bE œ E � Fs ^ ^ ea b a b a b^ ^ ^ß G œ G � H V œs s and . Then

(i) LQ (26)a b] � ^

œ E T � T E � G T G �s s s s

LQa b‡ ‡ ˆ ‰ ˆ ‰ ˆ ‰F T � H T G V � H T H F T � H T Gs s sX X X X XX �

(ii) If is given, then satisfies (1) if and only if ^ ] œ �equation T ] ^ satisfies

ÚÝÛÝÜa b

a bT E T � T E � G T G � Ts s s s

�

T > œ ^ >

w ‡ ‡

" "

� K �s Cˆ ‰ ˆ ‰ ˆ ‰a bF T � H T G V � H T H F T � H T G œ !ßs s s

�

X X X X XX �

R ,

(27)

where K œ ^ ^ ^s w � � ÞLQa b a bC

Proof. Let and in Proposition 4 (i), respectively. Then we getT œ ] ^

LQa ba b] �

] � O ] ] � O � ^ � O ^ ^ � O Þ

LQa b^ œ

O à T �_ a b a ba b a b a ba ba b a b a b a b^ e ^ ^ e ^X X

(28)

Setting in (28), we obtainO œ ^a b^

LQa b a b] ^ ] ^ ] ] ^� œ à TLQ (29)a b a b a b a b^ _ ^ ^ ^� � �a b a ba ba b a b^ e ^X .

To continue the proof, we first simplify e ^ ^a bc da b a b] ] ^� . Note that

e e f fa b a b a b a b] ^ ] ^� œ H T H � œ F T � H T GX X X and . The feasibility of and implies that] ^

e ^ f e ^ fa b a b a b a b a b a b] œ œ] ] ^ ^ ^ and . Therefore

e ^ ^ e ^ e ^

e ^ e ^ ^

^

a bc d a b a b a b a ba b a ba b a b a b a b a ba b

] ] ^ ] ] ^

] ^ ^ ^

^

� œ ] �

œ ] � � H T H

œ F T � H T G � H T H œ F T � H T Gs

X

X X X X X .

(30)

Now using the fact e e e ea b a b a b a b] ] ] ]� œ and (30), we obtain that

a b a ba b a ba b a b^ e ^ e] � ^ ] ] � ^ œ ]^ ^a b a b ˆ ‰ ˆ ‰X X X X XXF T � H T G F T � H T Gs s� . (31)

Substituting (31) and into (29), we obtain (i)._a b^a b^ à T œ E T � T E � G T Gs s s s‡ ‡

To show (ii), denote Xa b] ] ] ]œ w � �LQ . a b a bC Then is a solution to ] equation (1) if and

only if and . Note thatXa b a b] œ ! ] > œ" R

X Xa b a b] � ^ œ T � w LQa b] � LQ ,a b a b^ � TC

where LQ is expressed in terms of as in (26). If is given, then andLQa b] � a b a b^ T ^ ] œ !X

] > œ T œ ] � ^a b" R if and only if satisfies (27). ¨

10

Remark 2. Note that equation (27) is (1) with parameters . ItEßFßGßHßKß R ^ >s s s Vß !ß �s C and a b"

follows that is a solution (upper solution, lower solution) to (1) if and only if is a solution] T ] ^œ �

(upper solution, lower solution, respectively) to equation (27). For example, if , then (1) is^ œ !

equivalent to (27) with E œs E � FV Wß G œ G � HV Wß K œ K � W V Wß V œ VÞs s s�" �" �"X In (27) the

"crossing term" .W œ !

If is a lower solution to (1), then is a lower solution to (27) because , ^ K ! R ^ >s!   �   !a b" .

Similarly, if is an upper solution to (1), then is an upper solution to (27). For convenience, we will^ !

say equation (1) is a if and is a lower solution or an upper solution. We havestandard case W œ ! !

Proposition 6. Equation (1) can be reduced to a standard case if and only if it has a lower or upper

solution.

Remark 3. Let , then it is easily seen that is a solution (upper solution, lower solution) to (1)] œ �^ ^

if and only if is a solution (lower solution, upper solution) to the following equation]

] ] ] ] ] ] ]w � � ] � K � œ !La b a b a bC F � H G � W �V � H H F � H G � WX X X X XX a b a b�"

with ] > R Kß Vß W R �Kß �Vß �W �Ra b" œ � . This is same as (1) with and replaced by and ,

respectively. When C œ !, this equation is precisely the differential Riccati equation associated with the

problem maximizing . Also note that As a�N ?a b V � H ^H   �V � H H Ÿ !X X! ] if and only if .

result, if a lower solution (if it exists) to (1) with has certain property, then an upper^ ^ea b � !

solution (if it exists) to (1) with also has the corresponding property. Because of this, we] ]ea b � !

will state properties of upper solutions without proof.

§ 3. Comparison Theorems, Upper-Lower Solution Theorem, Existence and Approximation of

Solutions

We first consider the linear matrix differential equation

T �w"E T � T E � G T G � T � œX X Ca b K !ß T > œ Ra b (32)

where and are as in (5) and .EßGß K R −C ’8

Proposition 7. (i) E .quation (32) has a unique solution T − P Mß"ß_a b’8

(ii) If is a lower solution to (32) then . If is strict, then and .! T   ! ! T � !

(iii) If is a pair of upper-lower solutions to (32), then . If one of and are strict, thena b] ß ^ ]   ^ ] ^

] � ^ .

11

Proof. Following the idea in [25], we let be the fundamental matrix of , that is,Ga b>ß = E

`

`>>ß = œ E > >ß = ß >ß > œ E > > Ÿ =ß > Ÿ ÞG G Ga b a b a b a b a b, ! >"

Note that and . It follows that (32) is equivalent toG G G Ga b a b a b a b a b>ß = œ =ß > =ß > œ �E > =ß >�" ``>

T a b a b a b a b e f a b( a b a b a b a b a ba b> œ ß > ß > � =ß > T = � = = = = =ß > .=G G G G> R > K" "

>X X X

"

>C G T G � (33)

The Volterra equation (33) has a unique solution which can be found by successive approximations;T

say, with . This shows (i).e fT À œ !ß "ß #ß â T œ !/ / !

In case is a lower solution (i.e., , we have for all , which! − T   !  ’ /8 K ! !  R   ! and ) /

implies that . is a strict lower solution (i.e., T > œ T >   ! !a b a blim/ /Ä_ If or is strict),K ! >  T   !a b"

then

T >   T > ´ ß > ß > � =ß > = =ß > .= � !a b a b a b a b a b a b a b(# G G G G> > K" "

>X XR

"

>. (34)

This proves (ii). If is a pair of upper-lower solutions to (32), then satisfies (32) witha b] ß ^ T œ ] � ^

some and . Applying the conclusions in (ii) to , we obtain (iii). K   ! T > œ ] > � ^ >   ! Ta b a b a b" " " ¨

As an application of Proposition 7, we consider the linear matrix equation

T T T T > Rw"� � � œ ! œ_ ^ Z ^a b a b a b a ba b a bT T! !; C , (35)

where is a given matrix function and is defined in (22). Denote by the uniqueT − P Mß T! !"ß_ 8a b a b’ _ [

solution to (35). The following property of will be used in the proofs of Theorem 12 andT [a bT!

Theorem 14.

Proposition 8. Let be the solution to (35).T œ T[a b!

(i) If (1) has a lower solution with , then is an upper solution to (1) and .^ ^ T   ^ea b � ! T

(ii) If (1) has an upper solution with , then is a lower solution to (1) and .] ] T T Ÿ ]ea b � !

Proof. (i) We first show that . By Proposition 4 (i) with and ea bT T ^ T� ! œ O œ ^a b! , we have

^ T ^ T ^

^ ^ T T ^

w

w

� à � �

œ � �

_ Za b a ba b^ ^ C

^ ^ e ^ ^ C

a b a b a bc d a bc d a ba b a b a b a b! !

! !LQ ^ � ^ ^ � �   !X ,

where the inequality follows from that So is also aea b^   ! ^ ^ and that is a lower solution to (1).

lower solution to (35). Proposition 7 applied to (35) implies that . In particular,T ^ 

e ea b a bT ^ T  � ! O œ. By (24.1) with (35), we have^a b! and

T T T T T T T Tw w� � � � � œ !LQa b a b a b a bC _ ^ Z ^a b a b a bŸ ! !; .C

12

So is an upper solution to (1). The proof of (ii) is similar; it also follows from Remark 3.T ¨

Now we prove a general comparison theorem for equation (1).

Theorem 9. Denote LQ , are feasible and satisfyXa b a b a bT ´ T − P MßT �w � TCa b. Suppose ] ^ "ß_ ’8

X Xa b a b a b a b] ^ ] > ^ >Ÿ ß  " " , (36)

and either or hen . If one of (36) is strict, then .e ea b^ ] ] ^ ] ^  !   � . Ta b Ÿ !

Proof. Let Then satisfies . Suppose T œ � T T   !] ^ > ^ ]. . Setting in (29), wea b" ea b a b  ! O œ ^

have

!   T � � � T

œ T �

  T �

X Xa b a ba b a b a ba ba ba b] ^ ] ^

] ] ]

]

� œ

à T �

à T �

w

w

w

LQ LQa b a b a bC

_ ^ ^ ^

_ ^

C

C

a ba bT �

T Þ

a b a ba ba b a b^ e ^^ � ^ ^ �X

(37)

So is a pair of upper-lower solutions to the equation a bT ß ! T �w _ ^a ba b] à T � Ca bT œ ! with

T > œ !a b" . By Proposition 7 (iii), we have . If one of (36) is strict, then will be a strict upper]   ^ T

solution, which implies , or equivalently, .T � ! ] � ^

If , then set in e ^a b a b] Ÿ ! O œ ^ (29) to get

!   T � � � T

œ T �

  T �

X Xa b a ba b a b a ba ba ba b] ^ ] ^

^ ] ]

^

� œ

à T �

à T �

w

w

w

LQ LQa b a b a bC

_ ^ ^ ^

_ ^

C

C

a ba bT �

T Þ

a b a ba ba b a b^ e ^^ � ] ^ �X

The rest of the proof is the same as the case .ea b^   !¨

The conclusion that ] ^ ] ^ ] ^    ! !   implies that either e e e ea b a b   a b a b .or In

other words, and turn out to have the same definiteness. and ] ^ Therefore, e ea b^ ]  ! a b Ÿ !

usually do not occur at the same time.

Theorem 9 immediately extends to two equations like (1):

X3a bT ´ T T Tw3� LQ3a b � œ ! T > œ TC3 " "3a b a b, , (38)

where the parameters , and satisfy (4) for , and .E ß F ß G ß H ß ß V ß W T 3 œ "ß # T   T3 3 3 3 3 3 3 3 "3 "" "#K C

Denote e3 3 33a bT ´ V � H T H 3 œ "ß #Þ , for X

Theorem 10. Suppose , are feasible for both equations, is an upper solution to] ^ − P Mß ]"ß_a b’8

(38) while is a lower solution to (38) . If one of the following conditions holds, then in ." #^ ]   ^ M

(i) X X e e" # # #a b a b a b a b] ]   ! Ÿ !  and either or .^ ]

(ii) X X e e" # " "a b a b a b a b^ ^   ! Ÿ !  and either or .^ ]

13

Proof. The proof is almost trivial. The assumptions imply that . Then in case (i), weX X" #a b a b] Ÿ ! Ÿ ^

have that ; that is, X X# #a b a b] ^Ÿ ! Ÿ ] ß ^a b is a pair of upper-lower solutions to (38) . The conclusion#

follows from Theorem 9 applied to . Similarly, in case (ii), we have that X X X# " "a b a b] ! ^Ÿ Ÿ . Soa b] ß ^ is a pair of upper-lower solutions to (38) and the " conclusion follows from Theorem 9 applied to

X".¨

Theorems 9 and 10 are very general comparison results. Comparison theorems have been proved

(e.g., in [10], [11], [19] and [24]) for solutions to (1) with G œ H œ W œ ! under the assumptions that

L   L   L œE �F V F

�K �E" # " # 33 3 3 3

�"

3 3 and , where . These assumptions imply thatC C Œ �X

X

X X" #a b a bT T  for all T − ’8, which is stronger than the conditions (i) and (ii). In this special case, the

conditions on and are not necessary; see [17].e e3 3a b a b] ^

From Theorem 9 and the local theory of ODE we prove the following upper-lower solution

Theorem.

Theorem 11. Suppose that is a pair of upper-lower solutions to (1).a b] ^,

(i) If either or , then one of is strict, then e ea b a b^ ] ] ^ ] ^ ] ^  ! Ÿ !   �. In addition, if and .

(ii) If either or , then equation (1) has a unique solution with e ea b a b^ ] ] ^� ! � ! T   T   .

Proof. Part (i) follows immediately from Theorem 9. For part (ii), we first consider the case M œ c d> ß! >" .

The local existence theory of ODE implies that equation (1) has a solution that exists in a maximal

interval . Part (i) implies that in . By using the equation, we see thatÐ ß Ó § M   T   Ð ß Ó7 7> ] ^ >" "

T ´ T >   T   T   � ! � � >7 7 77

lim>Ä �

!a b a b a b a b a ba b exists and . In particular, . If ,] ^ ^ >7 7 e e 7 7"

then the local existence theory of ODE again shows that can be extended further left beyond . ThisT 7

would contradict the definition of . Therefore, exists in 7 7 œ >> T >! " and so a b c d> ß! . For the case

M œ Ð�_ß Ó T > ß > �> > >" " ", the same argument shows that the solution exists on for every . So thec d! !

solution exists on .Ð�_ß Ó>" ¨

Theorem 11 (ii) implies that a necessary and sufficient condition for the existence of a solution T

to (1) with either or is the existence of a pair of upper-lower solutions toe ea b a bT � ! T � ! a b] ^,

(1) satisfying either or , respectively. Proposition 8 shows that, in fact, the existencee ea b a b^ ]� ! � !

of one of and is sufficient for the existence of a solution. We have] ^

Theorem 12.

(i) Equation (1) has a solution with if and only if it has a lower solution with T � �e ea bT ! ! .^ ^a b

14

(ii) Equation (1) has a solution with if and only if it has an upper solution withT �ea bT ! ]

ea b] � !.

Proof. The necessity in both cases is trivial. To show the sufficiency in both cases, use Proposition 8 to

conclude that equation (1) has an upper solution (lower solution ) with ( ). So] ^ ] � ! ^ � !e ea b a bequation (1) has a solution with ( , respectively) by Theorem 11. T T � ! � !ea b ¨

Theorem 12 is not true under the weaker assumptions or e ea b a b^ ]  Ÿ! !. Consider the

scalar case of equation (1) with ; that is,E œ �"ß F œ K œ œ "ß G œ H œ V œ W œ !ß œ !R C

T � " � #T œ !ß T > œ " ! ! œ ! œ !w"a b a b a b. Obviously, is a lower solution. It is feasible because .e f

However, the solution is not feasible because but . T > œ � / T œ ! T œ Ta b a b a b" "# #

#> e f It is proved in

[8, Thm 4.1] (1) has a solution with that T ea bT � ! W œ ! H H � ! ! if , and is a lower solutionX strict

with (e.g., , , or Next theorem generalizesea b! œ V   ! V   ! K ! R K ! R    ! � � ! and either ).

this result to equation (1) with an arbitrary lower solution. Our proof is based on Theorem 12.

Theorem 13. Recall . Suppose ea b a b a b a bR > > R > !œ V � H H H H �" " "X X .

(i) If and is a strict lower solution to (1) with , then (1) has a solution withea b a bR ^ ^ !� !   Te

ea bT !� .

(ii) If and is a strict upper solution to (1) with , then (1) has a solution withea b a bR ] ] !� ! Ÿ Te

ea bT !� .

Proof. By Remark 3, we show (i) only. We first assume and . In this case the assumptionW œ ! œ !^

implies that and either is strict. We will show that (1) has another lower solutionV       !! K ! R or

^ � Ò> ß Ñ ^ � Ò> ß Ó T+ +! > ! > in with in .! " ! " . Then by Theorem 12, (1) has a solution ea b with ea bT !�

If , then let , where is the unit matrix, R � ! ^ œ / I I 8 ‚ 8+ & &!a b>�8 8

>" is the minimum

eigenvalue of and is an undetermined number. Using that , we haveR   H H � !! e &a b^ /+!a b>�>" X

X &a b a b a b^ ^ � ^ � ^   / Q � K+ + ++´ w >LQ ,C !a b>� "

where Taking aQ œ I � E � E � G G � F � H G H H F � H G � I! 8 8�"X X X X X X XXa b a b a b Ca bÞ

sufficiently large so that , we have ! XQ � ! a b^ � !+ as desired.

If , then let , where such that K � ! R� ! ^ œ ^ > ^ − P Mß ^ > œ+ & & ’! ! ! ""ß_a b a b a b and and8

^ > � ! > − Ò> ß Ñ ^ ^ Ò> ß Ó! ! ! !a b for . >   � ! >" "Using that in , we havee &ea b a b+

X &a b a b a b^ ^ � ^ � ^ œ+ + ++´ Q � Kw LQ C , (39)

where Q œ � E � E � G G � �^ ^ ^ ^ ^ ^ ^ ^ Þ K � !! ! ! ! ! ! ! !X X Xf e fa b a b a b�" Ca b Since we can

make by taking a X &a b^ � !+ sufficiently small .� !

15

If or , then ^ Á ! W Á ! consider . Equation (1) for is equivalent to (27) for .\ œ T � ^ T \

Now (27) because is a strict lower solution to has a strict lower solution (1). From what we just! ^

proved, (27) has a solution such that is a\ ^ \ ^ \ ^ \! � � H H œ T œ �e ea b X a b� . Therefore,

solution to (1) with ea bT � !.¨

Next we show that the solution to (1) can be approximated by solutions to (35).] ]3 3�"œ [a b

Theorem 14. Suppose and is a lower solution to (1) such that . LetM œ > ß � !c d a b! > ^ ^" e

] ] ]! 3 3�""ß_ 8− P Mß œ 3   "a b a b’ and for . Then[

(i) .] ] ] ^" # $      â  

(ii) is a solution to (1) and there exist constants and such that] ]œ 5 -lim3Ä_

3

l > � > l Ÿ 5 � > > − MÞ-

4 � # x] ] >3

4œ3

_ 4�#4�#a b a b a b"a b " , (40)

Proof. Proposition 8 implies that each is an upper solution and for . To show other] ] ^3 3   3   "

conclusions, we first derive an equation satisfied by ?3 3 3�" 3 3�"œ �] ] ] ]. By the definitions of and ,

they satisfy

] ] à ] ] ]3w

3�" 3 3�" 3� � � œ !Þ_ ^ Z ^a b a b a ba b a b C (41)

] ] à ] ] ]w3�" 3 3�" 3 3�"� � � œ !Þ_ ^ Z ^a b a b a ba b a b C (42)

By Proposition 4 (i) with , we haveT œ O œ] ]3 3�" and ^a b_ ^ Z ^a b a b a ba b a b] à ] ] ] ] ] ] ] ]3�" 3 3�" 3�" 3�"� œ � ÞLQ 3 3 3 3a b a ba ba b a b^ e ^� �^ ^a b a bX

However, by (24.3) Thus (41) becomesLQa b a b a b] œ ] à ] ]3 3 3 3_ ^ Z ^a b a b� Þ

] ] à ] ] ] ] ] ] ] ]w3 3 3 3 3�" 3�"3 3 3 3� � � � œ !_ ^ Z ^a b a b a ba b a b C a b a ba ba b a b^ e ^� �^ ^a b a bX . (43)

Subtracting (42) from (43) we obtain

? C3 3 3 3w

3 3 3 3�" 3�"� à � �_ ^ ? ?a b a ba b] ] ] ] ] ]a b a ba ba b a b^ e ^� � œ !Þ^ ^a b a bX (44)

Note that and because ?3 3 3 3a b> ] ] ] ] ] ! ]   !Þ" 3�" 3�" 3œ !  a b a ba ba b a b^ e ^� �^ ^a b a bX ea b By

Theorem 9, ?3   !. This finishes the proof of (i).

Now (i) implies that and are all uniformly bounded in . It follows thate f e f e f] ] ]3 ß Me ^a b a b3 3

l l Ÿ 5l lß l à � l Ÿ 5l la b a ba ba b a b^ e ^] ] ] ] ] ]3 3 3 3�" 3� �^ ^a b a b3�" 3�" 3 3 3X ? C ?_ ^ ? ?a b a ba b ,

16

for some constant . Thus from (44) we obtain that5 � !

l > l Ÿ 5 l = l � l = l .=? ? ?3 3 3�"

>

>a b a b( a b a b"

. (45)

The rest of the proof follows exactly the argument in [26, p.324]. We repeat it here for reader's

convenience. Denote . Then (45) reduces t@ > œ l = l.=3 3>

>a b a b' " ?

@ > � 5@ > � 5@ >   !ß3w

3 3�"a b a b a bwhich implies that

@ > Ÿ 5/ @ = .= œ - @ = .=ß3 3�" 3�"5>

> >

> >a b a b a b( ("

" "

where By induction, we deduce that- œ 5/ Þ5>"

@ > Ÿ > � > @ > Þ-

3x3�" " " !

33a b a b a b

It follows then from (45) that

l > l Ÿ 5 > � > � > � > @ > Þ- -

3 � " x 3 � # x?3 " " " !

3�" 3�#3�" 3�#a b a b a b a bœ �a b a b

This yields (40). ¨

By Remark 3, we also have an approximation theorem for the solution with .^ ^ � !ea bTheorem 15. Suppose is an upper solution to (1) with . Let and] ] ^ea b � ! !

"ß_ 8− P Mßa b’

^ ^3 3�"œ 3   "[a b for . Then

(i) .^ ^ ^ ]" # $Ÿ Ÿ Ÿ â Ÿ

(ii) is a solution to (1) and there exist constants and such that^ ^œ 5 -lim3Ä_

3

l > � > l Ÿ 5 � > > − M œ > ß Þ-

4 � # x^ ^ > >3 !

4œ3

_ 4�#4�#a b a b a b c d"a b " ",

§ 4. Estimates of Maximal Intervals of Existence

Suppose ( is an upper (lower) solution to the equation] ^Ñ

Xa bT ´ T � � T œ !ß T œw"LQ (46)a b a b a bT > RC

on an interval . If or , then Theorem 11 implies the existence of a solution in M � ! � ! T Mea b a b] ^e

with or , respectively. If or , then it is well-know that thee ea b a b a b a bT � ! T � ! � ! � !e e] ^

17

solution may blow up in . This happens, for example, to Riccati equations from differential games [2]. InM

this case, it would be of interest to estimate the existence interval of the solution to (46).maximal Ð ß Ó7 >"

By Theorem 11, the solution to (1) exists in the intersection of the existence intervals of a pair of

upper-lower solutions. So the maximal existence interval of the solution to (1) can be estimated by

constructing their upper and lower solutions. Theoretically, such estimates can be made as accurate as

possible if we can find upper and lower solutions that are close enough to the solution. Upper and lower

solutions can be constructed by using the solutions of comparison equations like those in Theorem 10. In

Theorems 16, 17 and Proposition 18 below, w scalar autonomouse demonstrate how to construct

differential equations in terms of . ] ^ > and to obtain explicit estimate for Ð ß Ó7 " By (27) and Proposition

6 we need to consider only the standard cases with , and or .W œ ! œ ! œ !] ^

Denote by , and the minimum eigenvalue, maximum eigenvalue and maximum- A 5a b a b a b† † †

singular value of a matrix, respectively. Suppose , and is an upper solution. LetW œ ! V � ! œ !]

+ß ,ß -ß .ß <ß :" be constant bounds such that

+   F � H G ß ,   E � E � G G ß .   H H ß

- Ÿ ! � < Ÿ Ÿ Þ

5 A C A

- - -

#

"

a b a b a ba ba b a b a bX X X X X� I

K V : R8

, , (47)

Consider the scalar equation

œ a ba b: 2 : >: > :

w"

"

� œ !ß > Ÿ ßœ ",

(48)

where . Note that , , . Suppose . Let be2 : : ! ! : : :a b œ � � , � - +   .   !ß - Ÿ Ÿ ! . � < � !+. �< " "

::

#

the unique solution to (48) on the interval Since is an upper solution tomaximal Ð ß Ó5 >" . 2 !a b! œ - Ÿ !,

(48), which implies that . Furthermore, for : > > : >a b Ÿ ! Ð ß Ó in by Theorem 115 5" ". > � < � ! > −a b Ð ß Ó.

Theorem 16. . Suppose is an upper solution to (46) with such that If (48) with! V :� ! . � < � !"

coefficients in (47) has a solution in an interval with then (46) has a solution in: > :Ð ß Ó T5 " . � < � !,

Ð ß Ó T � !5 >" with .ea bProof. Let . We only need to show that is a lower solution in to (46) with .^ :I ^ > ^œ Ð ß Ó � !8 "5 ea bNote that

^ > : I Rß

^ : : I

a ba b a b" 8

8

œ Ÿ

œ � H H   < � . � !"

e V X .(49)

The second inequality in (49) implies that . By the definition of , wea b a bV � H H Ÿ < � .: : I 2X �" �"8

18

have

Xa b a b a ba ba ba b a b a b

” • c da b

^ ^ ^ ^

: I : I K

: :

: : I : 2 : I:

:

œ � �

œ � � E � � �

� F � H � H H F � H

  � , � - � œ � œ !Þ+

< � .

w

w8 8

w w8 8

LQ C

CE G G

G V G

X X

X X X X XX# �"

#

This shows that is a lower solution. By Theorem 11, (46) has a solution in . ^ >Ð ß Ó5 " ¨

Similarly, suppose , and is a lower solution. W œ ! V � ! ! Let be constant bounds+ß ,ß -ß .ß <ß :"

such that

+   F � H G ß ,   � E � G ß .   H H ß

-   ß ! � <   ß   Þ

5 A C A

A A A

#

"

a b a b a ba ba b a b a bX X X X XE G I

K V : R

� 8 (50)

Let . Consider (48) with coefficients in (50). Note that , ,2 : : ! !a b œ � � , � - +   .   !ß -  +. �<

::

#

: : : >" "  ! . � < � !. Suppose . Let be the unique solution to (48) on the interval Sincemaximal Ð ß Ó5 " .

2 !a b! œ -   !   ! Ð ß Ó, in by Theorem 11 and is a lower solution to (48), which implies that : > >a b 5 "

. > � < � ! > −: >a b for Ð ß Ó5 " .

Theorem 17. Suppose is a lower solution to (46) with If (48) with coefficients in (50) has a! V � !.

solution in an interval with then (46) has a solution in with .: > : >Ð ß Ó T Ð ß Ó T � !5 5" ". � < � !, ea bProof. The proof is similar to that of Theorem 16. One only needs to show that is an upper solution.:I8

¨

We now give an integral representation for the maximal existence interval of the solutionÐ ß > Ó5 "

: > 2 : 5   ! 2 : 5 � #a b a b a b to (48) for a general rational function . Assume for some , has distinct zeros

and poles , including .�_ œ D � D � â � D � D œ _ „ _! " 5 5�"

Proposition 18. Suppose for some . Then the solution of (48) exists on: − D ß D 3 œ !ß #ßâß5 : >" 3 3�"a b a ba maximal interval withÐ ß > Ó § Ð�_ß > Ó5 " "

> � œ.:

2 :"

:

:5 ( a b

"‡

"

,

where ; that is, if and if .: œ : > : œ D 2 : � ! : œ D 2 : � !"‡ "‡ 3�" " "‡ 3 ">Ä �lim

5a b a b a b

Proof. If , then the solution is strictly increasing at decreases in with range2 : � ! : > >a b a b" "Ð ß > Ó5

Ò: ß D Ñ : > œ D � .> œ ! : D" 3�" 3�" " 3�">Ä �

.:2 : and . Write (48) as . Integrating this equation from to forlim

5a b a b

19

: > > and from to for , we get" 5

( a bD

:"

3�"

"

.:

2 :� � > œ !Þ5

If , then the solution is strictly decreasing at decreases in with range with2 : � ! : > > ÐD ß : Óa b a b" "Ð ß > Ó5 3 "

lim>Ä �

3 " 3 ".:

2 :5: > œ D � .> œ ! : D : > >a b . Integrating from to for and from to for , we obtaina b 5

( a bD

:"

3

"

.:

2 :� � > œ !Þ5

The Proposition is proved.¨

Example. We take a simple example from [26, Ch. 6, Example 7.8] to illustrate an application of

Proposition 17. Consider a scalar case of problem (3) with E œ G œ K œ W œ !ß

F œ H œ œ > œ " V œ �< < − !ß "R " and with . The Riccati equation becomesa b: : : :w #� Î � < œ ! " œ "a b a b, .

This equation is like (48) with . Since , the solution decreases to as2 œ � Î � < 2 " � ! > <a b a b a b a b: : : :#

> Ä � œ5 5 5. By Proposition 18, " � . œ # � < � < œ !' a b<" �<:

:# : ln . Note that when

< œ < ´ !Þ"&)&*%â !ß " < � <! !. Therefore, the equation has a solution on if and only if . The estimatec dgiven in [26] for appears to be too big.<

Note that Theorems 16 and 17 continue to hold even if the bounds and are time+ß ,ß -ß . <

dependent.

§ 5. An Extension

All of the results in Sections 1-4 are true for (1) with and ; that is,multidimensional G H

G œ G ßâßG H œ H ß â ß Ha b a b" 7 " 7, , where G − P Mß ß H − P ÐMß Ñ4 4_ _a b‘ ‘8‚8 8‚5 and

G T G œ G T G ß H T G œ H T G ß H T H œ H T H ÞX X X X X X" " "4œ" 4œ" 4œ"

7 7 7

4 4 44 4 4 (51)

With the notations in (51), the statements and proofs of the results for the case are identical to the7 � "

case in Sections 1-4. This equation is associated with the problem (2) with multidimensional7 œ " LQ

noise [ œ [ ßâß[a b" 7 ; . The state equation has the same form exceptsee [26, Ch.6, Sec. 3] for details

a b a bGB � H? .[ G B � H ? .[ is defined as .!4œ"

7

4 4 4

References

20

[1] M. Ait Rami and X.Y. Zhou, Linear Matrix Inequalities, Riccati Equations, Indefinite Stochastic

Linear Quadratic Controls, , Vol. 45, No.6 (2000)IEEE, Transactions on Automatic Control

1131-1143.

[2] T. Basar and P. Bernhard, H -Optimal control and related minimax design problems, Birkhauser,_

1995.

[3] A. Bensoussan, Stochastic control of partially observable systems. London, Cambridge University

Press, 1992.

[4] S. Bittanti, A.J. Laub, and J.C. Willems, The Riccati equations, Springer, 1991.

[5] J.M. Bismut, Linear quadratic optimal control with random coefficients, .,SIAM J. Contr. Optim

vo. 14 (1976) 419-444.

[6] R. W. Brockett, Finite dimensional linear systems, J. Wiley, New York, 1970.

[7] S. Chen, X. Li and X.Y. Zhou, Stochastic linear quadratic regulators with indefinite control

weight costs, 36: 1685-1702 (1998).SIAM J. Control and Optimizations

[8] S. Chen and X.Y. Zhou, Stochastic linear quadratic regulators with indefinite control weight

costs, II, ., vol 39, 4(2000) 1065-1081.SIAM J. Control Optim

[9] M.D. Fragoso, O.L.V. Costa and C.E. de Souza, A new approach to linearly perturbed Riccati

equations arising in stochastic control, , 37 (1998) 99-126.Applied Mathematics & Optimization

[10] G. Freiling and G. Jank, Existence and comparison theorems for algebraic Riccati equations and

Riccati differential and difference equations, , 2 (1996) 529-547.J. Dynam. Contr. System

[11] G. Freiling, G. Jank and H. Abou-Kandil, Generalized Riccati difference and differential

equations, , 241-243 (1996) 291-303.Linear Algebra and Its Applications

[12] I. Gohberg, P. Lancaster, and L. Rodman, On Hermitian solutions of the symmetric algebraic

Riccati equations, . 24 (1986), 1323-1334.SIAM J. Contr. Optimiz

[13] V. Ionescu, C. Oara and M. Weiss, Generalized Riccati theory and robust control, A Popov

Approach, John Wiley & Sons, 1999.

[14] P. Lancaster and L. Rodman, The Algebraic Riccati equations, Oxford University Press, Oxford,

1995.

[15] M. McAsey and L. Mou, Riccati differential and algebraic equations from stochastic games. In

Preparation, 2001.

[16] L. Mou, General algebraic and differential Riccati equations from stochastic LQR problem.

Preprint, 2001.

[17] L. Mou and S. R. Liberty, Estimation of Maximal Existence Intervals for Solutions to a Riccati

Equation via an Upper-Lower Solution Method. To appear on the Proceedings of Thirty-Nineth

Allerton Conference on Communication, Control, and Computing, 2001.

[18] R. Penrose, A generalized inverse of matrices, ., 52 (1955), 17-19.Proc. Cambridge Philos. Soc

[19] A.C.M. Ran and R. Vreugdenhil, Existence and comparison theorems for algebraic Riccati

equations for continuous- and discrete-time systems, . 99 (1988), 63-83.Linear Algebra Appl

21

[20] W.T. Reid, Riccati differential equations. Academic Press, New York and London, 1972.

[21] C. Scherer, The solution set of the algebraic Riccati equation and the algebraic Riccati inequality.

Linear Algebra and its applications, 153 (1991) 99-122.

[22] D.D. Sworder, Feedback control of a class of linear systems with jump parameters, IEEE Trans.

Automat. Contr., Vol. AC-14 (1969) 9-14.

[23] J. C. Willems, Least squares stationary optimal control and the algebraic Riccati equation. IEEE

Trans. Automatic Control, AC-16 (1971), 621--634.

[24] C. W. Wimmer, Monotonicity of maximal solutions of algebraic Riccati equations, Syst. Contr.

Lett. 5 (1985), 317-319.

[25] W.M. Wonham, On a matrix Riccati equation of stochastic control, , Vol. 6,SIAM J. Control

No.4 (1968) 681-697.

[26] J. Yong and X.Y. Zhou, Stochastic Controls, Hamiltonian Systems and HJB Equations, Springer

1999.

[27] K. Zhou with J. Doyle and K. Glover, Robust optimal control, Prentice Hall 1995.