PREFACE
This collection of papers deals with Llw general role of singular perturbation techniques in control systems analysis and design problems. These methods have proven useful in tIle construction or "reduced order models" and the evaluation of cont-rol system designs based on those models. We have collected here the usc at' these techniques which will be informat;ive
1,0
11
representa,tive sampling of
t,lwse readers interested in
acquiring a taste ror the theory and its applications. We have also addressed those doing research in the subject matt;er by including some new results and methods not published elsewhere. The first paper in this collection,
Singular Perturbation Techniques ill Control
Theory is a survey of the role of singular perturbation ideas in engineering control sy:;tern design. The analysis and examples which it contains summarizes much of the work in the field prior to this volume. It sets the stage for the detailed treatment of more specialized topics in the subsequent papers.
In Part I we treat optimal control problems with small parameters. The paper Singular Perturbations for Deterministic Control Problems provides a comprehensive treatment of deterministic optimal control problems with "fast" and "slow" states. It is based on the asymptotic analysis of both necessary conditions and l;ho associated Hamilton-.Jucobi-Bellmun equation - that is, direct evaluation of the optimal cost runeLion. The treatment using a dlWlity for this equation is new. As a consequence, one can extend the concept of composite feedback involving "separation" or controls ror fast and slow states which had been derived earlier for quusi-linear systems the fuJI nonlinear case.
v larly Perturbed Systems nonlinear, non-autonomous singula.rly perturbed sysl,cms axe considered at I,lle outset. Tbe methods are I,hen exl,ended
1,0
treat multiparamctcr pcr-
turba.tion problems. In New Stability Theorems Jor Averagillg and Their Application to
lhe Convergence Analysis oj Adaptive Identification alld Oontrol Schemes multi-time scale methods are llsed
1,0
GreaL time varying nonlinear systems wHh applical,ions
1,0
csl,i-
mat,ion oj' the mLcs of convergence of adaptive identification a.nd conl,rol algoriLhms. These papers provide just a.pplied mathematics.
I'\'
sampling of the methods :lvILlla.ble
ill I,his
rich a.rea. or
Some of' the Impcrs indicate the broader range or mel,hods and
applications which lie outside con trol Uleory.
We trust that those readers who' have
round the papers in Irllis volume interesting will be motivated to explore the many important, contributions which treat relat,cd applica.tions in ellgineering and applied phy-
sics.
P.V. Kokotovlc A. Bcnsoussan G.L. Blankenship
SINGULAR PERTURBATION TECHNIQUES IN CONTROL THEORY
P. Y ](okotovic t
Abstract This paper discusses typical applications of singular perturbation techniques to control problems in the last fifteen years. The first three sections are devoted to the standard model and its convergence, stability and controllability properties. The next two sections deal with linear-quadratic optimal control and one with cheap (nearsingular) control. Then the composite control and trajectory optimization are considered in two sections, and stochastic control in one section. The last section returns to the problem of modeling, this time in the context of large scale systems. The bibliography contains more than 250 titles. Introduction For the control engineer, singular perturbations legitimize his ad hoc simplifications of dynamic models.
One of them is to neglect some "small" time constants.
masses, capacitances, and similar "parasitic" parameters which increase the dynamic order of the model.
However, the design based on a simplified model may result in a
system far from its desired performance or even an unstable system.
If this happens,
the control engineer needs a tool which will help him to improve his oversimplified design.
He wants to treat the simplified design as a first step, which captures the
dominant phenomena.
The disregarded phenomena, if important, are to be treated in
the second step. It turns out that asymptotic expansions into reduced (,touter") and boundary layer ("innerl!) series, which are the main characteristic of singular perturbation techniques, coincide with the outlined design stages.
Because most control systems are
dynamic, the decomposition into stages is dictated by a separation of time scales. Typically, the reduced model represents the slowest (average) phenomena which in most applications are dominant.
Boundary layer (and sublayer) models evolve in faster
tCoordinated Sciences Laboratory and Electrica.l Engineering Department, University 01 Illinois, 1101 W. Springlleld Avenue, Urbana, IL 61801. This paper Is based on the amhor's survey In Lhe SlAM Relliew, Vol. 6, No. -I, October Hl8·1, pp. 501-550,
3
the state space of (1), (2) reduces from n + m to n because the differential equation
(1.2) degenerates into an algebraic or a transcendental equation
o
g(x,z,u.o,t),
(1. 3)
where the bar indicates that the variables belong to a system with
E
= O.
He tll!ll
say that the model (1.1), (1. 2) is in the stctnda2,d fonn i f and only if the following crucial assumption concerning (1.3) is satisfied.
In a domain of interest equation (1.3) has k > 1 distinct ("isolated") real roots i
1,2, ... ,k.
(1.4)
This assumption assures that a well defined n-dimensional reduced model will dorrespond to each root (1.4).
To obtain the i-th reduced model we substitute (1.4)
into (1.1), (1.5)
In the sequel we will drop the subscript i and re,rrite (1.5) more compactly as
.:. x
= £(i.u,t).
(1. 6)
This model is sometimes called quasi-steady-state model, because z, whose velocity
z .s. EO:
is large when
E
is small, may rapidly converge to a root of (1.3). which
the quasi-steady-state form oE (1.2). of (1.1), (1.2) in the
na~t
i8
We will discuss this two-time-scale property
section.
The convenience of using a parameter to achieve order reduction has also a drawback:
it is not always clear how to pick the parameters to be considered as small.
Fortunately, in marty applications our knowledge of physical processes and components of the system suffice to be on the right track.
Let us illustrate this by examples.
Example 1.1
A well-known model of an armature controlled DC-motor is (1. 7)
ax
Lz
bx
Rz + u
(1. 8)
where x. z. and u are respectively, speed, current. and voltage, Rand L are armature resistance and inductance, and a and b are some
m~tor
is a Ilsmall parameter" which is often neglected. £:=L.
constants.
In most DC-motors L
In this case equation (1.3) is
5 network is
(1.17) = v
1
-
(1 +~)v + -R~ u.
u +
R
(1.18)
2
+
(0)
{b} Fig. 1.
System with a high gain amplifier:
(a) full model, (b) reduced model.
If this model were in the form (1.1). (1.2), both
and v
2
would be considered as
z-variab1es and (1.3) would be
o
(1.19)
(1. 20)
However, Assumption 1.1 would then be violated because the roots of (1.3). in this
v
case vI 2 , are not distinct. The question remains whether the model of this RCnetwork can be simplified by singular perturbation E = 0, that is, by neglecting the small parasitic resistance r?
Without'hesitation the answer of the electrical engi-
neer is yes, and his simplified model is given in Fig. 1.2b.
To justify this simpli-
fied model a choice of state variables must be found such that Assumption 1.1 be satisfied.
As will be explained in Section 10 a good choice of the x-variable is the
"aggregate" voltage
7 Most of the quoted singular perturbation literature assumes that model (1.1), (1.2) is in the standard form, that is, it satisfies Assumption 1.1.
The importance
of Example 1.3 is that'it points out the dependence of Assumption 1.1 on the choice of state variables.
In most applications a goal of modeling is to remain close to
original "physical" variables.
This was possible in our Examples 1.1 and 1.2, but
not in Example 1.3. where a new voltage variable (1.21) had to be introduced.
However,
few engineers, accustomed to the simplified "equivalent" circuit in Fig. 1.2b. would question the "physicalness" of this new variable.
On the contrary. physical proper-
ties of the circuit in Fig. 1.2a.are more clearly displayed by the standard form (1.22). (1.23).
Nevertheless the problem of presenting and analyzing singular per-
turbation properties in a coordinate-free form is of fundamental importance.
A
geometric approach to this problem has recently been developed by Fenichel (1979) Kopell (1979) and Sobolev (1984).
Nore common are indirect approaches which deal
with singular singularly perturbed problems. such as in
O'~mlley
the original "nonstandard l l model into the standard form (1.1)
t
(1979). or transform
(1.2). such as in
Peponides, Kokotovic. and ehol" (1982). or Campbell (1980, 1982).
He will return to
this modeling issue in Section 10.
Singular perturbations cause a multi-time-scale behavior of dynamic systems characterized by the presence of both slow and fast transients in the sy,stem response to external stimuli.
Loosely speaking, the slow response, or the "quasi-steady-state, II
is approximated by the reduced model (1.6), while the discrepancy between the response of the reduced model (1.6) and that of the full model (1.1), (1.2) is the fast transient.
To see this let us return to (1.1)-(1.6) and examine variable z which has been
e1lOcluded from the reduced model (1. 6) and substituted by its "quasi-steady-state" z. In contrast to the original variable z, starting at t quasi-steady-state z is not free to start from
ZO
from a prescribed zo, the o and there may be a large discrepancy
between its initial value ~(x(t
o
and the prescribed initial condition z.
(2.1)
),u(t 0 ),t 0 ) ZO
Thus
z cannot
be a uniform approximation of
The best we can expect is that the approximation
z
z(t)
+
(2.2)
a(e)
will hold on an interval excluding to' that is, for tE[tl,T] where tl > to' we can constrain the quasi-steady-state dition x
O
x to
However,
start from the prescribed initial con-
and. hence the approximation of x by
x may
be uniform.
In other lrords,
9 If this assumption is satisfied, that is, if
(2.8)
uniformly in x tl
>
to'
small.
O
to' then z will come close to its quasi-steady-state z at some time
,
Interval [to,t l ] can be made arbitrarily short by making £ sufficiently To assure that z stays close to z, we think as if any instant t E [tl,T] can
be the initial instant.
At such an instant z is already close to
z,
\~hich motivates
the following assumption about the linearization of (2.6). Assumption 2.2 The eigenvalues of
ag/cz
evaluated along x(t), z(t), u(t) for all t E [to,T]
have real parts smaller than a fixed negative number
ReA{~} <-c
<
O.
(2.9)
Both assumptions describe a strong stability property of the boundary layer system (2.6).
If
ZO
is assumed to be sufficiently close to z(t ), then Assumption 2.2 a
encompasses Assumption 2.1.
We also note from (2.9) that the non-singularity of
ag/az along x(t), z(t), z(t) implies that the root z(t) is distinct as required by Assymption 1.1.
These assumptions are common in much of the singular perturbation
literature (Tichonov (1948,1952), Levinson (1950), Vasileva (1963), Hoppensteadt (1971), et al.).
These references contain the proof and refinements of the following
result, frequently referred to as Tichnov's theorem. Theorem 2.1: If Assumptions 2.1 and 2.2 are satisfied, then (2.3) and (2.7) hold for all t E
[to,T], while (2.2) holds for all t E [tl,T], where the "thickness of the boundary
layer" tl-t
can be made arbitrarily small by choosing small enough E. o As we shall see, "many control applications of singular perturbations make use of
this theorem.
Let us first specialize (1.1), (1.2) to linear systems
+ Hz,
(2.10)
Cx + Dz,
(2.11)
Ax
assuming first that A, B, C, and D are constant matrices. if ReA{D}
<
-
O.
z = -D
Clearly, Theorem 2.1 holds
The root of (1.3), -1 -
Cx
substituted in (2.10) yields the reduced model
(2.12)
11
E:
For
dz dt
D(t)z.
(2.22)
1. even if
E:
ReA{D(t)} ::.
<
0,
system (2.22) can be unstable.
Vt > t
(2.23)
o
However, when
E
is small, the following result holds.
Theorem 2.2 If, in addition to (2.23), the derivative for all t > to' then there exists E:l
>
net)
of net) is bounded, IIO(t)!! < c
0 such that for all 0 <
E: ~
2
E:l the system
(2.22) is uniformly asymptotically stable. To prove this theorem we define ll(t) for all t ..:: to by D' (tHHt)
+ N(t)D(t)
(2.24)
-1.
In view of (2.23) B(t) is positivedeEinite and its derivative M(t) is bounded, that is, (2.25) Theorem 2.2 follows from the fact that the derivative v DE the Lyapunov function v
z'M(t)z
(2.26)
for (2.22) is bounded by v < -
(
1c -
c )z ' z 3
This analysis reveals the meaning of the boundary layer stability assumption of Theorem 2.1. 1
E D(t).
For
E:
sufficiently small, the "frozen ll spectrum of
1 E
lIfrozen" stability condition (2.23) applies.
*
¥ ' in this case
is sufficiently faster than the variations of the entries of
Z
d
and the
He are now in the position to generalize the transformation (2.17) to the timevarying system (2.10), (2.11) that is when A
A(t),
B
B( t) ,
If the transformation matrix L equation
c
e(t),
D
net) .
(2.28)
L(t) in (2.17) satisfies the matrix differential
13
with a change of notation suitable for control applications.
Following Chang (1972),
we let L(t) satisfy (2.29) in the new notation and we also define R(t) as a solution of (3.3) which can be approximated by (3.4)
RCt)
Denoting by
a \c.xk identity we introduce the transformation
[:]
-ERL (3.5) L
whose inverse is
(3.6)
In the new coordinates
~,
n, the system (3.1), (3.2) separates into two subsystems (3.7)
(3.8)
Taking into account (2.29) and (3.4) '-Ie readily obtain the following result: Theorem 3.1 For
E
small a sufficient condition for the controllability of the full system
(3.1), (3.2) is the controllability of the slow (reduced) subsystem
.:.
and the fast (boundary layer) subsystem
(3.9)
15
xes)
(3.12)
Z(5)
(3.13) ~G.
Defining the transfer function matrices G and
(3.14 )
G(s)
~G(s,£)
-£s(£sl - A
=
2l
)
-1
(3.15)
and denoting by a and a the largest and the smallest singular values, respectively. the robustness conditions due to Sandell (1979) is stated as follows. Theorem 3.3
If the reduced system (3.9) is stable, the full system (3.1), (3.2) remains stable for all ;:: >
a
satisfying
-
o(6G(jw,e» for all
!ll>
:
~(1
-1
+ G (jw»
(3.16)
O.
For nonlinear singularly perturbed systems the stability is frequently analyzed using separate Lyapunov functions for the reduced system and the boundary layer system and composing them into a single Lyapunov function for the full system.
Let us first
illustrate this on a nonlinear system which is linear in z,
x £2
where
+ F(x)z
(3.17)
g(x) + G(x)z
(3.18)
f(x)
exists for all x.
of two functions.
Lyapunov function introduced by Chow (1978) consists
The first function (3.18)
a'(x)Q(:{)a(x)
v
.:. establishes the asymptotic stability oE the reduced system x
a(x) and Q(x)
> 0
f(x) - F(x)G
-1
where (3.19)
(x)g(x)
satisfies, for some differentiable C(x) - C(x)
= a(x),
a
x
>
D.
(3.20)
17-' (3.29) such that
a.(~z
- ip(x,t)ll)
<
'W(x,z,t)
<
(3.31)
b(llz - ip(x,t)P
here a and b are positive nondecreasing scalar functions.
Furthermore, suppose that
positive constants kl and k2 exist such that (3.32) (3.33)
(3.34)
I~~I II
~
k2ilz - 'P(x,t)!l(~z - ip(x>t)~ + Ilxn,
<
< k2 h
(3.35) (3.36)
- ip (x, t ) II ,
~f(x,z,t) - f(x,tp(x,t)
,tH .: kzllz - tp(x,tH,
(3.37)
~f(x.z,t)~ .: k2(~x~
Iz -
(3.38)
+
tp(x,t)~),
(3.39)
11'P(x,t)11 .: b{llxll), where
v
in (3.32) denotes the t-derivative Eor the reduced system (3.27), while 'W in
(3.34) denotes the T-derivative for the boundary layer system (3.29).
(3.39) are satisfied, then there exists
If (3.33) to
such that for all EE {O,E*} the equilibrium
x= 0, z= 0 of (3.25), (3.26) is uniformly asymptotica.lly stable. Obtaining more easily verifiable stability conditions is an active research topic, of major interest in robustness studies of adaptive systems, Ioannou and Kokotovic
(1982,1983).
A possible approach to these problems is to investigate the perturbations
of the aboslute stability property as in Siljak (1972), Ioannou (1981), and Saksena and Kokotovic (1981). 4.
Optimal Linear State Regulators
One of the basic results of control theory is the solution of the optimal linear state regulator problem by Kalman (1960), 'Which reduces the problem to the solution of a matrix Riccati equation.
For the linear singularly perturbed system (3.l), (3.2)
this equation is also singularly perturbed. Kokotovic (1969)
t
It was investigated by Sannuti and
Yackel (l971)} Haddad and Kokotovic (1971). Koltotovic and Yackel
(1972), O'Malley (1972)} and
Ol}~lley
and Kung (1974).
Another form of the regulator
solution is obtained via a Hamiltonian boundary value problem which in this case is
19
(4.6)
E
(4.7)
(4.8) with the end condition
o.
0,
(4.9)
This is clearly a singularly perturbed system of the type (1.1), (1.2) and we can apply Theorem 1.1.
~~hen
'ole set
E =
0, we ge t
o,
(4.10)
°
(4.11)
o
(4.12)
The only end condition to be imposed on this algebraic-differential system is Kll(t ) = ~ f
while (4.11) and (4.12) now play the role of (1.3).
A crucial property of this system
is that (4.12) is independent of (4.10) and (4.11).
To satisfy Assumption 1.1 we need
a unique positive definite solution K22 of (4.12) to exist.
For each fixed tE [to,tfl the pair A22 (t), B (t) is stabilizable and pair A22 (t), 2 C (t) is detectable. 2 For this assumption to hold it is sufficient that the controllability condition (3.11) and (4.13)
In
hold for all tE (to,t ]. E
Under Assumption 4.1 eigenvalues of A2l -
negative real parts and (4.11) can be solved for K12 in terms of
K22 ,
all have known from (4.12),
21
that (4.15) represents the time-invariant Riccati equation depending on the fixed parameter t, which is, in fact, an independent optimality condition for the boundary layer regulator problem (3.10) in fast time scale
T.
Then the resulting feedback matrix
A22 - S22K22 satisfies Theorem 2.2, that is, it guarantees the uniform asymptotic stability of the boundary layer.
This is the stabilizing role of the fast regulator
feedback K . He reiterate that the weakly controllable (stabilizable case is excluded, 22 that is, Theorem 4.1 requires that the fast modes be controlled directly, rather than through the slow subsystem. robust design. (4.12).
Although not necessary, this requirement is needed for a
The slow regulator is defined by the reduced system (4.10), (4.11),
At the first glance it appears that it depends on the quasi-steady-state
solution K22 of the fast regulator. solution for the problem in which
This would allow it to differ from the regulator is neglected already in the system (3.1), (3.2)
E
and in the cost (4.1), rather than later in the Riccati equation.
The difference
between the two reduced solutions would indicate nonrobustness. because the result would depend on the moment when
E
is neglected.
The robustness of the optimal state regulator problem with respect to singular perturbations is established by Haddad and Kokotovic (1971). property is not automatic in other feedback designs.
The same robustness
Khalil (1981) gives examples of
non-robust feedback designs using reduced order observers or static output feedback. Gardner and Cruz (1978) show that, even with the state feedback, Nash games are nonrobust with respect to singular perturbations. Once the robustness of the optimal state regulator is established. we can proceed with the design which consists in implementing the control law (4.2) with approximate feedback gains (4.16), (4.17), (4.18). feedback gains depending on t and
This is a two-time scale design because the
are obtained separately.
T
However, an equivalent,
but more direct approach is the so-called composite control approach developed by Suzuki and Hiura (1976) and Chow and Kokotovic (1976). in the section on nonlinear control.
tole will present this approach
The singularly perturbed optimal regulator problem
for linear difference (rather than differential) equations was solved by Blankenship (1981), and Litkouhi and Khalil (1983). 5.
Linear Optimal Control
Although convenient for the feedback solution of linear optimal control problems ,~ith
free endpoints, the Riccati equation approach must be modified in order to apply
to problems with fixed endpoints.
Two such modifications were developed by Wilde and
Kokotovic (1973) and Asatani (1976).
In general, endpoint constraints require the
solution of Hamiltonian boundary value problems, which are in our case singularly perturbed.
Various forms of singularly perturbed boundary value problems, not directly
related to control applications, were studied earlier by Levin (1957), Vishik and Liusternik (1958), Harris (1960), Vasileva (1963), Wasow (1965), O'Halley (1969), Chang (1972), and others.
Host of these works develop "inner" (in
T
and
0)
and "outer" (in t)
23 appear at both ends of the optbnal trajectory.
The layer at the left end point must
be uniformly asymptotically stable in the direct, and the layer at the right end point in the reverse time. The two-time scale design of a near optimal trajectory is summarized in the following theorem.
Suppose that Assumption 4.1 is satisfied and x(t) and
pet) uniquely satisfy (5.4)
Denote by P the positive definite root of the Riccati equation (4.l2) at 22 to and by N22 its negative definite root at t= t . Let L(t) and RCa) be the solutions
and (5.5). t
f
of tll10 mutually independent time-invariant initial value problems dL(t)
[A
ciT L(O)
:=
z
o
22
(5.6)
(t ) - S2Z(t )P22]L(T) o o
(5.7)
- z(t ) 0
and
dR(a)
dO
[A
22
(t ) - S22(t )N ]R(a) f f 2Z
(5.8)
(5.9)
RCO) = z f - z{t ) f where
T ""
(t-tO)/E and a
:=
(t-tr)!E are the tlstretched" time scales.
Then there exists
> 0 such that for all tE [to,trL EE (0,£'*]
x{t,e::)
x( t) +O(e::)
z(t,e::)
:let) + L(T}
p(t,t)
p (t) + OCt)
q(t,£)
q( t) +
U(t,E)
ti(t) + U eT) + U (rr) + O(E) L R
(5.10)
+
RCa) +
oed
(5.11)
(S.12)
PnL(-r)
+
N R(rr) 22
+ O(c)
(5.13) (5.U)
where li (t) UL(T) URea)
- R-l(B{P +
Bi q)
(5.15)
R-l(t 0 )B~(t }P22 L (T) _0 -1
- R
(5.16) (5.l7)
(tf)B2(tf)N22R(a).
The time scales for these two operations can be selected to be independent. reduced problem, a standard tll10 point boundary value technique is used.
For the
The advantage
over the original problem is th;>t: the order is lower, and the fast phenomena due to are eliminated. Example 5.1 We illustrate the procedure using the system and the cost
t:
25 in real time t, that is, asymptotically stable in the reverse time tf-t. We can use the same example to illustrate the more common approach by (1972b).
Starting with (5.20) an asymptotic series in t,
for each of the-variables and the terms with like powers of
Ot~fu1ley
and a would be substituted
T, E
are identified.
The first
terms i(t), z(t), p(t), q{t) in the t-series are obtained from (5.21) and (5.22), as in this approach.
However, instead of using the Riccati and the boundary layer systems,
(5.23) and (5.25), the first terms Z(T), q(T), z(a), q(a) in the T- and the a-series would be obtained from the
T-
and the a-form of (5.20), subject to appropriate matching
of their initial and end conditions.
This approach can handle any other type of con-
sistent initial and end-conditions.
Both approaches lead to the same asymptotic
solution, but under different hypotheses.
The relationship of the hypotheses was
investigated by O'Halley (1975). 6.
Singular, Cheap. and High Gain Control
In our discussions thus far the singular perturbation properties of t1le system to be controlled l>lere not altered by the control law.
However. even if the original system
is not singularly perturbed, a strong control action can force it to have fast and slow transients, that is. to behave like a singularly perturbed system. -the strong control action is achieved by high feedback gain.
In feedback systems,
For a high gain system to
emerge- as a result of an optimal control problem, the control should be "'cheap, It that is, instead of u'Ru, its cost in (4.1) should be only
E
2
U'Ru, t>lhere
E>
0 is very small.
On the other hand, an optimal control problem (3.1), (3.2). and (4.1) with det R= 0 is singular in the sense that the standard optimality conditions do not provide adequate information for its solution.
Singular optimal controls and resulting singular arcs
have been a control theory topic of considerable research interest, see for example Bell and Jacobson (1975).
By formulating and analyzing the cheap control problem as a
singular perturbation problem O'Nalley and Jameson (1975,1977), Jameson and O'Ha11ey (1975), and O'Malley (1976) llave developed a new tool for a study of singular controls as the limits of cheap controls.
The application of these results to the design of
high gain and variable structure systems was discussed in Young, Kokotovic. and Utkin (1977).
Here we closely follow a presentation in
Ot}~lley
(1978).
The cheap (near-s'ingu1ar) control problem for a linear system
'* = A(t}x + B(t)u,
(6.1)
is characterized by the presence of
E
in the cost functional
t J =
f
1
2 J t
2
[x'Q(t)x + £ u'R(t)u]dt
(6.2)
a
where Q and R are as usual symmetric positive definite. timality conditions hold,
For
E>
0 the standard op-
27
o.
nand det B 1: 0 whic.h is a very special
This equation is in the standard form only if r and unlikely situation.
(6.12)
Por r
Henc.e this
equation is not i.n the standard form and the procedure in Section 4 does not apply. that a reduced solution Ko satisfies
\.Je see, hm.;ever,
o
n'Ko
(6.13)
but this Ko is not fully defined.
Since B 'KE can be nonsingular, \.Je pre- and post-
multiply (6.12) by S' and B. respectively, ?
•
c-[S'(KB) + B'1(1\B + B'A'KB + B'qBJ Substituting K =
B'QB
B
'KB.
(6.14)
and letting c =0 we obtain the reduced equation (6.15)
(8
\"bicb in Case 1; \.Jhen B' QD > 0, has a unique solution
;.- O.
(6.16)
Such an analysis suggests that K be sought in the form (6.17)
where
IT
= (tr-t)/c and Kl(a) is the boundary layer correction at t=t •
(6.17) into (6.12) and equating the terms of like pO\.;ers in
K
0
+
+
- Ko S1 K0 +
Q1
o
0,
E,
Substituting
f we obtain at
0
(6.18)
where
(B'QB)-lB'q
(6.19)
QB(n'QB)-IB'Q
(6.20)
Al
A -
Ql
Q
S1
Bl (B'QB)-lBi::' 0
(6.21)
and
(6.22) l
B'(t ). I t can be shown that Ko(t) is defined by (6.13) and f (6.18) and that (6.22) and B'(t"[~)Kl(O) + B'(t )K ( 0 uniquely define in
where Sf
B(tf)R- (
f
1
29 (6.30) and the boundary layer (fast) subsystem (6.31)
Taking Fl (6.32) It can be shown, Kokotovic (1984), that the feedback matrices can be separately chosen, to place the eigenvalues of EZF 2 and F 5 to place eigenvalues of
- A12 F2 •
Such
a design procedure \"fas proposed by Young, Kokotovic, and Utkin (1977).
High gain systems have good disturbance rejection properties.
They have been
extensively studied in control literature. most recently by Sastry and Desoer (1983) and, using a geometric approach, by Hil1ems (1981, 1982).
Insensitivity and disturbance
decoupling properties are analyzed by Young (1976, 1982a ,b). suffer because of neglected high frequency parasitics.
High-gain systems may
This aspect was addressed by
Young and Kokotovic (1982). 7.
Composite Feedback Control of Nonlinear Systems
In the preceding three sections appro:·dmations of both the optimal feedback control and the optimal trajectory consisted of slow and fast parts.
They are obtained from
singularly perturbed Riccati equations or thlo-point boundary value problems. optimality conditions also consisted of slow and fast parts.
These
A further. step toward a
final decomposition of the t,,,o time scale design has been made which decomposes the optimal control problem itself into a slow subproblem and a fast subproblem.
Separate
solutions of these subproblems are then composed into a eompo8'ite feedback conb'ot which is applied to the original system.
As an engineering tool the composite control
approach has both conceptual and practical advantages.
The fast and slmv controllers
appear as recognizable entities which can be implemented in separate hardware or software. The composite control was first developed for time-invariant optimal linear state regulators by Suzuki and l'-liura (1976), Chm. . (1977), and Chatv and Kokotovic (1976), and then for nonlinear systems by Chm., and Kokotovic (1978a, b t 1981). Suzuki (19Bl) and Saberi and Khalil (1985). and
r~gni
(1980).
A frequency domain composite design {\Tas developed by Fossard
Extensions to stochastic control problems are due
(1981) and Khalil and Gajic (1984).
to Bensoussan
The composite control bas also been applied to
large scale systems, as will be discussed in a subsequent section.
The composite
control approach is now presented following Chow and Kokotovic (1981).
31
x
(7.7)
o
(7.8)
and, since A-I is assumed to exist, 2.
(7.9)
is eliminated from (7. 7) and (7.3).
Then the slot., subproblem is to optimally control
the "slmv subsystem" x (0)
(7.10)
'.'
"0
s
'''ith respect to "slow cost"
J
s
J [p
o
+
(x ) 0
I(X)U
s
S
S
+
u'R (x )u ldt
s
0
5
(7.11)
S
t..rhere
p
s
o
ntA,-l(OA-1a 2 2.
• 2
2
- -.~s), !.
R
(7.12)
o
{'Je note that. in vietV' of Assumption 7.1, the equilibrium of the slotV' subsystem (7.10)
for all
ED
is
0, and
(7.13)
Our cLucial Assumption 7.2 concerns the existence of the optimal value function L(x ) satisfying the optimality principle
s
o
(7.14)
where Lx denotes the derivative of L with respect to its argument
The elimination
of the mini.mizing control
u
s
(s
o
+lnILt) 2 0 x
(7.15 )
from (7.14) results in the Hamilton-Jacobi equation
a
L(O)
o
(7.16)
33 (7.15)
has the property that ReA
tAz (x) 1 < 0 >
¥.ltE D.
\ve no\.; form a "composite" control
-1 zf by z + A2 (3
+
2
U
c
Us
=
+
u ' in which Xg is replaced by x and f
8 u (x», that is 2 s
u (x,z)
c
(7.26) t.;rhere 1:.
az(x)
2
n2 R-1 (b'L' 1 x + BiVl) ,
a2 (0)
0,
Al = Al
- Blf{-1 2
-1
V'1 = - (s t + 2a 1(f +
2
A2
(7.27)
B [C[,
The properties of the system controlled by the composite control are summarized in the follm..ring theorem. Theorem 7.1 '''hen Assumptions 7.1,7.2, and 7.3 are satisfied then there exists s
.'.
¥EE (O,E-J. the composite control u (7.1).
c
such that
defined by (7.26) stabilizes the full system
(7.2) in a sphere centered at .)-;=0,
Z
O.
The corresponding cost J
c
is bounded.
is near opti.mal in the sense that Jc';"'J as ;;:--'--0. c s This theorem shows that the considered nonlinear regulator problem is well-posed
horeover, J
It is the basis for a two-time scale design procedure developed by
wit h respect to
ChmJ and Kokotovic (1981) and Saberi and Khalil (1985).
8.
Nonlinear Trajectory Optimization
\,113 now consider a more general class of nonlinear optimal control problems on a
finite interval lto,tfl, frequently encountered in flight dynamics and start-up or shut-dm..rn operations for industrial plants.
In Section 5 we have discussed such prob-
lems fo!:" linear systems and quadratic functionals.
In this section \.;e deal tJith non-
linenr systems in the form
f(x,2,u),
xE
g(x,z,u),
(8.1)
(8.2)
and the functional to be minimized t J
f
f
o
v(x,z,u)dt
(8.3)
35 problem decomposes into onl'! slow and two fast subproblems.
The slow ("outer")
sub-
problem (8.9)
i.s 2n-dimensional.
To satisfy the remaining 2m boundary conditions, the layer ("inner")
corrections 2 ("1 ), Zn(T ) for z, and qL{T ), qR(T R) for q nre determined from the L R L 1. initial (1) and flnal (It) boundary layer systems \vith appropriately defined Hamiltonians R HL and H , that is, dZ
L
ClH
dTL
()CJ
L
dqL
,
dZ
dqR
R dTR
where
(8.10)
d!~
L
(8.11)
dTR
"" ,_ tvhile Tn;; c is the reversed fast time scale. L subproblems are used to form approximations -t
U
:=
The results of these
us(t) + u L(T L) + llR(T H) + aCE)
}:
Xs (t)
+
O(£)
z
z (t)
+
zL (-lL)
s
(8.1.2)
(8.13)
+
zR (T ) R
+
(8.14)
O{,c;).
As ,.;as already discussed in Section 5, the L-Iayer must asymptot ically decay. that is, the initial condition at
a
for (8.10) must be on a stable manifold.
ZR(T ) must asymptotically decay as R
on
11
Ul,
that is, as t-i-
The end layer
and hence za(O) must lie
totally unstable manifold of (S.ll). In realistic nonlinear problems the matching of layers and reduced sQlutions is
not nn easy task. arcs occur.
It is more complex if the control is constrained and if singular
For this reason practical approaches are problem-dependent and based on
prior experience.
This is particularly true in flight dynamics, where reduced oruer
approximations based on lIenergy state" or "point mass" and "rigid body" models are common. two tvays:
In flight dynamics singular perturbations facilitate numerical computations in first, they reduce the number of costate i.nitial values that must be deter-
mined simultaneously, and, second, they improve the conditioning of the boundary value problem.
For example, the \.Jlld, undamped phugoid-like oscillations characteristic of
the system (8.5), (8.6) for lifting atmospheric flight is avoided for the most part, being relegated to boundary layer corrections.
Kelley, Calise and Ardema contain
details of several Clpplications containing the layers not only at the ends, but also at some inner points tvhere the reduced trajectory is permitted to be discontinuous. Another difficulty in these applications is a proper choice of fast and slow variables,
37 (1984) approached this problem via singularly pertur-bed Lyapunov equations. (1978) and Singh and [{am-Nandan (1982) have establ ished the
\~enk
Hazevig
convergence, as c -l" 0,
of the fast stochastic variable z which satisfies the Tto equati.on
(9.6)
ReA (A) < 0
where
~(t)
is Gaussian white noise with covariance W, that is, lim ,,>~
where
z
is a
z weakly
z(t;e}
(9.7)
0 const,~nt
Gaussian random vector t1lth covariance P snt.isfylng the Lyapunov
equati.on AP
+
o.
PAl +G\.](;1
Khalil (1978) assumes
11
(9.8)
colored noise disturbance in the fast subsystem to account for
situat Lons h1hen the correlation time of the input stochast il' pr-ocess is longer than the time constants of f
Thus the optimal solution to the stodwstic
regulator probLem can be approximated by the optimal solution of the slow subproblem and optimal cost J does not diverge.
A composite control approach to a class
or
nonlinear systems dr.iven by tIIhite noise
disturbances, as a stochastic version of the results reviewed in Section 8, was develope by UenSOUSRan (L981).
He considered
(9.9) (a(x)z + b(x) + 2c«:!.)u)dt + e/f
cdz
,z
(u(·»
E
f
e-Vt[(f(x)
(9.10)
+ h(x)z)2 + u 2 ldt
o
(9.11)
wiler-e w1(t), w (t) are standard Wiener processes independent of each other-. 2 The opt lUlal feedback 1.1\1 is obtained ilS
,z) (x,z)
(9.12)
where VE(x,z) is the Bellman function.
solutions of the nw subproblems.
As ;::-:..0. the optimal solution converges to the
The slow subproblem is
dx
(9.13)
]dt.
(9.14)
39 N
N
(10.1) N
(10.2)
AiO x + Ariz i + E
j'='l j;fi
This model allO\o,Is us to assume that each controller neglects all other fast subsystems and concentrates on its own subsystem, plus the interaction with others through the slow core.
For the 1-th controller, this is simply effected by setting E-parameters to
zero, except for si"
The i-th controller's simplified model is then N
(10.3)
I
lllhich is often all the i-th controller knmlls about the whole system.
The It-th control-
ler, on the other hand, has a different model of the same large scale system. u
i
can
Control
be divided into a slow part, which contributes to the control of the core, and
a fast part controlling only its own fast subsystem.
The multiparameter perturbation
problem has been solved under rather restrictive D-stabil.ity assumptions, Khalil and lCokotovic (1979), Ozguner (1979), and Khalil (1981).
Stochastic multimodeling problems
are even more complex, because of the so-called nonclassical information patterns, Saksena and
Ba~ar
(1983).
Singular perturbation problems for multiple controllers with different cost [unctionals (e.g., differential games) are complex even with a single perturbation parameter. I~e
have already mentioned the ill-posedness a f linear Nash games with respect to sin-
gular perturbations, Gardner and Cruz (1978).
Singularly perturbed differential games
Here further investigated by Salman and Cruz (1979), Khalil and Kokotovic (1979), Khalil and Nedanic (1980).
Singularly perturbed pursuit-evasion problem tl1as studied by Farber
and Shinar (1980) and Shinar (1981). Let us conclude this section and the
~l1hole
survey with " closer look at a funda-
mental property of large scale systems--the fact that the time scales are caused by weak connections, Kokotovic (1981).
Although this is a property of a wide class of nonlinear
systems, such as pOt"er systems, Peponides, Kokotovic, and Chmv (1982). and multimarket economies, Peponides and Kokotovic (1983), t"e restr.ic.t our discussion to linear timeinvariant systems in the form
EV
= [A +
sB(s)jv,
(l0.5)
,,,here A represents strong in.ternal connections tl1ithin a subsystem while ell are tveak e:.I:te~f'nal
c.onnec.tions among subsystems.
Tf A is singular, this is not a standard form
41
which is a standard form because QAT is nonsingular due to (10.7).
This defines the fastest tbne scale
T
= ~E
and
PB(O)Sx
(10.16)
is the slow (reduc.ed) subsystem of (10.14), (10.15).
If P13(O)S is singular, there will
be time scales slower than t and the same procedure can be continued.
This is the
essence of a sequential determination of time scales by Coderich, et a1.. (1983), Delebecque (19B3), and Khalil (1984<1). Example 10.1 Let us re-examine the RC-network in Fig. 1.2 and its model (1.17), (1.18).
In
this case
(10.17)
B
A
and Q and P can be defined as
Q
-1],
[1
For (1.21) \vith C
P
= [p
p] .
(10.18)
1
C c.oefficient p is 2' The near conservation property of the 2 net\vork in Fig. 1.2 refers to the fact that i f R =' co , the total charge on the capacitors I
:::::
and the "aggregate" voltage x is the voltage on the sum of the capacitors with that charge.
During the fast transient this voltage remains essentially constant, while the
actual voltages VI and v
2
converge to their quasi-steady state VI = v . 2
Their differ-
ence z
(10.19)
Qv
is the fast variable.
Its substitution into (1.22), (1.23) \IIould put the network model
in the form (10.15).
In networks and Narkov chains, A is often block-diagonal and each of its N blocks Ai represents a local network or l-Iarkov chain lvith the property that det A.
L
0,
(10.20)
1=1, ... ,N.
The 1Il0st interesting case is 'ilhen dim N(A ) := 1 for all 1=1, ... , N LInd hence \l = N. Then i P is an N ;-: n dimensional aggr'egation matrix and x Pv defines one aggregate variable for each subsystem.
In Narkov chains the aggregate variable :{i is the. probability for
the Narkov process to be in the class i of the strongly interacting states.
For the
multilllodeling approach to decentralized control it is of crucial importance that QAT is block diagonal, that is, the fast subsystems are indeed "local."
The variables in the
43 REFEREtlCES Allemong, J. J. 2nd P. V. Kokotovic (1980), "Eigensensitivities in reduced order modeling,"..I.llliE. Trans. Automat. Control, AC-25, pp. 621-822. Al tshuler I D. and A. H. Haddad ( 197 B) , "Near optimal smoothing for singularly perturbed linear systems,·t Automatica, 14, pp. 81-87. Anderson, B. D. O. and J. B. Hoore (1971), Linear Optimal Control, Prentice Hall, Englewood Cliffs J New Jersey. Anderson, P. M. and A. A. Fouad (1977), ~ System Control ~ Stability, Iowa State University Press. Anderson, L. (1978), I1Decomposition of two time scale linear systems,tI J:r.QQ. ~, pp. 153-163. Andreev, Yu H. (1982), "Differential geometry method, in oontrol theory a survey," Automatika Telemachnika, 10, pp. 5-46. Ardema, M. D. (1976), "Solution of the m~n~mum time-to-climb problem by matched asymptotic expansions, 1t.AI.A.A.J., 14, pp. 843-850. Ardema, H. D. (1979), "Linearization of the boundary layer equations for tbe minimum time to climb problem,11 .A1..A.A.J. Guidance .iillQ Control. 2, pp. ~34-436. Ardema, M. D. (1980), "Nonlinear singularly perturbed optimal control problems 'Ylith singular arcs,1I Automatics, 16, pp. 99-104. Arderna, M. D. (1983), "Singular pertUrbations in Systems and Control, n.Q.SM Courses ..s..ruI. Lectures, 180. Springer, Ne\-l York. Asa tani, K. (1974), "Suboptimal control of fixed-end-point minimum energy problem via singular perturbation theory, 1f.J.. Hath. Anal. .A.Iml.. I 45, pp. 684-6gr. Asatani, K. (1976), "Near-optimum control of distributed parameter systems via singular perturbation theory."~, Math. Anal. ARnl., 54, pp. 799-819. Asatani, K., ~l. Shiotani, and Y. Huttoni (1977), "Suboptimal contI'ol of nuclear reaotors with distributed parameters using singular pertul'bation theory, 11 Nuclear Science and Engineering. 6, pp. 119-xxx. Athans, M. and P. L. Felt (1956) I Optimal Control: .An. Introduction to .t.M. Theory ~ ~ Applications, McGraw-Hill, New York. Avramov ie, B. (1919), "Subspace iteration approach to the time scale separation, II ~. IEEE Conference Qn Decision snQ Control, pp. 684-697. Avramovic, B. (1980), "Time scales, coherency, and weak coupling,l1 Ph.D. Thesis, Coordinated Science Laboratory, Report R-895, Univ. Illinois. Urbana. Avramovic, B., P. V. Kokotovlc, J. R. Winkelman and J. H. Chow (1980), I1Area decomposition of electromechanical models of pO~leI' syst.ems," Automatioa, 16 f pp. 637-640. Eagirova, N., A. B. Vasileva and H. I. Imanaliev (1967), ItThe problem of asymptotic solutions of optimal control problems," Differential Equations, 3, pp. 985-
988. Ealas, H. J. (1978), "Observer stabilization of singularly perturbed systems, II J!. Guidance .e.rui Control, 1, pp. 93-95. Balas, M. J. (1982), IIReduced order feedback control of distributed parameter systems via singular perturbation methods, "~. Math. Anal. .Arul!., 87, pp. 281-294. Eell, D. J. and D. H. Jacobson (1975), Sjngular Optimal Control Problems, Academic Press I New York. Bensoussan, A., J. L. Lions and G. C. Papanicolaou (1978), Asymptotic Analysis for Periodic Structures, Nor th-Holland, New York. Bensoussan, A. (1981), "Singular perturbation results for a class of st.ochastic control problems, II ~ Trans. Automat. Control, AC-26 , pp. 1071-1080. Bensoussan, A. (1984). "On some singular perturbation probelrns ariSing in optimal control,n Stochastic An2l. and Applic" 2, pp. 13-53. Blankenship, G. and D. Meyer (1977), "Linear filtering with wide band noise disturbances," J..6..t.t! IEEE Conf. Qn Decision .e.rui Control, pp. 580-584. Blankenship, G. and G. C. Papanicolaoi (1978), "Stability and control of stochastic systems with wide-band noise disturbance, SIAM~. ~. Math. I 3~, pp. 437476. Blankenship, G. (1978), "On the separation of time scales in stochastio differential equations," Proc. 1.ih IFAC Congress, Helsinkii, pp. 937-9q4. Blankenship, G. (1979), I1Asymptotic analysis in mathematical physics and control
45 801, Univ. Illinois, Urbana. Chow, J. H. (1977b), npreservation of controllability in linear time invariant pertur bed systems, II .I.!lt • ..4.. Control, 25, PP. 6 gr -704. Chow, J. H. (1gr8), "Asymptotic stability of a class of nonlinear singularly perturbed systems,".l. Franklin Inst., 306, pp. 275-278. Chow, J. H. (1978), "Pole-placement design of multiple controllers via weak and strong controllability," lnl . ..4.. ~. Sci., 9, pp. 129-135. Chow, J. H. and P. V. Kokotovic (1978a), "Near-optimal feedback stabilization of a class of nonlinear singularly perturbed systems, II SIAM..4.. Control M.irn, I 16, PP. 756 -77 O. Chow, J. H. (1grBb), "Two time scale feedback design of' a class of nonlinear systems," IEEE.IJ:.e..n.§.. Automat. Control, AC-23, pp.' 438- 1J113. Chow, J. H., J. J. lUlemong and P. V. Kokotovic (1 W8), "Singular perturbation analysis of systems with sustained high frequency OSCillations, Automatics, 14, pp. 27 1-27 9. Chow, J. H. (1 gr 9), "A class of singularly perturbed nonlinear, fixed endpoint control problems," .l • ..Q.Q.lim. Theory ~., 29, pp. 231-251. Chow, J. H. and P. V. Kokotovic ( 1981) , "A two-stage Lyapunov-Eellman feedback design of a class of nonlinear systems," ~ Trans. Automat. Control, AC26, pp. 656 -66 3. Chow, J. H., Ed. (1982), ~ Scale Modeling of Dynamio Networks, Lecture Notes in Control Information Soiences, 47, Springer-Verlag, New York. Chow, J. H. and P. V. Kokotovic (1983), nSparsity and time scales, fl Proc. of 1..9ll American Control Conference, 2, San FranCiSCO, pp. 656-661. Chow, J. H' I P. V. Kokotovic and Y. K. Hwang (1983), "Aggregate modelling of dynamic networks with sparse interconnections," Proe. 22nd Decision.e.rui Control Conference, San AntoniO, pp. 223-229. Chow, J. H' l J. Cullum and R. A. Willoughby (1984), "A sparity-based technique for identifying slow-coherent areas in large power systems,ff lEEE Trans. ~ .AIl.mlx.. Systems, PAS-103, pp. ,lj63-471. Cobb, D. (19811) I "Slowest fast stability eigenvalues systems," l.f::.Q..G.. 23..r& Decision and Control Conference, Las Vegas, Nevada, pp. 280-282. Code1"ch, H., A. S. Willslcy, S. S. Sastry and D. A. Castanon (1983) t If Hierarchical aggrega tion of linear systems \-Ii th mul tiple tilLe scales, If IEEE ~. Automat. Control, AC-28, pp. 1017-1030. Collins, ~l. B. (1973), IfSingular perturbations of linear time-optimal control," ~ Mathematical Deyelopments in Control, D. J. Bell, ed. I Academic Press, Ne\-l York, pp. 123-136. Cappel, W. A. (1965), Stability snQ Asymptotic Behavior of Differential EQuations, D. C. Heath and Company, Boston. Coppel, W. A. (1 ~7), "Diochotomies and reducibility,IT.J.. Differentialll Eguations, 3, pp. 500-521. Cori, R. and C. Haffezzoni (1984), "Practical optimal control of a drum boiler power plant," Automatica, 20, pp. 163-173. Delebecque I F. ana J. P. Quadrat ( 197B) , "Contribution of stochastic contraIl singular perturbation averaging and team theories to an example of large scale systems: management of hydropower' production, IT mE.~. Automat. Control, AC-23, pp. 209-222. Delebecque, F. and J. P. Quadrat {1gB1}, "Optimal control of Markov chains admitting strong and weak interactions," Automatica, 17 I pp. 281-295. Delebecque, F. (1983), IIA reduction process for perturbed Harkov chains, n .illl:! .J..• .ll.ru2l. Math, ,43, pp. 325-350. Delebecque, F., J. P. Quadrat and P. V. Kokotovic (1984) I If unified view of aggregation and coherenoy in networks and Markov chains, II Int • .![. Control,
40, pp. 939-952. Demello, F. P. and C. Concordia (1969), "Concepts of synchronous machine stability as affected by exei tation control, n IEEE Trans. Power Applications Ji!..Il4 Systems, PAS-88, pp. 316-329. Desoer, C. A. (1970), "Singular perturbation and bounded input bounded state stability," Electronic Letters, 6, pp. 16-17. Desoer, C. A. and M. J. Shensa (1970), "Network \-lith very small and very large parasitics: natural frequencies and stability,lt Proe. IEEE, 58, pp. 19331938.
47
function method," ~. l:Is.th. l>.Q.Itl.. I 17, pp. 1503-1505. Glizer, V. J. (1g]7), "On the continuity of the regulator problem with respect to singular perturbations," .A.P.PJ... Math. Mech., PMH, ,41, pp. 573-576. Glizer, V. J. (1978), "Asymptotic solution of a singularly perturbed Cauchy problem in optimal control," Differential Equations, 14, pp. 601-612. Glizer, V. J. (197 9), "Singular perturbations and generalized functions,"~. Math. DQkl., 20, pp. 1360-1364. Grasman, J. (1982), "On a class of optimal control problems with an almost cost-free solution, n ~..I.r::.s..rul. Automat. Control, AC-ZT, pp. 441-1145. Grishin, S. A. and V. I. Utkin (1980), "On redefinition of discontinuous systems," Differential Equations, 16, pp. 227-235. Grujic, L. T. (1979), "Singular perturbations, large scale systems and asymptotic stability of invariant sets," .In.:t.. Jl.. Systems Science, 12, pp. 1323-1341. Grujic, L. T. (1981), "Uniform asymptotic stability of nonlinear singularly perturbed large-scale systems," ln1 . .!l.. Control, 33, pp. 481-504. Habets, P. (1974), nStabilite asyrnptotique pour des problemes de perturbations singulieres,1t in Bressanone, Edizioni Cremonese, Rome, Italy, pp. 3-1B. Haddad, A. H. and P. V. Kokotovic (1971), "Note on singular perturbation of linear state regulators, II ~ Trans. Automat. Control, AC-16 , 3, pp. 279-281. Haddad, A. H. (1976), "Linear filtering of singularly perturbed systems, It IEEE ~. Automat. Control, AC-31 , pp. 515-519. Haddad, A. H. and P. V. Kokotovic (1977), "Stochastic control of linear singularly perturbed systems," IEEE Trans. Automat. Control, AC-22, pp. 815-821. Hadlock, C. R. (1 gr 0), "Singular perturbations of a class of two point boundary value problems ar~s~ng in optimal control," Ph.D. TheSiS, Coordinated Science Laboratary, Report R-481, Univ. Il11nois, Urbana. Hadlock, C. A. (1973), "Existence and dependence on a parameter of solutions of a nonlinear two-point boundary value problem,"A. Differential Equations, 111, pp. 498-517. Halanay, A. and St. Mirica (1979), "The time optimal feedback control for singularly perturbed linear systems," Rev. Boum. l:!e..t.. Pures n AIm.!., 24, pp. 585-595. Hale, J. K. (1980), Ordinary Differellt.ill Equations, Krieger Publishing Company. Harris, W. A., Jr. (1 £6 0), nSingular perturbations of two-point boundary problems for systems of ordinary differential equations, II Arch. Rat. Mech • ..An.a.l. I 5, pp. 212-225. Hopkins, W. E., Jr. and G. L. Blankenship (1981), "Perturbation analysis of a system of quasi-variational inequalities for optimal stochastic scheduling, fI IEEE ~. Automat. Control, AC-26, pp. 1054-1070. Hoppensteadt, F. (1967), "Stability in systems with parameters," A. ~. An£l. ~., 18, pp. 129-134. Hoppensteadt, F. (1971), "Properties of solutions of ordinary differential equations with small parameters, II l&mm. ~~. ~., 34, pp. 807-840. Hoppensteadt, F. (1974), flAsyrnptotic stability in singular perturbation problems, II, 111... Differential Eouations, 15, pp. 510-521Howes, F. A. (1976), "Effective characterization of the asymptotic behaviour of solutions of singularly perturbed boundary value problems,!! SIAH.!l.. AllP.l. Hath., 30, pp. 2~-306. Ioannou, P. (19B 1), !!Robustness of absolute stability, IT Int. .!l.. Control, 34, pp. 1 OZT -1033. Ioannou, P. A. (1982), "Robustness of model reference adaptive schemes with respect to modeling errors, II Ph.D. TheSiS, Coordinated Science Laboratory, Report R-955, Univ. Illinois, Urbana. Ioannou, P. and P. V. Kokotovic (1982) I "An asymptotic error analysis of identifiers and adaptive observers in the presence of parasi tics, " ~ Trans. Automat. Control, AC-27 , pp. 921-927. Ioannou, P. A. and P. V. Kokotovic (1983), Adaptive Systems with Reduced Hodels, Lecture Notes in Control and Information Sciences,~, Springer-Verlag, New York. Ioannou, P. A. (1984), "Robust direct adaptive controller," Proc. 23rd IEEE 1&r!.f • .QI! Decision ~ Control, Las Vegas, Nevada, pp. 1015-1019. Ioannou, P. A. and P. V. Kokotovic (1984), hRobust redeSign of adaptive control," IEEE~. Automat. Control, AC-29, pp. 202-211. Ioannou, P. A. and P. V. Kokotovic (1985), "Decentralized adaptive control of
49
singularly perturbed systems," Automatica. 17, pp. 797-804. Khalil, H. K. (19B1 b), "On the robustness of output feedback control methods to modeling errors," ~~. Automat. Control, AC-28, pp. 524-528. Khalil, H. K. and Z. Gajic (1984), tINeal" optimal regulators for stochastio linear singularly perturbed systems," IEEE ~. Automat. Control, AC-29, pp. 531-511 1. Khalil, H. (1984a), "A further note on the robustness of output feedbaok control to modeling errors," ~~. Automat. Control, AC-29, pp. 861-862. Khalil, H. K. (1984b), "Time scale decompositin of linear impliot singularly perturbed systems," ~~. Automat. Control, AC-29, pp. 1054-1056. Khalil, H. K. (1984c). "Feedback control of implicit singularly perturbed systems," Proc. ~ IEEE Conference Qn Decision and Control, Las Vegas, Nevada, pp. 1219-1223. Khalil, H. K. (1985), "Output feedback control of linear two time scale systems," ~ American Control Conference, Boston. Klimushev, A. I. and N. N. Krasovskii (1S62), ITUniform asymptotic stability of systems of differential equations with a small parameter in the derivative terms,If..!I . .A.Q.Q.l. Hath. l~ech., 25, pp. 1011-1025. Koda, M. (1982), "Sensitivity analysis of singularly perturbed systems," Int. ~. Systems Science, 13, PP. 909-919. Kokotovic, P. V. and P. Sannuti (196B), "Singular perturbation method for reducing the model order in optimal control deSign, II ~ 1.r..e.ru!. Automat. Control, AC-13, PP. 377-38~. Kokotovic, P. V. and R. A. Yackel (197 2) , tlSingular perturbation of linear regulators: basic theorems," IEEE Trans. Automat. Control, AC-17, pp. 2937. Kokotovic, P. V. (1gJ'5) J tlA Hiccati equation for block-diagonalization of illconditioned systems," ~~. Automat. Control, AC-20, pp. 812-81~. Kokotovic, P. V. and A. H. Haddad (1975a) t "Controllability and time-optimal control of systems \>lith slow and fast modes, II ~.Ir.rul.§.. Automat. Control, AC-20, pp. 111-113. Kokotovic, P. V. and A. H. Haddad (1975 b ), "Singular perturbtaions of a class of time-optimal controls," ~ Trans. Automat. Control, AC-20, pp. 163-16~. Kokotovic, P. V., R. E. O'Malley, Jr. and P. Sannuti (1976), "Singular perturbations and order reduction in control theory--an overview," Automatica, 12, pp. 123-132. Kokotovic, P. V., J. J. Hlemong, J. R. Winkelman and J. H. Chow (1980), tlSingular perturbation and iterative separation of time scales," Automatica, 16, pp.
23-33.
Kokotovlc, P. v. (1981), "Subsystems, time-scales and multimodeling," Automatics, 171 pp. 789-795. Kokotovic, P. V., B. Avramovic t J. H. Chow and J. R. Winkelman (1982), "Coherenoybased decomposition and aggregation," Automatice.t 17, pp. 47-56. Kokotovic, P. v. (1984), "Applications of singular perturbation techniques to control problems, tI ~ Eeview, 26, pp. 501-550. Kokotovic, P. V. (1985), "Control theory in the 80's: trends in feedback design," Automatica, 21, pp. xx-xx. Kokotovic, P. V•• K. Khoresan! and H. Spong (1985), "A slow manifold approach to nonlinear two time scale systems," ~ American Control Conference, Boston. [(opel, N. (1 gJ' 9), "A geometric approach to boundary layer problems exhibiting resonance, II .§lAM.J!• ..A.I2Il.l. • .Me..t.h., 37, pp. 436-458. Kouv?l'itakis, B. (1978), "The optimal root loci of linear multivariable systems," lnl. ~. Control, 28, PP. 33-62. Kouvaritakis, B. and J. M. Edmunds (1979), "A mUltivariable root loci: a unified approach to finite and infinite zeros, Int • ..!I. Control, 29, pp. 393-428. Krtolica, R. (1984), ITA singular perturbation model of reliability in systems control," Automatica, 2, pp. 51-57. Kung, C. F. (1976), "Singular perturbation of an infinite interval linear state regulator problem in opt.imal control,"..J.. Math. Anal. .A.!m.l., 55, pp. 365374. Kurine, G. A. (1 rtf7) t ItAsymptotic solution of a classical singularly perturbed optimal control problem," Sov. Math. Dokl., 18, pp. 722-726. Kuruoghu, N., D. E. Clough and W. F. Ramirez (1981), "Distributed parameter
51 linear state regulator problem," SIAM~, 17, pp. 16-37. OINalley, R. E" Jr. end A. Jameson (1!!l5), "Singular perturbations and singular arcs--part I," ~~. Automat. Control, 20, pp. 218-226. O'Malley, R. E' I Jr. (1976), "A more direct solution of the nearly singular linear regulator problem," SIAM~. Control Option, 1~, pp. 1063-1077. O'Malley, R. E., Jr. and A. Jameson (1977), "Singular perturbations and singular arca--part II, It .llillE...'r.r.e.n.§.. Automat. Control, 22, pp. 328-337. 0 1 ~lalley, R. E. I Jr. and J. E. Flaherty (1977), "Singular singular perturbation problems, TI Singular Perturbations and Boundary ~~, Lecture ~ ~ MathematiCS, 594, Springer-Verlag, New York, pp. ~22-436. O'Malley, R. E., Jr. (1978), "Singular perturbations and optimal cont.rol," in Mathematical Control ~, Lecture Notes in Mathematics, 680 , SpringerVerleg, New York. DIHalley, R. E., Jr. (1978), "On singular singularly-perturbed initial value problems," Applicable Anal. 1 8, pp. 71-81. Ot~lalley, R. E., Jr. and R. L. Anderson (1978), "Singular perturbations and slow mode approximation for large scale linear systems, If Proe • .Qf IFAC!11lIA Workshop Qn Singular Perturbations in Control, France, pp. 113-121. O' Nalley, R. E., Jr. (1 fJl9), ITA singular singularly-perturbed linear boundary value problem, If ..Q.IPJ1;t. Math • ..An.el.., 10, pp. 695-708. O'Malley, R. E., Jr. and J. E. Flaherty (1980), "Analytical and numerical methods for nonlinear singular singularly perturbed initial value problems, II SIAM.J. AQRl. Mslh. t 38, pp. 225-2~8. O'Nalley, R. E. t Jr. (1982), .RQ.Qk Reyiews, Bull. (llew Series), Amer. Hs.th. . .QQ.Q.., 7, 2, pp. Jl14-~20. O'Malley, R. E., Jr. ( 1983) , "Slow! fast decoupl iug analytical and numerical aspects, tI .kI.S.M Courses.ru:!!i Lectures, 280, M. Ardema, ed., Springer-Verlag, pp. 1~3-159. 0 1 Reilly, J. (1 fJl9a) , "Two time scale feedback stabilization of linear time varying singularly perturbed systems, It~. Franklin 'Inst. I 308, pp. 465-474. O'Reilly, J. (1979b), "Full order observers for a class of singularly perturbed linear time varying systems, II .In.t. . .J.. Control, 3D, pp. 745-756. O'Reilly, J. (1980), "Dynamical feedback control for a class of singularly perturbed linear systems using a full order observer, II Int. ;I. Control, 31, pp. 1-10. O'Reilly, J. (1983a), TlPartial cheap cont.rol of the time-invariant regulator, II .Int. 1.. Control, 37, pp. 909-9Z7. OIReilly, J. (1983b), Observer for Linear Systems, Academic Press, London. O'Reilly, J. (1985), "The robustness of linear feedback control systems to unmodeled high frequency dynamics, n Proe. ~ I Control 1l5.. f Conference, Cambridge. Ozguner, U. (197 9) I lINear-optimal control of composite systems: the multi-time scale approach," ~~. Automat. Control, AC-2~, pp. 652-655. Peponides, G., P. V. Kokotovic and J. H. Chow (1982), "Singular perturba tiona and time scales in nonlinear models of power systems," IEEE~. Circuits ~ Systems, CAS-29, pp. 758-767. Peponides, G. M. (1982), "Nonexplicit singular perturbations and interconnected systems," Ph.D. TheSiS, Coordinated Science Laboratory, Report R-960, Univ. Illinois, Urbana. Peponides, G. and P. Kokotovic (1983), "Weak connections, time scales and aggregation of nonlinear systems," ~ Trans. Automat. Control, AC-28, pp. 729-735. Pervozvanskii, A. A. and V. G. Gaitsgori (1978), "Suboptimization, decomposition and aggregation, n .Lt.h .I.EAC .H.Qr.lQ. Congress, Helsinki. Pervozvanskii, A. A. (1979a), DecompOSition, Aggregation and Suboptimization, Nauka, Moscow. (In Russian.) Pervozvanskii, A. A. (1979b), "Perturbation method for LQ problems: duality in degerate cases," Proe. ~~ Workshop on Multileyel Control, Warsaw. Pervozvanskii, A. A. (, 980) f nOn aggregation of linear control systems," Autom. .sru;! ~ Control, 8, pp. 88-95. Pervozvanskii, A. A. (1981), "Degeneracy in LO and LQG problems of optimal cont.rol; possibilities to simplify the syntheSis, IT Proc. lli.I.E.A.Q Congress, .KyQtQ. Phillips, R. G. (1980), "Reduced order modeling and cont.rol of two time scale discret.e systems," ln1. ;t. Cont.rol, 31, pp. 765-780. Phillips, R. G. (1980) , "Decomposition of time scales in linear systems and
53 Conference~. Science, Princeton. Sannuti, P. (1985), ITNul ti-time scale decomposition in cheap control problems I" ~ ~. Automat. Control, AC-3D, pp. xxx-xxx. Sastry, 8. 8. and C. A. Desoer (1981), "Jump behav iour of cireui ts and sy stems, IT ~~. Circuits and Systems, CAS-28, pp. 1109-1124. Sastry, S. 8. (1983), "Asymptotic unbounded root loci-formulas and computation," ~ Trans. Automat. Control, AC-2B, 5, pp. 557-568. Schuss, Z. (1980}, "Singular perturbation methods in stochastic differential equations of mathematical physics," SlAH Review, 22, pp. 119-155. Sebald, A. V. and A. H. Haddad (1978) I "state estimation for singularly perturbed systems with uncertain perturbation parameter," IEEE ~. Automat. Control, AC-23, pp. 1164-469. Shalced, U. (1976), 'IDesign techniques for high-feedback gain stability," .Int . .sI.. Control, 24, pp. 137-144. Shaked, U. (1978), liThe asymptotic behaviour of the root loci of multivariable optimal regulators, II IEEE Trans. Automat. Control, AC-23 , pp. 425- 1130. Shinar, J. (1981), "Solution techniques for realistic pursuit-evasion games, II Adyances in Control ~ Dynamic Systems, C. T. Leondes, ed., 17, Academic Press, New York, pp. 63-124. Shinar, Jr. ( 1983), "On appliea tions of Singular perturbation techniques in nonlinear optimal control," Automatica, 19, pp. 203-211. Shinar, J. and N. Farber (1984), "Horizontal variable-speed interception game solved by forced singular perturbation technique, .sI.. Optim. Theory ~ Control, 42, pp. 603-636. SUjak, D. D. (1g] 2), "Singular perturbation of absolute stability, n .I1ili.E ~. Automat. Control, AC-17, p. 720. Simon, H. A. and A. Ando (1S61}, "Aggregation of variables in dynamic systems,lI Econometrica, 29, pp. 111-138. Simon, H. A. (1 S6 2), "The architecture of complexi ty, II~. American Philosophical SOCiety, 104, pp. 467-482. Singh, R. -N. P. (1982), liThe linear-quadratic-Gaussian problem for singularly perturbed systems, n .l.r!t.• ..J.. Systems Science, 13, pp. 93-100. Slater, G. L. (1984), "Perturbation of optimal integral controls, 11 ~ Trans., ~. ]xn. Systems, Heas. Control, 106, pp. 114-116. Sobolev, V. (1984), "Integral manifolds and decomposition of singularly perturbed systems," Systems £ill! Control 4, pp.169-179. Sridhar, B. and N. K. Gupta (1980) , guidance laws based on singular perturbation methodology,".sI.. Guidance snQ Control, 3, pp. 158-165. Suzuki, H. and H. Hiura (1 '!l6), "Stabilizing feedback controllers for singularly perturbed linear constant systems," IEEE Trans. Automat. Control, AC-21 , pp.
123-12~.
Suzuki, M. (1981), "CompOSite controls for singularly perturbed systems," ~ ~. Automat. Control, AC-26, pp. 505-507. Syrcos, G. P. and P. Sannuti (19B1!), "Near-optimum regulator design of singularly perturbed systems via Chandrasekhar equations, n Int • .4.. ~, 39, pp. 1083-1102. Teneketzis, D. and N. R. Sandell, Jr. (1 '!l7) t "Linear regulator deSign for stochastic systems by a multiple time scale method," ~~. Automat. Control, AC-22, pp. 615-621. Tikhonov, A. (1948), "On the dependence of the solutions of differential equations on a small parameter, It 1:le.t.. Sb., 22, pp. 193-204. (In Russian.) Tikhonov, A. N. (1952), "Systems of differential equations containing a small parameter multiplying the derivative, II Mat. Sb., 31, 13, pp. 575-586. (In Russian. ) Tsa!, E. P. (1 g]8) t "Perturbed stochastic linear regulator problems," SIAM .sI.. Control, 16, pp. 3 £6- 1/1 O. Utkin, V. 1. (1lJ77a), Sliding Hodes and Their Application .tQ. Variable Structure Systems, Mil' Hoscow. (In English.) Utkin, V. I. (1977b), "Variable structure systems with sliding modes: a survey," IEEE~. Automat. ~ontrQI, AC-22, pp. 212-222. Utkin, V. I. (1983), "Variable structure systems: state of the art and perspectives," Automation and Remote Control, 9, pp. 5-25. Van Harten, A. (19811), "Singularly perturbed systems of diffusion type and feedbaclt
55 22, pp. 931-938. Young, K. D. (1977), "Analysis and synthesis of high gain and variable structure feedback systems," Ph.D. Thesis, Coordinated Science La.boratory, Report R800, Univ. Illinois, Urbana. YounS, K. D. (1978), "Hultiple time-scales in single-input single-output high-gain feedback systems, n..J.. Franklin Inst., 306. pp. 293-301. Young, K. D. (1982), "Near insensitivity of linear feedback systems, Tf ..J.. Franklin lost. I 31~, pp. 129-1~2. Young, K. D. (1982), "Disturbance decoupl ing by high gain feedback," IEEE ~. Automat. Control, AC-Zl, pp. 970-!171. Young, K. D. and P. V. Kokotovic (1982), "Analysis of feedback loop interactions with actuator and sensor parasitics , " Automatios, 1S, pp. 577-582. Zien, L. (1973), "An upper bound for the singular parameter in a stable, singularly perturbed system, ".!L. Franklin Inst. I 295, pp. 373-381.
Part I: OPTIMAL CONTROL
SINGULAR PERTURBATIONS FOR DETERl\1INISTIC CONTROL PROBLEMS
A.
BenSOUSSllllt
INTRODUCTI ON. The problems considered in this article are of the following type. Consider a dynamic system whose evolution is governed by dx C
err
,yE ,v)
xE(0) ::: Xo
£~ dt
g(XE,yE,V)
y£(o) = Yo
( 1)
in which v{t) represents a control. The parameter E tends to O. The state of the (t)) contains one part xE(t) which varies slowly, and one part system (x£(t), (t) which varies fastly. Such a situation is common in the applications. It appears for example in economic models to take into account long term and short term variations, but also in many problems of engineering. biology, mechanics ... The terminology "singular" explains as follows E = 0, namely
the problem corresponding to
f{x,y,v) (2)
g(x,y,V)
0
is of a type different from the case rential equation).
E >
0 (an algebraic equation replaces a diffe-
'F'TP.IA. Domaine de Voluceau, nOcrttH!fIt:Ourt. rtp, 105. 78150 LE CllESNAY CEDEX. France and de Pa,i.~ - iJullphillc.
U/liVfr.~ilc
61
(cf. P. FAURRE. M. CLERGET, F. GERMAIN [IJ) and the structure of the set of solutions is interesting. We have presented it beyond what is strictly necessary to solve the boundary layer problems. The non linear case (often referred as the trajectory optimization in the litterature) has been considered in particular by P. SANNUTI [lJ, [2J. R.E. O'MALLEY [3J. [5J, P. SANNUTI - P.V. KOKOTOVIC [IJ, C.R. HADLOCK [lJ. M.I. FREEDMAN. B. GRANOFF [IJ. ~1.L FREEDtMN - J. KAPLAN [1], A.B. VASILEVA, V.A. ANIKEEVA [1], P. HABETS [l], M. ARDEMA [lJ, [2J ... ). In general the poi nt of view is to \'Jrite the necessary conditions of optimality and to find expansions. A pl'oblem \,Ihich is considered is to solve the necessary conditions of optimality for the E problem by perturbation techniques. ~Je do not treat this problem here. On the other hand the evaluation of the cost function for "good" controls does not seem very much considered in the litterature. nor the expansion of the optimal cost. The fact that the control Uo itself yields an approximation of order E was known at least in the L.Q case, although the proof given relies on the boundary layer analysis. ~Je show this fact in general \oJithout using the boundary layer. The presentation of the convergence in the "constraints" case (lack of regularity) has not either appeared in the litterature. The study of Bellman equations in duality seems also original. It should be interesting to study the complete structure of the set of solutions. In the Dynamic Programming approach, the main concept is that of feedback, due to J. CHOW - P.V. KOKOTOVIC [lJ. We extend this work and particular that the decomposition of the composite feedback as the sum feedback and a complementary term involving the fast state ;s general, tricted to a quasi linear structure of the dynamics.
composite prove in of the limit and not res-
63
1.2. The limit problem Consider first the algebraic equation (1.6)
g(x,y,v) = 0
in which x,v are parameters and we solve (1.6) in y. By virtue of (1.2), the equation (1.6) has a unique solution ~(x!v). Moreover differentiating formally (1.6) with respect to x,v we obtain (1. 7)
o
These formulas show that
~x' ~v
are continuous functions of x,v, and bounded.
Consider then the system. for v(.) (1. 8)
2 (O,T;R k)
c L
dx = f(x,y(x,v).v) at x(O) = xo'
- (l.B) has one and only one solution xC.) in H1 (O,T;R n ). By the properties of y. The limit problem consists in minimizing
(1.9)
faT 9-(x(t).~(t),v(t»dt
J(v(.»
+ h(x(T})
in which we have set ( 1.10)
~(t)
~(x(t),v(t)}.
We shall make assumptions on the limit problem. We shall assume basically that the necessary conditions of optimality (Pontryagin principle) are satisfied, as well as 2nd order conditions. This will imply, among other things, that the limit problem has a unique optimal solution. We shall define the Hamiltonian (1.11)
H(x.y,v,p,q}
= £(x,y,v)
+ p.f(x.y.v) + q.g(x,y,v).
65
and (1. 16)
9.
n.
xx - (9. xy Q, xv )
Conditions (1.15). (1.16) coincide with (1.12). (1.13). when Xo = x*. Therefore . It is possible to the constant control u* is optimal for (1.8). (1.9) when x show, at least v/hen Uad = Rk and for data sufficiently smo~th (cf A. BENSOUSSAN [lJ) that taking Xo - x'" sufficiently small. there exists a function wo(t) satisfying conditions (1.12), (1.13). L,3.
We can state the following convergence result Assume (1.1). (I.2) and the exi (1.13). (1.14) hold. Then one has (1.17)
Inf J€(v(.))
-+
uE - Uo
-+
a
in L2 (O.T;R k)
- Yo
-+
a
in
- Xo
-'r
a
in H1(O,T;R n )
of wo{t) such that (1.12),
inf J(v(.).
( 1.18)
(1.19 )
L2 (O.T;Rm)
The proof of Theorem 1.1 is done in several Lemmas
0
67
and from the 1st differential equation (1.21). we get also (1. 22)
T dxE: 2
Io Ierr I
dt ;::;
From the estimates sequence
Ko'
(1.21)~
(1.22)~
we can assert
that~
at least for a sub-
(1. 23)
Co ns i der; ng
nO\l1
from (1.23) it remains in a bounded set of L2(O~T;Rm}. But from the second differential equation (1.20)~ taking ¢ C~(O~T;Rm) rl~dt
f
To
hence (1.24)
. .;. 0
in
To proceed we use the classical technique of MINTY [1J (cf also J.L. LIONS [IJ). Let m 2 Z E L (O,T;R ). We have from (1.2)
hence (1. 25)
From the 2nd differential equation (1.20) we deduce
69
It is then possible to pass to the limit in the 1st differential equation (1.20) and to deduce
which together with (1.26) implies From the uniqueness of the limit we can assert that
and thus the desired result obtains
o
Lemma 1.2. The functions uE,yE remain bounded in L2(O,T;Rk) and L2 (O,T;Rm) respectively. The function xE remains bounded in H1(O.T;R n). --Proof. Let us set
It will be convenient to use the notation a = (x,y,v)(recalling that w = (x.y,v,p,q)). He thus write
Let us establish the formula (1. 27) 1
+
1
fTo fa fa
.a dtdAd~ -E-E • H (w,E }a
A
acr
IIIJ
71
Then
(H )-l() yy Hyv
+ (H
xx -
(H
H ) xy xv
H HVY vv
where y ;s a positive number independant of
Hyx
(XE)2
;:::
ylz E I2
Hvx
A,~.
Therefore we deduce from (1.27) and from the last condition (1.12) (1.29)
g(o
o
(note that
depends on
A,~).
Noting that
and the assumtion (1.18) on uE ,
we obtain
I2dtdAd)J. On the other hand
T [1 J1
Ja Ja a ~IZEI
))dt + Y
2
dtdAd~
73
and also
since yE is bounded. Therefore n (1.32) that
= O. Using this fact and Lemma 1.1, we deduce from
Using (1.31) and the definition of Since there exists always
• we easily prove (1.19).
such that
and
Taking account of (1.19). we have JE(U E) ~ J(u )' hence o
Since also
we also have
which completes the proof of (1.7).
o
1.4. Stronger convergence results in the case of regularity.
One can improve the convergence result of Theorem 1.1, when the following additional regularity is satisfied (1. 33)
dQo
E
2 m L (O,T;R )
75
On the other hand
with y
=
hence from (1.13)
n
The proof is now complete Our objective is to prove the follmv;ng
Theorem 1.2. We make the assumptions of Theorem 1.1. and the regularity assumption (1.33). Then we have ( 1. 36)
IInf JE(v{.}) - inf J(v(.)}i
$
CEo
If u _ _ _ _ (1. 18) then (1. 37)
o The proof relies on the following imProvement of Lemma 1.1, Lemma 1.4. We have (1. 38)
Proof. Let us improve the convergence of XE,yE to xo'Yo (cf Lemma 1.1) and in fact simplify the proof, thanks to the regularity {1.33}. We have in fact
77
This and the t\vO first estimates (1.39) imply at once from the expression (l.40) that (1.38) holds true o Proof of Theorem 1.2. Consider (1.27) which is now written as ( 1.41)
We use next the relations
to deduce as in Lemma 1.4,
Taking account of the definition of Z£, we derive the inequality
+
hence also (1. 42)
S,A.ll)1 2ds]
79
The assumption (1. 2) means (2.5)
o
~
- jJI
and \lIe have
(2.6)
Hxx Hxy
Q,
Hyy -- R • Hvv :: N
= Hxv = HyV
O.
He assume
(2.7)
M,Q,R.N symmetric. R,N positive definite, Q.M non negative.
The assumption (1.12), (1.13). (1.14) are clearly satisfied. In this case we can also assert that there exists an optimal control for the solution of payoff JE{v{.)) itself. Therefore these exists (XE,yE,u E ,qE) = the system
(2.8) d £ £~ -
E dt - Cx +
(2.9)
(2.1O)
and eliminating (2.11)
yields
+ HUE +
b ,
(0) = Yo
81
(2.16) Yo(O)
= Yo - Yo(O)
dl - --.9 = de
D*L
0
(2.17) D*Q o
+
R Z0
(2.1B) (2.19) dM
(2.20)
- erro dK
(2.21)
C*l
o
~
-- e*Q 0
dY 1 -dT
= DY 1 - H N- H*l 1
dT
(2.22)
(2.23)
(2.24)
1
+
1
ex 0 - H N- G*M0
83
Proof. a) _ _ _ _ _ __
Let us show the existence of a non negative solution. Consider indeed the control problem (2.27)
dY ar t;.
D Y + W-s to
Y(O)
h
L2 (O,tlJ;R k )
h(~)
~
f:(R y2
+
N ~2)dT.
Note that by virtue of (1.2), Y ~ H1 (O.oo;Rm) and the control problem is well defined. The optimal control is unique. We can write the Pontryagin conditions of optimality, name1y {2.28}
Note that the set of conditions (2.28) defines a unique pair Yo' La. Writing
one easily checks that in fact
and using this in (2.28), necessarily
is an optimal control for (2.27) and (2.29)
TI
is a solution of (2.26). Note that
85
and thus if
TI
satisfies (2.26)
.l -rrh2 2"
1;1/ (r)
,vh ""
hence 1 - 2 lTh
1
2"
2"
1rh
2
b) negative solution This is slightly more (2.32)
- dY aT
intricate. Let us conside.· the dynamic system
DY + HE:
T
(0
2 m YcL(O,co;R).
Note that for ~ c L2 (O.oo,R k} ,there exists one and only one Y solution of (2.32), ~hich is square integrable. Hence Y(O} has a unique value. Define
Define next
E(h)
{~rY(O)
= h}.
Assume the controllabili (2.33)
E(h) is not empty,
condition for the pair (O,H) ~
h.
and consider the problem (2.34) ~
(: E(h)
It is easy to check that S((h) is a quadratic function and thus can be (2.35)
\'JI"~tten
87
therefore u~o(+T)2 increases as T which implies that t:o
l 2 (O,rn;R k ), -Vo
~
t
+ =. Since it is negative, there is a limit,
L2 (O,oo;Rm), hence
;0
E
E{h),
and
(2.38) Let
us
check that f,;o is optimal. Indeed take
f, c
E(h), and set
V :: V - Vo
l~e
E, + N- 1H* UY
+ N-1 H*lTV
~ = f,
~o'
have
-V(O)
(2.39)
:: O.
and
and from (2.39)
froo 7rY H~ 0
dT::
foo
lTV {-
~
DOT
Collecting results we obtain
- (0 - HN-1H*lT)Y)dT
89
and
(h) = { n I L(O)
~ ~
2 E h
"21
h}.
. f ~ (i't*h (T1 )
ln
we obtain the unique non negative solution of (2.40). To get E positive definite it is sufficient to assume that.
H H*
(2.43)
has a left inverse.
This suffices to verify the controllability condition (2.33), by virtue of Remark 2.1. In fact the special representations of E related to the control problems (2.41),
(2.42) cannot be generalized and are limited to the L.a. case. However it is possible to give two interpretations which will be generalizable. Consider the control prob1em (2.44)
dZ - dT
DZ + HIl
Kh ( rj)
1 2
fto ( RZ 2 0
+
2
Nll ) dT -
Z( 0 ) h
in which 11 IE L2 (O.w;R k) (note that Z is uniquely defined in H1(O,m;Rm), and in particular Z(O) is completely determined). Let us set inf
(2.45)
Kh{q)
11
then
[
.. IT
-1
-1- E h2
is the positive solution of (2.40).
Define next the problem (2.46)
dZ dT
DZ + Hn
• Z(O)
91
and
and from the invertibility of 1i~ the 1st and the 2nd estimates (2.49) fo1101'I. The two last are proved in a similar way o ----
: One has
(2.50)
Proof.
and from (2.49) one deduces the 1st estimate (2.50). Similal- considerations hold
o
for the other quantities The system
(2.22)~
(2.23) is similar to
(2.14)~
(2.15) and has a unique solution
related to a linear quadratic control problem. Lemma 2.3. VI' L1 _ _-"'-_ _ _ _ _ _ _ _ _ Yo,L o and ZI,Q1 the same as Zo' Qo (cf (2.49). Proof. One has
and PI ;s the solution of p
- d (f-[1
= (0'* - nHN -1 H* )PI + (8 * - nHN -1 G* }Mo + nCXo
93
and (2.17) to the control problem (2.53)
1 -2
f'''' o
(RZ 2 + Nr, 2 )dr - Z (0)
a
a
0
qo(T).
The relations (2.22), (2.23) relate to the control problem dX (2.54)
1
crt
AX l
+
BY1 + GU 1
Finally for (2.24), (2.25) we get the problems (2.55)
(2.56)
The various optimal controls are obtained in function of the adjoint variables by the fOl'mul as (2.57)
SO(T)
= - N-lH*Lo(T)
no(t)
N-1H*Qo(t)
95
and clearly one has (2.65)
Let us then define
we derive the following equations from (2.11). (2.12) (2.66)
(2.67)
(2.68)
- E
2
d'':S _9.E::dt -
(2.69)
qE(T) From (2.66).
(2.69) one deduces the relation
(2.70)
= cyS(T)(L (l) + EL (l)) 1 E o E +
f: ["
N" IG*
(G*pE +
(O)(ZO(l) +
"
dq1 Cit )dt.
QX~,pE
E
Zl
E
)
+
+ CqSjE " B*~,;y£]dt
97
and this comp1etes the proof of the desired results
0
We turn now to the expansion of the optimal cost JC fo 11 0I11i ng terms
). We shall need the
(2.75)
f0 L &r U
- (d:to(O) + cxo(o))
x3
= .:;) ([ )
"'- 1 -1
2 d y
- ---22(0)
dt
+
+
]
0
'
+ 2 d y fCC dP l Jet) TLod T + ---2 {T) TQ dT + ---dt (O) T(BY o + G~o 2 0 dt 0 0 0
fco
fa
(I·Vxo + KoAUo)dT - d:t1(T)
f
-21 Q(X 2 + U2 )dT 0 o
Cto)
IT0
J:
l q1 dY crt dt
T(BZ o + GnO)dT
-
+
Y1(0) q1 (0) +
O)
fo K dT. 0
One can then state the Theorem 2.2. Undet the assumptions of Theorem 2.1, _ _ _ _ _ _ __
(2.76) Remark 2.3. We have the related estimates
+
99
+ 2r yE + 2n
He note that
(2.80)
(0)
=-
(2.81)
We compute the quantity
(2.82)
+
101
Considering
+
fil~st
the
1.. 2
0
tel~ms
which may be of otder
• vie get
R(y2 +
But
T
fa
[Ex1(Qx o + q) + (EYI + Yo + Zo)(Ryo + r) + (cUI
T -
E
fo
dX 0
E
f
Collecting results we obtain the term T
E
dpo
f0 err
dY
err(V o
0
qo (Cit
eX
dqo (Xo + Uo}dt +
dU 0
P ( - + -}dt a dt dt
T -
I
+
dZ
dyo
a
q
,w-e
t: 2
IT (L a
a
d'C
0
and a term of order 2,
get - E
-
- . dt-
+ CiT) dt
+ Zo)dt.
Considel~ing next the terms of ordel~
+
0
2
103
We now collect the terms of order 3. Letting aside those in the last bracket of the preceding expression! we obtain
Nothing that
Gnl)Jdt =
+ P1(AX o + AU o + BY 1 + BZ 1 + G~l
£2 J:
:::: -f;:
r:(-
d:l
_A*Pl - C*qj)(XO + Uo ) + (-B*pj
2 rT dPl j o dt (X 0 + U0 )dt -
E
3
Collecting results we indeed obtain of {2.76)
IT 0
E
q
dY 1
dt - c 1 dt
3
O*qj)(Y j + I j ) +
IT 0
q
1
dt
3X + O(E 4 )! which completes the proof 3 0
105
We shall write (using a notation of HABETS [IJ)
These functions depend on an argument T which plays the mle of ~ or t..~ E The first case corresponds to the upper index 0 and the second to the upper index T. With this notation we have the asymptotics (3.3)
We shall also use the notation
Ii
and similar definitions for (t), [~(T) ~ Cr). The key to wl"ite the expansion 1ies in the following formal consideration. Let be an expansion (t)).
107
1 + -2
{II
\1M
T T T t- T • (VI 0 ( T) + l'l 0 ) I·J 1 (\'1 + 2 ( E -O 'Ii (T) + 1
Using this expansion in (3.1) we obtain (3.5)
He then have (3.6)
dV O dT
=
g(oo(O) + I:~(-r)) ~ Yo(O}
=
H, (\./ (T) + H (T)) Y 0 0
= Yo
(3.7)
dQ o -d T
T
(3.8)
dU (3.9)
- ---dO = f(a (T) + IT(T)} - f(oo(T}} TOO
- Yo(O)
\'i 1 ( T)
))
109
I'le note also the relations, obtained by expanding the last relation (3.1) (3.16) (3.17)
(3.18)
(3.19) - Hvw (w0 (T)))(- T~ 0 (T) + wI(T))
=
a
Note also that in fact
Vt
(3.20)
3.2. Optimization problems We shall now relate the relations (3.5) to (3.20) to some optimization problems which will allow us to prove the existence of solutions. The system (3.5) relates to the limit problem (3.21)
::: X
o
·T J 1(X ,y ,u )dt
,) 0
0
0
0
+ h(x (T)). 0
The relations (3.7) together with (3.16) relate to the control problem (3.22)
dY o
err
g(xo(O),yo(O)
+
Yo,uo(O)
+ ~o)
111
where we have used the b/o last relations (3.13) as \'/ell as (3.20). Similarly the conditions (3.15), (3.19) are related to the control problem in which the state equation is given by the 1st equation (3.15) and the cost functional is given by (3.26)
3.3. Solution of the optimization problems. Under the conditions of Theorem 1.1 we know that the limit problem (3.21) posesses a unique solution uo ' We shall then concentrate on (3.22), (3.23). As we have introduced Riccati equations in the L.Q. case (cf (2.26), (2.47)) we shall consider analogously Bellman equations in duality. 3.2.1.
9~1.1!f1,~~n
equations in duality.
Let us introduce some notation (3.27)
F(Y
(3.28) We note the properties (3.29)
F(O,O) = 0, Fy(O.O) = 0,
F~(O,O)
=0
113
Let us prove (3.35). Set
then from (3.33) we have
~(h)
inf
f:
[F(Y.() +
2aY.G(Y~~)]d~
and ~
0
for a sufficiently small, hence (3.35)
[1
Remark 3.1. As in the L.Q. case it is possible to introduce the problem dY - err
= G(Y
2 k L (O~m;R )
, E,
and to minimize
on the set E(h) = C~I Y(O)
hJ
assuming this set not empty for any h (controllability condition). Let us term
(3.36)
~ (h) = - I nf;i)U;) €;EE(h)
O.
Then we may wonder wether ~(h) is the minimum solution of (3.31). One of the main difficulties lies in the fact that it is not possible easily to establish that 1: belongs to the class (3.34). Note hO~lever the optimality pt-inciple .·,5
T(h)
i nf r ~ (.)
F(Y ,EJ dT -
J'
~ (Y (6)
)
0
from which (3.31) can be derived in any point of differentiability of
~.
115
This gives the 2nd estimate (3.34). The Lipschitz property (3.34) is almost obvious from the definition of,jt,,(n). Pick a point h \1here q) is differentiable. and for any E a control IlE such that
Let ZE be the corresponding trajectory. One can assert that (3.41) Indeed ZE(O) is bounded, since
ITiEI :; Clhj, and -
E
2
Pick a cluster point of Z (0), Z*. using the differentiability in h. we get E after dividing by E and letting E tend to 0 Dl/J(h) k
?::
kZ* D~}(h). By the uniqueness of the limit (3.41) follows.
and since k is arbitrary Z* Note that
[F(Z E ,n E ) - hG(Z E ,n S )JdT But setti ng
we have
hence
f
OO
E:
F(Z ,n )dT - hZ~(E) E
E
!:..
(n- ) E
+
F(Z.q )dT - hZ (E). E E £
117
hence (d=Z(O). E
Therefore Jj{h(n.J
fE [F(Z (s) ,rIo) - h G(Z (s)'no)Jds +d,:tt'l(n ). 0
r.
E E '
E
It ;s aasy to check that
hence
'~Jcn
s [F(Zc(O).ll ) - h G(Z (O),n }J + Cs o E: 0
2
+:~~'n ('1 E: )
and from the defi nition of
or
and letting
E
tend to O. we deduce
Since no is al"bitrary, we have proved the reverse inequality of (3.43), hence the result desired 0 Consider next the control problem (3.44)
~
= G(Z,n-)
Z(O) :: ;:;
119
Let us set
and assume that there exists a triple (3.47)
(Yo'~o,Lo)
such that
dYo dT :::
G(Ya,t;;o)
(3.48)
Remark 3.4. For h = 0, the relations (3.47), (3.48) are satisfied with Yo'~o' La equal to D. From the assumptions (3.29), (3.30) it can be established that (3.47). (3.48) hold for h small.
o
When (3.47). (3.48) hold it follows from classical results on Bellman equations that : (3.49)
¢ is differentiable at any point YO{T) of the trajectory and
We can then prove the exponential decay Proposition 3.4. Assume (3.47). (3.48) then one has (3.50) Proof. From (3.49) we can asset"t that @(YO(T}) ;s a.e. differentiable and
i 21 I~d
The 3 the proof
equation (3.47) implies the same estimate for so(t). which completes
o
We next make ,similar consideration for 1/)(h). We assume that there exists a trip1e Zo.Qo,n o such that (3.52)
(3.53)
l>le then have
Proposition 3.5. When (3.52), {3.53} hold; no is the unique optimal control for (3.38). Moreover ~ is differentiable along the trajectory Qo(t) and (3.54)
Proof. One easily checks the formula (3.55)
Therefore if
h = h. (3.53) implies
which proves that no is the unique optimal control.
123
and thus (3.56) is satisfied whenever (2.60) holds. We then have Proposition 3.6. Assume (3.52), (3.53)
(3.56). _ _o_n_e_h_a_s
(3.57)
Proof.
We first prove that (3.58)
Indeed, notice that (3.59)
¢(h) ::
J~
(-
and from the last relation (3.52) and (3.53)
On the other hand, thanks to (3.56) and the 1st and 3rd relations (3.52) one has
and from the 2nd relation (3.52)
or
Collecting results (3.58) obtains.
125
3.3.3. Application to the boundary layer problems We assume that there exist solutions of the systems (3.5), (3.6) ~nd (3.16), (3.7) and (3.17). We need also 2nd order conditions like {3.48}, (3.53). In fact for the convergence we shall need more stringent assumptions, which we make right now for the sake of brevity. (3.62)
C yy
Hvy
yv H
)
(x,y,v,po(t), qo(t} + Lo(T))
~
SI
(x,y,v,po(t), qo(t} + QO(T)}
~
SI
Hvv
yy
C vy
:yv) vv
(3.63) ~
with the same arguments as in (3.62),
0
V x,y,V,t,T.
Note that (3.62). (3.66) contain (1.13), since Lo' Qo vanish at 00. Using them with either x = xo(D), t = 0, or x = xo(T}. t T. we recover (3.48) and (3.53) correctly interpreted with F, G given by (3.27). (3.28) or the analogous ones with Wo(T) instead of wo(D). We also need (3.56) only for the boundary layer at T. It reads (3.64)
g:(XO(T),Z,n} has a left inverse which is bounded in Z.n.
Under these assumptions we can assert the following exponential decay properties (3.65)
Ce -Y1: ,y
From this and (3.8) ... ,(3.11) we deduce (3.66)
>
o.
127
(4.3)
Moreover we can express Je(u E ) as (4.4 )
J E(uC::)
= J U(o) -
£
JT0
qo ~ dt dt + fT0 [H( 0 E: ,po,qo ) - H( Wo )
-
We shall make use of the following properties (4.5)
I
T
IP
IOE
DE:
I tnJX (!) I (4.6)
I
(t)X {.!.)U (T-t)dt
0 E
OE
;<;;
e
-KJlE: IT I{p (t)ldt 0 E
::; CEmt-l Ll
ItnlX o(i) I
2 L
CErn/- 1/2.
5
m~ 0
and analogue properties for Uo(T~t). In (4.5). (4.6) Xo' uo are generic functions with exponential decay. (4.7)
1fT F {t;X (Jl) + U (T-t))dt OE
-fTo F (t;U E
DE:
0
DE
(T -t) )dt + E
refers to the 2nd argument.
-
-k l
f0T
provided all integrals of the form
ITOE F (t;Xo(Jl)dt E
(t;O)dtl ·T
J
o
IIa
$
Ce
E:
IFE,;(t,X o + ~Ua)ldtd~ are bounded and;
129
(4.12 )
which easily follows from (4.5), (4.6), (4.7)
o
4.2. Expansions We write the 3rd integral at the right side of (4.4) as follows ( 4.13)
This notation deserves some explanation. Remember that the variables w have 5 components, and C1 (or I) have 3, which are the first three of VI. l~hen we add w + a we implicitly imbed C1 in a 5 component vector, by equating to a the last two. So, for instance W
a
+ EO +
+
0
The expression {4.13} is written as
(4.14 )
+
Ha (w0
+ EO + 0
in which we have set (4.15 )
\4e next concentl-ate on the terms of order 2 in el ,namely
131
But
From (4.2) we derive, taking account of (4.5) (4.18)
- e::fo (oo)01 - (f(oo(O) + E~) - f(ao(O) )Jdt + T
+ E
fo MO(fy(oO(O)
+
E~)yE
+
fv(oo(O)
+
E~)UE)dt
and
(4.19)
o t . + gva(oo(O} + EO}(E: 00(0) + °1(0) + +
fTo f1 f1 A Log 0
0
00
(W~II ) (cr E }2dtdAdll
+
+ O(
I + I
1+£2),
+
133
in
a
C
the expressions (4.19) and (4.20) being a contribution to the quadratic form entering in (4.14). Adding up we g~t the ~uadratic furm )2dtdAdW where we have set
(4.22)
-E a T \v,AlJ = \v a + \~ a + \>J 0 +
E (0
1
T E + r"0 ) + AjlO •
+
Collecting the terms in and liE in the expression of • by (4.16), (4.18). (4.19), (4.20), (4.21) we obtain, considering the equations defining L , Q (3.14). 1 1 (3.15) as I-Iell as the relations (3.1S). (3.19).
(4.23)
dt +
- cQ1g (0 (T) + ET)]dt + Y
0
0
IT
0
The expression (4.13) contains also a term which does not depend on the quantities . Collecting terms arising from (4.14). (4.18), (4.19), (4.20), (4.21), we obtain T
(4.24)
{H(w
+
WO
J0 0 0
+
T+ W 0
135
where we have set
Collecting terms we deduce the expression
(4.28)
- £g (0 (0) + LOO)I0 - E(g 00 1 0
+
T
fo [H(w°
+
0
(OO{O)+r.°)~g (OO(O)))(Ol(O)+~crO(O))]dt+ 0'0 e:
ToT + EW + £W ) 1 1
W + W + EW 1 0 0
- H(w ) 0
137
TW (T) + \'Jl(T) + woo
- H (\., (T))) (-
HT1)]d T +
f0T
Finally the term of order E3 is the following (4.31)
q0 (0) foo
0
Y1dT +
q0 (T)
+ U (0))3 - Y1(0) f"" Lod T o
o
-
'y
a
(0)
f<:L)
a
TL dT + 'q' (0)
a
a
f=0 TV dT 0
139
+
fooo
1 [-2 H x(w (T) + x
a
l~TO)U2a
+ K f (0 (T) + I:T)U ]dT + ox
0
00
To recover in the linear quadratic case the formulas (2.75) we notice that in this case one has
141
and
We first give the analogue of Lemma (1.4), namely denoting (4.41)
one has
U.
Lemma
The following estimate holds
( 4.42)
Let us term
xC:,
yEO:
the trajectories corresponding to the control
define
By analogy with Lemma 4.1. we establish ( 4.43)
dx c 1
E
crt = AX l
E
+ BYl +
\'Ihere 1)'£ is given by (4.11) and (4.44)
c
,and
143
(4.48)
which implies easily
(4.49)
Nm'l we can compute
be replaced by
(G E ) by a formula similar to (4.37) in It/hich
has to
. Taking account of (4.49). we easily obtain (4.42).
o Proof of Theorem 4.1. Considering
xi::,
given by (4.2). (4.3). vie start by showing like for (4.48) that
(4.50)
Define next ZE{A.~)t) by the formula
hence from (4.50)
therefore
145
and thus obtain
which proves (4.38). This completes the proof of the results desired [J
5. DYNAMIC PROGRAMMING. 5.1. Setting of the problem. Let us consider the family of control problems (5. 1)
s" (t ,T)
,
t)
=x
• yE(t)
=Y
T
(5.2)
J~,y,t(V(.) = It
t(xE(s),yC(s}.v(s)ds + h(xE(T»
4E(x,y,t) = Inf
,y,t(V(.».
and define
(5.3)
v(. )
We shall assume besides (1.1), (1.2) that (5.4)
(5.5)
(a
(x.y,v»
the 2nd derivatives of f,g are bounded by large compared to y.
where Co is not too
147
Regular means here that (5.12)
Define u(x,t} to be the unique optimal feedback in (5.11), and (5.13)
It will be also convenient to introduce q(x.t} defined uniquely by the relation (5.14)
then one has (5.l5)
Y.U,q are continuously differentiable and
In the sequel the regularity properties (5.12). (S.lS) can be assumed, in which case {5.4}. {5.5} can be omitted provided one can guarantee (5.6). Our objective is to study the convergence of ~E to
$
Remark 5.1. The regularity assumptions on the limit HJB equation playa role similar to (1.12), {1.13}. Note that they imply that the optimal control for {5.B}. (5.9) exists and is uniquely defined.
149
(5.21)
We consider the following problem (5.22)
.. £x.. ae + I nf [ F( x.•t:, Y x(x,t;y,O} = O.
This problem has a lot in common with that introduced in (3.31), which may be viewed as the limit of (5.22) as 0 * We associate to (5.22) the control problem 00.
(5.23)
dY dt = G(x,t;Y,t;)
Y(O) ::: Y 8
X(x,t;Y,O)
info ~
~~e
(.).fa
.
F(x,t;Y(T),t:(T))dT = inf ::~(x,t;y,e;~(.)) ~(.)
have the
Under the assumption (1.1), (1.2), (5.4), (5.5), the function X defined by (5.23) _ _ __ (5.24)
and is the maximum solution of (5.22). The constants in (5.24) do not depend on the the x,t. arguments Y.S, Proof. The fact that a ~ x ~ C!YI 2 is clear from the definition. Noting that the admissible controls can be restricted to satisfy (5.25)
one derives as for (3.34) the estimate on DyX.
151
We then have Lemllla 5.1. The follovling estimates hold (5.27) Proof. From the relations (5.20) and differentiating in t,x, we have
Fty(x,t;O,O) = 0 ,
(x,t;O,O)
=
0
Therefore
and evaluating the derivatives Ftyy ... , using (5.26) we deduce
and s imil arly
Similarly one can show that
Recalling the property (5.25) and using the preceding estimates, we deduce from the definition (5.23) of X the results desired
o
153
where xo' Yo are fixed. Call the trajectories (5.1) with the initial conditions xo' Yo at time t. It ;s easy to check that (5.31)
lx E (s)1
2 , E1y2(s)12,
It
T
2
lyE(s)1 ds
C(l + 1xol2 + EIY o I2).
From (5.16) we can then assert that ) f (x ,y , v ( s )) +
((p +
+
0y ('II +
)g(x,y,v(s)) +
h(x) + Eq(x,T)(y - y(x,T))
)(x,y,T)
Vlhich implies (5.32)
But from the estimates (5.29) we deduce Tt
I
f
a(lll
(""55 + Dx411 .f)(x E (s),yE(S),V(S)):;
10·T cn + 1/::(s)1 3 +
+ jv(s)I(l + I
and from the estimates (5.31) C T (1 + ~E
2
Ix 0 I
2 3/2 + ElY 0 I) .
Similarly
There estimates in (5.32) yield
(5)1
2
Il:(5)1 +
3
2
1y'=:(s)1 }]ds
155
hence
Combining this estimate with (5.33) yields the result (5.30)
o 5.3. Composite feedback. Consider the function solution of (5.34)
inf
Rwhich
is the limit of X as 8
[F(x,t;Y,~) + DyX.G(x,t;Y,~)J
~
ro
in (5.22). It is the
= O.
E;,
The infimum is attained in a point S(x,t;Y) and it is easy to check that (5.35)
1~(x,t;Y)1 ~ CIYI·
We consider a situation where
Xis
C2 and
€ is
C1 in y, with
(5.36) (5.37) It is possible to show that these properties hold when we reinforce (5.5) by d Co (5.38) the 2n deri ~ati ves of g are bounded by ---=---- with Co not too large.
1 +
FollO\.;;ng a concept introduced by J. composite feedback, the function (5.39)
CHm~
Ixl
+
Iyl
and P.V. KOKOTOVIC [1], we shall term
uc(x,y,t) = u(x,t) + ~(x.t;y - Y(x,t)).
Considering the function (cf. also (5.28)) (5.40)
~1(x,y,t)
= q(x.t)(y - y(x,t))
it satisfies the equation
+ X(x,t;y - y(x,t))
157
Let us set
hence using the estimates on
+ I;EI
+
~
Ix E - xol]
+ I~CI + Ix C
-
as well as on F
+
CI
xo IJ -
E
(5) -
(I
y(xE)I[l + y(xE)i2
(5) -
lyE -
+
On the other hand
We first pick k such that
c~
k ek(T-t)
•
C:s:
+
We deduce from (5.45), (5.46) that
ek(T-t)
,
k
::; "2
y(xE)I+
ISEI2)
159
which implies in particular
Therefore we can proceed beyond t 1 , and prove finally that (5.50) holds on [t,T). This implies the results desired
fJ
We can deduce from Proposition 5.2 a slight improvement of Theorem 5.1, as follows D .... ".nl''H·+.;'''n
5.3. Under the assumptions of Theorem 5.1. one has
(5 51)
Proof From (5.41) we deduce
Considering
we obtain the inequality (5.52)
161
(6.2)
We are interested in the function (6.3)
(x,y) = Inf JE (v(.}) v(.) x.y
which will appear as the (6.4)
maxim~m
solution of
I nf [D I;) E • f (x ,y, v) + -1 D IfIis • g (x.y, v) + 9, (x ,Y. v) ] v X E: Y
o.
Let us make the following assumptions
(6.5)
(5.6)
f. g.1 are C2 ; the derivatives of f~g of all orders are bounded second derivatives of t are bounded 11.(0,0,0)
;0
0
9,
a
(0.0.0)
=0
(0
the
(x,y,v)).
f(o,o,o) = g(o,o,o) = 0
(6.7)
?:!yI,
y>O
(6.8)
(6.9)
the 2nd derivatives of f,g are bounded by ~ where C is not too 1 + Ix I 0 large compared to y.
For the control problem (6.2) the admissible controls satisfy (6.10)
and defined by (6.3) is the maximum solution of (6.4) among the class of Lipschitz functions satisfying the growth condition given at the right hand side of (5.10). Consider then the limit problem
163
6.2. Expansion
We consider the problem (6.18)
InfCOIjl.f(x,y,v) + 0ll. g (x,y,v) + l(x,y,v)] = 0 v
which is solved as follows. Let us set (6.19)
G(x;Y,~)
=
g(x,~(x) +
Y,G(x) +
~)
(6.20) + q(x).g(x,~(x) + Y,u(x) + ~) +
+ t(x,~(x) + Y,G(x) + ~)
-
H(x,~(x)
,u(x) ,q(x)).
We solve the problem (6.21)
Tnf [F(x;Y,s)
s
DyA.G(x,Y,G)] = 0
+
related to the control problem (6.22)
dY dT = G(x;Y,~) x(x;Y)
inf .;(.)
,Y(O) = Y
r'
F(X;Y(T),';(T))d-r = inf ,:;J(x;Y;';(.)) .
0
~(.)
We can prove as in Proposition 5.1 that (6.23)
with constants independant of x. Furthermore assuming (5.26) we also have, as in Lemma 5.1
165
then one has Proposition 6.1. _ _ _ _ _ __
I XE(S) I ' 1/: (s) I
(6.31)
f:
E
2
Ix (t}1 dt
~
f:
C(
Ixl
+
Iyl)
lyE(t)12 dt
~
C(lxl 2
+
lyl2)
We set
and compute
- ~ (lyE - Y(X£)12
~ COo(lx E 12 C' +
if
+
- E) asd !jl(X E (5)) =E Dql(X ).f(x £( S).y E(S),U{X
+
lyE _ y(xE)12)
6 E (lyE - y(xE:)12 +
On the other hand (6.33)
I~EI2)
ly(x E }1 2 + lu{x E )1 2 )+
(lyE - y{xE)14
+ C~'It:EI2 -
+
+
:::
~~)
I
+
2).
1
167
From the estimates on X, ¢, and arguing as in the proof of Proposition 5.2, we deduce that lyE(sl} }}I p and step by step, it follows that (6.35) holds for any s. From this the results desired obtain
o We can finally give the analogue of Theorem 5.1, namely Theorem 6.1. 14e assume (6.5); (6.6). (6.7). (6.B). (6.9), (5.26). (6.26). Then one has
)I
(6.36)
Proof. We have the relation (6.37)
and
Compared to (6.4). as in Proposition 5.3, yields E
(I' (x,y) -
$
2 E -IS 2 C£ E froa [lxL-(s)1 + Iy (5) - y(x' (s))1 +
+ lyE(s) - y(x£(s))1 3]dS
where xE,y£ refer to the system when controlled by the composite feedback, i.e. (6.30). From the estimates (6.31) follows, (6.38) On the other hand, for an admissible control of the class (6.10), which suffi ces, the corresponding states (also noted xE,yE for brevity) satisfy (6.39)
169
HABETS P. [1], Singular Perturbations in Non linear Systems and Optimal Control, in M. ARDEMA (edition, see above). pp. 103-143 HADDAD A.M., KOKOTOVIC P.V. [lJ, Note on Singular perturbations of linear state regulators. IEEE Trans.Auto.Control, AC-16, 3, pp. 279-281, (1971) HADLOCK C.A. [lJ, Existence and dependence on a parameter of solutions of a non linear two point boundary value problem. J. Diff Equat., 14, (1973), pp. 498-517 KOKOTOVIC P.V. [lJ. Applications of Singular perturbation techniques to control problems, SIAM Review, (1984) KOKOTOVIC P.V., Olt~ALLEY Jr R.E., SANNUTI P.• UJ, Singular perturbations and order reduction in Control theory, an overview Automatica, 12, (1976), pp. 123-132 KOKOTOVIC P.V., SAKSENA V.R .• [1], Singular perturbations in Control theory. Survey 1976. 1982 KOKOTOVIC P. V.• YACKEL R.A .• [lJ. Si ngul al" pel turbati on of linear regul ators, IEEE Trans.Auto.Control. (1972). AC-17. pp. 29-37 1
LIONS J.L. , [1], Quelques m!thodes de r~solution des linAaires. Dunod. Paris (1969)
probl~mes
aux limites non
MINTY G.J. [lJ. Monotone (non linear) operators in Hilbert Spaces, Duke Math-Journal, 29 (1962), pp. 341-346 O'MfdlEY Jr R.E. [1], The singularly perturbed linear state regulator problem, SIAM Cant, 10, (1972), pp. 399-413 [2], Singular perturbations of the time invariant linear state regulator protilem, J. Diff equat. (1972) 12, pp. 117-128
[3J, Boundary layer methods for certain non linear singularly perturbed optimal control problems, J. ~1ath anal. App. 45, (1974), pp. 468-484 [4J, Introduction to Singular Perturbations, Academic
PI~ess,
1974, N.Y.
control. In Mathematical Control 680, Springer, N.Y. O'MALLEY Jr R.E. ,KUNG CF.[llJhe matrix Riccati approach to a singularly perturbed regulator proble~. J. Diff equat. (1974), 17, pp. 413-427 [2J. The singularly perturbed linear state regulator problem, SIAM cant. (1974), 13, pp. 327-337 SAKSENA V.R., O'REILLY J .• KOKOTOVIC P.V., Singular Perturbations and Time-scale methods in Contl'ol Theory. SUI"Vey 1976, 1982 SANNUTI P. [1], Asymptotic Solution of singularly perturbed optimal control problems, Automatica, (1974). 10. pp. 183-194 [2]. Asymptotic expansions of singularly perturbed quasi linear optimal systems, SIAM Cant. (1975). 13, 3 • pp. 572-592
SANNUTI P., KOKOTOVIC P.V., [lJ, Near Optimum design of linear systems by a singular perturbations method, IEEE Trans -Auto-Control (1969). AC-14, pp. 15-22
SINGULAR PERTURBATIONS IN STOCI-IASTIC CONTROL
A. Bensoussan t
G.L. Blankenshl p tt
Abstract: \Ve consider a class of problems in stochastic conl,ral theory Involving stochastic systems with small parameters. Using both analytical and prouablllsUc methods adapLed to Lhe special structures of singularly pcrt.ul'bcd stochastic control problems, we develop a sysl,emal.lc methodology for their analysis.
Introduction In this article we address the following class of
COil trol
problems. \Vc have a system
governed by dx =
fdy
=
f
(x ,Y ,v ) dt
9 (x.y ,v )dt
x (0)
+ J2 rl w + /2itlb
(1 )
x. y (0) = y.
where wand bare independen t; vViener processes. The state x (i) represents the slow system, while the state y (t) represents the fast system. The scaling is such that the variaLions of the fast system
PCI'
unit; of time, in average as well as in variance, are of order
The dynamics are controlled via the parameter
'IJ
(t).
There is full information and
IINRIA, Domain!! de Voluccll.u. Rocquencourt, D.P. 105, 7815:1 LE CHESNAY CEDEX FRANCE. Also with Lite
Uniufrsile' de Paris - Dauphine. The research of this Il.uthor was sUl"portei.! In pnrl by the U,S. Department or Energy. ttElect.rlcal Engineering Department, University of Maryland, College Park. 1\1D ~Oj.j:!. This fl':'lf;rlrch was support.ed In part. by the U.S. Army Hesc!l.rch OUlce and t.he Army Night Vision and Electro-OptIcs Lauomlury ullder contract DAA.G 2!l-S3-C-OO:!8 with Systems Engineering, Tnc., Greenbelt, MD.
173
One can then consider the Bellman equation of ergodic cont;rol relative to (G). It is defined are follows: pick a constant X (constant wil.h respect to y) and a function
Suppose llhat one can 11 nd a pair
x..cP
tP + X = H{y ,Dllf/J)·
(7)
depending parametrically on x
x=
,p ;
hence.
X(x .p ).
If we choose u so that - All
7-
f3u = X(x ,D;r
(8)
11 },
then the pair u ,cP will satisfy (5). Onc can Lhus expect a solution or (8), vanishing on the boundary r of 0 to be the limit; of
1t '.
This procedure depends on the possibiliLy of being able
(,0
solve ergodic contl'ol
problems of the t,ype (7). Tho cont,rol problem iLselr is as follows: Consider dy = G (y
,11
)d T
+
v?idb. y (0)
lim
1
E
= ()
Co)
T
T-+oo
f
D (y , I)
}
if
T
o
then in general
x
In! ( Nil (v (.»)} independent. v (0)
of
y.
The interpretation of tjJ is more delicate (eL sect. 2.G). Pick a feedback
11
(!I) and consider
th e con trolled state ely =G(y,v(y))dr
+
~db.
y(o):::=y.
(10)
It seems inevit,able to requ ire crgodicity of the process y.
This means that as T -
00,
Y (T) behaves like a random variable following a proba-
bUity In;:V(')(y), depending on the choice of v (.) l\,nd of the paramet,er entering into the deOnition or G. Suppose, moreover, t;hat, m is a. probability density WiUl respect t,o Lebesgue measure; it is possible to give another inLerpretalioll or \: as follows:
175
case, while in sections 3 and
the case of reflected dHTusions is treated.
-1
In section 5 we consider a fast system of the form iii
dy =
G (v)y dt
E
+
III
urn 7J db m (t).
Y (0)
!I.
(13)
=1
Defining the norm and the angular velocity
I y (t) I
fl{t )
~(t )
J
y (t )
I y (l ) I
=
then by linearity of (13). eO) is itself a diffusion taldng place on a sphere; hence, ergodicity will hold for e(t}. \I'Ve assume that the slow system depends separat;ely on the norm and the angular velocity of the fast system. namely dx =
I
(x ,p.e,u) dt
-I-
"f2 dw,
x(o) =
(14)
x
and we minimize a cost function of the form
E
Jo ".. - PI
I ( x .p ....t:) ,v dt .
\Ve treat this model in situations where
p -+ 0
as
l
---+
co, and
~
is ergodic. In sec-
tions 6 and 7 we consider (10) in the whole space. To get an invariant probability on the whole space, we follow the theory developed by Khas'minskii [2] in the case without Note that G
troL Fy
+
COll-
cannot be bounded; hence we consider a drift ot' the form
G (y ,v (y)). where F
is a stable matrix. This n.1't.icle covets most cases where a
natural ergodic fast system governs the evolution 01' Lhe sl,atc. There may be many other situatiolls where different techniques of singular perturbations are used. Examples of such situations may be found in the paper of ft. approaches to ergod ic con trol. see
Jensen and P.L. Lions [3j. For other
Hl. Ack now ledgement
The first au thor has bencH ted from useful discussions with H. Brezis, L.C. Evans, and J.L. Lions. The second author would
lil~e
to thank G.C. Papanicolaou and P.V. Koko-
177
(O.A ,F t .Pt), btJ (t) becomes a Wiener process and the process y (l) is the solution of
dy =
(l
+ Vi. db g (t).
{y (t ),v (t)) dt
y (0) = y.
(1.8)
Let us consider the cost function 00
J(lI ll'(v{.))=Elje o
otl(y(t),v(t) dt
(1.Q)
\Ve set
{1.1O} \Ve shall be interested in the behavior of rPo(Y) as
Q'
tends to O.
1.2 The Hamilton-Jacobi-Bellman Equation Let us consider the Hamiltonian H(y,q)=
In( {l{y,v)
+
q.g(y,tJ)}
v E tJ,2r/
J..nt
= ~
I;:
{L(y ,q
(1'.11)
.v)}.
~d
From the assumptions (1.3) and (1.5) there exists a Dorcl map Y
with values In U ad
,
such that
-
lI(y,q)
L(y,q, Y(y.q».
(1.12)
The following is a classical result in stochrusl;ic 'control theory:
Theorem 1.1.
The Junelion tPa
is
the
uniqlle
periodic jlmclion beiongino to
(1.13)
A1oreover,
1/ we
sel v o(Y )
then the process
-i}
179
(l - AG{)m =0.
(1.19)
The number of linearly independent solutions of (1.10) is the same as that of
Le .•
(1.20)
z periodic, ;; E Hl(y). But the solution of (1.20) is more regular,
Z
E WI~it (lRl'I). \::1
p. 2 ~ 1) ~ 00.
The
desired result follows. 0
1.4 Fundamental solution Let us consider the Cauchy problem f}z
gV.Dz = 0
al Z(y,O)
(1.21)
tMy)
where ¢ is Borel bounded. Then there exists the following represen tation formula
z (y ,t) =
f P v (y ,l,'I)¢(11}d 11
(1.22)
Ei
where the funct.ion 1) (y ,t ,11) is the fundamental solution and has the following properties
\10> 0, Ii < T, \1 g, pll(x;.,.) E L a(8,T;H 1(lRI'I}) Cn exp(
-
Q'~
-
where
0'1' QI::J
>
0,
C I' C'J
>
0
(1.23)
I Y -'I I ~ ] l
and they depend only on the bound on g G
•
(1.2-:1)
Tn particular,
they do not depend on the particular feedback v (.). This result is due to D. Aronson [5]. Now note that; if ¢ is periodic, z is period ie, and we can write
z (y ,t ) =
f PD~(Y ,l,1/)¢(11)d'l l'
(1.25)
181
I z(y,l) Taking
11
=
J4>(t})TI(d'l) y
I
~ K
Iltbll
e
p.
[l), we deduce I z(y.t) -
JrfJ('1)TI (dry) 1
<
J( c P 114>11 t!
-pl.
Y
Using the invariant measure m
Tn
11
(1.31 )
defined in (1.IG), we also see easily from
(1.113) and (1.21) that Jz(y,t) m(y) dy = J~6(y) m(y) dy. y
y
(1.32)
Using (1.31) in (1.32), we deduce J
Tn
(y ) dy J cP(77)
y
which proves th at.
n
(d '1) = J ¢(y)
y
Jm (y )dy
0,
'Tn
(y ) t/y
y
since
is not a. e. O. Normalizing the in tegral to be I,
111
r
we see that IT(dy)
=
m (y)dy
and th us {l.31} yields Izt>(y.i) -
Jq,(y}mtl{y)dy I
<
J(
ePllrfJlle- Pt •
y
Now recalling (l.30)
Jy ,p(y )m (y }dy
=
J}' z {y ,1)111 (y )(iy
and from (1.25) and (1.27) {j
Jy ¢(y }dy
~
<
81
~ J ¢(Y)1I1 (y )dy
<
Z
(y ,1)
Jy rfJ(y )dy.
Therefore, 8J q,(y )dy }I"
Hence,
Y
01 J ,p(y )lly. Y
(1.33)
183
JU(y.vo(Y)) - atPo(y))mcx(y)dy
=0,
y
Considering
;];0'
=
Jy ~Q(Y )r/y
we deduce rrom (l.38) that
But rrom Theorem 1.3 we can assert that
where 0. 01 do not depend on
0'.
Therefore, it follows that;
J I D tPa I
:::; C
ely
:l
I r/J" - ¢a I L!lP")
Y
::5
a [ ~ I D ~" I 'dy
1/2
1
By Poincare's inequality we thus obtain the estimate
I D rPo I L:l
:::;
C.
(l.30)
Let us consider
Hence,
-
rPa is bounded in H 1CV) and
Xa is bounded. Note that Jf(y.D
,po} is also bounded
Consider a subsequence such that Xa -
x'
4>0 -
H (y ,D 4>0)
r/J in }[l{Jr) wellkly and a.c. -10
e In
L 2( Y) weakly.
Passing to the limit in (1.13) yields
-
~t/>
+ X=
e.
4> periodic.
Jy 4>dy
o.
(lAO)
,185
Considering m ~ thejnvariant pt'obability corresponding to v:!. we deduce by multiplying
Iy m~ I D (¢1
I ':!dy
- ¢:2) +
= 0
tP'J) + = o.
There(ore,
constant. 0
¢'J
1.0 The-ergodic control problem \Ve can now interpret the pair (X, ¢). \Ve already know
x= =
lim
Ci tl
o(y')
0 ..... 0
11m
Ci
0 .... 0
1111 Ky'Y(v (.».
In fact, one can,be slightly more ,precise. \Ve have
Theorem 1.5. 'Under the assumptions of Theorem
1.4
we have
00
'X = 'in] { 11m 0(0)
Ci
Ej'Ie -
0-0
= Illf{ 11m v (0)
r::;-oo
ot
I (y, (f).v' (t» tit }
(1..12)
0
1
T
1, EoVII(y(t),v(l» lit} 0
'Nloreover, choosing the undetermined constant for ¢ such ,that
J~ry(y )11~ (y)i/y
=,0,
we
y
have T
I
t/>Ur)=Inj {l!ill 'E!f(l(y(t),v{l)) - x)ut 7 .... 00
Proof.
v(.): E,J't/J(y(T»-o}
0
Let us simply prove (1.-13). For any control
11
(0) we have
T
¢(y)
<
EJ'
I( /(y{t),v(t)) o
;Hence, if
E~1J¢(y
(T)) -+ 0, 'then
- X) lit
+
E~!JtP(y(T»
(1..13)
187
= -
b/(t)
(J
tV (/)
(x (s ),g t(s ),v (.5
J
-
»ds
(2..1)
(2.5)
{x (s ),y (.'I ),V (8)) lis
Let us consider a bounded smooth domain 0 of
m
B
,
and
denotes the flrst exit time
r=T;:
of the process x (t) from the domain O. Since we are not going to consider the process x outside 0, we ma,y without loss of generality assume that JJ OJ and I are bounded functions. Let us define the probability pf (which depends also on the control
dp E dP
exp ([ [
1 f'1
+ [
- ~ j [~ 1 9 ·1
0
!
1
(x (s
1
).v (8)) 12 -}-
f
For the system (O.A ,FI·P f),
and (x ,y))
a (x (. l,y ,(a Lv (, )). db (, 1 ]
(x (. l,y ,(, l,v (.)). ,(w (.)
),Y t(1I
11 (.),
1
(1.6)
I J (x (8 ),y «s),v (.5» 12]dS }
the processes b l(t) ilnd
w £(t)
become standard
independen t \Viener processes and the processes x (t ),Y ,(t ) appear ns the solutions of
(2.7) x (0) = x, y,(O) =
]I
Our objective is to minimize the pn.y olT function
.l/l1(11(.»=E'
J
l(x(t),Yt(l),v(t»e-t/t
ilt
(2.8)
o
wh ere /1 > o. Let us set U
,(x ,Y) = InJ J.'.,(v to (a) •• ,
(.»
Then u, is the unique solution of the H.J.B. equation
(2.0)
189
iJ:!
Ir =
av
0, z is periodic in y
and the solu tion is necess(l,rily a constan t. vVe have the following important estimate
Lemma 2.1
lYe have (2.16)
where 5 and 01 are the same as in (1.34). Proof. Let us consider the fundamental solution of the Cauchy problem.
o iJz I r = 0, -8 1.1
. dl em . =. IS perlO
z (x ,V ,0)
where we h ave set 9 c
=
(2.17)
y
1/J(x tY)
9 (x ,y ,v (x ,y ).
'Ve are going to use the probabilistic interpretation of (2.17). Consider process reflected at the boundary of O. Namely,
J2€
iIx =
dU! - Xr(x (t» v d ~(t) x(O) =
where
e is an
x
increasing process. Consider next dy(t) =
V2 db{l),
yeo) = y
and assume wand b are independent. Let us define the pl'oeesE t
bt(t)= bet) -
~
£g((x(S).Y(8).V,(X(s},y(s))) tis
and the change of probability t
dP' IF"
exp{[~
t
g(x(,r;),y(,r;)).db
-
+[
1!1{I~cl.s}
11
\Viener
191
8¢
at -
AtjJ
f
q,(x ,0)
=
=
a~
°
av I r
0,
(~.~1)
Jy 1f;(x ,'I) d t]
But we notice that
JJ
8
III f(X ,1)
JJ
111
Ax
(x
Z
,1)
,1) dxdy
,1)
)1/1(X ,1) dxdy
=
JJ
S
JJ
TIt
111
,(x
1f;(x
,1)
rlxdy
,1)
,(x ,y) '1Nx ,y ) drily
However, from (2.15) we have
-
f.
A;t
J1" m ((x ,y) dy
8
= 0, -a v
Jl'
TTl
,(x ,y )dy
0011
r
hence
Jm
f(X ,1)
dy = constant = 1.
Y
We then deduce from (2.22) 8
Jq,(x ,1) dx o
~
JJ
111
s 8 Jr/J(x ,1) dx
kr,y) 1f;(x ,V) dxdy
1
0
Bu t i'rom (2.21)
J ¢(x ,1) dx
o
J tjJ(x ,0) dx = JJ 1/J(x o
which, used in (2.33). implies the desired properLy
,1)
dxdy
0,."
(~.lG).
D
2.3 A priori estimates \Ve start with
Lemma 2.2
The following estimales hold
2 Here we use the positivity of
m (/
which follows from ergodic theory.
ll.5
III secLloll 1.'1.
(2.23)
193
+
I I Dz rP I ~
+
dx
j3
I
if/ dz
Proof. We multiply (2.25) by m c¢,l and in tegrate. \Ve have
(2.20)
Now mul!;iply (2.15) by
11
L¢ and integrate. Using t,he fact that
tP does not depend
on y, we obtain
II
+ .!.. E
D" m (DII u! tP dxdy =
II
1 f
TIl f
9
(2.30)
D" u ( rP rlxdy.
But (2.31)
- II
m ( ~¢
Tt (
tlxdy.
+
II
Using (2.30). (2.31) in (2.2\)) yields 2
II
Dz u (D z ¢ m (dxdy
III
f
ttl
1::..q)
(2.32)
dIdy
But now
(2.33)
II +
m
I I D::
I
£
t/J
Dz
I2
tt (
dx
I +
2
clxdy j3
I
+ .!.. €
II
m
f
I
D1/
It
£
l!l dxdy + I I
t/J2 dx
Using (2.26) and (2.32) in (2.33). the result (2.28) follows. 0
j3
111 f It (2
dxdy
195
implies
f fm o
I
f
(x ,y ) 1jJ( x ) dx
tf;( x) fix
\;! fjJ ELI.
o
y
Letting e tend to 0 in (2.35), we deduce
+
jJtP,. ) dxdy
I~
u) =
+
2
f3
Letting k tend to
'}.
f +
J Du
D,pl; dx - 2/3 f
,p/:2 dx + 00,
2
f I D"
(h
it
(4J k
-
I
tt)
f
I D"
rPk
f
(,p,.
dx + 2jJ
+
dx
2
f I
2
D",p,.
+ 2jJ f (111:
~ dx
I~
dx
I
dx
-
:l
1I)~ rl.r..
we deduce
Since clearly 1t)
I
2
+ jJo
dxlly
JJ (ll
c -
11)~ dxd y
the strong convergence property is established. 0 "Ve are now going to iden tify the limit. \Ve shall llse the notation or section 1. Let us consider x
E 0, and
tion 2. with
9 (x 'v ,v)
[J
E Un as parameters. \Vc shall consider the SitlHltioll or sec-
instead or 9 (y ,f}) and
I (x ,y • v) -I-
p.J
(x ,y ,v)
instead or
I(x.y,v).
A feedback now is a Borcl function or y, which mas be indexed by xJp. feedback v consider the invariant. probability m ~ (x ,y), which is the solution of - ~v m In
+
dlv g {111 9 (x ,Y
E HI(y'l,
til
,1) ( Y
))) = 0
periodic.
For I,p fixed we can consider the scalar X defined in (1.3G), namely
For any
197 fJ¢c
Ir
(2.,13)
0, ¢( periodlc in y
This problem is studied using ergodic control theory as in section I.
111;
have (recalling that the integral of
Ai
= 111/
lI(.,.j
I0
is
,)
m: (x ,y )
II
In particular, we
(2,-1-1)
F(x,y ,v (x,y)) dxlly.
Consider now for x rrozen the problem - Q.lJ
¢
H(x ,y .DJI
¢ periodic in y. ¢ E \Ve want to study the limit of A( as
Lemma 2.5
~Ve
f
q,) -
(2A5)
A(x)
tv'.!·l' (Y), \-J x
tends to o. We shall need Lhe
have
I0 I
A,""'"
J A( x) rlx
(:3..I6)
o
Proof. Note that A( x )
=
Jn / fI(.)
J 111
II
(x ,11 ) F (x •y , v (y
»tl 11
where tn is donned by (2.36). Hence A E L ~(O). Since we can write
-
-
DII ¢.g (x ,1/ , v) - A{x)
-Q. v ¢=P(x,y.v)
Considering 11''' v,
\VC
deduce
J mil
I Dv ¢ 1:1 ely
=
J 1It II
{F (x .1/ ,II)
A(x ))(rp -
-;j;) dy
r
l'
where
~(x )
=
Jy ¢(x ,y) dV·
Hence,
f
I Dy
4)
l!.l dy $;
CF {x f.
}'
Therefore, Dy IjJ E L2{O X Y). From the eqUation (2.-15) it also roll0\v8 t.hat
199
hence
Letting k
-+ 00,
we obtain A,
I 0 I ::; J A(x)
fix
which with (2.50) completes the proof of the desired result (2.-16), 0 \Ve can now proceed with t,he
Proof of Theorem 2.1 Let. us consider a subsequence of
u[
which converges to u in [JI{O X Y) weakly.
Such subsequences exist, by virtue of Lemma 2.2. 1vloreover, from Lemma 2A, we can
assert that u does not depend on y [wd that the convergence is strong. u
E HoI (0).
Let,p E Coco (0).
,p
~
O.
~roreover,
Consider the relation (2.25) and multiply by
cP m (" ActuaIJy, we have already done the calculation in Lemma 2.3 (c.l'.
(2.20). a.nd
(2,32)). \Vc thus have (c.r. (2.32». 2
JJ Dz
uc
D z rP
111
(chtly
-I-
ff
III ( It <
.6.
q,
drily
(2.51)
Now consider F(x,y.v)=q,(x)(I(x,y,v) -I- Duo! (x,y,v))
which satisfies (2."11). Obviously. (2.52)
Note that A(x)
lIence from Lemma 2.5
X(x ,DIL ) ,p(x )
201
f D: u D;z ¢ tlx + J :::; f J (I (X ,V ,V (X ,y}) + D:z :3
11
.6.t/J dx
+
fJ
f
tl.f (X ,y ,V))
tl
(2.56)
¢ elx
111 V
qJ tlxdy.
Therefore, -
Au
+
fJ u :::;
f (l (x .y ,v (x ,V))
+ D;:
u.J (x ,y
,tJ ))
m v dy
r' and since v is arbitrary, we deduce
- .6. u
+
fJ u :::; X(x ,Du ).
Therefore, u coincides with the solution of (2.38).
By uniqueness the desired result
obtains. D
2.5 Asymptotic expansion The result of Theorem 2.1 does not give any estimate of Lhe rate of convergence. Bu t its advantage is that it requires no particular reglllariLy assumptions. It is possible to give a rate or convergence and to proceed dHferenLly relying on maximum principle type of arguments, when additionaJ smoothness is available. The situation is somewhat similar to what arises in homogenization (c.f. Bensoussan Lions Papanicolaou [GJ). \Ve shall outline the argument, and in particular we shall not try to present a.':;sumptions under which the regularity reqnil'cmen ts are satisfled (see for til at aspect (71). Consider t/J(x ,y) such that
All ¢
+
X(x ,Du ) = H (x .D"
1l
tY
,D" t/J)
(2.57)
t/> periodic in y.
\Ve define u [ by u!=tt
Then
11
l
+tt/J-f-1tc;
(2.58)
satisfies (2.59) -I- H(x
203
V"(x .Du ,y ,DII q,)
v (x ,Y )
(2.63)
is an optimal feedback for the limit problem. In fact, this is the feedback Lo be applied on the real system as a surrogate for u «(x ,y) den ned in (2.13). One can show by techniques similar to those used in previolls paragraphs to obt.aJll Theorem 2.1, t,hat the corresponding cost function will converge as
tends
E
a
to
11
in Hl(O X V).
Note t.hat
unlike the deterministic situation the optimal feedback for the limit problem is not a function of x only.
In fact (2.63) corresponds to the composite feedback ot'
Kokotovic [8] (c.r. also
17\
Chow~
in the deterministic case).
3. Ergodic. Control for Reflected Diffusions 3.1 Assumptions and notation Our objective in this section is to describe another class or crgod ic con t,l'ol problems
and to consider stochastic can trol problems -with singular perturbations wll kh call be associated to them, as in section 2. \Ve shall consider ditTllSiolls with reOeetion. Let B be a smooth bounded doma.in in nd, whose boundary is denoted by DB. Lct g and
I be continuous funct;ions g (y,v):
Ii
X U
->
n. d
I (y .v):
iJ
X U ->
nd
(3.1 )
where U is as in (1.2) and Uall is as in (1.5). Consider (O,A. ,P ,FI ,6 (l)) as in section 1.1. and let y (/) represent t.he dilrllsion process reflected at the boundary of B ely
.J2 db
- XlJn (y (l ))/Ld lJ
yeo) =
(3.2)
y.
where II is t,he outward unit normal at. the boundary of B, and ,,(l) is an increasing process. Admissiblc con('rols are defined as in section 1.1. Lor; us consider next the process be (l) defined in (1.6). and t.he cbange of probabilit.y dotlned in (1.7). For the system
205 the coeJficicn ts are not smooth. These properties have not, been clearly stated in the literature. \Ve shall proceed differently using some ideas at' Y. Kogan [oj. Consider the parabolic problem 8z 8z a; I
IlB
= 0,
~z
-
Z
-
g II.D::
o
(3.7)
4>(1)), ¢ Borel bounded
(y ,0)
and we slla1l define the operator, as in (1.2S), PI/>(y) =
Let us write for r a Borel subset of A...~(r)
(3.8)
z (y ,1).
11
P Xr(x) - P Xr(Y). Y,z
E
B
(3.0)
\Ve have
Lemma 3.1 81t 1l
{ A;:~ (n
I
t}, X
,y ,f'} <
(3.10)
1.
Proof. Suppose that (3.10) does not hold. Then there exists a sequence {
Ilk ,:fk
,YI:
,r k
}
such that
Hence,
Define
Zk
(y
initial data
,t ) to be tIle solution of (3.7) corresponding to 1;lw control Xf't
(y) =
t'l: (.),
and to the
tPJ: (y). \Ve then have (3.11 )
Bu t the classical estimates on parabolic equations yield ZI;
bounded in W 2 •P ,1(B X(5, T)), \-1 8 > 0
207
Therefore, (1.33) also holds. This estimate implies in particular Lhat
In v
;:::
0 and can be
taken to be a probability. To prove the estimate (1.3-1), we rely on the foHowin g Lemma.
Lemma 3.2. The solution oj (3.5) (normalized as n probab£lilY) satisfies (3.17)
wilh a norm bounded in v (.).
Proof. Let us consider the problem o If
l/J E L' (8) then
tP
E
W~,'
(B).
:s .s
1
<
00.
Moreover, we have
Jm tP dy = J111 ,pdy. B
(3.18)
B
We deduce, since m is a probability,
:s
I Jm ~,dy I B
I r/; I o"(li)
< ClltPll w''2.' •
with
-
-< C 1
Therefore, mEL '., with
1
1 /1
(3.17) is proved.
Suppose d
3,
I
()
'
8'
a
d
>
2
IV)I L" " . 1>_1_>1_2.. It' d
then we have
111
1,2.
d
&/}
E L",
1:S
So
<
the result.
3. \Ve
proceed
using (3.18) together with a boot strapping argument. \Ve Ilave
with
1
~
1
< -;-;-:- +
2 d;
and (3.17) is proved, If tl =
h
1
1
ence, ---;:: > ~ > '1, . . . ,
1 8"
we proceed in the
finite number of steps suffice to imply (3.17).
2
J'
Since tl
SLime
3.
S I
is i.1l'bil,rary
way. For a.ny value of d, a
209 If h > k >
0,
then A (h )
c
A (k). and we have
<
k} (Aleas A (h ))'
{ll
ff
::; !J(k ){m - k}"
dy l'
It
and from (3.20) it follows that
(It
-
k) (Aleas .A (11
»'
J..
< elm I L'
(Afeas A (~.))!!
or G I m Il: (J.. _ J..),' , Afeas A (h) ::; - - - - (Mens A (k» ~ (Il - k)'
Pick s > d. then 1f;(t). ko ::; t
<
00
(2.. 2
.!..)s· > s
1.
\Ve use the following result from
[llJ (p. 63): Let
be nonnegative and non increasing, slIch that (3.21)
where C ,01, and f3 are positive constants with f3 >
1.
Then (3.22)
where (3.23)
It is clear that this result applies, and thus A1eas A (k) =
k
=
G 1111
I L'
I
0
where
J..
(3.2-1)
(Aleas B) d
The second estimate is thus proved. The proof of the second estimate is more involved. \Ve refer t,O [7].
Remark 3.1. The fUllction result in Lions -
~!lagenes
1)1
l12J
v
E
WI,IJ (B).
\1 p E
(I.OO).
T'his follows from a general
Teo. 6.1, p. 33. Indeed, we write (3.5) as follows
211 i)z at i)z -a IBB II
with tP E L l(B}. tP
~ O.
9~
.6.z =
0,
Z
.Dz
=
(y ,0)
(3.27)
0
4>(y}
Let us assume that (3.28)
where
Co
does nOL depend
011
tP, nor on
v (.).
v.
vVc then have
Proposition 3.1. The following estimate holds
where c is i'ndependenl of ¢. v (.). and y.
Proof. ''\Tc shall prove that lu! {:(y.l) ly,l)(·).I/I~O, ItPlLI
l,pk ILl =
if this is false. there exists a sequence I/Ik ~ 0, jng
=k
the solution of (3.27) corresponding- to 1/11:.
Vk (.).
I}
(3.30)
1. Vk.
tIl: (.)
such that, denot-
then one has (3.31 )
\Vriting i)ZI:
~
O.
I.IJI
lOB
=
O.
=1;
(y ,0)
tPJ: (y)
and making use of (3.28), we can asserL Lhat zdy ,I) is bOll nded in L =(fl X(8. T
then
ZJ:
remains bounded in
w~,llr (B
X (8,1')). \-f p.
2:::;
[J
<
n.
Bu L
00.
Reasoning as in Lemma 3.2, we identify a limit fundion z' such that
az •
-.6.z
•
•
- 9 ".Dz
a•
~1/lB ap'
•=
0,
0
and as a conseqnence of (3.31) we have z'(y' .1)
O.
t
E [ti.T
1
(3.32)
213
To prove (3.34), we consider (~
,t )
r (y
t}q(y,t)
2
which is the solution of
aT
-
Ar
+
r g oil
dlv (rg )
I DB
=
q
(3.35)
= 0
r (y ,.!..) = 0 2
and we know a pr£ori thilt (3.3G)
The result of Proposition
3.~
follows from the following result of LadyzenskilYil, SOIOIllli-
kov, and Ural'tseva [10].
Lemma 3.3. The solution r oj (9,95) belongs to L =(B X (O,..!:..)). 2
Proof. The argument is in the spirit of Theorem 3.1 or cnce we take T instead of
2-, 2
[111. For notational conveniE L 00(0.1': L 'I-.(B )). by
and assume q ;::: o. NOLe thnt r
the definition of r. Let k be a constant and
vVe easily deduce from (3.5) that t
..!:..
I t](t ) I ~ + f I D '] I ~
2
0
ds
= f ds f(l·g =
f(lSf o
whel"c X E L 00(0, T ; L >-(B )}.
Let us introduce the following nOl'ms
+ q)
n
0
n
X
'I
tlx
t]
elx
(3.37)
215
(3.40)
2::
For k
1
we can also write T
III til II <
Ok
It then follows from the theorem in
[
)..
[ dt CAfe-as ilk (t
1 II
I,
»~p
]
[101 loc. cil., p. 102, ('I'heorem 6.1) that if we can
write ).. -
1
---p' 8' ).. with
_1_ ~
>
2.. , then r
r
1
=q
r
+
fJ is bounded. Expressing.!:.. = q
1
d ).. - 1
-+ :2 )..
L, ,~
d
d
2q
-!
)..
1 r
• then
tl -r.
(3.·11)
·1
Now from (3.37) it follows easily that
hence
[
1
+
d ·1
since).. > .!!:.. . 0 2
4. Singular Perturbations with Reflected Diffusion 4.1 Assumptions and notation One can apply the ergodic theory of Theorem 3.2 to solve some problems oj' singular perturbations in a similar way as in section 2. LeL us consider fUnctions
I . !1.
and It cOIJLinuOlIs
217
One can also choose a Borel runction v (x ,y) such that (4.8)
4.2 Convergence Now the same theory as the one developed from section 2.2 to 2,4 can be carried over to study the limIt· of' ('-r.7). \\Fe just state the result. For x ,p parameters we solve the ergodic problem of the type (3.213)
q, +
All
X(x.p)
=
a¢ ap
H(x.p .y .DJI t/J), - l o B
=
0
(-1.0 )
and the limit problem is given by - A u
+ f3
u
=
X(x .Du).
1L
r
=
0,
Tt
E
W~·P (0)
(-1.10)
Theorem 4.1. Assume ({. l)(f. 2). Then one has 11 ( - -
(-1, ll)
u in 1[1(0 X B) strongly
The same considerations as in sectlons '2.5 and 2.6 carryover to this case.
5. Singular perturbations in the case of a linear fast system 5.1 Study of a linear system Let us consider the following linear system dy
=
G(v)y
tit +
r
'E
CI"r
Y dbr(t), yeO)
= y.
(5.1)
r=l
where G (V): Ua.d -
Uaa
L (IRa; lR d
):
is a continuous bounded function
compact subset of a met,ric space U.
(5.2)
(5,3)
br (t ) independent standard scalar \Viener processes on (O.A ,P .Ft ) An admissible control is a process v(l) which is adapced to FI and takes values in UtJd
•
219
Hence,
It is convenient to introduce the vector
-~( t ) ...
( - sinO )
=
(5.0)
cosO
which is orthogonal to e(t). We deduce
~ (€·O'r
[e.g e
dO(t) =
-
e) (e·O'r
e) ]
at
(5.]0)
We shall assume that
E
-
(e·O'r e)~ ~ cr
>
0,
\1 O.
(5.11)
Therefore, O(t) is a non degenerate diffusion, which is periodic with period 211".
Remark 5.1 When d >
2.
we shall have a local representation
Consider
then d €( t ) = D
r
d0 +
.!.. D 2r 2
dBd0
hence dO = (D r" D rr L Dr" d
Note that the relation
I reo) I!! =
1,
e-
..!... (D r" D rr 1 Dr· D 2r cl 0 dO. 2
implies
Dr'" r =
0, i.e.,
Dr"
e= o.
We finally obtain
(5.12)
In the case (5.10) we have
-e =
Dr and Dr" Dr =
1,
which implies Dr'
n'2 r
= o. The
221
- -?; E IE, -
(ir
€( I :l] ell +
p£
JE
€c (ir €(
db r
r
(5.10)
Let
T(
be the first exit time of x (t) from 0 ; we consider the cost functional
/:~p,O (v (.)) =
E
J e -/31
l(x({i ),Pf(t ),0(1 ),v (t)
(5.20)
tit
o
and we set (5.21 )
Remark 5.2 The assumption on the derivatives of ]; G, and I allows us to consider a. strong formulation of the state equations. Consider now an ergodic control problem related to (I, according to the theory developed in section 1. Let us write F(x ,1) ,O,v)
I (x ,0,0,1/)
+
(5.22)
l'.f (x ,0,O,t})
where X,p are parameters. Let us also define (5.23)
+E
a (0)
...
III f
H (x ,p ,O,q ) = II
(€
rir
e)2
r
[F (x .p ,O.v) + q.g (x ,O,v ) ]
E uc:t
(here q is a scalar). A feedback is a Borel function v (0) with values in U lJd
invariant probability
111
II
==
111
11
(x
a~
T'o any
11 (.)
we associate the
,0), which is the solution of
- " (a (0) m to)
ao-
•
a + -ao
(IT!
v
g)
0
(5.26)
m periodIc.
Since we are considering a one dimensional problem, we have in fact an explicit I'ormula of the form
223
(5.34) He
Ir
0,
1l
(periodic In 0
where we have set H(x ,p JJ,q ,p,A) =
[I (x ,p,O,v)
In f II
E fl.1l
+ q 9 (x
,O,v)
+
+
p.f (x ,p,O,v)
A k (x ,O,lI)
(5.35)
1
and clearly the function defined in (5.35) coincides with H (x ,1' ,O,q ,0,0).
Let us set 'It
((x ,0) =
'It
Ax ,0,0)
which is the solution of -
1
DUe
H (x .Du (JO, -; ;)0,0,0)
-
u, I r = 0,
11 {
periodic in
(5.30)
O.
Let us also define 'It
!ic(x ,p,O) =
r(x ,p,O) -
11
,0)
(5.30)
--------
p
Then we have the following
Lemma 5.1 Assume
'Y
stl.ff£ciently large, then one has
1S"(e- JlP I L 2::;
I a~c Proof.
The function
e -
.~
C VE.,
~p I L::! ::;
c
a~~ PP IpapeIL::!::;c vt
Je. I D )f e - ~p I L!! ::;
(5.38)
c.
rl satisfies
(5.30)
-
1
- H (x ,Du (00.-; Therefore, for a convenient feedbaclc v r(x .P.O)
au (
;)O,O,O) ].
225 k -
e+
E·G
t: I
1 -;:;-E ! a r EI'.:! -
r
For I sufficiently large we can then deduce that
I ~f + I L ~
a~ +
:::;
a.;e. I P --/;p ILl!:::; C .;e
a~ +
I +0 I :::; a JE,
ID
~c +
I :::;
(5..11)
G .
Similarly, we can pick a feedbacl( for which the reverse or (5040) holds. "Multiplying by ~c -
e-'.:!I1P
and making similar calculations, we obtain the sa.me estimates as (SAl) for
~f - .
The desired result follows. 0 \Ve can then state the following
Theorem 5.1. Assume (5.1.1), (5.15), (5.16), (5.17) and (5.81) wilh I sllfficiently large. Then we have (Ul(X
,p,O) -
'U
(x)) e
-I'P __
0 in
Ll!
(5 A::!)
where Jl > 0 £s arb ilrary.
Proof. From Theorem 2.1, we have
kr
11
,0) -
u (x) -- 0 in Hl.
This result combined
with the estimates of Lemma 5.1 implies the desired result (SA::!).
Note that JI is arbi-
trary > o. 0
5.3 The smooth case \Ve shall make some formal calculations in the smooth case and derive estimates on the rate of convergence. \Ve define tPo{x ,0),
-
Al!
tPl(X ,0) as follows: +
[)~tPo
(3u
- a
a2,p,
- a alP
[)4>o
-I'l-
[)o-
- c
= H (x .Du ,0, -;-0 ,0,0). u
atPl
eo =
tPo
at/Jo
H /X ,Drl ,0, DO
.0,0)
periodic
(5..13)
(5.-1-1)
227
we have
I
u ~(x ,p,O) = E
e - PI Xl(X c(t), o[(t). Pe{t» dt
a
-
+ E u ~(x f(T!)JO~(Tc)'
Pl(Tl )) e
- (Jr C
and from (5.46), (5.48) it follows that
I uc(x,p,O) I ::5
O(e
+
fP
+ E Ie-PI
p,(tfl[l).
o
Using the expression for Pl' and assuming that
we can conclude that
which implies It l
(x ,p,O)
which justifies the convergence of
1L
l
to
I ::5
0 (e +
~
(J
+
e p~)
(5..10)
11.
O. Ergodic Control for diffusions in the whole space 0.1 Assumptions - Notation Let us consider basically the situation of section 1, except Lhat now Lhe periodicity is left out.
Of course, additional assumptions are necessary to recover the ergodicity.
\Ve consider 9 (y ,v): lR d X U _
lR d
I (y ,v): IntI X U
lR d
-J.
(0.1)
continuous and bounded Uad compact subset of U a met,ric space.
(6.2)
For a given feedback v (y), which is a Borel function with values in [lad, we shall solve in a weak sense the stochastic differential equation.
229
In genera1 one can try to flnd 1/1 of the fOfm
=
t/J{y)
+
LogQ (y)
k
(6.0)
where
~ Afy • y +
Q (y)
fit •
2
Y
+
P
(6.10)
/\1 symmetric positive definite Q :;::: 0; D is a region containing lhe zeros of Q. The following condition must be achieved to get (6.8)
1
2
~ for
a convenient
11.£ = I. m
=
- tJ'M
~My'J -I- my }
+
(Fy
+
g (y
,1) )) •
(My
+
III)
(G.Il)
p
},fy'1
choice of /\1,
+
my
111,
+
and
p. \1 y
E 117(d -
D
For instance if d
p.
2,
we can
take
0, andp = 0 and (G.D) is saliisfied provided that for instance
F < -
(_2:.. 2
6) I
(6.12)
and D is a neighborhood or 0 suH'icicntly big. 'Ve now follow Khas'mins)rii 12] to introduce a compact; set with an ergodic Ivlarlcov chain as in sect,ion 1..1. \Ve mal,-c the I'ollowing assumption: There exists a bounded smooth domain D fwd a function Ij. is continuolls and 10cal1y bounded on lR d
A 1/1 - 9 (y IV) D 1jJ
I/J> 0< 1/1 -
co as
-
?:.
~ 0
which
D and
1,
Iy I -
'v1.
V.
Y E IR J
co and
I D'I/; I. ~ I/J
D
(6.8)
hounded
In general, one can try to find 1jJ of the form
VJ( Y ) = log Q (y)
where
+ k
(6.0)
231
I
,
T 1- T
:1' ••••
such that 0
To
In! {t >
T 11+1
=
, T
In! {t ~
u ~ 1
fl
Tn
I
y (t)
ED}.
n
~ 0
The process y (t) in the brackets is the process defined by (G.3), i.e. with initial condition y. Let us set Yn = Y (Tn
).
n ~
1.
Then Y n E PI and is a 1vlarkov chain with tmn-
sition probability defined by {B. IS)
\Ve define the following operator on Borel bounded functions on PI (B.IO)
We can give an analytic formula as follows. Consider the problem A ~ - g (y ,v (y)) • D ~ = 0 in D I •
d
I\
ifJ.
(6.20)
We first note that Eti= ifJ(y;: (O(a: »)) = Ev;: dy.l (0' (a:)
therefore taking account of (6.17) , we have P ¢(a:) = 17(a:)
(6.21)
where 17 denotes the solution of (6.16) corresponding to the boundary condition It Of course, in (6.21) x
E
rl
=~.
are the only relevant points. \'\'e then have
Lemma 6.1. The operator P is ergodic. Proof. \Ve proceed as in the proof of Lemma 3.1. Indeed, defining
\-f a:.y E r
l•
B Borel subset of P 1
everything amounts to showing that sup
U
,I.l/,e
>";:~(B)
<
L
(6.22)
233
From ergodic theory, it follows (c.f. (1.30))
I where
J(,
pn r/J(y) -
J r/J(11) n(d cr) I ::; J( 11 q, II e -P'"
E 1\
x
(6.25)
'.
p are uniform with respect to the feedback control v ( • ), and iT =
Jr"
denotes
the invariant probability on r l •
It follows that, since
we can write
I
Evll~(Y(TR)) -
J t/J(I/)iT(dcr) I
'.
We can then define a probability on IR. d
,
II
~ J(
II
t/J
e -PRo
(6.20)
by the formula 8(.,)
J [Evil J A(y,,(t)) dt J A(y) d p(y) =
'.
I iT(d cr)
0
----------
J E/' O(JJ)1i(d £1")
Rd
(6.27)
r. \-1 A Borel bounded In lR d • Following Khas'minskii, one can then prove that the invariant probability is unique, has a density with respect to Lebesgue mCU.'SHre, denoted by m =
m
Ii
which is the solution
of A • m
+
dlv (mg ") =
J m (y)
dy =
0,
III
>
0,
(6.28)
1.
(Rd
where
A -
- ~
+
div (Fy .).
Consider now the Cauchy problem
8z
at +
A z - 9 v Dz z (y ,0) = t/J(y )
=
0
(G.2{l)
235 Hence, we deduce from (0.34) that 1)
f
I DIt I 2 tly
v~
ltd I)
-
+
1)
- ('2p P
Jv
2
J 11
Dv • Dh dy -
(2p
I)
-
nd [
-
nil
Il t
-
f
v ( .:!:..
nd
t::..h
+
- 1 tl" F - Fy • DII
'2p
DIJ
g
Dh ) tly
v 9
P
-
'2 p g •
Dit
1 tly
o.
2p
Simplifying, one obtains
J I Dv I
--:.--..,..-_1
+
Jv
2 [
-
ht
+
_P_ _ l t::..h
+
2p
P
JR4
2.
-
1
-
dy -
nd
P
p
1
Jv
nd
I Dh I ~ -
tr F - Fy • Dh
2p
We now fix the function h, setting h(y.l)={ -
Q(t)(y - r(i))!! - pel)]
and choosing the fUnctions Q(t), r(t) and p(t) in order that ht
+
t::..h
'2
I Dh 12 +
Fy • Dh = o.
Performing the calculations we obtain F Q =0
.
F r
r
-
p
+
=0
'2 tr Q
o.
vVe take
Q (1) It follows that
I,
T
(0) =
E arbitrary,
pea)
o.
Q (t) is the solution of a Riccati cquaLion and satisfies
\¥e deduce from (6.35)
Dv • y tly (6.35)
!1 Dh J dy
o
237
I Let us set .!L = 1 2
+
O!,
v
+
2,2 ::; C
I
then we can assert tha,t
• I •. , ::; , \Ve tihen take p = (1
I
a)k,
P (
ClI
I"
1 +a+ q ,r
1 ]
k = 1,2 ... and note
From (0.30) we deduce the induction relation +
a 1
+l)k~1.
(6.-10)
Note that by (6.30)
,pI ::;
C
I II I
u c II
I I + a III <
C
(although we cannot assert that 4>0 is ft nUe). From Co (1
+
0')
a Lemma in
?:
[10J
'
p.
05,
we
deduce
from
(G.-1O)
t,ha.t (assuming
I),
with
Therefore,
(0.-11)
Since
we get
239
+
2
J
I
~
IP
J I DIS' I I ~ dy
_-2-!.P~(2.!-P _ _l~)
d
-dtE:;.p(l)
p~
I
g D
~
IP
p
Rd
dy
+
Ra
J
I
S' 1 2P (-2k 0 P -
tr F) dy.
Rd
Choosing koso that 2
k0
+ II iJ II -
2
;::::
tr
F
we deduce 2p
-
1
[J
JR.'
P
I DIS' I P I ~
tty
+
I I S' l!lp
dy
j.
(BAS)
R4
Using the in terpolation inequality
I u I;:! ~ applied with
IJ
I~IP ,
0
Ilu I
Iu I
I
we get
hence, from (6.'15) the inequality --'---- (E,l (t )) - d"' p
or also
(6,46)
Applying (6,46) with k = I, yields (taking account of (6.44))
and integrating, one obtains
Following Besala [14], we deduce by induction that
241
From results on the Dirichlet problem, it follows that p, 1
<
p
<
In particular,
00.
m" (y)
Tn
2::
0 belongs to
111
ok
>
\1 11 E
0,
J(.
11 ( • ).
\Ve also shall consider the fo1lowing approximation to mR
nonempty. Otherwise
0
= 0, which is impossible.
radius R, centered at o. Let us consider
(5.40)
compact
Remark 6.2. The assumption ((1.8) cannot be made without D
fin ay
(JR."). \7
0 is continuous. Therefore, we deduce that
where the constant Ok does not depend· on
(6.8) and (6.28) yield
WI,II
'/Il.
Let Bil be the baH of
defined by (6.50)
in which A is sufficiently large so that
e. g 0
+
(A
+ ~
tr F }O'.l
2:
I eI 2 +
C(
02 )
\1 ~ E lR d , 0 E IR
'Moreover,
TR
(y ) = r( ~} where
r( y ) Is smooLh T( 11 ) = 0 for
r( 11 ) = 1, for
I y I :c:; 2"1
I 11 I 2:
and 0 ~
I,
~ L
T
\Ve have
Lemma 6.3. m
R
-
strongly and m
the exlension of mil fU~,
q
by 0 outside B n
,
converges to m
converges monotonically increasing to m.
Proof. \Ve compute A -
(TR m)
+
dlv (1'R
+ D Til hDnce
111
•
g
g) = 111
-
~TR
+ D Tn
•
tTl
Fy
-
111
2 D 1)~
•
Dm
In JIl{lR
d)
243 f(mR
-11I)+g.D(mR
-m)+£ly
DR
+
(1
+ ~tr F) 2
+
A
J (m
-
f
- m) + ~ tly
(mn
DR
Tn m) (mn
-
111)
+
dy = O.
DR
Hence, (rna
m) + = 0 in
Ea.
In a. similar way we have
Indeed multiplying by mn- and integrating yields
J I DmR- I 2dy
+
DR
f
mn- g • Dmn- dy
Dn
Hence, ma - = o. Let us next prove that
mq1 ~
mil!!
if q 1 ~ q!.l'
- m q ., -
Multiplying by
(1IIq 1
pletes the proof. 0
o.
18B 2'•1
mq(!)+ and integrating, we deduce
(111
91
-
m q2 )+
0,
which com-
245
Lemma 6.5.
lVe have D
(6.58)
where the constant does not depend on a, nor y. Proof. We write (13.54) as follows A 4t0! - D
I ,paCy)
cPa • 9 (y ,v 0:)
J cPo(71) il'o(d (1) I ::;
(5.50)
= T....{y).
G1/-O( y ) in IR d { C inD
-
D
(6.58)
PI
with
From Lemma 6.-1. we have (6.60) :Moreover, clea.rly (5.61) Using the sequence of stopping times
Tn
defined in
(O.~),
we can write
f'N
Eel/a ¢,)(y
(TN))
-
tPo(Y) = E.Yer
J lo(Y (l ») tit.
(6.£32)
Now from (6.26)
EvYr:. tPo(Y
TN)
-"
J ,po:(rJ) il'o.(il a) as N
-
00.
rl
(0.03)
Let us prove th at rN
Ev"a JTa(y) til o
Indeed
I ::;
C' 11'(y} \-! a, \-1 N. \-! y E IR.'f - D .
(6.0-1)
247
r" + 1
N
J
E1a
:E n=]
To{Y) tIt
j
I :::;
0
:::; 0
1
independent; of 11 • N.
1"1'1
Moreover, 1"1
I
E!CI JTcr(y) tIt o
I
E!a O(y)
I
~ 0 t/;{y)
which implies (6,64) and the desired result. 0
Proof of Theorem 6.1.
Existence Let us set tPo = tPo
J tPo{I1) 1rCl(J u).
Then
Al
II ~I~ IlL"" ~ '0.
Moreover, from (6.6)
If'
we also have (6.68) in which Xa
J tPQ(ll)1ra(d17).
C¥
At
It readily follows from (6.68) that
bounded in W!!·" .11(IIl d ).
2 ~ p :::; 00, JL
>
O.
\Ve can extract a subsequence such that XD' -+ X
tPOi -+ cP in W 2 ,p 'Jl(IR Ii) weakly.
We can assert that tPQ' D t/!OI
-+
cPo D rP pointwise.
hence,
H (y ,D cPo) -- H (y .D ifJ) pointwise, Noting that H(y.D¢a) is bounded in LP.JJ, we can pass to the limit in (0 (iR), and the
249
J a4>o(Y) mtl(y) (ly
J I(y,v(y)) mfl{y) dy.
~
Hence. as a tends to 0, x ~
and since
11 ( • )
J I(y,v(y)) mil
(y) dy
is arbitrary X ~ X'. Therefore, (B.5\)) is proved .
.
Let v be a feedback associated to tP. where 4> is any solu tion of (6.53). Let us show that
(6.73)
Indeed call X the right hand side of (6.50). \Ve have A 4> - (} v
..
•
D 4> + X - X
=
-
f (y ,v) -
-
and Ji(y)
m~ (y) dy = o.
\Ve deduce as in (6.62)
r_
= -
J
Ef
l(y(t))tlt
Q
bence,
-
(X - X ) E;!'
TN
bounded in N.
However.
E;!'
TN -
+
00
as N -
since
= E; Z (Y:r (II and
(x))
I
:z
00,
X
-
= I (y)
251
D (rpt -
interchanging
,pl
4/2 ) +
o.
and ,p':l, leads ftnally to D (,p1 - ¢'2)
0,
and this completes the proof of
the uniqueness. 0
7. Singular perturbations with diffusions in the whole space 7.1 Setting of the problem Again we basically consider the setting of section 2, in which we shall drop the aBsurnptions of periodicity as far as the fast system is concerned. We consider / (x ,y ,v) : JRfl X n 4 X U _ 11111
U (x ,Y ,v) :
nil x ill:' X U _
I (x ,y ,v) : nil X n
J
X U -
(7.1)
JRd JR
continuous bounded
Uad compact of U (metrIc space).
On a convenient set (O,A ,Ft .PC) (c.f.
(7.2)
section 2.1), we deftne a dynamic system,
composed of a slow and a fast system described by the equations (2.7), with g replaced by Fy
+
g (x.y.v). The cost function is defined by (2.8). and we are interested in the
behavior of the value function u (x ,y). It is given as the solution of the H.J.B. equation (noting
Ay
= -
~y
-
Fy • D) A:t tiE -
.!.A f
II
U
It,
Uf
+ f3
tl ,
c = 0 for x
= If (x
,D,z
tI
,.y. -1 f
E r,
Dv u ,)
(7.3)
\1 y
E W 2,P,P(O X n d ). 2~p
<00
By W 2 •r ,1l{O X ni) we mean in fact, (since 0 is bounded) the set of functions z such
that z "p(y) belongs to W 2 ,p (0 X IR d ). We shall denote by v ~(x ,y) the optimal feedback attached to (2.3), as defined in (2.13). The assumption (6.8) is replaced by
253 and we have (cor. (6.15)l
if
E
(x ,V) ~ 1jJ(y).
Define the sequence of stopping times l\.1arkov chain Xn = X (Til)' 1'"11
]I
To =
O.
1"f1 ,
T n'I-1 as in section 6.2. and the
(Tn) which is a Markov chain on 0 X "'11-
define the linear opemtor on Borel bounded functions on 0 X
p ( ~(x ,y) = E:tJ
fjJ(x (O).y
II
vVe then
by the rcla.tion
(O».
(7.7)
\Ve deduce the analytic formula (c.r. (6.21))
(7.S) where -
e.Ll:f 11
+
f1 -
f
9G
All '1 -
~~ ~
•
DlI 11 = 0, on 0 X (lll"
D)
(7.0)
8'1 1'1= f, -1[,=0 8v
+ Av
!: -
gv
•
DII ~
0 on
0 X DI
(7.10)
o The ergodicity of P' is proved like that of P (c.l'. Lemma 6.1). Let lI"C{clx ,ll 0') be
the corresponding invariant probability on 0 X II. \Ve then define the probability JJ'(dx ,ely) on
a x
IRd
by the formula
fI
II
A(x ,y ) d J'(X ,v )
8(~.!,)
[E/rJ
07l
I
A{x (l ),y (t))
tit 111"!(rJ e.d 11)
0
JI E/"
°R d
(7.11) O(e,ll) lI"!(d e,d ")
0'1
for any A Borel bounded on
a
X JRd
Let us note that we can also give an analytic formula for the quantity 8{=.lJ ) aC(x ,'9)
Ev:flJ
f
o
namely ~ereE
E,;Z 'V for short.
A(x (t ).y (t») ill
255
Lemma 7.1. Let Bp be the ball of radius pin IRd
J
lip
and
IRd
Bp_ Then {7.I8}
where 6(p)
-J.
0
as p
-+ 00.
Proof. Consider in (7.12), A(x ,y)
Ii" n
Dt
rjJ, we deduce that" -
f~;:
+
a
0
a C is a solution of
and thus a
All ()( -
gv •
D)
Dy a =
oa ov Ir
0,
Let us consider the solu tion
Assuming p sufficiently large so that
XBp(Y).
(7.H)
o.
It
Av u - ko 1 DJI u I = Xii p
u I..,
(7.20)
o.
\Ve have
(7.21)
since clearly - c ~;: (a - 'U) g fI
+
Du - ko (0' -
I
All (0:' - u) - g" • D (0:' - u)
Du
1
u) I.., = 0,
~ 0 In 0 X (n d
a(~~U)
1r
D), 0
Note also that
since
All (u -
Du
t/J) - ko 1
Du
(u
As p tends to
+
00,
(Du
- D1jJ) ~ ainIrr d -
1
-1/;) 11'~o.
up remains bounded in W::l,1' .11(Irr d
-
D).
D
257 meR -+
m
t
in LIn fli as R -+!Xl.
(7.2G)
7.3 A priori estimate \Ve shall need the approximation of
given by
tic
(7.27)
=
'U,
0
on 0(0 X B R
)
and U til
-+ 1£
(in 'VI~c"
weakly and In L 00 wenl.;: sLat'
(7.28)
where loc is meant only for trhe y variable. \Ve shall need also a similar approximation in the case of' explicit feedbacks; in particular v
f
Lemma 7.1. The following eslim,ates hold
I
D;r
U
f
I ll~c < c.
I
11 (
!L
00
:::;
C
(7.20)
Proof. Using the feed back v E' equation (7.3) reads -
~;r
1
'U£
-
-;AlI
11£
+
+ ];..r; D11
Similarljr, define
-
.6.%
11 eR
11
l
•
g (x.Y."
l
)
corresponding to (7,30) (7.31)
U (R
U (
Consider similarly me and meR' obtains
(7.30)
(ju f
= 0 on a(O X Bn ).
:t-.1ultiplying (7.31) by nt,R
tlcR
and int,egrnt.ing. onc
259
7.4 Convergence We have
Lemma 7.3. Let us consider a subsequence of 11! such that tL £
--
11
in 1I,~c (0 X lR d
Then u is a function oj x only, belongs to
[J 01 (0),
)
(7.37)
weakly.
and fhe convergence (7.37) i,') strong.
Proof. Setting u)
I:!
+
dx dy
1
e
I I m ( I VII
11 [
I!! dx
dy
and making use of Lemma 7.1, one can prove as in Lemma 2.-1. thaL lim
Xc
0
' .... 0
and using the eSLimates (7.15) \Ve deduce
I oxK I I D"Z
(tL, - u)
I:! dx
fly
+
1
e
I I I Dy
11
oxK
U)2 dx tly --
f
I:!
r/;r: ely
+
0,
ror any J( compact subset of IR. d • This proves the desired result. 0
We now identify the limit. Let us recall the clefinit,ion or
x(x ,p ) by
III v
given in (7.5). Define
the formula
x(x,p)=Inf P
(.J
I
m,,(%,vl(l(x,y,v(y))
+
p ·f(:c,y,u(y)})dy
IRd
(7.38)
and consider the Dirichlet problem -~lj
+f31t
xCx ,Du).
11
Ir =
0,
tt
E
tV!!·I'
(0)
(7.30)
\Ve can then state the following
Theorem 7.1. We assume (7.1), (7.2) and (7.16). Then we have (7.40)
Part II: LARGE SCALE SYSTEMS
SINGULAR PERTURBATION OF MARKOV CHAINS
F. Delebecque, O. Muron, J.P. Quadrat
INRIA Domaine de Voluceau
B. P. 105 78150 LE CHESNAY, FRANCE
ABSTRACT.
This
p8.per
f1arkov chains. and
shaH how
SC;:J.les.
studies
these chains arise
In the second part
chai ns to
some aspects of perturbation theory applip.d
to
In the first part we introduce the notion of agregated chain
I
in the cont8xt of perturbation and
time
we study some applications of perturbed t1ar\
the ReI iabil i ty of large scale repairable systems. In the third
part we give some applications to optimal control.
265 Properties of A
Clearly A1
0
i. e.
the zero
off-diagonal
elements
of
A
are
non
1s an eigenvalue of A with eigenvector
negative. 1.
Also
In general
however there are several eigenvectors associated wi th the zero eigenvalue of A.
After reordering the elements of E one can get the following figure
for A
A state x 1s either recurrent
(x
€
R) or transient (x
€
T). I f x
is recurrent we will denote by x its recurrent class (independent block in the preceding figure),
We will denote by m
the number of such blocks. The events Ax
1
is absorbed 1n the class th~
x"
function defined by q-(x)
These
x
m,
funct ions
are
eigenvalue of A. Aq; ~ O. ~
: ItX n make a partition of E. We will denote by q'X
=
P
right
X•
. Clearly
eigenvectors
associated
wi th
the
zero
267 1
P.
For
a
1
2 in'
D.1
2h
1
Markov
RO)dA
f
Residue of R(A) at A
(Ai)
J (Ai)
}"R(}..)dA - Ai P i
generator
A we
have
L. 1
A, }P .•
(A
1
Do
1
O.
This means
that
the
eigenvalue 0 of A is diagonalizable (no Jordan blocks).
Moreover one knows an explicit full rank factorisation of P (mean ergodic theorem) lim lR(}..)
P
}.. . . o
So the m right eigenvectors q; make a basis of Ker A 1
the left eigenvectors make the dual basis
o
y
-;t
x
We have the direct sum decomposition
E
n
With
Ker A + ~ (A) respect
to
R (P)
+
any basis
Ker P. compatible with
operator (P - A) is represented by the matrix:
Ker A
R(A)
1 0
Ker A 1
R(A)
0
(P - A)
-A
this
decomposition the
269
Let us set A'
P AlP.
0
Then the resolvent of Al may be written 0
deduce
\'le
P
that
th~
spectral projection R for
t)le
eigenvalue 0 of
P may be written
A,
R
pl + (I - P)
,.M
where pl
Q
1
Q
1
t11
QQ
and Q and 11 are obtained from the
f·1 M
agregate A 1 '" M A1 Q in the same manner as Q and t1 were from Q.M. spectral projection P for A admits the factor i za Uon P 1 0
the number of recurrent classes Df the agregate
II ,.
the
The redL;C0.d resDlvent
of Al is : o
R
where
31
is the reduced resolvent at zero of A l'
One
,,1
has
.J
o
•
and R{P) may be
decomposed into a direct sum : p
(2)
This decomposltion Is similar to (1), Since pl may be interpreted as
the spectral projection for the zero used p1A P
elgenv~lue
of a Markov chain it may be
to aggregate a new generator A2 as done above wi th p, Tn other words 1
2
'" MNA
/
2
.R
(P')
may
be
interpreted
as
the
generator
of
a
neh'
chain
QQ. The process may be iterated, Let us eX3mine for insbmce the
previous example.
271
we obtain the following system of equations A V
1)
0
o a
o ...
(1) (I -
This
system
V.
PV.
1
0
P) V
0
may
solved
be
easily
For
S(-A )V ..
+
thanks
instance
the
1 o 1 and the second one impl ies
-APV
P( ii)
We
etc ...
o
PA P V
+
1
+
0
P
0
f
to
decompos i tion
the
first
equation
gives
0
obtain the aggregate Markov chain PA P of the preceding section 1
and P(li) uniquely defines V
ii)
also
PV
o
o
S{-A )V
(I
o
The
above
part
the
determines
SA
1
process
1
is
V0
+
of
V,
in
R
because
the
computation
( 1-P)
Sf.
interesting
of
(Va ,V 1 • ... ) is made in a decentral ized-aggregated way : first we compute A 1 and then we solve the aggregate system. r-lore generally the computation of €:k (Ek,\ -
{Ao
+
€: 111
)}-l
may be
done by recurrence on k. Let us consider the case k=2. To compute Va one has to solve the system (i)
AV
(ii)
AV
+
{iii}
-'wo
+
o
0
0
a 1
AV 1
0
0
II V
a 1
+
AV
1 1
+ f
a
273 \~e will denote by E the matrix E '" [ell ..• I e ] . m 1
The eigenvalue equation may be written
where
diRg(A~>""X~).
II'
X
By identification of the factors of z, we obtain
A Q o
The first
1
+ A E
En 1
1
~quation
is trivially
s~tisifed
if 8 is obtained from Q by
a change a variables E '" QT with T invertible. I f we multiply from the left
the second equation by MI we obtain f1 Q T ",'
and
so
vanish. known
we
if
~ggregate
choose
T
the
matrix
operator M A1 Q that Is T
These in
formal
which
diagonalize
(M A1 Q)T '" II
1
theory
if
the
(if
possible)
the
we see that the z-term
calculations show the following
perturbation
described above then
-1
important
eigenvalues
split
in
fact well
the
form
they are solution of an aggregate eigenvalue problem
and. also a bi'ls is QT of eigenvectors of Ao may be constructed made of limit
of
eigenvectors
for
the
perturbed
eigenvalue
problem
(the
first
order
eigenvalues fix the zero order eigenvectors).
For perturbed eigenvalues of Markov chains one is mainly interested in computIng the limit of me Where mE: is the invariant pr'obability measure (left-eigenvector)
corresponding
to the zero eigenvalue of Ao +
E:A,.
The
most common situ3tion is the case of a generator A0 with a block-diagonal structure with weak interactions between the blocks. In ego [5J it is shown that
lim
m£ is -':l.Ssociated wi th the eigenvalue zero of the aggregate 1-1 Al Q
,
th:at is m
, perturbed eigenvalues split and lim m£
Invariant probability measure of MAl Q.
E-+O
,. mWhere
m is the
275
Ql
Here
Ai
eigenvalue
is
the
of A1(O)
total
= PA,P
projection
associated
wi th
the unperturbed
for the series A'(z).
(When 0 is an eigenvalue of A'(z)/Im Q(z) then one has to define Q1 (z) b
o
The operators
1 y 2i1r
1
Qi
Q () 0
(z)
z
. (f
0. -
(0)
are
holomorphic subprojections of Q(z) which
reduce A(z) and the characteristic polynomial of A(z). (which is p('\,A(z» r ITo i'="
thanks
pC'\,A(z) I 1m
This
to
lj)
may
be
factorized
once
more
1
Q. (z))
P(A.A(z)/Qi (z».
1
"process of reduction"
eigenvalue of the first
(Kato [9]) may be pursued whenever an , 2 term of the aggregate series A (z). A (z), ..• is
diagonali zable. In [2J, [5J. it is shown that this is always
o
of a
perturbed Harkov chain.
In particular,
if
true for the eigenvalue
An
the n-th aggregate
chain admi ts ,\...
0 as eigenvalue with multiplid ty m then there n repeated eigenvalues of A(E) of the form Snh + o(c ). In
the
next
paragraph
we
will
exist m
derive a simpler way to compute a
perturbed eigenvalue, for systems with particular structure.
277 Assume the components are in "parallel" mode that is the system fails only when the two components are failed. The reliability engineers are interested in three parameters :
=
Avai13bility Reliability MTT!?
=
P (the system works at time t)
l-P(X
P (the system works between 0 and t)
=
t P(Xs
0) ~
s
0,
~
t}
Hean time before the first failure of the system (the ini tial state
being the state where both components works). ylhen the system is large (the number of components is more than 10) an exact computation of these parameters is not possible ; a perturbation method will provide approximatIons for then. 2.2. Model of the system The large scale repairable systems involving a repair poli'cy with the constitution of queues can in the exponential case be modelled by a Harkov chain
X with the following properties : t
There exists a partition of the set E of states in subsets G i
E
and the following hypothesis are made
Hl
: the only nonzero transitions occurs between neighbooring subsets (from G i
to 01-1 or H2
:
from any state e in G
t
(1
>
0)
there exists an nonzero transition to the
subset G - • i 1
H3
:
the largest of the transi tion rates
smallest of the transItion rates G
i
H4
Any state e in G
1
+
-+
G+
1 1
is much smaller than the
G- , i 1
(I ) 0) can be reached from at least one state In G - • 1 1
279
The submatrices constituting A have the following properties
a)
Ai (i
0,
k-l)
are of dimension
(01+1'
their elements are between 0
ni'
and 1. b) 8 (i '" 1, k) are of dimension (oi-1' ni' ; their elements are greater than 1 one; moreover the sums of each column is strictly positive by H2. c)
The di agonal el emen ts are such tha t the sum of the col umns 1s zero. Di (d O,k) can therefore be expressed as a function of Ai and B , 1
(1
D.(O)
Let Di d)
1
(i
O,k)
The sum of each line of Ai (i
O,k-l) is non zero by H4,
2.3. Approximates for availabilities
Availabilities
can
be
expressed
as
function
of
the
stationary
probabilities of the Markov chain. The first result gIves approximates for them when
E
is small.
Theorem 1 Let
p.(e) 1
(1
O,k) be the column vector of the stationary probabilities for the
st.ates in G.
1
and let Ql{E) (1
= O,k) be the columns vectors defined by
Q (E) o
1,
0i (E)
E
i
j=1 IT 1=1
(D-:
(i J A j-l )
1 ,k)
281
o Ci
0, k-2)
Dt(E) is inversible : moreover
From the last equation one can deduct
substituting in the preceeding one
Let 0'
k-2
(E)
By induction one gets then
D' i+1 (c) P i+1 Ce}
(1
o.
k-1).
An explicit formula for P.(s) is then: 1
P. (IE:) 1
E
i
(0' -1 j
p (r:::)
o
(i
o
t
k).
283 Let A'
be the submatrix of A corresponding to pC the complementary
subset of P in E. The structure of A is similar to that of A : D~ (E). B'l EA~,
DI, (d,
-
81 2
Let Qt(E) be the vector whose components are the mean time spent in the states of Pc between 0 and t. Qt(c) is solution of :
Let 11m
T
t,""rn
T is solution of AI T
Ti
(i
=
0,
k')
is a column vector of dimension n l , Solving the system as in?, i
one gets : 1, k')
with 0,,-1
1, 1(1)
i
f1oreover
I
(
)
'""
C~
as
E '"" 0
C' o
285 Let
G.
be
1
the
subset.
of
states
where
components
are
('?ither awaiting repair or in repair). To define a state in G
< ••• <
enough to give the list j,
(~) 1
of such states is
not Horking
$ r, i t is i ji of components in repair. The number
i
•
In the subsets G
i)
i
r.
it
is necessary to know the components in
repair and the order of the queue. There are n-r A i-I'
H3
such states.
The corresponding process is easily seen to satisfy assumptions H , H , 2 1 and Hit and approximates ca.n therefore be found for the avai 13bil tty of the
system.
Let p
us
[ 11
deftned
12 by
n
get
for
<
example
<
[jl
the
probabil i ty
is not available. I f i
(
(
jrJ
the
)
that
the
subset
r a state e of P is
list of the components being
repaired and by the order of the queue for the components awaitjng repair.
Applying Theorem 1 to each of these states and summing one gets the following approxim3tion for the unavailability of P :
Q
(i - r)!
IT A" j6? J
Similar results can be obtained for various repair policies (see [14] fOl'
det3ils).
Approximat ion easily
be
obtained
for the reI iabil i ty pElrameters of a system can therefore for
large
scale
systems by
transition graph of the associated MarkOV Chain.
a
simple
Inspectrion
of the
287 [11] R. PHILIPS, P. KOKaroVIC. A singular perturbation approach to rnodclline
and control of Harkov, chains IEEE A.C. Bellnan issue, '981. [12J H. SnON, A. ANOO. Aggregation of variables in d}7lamic syster.lS, Econometrica, 29, 111-139, 1961. [13J J. KDtENY J 1.. SNELL. Finite :f-.1arkov chains J Van Nostr:md, 1960. [ 14 J O. lvlURDN. Evaluation de pol i tique 5 de maintenance pour
lU1
sys teffie
complexe, RIIID, voL 14, n° 3, pp~ 265-282, 1980. [15] S.L. CAHBELL, C.D. l'-1.EYER jr. Generalized inverses of linear transformations. Pitman, London, 1979. [16J TKIOUAT. These Rabat
a paraitre.
[17J J.P. QUADRAT. Commande Outils et
[un
~bdiHes
ortir.ale de chaines de
~.farkov
perturbces
1'-lath. pour l'automatique ... t3 edition CNRS 1983.
J.P. QUADRA..T Optimal control of perturbed, lo:!arkov chain the lTu11titime scale case. Sin::;ular pertubation in systems :md control. CISfv1 courses and lectures n° 280, Springer Verlag 82.
[19J F. DELEBECQUE, J.P. QUADRAT. COIltribution of stochastic control, teaJ11 theory and singular perturbation to an example of laY,Ge scale systems Management of hydropower production. IEEE AC avril 1973.
289
PLA..1>.J
- Introduction 2 - Notations and statement of the problem 3 Perturbed Markov chains 4 - Reviel'l of controlled l!arkov chains 5 - Control of perturbed Markov chains 6 - Example and application.
1 - INTRODUCfION
Stochastic or deterministic control problems can be reduced after discretization to the control of }·'-arkov chains. This approach leads to control of ~,!arkov chains which have a larp:e number of states. An attempt to solve this difficulty is to see the initial ~!arkov chains as the perturbation of a simpler one, and to design algori thtts lvhich use the hierarchical structure of more and more aggregated P.lodels, described in the previous paper of Delebecque, to increase the computation speed of the optmal control. The two time scale control problem (actualization rate of order £) is solved in Delebecque-Quadrat [6J , [7]. The ergodic control problem when the unperturbed chain lUiS no transient classes has been studied in Philips-Kokotovic [19]. In this paper we give the construction of the complete expansion of the optimal cost of the control nroblem in the general multi-tine scale situation. This presentation is a very little improved version of 0;uadrat [17J , [18J. For that, we use three kinds of results: - the Delebecque's result discussed in the previous
~aper.
- the realization theory of ~licit systew5 developed by Bernhard [1J. Indeed this method gives a recursive mean of co~putinr the complete cost expansion in the uncontrolled case. the r,lille -Veinott [10J way of constructing the opti!!1al cost expansion of an unperturbed ~!~rkov chain having a small actualization rate.
29'
The conditional e).-pected cost knoh'ing X(O,w) is a defined by :
wx :
E[j (lI.))
I X(O ,w)
= xJ,
Vx
E
Ixl-
Z
vector denoted h'
(2.2)
The Hawiltonian is the operator
IRlxl
h
lR
-+
w
Ixl
[m - (1 +A)iJ w + c
where i denotes the identify of the
(2.3)
(lxi, Ixl) -
matrices set.
Then w defined by (2.2) is the unique solution of the KollTDgorov equation hew)
o
(2.4)
2.2 - In the perturbed situation the n-tuple defining the perturbed
~nrkov
chain is (T, X, 8, m(e), c (e), A(c))
- tJ is now the space of the perturbations ; in all the following it is m,+ ;
m(E), C(E), A(E) have the same definition as previously but depends on the parameter variable.
E
E:
a , and we suppose that they are polynomials in this
l'/e denote by d
the degree of a polynomial and by v its valuation (the smallest non zero power of the polynominal). In the follO\>,ing d(m) = 1, vern) = 0, V(A) = v(c) = d(A) =i . From this particular case the general case can be understood. The Hamiltonian of the perturbed problem is denoted by h(w,c)
= [m(E) -
(1 + A(c)) iJ W + C(E)
(2.5)
293
Then with H(W)
[lvl-(I+J\)JW+C,
(2.11)
\vhere
c
(~,
I
the identity operator
A
n
t:
IN, cn
the operator
lx' _[:ec~or~)
are
o
i
0
0
0
....
0
... .
0 i
~ th IzI-blocti~~
J
1
an e).:pansion of the cost is obtained by solving HeW)
o
(2. 12)
f\brever the sequence (\"i' i
E:
n.J) can be computed recursively. These two
resul ts \."ill be shown in part 4.
2 . .3 - For the control prahl em u (T, X', 'U-' rn,
\ole
need the introduction of the n-tuple
, A)
- 'U is the set of control l'Ihich is here a finite set. luI denotes the cardinal of '4- . Its generic element is denoted by u. - m denotes the (I'l..d ,lxi, Ix!) tensor of entries mU xx' go in Xl, starting from x, the control being u. c denotes the control
bein~
(lUI, 'Xl) u.
matrLx of entries c
U
x
,
the probability to
the cost to be in x, the
295
h*(W) = min hU (w), Vx £~
W
X
U
(2. 18)
x
The optimal expected cost w* is the unique solution of the dynamic prograrrnning equation (2. 19) 1m ......... ~..
I ...Ui.........
policy is given by
x
2.4 - The perturbed control problem is defined by the n-tuple
Its interpretation is clear from the previous paragraphs. . u * *E U analogy the notatlon H (W,E), h (W,E) , W ,H 0'1) are clear, but we need a definition of H*(\~. For that let us introduce the lexicographic order, ~ , for sequences of real numbers, tlmt is :
By
(Yo'y,,···) ~ (yo'y"yZ"") is true (if Yn = yin' Yn
<
m
<=>
then Ym ~ ylm) Ym
€
(2.20)
IN.
We denote by min the min.imtnn for this order. Then we define H* by :
*
..,.
H (\\1) = min x
Hu.X Oil)
(2.21)
297 '
with
nu - i
a O ::
(3.4)
ao~
~~
E
~ml 0
aO
~
ml~
J
[! ]}!+1l Gi
aO
CR. + 1) blocks
(3.6)
0
0
G
aO
[0\\
F
(3.5)
(.2.+ 1) blocks
(3.7)
blocb
O--OJ (2+ 1) blocks
(3.8)
Indeed if W is a solution of (3. 1) yn
= (Wn ,
n +1' •••• J Wn +9.)
\'of
is a solution of (3.3). Conversely if \>Je
t~
is a solution of (3.3), by
el~ination
of the variables y
see that W satisfies (3.1).
Let us denote by £ the space
m.1X'1 x(2 + 1) in
'vhich lives y.
299
Now the fact that ?'l (J) :J?'l (E) implies that the sequence Wn is lIDiquely defined. We have proved the Theorem 1 : The solution
of :
WE
h(W,E) : ::: (r.1(E) - i
A(E) \'I + C(E) =
a
(3.13 )
admits an e:x.)?ansion WeE) lvhich is the unique solution of
H(W): = (M-I-A)W
+
C= 0
( 3.14 )
Jvbreover W can be computed recursively by constnlcting the implicit system realiz.ation of y
-1
o (3.15)
\.;here E, F. 0, H are defined in (3.5) to (3.8). This implicit system has an output uniquely defined and it admits a strictly causal realization. A specific algorithm is given in Tkiovat [16J
4 - REVIBV OF CONTROLLED f','lARKOV mAINS U
Given the controlled Markov chain n-tuple : (T,X, U , m
J
C
U J
A). The opt:i.nal
conditional expected w* cost is the unique solution in w of the dynamic programming equation h*(\'I) X
u min [(m - 1 - 1.)\<1 + cU]x u
O,VX(Z.
(4.1)
This result can be proved using the Howard algoritrun Z Step 1 : Given a policy s f.U
linear equation :
, let us compute \"', solving, in w, the
301
The existence and the tmiqueness of a solution in
\V
of (4.1) folloNS easily
from this result.
5
COm-ROL OF PERTURBED J-.1A.RKOV Gl!\INS U
Given the perturbed controlled ~larkov chain n-tuple (T,X lU ,a , m (A) J cU(e) , A (E)). TIle optimal cost is the unique solution in \'J of the dynamic programming equation : h
*..c\11 ) d ::
':X:
u nUn u [(m (E) -1 - A(E) ) \11
+
U
C (E) ] X
0, Vx
IS X
5 ( .. 1)
We have the Theorem 2 : The solution of (5.1) denoted by
'II*E
admits an eA~ansion in
E
denoted by W* (E) lvhich is the unique solution in 1'! of the vectorial dyna-
mic nrogramming equation : H*.X 0'1) ==
min
[(t-P- I - /1.)1" +
U
eU ].x = 0,
Vx
E"""
....
(5.2)
+
Let us remember that min means the minimum for the lexicographic order on the sequence of real numbers. The solution W* can be computed by the vectoriel HO\vard algorithm ~
x
Given a policy s
IS
U ,let us compute tv using the results of
part 4 Hos (w) = 0
(5.3)
Step 2 : Given a conditional expected cost Wl let us :ir.1prove the policy by computing +
U
min H U
Ne change
.x
( 5.4)
0'1)
U
s (x) only if H
.X
(W)< O. TIlen we return to step 1 .
303
By (3.15) we know that the order of the matrix E is smller than (v CA) + The entry C .+ is equal to zero for n ~ d (c) - £ - 1 nH 1
1)!:t1.
We add IZj new states to z, denoted by z'with :
z'n+l
= Wn •
With the new states
(5.8)
Zll =
(z, z') the second part of (6.6) can be l>.rritten : (5.9)
(5.9) has the form
J'z"
n
a
(S .10)
with J' an observation matrLx of the dynamical system of state z", It n follows by the Cayley-H8..!~lton theorem that if (5.10) is true \In :
n
~
n > d (c) then (5.10) is true Vn> dec). The theorem 3 is deduced
easily from this result.
Remark: In Tkiouat [16] a method is given to compute for each state x a bound q(x) on the size of vectors on which ,,,e have to make the vectorial minimization. 6-Example and application Let us show on a trivial 2 time scale example hO\II these results can be useful to design fast algoritlnn to solve stochastic control problem. Let us take the most simple example
305
using the particular structure of \Ii, and the results explained in Delebecque, that is \'1e compute solution of:
and P2 solution of :
. P,- ['] 1
1
=
Then \ye compute the aggregate \<Jhain transition matrix
0
P,
0
A= 0
0
0
0
A
Pz
0 0
and the aggregated cost
C, = P,
[CC']'
z
c3
z = Pz rtc
C
1
4
307
Then it is possible to improve the strategy by minimizing in u all the entries of the 4 - vector
\'Ie have seen that in this process we have never solved a linear of size 4 but
three systems of size 2, and 4 minimizations. generaly when the matrice mO has a block diagonal structure, this perturbation method ir.1prove the speed of the 1-I0Wllrd algorithn. In the best situation \<Je can obtain a [Z [2 algorithr.1 to solve the probler.l.
~bre
In this discussion we have only compute the first tem. of the expansion "there the vectorial minimization defines completely the control after computing only the two first terms of the expansion Wo and W . 1 The algorithm is complicated to be implemented. It is done inTkiouat [16]. This method can be applied for discrete version of the followinrr diffusion process : dX t
= b,(Xt,Yt,Ut ) ,
dt
+
dYt = £" b 2 (Xt ,Yy 'Ut ) dt
t
a 1 (Xt ,Yt , Ut ) ill1 1
+ ~
?
aZ(Xt,Yt,l't J d1'J~
that is diffusion process having b~o time scales. Some dam management problems can be described in this fOTQalism see Delebecque-Quadrat [19J
309
[11] R. PHILIPS, P. KOKafOVIC. A singular perturbation approach to modellin!) and control of
[12J H. SHDN, A.
~
ANlx)~
chains IEEE A. C. Bellr.tan issue, 1.98 t.
Aggregation of variables in dynamic systerns,
Econometrica, 29, 111-139, 1961. [13J J. KBIENY, L. SNELL. Finite Markov chains, Van Nostrand, 1960.
[14J O. MURON. Evaluation de politiques de maintenance pour un systeme complexe, RlRO, vol. 14, nO 3, pp. 265-282, 1980.
[15J S.l. CAJillELL, C.D.
~~R
jr. Generalized inverses of linear transfor-
mations. Pitman, London, 1979. [16J TKIOUAT. These Rabat
a paraitre.
[17J J.P. QUADRAT. Corrunande
ortir..ale de chaines de r·(arkov perturbees
Outils et 1-1odiHes Math. pour I' automatique . .. t3 edition CNRS 1983. [1SJ J.P. QUADRAT Optjrnal control of perturbed,
~~rkov
chain the Jnultitime
scale case. Sin!:Ular pertubation in systems and control. CISM courses and lectures n° 280, Springer Verlag 82. [19] F. DELEBECQUE, J.P. QUADRAT. Contribution of stochastic
control~
teaM
theory and singular perturbation to an example of large scale systems }'1anagement of hydropower production. IEEE AC avril 1978.
311
but
they
words.
may
weak
be
the network. assuming forces
coupled
in
determine
a
the
slow
time-scale.
long-term dynamic
In
other
behavior
of
The aggregate model captures this long-term behavior by
that its
the
nodes
"aggregate" external
strongly
connections
node
strength
of
to
their
lose
connected
connections.
to
The
internal other:
same
connections
identity
and
"aggregate"
weak
in
act nodes
external
each
as
a
area
single
through
connections
weak
have
a
negligible effect on the short-term behavior of the network which is modeled with the areas disconnected from each other. The outlined methodology is a analysis with connections E~O
in
model
two
of
respect
[1].
time-scales.
the ne two rk
parameter
is
t
~
and
singular
[:
obta ined
tIt.
perturbation
representing the weak
itself
groups
is
the
A
singularly
by transforming the
set of aggregate
transformation
algorithm which
scalar
of a
This asymptotic analysis investigates the limit as
variables into a The
to a
result
(slow) and local
found
nodes
areas.
r igina 1 s ta te
0
(fast) variables.
automatically
into
perturbed
The
by
a
computer
detai Is
of
this
grouping algorithm and its applications to large power systems can be found in [6}.
A summary of the algorithm is given in Appendix
A limi ta tion individual
of
the
external
proportional
to
methodology
connection
the
small
presented
is
assumed
parameter
c.
in
to
This
[1]
be
is
c.
tha teach
weak.
that
formulation
is,
excludes
more common situations in which the weakness of external connections is due to their
sparsity.
In many applications.
individual external
connections
as
as.
than,
are
strong
connections.
In
connections.
Relating
this
paper. the
but we
much allow
sparsity
sparser strong
pattern
but
of
a
the
internal
sparse network
external with
time scale properties. we extend the asymptotic analysis of larger class of our
analysis
networks.
also
[I}
its to a
From the graph theory point of view [7]
contains
a
new
result
on
the
dependence
of
r
the
eigenvalues of a graph on its sparsity pattern. We and
~.
first and
characterize show
that
the
the
sparse
time-scale properties similar connections. a
singular
network
topology
external
by
two
parameters
connections
induce
d
the
to those of dynamic networks with weak
Although the aggregate model depends on two parameters, perturbation
model
is
still
obtained
by
the
same
decomposition-aggregation transformation. An application of pattern of a
large network.
our grouping algorithm. system of
these
is
to find
We show that
the unknown sparsity
this can be determined by
The network considered represents
the Western U.S .•
In this case,
results
but with connections
the power
treated as 1 or O.
the algorithm produces a grouping into nine areas quite
313
£.
I
min a,i
max a, i
It is desired
(2.:2 )
that a partitioning into areas be such that even the
worst node has mare internal than external connections.
that is.
the
.!lode parametet.
d
=:
-E 1 c I£.
(2.3)
should be small, may
violate
d«l.
this
1n
a la rge ne twork.,
requirement.
E
can be used and c. al fu.rther discussed be example in Section 5. The a-th area connections if
to in
the
Then
a small number of nodes values
of
c
'I
ai parameter d. This wi 11 the large power network
define the conjunction
node with
sparse
external
has
average
and
internal
dense
(2. '1 )
where E
'Y a
the total number of external connections in area a,
Y!
the total number of internal connection~ in area
The
least
favorable
areas
are
those
with
the
connections yl, and the densest exter.nal connections
Y
I
mi.n a
Cleacly, y I I
Y
~ ill £.
{Y
1 } , a
-E
Y
max
{
E}
'Va .
11.
sparsest
yE,
internal
where
(2.5)
11
is bounded below by ill £. I , that is. I
(2.6)
where ill
min {mal, a
m
Il
the number of nodes in area a.
Our goal is to find a partitioning in which even the worst area has more internal than external connections, that is, the "area parameter"
315
,
Area 1
/'
(
-----
Area 2
/--1---\
1
1
I
I
1
I
I
I I
1
I
I
I
1
I
I
I
I 1
I
I
1
I I
I
I
I
1
I
1
I
I
I
l, - - - - - _ . / 5
I ./
1
o
Node
- - - Connection Figure 2.1
A lO-Node. 2-Area Network
Area 1
r---------, 11
l0
Area 2 r---------~
2
3
4111
2
3
4
0
0
0>--1-1-<0
0
0
0
l 1
'--------_./ '---------_./ o
Node
- - - Connection Figure 2.2
An B-Node. 2-Area Longitudinal Network
is still applicable,
but with less accuracy.
Additional examples of
sparse networks are given in [8]. 2.3 Dynamic Networks Al though clarity we To
stress
our
ana lys is
present the
it
applies
in the
structural
to
other
context of aspect
of
types
of
networks.
for
electromechanical networks. the
problem.
we
make
the
317
where K! is the maxma internal connection matrix of area a. Since an area itself is a dynamic network, the i-th diagonal entry of is
K!
-c!i'
that is the sum of the other entries in row i.
off-diagonal
entries of Kl is 2yl
The sum of the
Although the diagonal
entries of
I . a. a K are much larger ln magnltude than any of the off-diagonal entries. 't . . . . . Ka is not dlagonally dominant because lts rank is rna-I, due to a zero
eigenvalue with the eigenvector u
[1
a
This
zero
mode
disconnected that,
(2.11)
1
is
from
the
the
equilibrium
rest
of
when isolated, area a
the
manifold
= x~, ]
the
area
when
a
expresses the fact a = 0, whenever all
x
=
j
1,2, ... ,m . a . . E . 2 . partition of K lnto r block matrlces KE , a, 1
. 'l'h e correspon d lng
1,2, ... ,r,
It
is at equilibrium,
its storage potentials are the same, x~
B
of
network. i.
aB
K~B contains the connections between areas
is such that
a and B.
We note that KE is a diagonal matrix whose i-th diagonal E aa entry is -cai' the negative of the sum of the other entries in row i.
It is also helpful to observe that the sum of the entries of K!B is equal to the number of external connections between areas a and fl, and t ' I . E . hence d«l and 0«1 imply that Ka lS dense and KaB 16 sparse, for all a and B. 3.
TIME SCALE SEPARATION We
areas
now demonstrate and
behavior
their of
decomposes
sparse
the
how the
equilibrium manifold
external connections
overall
dynamic
network.
The
this slow motion all is,
the nodes
because
the
dense
equilibrium.
areas This
is
connections
During
negligible
due
the to
fast the
motion,
sparsity
exchange becomes significant over a
motion
of
the
two- time-sea Ie variables.
motion
areas behavior
as
is
the areas.
Intuitively.
internal
potentials in the same area to rapidly equalize. area
network
During
in the same area are "coherent."
the areas move as IIrigid bodies." fast
the
two-time-scale
into the fast local motions of the storage elements within
the same area and the network-wide slow motion of
are
property of
induce a
heavier made
allow
bodies by
a
new
is
the
node
to reach an
exchange
external
longer period,
rigid
apparent
the local motions
that is,
the of
that
with other connections.
that is. slow.
choice
of
the This
s ta te
319 We have thus defined a transformati.on into the aggregate and local variables
y
c
z
G
of
the
original
x .
state
{3. B}
The inverse of this transformation is explicitly known
x
(3.9)
where G+ is block diagonal (3.10)
L
G+ a.
m
-1
-1
-1
ma. -1
-1
-1
-1
ma.- 1
-1
-1
-1
a.
(3.11)
ma. -1
In the new variables y and z. the network model (2.9) becomes
y
All
A12
y
{3.12}
z
A21
A22
z
where All
CKEU
A12
CKEG+ (3.13)
A21
GKEU
A22
G(K
1 + KE}G+
x
321
I All/{£. 0).
All
(3.21) I A21 -= AZ1/{£ d),
so that the norms of All' A12 , 1\21 This scaling reveals that the model (3.12) the parameter 0,
and 1\22 are all 0(1).' is singularly pecturbed by
that is,
(3.22 )
In the slow time-scale t
(3.23)
S
the same model has the 50-called explicit singular perturbation form
(3.24)
This
well-known
results.
One
of
model
allows
them
is
us
the
to
make
following
use
Theorem
of
many
[1,9,20]
existing about
the
<
0*.
time-scale properties of (3.24). Theorem 3.1.
a < d
},.6
~
There exist
0*
and d*
such
that
for
all 0
0 =5
d*. the system (3.24) has r slow eigenvalues
= },.[ (All
-1
dA12A22A21) + O(6d)]
(3.25)
and n-c fast eigenvalues
},.f and,
= }"[A 22 hence.
+ O(od)]/o
I },.61/1},.f I
(3.26)
O(o}.
Furthermore.
the
slow
fast subsystems of (3.24) are
y( 0) ,
(3.27)
and
323 is based on Theorem 1 in [211. In this case. the s low-fas t time relation (3.23) is ts vb tt' the eigenvalue approximations (3.25). (3.26) are only O{vOd}. The state approximations (3.27}-(3.30) are also O(vbd). but because of the oscillatory character.
are valid only
for a finite time interval. Our development sparsity
and
remainder
of
thus
far
time-scale the
paper
properties
we
decomposition-aggregation
has demonstrated a of
dynamic
show how these procedure
relationship between
and
networks.
properties
for
a
are
In used
grouping
into
the in a
areas
when the partitioning is unknown. 4.
DECOMPOSITION AND AGGREGATION The
presence
of
the construction of
the
node
parameter
d
in
Ao
several approximate models foe
and
1\22
suggests
the slow and fast
subsystems. 4.1 Aggregate Models Since centers
of
the
states
of
the
inertias,
we
treat
slow
subsystem
(3.27)
as
an
(3.27)
are
aggregate
the
area
model.
To
simplify (3.27). we first express the terms in AO in a clearer form:
(4.l) which is derived in [1], and
1
where using (3.2) and (3.l0).
(4.3 )
Therefore.
the
slow SUbsystem can be rewritten in a form similar
to
that of the original system (2.9). namely,
(4. II)
First. we note that Ka and K! are symmetric. Second. the row sums of each of these two matrices are zero. This can be verified by post-multiplying Ka and K! by the r-dimensional vector
325 which makes the effect of the internal connections more exp 1 ic itof KI is this simplified form The computation of even a time-consuming for large networks. To avoid it. and also to improve the
accuracy
of
the
aggregate
(4.?).
we
propose
a
compensated
aggregate
(4. g)
where c
is a compensation factor chosen to minimize the discrepancies
between
the
The
eigenvalues
actual
[6],
slow
while
require
much to
a
(4.9)
and
are
known
eigenvalues
Ma'
straightforward leads
of
Ka
to
and
calculate.
additional major:
the
the
from
TheI:"efore, of
the
slow
the
the as
CMa Ka of
c
examples
accuracy
o(
algorithm --1
of
choice
the
eigenvalues.
grouping
eigenvalues
computation and.
improvement
actual
the
are
does
will
not
show.
compensated
aggregate over the rigid aggr:egate (4.4). 4.2 Local Models The fast individual
subsystem variables
reference states.
(3.28)
in
the
represents areas
the
with
r:elative motions of the respect
to
their
local
The local motions can be simplified by recognizing
that
(4,10) where
(4.11) (4.12) are
0(1).
Since A
220
is a
blOCk-diagonal matr:ix.
the fast subsystem (3.28) decouples i.nto r
a.
1.2 •••.• r,
areas,
and
are
the
obtained
frequencies
220 by discarding the of
the
modes
d
small.
(4.13)
where zfa. are the area a. states of Z and A These models
then for
local. models
of
=
d'lag
(A 1
220 connections (4.12)
will
• . . . • Ar ). 220 between the be
somewhat
lower than those of the fast modes of the full system [13. p. 331]. A correction for the mode shirt is to include a. blocks. A , of A in the compensated local models 22d 22d
the
diagonal
327
Table 4.1 Eigenvalues of Slow Models (IO-Node Network)
Matrix
Eigenvalues
Exact
K
0.00. -0.61
Complete Aggregate
AO
0.00. -0.63
All
0.00, -0.80
Slow Model
Rigid Aggregate
Table 4.2 Eigenvalues of Fast Models (IO-Node Network)
~atLix
Fast Model
Eigenvalues Area 2
Area 1 Exact
-3.34.-5.00,-5.00,-6.16 -3.00,-5.00.-5.00.-6.90
K
Past Subsystem A22
a.
A220 + -3.42,-5.00,-5.28.-5.89 -3.00,-5.00,-5.60,-6.00
Compensated Local Models
dA
and
4.2.
Note to
0.
22d
A~20
Local Models
ass igned
3.00.-5.00,-5.00,-6.89
-3.31,-5.00,-5.00.-6.00
the
-3.00,-5.00.-5.00,-5.00 -3.00, 5.00,-5.00,-5.00
that
the
areas.
fast
eigenvalues
of K and A • while 22 depend on the other areas. The
in general,
time scale separation is \-3.00\/1-0.611 The
relative
I: igid
errors
aggrega te and
4.9
in
0(1/0).
the
the fas t
slow
eigenvalue
approximations
by
e igenva lue approxima t ions by the
the
loca I
models are of Oed). the
slow
0.61/0.80
the
exact
aggregates. the
and the relative errors of the approximations by By choosing c and fast subsystems are of O(od). 0.76, the eigenvalues of the compensated aggregate are slow The
compensated
eigenvalues,
which
is
achievable
eigenvalue approximations local
models
are
of
results aLe verified for this example.
of
2 0{d ).
for
the fast Thus,
all
2-area
subsystem by
the
time-scale
329 experience motivates the modification of the theoretical results for practical applications to large networks. The power network considered is a 411-machine, 17S0-bus. 2800-line model of the Western Systems Coordinating Council
(WSCC) system (Figure 5.1).
The total number of nodes is
machines are shown as dots on the map. 411
1750
+
preserve allow
2161.
the
the
The
overall
use
of
nodes
bus
sparsity
the
sparsity-based
reduced
not
are
in
to
order
thus
network
structure,
and
grouping
algorithm,
which
the
of
Most of the
is
summarized in Appendix C for completeness. The
WSCC
dispersed
system
consists
generation
centers.
of
sites.
several
major
Connecti.ons
are
load
dense
centers
about
with
the
load
The centers are interconnected with a few long transmission
lines along system.
the Pacific Coast and around
forming
transmission
the
lines
so-called
are
the eastern portion of
IIdonut "
sparse,
but
pattern
[14].
each
individually
K was
constructed
the
These
long
strong.
and
cannot be modeled as E-connections. The
2161x2161
storage.
connection
matrix
using
sparse
Then all the non-zero Off-diagonal entries of K were set to
1. while the diagonal entries were set to be the negative of the row sums of the oCf-diagonal entries. set
to 1.
The inertias at the nodes were all
The computation time required to calculate the 15 slowest
eigenvalues
and
their
eigenvectors
using
the
about 6 minutes on a VAX 11/760 computer. to use
partition WSCC the
9-area
procedure.
into
several different
partition
shown
in
sparse
algorithm
was
The eigenvectors were used numbers
Figure
of areas.
5.1
to
We will
illustrate
the
The same technique can be applied to other partitions.
The algorithm partitioned the 9 areas of WSCC along boundaries of well-derined uti.lity
geographical
systems.
are
denser
the
9 areas
than are
Since the
regions
the
connections
partitioned
approximately
connections
within
between
along
corresponding
the
different
boundaries
of
utility uti.lity
to
systems systems,
sparse connections.
The 9-area partition will be compared to the II-area partition based on
actual
connection
strength
in
Section
7
to
identify
the
weak
connections. Let us pa rameter
now examine Sand d.
connections for external Table 5.2. .
The numbers
o(
nodes,
and internal and external
the areas are shown in Table 5.1,
connections between the areas are
and the numbers of
shown by the Ka matrix in
In this system. some internal nodes have .
connectlons. that 1S. £ we still have small violates
the sparsity pattern using the area and node
1
=
1.
b = 20/104
the requirement d«l,
Since
m = 104
0.192.
.
1S
single internal
E
However, with
the assumption
-E y.
much larger than
»c
E
c
2,
d
~
2
is no longer
331
Table 5.1 Numbers of Nodes, and Internal and External connections, for 9-Area Partition of WSCC Number of Average Number Number of Average Number Internal of of External Internal External Number Connections. Connections. Connections, of Nodes, Connections, Area E E I yI m (ca,)ave (ca)ave Yo. a a n 1
274
352
2.57
20
0.073
2
104
124
2.38
13
0.125
3
176
221
2.51
18
0.102
4
234
290
2.48
17
0.073
5
166
208
2.51
3
0.018
6
283
357
2.52
11
0.039
7
405
542
2.68
15
0.037
B
232
306
2.64
5
0.022
9
287
371
2.59
12
0.042
Table 5.2 Aggregate Connection Matrix Ka
Column Row
1
2
3
4
5
6
7
8
9
1
-20.0
0.0
1.0
10.0
0.0
9.0
0.0
0.0
0.0
2
0.0
-13.0
3.0
0.0
0.0
0.0
6.0
0.0
4.0
3
1.0
3.0
-18.0
2.0
0.0
1.0
5.0
3.0
3.0
4
10.0
0.0
2.0
-17.0
0.0
0.0
0.0
0.0
5.0
5
0.0
0.0
0.0
0.0
-3.0
0.0
3.0
0.0
0.0
6
9.0
0.0
1.0
0.0
0.0
-11.0
0.0
1.0
0.0
7
0.0
6.0
5.0
0.0
3.0
0.0
-15.0
1.0
0.0
8
0.0
0.0
3.0
0.0
0.0
1.0
1.0
-5.0
0.0
9
0.0
4.0
3.0
5.0
0.0
0.0
0.0
0.0
-12.0
333
6.
NE'l'WORKS WI'l.'H SPARSE AND WEAK CONNECTIONS
In sparse
our presentation of connections, we have
the area decomposition simplified the analysis
results [or by assuming
. k aB . . t h at eac h connect ton ij between node 1. tn area a and node J. 1n area B normalized with respect to the inertias is unity. To be more precise
of areas, the actual in determining time-scale decomposition Of connection strength and inertia values have to be used. particular importance are the weak and strong connections. For many practical power systems,
while
the
ucores"
o(
the areas aLe largely
the sparse and dense connection patterns. the determined by boundaries of the areas are determined by the connection strength and inertias.
In some cases, weak and strong connections may cause large
perturbations in the area partition. from
sparse
connections
may
split
For example, an area determined into
several
areas
when
actual
connection strength is considered, because it possesses weak internal connections. As another example, areas determined (rom sparse connections
may
combine
into
a
single
area
connection strength between the areas is strong. section is
to
pLesent
because
the
external
The pULpose of this
time- scale decomposition of areas
taking into
account simultaneously sparse and dense connections, and weak and strong connections. We will summarize the weak connection results, analyze a special class of systems with sparse and weak connections, and
use
the WSCC
examp le
to discuss a
gene I:a 1 a ppI:oach
to ana 1 yze
practical systems using the grouping algorithm in Appendix C. 6.1 Weak Connections
The aI:ea decomposition results for networks with weak connections in [1] can be readily deI:ived [ollowing the analysis in Sections 2, 3 and 4 (or sparse connections. In the weak connection analysis, we use the actual inertia values and connection strength. system are
~
a . where m 15
t
The network dynamic equations [or an r-area
= 1.2, ... ,I:I and BFa when
i,
(6.1)
. . . the lnert1a or capacltance of the storage element of
~
Xi'
and k~~ is the actual connection strength between node i of area a and node j of area B. [-'allowing the notation of (2.9) and (2.10), we write (6.1) in the compact form
335
where
(6.9)
System (6.8)
is a singularly perturbed system in the fast time scale
t = t. and is similar to system (3.22) f except that the weak. connection parameter area parameter b and the node parameter d. In the slow time-scale ts = Et , f explicit singular perturbation form
for sparse functions
£"
system
connections. as both the the
has
(6.8)
(G.IO)
The ti.me-scale results as in Theorem 3.1.
are
now
in
terms
of
that
for
Theorem 6.1.
'I'here exists £* such system (6.8) has r slow eigenvalues
"- s
-1
f:A1ZA2ZA21) +
"-[ (All
0(£
2
t:
instead
all
0
<
of
t:
d
and
5. E*,
<5
the
)]
(6.11) "- [All + O{E:)] and n-r fast eigenvalues
l..f
2 l..[A 22 + O(E: }]/£ (6.12) I
-I
l.. [GK G
+ O(t}]/t
and, hence. Il..sl/Il..fl fast subsystems of (6.8) are
o (£) .
Furthermore.
the
slow
yeO) ,
(6.13) (6.14)
and
337
dy/dt
eoA
f
l1
Y ... COA
12
Z
(6.17) dz/dt
cdA
f
where tf
21
~
y
A Z 22
£.1 t and
All
ci{Eu / [£.1 0 ) , A12
A21
Gi(Eu / (£.1 d ) , A22
ci{EG+1 (£.1' O), (6.18)
System fast
(6.17)
is a
scale
t
time
separation
and
time-scale
ts
=
G(K
+ ti(E)G+/£.I.
singularly perturbed
where
f
the
=
1
the
product
(to)t ,
cd
produc.t denotes
system
f
di
weak
(6.17)
system expressed denotes
the
coupling. becomes
in
the'
time-scale In
the
the
slow
explicit
singular perturbation form
(6.19)
Theorems
3.1 and 6.1 can now be combined to obtain the following
result. The~~.
o < d 5 d* and
There exist &*, d* and c* such 0
<
t
5
c*.
that for all 0
< 0
~
0*,
the system (6.19) has r slow eigenvalues
(6.20)
= O(A l1
+ O(ed)]
and n-r fast eigenvalues
(6.21)
and,
hence.
1)..6 11 I)..r
1
O(di).
fast subsystems of (6.19) are
Furthermore,
the
sloW'
and
339 Table 6.2 Eigenvalues of Past Models (lO-Node Network with Weak connections)
Fast Model
Matrix
Eigenvalues Area 2
Area 1 Exact
-3.05,-5.00,-5.00.-5.09 -3.00.-5.00, 5.00,-5.18
K
Fast Subsysterr A22 Compensated Local Models
A + -3.05,-5.00, 5.02,-5.09 -3.00,-5.00,-5.07,-5.07 220 cdA
From the the
(1
22d
A~20
Local Models
-3.00.-5.00, 5.00,-5.00 -3.00,-5.00. 5.00,-5.00
results of Theorem 6.2 in which the product co denotes
time-scale
existence
3.05.-5.00,-5.00,-5.09 -3.00,-5.00,-5.00, 5.1B
(1
of
separation,
multiple
we
time
can
readily
infer
the
possible
scales.
Consider the case when some external connections are sparse, some weak and others both sparse and weak.
Assuming
that
the
c,
0«
slow
time-scale
has
a
three-level hierarchy
(6.26)
in
which
1
In this four time-scale s is the slowest time-scale. 3 system. a nested aggregation starting from t s can be used to obtain subsystems for the different time-scales. t
6.3 A Procedure
to
Ident i fy
Sparse
and
Weak
Connections
in
Large
Scale Networks Large
scale
connections
decomposition. for
power
power
which
networks
together
often
contain
determine
the
both time
sparse scales
and and
weak area
Information on sparse and weak connections is useful
system
planning
and
operation.
setting of power system stabilizers and results in 'l.'heorem 6.2 only deal with the the sparse connections are also weak.
such
as
the
siting
and
protective relays. The simplest situation where
For more general situations we
propose to use the grouping algorithm to identify the weak and sparse connections. The first
We will use the WSCC example as an illustration.
identification
step,
the
procedure
grouping
consists
algorithm
is
of
three
applied
to
steps. the
In
the
system with
341
Figure 6.1
An II-Area Partition of WSCC
Figures S.l and 6.1 of the WSCC system illustrate that our 3-step identification weak
approach
connections.
design
of
the
The WSCC
provides results Intertie
valuable here
would
Generator
information have
on sparse
been useful
Dropping
and
large disturbances described in [22].
the
connections
and
weak
will
also
system planning, eKpansion and control design.
be
helpful
and the
Controlled
Separation scheme for sparse
in
for
Knowing future
343
where x~ is the machine angle. m~ the inertia constant. d~ the damping ~ 1 1 constant. the mechanical input power and v~ the voltage at node i 1 aB in area a. As in the linear case. we assume that the admittance Bij be tween node i in a rea Cl and node in a rea 13 is dense wi thi.n an area and sparse between the areas such that the node parameter d (2.3) and the area parameter b (2.7) defined according to the number of connections are small. Without the external connections. each of the r areas is isolated. The dynamic equation of area d is
pi
-d~x'?" + p,?"
i =1.2 •••• , rna'
111
(7.2) Area
d
has a continuum of equilibr.ium points
-a.
(7.3)
X
a where Xo is an equilibrium point. Cd a
scalar. and
un as
given
in
(2.11)
(7.4)
Equation {7.3} implies that if (7.2) is at equilibrium. then (7.2) is still at equilibrium if all the components of xa. are increased or decreased by an equa 1 amount. This is the egu i 1 ibr i urn proper ty and (7.3) defines the equilibrium manifold for area n. On this manifold, the potentials within an area are equalized. A second property is the conservation property that the state
ya.
m La. C1 n a. mC1 rnixi/rn • i=l
m La. m~ 1
(7.S)
i=l
remains unchanged when m
La. P'?"
i=l d~1
1
Of
O.
i
(7.6 )
1.2 •...• rna'
(7.7 )
345
From
these
results.
subsystems.
It
we
is
can
also
derive
possible
the
to
nonlinear
derive
slow
various
and
fast
aggregate
and
ana lys is
for
local models following the analysis in Section 4. The
time-sea 1e
decompos i t i on
and
aggrega tion
nonlinear dynamic networks with both sparse and weak connections will a Iso
make
use
of
the
conservation and
equ il i br ium
proper ties.
The
derivation can follow the steps in Section 6. 8.
CONCLUSIONS The
time-scale
sparse
dynamic
method
[1]
approach
networks
for
to
the
developed
dynamic
decomposit1.on-aggregation
here
networks
extends
with
and
of
complements
£-connections.
the
These
new
results are mote broadly applicable. since most practical large scale networks
have
the
characterized d.
Although
they
can
using
terms
the
be
a
in
be
readily
nested
removed
of
pa t tern
an area
cesults ar:e
transformation. can
sparsi ty
for
extended
Furthermore, in
a
parameter networks
of the
manner
in
this
.5
are
time-scales. time-scales
decomposition-aggregation
assumption to
and
node pacameter
two
with multiple
the
simi lar
paper
and a
a separation into
to
application
assumed
the
of
linear
nonl inea r
connections
ne two rks
wi th
£-connections [5.1]. Using
the
descr ibing revea 1 Some
the
the
of
parameter
s low
and
r:oles
the
networks
node
played
cesults
with
f as t
by
also
d.
new
aggregate
dynamics the
internal
improve
£-connections.
are
the
The
and
proposed.
local
models
Thes e
models
and
externa 1
connections.
models
proposed
earlier
application
of
the
for
results
to
large networks is illustrated with a 2000-node power system. 'The
sparse
connection whose both. is
connection
results
time- sea les Only the
analyzed.
2000-node
power
algorithm
for
to are
are
a
due
either:
special More
results
provide to
case where
general
system.
more
then
weak
the
0
r
with
treatment sparse
of
the
weak
net'Wor:ks
connee t ions
or
sparse connections are also weak
situations
A three-step
identifying
combined
general
sparse
are
illustrated
procedure
using
connections
the and
with
the
grouping tbe
weak
connections has been proposed and shown to pr:ovide useful information about the 2000-node system.
347
In the row of GK responding to
E
to the first node
of area « is -eEl' .
z~. l
corresponding to the variable
node i+l of area ~ is c;(i+l), E
E
the entry car-
the entry corresponding
and the entry corresponding
~
to
E
node j in area B lS C«(i+l)Bj-c«lBj' where e«iBj is the number of connections between node i of area « and node j of area E. Then in the row of
A21
sponding to
= y~
GKEU corresponding to the variable z~,
the entry corre-
is
(A. 6)
and the entry corresponding to YB is
(A. 7)
Hence max [ E «, i (A. B)
In
corresponding
the
to
the
variable
the entry corresponding to z~ is 1
(A. 9)
a. the entry corresponding to zk' k I: L
is
(C E - ca.(k+l»/ma. a.l E
(A.IO)
and the entry corresponding to z~ is J c
cE (c E «(i+l)E( j+l) alB (j +1) + alB
E
-
E
Ca.(i+l}B)/m,B'
(A.ll)
349
APPENDIX B PROOF OF THEOREM 3.1
We and [1].
show the O(od} approximations of ~s and ~f in (3.25) by an iterative upper block triangularization process in Introducing the new fast variables
(3.26)
ill = Z +
we transform dY/dt s
=
-1
dA 22 A21 y (3.24)
(B.I)
into
1
AllY +- A12 Til' (B.2 )
where
(B. 3)
A second transformation using the revised fast variables
(B. 4)
yields
(B. 5)
where
(B.6)
351
APPENDIX C GROUPING ALGORITHM
For
a
large
identifying
scale
sparsely
dynamic
connected
network,
areas
is
an the
automated grouping
tool
for
a 19ori thm
in
[1,6], which is summarized as follows: step 1:
Choose the number of areas.
Step 2:
Compute a basis matrix V of the eigenspace of the r slowest
t.
eigenvalues, using either EISPACK [15] for dense matrices or the Lanczos algorithm in [16] for sparse matrices. Step 3:
Apply Gaussian elimination with complete obtain
the
states
used
for
the
pivoting
pivots
as
to V and
the
reference
if
the
row
all
the
reference
states. Step 4:
Assign
a
state
corresponding
i
to
to
reference
state
a.
is.
a.
state among
in
V
states. closest to that corresponding to state i. When
the
large
r
number
of
for Step 2.
of r to find an r and
area
areas
for
entries
per
a
is
not known,
we
choose
a
sufficiently
Then Step 3 can be repeated for various values
that yields a partition with small node parameter d
parameter
computer
r
().
Typical
2000-node
row
of
K
eigenvalues/eigenvectors.
computation
network are
with
about
4
an
times
on
average
minutes
of
a of
VAX 4
CPU
11/780
non-zero per
10
353 16. J. Cul11lm and R.A. Willoughby, IIComputing Ei.genvalues of Very Large Symmetric Matrices," J. Computational Physics. Vol. 44. pp. 329-358. 1981. 17. H.A. Simons. liThe Architecture of Complexity," Proceedings of the American Philosophical Society, Vol. 104. pp. 467-482. 1962. 18. R.M. May, Stability and Complexity in Model Ecosystems, Princeton University Press, 1973. 19. D.G. Feingold and R.S. Varga, "Block Diagonally Dominant Matrices and Generalizations of the Gerschgorin Circle Theorem, II Journal of Mathematics. Vol. 12. pp. 1241-1250, 1962. 20. P.V. Kokotovic, "A Riccati Equation for Black-Diagonalization of Ill-Conditioned Systems.1I IEEE Transactions on Automatic Control, Vol. AC-20, pp. B12-B14, 1975. 21. J.H. Chow. J.J. Allemong and P.V. Kokotovic. Perturba t ion Analys is of Sys terns wi. th Sus tained High Oscillations," Autornatica, Vol. 14, pp. 271--279, 1978.
"Singular Frequency
22. L. H. Fi nk, II Emergency Control P rac t ices. II 'I'as k Force on Emergency Control Report. Paper BSWM034-4. IEEE Winter Power Meeting, 1985.
STABILITY ANALYSIS OF SINGULARLY PERTURBED SYSTE"NIS
HI\.. I\.h alilt
The
scale decompositJon or a singularly perturbed system
into reduced and boundary layer stability analysts.
gy~1tems
provide::.; a strong
for
KLIMUSHCHEV and KRASOVSKII (1961) employed
Lyapunov functions to show that asymptotic stability of
equili-
brium of a singularly perturbed system can be established, for sufficently small perturbation parameter E, by investigating equilibria
r
the reduced and boundary-layer systems.
Similar
results for linear systems were given in DESOER and SHENSA (1970) and WILDE and KOKOTOVIC (1972).
HOPPENSTEADT (1966) gave what are
probably the weakest "conceptual" cond:i. t Ions under which unif'o r'm asymptotic stabil:i ty of the equi librium of a singularly perturbed system is confirmed for sufficiently small E. com~rise
The conditions
uniform asymptotic stability of tHe equilibrium of
reduced system, unlf.'orm asymptotic stability
f the equilibrium
of' the boundary-layer system. wliformly in the frozen slow parameters, and growth concH tions on r'ight-hand side rune tions. Stability investigations in which conditions, guaranteeing those of HOPPENSTEADT (1966), are imposed on Lyapunov functions for the reduced and boundary-layer systems have been pursued by a few researchers.
Examples can be found in HAB8TS (1974), CHOW (1978).
GRUJIC (lg81) and SABERI and KHALIL (19SQ).
Stability results for
linear multiparameter singularly perturbed systems are reported in KI-[ALIL and IWKOTOVIC (1979). LADDE and
SILJ AJ{ (
) and ABED
(1985), and for a class of nonlinear systems which are linear in 'Elect,rical Engineering Dcparr.ment" hflchigan 51,al-;: lJniver"!ty, East. LaIlSltll!" l\.Jr_
359
We consider a nonlinear nonautonomous singularly perturbed system
f( t , x,
Z, E ) ,
(1.1 ) EZ
=
g(t,x,Z,E),
£>0,
where the functions f and g are smooth enough to ensure that, for specified initial conditions,
(1.1) has a unique solution.
Suppose
that (1.1) has an isolated equilibrium point at the origin. Stability of' the origin is investigated by examining the reduced system x where z
f(t,x,h(t,x),o)
(1. 2)
h(t,x) is an isolated root of
g(t,x,s,o) and
0
the boundary-layer system
dT
g(t,x,Z('r),o)
,
T=
tie:
where t and x are treated as fixed parameters.
(1. 3)
Theorem 1 states,
essentially, that if x=o is a uniformly asymptotically stable equilibrium of the reduced system (1.2), z
= h(t,x)
is an asymp-
totically stable equilibrium of the boundary-layer system (1.3), uniformly in t and x, and f and g satisfy certain growth conditions, then the origin is a uniformly asymptotically stable equilibrium of the singularly perturbed system (1.1), for sufficiently small E. In Theorem 1, asymptotic stability reqUirements on the reduced and boundary-layer systems are expressed by requiring the existence of Lyapunov functions for each system.
The growth requirements on f
and g take the form of inequalities satisfied by the Lyapunov functions, which we call interconnection conditions. We assume that the following condltions hold for all
361
where
~(.)
is a continuous function of an Rm-vector
which vanishes only at zf= O. Interconnection Conditions
V and W satisfy the following inequalities:
+
(i)
(x)
(1. 9)
(ii)
(z -
h(t,x»
+ £B;~(X). (1.10)
(iii)
altJ
at
+
(t,x,Z,£) < Y~t2(z - h(t,x»
+ B~~(x)¢(z - h(t,x)
2,
(1.11)
Y2
Y2
For simplicity the constants , ~2' 8 Yl , and are assumed to be nonnegative. The term £Y $ in (1.9) allows for more general l dependence of f on £. It drops cut when f is Jndependent of E. Similarly, (1. 1. 0) drops
Oil t
Nhsn g is :l ndependent or
at the above inequalities, one might have to obtain on £-dependent terms like Kl + an interval [0, £1"
E
Lhat is why
;
E
£.
In arriving
unlfo~m
bounds
may be restricted to
\
boundary-layer systems, V and W, in hand, we consider a Lyapunov function candidate v(t,x,z)
del~ined
by a weighted sum of V and H
v( ,x,z) = (1 - d)V(t,x) + dW(t,x,z),
o
(1.12)
From the properties of V and Wand inequality (1.6), it follows that v (t,x,z) is positive-definite and decrescent. with respect to (1.1) and using (1.7) -
\J
where
<
[1 8 2f +
[l-d)(a, 2
Sit and Y 2 2
(1.11) we obtain
~(I-d)Bl-
-
- 1:(l-d)Sl- !d B2 2
= Y2 +
It can be easily seen that for all
2"
Y
.
Computing v
d
e:;(a 2 -£y )
2
~df21l:'1
(1.13)
363
[(l-d)
+ 4(1-d)d
(1.18)
Several examples of antonomous systems were studied in SABERI and KHALIL (1984), including estimating the region of attraction of a synchronous generator connected to an infinite bus. For the linear time-varying system x
l\ll(t)
x +
(t)
(1.19 ) (t) x
EZ
+
(t)z
where Aij(t) are continuously differentiable,
He
~(A22(t»
< -
C
< 0
and the reduced system b-
-1
x = [All(t)
(t) are bounded,
(t)
(t)
is uniformly asymptotically
A21 (t)]x
stable~
Ao(t) x
(1.20)
the functions V and W can be
taken as
V(t,x)
x Tp (t)x s
(1.21)
(1. 22)
where Ps(t
and
(t) -=-cIm>o satisfy the I'.,.yapunov equations (t)+A
(t)
T
o
(t)
o
(1.
(1.
4)
Then A;;;sumptions 5.2 -
(Euclidean
norm), [I)
[12=1,
(z-h(t
,x» and Y2 are bounds on the time functions
365
The analysis of Section
extended to multiparameter
singularly perturbed systems.
convenience} we consider an
autonomous system x
.
( Ill.
>0, ~iE.R l)J=I~ ... N
, If
s are known, they can be represented as known multiples of a
single parameter, i.e.,
E,
and the problem reduces to the
single parameter case treated in section 1.
In same applications
it is important to perform stability analysis without assuming the knowledge of the mutual ratIos of the perturbation parameters. In this section we extend the stabl1Lty analysis of section 1 f
to the rnult1parametel' system (2.1) 'tJhen the parameters
E
are
unknown. We want to study the stability of an isolated equilibrium at the origin (x=o, zi=o Vi). required to hold for all (x,sl"
The following assumptions are
.,,)€
x .. xB
~l
:";N
The origin is the unique equilibrium of (2.1) algebraic equations
o
(2.2)
have a unique root z.=h.(x) such that l
l
(0 )
o.
The reduced system is given by
f(x,h (x), ... 1
,hl~(x))A ~'l ==
and has equilibrium at x=o.
(x)
(2.3)
367 Viewing gi as the interconnection term. the ith isolated subsystem is given by
(2.10)
which has an equilibrium at constant
(2.10).
Suppose now that we can find a Lyapunov function
Hi (x~
)
satisfying
2
01:1 i
(2.11)
rp i '
dZ 1
hi(x»
is a continusous scalar function of
(x) which vanishes only at
iV
i
t
S
Notice that the positive
(x).
does not affect the stability of the equilibrium of
hi(X).
Suppose further that
satisfy the "spatial" interconnection condition N
< i
""""" ~
b .. I~ J.;)
.Ip J. ,
l
b .. >0 lJ-
(2.12)
'faking N
L
(x,
),
o
(2.13)
i=l
with unspecified e
I
1
s as a Lyapunov function candidate for (2.9),
it can be shown that the derivative of W along the trajectDry of
(2.9) satisfies dW
\'There (p T
<:
¢ T (E N R + RTEN ) ¢
_
(
<11
(2.14)
1 " " , {PN)' E
and :j=i
R
(r.o ) ; 1.•)
369
ties (2.18) and (2.19) are stright-forward extensions or
(l.g) and (l.ll) when restricted to the autonomous case.
A.
Lyapunov function candidate for the singularly perturbed system (2.1) is taken as v(x"
,,'"
,
)
'ZN)
(2.20)
The derivative of v along the trajectory of (2.1)
for some do>O. satisfies
v
[II:: IV ]
< -
r
I L
where
doct o
-u
-u
1
S
E
(u l ' ... ,uN) ,
Iz(doP i
DR + RTD - DE E= diag (~, ... ,
T
r
I[~~ I +
(2.21 )
diEici)~ E:
E D,
EN)' and
r
PH ij ).
Since D R + RT D is positive-definite, S is positive-definite for Moreover~
sufficiently small £i'
o~
are bounded and for any there exists
E.· 1
>0
< 1,
since
doYo -
such that whenever
T -1 u S U
E
E
equilibrium of (2.1) is asymptotically stable. su~narized
the elements of u >0 as
of
Thus
, v
in Theorem 2. Suppose that Assumpt10ns 2.1 - 2.11 hold.
origin (x
E. -j.-O. l
=
0,
zi
=
Vi)
0
Then the
an asymptotically stable equilibrium
singularly perturbed system (2.1) for sufficiently small
Calculating the bourtds numbers £i for
whic~
is positive-definite.
can be done by determining the largest
the matrix on the right-hand side of (2.21) An easier to calculate, yet more conservative,
bound can be obtained as follOWS.
Using that
i'le obtain
371
1.
Extension of Theorem 2 to nonautonomous systems is straightforward.
2.
For N=l, the bound (2.23) reduces to the bound (1.17) derived in the single parameter case.
3.
The analysis is valid for all £i>o including cases IH::e Ei«
E
j
.
However, when the perturbation parameters are of
different orders of magnituate, less conservative results can be obtained by treating the problem as nested single parameter perturbations.
4.
The analysis does not involve bounds on the ratio between parameters,
i.e.~
(2.24)
as in the earlier work of KHAI,JL and KOKOTOVIC (1979) and KHALIL (1981).
This should come as no surprise.
The use of the (2.24)
bounds in KHALIL and KOKOTOVIC (1979) and KHALIL (1981) was a matter of convenience.
It was a way
at
excluding cases when
perturbation parameters are of different orders of magnitude since, as we pointed out in Remark 3, such cases are better treated as nested single parameter perturbations.
Actually in
KHALIL and KOKOTOVIC (1979) and KHALIL (1981) the numbers lliij and M1j were allowed to take any finite positive values. Recently, ABED (1985) emphasized that bounds like (2.24) are not needed by performing the analysis for the linear case without sueh bounds.
373
Ladde, G.S. and D.D. Siljak (1983), Multiparameter singular perturbations or linear systems with multiple time scales, Automatica~ 19, pp. 385-394.
A.N. and R. K. Miller (1977), Qualitative Analysis Large Scale Dynamical Systems, Academic Press.
Michel~
or
A. (1983), Stability and Control of Nonlinear Singularly urbed Systems, with Application to High-Gain Feedback, Ph.D. Dissertation, Michigan State University.
Saberi
Saberi, A. and H. Khalil (1984), Quadratic-type Lyapunov functions for singularly perturbed systems, IEEE Trans. on Auto. Control, AC-29, pp. 542-550. Saksena, V.R., J.OtReilly and P.V. Kokotovic (1984), Singular perturbations and time-scale methods in control theory: survey 1976-1983, Automatlca, 20, pp 273-293. Siljak, D.D. (1978), Large-Scale Dynamic Systems: Structure, New York: North-Holland.
Stability and
Wilde, n.R. and P.V. Kokotovic (1972), Stability of Singularly perturbed systems and networks with ics, IEEE Trans. on Auto. Control, AC-17~ pp. 245-24 Zien
s
An upper bound for the perturbed system, J.
parameter in a Inst.~
375
side of (1.2). We relax this assumption in Section 2 of this paper. Our theorems are rather different in hypothesis and rather simpler than those of [4.5] in this regard. Another important contribution of Section 2 is to relate the exponential stability of the averaged system (1.2) to the exponential stability of the unaveraged system (1.1). using a converse theorem of Lyapunov. As such, these theorems are a considerable extension of the local stability theorems of Hale. In Section 4. we extend all of these results to tWo-time scale state space systems and the results are generalizations in the sense mentioned above of those of Hale and Sethna. (B)
Our development of these theorems on averaging was heavily motivated by recent
literature on the application of averaging techniques to adaptive control--notably the work of Krause et al [9], Astrom [10]. Riedle and Kokotovic [11]. Averaging methods have been more prevalent in the stochastic adaptive control literature. ego Ljung [12] and the first attempts to apply averaging were made heuristically in [9]. and increasingly rigorously in [10] and [11]: The primary focus of the efforts in (9-11] is to use averaging to explain instability mechanisms in adaptive control arising from unmodelled dynamics. a phenomenon popularized by Rohrs et al [13]. In this paper, we content ourselves with applying our results on averaging theory along with techniques of generalized harmonic analysis introduced in Boyd and Sastry [14]. We study convergence rates of adaptive identification schemes and linearized adaptive control schemes without unmodelled dynamics and in the presence of persistent excitation. Estimates of convergence rates are of interest in the determination of optimal input signals for identification. In earlier work (Bodson and Sastry [15]). we also showed how persistent excitation guarantees a margin of robustness to unmodelled dynamics and established connections between the rate of convergence of the adaptive schemes and their robustness margins. A more detailed study of instability theorems for averaging and their application to understanding the mechanism of slow drift instability pointed out by Riedle and Kokotovic [16] is an interesting avenue of future work. In adaptive systems. averaging has usually been associated with slow cu1aptation. Since the parameter e appears in the right-hand side of the differential equation governing the adaptive parameters. and since averaging is considered as a perturbation technique. it is frequently understood that the results are valid only for e small (if not infi.nitesimaI), Le. for slow adaptation. However. simulations of adaptive systems show that averaging often provides a good approximation for relatively large (of order 1) values of the parameter • After this manuscript was written, new and related work of KOSUl and Anderson [11] was communica led 10 us for system (1.1) with f (t .X) linear in x . but wilh weaker conditions in the limit in 0.3).
€.
377
2. Basic Averaging Theory In this section. we consider differential equations of the form:
i = e f (t .x .e) where x ERn. t ~O. O<e ~€o. and
f
(2.1)
is piecewise continuous with respect \0 time. We
will concentrate our attention on the behavior of the solutions in some closed ball Bil of radius h . centered at the origin. For small E, the variation of x with time is slow. as compared to the rate of time variation of f. Such systems can be conveniently studied using the method of averaging (see e.g. [1], [3]. [6], [7J). The theory relies on the assumption of the existence of the mean value of
f
(t .x ,0) defined by the limit: t+T
f
(n'
(x)
= lim 1 T-=
assuming that the limit exists uniformly in
Jf
(T .x
.0) d
(2.2)
T
t
t
and x. This is formulated more precisely in
the following definition:
Deftnition 2.1 The function
Mean Value of a Function, Convergence Function
f
(t .x .0) is said to have mean value
f al' ex)
if there exists a continuous
function y(T): R + ...... R +, strictly decreasing, such that y(T)- 0 as T -+ tXJ, and: 1 t+T
Jf
(T .x .0) d
1" -
f
(I"
(x ) II ~ Y (T)
(2.3)
for all t .T ~O, x EBh . The function y(T) will be called the convergence function. Note that the function {(t .x .0) has mean value d(t.x)=j(t.x.O)
f
ai'
(x) if and only if the function:
ja,,(x)
(2.4)
has zero mean value. The following definition ([20]. p 7) will also be useful:
Deftnition 2.2 Class K Function A function a(e): R+-R+ belongs to class K
(aCe) E K) . if it is continuous. strictly
increasing. and a(O)=O.
It is common. in the literature on averaging. to assume that the function j (t .x .e) is periodic in 1 , or almost periodic in t. Then. the existence of the mean value is guaranteed.
379
(2.7)
for all
t ~O.
x EB1, • Moreover. w eCO.x )=0. for all x EBI! .
If, moreover: y(T)=a / T r for some a ~O, r ECO,l],
Then.: The function g(e) can be chosen to be 2a e r
•
The proof of Lemma 2.1 is provided in the appendix. The construction of the function we(r .x) is identical to that in [2.11 but the proof of (2.6), (2.7) is different. and leads to the relationship between the convergence function yeT) and the function gee). The main point of Lemma 2.1 is that. although the exact integral of d
Ct
.x) may be
an unbounded fUnction of time. there exists a bounded function we(t .x ). whose first partial derivative with respect to t is arbitrarily close to d Cr .x). Although the bound on we(t .x) may increase as e-O. it increases slower than 1/ e, as indicated by (2.6).
It is necessary to obtain a function w t:(t ,x). as in Lemma 2.1, that has some addi-
tional smoothness properties. A useful lemma is given by Hale in [3] (lemma 5. p 349). For the price of additional assumptions on the function d (l .x ). the following lemma leads to stronger conclusions that are useful in the sequeL
Lemma 2.2 Smooth Approximate Integral of a Zero Mean Function If: d (t .x ): R +X Bh - R n is piecewise continuous with respect to t. has bounded and con-
tinuous first partial derivatives with respect to x, and d
Ct
,O}==O for all t ~O. Moreover.
d (t .x ) has zero mean value. with convergence function y(T) II x U. and ad (t .x) has zero
ax
mean value, with convergence function y(T), Then.: There exists
g( e) E K
. and a function
We (t
.x ): R +x Bh
- R 11
Uewe(t .x ) fi ~ fCe) II x 0 II
awe(t ,x)
at
U
e
-d (t.x) II ~ g(eH xU
aw E(t ,x)
ax
II
~ gee)
•
such that: (2.8) (2.9)
(2.10)
for all t ~O. x EBn . Moreover. w E(O.x )=0, for all x EBh .
If, moreover: y(T )=a / T r for some a ~O, r E(O.l]. Then.: the function fCe) can be chosen to be 2a el' . The proof of Lemrru:t. 2.2 is provided in the appendix. The difference from Lemma 2.1 is in the condition on the partial derivative of
W E(l
.x) with respect to x in (2.10), and the
381
Comments The proof of Lemma 2.3 is provided in the appendix. A similar lemma can be found in [3J (lemma 3.2. p 192). Inequality (2.17) is a Lipschitz type of condition on p (t .z .e). which is not found in [3], and results from the stronger conclusions of Lemma 2.2.
Lemma 2.3 is fundamental to the theory of averaging presented hereafter .. It separates the error in the approximation of the original system by tbe averaged system (x -xu,,) into two components: x-z and
Z-XQ1 ••
The first component results from a
pointwise On time) transformation of variable. This component is guaranteed to be small by inequality (2.8). For e sufficiently small (e ~e 1). the transformation z -x is invertible. nnd as e-O. it tends to the identity transformation. The second component is due to the perturbation term p (t .z .€).
Inequality (2.17) guarantees that this perturbation is
small as e -+ O. A t this point. we can relate the convergence of the function y(T) to the order of the two components of the error x -xcv in the approximation of the original system by the averaged system. The relationship between the functions yeT) and
tee)
was indicated in
Lemma 2.1. Lemma 2.3 relates the function g(e.) to the error due to the averaging. If d (t .x) has a bounded integral (i.e. y(T }--1/ T), then both x -z and p (t
order of e with respect to the main term
f
QtJ
(z).
e-+O. but possibly more slowly than linearly ( as
,Z
.e) are of the
In generaL these terms go to zero as
-J'E for
example). The proof of Lemma
2.1 provides a direct relationship between the order of the convergence to the mean value, and the order of the error terms. We now focus attention on the approximation of the original system by the averaged system. Consider first the fallowing assumption: (AS)
Xo
is sufficiently small that. for fixed T. and some h'
t E[O.T / e) (tbis is possible. from (A3)).
Theorem 2.4 Basic Averaging Theorem If: The original system (2.1). and the averaged system (2.5) satisfy assumptions (Al)-
(AS).
Then: There exists W(e) as in Lemnw. 2.3 such that. given T ~O: D
x (t )-X01 ' (t ) n ~ wCe) br
for some br , er >0. and for all t e[O.T/ e]. e~eT'
(2.18)
383
II x(t )-xo,,(t) II
~tP(e)b
(2.24)
for all t ~O. and some b. Consequently. Theorem 2.4 does not allow us to relate the sta.bility of the original and of the averaged system.
This relationship is investigated in
Theorem 2.5, after a preliminary definition.
Definition 2.3
Exponential Stability, Rate of Convergence
The equilibrium point x =0 of a differential equation is said to be exponentially stable, with
rate of convergence
0/
(01 >0). if: II
x (t )
II
~ m II x (t 0) Ue -0(1-1 0)
(2.25)
for all t ~to~O. x(to)EBho ' and some m ~l. We assume that ho~h/ m. so that all trajectories are guaranteed to remain in B h
•
Theorem 2.5 Exponential Stability Theorem If: The original and averaged systems satisfy assumptions (A1)-CAS), the function
f ov (x)
has continuous and bounded :first order partial derivatives in x. and x =0 is an exponentially stable equilibrium point of the averaged system. Then: There exists e2>0 such that the equilibrium paint x =0 of the original system is exponentially stable for all e ~E2' Proof: The proof relies on a converse theorem of Lyapunov for exponentially stable systems (see for example [201. p 273). v (X 01 '
):
Under the hypotheses, there exists a function
R" -R +. and strictly positive constants
OIl.0!2.0:'3'0:'4.
such that. for all
XOl'
EBh 0: (2.26) (2.27) (2.28)
The derivative in (2.27) is to be taken along the trajectories of the averaged system (2.5). The function v is now used to study the stability of the perturbed system (2.16). Considering v (z). inequalities (2.26) and (2.28) are still verified. with z replacing X"", The derivative of v (z ) along the trajectories of (2.16) is given by: V(Z)I(2.16)=V(z)b.s)+(
~~
)(ep(t.z,e))
and. using previous inequalities (including those from Lemma 2.3 ):
(2.29)
385
and any such function will provide a bound on the rate of convergence of the original system for e sufficiently small. 3) The conclusion of Theorem 2.5 is quite different from the conclusion of Theorem 2.4. Since both x and
X a ,.
go to zero exponentially with t . the error x
- Xa ,.
also goes to
zero exponentially with t. Yet. Theorem 2.5 does not relate the bound on the error to E.
It is possible, however. to combine Theorem 2.+ and Theorem 2.5 to obtain a uni-
form approximation result. with an estimate similar to (2.24).
387
r
PLANT
(5I-Ar'b
v(I)
+
Fig 3.1 Block diagram of ada.ptive identi6er.
389
in which instance. tbe limit Ru (T) is called the autocovariance of u. It may be verified that the autocovariance matrix of a stationary signal w is a posi-
tive semidefinite function RII' (T). and that w is persistently exciting if and only if the autocovariance at 0 is positive definite [14]. Also. R".(T) can be written as the inverse Fourier transform of a positive spectral measure SII' (d v): tQ
R". ('T )
j
e il'T S", Cd v)
(3.16)
Further. if the input r is also stationary. S... (d v) can be computed. using the fact that the transfer function from r to w is given by:
(3.17)
so that: S...
Cd v) =qCj v) q' (j v) sr Cd v)
(3.18)
Using eqns. (3.16) and (3.17). we can conclude that: =
R".(O)
= jq(jv)q'Civ)sr(dV) >0
This in turn is assured [14] if the support of points (the dimension of w
=
Sr
Cd v)
(3.19)
is greater than or equal to 2n +1
the number of unknown parameters
=
2n +1).
With these definitions. the averaged system corresponding to (3.14) is simpJy: (3.20)
This system is particularly easy to study. since it is linear, and when w is persistently exciting. R". (0) is a positive definite matrix. A natural Lyapunov function for (3.14) is: (3.21)
and: (3.22)
where
"'min
and
"'max
are respectively the minimum and maximum eigenvalues of RI/.' (0).
Thus. the rate of exponential convergence of the averaged system is at least
E)..min(R",
(0)).
and at most €"'maiR", (0)). By the comments after theorem 2.5. we can conclude that the rate of convergence of the unaveraged system for e small enough is close to the interval [EAmi!l(R~. (0)). EAmll,,(R w (0))].
391
s.
Ca) 6.
(b)
c, ~
,
I.
(c) Fig 3.2 Trajectories of parameter error fll(= Cl-C:) and ¢lliJl with three different adaptation gains. (a) £=1 (b) (=0.5 (c) £=0.1
393
c. t, 2. I. -2. -\,
-£.
(a)
6•
•• 2.
e, ·~I
.,. (b)
3,7:
2. : 1.1£ &•
.
:.~:
"'~':
(c)
If.
• :
I
hi .
Fig 3.4 Trajectories or Lyapunov (unction V(¢) and V(¢u) with three adaptation gains (a) l=1 (b) £=0.5 (c) £=0.1 using log ocale.
395
(B3)
the function d (t .x) =
(B4)
A ERm Xm is Hurwitz.
(B5)
Xo
f
(t
.x .0)- f
aI'
(X) satisfies the conditions of Lemma 2.2
is sufficiently small that. for T fixed. and some h '
X al ,
(t) EEn' for all
t E[O. TIe] (this is possible. from (B2)). \Ve will also assume that YoEBh '. the corresponding closed ball in Rm.
Theorem 4.1 Basic Averaging Theorem for Two-Time Scale System If: The original system (4.1). (4.2). and the averaged system (4.3), satisfy assumptions
(B1)-(B5). Then: There exists I/1Ce)EK such that. given T ~O: (4.8) for some br • Er >0. and for all t E[O. TIe]. e ~er. and Yo sufficiently small. Further.l/I(e) is of the order of e+E(e) (as defined in Lemma 2.2).
Proof: We first apply Lemmo. 2.2, and obtain a result similar to Lemmo. 2.3. Consider the transformation of variable: x
= z + Ew f(t .z)
(4.9)
with e ~E 1. This transformation leads to:
i = (I + e
a:
uZ
€
)-le
If
01'
Cz ) + (
f
(t
.z .0) - f
av
(Z ) _ OW £
+ Cf Ct .z+ewE.O)
+(f
(t .z Hw ,.y) -
at
f
)
(t .z ,0))
f
(t .z +ew ,.0))
I
(4.10)
or: Z (O)=XII
(4.11)
where: (4.12) and:
(4.13)
397 Using this estimate in (4.16), and using the Generalized Bellm.aJl-GronwaU Lemma again:
(4.23) As in Theorem 2.4, it follows that. for some bT : Ux
By assumption. I
Xl.ll'
(4.24)
(t
(t) U~h '
let Yo. and Er sufficiently small that. by (4.22). y (t )EBh • for all t e[O.T J e]. It follows. from a simple contradiction argument. that the estimate in (4.24) is valid for all t E [0. T J e]. whenever e ~er .
Theorem 4.2 Exponential Stability Theorem for Two-Time Scale Systems If: The original system (4.1). (4.2). and the averaged system (4.3) satisfy assumptions
(Bl)-(B4). the function fat.' (x ) has continuous and bounded first partial derivatives in x. and x =0 is an exponentially stable equilibrium point of the averaged system. Then: There exists E4>O such that the equilibrium point x =0 of the original system is exponentially stable for all e ~e4'
Proof: Since
XU\'
=0 is an exponentially stable equilibrium point of the averaged system.
satisfying (2.26 )-(2.28). On the other hand. since A is Hurwitz. there exist matrices P,Q >0. such that A r P+PA =-Q. Denote by Pl,P2.tJ l.Q2 there exists a function v (X/.l v
)
the minimum and maximum eigenvalues of the P and Q matrices. We now study the stability of the system (4.11). (4.2), and consider the following Lyapunov function: VI (z
.y ) = v (z )
a.,
+ _- y T P Y
(4.25)
P2
so that: (4.26) where
Q'I=min(al,~pl)'
The derivative of
P'2
be estimaled. using the previous results:
VI
along the trajectories of (4.11). (4.2) can
399 Mixed Time Scales We now discuss a more general class of two-time scale systems. arising in adaptive control:
:i
sf '(t ,x.y ')
y' = Ay + h (t I
(4.32)
+ eg r(t
,x)
(4.33)
.x .y')
\Ve will show that system (4.32)-(4.33) can be transformed into the system described in the previous section. In tbis case. x is a slow variable. but y' has both a fast. and a slow component. The averaged system corresponding to (4.32). (4.33) is obtained as follows. Define the function: t
vet ,x) =
J
eA(l-T)h
(r.x)d
(4.34)
T
(I
and assume that the following limit exists uniformly in t and x: 1 t+T
f
al'
(x )
=T-c:o lim -T J f
( 4.35)
'('1' oX .v ('1' oX )) d T
I
Intuitively, v (t ,x) represents the steady-state value of the variable y with x frozen nnd E=O in (4.33).*
To show that the averaged system of (4.35) is the right one. we transform the system (4.32). (4.33) to the form (4.1), (4.2). using the transformation: y
=y , -
(4.36)
v (t oX )
From (4.34). v (t .x) sat-isfies:
:t
v (t
.x ) = A v (t ,x ) + h (t
oX )
l'
(4.37)
(t .0)=0
Differentiating (4.36), we have that:
.= I a
y
Ay
+e -
1'
(t aXoX)
f
'(t ,X .y
+v (t
oX))
+ g '(t
oX
.y +v (t ,x))
I
(4.38)
so that system (4.32). (4.33), is of the form (4.1). (4.2), with:
f g (t .x .y) =
(t ,x ,y) =
f 'et.x.y +v (t ,x))
8v ~:) f '(t .x .y +v (t .x )) + g I(t .x .y +v (t ,x ))
'This choice of transformation was pointed out to us by B. Ril!dJe & P. Kokoto'\'ic.
(4.39) (4.40)
401
5. Two-Time Scale Averaging Applied to Model Reference Adaptive Controller To apply the theory of Section 4 to model reference adaptive controllers we review the model reference adaptive system of Narendra. Va]avani [18) for the relative degree 1 case (our notation is however consistent with Sastry [19]). Consider a plant with transfer function (5.1)
where
np.
d p are relatively prime monic polynomials of degree n-l, n respectively and
kp is a scalar (the representation in (5.1) is assumed minimal). The following are assumed to be known' about the plant transfer function:
dp • np
(el)
The degrees of the polynomials
are known.
(e2)
The sign of kp is known (say kp >0).
(e3)
The plant transfer function is assumed to be minimum phase.
The objective is to build a compensator so that the plant output asymptotically
Tn
matches that of a stable reference model
(s) with input r (t). output Ym (t) nnd
transfer function
m(s) where k m >0 and
nm • dm
are monic polynomials of degree n -1. n respectively (not
necessarily relatively prime but both Hurwitz). If we denote the input and output of the plant u (t) and Yp (t) respectively. the objective may be stated as: find u (t) so that Yp (t )- Ym (t ) -0 as t -
o:::t
By using suitable prefiltering of the reference signal if neces-
sary. we may assume that the model
m(s) is strictly positive real.
The scheme is shown in Figure 5.1. The dynamical compensator blocks F 1 and F 2 (reminiscent of those in Section 3) are identical one input. n-1 output systems. each with transfer function (s1 -A)-lb eigenvalues are the zeros of
: A E Rn.-1Xn-l • b E R rt -
nm • The pair A
1
where A is chosen so that its
b is assumed controllable and. for ease of
book-keeping On the algorithm proof alone). we assume that they are in controllable form so that
1
(s1 -A)-l
= _1_ nmes)
.r
403
The parameters e E R TI zeros: d E R
l1
-
The parameter
1
1
in the precompensator block serve to tune the closed-loop plant
•
doE R in the feedback compensator assign the closed loop plant poles.
Co
adjusts the overall gain of the closed loop plant. Thus. the vector of 2n
adjustable parameters denoted B is
with the signal vector w E R211 defined by
The input to the plant is seen to be
and the stnte equations of the plant loop are given by
:ip
Ap
0 0
Xp
0
A 0
v(l)
beT p
0 A
V(2)
=
\~(l) \;(2)
bl'
+
b
BTw
(5.2)
0
It may be verified that there is a unique constant B· E R '2n such that. when B
transfer function of the plant plus controller equals when
m. (5). It can
B'. the
also be shown {18] that
is bounded and the parameter update law is given by
T
(5.3)
with
r
E R2nX2n • a positive definite matrix. all signals in the loop. i.e.
u. v. v(l).1'(2),yp .Ym
are bounded. In addition. lim e l(t) = 0 so that asymptotically Yp (t) approacbes Ym (t ). 1-.:>:.1
The proof of this fact used the following procedure: represent the model ( in non-minimal form) as the plant loop with B set equal to
e' . The state equations for the model
loop are
given by ' • T oCp
.im . (1) Vm
• (2)
Vm
bpC 'T
bpd*T
Xm
hd~cJ
A+bc'r
\' (J)
bc:'
0
bd' r A
AI' +bp d
=
m
v
(2) m
+
hI' h 0
Co T
(5.4)
The 3n-2 X 3n-2 matrix in (5.4) is henceforth referred to as A . and the 3n-2 vector in (5.4) as
b.
Then. subtracting (5.4) from (5.2) with
we have lhat
(5.5) and
· 405
and the eigenvalues of A).
Wm
is bounded. Hence it is easy to see that the equations (5.12).
(5.13) are of the form of (4.33). (4.32) with the functions
f ' and h
satisfying the condi-
tions of Section 4. To establish the averaging results. we assume that r is stationary. This implies. as has been shown in Boyd and Sastry [14]. that
Wm
is stationary. Its spectral measure is
related to that of r by (5.14) with
1
an exponentially stable transfer function. The function v (t
.rp) of Section 4 for the system (5.12), (5.13)
is
r
v(t.rp):=[! eA(t-T)bw!(T)d'i]rp o and the averaged
f
is given by (5.15)
Since wm is stationary, the limit in (5.15) may be shown to exist as follows. Define a filtered version of
Wm
to be t
w l1l! (t) =
Since
cT (sJ - A )-11; = ].m (s)
!o
(5.16)
is stable. it follows that wm! (t) is also stationary. The
Co
quantity inside 1.he square bracke1.s in (5.15) is s+T
lim T1
T--.
!
wm(t
s
i.e. the cross correlation between
Wm
)w~j (t )dt = R",m
\0'
mt
(0)
(5.17)
and wntf evaluated at O. Consequently. we may use
(5.14) and (5.16) to obtain a formula for R..,~, III
111/
(0) as (5.18)
407
using equation (5.18) .. With a m =3. k m =3, ap =1. k p =2. a=3. w=2. the two eigenvalues of the averaged system are computed to be -3.10e and -0.43e. both real negative. Figs 5.2. 5.3 show the plots of the parameter errors of
Co
and do for the original and averaged
system. with three different adaptation gains. Fig. 4 corresponds to a higher frequency input signal w = 4 such that the eigenvalues of the matrix R ....m ,,'ml (0) are complex
(-0.49±0.30i )e. and explains the oscillatory behavior of the original and averaged systems.
Using the results of Boyd and Sastry [14]. it is easy to verify the following facts (i)
R"'mwm'(O) is singular unless R",(O) >0. i.e. wet) is persistently exciting. Thus persistent excitation of w is a necessary condition for exponential stability of (5.19).
(ii)
If
m(s)
is strictly positive real and w (t ) is persistently exciting, then R".m \4"m! (0) is
Hurwitz. Hence
m(s)
being strictly positive real is a sufficient condition for stabil-
ity of (5.19), given that w (t ) is persistently exciting.
It is intuitive that if w is persistently exciting and
m(s)
is close in some sense to
being strictly positive real that R".m Ii"tn! (0) will be Hurwitz (in particular. this is the case if Re m. (j v) fails to be posi~ive at frequencies where
n (j v)
is small enough). More specific
results in this context are in [11.17). In view of the results stated at the end of section 5. averaging can also be applied to the nonlinear system described by (5.10)-(5.11). with A {x
)=..1 +bx T Q.
Consequently.
the nonlinear time varying adaptive control scheme can be analyzed through the autonomous averaged system (a generalisation of the ideas of [24]). However. due to the nonlinearity of the system. the frequency domain analysis. and the derivation of guaranteed convergence rates are not straightforward.
409
e. -1.5 -3,
·~,S
·it
\ r
-7.5
\/
(a)
e, -l.:~
-2,~
-l,iS
-~.
-:.:: I
1
(b)
·7« S ~ O.
(c) Fig 5.3 Trajectories of parameter error q,i:= da-do ) and ¢'1J2 with three adapta.tion gaiDs (a) t==l (b) f=0.5 (e) f=O.l using log scale.
411
6. Concluding Remarks We have presented in this paper new stability theorems for averaging analysis of one and two time scale systems. We have applied these techniques to obtain bounds on the rates of convergence of adaptive identifiers and controllers of relative degree L We feel that the techniques presented here can be extended to obtain instability theorems for averaging. Such theorems could be used to study the mechanism of slow drift instability in adaptive schemes in the presence of unmodelled dynamics. in a framework resembling that of [10].
413
Thjs. in turn. implies that
(LIO) Clearly g(e)EK. From (Ll), it follows that:
aWE(t,X)
---::-,--- - d (t ,x ) = -e
at
W
E(t
.X )
(Lll)
so that both (2.6) and (2.7) are satisfied. If y(T )=a / T r , then the right-hand side of (L8) can be computed explicitly:
(L12) and. with
r
denoting the standard gamma function: co
Ja e
r
('J",)l-r e-r'd T';; a e r r(2-r) ~a e r
(L13)
o
Defining g(e)=2a e r • the second part of the lemma is verified.
Proof of Lemma 2.2 Define W E(t..x) as in Lemma 2.1. Consequently,
(L14) · ad OX (t .X) IS . zero mean. an d'IS b ound e. d Lemma. 2 1 can be app I"Ie d to ad aX (t .x) • an d Slnce
inequality (2.6) of Lemma 2.1 becomes inequality (2.10) of Lemma 2.2. Note that since ad ~t;) is bounded. and d (t ,0)=0 for all t
~O. d (t ..x)
is Lipschitz. Since d (t..x) is zero
mean, with convergence function yeT) Ox D. the proof of Lemma 2.1 can be extended. with an additional factor Ox H. This leads directly to (2.8) and (2.9) (although the function
gee)
od ~~,x) • these functions can be replaced
by a
may be different from that obtained with single
gee )).
415
+E ( f
(t .z .€)- f (t ,z .0) )
:= ef av (z)
+ Ep '(t
(L18)
,x ,z .e)
where. using the assumptions. and the results of Lemma. 2.2: (L19)
For E ~El' (2.10) implies that (J +E
OlY az) E
,
has a bounded inverse for all t ~O. z EBh .
r
Consequently. z satisfies the differential equation:
i = [I+e =
Ef
av
1l;:'
(z )
(e!,,(z)+ep'(t,z,e))
+ E P (t ,z .E)
Z
(L20)
(O)=xo
where: p(t.z,E)=
!
aWE
l+e-_a~
-1
OWe Ip'(t.z.e)-e-_-!a\'(Z) I a~
1
(L21)
and:
:= .pCe) II z I
(L22)
Generalized Bellman-Gronwall Lemma (cf. [71 P 169) If: x (t ). a (t ), u (r ) are positive functions satisfying: t
x (t ) :£;.
Jo a (
T )x ( T )d T
+ u (t )
(L23)
for all t e[O.T). and u(t) is differentiable.
Then: t
x(t) ~u(O)e
ja(a)dCT 0
t
+
z
J ti(r)c o
for all t E[O.T].
jaCa)da T
dT
(L24)
417
[15] Bodson. M. and S. Sastry. liSman Signal 1/0 Stability of Nonlinear Control Systems: Application to the Robustness of n MRAC Scheme." Memorandum No. UCB/ERL
M84/70. Electronics Research Laboratory, University of California. Berkeley, 1984. [16] Riedle. B. and P. Kokotovic. "Stability Analysis of Adaptive System with Unmodelled Dynamics." to appear in 1nt. 1. of Control. Vol. 41. 1985. [17] Kosut. R .. B.D.O. Anderson and I. Mnreels. "Stability Theory for Adaptive Systems: Method of Averaging and Persistency of Excitntion/' Preprint. Feb. 1985. [18] Nnrendra, K. and L Valavani ... Stable Adaptive Controller Design-Direct Control,'·
IEEE Trans. on Autonuztic Control, Vol. AC-23 (1978). pp 570-583. [19] Sastry. S., "Model Reference Adaptive Control - Stability. Parameter Convergence. and RObustness." IMA Journal of Mathemotical Control & Information, Vol. 1 (1984), pp 27-66.
[20] Hahn. W.o Stability of Motion, Springer Verlag. Berlin. 1967. [21] Luders. G. and K.S. Narendra. II An Adaptive Observer and Identifier for a Linear System," IEEE Trans. on Automatic Control, Vol. AC-18 (1973). pp. 496-499. [22] Kreisselmeier. G.
\I
Adaptive Observers with Exponential Rate of Convergence."
IEEE
Trans. on Automatic Control, Vol. AC-22 (1977). pp. 2-8. [23] Goodwin. G. C. and R. L Payne. Dynamic System Identification, Academic Press. New York. 1977. [24] Riedle B. and P. Kokotovic. "Integral Manifold Approach to Slow Adaptation," Report DC-80. University of Illinois. March 1985.
