Singular Optimal Control Problems
This is Volume 1I7 in MATHEMATICS 1N SCIENCE AND ENGINEERING A Series of Monographs...
12 downloads
182 Views
5MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
Singular Optimal Control Problems
This is Volume 1I7 in MATHEMATICS 1N SCIENCE AND ENGINEERING A Series of Monographs and Textbooks Edited by RICHARD BELLMAN, University of Southern California The complete listing of books in this series is available from the Publisher upon request.
Singular Optimal Control Problems
DAVID J. BELL
Department of Mathematics and Control Systems Centre University of Manchester Institute of Science and Technology Manchester, England
and
DAVID H. JACOBSON
National Research Institute for Mathematical Sciences Councilf o r Scientific and Industrial Research, South Africa (Honorary Professor in the Universiiy of the Witwatersrand)
1975
Academic Press London
: New York
: San Francisco
A Subsidiary of Harcourt Brace Jovanovich, Publishers
ACADEMIC PRESS INC. (LONDON) LTD. 24-28 Oval Road 1,ondon NWI
U . S . Edition published hi. ACADEMIC PRESS INC. I I I Fifth Avenue New York. New York 10003
Copyright 0 1975 by ACADEMiC PRESS INC. (LONDON) LTD.
All Righis Re.wr\<ed Y o part of this hook may be reproduced in any form by photostat, microfilm. or any other
means. without written permission from the publishers
Library of Congress Catalog Card Number. 75-21610 I S B N : 0 - 12-085060-5
I’rinlcd in (;rc;it Hritain hk Galliard (Pi-inter\) L.td. Great Yarrnouth
PREFACE The modern t h e o r y of o p t i m a l c o n t r o l h a s i t s b e g i n n i n g s i n t h e p i o n e e r i n g work of men s u c h a s Hohmann i n Europe and Goddard i n t h e U n i t e d S t a t e s . These men, and o t h e r s l i k e them, dreamt of t h e day when space t r a v e l would become a r e a l i t y .
I t i s t h e work o f
t h e s e p i o n e e r s which h a s h e l p e d t o b r i n g t h e i r dreams t o p a r t i a l f u l f i l l m e n t i n s u c h a b r i e f p e r i o d of t i m e . The o p t i m i z a t i o n problems of s p a c e f l i g h t dynamics such a s minimum f u e l and minimum t i m e were q u i c k l y r e a l i z e d t o b e problems i n t h e c a l c u l u s of v a r i a t i o n s .
It w a s
t h u s t h a t t h i s v e r y o l d and e s t a b l i s h e d b r a n c h of Mathematics r e c e i v e d y e t a n o t h e r new lease of l i f e , a p a t t e r n which h a s b e e n r e p e a t e d s i n c e i t s b i r t h i n t h e seventeenth century. Problems of h i g h performance a i r c r a f t became i m p o r t a n t n e a r t h e end of t h e Second World War.
Maximum
range of a i r c r a f t f o r a g i v e n q u a n t i t y of f u e l and minimum t i m e t o climb were t y p i c a l o p t i m i z a t i o n problems which a r o s e .
These t o o were c l e a r l y problems b e l o n g i n g
t o t h e same class as t h o s e from s p a c e dynamics a l t h o u g h c o m p l i c a t e d by t h e p r e s e n c e o f aerodynamic l i f t and d r a g . Much of t h e m a t h e m a t i c a l t h e o r y f o r such problems had a l r e a d y b e e n developed e a r l y i n t h e t w e n t i e t h c e n t u r y by P r o f e s s o r G. A. B l i s s and h i s s t u d e n t s a t t h e U n i v e r s i t y of Chicago.
I n p a r t i c u l a r , t h e optimiza-
t i o n of a i r c r a f t and s p a c e v e h i c l e f l i g h t p a t h s f o r which t h e c o n t r o l s are f i n i t e and bounded was a n a l y s e d V
PREFACE
vi
by a technique d e s c r i b e d i n a d i s s e r t a t i o n by F. A. V a l e n t i n e , a r e s e a r c h s t u d e n t of B l i s s , i n 1937. However, i n 1959 L. S. P o n t r y a g i n p r e s e n t e d h i s Maximum P r i n c i p l e which c o n s o l i d a t e d t h e t h e o r y f o r c o n s t r a i n e d problems. N e v e r t h e l e s s , i t soon became a p p a r e n t t h a t t h e mathematical t h e o r y a v a i l a b l e w a s n o t s u f f i c i e n t f o r c e r t a i n s p e c i a l c o n t r o l problems i n which t h e P o n t r y a g i n P r i n c i p l e y i e l d e d no a d d i t i o n a l i n f o r m a t i o n on t h e stationary control.
These problems were d e s c r i b e d as
s i n g u l a r problems and they have a r i s e n i n many engineeri n g a p p l i c a t i o n s i n f i e l d s o t h e r than Aerospace and a l s o more r e c e n t l y i n non-engineering a r e a s such as Economics.
I n t h e l a s t t e n t o twelve y e a r s much e f f o r t
has been p u t i n t o t h e development of new t h e o r y t o d e a l w i t h t h e s e s i n g u l a r problems.
F i r s t , new n e c e s s a r y
c o n d i t i o n s were found f o r s i n g u l a r e x t r e m a l s t o be c a n d i d a t e optimal a r c s .
Secondly, and more r e c e n t l y ,
new s u f f i c i e n t c o n d i t i o n s and n e c e s s a r y and s u f f i c i e n t c o n d i t i o n s have been found f o r such e x t r e m a l s t o be optimal , We f e e l t h a t t h e t i m e i s now r i g h t f o r t h e t h e o r y of s i n g u l a r problems t o b e c o l l e c t e d t o g e t h e r , s c a t t e r e d
a s i t i s i n numerous d i f f e r e n t j o u r n a l s , and p r e s e n t e d under one cover. volume.
T h i s i s t h e purpose of t h e p r e s e n t
We a r e p a r t i c u l a r l y p l e a s e d t h a t t h i s book
should t a k e i t s p l a c e a l o n g s i d e s o many well-acclaimed t e x t s i n Richard Bellman's s e r i e s which h a s proved i t s
PREFACE
worth many times over.
vii
Our gratitude goes to all o u r
colleagues and students, past and present, who have stimulated us both in our own researches. In particular, we would like to mention Y. C. Ho, D. Q. Mayne, J. L. Speyer, W. Vandervelde, D. F. Lawden, R. N. A. Plimmer
and B. S. Goh.
Finally, we acknowledge the help and
encouragement received from Academic Press during the preparation of this book and our special thanks go to Mrs. G. M. McEwen and Mrs. M. E. Hughes who typed the
manuscript.
D. J. BELL dune t 9 7 5
D. H. JACOBSON
This page intentionally left blank
CONTENTS
Page Preface
V
Chapter 2
An H i s t o r i c a l Survey o f S i n g u l a r C o n t r o l Problems
1.1 I n t r o d u c t i o n 1.2 S i n g u l a r C o n t r o l i n Space Navigation 1.3 Method o f Miele v i a Green's Theorem
1.4 1.5
1.6
-
Q u a d r a t i c Cost L i n e a r Systems Necessary Conditions for S i n g u l a r Optimal C o n t r o l 1 . 5 . 1 The Generalized LegendreClebsch Condition 1 . 5 . 2 The Jacobson Condition S u f f i c i e n t Conditions and Necessary and S u f f i c i e n t Conditions for Opt i m d i t y References
1
6 8
10
12 12
18 19
28
Chapter 2 Fundamental Concepts 2.1 2.2 2.3 2.4 2.5
Introduction The General Optimal C o n t r o l Problem The F i r s t V a r i a t i o n o f J The Second V a r i a t i o n of J A S i n g u l a r Control Problem References
Chapter
37 39 41 49 56 59
3
Necessary Conditions for S i n m a r Optimal C o n t r o l
3.1 I n t r o d u c t i o n
3.2
The G e n e r a l i z e d Legendre-Clebsch Condition 3.2.1 S p e c i a l C o n t r o l V a r i a t i o n s 3.2.2 A Transformation Approach ix
61 62 62
a1
CONTENTS
X
page
3.3 J a c o b s o n ' s Necessary Condition R e f e re nc e s
91 98
Chapter 4 S u f f i c i e n t Conditions and Necessary and S u f f i c i e n t Conditions f o r Non-Negativity of Nonsingular and S i n g u l a r Second Variations
4.1
Introduction 4.2 P r e l i m i n a r i e s 4.3 The Nonsingular Case 4 . 4 S t r o n g P o s i t i v i t y and t h e T o t a l l y S i n g u l a r Second V a r i a t i o n 4 . 5 A General Suffici-ency Theorem f o r t h e Second V a r i a t i o n 4 . 5 . 1 The Nonsingular S p e c i a l Case 4.5.2 The T o t a l l y S i n g u l a r S p e c i a l Case 4.6 Necessary and S u f f i c i e n t Conditions f o r Non-negativity of t h e T o t a l l y S i n g u l a r Second V a r i a t i o n 4.7 Necessary Conditions f o r O p t i m a l i t y 4.8 Other Necessary and S u f f i c i e n t Conditions 4.9 S u f f i c i e n t Conditions f c r a Weak L o c a l Minimum 4 . 1 C E x i s t e n c e Conditions f o r t h e Matrix R i c c a t i D i f f e r e n t i a l Equation 4.10.1 An Example 4.11 Conclusion References
101 103 106 112
115 117
119 122 137
140
140 142
146
148 149
Chapter 5 Computational Methods f o r S i n g u l a r Control Problems
5.1 5.2
I n t r o duct i on Computational A p p l i c a t i o n o f t h e S u f f i c i e n c y Conditions o f Theorems 4.2
153
CONTENTS
5.3
5.4 5.5
and 4.5 5.2.1 The Nonsingular Case 5.2.2 The T o t a l l y S i n g u l a r Case 5.2.3 A L i m i t Approach t o t h e Construct i o n of P ( * ) Computation o f Optimal S i n g u l a r C o n t r o l s 5.3.1 Preliminaries 5.3.2 An €-Algorithm 5.3.3 A G e n e r a l i z e d Gradient Method f o r S i n g u l a r Arcs 5.3.4 Function Space Quasi-Newton Methods 5.3.5 Outlook f o r F u t u r e J o i n i n g o f O p t i m a l S i n g u l a r and N on s i n g u l ar Sub- a r c s Conclusion References
xi
155
155 155
157 159 159
160
165
165 166
166 168 169
Chapter 6 Conclusion
6.1 6.2 6.3
6.4 6.5 6.6
The Importance of S i n g u l a r Optimal C o n t r o l Problems Necessary Conditions Necessary and S u f f i c i e n t Conditions Computational Methods Switching C o n d i t i o n s Outlook f o r t h e F u t u r e
AUTHOR INDEX SUBJECT INDEX
173 174 175 181 181 182
183 187
This page intentionally left blank
CHAPTER 1
An H i s t o r i c a l Survey of S i n g u l a r C o n t r o l Problems
1.1
Introduction It i s w e l l known t h a t t h e fundamental problem of
o p t i m a l c o n t r o l t h e o r y can b e formulated as a problem of Bolza, Mayer o r Lagrange.
These t h r e e f o r m u l a t i o n s
a r e q u i t e e q u i v a l e n t t o one a n o t h e r ( B l i s s , 1946).
We
s h a l l d e s c r i b e t h e Bolza problem s i n c e t h e a c c e s s o r y minimum problem, and t h e a s s o c i a t e d second v a r i a t i o n w i t h which w e s h a l l b e much concerned i n t h i s book, appears as such a problem. The problem of Bolza i n o p t i m a l c o n t r o l t h e o r y
i s t h e following.
Determine t h e c o n t r o l f u n c t i o n
u ( * ) which minimizes t h e c o s t f u n c t i o n a l
r tf
(1.1.1)
where t h e system e q u a t i o n i s
H
= f (X,U,t)
(1.1.2)
subject t o the constraints x ( t o > = xo
(1.1.3) (1.1.4)
Slhii;U!2.4R OPTIMAL CONTROL PROBLEMS
2
u(*> is a member of the set U, t a member (1.1.5)
of [to, tfl.
Here x is an n-dimensional state vector and u is an m-dimensional control vector.
The functions L and F
are scalar and the terminal constraint function $ is an s-dimensional column vector function of x(t ) at f The functions L, F and $ are assumed smooth. The tf' set U is defined by
u
E {u(-)
: u . ( * ) is piecewise continuous 1
...
i = l , 2, The initial time t
,ml.
in time,
(1.1.6)
i s given explicitly but the final
0
may be unspecified. f The Hamiltonian for this problem is
time t
and the following necessary conditions (Pontryagin's principle) hold along an optimal trajectory:
-
=
--
Hx (x,u,A,t)
(1.1.8)
1.
AN HISTORICAL S U R V E Y
A(tf)
=
Fx[x(tf)
, tfl
+ QXTV
3
(1.1.9)
where
-u =
arg min H(';;,u,X,t)
(1.1.11)
U
u(*) a member of U. Here
; ( a ) ,
u(-) denote the candidate state and control
functions respectively, A ( * )
denotes an n-dimensional
vector of Lagrange multiplier functions of time, and v is an s-dimensional vector of Lagrange multipliers
associated with JI. A singular minimizing arc for the problem of
Bolza is defined by Bliss (1946) as one for which the Legendre-Clebsch necessary condition is not satisfied with strict inequality.
For the optimal control
problem as formulated above the definition of Bliss is equivalent to the following. An extremal arc of the control problem is said to be singular if the m
x
m determinant det(H
along it.
) vanishes at any point uu Otherwise it is said to be nonsingular.
In particular, if the Hamiltonian H is linear in one or more elements of the control function then the extremal is singular (Goh, 1966b).
4
SINGULAR OPTIMAL CONTROL PROBLEMS
The following definitions are used in the sequel: Definition 1.1
Let u
k
be an optimal singular element
of the control vector u on the interval [tl, t2] which appears linearly in the Hamiltonian. time derivative of H derivative in which
Uk
\
Let the 2q th
be the lowest order total appears explicitly with a
coefficient which is not identically zero on [tl’ t2]. Then the integer q is called the order of the singular arc.
The control variable
%
is referred to as a
singular control. Definition 1.2
Assuming all the elements u1,u2, ...,%
of the control vector u are singular simultaneously then u is called a totally singular control function when HU(y,h,t)
=
(1.1.12)
0
for all t in [to, tfl. Definition 1.3
A partially singular control function
is one along which (1.1.12) holds for k subintervals of length Ti, i = 1, 2,
...
,k
and where
k
The concepts of total and partial singularity can be applied also to the accessory minimum problem in which the second variation of the cost functional
(1.1.1) is to be minimized (see Sections 4 . 4 and 6.1).
1.
AN HISTORICAL SURVEY
5
I n t h e important s u b c l a s s of optimal c o n t r o l problems known as r e l a x e d v a r i a t i o n a l problems d i s c u s s e d by S t e i n b e r g (1971) some of t h e elements of t h e c o n t r o l v e c t o r u appear l i n e a r l y and o t h e r s appear n o n l i n e a r l y . I n t h i s book we s h a l l be mainly concerned w i t h problems which are t o t a l l y s i n g u l a r f o r a l l
1 ( b u t see S e c t i o n 5 . 4 ) . I n t h e p a r t i a l l y f s i n g u l a r case t h e n o n s i n g u l a r c o n t r o l s t h a t are t i n [to, t
p r e s e n t can b e e l i m i n a t e d v i a well-developed nons i n g u l a r t h e o r y , l e a v i n g a problem t o t a l l y s i n g u l a r i n t h e remaining c o n t r o l v a r i a b l e s (Robbins, 1967).
The
s i n g u l a r c o n t r o l t h e o r y t o b e d i s c u s s e d has been developed f o r and motivated by e n g i n e e r i n g problems. N e v e r t h e l e s s , s i n g u l a r problems may a r i s e i n any d i s c i p l i n e where c o n t r o l t h e o r y i s a p p l i e d , evidence f o r which can b e s e e n i n economics (Dobell and Ho,
1967) and p r o d u c t i o n from n a t u r a l r e s o u r c e s (Goh, 1969/70).
I n t h e f o l l o w i n g c h a p t e r s n e c e s s a r y and
s u f f i c i e n t c o n d i t i o n s W i l l be developed f o r s i n g u l a r optimal c o n t r o l of systems governed by o r d i n a r y d i f f e r e n t i a l equations.
However, i n v e s t i g a t i o n i s
b e i n g c a r r i e d o u t i n t o s i n g u l a r c o n t r o l of d i s c r e t e t i m e systems (Tarn e t a l . , 1971; Graham and D'Souza,
1970) and of systems w i t h d e l a y (Soliman and Ray, 1972; Connor, 1974).
SINGULAR OPTIMAL CONTROL PROBLEMS
6
1.2
Singular Control in Space Navigation Practical problems involving singular controls
arose early in the study of optimal trajectories for space manoeuvres.
Trajectories for rocket propelled
vehicles in which the thrust magnitude is bounded exhibit singularity in the rate of fuel consumption. Lawden (1963) formulates the fundamental problem of space navigation in the following way.
Ox1x2x3 is an
inertial frame in which a space vehicle has position coordinates ( x l , x2, x3) and velocity components The rocket thrust has direc-
(vl, v2, v3) at time t. tion cosines ( E l ,
R3)
L!,
has components (gl, g,,
and the gravitational field
g3).
The mass rate of
propellant consumption, bounded above by some finite
-
constant m, is denoted by m.
If the rocket has an
exhaust velocity c and the mass of the vehicle is M then its equations of motion are
G.1
=
cmR./M 1 + gi(xl, x2, x3, t)
x. = v 1
i’
i
=
1, 2, 3
M = - m .
(1.2.1)
(1.2.2) (1.2.3)
The propellant consumption rate m must satisfy the inequality constraints (1.2.4)
1.
AN HISTORICAL S U R V E Y
7
whilst the direction cosines may be written in the form
R1
=
sine cos$,
where
R2 = sine sin$, R 3 =
case
(1.2.5)
e , + are spherical polar coordinates. It is
required to choose the control variables m(-), O ( . ) , $ ( * ) in order to minimize the fuel used in transfer-
ring the vehicle between two given positions at each of which the vehicle's velocity is specified. The final time tf may or may not be given explicitly. State variables x. ( - ) , vi(*), M(*) must satisfy 1 boundary condit ions x.
x.(t ) 1 0
=
Xi(tf)
= x
10'
if'
v.(t ) 1 0
=
v io'
v.(t ) 1 f
=
v if
M(to)
= Mo
(1.2.6) (1.2.7)
and the cost functional can be written as
In this problem the Hamiltonian is linear in the rate of fuel consumption and this control variable turns out to be a singular control. During the 1950's it was not known whether a given manoeuvre in space using a small, continuous thrust would be more economical in fuel than the previously accepted optimal procedure of impulsive boosts (Hohmann
SINGULAR OPTIMAL CONTROL PROBLEMS
8
transfers).
Some numerical r e s u l t s (Forbes, 1950)
suggested t h a t i n c e r t a i n circumstances l e s s f u e l would be used i n an o r b i t a l t r a n s f e r by f o l l o w i n g a s p i r a l p a t h u s i n g a s m a l l b u t continuous t h r u s t throughout t h e maneouvre t h a n by a Hohmann t r a n s f e r . Other a n a l y s i s (Lawden, 1950, 1952) appeared t o show t h a t the so-called intermediate-thrust a r c s (along which t h e f u e l e x p e n d i t u r e r a t e i s non-zero b u t less than
m)
were i n a d m i s s i b l e i n a f u e l optimal t r a j e c t o r y .
The t r u e s t a t u s of t h e i n t e r m e d i a t e - t h r u s t a r c s remained hidden u n t i l t h e mathematical t h e o r y had been f u r t h e r developed ( s e e S e c t i o n 1.5.1 and Chapter 3). Much of t h e e a r l y work i n a e r o s p a c e has been surveyed elsewhere ( B e l l , 1968). 1.3
Method of Miele v i a Green's Theorem A method which d e a l s s u c c e s s f u l l y w i t h problems
i n v o l v i n g s i n g u l a r c o n t r o l s i s due t o Miele (1950-51). It i s based upon a t r a n s f o r m a t i o n u s i n g Green's
theorem r e l a t i n g l i n e and s u r f a c e i n t e g r a l s .
As
developed by Miele t h e method i s a p p l i c a b l e o n l y t o a p a r t i c u l a r c l a s s of l i n e a r problems i n two dimensions. The problems are l i n e a r i n t h e s e n s e t h a t t h e c o s t f u n c t i o n a l and any i s o p e r i m e t r i c c o n s t r a i n t a r e l i n e a r i n t h e d e r i v a t i v e of t h e dependent v a r i a b l e . The g e n e r a l c o s t f u n c t i o n a l i n t h i s c l a s s of problems can be w r i t t e n as
1.
AN HISTORICAL SURVEY
9 ( 1.3.1)
J = ~~$(x,y) + $(x,y)dy/dxldx XO
where $ and
)I
are known functions of x and y.
An
admissible region in (x,y)-space is determined from the constraints in the problem formulation and the optimal trajectories within this region are determined by the sign of the function w(x,y)
=
a$ - a$ . ax
(1.3.2)
ay
If this function changes sign within the admissible region then a typical trajectory will contain two bang-bang arcs separated by a singular arc, the latter being defined by the equation w(x,y)
(1.3.3)
= 0.
Many problems in flight mechanics can be formulated in linear form of the above type provided suitable assumptions are made.
Such problems are
discussed by Miele (1955a, 1955b).
The general
theory of the extremization of linear integrals by Green's theorem for the two-dimensional case has been presented together with a number of applications to aerospace systems by Miele (1962).
The synthesis
of the optimal control by Miele's method including a demonstration of uniqueness of the singular control
SINGULAR OPTIMAL CONTROL PROBLEMS
10
and t h e d e t e r m i n a t i o n of t h e o p t i m a l s t r a t e g y h a s b e e n d i s c u s s e d by Hermes and Haynes (1963).
The g e n e r a l i z e d
S t o k e s ' s theorem of t h e e x t e r i o r c a l c u l u s h a s e n a b l e d Haynes (1966) t o e x t e n d Miele's method t o n dimensions.
1.4
L i n e a r Systems
-
Q u a d r a t i c Cost
Singular control f o r time-invariant,
linear
systems w i t h q u a d r a t i c c o s t f u n c t i o n , f r e e t e r m i n a l t i m e and f i x e d e n d p o i n t h a s b e e n s t u d i e d q u i t e extensively.
I n p a r t i c u l a r , Wonham and Johnson (1964) and
Bass and Webber (1965) h a v e s t u d i e d t h e s y s t e m
k
= Ax
+ Bu
i = 1, 2,
(1.4.1)
..., m
(1.4.2)
with cost function J =
itf J
T
x Qx d t
(1.4.3)
t0
under t h e a s s u m p t i o n s i ) u i s a s c a l a r , i i ) matrices A , B and Q are c o n s t a n t , i i i ) Q i s p o s i t i v e d e f i n i t e .
They have shown t h a t t h e o p t i m a l c o n t r o l ;
i s of t h e
form
-
T u = k x
on t h e s i n g u l a r s t r i p
(1.4.4)
AN HISTORICAL S U R V E Y
1.
lkTxl < 1
,
11
T k 1 x :0
where k and k, a r e c o n s t a n t v e c t o r s .
(1.4.5) The s i n g u l a r
c o n t r o l i s o p e r a t i v e w i t h i n t h e r e g i o n of s t a t e space d e f i n e d by t h i s s t r i p .
Johnson and Gibson (1963),
Athans and Canon (1964) have c a r r i e d o u t a s s o c i a t e d I n the l a t t e r reference a cost function i s
work.
taken i n t h e form
1
tf
J =
(1.4.6)
(k + I u ( t > l ) d t
t0
w i t h k a p o s i t i v e c o n s t a n t and u ( t ) a s c a l a r s a t i s f y ing trol
u[ < 1.
It i s found t h a t optimal s i n g u l a r con-
< 1 but n o t otherwise. exist i f k -
In a l l the
r e f e r e n c e s quoted above i n t h i s s e c t i o n t h e methods of a n a l y s i s a r e r a t h e r s p e c i a l and a r e n o t d i r e c t l y a p p l i c a b l e t o time-varying systems w i t h non-positive d e f i n i t e performance w e i g h t i n g m a t r i c e s . An e a r l y paper d i s c u s s i n g t h e time-varying problem w a s by Johnson (1965).
Assumptions i ) and
i i ) mentioned above were r e l a x e d i n a paper by Rohrer and S o b r a l (1966) whereas r e s u l t s o b t a i n e d by S i r i s e n a (1968) f o l l o w from a r e l a x a t i o n of i i i ) . An a n a l y s i s i n which a l l t h r e e assumptions were
r e l a x e d was given by Moore (1969) and Moylan and Moore (1971).
An e x c e l l e n t book which d i s c u s s e s t h e t o p i c s
of t h i s s e c t i o n i s t h e one by Anderson and Moore (1971).
SINGULAR OPTIMAL CONTROL PROBLEMS
12
1.5 1.5.1
Necessary Conditions for Singular Optimal Control The Generalized Legendre-Clebsch Condition In 1959 Leitmann suggested in private correspond-
ence that the intermediate thrust (IT) arcs arising from minimum fuel problems of space navigation may after all be candidate sub-arcs for optimal trajectories.
The first development following this sugges-
tion was the discovery of the form of these IT-arcs in two dimensions and in an inverse square law gravitational field (Lawden, 1961, 1962, 1963).
It
was found that if the time of transit is not predetermined then the IT-arc is a spiral with its pole at the centre of attraction, a trajectory which has since become known as Lawden’s spiral.
The status of
general IT-arcs in a central, time-invariant force field is given by Archenti and Vinh (1973). The difficulty at this stage of the investigations was that the classical Legendre-Clebsch condition
a
-H au u
-> O
(1.5.1)
which yields further information for nonsingular problems is trivially satisfied along a totally singular arc.
More and more researchers thus became
interested in these problems of singular control (e.g. Snow, 1964; Hermes, 1964) and particularly in the search for further necessary conditions which would help to establish the status of singular trajec-
1.
AN HISTORICAL SURVEY
tories such as Lawden's spiral.
13
In order to deduce
new conditions for optimality it was necessary to study the second variation of the cost functional which must be non-negative for a minimum value of that functional.
In certain cases it is possible to use
the second variation directly in order to establish the optimality or otherwise of both nonsingular (Bell, 1965) and singular arcs (Bell, 1971a).
However, the
main effort during the last decade has been to deduce new necessary, sufficient, and necessary and sufficient conditions for singular problems from the second variation.
A full discussion of the second variation
will be found in Chapters 2 and 4 . When the control is a scalar, Kelley (1964a) deduced a new necessary condition for singular optimal control by studying the second variation under a special control variation.
Kelley's method was
generalized by Tait (1965),
Kopp and Moyer (1965),
Kelley et al. (1967) to give what has since become known as the generalized Legendre-Clebsch condition (Kelley-Contensou test):-
(1.5.2)
where the integer q is the order of the singular problem.
SINGULAR OPTIMAL CONTROL PROBLEMS
14
The generalized Legendre-Clebsch condition for a vector control was obtained by Robbins (1967) and Goh (1966b).
In this case the controls can appear in an
odd time derivative of H
U
but if this does occur then
there necessarily exists a control u in U such that the second variation is negative (for a minimization problem).
Hence, the generalized Legendre-Clebsch
condition f o r vector control is
p odd
(1.5.3)
and (-1)q
ad2q H
au dt2q u
-> 0
for all t in (1.5.4)
assuming that the matrices H H fx and fU are xx’ ux’ sufficiently differentiable with respect to time. For p
=
1, eqn(1.5.3)
implies Huxfu is symmetric, a
powerful necessary condition in the case of vector controls (Speyer, 1973).
On the other hand, eqn
(1.5.3) is trivially satisfied when u is a scalar. Incidentally, the result of Speyer (1973) is contrary to Schultz and Zagalsky (1972) but consistent with the findings of Bryson et al. (1969). The generalized Legendre-Clebsch condition or its equivalent was used by Robbins (1965) and by Kopp and
1.
AN HISTORICAL S U R V E Y
15
Moyer (1965) to test the optimality of Lawden's singular spiral, a problem of order q condition was not satisfied and
so
= 2.
The new
the spiral cannot
form part of a minimum fuel trajectory.
The same
result was obtained by other workers using a special variation in the control (Keller, 1964) and by canonical transformations of the Hamiltonian (Fraeijs de Veubeke, 1965). An alternative approach to Singular problems was by a transformation on the state and control variables (Kelley 1964b; Speyer and Jacobson, 1971).
IJnder this
transformation a state space of reduced dimension i s obtained and the singular problem (hopefully) changed to a nonsingular one in which the classical LegendreClebsch condition can be applied.
If this is not the
case then a further transformation to a state space of even smaller dimension is indicated. This method was used to investigate Lawden's spiral (Kelley, 1963). A generalization of Kelley's transformation has been
given by Mayne (1970). The general theory for both the generalized Legendre-Clebsch condition and Kelley's transformation has been given by Kelley et al. (1967) including the application of these theories to Lawden's spiral.
In
this last reference (amongst others) a plausibility argument was put forward concerning the satisfaction of terminal conditions associated with the second
16
SINGULAR OPTIMAL CONTROL PROBLEMS
variation.
It was suggested that the special control
variations used in the derivation of the generalized Legendre-Clebsch condition could be altered by the addition of appropriate functions of time.
Such
additional functions would be of an order of magnitude which allowed the terminal conditions to be satisfied exactly but would not affect the sign of the second variation. However, it is sometimes possible to generate special state variations which satisfy both the equations of variation and the terminal conditions. This has been done in the case of Lawden's spiral (Bell, 1971b). The existence of sufficient control variations to enable one to cancel out undesirable low order effects of the special control variations, as discussed above,
is equivalent to a normality assumption. No such assumption is necessary in the Pontryagin Principle and recently this Principle has been extended t o a so-called High Order Maximal Principle (Krener, 1973). This new Principle includes the generalized LegendreClebsch and Jacobson (see Section 1.5.2 below) conditions when they apply, together with other conditions, without the assumption of normality. The difficulty caused by the fact that singular extremals may be transformed into singular extremals under a single application of Kelley's transformation was investigated by Goh (1966a) using a procedure by which the generalized Legendre-Clebsch condition for
1.
A N HISTORICAL S U R V E Y
17
singular extremals can be deduced and, unlike Kelley's transformation, retains the full dimensionality of the original problem.
It was again proved that Lawden's
spiral is non-optimal and furthermore IT-arcs are candidates for an optimal trajectory only if the thrust possesses a component which is inwardly directed to the centre of attraction, a result which had already been noted by Robbins (1965).
The transformed accessory
minimum problem discussed by Goh was also used to further develop the generalized Legendre-Clebsch necessary condition for vector control problems (Goh, 1966b).
This new set of conditions was applied to
several problems including a doubly singular problem in interplanetary guidance (Breakwell, 1965) and to a class of identically singular problems, certain members of which had already been studied by Haynes (1966) using an extension of Miele's method discussed above in Section 1.3.
The methods of Kelley and Goh
have also been applied to a singular arc arising from the problem of a stirred tank reactor in the field of Process Control (Bell, 1969). A number of investigations into singular problems
have also been carried out by Russian workers (Bolonkin, 1969; Gabasov, 1968, 1969; Gurman, 1967; Vapnyarskiy, 1966, 1967).
In particular, Gabasov
(1968, 1969) obtained a slightly stronger version of a second necessary condition which is discussed in the following section. A comparison of the results of
SINGULAR OPTIMAL CONTROL PROBLEMS
18
Vapnyarskiy (1967) and Bolonkin (1969) with those of Goh (1966b) is given by Goh (1973). Once the generalized Legendre-Clebsch condition had been established it was possible to lay down a procedure for the derivation of the singular extremals. This was done for the linear, time-varying system with quadratic cost functional by Goh (1967).
1.5.2
The Jacobson Condition
Having established the generalized LegendreClebsch condition researchers began to look for generalizations of the known sufficient conditions for the nonsingular problem.
However, before such a
generalization was found, a new necessary condition for non-negativity of the singular second variation, not equivalent to the generalized Legendre-Clebsch condition, was derived (Jacobson, 1969, 1970b).
This
new condition, known as Jacobson's condition, is as f0llows
.
H
f
ux u
+ f TQfu > 0 U
(1.5.5)
where T - Q = Hxx + fx Q + Qf,
(1.5.6)
(1.5.7)
1.
A N HISTORICAL SURVEY
and the partial derivatives fu, H
and f are all
x(-),-u( - > . UX
evaluated along the singular arc
19 X
A
strong version of this condition is given by Mayne (1973). In the classical calculus of variations problem the Jacobson condition can be replaced by an equivalent necessary condition on the terminal point of the singular extremaf (Goh, B.S.
On Jacobson's Necessary
Condition for Singular Extremals. Note).
Unpublished Research
A generalization of this end condition leads
to a result originally given by Mancill (1950).
It has
been shown by Jacobson (1970a) that in general the generalized Legendre-Clebsch condition and the Jacobson condition together are not sufficient for optima1ity 1.6
.
Sufficient Conditions and Necessary and Sufficient Conditions for Optimality A sufficient condition for a weak local minimum
in a nonsingular problem is that the second variation be strongly positive (Gelfand and Fomin, 1963).
This
condition gives rise to a well known matrix Riccati differential equation.
In singular problems the
second variation cannot be strongly positive (Tait, 1965; Johansen, 1966) but investigations have shown that Riccati-type conditions do exist for the singular case.
Jacobson (1970b), using a direct approach,
derived sufficient conditions for the second variation
SINGULAR OPTIMAL CONTROL PROBLEMS
20
to be non-negative in both singular and nonsingular control problems.
These conditions are that there
exist a real symmetric bounded, matrix function of time P(*) such that H
UX
+ f T~ U
P + H xx
=
+
o
for all t in [to, tfl
(1.6.1)
TP + Pf > 0 fx x-
for all t in [t
(1.6.2)
0’
and (1.6.3) where Z is the n
x
(n-s) matrix
(1.6.4)
and the s
x
s matrix D 1 and the s
x
(n-s) matrix D 2
are such that
The above conditions in strengthened form are sufficient conditions for a weak relative minimun.
It is
demonstrated that the two necessary conditions for
1.
A N HISTORICAL SURVEY
21
singular optimal control, discussed in Section 1.5 above, are implied by the new conditions for the case of totally singular control and unconstrained terminal state. Related to these sufficient conditions of Jacobson, with the matrix $x the null matrix, are conditions found in the network theory literature in connection with passivity of electrical networks; see
Rohrer and Sobral (1966) , Rohrer (1968) , Silverman
(1968), Anderson and Moore (1968a, 1968b), Moore and Anderson (1968).
The sufficient conditions (1.6.1-3)
are very similar in form to certain necessary and sufficient conditions for positive real matrices (Anderson, 1967) which suggested that perhaps Jacobson's conditions may also be necessary.
An
alternative approach to the derivation of sufficient conditions for non-negativity of the second variation, in which the generalized Legendre-Clebsch condition (1.5.2) with q and $
=
1 is satisfied with strict inequality
0, is by the transformation method of Goh or
Kelley. The approach used by Goh transforms the singular accessory minimum problem (AMP) into a nonsingular, non-classical one.
The transformed problem is non-
classical in that the control variations appear in the boundary conditions and outside of the integral in the second variation.
Nevertheless, for certain classes
of totally singular problems the transformed AMP has been used by McDanell and Powers (1970) to develop a
SINGULAR OPTIMAL CONTROL PROBLEMS
22
sufficient condition which includes a Jecobi-type condition in the form of a matrix Riccati equation. A weakened form of this sufficiency condition is shown
to be necessary for a smaller class of singular problems.
The results of McDanell and Powers have
been generalized by Goh (19.70) and Speyer and Jacobson
(1971). The transformation proposed by Kelley (1964b) has been applied to the AMP of a totally singular optimal control problem (Speyer and Jacobson, 1971).
The
transformation can be obtained in closed form and the resulting problem is nonsingular.
The generalized
Legendre-Clebsch condition and Jacobson's necessary condition (for the unconstrained problem) are identified without the need for special variations.
The
results obtained are similar to those of Goh, McDanell and Powers mentioned above.
Eqns(1.6.1-3)
in strong
form (strict inequality) and with unconstrained terminal state ($ :0, Z = I), applied to the transformed AMP, are equivalent to the generalized Legendre-Clebsch condition and to the Jacobi condition (matrix Riccati equation) for the transformed nonsingular problem (Jacobson, 1970~). The sufficiency conditions of Jacobson are thus seen to be both necessary and sufficient.
Since the transformation is nonsingular
and Jacobson's conditions are coordinate independent these conditions are necessary and sufficient for optimality of singular problems in the original state
1.
space.
AN HISTORICAL S U R V E Y
23
This result is true for first-order arcs and
for problems with or without terminal state constraints.
If the problem is singular of order higher
than one then the reduction of the state space to achieve a nonsingular problem requires repeated application of the transformation technique.
This is
cumbersome especially if there are multiple control variables.
Furthermore, the method requires the
coefficients of the time dependent variation to be many times differentiable. Jacobsonts sufficiency conditions can also be shown to be necessary by a limit approach (Jacobson
and Speyer, 1971) which avoids the above difficulties. Here the singular second variation is made nonsingular by the addition of a quadratic functional of the controls :
1 2E 1 By allowing
E -+
6 ~ dt. ~ 6 ~ the optimality conditions for the
singular problem are deduced from the limiting optimality conditions of the synthesized nonsingular second variation.
This limit approach had been used previously
(Jacobson et al., 1970) as a computation technique. The sufficiency conditions of Jacobson (1970b) are again shown to be necessary for non-negativity of the singular variation.
Indeed, these conditions are
shown to be necessary and sufficient for a weak
24
SINGULAR OPTIMAL CONTROL PROBLEMS
minimum for a class of singular control problems and in certain cases a strong minimum is implied.
In
this limit approach a direct proof of necessary and sufficient conditions for optimality is obtained without the need to transform the problem to a reduced state space, Moreover, the concept of order of the singular arc is not required in the proof and differentiability requirements are not as severe as those demanded by Speyer and Jacobson (1971).
In particular,
the strong generalized Legendre-Clebsch condition is relaxed and a slightly more abstract version of condition (1.6.3)
and condition (1.6.6)
below is obtained.
This set of conditions is considered in Section 4.6, Chapter 4 (Theorem 4 . 5 ) .
They have been extended to
the partially singular case by Anderson (1973). Following on immediately from the results discussed in the last two paragraphs, a general sufficiency theorem for non-negativity of a large class of second variations was presented by Jacobson (1971a) for the partially singular case.
It is that there should exist
3 a continuously differentiable, f symmetric, matrix function of time P(*) such that for all t in [t
0’
t
+ Hxx + Pfx + fxTP H
ux
T
+ fu’P
H + Pfu xu H uu
1.
AN HISTORICAL SURVEY
25
for all t in [to, tf], together with (1.6.3-5).
A
proof of this condition in the case of unconstrained terminal states is given in Section 4.5 of Chapter 4 (Theorem 4 . 2 ) .
Jacobson then applied this condition
to both the totally singular case and to the nonsingular case.
Sufficient conditions were thus
obtained for the two special cases, demonstrating that both singular and nonsingular second variations can be treated in a common general framework.
The results
developed by Jacobson and Speyer through the transformation approach and the limit approach lead to the conclusion that the sufficiency conditions are also necessary for optimality for a large class o f problems.
In the nonsingular case the well known Riccati differential equation emerges and, since this is known to be a necessary condition for the nonsingular second variation, this implies that the sufficiency conditions of the theorem are also necessary.
In the
singular case the algebraic and differential inequalities (1.6.1-3)
in strengthened form are
obtained and these have been proved necessary and sufficient conditions for optimality in the 1971 papers of Jacobson and Speyer. Thus, in the singular case, the sufficiency conditions are also necessary. Assuming that the generalized Legendre-Clebsch condition is satisfied with strict inequality, a differential equation of Riccati type is obtained which implies and is implied by (1.6.1-3);
see Section 4.5, Chapter 4
SINGULAR OPTIMAL CONTROL PROBLEMS
26
(Theorem 4 . 4 )
.
Further results arising from the investigations of Jacobson and Speyer are obtained for a class of quadratic minimization problems whose optimal control functions are partially singular (Jacobson, 1 9 7 2 ) .
Ap
explicit expression for the time-varying, singular surface (hyperplane) is obtained.
Sufficient condi-
tions for totally bang-bang arcs and for totally singular arcs are matched for the case of a firstorder singular problem to yield sufficient conditions for optimality of partially singular problems.
The
investigations do not include terminal constraints but do allow non-positive definite, time-varying, weighting matrices in the performance criterion and fixed terminal time.
They therefore contribute to the under-
standing of problems discussed in Section 1.4 above. An important paper by Moyer ( 1 9 7 3 ) derives the conditions guaranteeing that a singular extremal which joins fixed end points provides a strong minimum for the independent time variable.
Under proper assump-
tions this paper extends the Weierstrass concept of a field.
Another interesting method for singular
problems has been suggested by Vinter ( 1 9 7 4 ) .
Instead
of modifying the cost functional as is done by
Jacobson and Speyer ( 1 9 7 1 ) an alternative approach is to modify the control constraint set.
The method is
applied to a class of linear systems and again the solution of the modified problem approximates the
1.
AN HISTORICAL S U R V E Y
27
solution of the original problem to any required accuracy.
Sufficient conditions for the singular
problem have also been derived by Mayne (1973) using differential dynamic programming. The mathematical characterization of optimal controls which contain both singular and nonsingular arcs is still not complete.
Preliminary results on
the matching of singular and nonsingular arcs at junction points were obtained by Kelley et al., (1967) and by Johnson (1965).
Further work clarified and
extended these results (McDanell and Powers, 1 9 7 1 ) . The main result of this latter paper is that the sum of order of the singular arc and lowest order time derivative of control which is discontinuous at the junction must be an odd integer when the strengthened Legendre-Clebsch condition is satisfied.
New neces-
sary conditions which do not require an analyticity assumption are developed.
These aid in character-
izing problems which may possess non-analytic junctions.
A l s o , McDanell and Powers (1971) have shown
that a continuous junction for a singular arc of odd order is realizable contrary to a result given by Kelley et al., (1967).
Results from two previous
papers (McDanell and Powers, 1 9 7 0 , 1971) are used to investigate the local switching behaviour of the singular guidance problem associated with the Saturn VAS-502 flight (Powers and McDanell, 1971).
A
numerical investigation is carried out to obtain a
28
SlNGULAR OPTIMAL CONTROL PROBLEMS
satisfactory suboptimal guidance scheme using the computational procedure suggested by Jacobson et al. (1970). Outlines of the methods of proof of the necessary and sufficient conditions mentioned above are given in the survey paper by Jacobson (1971b). Other survey papers on the singular control problem have been written by Gabasov et al. (1971) and Gabasov and Kirillova (1972), in which more references to the Russian literature will be found, and by Marchal (1973) which discusses the two different meanings of the term 'chattering' in optimization theory. References Anderson, B. D. 0. (1967). A System Theory Criterion 5, for Positive Real Matrices, SIAM J. Control 171- 182. Anderson, B. D. 0. (1973). Partially Singular LinearQuadratic Control Problems, IEEE Trans. autom. Control AC-18, - 407-409. Anderson, B. D. 0. and Moore, J . B. (1968a). Network Realizations of Time-Varying Passive Impedances, Tech. Rep. EE-6810, Univ. Newcastle, Australia. Anderson, B. D. 0. and Moore, J. B. (1968b). sions of Quadratic Minimization Theory. Infinite Time Results, Int. J. Control:, 473- 480.
ExtenI1
Anderson, B. D. 0. and Moore, J. B. (1971). "Linear Optimal Control", Prentice-Hall, Englewood Cliffs, N.J. Archenti, A. R. and Vinh, N. X. (1973). IntermediateThrust Arcs and their Optimality in a Central, Time-Invariant Force Field, J. Opt. Th. Applic. 11, 293-304. -
I.
AN HISTORICAL S U R V E Y
29
A t h a n s , M. and Canon, M. D. (1964). On t h e FuelOptimal S i n g u l a r C o n t r o l of N o n l i n e a r SecondOrder Systems, IEEE T r a n s . autom. C o n t r o l AC-9, 360-370.
-
Bass, R. W. and Webber, R. F. (1965). On S y n t h e s i s of Optimal Bang-Bang Feedback C o n t r o l Systems w i t h Q u a d r a t i c Performance I n d e x , P r o c . 6 t h JACC, 213-219. B e l l , D. J . (1965). Optimal T r a j e c t o r i e s and t h e 16 A c c e s s o r y Minimum Problem, Aeronaut. Q. -' 205- 2 20.
B e l l , D. J . (1968). Optimal Space T r a j e c t o r i e s - A Review of P u b l i s h e d Work, Aero. J1. R. a e r o n a u t . SOC. 72, 141-146.
-
B e l l , D. J. (1969). S i n -g u l a r Extremals i n t h e C o n t r o l of a S t i r r e d R e a c t o r , Chem. Engng. S c i . 24,
521-525.
B e l l , D. J. (1971a). The Second V a r i a t i o n and S i n g u l a r Space T r a j e c t o r i e s , I n t . J. C o n t r o l 14, 69 7- 703.
-
B e l l , D. J . (1971b). The Non-Optimality of Lawden's S p i r a l , A s t r o n a u t i c a Acta 1 6 , 317-324.
-
B l i s s , G. A. (1946). " L e c t u r e s on t h e C a l c u l u s of V a r i a t i o n s " . Univ. Chicago P r e s s , Chicago.
B o l o n k i n , A. A. (1969). S p e c i a l Extrema i n Optimal C o n t r o l Problems, Engng. Cybern. No. 2 , 170-183. B r e a k w e l l , J. V. (1965). A Doubly S i n g u l a r Problem i n Optimal I n t e r p l a n e t a r y Guidance, SIAM J . Control 3, 71-77. Bryson, A. E. and Ho, Y. C. (1969). "Applied Optimal Control". B l a i s d e l l , Waltham, Mass. Bryson, A. E . , Desai, M. N. and Hoffman, W. C. (1969). Energy-State Approximation i n Performance O p t i m i z a t i o n of S u p e r s o n i c A i r c r a f t , J. A i r c r a f t 6 , 481-488.
-
30
SINGULAR OPTIMAL CONTROL PROBLEMS
Connor, M. A. (1974). Singular Control of Delay Systems, J. Opt. Th. Applic. 13, 538-544. Dobell, A. R. and Ho, Y. C. (1967). Optimal Investment Policy: An Example of a Control Problem in Economic Theory, IEEE Trans. autom. Control AC-12, - 4-14.
Forbes, G. F. (1950). The Trajectory of a Powered Rocket in Space, J. Br. Interplanet. SOC. 9, 75-79.
Fraeijs de Veubeke, B. (1965). Canonical Transformations and the Thrust-Coast-Thrust Optimal 11, 271-282. Transfer Problem, Astronautica Acta Gabasov, R, (1968). Necessary Conditions for Optimality of Singular Control, Engng. Cybern. No.5, 28-37. Gabasov, R. (1969). On the Theory of Necessary Optimality Conditions Governing Special Controls, 13, 1094-1095. Sov. Phys.-Dokl. Gabasov, R. and Kirillova, F. M. (1972). High Order Necessary Conditions for Optimality, SIAM J. Control 10, 127-168. Gabasov, R., Kirillova, F. M. and Strochko, V. A. (1971). Conditions for High-Order Optimality (Review), Automn remote Control 32, 689-704, 857-875, 1013-1040. Gelfand, I. M. and Fomin, S. V. (1963). "Calculus of Variations". Prentice-Hall, Englewood Cliffs, N. J. Goh, B. S. (1966a). The Second Variation for the 4, Singular Bolza Problem, SIAM J. Control 309-325. Goh. B. S. (1966b). Necessary Conditions for Singular Extremals Involving Multiple Control Variables, SIAM J. Control 4, 716-731. Goh, B. S. (1967). Optimal Singular Control for MultiInput Linear Systems, J. math. Analysis Applic. 20, - 534-539.
1.
AN HISTORICAL SURVEY
31
Goh, B. S. (1969/70). Optimal Control of a Fish Resource, Malay. Scientist 5, 65-70. Goh, B. S. (1970). A Theory of the Second Variation in Optimal Control, unpublished report, Division of Applied Mechanics, Univ. California, Berkeley. Goh, B. S. (1973). Compact Forms of the Generalized Legendre Conditions and the Derivation of Singular Extremals, Proc. 6th Hawaii International Conference on System Sciences, 115-117. Graham, J. W. and D'Souza, A. F. (1970). Singular Optimal Control of Discrete-Time Systems, Proc. 11th JACC, 320-328. Gurman, V. I. (1967). Method of Multiple Maxima and the Conditions of Relative Optimality of Degenerate Regimes, Automn remote Control 28, 1845-1852. Haynes, G. W. (1966). On the Optimality of a Totally Singular Vector Control: An Extension of the Green's Theorem Approach to Higher Dimensions, 4, 662-677. SIAM J. Control Hermes, H. (1964). Controllability and the Singular Problem, SIAM J. Control 2, 241-260.
-
Hermes, H. and Haynes, G. W. (1963). On the Nonlinear Control Problem with Control Appearing Linearly, SIAM J. Control 1, 85-108. Jacobson, D. H. (1969). A New Necessary Condition of Optimality for Singular Control Problems, SIAM J. Control 7, 578-595.
-
Jacobson, D. H. (1970a). On Conditions of Optimality for Singular Control Problems, IEEE Trans. autom. - 109-110. Control AC-15, Jacobson, D. H. (1970b). Sufficient Conditions for Nonnegativity of the Second Variation in Singular and Nonsingular Control Problems, SIAM J. Control 8, - 403-423.
32
SINGULAR OPTIMAL CONTROL PROBLEMS
Jacobson, D. H. (1970~). New Conditions for Boundedness of the Solution of a Matrix Riccati Differential Equation, 3. Differential Equations 8, 258-263. Jacobson, D. H. (1971a). A General Sufficiency Theorem for the Second Variation, J. math. Analysis Applic. 34, 578-589. Totally Singular Quadratic Jacobson, D. H. (1971b). Minimization Problems, IEEE Trans. autom. Control AC-16, - 651-658. Jacobson, D. H. (1972). On Singular Arcs and Surfaces in a Class of Quadratic Minimization Problems, 37, 185-201. J. math. Analysis Applic. Jacobson, D. H. and Speyer, J. L. (1971). Necessary and Sufficient Conditions for Optimality for Singular Control Problems: A Limit Approach, J. math. Analysis Applic. 34, 239-266. Jacobson, D. H., Gershwin, S . B. and Lele, M. M. (1970). Computation of Optimal Singular Controls, IEEE Trans. autom. Control AC-15, 67-73. Johansen, D. E. (1966). Convergence Properties of the Method of Gradients, in "Advances in Control Systems" (C. T. Leondes, ed) Vo1.4, pp.279-316. Academic Press, New York and London. Johnson, C. D. (1965). Singular Solutions in Optimal Control Problems, in "Advances in Control Systems" (C. T. Leondes, ed) V01.2, pp.209-267. Academic Press, New York and London. Johnson, C. D. and Gibson, J. E. (1963). Singular Solutions in Problems of Optimal Control, IEEE - 4-15. Trans. autom. Control AC-8, Keller, J. L. (1964). On Minimum Propellant Paths for Thrust Limited Rockets, Astronautica Acta -910 262-269. Kelley, H. J. (1963). Singular Extremals in Lawden's 1, Problem of Optimal Rocket Flight, AIAA J. 1578- 1580.
1.
AN HISTORICAL SURVEY
33
Kelley, H. J. (1964a). A Second Variation Test for Singular Extremals, AIAA J. 2, 1380-1382. Kelley, H. J. (1964b). A Transformation Approach to Singular Subarcs in Optimal Trajectory and 2, 234-240. Control Problems, SIAM J. Control Kelley, H. J., Kopp, R. E. and Moyer, H. G. (1967). Singular Extremals, in "Topics in Optimization" (G. Leitmann, ed) pp.63-101. Academic Press, New York. Kopp, R. E. and Moyer, H. G. (1965). Necessary Con3 ditions for Singular Extremals, AIAA J. -' 1439-1444. Krener, A. J. (1973). The High Order Maximal Principle, in "Geometric Methods in Systems Theory" (D. Q. Mayne and R. W. Brockett, eds) pp.174-184. NATO Advanced Studies Institute Series: Mathematics and Physics. Lawden-, D. F. (1950). Note on a Paper by G. F. Forbes, J. Br. Interplanet. SOC. 2, 230-234. Lawden, D. F. (1952). Inter-Orbital Transfer of a Rocket, Annual Report, Br. Interplanet. Soc., 321-333. Lawden, D. F. (1961). Optimal Powered Arcs in an Inverse Square Law Field, J. Am. Rocket Soc. 31, 566-568. Lawden, D. F. (1962). Optimal Intermediate-Thrust Arcs 8, in a Gravitational Field, Astronautica Acta 106-123 . Lawden, D. F. (1963). "Optimal Trajectories for Space Navigation", Butterworth, Washington, D.C. Mancill, J. D. (1950). Identically Non-Regular Problems in the Calculus of Variations, Revista Matematica Y Fisica Teorica, Ser. A 7, 131-139. Marchal, C. (1973). Chattering Arcs and Chattering Controls, J. Opt. Th. Applic. 11,441-468.
34
S I N G U L 4 R OPTIMAL CONTROL P R O B L E M S
Mayne, D. Q. (1970). Sufficient Conditions for Optimality for Singular Control Problems, Report 18/70, C.C.D., Imperial College, London. Mayne, D. Q. (1973). Differential Dynamic Progranr ming - A Unified Approach to the Optimization of Dynamic Systems, in "Advances in Control Systems (C. T. Leondes, ed) Vol.10, pp.179-254. Academic Press, New York and London. McDanell, J . P. and Powers, W. F. (1970). New JacobiType Necessary and Sufficient Conditions for 8, Singular Optimization Problems, AIAA J. 1416-1420. McDanell, J . P. and Powers, W. F. (1971). Necessary Conditions for Joining Optimal Singular and Nonsingular Subarcs, SIAM J. Control 9, 161-173. Miele, A. (1950-51). Problemi di Minim0 Tempo nel Volo Non-Stazionario degli Aeroplani, Atti Accad. Sci. Torino 85, 41-52. Miele, A . (1955a). General Solutions of Optimum Problems in Nonstationary Flight, NACA Tech. Memo. No. 1388. Mizle, A . (1955b). Optimum Flight Paths of Turbojet Aircraft, NACA Tech. Memo. No.1389. Miele, A. (1962). Extremization of Linear Integrals by Green's Theorem, in "Optimization Techniques" (G. Leitmann, ed) pp. 69-98. Academic Press, New York. Moore, J. B. (1969). A Note on a Singular Optimal 5 , 857-858. Control Problem, Automatica Moore, J . B. and Anderson, B. D. 0. (1968). Extensions of Quadratic Minimization Theory. I Finite 7, 465-472. Time Results, Int. J. Control Foyer, H. G. (1973). Sufficient Conditions for a Strong Minimum in Singular Control Problems, 11, 620-636. SIAM J. Control -
1.
AN HISTORICAL S U R V E Y
35
Moylan, P. J. and Moore, J. B. (1971). Generalizations 7, of Singular Optimal Control Theory, Automatica 591-598. Powers, W. F. and McDanell, J. P. (1971). Switching Conditions and a Synthesis Technique for the Singular Saturn Guidance Problem, J. Spacecraft Rockets 8, 1027-1032. Robbins, H. M. (1965). Optimality of IntermediateThrust Arcs of Rocket Trajectories, AIAA J. 3, 1094-1098. Robbins, H. M. (1967). A Generalized LegendreClebsch Condition for the Singular Cases of 3, 361-372. Optimal Control, IBM J1 Res. Dev. Rohrer, R. A. (1968). Lumped Network Passivity Criteria, IEEE Trans. Circuit Theory CT-15, - 24-30. Rohrer, R. A. and Sobral, M. (1966). Optimal Singu1ar Solutions for Linear Multi-Input Systems, Trans. 88, 323-328. ASME, J. Basic Eng. Schultz, R. L. and Zagalsky, N. R. (1972). Aircraft 9, 108-114. Performance Optimization, J. Aircraft Silverman, L. M. (1968). Synthesis of Impulse Response Matrices by Internally Stable and Passive Realizations, IEEE Trans. Circuit Theory CT-15 -’ 238-245. Sirisena, H. R. (1968). Optimal Control of Saturating Linear Plants for Quadratic Performance Indices, Int. J. Control 8, 65-87. Snow, D. R. (1964). Singular Optimal Controis for a Class of Minimum Effort Problems, SIAM J. Control -2, 203-219. Soliman, M. A. and Ray, W. H. (1972). On the Optimal Control of Systems having Pure Time Delays and Singular Arcs. I Some Necessary Conditions f o r Optimality, Int. J. Control 16, 963-976. Speyer, J. L. (1973). On the Fuel Optimality of 10, 763-765. Cruise, J. Aircraft -
36
SINGULAR OPTIMAL CONTROL PROBLEMS
Speyer, J. L. and Jacobson, D. H. (1971). Necessary and Sufficient Conditions for Optimality for Singular Control Problems: A Transformation 33, 163-187. Approach, J. math. Analysis Applic. Steinberg, A. M. (1971). On Relaxed Control and Singular Solutions, J. Opt. Th. Applic. 8, 441-453. Tait, K. S. (1965). Singular Problems in Optimal Control, Ph.D. dissertation, Harvard Univ., Cambridge, Mass. Tarn, T. J., Rao, S. K. and Zaborszky, J. (1971). Singular Control of Linear-Discrete Systems, IEEE Trans. autom. Control AC-16, - 401-410. Vapnyarskiy, I. B. (1966). Solution of Some Optimal Control Problems, Engng. Cybern. No.4, 37-43. Vapnyarskiy, I. B. (1967). An Existence Theorem for Optimal Control in the Bolza Problems, some of its Applications and the Necessary Conditions for the Optimality of Moving and Singular Systems, USSR Comp. Math., Math. Phys. 7, 22-54. Vinter, R. B. (1974). Approximate Solution of a Class of Singular Control Problems, J. Opt. Th. Applic. 13, 461-483. Wonham, W. M. and Johnson, C. D. (1964). Optimal Bang-Bang Control with Quadratic Performance 86, 107-115. Index, Trans. ASME, J. Basic Eng. -
CHAPTER 2 Fundamental Concepts 2.1
Introduction A s has been d e s c r i b e d i n Chapter 1 t h e r e w a s no
s t a n d a r d mathematical procedure f o r a n a l y s i n g s i n g u l a r c o n t r o l problems b e f o r e t h e e a r l y 1960's a p a r t from
Miele's 'Green's Theorem' approach.
However, from
about 1963 onwards r e s e a r c h e r s began t o s t u d y t h e second v a r i a t i o n of t h e g e n e r a l performance index of optimal c o n t r o l i n an a t t e m p t t o f i n d new n e c e s s a r y c o n d i t i o n s f o r a s i n g u l a r c o n t r o l t o be optimal.
The
t h e o r y of t h e second v a r i a t i o n of a f u n c t i o n a l had been
w e l l e s t a b l i s h e d i n t h e c l a s s i c a l c a l c u l u s of v a r i a t i o n s and had f i g u r e d prominently i n t h e work c a r r i e d out by P r o f e s s o r G. A . Bliss and h i s s t u d e n t s a t t h e U n i v e r s i t y of Chicago d u r i n g t h e f i r s t h a l f of t h e present century.
But a l t h o u g h t h e g e n e r a l t h e o r y f o r
t h e problem of Bolza reached a h i g h degree of s o p h i s t i c a t i o n under t h e a t t e n t i o n of t h e Chicago School no a p p l i c a t i o n s of t h i s t h e o r y had been attempted. Indeed, i n t h e p r e f a c e t o h i s book ( B l i s s , 1946) P r o f e s s o r Bliss makes an appeal f o r s u i t a b l e examples t o be l i s t e d which would i l l u s t r a t e t h e theory.
This
appeal has t o a l a r g e e x t e n t been answered over t h e
l a s t t h i r t y y e a r s by t h e enormous r e s e a r c h e f f o r t engendered by c o n t r o l problems a r i s i n g from t h e o p t i m i z a t i o n of dynamical systems. 37
38
SINGULAR OPTIMAL CONTROL PROBLEMS
Until the early 1960's the theory of the second variation of a functional had rarely been applied to any practical problems.
Even in the field of
Mathematical Physics it had often been felt that the additional complexity of the second variation outweighed any possible benefit which might accrue from its use.
However, with the advent of such singular
problems as Lawden's intermediate-thrust arcs (Lawden, 1961, 1962, 1963) in Aerospace and Siebenthal and Aris's stirred tank reactor (Siebenthal and Aris, 1964) in Chemical Engineering it was clear that a satisfactory mathematical analysis of such problems lay in the theory of the second variation.
This approach has been
fully justified as will be seen in this book.
Not only
has t h e study of the second variation of a general performance index yielded necessary conditions for optimality of singular arcs but it has played a no less important part in the derivation of sufficient conditions for such arcs and in the production of algorithms for the numerical solution of non-singular problems. This present chapter sets before the reader a few fundamental concepts necessary for an understanding of what is to follow.
First, the general optimal control
problem mentioned in Chapter 1 is reiterated and placed on a firm mathematical foundation.
It should be
emphasized here that we give a general statement of what is usually referred to as the optimal control problem.
A singular problem is a special case of this
2.
FUNDAMENTAL CONCEPTS
general statement.
39
Next, t h e f i r s t and second
v a r i a t i o n s of t h e c o s t f u n c t i o n a l from t h e g e n e r a l The method o f
optimal c o n t r o l problem a r e d e r i v e d .
g e n e r a t i o n of t h e s e two v a r i a t i o n s f o l l o w s c l o s e l y t h a t used by B l i s s (1946) a l t h o u g h , of c o u r s e , h i s a n a l y s i s does n o t i n c l u d e s p e c i f i c mention of c o n t r o l variables.
Much of t h e n o t a t i o n used i n t h i s book
c o i n c i d e s with t h a t used by Bliss b u t where t h e r e a r e d i f f e r e n c e s w e have changed d e l i b e r a t e l y t o be i n keeping w i t h t h a t used i n t h e modern c o n t r o l l i t e r a t u r e . Having d e r i v e d g e n e r a l e x p r e s s i o n s f o r both t h e f i r s t and second v a r i a t i o n s , shown t h a t t h e f i r s t v a r i a t i o n must be z e r o and t h e second v a r i a t i o n nonn e g a t i v e f o r a minimizing a r c , t h e f i n a l s e c t i o n of t h e p r e s e n t c h a p t e r f o r m u l a t e s t h e g e n e r a l statement of a s i n g u l a r optimal c o n t r o l problem.
The correspond-
i n g forms f o r t h e f i r s t and second v a r i a t i o n s i n t h e s i n g u l a r case are s t a t e d .
A number of examples a r e
p r e s e n t e d i n t h i s c h a p t e r t o i l l u s t r a t e t h e many a s p e c t s of b o t h v a r i a t i o n s .
2.2
The General Optimal C o n t r o l Problem
Consider an n-dimensional s t a t e v e c t o r space X T w i t h time-varying elements x = ( x l x 2...x,) ,
\
= %(t),
k
= 1,2
,...n ,
and an m-dimensional c o n t r o l T v e c t o r space V w i t h elements u = (ulu2 Urn) ,
u. = u i ( t ) , 1
d e f i n e d by
i = 1,2,
...m,
-
to < t
...
tf.
The s e t V i s
40
SINGULAR OPTIMAL CONTROL PROBLEMS
V = (u(t) : a. 1 -< ui
bi,
i
= 1,2
,...,ml
(2.2.1)
where a;, bi can be known functions of time but are usually constants. Since the major portion of this book will be discussing the case of singular control we shall assume, unless otherwise stated, that a control vector u belongs to the interior of space V so that a. < ui < bi,
i
= 1,2,
...,m.
1
Should some of the ui's become equal
to the corresponding bounds ai or bi then either the
technique of Valentine (1937) may be employed or those variations
Bi(*)
(see below) of the control variables
which attain their bounds can be put to zero.
Further-
more, it may sometimes be convenient (in Section 3.2.2 for example) to update the control vector u to the status of a derivative and write u = v,
v(to)
=
0
(2.2.2)
with v(t ) arbitrary. This transformation will be f
seen to bring the control problem more in line with the classical problem of Bolza. Suppose the behaviour of a dynamical system is governed by differential equations
H
= f(x,u,t)
(2.2.3)
2.
41
FUNDAMENTAL CONCEPTS
and boundary conditions
where to and xo are specified and {to, x(to>, tf, x(tf)) belongs to S, a closed subset of R2n+2.
The terminal
constraint function IJ is an s-dimensional column vector function of x(tf)
and tf.
The final time tf may o r may
not be specified. We suppose further that the performance of the system is measured by a cost functional of the form
J = F[x(tf),
tf] +
I'
L(x,u,t)dt.
t0
(2.2.6)
The n-dimensional vector function f of eqn(2.2.3)
and
the scalar functions F and L are assumed to be at least twice continuously differentiable in each argument. The general problem of optimal control is to find an element of U which minimizes the cost functional J of (2.2.6) subject to (2.2.3-5). 2.3
The First Variation of J Define a one-parameter family of control vectors
42
SINCilJLAR OPTIMA12 CONTROL PROBLEMS
w i t h t h e o p t i m a l v e c t o r g i v e n by u ( . , O ) .
The c o r r e s -
ponding s t a t e v e c t o r
,E )
X('
(2.3.2)
w i l l a l s o b e a f u n c t i o n of t i m e t ( t o < t < tf(E) ) and p a r a m e t e r
E
In a l l cases u(-
with ,E)
E
= 0 along t h e optimal t r a j e c t o r y .
b e l o n g s t o U, x ( * ,E)
b e l o n g s t o some
D i f f e r e n t i a l s of f a m i l y ( 2 . 3 . 2 )
f u n c t i o n s p a c e Y.
are
A s i n t h e c l a s s i c a l c a l c u l u s of v a r i a t i o n s ( B l i s s , 1 9 4 6 ) t h e symbol 6 d e n o t e s d i f f e r e n t i a l s o n l y w i t h r e s p e c t t o t h e parameter
E.
We now i n t r o d u c e a s e t of s t a t e
v a r i a t i o n s and f i n a l - t i m e v a r i a t i o n s d e f i n e d a l o n g t h e optimal t r a j e c t o r y a s
i n which case d t f = SfdE and 6x = vde. S i m i l a r l y , w e can d e f i n e a c o n t r o l v a r i a t i o n a l o n g t h e o p t i m a l t r a j e c t o r y as
2.
FUNDAMENTAL CONCEPTS
43
B = (aU/aE)E=o.
(2.3.5)
From the system equatj.ons ( 2 . 2 . 3 ) state variation
rl
it follows that the
must satisfy the equation of
variation
;1 where f,,
= f , r l
f, are n
x
(2.3.6)
+ fuB
n and n
x
m matrices respectively.
Because of the boundary conditions ( 2 . 2 . 4 - 5 ) set of variations Sf,
rl
the
must satisfy end conditions of
the form rl(to>
= 0
.
where $Jt is an s-dimensional vector and f matrix.
(2.3.7)
xf
an s
We now adjoin the system equations ( 2 . 2 . 3 ) terminal constraints ( 2 . 2 . 5 ) of ( 2 . 2 . 6 )
x
n
and the
to the cost functional J
by A , an n-vector of Lagrange multiplier
functions of time, and by v, an s-dimensional constant vector of Lagrange multipliers respectively. functional may then be written as
The cost
SINGULAR OPTIMAL CONTROL PROBLEMS
44
where H(x,u,X,t)
T + A f(x,u,t).
= L(x,u,t)
When the vectors u ( * ,E)
and
are substituted into eqn(2.3.9)
x(*,E)
(2.3.10)
of (2.3.1-2)
the cost functional J
may be looked upon as a function of the single parameter
E
and from J(E) one can easily calculate the
first differential dJ. dJ
=
[(Ft
+
T v $t + H
In fact,
-
ATg)dt
T
+ (Fx + v $x)dx] t=tf
By integrating the term in 6; in the integrand of eqn(2.3.11)
by parts, using (2.3.3)2
to eliminate
6x(tf) and noting that 6x(to) = 0 since x(to) is specified, we obtain dJ
= [ (Ft
T + v Jlt + H)dt + (Fx + vT$x
-
AT)dx] t'f
On the optimal trajectory where the parameter
E:
is zero this differential takes the form dJ = J1(cf,n,B). dE:. The second differential d2J on the optimal trajectory can similarly be written as d2J = 2J2(Sf,~,B)d~2. A Taylor series for the func-
tion J(E) may then be written as
2.
FUNDAMENTAL CONCEPTS
J ( E ) = J ( o ) + E J +~ c2J2 +
The function J,(Sf,q,B)
45
...
(2.3.13)
is called the first variation
of J on the optimal trajectory and from its definition and eqn(2.3.12)
it is clear that
Bearing in mind the assumption made in Section 2.2 that u belongs to the interior of V it follows from
(2.3.13) that a necessary condition for u(*,O) to be a control vector which minimizes J is J1
= 0.
That is,
a necessary condition for optimal control is that the first variation should vanish for all admissible variations.
By choosing the adjoint vector X and the
vector v so as to make the coefficients of
0,
n(tf)
and Sf vanish in ( 2 . 3 . 1 4 ) we obtain the following results:
-iT= XT(tf)
=
Hx(x,u,X,t) -F,(;;(tf),
(2.3.15) tf)
+
*Xf
T
(2.3.16)
SINGULAR OPTIMAL CONTROL PROBLEMS
46
(2.3.17) The first variation of J then reduces to J 1 = ItfHU6 dt.
(2.3.18)
t0
With the control variables away from any bounds the variation 8 in the integrand of (2.3.18) is arbitrary. Since J 1 is to vanish for all admissible variations B the fundamental lemma of the calculus of variations (Bliss, 1 9 4 6 ) yields the condition
Hu
(2.3.19)
= 0.
Of course, when u, a member of V, is allowed to attain its bounds we are led to Pontryagin’s Minimum Principle, namely
-u =
arg min H(;,U,A,~).
(2.3.20)
Throughout the above discussion
x(*)and U(*) denote
u
the candidate state and control functions respectively. A further first order condition is the necessary
condition of Clebsch (Bliss, 1946).
In the control
formulation this condition may be written
Tr
T
O
O
L
O
(2.3.21)
2.
FUNDAMENTAL CONCEPTS
for all (n+m)-vectors
TT
(-In fU)T
47
satisfying the equation =
(2.3.22)
0
where In is the nth order identity matrix. We now illustrate the use of necessary conditions (2.3.15)
,
(2.3.19) to obtain a candidate arc for
optimality by applying them to a rocket problem. Example 2.1
The problem of finding the thrust direc-
tion programme necessary to maximize the range of a rocket with known propellant consumption is considered by Lawden ( 1 9 6 3 ) .
The thrust direction is limited to
lie in a vertical plane through the launching point. The acceleration due to gravity is assumed constant and flight takes place in vacuo over a flat earth. rocket is launched with zero initial velocity at t and burn-out occurs at a known instant t
=
T.
The = 0
The
vehicle continues under gravity along a ballistic trajectory until impact.
The acceleration f caused by
the motor thrust, essentially positive, is a given function of time. With Ox and Oy horizontal and vertical axes through 0 and lying in the plane of flight, the equations of
motion for this problem are
(2.3.23)
SINGULAR OPTIMAL CONTROL PROBLEMS
48
where f
=
cm/M and 0 , the control variable, is the
angle made by the thrust direction with Ox (Lawden, 1963). The initial values of the state variables u, v (horizontal and vertical velocity) and x, y (horizontal and vertical displacement) are specified of flight T to burn-out.
as
is the time
There are no end values
specified at the final end-point except T.
The
boundary conditions for the problem are then v(0) = 0
u ( 0 ) = 0,
to = 0,
(2.3.24) y(0)
x(0) = 0,
tf = T.
= 0,
It is required to maximize the total range, which is a function of the values of the state variables at burnout.
This is equivalent to minimizing the cost
function
The Hamiltonian H of eqn(2.3.10) H
=
is
AU fcose + XV (fsine - 8 ) + A u + X v.
Eqn(2.3.15)
X
Y
(2.3.26)
then yields
-xu ix
- A,, = 0,
-Av
=
xY
iY = o .
(2.3.27)
49
2. FUNDAMENTAL CONCEPTS
Eqn(2.3.19)
leads to the result tan8
=
Av/Au.
(2.3.28)
The end conditions given by eqn(2.3.16) Au(tf>
= -(vf+r>/g,
Av(tf)
=
are
-uf(vf+r)/gr, (2.3.29)
Ax(tf)
=
-1,
where r
=
J(vf2 + 2gyf).
Ay(tf)
=
-uf/r A s in (Lawden, 1963)these
results lead to tan8
=
uf/r.
(2.3.30)
Pontryagin's Principle (2.3.20) or the Clebsch condition (2.3.21-22) are satisfied if 8 takes the positive acute angle solution of eqn(2.3.30).
The
extremal is therefore a trajectory along which the thrust direction remains at a constant positive acute angle to Ox.
Whether this extremal is an optimal
trajectory is still to be decided and this will be the subject of a further example in the next section. 2.4
The Second Variation of J In the previous section it was shown that the
augmented cost functional J of (2.3.9) can be thought of as a function J ( E ) of the single parameter
Eqn(2.3.11)
E.
gives the first differential of J and is
quoted again here for convenience:
SINGULAR OPTIMAL CONTROL PROBLEMS
50
dJ
=
[ (Ft + v T+t + H - A T%)dt
+ (Fx + v T+x)d~It,tf
+ jtf{HXGx + Hu6u - A T 6x)dt *
(2.4.1)
t0
This formula is valid along all arcs x(-,E), u(*,E) belonging to the one-parameter families (2.3.1-2).
In
particular, we have seen that along the optimal trajectory (where
the first differential dJ vanishes.
E=O)
From eqn(2.4.1)
one may calculate the second
differential of J as
J'd
=
[(Ft + v +
+t
T
(Ftt
T + H -A %)d2t +
v +tt
T + (Fx + v 11, )d2x X
+ fi -XTx)dt2
+ 2H Sxdt + 2H Gudt -2ATSkdt] U
X
1
.tF
+
IIHxG2x + H 62u - A U
T
G2t +
t0
t=tf
T G X Hxx6x
T T + 26u HuxSx + 8u H 6u)dt. uu By expanding the term
fi
=
(2.4.2)
dH/dt into its constituent
parts, integrating by parts the term in S 2 $ ,
eliminat-
2.
51
FUNDAMENTAL CONCEPTS
i n g t h e & d i f f e r e n t i a l s o u t s i d e t h e i n t e g r a l s i g n by means of e q n s ( 2 . 3 . 3 )
and f i n a l l y by u s i n g e q n s ( 2 . 3 . 1 5 ,
19), w e f i n d t h a t T T d 2 J = [(H + Ft + v J l t ) d 2 t + (Fx + v $
J
~
-A
T
)d2x]
tf
T
T
+ d x (Fxx + v $Jxx)dx1
+ [ (Ht
-
H K)dt2 X
tf
+ 2H d x d t ] X
tf
(2.4.3) The c o e f f i c i e n t s o f t h e terms i n d 2 t f and d 2 x ( t f ) are zero because of t h e t r a n s v e r s a l i t y conditions (2.3.16-17).
The second d i f f e r e n t i a l t h u s r e d u c e s t o
+
[(Ht
-
H & ) d t 2 + 2Hxdx d t I ,
+ ftf(6xTH t0
X
f
6~ + 26uTHux6x + Bu TH xx
6u)dt uu
(2.4.4)
52
SINGULAR OPTIMAL CONTROL PROBLEMS
where @(x(tf),
tf) = F + v
T
$J
and xf
1
x(tf).
A s men-
tioned in Section 2.3 the second differential d2J on the optimal trajectory may be written as d2J
=
2J,(Cf,q,B)d~2
and from eqn(2.4.4)
it is seen
that
J2 = IQtftfCf
+
1
+
x 5f GfCf f f
0t
.tc
LfiriTHxxn + BTHuxrl
+
t0
+
rl(tf))
JBTH Bldt.
(2.4.5)
uu
This expression J2 is the coefficient of
E~
in the
Taylor series (2.3.13) and is called the second variation of the cost functional J along the optimal trajectory.
It follows that a necessary condition
for u ( * , O ) to be a control vector which minimizes J is d2J/dE2 > 0. That is to say the second variation J, must be non-negative. A particular set of admissible variations satisfy-
ing eqns(2.3.6-8)
is given by
2.
FUNDAMENTAL CONCEPTS
53
For this set of variations the second variation J, vanishes.
Thus, if the candidate arc obtained from the
vanishing of the first variation satisfies the condition J, 2 0 then the set of variations ( 2 . 4 . 6 ) minimize J,.
must
We are accordingly led to an auxiliary
problem known as the accessory minimum problem.
This
is the problem of minimizing J2 with respect to the set of variations
Ef,
q(*),
B(*)
satisfying the
equations of variation ( 2 . 3 . 6 - 8 ) . A further necessary condition for a minimizing
arc is known in the classical literature as the Jacobi condition.
In the optimal control problem with second
variation J2 given above ( 5f = 0 ) the Jacobi condition may be stated as follows (Bryson and Ho, 1969, but see also Chapter 4 ) : an optimal trajectory contains no conjugate point between its end points.
This will be
the case if the matrix S remains finite for
- -
to < t < tf where S satisfies the matrix differential
equation -S
= H
xx
TS -(Hxu + SfU)HUU -1 (HUx + + Sfx + fx
f
TS )
U
(2.4.7)
and end condition S(t,>
= @XX[X(t,>
tf I.
(2.4.8)
SINGULAR OPTIMAL CONTROL PROBLEMS
54
Example 2.2 (Bell, 1965)
Consider again the problem
of maximum range of a rocket vehicle discussed in the
example of Section 2.3. The equations of variation on the extremal are, from eqn~(2.3.23)~
hu
-8,
=
fsine
,
TIv =
Be fcos9
The equations of variation on the extremal of the end conditions are, from eqns(2.3.241,
5,
= 0,
5f
= 0,
rl ( 0 ) = 0 ,
u
Using eqns(2.3.25-26, (2.4.5) may b e written as
- lJ
f ( t - k ) B e 2 sece dt 0
'lJ0)
= 0,
29) the second variation
2.
FUNDAMENTAL CONCEPTS
55
1
T + -(v + r). This variation has to be nong f negative for the extremal found in Example 2.1 of
where k
=
Section 2.3 to be optimal. To demonstrate that it is indeed always non-negative we integrate eqns(2.4.9) with 8 having the positive, acute angle solution of eqn(2.3.30).
where
I1
This gives
=
1
rT fBedt
and
I2 =
0
0
Substituting for nU(T), expression for J
2
nv(T),
0
and 11 (T) in the Y
T + r)j fBe2dt 1.
g f
Thus, J2 > 0 and, moreover, J, 0.
“(T)
we obtain
T + sec8[1 f (T-t) Be2dt + -1(v
$,
rT f(T-t)Bedt.
J
0
= 0
if and only if
The second variation is therefore always
non-negative and the extremal found from the first variation satisfies the necessary condition associated with the second variation.
It is worth pointing out at this stage that the following notation for the second variation will
SINGULAR OPTIMAL CONTROL PROBLEMS
56
normally be used throughout this book:
Q
=
Qf
C = H
Hxx’
-
ux’
A = f
*x x ’ f f
X’
R = H
uu ’
B = f
Furthermore, the variations
U’
IT
D = +
X
f
.
(2.4.10)
in state and B in
control will be denoted by x and u respectively. No confusion will result when the second variation is being considered as a new cost function J[u(*)] in the accessory minimum problem.
The form of the second
variation for a constrained optimal control problem, from eqns(2.4.5) as (5,
and (2.3.6-8)’
may then be written
= 0)
(2.4.11) subject to and 2.5
k
=
Ax + Bu,
Dx(tf)
=
x(to)
0
= 0.
(2.4.12) (2.4.13)
A Sirgular Control Problem
We consider here the class of control problems where the dynamical system is described by the ordinary differential equations
2.
FUNDAMENTAL CONCEPTS
57
= fl(x,t) + fu(x,t)u.
(2.5.1)
where f(x,u,t)
The performance of the system is measured by the cost functional J =
L(x,t)dt
+
F[x(tf),
tf]
(2.5.2)
t0
and the terminal states must satisfy +[x(t,),
(2.5.3)
tfl = 0.
The final time tf is assumed to be given explicitly. Thus, the Hamiltonian H for this problem formulation is linear in the control variables, and the problem turns out t o be singular. It is clear from eqn(2.4.5)
that the second
variation is
subject to eqns(2.3.6-8).
In terms of the notation
mentioned at the end of Section 2.4 this second variation is J[(.)]
=
/tf(hxTQ~ + uTC d d t +
&XT (tf)Qfx(tf)(2.5.5)
t0
subject to
k
=
Ax + Bu,
x(to) = 0
58
SINGULAR OPTIMAL CONTROL PROBLEMS
Dx(t ) = 0. f We conclude this section with a simple example: Example 2.3
Consider the following scalar control
problem: minimize
J subject to
k
=
;[
= u,
2
x2dt x(0) = 1,
-
IuI < 1.
This problem is linear in u with the Hamiltonian
where
H
=
fx2 + Xu
x
=
-x.
A singular arc is one along which
H u = X = O for a finite interval of time.
During this interval
we have H u = X = O which implies
x = 0.
In this case x vanishes identically and s o does u. The arc in (x,t)-space along which u is zero is thus a singular arc.
2.
FUNDAMENTAL CONCEPTS
59
References Bell, D. J. (1965). Optimal Trajectories and the Accessory Minimum Problem, Aeronaut. Q. -' 16 205-220. Bliss, G. A. (1946). "Lectures on the Calculus o f Variations". Univ. Chicago Press, Chicago. Bryson, A. E. and Ho, Y. C. (1969). "Applied Optimal Control". Blaisdell, Waltham, Mass. Lawden, D. F. (1961). Optimal Powered Arcs in an Inverse Square Law Field, J. Am. Rocket SOC. -' 31 566-568. Lawden, D. F. (1962). Optimal Intermediate-Thrust Arcs in a Gravitational Field, Astronautica Acta -8, 106-123. Lawden, D. F. (1963). "Optimal Trajectories for Space Navigation", Buttemorth, Washington, D.C. Siebenthal, C. D. and Aris, R. (1964). Studies in Optimisation - VI. The Application of Pontryagin's Methods to the Control of a Stirred Reactor, Chem. Engng. Sci. 19, 729-746.
-
Valentine, F. A. (1937). The Problem of Lagrange with Differential Inequalities as Added Side Conditions, in "Contributions to the Theory of Calculus of Variations (1933-1937)" pp 403-447. Univ. Chicago Press, Chicago.
This page intentionally left blank
CHAPTER 3 Necessary Conditions for Singular Optimal Control 3.1
Introduction
It has already been stated in the previous two chapters that problems involving singular controls do arise in the optimization of engineering, economic and ecological systems. Because of this it is important that such problems be amenable to mathematical analysis in the same way as nonsingular problems are investigated.
However, since Pontryagin's Principle does not
yield any information directly on singular controls the first task for researchers was to discover new necessary conditions for optimality in the singular case. This chapter describes some of the methods used to derive the two necessary conditions known as the generalized Legendre-Clebsch and Jacobson conditions. One approach for obtaining the former condition was based on a transformation of the singular problem to a nonsingular one (Kelley, 1964b).
This transforma-
tion initially was applicable to problems with scalar control.
It allows analysis of singular problems in a
reduced state space but has the disadvantage of requiring the solution of a system of nonlinear differential equations in closed form. The generalized Legendre-Clebsch condition for scalar control was also obtained by using special control variations (Kelley, 1964a; Kopp and Moyer, 1965; Robbins, 1965). 61
This
62
SINGULAR OPTIMAL CONTROL PROBLEMS
method is described in Section 3.2.1 of the present chapter.
The same condition but for vector control
has been deduced by using the transformations of Goh (1966a, 1966b) and Kelley (Speyer and Jacobson, 1971). Section 3 . 2 . 2
of this chapter describes the transforma-
tion of Goh. By using a particularly simple special control variation Jacobson deduced h i s necessary condition.
In Section 3.3 Jacobson's condition is derived and shown to be a different necessary condition to the
generalized Legendre-Clebsch; together they are insufficient for optimality. Again a number of examples are presented in this chapter both to illustrate the theory and t o draw attention to certain practical problems which give rise to singular control.
Much of the work presented
in this chapter plays an important part in later chapters where necessary and sufficient conditions are discussed. By using the necessary and sufficient conditions for the non-negativity of the second variation the two necessary conditions mentioned above are again derived in Chapter 4 for the totally singular case. 3.2 3.2.1
The Generalized Legendre-Clebsch Condition Special Control Variations Two different methods were used by those research
workers who first derived a new necessary condition for singular optimal control problems with scalar
3.
control.
NECESSARY CONDITIONS
63
This condition has since become known as the
generalized Legendre-Clebsch condition due to its similarity in form to the classical conditions of Legendre and Clebsch.
The two methods were due to
Kelley (1964b) using a transformation to a state space of reduced dimension and to Robbins (1965), Kelley
(1964a), Kopp and Moyer (1965) in which special control variations were employed.
In the present section we
give the derivation due to Kopp and Moyer.
Both
methods are discussed in detail by Kelley et al. (1967). The method adopted to find new necessary conditions in the singular case is to generate special control variations and then to use the condition that the associated second variation must be non-negative. It is a technique that has been used to good effect
in the classical literature on the nonsingular problem. The special control variations s o generated must of course give rise to state variations which satisfy both the equations of variation (2.3.6) and the end conditions (2.3.7-8).
In the following derivation of the generalized Legendre-Clebsch condition a sequence of special control variations will be constructed which in turn will generate a sequence of necessary conditions.
Should
the first condition of this sequence be trivially satisfied then the second condition is tested and so on until new information is obtained.
SINGULAR OPTIMAL CONTROL PROBLEMS
64
Theorem 3.1.
A necessary condition for the scalar
control u in the control problem ( 2 . 5 . 1 - 3 )
to be
optimal is that
for all t in [to,tfl. A special scalar control variation, denoted by
Proof:
@ o l ( ~ , ~ ) is
shown in Fig.1.
F i g . 1.
Control variation
$
0
( T , E)
The centre of this double pulse variation is at any point t2 in (to,tf). T =
t
-
Introduce the fictitious time
t2 so that the variation is symmetrical about
the origin
T =
0 as shown.
The interval of the double
pulse is 2~ and the duration of each pulse is
E’.
Successive integrations of +ol with respect to t are designated @ v l ( ~ , ~ ) ,
65
3. NECESSARY CONDITIONS
With the control variation u of variation ( 2 . 4 . 1 2 )
i =
AX
+
=
$0
the equations
’(T,E)
become
,
x(to)
B+~~(T,E>
=
o
(3.2.1)
The end conditions at tf ( 2 . 4 . 1 3 ) will be ignored for the time being. Assuming the necessary smoothness properties of the coefficients of ( 3 . 2 . 1 ) it can be demonstrated by direct substitution that a solution of (3.2.1)
is given by x = B$l’ + A2$z1 + O 1
where
(3.2.2)
(3.2.3)
A2zAB-i
With this expression for x substituted into the second variation ( 2 . 5 . 5 ) we obtain
J[(*)I
E
=
v[Sf, x(tf>I
+
+
+ contribution over
[E,
tf-t21.
A242l + 0 ’ )
(3.2.5)
SINGULAR OPTIMAL CONTROL PROBLEMS
66
Integrating by parts the terms 9 0 ’$1’CB and $01@21CA2 and using the definitions of $1’ and $2’ the second variation may be expanded as a power series in assuming the Taylor series in of convergence.
-2C(AB-i)
dt
has a nonzero interval
Retaining the terms in ($1’)’
are the dominant terms in d J = [ - -(CB)
E
E,
E,
which
we have
T
+ B QB]
E~
+
O(E~)
(3.2.6)
T=O
Here we have used the fact that from equation (3.2.2)
and from equation (3.2.4) el(t)
is or order
E ~ .
It
f) with 5 f = 0, and the contribution to J over the time interval f o l l o w s that y, the quadratic form in x(t
’
are of order E ~ . However, the state fJ variations x of equation (3.2.2) corresponding to the [t2
+E,
t
special control variation 0
0
,
(T E)
will not in general
satisfy the terminal constraints (2.4.13); order
E ~ .
they are of
The question thus arises as to the admissiHowever, since x(t ) f we may alter the special control varia-
bility of the variation $ is of order
E~
0
’(T,E).
tion over the interval [t2 +
E,
tf] by the addition of
an appropriate function of time which need only be of order
E ~ .
This additional functi.on is chosen to ensure
the satisfaction of the terminal constraints (2.4.13).
67
NECESSARY CONDITIONS
3.
The resulting changes in the second variation (3.2.5) are of order c 6 so that the dominant terms of that variation, stated in (3.2.6),
are unchanged.
The
existence of such control correction functions is ensured by our assumption of normality. Remembering that
T =
0 is any interior point on the
singular arc, a necessary condition for that arc to be minimizing is thus, from (3.2.6),
d
-(CB) dt
+ 2C(AB
- i) -
T
-
B QB < 0
(3.2.7)
This is identical to the result of Kelley (1964a) and was recognized by A. E. Bryson, Jr. as being equivalent to the compact form
-a d2H au
dt2
u
< O
-
for all t in [to,tf].
(3.2.8)
This condition is the first of the sequence of conditions which make up the generalized Legendre-Clebsch condition. Should this condition (3.2.8) be trivially satisfied (equality) then no further information with which to test the singular arc has been obtained. case a second scalar control variation 4 structed and is shown in Fig.2.
0
In this
2 ( ~ , ~is)
con-
*
-E lI
-
I-
m
6
2
4
t-
I
I
-
0 e3-
-
-I
F i g . 2.
-
Control variation 9
Successive integrations of 9
i. e.
f-
-
denoted by 4 2 ( r , ~ ) , v
-
0
0
(T
,E )
with respect to time are
dV4v2( T ,€1 = 9,2(T,E).
dTv
Proceeding exactly as in the proof of Theorem 3.1, ignoring terminal constraints and with 4
x = ~
9 +~
1
2A . + . ~+ 1 1 i=2
e2,
one is led to the necessary condition
3.
a d4H > O a u g U-
NECESSARY CONDITIONS
69
for all t in [to,tfl.
(3.2.9)
Details of the derivation of (3.2.9) are given by Kelley et al. (1967).
Again, the satisfaction of terminal con-
straints may be accomplished with control correction functions which do not affect the dominant term of the second variation from which the condition (3.2.9) is obtained. If the condition (3.2.9) is trivially satisfied then a third control variation $ , 3 ( ~ , ~ )is constructed and so on. Details of the general analysis are also given by Kelley et al. (1967).
The general result may
be written as (3.2.10) It can be shown (Kelley et al. 1967) that the control u cannot appear in an odd time derivative H when u is U
a scalar. Example 3.1.
The problem of control of a stirred
reactor as defined by Siebenthal and Aris (1964) is that of determining the heat removal rate u and the associated state variables x1 and x2 denoting extent of reaction and temperature respectively, which satisfy differential equations
70
SINGULAR OPTIMAL CONTROL PROBLEMS
51
= -XI
+ R(x1,xZ)
where a and b are c o n s t a n t s and R r e p r e s e n t s t h e r a t e of r e a c t i o n such t h a t a c e r t a i n performance i n d e x may be extremized.
I n ( S i e b e n t h a l and A r i s , 1964) i t i s t h e
t i m e t a k e n f o r s t e a d y s t a t e t o be achieved from a given
i n i t i a l s t a t e which i s t o b e minimized.
A l l variables
a r e measured from t h e s t e a d y s t a t e . The Hamiltonian f o r t h i s problem i s H = Al(-xl
+ R) + X2(R
-
-
ax2
(x2 + b ) u ) .
(3.2.11)
The a d j o i n t v a r i a b l e s A 1 and A2 a r e governed by t h e equations
i2 = -X1R2 where R1
=
+ X2(a + u
-
R2)
aR/3x1 and R 2 = 3R/3x2.
Since t h e Hamiltonian i s l i n e a r i n t h e c o n t r o l u t h e r e i s t h e p o s s i b i l i t y of optimal s i n g u l a r c o n t r o l when t h e e x p r e s s i o n A2(x2 + b) v a n i s h e s i d e n t i c a l l y over a f i n i t e i n t e r v a l of t i m e .
Furthermore i t is
shown by S i e b e n t h a l and A r i s (1964) t h a t such a
3.
NECESSARY CONDITIONS
71
singular control is possible in a reversible, exothermic reaction provided that
X2 :0, R2
3
0 and XI # 0.
(3.2.12)
We shall now apply the generalized Legendre-Clebsch condition to this singular arc (Bell, 1969). From ( 3 . 2 . 1 1 ) we see that
d H dt u
= A 1 (x2+b)R2
-A2(ab-x~R~-bR2+R)
+ R22{R-ax2 -(x2+b)u)l
where
R21 =
a2R and R22 ax2axl
+ XlRz[R-ax2 -(x2+b)u]
=
a 2R ax22
Using the identities in ( 3 . 2 . 1 2 ) we see that
SINGULAR OPTIMAL CONTROL PROBLEMS
72
Since x2+b # 0 (Siebenthal and Aris, 1964) the generalized Legendre-Clebsch condition (3.2.8) thus yie Ids
along the singular arc.
This is a new result, not
obtainable from Pontryagin's Principle. Example 3.2.
Lawden (1963) has shown that a singular
trajectory in the form of an intermediate-thrust arc satisfies Pontryagin's Principle and is a candidate for a minimum-fuel transfer orbit in space navigation when the final time is unspecified. has become known as Lawden's spiral.
This singular arc The non-optimality
of this singular arc has been demonstrated, for example by Kelley et al. (1967), using the generalized LegendreClebsch condition discussed above. This present example shows that Lawden's spiral is non-optimal by constructing special state variations which satisfy exactly the necessary end conditions on the variations (Bell, 1971). The notation is that used by Lawden (1963).
At
time t the vehicle R of mass M has polar coordinates r,8 on the singular arc. At this instant the direction of the rocket's thrust makes an angle 4 with the perpendicular to the radius vector drawn in the same sense as the motion.
With 0 as pole and OX as initial line
the angle made by the thrust direction and OX is $
(Fig. 3)
3.
.
73
NECESSARY CONDITIONS
Lmden's s p i r a l
Fig. 3.
The vehicle's velocity components in the direction of and perpendicular to the thrust are (u,w).
The
equations of motion of the vehicle resolved in directions along and perpendicular to the thrust are cm +-
G
=
G
= -$u
i-
= usin+
$w
M
r2
sin+
+ -L cos+ r2
-
wcos+
rt, = ucos+ + wsin+
M = -m
(3.2.13)
74
S I N G U L A R OPTIMAL CONTROL PROBLEMS
where c is the constant exhaust velocity of the rocket, m is the propellent mass flow rate and y is the The initial values of t, M, r,
gravitational constant.
0 and the final values of r, 0 are specified while the
initial and final values of u, w and $ are given by the equations u w
0
0
=
i-0 sin+0 + r0.0 0 cos$ 0
=
-r cos@
0'0
' 0
+ r 6 sin$
0
+ +o = 0
O
0
+T
r 2
(3.2.14)
u w
f f
$bf
=
.f f-f r sin@ + r 0 C O S + ~ f
=
*f -r cos$
+$f = 0
f
f*f + r 8 sin$f
f T r + 2 -
Suffices o and f denote as usual initial and final end points respectively. Superscripts o and f denote specified values at t variables, namely
+
0
and tf.
There are two control
and m.
It is required to determine the thrust magnitude and thrust direction along an intermediate-thrust arc which transfers the vehicle between the two fixed terminals with mininurn consumption of fuel.
A suitable
cost function which is to be minimized is given by
3.
75
NECESSARY CONDITIONS
J
= c
ln(Mo/Mf).
It is clear from equations (3.2.13) that the Hamiltonian for this problem is linear in the mass flow rate m. This control variable m is singular.
Results from the
vanishing of the first variation J1 are (Lawden, 1963)
Y sin+ = 0 wI)* + 7 r
w + rI)sin+ = A
.. $
=
3Y sin4 -3
r
COS~
where A is a constant of integration. The equations of variation along the singular extremal are (2.3.6)
nu
= $qw
riw =
rlu nW
=
+ wq $- -(d-nd t cM M + 2Y3 sin+ nr -
-$nu - un+
-
2Y r3
7
‘OS+
‘r
-we4 + sin+ nr + 8cos+
= UB
0
-
cos+
-
- 1sin+ B r2
n,
+
rcos+
: r + ;sin4 nr + rsin+ ;I8
+ (3.2.15) 8
SINGULAR OPTIMAL CONTROL PROBLEMS
76
II
$
+
B
0
=
nO.
The e q u a t i o n s of v a r i a t i o n on t h e e x t r e m a l arc of t h e end c o n d i t i o n s are ( 2 . 3 . 7 - 8 )
E0
= n,(O)
= nr(0> =
17 ( 0 ) =
-w B (01,
U
=
p
p
= 0,
llW(O) = u B (0)
0 4
0 4
n “(tf)
n
= -
B p
(3.2.16)
n p f )
-GfEf,
IIU(tf) =
-Cfsf -
IIW(tf) =
-GfSf
wf(+f5f
+
=
-6 f 5 f
B4(tf))
+ Uf(ifSf + B 4 ( t f ) ) .
It can be shown ( B e l l , 1971) t h a t t h e second v a r i a t i o n
reduces t o
tf 5 f 2
-
3Y sin+ ne2) r2
dt.
(3.2.17)
3. NECESSARY CONDITIONS
77
We now generate a special set of state variations satisfying the end conditions (3.2.16), to show “r, “8 that the second variation J2 of (3.2.17) can assume a negative value, thus violating the necessary condition that it should be positive for all admissible variations.
Choose
..
‘r
= plG
..
Q9
where G(t)
= q,G
+ p2k
-
PG (3.2.18)
+ q2k
-
ifG
is an arbitrary function of time, except for
certain end conditions specified below, and the functions pl(t), p2(t), q1(t), q2(t) are also arbitrary but finite functions of time subject only to the continuity
vr, n g ,
conditions on the variations
OU, “w
and
rl
M’ The end conditions (3.2.16) on the chosen varia-
tions Q r’ ~~(3.2.18)are satisfied provided function G(t) satisfies the boundary conditions G(o)
= 0,
6(0) =
0,
G(0) = 0
Differentiating equations (3.2.18) with respect to time, eubstituting into equations (3.2.15) employing equations (3.2.13)
3-4
and 3-4 yields expressions for
78
SINGULAR OPTIMAL CONTROL PROBLEMS
the variations n
U
nW'
and
It turns out that these two
latter variations satisfy end conditions ( 3 . 2 . 1 6 ) provided the function G(t)
satisfies the further
boundary conditions
Equation (3.2.15) 2 then yields an expression for fB 4 as a function of G and its derivatives up to and including G ( iV>
.
Now we choose the functions pl(t),
e(t) s o that ,(iv) , G(iii)
p2(t),
ql(t),
the coefficients of the derivatives and G vanish in the expression for
It can easily be deduced that this implies f(@+-qo). the following equations:
pzcos+ 2$(p2sin+
+
-
q2rsin+
q2rcos$)
=-
2fG
(3.2.19)
cos+ q1
d 2r$ ql) dt cos$
= -(-
+
w
6Y -7 sin4 q,. r
From equations (3.2.19)
the functions p1, p2 and q2 can be found in terms of ql. It is clear that the function ql(t) may be chosen arbitrarily provided it is
non-zero.
With these choices for the functions p l y p2
and q2 the variation
rl
dJ
is a function only of G(t)
and
3. NECESSARY CONDITIONS
its first derivative k(t). given by (3.2.17),
79
The second variation
after integration by parts and using
end conditions on k(t),
may then be written as
0
+ terms in
h2
and G2)dt. (3.2.20)
Now choose G(t)
where
E
such that
is an arbitrarily small parameter.
within the interval [t,, t l + derivative G(iv)(t>
Fig.4.
E]
Furthermore,
choose the fourth
to be of the form shown in Fig.4.
The derivative G ( i v ) ( t ) ,
t, < t < tl +
E.
S l N G U L A R OPTIMAL CONTROL PROBLEMS
80
The function G(t) and its first three derivatives are continuous. It is easy to verify that G(t) satisfies all the required boundary conditions and furthermore, G(t)
-
0(c8),
In particular, 5 ,
6(t) -
-
O(E~),
O(E~).
G(t)
-
Oh4),
The sign of the second
variation is thus determined by the term in G2 in (3.2.20); the dominant term, when
E
is sufficiently
small, being 3y sin+
( 3-5sin2$) q 1
G2 dt ,
(3.2.21)
Lawden (1963) has shown that the control variable
+
must satisfy the inequalities
Therefore, the term in (3.2.21) is negative and the second variation 5 2 given by (3.2.20) can be made negative by choosing
E
sufficiently small.
This violates
the necessary condition for optimal control that the second variation must be non-negative for a minimum value of J.
Lawden's spiral cannot therefore form part
3.
NECESSARY CONDITIONS
81
of a minimunrfuel trajectory. Note that this result has been obtained with special variations which satisfy exactly the initial and terminal conditions (3.2.16). 3.2.2
A Transformation Approach
The generalized Legendre-Clebsch condition was derived n an alternative, indirect way by first transforming the singular problem into a nonsingular one and then applying the classical Legendre-Clebsch necessary condition (Kelley, 1964b; Kelley et al. 1967; Goh, 1966a, 1966b).
This condition for the transformed
problem is the generalized Legendre-Clebsch condition for the original singular problem.
There have been two
different transformations given in the literature. Kelley's transformation (Kelley, 1964b; Kelley et al., 1967; Speyer and Jacobson, 1971) reduces the dimension of the state space to n-p, where p is the order of the
singular arc. The other transformation is due to Goh (1966a, 1966b) and is the one discussed in this section.
It retains the full dimensionality of the original problem and is simpler in the case of vector controls (see also Chapter 6, Section 6.3). A s in Section 3.2.1 we start from the second
variation for the singular arc J2 =
jtf{ixTQx + uTCxldt +
IXT (tf)Qf
x(tf)
t0
subject to the equations of variation
(3.2.22)
SINGULAR OPTIMAL CONTROL PROBLEMS
82
L
=
Ax + Bu,
x(to)
=
0
(3.2.23)
and terminal conditions Dx(tf) = 0.
(3.2.24)
The basic idea behind Goh's transformation is to update the control vector to the status of a derivative of a new vector followed by the elimination of this derivative from both the second variation and equations of variation.
A further updating of the new vector to
a derivative may then yield a non-singular problem to which the classical Legendre-Clebsch condition may be applied.
If this is not the case, then the procedure
is repeated. Define a vector v such that v = u
(3.2.25)
and, without l o s s of generality, set V(t0)
=
0.
(3.2.26)
Furthermore, make the transformation from x to a new variable z: z=x-Bv in order to eliminate (3.2.23).
i7
from the equations of variation
These equations, together with the terminal
83
NECESSARY CONDITIONS
3.
conditions (3.2.24), become
5
=
-
Az + (AB
D[z(tf)
i)v,
z(to)
+ Bv(tf)]
= 0.
(3.2.27)
= 0
(3.2.28)
The second variation becomes J2[v(*)]
=
T T T T T jtf{4z Qz + v B Qz + iv B QBv t0
+ GTC(z
+ Bv))dt
(3.2.29) .T Integrate by parts the term v Cz by parts. J,[v(.)]
T
T T
= Itf{iz Qz + v (B
Q - CA -
This gives
c)z
t0
+ ivT [BTQB
-
2C(AB
-
i)]v
+ GCBvJdt
rn
rn
3.2. 1)
But it can be shown that T B QB
-
2C(AB
-
f3)
-
d x(CB) =
a;; -au
u
SINGULAR OPTIMAL CONTROL PROBLEMS
84
where, because of t h e i d e n t i t i e s ( 2 . 4 . 1 0 ) , T
H(x,u,A,t)
T
-
A = H
T
+ u Cx + X ( A x + Bu),
= { x Qx
X'
T h e r e f o r e , t h e second v a r i a t i o n g i v e n by (3.2.30)
can
be e x p r e s s e d as J 2 [ v ( * ) ] = I t f { $ z T Q z + v T (BT Q
-
CA
- t)z
t0
+
T
IV
[-
a H.. au
u
T d + GCBV + {V x ( C B ) v } d t
]V
F i n a l l y , i n t e g r a t i n g t h e l a s t t e r m i n t h e i n t e g r a n d by parts, J2[v(*)]=
[t f
(42
T
T T Qz + v (B Q
-
CA
-
?)z
t0
+
T
IV
[-
a H.. au
u
]V +
;4
T
[CB
-
T (CB) ] v ) d t
3.
NECESSARY CONDITIONS
85
We pause at this point to deduce condition ( 1 . 5 . 3 ) . Using an appropriate choice of control v(s) we can make the term in ir dominate the other second-order quantiT ties in ( 3 . 2 . 3 2 ) . Since CB - (CB) is skew-symmetric this means we can make J, either positive or negative. This violates the condition that J2 must be of one sign for a minimizing or maximizing arc.
We are thus led to
the necessary condition that CB
=
(CB)T
=
for all t in [to,tfl. ( 3 . 2 . 3 3 )
B C
This is a very powerful condition in the case of vector controls.
It is easily verified (see proof of Lemma
4 . 2 in Chapter 4 ) that
CB
-
BTCT
=
aua .HU.
Thus, ( 3 . 2 . 3 3 ) corresponds to ( 1 . 5 . 3 ) with p
=
If
1.
u is a scalar then ( 3 . 2 . 3 3 ) is satisfied trivially.
Returning to equation ( 3 . 2 . 3 2 ) the term in vanishes by ( 3 . 2 . 3 3 ) .
The derivative
now
has thus been
eliminated completely from the second variation and from the equations of variation.
We may therefore
update the control v to the status of a derivative, say
G.
The second variation and differential con-
straints of the accessory minimum problem may now be written in the form
86
SINGULAR OPTIMAL CONTROL PROBLEMS
q 1 ,
+ YCfi(tf>¶
+rl +
en
(3.2.35)
= 0
where > 0
0
R =
¶
0
- -3 ; ; au
u
0
0
(B~Q-cA-~)
0
L =
J
M =
(3.2.34)
3.
87
NECESSARY CONDITIONS
Applying the classical Clebsch condition (2.3.21-22)
to the transformed accessory minimum
problem ( 3 . 2 . 3 4 - 3 5 )
we obtain the following result:
Along a minimizing arc T
-
(3.2.36)
I T R T > O
for all (n+m)-vectors $TI
satisfying
IT
(3.2.37)
= 0.
...)
IT to be arbitrary, condition Taking ITn+l ' TIn+2 ' n+m ( 3 . 2 . 3 6 ) subjected to ( 3 . 2 . 3 7 ) implies that
- - aH > O au u -
(3.2.38)
for all t in [to,tf]
which is ( 3 . 2 . 1 0 ) with q
=
1.
If ( 3 . 2 . 3 8 )
is met with
equality for all t in [tost 1 then the functional f J2[v(.)] is totally singular and another transformation must be made.
In this way the generalized necessary
conditions ( 1 . 5 . 3 - 4 )
are obtained.
It should be noted that the derivative $(tf) occurs in the terminal quadratic form y of ( 3 . 2 . 3 4 ) and also in the terminal constraint ( 3 . 2 . 2 8 ) with v
=
&.
However, the class of admissible vectors w satisfying the differential constraints ( 3 . 2 . 3 5 )
is not reduced by
restrictions placed upon the end values 6(tf).
This is
because any such restrictions can be met by infinitesimal adjustments over a sufficiently small neighbourhood
SINGULAR OPTIMAL CONTROL PROBLEMS
88 [tf-E,
t ] and these adjustments will only affect the
f integral in J 2 infinitesimally. Consequently, the
elements of the vector 6(t f) can be treated just as one would treat the parameter 5 f (Goh, 1966a, 196613; Speyer and Jacobson, 1971; McDanell and Powers, 1970). Example 3 . 3
The problem of control of a particular
aircraft model as defined by Schultz and Zagalsky (1972) is that of determining the minimum fuel required for a climb-cruise-descent trajectory between specified values of altitude, range and velocity.
The control
variables are thrust and flight path angle and the terminal time is not specified although the final result is valid also for fixed time.
The system is
governed by the differential equations and end conditions
i
=
(T-D)v/M,
i = vy, x =
E(to)
= Eo,
h(t )
=
0
v,
ho, 0
x(to) = x
y
f
E(tf)
=
E
h(tf)
= h
X(tf)
=
f
.
f x
Superscripts o and f denote specified values at t0 and t
f
respectively.
E is the specific energy defined as
E = V2/2
+
gh,
3.
NECESSARY CONDITIONS
89
h i s t h e a l t i t u d e , V t h e v e l o c i t y , x t h e range, D(E,h) t h e aerodynamic d r a g and M t h e mass of t h e a i r c r a f t . The mass M w i l l be assumed c o n s t a n t although t h e conc l u s i o n s remain t h e same when t h i s r e s t r i c t i o n i s lifted.
The c o n t r o l v a r i a b l e s , t h r u s t T and f l i g h t
p a t h a n g l e y, are bounded as
It i s r e q u i r e d t o minimize t h e f u e l J = jtfa(E,h)T d t to
used d u r i n g t h e manoeuvre, where a(E,h) i s t h e f u e l
rate per u n i t thrust. The Hamiltonian f o r t h i s problem is H = UT
+ Al(T-D)V/M +
A2Vy + X3V.
The a d j o i n t v a r i a b l e s A 1 , A2 and A3 a r e governed by t h e equ a t i o n s i1 = - a H / a E ,
i2 = -aH/ah
and
S i n c e t h e Hamiltonian i s autonomous and t
A 3 = -aH/ax.
is f u n s p e c i f i e d a f i r s t i n t e g r a l of t h e motion e x i s t s and
is
90
SINGULAR OPTIMAL CONTROL PROBLEMS H = O
for all t in
[to,t,l.
The H a m i l t o n i a n i s a l s o l i n e a r i n t h e c o n t r o l v a r i a b l e s s o t h a t t h e r e i s t h e p o s s i b i l i t y of o p t i m a l s i n g u l a r c o n t r o l when t h e e x p r e s s i o n s
and
x2v
away =
v a n i s h i d e n t i c a l l y o v e r a f i n i t e i n t e r v a l of t i m e . This implies t h a t
A2 = 0 .
It i s shown by S c h u l t z and Zagalsky (1972) t h a t when
t h e c o n t r o l s assume v a l u e s which s u s t a i n t h e c r u i s e p o r t i o n of t h e t r a j e c t o r y T = D,
y = 0.
Since t h e Hamiltonian i s zero, d(aH/aT)/dt
=
a(oD/V)/aE = 0
3.
NECESSARY CONDITIONS
91
which imply that the cruise is flown at the energy and altitude which satisfy min(oD/V).
The cruise condition
is thus a doubly singular arc which satisfies the first order necessary conditions (Schultz and Zagalsky, 1972). We shall now apply the generalized Legendre-Clebsch condition to this singular arc (Speyer, 1973). T For the control vector u = (T y), eqn(3.2.33) equivalently eqn(1.5.3))
(or
is not satisfied since
and this is not the null matrix because the term (aa/ah + ag/V2)V in general is not zero.
This result
remains true if t is specified. The cruise condition f will not therefore yield a minimizing singular arc. This result is contrary to the findings of Schultz and Zagalsky (1972). 3.3
Jacobson's Necessary Condition A further necessary condition for non-negativity
of J2[u(.)]
was discovered by Jacobson (1969, 1970b),
different from the generalized Legendre-Clebsch condition.
This new condition is stated and proved below
for the free-endpoint problem, that is the matrix D in (3.2.24)
is the null matrix.
The appropriate condition
SINGULAR OPTIMAL CONTROL PROBLEMS
92
for the constrained-endpoint problem is derived by Jacobson (1969).
A slightly stronger version of
Jacobson's condition for the free-endpoint problem was obtained by Gabasov (1968, 1969) although the conditions are identical for quadratic problems.
A necessary condition for J2[u(.)]
Theorem 3 . 2 .
to be
non-negative for all u(*) belonging to U, for the case where D = 0, is that
T
CB + B WE5 > 0 where
Proof.
-fi
+ ATw
Q
=
for all t in [to,tfl
+ WA;
W(tf> =
Q,.
(3.3.1)
(3.3.2)
Adjoin the equations of variation ( 3 . 2 . 2 3 )
to
the expression J, of ( 3 . 2 . 2 2 ) by an adjoint vector of T the form JW x, where W(-) is an arbitrary n x n symmetric, continuously differentiable matrix function of time.
J~[u(.)]
Then =
T T T jtf{4x Qx + u Cx + ix W(Ax+Bu-k)ldt
t0 (3.3.3)
Integrate by parts the term in 2 to give J2[u(*)]
=
1
tf
T {4xT(fi + Q + A W + WA>x
t0
T T + u (C + B W)x)dt
T
+ 4x (t,>[Qf
-
W(t,)lx(t,)
(3.3.4)
3.
93
NECESSARY CONDITIONS
Now choose W to satisfy equations (3.3.2). Then J,[U(-)]
=
jLfuT(C + BTW>x dt.
(3.3.5)
0
Choose a special variation u which is zero except strictly contained in during an interval [t,,t,+A],
1, when it assumes a constant but arbitrary f magnitude K. Then [to,t
J, = It'+'KT(C
+ BTW)x dt
(3.3.6)
tl
and the dominant term in the expansion of this integral for sufficiently small A is
.
AKT (C + BTW)BKA2It
(3.3.7)
1
A necessary condition for J2 to be non-negative is
therefore the condition (3.3.1) since tl can be chosen as any point in the interval [t ,t 1. (3.3.1) and o f (3.3.2) together form Jacobson's necessary condition. The following example illustrates the nonequivalence of Jacobson's condition and the generalized Legendre-Clebsch condition.
Furthermore, it demon-
strates that these two necessary conditions are in general insufficient for optimality.
SINGULAR OPTIMAL CONTROL PROBLEMS
94
Example 3 . 4 (Jacobson, 1970a).
Minimize
(-x12 + x 2)dt
(3.3.8)
2
subject to
(3.3.9)
;r2=u
,
x2(o) = 1.
The Hamiltonian is given by
H
4(-xl2
=
+
x2
2)
+
x1x2 +
x2u.
The adjoint equations for this problem are:
It i s easy to verify that
-
u(t)
=
-
-sint ( 3 . 3 . lo)
x,(t) = -sint
-
x,(t)
=
cos t
is a singular solution satisfying Pontryagin’s prin-
ciple.
For this solution, the magnitude of J is zero.
Here, C = H
ux
= (0 0),
B = f
U
=
T
(0 1)
95
3. NECESSARY CONDITIONS
and it follows that T
(C + B W)B
=
w22
where
w
=
(w. . ) ,
i,j = 1,2.
1J
Using (3.3.2) it is easily verified that w22(T)
= T
-
T1 T
3
where T =
3 ~ / 2- t,
i.e., reverse time.
Hence, w22 is negative for
so that (3.3.1) ceases to be satisfied.
T
> 43
The singular
solution is then nonoptimal. Also, for this problem HU
..
fiu
= 12,
HU=-
U -
= -x2
-
A]
x1
so that the generalized Legendre-Clebsch condition
(3.2.38) is satisfied for all
T.
If (3.2.38) and (3.3.1) were together sufficient for optimafity then the singular solution for 0 < T <
43 would be optimal.
That this is not the case
can be demonstrated by transforming the original problem to a nonsingular one in the following way.
SINGIJLAR OPTIMAL CONTROL PROBLEMS
96
Write z so
x1 and v = x2
=
that the original problem becomes:
-
Minimize J
4
=
1
3~12 (-z2
+ v2)dt
(3.3.11)
0
subject to z = v,
in which variable.
z
z(0)
=
(3.3.12)
0
is now a state variable and v a control The problem is now nonsingular.
Clearly, v(t)
=
cos t
-
satisfies Pontryagin’s principle and J control. ‘Hvv
(3.3.13) =
0 for this
The strengthened Legendre-Clebsch condition
> 0 ) and the boundedness of the solution of the
associated Riccati differential equation can be used to determine whether or not the control (3.3.13) is optimal since the problem is now nonsingufar. We have Hvv
= 1> 0
and the Riccati equation is
- -ds -d.r
1 +
s2,
s = S(T),
s ( 0 ) = 0.
The solution of this differential equation is S(T)
=
-tan
T
3.
NECESSARY CONDITIONS
which becomes unbounded at
T =
97
~ / 2 . Therefore, the
control (3.3.13) ceases to be optimal for
T > ~ / 2 .
This means that the singular solution (3.3.10) ceases to be optimal for
T >
n/2, since the two problems
(3.3.8-9) and (3.3.11-12)
are equivalent.
But we have
already shown from Jacobson's condition that (3.3.10) is nonoptimal for
T
> 43.
Thus, we have the situation
whereby the two necessary conditions are both satisfied for -m/2 <
T <
43 but the singular solution is nonoptimal.
This shows that the satisfaction of both the generalized Legendre-Clebsch condition and the Jacobson condition is not sufficient for optimality in singular control problems. In the derivations of the generalized LegendreClebsch condition it has been assumed that the problem is normal (Bliss, 1946).
That is to say sufficient
control variations exist to modify the special control variations in order to satisfy terminal constraints. Even in the absence of terminal constraints the assumption of normality is still necessary to insure the correct form of the test.
In the proof of the
Pontryagin Maximum Principle (Pontryagin et al., 1962) no assumption of normality is necessary since it is taken care of by the transversality condition on the adjoint variable and the additivity of first order variations.
An extension of the Pontryagin Maximum
Principle has been obtained by Krener (1973) and is called the High Order Maximal Principle.
This extended
98
SINGULAR OPTIMAL CONTROL PROBLEMS
3rinciple includes not only the generalized LegendreClebsch and Jacobson conditions when they apply but also a number of other conditions which can be constructed for a specific problem without the assumption of normality. References Bell, D. J. (1969). Singular Extremals in the Control of a Stirred Reactor, Chem. Engng. Sci. 24, 521-525. Bell, D. J. (1971). The Non-Optimality of Lawden's 16, 317-324. Spiral, Astronautica Acta Bliss, G. A. (1946). "Lectures on the Calculus of Variations". Univ. Chicago Press, Chicago. Gabasov, R. (1968). Necessary Conditions for Optimality of Singular Control, Engng. Cybern. No.5, 28-37. Gabasov, R. (1969). On the Theory of Necessary Optimality Conditions Governing Special Controls, Sov. Phys.-Dokl. 13, 1094-1095. Goh, B. S. (1966a). The Second Variation for the Singular Bolza Problem, SIAM J. Controli, 309-325. Goh, B. S. (1966b). Necessary Conditions for Singular Extremals Involving Multiple Control Variables, SIAM J. Control 4, 716-731. Jacobson, D. H. (1969). A New Necessary Condition of Optimality for Singular Control Problems, SIAM J. Control 7, 578-595. Jacobson, D. H. (1970a). On Conditions of Optimality for Singular Control Problems, IEEE Trans. autom. Control AC-15, - 109-110. Jacobson, D. H. (1970b). Sufficient Conditions for Nonnegativity of the Second Variation in Singular and Nonsingular Control Problems, SIAM J. Control 8, 403-423. -
3.
NECESSARY CONDITIONS
99
Kelley, H. J. (1964a). A Second Variation Test for Singular Extremals, AIAA J. 2, 1380-1382. Kelley, H. J. (1964b). A Transformation Approach to Singular Subarcs in Optimal Trajectory and Control 234-240. Problems, SIAM J. Control;, Kelley, H. J., Kopp, R. E. and Moyer, H. G. (1967). Singular Extremals, in "Topics in Optimization" (G. Leitmann, ed) pp 63-101. Academic Press, New York. Kopp, R. E. and Moyer, H. G. (1965). Necessary Conditions for Singular Extremals, AIAA J. 3, 1439-1444. Krener, A. J. (1973). The High Order Maximal Principle, in "Geometric Methods in Systems Theory" (D. Q. Mayne and R. W. Brockett, eds) pp 174-184. NATO Advanced Studies Institute Series: Mathematics and Physics. Lawden, D. F. (1963). "Optimal Trajectories for Space Navigation", Butterworth, Washington, D.C. McDanell, J. P. and Powers, W. F. (1970). New JacobiType Necessary and Sufficient Conditions for 8, Singular Optimization Problems, AIAA J. 1416-1420. Pontryagin, L. S., Boltyanskii, V. G., Gamkrelidze, R. V. and Mishchenko, E. F. (1962). "The Mathematical Theory of Optimal Processes". Interscience, John Wiley & Sons, Inc., New York, London and Sydney. Robbins, H. M. (1965). Optimality of Intermediate3, Thrust Arcs of Rocket Trajectories, AIAA J. 1094- 1098. Schultz, R. L. and Zagalsky, N. R. (1972). Aircraft Performance Optimization, J. Aircraft 9, 108-114. Studies in Siebenthal, C. D. and Aris, R. (1964). Optimisation - VI. The Application of Pontryagin's Methods to the Control of a Stirred Reactor, Chem. Engng. Sci. 19, 729-746.
-
I00
SINGULAR OPTIMAL CONTROL PROBLEMS
Speyer, J. L. (1973). On the Fuel Optimality of Cruise, J. Aircraft 10, 763-765. Speyer, J. L. and Jacobson, D. H. (1971). Necessary and Sufficient Conditions for Optimality for Singular Control Problems; A Transformation Approach, J. math. Analysis Applic. 33, 163-187.
-
CHAPTER 4 Sufficient Conditions and Necessary and Sufficient Conditions for Non-Negativity of Nonsingular and Singular Second Variations 4.1
Introduction A s indicated in Chapter 2, non-negativity of the
second variation is a (second-order) necessary condition for optimality in control problems.
It is known,
also, that a sufficient condition for a weak local minimum is that the second variation be strongly positive (Gelfand and Fomin, 1 9 6 3 ) .
Unfortunately, it turns
out that in singular control problems the second variation cannot be strongly positive (Tait, 1965; Johansen, 1966) and so different tests for sufficiency have to be
devised.
Despite this fact, the second variation con-
tinues to play an important role in second-order optimality tests and it is for this reason that this chapter explores the second variation fully.
In the case of nonsingular optimal control problems, where Huu > 0 for all t in [to,tf],
it is
known that a sufficient condition for strong positivity
of the second variation, and hence for a weak local minimum, is that the matrix Riccati differential equation associated with the second variation should have a solution for all t in [to,tf].
Clearly, this
condition is inapplicable in the singular case owing -1
to the presence of Huu in the matrix Riccati equation. For a long time, therefore, it was felt that no Riccatilike condition existed for the singular case but this 101
102
SINGULAR OPTIMAL CONTROL PROBLEMS
has turned out to be not true (Goh, 1970; Jacobson, 1970a, 1971a, 1971b; Jacobson and Speyer, 1971; McDanell and Powers, 1970; Speyer and Jacobson, 1971). The result is that sufficiency conditions for nonnegativity of singular and non-singular second variations are rather closely related. We begin by proving that a necessary and sufficient condition for strong positivity of the nonsingular second variation is that a solution exists to the well known matrix Riccati differential equation.
The
sufficiency part of this theorem is very well known and documented (Brockett, 1970) but though the necessity part is well known it is not, in our opinion, proved convincingly elsewhere except in certain special cases. Following this result, we then illustrate that the singular second variation cannot be strongly positive and confirm that the standard Riccati equation is of no direct u s e .
Next, we provide a sufficient condition
(Jacobson, 1971a) for the partially singular second variation which is in the form of a set of differential and algebraic inequalities. We show how, in the nonsingular case, the standard Riccati equation is related to these conditions, and thereby implicitly establish also the necessity of these conditions.
In the totally
singular case we show that a new Riccati differential equation is related to the sufficiency conditions. We then turn our attention to necessary and sufficient conditions for non-negativity of the totally singular
4. SUFFICIENT CONDITIONS
103
second variation and produce again a set of differential and algebraic inequalities. These conditions are slightly more abstract in form than the ones presented earlier and, because they are necessary as well as sufficient, all known necessary conditions can be deduced from these.
Recent extensions of the condi-
tions (Anderson, 1973) are mentioned.
Finally, the
role of the conditions in deducing sufficient conditions f o r a weak local minimum in singular control problems and in deducing existence conditions for a solution to the standard matrix Riccati equation, is investigated. 4.2
Preliminaries The form of the second variation for an uncon-
+
strained optimal control problem is, from Chapter 2 ,
&
=
Ax + Bu; x(to)
= 0
+ We use here the notation J[u(*)]
variation. t ion.
(4.2.2)
for the second This is done for simplicity of presenta-
SINGULAR OPTIMAL CONTROL PROBLEMS
I04
n where x is in R , u is in Rm, Q,C,R,A,B are continuous matrix functions of time of appropriate dimensions, and where Q
f
is in Rnxn and is constant.
Without loss of
generality, it is assumed that Q, R and Qf are
.
symmetric
We now make three explicit assumptions; one on the class of allowed controls and the others on the nature of R and of (4.2.2). Assumption 4.1
The class of controls U from which u(-)
is drawn is the class of piecewise continuous m-vector functions of time on [to,tfl. Though a less restrictive class of controls could be assumed, the class of piecewise continuous functions
is adequate for the problems treated in this book. Assumption 4.2
A basic assumption that we make is that
In fact this assumption is not at all restrictive since if R were not positive semi-definite the criterion (4.2.1) would, regardless of any other conditions, be able to take on arbitrarily large, negative, values. Assumption 4.2 is the Legendre-Clebsch necessary condition for oprimality in the calculus of variations (Gelfand and Fomin, 1963). Assumption 4.3
We assume that the dynamic system
(4.2.2) is completely controllable on the interval
4. SUFFICIENT CONDITIONS
where t0 < t' < tf.
[to,t'],
jt",a)13(o)BT(o)@T(t'
105
That is,
(4.2.4)
,o)do > 0
where
a
E@(t,a)
= A(t)@(t,a)
; @(a,a) = I
(4.2.5)
for all t' in (to,tfl. Note that this assumption is used in only some of the results derived in this chapter. As the main objective of this chapter is to develop conditions under which ( 4 . 2 . 1 )
is non-negative,
positive definite, and strongly positive, we now define these terms precisely. Definition 4.1 J[u(-)]
is said to be non-negative if
for each u(*) in U its value is non-negative. Definition 4 . 2 J[u(*)] is said to be positive definite if for each u(*) in U, u(*) # 8 (null function), J[u(*)I
> 0.
Definition 4 . 3
JLu(*)] is said to be strongly positive
if for each u(-) in U, and some k > 0,
where 11u(.)
II
is some suitable norm defined on U.
I06
4.3
SINGULAR OPTIMAL CONTROL PROBLEMS
The N o n s i n g u l a r Case
D e f i n i t i o n 4.4
We r e f e r t o ( 4 . 2 . 1 )
R(t) > 0,
as n o n s i n g u l a r i f
for all t in [to,t
f
1.
(4.3.1)
I n t h i s case w e have t h e f o l l o w i n g theorem.
Theorem 4 . 1
A n e c e s s a r y and s u f f i c i e n t c o n d i t i o n f o r
J [ u ( . ) ] t o b e s t r o n g l y p o s i t i v e i s t h a t t h e r e exists for a l l t i n [t
t ]a f u n c t i o n S ( * )
which s a t i s f i e s
f t h e Riccati e q u a t i o n 0'
T T T -1 T - ~ = Q + s A + A S-(C+B s> R (C+B s)
(4.3.2) (4.3.3)
S ( t f ) = Qf.
I n o r d e r t o p r o v e t h i s theorem w e r e q u i r e t h e f o l l o w i n g lemma. Lemma 4 . 1
Suppose t h a t t
0
< t'
2
t f and t h a t ( 4 . 2 . 2 )
i s c o m p l e t e l y c o n t r o l l a b l e on [ t o , t '
1.
Then, there
- -
e x i s t s u * ( t ) , t o < t < t ' , which t r a n s f e r s x ( t o ) = 0 t o any d e s i r e d x a t t = t ' such t h a t t'
I bt'!1
(4.3.4)
= 1
and
I lU*W I I
2
P(t'>
(4.3.5)
where p ( * ) i s a c o n t i n u o u s f u n c t i o n d e f i n e d on ( t o , t ' l .
SUFFICIENT CONDITIONS
4.
Proof:
I07
Define
W(t 0 , t ' ) =
it
' Q ( t ' , t ) B ( t ) B T ( t ) Q T ( t ' ,t ) d t
(4.3.6)
t0
and
It i s a t r i v i a l matter t o v e r i f y t h a t t h e c o n t r o l
f u n c t i o n u * ( - ) t r a n s f e r s t h e s t a t e of system ( 4 . 2 . 2 ) from t h e o r i g i n a t t since O(t'.)
1 Ixtl I I
=
0
and W-l(t
to x 0
a t t=t'.
t'
,t')
Furthermore,
are continuous i n t ' and
1, i t i s clear t h a t t h e r e i s a continuous
f u n c t i o n p ( * ) such t h a t
Proof of Theorem 4 . 1 s a t i s f i e s (4.3.2)
F i r s t w e prove t h a t i f S ( * )
and ( 4 . 3 . 3 )
t h e n J [ u ( * )1 i s p o s i t i v e
definite. Consider t h e i d e n t i c a l l y z e r o q u a n t i t y
(4.3.9)
which, r e - w r i t t e n u s i n g ( 4 . 3 . 2 )
and (4.3.3)
is
108
SINGULAR OPTIMAL CONTROL PROBLEMS
T
T
T
+ 4x (SA+A S)x+x SBuIdt. A d j o i n i n g (4.3.10)
+
J
t o (4.2.1)
(4.3.10)
gives
4 [ u+R%+BTS) x 3 TR [ u+R'-'
( C+BTS) x 1d t
(4.3.11) which t a k e s on i t s minimum v a l u e of z e r o ( b e c a u s e x ( t o ) = 0 ) i f and o n l y i f u ( t ) = -R-'(t)
[ C ( t ) + B T ( t ) S ( t )] x ( t > ,
(4.3.12)
- -
to < t < t f .
That i s , i f and o n l y i f , u(*)
=
8 (null function).
Hence, J [ u ( * ) 1 i s p o s i t i v e d e f i n i t e .
(4.3.13)
109
4. SUFFICIENT CONDITIONS
Now consider
- uTRu)dt
T T J[u(*),E] = jtfIlx Qx+u Cx+ 2-
E
t0 (4.3.14)
This functional is positive definite if 2 - ~ > 0 and i f -i(t,E)=Q+S(t,E)A+A
T
S(t,E)-[C+B
T
S(t,E)
( ~ - E ) R-l 1T2
[C+BTS(teE) 1 S(tf,E)
=
(4.3.15)
Q,
(4.3.16)
has a solution S(t,E) defined for all t in [to' tf 1 Now, since Q,C,R,A,B are continuous in t and the right hand side of (4.3.15) is analytic in S(t,E) and E,
and since S(*,O)exists, we have that
S(*,E)
is a
continuous function of E at E=O (Coddington and Levinson, 1955). So, for E sufficiently small, S(t,E) exists for all t in [to,tf].
Therefore, for E
sufficiently small, J[u(*),E]
is positive definite.
Nerrt, we n o t e that
go
that
SINGULAR O P T I M A L C O N T R O L P R O B L E M S
110
(4.3.18)
and because of ( 4 . 3 . 1 )
we conclude from (4.3.18)
that
so that
where (4.3.21)
and
J
(4.3.22) t0
which i m p l i e s t h a t J[u(*)l i s s t r o n g l y p o s i t i v e . Now w e prove t h e converse; namely, t h a t i f J [ u ( * ) ] i s s t r o n g l y p o s i t i v e , t h e n an S ( * ) e x i s t s which s a t i s f i e s ( 4 . 3 . 2 )
and ( 4 . 3 . 3 ) .
Define
Jt' + j xT ( t , ) Q , x ( t , ) .
(4.3.23)
SUFFICIENT CONDITIONS
4.
111
Then, f o r t ' s u f f i c i e n t l y c l o s e t o t f we have by s t a n d a r d d i f f e r e n t i a l e q u a t i o n t h e o r y (Coddington and
- -
Levinson, 1 9 5 5 ) t h a t S ( t ) e x i s t s , t ' < t < t f . s e q u e n t l y , t h e minimum v a l u e of ( 4 . 3 . 2 3 ) ,
Con-
denoted by
J [ u o ( * ) , x ( t l ) , t ' ] , i s g i v e n by ( s e e ( 4 . 3 . 1 1 )
and
(4.3.12)) J [ u o ( * ), x ( t ' ) , t ' ] =
T
JX
(t')S(t')x(t')
(4.3.24)
where [ C ( t ) + B T ( t ) S ( t ) ] x ( t >,
u o ( t ) = -R-'(t)
(4.3.25)
t ' < t < t
-
f'
Now suppose t h a t S ( t ) c e a s e s t o e x i s t a t t e where < te < t '
By Lemma 4 . 1 , having
2
tf.
(4.3.26)
w e can c o n s t r u c t any d e s i r e d x ( t ' ) ,
I Ix(t') 1 I
-
= 1, u s i n g some c o n t r o l f u n c t i o n
u * ( t > , to < t < t ' which s a t i s f i e s
Now, J[u(*)] =
[t'
[I,x T Qx+u* T CX+I,U* TRu*]dt (4.3.28)
SINGULAR OPTIMAL CONTROL PROBLEMS
112
where
(4.3.29) t' < t
= uO(t)
Note t h a t because of (4.3.27)
tf.
t h e i n t e g r a l on t h e
i s f i n i t e f o r each
r i g h t hand s i d e of (4.3.28)
3 and t h a t by assumption, (4.3.28) i s nonf T n e g a t i v e . However, i f i t were t r u e t h a t x ( t ' ) S ( t ' ) x ( t ' ) t' in (t
0'
f o r some x ( t ' ) then (4.3.28) would e tend t o minus i n f i n i t y a s t ' -F te. Thus we conclude T t h a t x ( t ' ) S ( t ' ) x ( t ' ) cannot tend t o -00. Furthermore,
-+
-00
as t '
t
-+
t
i t i s c l e a r t h a t J [ u o ( * ), x ( t ' ) , t ' J'
, x ( t ' ) , t ' ] where
[;(a)
G(*)
1
i s bounded above by
i s any given piecewise
continuous f u n c t i o n d e f i n e d on [ t ' ,tf 1. Hence T x ( t ' ) S ( t ' ) x ( t ' ) cannot tend t o +co f o r some x ( t ' ) a s S i n c e , by our c o n t r o l l a b i l i t y assumption, e [x(t') = 1, we conclude t h a t x(t') i s arbitrary, t'
+
t ,
I
II
. . (to, tfl
S ( t ' ) cannot tend t o i n f i n i t y as t words, S ( t ) e x i s t s f o r a l l t i n
-f
t
e
In other We l e a v e i t
a s an e x e r c i s e f o r t h e r e a d e r t o v e r i f y t h a t , i n f a c t , S ( t ) e x i s t s f o r a l l t i n [ t , t f ] ( s e e Gelfand and Fomin, 0
1963).
4.4
S t r o n g P o s i t i v i t y and t h e T o t a l l y S i n g u l a r Second Variation I n S e c t i o n 4.3 w e demonstrated t h a t i f t h e m a t r i x
4.
SUFFICIENT CONDITIONS
113
Riccati equation has a solution for all t in [t0 then J[u(-)]
4
is not only positive definite but also
strongly positive.
Clearly, in a finite dimensional
vector space positive definiteness is equivalent to strong positivity, but in our space of piecewise continuous control functions this is not so.
In this
section we illustrate the difference between positive definiteness and strong positivity by means of a simple example.
In addition, the example illustrates the fact,
see Jacobson and Speyer (1971) and Tait (1965) for general proofs, that the totally singular second variation cannot be strongly positive.
This is of course
consistent with the fact that in the totally singular case the matrix Riccati equation is undefined (because rlR-1 = , i t ) . Before presenting the example we define "totally singular" precisely as follows: Definition 4.5
J[u(*)]
is said to be totally singular
if R(t)
=
0
for all t in [to,tfl.
(4.4.1)
Now, we consider the following totally singular functional (4.4.2) ' 0
subject to
SINGULAR OPTIMAL CONTROL PROBLEMS
I14
x(0) = 0
% = u *
u ( - ) i s a member o f U.
(4.4.3) (4.4.4)
Clearly J [ u ( * )1 i s p o s i t i v e d e f i n i t e . Set
u(t) =
COSWt,
O < t < t f
- -
(4.4.5)
-
(4.4.6)
so that
1 x ( t ) = -sin w
at,
O < t < tf '
With t h i s c h o i c e of c o n t r o l J [ u ( * ) ] becomes (4.4.7)
and
By d e f i n i t i o n , if J [ u ( . ) ] were s t r o n g l y p o s i t i v e t h e n
-
f o r some k > 0 and a l l w > 0 ,
w2
fsin2wtdt > k
(4.4.9)
(4.4.10)
4.
115
SUFFICIENT CONDITIONS
But this is impossible because the left hand side of (4.4.10) (4.4.2)
tends to i k t
as w -t to. In other words, f is not strongly positive.
Note, however, that the functional (4.4.11) ' 0
is strongly positive.
This follows directly because
> u2 x2 + u2 -
.
(4.4.12)
The fact that the totally singular second variation cannot be strongly positive implies that in the totally singular case we should seek (necessary and) sufficient conditions only for non-negativity and for positive definiteness of J[u(*)]. We begin this task in the next section. 4.5
A General Sufficiency Theorem for the Second
Variation Here we present and prove a general sufficiency theorem for the partially singular case, which is defined precisely as follows: Definition 4 . 6
J[u(*)
1
is said to be partially
singular if R(t)
2
0
for all t in [to,tf].
(4.5.1)
116
SINGULAR OPTIMAL CONTROL PROBLEMS
Following the statement and proof o f the general theorem we particularise the result to both the nonsingular and the singular special cases.
Again, we
confine our attention to the situation in which there are no terminal constraints.
See Jacobson (1971a) for
a more general treatment. Theorem 4 . 2
A sufficient condition for non-negativity is that there exists for all t in[t
of J[u(.)]
t,]
0’
a
continuously differentiable, symmetric, matrix function of time P ( * ) such that T
(C+B P)
1
C+BTP
0’
Proof:
> 0
(4.5.2)
R
for all t in [t t
Qf
T
f
-
1 and
P(tf)
(4.5.3)
0.
First, we note that for any nxn, symmetric,
continuously differentiable, matrix function of time
P(. ) , we have that
4 IXTP(Ax+Bu-~) )dt
itf
= 0.
(4.5.4)
Mow, adding this identically zero integral to J [ u ( * ) ] yields
117
4. SUFFICIENT CONDITIONS
T
T
T
T
= jtf{ix QX+U C X + ~ URu+lx P(Ax+Bu-5))dt
J[U(*)I
In view of our assumptions on u(*) and P ( * ) we can T integrate the term x P% by parts to obtain t
J[u(*)]
=
[ fii~T(t+Q+PA+ATP)x+u T (C+BTP)x+iu TRuldt
Jt
0
(4.5.6)
By inspection, then, conditions ( 4 . 5 . 2 )
and ( 4 . 5 . 3 )
f01 low. 4.5.1
The Nonsingular Special Case We now show that if ( 4 . 3 . 1 )
of a solution S(-) to ( 4 . 3 . 2 1 ,
holds then existence (4.3.3)
there exists P(*) such that ( 4 . 5 . 2 ) ,
implies that (4.5.3)
hold.
It
then follows from Theorem 4 . 1 that the conditions of Theorem 4.2 are both necessary and sufficient for positive-definiteness of J[u(*)]
in this nonsingular
case. Theorem 4.3 fies ( 4 . 3 . 2 ) ,
Suppose that an S ( * ) exists which satis(4.3.3).
Then, there exists a P(-) such
that the conditions of Theorem 4 . 2 are satisfied.
SINGULAR OPTIMAL CONTROL PROBLEMS
118
Proof:
Consider the quadratic form
where ( a l T
c1
2
T,
row vector.
is a partitioned, arbitrary, (n+m)-
Since R is positive definite the a2 that
minimizes ( 4 . 5 . 7 )
for fixed, but arbitrary, a 1 is
and the minimum value of ( 4 . 5 . 7 ) ,
using this expression,
is
T [ i + ~ + ~ ~ + ~ TT~s)-TR(-1~ (C+B + ~ T s>la1
a 1
But this is zero for all a from ( 4 . 5 . 7 )
1,
by ( 4 . 3 . 2 ) .
.
(4.5.9)
Hence we have
that
k+Q+SA+A
T
T T ’ (C+B S)
s
-> o T C+B S
(4.5.10)
R J
and
Q, - S(tf)
= 0.
(4.5.11)
SUFFICIENT CONDITIONS
4.
Identifying
I19
with P ( * ) we see that the conditions
S(*)
of Theorem 4.2 are satisfied.
4.5.2
The Totally Singular Special Case Here we have that (4.4.1) holds.
A s a consequence,
the conditions of Theorem 4.2 become for all t in [to,tfl (4.5.12)
C+BTP = 0,
T
++Q+PA+A P > 0,
for all t in [to,tf] (4.5.13)
and (4.5.14) A s in the nonsingular case, it is possible to
obtain a Riccati-type differential equation which, if it has a solution, implies the existence of P ( . ) .
In
deriving this Riccati equation we assume that B and C are once continuously differentiable with respect to
.
Furthermore, we introduce the follow0Stf1 ing, rather important, assumption. time on [t
Assumption 4.4 The first generalized Legendre-Clebsch necessary condition for optimality is satisfied in strong form for all t in [to,tf].
a .
-H
au u
which implies
=O,
t
0 -
That is, (4.5.15)
SINGULAR OPTIMAL CONTROL PROBLEMS
120
CB symmetric,
for all t in [t ,tfl 0
(4.5.16)
and -1-
a H u -*
for all t in [to,tfJ
> 0,
aU
(4.5.17)
which implies *T T - B ~ Q B + ~ B + C A B - Bc + B ~ A ~< co ~
(4.5.18)
where
H
A X T Q x + uT Cx+X T (Ax+Bu)
=
(4.5.19)
and -A
T
= Qx+A X+C
T
.
u
(4.5.20)
We shall require the following preliminary lemmas. Lemma 4.2 Proof:
T h e matrices CB and CB+CB are symmetric.
From ( 4 . 5 . 1 5 )
and ( 4 . 5 . 1 9 1 ,
(4.5.20)
we have
that
a fi au u
a
= -[-(B
d
au dt
a
=-[B au
a
= -[B
au
T
A+Cx) ]
*T A+B X+CX+CAX+CBU] TO
(4.5.21)
(4.5.22)
- A-B T~ QX-B TATX-B T CTu+~x+CAX+CBU1
(4.5.23)
4. SUFFICIENT CONDITIONS
121
T T
(4.5.24)
= C B - B C
I
=
o
for all t i n
That i s , CB i s symmetric.
[to,tfl.
(4.5.25)
Now, as C and B are con-
t i n u o u s l y d i f f e r e n t i a b l e t h i s i m p l i e s t h a t t h e lemma
i s proved. Lemma 4.3
a ii au u
*TT T T T c +B A c
T
= -B QB+~B+CAB-B
. (4.5.26)
By d i r e c t d i f f e r e n t i a t i o n and Lemma 4.2.
Proof:
Theorem 4.4
Suppose t h a t t h e r e e x i s t s a f u n c t i o n S(*)
which s a t i s f i e s f o r a l l t i n [ t f,tf1, t h e e q u a t i o n s 0
a HU1-’ [x
-i=Q+SA+ATS+[ (AB-5)T S+BTQ-CA-C]- T T
T
[ ( A B - ~ ) S+B
Q-CA-t]
(4.5.27)
(4.5.28)
-
Qf
s(tf)
2
0
.
(4.5.29)
Then, t h e r e exists a P ( * ) which s a t i s f i e s ( 4 . 5 . 1 2 )
-
(4.5.14).
Proof:
From ( 4 . 5 . 1 7 )
T i+Q+SA+A S > 0 ,
-
and ( 4 . 5 . 2 7 )
f o r a l l t i n [ t ,t o f
Next, p r e m u l t i p l y i n g ( 4 . 5 . 2 7 ) 4.2,
4.3, we obtain
we see that
T
by B
1.
(4.5.30)
and u s i n g Lemmas
SINGULAR OPTIMAL CONTROL PROBLEMS
122
- ~ = ~(C+BiTs ) A + ~ + 'TB s
(4.5.31)
which, re-arranged, is
d - -(C+B dt
T
T = s)=(c+B s ) .IA+(AB-B)
[ (AB-~)
a
-1
HuI
T ~ + ~ T ~ 1-1.~ ~ - (t4 . 5 . 3 2 )
Now this is an ordinary linear homogeneous differential T
equation for (C+B S) which has boundary condition (4.5.28).
Consequently, C+B
Ts
= 0,
for all t in [to,tfl.
(4.5.33)
Identifying P ( * ) with S ( * ) we see that the theorem is true. 4.6
Necessary and Sufficient Conditions for Nonnegativity of the Totally Singular Second Variation We have seen via Theorem 4 . 3 that, in the non-
singular case, Theorem 4 . 2 provides conditions which are both necessary and sufficient for positive-definiteness of J[u(*)].
A s a consequence of the necessity of con-
ditions ( 4 . 5 . 2 7 - 2 9 )
(McDanell and Powers, 1 9 7 0 ) subject,
4.
123
SUFFICIENT CONDITIONS
of course, to Assumption 4.4, we have necessity and sufficiency also in the totally singular case (see also Speyer and Jacobson (1971)).
The question now
arises as to the possible necessity and sufficiency of conditions (4.5.2) and (4.5.3) in the totally singular case, in the absence of Assumption 4.4 and the continuous differentiability of B and C.
It turns
out that a slightly more abstract version of these conditions is both necessary and sufficient (Jacobson and Speyer, 1971) and it is this set of conditions which occupies us in this section.
i) Necessary Condition. Under assumptions
Theorem 4.5
4.1-4.3, a necessary condition for J[u(.)
1
to be non-
negative in the totally singular case is that there exists for all t in (t 0’ tf 3 a real symmetric nxn matrix ) is monotone increasingt in function of time @ < - which t
such that T
for all t in (to,tfl
C+B P = 0, h
Qf-P(tf)
= -p(tf>
2
(4.6.1) (4.6.2)
0
where P
t i.e.,
= S+$
Th
ThP$,
z P(t)z
for all t in (t
OYtf1
(4.6.3)
is monotone increasing in t, where z belongs to Rn and is fixed but arbitrary.
SINGULAR OPTIMAL CONTROL PROBLEMS
124
i=
-$A,
$(tf)
=
(4.6.4)
I
and
-;=
T
Q+SA+A S,
s(tf) =
Qf
ii) Sufficient Condition.
(4.6.5)
In addition to
h
the above stated conditions, P ( * ) exists for all t in [to,t ] (strengthened existence condition).
f Note that the gap between the necessary and the
sufficient condition is minimal. Before proceeding to the proof of this main theorem, we verify that ( 4 . 5 . 1 2 ) - ( 4 . 5 . 1 4 ) by ( 4 . 6 . 1 ) - ( 4 . 6 . 5 )
are implied
in the case that ? ( * I is continClearly, ( 4 . 6 . 1 )
uously differentiable.
dition ( 4 . 5 . 1 2 ) and ( 4 . 6 . 2 )
is just con-
is condition ( 4 . 5 . 1 4 ) .
Now, differentiating ( 4 . 6 . 3 ) with respect to time yields
i,
=
+
iTi$+
T: $I p$I
+
$Tiql
(4.6.6)
which is T TA
T:
F+Q+SA+A~S+A$ ~ 4 - 4 P ~ + + ~ G + A= o
(4.6.7)
i.e.,
A
which, because of the monotonicity of P and the
125
4. SUFFICIENT CONDITIONS
expression ( 4 . 6 . 3 )
implies T
-o
(4.6.9)
~ + Q + P A + AP >
which is (4.5.13). The question now arises as to whether or not there are problems for which the conditions of Theorem 4 . 5 are satisfied, but for which no continuously differentiable
We supp1.y such an example below:
exists.
$ ( a )
jtfuCxdt
(4.6.10)
Jt
0
subject to ; x(to)
X’U
=
0
(4.6.11)
where we assume that C(*) is monotone decreasing in t and that
atf)
0
(4.6.12)
It is clear that
P
=
-c,
for all t in [to,tfl
A
P = P
satisfy the conditions of Theorem 4 . 5 .
(4.6.13) (4.6.14)
However, if
C is not continuously differentiable there does not exist a continuously differentiable function P(*>
I26
SINGULAR OPTIMAL CONTROL PROBLEMS
which s a t i s f i e s (4.5.12)-(4.5.14).
I n o t h e r words,
f o r t h i s example, t h e more a b s t r a c t c o n d i t i o n s of Theorem 4.5 a p p l y whereas t h e g e n e r a l s u f f i c i e n c y Theorem 4 . 2 i s i n a p p l i c a b l e . We commence t h e proof of Theorem 4.5 w i t h a
sequence of Lemmas. Lemma 4 . 4
The t o t a l l y s i n g u l a r q u a d r a t i c f u n c t i o n a l
i s e x p r e s s i b l e i n t h e e q u i v a l e n t c a n o n i c a l form
J[u(*)] = \:fuTDydt
(4.6.15)
0
subject to y = E u ;
Y(tJ
(4.6.16)
= 0
where
D
=
(4.6.17)
[C+BTS]+-l
(4.6.18)
E = +B
and where
y(t>
=
@(t)x(t), for all t in [ t
t,].
0’
(4.6.21)
4.
SUFFICIENT CONDITIONS
127
Proof: Adjoin (4.2.2) to (4.2.1) using a continuously differentiable vector multiplier A ( * )
to form
Integrating by parts and setting h(t)
=
fS(t)x(t)
(4.6.23)
yields J[u
Now, using (4.6.20) in (4.6.24) yields J[u(*)l
=
T
T
/tfu (C+B S)xdt.
J
(4.6.25)
t0
Differentiating (4.6.21) with respect to time we obtain easily the result that J[u(*)
3
=
[:fuT(C+BTS)$-lydt 0
subject to
(4.6.26)
SINGULAR OPTIMAL CONTROL PROBLEMS
128
9
+Bu ;
=
y(to) = 0
.
(4.6.27)
For t h e c o n v e r s e , we n o t e t h a t b e c a u s e (9 i s i n v e r t i b l e (4.6.26)
is just (4.6.28)
and
5
= Ax
+ Bu ;
Now, a d j o i n i n g ( 4 . 6 . 2 9 ) ”
function A(*),
(4.6.29)
x(to) = 0 t o (4.6.28)
with a multiplier
i n t e g r a t i n g by p a r t s and s e t t i n g (4.6.30)
X(t) = i ? ( t ) x ( t ) yields
J[u(-)] = j t f [ 4 x T (S+SA+ATS)x+uT(C+BTS+BT?)xldt - t0
-
- I XT ( t , ) S ( t , ) x ( t , )
.
(4.6.31)
Setting
-
S(.)
yields
=
-S(*)
(4.6.32)
SUFFICIENT CONDITIONS
4.
129
T T T J [ u ( . ) ] = j t f ( & x Qx+u C x ) d t + & x ( t f ) Q f x ( t f ) ( 4 . 6 . 3 3 ) t0
s o t h a t t h e l e m a i s proved. Lemma 4 . 5
Condition ( i ) of Theorem 4.5 i s e q u i v a l e n t
t o t h e e x i s t e n c . e f o r a l l t i n ( t , t 1 of a r e a l symmeo f t r i c monotone i n c r e a s i n g m a t r i x f u n c t i o n of t i m e P ( . ) A
such t h a t TA D+E P = 0 ,
-iqtf)
for all t in (tortf]
(4.6.34) (4.6.35)
0 .
Proof: L e t P =
$)Ti%$)+ s
(4.6.36)
and s u b s t i t u t e i n t o t h e c o n d i t i o n s of Theorem 4 . 5 and the conditions (4.6.34),
(4.6.35).
We now d e f i n e t h e r e l a t e d n o n s i n g u l a r f u n c t i o n a l
i.e.
J"U(*),E]
subject t o
=
-
Itf[uTDy + 1 u Tu l d t 2E to
(4.6.38)
130
SlNGULAR O P T I M A L C O N T R O L P R O B L E M S
(4.6.39)
Y(t0> = 0
~ = E u ;
which l e a d s t o t h e f o l l o w i n g lemmas. I f J [ u ( * ) l i s non-negative t h e n J " u ( * ) , E ]
Lemma 4 . 6
i s positive definite. Lemma 4 . 7
The proof i s obvious.
I f J [ u ( - ) ] i s non-negative t h e n t h e m a t r i x
Riccati equation
-; = E
T
-(D+E SE)
T
T
(D+E S E ) &
;
sE ( t , )
= 0
(4.6.40)
a s s o c i a t e d w i t h JN[u(*),~] h a s a s o l u t i o n which e x i s t s f o r a11 t i n [ to & f l . See Theorem 4 . 1 and Gelfand and Fomin ( 1 9 6 3 ) .
Proof:
Lemma 4 . 8
I f the matrix R i c c a t i equation (4.6.40)
has
a s o l u t i o n which e x i s t s i n t h e i n t e r v a l [ r , t f ] , t
< T < t
0 -
-
f
t h e n t h e c o n t r o l f u n c t i o n t h a t minimizes
J N T [ y ( ~ ) , u ( * ) , ~ , ~ ]=
-
Dy + 1
Tu I d t
(4.6.41)
2E
(4.6.43)
131
SUFFICIENT CONDITIONS
4.
and moreover,
(4.6.44) See Theorem 4.1.
Proof:
Lemma 4.9 parameter Proof: S,(*)
S
E
(
0
is a continuous function of the
)
E.
1 is
If J[u(.)
exists for all
non-negative it follows that E.
Furthermore, the right hand
side of (4.6.40) is analytic in S
E
and €(Coddington
and Levinson, 1955). Lemma 4.10 J'[Y(T) ,€,TI is a monotone decreasing function of E . Hence S E ( r ) is, by definition, a monotone decreasing matrix function of Proof:
For some arbitrary y(?),
Jo[y(~),
cl,
TI
=
E
E. ~
T,
we have
min ltf[uTDy +
1 uTu]dt.(4.6.45)
U('>
2E 1
Let the control which minimizes (4.6.45) be denoted u l ( * ) and its associated trajectory be y,(*).
for any
E~
->
E
1
Then,
it is clear that
(4.6.46) and by definition,
SINGULAR OPTIMAL CONTROL PROBLEMS
132
Thus, for any
-
E~ > E ~ ,we
have
A s Y(T)
and T are arbitrary, the lemma is proved. Lemma 4.11 Under Assumptions 4.1 and 4 . 3 , if J[u(.)]
is non-negative then Sm(t)
=
lim SE (t) exists for all €+a
t in (to,t ] and is negative semi-definite.
f
Proof:
From (4.6.401, SE(r)
for all
E,
0 < E <
0 for all
T
in [to,tf],
00.
By Lemma 4.10, S E ( T ) is a monotone decreasing function of
E
so that it has a limit (possibly
Now, given an arbitrary time
-a).
in the interval (t0Stf)
T
we can, by Assumption 4.3, construct u,(t),
to
2
-
t < T
such that (4.6.49)
and such that Y3(T> = Suppose that
I I S€(T)
II
arbitrary.
Y(T)
Y(T>,
-f
as
E
-f
m.
(4.6.50)
Then, by Lemmas
4 . 9 , 4.10, 4.11, Jo[y,(~),~,~]can be made large and
4.
SUFFICIENT CONDITIONS
negative for some y ( r ) and for some
E
133
sufficiently
large. This implies that one can make the following inequality hold, for
1 2~
Dy3 +
E
sufficiently large,
u3Tu33dt + J
0
[Y~(T),E,T]
< 0 . (4.6.51)
But this contradicts the fact that J[u(-)] is non-
I Is~(T) II
negative, so
+
m
as
E + m.
Since
arbitrary and since S (t ) = 0 for all E f that S,(T) exists for all T in (to,tf]. Lemma 4.12
If S,(T)
function of
T.
Proof:
E
T
is
we conclude
exists it is a monotone increasing
From (4.6.40), T
S,(d
E-tco lim[It (D+ETSE)T(D+ETSE) Edt 1 f
=
(4.6.52)
and, because of the existence of each limit, the right hand side of (4.6.52) becomes Elim[ [T+A(D+ETSE)T (D+ETSE)Edt ] tf
+ lim[ E-
T
(D+ETSE)T (D+ETSE)Edt 1.
(4.6.53)
T+A
That is, S m ( - t > = S,(T+A)+
E”
[ (D+E~S)T(D+ETS E
E
) Edt 1
(4.6.54)
SINGULAR O P T I M A L C O N T R O L PROBLEMS
134
and t h e lemma follows. I f S,(T)
Lemma 4.13
exists for a l l
T
i n (to,t,l
then
Proof: T E*
Suppose t h e c o n t r a r y .
Then, f o r some
i n ( t o , t f ) , by Lemma 4 . 9 , we have t h a t t h e r e e x i s t s > 0 such t h a t
(4.6.56) so t h a t
I,/
T
liml
E-
( D + E ~WS)T(D+ETSm)EdtI1 =
O0
(4.6.57)
tf
which c o n t r a d i c t s t h e f a c t t h a t SW(.r) e x i s t s , to <
T
1. t f .
We a r e now i n a p o s i t i o n t o prove Theorem 4 . 5 . Proof of Theorem 4 . 5
i ) Necessary Condition.
If
J [ u ( . ) ] i s non-negative t h e n by Lemmas 4.11-4.13 e x i s t s S,(T),
-
there
t o < T < t f which i s a r e a l symmetric
monotone i n c r e a s i n g m a t r i x f u n c t i o n of time such t h a t T D + E S m = O
a.e.
in [to,tfl
(4.6.58)
4. SUFFICIENT CONDITIONS
and SW(tf)
= 0
.
135
(4.6.59)
Now, since S w ( * ) is a monotone function, we can define (4.6.60) L
and consequently we have (4.6.61) Furthermore, defining < t < t
f
(4.6.62)
yields rn
D + E ~ P=
o
a.e. in [to,tfl.
(4.6.63)
is defined for all t in (to,tf].
where $(t>
In order to establish the necessary conditions we need only show that TD+E P
=
0,
for all t in (t o,tfl.
(4.6.64)
In order to do this, suppose that for some t in (t
zo
D+E~~,
.
09tf)
(4.6.65)
A s D, E are continuous in t, (4.6.65) can occur o n l y if h
a jump in P occurs a t time t that does not lie in the
I36
SINGULAR OPTIMAL CONTROL PROBLEMS
T
null space of E (t).
,.
Moreover, as P is monotone
increasing in t, it follows that
DE + E ~ + E>
o
.
(4.6.66)
Again, since $ is monotone and D and E are continuous we have that ( 4 . 6 . 6 6 ) holds during a time interval A>O,
[t, t + A ] ,
contradicting ( 4 . 6 . 6 3 ) .
D + E'?
= 0,
Equation ( 4 . 6 . 6 4 )
ii)
Hence,
for all t in (t o,tf)
.
(4.6.67)
now follows because of ( 4 . 6 . 6 0 ) .
Sufficient Condition.
Suppose now that f(t)
exists for all t in [t0' t f ] (strengthened existence condition). Adjoin the dynamics ( 4 . 6 . 1 6 ) to J[u(*) ] as follows. (4.6.68)
=
ItfuT(D+E TP>y dt
-
(4.6.69)
t0
The first integral is zero because of ( 4 . 6 . 3 4 ) .
The
remaining integral can be written in Stieltjes form as
-
yTidy
(4.6.70)
4.
I37
SUFFICIENT CONDITIONS
which, upon integration by parts, becomes (tfiyTd?y
-
iyT?yl tf
(4.6.71)
which is non-negative owing to the monotonicity of :(-)
and the fact that -?(tf)
-> 0 and
y(to)
= 0.
This concludes the proof of Theorem 4.5. It is worth noting that an extension of Theorem 4.5 is known, for the case where terminal equality constraints are present in the formulation of the optimal control problem (and hence in the second variation), (Jacobson and Speyer, 1971). 4.7
Necessary Conditions for Optimality A s Theorem 4.5 provides necessary and sufficient
conditions for non-negativity of the second variation, one would expect to be able to derive known necessary conditions, for the totally singular case, from these. This is indeed possible, as the next several results show. Theorem 4.6
(Robbins, 1967; Goh, 1966)
A necessary
condition for J [ u ( - ) ] to be non-negative is that CB be symmetric for all t in [to,tf3* Proof: From (4.6.1) T CB + B PB
=
0,
for all t in (t ,tfl. 0
(4.7.1)
Since P is symmetric it follows that CB is symmetric
SINGULAR OPTIMAL CONTROL PROBLEMS
138
.
The theorem follows from the tf3 continuity of CB with respect to time. for all t in [to,
Theorem 4.7
(Jacobson, 1969; Gabasov, 1968) A neces-
sary condition for J[u(-)] CB
+ BTSB > 0,
to be non-negative is that
for all t in [to,tfl (4.7.2)
where
.
(4.7.3)
(C+B P ) B = (C+B S+B 4 P4)B = 0,
(4.7.4)
-6 Proof:
T
= Q+SA+A S ;
S(tf)
=
Q,
From (4.6.1) and (4.6.31,
T
T
T TA
for all t in (t ,tfl. 0
AS
?(t)
-< 0 ,
for all t in [to,tfl it follows from
(4.7.4) that T
CB + B SB > 0,
for all t in (t o,tfl.
(4.7.5)
By continuity of C, B and S, inequality (4.7.2) follows. Theorem 4.8
(Kelley et al., 1967; Robbins, 1967;
Tait, 1965, generalized Legendre-Clebsch condition) necessary condition f o r J[u(*)]to
be non-negative is
that
a -. -
(-1) au Hu > 0
,
for a l l t in [to,tf]
where H i s defined by (4.5.19).
(4.7.6)
A
4.
Proof:
I39
SUFFlCIENT CONDITIONS
From ( 4 . 6 . 1 1 , ( 4 7.7)
which is dPB = 0. ( 4 . 7 . 8 )
j17(CB+lb)dt+j::(HT~B+~T~b)dt+ tl
Using ( 4 . 6 . 3 )
It2
in (4.7.8) yields
''1
( b + C i + i T P B + B T P i ) dt-
( B TQB+BT PAB+BTATPB) dt
(4.7.9)
Using (4.6.1) in ( 4 . 7 . 9 )
and collecting terms yields, (4.7.
lo)
,.
Since P is monotone increasing in t we obtain the result that (-1)
a -. H > 0, au u -
(4.7.11)
The continuity with respect to time of the left hand side of (4.7.11) yields ( 4 . 7 . 6 ) .
I40
SINGULAR O P T I M A L C O N T R O L P R O B L E M S
4.8
Other Necessary and Sufficient Conditions The necessary and sufficient conditions presented
and proved in Section 4.6 do not require that (4.5.17) hold.
However, if this strong form of the generalized
Legendre-Clebsch condition is assumed then either Goh's or Kelley's transformation technique can be used to transform the totally singular second variation into a nonsingular one.
In the case of Goh's transformation
the dimension of the state space is preserved and the ensuing analysis (Goh, 1970; McDanell and Powers, 1970) results in the Riccati differential equation of (4.5.27) Application of Kelley's transformation technique yields a nonsingular problem in a reduced dimensional state
space and hence a Riccati differential equation of reduced dimension results (Speyer and Jacobson, 1971). Both these Riccati equations imply that the second variation is strongly positive with respect to the control variable in the transformed spaces. Note that our conditions, Theorem 4.5, have been extended recently to the partially singular case by Anderson (1973). 4.9
Sufficient Conditions for a Weak Local Minimum A s pointed out in the introduction to this
chapter, a sufficient condition for a weak local minimum in the nonsingular optimal control problem is that a solution exists for all t in [to,tf] to the
4. SUFFICIENT CONDITIONS
matrix Riccati equation (4.3.2).
141
In other words, a
sufficient condition is that J[u(*)]
given by (4.2.1)
be strongly positive, where
(4.9.1)
and where (4.9.2)
In the singular case the second variation cannot be strongly positive, but sufficiency is nevertheless ensured by the strong positivity of the transformed second variation (Goh, 1970; Jacobson and Speyer, 1971; McDanell and Powers, 1970; Speyer and Jacobson, 1971). A recent advance in the theory of singular optimal
control problems is provided by Moyer (1973) who gives sufficient, but generally not necessary, conditions f o r a strong minimum. theory.
This is accomplished by a field
S I N G U L A R OPTIMAL CONTROL PROBLEMS
I42
4 . 1 0 Existence Conditions for the Matrix Riccati
Differential Equation This chapter is concerned mainly with (necessary and) sufficient conditions for non-negativity of the singular second variation.
It is an interesting fact
that the sufficient conditions ( 4 . 5 . 1 2 ) - ( 4 . 5 . 1 4 )
play
an important role in deducing conditions which guarantee existence of a solution to ( 4 . 3 . 2 - 3 ) .
The
conditions so deduced (Jacobson, 1 9 7 0 b ) are less restrictive than those known heretofore (Kalman, 1 9 6 0 ; Breakwell and Ho, 1 9 6 5 ) which we state here without proof. Theorem 4 . 9
Sufficient conditions for the existence
< t < tf which satisfies of S(t>, to -
-;=
T T -1 T T Q+SA+A S-(C+B S ) R (C+B S )
(4.10.1)
are that Q
-
-1
R
T -1
C R
-
C > 0,
> 0,
for all t in [t03tf1
(4.10.3)
f o r all t in [t
(4.10.4)
0’
tf]
(4.10.5)
SUFFICIENT CONDITIONS
4.
I43
Clearly, (4.10.1) can be written in the form
-B
- -TS-SBR-1 BTs Q+SA+A
=
(4.10.6)
where T -1
G=Q-CR
-
C
-1
A=A-BR
(4.10.7)
C.
(4.10.8)
In view of this, we shall study the question of existence of a solution to (4.10.6) but, for simplicity of notation we shall simply refer to
and
6 as A
and Q.
We then have the following theorem. Theorem 4.10 A sufficient condition for the existence
-5
=
-1 T T Q+SA+A S-SBR B S
s(tf)
(4.10.9)
Qf
=
(4.10.10)
is that there exists an nxn symmetric matrix function of time P ( - ) whose elements are continuously differentiable functions of time in the interval [t
0’
tf] such
that T B P = 0, :+Q+PA+A
T
for all t in [t o*tfl
(4.10.11)
P=M(t) > 0, for all t in [to,tf] (4.10.12)
SINGULAR OPTIMAL CONTROL PROBLEMS
144
Q,
-
P(tf)
= G,
1. 0 .
(4.10.13)
Proof: Let
where
Y(.)
of t i m e .
-. -. =
-P-S
=
and
s ( * are ) real symmetric matrix functions
Then, from ( 4 . 1 0 . 1 1 ) ,
-_
-IT--
-1 TB S PBR B P.
and ( 4 . 1 0 . 1 4 )
Q+A~(F+F)+(F+T)A-M-TBR
(4.10.15)
B (P+S)
-1 T---
Using ( 4 . 1 0 . 1 1 ) =
_-
and ( 4 . 1 0 . 1 4 )
T--Q+A (P+S) + (P+S A-M-FBR-I B~T-TBR-IB
-
..
(4.10.12)
T-Q+A (P+S)+(P+S)A-M-(P+S)BR
-PBR
-F-T
(4.10.14)
for a l l t in [t 0 ,tf]
P(t)+F(t)=P(t),
~F (4.10.16)
in ( 4 . 1 0 . 1 6 )
yields
-1 TB S+PBR B P.
-1 T--
(4.10.17)
Now choose,
-=
-P and
T-- - - I T-M+A P+PA+PBR B P
-P(tf)
= -Gf
From ( 4 . 1 0 . 1 8 ) ,
.
(4.10.19)
(4.10.18)
(4.10.19)
we see that the variable
(-F)
satisfies a Riccati equation for which the conditions of Theorem 4 . 9 hold, namely
4. SUFFICIENT CONDITIONS M(t) > 0, R-’(t)
I45
f o r a l l t i n [ to * t f l
(4.10.20)
> 0, f o r a l l t i n [ t o , t f ]
(4.10.21)
-
(4.10.22)
Gf > 0
s o t h a t F ( t ) e x i s t s f o r a l l t i n [ to’ t f 1 Now, u s i n g (4.10.18)
i n (4.10.13)
i n (4.10.17)
and (4.10.19)
yields
T---1 T-s- = Q+FA+A s SBR B s
-S ( t f )
=
(4.10.23)
Qf
(4.10.24)
which a r e i d e n t i c a l t o (4.10.91,
(4.10.10).
Further-
more, b e c a u s e P ( t ) and P ( t ) e x i s t f o r a l l t i n [ tOStf1 it follows from (4.10.14) t h a t y ( t ) exists f o r a l l t in [to,tf],
and the t h e o r e m i s proved.
The next t h e o r e m and t h e f o l l o w i n g example emphas i z e t h e v a l i d i t y o f Theorem 4.10. Theorem 4.11
The c o n d i t i o n s of Theorem 4.10 are n o t
more s t r i n g e n t t h a n t h o s e of Theorem 4.9. T -1 P r o o f : W r i t i n g Q-C R C f o r Q and A-BR-IC
for A in
t h e c o n d i t i o n s of Theorem 4.10 y i e l d s BT p = 0 ,
;+Q-C
(4.10.25)
-for a l l t in [ t o , t f l
T -1 -1 T R C+(A-BR C) P+P(A-BR-’C)=M(t)
-> 0
(4.10.26)
SINGIJLAR O P T I M A L C O N T R O L PROBLEMS
146
for all t in [t ,tf] and, 0
(4.10.27)
Clearly ( 4 . 1 0 . 2 5 ) - ( 4 . 1 0 . 2 7 ) P(.)
5
are satisfied with
0, if the conditions o f Theorem 4.9 are
satisfied. The following example illustrates that the conditions of Theorem 4.10 are less r e s t r i c t i v e than those of Theorem 4 . 9 .
4.10.1 An Example Let n=2, m=l, t =0, t =1 C=O,
Q = [
(4.10.28)
f
0
-
T
R '=1,
B =(0,1),
-1
0
0
4
(4.10.29)
Q =O
f
l o
* = l o
1 (4.10.30) 0
Clearly these values do not satisfy the conditi-ons of Theorem 4 . 9 ; that is, the known sufficiency conditions are violated. The conditions of Theorem 4.10 become
(P12
P22)
=
0,
for all t in [ 0 , 1 ]
(4.10.31)
4. SUFFICIENT CONDITIONS
for a l l t in
147
[0,11
and
(4.10.33)
Let u s now choose PJl)
(4.10.34)
= 0
which s a t i s f i e s (4.10.33)
with equality.
I f w e t h e n choose = 2
(4.10.35)
P l 1 ( 0 ) = -2
(4.10.36)
;ll
we get
and (4.10.32)
becomes -2+2t
4
> o .
(4.10.37)
I48
SINGULAR OPTIMAL CONTROL PROBLEMS
Inequality (4.10.37) holds for all t in [0,1]
so
that
the Riccati equation associated with the parameter values given in (4.10.28)-(4.10.30)
has a solution for
all t in [0,1], despite the fact that the conditions of Theorem 4.9 are violated.
4.11
Conclusion In this chapter we studied rather thoroughly the
quadratic functional that arises as the second variation in optimal control problems.
First, we proved
certain necessary and sufficient conditions for strong positivity of the nonsingular second variation and illustrated by means of an example that the totally singular second variation cannot be strongly positive. Next, we presented and proved a general sufficiency theorem for the partially singular case and specialized this in both nonsingular and totally singular cases. Following a detailed progression of lemmas we proved our main theorem in Section 4 . 6 , providing necessary and sufficient conditions for non-negativity of the singular second variation.
In the next section we
deduced certain well known necessary conditions for optimality in singular control problems.
Other neces-
sary and sufficient conditions not derived in this chapter were then referred to and briefly discussed and the question of sufficient conditions for both a weak and a strong minimum were examined.
Finally, we
4. SUFFICIENT CONDITIONS
I49
showed that the conditions of Theorem 4.2 allow us to deduce conditions for existence of a solution to the (nonsingular) matrix Riccati differential equation. These conditions are less stringent than those known heretofore. References Partially Singular LinearAnderson, B. D. 0. (1973). Quadratic Control Problems, IEEE Trans. autom. Control AC-18, 407-409.
-
Breakwell, J. V. and Ho, Y. C. (1965). On the Conjugate Point Condition for the Control Problem, Int. J. Engng. Sci. 2 , 565-579. Brockett, R. W. (1970). "Finite Dimensional Linear Systems". John Wiley & Sons, Inc., New York. "Theory of Coddington, E. A. and Levinson, N. (1955). Ordinary Differential Equations". McGraw-Hill, New York. Gabasov, R. (1968). Necessary Conditions for Optimality of Singular Control, Engng. Cybern. No.5, 28-37. &Ifand, I. M. and Fomin, S. V. (1963). "Calculus of Variations". Prentice-Hall, Englewood Cliffs, N.J. Goh, B. S. (1966). The Second Variation for the Singular Bolza Problem, SIAM J. Control 4, 309-325. Goh, B. S. (1970). A Theory of the Second Variation in Optimal Control, unpublished report, Division of Applied Mechanics, Univ. California, Berkeley. Jacobson, D. H. (1969). A New Necessary Condition of Optimality for Singular Control Problems, SIAM J. Control 7, 578-595.
-
I50
SINGULAR OPTIMAL CONTROL PROBLEMS
Jacobson, D. H. (1970a). Sufficient Conditions for Nonnegativity of the Second Variation in Singular and Nonsingular Control Problems, SIAM J. Control 8 , 403-423. Jacobson, D. H. (1970b). New Conditions for Boundedness of the Solution of a Matrix Riccati Differen8, tial Equation, J. Differential Equations 25 8-263. Jacobson, D. H. (1971a). A General Sufficiency Theorem for the Second Variation, J. math. Analysis Applic. 34, 578-589. Jacobson, D. H. (1971b). Totally Singular Quadratic Minimization Problems, IEEE Trans. autom. Control AC-16, 651-658.
-
Jacobson, D. H. and Speyer, J. L. (1971). Necessary and Sgfficient Conditions for Optimality for Singular Control Problems: A Limit Approach, J. math. Analysis Applic. 34, 239-266. Johansen, D. E. (1966). Convergence Properties of the Method of Gradients, in "Advances in Control Systems" (C. T. Leondes, ed.)Vol. 4 , pp.279-316. Academic Press, New York and London. Kalman, R. E. (1960). Contributions t o the Theory of 5, Optimal Control, Bol. SOC. Mat. Mexicana 102-119. Kelley, H. J., Kopp, R. E. and Moyer, H. G. (1967). Singular Extremals in "Topics in Optimization" (G. Leitmann, ed.) pp.63-101. Academic Press, New York. McDanell, J. P. and Powers, W. F. (1970). New JacobiType Necessary and Sufficient Conditions for Singular Optimization Problems, AIAA J. 8, 1416-1420. Moyer, H. G. (1973). Sufficient Conditions for a Strong Minimum in Singular Control Problems, SIAM J. Control 11, 620-636.
-
4.
151
SUFFICIENT CONDITIONS
Robbins, H. M. (1967). A Generalized Legendre-Clebsch Condition for the Singular Cases of Optimal Control, IBM J1 Res. Dev. 3, 361-372. Speyer, J. L. and Jacobson, D. H. (1971). Necessary and Sufficient Conditions for Optimality for Singular Control Problems; A Transformation Approach, J. math. Analysis Applic. 33, 163-187.
-
Tait, K. S. (1965). Singular Problems in Optimal Control, Ph.D. dissertation, Harvard Univ., Cambridge, Mass.
This page intentionally left blank
CHAPTER 5 Computational Methods for Singular Control Problems 5.1
Introduction
It turns out that computational methods for singular optimal control problems have not been fully developed and there remains, therefore, considerable opportunity for further, but rather difficult, research in this area.
In this chapter, we survey certain numerical methods which have appeared during the past several years.
First, we look at the problem of constructing
a matrix function P(*) which satisfies the differential inequalities of Theorem 4 . 2 and of Theorem 4 . 5 .
This
is an important numerical problem as these theorems state only that if a P(*) exists, then the second variation is non-negative; there is no hint as to how to
construct P ( * ) .
Next, we look at the problem of
computing controls which are, possibly, singular.
In
nonsingular situations there are numerous algorithms available, for example Jacobson and Mayne (1970), which iteratively improve a nominal, guessed, control function. However, in the singular case, most of these algorithms are not usable owing to the fact that certain steps in the algorithms become undefined.
The
remaining algorithms which are able to handle the singular case are of the first-order, or gradient, type, and these have been shown to behave particularly poorly, as far as rate of convergence is concerned, I53
154
SINGULAR OPTIMAL CONTROL PROBLEMS
when a p p l i e d t o s i n g u l a r o p t i m a l c o n t r o l problems (Johansen, 1966).
We t h e r e f o r e p r e s e n t a method
(Jacobson e t a l . , 1970) which c o n v e r t s a s i n g u l a r problem i n t o a sequence of n o n s i n g u l a r o n e s by t h e a d d i t i o n of a t e r m
.it
uTudt
.
T h i s e n a b l e s us t o u s e powerful second-order a l g o r i t h m s (Jacobson and Mayne, 1970).
As
E
i s progressively
reduced toward z e r o w e f i n d t h a t t h e s o l u t i o n of t h e E-problem t e n d s t o t h a t o f t h e o r i g i n a l , p o s s i b l y s i n g u l a r , one.
I n t h i s c o n n e c t i o n I$C r e f e r t o Powers
and McDanell (1971) who have s u c c e s s f u l l y u s e d t h i s t e c h n i q u e i n t h e s i n g u l a r S a t u r n g u i d a n c e problem,
A r e c e n t g e n e r a l i z e d g r a d i e n t method due t o Mehra and Davis (1972) i s t h e n mentioned as are c e r t a i n r e c e n t experiments u s i n g f u n c t i o n space v e r s i o n s of Davidon's and Broyden's p a r a m e t e r o p t i m i z a t i o n methods (Edge and Powers, 1974a, 1974b). We t h e n t u r n o u r a t t e n t i o n t o t h e s i n g u l a r l i n e a r -
q u a d r a t i c o p t i m a l c o n t r o l problem and g i v e r e c e n t , r e l e v a n t , r e f e r e n c e s ( J a c o b s o n , 1972; Moore, 1969; Moylan and Moore, 1971; Wonham and Johnson, 1964). F i n a l l y , w e emphasize c e r t a i n work by McDanell and Powers (1971) and Maurer (1974) who have o b t a i n e d i n t e r e s t i n g c o n d i t i o n s which i n c r e a s e u n d e r s t a n d i n g of j u n c t i o n s between s i n g u l a r and n o n s i n g u l a r arcs.
5. COMPUTATIONAL METHODS 5.2
155
Computational Application of the Sufficiency ‘Conditionsof Theorems 4.2 and 4.5 In this section we investigate certain methods for
constructing a matrix function P ( - ) which satisfies the conditions of Theorem 4.2 and/or Theorem 4 . 5 . 5.2.1
The Nonsingular Case
In the nonsingular case it is known, and proved in Theorem 4 . 3 , that if S ( * )
exists which satisfies
TS-(C+B Ts) TR-1 (C+BTs) -~=Q+sA+A
(5.2.1)
(5.2.2)
then P ( * )
= S(*)
satisfies the conditions of Theorem
4.2.
Furthermore, see Theorem 4 . 1 ,
S(t),
to < t < tf is also a necessary condition for
-
the existence of
non-negativity of the second variation so that necessity of the conditions of Theorem 4.2 in the nonsingular case is implied.
Thus it turns out that, in the non-
singular case, integration of ( 5 . 2 . 1 1 ,
which can be
accomplished numerically, is an altogether satisfactory way of constructing P ( * ) . 5.2.2
The Totally Singular Case If we are prepared to make the assumption that (-1)
a H*. au u
> 0,
for all t in [t
0’
tf1(5.2.3)
156
SINGULAR OPTIMAL CONTROL PROBLEMS
we have, from Theorem 4 . 4 , that the existence of a function S ( . ) which satisfies, for all t in [t.,tfl, the equations
T T T * T -;=Q+SA+A S + [ (AB-i) S+B Q-CA-C]
a ..
-1
[uHU 1
[ (AB-i) TS+BTQ-CA-t 1
T C(t ) + B (tf)S(tf> E
(5.2.4)
(5.2.5)
= 0
implies that P ( * ) = S ( * ) satisfies the conditions of Theorem 4 . 2 .
This is a Riccati differential equation
but, unlike ( 5 . 2 . 1 1 ,
equation ( 5 . 2 . 4 )
constructs an
S ( - ) which satisfies
T
C+B S = 0
,
for all t in [to,tfl*
Again, see McDanell and Powers (19701,
(5.2.7)
existence of
S ( * ) turns out to be a necessary condition for non-
negativity of the second variation s o that the conditions of Theorem 4 . 2 are also necessary, subject to condition ( 5 . 2 . 3 ) , variation.
for non-negativity of the second
The necessity of the existence of S(*) is
obtained by means of Goh's transformation technique in (McDanell and Powers, 1970).
5. COMPUTATIONAL METHODS
157
The f a c t t h a t t h e S ( * ) which s a t i s f i e s ( 5 . 2 . 4 ) (5.2.6)
a l s o s a t i s f i e s (5.2.7)
-
r a i s e s t h e q u e s t i o n of
whether o r n o t , i n t h e s i n g u l a r c a s e , t h e r e e x i s t s a R i c c a t i d i f f e r e n t i a l e q u a t i o n of lower dimension, of t h e same form as (5.2.1).
That t h i s i s indeed t r u e i s
demonstrated by Speyer and Jacobson (1971) u s i n g K e l l e y ' s t r a n s f o r m a t i o n technique.
It i s t h i s t y p e of
r e d u c t i o n of t h e dimension of t h e s t a t e space t o produce a n o n s i n g u l a r second v a r i a t i o n which i s e x p l o i t e d by Anderson and Moylan (1973, 1974). I f (5.2.3)
i s not s a t i s f i e d then the singularity
i s deeper and one of t h e h i g h e r o r d e r g e n e r a l i z e d Legendre-Clebsch c o n d i t i o n s must t u r n o u t t o b e s a t i s f i e d w i t h s t r i c t i n e q u a l i t y i f a Riccati-approach t o t h e c o n s t r u c t i o n of P(*) i s t o b e used.
It i s t h e r e -
f o r e c l e a r from t h e s e remarks t h a t a method f o r cons t r u c t i n g P ( * > which i s g e n e r a l , i n t h e s e n s e t h a t i t does n o t depend upon (5.2.3),
is preferable.
Such a
method i s d i s c u s s e d next. 5.2.3
A L i m i t Approach t o t h e C o n s t r u c t i o n of P(*)
The proof of Theorem 4.5 i s based on t h e f a c t t h a t t h e s i n g u l a r second v a r i a t i o n can b e made n o n s i n g u l a r by a d d i t i o n of t h e term
and t h a t
E
can b e p r o g r e s s i v e l y i n c r e a s e d s o t h a t
SINGULAR OPTIMAL CONTROL PROBLEMS
158
lim s (.) E”
= p(.)
(5.2.8)
&
where
SE(tf)
=
Qf
.
(5.2.10)
Computationally, constructing P ( - ) according to (5.2.8)
has both an advantage and a disadvantage.
disadvantage is rather obvious; namely, as right hand side of ( 5 . 2 . 9 )
E-,
The
the
becomes ill defined.
However, possible numerical difficulties can be reduced by solving ( 5 . 2 . 9 )
iteratively using a Newton or quasi-
linearization method in which the solution for c=cl is used as a starting solution for the solution for E = E ~ > E ~ .
This is, in principle, a more stable approach
than simply setting grating ( 5 . 2 . 9 )
E
to a very large value and inte-
backwards from t
f numerical integration techniques. The advantage of using (5.2.8)
using standard is as follows:
Often, we are not interested in the actual value of
P(-) but rather whether or not a P ( * ) exists which satisfies the conditions of Theorem 4 . 5 .
In other words we may wish to construct P ( . ) only to verify its existence. Now, if no such P ( * ) exists we have from the lemmas preceding the proof of Theorem 4 . 5 , and because of the continuity of S ( * ) with respect to c , E
5. COMPUTATIONAL METHODS
that for some
E
159
sufficiently large but f i n i t e , SE(t)
will cease to exist for some t in (to,tf].
Thus the
non-existence of a P(*) which satisfies the conditions of Theorem 4.5 can be inferred from the non-existence of S E ( * ) ,
~ < m ,E
sufficiently large.
In other words,
if no P(*) exists, it is not necessary to compute
lim S , ( * ) to confirm this; rather the integration of
€-
(5.2.9)
may be halted as soon as a finite escape time
is detected in S E ( * ) . To our knowledge, this limit approach for constructing, or testing the existence of, P ( * ) is the only one which is independent of assumptions such as ( 5 . 2 . 3 ) . 5.3
Computation of optimal Singular Controls We now corn to the question of synthesis of
optimal control functions for, possibly, singular control problems.
Of course, if it is known a-priori
that the optimal control function has singular sub-arcs, and if the number and location of these sub-arcs are known, then special techniques can be devised to deal with this situation (Anderson, 1 9 7 2 ) .
However, if
little a-priori knowledge is at hand, an algorithm which can cope with singular sub-arcs, as and if these arise, is most attractive. 5.3.1
Preliminaries We shall consider the problem of controlling
I 60
SINGULAR OPTIMAL CONTROL PROBLEMS
jr=fl(x,t) + fu(x,t)u;
(5.3.1)
x ( t o ) = x0
where x ( t ) i s i n Rn, u ( t ) i s i n Rm and where u ( t ) i s c o n s t r a i n e d i n t h e f o l l o w i n g way: .lui(t)l
2
1 for a l l t i n
[to,tfI,
...,m.
i=l,
(5.3.2) The performance index, o r c o s t f u n c t i o n a l , i s
where t o and t f are given.
The f u n c t i o n s f l ( x , t ) ,
f U ( x , t ) , L ( x , t ) and F [ x ( t f ) ] are assumed t o b e a t
least once c o n t i n u o u s l y d i f f e r e n t i a b l e i n each argument. Our problem i s t o choose u(*), p i e c e w i s e cont i n u o u s i n t i m e , which s a t i s f i e s (5.3.2),
t o minimize
J [ u ( * ) 1. It i s w e l l known t h a t , i n t h e absence of c e r t a i n
f u r t h e r assumptions, t h e optimal c o n t r o l f u n c t i o n f o r t h i s c l a s s of problems c o n s i s t s of bang-bang and s i n g u l a r sub-arcs.
Of c o u r s e , i f t h e o p t i m a l c o n t r o l
i s p u r e l y bang-bang
(i.e.
nonsingular) t h e techniques
of Jacobson and Mayne, (1970) can handle t h e problem. 5.3.2
An €-Algorithm We develop h e r e an a l g o r i t h m f o r s o l v i n g t h e
COMPUTATIONAL METHODS
5.
161
problem formulated in Section 5.3.1 which is applicable whether or not singular sub-arcs exist in the optimal control.
Basically, we convert (5.3.3)
into a non-
singular functional by the addition of a term
“itfT 2
u udt.
t0
Minimizing this nonsingular functional, using for example the techniques described by Jacobson and Mayne (1970),
for a sequence of
lim ‘k = 0 kNo assumptions as
such that
E ~ ’ S
yields the optimal value of (5.3.3).
to the number and position of singular sub-arcs need be made. First we define J[u(*)
, E ~ ]
=
If‘ L
E
[L(x,t)
k T + -U2 u]dt+F[x(tf)
I
(5.304)
0
where Ek > 0
(5.3.5)
Clearly this ck-problem is nonsingular and can be solved using the methods of Jacobson and Mayne (1970). Our E-Algorithm is then as follows: Step 1.
Choose a starting value control function Z1( * I .
Step 2.
E
1
> 0 and a nominal
Solve the resulting Ek-problem ( k = l initially)
S I N G U L A R OPTIMAL CONTROL PROBLEMS
I62
using the methods of Jacobson and Mayne
(1970); this yields a minimizing control function Step 3 .
% ( a ) .
< Choose E k+l -
E
k
(for example,
(*)=u,(-), k=k+l, and
set %+I
E
k+l
=E
/lo),
k
go to Step 2.
In a practical case the algorithm can be halted, after Step 2 , when E
k
where
CJ
(5.3.6)
<(s
is a small, positive, pre-determined number.
We prove the convergence of the €-Algorithm under the following assumptions. Assumption 5.1
Let U be the set of piecewise contin-
uous m-vector functions of time defined on [t OStf1 Then which satisfy (5.3.2). (5.3.7)
where u(*) belongs to U and v0 belongs to R1. Assumption 5.2 inf J[u(*),E~] = J[u~(.),E k 1 is obtained in U.
U(')
That is, \ ( * ) ,
the control which minimizes J[u(-),E
belongs to U. These two assumptions allow us to prove the following lemma. Lemma 5.1
For k>R(Ek<EL)
we have
k
]
5.
J[q(*>
COMPUTATIONAL METHODS
,Ek1
That is, J[q(*) function of
E
k'
I63
(5.3.8)
J[uQ(*) , E ~ ]
is a monotonically decreasing
Moreover, we have that (5.3.9)
u(*) belongs to U. Proof: We have that
(5.3.10)
In addition,
(5.3.11)
We now state and prove our main theorem. Theorem 5.1
For a positive sequence
{E
1,
E >E
k k k+1" and lim E:k =0, and under Assumptions 5.1, 5.2, k(5.3.12)
Proof:
Since, by Lemma 5.1, (J[%(-),E~]~ i s monotone
decreasing and bounded below by vo, it must converge.
164
SINGULAR O P T I M A L CONTROL P R O B L E M S
Suppose, then, that (5.3.13)
But then, because of (5.3.2)
and the fact that
-m
E
R
belonging to
{E
k
1,
E
R
where uo(-) is defined by min J[u(.)]
= J[uo(*)]
U(.>
Inequality (5.3.14)
= v
0 '
u(-) in U.
(5.3.15)
then implies that
But this implies, since {J[u~(.),E~]} is a monotone decreasing sequence that (5.3.17)
Hence, v7e have a contradiction and so (5.3.12) must be true.
It can happen that, as E k approaches zero, numerical ill-conditioning manifests itself. This is especially the case if second-order numerical methods
are used, as then terms involving - occur (Jacobson E k 1
5.
et al., 1970). I1
COMPUTATIONAL METHODS
165
However, it must be acknowledged that
a sufficiently good approximation" to the solution can
often be obtained by reducing Ek to a small, but still numerically stable, value. Indeed, Powers and McDanell (1971) report a successful application of the €-Algorithm to the important Saturn guidance problem. 5.3.3
A Generalized Gradient Method for Singular Arcs
Recently, Mehra and Davis (1972) exploited the observation that the independent variables in a computational procedure for determining optimal control do not have to be only the control variables.
They show
that a judicious mixture of control and state variables as independent variables can result in the control problem being nonsingular in these variables, and hence standard algorithms such as gradient or conjugate gradient can be used to advantage.
Though various,
relatively simple, problems are solved by Mehra and Davis (1972) using this idea, it seems to us that the procedure is not yet systematized to a point where it can be programmed into a package and used routinely. Thus it appears that additional research, in developing further these ideas, is necessary. 5.3.4
Function Space Quasi-Newton Methods Perhaps the most encouraging numerical experiments
in the computation of optimal (singular) controls are those of Edge and Powers (1974a, 1974b).
The approach
166
SINGULAR OPTIMAL CONTROL PROBLEMS
h e r e i s t o u s e t h e powerful Davidon and Broyden parameter o p t i m i z a t i o n methods by s u i t a b l y e x t e n d i n g t h e s e t o f u n c t i o n space.
The r a t e of convergence
experienced when u s i n g t h e s e methods i s c o n s i d e r a b l y f a s t e r t h a n i n t h e case of g r a d i e n t and c o n j u g a t e g r a d i e n t methods, and t h e f u n c t i o n space quasi-Newton methods a r e s i m p l e r t o program than f u l l second-order algorithms.
I n addition, only gradient information
needs t o be c a l c u l a t e d , whereas second d e r i v a t i v e s of t h e dynamic system and performance i n t e g r a n d need t o be o b t a i n e d e x p l i c i t l y i n t h e c a s e of second-order a1 gor i thms 5.3.5
.
Outlook f o r F u t u r e I t i s c l e a r from t h e above sub-sections
t h a t much
r e s e a r c h needs t o be done i n t o t h e problem of computat i o n of optimal ( s i n g u l a r ) c o n t r o l s ,
T h i s i s empha-
s i z e d by t h e f a c t t h a t t h e most promising experiments (Edge and Powers, 1974a, 1974b) a r e only of r e c e n t vintage. 5.4
J o i n i n g o f Optimal S i n g u l a r and Nonsingular Sub- A r c s I n t h i s book our a t t e n t i o n h a s been focussed mainly
on problems t o t a l l y s i n g u l a r f o r a l l t i n [ t o , t f
I.
However, p r a c t i c a l problems u s u a l l y e x h i b i t a m i x t u r e ,
o r c o n c a t e n a t i o n , of n o n s i n g u l a r and s i n g u l a r sub-arcs i n t h e i r optimal c o n t r o l f u n c t i o n s .
Here we g i v e a
5. COMPUTATIONAL METHODS
167
b r i e f survey of r e s u l t s i n t h i s area. Study of j u n c t i o n c o n d i t i o n s i n l i n e a r - q u a d r a t i c problems, where c o n d i t i o n (5.2.3)
i s s a t i s f i e d , has
y i e l d e d a r a t h e r complete p i c t u r e (Jacobson, 1972; Moore, 1969; Moylan and Moore, 1971; Wonham and Johnson, 1964), b u t i n higher-order
s i n g u l a r problems, j u n c t i o n s
between s i n g u l a r and n o n s i n g u l a r segments are n o t It i s f a i r t o s a y , however, t h a t
e n t i r e l y understood.
McDanell and Powers (1971) have provided numerous i n s i g h t s i n t o j u n c t i o n c o n d i t i o n s v i a a number of theorems, one of which w e s t a t e below w i t h o u t proof. It i s important t o n o t e t h a t t h e work of McDanell and
Powers w a s s t i m u l a t e d by t h e p i o n e e r i n g c o n t r i b u t i o n s of Kelley e t a l . , Theorem 5.2
Let t
(1967). C
be a p o i n t a t which s i n g u l a r and
n o n s i n g u l a r sub-arcs of an optimal c o n t r o l u ( * ) are j o i n e d , and l e t q be t h e o r d e r of t h e s i n g u l a r a r c . Suppose t h a t t h e s t r e n g t h e n e d , g e n e r a l i z e d LegendreClebsch n e c e s s a r y c o n d i t i o n i s s a t i s f i e d a t t c ; i . e . (5.3.18) and assume t h a t t h e c o n t r o l i s piecewise a n a l y t i c i n a neighbourhood of tc.
-
L e t u ( ~ )( r > 0 ) b e t h e lowest
o r d e r t i m e d e r i v a t i v e of u t h a t i s d i s c o n t i n u o u s a t t
C
.
Then, q + r i s an odd i n t e g e r . Corollary 5.1
I n q-even problems, assuming u i s piece-
wise a n a l y t i c and t h e s t r e n g t h e n e d , g e n e r a l i z e d Legendre-
168
SINGULAR OPTIMAL CONTROL PROBLEMS
Clebsch condition is satisfied, the optimal control is continuous at each junction.
See Maurer ( 1 9 7 4 ) for an
example. Corollary 5.2
In q-odd problems, assuming u is piece-
wise analytic, and the strengthened, generalized Legendre-Clebsch condition is satisfied, the optimal control either has a jump discontinuity at each junction, or else the singular control joins the boundary smoothly; i.e. with a continuous first derivative.
5.5
Conclusion In this Chapter we first examined the question of
the construction of matrix functions of time which satisfy certain (necessary and) sufficient conditions for non-negativity of the second variation.
It turns
out, as is well known, that integration of the conventional matrix Riccati differential equation accomplishes this construction in the nonsingular case.
In the
totally singular case, under certain assumptions, a new Riccati differential equation is used in this role.
In
general, however, in the absence of these assumptions, the construction o f the required matrix functions of time is not routine. approach
-
The suggested method
-
a limit
while having certain numerical difficulties
associated with it, appears to be the most useful general technique available. Next, we considered the problem of computation of optimal singular controls.
This is, in many respects,
5. COMPUTATIONAL METHODS
169
a similar problem to that discussed above in that the most flexible technique available turns out to be the €-Algorithm, which is also a limit method.
However,
recent experiments using Davidon and Broyden algorithms in function space are very encouraging, Finally, we discussed the joining of singular and nonsingular sub-arcs.
In the linear-quadratic problem,
under the assumption that the generalized LegendreClebsch condition is satisfied for q=l with strict inequality, the situation is rather well understood. However, many difficult questions remain in the case of higher-order singular arcs. We conclude, then, by noting that it is rather clear that further research needs to be done on computational methods in general and on computation of optimal singular controls in particular. References Anderson, B. D. 0. and Moylan, P. J. (1973). Spectral Factorization o f a Finite-Dimensional, Nonstationary Matrix Covariance, Tech. Rep. EE-7303, Univ. Newcastle, Australia. Anderson, B. D. 0. and Moylan, P. J. (1974). Synthesis of Linear Time-Varying Passive Networks, IEEE Trans. Circuits and Systems CAS-21, 678-687.
-
Anderson, G. M. (1972). An Indirect Numerical Method for the Solution of a Class of Optimal Control Problems with Singular Arcs, IEEE Trans. autom. Control AC-17, 363-365.
-
Edge, E. R. and Powers, W. F. (1974a). Shuttle Ascent Trajectory Optimization with Function Space Quasi-
170
SINGULAR OPTIMAL CONTROL PROBLEMS
Newton Techniques, AIAA Mechanics and Control of Flight Specialist Conference, Anaheim, California. Edge, E. R. and Powers, W. F. (1974b). Function Space Quasi-Newton Algorithms for Optimal Control Problems with Bounded Controls and Singular Arcs, unpublished paper, Univ. Michigan, Ann Arbor, Jacobson, D. H. (1972). On Singular Arcs and Surfaces in a Class of Quadratic Minimization Problems, J. math. Analysis Applic. 37, 185-201.
Jacobson, D. H. and Mayne, D. Q. (1970). "Differential Dynamic Programming", Elsevier, New York.
Jacobson, D. H., Gershwin, S. B. and Lele, M. M. (1970). Computation of Optimal Singular Controls, IEEE Trans. autom. Control AC-15, - 67-73. Johansen, D. E. (1966). Convergence Properties of the Method of Gradients, in "Advances in Control Systems" (C. T. Leondes, ed) V o 1 . 4 , pp.279-316. Academic Press, New York. Kelley, H. J., Kopp, R. E. and Moyer, H. G. (1967). Singular Extremals, in ''Topics in Optimization" (G. Leitmann, ed) pp.63-101. Academic Press, New York. Maurer, H. (1974). An Example of a Continuous Junction for a Singular Control Problem of Even Order, Department of Mathematics, Univ. British Columbia, Vancouver, Canada. McDanell, J. P. and Powers, W. F. (1970). New JacobiType Necessary and Sufficient Conditions for Singular Optimization Problems, AIAA J. 2, 1416-1420. McDanell, J. P. and Powers, W. F. (1971). Necessary Conditions for Joining Optimal Singular and Nonsingular Subarcs, SIAM J. Control 9, 161-173.
-
Mehra, R. K. and Davis, R. E. (1972). A Generalized Gradient Method for Optimal Control Problems with Inequality Constraints and Singular Arcs, IEEE Trans. autom. Control AC-17, 69-79.
-
5.
171
COMPUTATIONAL METHODS
Moore, J. B. (1969). A Note on a Singular Optimal Control Problem, Automatica 5, 857-858. Moylan, P. J. and Moore, J. B. ( 1 9 7 1 ) . Generalizations of Singular Optimal Control Theory, Automatica L, 591-598.
Powers, W. F. and McDanell, J. P. ( 1 9 7 1 ) . Switching Conditions and a Synthesis Technique for the Singular Saturn Guidance Problem, J. Spacecraft Rockets 8, 1027-1032.
-
Speyer, J. L. and Jacobson, D. H. (1971). Necessary and Sufficient Conditions for Optimality for Singular Control Problems; A Transformation Approach, J. math. Analysis Applic. 33, 163-187. Wonham, W. M. and Johnson, C. D. ( 1 9 6 4 ) . Optimal BangBang Control with Quadratic Performance Index, Trans. ASME, J. Basic Eng. 8 6 , 107-115.
-
-
This page intentionally left blank
CHAPTER 6 Conclusion 6.1
The Importance of S i n g u l a r Optimal Control Prob l e m s The word " s i n g u l a r " i n t h e c o n t e x t of optimal
control theory has i t s o r i g i n s i n t h e c l a s s i c a l calculus of v a r i a t i o n s i n which a v a r i a t i o n a l problem i s r e f e r r e d t o as s i n g u l a r i f t h e Legendre-Clebsch n e c e s s a r y condition i s not s a t i s f i e d with strict inequality.
I n con-
temporary c o n t r o l t h e o r y we are c o n s i s t e n t w i t h t h i s d e f i n i t i o n of s i n g u l a r i t y i n r e f e r r i n g t o t h e second v a r i a t i o n as t o t a l l y s i n g u l a r i f Huu = O
f o r a l l t i n [ tO ' t f l ,
(6.1.1)
as p a r t i a l l y s i n g u l a r i f HUU
-> o
for a l l t i n [t ,t o f
3,
(6.1.2)
and as n o n s i n g u l a r i f Huu > O
for all t in [to,tfl.
(6.1.3)
Thus, s i n g u l a r optimal c o n t r o l problems are " s i n g u l a r " i n a mathematical s e n s e ; i . e . det(HUu) = 0.
It t u r n s
o u t , though, t h a t s i n g u l a r problems a r i s e f r e q u e n t l y i n e n g i n e e r i n g and are, t h e r e f o r e , n o t a t a l l " s i n g u l a r " o r uncommon i n t h e realm of a p p l i e d optimal c o n t r o l theory.
It was t h i s r e a l i z a t i o n , which probably began
w i t h Lawden's r o c k e t problem, which s t i m u l a t e d t h e I73
SINGULAR O P T I M A L C O N T R O L PROBLEMS
174
l a r g e r e s e a r c h e f f o r t on t h e s e problems d u r i n g t h e 1 9 6 0 ' s and e a r l y 1 9 7 0 ' s and which s t i l l m o t i v a t e s u s t o develop e f f i c i e n t computational techniques. The importance of s i n g u l a r o p t i m a l c o n t r o l problems
is therefore evident.
On t h e one hand, s i n g u l a r
problems e x h i b i t many i n t e r e s t i n g and deep t h e o r e t i c a l n i c e t i e s and on t h e o t h e r hand t h e y a r i s e , and a r e t h e r e f o r e of p r a c t i c a l s i g n i f i c a n c e , i n e n g i n e e r i n g and other disciplines.
Our a i m i n t h i s monograph, t h e r e -
f o r e , i s t o g i v e b o t h s i d e s of t h e s t o r y by r e f e r r i n g i n C h a p t e r s 1-3
t o c e r t a i n of t h e s i n g u l a r problems o f
p r a c t i c a l importance, i n Chapter 5 t o computational t e c h n i q u e s f o r t h e i r s o l u t i o n , and i n C h a p t e r s 2-4
to
n e c e s s a r y , and n e c e s s a r y and s u f f i c i e n t c o n d i t i o n s f o r optimality.
I n t h e remainder of t h i s c h a p t e r w e d i s c u s s
c e r t a i n a s p e c t s of t h e r e s u l t s p r e s e n t e d i n C h a p t e r s
1-5 i n o r d e r t o s t r e s s t h e i r r a t h e r c e n t r a l t h e o r e t i c a l and p r a c t i c a l r o l e s . 6.2
Necessary C o n d i t i o n s I t i s w e l l known t h a t t h e a p p l i c a b i l i t y of
P o n t r y a g i n ' s P r i n c i p l e i s u n a f f e c t e d by s i n g u l a r i t y , o r lack thereof.
However, i n t h e ( p a r t i a l l y ) s i n g u l a r
i s n o t i n v e r t i b l e and t h e r e f o r e t h e s u f f i c i e n t uu c o n d i t i o n s developed e s p e c i a l l y f o r t h e n o n s i n g u l a r case H
case a r e inapplicable.
T h i s l a c k of s u f f i c i e n t condi-
t i o n s meant t h a t t h e o p t i m a l i t y , o r l a c k t h e r e o f , of Lawden's s p i r a l i n t h e a e r o s p a c e f i e l d c o u l d n o t b e
6.
CONCLUSION
175
checked and this stimulated researchers in the quest for further necessary conditions.
Research then
yielded the generalized Legendre-Clebsch necessary condition of Goh, Kelley et al., and Robbins, which decided the fate of Lawden's spiral.
It is rather
interesting to note that Kelley et al. utilized special control variations of complicated nature to yield the generalized Legendre-Clebsch condition, but these researchers neglected to investigate further the affect of the simplest variation
-
a rectangular pulse.
Presumably, they reasoned that this simple variation, which yields the classical Legendre-Clebsch condition in the nonsingular and partially singular cases, could not yield any additional useful information. Remarkably, use of this simple variation does yield additional results as evidenced by Jacobson's necessary condition which emerged circa 1969 and which paved the way for the development of necessary and sufficient conditions. A further consequence of the search for additional
necessary conditions was the development of Kelley's and Goh's transformation techniques. These, in turn, played an important role in the development of necessary and sufficient conditions. 6.3 Necessary and Sufficient Conditions In the (partially) singular control problem the conventional Riccati differential equation is not -1
defined, owing to the presence of Huu in its right hand
176
SINGULAR OPTIMAL CONTROL PROBLEMS
side.
F o r a long time, then, it was felt that no
sufficiency theory analogous to that for nonsingular problems could be expected in the singular case.
This
is, in retrospect, a somewhat surprising conclusion to reach as Riccati-like inequalities were already known in network theory in connection with passivity which is closely related to the non-negativity of a singular quadratic functional. Shortly after the advent of Jacobson’s necessary condition a set of algebraic and differential inequalities, closely related to conditions in network theory, were developed which are sufficient for non-negativity of the singular second variation.
At this point in
time the quest for conditions which are both necessary and sufficient intensified.
Two different approaches
were used; namely, a limit approach of Jacobson and Speyer and the transformation approaches of Goh, Powers, and Speyer and Jacobson. The limit approach of Jacobson and Speyer, fully detailed in Chapter 4 , converts the singular second variation into a nonsingular one by the addition of the term
2 21 E
uTud t t0
.
The conventional Riccati equation for this nonsingular problem is then studied in the limit as
E-,
yielding
6. CONCLUSION
177
necessary and sufficient conditions for non-negativity of the original singular variation.
This limit approach
has proved to be rather powerful in that minimal differentiability assumptions are required.
Furthermore, all
known necessary conditions can be recovered by manipulating, in a rather simple way, these conditions.
In
certain cases it is possible to exhibit the necessary and sufficient conditions in Riccati differential equation form thereby demonstrating the close relationship between singular and nonsingular problems. The transformation approaches are designed to transform the singular problem into a nonsingular one; the conventional necessary and sufficient conditions (conventional Riccati equation) are then applicable. In the Speyer and Jacobson approach, Kelley's transformation technique converts the singular problem into a nonsingular one in a state space of reduced dimension. Goh's transformation, on the other hand, as exploited by McDanell and Powers, preserves the dimensionality of the original state space. A s Goh's transformation is simpler to apply than Kelley's we use it here to illustrate the role of transformation techniques in the development of the Riccati equation for singular problems which is stated and used, but not derived, in Chapter 4 . We consider the totally singular quadratic prob 1em,
178
SINGULAR OPTIMAL CONTROL PROBLEMS
J[u(.)]
=
I t f ( & x T Q x +TuCx)dt + i xT ( t , ) Q f x ( t f )
(6.3.1)
where j , = Ax+Bu ;
x(t ) = x 0
0
.
Goh's t r a n s f o r m a t i o n technique i s a s follows.
(6.3.2) First,
define
(6.3.3) and z = x-Bv
.
(6.3.4)
Then, we o b t a i n e a s i l y t h e d i f f e r e n t i a l e q u a t i o n ,
H
= Az
+ (AB-6)v ; z(t,)
= 0
(6.3.5)
and T T T T T T *T J [ v ( * ) ] = I t f ( i z Qz+v B Qz+iv B QBv+; Cz+v CBv)dt
(6.3.6) I n t e g r a t i n g the terms i n yields
by p a r t s and r e - a r r a n g i n g
6 . CONCLUSION
179
T
+ icT[CB-(CB) ]v)dt
+ i vT (t,) [CB+BTQfBlv(tf) (6.3.7) Now, i f the f i r s t generalized Legendre-Clebsch neces-
sary condition is s a t i s f i e d i n s t r o n g form, i . e . CB = (CB)
T
(6.3.8)
and (6.3.9) and i f we assume t h a t
T
C + B Q f = O
(6.3.10)
at tf,
then, ~[V J [ v ( * ) ] = I t f i i z TQz+vT (BTQ - C A - ~ ) Z +~a t0
-H au
u
]v)dt
SINGULAR OPTIMAL CONTROL PROBLEMS
180
which is a standard, nonsingular quadratic functional. A necessary and sufficient condition for strong positivity for (6.3.11) is, then, that there exists S ( - ) which satisfies T
C+B S(tf)
=
0
(6.3.12)
T T T T *T T -~=Q+sA+AS+(B Q-CA-~+BA S-B s>
a H.. -1 (BTQ - c A - ~ + B ~ A ~ s - ~ ~ s ) au u
(6.3.13)
where
On the application of Kelley's transformation rather than Goh' s, Speyer and Jacobson's Riccati equation which is of smaller dimension, but otherwise equivalent, is obtained. If it turns out that the first generalized Legendre-Clebsch condition is not satisfied in strong form the transformation techniques have to be applied repeatedly until one of the higher-order generalized Legendre-Clebsch conditions is satisfied in strong form. This can only be done if the system equations and performance integrand are many times differentiable; a significant disadvantage which is not present in the more direct limit approach of Chapter 4.
6.
6.4
CONCLUSION
181
Computational Methods Computational problems i n s i n g u l a r c o n t r o l f a l l
r a t h e r n a t u r a l l y i n t o two areas.
The f i r s t i s t h e
computation of optimal s i n g u l a r c o n t r o l s and t h e second
i s t h e c o n s t r u c t i o n of c e r t a i n m a t r i x f u n c t i o n s of t i m e i n n e c e s s a r y and s u f f i c i e n t c o n d i t i o n s which check a particular singular control function f o r optimality. It t u r n s o u t t h a t t h e l i m i t approach, i n which t h e
s i n g u l a r problem i s made n o n s i n g u l a r by t h e a d d i t i o n s of 22 El t f u T u d t
,
t0
i s u s e f u l i n both of t h e s e problem areas.
Indeed, i t
i s worth mentioning t h a t Powers and McDanell have r e p o r t e d u s i n g t h i s method s u c c e s s f u l l y i n a s i n g u l a r S a t u r n guidance problem.
D e t a i l s of t h e l i m i t approach
are given i n Chapter 5 where i t i s p o i n t e d o u t t h a t t h i s i s t h e o n l y g e n e r a l method c u r r e n t l y a v a i l a b l e . However, t h e r e c e n t experiments of Edge and Powers i n v o l v i n g quasi-Newton methods i n f u n c t i o n s p a c e are encouraging and augur w e l l f o r t h e f u t u r e .
6.5
Switching Conditions The j o i n i n g of s i n g u l a r and n o n s i n g u l a r sub-arcs
i s i m p o r t a n t , and n o t y e t f u l l y understood.
However,
McDanell and Powers have provided a number of u s e f u l , g e n e r a l , theorems and Jacobson h a s o b t a i n e d s a t i s f a c -
182
SINGULAR O P T I M A L C O N T R O L P R O B L E M S
tory switching strategies in a class of quadratic problems.
6.6 Outlook for the Future It seems to us that the areas of optimal (singular) control which require greatest attention are those which involve the computation of optimal controls and the joining of singular and nonsingular segments.
The
experiments of Edge and Powers in control function computation and the work of Anderson and Moylan in network synthesis are encouraging. In closing, we mention the recent work of Moyer in connection with sufficient conditions for a strong minimum in singular optimization problems.
This paper
could stimulate further developments in optimal control theory
.
AUTHOR
INDEX
Desai, M. N. , 14, 29 Dobell, A. R., 5, 30 D'Souza, A. F. , 5, 31
A Anderson, B. D. O . ,
11,
21, 24, 28, 34, 103,
E
, 154, 165, 166, 169, 170, 181,
A r c h e n t i , A. R . , 12, 28 A r i s , R., 38, 59, 69, 70,
Edge, E. R.
72, 99
Athans , M.
182
, 11, 29
F
B
19, 30, 101, 104, 112, 130, 149
Fomin, S. V . ,
Bass, R. W . , 10, 29 B e l l , D. J . , 8, 13,
16, 17, 29, 54, 59, 71, 72, 76, 98
G
1, 3, 29, 37, 39, 42, 46, 59,
1 7 , 28, 30, 92, 98, 138, 149 Gamkrelidze, R. V., 9'7, 99 Gelfand, I. M., 19, 30, 101, 104, 112, 130,
B l i s s , G. A.,
97, 98
Bolonkin, A. A . ,
29
17, 18,
B o l t y a n s k i i , V. G . ,
99
Breakwell, J. V.
142, 149
Gabasov, R . ,
97,
149
, 17, 29,
Gershwin, S. B . ,
23, 32,
154, 165, 170
B r o c k e t t , R. W., 102, 149 Bryson, A. E . , 14, 29,
Gibson, J. E . , 11, 32 Goh, B. S . , 3, 5, 14,
16,
17, 18, 19, 22, 30, 31, 62, 81, 88, 98, 102, 137, 140, 141, 149 Graham, J. W. , 5, 31 Gurman, V. I., 17, 31
53, 59, 67 C
Canon, M. D . , 11, 29 Coddington, E. A., 109,
111, 131, 149 5, 30
H
Connor, M. A.,
Haynes, G. W. , 10, 1 7 , 31 Hermes, H. , 10, 12, 31
D Davis, R. E . , 154, 165,
5, 29, 30, 539 59, 142, 149
Ho, Y* C.,
170 I83
AUTHOR INDEX
184
L
H o f f m a n , W. C., 14, 29
J
4, 15, 18, 19, 22, 23, 24, 25, 26, 28, 31, 32, 36, 62, 81, 88, 91, 92, 94, 98, 100, 102, 113, 116, 123, 137, 138, 140, 141, 142, 149, 150, 151, 153, 154, 157, 160, 161, 162, 164, 167, 170, 171, 176, 177, 180,
Jacobson, D. H.,
181
J o h m s e n , D. E. , 19, 32, 101, 150, 154, 170 Johnson, C. D . , 10, 11,
27, 32, 36, 154, 167, 171 K
Kalman, R. E . , 142, 150 Keller, J. L. , 15, 32 Kelley, H. J., 13, 15, 17, 22, 27, 32, 33, 61, 63, 67, 69, 72, 81, 99, 138, 150, 167, 170, 175
Kirillova, F. M., 28, 30 Kopp, R. E., 13, 14, 15, 27, 33, 61, 63, 69, 72, 81, 99, 138, 150, 167, 170 Krener, A. 3 . , 16, 33, 97, 99
Lawden, D. F.,
6, 8, 12,
33, 38, 47, 48, 49,
59, 72, 75, 80, 99
Leitmann, G. , 12 Lele, M. M. , 23,
165, 170
Levinson, N.
32, 154,
, 109, 111,
131, 149 M
M a n c i l l , J . D. , 19, 33 Marchal, G. , 28, 33 Maurer, H., 154, 168, 170 Mayne, D. Q., 15, 19, 27,
34, 153, 154, 160, 161, 162, 170 McDanell, J. P. , 21, 2 2 , 27, 34, 35, 88, 99, 102, 122, 140, 141, 150, 154, 156, 165, 167, 170, 171, 177, 181 Mehra, R. K . , 154, 165, 170 Miele, A . , 8, 9 , 17, 34 Mishchenko, E. F., 97, 99 Moore, J. B., 11, 21, 28, 34, 35, 154, 167, 171
13, 15, 26, 27, 33, 34, 61, 63, 69, 72, 81, 99, 138, 141, 150, 167, 170, 182 Moylan, P. J . , 11, 35, 154, 157, 167, 1-69, 171, 182
Moyer, H. G . ,
AUTHOR INDEX P
,
Pontryagin, L. S . 97, 99 Powers, W. F., 21, 22, 27,
34, 35, 88, 99, 102,
122, 140, 141, 150,
154, 156, 165, 166, 167, 169, 170, 171, 177, 181, 182
Rao, S . K., 5, 36 Ray, W. H., 5, 35 Robbins, H. M., 5, 14,
17, 35, 61, 63, 99, 137, 138, 151 35
32, 35, 36, 62, 81, 88, 91, loo, 102, 113, 123, 137, 140, 141, 150, 151, 157, 171, 176, 177, 180 S t e i n b e r g , A. M., 5, 36 Strochko, V. A., 28, 30 T
13, 19, 36, 101, 113, 138, 151 T a r n , T. J . , 5, 36
T a i t , K. S . ,
R
Rohrer, R. A . ,
185
11, 21,
S
14, 35, 88, 90, 91, 99 S i e b e n t h a l , C. D., 38, 59, 69, 70, 72, 99 Silverman, L. M . , 21, 35
S c h u l t z , R. L.,
S i r i s e n a , H. R . , 11, 35 Snow, D. R . , 12, 35 Sobral, M., 11, 21, 35 Soliman, M. A., 5, 35 Speyer, J. L., 14, 15,
22, 23, 24, 25, 26,
V
Valentine, F. A . , 40, 59 Vapnyarskiy, I. B., 17,
18, 36
Vinh, N. X., 12, 28 V i n t e r , R. B., 26, 36 W
Webber, R. F . , 10, 29 Wonham, W. M., 10, 36,
154, 167, 171 Z
Zaborszky, J . , 5, 36 Zagalsky, N. R., 1 4 , 35
88, 90, 91, 99
This page intentionally left blank
SUBJECT INDEX
t e r m i n a l , 1, 7, 16, 1 9 t r a n s v e r s d i t y , 45 46
A
Accessory problem, 4, 17, 21, 53, 56, 85, 87 Adjoint e q u a t i o n , 94 v a r i a b l e , 70, 89, 97 v e c t o r , 45 Admissible r e g i o n , 9 U g o r i t h m , 38, 153, 154,
Conjugate g r a d i e n t , 166 p o i n t , 53 C o n t r o l l a b i l i t y 104 106, 112 Cost f u n c t i o n a l , 1, 'j'ff.,
,
B
D D i f f e r e n t i a l , first , 44, 49, 50 second, 44, 50ff. D i f f e r e n t i a l Dynamic Programming, 27, 154, 160ff.
Bang-bang a r c , 9, 26, 160 Bolza problem, 1, 3, 40
C Condition, boundary, 43,
48, 63, 74, 77
Clebsch, 46, 49, 63, 87 existence, 142ff. J a c o b i , 22, 53 Jacobson, 16ff., 61, 62,
175, 176
j u n c t i o n , 27, 154, 167ff. , 181, 182 n e c e s s a r y , 1 2 f f . 21ff.
,
38, 55, 61ff., 85, 101, 102, 123, 13b, wff.,155, 156, 174ff.
,
E
,
E-algorithm, 1 6 0 f r . 169 convergence o f , 162 Economics, 5, 61 E x t e r i o r c a l c u l u s , 10 F
F i e l d t h e o r y , 26, 1 4 1 F l i g h t mechanics, 9 , 47,
n e c e s s a r y and s u f f i c i e n t ,
5, 13, 1 9 2 ~ ,. l o i f f . , , 168, 175ff. s u f f i c i e n t , 13, l g f f . , 38, i o i f f , , 124, 136, 1 2 2 f f . , 140ff.
141ff.
,
41ff., 49, 57, 75, 160
159
giff.
, ,
51, 97
, 182
167
54, 88, 165, 181
Function space method,
154, 165ff., 181, 182
convergence o f , 166 Fundamental lemma, 46
SUBJECT INDEX
188
L i m i t approach, 23, 28,
G
Gradient method, 154, 165 Green's theorem, 8, 9
157ff. , 168, 169, 176, 177, 181 M
H
Hamiltonian, 2 , 3, 7, 15,
44, 48, 57, 58, 70, 75, 89, 90, 94 Hohmann t r a n s f e r , 7, 8
15, 72, 74, 88, 89
I l l - c o n d i t i o n i n g , 164 Intermediate-thrust a r c ,
8, 1 2 , 17, 38, 72, 74
I n t e r p l a n e t a r y guidance, 17, 27 Isoperimetric constraint ,
8
s t r o n g , 24, 26, 141, 182 weak, 19, 20, 24, 101, 103, 140 N
Network t h e o r y , 21, 182 Newton method, 158, 165,
181
K
Nonsingular a r c , 3, 1 2 ,
Kelley-Contensou t e s t , 1 3 L
Lagrange m u l t i p l i e r , 3,
43, 127, 128
problem 1 Lawden's s p i r a l , 1 2 , 1 5 ,
17, 72, 80, 174, 175
Legendre- Cleb sch condition, 3, 1 2 , 15, 61, 81, 82, 96, 104,
173, 175
37
Minimum, fuel, 7, 8, 1 2 ,
I
generalized, 1 2 f f .
M a x i m a l p r i n c i p l e , high o r d e r , 16, 97 Mayer problem, 1 Miele's method, 8 f f . , 17,
,
62ff., 9 3 f f . , 119, 140, 157, 167ff. , 175, 179, 180
13, 27, 154, 166
control, 5 f u n c t i o n a l , 129, 161 problem, 19, 22, 25, 61, 63, 95, 96, 101,
140, 161, 174, 176
Normality, 16, 67, 97 0
Order of s i n g u l a r a r c , 4, 13, 1 5 , 23ff. P
Passivity, 21 Pontryagin's p r i n c i p l e , 2 , 16, 46, 49, 61,
SUBJECT INDEX
4 , 7 f f . , 61, 70, 75, 153
72, 9 4 f f . , 174 P o s i t i v e real matrix, 21 Process c o n t r o l , 1 7
control,
p a r t i a l l y , 4 , 5, 24, 26,
175
Q
problem, 20, 22, 38, 39, 56ff., 154, 173,
Q u a s i - l i n e a r i z a t i o n , 158
177
R
Relaxed problem, 5 Resources , n a t u r a l , 5 R i c c a t i equation, 1 9 , 22, 25, 53, 96, i o i f f . , 113, 119, 130, 1 4 0 f f . , 156, 157,
168, 1 7 5 f f . , 180
Second v a r i a t i o n , nonnegative, 2 0 f f . , 39, 52, 55, 63, 80, 91, 93, l O l f f . , 115ff., 130ff. , 153ff. n o n s i n M a r , 102, 106, U T f f . , 1 5 5 f f . , 173 p a r t i a l l y s i n g u l a r , 102, 105, 140, 173, 174 positive definite, l 0 5 f f . 130 s i n g u l a r , 18, 23, 25, 81, 87, 101, 102,
,
1 4 1 , 157
173
Singular arc, 3, 9 , 1 2 , 13, 27, 58, 72, 91, 154, 160, 161, 166
s o l u t i o n , 94, 95, 97 s t r i p , 10, 11 s u r f a c e , 26 t o t a l l y , 4, 5, 2 1 f f . , 26, 102, 177ff. Space n a v i g a t i o n , 6 f f . , 12, 72 S p e c i f i c energy, 88 S t i e l t j e s i n t e g r a l , 136 S t i r r e d t a n k r e a c t o r , 38,
69ff.
S
strongly p o s i t i v e , 19, l O l f f . , 1 4 1 , 180 t o t a l l y s i n g u l a r , 103, 1 1 2 f f . , 137, 1 5 5 f f .
189
,
S t o k e s ’ s theorem, 10 System, aerospace, 9 , 154 delay, 5 discrete, 5 e c o l o g i c a l , 61 T
Terminal c o n s t r a i n t , 43, 57, 66, 69, 82, 83, 97, 137 Transformation, canonical,
15
o f Goh, 1 6 , 21, 62, 81ff., 140, 156,
174ff.
,
of Kelley, 1 5 f f . 61ff. 81, 140, 157, 175, 177, 180
,
V Valentine technique , 40 V a r i a t i o n , admissible, 52,
SUBJECT INDEX
190
77
c o n t r o l , 1 3 , 15, 1 6 , 4 2 ,
7 5 6 1 f f . , 93, 175
e q u a t i o n s o f , 1 6 , 43,
53, 54, 63, 65, 75, 76, 82
f i n a l t i m e , 42
f i r s t , 39, h l f f . , 75 second, 4, 13, 1 5 , 1 6 ,
1 9 , 37ff., 49ff., 6 5 f f . , 7 6 f f . , 92,
10lff. s t a t e , 1 6 , 42, 56, 63,
66, 72ff.