Singular Optimal Control Problems (Mathematics in Science and Engineering, Vol. 117)

Singular Optimal Control Problems

This is Volume 1I7 in MATHEMATICS 1N SCIENCE AND ENGINEERING A Series of Monographs and Textbooks Edited by RICHARD BELLMAN, University of Southern California The complete listing of books in this series is available from the Publisher upon request.

Singular Optimal Control Problems

DAVID J. BELL

Department of Mathematics and Control Systems Centre University of Manchester Institute of Science and Technology Manchester, England

and

DAVID H. JACOBSON

National Research Institute for Mathematical Sciences Councilf o r Scientific and Industrial Research, South Africa (Honorary Professor in the Universiiy of the Witwatersrand)

1975

Academic Press London

: New York

: San Francisco

A Subsidiary of Harcourt Brace Jovanovich, Publishers

ACADEMIC PRESS INC. (LONDON) LTD. 24-28 Oval Road 1,ondon NWI

U . S . Edition published hi. ACADEMIC PRESS INC. I I I Fifth Avenue New York. New York 10003

Copyright 0 1975 by ACADEMiC PRESS INC. (LONDON) LTD.

All Righis Re.wr\<ed Y o part of this hook may be reproduced in any form by photostat, microfilm. or any other

means. without written permission from the publishers

Library of Congress Catalog Card Number. 75-21610 I S B N : 0 - 12-085060-5

I’rinlcd in (;rc;it Hritain hk Galliard (Pi-inter\) L.td. Great Yarrnouth

PREFACE The modern t h e o r y of o p t i m a l c o n t r o l h a s i t s b e g i n n i n g s i n t h e p i o n e e r i n g work of men s u c h a s Hohmann i n Europe and Goddard i n t h e U n i t e d S t a t e s . These men, and o t h e r s l i k e them, dreamt of t h e day when space t r a v e l would become a r e a l i t y .

I t i s t h e work o f

t h e s e p i o n e e r s which h a s h e l p e d t o b r i n g t h e i r dreams t o p a r t i a l f u l f i l l m e n t i n s u c h a b r i e f p e r i o d of t i m e . The o p t i m i z a t i o n problems of s p a c e f l i g h t dynamics such a s minimum f u e l and minimum t i m e were q u i c k l y r e a l i z e d t o b e problems i n t h e c a l c u l u s of v a r i a t i o n s .

It w a s

t h u s t h a t t h i s v e r y o l d and e s t a b l i s h e d b r a n c h of Mathematics r e c e i v e d y e t a n o t h e r new lease of l i f e , a p a t t e r n which h a s b e e n r e p e a t e d s i n c e i t s b i r t h i n t h e seventeenth century. Problems of h i g h performance a i r c r a f t became i m p o r t a n t n e a r t h e end of t h e Second World War.

Maximum

range of a i r c r a f t f o r a g i v e n q u a n t i t y of f u e l and minimum t i m e t o climb were t y p i c a l o p t i m i z a t i o n problems which a r o s e .

These t o o were c l e a r l y problems b e l o n g i n g

t o t h e same class as t h o s e from s p a c e dynamics a l t h o u g h c o m p l i c a t e d by t h e p r e s e n c e o f aerodynamic l i f t and d r a g . Much of t h e m a t h e m a t i c a l t h e o r y f o r such problems had a l r e a d y b e e n developed e a r l y i n t h e t w e n t i e t h c e n t u r y by P r o f e s s o r G. A. B l i s s and h i s s t u d e n t s a t t h e U n i v e r s i t y of Chicago.

I n p a r t i c u l a r , t h e optimiza-

t i o n of a i r c r a f t and s p a c e v e h i c l e f l i g h t p a t h s f o r which t h e c o n t r o l s are f i n i t e and bounded was a n a l y s e d V

PREFACE

vi

by a technique d e s c r i b e d i n a d i s s e r t a t i o n by F. A. V a l e n t i n e , a r e s e a r c h s t u d e n t of B l i s s , i n 1937. However, i n 1959 L. S. P o n t r y a g i n p r e s e n t e d h i s Maximum P r i n c i p l e which c o n s o l i d a t e d t h e t h e o r y f o r c o n s t r a i n e d problems. N e v e r t h e l e s s , i t soon became a p p a r e n t t h a t t h e mathematical t h e o r y a v a i l a b l e w a s n o t s u f f i c i e n t f o r c e r t a i n s p e c i a l c o n t r o l problems i n which t h e P o n t r y a g i n P r i n c i p l e y i e l d e d no a d d i t i o n a l i n f o r m a t i o n on t h e stationary control.

These problems were d e s c r i b e d as

s i n g u l a r problems and they have a r i s e n i n many engineeri n g a p p l i c a t i o n s i n f i e l d s o t h e r than Aerospace and a l s o more r e c e n t l y i n non-engineering a r e a s such as Economics.

I n t h e l a s t t e n t o twelve y e a r s much e f f o r t

has been p u t i n t o t h e development of new t h e o r y t o d e a l w i t h t h e s e s i n g u l a r problems.

F i r s t , new n e c e s s a r y

c o n d i t i o n s were found f o r s i n g u l a r e x t r e m a l s t o be c a n d i d a t e optimal a r c s .

Secondly, and more r e c e n t l y ,

new s u f f i c i e n t c o n d i t i o n s and n e c e s s a r y and s u f f i c i e n t c o n d i t i o n s have been found f o r such e x t r e m a l s t o be optimal , We f e e l t h a t t h e t i m e i s now r i g h t f o r t h e t h e o r y of s i n g u l a r problems t o b e c o l l e c t e d t o g e t h e r , s c a t t e r e d

a s i t i s i n numerous d i f f e r e n t j o u r n a l s , and p r e s e n t e d under one cover. volume.

T h i s i s t h e purpose of t h e p r e s e n t

We a r e p a r t i c u l a r l y p l e a s e d t h a t t h i s book

should t a k e i t s p l a c e a l o n g s i d e s o many well-acclaimed t e x t s i n Richard Bellman's s e r i e s which h a s proved i t s

PREFACE

worth many times over.

vii

Our gratitude goes to all o u r

colleagues and students, past and present, who have stimulated us both in our own researches. In particular, we would like to mention Y. C. Ho, D. Q. Mayne, J. L. Speyer, W. Vandervelde, D. F. Lawden, R. N. A. Plimmer

and B. S. Goh.

Finally, we acknowledge the help and

encouragement received from Academic Press during the preparation of this book and our special thanks go to Mrs. G. M. McEwen and Mrs. M. E. Hughes who typed the

manuscript.

D. J. BELL dune t 9 7 5

D. H. JACOBSON

This page intentionally left blank

CONTENTS

Page Preface

V

Chapter 2

An H i s t o r i c a l Survey o f S i n g u l a r C o n t r o l Problems

1.1 I n t r o d u c t i o n 1.2 S i n g u l a r C o n t r o l i n Space Navigation 1.3 Method o f Miele v i a Green's Theorem

1.4 1.5

1.6

-

Q u a d r a t i c Cost L i n e a r Systems Necessary Conditions for S i n g u l a r Optimal C o n t r o l 1 . 5 . 1 The Generalized LegendreClebsch Condition 1 . 5 . 2 The Jacobson Condition S u f f i c i e n t Conditions and Necessary and S u f f i c i e n t Conditions for Opt i m d i t y References

1

6 8

10

12 12

18 19

28

Chapter 2 Fundamental Concepts 2.1 2.2 2.3 2.4 2.5

Introduction The General Optimal C o n t r o l Problem The F i r s t V a r i a t i o n o f J The Second V a r i a t i o n of J A S i n g u l a r Control Problem References

Chapter

37 39 41 49 56 59

3

Necessary Conditions for S i n m a r Optimal C o n t r o l

3.1 I n t r o d u c t i o n

3.2

The G e n e r a l i z e d Legendre-Clebsch Condition 3.2.1 S p e c i a l C o n t r o l V a r i a t i o n s 3.2.2 A Transformation Approach ix

61 62 62

a1

CONTENTS

X

page

3.3 J a c o b s o n ' s Necessary Condition R e f e re nc e s

91 98

Chapter 4 S u f f i c i e n t Conditions and Necessary and S u f f i c i e n t Conditions f o r Non-Negativity of Nonsingular and S i n g u l a r Second Variations

4.1

Introduction 4.2 P r e l i m i n a r i e s 4.3 The Nonsingular Case 4 . 4 S t r o n g P o s i t i v i t y and t h e T o t a l l y S i n g u l a r Second V a r i a t i o n 4 . 5 A General Suffici-ency Theorem f o r t h e Second V a r i a t i o n 4 . 5 . 1 The Nonsingular S p e c i a l Case 4.5.2 The T o t a l l y S i n g u l a r S p e c i a l Case 4.6 Necessary and S u f f i c i e n t Conditions f o r Non-negativity of t h e T o t a l l y S i n g u l a r Second V a r i a t i o n 4.7 Necessary Conditions f o r O p t i m a l i t y 4.8 Other Necessary and S u f f i c i e n t Conditions 4.9 S u f f i c i e n t Conditions f c r a Weak L o c a l Minimum 4 . 1 C E x i s t e n c e Conditions f o r t h e Matrix R i c c a t i D i f f e r e n t i a l Equation 4.10.1 An Example 4.11 Conclusion References

101 103 106 112

115 117

119 122 137

140

140 142

146

148 149

Chapter 5 Computational Methods f o r S i n g u l a r Control Problems

5.1 5.2

I n t r o duct i on Computational A p p l i c a t i o n o f t h e S u f f i c i e n c y Conditions o f Theorems 4.2

153

CONTENTS

5.3

5.4 5.5

and 4.5 5.2.1 The Nonsingular Case 5.2.2 The T o t a l l y S i n g u l a r Case 5.2.3 A L i m i t Approach t o t h e Construct i o n of P ( * ) Computation o f Optimal S i n g u l a r C o n t r o l s 5.3.1 Preliminaries 5.3.2 An €-Algorithm 5.3.3 A G e n e r a l i z e d Gradient Method f o r S i n g u l a r Arcs 5.3.4 Function Space Quasi-Newton Methods 5.3.5 Outlook f o r F u t u r e J o i n i n g o f O p t i m a l S i n g u l a r and N on s i n g u l ar Sub- a r c s Conclusion References

xi

155

155 155

157 159 159

160

165

165 166

166 168 169

Chapter 6 Conclusion

6.1 6.2 6.3

6.4 6.5 6.6

The Importance of S i n g u l a r Optimal C o n t r o l Problems Necessary Conditions Necessary and S u f f i c i e n t Conditions Computational Methods Switching C o n d i t i o n s Outlook f o r t h e F u t u r e

AUTHOR INDEX SUBJECT INDEX

173 174 175 181 181 182

183 187

This page intentionally left blank

CHAPTER 1

An H i s t o r i c a l Survey of S i n g u l a r C o n t r o l Problems

1.1

Introduction It i s w e l l known t h a t t h e fundamental problem of

o p t i m a l c o n t r o l t h e o r y can b e formulated as a problem of Bolza, Mayer o r Lagrange.

These t h r e e f o r m u l a t i o n s

a r e q u i t e e q u i v a l e n t t o one a n o t h e r ( B l i s s , 1946).

We

s h a l l d e s c r i b e t h e Bolza problem s i n c e t h e a c c e s s o r y minimum problem, and t h e a s s o c i a t e d second v a r i a t i o n w i t h which w e s h a l l b e much concerned i n t h i s book, appears as such a problem. The problem of Bolza i n o p t i m a l c o n t r o l t h e o r y

i s t h e following.

Determine t h e c o n t r o l f u n c t i o n

u ( * ) which minimizes t h e c o s t f u n c t i o n a l

r tf

(1.1.1)

where t h e system e q u a t i o n i s

H

= f (X,U,t)

(1.1.2)

subject t o the constraints x ( t o > = xo

(1.1.3) (1.1.4)

Slhii;U!2.4R OPTIMAL CONTROL PROBLEMS

2

u(*> is a member of the set U, t a member (1.1.5)

of [to, tfl.

Here x is an n-dimensional state vector and u is an m-dimensional control vector.

The functions L and F

are scalar and the terminal constraint function $ is an s-dimensional column vector function of x(t ) at f The functions L, F and $ are assumed smooth. The tf' set U is defined by

u

E {u(-)

: u . ( * ) is piecewise continuous 1

...

i = l , 2, The initial time t

,ml.

in time,

(1.1.6)

i s given explicitly but the final

0

may be unspecified. f The Hamiltonian for this problem is

time t

and the following necessary conditions (Pontryagin's principle) hold along an optimal trajectory:

-

=

--

Hx (x,u,A,t)

(1.1.8)

1.

AN HISTORICAL S U R V E Y

A(tf)

=

Fx[x(tf)

, tfl

+ QXTV

3

(1.1.9)

where

-u =

arg min H(';;,u,X,t)

(1.1.11)

U

u(*) a member of U. Here

; ( a ) ,

u(-) denote the candidate state and control

functions respectively, A ( * )

denotes an n-dimensional

vector of Lagrange multiplier functions of time, and v is an s-dimensional vector of Lagrange multipliers

associated with JI. A singular minimizing arc for the problem of

Bolza is defined by Bliss (1946) as one for which the Legendre-Clebsch necessary condition is not satisfied with strict inequality.

For the optimal control

problem as formulated above the definition of Bliss is equivalent to the following. An extremal arc of the control problem is said to be singular if the m

x

m determinant det(H

along it.

) vanishes at any point uu Otherwise it is said to be nonsingular.

In particular, if the Hamiltonian H is linear in one or more elements of the control function then the extremal is singular (Goh, 1966b).

4

SINGULAR OPTIMAL CONTROL PROBLEMS

The following definitions are used in the sequel: Definition 1.1

Let u

k

be an optimal singular element

of the control vector u on the interval [tl, t2] which appears linearly in the Hamiltonian. time derivative of H derivative in which

Uk

\

Let the 2q th

be the lowest order total appears explicitly with a

coefficient which is not identically zero on [tl’ t2]. Then the integer q is called the order of the singular arc.

The control variable

%

is referred to as a

singular control. Definition 1.2

Assuming all the elements u1,u2, ...,%

of the control vector u are singular simultaneously then u is called a totally singular control function when HU(y,h,t)

=

(1.1.12)

0

for all t in [to, tfl. Definition 1.3

A partially singular control function

is one along which (1.1.12) holds for k subintervals of length Ti, i = 1, 2,

...

,k

and where

k

The concepts of total and partial singularity can be applied also to the accessory minimum problem in which the second variation of the cost functional

(1.1.1) is to be minimized (see Sections 4 . 4 and 6.1).

1.

AN HISTORICAL SURVEY

5

I n t h e important s u b c l a s s of optimal c o n t r o l problems known as r e l a x e d v a r i a t i o n a l problems d i s c u s s e d by S t e i n b e r g (1971) some of t h e elements of t h e c o n t r o l v e c t o r u appear l i n e a r l y and o t h e r s appear n o n l i n e a r l y . I n t h i s book we s h a l l be mainly concerned w i t h problems which are t o t a l l y s i n g u l a r f o r a l l

1 ( b u t see S e c t i o n 5 . 4 ) . I n t h e p a r t i a l l y f s i n g u l a r case t h e n o n s i n g u l a r c o n t r o l s t h a t are t i n [to, t

p r e s e n t can b e e l i m i n a t e d v i a well-developed nons i n g u l a r t h e o r y , l e a v i n g a problem t o t a l l y s i n g u l a r i n t h e remaining c o n t r o l v a r i a b l e s (Robbins, 1967).

The

s i n g u l a r c o n t r o l t h e o r y t o b e d i s c u s s e d has been developed f o r and motivated by e n g i n e e r i n g problems. N e v e r t h e l e s s , s i n g u l a r problems may a r i s e i n any d i s c i p l i n e where c o n t r o l t h e o r y i s a p p l i e d , evidence f o r which can b e s e e n i n economics (Dobell and Ho,

1967) and p r o d u c t i o n from n a t u r a l r e s o u r c e s (Goh, 1969/70).

I n t h e f o l l o w i n g c h a p t e r s n e c e s s a r y and

s u f f i c i e n t c o n d i t i o n s W i l l be developed f o r s i n g u l a r optimal c o n t r o l of systems governed by o r d i n a r y d i f f e r e n t i a l equations.

However, i n v e s t i g a t i o n i s

b e i n g c a r r i e d o u t i n t o s i n g u l a r c o n t r o l of d i s c r e t e t i m e systems (Tarn e t a l . , 1971; Graham and D'Souza,

1970) and of systems w i t h d e l a y (Soliman and Ray, 1972; Connor, 1974).

SINGULAR OPTIMAL CONTROL PROBLEMS

6

1.2

Singular Control in Space Navigation Practical problems involving singular controls

arose early in the study of optimal trajectories for space manoeuvres.

Trajectories for rocket propelled

vehicles in which the thrust magnitude is bounded exhibit singularity in the rate of fuel consumption. Lawden (1963) formulates the fundamental problem of space navigation in the following way.

Ox1x2x3 is an

inertial frame in which a space vehicle has position coordinates ( x l , x2, x3) and velocity components The rocket thrust has direc-

(vl, v2, v3) at time t. tion cosines ( E l ,

R3)

L!,

has components (gl, g,,

and the gravitational field

g3).

The mass rate of

propellant consumption, bounded above by some finite

-

constant m, is denoted by m.

If the rocket has an

exhaust velocity c and the mass of the vehicle is M then its equations of motion are

G.1

=

cmR./M 1 + gi(xl, x2, x3, t)

x. = v 1

i’

i

=

1, 2, 3

M = - m .

(1.2.1)

(1.2.2) (1.2.3)

The propellant consumption rate m must satisfy the inequality constraints (1.2.4)

1.

AN HISTORICAL S U R V E Y

7

whilst the direction cosines may be written in the form

R1

=

sine cos$,

where

R2 = sine sin$, R 3 =

case

(1.2.5)

e , + are spherical polar coordinates. It is

required to choose the control variables m(-), O ( . ) , $ ( * ) in order to minimize the fuel used in transfer-

ring the vehicle between two given positions at each of which the vehicle's velocity is specified. The final time tf may or may not be given explicitly. State variables x. ( - ) , vi(*), M(*) must satisfy 1 boundary condit ions x.

x.(t ) 1 0

=

Xi(tf)

= x

10'

if'

v.(t ) 1 0

=

v io'

v.(t ) 1 f

=

v if

M(to)

= Mo

(1.2.6) (1.2.7)

and the cost functional can be written as

In this problem the Hamiltonian is linear in the rate of fuel consumption and this control variable turns out to be a singular control. During the 1950's it was not known whether a given manoeuvre in space using a small, continuous thrust would be more economical in fuel than the previously accepted optimal procedure of impulsive boosts (Hohmann

SINGULAR OPTIMAL CONTROL PROBLEMS

8

transfers).

Some numerical r e s u l t s (Forbes, 1950)

suggested t h a t i n c e r t a i n circumstances l e s s f u e l would be used i n an o r b i t a l t r a n s f e r by f o l l o w i n g a s p i r a l p a t h u s i n g a s m a l l b u t continuous t h r u s t throughout t h e maneouvre t h a n by a Hohmann t r a n s f e r . Other a n a l y s i s (Lawden, 1950, 1952) appeared t o show t h a t the so-called intermediate-thrust a r c s (along which t h e f u e l e x p e n d i t u r e r a t e i s non-zero b u t less than

m)

were i n a d m i s s i b l e i n a f u e l optimal t r a j e c t o r y .

The t r u e s t a t u s of t h e i n t e r m e d i a t e - t h r u s t a r c s remained hidden u n t i l t h e mathematical t h e o r y had been f u r t h e r developed ( s e e S e c t i o n 1.5.1 and Chapter 3). Much of t h e e a r l y work i n a e r o s p a c e has been surveyed elsewhere ( B e l l , 1968). 1.3

Method of Miele v i a Green's Theorem A method which d e a l s s u c c e s s f u l l y w i t h problems

i n v o l v i n g s i n g u l a r c o n t r o l s i s due t o Miele (1950-51). It i s based upon a t r a n s f o r m a t i o n u s i n g Green's

theorem r e l a t i n g l i n e and s u r f a c e i n t e g r a l s .

As

developed by Miele t h e method i s a p p l i c a b l e o n l y t o a p a r t i c u l a r c l a s s of l i n e a r problems i n two dimensions. The problems are l i n e a r i n t h e s e n s e t h a t t h e c o s t f u n c t i o n a l and any i s o p e r i m e t r i c c o n s t r a i n t a r e l i n e a r i n t h e d e r i v a t i v e of t h e dependent v a r i a b l e . The g e n e r a l c o s t f u n c t i o n a l i n t h i s c l a s s of problems can be w r i t t e n as

1.

AN HISTORICAL SURVEY

9 ( 1.3.1)

J = ~~$(x,y) + $(x,y)dy/dxldx XO

where $ and

)I

are known functions of x and y.

An

admissible region in (x,y)-space is determined from the constraints in the problem formulation and the optimal trajectories within this region are determined by the sign of the function w(x,y)

=

a$ - a$ . ax

(1.3.2)

ay

If this function changes sign within the admissible region then a typical trajectory will contain two bang-bang arcs separated by a singular arc, the latter being defined by the equation w(x,y)

(1.3.3)

= 0.

Many problems in flight mechanics can be formulated in linear form of the above type provided suitable assumptions are made.

Such problems are

discussed by Miele (1955a, 1955b).

The general

theory of the extremization of linear integrals by Green's theorem for the two-dimensional case has been presented together with a number of applications to aerospace systems by Miele (1962).

The synthesis

of the optimal control by Miele's method including a demonstration of uniqueness of the singular control

SINGULAR OPTIMAL CONTROL PROBLEMS

10

and t h e d e t e r m i n a t i o n of t h e o p t i m a l s t r a t e g y h a s b e e n d i s c u s s e d by Hermes and Haynes (1963).

The g e n e r a l i z e d

S t o k e s ' s theorem of t h e e x t e r i o r c a l c u l u s h a s e n a b l e d Haynes (1966) t o e x t e n d Miele's method t o n dimensions.

1.4

L i n e a r Systems

-

Q u a d r a t i c Cost

Singular control f o r time-invariant,

linear

systems w i t h q u a d r a t i c c o s t f u n c t i o n , f r e e t e r m i n a l t i m e and f i x e d e n d p o i n t h a s b e e n s t u d i e d q u i t e extensively.

I n p a r t i c u l a r , Wonham and Johnson (1964) and

Bass and Webber (1965) h a v e s t u d i e d t h e s y s t e m

k

= Ax

+ Bu

i = 1, 2,

(1.4.1)

..., m

(1.4.2)

with cost function J =

itf J

T

x Qx d t

(1.4.3)

t0

under t h e a s s u m p t i o n s i ) u i s a s c a l a r , i i ) matrices A , B and Q are c o n s t a n t , i i i ) Q i s p o s i t i v e d e f i n i t e .

They have shown t h a t t h e o p t i m a l c o n t r o l ;

i s of t h e

form

-

T u = k x

on t h e s i n g u l a r s t r i p

(1.4.4)

AN HISTORICAL S U R V E Y

1.

lkTxl < 1

,

11

T k 1 x :0

where k and k, a r e c o n s t a n t v e c t o r s .

(1.4.5) The s i n g u l a r

c o n t r o l i s o p e r a t i v e w i t h i n t h e r e g i o n of s t a t e space d e f i n e d by t h i s s t r i p .

Johnson and Gibson (1963),

Athans and Canon (1964) have c a r r i e d o u t a s s o c i a t e d I n the l a t t e r reference a cost function i s

work.

taken i n t h e form

1

tf

J =

(1.4.6)

(k + I u ( t > l ) d t

t0

w i t h k a p o s i t i v e c o n s t a n t and u ( t ) a s c a l a r s a t i s f y ing trol

u[ < 1.

It i s found t h a t optimal s i n g u l a r con-

< 1 but n o t otherwise. exist i f k -

In a l l the

r e f e r e n c e s quoted above i n t h i s s e c t i o n t h e methods of a n a l y s i s a r e r a t h e r s p e c i a l and a r e n o t d i r e c t l y a p p l i c a b l e t o time-varying systems w i t h non-positive d e f i n i t e performance w e i g h t i n g m a t r i c e s . An e a r l y paper d i s c u s s i n g t h e time-varying problem w a s by Johnson (1965).

Assumptions i ) and

i i ) mentioned above were r e l a x e d i n a paper by Rohrer and S o b r a l (1966) whereas r e s u l t s o b t a i n e d by S i r i s e n a (1968) f o l l o w from a r e l a x a t i o n of i i i ) . An a n a l y s i s i n which a l l t h r e e assumptions were

r e l a x e d was given by Moore (1969) and Moylan and Moore (1971).

An e x c e l l e n t book which d i s c u s s e s t h e t o p i c s

of t h i s s e c t i o n i s t h e one by Anderson and Moore (1971).

SINGULAR OPTIMAL CONTROL PROBLEMS

12

1.5 1.5.1

Necessary Conditions for Singular Optimal Control The Generalized Legendre-Clebsch Condition In 1959 Leitmann suggested in private correspond-

ence that the intermediate thrust (IT) arcs arising from minimum fuel problems of space navigation may after all be candidate sub-arcs for optimal trajectories.

The first development following this sugges-

tion was the discovery of the form of these IT-arcs in two dimensions and in an inverse square law gravitational field (Lawden, 1961, 1962, 1963).

It

was found that if the time of transit is not predetermined then the IT-arc is a spiral with its pole at the centre of attraction, a trajectory which has since become known as Lawden’s spiral.

The status of

general IT-arcs in a central, time-invariant force field is given by Archenti and Vinh (1973). The difficulty at this stage of the investigations was that the classical Legendre-Clebsch condition

a

-H au u

-> O

(1.5.1)

which yields further information for nonsingular problems is trivially satisfied along a totally singular arc.

More and more researchers thus became

interested in these problems of singular control (e.g. Snow, 1964; Hermes, 1964) and particularly in the search for further necessary conditions which would help to establish the status of singular trajec-

1.

AN HISTORICAL SURVEY

tories such as Lawden's spiral.

13

In order to deduce

new conditions for optimality it was necessary to study the second variation of the cost functional which must be non-negative for a minimum value of that functional.

In certain cases it is possible to use

the second variation directly in order to establish the optimality or otherwise of both nonsingular (Bell, 1965) and singular arcs (Bell, 1971a).

However, the

main effort during the last decade has been to deduce new necessary, sufficient, and necessary and sufficient conditions for singular problems from the second variation.

A full discussion of the second variation

will be found in Chapters 2 and 4 . When the control is a scalar, Kelley (1964a) deduced a new necessary condition for singular optimal control by studying the second variation under a special control variation.

Kelley's method was

generalized by Tait (1965),

Kopp and Moyer (1965),

Kelley et al. (1967) to give what has since become known as the generalized Legendre-Clebsch condition (Kelley-Contensou test):-

(1.5.2)

where the integer q is the order of the singular problem.

SINGULAR OPTIMAL CONTROL PROBLEMS

14

The generalized Legendre-Clebsch condition for a vector control was obtained by Robbins (1967) and Goh (1966b).

In this case the controls can appear in an

odd time derivative of H

U

but if this does occur then

there necessarily exists a control u in U such that the second variation is negative (for a minimization problem).

Hence, the generalized Legendre-Clebsch

condition f o r vector control is

p odd

(1.5.3)

and (-1)q

ad2q H

au dt2q u

-> 0

for all t in (1.5.4)

assuming that the matrices H H fx and fU are xx’ ux’ sufficiently differentiable with respect to time. For p

=

1, eqn(1.5.3)

implies Huxfu is symmetric, a

powerful necessary condition in the case of vector controls (Speyer, 1973).

On the other hand, eqn

(1.5.3) is trivially satisfied when u is a scalar. Incidentally, the result of Speyer (1973) is contrary to Schultz and Zagalsky (1972) but consistent with the findings of Bryson et al. (1969). The generalized Legendre-Clebsch condition or its equivalent was used by Robbins (1965) and by Kopp and

1.

AN HISTORICAL S U R V E Y

15

Moyer (1965) to test the optimality of Lawden's singular spiral, a problem of order q condition was not satisfied and

so

= 2.

The new

the spiral cannot

form part of a minimum fuel trajectory.

The same

result was obtained by other workers using a special variation in the control (Keller, 1964) and by canonical transformations of the Hamiltonian (Fraeijs de Veubeke, 1965). An alternative approach to Singular problems was by a transformation on the state and control variables (Kelley 1964b; Speyer and Jacobson, 1971).

IJnder this

transformation a state space of reduced dimension i s obtained and the singular problem (hopefully) changed to a nonsingular one in which the classical LegendreClebsch condition can be applied.

If this is not the

case then a further transformation to a state space of even smaller dimension is indicated. This method was used to investigate Lawden's spiral (Kelley, 1963). A generalization of Kelley's transformation has been

given by Mayne (1970). The general theory for both the generalized Legendre-Clebsch condition and Kelley's transformation has been given by Kelley et al. (1967) including the application of these theories to Lawden's spiral.

In

this last reference (amongst others) a plausibility argument was put forward concerning the satisfaction of terminal conditions associated with the second

16

SINGULAR OPTIMAL CONTROL PROBLEMS

variation.

It was suggested that the special control

variations used in the derivation of the generalized Legendre-Clebsch condition could be altered by the addition of appropriate functions of time.

Such

additional functions would be of an order of magnitude which allowed the terminal conditions to be satisfied exactly but would not affect the sign of the second variation. However, it is sometimes possible to generate special state variations which satisfy both the equations of variation and the terminal conditions. This has been done in the case of Lawden's spiral (Bell, 1971b). The existence of sufficient control variations to enable one to cancel out undesirable low order effects of the special control variations, as discussed above,

is equivalent to a normality assumption. No such assumption is necessary in the Pontryagin Principle and recently this Principle has been extended t o a so-called High Order Maximal Principle (Krener, 1973). This new Principle includes the generalized LegendreClebsch and Jacobson (see Section 1.5.2 below) conditions when they apply, together with other conditions, without the assumption of normality. The difficulty caused by the fact that singular extremals may be transformed into singular extremals under a single application of Kelley's transformation was investigated by Goh (1966a) using a procedure by which the generalized Legendre-Clebsch condition for

1.

A N HISTORICAL S U R V E Y

17

singular extremals can be deduced and, unlike Kelley's transformation, retains the full dimensionality of the original problem.

It was again proved that Lawden's

spiral is non-optimal and furthermore IT-arcs are candidates for an optimal trajectory only if the thrust possesses a component which is inwardly directed to the centre of attraction, a result which had already been noted by Robbins (1965).

The transformed accessory

minimum problem discussed by Goh was also used to further develop the generalized Legendre-Clebsch necessary condition for vector control problems (Goh, 1966b).

This new set of conditions was applied to

several problems including a doubly singular problem in interplanetary guidance (Breakwell, 1965) and to a class of identically singular problems, certain members of which had already been studied by Haynes (1966) using an extension of Miele's method discussed above in Section 1.3.

The methods of Kelley and Goh

have also been applied to a singular arc arising from the problem of a stirred tank reactor in the field of Process Control (Bell, 1969). A number of investigations into singular problems

have also been carried out by Russian workers (Bolonkin, 1969; Gabasov, 1968, 1969; Gurman, 1967; Vapnyarskiy, 1966, 1967).

In particular, Gabasov

(1968, 1969) obtained a slightly stronger version of a second necessary condition which is discussed in the following section. A comparison of the results of

SINGULAR OPTIMAL CONTROL PROBLEMS

18

Vapnyarskiy (1967) and Bolonkin (1969) with those of Goh (1966b) is given by Goh (1973). Once the generalized Legendre-Clebsch condition had been established it was possible to lay down a procedure for the derivation of the singular extremals. This was done for the linear, time-varying system with quadratic cost functional by Goh (1967).

1.5.2

The Jacobson Condition

Having established the generalized LegendreClebsch condition researchers began to look for generalizations of the known sufficient conditions for the nonsingular problem.

However, before such a

generalization was found, a new necessary condition for non-negativity of the singular second variation, not equivalent to the generalized Legendre-Clebsch condition, was derived (Jacobson, 1969, 1970b).

This

new condition, known as Jacobson's condition, is as f0llows

.

H

f

ux u

+ f TQfu > 0 U

(1.5.5)

where T - Q = Hxx + fx Q + Qf,

(1.5.6)

(1.5.7)

1.

A N HISTORICAL SURVEY

and the partial derivatives fu, H

and f are all

x(-),-u( - > . UX

evaluated along the singular arc

19 X

A

strong version of this condition is given by Mayne (1973). In the classical calculus of variations problem the Jacobson condition can be replaced by an equivalent necessary condition on the terminal point of the singular extremaf (Goh, B.S.

On Jacobson's Necessary

Condition for Singular Extremals. Note).

Unpublished Research

A generalization of this end condition leads

to a result originally given by Mancill (1950).

It has

been shown by Jacobson (1970a) that in general the generalized Legendre-Clebsch condition and the Jacobson condition together are not sufficient for optima1ity 1.6

.

Sufficient Conditions and Necessary and Sufficient Conditions for Optimality A sufficient condition for a weak local minimum

in a nonsingular problem is that the second variation be strongly positive (Gelfand and Fomin, 1963).

This

condition gives rise to a well known matrix Riccati differential equation.

In singular problems the

second variation cannot be strongly positive (Tait, 1965; Johansen, 1966) but investigations have shown that Riccati-type conditions do exist for the singular case.

Jacobson (1970b), using a direct approach,

derived sufficient conditions for the second variation

SINGULAR OPTIMAL CONTROL PROBLEMS

20

to be non-negative in both singular and nonsingular control problems.

These conditions are that there

exist a real symmetric bounded, matrix function of time P(*) such that H

UX

+ f T~ U

P + H xx

=

+

o

for all t in [to, tfl

(1.6.1)

TP + Pf > 0 fx x-

for all t in [t

(1.6.2)

0’

and (1.6.3) where Z is the n

x

(n-s) matrix

(1.6.4)

and the s

x

s matrix D 1 and the s

x

(n-s) matrix D 2

are such that

The above conditions in strengthened form are sufficient conditions for a weak relative minimun.

It is

demonstrated that the two necessary conditions for

1.

A N HISTORICAL SURVEY

21

singular optimal control, discussed in Section 1.5 above, are implied by the new conditions for the case of totally singular control and unconstrained terminal state. Related to these sufficient conditions of Jacobson, with the matrix $x the null matrix, are conditions found in the network theory literature in connection with passivity of electrical networks; see

Rohrer and Sobral (1966) , Rohrer (1968) , Silverman

(1968), Anderson and Moore (1968a, 1968b), Moore and Anderson (1968).

The sufficient conditions (1.6.1-3)

are very similar in form to certain necessary and sufficient conditions for positive real matrices (Anderson, 1967) which suggested that perhaps Jacobson's conditions may also be necessary.

An

alternative approach to the derivation of sufficient conditions for non-negativity of the second variation, in which the generalized Legendre-Clebsch condition (1.5.2) with q and $

=

1 is satisfied with strict inequality

0, is by the transformation method of Goh or

Kelley. The approach used by Goh transforms the singular accessory minimum problem (AMP) into a nonsingular, non-classical one.

The transformed problem is non-

classical in that the control variations appear in the boundary conditions and outside of the integral in the second variation.

Nevertheless, for certain classes

of totally singular problems the transformed AMP has been used by McDanell and Powers (1970) to develop a

SINGULAR OPTIMAL CONTROL PROBLEMS

22

sufficient condition which includes a Jecobi-type condition in the form of a matrix Riccati equation. A weakened form of this sufficiency condition is shown

to be necessary for a smaller class of singular problems.

The results of McDanell and Powers have

been generalized by Goh (19.70) and Speyer and Jacobson

(1971). The transformation proposed by Kelley (1964b) has been applied to the AMP of a totally singular optimal control problem (Speyer and Jacobson, 1971).

The

transformation can be obtained in closed form and the resulting problem is nonsingular.

The generalized

Legendre-Clebsch condition and Jacobson's necessary condition (for the unconstrained problem) are identified without the need for special variations.

The

results obtained are similar to those of Goh, McDanell and Powers mentioned above.

Eqns(1.6.1-3)

in strong

form (strict inequality) and with unconstrained terminal state ($ :0, Z = I), applied to the transformed AMP, are equivalent to the generalized Legendre-Clebsch condition and to the Jacobi condition (matrix Riccati equation) for the transformed nonsingular problem (Jacobson, 1970~). The sufficiency conditions of Jacobson are thus seen to be both necessary and sufficient.

Since the transformation is nonsingular

and Jacobson's conditions are coordinate independent these conditions are necessary and sufficient for optimality of singular problems in the original state

1.

space.

AN HISTORICAL S U R V E Y

23

This result is true for first-order arcs and

for problems with or without terminal state constraints.

If the problem is singular of order higher

than one then the reduction of the state space to achieve a nonsingular problem requires repeated application of the transformation technique.

This is

cumbersome especially if there are multiple control variables.

Furthermore, the method requires the

coefficients of the time dependent variation to be many times differentiable. Jacobsonts sufficiency conditions can also be shown to be necessary by a limit approach (Jacobson

and Speyer, 1971) which avoids the above difficulties. Here the singular second variation is made nonsingular by the addition of a quadratic functional of the controls :

1 2E 1 By allowing

E -+

6 ~ dt. ~ 6 ~ the optimality conditions for the

singular problem are deduced from the limiting optimality conditions of the synthesized nonsingular second variation.

This limit approach had been used previously

(Jacobson et al., 1970) as a computation technique. The sufficiency conditions of Jacobson (1970b) are again shown to be necessary for non-negativity of the singular variation.

Indeed, these conditions are

shown to be necessary and sufficient for a weak

24

SINGULAR OPTIMAL CONTROL PROBLEMS

minimum for a class of singular control problems and in certain cases a strong minimum is implied.

In

this limit approach a direct proof of necessary and sufficient conditions for optimality is obtained without the need to transform the problem to a reduced state space, Moreover, the concept of order of the singular arc is not required in the proof and differentiability requirements are not as severe as those demanded by Speyer and Jacobson (1971).

In particular,

the strong generalized Legendre-Clebsch condition is relaxed and a slightly more abstract version of condition (1.6.3)

and condition (1.6.6)

below is obtained.

This set of conditions is considered in Section 4.6, Chapter 4 (Theorem 4 . 5 ) .

They have been extended to

the partially singular case by Anderson (1973). Following on immediately from the results discussed in the last two paragraphs, a general sufficiency theorem for non-negativity of a large class of second variations was presented by Jacobson (1971a) for the partially singular case.

It is that there should exist

3 a continuously differentiable, f symmetric, matrix function of time P(*) such that for all t in [t

0’

t

+ Hxx + Pfx + fxTP H

ux

T

+ fu’P

H + Pfu xu H uu

1.

AN HISTORICAL SURVEY

25

for all t in [to, tf], together with (1.6.3-5).

A

proof of this condition in the case of unconstrained terminal states is given in Section 4.5 of Chapter 4 (Theorem 4 . 2 ) .

Jacobson then applied this condition

to both the totally singular case and to the nonsingular case.

Sufficient conditions were thus

obtained for the two special cases, demonstrating that both singular and nonsingular second variations can be treated in a common general framework.

The results

developed by Jacobson and Speyer through the transformation approach and the limit approach lead to the conclusion that the sufficiency conditions are also necessary for optimality for a large class o f problems.

In the nonsingular case the well known Riccati differential equation emerges and, since this is known to be a necessary condition for the nonsingular second variation, this implies that the sufficiency conditions of the theorem are also necessary.

In the

singular case the algebraic and differential inequalities (1.6.1-3)

in strengthened form are

obtained and these have been proved necessary and sufficient conditions for optimality in the 1971 papers of Jacobson and Speyer. Thus, in the singular case, the sufficiency conditions are also necessary. Assuming that the generalized Legendre-Clebsch condition is satisfied with strict inequality, a differential equation of Riccati type is obtained which implies and is implied by (1.6.1-3);

see Section 4.5, Chapter 4

SINGULAR OPTIMAL CONTROL PROBLEMS

26

(Theorem 4 . 4 )

.

Further results arising from the investigations of Jacobson and Speyer are obtained for a class of quadratic minimization problems whose optimal control functions are partially singular (Jacobson, 1 9 7 2 ) .

Ap

explicit expression for the time-varying, singular surface (hyperplane) is obtained.

Sufficient condi-

tions for totally bang-bang arcs and for totally singular arcs are matched for the case of a firstorder singular problem to yield sufficient conditions for optimality of partially singular problems.

The

investigations do not include terminal constraints but do allow non-positive definite, time-varying, weighting matrices in the performance criterion and fixed terminal time.

They therefore contribute to the under-

standing of problems discussed in Section 1.4 above. An important paper by Moyer ( 1 9 7 3 ) derives the conditions guaranteeing that a singular extremal which joins fixed end points provides a strong minimum for the independent time variable.

Under proper assump-

tions this paper extends the Weierstrass concept of a field.

Another interesting method for singular

problems has been suggested by Vinter ( 1 9 7 4 ) .

Instead

of modifying the cost functional as is done by

Jacobson and Speyer ( 1 9 7 1 ) an alternative approach is to modify the control constraint set.

The method is

applied to a class of linear systems and again the solution of the modified problem approximates the

1.

AN HISTORICAL S U R V E Y

27

solution of the original problem to any required accuracy.

Sufficient conditions for the singular

problem have also been derived by Mayne (1973) using differential dynamic programming. The mathematical characterization of optimal controls which contain both singular and nonsingular arcs is still not complete.

Preliminary results on

the matching of singular and nonsingular arcs at junction points were obtained by Kelley et al., (1967) and by Johnson (1965).

Further work clarified and

extended these results (McDanell and Powers, 1 9 7 1 ) . The main result of this latter paper is that the sum of order of the singular arc and lowest order time derivative of control which is discontinuous at the junction must be an odd integer when the strengthened Legendre-Clebsch condition is satisfied.

New neces-

sary conditions which do not require an analyticity assumption are developed.

These aid in character-

izing problems which may possess non-analytic junctions.

A l s o , McDanell and Powers (1971) have shown

that a continuous junction for a singular arc of odd order is realizable contrary to a result given by Kelley et al., (1967).

Results from two previous

papers (McDanell and Powers, 1 9 7 0 , 1971) are used to investigate the local switching behaviour of the singular guidance problem associated with the Saturn VAS-502 flight (Powers and McDanell, 1971).

A

numerical investigation is carried out to obtain a

28

SlNGULAR OPTIMAL CONTROL PROBLEMS

satisfactory suboptimal guidance scheme using the computational procedure suggested by Jacobson et al. (1970). Outlines of the methods of proof of the necessary and sufficient conditions mentioned above are given in the survey paper by Jacobson (1971b). Other survey papers on the singular control problem have been written by Gabasov et al. (1971) and Gabasov and Kirillova (1972), in which more references to the Russian literature will be found, and by Marchal (1973) which discusses the two different meanings of the term 'chattering' in optimization theory. References Anderson, B. D. 0. (1967). A System Theory Criterion 5, for Positive Real Matrices, SIAM J. Control 171- 182. Anderson, B. D. 0. (1973). Partially Singular LinearQuadratic Control Problems, IEEE Trans. autom. Control AC-18, - 407-409. Anderson, B. D. 0. and Moore, J . B. (1968a). Network Realizations of Time-Varying Passive Impedances, Tech. Rep. EE-6810, Univ. Newcastle, Australia. Anderson, B. D. 0. and Moore, J. B. (1968b). sions of Quadratic Minimization Theory. Infinite Time Results, Int. J. Control:, 473- 480.

ExtenI1

Anderson, B. D. 0. and Moore, J. B. (1971). "Linear Optimal Control", Prentice-Hall, Englewood Cliffs, N.J. Archenti, A. R. and Vinh, N. X. (1973). IntermediateThrust Arcs and their Optimality in a Central, Time-Invariant Force Field, J. Opt. Th. Applic. 11, 293-304. -

I.

AN HISTORICAL S U R V E Y

29

A t h a n s , M. and Canon, M. D. (1964). On t h e FuelOptimal S i n g u l a r C o n t r o l of N o n l i n e a r SecondOrder Systems, IEEE T r a n s . autom. C o n t r o l AC-9, 360-370.

-

Bass, R. W. and Webber, R. F. (1965). On S y n t h e s i s of Optimal Bang-Bang Feedback C o n t r o l Systems w i t h Q u a d r a t i c Performance I n d e x , P r o c . 6 t h JACC, 213-219. B e l l , D. J . (1965). Optimal T r a j e c t o r i e s and t h e 16 A c c e s s o r y Minimum Problem, Aeronaut. Q. -' 205- 2 20.

B e l l , D. J . (1968). Optimal Space T r a j e c t o r i e s - A Review of P u b l i s h e d Work, Aero. J1. R. a e r o n a u t . SOC. 72, 141-146.

-

B e l l , D. J. (1969). S i n -g u l a r Extremals i n t h e C o n t r o l of a S t i r r e d R e a c t o r , Chem. Engng. S c i . 24,

521-525.

B e l l , D. J. (1971a). The Second V a r i a t i o n and S i n g u l a r Space T r a j e c t o r i e s , I n t . J. C o n t r o l 14, 69 7- 703.

-

B e l l , D. J . (1971b). The Non-Optimality of Lawden's S p i r a l , A s t r o n a u t i c a Acta 1 6 , 317-324.

-

B l i s s , G. A. (1946). " L e c t u r e s on t h e C a l c u l u s of V a r i a t i o n s " . Univ. Chicago P r e s s , Chicago.

B o l o n k i n , A. A. (1969). S p e c i a l Extrema i n Optimal C o n t r o l Problems, Engng. Cybern. No. 2 , 170-183. B r e a k w e l l , J. V. (1965). A Doubly S i n g u l a r Problem i n Optimal I n t e r p l a n e t a r y Guidance, SIAM J . Control 3, 71-77. Bryson, A. E. and Ho, Y. C. (1969). "Applied Optimal Control". B l a i s d e l l , Waltham, Mass. Bryson, A. E . , Desai, M. N. and Hoffman, W. C. (1969). Energy-State Approximation i n Performance O p t i m i z a t i o n of S u p e r s o n i c A i r c r a f t , J. A i r c r a f t 6 , 481-488.

-

30

SINGULAR OPTIMAL CONTROL PROBLEMS

Connor, M. A. (1974). Singular Control of Delay Systems, J. Opt. Th. Applic. 13, 538-544. Dobell, A. R. and Ho, Y. C. (1967). Optimal Investment Policy: An Example of a Control Problem in Economic Theory, IEEE Trans. autom. Control AC-12, - 4-14.

Forbes, G. F. (1950). The Trajectory of a Powered Rocket in Space, J. Br. Interplanet. SOC. 9, 75-79.

Fraeijs de Veubeke, B. (1965). Canonical Transformations and the Thrust-Coast-Thrust Optimal 11, 271-282. Transfer Problem, Astronautica Acta Gabasov, R, (1968). Necessary Conditions for Optimality of Singular Control, Engng. Cybern. No.5, 28-37. Gabasov, R. (1969). On the Theory of Necessary Optimality Conditions Governing Special Controls, 13, 1094-1095. Sov. Phys.-Dokl. Gabasov, R. and Kirillova, F. M. (1972). High Order Necessary Conditions for Optimality, SIAM J. Control 10, 127-168. Gabasov, R., Kirillova, F. M. and Strochko, V. A. (1971). Conditions for High-Order Optimality (Review), Automn remote Control 32, 689-704, 857-875, 1013-1040. Gelfand, I. M. and Fomin, S. V. (1963). "Calculus of Variations". Prentice-Hall, Englewood Cliffs, N. J. Goh, B. S. (1966a). The Second Variation for the 4, Singular Bolza Problem, SIAM J. Control 309-325. Goh. B. S. (1966b). Necessary Conditions for Singular Extremals Involving Multiple Control Variables, SIAM J. Control 4, 716-731. Goh, B. S. (1967). Optimal Singular Control for MultiInput Linear Systems, J. math. Analysis Applic. 20, - 534-539.

1.

AN HISTORICAL SURVEY

31

Goh, B. S. (1969/70). Optimal Control of a Fish Resource, Malay. Scientist 5, 65-70. Goh, B. S. (1970). A Theory of the Second Variation in Optimal Control, unpublished report, Division of Applied Mechanics, Univ. California, Berkeley. Goh, B. S. (1973). Compact Forms of the Generalized Legendre Conditions and the Derivation of Singular Extremals, Proc. 6th Hawaii International Conference on System Sciences, 115-117. Graham, J. W. and D'Souza, A. F. (1970). Singular Optimal Control of Discrete-Time Systems, Proc. 11th JACC, 320-328. Gurman, V. I. (1967). Method of Multiple Maxima and the Conditions of Relative Optimality of Degenerate Regimes, Automn remote Control 28, 1845-1852. Haynes, G. W. (1966). On the Optimality of a Totally Singular Vector Control: An Extension of the Green's Theorem Approach to Higher Dimensions, 4, 662-677. SIAM J. Control Hermes, H. (1964). Controllability and the Singular Problem, SIAM J. Control 2, 241-260.

-

Hermes, H. and Haynes, G. W. (1963). On the Nonlinear Control Problem with Control Appearing Linearly, SIAM J. Control 1, 85-108. Jacobson, D. H. (1969). A New Necessary Condition of Optimality for Singular Control Problems, SIAM J. Control 7, 578-595.

-

Jacobson, D. H. (1970a). On Conditions of Optimality for Singular Control Problems, IEEE Trans. autom. - 109-110. Control AC-15, Jacobson, D. H. (1970b). Sufficient Conditions for Nonnegativity of the Second Variation in Singular and Nonsingular Control Problems, SIAM J. Control 8, - 403-423.

32

SINGULAR OPTIMAL CONTROL PROBLEMS

Jacobson, D. H. (1970~). New Conditions for Boundedness of the Solution of a Matrix Riccati Differential Equation, 3. Differential Equations 8, 258-263. Jacobson, D. H. (1971a). A General Sufficiency Theorem for the Second Variation, J. math. Analysis Applic. 34, 578-589. Totally Singular Quadratic Jacobson, D. H. (1971b). Minimization Problems, IEEE Trans. autom. Control AC-16, - 651-658. Jacobson, D. H. (1972). On Singular Arcs and Surfaces in a Class of Quadratic Minimization Problems, 37, 185-201. J. math. Analysis Applic. Jacobson, D. H. and Speyer, J. L. (1971). Necessary and Sufficient Conditions for Optimality for Singular Control Problems: A Limit Approach, J. math. Analysis Applic. 34, 239-266. Jacobson, D. H., Gershwin, S . B. and Lele, M. M. (1970). Computation of Optimal Singular Controls, IEEE Trans. autom. Control AC-15, 67-73. Johansen, D. E. (1966). Convergence Properties of the Method of Gradients, in "Advances in Control Systems" (C. T. Leondes, ed) Vo1.4, pp.279-316. Academic Press, New York and London. Johnson, C. D. (1965). Singular Solutions in Optimal Control Problems, in "Advances in Control Systems" (C. T. Leondes, ed) V01.2, pp.209-267. Academic Press, New York and London. Johnson, C. D. and Gibson, J. E. (1963). Singular Solutions in Problems of Optimal Control, IEEE - 4-15. Trans. autom. Control AC-8, Keller, J. L. (1964). On Minimum Propellant Paths for Thrust Limited Rockets, Astronautica Acta -910 262-269. Kelley, H. J. (1963). Singular Extremals in Lawden's 1, Problem of Optimal Rocket Flight, AIAA J. 1578- 1580.

1.

AN HISTORICAL SURVEY

33

Kelley, H. J. (1964a). A Second Variation Test for Singular Extremals, AIAA J. 2, 1380-1382. Kelley, H. J. (1964b). A Transformation Approach to Singular Subarcs in Optimal Trajectory and 2, 234-240. Control Problems, SIAM J. Control Kelley, H. J., Kopp, R. E. and Moyer, H. G. (1967). Singular Extremals, in "Topics in Optimization" (G. Leitmann, ed) pp.63-101. Academic Press, New York. Kopp, R. E. and Moyer, H. G. (1965). Necessary Con3 ditions for Singular Extremals, AIAA J. -' 1439-1444. Krener, A. J. (1973). The High Order Maximal Principle, in "Geometric Methods in Systems Theory" (D. Q. Mayne and R. W. Brockett, eds) pp.174-184. NATO Advanced Studies Institute Series: Mathematics and Physics. Lawden-, D. F. (1950). Note on a Paper by G. F. Forbes, J. Br. Interplanet. SOC. 2, 230-234. Lawden, D. F. (1952). Inter-Orbital Transfer of a Rocket, Annual Report, Br. Interplanet. Soc., 321-333. Lawden, D. F. (1961). Optimal Powered Arcs in an Inverse Square Law Field, J. Am. Rocket Soc. 31, 566-568. Lawden, D. F. (1962). Optimal Intermediate-Thrust Arcs 8, in a Gravitational Field, Astronautica Acta 106-123 . Lawden, D. F. (1963). "Optimal Trajectories for Space Navigation", Butterworth, Washington, D.C. Mancill, J. D. (1950). Identically Non-Regular Problems in the Calculus of Variations, Revista Matematica Y Fisica Teorica, Ser. A 7, 131-139. Marchal, C. (1973). Chattering Arcs and Chattering Controls, J. Opt. Th. Applic. 11,441-468.

34

S I N G U L 4 R OPTIMAL CONTROL P R O B L E M S

Mayne, D. Q. (1970). Sufficient Conditions for Optimality for Singular Control Problems, Report 18/70, C.C.D., Imperial College, London. Mayne, D. Q. (1973). Differential Dynamic Progranr ming - A Unified Approach to the Optimization of Dynamic Systems, in "Advances in Control Systems (C. T. Leondes, ed) Vol.10, pp.179-254. Academic Press, New York and London. McDanell, J . P. and Powers, W. F. (1970). New JacobiType Necessary and Sufficient Conditions for 8, Singular Optimization Problems, AIAA J. 1416-1420. McDanell, J . P. and Powers, W. F. (1971). Necessary Conditions for Joining Optimal Singular and Nonsingular Subarcs, SIAM J. Control 9, 161-173. Miele, A. (1950-51). Problemi di Minim0 Tempo nel Volo Non-Stazionario degli Aeroplani, Atti Accad. Sci. Torino 85, 41-52. Miele, A . (1955a). General Solutions of Optimum Problems in Nonstationary Flight, NACA Tech. Memo. No. 1388. Mizle, A . (1955b). Optimum Flight Paths of Turbojet Aircraft, NACA Tech. Memo. No.1389. Miele, A. (1962). Extremization of Linear Integrals by Green's Theorem, in "Optimization Techniques" (G. Leitmann, ed) pp. 69-98. Academic Press, New York. Moore, J. B. (1969). A Note on a Singular Optimal 5 , 857-858. Control Problem, Automatica Moore, J . B. and Anderson, B. D. 0. (1968). Extensions of Quadratic Minimization Theory. I Finite 7, 465-472. Time Results, Int. J. Control Foyer, H. G. (1973). Sufficient Conditions for a Strong Minimum in Singular Control Problems, 11, 620-636. SIAM J. Control -

1.

AN HISTORICAL S U R V E Y

35

Moylan, P. J. and Moore, J. B. (1971). Generalizations 7, of Singular Optimal Control Theory, Automatica 591-598. Powers, W. F. and McDanell, J. P. (1971). Switching Conditions and a Synthesis Technique for the Singular Saturn Guidance Problem, J. Spacecraft Rockets 8, 1027-1032. Robbins, H. M. (1965). Optimality of IntermediateThrust Arcs of Rocket Trajectories, AIAA J. 3, 1094-1098. Robbins, H. M. (1967). A Generalized LegendreClebsch Condition for the Singular Cases of 3, 361-372. Optimal Control, IBM J1 Res. Dev. Rohrer, R. A. (1968). Lumped Network Passivity Criteria, IEEE Trans. Circuit Theory CT-15, - 24-30. Rohrer, R. A. and Sobral, M. (1966). Optimal Singu1ar Solutions for Linear Multi-Input Systems, Trans. 88, 323-328. ASME, J. Basic Eng. Schultz, R. L. and Zagalsky, N. R. (1972). Aircraft 9, 108-114. Performance Optimization, J. Aircraft Silverman, L. M. (1968). Synthesis of Impulse Response Matrices by Internally Stable and Passive Realizations, IEEE Trans. Circuit Theory CT-15 -’ 238-245. Sirisena, H. R. (1968). Optimal Control of Saturating Linear Plants for Quadratic Performance Indices, Int. J. Control 8, 65-87. Snow, D. R. (1964). Singular Optimal Controis for a Class of Minimum Effort Problems, SIAM J. Control -2, 203-219. Soliman, M. A. and Ray, W. H. (1972). On the Optimal Control of Systems having Pure Time Delays and Singular Arcs. I Some Necessary Conditions f o r Optimality, Int. J. Control 16, 963-976. Speyer, J. L. (1973). On the Fuel Optimality of 10, 763-765. Cruise, J. Aircraft -

36

SINGULAR OPTIMAL CONTROL PROBLEMS

Speyer, J. L. and Jacobson, D. H. (1971). Necessary and Sufficient Conditions for Optimality for Singular Control Problems: A Transformation 33, 163-187. Approach, J. math. Analysis Applic. Steinberg, A. M. (1971). On Relaxed Control and Singular Solutions, J. Opt. Th. Applic. 8, 441-453. Tait, K. S. (1965). Singular Problems in Optimal Control, Ph.D. dissertation, Harvard Univ., Cambridge, Mass. Tarn, T. J., Rao, S. K. and Zaborszky, J. (1971). Singular Control of Linear-Discrete Systems, IEEE Trans. autom. Control AC-16, - 401-410. Vapnyarskiy, I. B. (1966). Solution of Some Optimal Control Problems, Engng. Cybern. No.4, 37-43. Vapnyarskiy, I. B. (1967). An Existence Theorem for Optimal Control in the Bolza Problems, some of its Applications and the Necessary Conditions for the Optimality of Moving and Singular Systems, USSR Comp. Math., Math. Phys. 7, 22-54. Vinter, R. B. (1974). Approximate Solution of a Class of Singular Control Problems, J. Opt. Th. Applic. 13, 461-483. Wonham, W. M. and Johnson, C. D. (1964). Optimal Bang-Bang Control with Quadratic Performance 86, 107-115. Index, Trans. ASME, J. Basic Eng. -

CHAPTER 2 Fundamental Concepts 2.1

Introduction A s has been d e s c r i b e d i n Chapter 1 t h e r e w a s no

s t a n d a r d mathematical procedure f o r a n a l y s i n g s i n g u l a r c o n t r o l problems b e f o r e t h e e a r l y 1960's a p a r t from

Miele's 'Green's Theorem' approach.

However, from

about 1963 onwards r e s e a r c h e r s began t o s t u d y t h e second v a r i a t i o n of t h e g e n e r a l performance index of optimal c o n t r o l i n an a t t e m p t t o f i n d new n e c e s s a r y c o n d i t i o n s f o r a s i n g u l a r c o n t r o l t o be optimal.

The

t h e o r y of t h e second v a r i a t i o n of a f u n c t i o n a l had been

w e l l e s t a b l i s h e d i n t h e c l a s s i c a l c a l c u l u s of v a r i a t i o n s and had f i g u r e d prominently i n t h e work c a r r i e d out by P r o f e s s o r G. A . Bliss and h i s s t u d e n t s a t t h e U n i v e r s i t y of Chicago d u r i n g t h e f i r s t h a l f of t h e present century.

But a l t h o u g h t h e g e n e r a l t h e o r y f o r

t h e problem of Bolza reached a h i g h degree of s o p h i s t i c a t i o n under t h e a t t e n t i o n of t h e Chicago School no a p p l i c a t i o n s of t h i s t h e o r y had been attempted. Indeed, i n t h e p r e f a c e t o h i s book ( B l i s s , 1946) P r o f e s s o r Bliss makes an appeal f o r s u i t a b l e examples t o be l i s t e d which would i l l u s t r a t e t h e theory.

This

appeal has t o a l a r g e e x t e n t been answered over t h e

l a s t t h i r t y y e a r s by t h e enormous r e s e a r c h e f f o r t engendered by c o n t r o l problems a r i s i n g from t h e o p t i m i z a t i o n of dynamical systems. 37

38

SINGULAR OPTIMAL CONTROL PROBLEMS

Until the early 1960's the theory of the second variation of a functional had rarely been applied to any practical problems.

Even in the field of

Mathematical Physics it had often been felt that the additional complexity of the second variation outweighed any possible benefit which might accrue from its use.

However, with the advent of such singular

problems as Lawden's intermediate-thrust arcs (Lawden, 1961, 1962, 1963) in Aerospace and Siebenthal and Aris's stirred tank reactor (Siebenthal and Aris, 1964) in Chemical Engineering it was clear that a satisfactory mathematical analysis of such problems lay in the theory of the second variation.

This approach has been

fully justified as will be seen in this book.

Not only

has t h e study of the second variation of a general performance index yielded necessary conditions for optimality of singular arcs but it has played a no less important part in the derivation of sufficient conditions for such arcs and in the production of algorithms for the numerical solution of non-singular problems. This present chapter sets before the reader a few fundamental concepts necessary for an understanding of what is to follow.

First, the general optimal control

problem mentioned in Chapter 1 is reiterated and placed on a firm mathematical foundation.

It should be

emphasized here that we give a general statement of what is usually referred to as the optimal control problem.

A singular problem is a special case of this

2.

FUNDAMENTAL CONCEPTS

general statement.

39

Next, t h e f i r s t and second

v a r i a t i o n s of t h e c o s t f u n c t i o n a l from t h e g e n e r a l The method o f

optimal c o n t r o l problem a r e d e r i v e d .

g e n e r a t i o n of t h e s e two v a r i a t i o n s f o l l o w s c l o s e l y t h a t used by B l i s s (1946) a l t h o u g h , of c o u r s e , h i s a n a l y s i s does n o t i n c l u d e s p e c i f i c mention of c o n t r o l variables.

Much of t h e n o t a t i o n used i n t h i s book

c o i n c i d e s with t h a t used by Bliss b u t where t h e r e a r e d i f f e r e n c e s w e have changed d e l i b e r a t e l y t o be i n keeping w i t h t h a t used i n t h e modern c o n t r o l l i t e r a t u r e . Having d e r i v e d g e n e r a l e x p r e s s i o n s f o r both t h e f i r s t and second v a r i a t i o n s , shown t h a t t h e f i r s t v a r i a t i o n must be z e r o and t h e second v a r i a t i o n nonn e g a t i v e f o r a minimizing a r c , t h e f i n a l s e c t i o n of t h e p r e s e n t c h a p t e r f o r m u l a t e s t h e g e n e r a l statement of a s i n g u l a r optimal c o n t r o l problem.

The correspond-

i n g forms f o r t h e f i r s t and second v a r i a t i o n s i n t h e s i n g u l a r case are s t a t e d .

A number of examples a r e

p r e s e n t e d i n t h i s c h a p t e r t o i l l u s t r a t e t h e many a s p e c t s of b o t h v a r i a t i o n s .

2.2

The General Optimal C o n t r o l Problem

Consider an n-dimensional s t a t e v e c t o r space X T w i t h time-varying elements x = ( x l x 2...x,) ,

\

= %(t),

k

= 1,2

,...n ,

and an m-dimensional c o n t r o l T v e c t o r space V w i t h elements u = (ulu2 Urn) ,

u. = u i ( t ) , 1

d e f i n e d by

i = 1,2,

...m,

-

to < t

...

tf.

The s e t V i s

40

SINGULAR OPTIMAL CONTROL PROBLEMS

V = (u(t) : a. 1 -< ui

bi,

i

= 1,2

,...,ml

(2.2.1)

where a;, bi can be known functions of time but are usually constants. Since the major portion of this book will be discussing the case of singular control we shall assume, unless otherwise stated, that a control vector u belongs to the interior of space V so that a. < ui < bi,

i

= 1,2,

...,m.

1

Should some of the ui's become equal

to the corresponding bounds ai or bi then either the

technique of Valentine (1937) may be employed or those variations

Bi(*)

(see below) of the control variables

which attain their bounds can be put to zero.

Further-

more, it may sometimes be convenient (in Section 3.2.2 for example) to update the control vector u to the status of a derivative and write u = v,

v(to)

=

0

(2.2.2)

with v(t ) arbitrary. This transformation will be f

seen to bring the control problem more in line with the classical problem of Bolza. Suppose the behaviour of a dynamical system is governed by differential equations

H

= f(x,u,t)

(2.2.3)

2.

41

FUNDAMENTAL CONCEPTS

and boundary conditions

where to and xo are specified and {to, x(to>, tf, x(tf)) belongs to S, a closed subset of R2n+2.

The terminal

constraint function IJ is an s-dimensional column vector function of x(tf)

and tf.

The final time tf may o r may

not be specified. We suppose further that the performance of the system is measured by a cost functional of the form

J = F[x(tf),

tf] +

I'

L(x,u,t)dt.

t0

(2.2.6)

The n-dimensional vector function f of eqn(2.2.3)

and

the scalar functions F and L are assumed to be at least twice continuously differentiable in each argument. The general problem of optimal control is to find an element of U which minimizes the cost functional J of (2.2.6) subject to (2.2.3-5). 2.3

The First Variation of J Define a one-parameter family of control vectors

42

SINCilJLAR OPTIMA12 CONTROL PROBLEMS

w i t h t h e o p t i m a l v e c t o r g i v e n by u ( . , O ) .

The c o r r e s -

ponding s t a t e v e c t o r

,E )

X('

(2.3.2)

w i l l a l s o b e a f u n c t i o n of t i m e t ( t o < t < tf(E) ) and p a r a m e t e r

E

In a l l cases u(-

with ,E)

E

= 0 along t h e optimal t r a j e c t o r y .

b e l o n g s t o U, x ( * ,E)

b e l o n g s t o some

D i f f e r e n t i a l s of f a m i l y ( 2 . 3 . 2 )

f u n c t i o n s p a c e Y.

are

A s i n t h e c l a s s i c a l c a l c u l u s of v a r i a t i o n s ( B l i s s , 1 9 4 6 ) t h e symbol 6 d e n o t e s d i f f e r e n t i a l s o n l y w i t h r e s p e c t t o t h e parameter

E.

We now i n t r o d u c e a s e t of s t a t e

v a r i a t i o n s and f i n a l - t i m e v a r i a t i o n s d e f i n e d a l o n g t h e optimal t r a j e c t o r y a s

i n which case d t f = SfdE and 6x = vde. S i m i l a r l y , w e can d e f i n e a c o n t r o l v a r i a t i o n a l o n g t h e o p t i m a l t r a j e c t o r y as

2.

FUNDAMENTAL CONCEPTS

43

B = (aU/aE)E=o.

(2.3.5)

From the system equatj.ons ( 2 . 2 . 3 ) state variation

rl

it follows that the

must satisfy the equation of

variation

;1 where f,,

= f , r l

f, are n

x

(2.3.6)

+ fuB

n and n

x

m matrices respectively.

Because of the boundary conditions ( 2 . 2 . 4 - 5 ) set of variations Sf,

rl

the

must satisfy end conditions of

the form rl(to>

= 0

.

where $Jt is an s-dimensional vector and f matrix.

(2.3.7)

xf

an s

We now adjoin the system equations ( 2 . 2 . 3 ) terminal constraints ( 2 . 2 . 5 ) of ( 2 . 2 . 6 )

x

n

and the

to the cost functional J

by A , an n-vector of Lagrange multiplier

functions of time, and by v, an s-dimensional constant vector of Lagrange multipliers respectively. functional may then be written as

The cost

SINGULAR OPTIMAL CONTROL PROBLEMS

44

where H(x,u,X,t)

T + A f(x,u,t).

= L(x,u,t)

When the vectors u ( * ,E)

and

are substituted into eqn(2.3.9)

x(*,E)

(2.3.10)

of (2.3.1-2)

the cost functional J

may be looked upon as a function of the single parameter

E

and from J(E) one can easily calculate the

first differential dJ. dJ

=

[(Ft

+

T v $t + H

In fact,

-

ATg)dt

T

+ (Fx + v $x)dx] t=tf

By integrating the term in 6; in the integrand of eqn(2.3.11)

by parts, using (2.3.3)2

to eliminate

6x(tf) and noting that 6x(to) = 0 since x(to) is specified, we obtain dJ

= [ (Ft

T + v Jlt + H)dt + (Fx + vT$x

-

AT)dx] t'f

On the optimal trajectory where the parameter

E:

is zero this differential takes the form dJ = J1(cf,n,B). dE:. The second differential d2J on the optimal trajectory can similarly be written as d2J = 2J2(Sf,~,B)d~2. A Taylor series for the func-

tion J(E) may then be written as

2.

FUNDAMENTAL CONCEPTS

J ( E ) = J ( o ) + E J +~ c2J2 +

The function J,(Sf,q,B)

45

...

(2.3.13)

is called the first variation

of J on the optimal trajectory and from its definition and eqn(2.3.12)

it is clear that

Bearing in mind the assumption made in Section 2.2 that u belongs to the interior of V it follows from

(2.3.13) that a necessary condition for u(*,O) to be a control vector which minimizes J is J1

= 0.

That is,

a necessary condition for optimal control is that the first variation should vanish for all admissible variations.

By choosing the adjoint vector X and the

vector v so as to make the coefficients of

0,

n(tf)

and Sf vanish in ( 2 . 3 . 1 4 ) we obtain the following results:

-iT= XT(tf)

=

Hx(x,u,X,t) -F,(;;(tf),

(2.3.15) tf)

+

*Xf

T

(2.3.16)

SINGULAR OPTIMAL CONTROL PROBLEMS

46

(2.3.17) The first variation of J then reduces to J 1 = ItfHU6 dt.

(2.3.18)

t0

With the control variables away from any bounds the variation 8 in the integrand of (2.3.18) is arbitrary. Since J 1 is to vanish for all admissible variations B the fundamental lemma of the calculus of variations (Bliss, 1 9 4 6 ) yields the condition

Hu

(2.3.19)

= 0.

Of course, when u, a member of V, is allowed to attain its bounds we are led to Pontryagin’s Minimum Principle, namely

-u =

arg min H(;,U,A,~).

(2.3.20)

Throughout the above discussion

x(*)and U(*) denote

u

the candidate state and control functions respectively. A further first order condition is the necessary

condition of Clebsch (Bliss, 1946).

In the control

formulation this condition may be written

Tr

T

O

O

L

O

(2.3.21)

2.

FUNDAMENTAL CONCEPTS

for all (n+m)-vectors

TT

(-In fU)T

47

satisfying the equation =

(2.3.22)

0

where In is the nth order identity matrix. We now illustrate the use of necessary conditions (2.3.15)

,

(2.3.19) to obtain a candidate arc for

optimality by applying them to a rocket problem. Example 2.1

The problem of finding the thrust direc-

tion programme necessary to maximize the range of a rocket with known propellant consumption is considered by Lawden ( 1 9 6 3 ) .

The thrust direction is limited to

lie in a vertical plane through the launching point. The acceleration due to gravity is assumed constant and flight takes place in vacuo over a flat earth. rocket is launched with zero initial velocity at t and burn-out occurs at a known instant t

=

T.

The = 0

The

vehicle continues under gravity along a ballistic trajectory until impact.

The acceleration f caused by

the motor thrust, essentially positive, is a given function of time. With Ox and Oy horizontal and vertical axes through 0 and lying in the plane of flight, the equations of

motion for this problem are

(2.3.23)

SINGULAR OPTIMAL CONTROL PROBLEMS

48

where f

=

cm/M and 0 , the control variable, is the

angle made by the thrust direction with Ox (Lawden, 1963). The initial values of the state variables u, v (horizontal and vertical velocity) and x, y (horizontal and vertical displacement) are specified of flight T to burn-out.

as

is the time

There are no end values

specified at the final end-point except T.

The

boundary conditions for the problem are then v(0) = 0

u ( 0 ) = 0,

to = 0,

(2.3.24) y(0)

x(0) = 0,

tf = T.

= 0,

It is required to maximize the total range, which is a function of the values of the state variables at burnout.

This is equivalent to minimizing the cost

function

The Hamiltonian H of eqn(2.3.10) H

=

is

AU fcose + XV (fsine - 8 ) + A u + X v.

Eqn(2.3.15)

X

Y

(2.3.26)

then yields

-xu ix

- A,, = 0,

-Av

=

xY

iY = o .

(2.3.27)

49

2. FUNDAMENTAL CONCEPTS

Eqn(2.3.19)

leads to the result tan8

=

Av/Au.

(2.3.28)

The end conditions given by eqn(2.3.16) Au(tf>

= -(vf+r>/g,

Av(tf)

=

are

-uf(vf+r)/gr, (2.3.29)

Ax(tf)

=

-1,

where r

=

J(vf2 + 2gyf).

Ay(tf)

=

-uf/r A s in (Lawden, 1963)these

results lead to tan8

=

uf/r.

(2.3.30)

Pontryagin's Principle (2.3.20) or the Clebsch condition (2.3.21-22) are satisfied if 8 takes the positive acute angle solution of eqn(2.3.30).

The

extremal is therefore a trajectory along which the thrust direction remains at a constant positive acute angle to Ox.

Whether this extremal is an optimal

trajectory is still to be decided and this will be the subject of a further example in the next section. 2.4

The Second Variation of J In the previous section it was shown that the

augmented cost functional J of (2.3.9) can be thought of as a function J ( E ) of the single parameter

Eqn(2.3.11)

E.

gives the first differential of J and is

quoted again here for convenience:

SINGULAR OPTIMAL CONTROL PROBLEMS

50

dJ

=

[ (Ft + v T+t + H - A T%)dt

+ (Fx + v T+x)d~It,tf

+ jtf{HXGx + Hu6u - A T 6x)dt *

(2.4.1)

t0

This formula is valid along all arcs x(-,E), u(*,E) belonging to the one-parameter families (2.3.1-2).

In

particular, we have seen that along the optimal trajectory (where

the first differential dJ vanishes.

E=O)

From eqn(2.4.1)

one may calculate the second

differential of J as

J'd

=

[(Ft + v +

+t

T

(Ftt

T + H -A %)d2t +

v +tt

T + (Fx + v 11, )d2x X

+ fi -XTx)dt2

+ 2H Sxdt + 2H Gudt -2ATSkdt] U

X

1

.tF

+

IIHxG2x + H 62u - A U

T

G2t +

t0

t=tf

T G X Hxx6x

T T + 26u HuxSx + 8u H 6u)dt. uu By expanding the term

fi

=

(2.4.2)

dH/dt into its constituent

parts, integrating by parts the term in S 2 $ ,

eliminat-

2.

51

FUNDAMENTAL CONCEPTS

i n g t h e & d i f f e r e n t i a l s o u t s i d e t h e i n t e g r a l s i g n by means of e q n s ( 2 . 3 . 3 )

and f i n a l l y by u s i n g e q n s ( 2 . 3 . 1 5 ,

19), w e f i n d t h a t T T d 2 J = [(H + Ft + v J l t ) d 2 t + (Fx + v $

J

~

-A

T

)d2x]

tf

T

T

+ d x (Fxx + v $Jxx)dx1

+ [ (Ht

-

H K)dt2 X

tf

+ 2H d x d t ] X

tf

(2.4.3) The c o e f f i c i e n t s o f t h e terms i n d 2 t f and d 2 x ( t f ) are zero because of t h e t r a n s v e r s a l i t y conditions (2.3.16-17).

The second d i f f e r e n t i a l t h u s r e d u c e s t o

+

[(Ht

-

H & ) d t 2 + 2Hxdx d t I ,

+ ftf(6xTH t0

X

f

6~ + 26uTHux6x + Bu TH xx

6u)dt uu

(2.4.4)

52

SINGULAR OPTIMAL CONTROL PROBLEMS

where @(x(tf),

tf) = F + v

T

$J

and xf

1

x(tf).

A s men-

tioned in Section 2.3 the second differential d2J on the optimal trajectory may be written as d2J

=

2J,(Cf,q,B)d~2

and from eqn(2.4.4)

it is seen

that

J2 = IQtftfCf

+

1

+

x 5f GfCf f f

0t

.tc

LfiriTHxxn + BTHuxrl

+

t0

+

rl(tf))

JBTH Bldt.

(2.4.5)

uu

This expression J2 is the coefficient of

E~

in the

Taylor series (2.3.13) and is called the second variation of the cost functional J along the optimal trajectory.

It follows that a necessary condition

for u ( * , O ) to be a control vector which minimizes J is d2J/dE2 > 0. That is to say the second variation J, must be non-negative. A particular set of admissible variations satisfy-

ing eqns(2.3.6-8)

is given by

2.

FUNDAMENTAL CONCEPTS

53

For this set of variations the second variation J, vanishes.

Thus, if the candidate arc obtained from the

vanishing of the first variation satisfies the condition J, 2 0 then the set of variations ( 2 . 4 . 6 ) minimize J,.

must

We are accordingly led to an auxiliary

problem known as the accessory minimum problem.

This

is the problem of minimizing J2 with respect to the set of variations

Ef,

q(*),

B(*)

satisfying the

equations of variation ( 2 . 3 . 6 - 8 ) . A further necessary condition for a minimizing

arc is known in the classical literature as the Jacobi condition.

In the optimal control problem with second

variation J2 given above ( 5f = 0 ) the Jacobi condition may be stated as follows (Bryson and Ho, 1969, but see also Chapter 4 ) : an optimal trajectory contains no conjugate point between its end points.

This will be

the case if the matrix S remains finite for

- -

to < t < tf where S satisfies the matrix differential

equation -S

= H

xx

TS -(Hxu + SfU)HUU -1 (HUx + + Sfx + fx

f

TS )

U

(2.4.7)

and end condition S(t,>

= @XX[X(t,>

tf I.

(2.4.8)

SINGULAR OPTIMAL CONTROL PROBLEMS

54

Example 2.2 (Bell, 1965)

Consider again the problem

of maximum range of a rocket vehicle discussed in the

example of Section 2.3. The equations of variation on the extremal are, from eqn~(2.3.23)~

hu

-8,

=

fsine

,

TIv =

Be fcos9

The equations of variation on the extremal of the end conditions are, from eqns(2.3.241,

5,

= 0,

5f

= 0,

rl ( 0 ) = 0 ,

u

Using eqns(2.3.25-26, (2.4.5) may b e written as

- lJ

f ( t - k ) B e 2 sece dt 0

'lJ0)

= 0,

29) the second variation

2.

FUNDAMENTAL CONCEPTS

55

1

T + -(v + r). This variation has to be nong f negative for the extremal found in Example 2.1 of

where k

=

Section 2.3 to be optimal. To demonstrate that it is indeed always non-negative we integrate eqns(2.4.9) with 8 having the positive, acute angle solution of eqn(2.3.30).

where

I1

This gives

=

1

rT fBedt

and

I2 =

0

0

Substituting for nU(T), expression for J

2

nv(T),

0

and 11 (T) in the Y

T + r)j fBe2dt 1.

g f

Thus, J2 > 0 and, moreover, J, 0.

“(T)

we obtain

T + sec8[1 f (T-t) Be2dt + -1(v

$,

rT f(T-t)Bedt.

J

0

= 0

if and only if

The second variation is therefore always

non-negative and the extremal found from the first variation satisfies the necessary condition associated with the second variation.

It is worth pointing out at this stage that the following notation for the second variation will

SINGULAR OPTIMAL CONTROL PROBLEMS

56

normally be used throughout this book:

Q

=

Qf

C = H

Hxx’

-

ux’

A = f

*x x ’ f f

X’

R = H

uu ’

B = f

Furthermore, the variations

U’

IT

D = +

X

f

.

(2.4.10)

in state and B in

control will be denoted by x and u respectively. No confusion will result when the second variation is being considered as a new cost function J[u(*)] in the accessory minimum problem.

The form of the second

variation for a constrained optimal control problem, from eqns(2.4.5) as (5,

and (2.3.6-8)’

may then be written

= 0)

(2.4.11) subject to and 2.5

k

=

Ax + Bu,

Dx(tf)

=

x(to)

0

= 0.

(2.4.12) (2.4.13)

A Sirgular Control Problem

We consider here the class of control problems where the dynamical system is described by the ordinary differential equations

2.

FUNDAMENTAL CONCEPTS

57

= fl(x,t) + fu(x,t)u.

(2.5.1)

where f(x,u,t)

The performance of the system is measured by the cost functional J =

L(x,t)dt

+

F[x(tf),

tf]

(2.5.2)

t0

and the terminal states must satisfy +[x(t,),

(2.5.3)

tfl = 0.

The final time tf is assumed to be given explicitly. Thus, the Hamiltonian H for this problem formulation is linear in the control variables, and the problem turns out t o be singular. It is clear from eqn(2.4.5)

that the second

variation is

subject to eqns(2.3.6-8).

In terms of the notation

mentioned at the end of Section 2.4 this second variation is J[(.)]

=

/tf(hxTQ~ + uTC d d t +

&XT (tf)Qfx(tf)(2.5.5)

t0

subject to

k

=

Ax + Bu,

x(to) = 0

58

SINGULAR OPTIMAL CONTROL PROBLEMS

Dx(t ) = 0. f We conclude this section with a simple example: Example 2.3

Consider the following scalar control

problem: minimize

J subject to

k

=

;[

= u,

2

x2dt x(0) = 1,

-

IuI < 1.

This problem is linear in u with the Hamiltonian

where

H

=

fx2 + Xu

x

=

-x.

A singular arc is one along which

H u = X = O for a finite interval of time.

During this interval

we have H u = X = O which implies

x = 0.

In this case x vanishes identically and s o does u. The arc in (x,t)-space along which u is zero is thus a singular arc.

2.

FUNDAMENTAL CONCEPTS

59

References Bell, D. J. (1965). Optimal Trajectories and the Accessory Minimum Problem, Aeronaut. Q. -' 16 205-220. Bliss, G. A. (1946). "Lectures on the Calculus o f Variations". Univ. Chicago Press, Chicago. Bryson, A. E. and Ho, Y. C. (1969). "Applied Optimal Control". Blaisdell, Waltham, Mass. Lawden, D. F. (1961). Optimal Powered Arcs in an Inverse Square Law Field, J. Am. Rocket SOC. -' 31 566-568. Lawden, D. F. (1962). Optimal Intermediate-Thrust Arcs in a Gravitational Field, Astronautica Acta -8, 106-123. Lawden, D. F. (1963). "Optimal Trajectories for Space Navigation", Buttemorth, Washington, D.C. Siebenthal, C. D. and Aris, R. (1964). Studies in Optimisation - VI. The Application of Pontryagin's Methods to the Control of a Stirred Reactor, Chem. Engng. Sci. 19, 729-746.

-

Valentine, F. A. (1937). The Problem of Lagrange with Differential Inequalities as Added Side Conditions, in "Contributions to the Theory of Calculus of Variations (1933-1937)" pp 403-447. Univ. Chicago Press, Chicago.

This page intentionally left blank

CHAPTER 3 Necessary Conditions for Singular Optimal Control 3.1

Introduction

It has already been stated in the previous two chapters that problems involving singular controls do arise in the optimization of engineering, economic and ecological systems. Because of this it is important that such problems be amenable to mathematical analysis in the same way as nonsingular problems are investigated.

However, since Pontryagin's Principle does not

yield any information directly on singular controls the first task for researchers was to discover new necessary conditions for optimality in the singular case. This chapter describes some of the methods used to derive the two necessary conditions known as the generalized Legendre-Clebsch and Jacobson conditions. One approach for obtaining the former condition was based on a transformation of the singular problem to a nonsingular one (Kelley, 1964b).

This transforma-

tion initially was applicable to problems with scalar control.

It allows analysis of singular problems in a

reduced state space but has the disadvantage of requiring the solution of a system of nonlinear differential equations in closed form. The generalized Legendre-Clebsch condition for scalar control was also obtained by using special control variations (Kelley, 1964a; Kopp and Moyer, 1965; Robbins, 1965). 61

This

62

SINGULAR OPTIMAL CONTROL PROBLEMS

method is described in Section 3.2.1 of the present chapter.

The same condition but for vector control

has been deduced by using the transformations of Goh (1966a, 1966b) and Kelley (Speyer and Jacobson, 1971). Section 3 . 2 . 2

of this chapter describes the transforma-

tion of Goh. By using a particularly simple special control variation Jacobson deduced h i s necessary condition.

In Section 3.3 Jacobson's condition is derived and shown to be a different necessary condition to the

generalized Legendre-Clebsch; together they are insufficient for optimality. Again a number of examples are presented in this chapter both to illustrate the theory and t o draw attention to certain practical problems which give rise to singular control.

Much of the work presented

in this chapter plays an important part in later chapters where necessary and sufficient conditions are discussed. By using the necessary and sufficient conditions for the non-negativity of the second variation the two necessary conditions mentioned above are again derived in Chapter 4 for the totally singular case. 3.2 3.2.1

The Generalized Legendre-Clebsch Condition Special Control Variations Two different methods were used by those research

workers who first derived a new necessary condition for singular optimal control problems with scalar

3.

control.

NECESSARY CONDITIONS

63

This condition has since become known as the

generalized Legendre-Clebsch condition due to its similarity in form to the classical conditions of Legendre and Clebsch.

The two methods were due to

Kelley (1964b) using a transformation to a state space of reduced dimension and to Robbins (1965), Kelley

(1964a), Kopp and Moyer (1965) in which special control variations were employed.

In the present section we

give the derivation due to Kopp and Moyer.

Both

methods are discussed in detail by Kelley et al. (1967). The method adopted to find new necessary conditions in the singular case is to generate special control variations and then to use the condition that the associated second variation must be non-negative. It is a technique that has been used to good effect

in the classical literature on the nonsingular problem. The special control variations s o generated must of course give rise to state variations which satisfy both the equations of variation (2.3.6) and the end conditions (2.3.7-8).

In the following derivation of the generalized Legendre-Clebsch condition a sequence of special control variations will be constructed which in turn will generate a sequence of necessary conditions.

Should

the first condition of this sequence be trivially satisfied then the second condition is tested and so on until new information is obtained.

SINGULAR OPTIMAL CONTROL PROBLEMS

64

Theorem 3.1.

A necessary condition for the scalar

control u in the control problem ( 2 . 5 . 1 - 3 )

to be

optimal is that

for all t in [to,tfl. A special scalar control variation, denoted by

Proof:

@ o l ( ~ , ~ ) is

shown in Fig.1.

F i g . 1.

Control variation

$

0

( T , E)

The centre of this double pulse variation is at any point t2 in (to,tf). T =

t

-

Introduce the fictitious time

t2 so that the variation is symmetrical about

the origin

T =

0 as shown.

The interval of the double

pulse is 2~ and the duration of each pulse is

E’.

Successive integrations of +ol with respect to t are designated @ v l ( ~ , ~ ) ,

65

3. NECESSARY CONDITIONS

With the control variation u of variation ( 2 . 4 . 1 2 )

i =

AX

+

=

$0

the equations

’(T,E)

become

,

x(to)

B+~~(T,E>

=

o

(3.2.1)

The end conditions at tf ( 2 . 4 . 1 3 ) will be ignored for the time being. Assuming the necessary smoothness properties of the coefficients of ( 3 . 2 . 1 ) it can be demonstrated by direct substitution that a solution of (3.2.1)

is given by x = B$l’ + A2$z1 + O 1

where

(3.2.2)

(3.2.3)

A2zAB-i

With this expression for x substituted into the second variation ( 2 . 5 . 5 ) we obtain

J[(*)I

E

=

v[Sf, x(tf>I

+

+

+ contribution over

[E,

tf-t21.

A242l + 0 ’ )

(3.2.5)

SINGULAR OPTIMAL CONTROL PROBLEMS

66

Integrating by parts the terms 9 0 ’$1’CB and $01@21CA2 and using the definitions of $1’ and $2’ the second variation may be expanded as a power series in assuming the Taylor series in of convergence.

-2C(AB-i)

dt

has a nonzero interval

Retaining the terms in ($1’)’

are the dominant terms in d J = [ - -(CB)

E

E,

E,

which

we have

T

+ B QB]

E~

+

O(E~)

(3.2.6)

T=O

Here we have used the fact that from equation (3.2.2)

and from equation (3.2.4) el(t)

is or order

E ~ .

It

f) with 5 f = 0, and the contribution to J over the time interval f o l l o w s that y, the quadratic form in x(t

’

are of order E ~ . However, the state fJ variations x of equation (3.2.2) corresponding to the [t2

+E,

t

special control variation 0

0

,

(T E)

will not in general

satisfy the terminal constraints (2.4.13); order

E ~ .

they are of

The question thus arises as to the admissiHowever, since x(t ) f we may alter the special control varia-

bility of the variation $ is of order

E~

0

’(T,E).

tion over the interval [t2 +

E,

tf] by the addition of

an appropriate function of time which need only be of order

E ~ .

This additional functi.on is chosen to ensure

the satisfaction of the terminal constraints (2.4.13).

67

NECESSARY CONDITIONS

3.

The resulting changes in the second variation (3.2.5) are of order c 6 so that the dominant terms of that variation, stated in (3.2.6),

are unchanged.

The

existence of such control correction functions is ensured by our assumption of normality. Remembering that

T =

0 is any interior point on the

singular arc, a necessary condition for that arc to be minimizing is thus, from (3.2.6),

d

-(CB) dt

+ 2C(AB

- i) -

T

-

B QB < 0

(3.2.7)

This is identical to the result of Kelley (1964a) and was recognized by A. E. Bryson, Jr. as being equivalent to the compact form

-a d2H au

dt2

u

< O

-

for all t in [to,tf].

(3.2.8)

This condition is the first of the sequence of conditions which make up the generalized Legendre-Clebsch condition. Should this condition (3.2.8) be trivially satisfied (equality) then no further information with which to test the singular arc has been obtained. case a second scalar control variation 4 structed and is shown in Fig.2.

0

In this

2 ( ~ , ~is)

con-

*

-E lI

-

I-

m

6

2

4

t-

I

I

-

0 e3-

-

-I

F i g . 2.

-

Control variation 9

Successive integrations of 9

i. e.

f-

-

denoted by 4 2 ( r , ~ ) , v

-

0

0

(T

,E )

with respect to time are

dV4v2( T ,€1 = 9,2(T,E).

dTv

Proceeding exactly as in the proof of Theorem 3.1, ignoring terminal constraints and with 4

x = ~

9 +~

1

2A . + . ~+ 1 1 i=2

e2,

one is led to the necessary condition

3.

a d4H > O a u g U-

NECESSARY CONDITIONS

69

for all t in [to,tfl.

(3.2.9)

Details of the derivation of (3.2.9) are given by Kelley et al. (1967).

Again, the satisfaction of terminal con-

straints may be accomplished with control correction functions which do not affect the dominant term of the second variation from which the condition (3.2.9) is obtained. If the condition (3.2.9) is trivially satisfied then a third control variation $ , 3 ( ~ , ~ )is constructed and so on. Details of the general analysis are also given by Kelley et al. (1967).

The general result may

be written as (3.2.10) It can be shown (Kelley et al. 1967) that the control u cannot appear in an odd time derivative H when u is U

a scalar. Example 3.1.

The problem of control of a stirred

reactor as defined by Siebenthal and Aris (1964) is that of determining the heat removal rate u and the associated state variables x1 and x2 denoting extent of reaction and temperature respectively, which satisfy differential equations

70

SINGULAR OPTIMAL CONTROL PROBLEMS

51

= -XI

+ R(x1,xZ)

where a and b are c o n s t a n t s and R r e p r e s e n t s t h e r a t e of r e a c t i o n such t h a t a c e r t a i n performance i n d e x may be extremized.

I n ( S i e b e n t h a l and A r i s , 1964) i t i s t h e

t i m e t a k e n f o r s t e a d y s t a t e t o be achieved from a given

i n i t i a l s t a t e which i s t o b e minimized.

A l l variables

a r e measured from t h e s t e a d y s t a t e . The Hamiltonian f o r t h i s problem i s H = Al(-xl

+ R) + X2(R

-

-

ax2

(x2 + b ) u ) .

(3.2.11)

The a d j o i n t v a r i a b l e s A 1 and A2 a r e governed by t h e equations

i2 = -X1R2 where R1

=

+ X2(a + u

-

R2)

aR/3x1 and R 2 = 3R/3x2.

Since t h e Hamiltonian i s l i n e a r i n t h e c o n t r o l u t h e r e i s t h e p o s s i b i l i t y of optimal s i n g u l a r c o n t r o l when t h e e x p r e s s i o n A2(x2 + b) v a n i s h e s i d e n t i c a l l y over a f i n i t e i n t e r v a l of t i m e .

Furthermore i t is

shown by S i e b e n t h a l and A r i s (1964) t h a t such a

3.

NECESSARY CONDITIONS

71

singular control is possible in a reversible, exothermic reaction provided that

X2 :0, R2

3

0 and XI # 0.

(3.2.12)

We shall now apply the generalized Legendre-Clebsch condition to this singular arc (Bell, 1969). From ( 3 . 2 . 1 1 ) we see that

d H dt u

= A 1 (x2+b)R2

-A2(ab-x~R~-bR2+R)

+ R22{R-ax2 -(x2+b)u)l

where

R21 =

a2R and R22 ax2axl

+ XlRz[R-ax2 -(x2+b)u]

=

a 2R ax22

Using the identities in ( 3 . 2 . 1 2 ) we see that

SINGULAR OPTIMAL CONTROL PROBLEMS

72

Since x2+b # 0 (Siebenthal and Aris, 1964) the generalized Legendre-Clebsch condition (3.2.8) thus yie Ids

along the singular arc.

This is a new result, not

obtainable from Pontryagin's Principle. Example 3.2.

Lawden (1963) has shown that a singular

trajectory in the form of an intermediate-thrust arc satisfies Pontryagin's Principle and is a candidate for a minimum-fuel transfer orbit in space navigation when the final time is unspecified. has become known as Lawden's spiral.

This singular arc The non-optimality

of this singular arc has been demonstrated, for example by Kelley et al. (1967), using the generalized LegendreClebsch condition discussed above. This present example shows that Lawden's spiral is non-optimal by constructing special state variations which satisfy exactly the necessary end conditions on the variations (Bell, 1971). The notation is that used by Lawden (1963).

At

time t the vehicle R of mass M has polar coordinates r,8 on the singular arc. At this instant the direction of the rocket's thrust makes an angle 4 with the perpendicular to the radius vector drawn in the same sense as the motion.

With 0 as pole and OX as initial line

the angle made by the thrust direction and OX is $

(Fig. 3)

3.

.

73

NECESSARY CONDITIONS

Lmden's s p i r a l

Fig. 3.

The vehicle's velocity components in the direction of and perpendicular to the thrust are (u,w).

The

equations of motion of the vehicle resolved in directions along and perpendicular to the thrust are cm +-

G

=

G

= -$u

i-

= usin+

$w

M

r2

sin+

+ -L cos+ r2

-

wcos+

rt, = ucos+ + wsin+

M = -m

(3.2.13)

74

S I N G U L A R OPTIMAL CONTROL PROBLEMS

where c is the constant exhaust velocity of the rocket, m is the propellent mass flow rate and y is the The initial values of t, M, r,

gravitational constant.

0 and the final values of r, 0 are specified while the

initial and final values of u, w and $ are given by the equations u w

0

0

=

i-0 sin+0 + r0.0 0 cos$ 0

=

-r cos@

0'0

' 0

+ r 6 sin$

0

+ +o = 0

O

0

+T

r 2

(3.2.14)

u w

f f

$bf

=

.f f-f r sin@ + r 0 C O S + ~ f

=

*f -r cos$

+$f = 0

f

f*f + r 8 sin$f

f T r + 2 -

Suffices o and f denote as usual initial and final end points respectively. Superscripts o and f denote specified values at t variables, namely

+

0

and tf.

There are two control

and m.

It is required to determine the thrust magnitude and thrust direction along an intermediate-thrust arc which transfers the vehicle between the two fixed terminals with mininurn consumption of fuel.

A suitable

cost function which is to be minimized is given by

3.

75

NECESSARY CONDITIONS

J

= c

ln(Mo/Mf).

It is clear from equations (3.2.13) that the Hamiltonian for this problem is linear in the mass flow rate m. This control variable m is singular.

Results from the

vanishing of the first variation J1 are (Lawden, 1963)

Y sin+ = 0 wI)* + 7 r

w + rI)sin+ = A

.. $

=

3Y sin4 -3

r

COS~

where A is a constant of integration. The equations of variation along the singular extremal are (2.3.6)

nu

= $qw

riw =

rlu nW

=

+ wq $- -(d-nd t cM M + 2Y3 sin+ nr -

-$nu - un+

-

2Y r3

7

‘OS+

‘r

-we4 + sin+ nr + 8cos+

= UB

0

-

cos+

-

- 1sin+ B r2

n,

+

rcos+

: r + ;sin4 nr + rsin+ ;I8

+ (3.2.15) 8

SINGULAR OPTIMAL CONTROL PROBLEMS

76

II

$

+

B

0

=

nO.

The e q u a t i o n s of v a r i a t i o n on t h e e x t r e m a l arc of t h e end c o n d i t i o n s are ( 2 . 3 . 7 - 8 )

E0

= n,(O)

= nr(0> =

17 ( 0 ) =

-w B (01,

U

=

p

p

= 0,

llW(O) = u B (0)

0 4

0 4

n “(tf)

n

= -

B p

(3.2.16)

n p f )

-GfEf,

IIU(tf) =

-Cfsf -

IIW(tf) =

-GfSf

wf(+f5f

+

=

-6 f 5 f

B4(tf))

+ Uf(ifSf + B 4 ( t f ) ) .

It can be shown ( B e l l , 1971) t h a t t h e second v a r i a t i o n

reduces t o

tf 5 f 2

-

3Y sin+ ne2) r2

dt.

(3.2.17)

3. NECESSARY CONDITIONS

77

We now generate a special set of state variations satisfying the end conditions (3.2.16), to show “r, “8 that the second variation J2 of (3.2.17) can assume a negative value, thus violating the necessary condition that it should be positive for all admissible variations.

Choose

..

‘r

= plG

..

Q9

where G(t)

= q,G

+ p2k

-

PG (3.2.18)

+ q2k

-

ifG

is an arbitrary function of time, except for

certain end conditions specified below, and the functions pl(t), p2(t), q1(t), q2(t) are also arbitrary but finite functions of time subject only to the continuity

vr, n g ,

conditions on the variations

OU, “w

and

rl

M’ The end conditions (3.2.16) on the chosen varia-

tions Q r’ ~~(3.2.18)are satisfied provided function G(t) satisfies the boundary conditions G(o)

= 0,

6(0) =

0,

G(0) = 0

Differentiating equations (3.2.18) with respect to time, eubstituting into equations (3.2.15) employing equations (3.2.13)

3-4

and 3-4 yields expressions for

78

SINGULAR OPTIMAL CONTROL PROBLEMS

the variations n

U

nW'

and

It turns out that these two

latter variations satisfy end conditions ( 3 . 2 . 1 6 ) provided the function G(t)

satisfies the further

boundary conditions

Equation (3.2.15) 2 then yields an expression for fB 4 as a function of G and its derivatives up to and including G ( iV>

.

Now we choose the functions pl(t),

e(t) s o that ,(iv) , G(iii)

p2(t),

ql(t),

the coefficients of the derivatives and G vanish in the expression for

It can easily be deduced that this implies f(@+-qo). the following equations:

pzcos+ 2$(p2sin+

+

-

q2rsin+

q2rcos$)

=-

2fG

(3.2.19)

cos+ q1

d 2r$ ql) dt cos$

= -(-

+

w

6Y -7 sin4 q,. r

From equations (3.2.19)

the functions p1, p2 and q2 can be found in terms of ql. It is clear that the function ql(t) may be chosen arbitrarily provided it is

non-zero.

With these choices for the functions p l y p2

and q2 the variation

rl

dJ

is a function only of G(t)

and

3. NECESSARY CONDITIONS

its first derivative k(t). given by (3.2.17),

79

The second variation

after integration by parts and using

end conditions on k(t),

may then be written as

0

+ terms in

h2

and G2)dt. (3.2.20)

Now choose G(t)

where

E

such that

is an arbitrarily small parameter.

within the interval [t,, t l + derivative G(iv)(t>

Fig.4.

E]

Furthermore,

choose the fourth

to be of the form shown in Fig.4.

The derivative G ( i v ) ( t ) ,

t, < t < tl +

E.

S l N G U L A R OPTIMAL CONTROL PROBLEMS

80

The function G(t) and its first three derivatives are continuous. It is easy to verify that G(t) satisfies all the required boundary conditions and furthermore, G(t)

-

0(c8),

In particular, 5 ,

6(t) -

-

O(E~),

O(E~).

G(t)

-

Oh4),

The sign of the second

variation is thus determined by the term in G2 in (3.2.20); the dominant term, when

E

is sufficiently

small, being 3y sin+

( 3-5sin2$) q 1

G2 dt ,

(3.2.21)

Lawden (1963) has shown that the control variable

+

must satisfy the inequalities

Therefore, the term in (3.2.21) is negative and the second variation 5 2 given by (3.2.20) can be made negative by choosing

E

sufficiently small.

This violates

the necessary condition for optimal control that the second variation must be non-negative for a minimum value of J.

Lawden's spiral cannot therefore form part

3.

NECESSARY CONDITIONS

81

of a minimunrfuel trajectory. Note that this result has been obtained with special variations which satisfy exactly the initial and terminal conditions (3.2.16). 3.2.2

A Transformation Approach

The generalized Legendre-Clebsch condition was derived n an alternative, indirect way by first transforming the singular problem into a nonsingular one and then applying the classical Legendre-Clebsch necessary condition (Kelley, 1964b; Kelley et al. 1967; Goh, 1966a, 1966b).

This condition for the transformed

problem is the generalized Legendre-Clebsch condition for the original singular problem.

There have been two

different transformations given in the literature. Kelley's transformation (Kelley, 1964b; Kelley et al., 1967; Speyer and Jacobson, 1971) reduces the dimension of the state space to n-p, where p is the order of the

singular arc. The other transformation is due to Goh (1966a, 1966b) and is the one discussed in this section.

It retains the full dimensionality of the original problem and is simpler in the case of vector controls (see also Chapter 6, Section 6.3). A s in Section 3.2.1 we start from the second

variation for the singular arc J2 =

jtf{ixTQx + uTCxldt +

IXT (tf)Qf

x(tf)

t0

subject to the equations of variation

(3.2.22)

SINGULAR OPTIMAL CONTROL PROBLEMS

82

L

=

Ax + Bu,

x(to)

=

0

(3.2.23)

and terminal conditions Dx(tf) = 0.

(3.2.24)

The basic idea behind Goh's transformation is to update the control vector to the status of a derivative of a new vector followed by the elimination of this derivative from both the second variation and equations of variation.

A further updating of the new vector to

a derivative may then yield a non-singular problem to which the classical Legendre-Clebsch condition may be applied.

If this is not the case, then the procedure

is repeated. Define a vector v such that v = u

(3.2.25)

and, without l o s s of generality, set V(t0)

=

0.

(3.2.26)

Furthermore, make the transformation from x to a new variable z: z=x-Bv in order to eliminate (3.2.23).

i7

from the equations of variation

These equations, together with the terminal

83

NECESSARY CONDITIONS

3.

conditions (3.2.24), become

5

=

-

Az + (AB

D[z(tf)

i)v,

z(to)

+ Bv(tf)]

= 0.

(3.2.27)

= 0

(3.2.28)

The second variation becomes J2[v(*)]

=

T T T T T jtf{4z Qz + v B Qz + iv B QBv t0

+ GTC(z

+ Bv))dt

(3.2.29) .T Integrate by parts the term v Cz by parts. J,[v(.)]

T

T T

= Itf{iz Qz + v (B

Q - CA -

This gives

c)z

t0

+ ivT [BTQB

-

2C(AB

-

i)]v

+ GCBvJdt

rn

rn

3.2. 1)

But it can be shown that T B QB

-

2C(AB

-

f3)

-

d x(CB) =

a;; -au

u

SINGULAR OPTIMAL CONTROL PROBLEMS

84

where, because of t h e i d e n t i t i e s ( 2 . 4 . 1 0 ) , T

H(x,u,A,t)

T

-

A = H

T

+ u Cx + X ( A x + Bu),

= { x Qx

X'

T h e r e f o r e , t h e second v a r i a t i o n g i v e n by (3.2.30)

can

be e x p r e s s e d as J 2 [ v ( * ) ] = I t f { $ z T Q z + v T (BT Q

-

CA

- t)z

t0

+

T

IV

[-

a H.. au

u

T d + GCBV + {V x ( C B ) v } d t

]V

F i n a l l y , i n t e g r a t i n g t h e l a s t t e r m i n t h e i n t e g r a n d by parts, J2[v(*)]=

[t f

(42

T

T T Qz + v (B Q

-

CA

-

?)z

t0

+

T

IV

[-

a H.. au

u

]V +

;4

T

[CB

-

T (CB) ] v ) d t

3.

NECESSARY CONDITIONS

85

We pause at this point to deduce condition ( 1 . 5 . 3 ) . Using an appropriate choice of control v(s) we can make the term in ir dominate the other second-order quantiT ties in ( 3 . 2 . 3 2 ) . Since CB - (CB) is skew-symmetric this means we can make J, either positive or negative. This violates the condition that J2 must be of one sign for a minimizing or maximizing arc.

We are thus led to

the necessary condition that CB

=

(CB)T

=

for all t in [to,tfl. ( 3 . 2 . 3 3 )

B C

This is a very powerful condition in the case of vector controls.

It is easily verified (see proof of Lemma

4 . 2 in Chapter 4 ) that

CB

-

BTCT

=

aua .HU.

Thus, ( 3 . 2 . 3 3 ) corresponds to ( 1 . 5 . 3 ) with p

=

If

1.

u is a scalar then ( 3 . 2 . 3 3 ) is satisfied trivially.

Returning to equation ( 3 . 2 . 3 2 ) the term in vanishes by ( 3 . 2 . 3 3 ) .

The derivative

now

has thus been

eliminated completely from the second variation and from the equations of variation.

We may therefore

update the control v to the status of a derivative, say

G.

The second variation and differential con-

straints of the accessory minimum problem may now be written in the form

86

SINGULAR OPTIMAL CONTROL PROBLEMS

q 1 ,

+ YCfi(tf>¶

+rl +

en

(3.2.35)

= 0

where > 0

0

R =

¶

0

- -3 ; ; au

u

0

0

(B~Q-cA-~)

0

L =

J

M =

(3.2.34)

3.

87

NECESSARY CONDITIONS

Applying the classical Clebsch condition (2.3.21-22)

to the transformed accessory minimum

problem ( 3 . 2 . 3 4 - 3 5 )

we obtain the following result:

Along a minimizing arc T

-

(3.2.36)

I T R T > O

for all (n+m)-vectors $TI

satisfying

IT

(3.2.37)

= 0.

...)

IT to be arbitrary, condition Taking ITn+l ' TIn+2 ' n+m ( 3 . 2 . 3 6 ) subjected to ( 3 . 2 . 3 7 ) implies that

- - aH > O au u -

(3.2.38)

for all t in [to,tf]

which is ( 3 . 2 . 1 0 ) with q

=

1.

If ( 3 . 2 . 3 8 )

is met with

equality for all t in [tost 1 then the functional f J2[v(.)] is totally singular and another transformation must be made.

In this way the generalized necessary

conditions ( 1 . 5 . 3 - 4 )

are obtained.

It should be noted that the derivative $(tf) occurs in the terminal quadratic form y of ( 3 . 2 . 3 4 ) and also in the terminal constraint ( 3 . 2 . 2 8 ) with v

=

&.

However, the class of admissible vectors w satisfying the differential constraints ( 3 . 2 . 3 5 )

is not reduced by

restrictions placed upon the end values 6(tf).

This is

because any such restrictions can be met by infinitesimal adjustments over a sufficiently small neighbourhood

SINGULAR OPTIMAL CONTROL PROBLEMS

88 [tf-E,

t ] and these adjustments will only affect the

f integral in J 2 infinitesimally. Consequently, the

elements of the vector 6(t f) can be treated just as one would treat the parameter 5 f (Goh, 1966a, 196613; Speyer and Jacobson, 1971; McDanell and Powers, 1970). Example 3 . 3

The problem of control of a particular

aircraft model as defined by Schultz and Zagalsky (1972) is that of determining the minimum fuel required for a climb-cruise-descent trajectory between specified values of altitude, range and velocity.

The control

variables are thrust and flight path angle and the terminal time is not specified although the final result is valid also for fixed time.

The system is

governed by the differential equations and end conditions

i

=

(T-D)v/M,

i = vy, x =

E(to)

= Eo,

h(t )

=

0

v,

ho, 0

x(to) = x

y

f

E(tf)

=

E

h(tf)

= h

X(tf)

=

f

.

f x

Superscripts o and f denote specified values at t0 and t

f

respectively.

E is the specific energy defined as

E = V2/2

+

gh,

3.

NECESSARY CONDITIONS

89

h i s t h e a l t i t u d e , V t h e v e l o c i t y , x t h e range, D(E,h) t h e aerodynamic d r a g and M t h e mass of t h e a i r c r a f t . The mass M w i l l be assumed c o n s t a n t although t h e conc l u s i o n s remain t h e same when t h i s r e s t r i c t i o n i s lifted.

The c o n t r o l v a r i a b l e s , t h r u s t T and f l i g h t

p a t h a n g l e y, are bounded as

It i s r e q u i r e d t o minimize t h e f u e l J = jtfa(E,h)T d t to

used d u r i n g t h e manoeuvre, where a(E,h) i s t h e f u e l

rate per u n i t thrust. The Hamiltonian f o r t h i s problem is H = UT

+ Al(T-D)V/M +

A2Vy + X3V.

The a d j o i n t v a r i a b l e s A 1 , A2 and A3 a r e governed by t h e equ a t i o n s i1 = - a H / a E ,

i2 = -aH/ah

and

S i n c e t h e Hamiltonian i s autonomous and t

A 3 = -aH/ax.

is f u n s p e c i f i e d a f i r s t i n t e g r a l of t h e motion e x i s t s and

is

90

SINGULAR OPTIMAL CONTROL PROBLEMS H = O

for all t in

[to,t,l.

The H a m i l t o n i a n i s a l s o l i n e a r i n t h e c o n t r o l v a r i a b l e s s o t h a t t h e r e i s t h e p o s s i b i l i t y of o p t i m a l s i n g u l a r c o n t r o l when t h e e x p r e s s i o n s

and

x2v

away =

v a n i s h i d e n t i c a l l y o v e r a f i n i t e i n t e r v a l of t i m e . This implies t h a t

A2 = 0 .

It i s shown by S c h u l t z and Zagalsky (1972) t h a t when

t h e c o n t r o l s assume v a l u e s which s u s t a i n t h e c r u i s e p o r t i o n of t h e t r a j e c t o r y T = D,

y = 0.

Since t h e Hamiltonian i s zero, d(aH/aT)/dt

=

a(oD/V)/aE = 0

3.

NECESSARY CONDITIONS

91

which imply that the cruise is flown at the energy and altitude which satisfy min(oD/V).

The cruise condition

is thus a doubly singular arc which satisfies the first order necessary conditions (Schultz and Zagalsky, 1972). We shall now apply the generalized Legendre-Clebsch condition to this singular arc (Speyer, 1973). T For the control vector u = (T y), eqn(3.2.33) equivalently eqn(1.5.3))

(or

is not satisfied since

and this is not the null matrix because the term (aa/ah + ag/V2)V in general is not zero.

This result

remains true if t is specified. The cruise condition f will not therefore yield a minimizing singular arc. This result is contrary to the findings of Schultz and Zagalsky (1972). 3.3

Jacobson's Necessary Condition A further necessary condition for non-negativity

of J2[u(.)]

was discovered by Jacobson (1969, 1970b),

different from the generalized Legendre-Clebsch condition.

This new condition is stated and proved below

for the free-endpoint problem, that is the matrix D in (3.2.24)

is the null matrix.

The appropriate condition

SINGULAR OPTIMAL CONTROL PROBLEMS

92

for the constrained-endpoint problem is derived by Jacobson (1969).

A slightly stronger version of

Jacobson's condition for the free-endpoint problem was obtained by Gabasov (1968, 1969) although the conditions are identical for quadratic problems.

A necessary condition for J2[u(.)]

Theorem 3 . 2 .

to be

non-negative for all u(*) belonging to U, for the case where D = 0, is that

T

CB + B WE5 > 0 where

Proof.

-fi

+ ATw

Q

=

for all t in [to,tfl

+ WA;

W(tf> =

Q,.

(3.3.1)

(3.3.2)

Adjoin the equations of variation ( 3 . 2 . 2 3 )

to

the expression J, of ( 3 . 2 . 2 2 ) by an adjoint vector of T the form JW x, where W(-) is an arbitrary n x n symmetric, continuously differentiable matrix function of time.

J~[u(.)]

Then =

T T T jtf{4x Qx + u Cx + ix W(Ax+Bu-k)ldt

t0 (3.3.3)

Integrate by parts the term in 2 to give J2[u(*)]

=

1

tf

T {4xT(fi + Q + A W + WA>x

t0

T T + u (C + B W)x)dt

T

+ 4x (t,>[Qf

-

W(t,)lx(t,)

(3.3.4)

3.

93

NECESSARY CONDITIONS

Now choose W to satisfy equations (3.3.2). Then J,[U(-)]

=

jLfuT(C + BTW>x dt.

(3.3.5)

0

Choose a special variation u which is zero except strictly contained in during an interval [t,,t,+A],

1, when it assumes a constant but arbitrary f magnitude K. Then [to,t

J, = It'+'KT(C

+ BTW)x dt

(3.3.6)

tl

and the dominant term in the expansion of this integral for sufficiently small A is

.

AKT (C + BTW)BKA2It

(3.3.7)

1

A necessary condition for J2 to be non-negative is

therefore the condition (3.3.1) since tl can be chosen as any point in the interval [t ,t 1. (3.3.1) and o f (3.3.2) together form Jacobson's necessary condition. The following example illustrates the nonequivalence of Jacobson's condition and the generalized Legendre-Clebsch condition.

Furthermore, it demon-

strates that these two necessary conditions are in general insufficient for optimality.

SINGULAR OPTIMAL CONTROL PROBLEMS

94

Example 3 . 4 (Jacobson, 1970a).

Minimize

(-x12 + x 2)dt

(3.3.8)

2

subject to

(3.3.9)

;r2=u

,

x2(o) = 1.

The Hamiltonian is given by

H

4(-xl2

=

+

x2

2)

+

x1x2 +

x2u.

The adjoint equations for this problem are:

It i s easy to verify that

-

u(t)

=

-

-sint ( 3 . 3 . lo)

x,(t) = -sint

-

x,(t)

=

cos t

is a singular solution satisfying Pontryagin’s prin-

ciple.

For this solution, the magnitude of J is zero.

Here, C = H

ux

= (0 0),

B = f

U

=

T

(0 1)

95

3. NECESSARY CONDITIONS

and it follows that T

(C + B W)B

=

w22

where

w

=

(w. . ) ,

i,j = 1,2.

1J

Using (3.3.2) it is easily verified that w22(T)

= T

-

T1 T

3

where T =

3 ~ / 2- t,

i.e., reverse time.

Hence, w22 is negative for

so that (3.3.1) ceases to be satisfied.

T

> 43

The singular

solution is then nonoptimal. Also, for this problem HU

..

fiu

= 12,

HU=-

U -

= -x2

-

A]

x1

so that the generalized Legendre-Clebsch condition

(3.2.38) is satisfied for all

T.

If (3.2.38) and (3.3.1) were together sufficient for optimafity then the singular solution for 0 < T <

43 would be optimal.

That this is not the case

can be demonstrated by transforming the original problem to a nonsingular one in the following way.

SINGIJLAR OPTIMAL CONTROL PROBLEMS

96

Write z so

x1 and v = x2

=

that the original problem becomes:

-

Minimize J

4

=

1

3~12 (-z2

+ v2)dt

(3.3.11)

0

subject to z = v,

in which variable.

z

z(0)

=

(3.3.12)

0

is now a state variable and v a control The problem is now nonsingular.

Clearly, v(t)

=

cos t

-

satisfies Pontryagin’s principle and J control. ‘Hvv

(3.3.13) =

0 for this

The strengthened Legendre-Clebsch condition

> 0 ) and the boundedness of the solution of the

associated Riccati differential equation can be used to determine whether or not the control (3.3.13) is optimal since the problem is now nonsingufar. We have Hvv

= 1> 0

and the Riccati equation is

- -ds -d.r

1 +

s2,

s = S(T),

s ( 0 ) = 0.

The solution of this differential equation is S(T)

=

-tan

T

3.

NECESSARY CONDITIONS

which becomes unbounded at

T =

97

~ / 2 . Therefore, the

control (3.3.13) ceases to be optimal for

T > ~ / 2 .

This means that the singular solution (3.3.10) ceases to be optimal for

T >

n/2, since the two problems

(3.3.8-9) and (3.3.11-12)

are equivalent.

But we have

already shown from Jacobson's condition that (3.3.10) is nonoptimal for

T

> 43.

Thus, we have the situation

whereby the two necessary conditions are both satisfied for -m/2 <

T <

43 but the singular solution is nonoptimal.

This shows that the satisfaction of both the generalized Legendre-Clebsch condition and the Jacobson condition is not sufficient for optimality in singular control problems. In the derivations of the generalized LegendreClebsch condition it has been assumed that the problem is normal (Bliss, 1946).

That is to say sufficient

control variations exist to modify the special control variations in order to satisfy terminal constraints. Even in the absence of terminal constraints the assumption of normality is still necessary to insure the correct form of the test.

In the proof of the

Pontryagin Maximum Principle (Pontryagin et al., 1962) no assumption of normality is necessary since it is taken care of by the transversality condition on the adjoint variable and the additivity of first order variations.

An extension of the Pontryagin Maximum

Principle has been obtained by Krener (1973) and is called the High Order Maximal Principle.

This extended

98

SINGULAR OPTIMAL CONTROL PROBLEMS

3rinciple includes not only the generalized LegendreClebsch and Jacobson conditions when they apply but also a number of other conditions which can be constructed for a specific problem without the assumption of normality. References Bell, D. J. (1969). Singular Extremals in the Control of a Stirred Reactor, Chem. Engng. Sci. 24, 521-525. Bell, D. J. (1971). The Non-Optimality of Lawden's 16, 317-324. Spiral, Astronautica Acta Bliss, G. A. (1946). "Lectures on the Calculus of Variations". Univ. Chicago Press, Chicago. Gabasov, R. (1968). Necessary Conditions for Optimality of Singular Control, Engng. Cybern. No.5, 28-37. Gabasov, R. (1969). On the Theory of Necessary Optimality Conditions Governing Special Controls, Sov. Phys.-Dokl. 13, 1094-1095. Goh, B. S. (1966a). The Second Variation for the Singular Bolza Problem, SIAM J. Controli, 309-325. Goh, B. S. (1966b). Necessary Conditions for Singular Extremals Involving Multiple Control Variables, SIAM J. Control 4, 716-731. Jacobson, D. H. (1969). A New Necessary Condition of Optimality for Singular Control Problems, SIAM J. Control 7, 578-595. Jacobson, D. H. (1970a). On Conditions of Optimality for Singular Control Problems, IEEE Trans. autom. Control AC-15, - 109-110. Jacobson, D. H. (1970b). Sufficient Conditions for Nonnegativity of the Second Variation in Singular and Nonsingular Control Problems, SIAM J. Control 8, 403-423. -

3.

NECESSARY CONDITIONS

99

Kelley, H. J. (1964a). A Second Variation Test for Singular Extremals, AIAA J. 2, 1380-1382. Kelley, H. J. (1964b). A Transformation Approach to Singular Subarcs in Optimal Trajectory and Control 234-240. Problems, SIAM J. Control;, Kelley, H. J., Kopp, R. E. and Moyer, H. G. (1967). Singular Extremals, in "Topics in Optimization" (G. Leitmann, ed) pp 63-101. Academic Press, New York. Kopp, R. E. and Moyer, H. G. (1965). Necessary Conditions for Singular Extremals, AIAA J. 3, 1439-1444. Krener, A. J. (1973). The High Order Maximal Principle, in "Geometric Methods in Systems Theory" (D. Q. Mayne and R. W. Brockett, eds) pp 174-184. NATO Advanced Studies Institute Series: Mathematics and Physics. Lawden, D. F. (1963). "Optimal Trajectories for Space Navigation", Butterworth, Washington, D.C. McDanell, J. P. and Powers, W. F. (1970). New JacobiType Necessary and Sufficient Conditions for 8, Singular Optimization Problems, AIAA J. 1416-1420. Pontryagin, L. S., Boltyanskii, V. G., Gamkrelidze, R. V. and Mishchenko, E. F. (1962). "The Mathematical Theory of Optimal Processes". Interscience, John Wiley & Sons, Inc., New York, London and Sydney. Robbins, H. M. (1965). Optimality of Intermediate3, Thrust Arcs of Rocket Trajectories, AIAA J. 1094- 1098. Schultz, R. L. and Zagalsky, N. R. (1972). Aircraft Performance Optimization, J. Aircraft 9, 108-114. Studies in Siebenthal, C. D. and Aris, R. (1964). Optimisation - VI. The Application of Pontryagin's Methods to the Control of a Stirred Reactor, Chem. Engng. Sci. 19, 729-746.

-

I00

SINGULAR OPTIMAL CONTROL PROBLEMS

Speyer, J. L. (1973). On the Fuel Optimality of Cruise, J. Aircraft 10, 763-765. Speyer, J. L. and Jacobson, D. H. (1971). Necessary and Sufficient Conditions for Optimality for Singular Control Problems; A Transformation Approach, J. math. Analysis Applic. 33, 163-187.

-

CHAPTER 4 Sufficient Conditions and Necessary and Sufficient Conditions for Non-Negativity of Nonsingular and Singular Second Variations 4.1

Introduction A s indicated in Chapter 2, non-negativity of the

second variation is a (second-order) necessary condition for optimality in control problems.

It is known,

also, that a sufficient condition for a weak local minimum is that the second variation be strongly positive (Gelfand and Fomin, 1 9 6 3 ) .

Unfortunately, it turns

out that in singular control problems the second variation cannot be strongly positive (Tait, 1965; Johansen, 1966) and so different tests for sufficiency have to be

devised.

Despite this fact, the second variation con-

tinues to play an important role in second-order optimality tests and it is for this reason that this chapter explores the second variation fully.

In the case of nonsingular optimal control problems, where Huu > 0 for all t in [to,tf],

it is

known that a sufficient condition for strong positivity

of the second variation, and hence for a weak local minimum, is that the matrix Riccati differential equation associated with the second variation should have a solution for all t in [to,tf].

Clearly, this

condition is inapplicable in the singular case owing -1

to the presence of Huu in the matrix Riccati equation. For a long time, therefore, it was felt that no Riccatilike condition existed for the singular case but this 101

102

SINGULAR OPTIMAL CONTROL PROBLEMS

has turned out to be not true (Goh, 1970; Jacobson, 1970a, 1971a, 1971b; Jacobson and Speyer, 1971; McDanell and Powers, 1970; Speyer and Jacobson, 1971). The result is that sufficiency conditions for nonnegativity of singular and non-singular second variations are rather closely related. We begin by proving that a necessary and sufficient condition for strong positivity of the nonsingular second variation is that a solution exists to the well known matrix Riccati differential equation.

The

sufficiency part of this theorem is very well known and documented (Brockett, 1970) but though the necessity part is well known it is not, in our opinion, proved convincingly elsewhere except in certain special cases. Following this result, we then illustrate that the singular second variation cannot be strongly positive and confirm that the standard Riccati equation is of no direct u s e .

Next, we provide a sufficient condition

(Jacobson, 1971a) for the partially singular second variation which is in the form of a set of differential and algebraic inequalities. We show how, in the nonsingular case, the standard Riccati equation is related to these conditions, and thereby implicitly establish also the necessity of these conditions.

In the totally

singular case we show that a new Riccati differential equation is related to the sufficiency conditions. We then turn our attention to necessary and sufficient conditions for non-negativity of the totally singular

4. SUFFICIENT CONDITIONS

103

second variation and produce again a set of differential and algebraic inequalities. These conditions are slightly more abstract in form than the ones presented earlier and, because they are necessary as well as sufficient, all known necessary conditions can be deduced from these.

Recent extensions of the condi-

tions (Anderson, 1973) are mentioned.

Finally, the

role of the conditions in deducing sufficient conditions f o r a weak local minimum in singular control problems and in deducing existence conditions for a solution to the standard matrix Riccati equation, is investigated. 4.2

Preliminaries The form of the second variation for an uncon-

+

strained optimal control problem is, from Chapter 2 ,

&

=

Ax + Bu; x(to)

= 0

+ We use here the notation J[u(*)]

variation. t ion.

(4.2.2)

for the second This is done for simplicity of presenta-

SINGULAR OPTIMAL CONTROL PROBLEMS

I04

n where x is in R , u is in Rm, Q,C,R,A,B are continuous matrix functions of time of appropriate dimensions, and where Q

f

is in Rnxn and is constant.

Without loss of

generality, it is assumed that Q, R and Qf are

.

symmetric

We now make three explicit assumptions; one on the class of allowed controls and the others on the nature of R and of (4.2.2). Assumption 4.1

The class of controls U from which u(-)

is drawn is the class of piecewise continuous m-vector functions of time on [to,tfl. Though a less restrictive class of controls could be assumed, the class of piecewise continuous functions

is adequate for the problems treated in this book. Assumption 4.2

A basic assumption that we make is that

In fact this assumption is not at all restrictive since if R were not positive semi-definite the criterion (4.2.1) would, regardless of any other conditions, be able to take on arbitrarily large, negative, values. Assumption 4.2 is the Legendre-Clebsch necessary condition for oprimality in the calculus of variations (Gelfand and Fomin, 1963). Assumption 4.3

We assume that the dynamic system

(4.2.2) is completely controllable on the interval

4. SUFFICIENT CONDITIONS

where t0 < t' < tf.

[to,t'],

jt",a)13(o)BT(o)@T(t'

105

That is,

(4.2.4)

,o)do > 0

where

a

E@(t,a)

= A(t)@(t,a)

; @(a,a) = I

(4.2.5)

for all t' in (to,tfl. Note that this assumption is used in only some of the results derived in this chapter. As the main objective of this chapter is to develop conditions under which ( 4 . 2 . 1 )

is non-negative,

positive definite, and strongly positive, we now define these terms precisely. Definition 4.1 J[u(-)]

is said to be non-negative if

for each u(*) in U its value is non-negative. Definition 4 . 2 J[u(*)] is said to be positive definite if for each u(*) in U, u(*) # 8 (null function), J[u(*)I

> 0.

Definition 4 . 3

JLu(*)] is said to be strongly positive

if for each u(-) in U, and some k > 0,

where 11u(.)

II

is some suitable norm defined on U.

I06

4.3

SINGULAR OPTIMAL CONTROL PROBLEMS

The N o n s i n g u l a r Case

D e f i n i t i o n 4.4

We r e f e r t o ( 4 . 2 . 1 )

R(t) > 0,

as n o n s i n g u l a r i f

for all t in [to,t

f

1.

(4.3.1)

I n t h i s case w e have t h e f o l l o w i n g theorem.

Theorem 4 . 1

A n e c e s s a r y and s u f f i c i e n t c o n d i t i o n f o r

J [ u ( . ) ] t o b e s t r o n g l y p o s i t i v e i s t h a t t h e r e exists for a l l t i n [t

t ]a f u n c t i o n S ( * )

which s a t i s f i e s

f t h e Riccati e q u a t i o n 0'

T T T -1 T - ~ = Q + s A + A S-(C+B s> R (C+B s)

(4.3.2) (4.3.3)

S ( t f ) = Qf.

I n o r d e r t o p r o v e t h i s theorem w e r e q u i r e t h e f o l l o w i n g lemma. Lemma 4 . 1

Suppose t h a t t

0

< t'

2

t f and t h a t ( 4 . 2 . 2 )

i s c o m p l e t e l y c o n t r o l l a b l e on [ t o , t '

1.

Then, there

- -

e x i s t s u * ( t ) , t o < t < t ' , which t r a n s f e r s x ( t o ) = 0 t o any d e s i r e d x a t t = t ' such t h a t t'

I bt'!1

(4.3.4)

= 1

and

I lU*W I I

2

P(t'>

(4.3.5)

where p ( * ) i s a c o n t i n u o u s f u n c t i o n d e f i n e d on ( t o , t ' l .

SUFFICIENT CONDITIONS

4.

Proof:

I07

Define

W(t 0 , t ' ) =

it

' Q ( t ' , t ) B ( t ) B T ( t ) Q T ( t ' ,t ) d t

(4.3.6)

t0

and

It i s a t r i v i a l matter t o v e r i f y t h a t t h e c o n t r o l

f u n c t i o n u * ( - ) t r a n s f e r s t h e s t a t e of system ( 4 . 2 . 2 ) from t h e o r i g i n a t t since O(t'.)

1 Ixtl I I

=

0

and W-l(t

to x 0

a t t=t'.

t'

,t')

Furthermore,

are continuous i n t ' and

1, i t i s clear t h a t t h e r e i s a continuous

f u n c t i o n p ( * ) such t h a t

Proof of Theorem 4 . 1 s a t i s f i e s (4.3.2)

F i r s t w e prove t h a t i f S ( * )

and ( 4 . 3 . 3 )

t h e n J [ u ( * )1 i s p o s i t i v e

definite. Consider t h e i d e n t i c a l l y z e r o q u a n t i t y

(4.3.9)

which, r e - w r i t t e n u s i n g ( 4 . 3 . 2 )

and (4.3.3)

is

108

SINGULAR OPTIMAL CONTROL PROBLEMS

T

T

T

+ 4x (SA+A S)x+x SBuIdt. A d j o i n i n g (4.3.10)

+

J

t o (4.2.1)

(4.3.10)

gives

4 [ u+R%+BTS) x 3 TR [ u+R'-'

( C+BTS) x 1d t

(4.3.11) which t a k e s on i t s minimum v a l u e of z e r o ( b e c a u s e x ( t o ) = 0 ) i f and o n l y i f u ( t ) = -R-'(t)

[ C ( t ) + B T ( t ) S ( t )] x ( t > ,

(4.3.12)

- -

to < t < t f .

That i s , i f and o n l y i f , u(*)

=

8 (null function).

Hence, J [ u ( * ) 1 i s p o s i t i v e d e f i n i t e .

(4.3.13)

109

4. SUFFICIENT CONDITIONS

Now consider

- uTRu)dt

T T J[u(*),E] = jtfIlx Qx+u Cx+ 2-

E

t0 (4.3.14)

This functional is positive definite if 2 - ~ > 0 and i f -i(t,E)=Q+S(t,E)A+A

T

S(t,E)-[C+B

T

S(t,E)

( ~ - E ) R-l 1T2

[C+BTS(teE) 1 S(tf,E)

=

(4.3.15)

Q,

(4.3.16)

has a solution S(t,E) defined for all t in [to' tf 1 Now, since Q,C,R,A,B are continuous in t and the right hand side of (4.3.15) is analytic in S(t,E) and E,

and since S(*,O)exists, we have that

S(*,E)

is a

continuous function of E at E=O (Coddington and Levinson, 1955). So, for E sufficiently small, S(t,E) exists for all t in [to,tf].

Therefore, for E
sufficiently small, J[u(*),E]

is positive definite.

Nerrt, we n o t e that

go

that

SINGULAR O P T I M A L C O N T R O L P R O B L E M S

110

(4.3.18)

and because of ( 4 . 3 . 1 )

we conclude from (4.3.18)

that

so that

where (4.3.21)

and

J

(4.3.22) t0

which i m p l i e s t h a t J[u(*)l i s s t r o n g l y p o s i t i v e . Now w e prove t h e converse; namely, t h a t i f J [ u ( * ) ] i s s t r o n g l y p o s i t i v e , t h e n an S ( * ) e x i s t s which s a t i s f i e s ( 4 . 3 . 2 )

and ( 4 . 3 . 3 ) .

Define

Jt' + j xT ( t , ) Q , x ( t , ) .

(4.3.23)

SUFFICIENT CONDITIONS

4.

111

Then, f o r t ' s u f f i c i e n t l y c l o s e t o t f we have by s t a n d a r d d i f f e r e n t i a l e q u a t i o n t h e o r y (Coddington and

- -

Levinson, 1 9 5 5 ) t h a t S ( t ) e x i s t s , t ' < t < t f . s e q u e n t l y , t h e minimum v a l u e of ( 4 . 3 . 2 3 ) ,

Con-

denoted by

J [ u o ( * ) , x ( t l ) , t ' ] , i s g i v e n by ( s e e ( 4 . 3 . 1 1 )

and

(4.3.12)) J [ u o ( * ), x ( t ' ) , t ' ] =

T

JX

(t')S(t')x(t')

(4.3.24)

where [ C ( t ) + B T ( t ) S ( t ) ] x ( t >,

u o ( t ) = -R-'(t)

(4.3.25)

t ' < t < t

-

f'

Now suppose t h a t S ( t ) c e a s e s t o e x i s t a t t e where < te < t '

By Lemma 4 . 1 , having

2

tf.

(4.3.26)

w e can c o n s t r u c t any d e s i r e d x ( t ' ) ,

I Ix(t') 1 I

-

= 1, u s i n g some c o n t r o l f u n c t i o n

u * ( t > , to < t < t ' which s a t i s f i e s

Now, J[u(*)] =

[t'

[I,x T Qx+u* T CX+I,U* TRu*]dt (4.3.28)

SINGULAR OPTIMAL CONTROL PROBLEMS

112

where

(4.3.29) t' < t

= uO(t)

Note t h a t because of (4.3.27)

tf.

t h e i n t e g r a l on t h e

i s f i n i t e f o r each

r i g h t hand s i d e of (4.3.28)

3 and t h a t by assumption, (4.3.28) i s nonf T n e g a t i v e . However, i f i t were t r u e t h a t x ( t ' ) S ( t ' ) x ( t ' ) t' in (t

0'

f o r some x ( t ' ) then (4.3.28) would e tend t o minus i n f i n i t y a s t ' -F te. Thus we conclude T t h a t x ( t ' ) S ( t ' ) x ( t ' ) cannot tend t o -00. Furthermore,

-+

-00

as t '

t

-+

t

i t i s c l e a r t h a t J [ u o ( * ), x ( t ' ) , t ' J'

, x ( t ' ) , t ' ] where

[;(a)

G(*)

1

i s bounded above by

i s any given piecewise

continuous f u n c t i o n d e f i n e d on [ t ' ,tf 1. Hence T x ( t ' ) S ( t ' ) x ( t ' ) cannot tend t o +co f o r some x ( t ' ) a s S i n c e , by our c o n t r o l l a b i l i t y assumption, e [x(t') = 1, we conclude t h a t x(t') i s arbitrary, t'

+

t ,

I

II

. . (to, tfl

S ( t ' ) cannot tend t o i n f i n i t y as t words, S ( t ) e x i s t s f o r a l l t i n

-f

t

e

In other We l e a v e i t

a s an e x e r c i s e f o r t h e r e a d e r t o v e r i f y t h a t , i n f a c t , S ( t ) e x i s t s f o r a l l t i n [ t , t f ] ( s e e Gelfand and Fomin, 0

1963).

4.4

S t r o n g P o s i t i v i t y and t h e T o t a l l y S i n g u l a r Second Variation I n S e c t i o n 4.3 w e demonstrated t h a t i f t h e m a t r i x

4.

SUFFICIENT CONDITIONS

113

Riccati equation has a solution for all t in [t0 then J[u(-)]

4

is not only positive definite but also

strongly positive.

Clearly, in a finite dimensional

vector space positive definiteness is equivalent to strong positivity, but in our space of piecewise continuous control functions this is not so.

In this

section we illustrate the difference between positive definiteness and strong positivity by means of a simple example.

In addition, the example illustrates the fact,

see Jacobson and Speyer (1971) and Tait (1965) for general proofs, that the totally singular second variation cannot be strongly positive.

This is of course

consistent with the fact that in the totally singular case the matrix Riccati equation is undefined (because rlR-1 = , i t ) . Before presenting the example we define "totally singular" precisely as follows: Definition 4.5

J[u(*)]

is said to be totally singular

if R(t)

=

0

for all t in [to,tfl.

(4.4.1)

Now, we consider the following totally singular functional (4.4.2) ' 0

subject to

SINGULAR OPTIMAL CONTROL PROBLEMS

I14

x(0) = 0

% = u *

u ( - ) i s a member o f U.

(4.4.3) (4.4.4)

Clearly J [ u ( * )1 i s p o s i t i v e d e f i n i t e . Set

u(t) =

COSWt,

O < t < t f

- -

(4.4.5)

-

(4.4.6)

so that

1 x ( t ) = -sin w

at,

O < t < tf '

With t h i s c h o i c e of c o n t r o l J [ u ( * ) ] becomes (4.4.7)

and

By d e f i n i t i o n , if J [ u ( . ) ] were s t r o n g l y p o s i t i v e t h e n

-

f o r some k > 0 and a l l w > 0 ,

w2

fsin2wtdt > k

(4.4.9)

(4.4.10)

4.

115

SUFFICIENT CONDITIONS

But this is impossible because the left hand side of (4.4.10) (4.4.2)

tends to i k t

as w -t to. In other words, f is not strongly positive.

Note, however, that the functional (4.4.11) ' 0

is strongly positive.

This follows directly because

> u2 x2 + u2 -

.

(4.4.12)

The fact that the totally singular second variation cannot be strongly positive implies that in the totally singular case we should seek (necessary and) sufficient conditions only for non-negativity and for positive definiteness of J[u(*)]. We begin this task in the next section. 4.5

A General Sufficiency Theorem for the Second

Variation Here we present and prove a general sufficiency theorem for the partially singular case, which is defined precisely as follows: Definition 4 . 6

J[u(*)

1

is said to be partially

singular if R(t)

2

0

for all t in [to,tf].

(4.5.1)

116

SINGULAR OPTIMAL CONTROL PROBLEMS

Following the statement and proof o f the general theorem we particularise the result to both the nonsingular and the singular special cases.

Again, we

confine our attention to the situation in which there are no terminal constraints.

See Jacobson (1971a) for

a more general treatment. Theorem 4 . 2

A sufficient condition for non-negativity is that there exists for all t in[t

of J[u(.)]

t,]

0’

a

continuously differentiable, symmetric, matrix function of time P ( * ) such that T

(C+B P)

1

C+BTP

0’

Proof:

> 0

(4.5.2)

R

for all t in [t t

Qf

T

f

-

1 and

P(tf)

(4.5.3)

0.

First, we note that for any nxn, symmetric,

continuously differentiable, matrix function of time

P(. ) , we have that

4 IXTP(Ax+Bu-~) )dt

itf

= 0.

(4.5.4)

Mow, adding this identically zero integral to J [ u ( * ) ] yields

117

4. SUFFICIENT CONDITIONS

T

T

T

T

= jtf{ix QX+U C X + ~ URu+lx P(Ax+Bu-5))dt

J[U(*)I

In view of our assumptions on u(*) and P ( * ) we can T integrate the term x P% by parts to obtain t

J[u(*)]

=

[ fii~T(t+Q+PA+ATP)x+u T (C+BTP)x+iu TRuldt

Jt

0

(4.5.6)

By inspection, then, conditions ( 4 . 5 . 2 )

and ( 4 . 5 . 3 )

f01 low. 4.5.1

The Nonsingular Special Case We now show that if ( 4 . 3 . 1 )

of a solution S(-) to ( 4 . 3 . 2 1 ,

holds then existence (4.3.3)

there exists P(*) such that ( 4 . 5 . 2 ) ,

implies that (4.5.3)

hold.

It

then follows from Theorem 4 . 1 that the conditions of Theorem 4.2 are both necessary and sufficient for positive-definiteness of J[u(*)]

in this nonsingular

case. Theorem 4.3 fies ( 4 . 3 . 2 ) ,

Suppose that an S ( * ) exists which satis(4.3.3).

Then, there exists a P(-) such

that the conditions of Theorem 4 . 2 are satisfied.

SINGULAR OPTIMAL CONTROL PROBLEMS

118

Proof:

Consider the quadratic form

where ( a l T

c1

2

T,

row vector.

is a partitioned, arbitrary, (n+m)-

Since R is positive definite the a2 that

minimizes ( 4 . 5 . 7 )

for fixed, but arbitrary, a 1 is

and the minimum value of ( 4 . 5 . 7 ) ,

using this expression,

is

T [ i + ~ + ~ ~ + ~ TT~s)-TR(-1~ (C+B + ~ T s>la1

a 1

But this is zero for all a from ( 4 . 5 . 7 )

1,

by ( 4 . 3 . 2 ) .

.

(4.5.9)

Hence we have

that

k+Q+SA+A

T

T T ’ (C+B S)

s

-> o T C+B S

(4.5.10)

R J

and

Q, - S(tf)

= 0.

(4.5.11)

SUFFICIENT CONDITIONS

4.

Identifying

I19

with P ( * ) we see that the conditions

S(*)

of Theorem 4.2 are satisfied.

4.5.2

The Totally Singular Special Case Here we have that (4.4.1) holds.

A s a consequence,

the conditions of Theorem 4.2 become for all t in [to,tfl (4.5.12)

C+BTP = 0,

T

++Q+PA+A P > 0,

for all t in [to,tf] (4.5.13)

and (4.5.14) A s in the nonsingular case, it is possible to

obtain a Riccati-type differential equation which, if it has a solution, implies the existence of P ( . ) .

In

deriving this Riccati equation we assume that B and C are once continuously differentiable with respect to

.

Furthermore, we introduce the follow0Stf1 ing, rather important, assumption. time on [t

Assumption 4.4 The first generalized Legendre-Clebsch necessary condition for optimality is satisfied in strong form for all t in [to,tf].

a .

-H

au u

which implies

=O,

t


0 -

That is, (4.5.15)

SINGULAR OPTIMAL CONTROL PROBLEMS

120

CB symmetric,

for all t in [t ,tfl 0

(4.5.16)

and -1-

a H u -*

for all t in [to,tfJ

> 0,

aU

(4.5.17)

which implies *T T - B ~ Q B + ~ B + C A B - Bc + B ~ A ~< co ~

(4.5.18)

where

H

A X T Q x + uT Cx+X T (Ax+Bu)

=

(4.5.19)

and -A

T

= Qx+A X+C

T

.

u

(4.5.20)

We shall require the following preliminary lemmas. Lemma 4.2 Proof:

T h e matrices CB and CB+CB are symmetric.

From ( 4 . 5 . 1 5 )

and ( 4 . 5 . 1 9 1 ,

(4.5.20)

we have

that

a fi au u

a

= -[-(B

d

au dt

a

=-[B au

a

= -[B

au

T

A+Cx) ]

*T A+B X+CX+CAX+CBU] TO

(4.5.21)

(4.5.22)

- A-B T~ QX-B TATX-B T CTu+~x+CAX+CBU1

(4.5.23)

4. SUFFICIENT CONDITIONS

121

T T

(4.5.24)

= C B - B C

I

=

o

for all t i n

That i s , CB i s symmetric.

[to,tfl.

(4.5.25)

Now, as C and B are con-

t i n u o u s l y d i f f e r e n t i a b l e t h i s i m p l i e s t h a t t h e lemma

i s proved. Lemma 4.3

a ii au u

*TT T T T c +B A c

T

= -B QB+~B+CAB-B

. (4.5.26)

By d i r e c t d i f f e r e n t i a t i o n and Lemma 4.2.

Proof:

Theorem 4.4

Suppose t h a t t h e r e e x i s t s a f u n c t i o n S(*)

which s a t i s f i e s f o r a l l t i n [ t f,tf1, t h e e q u a t i o n s 0

a HU1-’ [x

-i=Q+SA+ATS+[ (AB-5)T S+BTQ-CA-C]- T T

T

[ ( A B - ~ ) S+B

Q-CA-t]

(4.5.27)

(4.5.28)

-

Qf

s(tf)

2

0

.

(4.5.29)

Then, t h e r e exists a P ( * ) which s a t i s f i e s ( 4 . 5 . 1 2 )

-

(4.5.14).

Proof:

From ( 4 . 5 . 1 7 )

T i+Q+SA+A S > 0 ,

-

and ( 4 . 5 . 2 7 )

f o r a l l t i n [ t ,t o f

Next, p r e m u l t i p l y i n g ( 4 . 5 . 2 7 ) 4.2,

4.3, we obtain

we see that

T

by B

1.

(4.5.30)

and u s i n g Lemmas

SINGULAR OPTIMAL CONTROL PROBLEMS

122

- ~ = ~(C+BiTs ) A + ~ + 'TB s

(4.5.31)

which, re-arranged, is

d - -(C+B dt

T

T = s)=(c+B s ) .IA+(AB-B)

[ (AB-~)

a

-1

HuI

T ~ + ~ T ~ 1-1.~ ~ - (t4 . 5 . 3 2 )

Now this is an ordinary linear homogeneous differential T

equation for (C+B S) which has boundary condition (4.5.28).

Consequently, C+B

Ts

= 0,

for all t in [to,tfl.

(4.5.33)

Identifying P ( * ) with S ( * ) we see that the theorem is true. 4.6

Necessary and Sufficient Conditions for Nonnegativity of the Totally Singular Second Variation We have seen via Theorem 4 . 3 that, in the non-

singular case, Theorem 4 . 2 provides conditions which are both necessary and sufficient for positive-definiteness of J[u(*)].

A s a consequence of the necessity of con-

ditions ( 4 . 5 . 2 7 - 2 9 )

(McDanell and Powers, 1 9 7 0 ) subject,

4.

123

SUFFICIENT CONDITIONS

of course, to Assumption 4.4, we have necessity and sufficiency also in the totally singular case (see also Speyer and Jacobson (1971)).

The question now

arises as to the possible necessity and sufficiency of conditions (4.5.2) and (4.5.3) in the totally singular case, in the absence of Assumption 4.4 and the continuous differentiability of B and C.

It turns

out that a slightly more abstract version of these conditions is both necessary and sufficient (Jacobson and Speyer, 1971) and it is this set of conditions which occupies us in this section.

i) Necessary Condition. Under assumptions

Theorem 4.5

4.1-4.3, a necessary condition for J[u(.)

1

to be non-

negative in the totally singular case is that there exists for all t in (t 0’ tf 3 a real symmetric nxn matrix ) is monotone increasingt in function of time @ < - which t

such that T

for all t in (to,tfl

C+B P = 0, h

Qf-P(tf)

= -p(tf>

2

(4.6.1) (4.6.2)

0

where P

t i.e.,

= S+$

Th

ThP$,

z P(t)z

for all t in (t

OYtf1

(4.6.3)

is monotone increasing in t, where z belongs to Rn and is fixed but arbitrary.

SINGULAR OPTIMAL CONTROL PROBLEMS

124

i=

-$A,

$(tf)

=

(4.6.4)

I

and

-;=

T

Q+SA+A S,

s(tf) =

Qf

ii) Sufficient Condition.

(4.6.5)

In addition to

h

the above stated conditions, P ( * ) exists for all t in [to,t ] (strengthened existence condition).

f Note that the gap between the necessary and the

sufficient condition is minimal. Before proceeding to the proof of this main theorem, we verify that ( 4 . 5 . 1 2 ) - ( 4 . 5 . 1 4 ) by ( 4 . 6 . 1 ) - ( 4 . 6 . 5 )

are implied

in the case that ? ( * I is continClearly, ( 4 . 6 . 1 )

uously differentiable.

dition ( 4 . 5 . 1 2 ) and ( 4 . 6 . 2 )

is just con-

is condition ( 4 . 5 . 1 4 ) .

Now, differentiating ( 4 . 6 . 3 ) with respect to time yields

i,

=

+

iTi$+

T: $I p$I

+

$Tiql

(4.6.6)

which is T TA

T:

F+Q+SA+A~S+A$ ~ 4 - 4 P ~ + + ~ G + A= o

(4.6.7)

i.e.,

A

which, because of the monotonicity of P and the

125

4. SUFFICIENT CONDITIONS

expression ( 4 . 6 . 3 )

implies T

-o

(4.6.9)

~ + Q + P A + AP >

which is (4.5.13). The question now arises as to whether or not there are problems for which the conditions of Theorem 4 . 5 are satisfied, but for which no continuously differentiable

We supp1.y such an example below:

exists.

$ ( a )

jtfuCxdt

(4.6.10)

Jt

0

subject to ; x(to)

X’U

=

0

(4.6.11)

where we assume that C(*) is monotone decreasing in t and that

atf)

0

(4.6.12)

It is clear that

P

=

-c,

for all t in [to,tfl

A

P = P

satisfy the conditions of Theorem 4 . 5 .

(4.6.13) (4.6.14)

However, if

C is not continuously differentiable there does not exist a continuously differentiable function P(*>

I26

SINGULAR OPTIMAL CONTROL PROBLEMS

which s a t i s f i e s (4.5.12)-(4.5.14).

I n o t h e r words,

f o r t h i s example, t h e more a b s t r a c t c o n d i t i o n s of Theorem 4.5 a p p l y whereas t h e g e n e r a l s u f f i c i e n c y Theorem 4 . 2 i s i n a p p l i c a b l e . We commence t h e proof of Theorem 4.5 w i t h a

sequence of Lemmas. Lemma 4 . 4

The t o t a l l y s i n g u l a r q u a d r a t i c f u n c t i o n a l

i s e x p r e s s i b l e i n t h e e q u i v a l e n t c a n o n i c a l form

J[u(*)] = \:fuTDydt

(4.6.15)

0

subject to y = E u ;

Y(tJ

(4.6.16)

= 0

where

D

=

(4.6.17)

[C+BTS]+-l

(4.6.18)

E = +B

and where

y(t>

=

@(t)x(t), for all t in [ t

t,].

0’

(4.6.21)

4.

SUFFICIENT CONDITIONS

127

Proof: Adjoin (4.2.2) to (4.2.1) using a continuously differentiable vector multiplier A ( * )

to form

Integrating by parts and setting h(t)

=

fS(t)x(t)

(4.6.23)

yields J[u

Now, using (4.6.20) in (4.6.24) yields J[u(*)l

=

T

T

/tfu (C+B S)xdt.

J

(4.6.25)

t0

Differentiating (4.6.21) with respect to time we obtain easily the result that J[u(*)

3

=

[:fuT(C+BTS)$-lydt 0

subject to

(4.6.26)

SINGULAR OPTIMAL CONTROL PROBLEMS

128

9

+Bu ;

=

y(to) = 0

.

(4.6.27)

For t h e c o n v e r s e , we n o t e t h a t b e c a u s e (9 i s i n v e r t i b l e (4.6.26)

is just (4.6.28)

and

5

= Ax

+ Bu ;

Now, a d j o i n i n g ( 4 . 6 . 2 9 ) ”

function A(*),

(4.6.29)

x(to) = 0 t o (4.6.28)

with a multiplier

i n t e g r a t i n g by p a r t s and s e t t i n g (4.6.30)

X(t) = i ? ( t ) x ( t ) yields

J[u(-)] = j t f [ 4 x T (S+SA+ATS)x+uT(C+BTS+BT?)xldt - t0

-

- I XT ( t , ) S ( t , ) x ( t , )

.

(4.6.31)

Setting

-

S(.)

yields

=

-S(*)

(4.6.32)

SUFFICIENT CONDITIONS

4.

129

T T T J [ u ( . ) ] = j t f ( & x Qx+u C x ) d t + & x ( t f ) Q f x ( t f ) ( 4 . 6 . 3 3 ) t0

s o t h a t t h e l e m a i s proved. Lemma 4 . 5

Condition ( i ) of Theorem 4.5 i s e q u i v a l e n t

t o t h e e x i s t e n c . e f o r a l l t i n ( t , t 1 of a r e a l symmeo f t r i c monotone i n c r e a s i n g m a t r i x f u n c t i o n of t i m e P ( . ) A

such t h a t TA D+E P = 0 ,

-iqtf)

for all t in (tortf]

(4.6.34) (4.6.35)

0 .

Proof: L e t P =

$)Ti%$)+ s

(4.6.36)

and s u b s t i t u t e i n t o t h e c o n d i t i o n s of Theorem 4 . 5 and the conditions (4.6.34),

(4.6.35).

We now d e f i n e t h e r e l a t e d n o n s i n g u l a r f u n c t i o n a l

i.e.

J"U(*),E]

subject t o

=

-

Itf[uTDy + 1 u Tu l d t 2E to

(4.6.38)

130

SlNGULAR O P T I M A L C O N T R O L P R O B L E M S

(4.6.39)

Y(t0> = 0

~ = E u ;

which l e a d s t o t h e f o l l o w i n g lemmas. I f J [ u ( * ) l i s non-negative t h e n J " u ( * ) , E ]

Lemma 4 . 6

i s positive definite. Lemma 4 . 7

The proof i s obvious.

I f J [ u ( - ) ] i s non-negative t h e n t h e m a t r i x

Riccati equation

-; = E

T

-(D+E SE)

T

T

(D+E S E ) &

;

sE ( t , )

= 0

(4.6.40)

a s s o c i a t e d w i t h JN[u(*),~] h a s a s o l u t i o n which e x i s t s f o r a11 t i n [ to & f l . See Theorem 4 . 1 and Gelfand and Fomin ( 1 9 6 3 ) .

Proof:

Lemma 4 . 8

I f the matrix R i c c a t i equation (4.6.40)

has

a s o l u t i o n which e x i s t s i n t h e i n t e r v a l [ r , t f ] , t

< T < t

0 -

-

f

t h e n t h e c o n t r o l f u n c t i o n t h a t minimizes

J N T [ y ( ~ ) , u ( * ) , ~ , ~ ]=

-

Dy + 1

Tu I d t

(4.6.41)

2E

(4.6.43)

131

SUFFICIENT CONDITIONS

4.

and moreover,

(4.6.44) See Theorem 4.1.

Proof:

Lemma 4.9 parameter Proof: S,(*)

S

E

(

0

is a continuous function of the

)

E.

1 is

If J[u(.)

exists for all

non-negative it follows that E.

Furthermore, the right hand

side of (4.6.40) is analytic in S

E

and €(Coddington

and Levinson, 1955). Lemma 4.10 J'[Y(T) ,€,TI is a monotone decreasing function of E . Hence S E ( r ) is, by definition, a monotone decreasing matrix function of Proof:

For some arbitrary y(?),

Jo[y(~),

cl,

TI

=

E

E. ~

T,

we have

min ltf[uTDy +

1 uTu]dt.(4.6.45)

U('>

2E 1

Let the control which minimizes (4.6.45) be denoted u l ( * ) and its associated trajectory be y,(*).

for any

E~

->

E

1

Then,

it is clear that

(4.6.46) and by definition,

SINGULAR OPTIMAL CONTROL PROBLEMS

132

Thus, for any

-

E~ > E ~ ,we

have

A s Y(T)

and T are arbitrary, the lemma is proved. Lemma 4.11 Under Assumptions 4.1 and 4 . 3 , if J[u(.)]

is non-negative then Sm(t)

=

lim SE (t) exists for all €+a

t in (to,t ] and is negative semi-definite.

f

Proof:

From (4.6.401, SE(r)

for all

E,

0 < E <

0 for all

T

in [to,tf],

00.

By Lemma 4.10, S E ( T ) is a monotone decreasing function of

E

so that it has a limit (possibly

Now, given an arbitrary time

-a).

in the interval (t0Stf)

T

we can, by Assumption 4.3, construct u,(t),

to

2

-

t < T

such that (4.6.49)

and such that Y3(T> = Suppose that

I I S€(T)

II

arbitrary.

Y(T)

Y(T>,

-f

as

E

-f

m.

(4.6.50)

Then, by Lemmas

4 . 9 , 4.10, 4.11, Jo[y,(~),~,~]can be made large and

4.

SUFFICIENT CONDITIONS

negative for some y ( r ) and for some

E

133

sufficiently

large. This implies that one can make the following inequality hold, for

1 2~

Dy3 +

E

sufficiently large,

u3Tu33dt + J

0

[Y~(T),E,T]

< 0 . (4.6.51)

But this contradicts the fact that J[u(-)] is non-

I Is~(T) II

negative, so

+

m

as

E + m.

Since

arbitrary and since S (t ) = 0 for all E f that S,(T) exists for all T in (to,tf]. Lemma 4.12

If S,(T)

function of

T.

Proof:

E

T

is

we conclude

exists it is a monotone increasing

From (4.6.40), T

S,(d

E-tco lim[It (D+ETSE)T(D+ETSE) Edt 1 f

=

(4.6.52)

and, because of the existence of each limit, the right hand side of (4.6.52) becomes Elim[ [T+A(D+ETSE)T (D+ETSE)Edt ] tf

+ lim[ E-

T

(D+ETSE)T (D+ETSE)Edt 1.

(4.6.53)

T+A

That is, S m ( - t > = S,(T+A)+

E”

[ (D+E~S)T(D+ETS E

E

) Edt 1

(4.6.54)

SINGULAR O P T I M A L C O N T R O L PROBLEMS

134

and t h e lemma follows. I f S,(T)

Lemma 4.13

exists for a l l

T

i n (to,t,l

then

Proof: T E*

Suppose t h e c o n t r a r y .

Then, f o r some

i n ( t o , t f ) , by Lemma 4 . 9 , we have t h a t t h e r e e x i s t s > 0 such t h a t

(4.6.56) so t h a t

I,/

T

liml

E-

( D + E ~WS)T(D+ETSm)EdtI1 =

O0

(4.6.57)

tf

which c o n t r a d i c t s t h e f a c t t h a t SW(.r) e x i s t s , to <

T

1. t f .

We a r e now i n a p o s i t i o n t o prove Theorem 4 . 5 . Proof of Theorem 4 . 5

i ) Necessary Condition.

If

J [ u ( . ) ] i s non-negative t h e n by Lemmas 4.11-4.13 e x i s t s S,(T),

-

there

t o < T < t f which i s a r e a l symmetric

monotone i n c r e a s i n g m a t r i x f u n c t i o n of time such t h a t T D + E S m = O

a.e.

in [to,tfl

(4.6.58)

4. SUFFICIENT CONDITIONS

and SW(tf)

= 0

.

135

(4.6.59)

Now, since S w ( * ) is a monotone function, we can define (4.6.60) L

and consequently we have (4.6.61) Furthermore, defining < t < t

f

(4.6.62)

yields rn

D + E ~ P=

o

a.e. in [to,tfl.

(4.6.63)

is defined for all t in (to,tf].

where $(t>

In order to establish the necessary conditions we need only show that TD+E P

=

0,

for all t in (t o,tfl.

(4.6.64)

In order to do this, suppose that for some t in (t

zo

D+E~~,

.

09tf)

(4.6.65)

A s D, E are continuous in t, (4.6.65) can occur o n l y if h

a jump in P occurs a t time t that does not lie in the

I36

SINGULAR OPTIMAL CONTROL PROBLEMS

T

null space of E (t).

,.

Moreover, as P is monotone

increasing in t, it follows that

DE + E ~ + E>

o

.

(4.6.66)

Again, since $ is monotone and D and E are continuous we have that ( 4 . 6 . 6 6 ) holds during a time interval A>O,

[t, t + A ] ,

contradicting ( 4 . 6 . 6 3 ) .

D + E'?

= 0,

Equation ( 4 . 6 . 6 4 )

ii)

Hence,

for all t in (t o,tf)

.

(4.6.67)

now follows because of ( 4 . 6 . 6 0 ) .

Sufficient Condition.

Suppose now that f(t)

exists for all t in [t0' t f ] (strengthened existence condition). Adjoin the dynamics ( 4 . 6 . 1 6 ) to J[u(*) ] as follows. (4.6.68)

=

ItfuT(D+E TP>y dt

-

(4.6.69)

t0

The first integral is zero because of ( 4 . 6 . 3 4 ) .

The

remaining integral can be written in Stieltjes form as

-

yTidy

(4.6.70)

4.

I37

SUFFICIENT CONDITIONS

which, upon integration by parts, becomes (tfiyTd?y

-

iyT?yl tf

(4.6.71)

which is non-negative owing to the monotonicity of :(-)

and the fact that -?(tf)

-> 0 and

y(to)

= 0.

This concludes the proof of Theorem 4.5. It is worth noting that an extension of Theorem 4.5 is known, for the case where terminal equality constraints are present in the formulation of the optimal control problem (and hence in the second variation), (Jacobson and Speyer, 1971). 4.7

Necessary Conditions for Optimality A s Theorem 4.5 provides necessary and sufficient

conditions for non-negativity of the second variation, one would expect to be able to derive known necessary conditions, for the totally singular case, from these. This is indeed possible, as the next several results show. Theorem 4.6

(Robbins, 1967; Goh, 1966)

A necessary

condition for J [ u ( - ) ] to be non-negative is that CB be symmetric for all t in [to,tf3* Proof: From (4.6.1) T CB + B PB

=

0,

for all t in (t ,tfl. 0

(4.7.1)

Since P is symmetric it follows that CB is symmetric

SINGULAR OPTIMAL CONTROL PROBLEMS

138

.

The theorem follows from the tf3 continuity of CB with respect to time. for all t in [to,

Theorem 4.7

(Jacobson, 1969; Gabasov, 1968) A neces-

sary condition for J[u(-)] CB

+ BTSB > 0,

to be non-negative is that

for all t in [to,tfl (4.7.2)

where

.

(4.7.3)

(C+B P ) B = (C+B S+B 4 P4)B = 0,

(4.7.4)

-6 Proof:

T

= Q+SA+A S ;

S(tf)

=

Q,

From (4.6.1) and (4.6.31,

T

T

T TA

for all t in (t ,tfl. 0

AS

?(t)

-< 0 ,

for all t in [to,tfl it follows from

(4.7.4) that T

CB + B SB > 0,

for all t in (t o,tfl.

(4.7.5)

By continuity of C, B and S, inequality (4.7.2) follows. Theorem 4.8

(Kelley et al., 1967; Robbins, 1967;

Tait, 1965, generalized Legendre-Clebsch condition) necessary condition f o r J[u(*)]to

be non-negative is

that

a -. -

(-1) au Hu > 0

,

for a l l t in [to,tf]

where H i s defined by (4.5.19).

(4.7.6)

A

4.

Proof:

I39

SUFFlCIENT CONDITIONS

From ( 4 . 6 . 1 1 , ( 4 7.7)

which is dPB = 0. ( 4 . 7 . 8 )

j17(CB+lb)dt+j::(HT~B+~T~b)dt+ tl

Using ( 4 . 6 . 3 )

It2

in (4.7.8) yields

''1

( b + C i + i T P B + B T P i ) dt-

( B TQB+BT PAB+BTATPB) dt

(4.7.9)

Using (4.6.1) in ( 4 . 7 . 9 )

and collecting terms yields, (4.7.

lo)

,.

Since P is monotone increasing in t we obtain the result that (-1)

a -. H > 0, au u -

(4.7.11)

The continuity with respect to time of the left hand side of (4.7.11) yields ( 4 . 7 . 6 ) .

I40

SINGULAR O P T I M A L C O N T R O L P R O B L E M S

4.8

Other Necessary and Sufficient Conditions The necessary and sufficient conditions presented

and proved in Section 4.6 do not require that (4.5.17) hold.

However, if this strong form of the generalized

Legendre-Clebsch condition is assumed then either Goh's or Kelley's transformation technique can be used to transform the totally singular second variation into a nonsingular one.

In the case of Goh's transformation

the dimension of the state space is preserved and the ensuing analysis (Goh, 1970; McDanell and Powers, 1970) results in the Riccati differential equation of (4.5.27) Application of Kelley's transformation technique yields a nonsingular problem in a reduced dimensional state

space and hence a Riccati differential equation of reduced dimension results (Speyer and Jacobson, 1971). Both these Riccati equations imply that the second variation is strongly positive with respect to the control variable in the transformed spaces. Note that our conditions, Theorem 4.5, have been extended recently to the partially singular case by Anderson (1973). 4.9

Sufficient Conditions for a Weak Local Minimum A s pointed out in the introduction to this

chapter, a sufficient condition for a weak local minimum in the nonsingular optimal control problem is that a solution exists for all t in [to,tf] to the

4. SUFFICIENT CONDITIONS

matrix Riccati equation (4.3.2).

141

In other words, a

sufficient condition is that J[u(*)]

given by (4.2.1)

be strongly positive, where

(4.9.1)

and where (4.9.2)

In the singular case the second variation cannot be strongly positive, but sufficiency is nevertheless ensured by the strong positivity of the transformed second variation (Goh, 1970; Jacobson and Speyer, 1971; McDanell and Powers, 1970; Speyer and Jacobson, 1971). A recent advance in the theory of singular optimal

control problems is provided by Moyer (1973) who gives sufficient, but generally not necessary, conditions f o r a strong minimum. theory.

This is accomplished by a field

S I N G U L A R OPTIMAL CONTROL PROBLEMS

I42

4 . 1 0 Existence Conditions for the Matrix Riccati

Differential Equation This chapter is concerned mainly with (necessary and) sufficient conditions for non-negativity of the singular second variation.

It is an interesting fact

that the sufficient conditions ( 4 . 5 . 1 2 ) - ( 4 . 5 . 1 4 )

play

an important role in deducing conditions which guarantee existence of a solution to ( 4 . 3 . 2 - 3 ) .

The

conditions so deduced (Jacobson, 1 9 7 0 b ) are less restrictive than those known heretofore (Kalman, 1 9 6 0 ; Breakwell and Ho, 1 9 6 5 ) which we state here without proof. Theorem 4 . 9

Sufficient conditions for the existence

< t < tf which satisfies of S(t>, to -

-;=

T T -1 T T Q+SA+A S-(C+B S ) R (C+B S )

(4.10.1)

are that Q

-

-1

R

T -1

C R

-

C > 0,

> 0,

for all t in [t03tf1

(4.10.3)

f o r all t in [t

(4.10.4)

0’

tf]

(4.10.5)

SUFFICIENT CONDITIONS

4.

I43

Clearly, (4.10.1) can be written in the form

-B

- -TS-SBR-1 BTs Q+SA+A

=

(4.10.6)

where T -1

G=Q-CR

-

C

-1

A=A-BR

(4.10.7)

C.

(4.10.8)

In view of this, we shall study the question of existence of a solution to (4.10.6) but, for simplicity of notation we shall simply refer to

and

6 as A

and Q.

We then have the following theorem. Theorem 4.10 A sufficient condition for the existence

-5

=

-1 T T Q+SA+A S-SBR B S

s(tf)

(4.10.9)

Qf

=

(4.10.10)

is that there exists an nxn symmetric matrix function of time P ( - ) whose elements are continuously differentiable functions of time in the interval [t

0’

tf] such

that T B P = 0, :+Q+PA+A

T

for all t in [t o*tfl

(4.10.11)

P=M(t) > 0, for all t in [to,tf] (4.10.12)

SINGULAR OPTIMAL CONTROL PROBLEMS

144

Q,

-

P(tf)

= G,

1. 0 .

(4.10.13)

Proof: Let

where

Y(.)

of t i m e .

-. -. =

-P-S

=

and

s ( * are ) real symmetric matrix functions

Then, from ( 4 . 1 0 . 1 1 ) ,

-_

-IT--

-1 TB S PBR B P.

and ( 4 . 1 0 . 1 4 )

Q+A~(F+F)+(F+T)A-M-TBR

(4.10.15)

B (P+S)

-1 T---

Using ( 4 . 1 0 . 1 1 ) =

_-

and ( 4 . 1 0 . 1 4 )

T--Q+A (P+S) + (P+S A-M-FBR-I B~T-TBR-IB

-

..

(4.10.12)

T-Q+A (P+S)+(P+S)A-M-(P+S)BR

-PBR

-F-T

(4.10.14)

for a l l t in [t 0 ,tf]

P(t)+F(t)=P(t),

~F (4.10.16)

in ( 4 . 1 0 . 1 6 )

yields

-1 TB S+PBR B P.

-1 T--

(4.10.17)

Now choose,

-=

-P and

T-- - - I T-M+A P+PA+PBR B P

-P(tf)

= -Gf

From ( 4 . 1 0 . 1 8 ) ,

.

(4.10.19)

(4.10.18)

(4.10.19)

we see that the variable

(-F)

satisfies a Riccati equation for which the conditions of Theorem 4 . 9 hold, namely

4. SUFFICIENT CONDITIONS M(t) > 0, R-’(t)

I45

f o r a l l t i n [ to * t f l

(4.10.20)

> 0, f o r a l l t i n [ t o , t f ]

(4.10.21)

-

(4.10.22)

Gf > 0

s o t h a t F ( t ) e x i s t s f o r a l l t i n [ to’ t f 1 Now, u s i n g (4.10.18)

i n (4.10.13)

i n (4.10.17)

and (4.10.19)

yields

T---1 T-s- = Q+FA+A s SBR B s

-S ( t f )

=

(4.10.23)

Qf

(4.10.24)

which a r e i d e n t i c a l t o (4.10.91,

(4.10.10).

Further-

more, b e c a u s e P ( t ) and P ( t ) e x i s t f o r a l l t i n [ tOStf1 it follows from (4.10.14) t h a t y ( t ) exists f o r a l l t in [to,tf],

and the t h e o r e m i s proved.

The next t h e o r e m and t h e f o l l o w i n g example emphas i z e t h e v a l i d i t y o f Theorem 4.10. Theorem 4.11

The c o n d i t i o n s of Theorem 4.10 are n o t

more s t r i n g e n t t h a n t h o s e of Theorem 4.9. T -1 P r o o f : W r i t i n g Q-C R C f o r Q and A-BR-IC

for A in

t h e c o n d i t i o n s of Theorem 4.10 y i e l d s BT p = 0 ,

;+Q-C

(4.10.25)

-for a l l t in [ t o , t f l

T -1 -1 T R C+(A-BR C) P+P(A-BR-’C)=M(t)

-> 0

(4.10.26)

SINGIJLAR O P T I M A L C O N T R O L PROBLEMS

146

for all t in [t ,tf] and, 0

(4.10.27)

Clearly ( 4 . 1 0 . 2 5 ) - ( 4 . 1 0 . 2 7 ) P(.)

5

are satisfied with

0, if the conditions o f Theorem 4.9 are

satisfied. The following example illustrates that the conditions of Theorem 4.10 are less r e s t r i c t i v e than those of Theorem 4 . 9 .

4.10.1 An Example Let n=2, m=l, t =0, t =1 C=O,

Q = [

(4.10.28)

f

0

-

T

R '=1,

B =(0,1),

-1

0

0

4

(4.10.29)

Q =O

f

l o

* = l o

1 (4.10.30) 0

Clearly these values do not satisfy the conditi-ons of Theorem 4 . 9 ; that is, the known sufficiency conditions are violated. The conditions of Theorem 4.10 become

(P12

P22)

=

0,

for all t in [ 0 , 1 ]

(4.10.31)

4. SUFFICIENT CONDITIONS

for a l l t in

147

[0,11

and

(4.10.33)

Let u s now choose PJl)

(4.10.34)

= 0

which s a t i s f i e s (4.10.33)

with equality.

I f w e t h e n choose = 2

(4.10.35)

P l 1 ( 0 ) = -2

(4.10.36)

;ll

we get

and (4.10.32)

becomes -2+2t

4

> o .

(4.10.37)

I48

SINGULAR OPTIMAL CONTROL PROBLEMS

Inequality (4.10.37) holds for all t in [0,1]

so

that

the Riccati equation associated with the parameter values given in (4.10.28)-(4.10.30)

has a solution for

all t in [0,1], despite the fact that the conditions of Theorem 4.9 are violated.

4.11

Conclusion In this chapter we studied rather thoroughly the

quadratic functional that arises as the second variation in optimal control problems.

First, we proved

certain necessary and sufficient conditions for strong positivity of the nonsingular second variation and illustrated by means of an example that the totally singular second variation cannot be strongly positive. Next, we presented and proved a general sufficiency theorem for the partially singular case and specialized this in both nonsingular and totally singular cases. Following a detailed progression of lemmas we proved our main theorem in Section 4 . 6 , providing necessary and sufficient conditions for non-negativity of the singular second variation.

In the next section we

deduced certain well known necessary conditions for optimality in singular control problems.

Other neces-

sary and sufficient conditions not derived in this chapter were then referred to and briefly discussed and the question of sufficient conditions for both a weak and a strong minimum were examined.

Finally, we

4. SUFFICIENT CONDITIONS

I49

showed that the conditions of Theorem 4.2 allow us to deduce conditions for existence of a solution to the (nonsingular) matrix Riccati differential equation. These conditions are less stringent than those known heretofore. References Partially Singular LinearAnderson, B. D. 0. (1973). Quadratic Control Problems, IEEE Trans. autom. Control AC-18, 407-409.

-

Breakwell, J. V. and Ho, Y. C. (1965). On the Conjugate Point Condition for the Control Problem, Int. J. Engng. Sci. 2 , 565-579. Brockett, R. W. (1970). "Finite Dimensional Linear Systems". John Wiley & Sons, Inc., New York. "Theory of Coddington, E. A. and Levinson, N. (1955). Ordinary Differential Equations". McGraw-Hill, New York. Gabasov, R. (1968). Necessary Conditions for Optimality of Singular Control, Engng. Cybern. No.5, 28-37. &Ifand, I. M. and Fomin, S. V. (1963). "Calculus of Variations". Prentice-Hall, Englewood Cliffs, N.J. Goh, B. S. (1966). The Second Variation for the Singular Bolza Problem, SIAM J. Control 4, 309-325. Goh, B. S. (1970). A Theory of the Second Variation in Optimal Control, unpublished report, Division of Applied Mechanics, Univ. California, Berkeley. Jacobson, D. H. (1969). A New Necessary Condition of Optimality for Singular Control Problems, SIAM J. Control 7, 578-595.

-

I50

SINGULAR OPTIMAL CONTROL PROBLEMS

Jacobson, D. H. (1970a). Sufficient Conditions for Nonnegativity of the Second Variation in Singular and Nonsingular Control Problems, SIAM J. Control 8 , 403-423. Jacobson, D. H. (1970b). New Conditions for Boundedness of the Solution of a Matrix Riccati Differen8, tial Equation, J. Differential Equations 25 8-263. Jacobson, D. H. (1971a). A General Sufficiency Theorem for the Second Variation, J. math. Analysis Applic. 34, 578-589. Jacobson, D. H. (1971b). Totally Singular Quadratic Minimization Problems, IEEE Trans. autom. Control AC-16, 651-658.

-

Jacobson, D. H. and Speyer, J. L. (1971). Necessary and Sgfficient Conditions for Optimality for Singular Control Problems: A Limit Approach, J. math. Analysis Applic. 34, 239-266. Johansen, D. E. (1966). Convergence Properties of the Method of Gradients, in "Advances in Control Systems" (C. T. Leondes, ed.)Vol. 4 , pp.279-316. Academic Press, New York and London. Kalman, R. E. (1960). Contributions t o the Theory of 5, Optimal Control, Bol. SOC. Mat. Mexicana 102-119. Kelley, H. J., Kopp, R. E. and Moyer, H. G. (1967). Singular Extremals in "Topics in Optimization" (G. Leitmann, ed.) pp.63-101. Academic Press, New York. McDanell, J. P. and Powers, W. F. (1970). New JacobiType Necessary and Sufficient Conditions for Singular Optimization Problems, AIAA J. 8, 1416-1420. Moyer, H. G. (1973). Sufficient Conditions for a Strong Minimum in Singular Control Problems, SIAM J. Control 11, 620-636.

-

4.

151

SUFFICIENT CONDITIONS

Robbins, H. M. (1967). A Generalized Legendre-Clebsch Condition for the Singular Cases of Optimal Control, IBM J1 Res. Dev. 3, 361-372. Speyer, J. L. and Jacobson, D. H. (1971). Necessary and Sufficient Conditions for Optimality for Singular Control Problems; A Transformation Approach, J. math. Analysis Applic. 33, 163-187.

-

Tait, K. S. (1965). Singular Problems in Optimal Control, Ph.D. dissertation, Harvard Univ., Cambridge, Mass.

This page intentionally left blank

CHAPTER 5 Computational Methods for Singular Control Problems 5.1

Introduction

It turns out that computational methods for singular optimal control problems have not been fully developed and there remains, therefore, considerable opportunity for further, but rather difficult, research in this area.

In this chapter, we survey certain numerical methods which have appeared during the past several years.

First, we look at the problem of constructing

a matrix function P(*) which satisfies the differential inequalities of Theorem 4 . 2 and of Theorem 4 . 5 .

This

is an important numerical problem as these theorems state only that if a P(*) exists, then the second variation is non-negative; there is no hint as to how to

construct P ( * ) .

Next, we look at the problem of

computing controls which are, possibly, singular.

In

nonsingular situations there are numerous algorithms available, for example Jacobson and Mayne (1970), which iteratively improve a nominal, guessed, control function. However, in the singular case, most of these algorithms are not usable owing to the fact that certain steps in the algorithms become undefined.

The

remaining algorithms which are able to handle the singular case are of the first-order, or gradient, type, and these have been shown to behave particularly poorly, as far as rate of convergence is concerned, I53

154

SINGULAR OPTIMAL CONTROL PROBLEMS

when a p p l i e d t o s i n g u l a r o p t i m a l c o n t r o l problems (Johansen, 1966).

We t h e r e f o r e p r e s e n t a method

(Jacobson e t a l . , 1970) which c o n v e r t s a s i n g u l a r problem i n t o a sequence of n o n s i n g u l a r o n e s by t h e a d d i t i o n of a t e r m

.it

uTudt

.

T h i s e n a b l e s us t o u s e powerful second-order a l g o r i t h m s (Jacobson and Mayne, 1970).

As

E

i s progressively

reduced toward z e r o w e f i n d t h a t t h e s o l u t i o n of t h e E-problem t e n d s t o t h a t o f t h e o r i g i n a l , p o s s i b l y s i n g u l a r , one.

I n t h i s c o n n e c t i o n I$C r e f e r t o Powers

and McDanell (1971) who have s u c c e s s f u l l y u s e d t h i s t e c h n i q u e i n t h e s i n g u l a r S a t u r n g u i d a n c e problem,

A r e c e n t g e n e r a l i z e d g r a d i e n t method due t o Mehra and Davis (1972) i s t h e n mentioned as are c e r t a i n r e c e n t experiments u s i n g f u n c t i o n space v e r s i o n s of Davidon's and Broyden's p a r a m e t e r o p t i m i z a t i o n methods (Edge and Powers, 1974a, 1974b). We t h e n t u r n o u r a t t e n t i o n t o t h e s i n g u l a r l i n e a r -

q u a d r a t i c o p t i m a l c o n t r o l problem and g i v e r e c e n t , r e l e v a n t , r e f e r e n c e s ( J a c o b s o n , 1972; Moore, 1969; Moylan and Moore, 1971; Wonham and Johnson, 1964). F i n a l l y , w e emphasize c e r t a i n work by McDanell and Powers (1971) and Maurer (1974) who have o b t a i n e d i n t e r e s t i n g c o n d i t i o n s which i n c r e a s e u n d e r s t a n d i n g of j u n c t i o n s between s i n g u l a r and n o n s i n g u l a r arcs.

5. COMPUTATIONAL METHODS 5.2

155

Computational Application of the Sufficiency ‘Conditionsof Theorems 4.2 and 4.5 In this section we investigate certain methods for

constructing a matrix function P ( - ) which satisfies the conditions of Theorem 4.2 and/or Theorem 4 . 5 . 5.2.1

The Nonsingular Case

In the nonsingular case it is known, and proved in Theorem 4 . 3 , that if S ( * )

exists which satisfies

TS-(C+B Ts) TR-1 (C+BTs) -~=Q+sA+A

(5.2.1)

(5.2.2)

then P ( * )

= S(*)

satisfies the conditions of Theorem

4.2.

Furthermore, see Theorem 4 . 1 ,

S(t),

to < t < tf is also a necessary condition for

-

the existence of

non-negativity of the second variation so that necessity of the conditions of Theorem 4.2 in the nonsingular case is implied.

Thus it turns out that, in the non-

singular case, integration of ( 5 . 2 . 1 1 ,

which can be

accomplished numerically, is an altogether satisfactory way of constructing P ( * ) . 5.2.2

The Totally Singular Case If we are prepared to make the assumption that (-1)

a H*. au u

> 0,

for all t in [t

0’

tf1(5.2.3)

156

SINGULAR OPTIMAL CONTROL PROBLEMS

we have, from Theorem 4 . 4 , that the existence of a function S ( . ) which satisfies, for all t in [t.,tfl, the equations

T T T * T -;=Q+SA+A S + [ (AB-i) S+B Q-CA-C]

a ..

-1

[uHU 1

[ (AB-i) TS+BTQ-CA-t 1

T C(t ) + B (tf)S(tf> E

(5.2.4)

(5.2.5)

= 0

implies that P ( * ) = S ( * ) satisfies the conditions of Theorem 4 . 2 .

This is a Riccati differential equation

but, unlike ( 5 . 2 . 1 1 ,

equation ( 5 . 2 . 4 )

constructs an

S ( - ) which satisfies

T

C+B S = 0

,

for all t in [to,tfl*

Again, see McDanell and Powers (19701,

(5.2.7)

existence of

S ( * ) turns out to be a necessary condition for non-

negativity of the second variation s o that the conditions of Theorem 4 . 2 are also necessary, subject to condition ( 5 . 2 . 3 ) , variation.

for non-negativity of the second

The necessity of the existence of S(*) is

obtained by means of Goh's transformation technique in (McDanell and Powers, 1970).

5. COMPUTATIONAL METHODS

157

The f a c t t h a t t h e S ( * ) which s a t i s f i e s ( 5 . 2 . 4 ) (5.2.6)

a l s o s a t i s f i e s (5.2.7)

-

r a i s e s t h e q u e s t i o n of

whether o r n o t , i n t h e s i n g u l a r c a s e , t h e r e e x i s t s a R i c c a t i d i f f e r e n t i a l e q u a t i o n of lower dimension, of t h e same form as (5.2.1).

That t h i s i s indeed t r u e i s

demonstrated by Speyer and Jacobson (1971) u s i n g K e l l e y ' s t r a n s f o r m a t i o n technique.

It i s t h i s t y p e of

r e d u c t i o n of t h e dimension of t h e s t a t e space t o produce a n o n s i n g u l a r second v a r i a t i o n which i s e x p l o i t e d by Anderson and Moylan (1973, 1974). I f (5.2.3)

i s not s a t i s f i e d then the singularity

i s deeper and one of t h e h i g h e r o r d e r g e n e r a l i z e d Legendre-Clebsch c o n d i t i o n s must t u r n o u t t o b e s a t i s f i e d w i t h s t r i c t i n e q u a l i t y i f a Riccati-approach t o t h e c o n s t r u c t i o n of P(*) i s t o b e used.

It i s t h e r e -

f o r e c l e a r from t h e s e remarks t h a t a method f o r cons t r u c t i n g P ( * > which i s g e n e r a l , i n t h e s e n s e t h a t i t does n o t depend upon (5.2.3),

is preferable.

Such a

method i s d i s c u s s e d next. 5.2.3

A L i m i t Approach t o t h e C o n s t r u c t i o n of P(*)

The proof of Theorem 4.5 i s based on t h e f a c t t h a t t h e s i n g u l a r second v a r i a t i o n can b e made n o n s i n g u l a r by a d d i t i o n of t h e term

and t h a t

E

can b e p r o g r e s s i v e l y i n c r e a s e d s o t h a t

SINGULAR OPTIMAL CONTROL PROBLEMS

158

lim s (.) E”

= p(.)

(5.2.8)

&

where

SE(tf)

=

Qf

.

(5.2.10)

Computationally, constructing P ( - ) according to (5.2.8)

has both an advantage and a disadvantage.

disadvantage is rather obvious; namely, as right hand side of ( 5 . 2 . 9 )

E-,

The

the

becomes ill defined.

However, possible numerical difficulties can be reduced by solving ( 5 . 2 . 9 )

iteratively using a Newton or quasi-

linearization method in which the solution for c=cl is used as a starting solution for the solution for E = E ~ > E ~ .

This is, in principle, a more stable approach

than simply setting grating ( 5 . 2 . 9 )

E

to a very large value and inte-

backwards from t

f numerical integration techniques. The advantage of using (5.2.8)

using standard is as follows:

Often, we are not interested in the actual value of

P(-) but rather whether or not a P ( * ) exists which satisfies the conditions of Theorem 4 . 5 .

In other words we may wish to construct P ( . ) only to verify its existence. Now, if no such P ( * ) exists we have from the lemmas preceding the proof of Theorem 4 . 5 , and because of the continuity of S ( * ) with respect to c , E

5. COMPUTATIONAL METHODS

that for some

E

159

sufficiently large but f i n i t e , SE(t)

will cease to exist for some t in (to,tf].

Thus the

non-existence of a P(*) which satisfies the conditions of Theorem 4.5 can be inferred from the non-existence of S E ( * ) ,

~ < m ,E

sufficiently large.

In other words,

if no P(*) exists, it is not necessary to compute

lim S , ( * ) to confirm this; rather the integration of

€-

(5.2.9)

may be halted as soon as a finite escape time

is detected in S E ( * ) . To our knowledge, this limit approach for constructing, or testing the existence of, P ( * ) is the only one which is independent of assumptions such as ( 5 . 2 . 3 ) . 5.3

Computation of optimal Singular Controls We now corn to the question of synthesis of

optimal control functions for, possibly, singular control problems.

Of course, if it is known a-priori

that the optimal control function has singular sub-arcs, and if the number and location of these sub-arcs are known, then special techniques can be devised to deal with this situation (Anderson, 1 9 7 2 ) .

However, if

little a-priori knowledge is at hand, an algorithm which can cope with singular sub-arcs, as and if these arise, is most attractive. 5.3.1

Preliminaries We shall consider the problem of controlling

I 60

SINGULAR OPTIMAL CONTROL PROBLEMS

jr=fl(x,t) + fu(x,t)u;

(5.3.1)

x ( t o ) = x0

where x ( t ) i s i n Rn, u ( t ) i s i n Rm and where u ( t ) i s c o n s t r a i n e d i n t h e f o l l o w i n g way: .lui(t)l

2

1 for a l l t i n

[to,tfI,

...,m.

i=l,

(5.3.2) The performance index, o r c o s t f u n c t i o n a l , i s

where t o and t f are given.

The f u n c t i o n s f l ( x , t ) ,

f U ( x , t ) , L ( x , t ) and F [ x ( t f ) ] are assumed t o b e a t

least once c o n t i n u o u s l y d i f f e r e n t i a b l e i n each argument. Our problem i s t o choose u(*), p i e c e w i s e cont i n u o u s i n t i m e , which s a t i s f i e s (5.3.2),

t o minimize

J [ u ( * ) 1. It i s w e l l known t h a t , i n t h e absence of c e r t a i n

f u r t h e r assumptions, t h e optimal c o n t r o l f u n c t i o n f o r t h i s c l a s s of problems c o n s i s t s of bang-bang and s i n g u l a r sub-arcs.

Of c o u r s e , i f t h e o p t i m a l c o n t r o l

i s p u r e l y bang-bang

(i.e.

nonsingular) t h e techniques

of Jacobson and Mayne, (1970) can handle t h e problem. 5.3.2

An €-Algorithm We develop h e r e an a l g o r i t h m f o r s o l v i n g t h e

COMPUTATIONAL METHODS

5.

161

problem formulated in Section 5.3.1 which is applicable whether or not singular sub-arcs exist in the optimal control.

Basically, we convert (5.3.3)

into a non-

singular functional by the addition of a term

“itfT 2

u udt.

t0

Minimizing this nonsingular functional, using for example the techniques described by Jacobson and Mayne (1970),

for a sequence of

lim ‘k = 0 kNo assumptions as

such that

E ~ ’ S

yields the optimal value of (5.3.3).

to the number and position of singular sub-arcs need be made. First we define J[u(*)

, E ~ ]

=

If‘ L

E

[L(x,t)

k T + -U2 u]dt+F[x(tf)

I

(5.304)

0

where Ek > 0

(5.3.5)

Clearly this ck-problem is nonsingular and can be solved using the methods of Jacobson and Mayne (1970). Our E-Algorithm is then as follows: Step 1.

Choose a starting value control function Z1( * I .

Step 2.

E

1

> 0 and a nominal

Solve the resulting Ek-problem ( k = l initially)

S I N G U L A R OPTIMAL CONTROL PROBLEMS

I62

using the methods of Jacobson and Mayne

(1970); this yields a minimizing control function Step 3 .

% ( a ) .

< Choose E k+l -

E

k

(for example,

(*)=u,(-), k=k+l, and

set %+I

E

k+l

=E

/lo),

k

go to Step 2.

In a practical case the algorithm can be halted, after Step 2 , when E

k

where

CJ

(5.3.6)

<(s

is a small, positive, pre-determined number.

We prove the convergence of the €-Algorithm under the following assumptions. Assumption 5.1

Let U be the set of piecewise contin-

uous m-vector functions of time defined on [t OStf1 Then which satisfy (5.3.2). (5.3.7)

where u(*) belongs to U and v0 belongs to R1. Assumption 5.2 inf J[u(*),E~] = J[u~(.),E k 1 is obtained in U.

U(')

That is, \ ( * ) ,

the control which minimizes J[u(-),E

belongs to U. These two assumptions allow us to prove the following lemma. Lemma 5.1

For k>R(Ek<EL)

we have

k

]

5.

J[q(*>

COMPUTATIONAL METHODS

,Ek1

That is, J[q(*) function of

E

k'

I63

(5.3.8)

J[uQ(*) , E ~ ]

is a monotonically decreasing

Moreover, we have that (5.3.9)

u(*) belongs to U. Proof: We have that

(5.3.10)

In addition,

(5.3.11)

We now state and prove our main theorem. Theorem 5.1

For a positive sequence

{E

1,

E >E

k k k+1" and lim E:k =0, and under Assumptions 5.1, 5.2, k(5.3.12)

Proof:

Since, by Lemma 5.1, (J[%(-),E~]~ i s monotone

decreasing and bounded below by vo, it must converge.

164

SINGULAR O P T I M A L CONTROL P R O B L E M S

Suppose, then, that (5.3.13)

But then, because of (5.3.2)

and the fact that

-m
E

R

belonging to

{E

k

1,

E

R

where uo(-) is defined by min J[u(.)]

= J[uo(*)]

U(.>

Inequality (5.3.14)

= v

0 '

u(-) in U.

(5.3.15)

then implies that

But this implies, since {J[u~(.),E~]} is a monotone decreasing sequence that (5.3.17)

Hence, v7e have a contradiction and so (5.3.12) must be true.

It can happen that, as E k approaches zero, numerical ill-conditioning manifests itself. This is especially the case if second-order numerical methods

are used, as then terms involving - occur (Jacobson E k 1

5.

et al., 1970). I1

COMPUTATIONAL METHODS

165

However, it must be acknowledged that

a sufficiently good approximation" to the solution can

often be obtained by reducing Ek to a small, but still numerically stable, value. Indeed, Powers and McDanell (1971) report a successful application of the €-Algorithm to the important Saturn guidance problem. 5.3.3

A Generalized Gradient Method for Singular Arcs

Recently, Mehra and Davis (1972) exploited the observation that the independent variables in a computational procedure for determining optimal control do not have to be only the control variables.

They show

that a judicious mixture of control and state variables as independent variables can result in the control problem being nonsingular in these variables, and hence standard algorithms such as gradient or conjugate gradient can be used to advantage.

Though various,

relatively simple, problems are solved by Mehra and Davis (1972) using this idea, it seems to us that the procedure is not yet systematized to a point where it can be programmed into a package and used routinely. Thus it appears that additional research, in developing further these ideas, is necessary. 5.3.4

Function Space Quasi-Newton Methods Perhaps the most encouraging numerical experiments

in the computation of optimal (singular) controls are those of Edge and Powers (1974a, 1974b).

The approach

166

SINGULAR OPTIMAL CONTROL PROBLEMS

h e r e i s t o u s e t h e powerful Davidon and Broyden parameter o p t i m i z a t i o n methods by s u i t a b l y e x t e n d i n g t h e s e t o f u n c t i o n space.

The r a t e of convergence

experienced when u s i n g t h e s e methods i s c o n s i d e r a b l y f a s t e r t h a n i n t h e case of g r a d i e n t and c o n j u g a t e g r a d i e n t methods, and t h e f u n c t i o n space quasi-Newton methods a r e s i m p l e r t o program than f u l l second-order algorithms.

I n addition, only gradient information

needs t o be c a l c u l a t e d , whereas second d e r i v a t i v e s of t h e dynamic system and performance i n t e g r a n d need t o be o b t a i n e d e x p l i c i t l y i n t h e c a s e of second-order a1 gor i thms 5.3.5

.

Outlook f o r F u t u r e I t i s c l e a r from t h e above sub-sections

t h a t much

r e s e a r c h needs t o be done i n t o t h e problem of computat i o n of optimal ( s i n g u l a r ) c o n t r o l s ,

T h i s i s empha-

s i z e d by t h e f a c t t h a t t h e most promising experiments (Edge and Powers, 1974a, 1974b) a r e only of r e c e n t vintage. 5.4

J o i n i n g o f Optimal S i n g u l a r and Nonsingular Sub- A r c s I n t h i s book our a t t e n t i o n h a s been focussed mainly

on problems t o t a l l y s i n g u l a r f o r a l l t i n [ t o , t f

I.

However, p r a c t i c a l problems u s u a l l y e x h i b i t a m i x t u r e ,

o r c o n c a t e n a t i o n , of n o n s i n g u l a r and s i n g u l a r sub-arcs i n t h e i r optimal c o n t r o l f u n c t i o n s .

Here we g i v e a

5. COMPUTATIONAL METHODS

167

b r i e f survey of r e s u l t s i n t h i s area. Study of j u n c t i o n c o n d i t i o n s i n l i n e a r - q u a d r a t i c problems, where c o n d i t i o n (5.2.3)

i s s a t i s f i e d , has

y i e l d e d a r a t h e r complete p i c t u r e (Jacobson, 1972; Moore, 1969; Moylan and Moore, 1971; Wonham and Johnson, 1964), b u t i n higher-order

s i n g u l a r problems, j u n c t i o n s

between s i n g u l a r and n o n s i n g u l a r segments are n o t It i s f a i r t o s a y , however, t h a t

e n t i r e l y understood.

McDanell and Powers (1971) have provided numerous i n s i g h t s i n t o j u n c t i o n c o n d i t i o n s v i a a number of theorems, one of which w e s t a t e below w i t h o u t proof. It i s important t o n o t e t h a t t h e work of McDanell and

Powers w a s s t i m u l a t e d by t h e p i o n e e r i n g c o n t r i b u t i o n s of Kelley e t a l . , Theorem 5.2

Let t

(1967). C

be a p o i n t a t which s i n g u l a r and

n o n s i n g u l a r sub-arcs of an optimal c o n t r o l u ( * ) are j o i n e d , and l e t q be t h e o r d e r of t h e s i n g u l a r a r c . Suppose t h a t t h e s t r e n g t h e n e d , g e n e r a l i z e d LegendreClebsch n e c e s s a r y c o n d i t i o n i s s a t i s f i e d a t t c ; i . e . (5.3.18) and assume t h a t t h e c o n t r o l i s piecewise a n a l y t i c i n a neighbourhood of tc.

-

L e t u ( ~ )( r > 0 ) b e t h e lowest

o r d e r t i m e d e r i v a t i v e of u t h a t i s d i s c o n t i n u o u s a t t

C

.

Then, q + r i s an odd i n t e g e r . Corollary 5.1

I n q-even problems, assuming u i s piece-

wise a n a l y t i c and t h e s t r e n g t h e n e d , g e n e r a l i z e d Legendre-

168

SINGULAR OPTIMAL CONTROL PROBLEMS

Clebsch condition is satisfied, the optimal control is continuous at each junction.

See Maurer ( 1 9 7 4 ) for an

example. Corollary 5.2

In q-odd problems, assuming u is piece-

wise analytic, and the strengthened, generalized Legendre-Clebsch condition is satisfied, the optimal control either has a jump discontinuity at each junction, or else the singular control joins the boundary smoothly; i.e. with a continuous first derivative.

5.5

Conclusion In this Chapter we first examined the question of

the construction of matrix functions of time which satisfy certain (necessary and) sufficient conditions for non-negativity of the second variation.

It turns

out, as is well known, that integration of the conventional matrix Riccati differential equation accomplishes this construction in the nonsingular case.

In the

totally singular case, under certain assumptions, a new Riccati differential equation is used in this role.

In

general, however, in the absence of these assumptions, the construction o f the required matrix functions of time is not routine. approach

-

The suggested method

-

a limit

while having certain numerical difficulties

associated with it, appears to be the most useful general technique available. Next, we considered the problem of computation of optimal singular controls.

This is, in many respects,

5. COMPUTATIONAL METHODS

169

a similar problem to that discussed above in that the most flexible technique available turns out to be the €-Algorithm, which is also a limit method.

However,

recent experiments using Davidon and Broyden algorithms in function space are very encouraging, Finally, we discussed the joining of singular and nonsingular sub-arcs.

In the linear-quadratic problem,

under the assumption that the generalized LegendreClebsch condition is satisfied for q=l with strict inequality, the situation is rather well understood. However, many difficult questions remain in the case of higher-order singular arcs. We conclude, then, by noting that it is rather clear that further research needs to be done on computational methods in general and on computation of optimal singular controls in particular. References Anderson, B. D. 0. and Moylan, P. J. (1973). Spectral Factorization o f a Finite-Dimensional, Nonstationary Matrix Covariance, Tech. Rep. EE-7303, Univ. Newcastle, Australia. Anderson, B. D. 0. and Moylan, P. J. (1974). Synthesis of Linear Time-Varying Passive Networks, IEEE Trans. Circuits and Systems CAS-21, 678-687.

-

Anderson, G. M. (1972). An Indirect Numerical Method for the Solution of a Class of Optimal Control Problems with Singular Arcs, IEEE Trans. autom. Control AC-17, 363-365.

-

Edge, E. R. and Powers, W. F. (1974a). Shuttle Ascent Trajectory Optimization with Function Space Quasi-

170

SINGULAR OPTIMAL CONTROL PROBLEMS

Newton Techniques, AIAA Mechanics and Control of Flight Specialist Conference, Anaheim, California. Edge, E. R. and Powers, W. F. (1974b). Function Space Quasi-Newton Algorithms for Optimal Control Problems with Bounded Controls and Singular Arcs, unpublished paper, Univ. Michigan, Ann Arbor, Jacobson, D. H. (1972). On Singular Arcs and Surfaces in a Class of Quadratic Minimization Problems, J. math. Analysis Applic. 37, 185-201.

Jacobson, D. H. and Mayne, D. Q. (1970). "Differential Dynamic Programming", Elsevier, New York.

Jacobson, D. H., Gershwin, S. B. and Lele, M. M. (1970). Computation of Optimal Singular Controls, IEEE Trans. autom. Control AC-15, - 67-73. Johansen, D. E. (1966). Convergence Properties of the Method of Gradients, in "Advances in Control Systems" (C. T. Leondes, ed) V o 1 . 4 , pp.279-316. Academic Press, New York. Kelley, H. J., Kopp, R. E. and Moyer, H. G. (1967). Singular Extremals, in ''Topics in Optimization" (G. Leitmann, ed) pp.63-101. Academic Press, New York. Maurer, H. (1974). An Example of a Continuous Junction for a Singular Control Problem of Even Order, Department of Mathematics, Univ. British Columbia, Vancouver, Canada. McDanell, J. P. and Powers, W. F. (1970). New JacobiType Necessary and Sufficient Conditions for Singular Optimization Problems, AIAA J. 2, 1416-1420. McDanell, J. P. and Powers, W. F. (1971). Necessary Conditions for Joining Optimal Singular and Nonsingular Subarcs, SIAM J. Control 9, 161-173.

-

Mehra, R. K. and Davis, R. E. (1972). A Generalized Gradient Method for Optimal Control Problems with Inequality Constraints and Singular Arcs, IEEE Trans. autom. Control AC-17, 69-79.

-

5.

171

COMPUTATIONAL METHODS

Moore, J. B. (1969). A Note on a Singular Optimal Control Problem, Automatica 5, 857-858. Moylan, P. J. and Moore, J. B. ( 1 9 7 1 ) . Generalizations of Singular Optimal Control Theory, Automatica L, 591-598.

Powers, W. F. and McDanell, J. P. ( 1 9 7 1 ) . Switching Conditions and a Synthesis Technique for the Singular Saturn Guidance Problem, J. Spacecraft Rockets 8, 1027-1032.

-

Speyer, J. L. and Jacobson, D. H. (1971). Necessary and Sufficient Conditions for Optimality for Singular Control Problems; A Transformation Approach, J. math. Analysis Applic. 33, 163-187. Wonham, W. M. and Johnson, C. D. ( 1 9 6 4 ) . Optimal BangBang Control with Quadratic Performance Index, Trans. ASME, J. Basic Eng. 8 6 , 107-115.

-

-

This page intentionally left blank

CHAPTER 6 Conclusion 6.1

The Importance of S i n g u l a r Optimal Control Prob l e m s The word " s i n g u l a r " i n t h e c o n t e x t of optimal

control theory has i t s o r i g i n s i n t h e c l a s s i c a l calculus of v a r i a t i o n s i n which a v a r i a t i o n a l problem i s r e f e r r e d t o as s i n g u l a r i f t h e Legendre-Clebsch n e c e s s a r y condition i s not s a t i s f i e d with strict inequality.

I n con-

temporary c o n t r o l t h e o r y we are c o n s i s t e n t w i t h t h i s d e f i n i t i o n of s i n g u l a r i t y i n r e f e r r i n g t o t h e second v a r i a t i o n as t o t a l l y s i n g u l a r i f Huu = O

f o r a l l t i n [ tO ' t f l ,

(6.1.1)

as p a r t i a l l y s i n g u l a r i f HUU

-> o

for a l l t i n [t ,t o f

3,

(6.1.2)

and as n o n s i n g u l a r i f Huu > O

for all t in [to,tfl.

(6.1.3)

Thus, s i n g u l a r optimal c o n t r o l problems are " s i n g u l a r " i n a mathematical s e n s e ; i . e . det(HUu) = 0.

It t u r n s

o u t , though, t h a t s i n g u l a r problems a r i s e f r e q u e n t l y i n e n g i n e e r i n g and are, t h e r e f o r e , n o t a t a l l " s i n g u l a r " o r uncommon i n t h e realm of a p p l i e d optimal c o n t r o l theory.

It was t h i s r e a l i z a t i o n , which probably began

w i t h Lawden's r o c k e t problem, which s t i m u l a t e d t h e I73

SINGULAR O P T I M A L C O N T R O L PROBLEMS

174

l a r g e r e s e a r c h e f f o r t on t h e s e problems d u r i n g t h e 1 9 6 0 ' s and e a r l y 1 9 7 0 ' s and which s t i l l m o t i v a t e s u s t o develop e f f i c i e n t computational techniques. The importance of s i n g u l a r o p t i m a l c o n t r o l problems

is therefore evident.

On t h e one hand, s i n g u l a r

problems e x h i b i t many i n t e r e s t i n g and deep t h e o r e t i c a l n i c e t i e s and on t h e o t h e r hand t h e y a r i s e , and a r e t h e r e f o r e of p r a c t i c a l s i g n i f i c a n c e , i n e n g i n e e r i n g and other disciplines.

Our a i m i n t h i s monograph, t h e r e -

f o r e , i s t o g i v e b o t h s i d e s of t h e s t o r y by r e f e r r i n g i n C h a p t e r s 1-3

t o c e r t a i n of t h e s i n g u l a r problems o f

p r a c t i c a l importance, i n Chapter 5 t o computational t e c h n i q u e s f o r t h e i r s o l u t i o n , and i n C h a p t e r s 2-4

to

n e c e s s a r y , and n e c e s s a r y and s u f f i c i e n t c o n d i t i o n s f o r optimality.

I n t h e remainder of t h i s c h a p t e r w e d i s c u s s

c e r t a i n a s p e c t s of t h e r e s u l t s p r e s e n t e d i n C h a p t e r s

1-5 i n o r d e r t o s t r e s s t h e i r r a t h e r c e n t r a l t h e o r e t i c a l and p r a c t i c a l r o l e s . 6.2

Necessary C o n d i t i o n s I t i s w e l l known t h a t t h e a p p l i c a b i l i t y of

P o n t r y a g i n ' s P r i n c i p l e i s u n a f f e c t e d by s i n g u l a r i t y , o r lack thereof.

However, i n t h e ( p a r t i a l l y ) s i n g u l a r

i s n o t i n v e r t i b l e and t h e r e f o r e t h e s u f f i c i e n t uu c o n d i t i o n s developed e s p e c i a l l y f o r t h e n o n s i n g u l a r case H

case a r e inapplicable.

T h i s l a c k of s u f f i c i e n t condi-

t i o n s meant t h a t t h e o p t i m a l i t y , o r l a c k t h e r e o f , of Lawden's s p i r a l i n t h e a e r o s p a c e f i e l d c o u l d n o t b e

6.

CONCLUSION

175

checked and this stimulated researchers in the quest for further necessary conditions.

Research then

yielded the generalized Legendre-Clebsch necessary condition of Goh, Kelley et al., and Robbins, which decided the fate of Lawden's spiral.

It is rather

interesting to note that Kelley et al. utilized special control variations of complicated nature to yield the generalized Legendre-Clebsch condition, but these researchers neglected to investigate further the affect of the simplest variation

-

a rectangular pulse.

Presumably, they reasoned that this simple variation, which yields the classical Legendre-Clebsch condition in the nonsingular and partially singular cases, could not yield any additional useful information. Remarkably, use of this simple variation does yield additional results as evidenced by Jacobson's necessary condition which emerged circa 1969 and which paved the way for the development of necessary and sufficient conditions. A further consequence of the search for additional

necessary conditions was the development of Kelley's and Goh's transformation techniques. These, in turn, played an important role in the development of necessary and sufficient conditions. 6.3 Necessary and Sufficient Conditions In the (partially) singular control problem the conventional Riccati differential equation is not -1

defined, owing to the presence of Huu in its right hand

176

SINGULAR OPTIMAL CONTROL PROBLEMS

side.

F o r a long time, then, it was felt that no

sufficiency theory analogous to that for nonsingular problems could be expected in the singular case.

This

is, in retrospect, a somewhat surprising conclusion to reach as Riccati-like inequalities were already known in network theory in connection with passivity which is closely related to the non-negativity of a singular quadratic functional. Shortly after the advent of Jacobson’s necessary condition a set of algebraic and differential inequalities, closely related to conditions in network theory, were developed which are sufficient for non-negativity of the singular second variation.

At this point in

time the quest for conditions which are both necessary and sufficient intensified.

Two different approaches

were used; namely, a limit approach of Jacobson and Speyer and the transformation approaches of Goh, Powers, and Speyer and Jacobson. The limit approach of Jacobson and Speyer, fully detailed in Chapter 4 , converts the singular second variation into a nonsingular one by the addition of the term

2 21 E

uTud t t0

.

The conventional Riccati equation for this nonsingular problem is then studied in the limit as

E-,

yielding

6. CONCLUSION

177

necessary and sufficient conditions for non-negativity of the original singular variation.

This limit approach

has proved to be rather powerful in that minimal differentiability assumptions are required.

Furthermore, all

known necessary conditions can be recovered by manipulating, in a rather simple way, these conditions.

In

certain cases it is possible to exhibit the necessary and sufficient conditions in Riccati differential equation form thereby demonstrating the close relationship between singular and nonsingular problems. The transformation approaches are designed to transform the singular problem into a nonsingular one; the conventional necessary and sufficient conditions (conventional Riccati equation) are then applicable. In the Speyer and Jacobson approach, Kelley's transformation technique converts the singular problem into a nonsingular one in a state space of reduced dimension. Goh's transformation, on the other hand, as exploited by McDanell and Powers, preserves the dimensionality of the original state space. A s Goh's transformation is simpler to apply than Kelley's we use it here to illustrate the role of transformation techniques in the development of the Riccati equation for singular problems which is stated and used, but not derived, in Chapter 4 . We consider the totally singular quadratic prob 1em,

178

SINGULAR OPTIMAL CONTROL PROBLEMS

J[u(.)]

=

I t f ( & x T Q x +TuCx)dt + i xT ( t , ) Q f x ( t f )

(6.3.1)

where j , = Ax+Bu ;

x(t ) = x 0

0

.

Goh's t r a n s f o r m a t i o n technique i s a s follows.

(6.3.2) First,

define

(6.3.3) and z = x-Bv

.

(6.3.4)

Then, we o b t a i n e a s i l y t h e d i f f e r e n t i a l e q u a t i o n ,

H

= Az

+ (AB-6)v ; z(t,)

= 0

(6.3.5)

and T T T T T T *T J [ v ( * ) ] = I t f ( i z Qz+v B Qz+iv B QBv+; Cz+v CBv)dt

(6.3.6) I n t e g r a t i n g the terms i n yields

by p a r t s and r e - a r r a n g i n g

6 . CONCLUSION

179

T

+ icT[CB-(CB) ]v)dt

+ i vT (t,) [CB+BTQfBlv(tf) (6.3.7) Now, i f the f i r s t generalized Legendre-Clebsch neces-

sary condition is s a t i s f i e d i n s t r o n g form, i . e . CB = (CB)

T

(6.3.8)

and (6.3.9) and i f we assume t h a t

T

C + B Q f = O

(6.3.10)

at tf,

then, ~[V J [ v ( * ) ] = I t f i i z TQz+vT (BTQ - C A - ~ ) Z +~a t0

-H au

u

]v)dt

SINGULAR OPTIMAL CONTROL PROBLEMS

180

which is a standard, nonsingular quadratic functional. A necessary and sufficient condition for strong positivity for (6.3.11) is, then, that there exists S ( - ) which satisfies T

C+B S(tf)

=

0

(6.3.12)

T T T T *T T -~=Q+sA+AS+(B Q-CA-~+BA S-B s>

a H.. -1 (BTQ - c A - ~ + B ~ A ~ s - ~ ~ s ) au u

(6.3.13)

where

On the application of Kelley's transformation rather than Goh' s, Speyer and Jacobson's Riccati equation which is of smaller dimension, but otherwise equivalent, is obtained. If it turns out that the first generalized Legendre-Clebsch condition is not satisfied in strong form the transformation techniques have to be applied repeatedly until one of the higher-order generalized Legendre-Clebsch conditions is satisfied in strong form. This can only be done if the system equations and performance integrand are many times differentiable; a significant disadvantage which is not present in the more direct limit approach of Chapter 4.

6.

6.4

CONCLUSION

181

Computational Methods Computational problems i n s i n g u l a r c o n t r o l f a l l

r a t h e r n a t u r a l l y i n t o two areas.

The f i r s t i s t h e

computation of optimal s i n g u l a r c o n t r o l s and t h e second

i s t h e c o n s t r u c t i o n of c e r t a i n m a t r i x f u n c t i o n s of t i m e i n n e c e s s a r y and s u f f i c i e n t c o n d i t i o n s which check a particular singular control function f o r optimality. It t u r n s o u t t h a t t h e l i m i t approach, i n which t h e

s i n g u l a r problem i s made n o n s i n g u l a r by t h e a d d i t i o n s of 22 El t f u T u d t

,

t0

i s u s e f u l i n both of t h e s e problem areas.

Indeed, i t

i s worth mentioning t h a t Powers and McDanell have r e p o r t e d u s i n g t h i s method s u c c e s s f u l l y i n a s i n g u l a r S a t u r n guidance problem.

D e t a i l s of t h e l i m i t approach

are given i n Chapter 5 where i t i s p o i n t e d o u t t h a t t h i s i s t h e o n l y g e n e r a l method c u r r e n t l y a v a i l a b l e . However, t h e r e c e n t experiments of Edge and Powers i n v o l v i n g quasi-Newton methods i n f u n c t i o n s p a c e are encouraging and augur w e l l f o r t h e f u t u r e .

6.5

Switching Conditions The j o i n i n g of s i n g u l a r and n o n s i n g u l a r sub-arcs

i s i m p o r t a n t , and n o t y e t f u l l y understood.

However,

McDanell and Powers have provided a number of u s e f u l , g e n e r a l , theorems and Jacobson h a s o b t a i n e d s a t i s f a c -

182

SINGULAR O P T I M A L C O N T R O L P R O B L E M S

tory switching strategies in a class of quadratic problems.

6.6 Outlook for the Future It seems to us that the areas of optimal (singular) control which require greatest attention are those which involve the computation of optimal controls and the joining of singular and nonsingular segments.

The

experiments of Edge and Powers in control function computation and the work of Anderson and Moylan in network synthesis are encouraging. In closing, we mention the recent work of Moyer in connection with sufficient conditions for a strong minimum in singular optimization problems.

This paper

could stimulate further developments in optimal control theory

.

AUTHOR

INDEX

Desai, M. N. , 14, 29 Dobell, A. R., 5, 30 D'Souza, A. F. , 5, 31

A Anderson, B. D. O . ,

11,

21, 24, 28, 34, 103,

E

, 154, 165, 166, 169, 170, 181,

A r c h e n t i , A. R . , 12, 28 A r i s , R., 38, 59, 69, 70,

Edge, E. R.

72, 99

Athans , M.

182

, 11, 29

F

B

19, 30, 101, 104, 112, 130, 149

Fomin, S. V . ,

Bass, R. W . , 10, 29 B e l l , D. J . , 8, 13,

16, 17, 29, 54, 59, 71, 72, 76, 98

G

1, 3, 29, 37, 39, 42, 46, 59,

1 7 , 28, 30, 92, 98, 138, 149 Gamkrelidze, R. V., 9'7, 99 Gelfand, I. M., 19, 30, 101, 104, 112, 130,

B l i s s , G. A.,

97, 98

Bolonkin, A. A . ,

29

17, 18,

B o l t y a n s k i i , V. G . ,

99

Breakwell, J. V.

142, 149

Gabasov, R . ,

97,

149

, 17, 29,

Gershwin, S. B . ,

23, 32,

154, 165, 170

B r o c k e t t , R. W., 102, 149 Bryson, A. E . , 14, 29,

Gibson, J. E . , 11, 32 Goh, B. S . , 3, 5, 14,

16,

17, 18, 19, 22, 30, 31, 62, 81, 88, 98, 102, 137, 140, 141, 149 Graham, J. W. , 5, 31 Gurman, V. I., 17, 31

53, 59, 67 C

Canon, M. D . , 11, 29 Coddington, E. A., 109,

111, 131, 149 5, 30

H

Connor, M. A.,

Haynes, G. W. , 10, 1 7 , 31 Hermes, H. , 10, 12, 31

D Davis, R. E . , 154, 165,

5, 29, 30, 539 59, 142, 149

Ho, Y* C.,

170 I83

AUTHOR INDEX

184

L

H o f f m a n , W. C., 14, 29

J

4, 15, 18, 19, 22, 23, 24, 25, 26, 28, 31, 32, 36, 62, 81, 88, 91, 92, 94, 98, 100, 102, 113, 116, 123, 137, 138, 140, 141, 142, 149, 150, 151, 153, 154, 157, 160, 161, 162, 164, 167, 170, 171, 176, 177, 180,

Jacobson, D. H.,

181

J o h m s e n , D. E. , 19, 32, 101, 150, 154, 170 Johnson, C. D . , 10, 11,

27, 32, 36, 154, 167, 171 K

Kalman, R. E . , 142, 150 Keller, J. L. , 15, 32 Kelley, H. J., 13, 15, 17, 22, 27, 32, 33, 61, 63, 67, 69, 72, 81, 99, 138, 150, 167, 170, 175

Kirillova, F. M., 28, 30 Kopp, R. E., 13, 14, 15, 27, 33, 61, 63, 69, 72, 81, 99, 138, 150, 167, 170 Krener, A. 3 . , 16, 33, 97, 99

Lawden, D. F.,

6, 8, 12,

33, 38, 47, 48, 49,

59, 72, 75, 80, 99

Leitmann, G. , 12 Lele, M. M. , 23,

165, 170

Levinson, N.

32, 154,

, 109, 111,

131, 149 M

M a n c i l l , J . D. , 19, 33 Marchal, G. , 28, 33 Maurer, H., 154, 168, 170 Mayne, D. Q., 15, 19, 27,

34, 153, 154, 160, 161, 162, 170 McDanell, J. P. , 21, 2 2 , 27, 34, 35, 88, 99, 102, 122, 140, 141, 150, 154, 156, 165, 167, 170, 171, 177, 181 Mehra, R. K . , 154, 165, 170 Miele, A . , 8, 9 , 17, 34 Mishchenko, E. F., 97, 99 Moore, J. B., 11, 21, 28, 34, 35, 154, 167, 171

13, 15, 26, 27, 33, 34, 61, 63, 69, 72, 81, 99, 138, 141, 150, 167, 170, 182 Moylan, P. J . , 11, 35, 154, 157, 167, 1-69, 171, 182

Moyer, H. G . ,

AUTHOR INDEX P

,

Pontryagin, L. S . 97, 99 Powers, W. F., 21, 22, 27,

34, 35, 88, 99, 102,

122, 140, 141, 150,

154, 156, 165, 166, 167, 169, 170, 171, 177, 181, 182

Rao, S . K., 5, 36 Ray, W. H., 5, 35 Robbins, H. M., 5, 14,

17, 35, 61, 63, 99, 137, 138, 151 35

32, 35, 36, 62, 81, 88, 91, loo, 102, 113, 123, 137, 140, 141, 150, 151, 157, 171, 176, 177, 180 S t e i n b e r g , A. M., 5, 36 Strochko, V. A., 28, 30 T

13, 19, 36, 101, 113, 138, 151 T a r n , T. J . , 5, 36

T a i t , K. S . ,

R

Rohrer, R. A . ,

185

11, 21,

S

14, 35, 88, 90, 91, 99 S i e b e n t h a l , C. D., 38, 59, 69, 70, 72, 99 Silverman, L. M . , 21, 35

S c h u l t z , R. L.,

S i r i s e n a , H. R . , 11, 35 Snow, D. R . , 12, 35 Sobral, M., 11, 21, 35 Soliman, M. A., 5, 35 Speyer, J. L., 14, 15,

22, 23, 24, 25, 26,

V

Valentine, F. A . , 40, 59 Vapnyarskiy, I. B., 17,

18, 36

Vinh, N. X., 12, 28 V i n t e r , R. B., 26, 36 W

Webber, R. F . , 10, 29 Wonham, W. M., 10, 36,

154, 167, 171 Z

Zaborszky, J . , 5, 36 Zagalsky, N. R., 1 4 , 35

88, 90, 91, 99

This page intentionally left blank

SUBJECT INDEX

t e r m i n a l , 1, 7, 16, 1 9 t r a n s v e r s d i t y , 45 46

A

Accessory problem, 4, 17, 21, 53, 56, 85, 87 Adjoint e q u a t i o n , 94 v a r i a b l e , 70, 89, 97 v e c t o r , 45 Admissible r e g i o n , 9 U g o r i t h m , 38, 153, 154,

Conjugate g r a d i e n t , 166 p o i n t , 53 C o n t r o l l a b i l i t y 104 106, 112 Cost f u n c t i o n a l , 1, 'j'ff.,

,

B

D D i f f e r e n t i a l , first , 44, 49, 50 second, 44, 50ff. D i f f e r e n t i a l Dynamic Programming, 27, 154, 160ff.

Bang-bang a r c , 9, 26, 160 Bolza problem, 1, 3, 40

C Condition, boundary, 43,

48, 63, 74, 77

Clebsch, 46, 49, 63, 87 existence, 142ff. J a c o b i , 22, 53 Jacobson, 16ff., 61, 62,

175, 176

j u n c t i o n , 27, 154, 167ff. , 181, 182 n e c e s s a r y , 1 2 f f . 21ff.

,

38, 55, 61ff., 85, 101, 102, 123, 13b, wff.,155, 156, 174ff.

,

E

,

E-algorithm, 1 6 0 f r . 169 convergence o f , 162 Economics, 5, 61 E x t e r i o r c a l c u l u s , 10 F

F i e l d t h e o r y , 26, 1 4 1 F l i g h t mechanics, 9 , 47,

n e c e s s a r y and s u f f i c i e n t ,

5, 13, 1 9 2 ~ ,. l o i f f . , , 168, 175ff. s u f f i c i e n t , 13, l g f f . , 38, i o i f f , , 124, 136, 1 2 2 f f . , 140ff.

141ff.

,

41ff., 49, 57, 75, 160

159

giff.

, ,

51, 97

, 182

167

54, 88, 165, 181

Function space method,

154, 165ff., 181, 182

convergence o f , 166 Fundamental lemma, 46

SUBJECT INDEX

188

L i m i t approach, 23, 28,

G

Gradient method, 154, 165 Green's theorem, 8, 9

157ff. , 168, 169, 176, 177, 181 M

H

Hamiltonian, 2 , 3, 7, 15,

44, 48, 57, 58, 70, 75, 89, 90, 94 Hohmann t r a n s f e r , 7, 8

15, 72, 74, 88, 89

I l l - c o n d i t i o n i n g , 164 Intermediate-thrust a r c ,

8, 1 2 , 17, 38, 72, 74

I n t e r p l a n e t a r y guidance, 17, 27 Isoperimetric constraint ,

8

s t r o n g , 24, 26, 141, 182 weak, 19, 20, 24, 101, 103, 140 N

Network t h e o r y , 21, 182 Newton method, 158, 165,

181

K

Nonsingular a r c , 3, 1 2 ,

Kelley-Contensou t e s t , 1 3 L

Lagrange m u l t i p l i e r , 3,

43, 127, 128

problem 1 Lawden's s p i r a l , 1 2 , 1 5 ,

17, 72, 80, 174, 175

Legendre- Cleb sch condition, 3, 1 2 , 15, 61, 81, 82, 96, 104,

173, 175

37

Minimum, fuel, 7, 8, 1 2 ,

I

generalized, 1 2 f f .

M a x i m a l p r i n c i p l e , high o r d e r , 16, 97 Mayer problem, 1 Miele's method, 8 f f . , 17,

,

62ff., 9 3 f f . , 119, 140, 157, 167ff. , 175, 179, 180

13, 27, 154, 166

control, 5 f u n c t i o n a l , 129, 161 problem, 19, 22, 25, 61, 63, 95, 96, 101,

140, 161, 174, 176

Normality, 16, 67, 97 0

Order of s i n g u l a r a r c , 4, 13, 1 5 , 23ff. P

Passivity, 21 Pontryagin's p r i n c i p l e , 2 , 16, 46, 49, 61,

SUBJECT INDEX

4 , 7 f f . , 61, 70, 75, 153

72, 9 4 f f . , 174 P o s i t i v e real matrix, 21 Process c o n t r o l , 1 7

control,

p a r t i a l l y , 4 , 5, 24, 26,

175

Q

problem, 20, 22, 38, 39, 56ff., 154, 173,

Q u a s i - l i n e a r i z a t i o n , 158

177

R

Relaxed problem, 5 Resources , n a t u r a l , 5 R i c c a t i equation, 1 9 , 22, 25, 53, 96, i o i f f . , 113, 119, 130, 1 4 0 f f . , 156, 157,

168, 1 7 5 f f . , 180

Second v a r i a t i o n , nonnegative, 2 0 f f . , 39, 52, 55, 63, 80, 91, 93, l O l f f . , 115ff., 130ff. , 153ff. n o n s i n M a r , 102, 106, U T f f . , 1 5 5 f f . , 173 p a r t i a l l y s i n g u l a r , 102, 105, 140, 173, 174 positive definite, l 0 5 f f . 130 s i n g u l a r , 18, 23, 25, 81, 87, 101, 102,

,

1 4 1 , 157

173

Singular arc, 3, 9 , 1 2 , 13, 27, 58, 72, 91, 154, 160, 161, 166

s o l u t i o n , 94, 95, 97 s t r i p , 10, 11 s u r f a c e , 26 t o t a l l y , 4, 5, 2 1 f f . , 26, 102, 177ff. Space n a v i g a t i o n , 6 f f . , 12, 72 S p e c i f i c energy, 88 S t i e l t j e s i n t e g r a l , 136 S t i r r e d t a n k r e a c t o r , 38,

69ff.

S

strongly p o s i t i v e , 19, l O l f f . , 1 4 1 , 180 t o t a l l y s i n g u l a r , 103, 1 1 2 f f . , 137, 1 5 5 f f .

189

,

S t o k e s ’ s theorem, 10 System, aerospace, 9 , 154 delay, 5 discrete, 5 e c o l o g i c a l , 61 T

Terminal c o n s t r a i n t , 43, 57, 66, 69, 82, 83, 97, 137 Transformation, canonical,

15

o f Goh, 1 6 , 21, 62, 81ff., 140, 156,

174ff.

,

of Kelley, 1 5 f f . 61ff. 81, 140, 157, 175, 177, 180

,

V Valentine technique , 40 V a r i a t i o n , admissible, 52,

SUBJECT INDEX

190

77

c o n t r o l , 1 3 , 15, 1 6 , 4 2 ,

7 5 6 1 f f . , 93, 175

e q u a t i o n s o f , 1 6 , 43,

53, 54, 63, 65, 75, 76, 82

f i n a l t i m e , 42

f i r s t , 39, h l f f . , 75 second, 4, 13, 1 5 , 1 6 ,

1 9 , 37ff., 49ff., 6 5 f f . , 7 6 f f . , 92,

10lff. s t a t e , 1 6 , 42, 56, 63,

66, 72ff.