Optimization of Discrete Time Systems: The Upper Boundary Approach

Lecture Notes in Control and Information Sciences Edited by A.V. Balakrishnan and M.Thoma

51 IIIII

III

Zbigniew Nahorski Hans E Ravn Ren6 V.V.Vidal

Optimization of Discrete Time Systems The Upper Boundary Approach IIII

Springer-Verlag Berlin Heidelberg New York Tokyo 1983

Series Editors A.V. Balakrishnan • M. Thoma Advisory Board L D. Davisson • A. G. J. MacFarlane • H. Kwakernaak J. L. Massey • Ya. Z. Tsypkin • A. J. Viterbi

Authors Dr. Zbigniew Nahorski Systems Research Inst. The Polish Academy of Sciences ul. Newelska 6 01-447 Warsaw Poland Dr. Hans E Ravn Danish Energy Agency 11 Landemaerket DK-1119 Copenhagen K Denmark Dr. Ren6 V.V.Vidal The Institute of Mathematical Statistics and Operations Research The Technical University of Denmark DK-2800 Lyngby Denmark

ISBN 3-540-12258-3 Springer-Verlag Berlin Heidelberg NewYork ISBN 0-387-12258-3 Springer-Verlag NewYork Heidelberg Berlin This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically those of translation, reprinting, re-use of illustrations, broadcasting, reproduction by photocopying machine or similar means, and storage in data banks. Under § 54 of the German Copyright Law where copies are made for other than private use, a fee is payable to 'Verwertungsgesellschaft Wort', Munich. © Springer-Verlag Berlin Heidelberg 1983 Printed in Germany Printing and binding: Beltz Offsetdruck, Hemsbach/Bergstr, 206113020-543210

PREFACE

The development of numerical methods for solving optimization problems has taken place with increasing speed along with the development of the electronic computer.

The underlying cause

for this to happen has been that a large number of technical and operational problems have shown themselves to be suitable for formulating and analyzing within the framework

of

optimi-

zation theory. However, the speed of development has been unequal for the different classes of methods.

Thus in the 1960s fundamental theo-

retical results were published in the area of control problems: the Maximum Principle and the Principle of Optimality.

There-

after numerical methods were developed and implemented, based on these principles and related theorems. In later years the main development within optimization theory has taken place in other areas,

leaving an impression that the

basic theoretical foundation for dealing with control problems had been successfully explored and mapped.

We do not share this

view.

In the present book we present a new approach to deal with the control problems

(specifically, the discrete time ones) which

indicates the possibility of exposing the fundamental theorems not only from control theory, but also from adjoining areas of mathematical programming.

But further it provides a convenient

base for formulating new and fundamental results,

in this book

exemplified by the theorem of the Nonlinear Maximum Principle. We call the approach the Upper Boundary Approach.

IV While thus the book sketches some promising perspectives and documents a number of new results, it does so, we admit, in a preliminary form. Rather than elaborating on presentation, we gave priority to the quick communication of the results.

The

reader will hopefully agree with us in the strategy chosen! The results presented here have their specific history in several years of research work at IMSOR, the Institute of Mathematical Statistics and Operations Research, The Technical University of Denmark.

Here Associate Professor, Lic. Techn. Ren~ Victor

Valqui Vidal in his lectures on optimization theory and applications pointed to some problems in the perception of basic properties and concepts in the theoretical foundation of the theory. They were basically the same in all areas of optimization theory, and not yet quite!

M.Sc. Hans Ravn, presently at the Danish

Energy Agency, at the time at IMSOR, was involved in the problems and worked with Vidal until the contours of the Upper Boundary Approach could be seen.

At this point Dr. Z. Nahorski,

of the Polish Academy of Sciences, joined the project and was main responsible that the ideas

(including some of his own} got

a firm shape at all.

We are very grateful that it was possible for the Danish Technical Research Council to support the later phase of the research economically.

Also we thank M.Sc. N.O. Olesen who con-

tributed during the later phases.

Last, but not least, we are

grateful to Miss Bente Wilkenschildt for her work with the preparation of this manuscript.

Copenhagen and Warsaw, July 1982 Z. Nahorski, H.F. Ravn, R.V.V. Vidal

CONTENTS

Chapter 1

INTRODUCTION

I

1.0

Introduction

3

i.i

Simple examples

4

1.2

The multistage optimization problem

12

1.3

Historical notes

14

1.4

The scope of the book

18

1.5

Literature

19

ONE-STAGE SYSTEMS

25

Literature

47

MULTISTAGE SYSTEMS

49

Literature

87

COMPUTER A L G O R I T H M

91

Chapter 2 2.1 Chapter 3 3.1 Chapter 4 4.1

Introduction

93

4.2

The general idea of the algorithm

93

4.3

The algorithm

4.4

Numerical examples

102

4.5

Complementary remarks

128

4.6

Literature

129

CONCLUSIONS AND FURTHER RESEARCH

133

Chapter 5

98

CHAPTER

I

INTRODUCTION

1.0

Introduction

Optimization

of discrete time systems is an activity which fre-

quently takes place as one of the central

steps in the design

process, when solving certain technical problems, stage problems. speaking, best

The purpose of this activity

called multi-

is, generally

to find a combination of parameter values, which will

(in some specified sense)

values, a mathematical

solve the problem.

model is constructed,

proposed solution to the problem. the multistage

the

of the discrete time

system.

Solution methods for the optimization is the subject of this book.

representing

The search for good parameter

values is taking place as an optimization model representing

To find these

of discrete time systems

We propose a new approach to this

area, the upper boundary approach,

which will allow us to derive

new and important results, while at the same time restating classical results within the same terminology. This first chapter serves as an introduction and to the book.

We shall therefore

examples of technical

and operational

to have some common properties

to the problem area

first give a number of problems, which can be seen

in their mathematical

representa-

tion. A subclass of these socalled multistage

optimization

will be identified.

and precision of this

After a discussion

subclass, we give a short outline of the historical

problems development

of solution methods for it, with emphasis on Dynamic Programming and the Maximum Principle.

1.1

Simple examples

Let us look to some problems which arise naturally in technical systems. I° Multistage Compression of a Gas A gas is to be isentropically compressed from the initial pressure Po to a final pressure PN" The compression proceeds in N stages.

In each stage the gas is first adiabatically compressed

and then isobatically cooled to its initial temperature. The energy consumption at the ith stage is given by

E i = mRT ¥/(y-I)[

(xi/xi_ I) (Y-I)/¥-I]

where m- the number of moles of gas compressed R- the universal gas constant T- the initial temperature of the gas ¥- the ratio of the specific heat of the gas at constant pressure to that at constant volume

(assumed to be

constant) xi-pressure of the gas at the end of the ith compression It is desired to determine the interstage pressure for which the total energy consumed in compression is minimal. variable in the ith stage u i is defined as

ui = xi+I/Xi then we can formulate our p r o b l e m as

N-I

min i=O

UOI---IUN_ I

u(Y-1 )/7 1

If the decision

subject

to

xi+1

= x.u. i i

Xo

= Po

XN

= PN

2° Transportation

Resources demand

i=0,1,...,N-I

Problem

are to be t r a n s p o r t e d

points

(sinks),

is only one type of r e s o u r c e to the total

from n depots

see Fig.

1.1.

and that

Demand

supply

points

• U

2

the total

J

•

uj 2 U..

i

UjN N

n

Fig.

•

1.1

A scheme

for t r a n s p o r t a t i o n

to N

that

demand.

Depots I

(sources)

It is s u p p o s e d

problem.

there

is equal

Let u~ - the q u a n t i t y i

of the resource

depot to the ith

sent from the jth

demand p o i n t

r~1 (u~)- the cost of this operation The p r o b l e m

is to determine

i=1,2,...,N

to minimize

the quantities

of u~l, j=1,2,...,n;

the total costs of transporting

the re-

sources

R

--

N ~

n ~

4 4 r~(u4J )

i=i j=1 subject to the c o n s t r a i n t s

u~>0 I -N

Z

U~ = W j, the supply of the resource

i=I En

jth depot, u~ = di, the demand

j=l

point,

j=1,2,...,n

at the

j=1,2,...,n for the resource

at the ith demand

i=1,2,...,N i=1,2,...,N

We can define

state v a r i a b l e s

which has been t r a n s p o r t e d mand points.

available

x~ as the total amount of resources l from the jth depot to the first i de-

Then we can write the following

equation

X~ = X j + U~ l i-I i xJ = 0 O j=1,2 .... ,n-1

XJ N = wJ

i=l,2,...,N

It s h o u l d

be n o t e d

there

n depots.

are

each d e m a n d nth d e p o t

point

there

This

mand by the

from

is p r e a s s i g n e d .

by all

rest

point

n = di _ ui

n-I E u3." i 9--I

to w r i t e

define

=

of n-1

the

2

above

. . ,x i

]

ri = [r11., r 2, .... rn] T w j = [w I, w 2, .... wn-1] T

N o w the p r o b l e m

c a n be

formulated

as

N

min Z u 1 , . . . , u N i=I subject

r i (u i)

to

xI

•

=

Xo

= 0

XN

= w

xi- I

U. > 0 , i --

÷

u.

di -

i = 1,2,...,N

the

although

the d e m a n d

supplies

the s u m of depots

problem

n-l~T

xi,-

variables

that

the

f r o m the

from

in the

supplies total

de-

d i i.e.

the v e c t o r s

I [x i,

state

fact

Therefore

2 n-l~T U i = [U 1, u i , . . . , u i J

xi

n-1

the

by subtracting

the

ith d e m a n d

convenient

Let us t h e n

are o n l y

arises

c a n be o b t a i n e d

of the r e s o u r c e

It is

that

n-1 Z u3"i -> 0 i--I

in v e c t o r

notation.

The last two constraints define for each i a region in the n-1 dimentional control space usually referred to as a set of admissible or feasible controls. n

~ j=l

U j. = d. 1 l

i=1,2,...,N The last five constraints define a region in the n-1 dimentional control space usually referred to as a set of admissible or feasible controls. 3 ° Catalyst replacement In a catalytic reactor the efficiency of the process gradually decreases as the catalyst gets older. Because of this, the best operating conditions change in time. The problem then is to find the best operating conditions

in some periods of time and the

best time for replacing the catalyst so as to obtain the maximum profit. Let us consider a system depicted in Fig. sists

1.2 which con-

of a tubular reactor and a distillation tower. In the

period i a material is feeded to the reactor with constant flow rate through the reactor F i. In the reactor a compound A cracks

I (I -C i )F i

i11

M.

1

O

FIJ TO1

Fi

O

Catalytic reactor Ti

I I I ei

I Fig.

1.2

Schematic flow sheet.

CiFi

to c o m p o u n d s

B and G. T h e n t h e y go to the d i s t i l l a t i o n

which the c o n v e r t e d m a t e r i a l CiF i and u n c o n v e r t e d recycled material (with rate Mi) have

(final product)

material

is r e c y c l e d w i t h rate

is added to the m a t e r i a l

=

(1-Ci)F i. The

f e e d e d to the p r o c e s

and inleted to the reactor.

Mi

tower f r o m

flows w i t h the rate

By m a t e r i a l

b a l a n c e we

C.F.

(1.1)

i i

(1.2)

F i = M i + (1-Ci)F i The c o n v e r s i o n

is a s s u m e d to be e x p r e s s e d

as

(1 .3)

C A = alTi-a2Fi-a3S i where T i is the exit t e m p e r a t u r e

in p e r i o d

tive flow rate t h r o u g h the catalyst,

Si =

S i represents

i and S i is the c u m u l a -

i.e.

i Z Fi j=l

(1.4)

the state of the s y s t e m w h i c h

age of catalyst, lance we have

is e q u i v a l e n t

al,a 2 and a 3 are g i v e n constants.

Qi = Fi Cp(Ti-To) where Qi is the heat input to the r e a c t o r reaction,

and T o is the t e m p e r a t u r e

(I .5)

+ hCiFi

in p e r i o d i, Cp is the

average heat c a p a c i t y of the r e a c t i n g mixture, reactor

to the

By e n e r g y ba-

h is the heat of

of the m i x t u r e

entering

the

(assumed constant).

The conversion, constraints

temperature,

and flow rates are subject to the

Cmi n _< C i <_ Cma x T m i n _< T

< Tma x

F



10

The p r o f i t

obtained

per

unit

time

is d e f i n e d

r i = CiFiv I - Miv 2 - Qiv3

-

as

(1-Ci)Fiv4-v5

where

v I - combined

value

of p r o d u c t s

v 2 - cost

of the

v 3 - cost

of h e a t i n g

v 4 - cost

of p r o c e s s i n g

distillation v 5 - fixed

B and G

feed

the

recycle

stream

through

the

tower

charges.

L e t us d e f i n e

a state

variable

xi = Si and the vector

Then

of d e c i s i o n

from equation

11.41

variables

we have

2 x i = xi_ I + u i and

from equations

r i ( x i , u i)

=

(1.1),

i=1,2,...,N (1.3),

the p r o b l e m

and

(alu.1 l - a 2 u i2 - a3xi) u2 2 ,u I - CpUi( i - To)V3

Then

(1.5)

c a n be

formulated

2 - uiv4

as

(1.6)

(v1_v2_hv3+v4)

- v5

follows

(I .6)

11

N

max u I,...,uN,N subject

r i ( x i , u i) i=1

to 2

x i = xi_ I + u i x

Cmin

o

= 0

~ a l u iI _ a 2 u ~

T m i n -< u i1 F

All the a b o v e (1964). where

~ C max --

Tmax

< u2 < F m i n -- i -- m a x

examples

The b o o k

these

_ a3xi

are t a k e n

contains

problems

were

from the book

references treated.

by Fan

& Wang

to the o r i g i n a l

papers

12

1.2

The m u l t i s t a g e

optimization

Let us try to s u m m a r i z e from

1.1.

The s y s t e m

(state equations) variables cision

are given.

is d e s c r i b e d

The p r o b l e m

at each

in such a w a y that

zation

stage,

This p r o b l e m

formulation

in the e x a m p l e s

by the e q u a t i o n s and/or

of m o t i o n s

final v a l u e s

is to find the v a l u e s subject

the o b j e c t i v e

to c e r t a i n

function

is o f t e n

of the de-

constraints,

(criterion)

called

of state

is m a x i m i z e d

the m u l t i s t a g e

optimi-

problem.

In this p a p e r we shall fined p r o b l e m s .

functions

given.

stage.

The n u m b e r

on d e c i s i o n

are considered.

be f o r m u l a t e d

r i for e a c h

x ° and the final

Only constraints

e a c h stage

a subclass

of the a b o v e

function

stage.

at m o s t on the values

u i in the same point

only

t h a t the c r i t e r i o n

functions

depends

the initial

discuss

We a s s u m e

of the c r i t e r i o n

sions

the p r o b l e m

and the initial

variables

or minimized.

proble~

of stages ~

x. and d e c i N is fixed

of the state

variables

Thus we a s s u m e

is a s u m

E a c h of these

of states

point

de-

separately

that

and

are in

the p r o b l e m

can

as follows N-I

max

E i=0

ri(xi,u i)

(1.7)

Xi+ I = gi(xi,ui) Xo = Xo' XN = XN u. 6 U. 1 l

Let us d i s c u s s to g e n e r a l

the l i m i t a t i o n s

multistage

to the 3 e x a m p l e s compression problem straints

of

of a gas

without

transportation

optimization

formulation

problems.

is f o r m u l a t e d

regions

problem.

as a s p e c i a l

on decision

in r e l a t i o n

To do this w e r e f e r

1.1. W e see that the p r o b l e m

constraints

form closed

of this

of m u l t i s t a g e

case of the above

variables.

of a d m i s s i b l e

controls

The c o n in the

13

Many d i f f e r e n c e s catalyst

in f o r m u l a t i o n

replacement

the d e c i s i o n

problem.

variables.

case the c o n s t r a i n t s state variables. -

the p r o d u c t

We can o b s e r v e

talyst

are of the

of the d e c i s i o n

xi =

with

(1.7),

tions of m o r e Therefore

and state

stages.

spaces.

particular

but also on

As we can always

(xi_2,ui_ 2) and so on, the by an a p r o p r i a t e f u n c t i o n

For the s p e c i a l

is linear

restrictions

as the one w i t h

In g e n e r a l

in p r o b l e m

in this

on

and f r o m

case of ca-

(1.4) we h a v e

i 2 Z uj 9=I

reformulated

riables.

in the

form h i ( u i , x i) 6 Z i c R 2 x R I

the f u n c t i o n

This w a y the p r o b l e m always

that

x±_ I = gi-1 be r e p l a c e d

in the p r e v i o u s

replacement

are met

are put not o n l y on d e c i s i o n

They

insert x i = g i ( x i _ 1 , u i _ 1 ) , state v a r i a b l e s can always of d e c i s i o n s

of the p r o b l e m

Let uS start w i t h c o n s t r a i n t s

on state v a r i a b l e s

restrictions

they n e e d not be, however,

because

they

involve

than one decision

the simple

on d e c i s i o n

va-

of the f o r m as

some c o n s t r a i n t s

vector

use of p e n a l t y

m a y be

on func-

, i.e. of ui, uj, i~j .

functions

may give better

re-

sults.

The s e c o n d

difference

in p r o b l e m

formulation

point of the e q u a t i o n

of m o t i o n

consider

in the p a p e r

this p r o b l e m

is not

framework.

it is u s u a l l y

to, was m a i n l y

works on d i s c r e t e blem w h e r e

the

maximum

final

in some a d m i s s i b l e

The last d i f f e r e n c e tion is that maximization.

regions

The

principle.

and/or

initial

Although

Sometimes, points

we do not

e a s y to h a n d -

free-end-point considered

final

problem,

as

in the p r e v i o u s however,

are not

the p r o -

fixed but

lie

were, studied.

in the c a t a l y s t

the n u m b e r

fixed.

it is r e l a t i v e l y

le it in the p r e s e n t e d referred

is t h a t the

of stages

problem

is n o t f i x e d b u t

This k i n d of p r o b l e m s

solving of f i x e d - s t e p s - n u m b e r

replacement

formula-

is s u b j e c t

to

m a y be c o p e d w i t h b y m u l t i p l e

problems

for d i f f e r e n t

valnes

of N.

14

Having discussed the limitations of the results presented in the paper and possibilities of extending the area of the applicability of the approach let us now proceed to presentation of the short history of development of the methods used to solve multistage optimization problems. 1.3

Historical notes

Many different techniques of dealing with multistage optimization problems were proposed.

In special cases some heuristic methods

or simply direct methods of calculation can be tried. Other cases admit application of the classical differential calculus or the calculus of variations.

The problems can be also treated using

linear or nonlinear programming methods. Two most specialised and yet general methods to cope with the problem were invented in late fifties. There are the Dynamic Programming and the Maximum Principle Methods. The Dynamic Programming Method was founded and developed mainly by Bellman

(1957, 1961, 1962). The method is based on the Bellman's

Principle of Optimality which was formulated by Bellman

(1957) as

follows: (| An optimal set of decision has the property that whatever the first decision is, the remaining decisions must be optimal with respect to the outcome which results from the first decision.~

Although the dynamic programming method was extended also to contlnuous systems it has found the main application in optimization of discrete systems or as we called them before, multistage optimization problems. The method is very powerful in treating these problems and its application is mainly limited by the extensive need for computer storage which can happen in some cases, the so called "course of dimensionality".

15

The Maximum Principle was first hypothesized

by Pontryagin

and then developed by him and his associates

see Boltyanski

(1956), Pontryagin

(1957,

1958b), Boltyanski

(1958), Pontryagin et al

of the maximum principle

1959,

1961), Gamkrelidze

is the construction

(1957,

(1956) et al

1958a,

(1962). The main idea of a special function

depending on controls and states and called the Hamiltonian. Knowing the optimal values of states the Hamiltonian

is optimized

at each stage by the optimal decision at this stage. Those first works confined to continuous

systems.

The first attempt to extend the maximum principle to the optimization of discrete

systems was made by Rozonoer

that the maximum principle

(1959).

systems although

it is valid for linear discrete

criterion.

(1960,

Chang

mum principle however,

1961) discussed the validity of the maxi-

to prove its applicability.

Or stationary

This final conclusion was that

(1962a,

1962b)

published an

derivation of the discrete maximum principle which was

(1964). The first examples

of the maximum principle kovski

either maximum

value.

then quickly extended and even published Wang

systems failing,

may take at the optimal decisions

In spite of those first findings Katz incorrect

concluded

systems with linear

in other special types of discrete

the Hamiltonian

He

is not generally valid for discrete

(1963, 1965)

in the book by Fan and

showing the general

inapplicability

for discrete systems were given by But-

and then by Horn and Jackson

These matters were also discussed by Denn

(1965a,

(1965)and

1965b).

Gabasov

(1968). A great progress in optimization 1966) and Propoi principle

in understanding of discrete (1964,

is valid

the role of the m a x i m u m principle

systems was done by Halkin

1965). They have shown that the maximum

when the set consisting

of states and criteria

of all possible values

in each stage is convex.

was then substantially weekened by Holtzman rectional convexity,

(1964,

This assumption

(1966a,

1966b)

to di-

i.e. convexity with regard to only one direc-

tion, this is the direction of increasing

the criterion value.

16

This subject was also treated by Holtzman and Halkin

(1966).

In the later works the use of nonlinear programming was stressed, see books by Canon et al.

(1970) and Propoi

(1973).

The most

popular theory at that time, the Kuhn-Tucker theory, combined with the useful approximation by cones were the basis for derivations presented in the books. So, apart from the use of heuristic or direct methods of calculation, a number of different techniques for solving the multistage optimization problem exists. Each technique has its weak and strong sides. The dynamic programming aproach is of extremely wide applicability since it poses minimum requirements to the functions of the problem:

they need neither be differential,

In this sense it is very powerful.

nor continuous.

The price paid for this is that

a lot of combinations of stages and criterion values are to be calculated and stored. Thus even fairly small problems require a great storage capacity and prove prohibitive for a solution. The maximum principle does not suffer from this weakness. To the contrary, breaks the problem to a sequence of smaller problems.

it

The drawback

is that the functions of the problem must be well behaved in terms of continuity, differentiability and, in many practical cases, convexity or linearity. Apparently the two methods have nothing in common unless we look to continuous-tim e problem§.

In continuous time both methods can

be applied. The maximum principle even in more general cases,

since

the fulfilment of the requirements of directional convexity is not necessary to guarantee a solution.

In continuous time the connec-

tion between the two methods is seen in e.g. the Hamilton-Jacobi equation which involve functions found both in the dynamic programming and the maximum principle methods.

On the other hand, a clear relationship between the maximum principle and nonlinear programming exists. The connecting element is the shadow price in nonlinear programming and the costate vector in the maximum principle.

In numerical methods based on the penal-

ty method the slope of the penalty function at the optimum point

17

assumes ~hesame

The reason

value

for not using nonlinear

maximum principle ables

involved.

it reduces made

as the costate vector

is obvious

programming

and well known:

instead of the

the number of vari-

The strength of the maximum principle,

the number of variables

it tempting

to overcome

of finding conditions timization

and the shadow price.

reducing

i.e.

that

at each stage of calculation,

its weaknesses.

Therefore

the idea

which could be used for stage by stage op-

this way the dimensionality

of the problem

was the main subject

of other papers connected with the problem.

One of them appeared

in the paper by Katz

the weak maximum principle. is in a stationary

and was called

It stated that the optimal

point of Hamiltonian.

led the local maximum

(1962b)

principle,

decision

The other condition,

was suggested by Butkovski

cal(1963)

and then was shown to be true only for some limited class of processes.

It stated that the Hamiltonian

optimal

decision.

quasimaximum

Then Gabasov and Kirillova

principle

which

an optimal

decision

difficult

(1966)

is bounded.

led high order conditions

in this direction.

rather

sofisticated

the Hamiltonian

nian

and Gabasov

More practically

generalization in Ravn

framework

of Hamiltonian

(1976), Vidal

(1977),

was to base the derivation Everett

(1963)

formulae

which

terms

leads

were t~e first is

which makes generalization

(1978).

A new Hamilto-

to the classical

by Ravn and Vidal (1978).

on the theory

(1980)

one.

in a natural way to

Ravn and Vidal

and then extended b y G o u l d

theorems

and

was elaborated

theory was then sketched by Ravn hypothetical

by Gabasov

(1972)

oriented

was p r o p o s e d by Yakovlev

theoretical

in a

So cal-

The idea of high order conditions

was defined by adding two simple

A broader

function

property.

developed

and uses complicated

to apply.

of the Hamiltonian

at

The value of the bound can be,

of optimality

and Ashchepkov

trials

it difficult

between

to estimate.

Another approach was to generalize

(1971)

the

and the value of Hamiltonian

way to keep its m a x i m u m - a t - t h e - o p t i m a l - d e c i s i o n

Tarasenko

in an

proposed

stated that the difference

the maximum value of Hamiltonian

however,

is locally maximal

Their

idea

started by the paper by (1969).

The proposed

in a concise

and named the Upper Boundar[

form of

Method.

18

1.4

The scope of the b0ok

In this

book

we intend to show the present status of works on the

upper boundary approach to optimal control of multistage problems. First we introduce the notion of upper boundary and discuss its application to optimization of static

(or one-stage)

systems and

more specifically to the problem of maximization of a function subject to equality constraints.

This is the content of Chapter 2.

The reasoning there follows in many aspects the lines of Gould (1969) although the definition of a support is generalized and a saddle-point theorem for the problems with equality constraints is introduced. Chapter 2 gives the necessary information for understanding the derivation for multistage systems which is the subject of Chapter 3. In Chapter 3 the mathematical treatment of the upper boundary approach to optimization of multistage systems is presented. Many basic notions of this chapter are the same as in Halkin although,

to our knowledge,

(1964)

most of the main results connected

with the generalized maximum principle, which is presented in this chapter, are new. In the formulation of the generalized maximum principle one of the more restricting assumptions of the classical discrete maximum principle i.e. the assumption of directional convexity is released. This considerably widens the area of applicability of the principle.

It is shown that the classical m a x i m u m

principle is the special case of the generalized one.

Apart from

the maximum principle also the dynamic programming approach is discussed as naturally placed within the upper boundary framework.

A computer algorithm for finding controls and states satisfying the conditions of the generalized maximum principle is proposed in Chapter 4. This algorithm was coded and run on

a computer and the

results of computations for three simple examples give the first evidence of the posibility to use the algorithm.

Lastly,

in Chapter 5 possibilities of further research on the upper

boundary approach are sketched.

19

1.5

Literature

L.T. Ashchepkov,

R. Gabasov: Optimization of Discrete Systems.

Differencyalnye Uravnenya, Vol. 8, No. 6, 1972 (in Russian). R. Bellman: Dynamic Programming. New Jersey,

Princeton Univ. Press,

1957.

R. Bellman: Adaptive Control Processes. New Jersey,

Princeton Univ. Press,

1961.

R. Bellman, S.E. Dreyfus: Applied Dynamic Programming. Univ. Press, New Jersey, V.G. Boltyanski: Processes.

Princeton

1962.

The Maximum Principle in the Theory of Optimum

Doklady Akad. Nauk SSSR, Vol.

119, No. 6, 1956

(in

Russian). V.G. Boltyanski,

R.V. Gamkrelidze,

Theory of Optimum Processes. No. I, 1956

L.S. Pontryagin:

On the

Doklady Akad. Nauk SSSR, Vol. 110,

(in Russian).

A.G. Butkovski:

Necessary and Sufficient Conditions of Optimality

of Discrete Control Systems. Automation and Remote Control, Vol. 24, No. 8, 963-970, A.G. Butkovski:

1963.

Theory of Optimal Control of Distributed Parameter

Systems. Nauka, Moskva,

1965

(in Russian).

M.D. Canon, C.D. Cullum Jr., E. Polak: Theory of Optimal Control and Mathematical Programming. McGraw-Hill,

New York,

1970.

S.S.L. Chang: Digitized Maximum Principle.

Proc. IRE, 2030-203~,

1960. S.S.L. Chang: Synthesis of Optimum Control Systems. McGraw-Hill, New York, 1961.

20

M.M. Denn: Discrete Maximum Principle. mentals,

Vol. 4, No. 2, p. 240,

H. Everett III: Generalized

Ind. & Eng.Chem.

Lagrange Multiplier Method for

Solving Problems of Optimum Allocation of Resources. Vol.

11, No. 3, 399-417,

L.T. Fan, Ch.S. Wang: New York,

Funda-

1965.

Opns. Res.,

1963.

The Discrete Maximum Principle,

Wiley,

1964.

R. Gabasov:

Uniqueness of Optimal Control in Discrete Systems.

Izd-vo Akad. Nauk SSSR.

Ser.Energetika

i Avtomatika,

No. 5, 1962

(in Russian). R. Gabasov: Mat.

Theory of Optimal Discrete Processes.

i Mat.Fiz.,

R. Gabasov, Principle

Vol.

F.M. Kirillova:

to Discrete

Vol. 27, No. R. Gabasov,

R. Gabasov, Vol. 32, No.

I, 50-57,

R.V. Gamkrelidze: Systems.

Automation and Remote Control,

Qualitative

Theory of Optimal Proces-

(in Russian).

N.W. Tarasenko: for Discrete

Maximum

1966.

F.M. Kirillova: 1971

(in Russian).

Extending L.S. Pontryagin's

Systems.

11, 1878-1882,

ses. Nauka, Moskva,

Optimality

8, No. 4, 1968

~urnal Vycisl.

Necessary High-Order Conditions

Systems.

Automation

of

and Remote Control,

1971.

On the Theory of Optimum Processes

Doklady Akad. Nauk SSSR, Vol.

116, No.I,

in Linear

1957

(in

Russian). R.V. Gamkrelidze: Linear Systems. No.4,

1958a

The Theory of Time Optimal Processes

Izv. Akad. Nauk SSSR,

Ser.Matem.,

Vol.

in 22,

(in Russian).

R.V. Gamkrelidze~

On the General Theory of Optimum Processes.

Doklady Akad. Nauk SSSR, Vol. English translation

123, No. 2, 1958b

in Automation

Express,

(in Russian).

Vol I, 37-39,

1959.

21

F.J. Gould: Extensions of Lagrange Multipliers gramming.

SIAM J. Appl. Math., Vol.

H. Halkin:

Optimal Control

Equations:

In: C.T. Leondes

Vol. I. Academic Press, H. Halkin:

(ed): Advances

Vol. 4, No.

1966.-

Convexity

IEEE Trans.

in Control Systems,

of the Pontryagin Type for Systems

Difference

Systems.

1969.

1964.

Described by Nonlinear

J.M. Holtzman:

17, No. 6, 1280-1297,

Pro-

for Systems Described by Difference

A Maximum Principle

I, 90-111,

in Nonlinear

Equations.

SIAM J. Control,

and the Maximum Principle

Autom. Control,

for Discrete

Vol. AC-11, No.

I, 30-35,

1966. J.M. Holtzman: Time Systems. 273-274,

263-275,

IEEE Trans. Autom.

for Nonlinear Discrete-

Control,

Vol. AC-11,

no. 2,

1966.

J.M. Holtzman, Principle

on the Maximum Principle

H. Halkin:

Directional

for Discrete Systems.

Convexity and the Maximum

SIAM J. Control,

Vol. 4, No. 2,

1966.

F. Horn, R. Jackson: Chem. Fundamentals, R. Jackson,

Discrete Maximum Principle. Vol. 4, No.

I, 1!0-112,

F. Horn: On Discrete Analogues

Maximum Principle.

Int. J. Control,

Vol.

Ind. & Eng.

No. 4, 487-488,

1965.

of Pontryagin's

1, No. 4, 389-395,

1965.

S. Katz: A Discrete Version of Pontryagin's Maximum Principle. Electronics

and Control,

Vol.

13, p. 179, 1962.

S. Katz: Best Operating Points for Staged Systems. Chem. Fundamentals. L.S. Pontryagin:

Vol.

~, No. 4, 226-240,

Some Mathematical

Academy of Sciences USSR on Scientific October

15-20,

1956

Ind. & Eng.

1962.

Problems Arising in Connection

with the Theory of Optimum Automatic Control

Industry,

J.

Systems.

Problems

(in Russian).

Session

of Automating

22

L.S. Pontryagin: Control.

L.S. Pontryagin: Nauk, Vol. Am.Math.

Basic Problems of Automatic

Izd-vo Akad, Nauk SSSR,

Optimal Regulation

14, No.

I, 1959

Soc. Trans.,

L.S. Pontryagin: Butterworths

1957

Processes.

(in Russian).

Ser. 2, Vol.

Regulation

18, 321-339,

in

1961.

I, p. 454,

1961.

L.So Pontryagin,

V.G. Boltyanski,

The Mathematical

Theory of Optimal Processes.

York,

Uspekhi Matem.

English translation

Proc. First IFAC Conf., Vol.

Publishing,

and

(in Russian).

R.V. Gamkrelidze,

E.F. Mishchenko:

Interscience,

New

1962.

A.I. Propoi:

On a Problem of Optimal Discrete Control.

Akad. Nauk SSSR, Vol. A.I. Propoi: Automation

The Maximum Principle

and Remote Control,

A.I. Propoi: Problems.

159, No. 6, 1964

for Discrete Control Systems. 26, No. 7, 1167-1177,

Methods of Feasible Directions

Automation

A.I. Propoi:

Vol.

Doklady

(in Russian).

and Remote Controls,

1965.

in Discrete Control

No. 2, 1967.

Problems of the Discrete Control with Phase Bounds.

~urnal Vycisl.

Mat.

i Mat. Fiz., Vol.

12, No. 4, 1972

(in

Russian). A.I. Propoi:

Elements of the Theory of Optimal Discrete Processes.

Nauka, Moskva,

1973

(in Russian).

H.F. Ravn: The Discrete Maximum Principle with Nonlinear Hamiltonian. University

Research Notes J. hr. 2376/041928.

of Denmark,

1976.

H.F. Ravn: Upper Boundary Methods. Technical University

IMSOR. Technical

of Denmark,

Research Report No.

1980.

10, IMSOR,

23

H.F. Ravn,

R.V. Valqui Vidal:

Everett's og Kuhn-Tucker's Research Report no. 1978

En elementar

multiplikatorer

introduktion

til

og deres anvendelser.

15, IMSOR, Technical University

of Denmark,

(in Danish).

L.I. Rozonoer:

The Maximum Principle of L.S. Pontryagin

Theory of Optimal Systems. 10-12,

in the

Automation and Remote Control,

Nos

1959.

R.V. Valqui Vidal: IMSOR, Technical W.M. Yakovlev: No. 34, 1978.

Notes on Static and Dynamic Optimization.

University of Denmark,

1977.

On Discrete Maximum Principle,

Problemy Kibernetiki

CHAPTER

2

ONE STAGE SYSTEMS

27 0.

In this chapter we shall investigate the problem of optimi-

zation of static systems subject to equality constraints.

This

problem is treated using the concept of a nonlinear support to the upper boundary of the extended set of feasible solutions. The approach of this chapter is a development of the ideas presented by Everett

(1963) and Gould

(1969).

We start with an introduction of some basic notions and then develop the main results which consist of four theorems dealing with maximization of the socalled Hamiltonian and relations between optimal solutions to the initial problem and solutions maximizing the Hamiltonian. Neither differentiability nor continuity assumptions for the optimized function is needed.

It is

believed that the saddle-point theorem in this part presents a new, not earlier published result. After proving the main theorem some important cases of linear and quadratic supports are discussed more thoroughly and the chapter ends with some examples illustrating the theory and indicating some possible applications.

The chapter is written in a very short way, presenting only necessary results without any broader discussion of them. All statements are numbered so that can help in discussion of the presented results. The numbers match those in the paper by Ravn

(1980).

New parts which have no strict correspondence in the paper by Ravn are marked with small letters a,b,c and so on.

28

i. C o n s i d e r

where

the

following

max

r (u)

We

(ii)

u 6 U

(iii)

r(-)

is a r e a l

f(u)

is a n - d i m e n s i o n a l

column

vector

function

u

is a m - d i m e n s i o n a l

column

vector

(m > n)

the

is a g i v e n

set

vector

space we call

axis,

and

PR-space

in P R - s p a c e

problem@. 102.

We call,

in R m in R n

in n + l - d i m e n s l o n a l

first

properties

funotion

is a g i v e n shall work

along

(i)

f (u) = x

U

i01.

problem:

for a g i v e n

f(u)

space, along

(P = p a y o f f , we

r(u)

the

being

measured

last n axes.

R = resource).

shall deduce

properties

This

From of t h e

u,

x = f (u)

a state,

and

Ii an extended

103.

If w e

state.

l e t u tO t a k e a l l v a l u e s

of x t h a t w e

shall call

the

t e d b y X. A n d

likewise

x will

we

the

shall A

by X.

call

in U w e g e t a set o f v a l u e s

set of reachable take

set o f r e a c h a b l e

states,

deno-

on a s e t of v a l u e s extended

states,

that

denoted

2g

103a.

A vector

u £ R m w i l l be said to be a f e a s i b l e

for the p r o b l e m ~ valently

if it s a t i s f i e s

if u £ U and f(u) = x.

said to be an o p t i ~ l

104.

solution

A vector

a n d r(u ~) > r(u), u £ U.

For a given

x we d e f i n e

u 6 U and

ub(x)

f(u)

solution (iii)

or equi-

u e E R m will

for the p r o b l e m ~ i f

is f e a s i b l e

for w h i c h

(ii) and

as the s u p r e m u m

= x. ub(-)

be it

of all r(u)

is thus d e f i n e d

for

all x 6 X.

Considering ub(-)

the r ( . ) - a x i s

the s u p r e m u m

as p o i n t i n g

function

"up" we can call

or c o n s i d e r e d

as a set of ^

points by

105.

we can call

it the u p p e r

boundary

of X, d e n o t e d

~b}.

We can thus

conceive

the u ~ that gives

the p r o b l e m

[A)

as that of f i n d i n g

the

r(u*)

R*=

I

t h a t on the one h a n d

satisfies

the con-

f(u*)

straint

f(u)

= x and on the o t h e r

hand

lies as high up

in X as p o s s i b l e .

106.

Theorem.

[ A ) has a f e a s i b l e

x 6 X a n d L'-~ ub(~)I

Proof.

107.

Follows

directly

and X.

Theorem.

If u • is an o p t i m a l

then

u if and o n l y

from the d e f i n i t i o n

solution

Q

solution

if

^ £ X

solution

to

r(u*) R~ =

is s i t u a t e d f(u*)

at

{ub}

of a f e a s i b l e

30 Proof. By contradiction.

Suppose ~ is an optimal

solution

and

x =

Ir (~) 1

Lf (~) is not situated at {ub}. Then from the definition of {ub} and Theorem 106 there is a u* 6 U for which f(u*)

= f(~) and

~*=

is situated at

{ub}. But then

f(u*) r(u*) 108.

> r(u) which contradicts

the optimality of ~-

To facilitate things we shall in the sequel assume that ub(x) < ~ for all x £ X and

ui(xI£ X^ for

all x £ X

This is not a real limitation of the analysis. also assume that x £ X. An optimal u* and the corresponding 109.

We shall

solution we shall call

extended state ~*.

We shall define a support n°(.) function of an n-dimensional

at the point x 0 as a

argument,

defined on the set

X. It has the form

= i=lz N°I(xi) n

O{x)

It has the property that there exist a real number k, such that (i)

~O(x°)

+ k = ub(x O)

(ii)

N°(x)+ k > ub(x)

for all x £ X

Sl

We say t h a t n°(x) w h i c h ~°(x)

s u p p o r t s ub(x)

+ k = ub(x).

at the p o i n t s x for

A s u p p o r t at the p o i n t x we

d e n o t e n*(x) 109a.

If the c o n d i t i o n k > ub(x)

109b.

109(ii)

(ii)' n°(x)

+

shall call a s u p p o r t

n°(x)

a s t r o n g S u p p o r t at a p o i n t x O. T h e n it s u p p o r t s

ub(x)

o n l y at a single p o i n t x °.

If there exists a n e i g h b o u r h o o d f u n c t i o n n°(x)

satisfies

then w e shall call n°(x) If n°(x) by i09c.

is r e p l a c e d by

for all x £ X and x $ x ° , w e

of x O, Y c X

conditions

a local s u p p o r t at the p o i n t x °.

satisfies on Y conditions

(ii)' then w e shall call n°(x)

The d e f i n i t i o n

§109 w i t h

(ii) r e p l a c e d

a s t r o n g local support.

of a s u p p o r t g i v e n in §109 is a g e n e r a l i -

zation of a c l a s s i c a l Gould

such that a

of §109 on the set Y

support d e f i n i t i o n

(1969). The c l a s s i c a l

support defined

as given by

s u p p o r t is a s p e c i a l case of

in §109 s u b j e c t to

k--0 •

n*

1

(x i) = A i(x i) - A i(xi) + n ub(x)

w h e r e i i is any f u n c t i o n

such t h a t i09(i)

and

(ii) are

satisfied.

AlSo a support obtained after a redefinition r a l i z e d L a g r a n g i a n as g i v e n by Evans at al special case of a support d e f i n e d

of a g e n e -

(1971)

is a

in § 109 s u b j e c t to

k=0 * (X)

x± (0)

= I i(xi - Xl) + 1 ub(x)

= 0

w h e r e I , i = 1 , 2 , . . . , n are any f u n c t i o n s 1

§ 109

(i),

(ii), and the a b o v e conditions.

that s a t i s f y

32

109d.

All

theorems

are v a l i d

given

respectively,

ii0.

Theorem.

below

for s u p p o r t s

for local s u p p o r t s

If

or strong

or strong local

if the set X is r e s t r i c t e d

Q

has a f e a s i b l e

supports

supports,

to Y.

solution

then

a

~*(.)

exists. Proof.

Suppose

x is a f e a s i b l e

~*(x)

value.

Take

0

x=x

sup ub(x)

x ~ x

=

l

xCX

T h e n ~* (x) is a s u p p o r t

iii.

Theorem.

If

~x)

and ub(x)

~°(x°)_ 3x

Proof.

The function K°(x)

= O(x)

z(x)

and ub(x)

at x ° t h e n

the f u n c t i o n

- ub(x)

is d i f f e r e n t i a b l e

are d i f f e r e n t i a b l e

at x ° b e c a u s e at x 9 F r o m

both

the defi-

of a s u p p o r t z(xO) = k

then

are d i f f e r e n t i a b l e

3 u b ( x O) ~x

L e t us c o n s t r u c t

z(x)

nition

w i t h k = ub(x).

a n d z(x)

z (x) has a local m i n i m u m

>_ k

for all o x

in the point

and ~z(x°)~x

~°(x°) ~x

~ub(x°)= ~x

0

x 6X

33

112.

If ub(x)

is d i f f e r e n t i a b l e

Dub (x)

at x we call

the

s h a d o w prices. n

113.

L e t ~(x)

be a n y f u n c t i o n

We d e f i n e

the H a m i l t o n i a n

H(u,~)

ll3a.

The f u n c t i o n

H(u,~)

Hamiltonian

in c o n n e c t i o n

For g i v e n tical

=

as

Z ~ i (Xi) . i=l

- ~(f(u))

was p r e v i o u s l y

see G o u l d

see C h a p t e r

H(-,-)

= r(u)

Lagrangian,

a generalized

114.

of the f o r m K(x)

(1969).

Hamiltonian

called

generalized

We shall

call H(u,K)

the

with dynamic

programming

where

is d e f i n e d

in a s i m i l a r

manner,

3.

u and

distance

~,H(u,~) between

c a n be i n t e r p r e t e d

as the v e r -

the p o i n t

^

X =

in x and the p o i n t

on ~.

Lf(u)]

f(u)

[~

(u)]

115.

Theorem.

If u m a x i m i z e s

situated

Proof. U that

H(u,~)

over

U then x =

at {ub}.

is

(u)J

By c o n t r a d i c t i o n .

Suppose

~ maximizes

H(u,~)

over

is

H(~,~)

> H(u,~)

and

for all u 6 U

is n o t s i t u a t e d

at

{ub}.

Lf (~)J B u t f r o m the d e f i n i t i o n is a u* C U for w h i c h

of { ub} and a s s u m p t i o n

f(u*)

= f(~)

and

108 there

34

X* =

[r(u*)l

is s i t u a t e d

at

{ub}.

Hence

tf (u*)J r (u*)

> r (~) a n d c o n s e q u e n t l y N

H(u*,~)

which

116.

contradicts

Theorem. there

n°(x)

exists

Proof.

the supposition.

is a s u p p o r t

u ° which

and s a t i s f i e s

Then

> H(u,u)

at a p o i n t

is a s o l u t i o n

if

to: m a x H ( u , n O) o v e r U

f(u O) = x O.

Suppose

~°(x)

is a s u p p o r t

f r o m the d e f i n i t i o n

ub(x)

- ~°(x)

f r o m the d e f i n i t i o n

at the p o i n t

xO

of a s u p p o r t

ub(x°) - ~ ° ( x ° ) =

Now,

x ° if a n d o n l y

k

x6X

< k

w

of ub(x)

and a s s u m p t i o n

108 there

a r e u °, u C U t h a t

Inserting

f(u °) = x ° a n d

r(u O) = ub(x O)

f(u)

r(u)

= x

the above

and

formulae

= ub(x)

for a n y x C X

into the p r e v i o u s

ones we

have

r ( u °) - u ° ( f ( u ° ) )

= k

r(u)

< k

- ~°(f(u))

B u t for a n y ~ ~ W there a n d f(~) mulae

= f(u)

is a l w a y s

which means

is s a t i s f i e d

r(u)

u C W = {u:

a u E W t h a t r(~)

t h a t the s e c o n d

o

(f(u))

< k

~ r(u)

of a b o v e

also by u ~ W. T h u s we c o n c l u d e

- ~

(u)J 6 { u b } }

u £ U

for-

that

35

and from the d e f i n i t i o n of H a m i l t o n i a n

H(u°,= °) = k H(U,nO

which means N O W let u

o

) < k

uEU

that uO m a x i m i z e s maximizes

H(u°, ° )

H(u,~°).

H(u,n°). Then

> ~(u,~ °)

u£U

so there is a k that H(uO, n O) = k H ( U , K O)

< k

and from the d e f i n i t i o n

of H a m i l t o n i a n

r(u O) - n°(f(u°))

= k

r(u)

< k

- n°(f(u))

uEU

But from the t h e o r e m 115

I~ (uO)

t h e r e f o r e we h a v e

ub(x O) - n * ( x O) = k

and, d e f i n i n g

(u° )

for any u E U, f(u)

sup r(u)

is s i t u a t e d

at {ub}

= x, we have

- n O(f(u))

< k

of ub(x)

and X

all u £ U

f (u) =x

or from the d e f i n i t i o n ub(x)

- O(x)

< k

all x £ X

38

Rearranging

terms

we h a v e

n°(x O) + k = ub(x °)

n°(x)

+ k > ub(x)

from w h i c h we c o n c l u d e

ll6a.

x ° = f(u°).

Corollary.

Let u* be an o p t i m a l

. T h e n n*(x)

only

is a s u p p o r t

if u* is a s o l u t i o n

Proof.

the c o n d i t i o n

r e m 116 leads

Theorem. only

n°(x)

at a p o i n t

to the p r o b l e m x = f(u*) over

if and

U.

to the p r o b l e m ( A )

f(u*)

is a s t r o n g

then

= x. N o w a p p l i c a t i o n

it sa-

of Theo-

support

at a p o i n t

x ° if and

u ° to: m a x H ( u , n °) over U s a t i s f i e s

= x°

Proof.

T h e proof

Suppose

t h a t n°(x)

from T h e o r e m and

solution

to ub(x)

to l16a.

if any s o l u t i o n

f ( u °)

is a s u p p o r t

to: m a x H(u,n*)

As u* is a s o l u t i o n

tisfies

117.

that n°(x)

at a point

Q

all x £ X

conclusion

is a s t r o n g

116 there

satisfies

maximizes

is a s i m p l e

exists

support

Then

from T h e o r e m

which

contradicts

f(~)

at a p o i n t

u° which maximizes

f(u °) = x °. S u p p o s e

H ( u , ~ °) but

there

% x °. D e f i n e

i16 n°(x)

from T h e o r e m

exists f(~)

is a s u p p o r t

the a s s u m p t i o n

i16.

x °. T h e n H ( u , ~ °)

~ which

= ~ # x 0.

at the p o i n t

that n°(x)

is a strong

support. Suppose

n o w t h a t any

satisfies support

f(u °) = x °. T h e n

at x O. S u p p o s e

that there

o)

from T h e o r e m

~°(x)

116 t h e r e

o v e r U and

the assumption.

u O to: m a x H ( u , n O) o v e r 116 n°(x)

it is not a strong

is ~ ~ x ° that

then from Theorem H(u,

solution

is a s u p p o r t

exists

satisfies

support,

f(~)

U

is a i.e.

at x. But

~ which maximizes = ~, w h i c h

contradicts

37 ll7a.

Corollary. only

n*(x)

is a strong

if any s o l u t i o n

f(u*)

= x. M o r e o v e r

Proof.

The

first

u* to: m a x H(u,~*)

x if a n d

over U satisfies

any such u* is an o p t i m a l

sentence

c a s e of T h e o r e m from Theorem

118.

Deleted.

119.

Let K(x)

117.

The

of c o r o l l a r y second

solution

to

be a n y s u p p o r t

to ub(x).

of the s u p p o r t

Call

of s u p p o r t s

the c l a s s

functional

~(-,-)

define

is a c o n c l u s i o n

{ub}.

L e t the n u m b e r

for g i v e n

= r(u)

a saddle-point

of

be c o n s t a n t

on the p r o d u c t

~(u,~)

is just a s p e c i a l

sentence

115 and the d e f i n i t i o n

the d e f i n i t i o n

We

at a p o i n t

problem@.

the

120.

support

k from

for all supports.

k, C k. D e f i n e

the

Ux C k as

- K*(f(u))

of ~(.,-)

+ ~(x)=H(u,~*)+K(x)

as a p a i r

(u°,~v)EUXCk

w i t h the p r o p e r t y

~(u,~ °) < ~(u°,~ °) < ~(u°,~) u C U, ~ £ Ck, 121.

k E R.

Let S k c C k be the subset Then

for

Theorem.

supports

in a set C k .

(u,~ °) £ Ux S k we have:

~*(x)

is a strong

a n d u* is a s o l u t i o n saddle-point

Proof.

of strong

to < A J

of ~(-,.)

L e t ~*(x)

u* be a s o l u t i o n

support

restricted

be a s u p p o r t to ~ k_/

H(U,~*)

. Then

< H(u*,~*)

m

at the p o i n t x=f(u*)

if and o n l y

if

(u*,~*)

is a

to Ux Sk, k E R.

at the p o i n t

x = f(u*)

from Corollary

l16a

all u E U

and

38

A d d i n g to b o t h sides n*(x) we get the left i n e q u a l i t y the s a d d l e - p o i n t observing

definition.

that from the d e f i n i t i o n

n*(x) Then a d d i n g

of

The r i g h t i n e q u a l i t y w e get of a s u p p o r t

< ~ (x)

all

to b o t h sides H(u*,n*)

n E S

we get the right

in-

equality.

Let uS s u p p o s e n o w that ~(-,.).

(u°,n °) is a s a d d l e - p o i n t

T h e n from the r i g h t i n e q u a l i t y

of

in the s a d d l e - p o i n t

d e f i n i t i o n we have

H ( u ° , n *) + n°(x)

_< H ( u ° , n *) + n(x)

all 7tES k

or ~°(x) Taking

< n(x)

n = ~* rc° (x)

all ~ e S

we have

< n*

S u p p o s e n o w that ~°(x)

(R) is a strong

s u p p o r t to ub(x)

p o i n t x O # x. T h e n from the d e f i n i t i o n

at the

of strong s u p p o r t s

rc° (~) > n* (~) which contradicts

the e a r l i e r

inequality.

This proves that

n°(x) m u s t be a support to ub(x) at the p o i n t x, i.e. o n = n*. Now, from the left i n e q u a l i t y in the s a d d l e - p o i n t definition

H(u,~*)

+ ~*(x)

_< H ( u ° , ~ * )

H(u,n*)

< H ( u ° , ~ *)

+ n*(~)

all uEU

or all uEU

39

i.e.

uO maximizes

o

u is a n o p t i m a l u o = u*.

122.

H(u,n*)

o v e r U. T h e n

solution

T h e c a s e of l i n e a r

~*(-)

deserves

®

from Corollary

to the p r o b l e m

special

l17a

, i.e.

attention.

In

this case H(u,p)

= r(u)

- pTf(u)

w h e r e p is a n - d i m e n s i o n a l notes

122a.

a transpose

two points

X l , X 2 E X the l i n e s e g m e n t

A function

in X, t h a t

f: X~R,

to be c o n c a v e real

l,

0 <

(l-l)

is,

x2 £ X

linear

A sufficient

support

[0,13

s u b s e t of R n, is said x I and x 2 in X a n d a n y

Af(xl)+(l-l)

condition

to ub(x)

L e t us c o n s i d e r

f(x 2)

for the e x i s t e n c e

at a n y point,

be finite and concave on a convex

Proof.

x I and x 2

have

f [ l x I + ( l - l ) x 2] ~ Theorem.

joining

for e v e r y I C

with X a convex

< lwe

if for a n y

if

if for a n y two p o i n t s l

T de-

o f a vector.

set X in R n is said to be c o n v e x

lx I +

123.

and superscript

A nonempty

is c o n t a i n e d

122b.

vector

of a

x C X is t h a t ub(x)

set X a n d x E Int X.

the set S c X d e f i n e d

in the f o l l o -

wing way

S = {(x,y) : xEX,

We

prove

from S

t h a t S is a c o n v e x

inf ub(x) x£X

set.

_< y _< ub(x) }

T a k e a n y two p o i n t s

(x I, yl ) , (x2,Y 2) a n d c o n s t r u c t

the

line s e g m e n t

40

l(xl,Yl)

+

(1-1) (x2,Y2)

= (~i

+(l-l)x2 ,

lyl + (l-l)y 2)

0<~
as a set X is convex

A

x = lx I +

(l-A) x 2 £ X

Then as Yl ~ ub(Xl) function, we have

and Y2 ! ub(x 2) and ub(x)

ly I + ( l - l ) y 2 <_ lub(Xl)+(l-l) Moreover

as inf ub(x) x£X

is concave

ub(x 2) < ub [lXl+(l-l)x 2]

~ Y l and inf ub(x) yEX

<_ Y2

we have ly I +(l-A)y 2 ~ A i n f ub(x)+(l-A) yEX

inf ub(x) xEX

= inf ub(x) x£X

Then the value

y = ly 1 +(l_~)y 2 satisfies inf ub(x) x£X

< y < ub(x)

But this means

that the point

(x,y) =

which

proves

(lXl+(l-A)x2,

the c o n v e x i t y

Ayl+(l-A)y2 ) E S for any 0<~
of the set S.

41

Then

from the Theorem

follows

that

the

A.5.42

set S has

p.

251

in C a n o n

supporting

et al

hyperplane

(1970)

L at

N

every

boundary

by the

point

L =

where such

(x,y)

6 S. T h e

hyperplane

is g i v e n

formula

{(x,y)

: ax + a0Y

a is a n - d i m e n s i o n a l

= b}

row vector

a n d a 0 is a s c a l a r ,

that ~

ax + a 0

= b for all

a x + a 0 Y _< b

Now, of

as a n y p o i n t

(x,y)

£ S

(x~,ub(x O) ), x O 6 I n t X, l i e s o n a b o u n d a r y

S then we have

ax°+

a 0 ub(x°) = b

a x + a 0 ub(x)

because

(x,ub(x))

L e t us n o t i c e ,

for all

<_ b

x 6 X

C S, x 6 X.

that

a 0 $ 0.

Indeed,

if a 0 = 0

then

a x °- b = 0 a n d a x - b < 0 f o r a l l x £ X

which

means

that

the hyperplane Then

x°~

all points

x E X are on

ax-b = 0 which

I n t X, w h i c h

In c o n s e q u e n c e

goes

contradicts

a 0 • 0 and we

the one

through

the point

the assumption.

have

cx O+ c o = ub(x O ) c x + c O ~ ub(x)

for all x C X

where a C

~-

~

b

a

o

c O = ao

side of x°

42

F r o m the a b o v e we conclude

that

a(x)

is a l i n e a r

124.

relation

=

cx

to ub(x)

In the c a s e of a q u a d r a t i c H(u,p,q)

where

125.

co

+

support

= r(u)

the s u p e r s c r i p t

mensional

vector

Theorem.

A sufficient

quadratic

support

ub(x)

be twice

function

x° .

support

- pTf(u)

T denotes

- ½qfT(u) f(u)

a transpose,

p is a n-di-

a n d q is a real number.

condition

to ub(x)

below

for the e x i s t e n c e

at any p o i n t

continuously

defined

at the p o i n t

x°£ X is that

differentiable

of the H e s s i a n

of a

in X and the

matrix

ub

(x) be XX

bounded

max

in X, that

zTub XX

(X) Z < M for all x £ X, M - r e a l z-real

zTz=I

Proof. p~"

is

n-dlmensional

L e t us t a k e q~ > M and

= ub

(x°) - q S x °. C o n s t r u c t

number vector.

for g i v e n

x°£ X take

the f u n c t i o n

X

G ( x , p ~ , q ~) = ub(x)

Differentiating

- p~Tx

- ½q~xTx

G ( x , p ~ , q *) w i t h r e s p e c t

Gx(X°,p~,q~)=

ub(x°) - p ~ T - q ~ x °

to x w e have

= 0

G x x ( X , p ~ , q ~) = U b x x ( X ) - q * I where

I is the i d e n t i t y

matrix

Gxx(x,p~,q~)

Indeed

taking

matrix.

is n e g a t i v e

z, zTz $ 0 we h a v e

Let us p r o o v e definite

that the

for any x E X.

43 T T

T

y

Y GxxY = Y Ubxx(X)y which,

putting

_ q,yTy

Ubxx (x) y T YY

=

-

q*

a new vector

z =

1 y yTy

T z z = 1

we c a n w r i t e

Y TGxxY

= zTubxx(X) Z - q* _< m a x T z

= m a x zTubxx(X) Z T z z=l which means

-q* <_ M - q* < 0

it s a t i s f i e s

48 p.

258 in C a n o n

and a s s u m p t i o n

=ub(x°). T h e n

necessary

G(x,p*,q*)

ub(x)

at al 108

u o maximizes H(u,p*,q*)

and T h e o r e m

116 leads ~(x)

is a s u p p o r t

we g i v e

z=l

that G

mum and moreover

126.

(1970). there

for a local m a x i -

see T h e o r e m

A. 6.

N o w f r o m the d e f i n i t i o n

is u ° t h a t

f(u°)=x°and

of

r ( u O)

the H a m i l t o n i a n

= r(u)-p*Tf(u~-%q*fT(u)f(u) to the c o n c l u s i o n

at the p o i n t

even

conditions

is concave,

that

= p * T x + ½q* x T x

two e x a m p l e s

a support

xO

that quadratic

for q u i t e

smooth

x~-0. We try to find a q u a d r a t i c a>0,

k=0.

y(x)

are d i f f e r e n t i a b l e

As u b ( 0 ) = 0

d ub(0) dx

= 0

d ~(0)

=b

function

functions. support

then

from Theorem

= 0

dx f r o m the d e f i n i t i o n

of a s u p p o r t

m a y not g i v e 4 ,

Let ub(x)=x

~(x)=a2x 2 + bx +c,

t h e n c=0 a n d b e c a u s e

and

Now,

- q*

is n e g a t i v e definite. B u t n o w we can xx that x ° m i n i m i z e s the f u n c t i o n G(x,p*,q*) o v e r X

conclude because

zTubxx(X)Z

b o t h ub(x) III we h a v e

and

44

ub(x)

< ~(x)

X 4 - a 2x 2 = x 2 (x 2 - a 2) which

Now

is s a t i s f i e d

only

for

< 0

Ixl <_ a.

l e t us t a k e X = < - l , l >

ub(x)

I

=

-x21nx

0<X

<

-I<x

0

1

< 0 ~x

From

lim

the d e

l'Hospital

-x21nx

I n1'x

= 1~

X+0 +

r u l e we h a v e

X ÷ 0 + ---~ X

so ub(x)

is c o n t i n u o u s

-2x

1 x2

= lim X~0 +

X

-i

=

3

lim

½ x 2

=

0

x+0 +

in X. D i f f e r e n t i a t i n g

inx - x

ub(x)

for x#0

0<x
d ub(x) dx =~[0

then

from the de

-l<x<0

l'Hospital

rule we have

lim - x inx = 0 x+0 +

that d ub(0)_ dx

and we conclude

0 and the

fist derivative

continuous.

Let ~(x)

us try to find a support = ax2+bx+c,

t h e n c = 0 a n d b=0. 2

k=0. Now

As

of t h e

ub(0)=0

from

form

and

the definition

2

ax

> -x i n x

for

0<x
> -inx

for

0<x
w

or a

d ub(0) =0 dx of a support

is

45

which

is i m p o s s i b l e

because

lim

(-inx)

= ®.

x÷0 + 127. B e l o w we g i v e e x a m p l e s

of f i n d i n g

the d i r e c t i o n

for c h a n g i n g

supports. Example ~(x)

i° L i n e a r

support.

Suppose

= a T x + b a n d we f o u n d that u,

Hamiltonian.

If n * ( x ) = a * T x

then from the definition

which

that we assumed f(u)=x,

+ b* is a s u p p o r t

of s u p p o r t s

a support

maximizes

the

at t h e p o i n t

(taking k=0)

leads to the c o n d i t i o n

(a*

-

a)T(~

_

~)

<

F o r the o n e - d i m e n s i o n a l

0

problem

we h a v e

N

which

a~ < a

for x > x

a~ > a

for x <

is i l l u s t r a t e d

9 !

on the f i g u r e s

J

~

x

In n - d i m e n s i o n a l condition Ila~-all

-

:

:

~

c a s e n>l a n y c h o i c e o f a • s a t i s f y i n g

(a~-a)T(~-~)

< 0 should give better

is n o t too big)

b y a ~ - a = -(~-~) ak+ 1 = a k + ~ ( x - ~ ) ,

below

which 9<0 •

b u t the o p t i m a l leads

results

direction

to the a l g o r i t h m

the (if

is g i v e n

48

Example ~(x)

2.

Quadratic

support.

= a xTx + bTx + c we

maximizes

the H a m i l t o n i a n .

is a s u p p o r t

at the p o i n t

Suppose

found Then,

which

leads

~(x)

fix a*=a

then

b*

< b

for x >

b*

> b

for x <

c a n be i l l u s t r a t e d

j

After that

.......

the

= ~, w h i c h

= a*xTx+b*Tx+c *

as b e f o r e

>

~*(~)

,

takes

< 0

the

form

case we have

as on the

figures

)

same argumentation

the adjustment

(b*-b) T (~_~)

< 0

the one-dimensional

which

+

the condition

(b*-b) T(~-~)

For

for a s u p p o r t f(~)

to t h e c o n d i t i o n

(a*-a) (~T~ - ~T~)

If w e

~,

if n*(x)

x, we h a v e

~(~) < ~*C~)

that

a point

t

as

of p a r a m e t e r s

b k ~ 1 = b k + S ( ~ - ~),

below.

in E x a m p l e b should

B< 0 .

I

1 we conclude take

form

>

47

2.1

Literature

This chapter is based on the report H.F. Ravn: Upper Boundary Methods,

IMSOR, Research Report

No. i0, 1980. see also

R.V.V.Vidal:

Notes in Static and Dynamic Optimization.

IMSOR, 1981. H.F. Ravn and R.V.V.Vidal: teori og dens anvendelse,

En introduktion til Everett's I - If, IMSOR.

The idea of the method comes from M. Everett III: Generalized Lagrange Multiplier Method for Solving Problems of Optimum Allocation of Resources. Opns. Res., Vol. II, No. 3, 399-417, 1963. F.J. Gould: Extensions of Lagrange Multipliers in Nonlinear Programming.

SIAM J. Appl. Math., Vol. 17, No.

6, 1280-1297,

1969.

J.P. Evans, F.J. Gould, S.M. Howe: A Note on Extended GLM. Operations Research, Vol.

19, No. 4, 1079 - 1080, 1971.

A concise review of theorems for convex sets, convex or concave functions and supporting hyperplanes can be found in M.D. Canon, C.D. Cullum Jr., E.Polak: Theory of Optimal Control and Mathematical Programming. McGraw-Hill,

The papers

devoted

to the

m u l t i p l i e r s , related to

the

1970.

qua-

dratic supports, numerical methods of its use and convergence of these methods are

M.T. Hestenes: Multiplier and Gradient Methods, No. 5, 303-320, 1969.

JOTA, Vol.4,

48

A. Miele, Method JOTA,

R.D.

Vol.

Rupp:

Hestenes NO.

P.E.

Moseley,

of M u l t i p l i e r s 10, No.

and Powell

R. F l e t c h e r :

zation

Problems

G.M.

Coggins:

O n the

Programming

Problems,

1972.

of the M u l t i p l i e r

with Newton's

A n Ideal

Powell:

Levy,

Method.

JOTA,

Method Vol.

of

15,

1975.

J. Inst.

M.J.D.

m i c Press,

i, 1 - 33,

O n the C o m b i n a t i o n

2, 167-187,

mization.

A.V.

for M a t h e m a t i c a l

Penalty

Maths.

A Method

Function

Applics,

for N o n l i n e a r

In: R. F l e t c h e r

283 - 298,

1969.

for C o n s t r a i n e d

Vol.

(ed.):

15,

319 - 342,

Constraints

Opti1975.

in M i n i m i -

Optimization.

Acade-

CHAPTER

MULTISTAGE

3

SYSTEMS

51

0.

In this chapter we consider a problem of dynamic optimization of discrete systems. The problem presented is the so called fixed-endpoint problem. To solve this kind of problems mainly dynamic programming algorithms were tried up to now. These algorithms are very succesful unless they meet the problem of too many dimensions.

The other

methods

based

on the maximum principle were rather applied to the free-endpoint problems. The new approach given in this chapter provides a broader theoretical fremework within which both dynamic programming and m a x i m u m principle can be included. More specifically the argumentation of this chapter leads to the generalized maximum principle.

It requires much less restricting

assumption to apply it than the classical maximum principle and the classical one is a special

(linear) version of it.

Then the forward and backward version of the dynamic programming are fitted in the framework. Many results of this chapter are new ones especially those concerning the generalized maximum principle. The argumentation of this chapter relies heavily on the results developed in chapter 2. The reason is that when using the maximum principle the overall optimization problem is split to a number of subproblems where the ideas of chapter 2 may be

readi-

ly applied. The reformulated chapter 2 theorems constitute a significant part of the present chapter. Like in chapter 2 the statements of this chapter are numbered. The numbers match those of the paper by Ravn

(1980).

52

2.

Consider

the

following

problem

O

:

N-I

max

F. r i ~x i ,ui) i=0

xi+1

- xi -- f i C x i ' u l )

Ui £ Ui x0 where

= x0 '

XN

= XN

ri(.,.)

is a r e a l

function

f. (.,-)

is a n - d i m e n s i o n a l

column

vector

function

1

u. i

is a m - d i m e n s i o n a l

columm

x.

is

column vector

U.

is a g i v e n

1

a n-dimensional set

in

vector

Rm

1

Xo for

201.

We

and

xN

are given vectors

i = 0,...,N-1

shall

work

in

.

n+1

dimensional

space,

where

the

x. 1

are measured mulated

along

the

last

n

axes,

and where

the

accu-

criterion

Rilu0,ul,...,ui_1)

is m e a s u r e d

along

the

=

first

i-I Z r Ixj,uj) j=0 J

axis.

This

i = I,.-.,N

space

we c a l l

PR-space.

202.

We call

the

x.

1

states,

and we define

i [Ri-1. I x i• as e x t e n d e d

203.

If w e xI

let

take

states.

u0

take

all. v a l u e s

o n a set of v a l u e s ,

in

U 0 , we

t h a t we

shall

shall call

see t h a t the

set of

53

states reachable Likewise

xI

from

~0 ' denoted

will take on a set of values,

call the set of extended denoted 204.

Xl(X0)

XI(x0)

In accordance

or

states

reachable

XI .

that we shall

0],

from

x 0 = Ix0

Xl "

w i t h the analysis

the upper boundary

or

is included

note this upper b o u n d a r y

of

O

in

we shall assume

that

We shall de-

X1(x01 .

by

M

ub I (~0,xl) conceived

as functions

of

and

x I , and

{ub(x01 } conceived 205.

in

xI =

[xR11] in

X1

we let

U 1 , and -we-thus generate from

x I X2(xl)

reachable

from

xl X21xl) ' and the

ub 2 ~xl 'x2 ~

or

This we do for all of reachable of extended boundary

207.

{UB~}

reachable

ry

206.

and

as sets.

From a given values

UB I (xl)

assume

all

the set of states

, the set of e x t e n d e d (smaller)

states

upper bounda-

{ub 2 Ixl) } .

Xl

states

in

X1

We thus generate

X 2 (i.e. reachable

reachable

UB2(x2)

u!

states

or

This can be g e n e r a l i z e d

X2

the set

from x0 ) , the set

and the (greater)

upper

{UB2} . to any stage

i .

Note the follow-

ing characteristics:

(i~

ub~(x0,x~) : UB~(xl)

(ii)

ub~(~i_1,xi) <_ uBi(xi)

or

{ub 1(x0)} : {UB I} Vx~ c xi(x~ I)

54

208.

We shall assume that the upper boundaries in the proper sets of reachable xN 6 XN .

An optimal

solution we call

N-1 , with the corresponding x~

and

are all included

extended states.

And that

u*~ , i = 0,1,-..,

states and extended states

x*~ , i = 1,2,..-,N , respectively.

208a. Let us notice that from the definitions in § 202 and criterion the difference

of extended states

in § 201 the extended

states satisfy

equation

xi+ I = x i ÷ F i I x l , u i) where

FiIXi'Ui)

209.

Theorem.

x$x

is situated at

This is Halkin's Proof.

Principle

N

{UBi} , i = 1,2,.--,N .

o f O p t i m a l E v o l u t i o ~ H a l k i n (1964).

We start from the stage N. Let us assume that

I°l

is not situated at XN

= IfilXi'Ui) ]i(Xi,ui)j

and assumption N

{U0,Ul,---,~N_I}

{UBN}.

Then from the definition of

208 there exist a control that the corresponding

xN = N satisfies

which contradicts

the optimality of

x"*N •

sequence

point

55

Let

{u0,u I, .-,u~_l}

quences of controls

and

be optimal se-

{x~,x~,--,x~}

and states.

Assume that for

I
LxIj is not situated at

{UBi} . Then from the definition of

Xi and assumption

208 there exist

{~0,~l,---,~i_l}

that the corresponding

a

control

sequence

point

N

Xi =

satisfies

Apply now the control sequence

{~0,~I,...,~i,ui+I,...,u~} Above control sequence the constraints

is admissible,

u. 6 U . .

that is it satisfies

It is now easily seen that the

above control sequence produces

the value of the objective

function N-I

9=i

J

But the value of the optimal objective R N = R~* +

N-1 ?.

function

is equal to

rjlx~,u~l

j=i and we have

which contradicts

the optimality of

{U~,U~,''',U~_I}

58

210.

Theorem.

x*i

Proof.

Let

states.

is s i t u a t e d at

{ubi(x~_l)},

{x~,x~,-..,x~}

i = 1,2,'.',N.

be a sequence of o p t i m a l

We have

Lx[j F r o m the d e f i n i t i o n

of

(smaller)

upper b o u n d a r y

R$ _< ub i [~*i-I ,x~) and from the T h e o r e m 209

N o w from 207

(ii)

ub i (xi ~ ,xi) < uBi (x~) = R~ Then w e c o n c l u d e

R*~ that is

x* i

is ~ i t u a t e d

211.

Deleted.

212.

We d e f i n e a support point

x °i+l

=

n ~ j=l

It has the p r o p e r t y such that

at

{ub i (x*i-i)}

E oi+l (xi+ll to {ubi+ 1 Ixi) } at a as a f u n c t i o n of a n - d i m e n s i o n a l argument,

d e f i n e d on the set o ~i+I (Xi+ll

ubi(~il,x ~)

Xi+llXil

.

It has the form

J J ~i+i(xi+i)

that there e x i s t s a real n u m b e r

ki+ I

s?

rl*i+l (Xi+l) * + ki+l = ubi+l
~*i÷I(xi÷i) ÷ ki÷1 -> uh÷1 (xi'xi÷i) Finally we require 213.

Theorem.

If

~

that

has a solution,

exists,

i = 1,2,.--,N .

Proof.

Suppose

@,

with

states.

~*i+l(X~+l)

u? , i

For any

i

1 6

Xi+ 1 (X~)

be finite.

~*(.)

then at least one

i = 0,I,...,N-1

x@i ' i = 1,2,--.,N

VXi+

,

i

is a solution

to

being the c o r r e s p o n d i n g

take

~*.~(xi) -- I

X. l

o

k

sup uh (xl I,xi)

-- X.~ l

x I ~x* 1

~x i E X i (i-I) x*

Then X*I

214.

~*[xi) with

is a support

k =

We define the H a m i l t o n i a n

HiIxl,ui,~i+ll 214a.

is a g e n e r a l i z a t i o n

literature

i

as

)]

of d e f i n i t i o n s support.

two d e f i n i t i o n s

When the e q u a t i o n

at stage

- ~i+l[xi+fi{xi,ui

for the case of the linear

in the earlier were used.

Hi{xi,ui,~i+l)

= rilxi,ui)

The above definition Hamiltonian

at the point

ubi[ X *i-l'Xi)

to

u b i ( iX *_ l , x * )

of

However,

of H a m i l t o n i a n

of motion was w r i t t e n

in the

form

xi÷ l = gi(xi,ui) as in Katz

(1962), Fan & W a n g

(1965), Denn & Aris tonian was defined

(1965), as

(1964),

Propoi

Jackson

(1965)

& Horn

then the Hamil-

58

Hi(xi'ui)

= ri~xi'ui)

-

Pi+l gi(xi'ui)

The definition of § 214 is the straight generalization of the above definition. On other papers by Halkin Canon et al.

(1964),

(1966), Holtzman

(1966),

(1970) the equation of motion was defined as

xi+ i =

xi

+ f±(xi,u±)

and the Hamiltonian as

Hi{xi'ui)

= ri(xi'ui)

- Pi+l fi[xi'ui)

The appropriate generalization of the above definition to the case of the nonlinear Hamiltonian would be

Hi(xi,ui]

= ri(xi,uiJ - ~ i + l [ X i . fi(xi,ui )] + ~i+IIXi)

See the discussion in § 223a.

We prefer definition 214

which is more straightforward and leads to simpler formulae. 215.

If we know x~1 and x*i+l and we want to find u~i we obviously have a problem of type Q . Therefore the analysis of Q

in chapter 2 is directly applicable.

Specifically we get: 216.

Theorem.

Under assumption of differentiability

~x~)

~

[x~l

~uBICx~

~x.

~x.

~x.

l

l

l

i = 1,2,''',N

59

Proof. 111.

The left equality To prove

is just the same as in T h e o r e m

the right equality

we observe

that

uB (x ) = ub iCx ) +

> uh

÷

--

for all

i-I

x i 6 X i(x~_1 )

and the function

U(xi)

Xi[.x*i_l].

has m i n i m u m on

~x which 217.

Theorem. function

Proof. Theorem.

If

~i

~i+l at

ax

Then we have

ax

l

= 0 l

Proof. Theorem.

maximizes over

Ui

Hi(x~,ui,Ki+l)

for a given

then the c o r r e s p o n d i n g

xi+ 1

{ubi+ 1(xl) } .

See proof of T h e o r e m ~*i+I IXi+l I

x*1 + fi(x*,u*) H i (x i,u i,ni+l)

218a.

= l

x?l .

at

leads to the right inequality.

situated

218.

= UB i[x i) - ub i[x i)

115 in chapter

is a support

if and only if over

U

at the point

x[+!

maximizes

116 and C o r o l l a r y

is a strong

support

116a.

at the point

x* if and only if any solution u~ to: i+l * * max Hi(xi,ui,ni+1) over U i satisfies f± * * * M o r e o v e r any sequence Xi* + (Xi'Ui) = Xi+l " i = 0,I,''',N-1 solution Proof.

satisfying

to the p r o b l e m

=

.

See proof of T h e o r e m n~+1(xi+1)

u*1

2.

above condition

u*i ' is an optimal

0"

See proof of T h e o r e m

117 and C o r o l l a r y

117a.

is

60 219.

Theorem.

Let

~i+IIxi+l)

ubi+ IIx~,xi+l) U~i 6 U.i "

Let the number

support be constant ~*i+I Ixi+l ) point x~+l

solution

~u*,K* i i+i.)

is a saddle-point

restricted k

Proof.

to the p r o b l e m

Hi{x~'Bi'Pi+I) Pi+l

supports w i t h constant

= ri(xl'ui)

A sufficient

support

to

T - Pi+l

vector

condition

[x~÷fi(x~,ui ) ]

and superscript

is that

T

set

See proof of T h e o r e m be any v e c t o r

Xi+ 1 (x~l

in

to be z - d i r e c t i o n a l l y there exists

a

~0

~a + ( I - ~ ) b

+ ~z6S

and

be finite

and

x Oi+l E Int X i+l [x~l "

123.

R n+l .

convex

of a

at any point

ubi+l(x~,xi+l)

Proof.

6 [0,1]

for the existence

ubi+iIx~,xi+l)

on a convex

z

+ T~i+l( x *i+l )

support we have

concave

Let

if

121.

is a n - d i m e n s i o n a l

x°i+l E Xi+ 1 Ix[l

220a.

if and only

a transpose.

Theorem. linear

Then

"

See proof of T h e o r e m

denotes

.

to the function

to the class of strong i+l

O

* * = Hi(xi,ui,r~i+l)

In the case of the linear

where

~i+iIxi+l)

of a

is a strong support to ubi+l[x~,xi+l) at a = x*l + filx~'u~ ) and u@l ' i = 0,1,-.-,N , is

an optimal

numbers

220.

0 = x~i + f i Ix~,u~), xi+l ki+ I from the d e f i n i t i o n

for all supports

(Pi(Ui,T~i+l)

219a.

be a strong support to

at any point

A set

S c R n+l

if for each such that

a,b E S

is said and each

61 220b.

It is seen that any convex for any vector relation

221 .

z

set is z - d i r e c t i o n a l l y

since in the case of a convex

of § 220a is always

satisfied

Theorem.

ubi+llX~,Xi+ll~

is concave

Xi+l(x~l~

~s e l - d i r e c t i o n a l l y

(I,0,''',0) Proof.

Assume

that

eI el

is the =

ubi.i[x:,xi+l) of a concave

is concave.

function

Then

(§ 122b)

the set

is convex then for any two points

Xl,X 2 £ Xi+i(x:) ~i : 0

I

where

that is

if

.

from the d e f i n i t i o n Xi+iIx:l

R n+l

set the

~ = 0 .

if and only

convex,

vector of the first axis in

with

convex

and for any

~ 6 [0,1]

there exists

that

uh., :x:,~x: . :i-.~x2] =~uh.: :x:,xl), c1-.~uh. ~ (x:,x~l*Bl Let us take any two points

~

=

Xl,X2 6 Xi+l(X:)

~2 =

x i,x 2 c x~. i (x:)

l For any

2

~ 6 [0,1]

Let us consider

the point

~x I + (I-~)x2 6 X i + l ( X * ] .

the value

ubi+ I [x*,~x I + (1-~)x2] + (1-~)ubi+l[x*'x2) +

= ~ubi+ 1(x~,x 1) +

+~i

~[ubi+ I Ix:,x,) - r,],

= ~rl + (I-~)r2 c~-.~

[ubi+~ :x: x21 -

As we have

{~x I + (I-~x2~:

x:, i(x:)

luh.l[x:,~x I * :I-~x2]}: ~:,I(;.:) ubi+IIX*,Xj]

>_ rj

+

j = 1,2

r~]

,~,

62

then for

S=

~Eubi+~(x~,xll-rl]+ (I-~)[ubi+I (x*,x2)-r2] + ~I >_0

there is

Ux I + (1-~)x 2 + Se I 6 Xi+1(x*)

that is the set

Xi+l(x~)

Assume now that the set

is el-directionally convex. Xi+l(X[)

is el-directionally

con-

A

vex, that is for each £ [0,1]

Xl" x2 E X i + l [ ~ )

there exist

B > 0

and each

such that

%IX1 + (I-~)X 2 + ~e I E Xi+I(X*) We shall show first that the set

Xi+1(x~)_

argument is taken from Canon et al. p. 84).

is convex

Let

x 1,x 2 ~ xi÷ I (xl) =

X2

=

1

2

Then from the directional there exist

B > 0

x~,x 2 c xi+ ~(x~)

convexity for any

~ E [0,1]

that

~r I + (1-~)r 2 + B] ~x I + (1-~)x 2 from which we conclude that

~x I + (1-~)x 2 c xi+1(x~)

that is, that

Xi+1(x* )

is convex.

(the

(1970), Theorem 4.2.3,

6 [0,1]

83

Let us now take ubi+ I (x*,x I) 9. 2 =

uh+Ix2(X~ ,x2) ]

Xl,X 2 E Xi+ 1 (X[)

xI from the directional exist B > 0 that ~ubi+l(x*,xl)

convexity for any

+ (1-~)ubi+l(x*,x2J

6 [0,1]

there

+ B1

~x I + (1-~)x 2 But from the definition of

Xi+l(X[)

~ubi÷iIx~,x~) ÷ (1-~)ubi÷~(x[,x21 ÷ ~ _< Uh÷l[X[,~ I + (I-~)x2] and, as

B > 0 , we conclude that the function

- i I] ubi+ 1 ~[x~'xi+

is concave.

221a. In the case of a quadratic

support

.iCx~,ui,pi÷1,qi+1) = riCxl,ui)

pTi+1[x[÷fiCxl,uD]

where the superscript T denotes a transpose, Pi+1 is a n-dimensional vector and Qi+1 is a nxn symmetric matrix. 221b. A sufficient condition for the existence of a quadratic support to ubi+lIX~,Xi+l) at any point x°i+l £ Xi+1(x~) is that ubi+IIx~,xi+1) be twice continuously differentiable in Xi+IIx~) and the function of the Hessian matrix, defined in Theorem 125

a2 uh÷1 (x. xi÷1) aXi+ 1

64

be b o u n d e d

in

Xi+iIx*) , that is

uh +i (xl'xi+1)

T max z z T z=l

Z < S _

~xi+ 1

z - real n - d i m e n s i o n a l Proof.

222.

for all

xi+ I E Xi+ 1 fxI)

M - real number vector.

See p r o o f of T h e o r e m

Theorem.

If

@

125.

has a s o l u t i o n

u ~ _ 1 , x [, i = 1,2 ..... N, then

there e x i s t s u p p o r t s

~*i+i '

(i)

u*1

Hilx*'ui' ~*i+ 11

(ii)

H i ( x * , u i , ~ * + l ) + ~i+l(X*+ll

maximizes

i = 0,1,---,N-1 over

so that Ui '

has the s a d d l e - p o i n t

* i) s

Ir~i+

(iii)

if the f o l l o w i n g

- ri(xi,u~)

and

differentiable

assumptions

are satisfied:

fi(xi,u~) , i = 0,I,.-.,N-I with

respect

to

x.

1

at

the

are point

x~ , 1

- UBi[xi) , entiable X.

1

i = 1,2,...,N in

a small

open

are c o n t i n u o u s l y neighbourhood

of

differx*

1

•

then ~x

~x.

l

1

This is the n o n l i n e a r M a x i m u m Principle. Proof.

(i)

F r o m the a s s u m p t i o n s

exist supports

~ i+I Ixi + i 1

ubi+l[x~,xi+ll

at the points

from T h e o r e m

218.

'

i

=

and T h e o r e m 213 there 0,I,...,N-I

x~+ 1 .

Then

,

to

(i) follows

in

65

(ii) Let the numbers ki+ 1 from the definition ports be constant for all supports ~i+l(xi+l) . let us restrict to the class of strong supports. (ii) follows from Theorem 219. (iii)

of supMoreover Then

From Theorem 209 for an optimal control we have UBi+I( x*i+l ) = UBI(xl) x*+1

Let us construct

* + ri[x*,ui)

= X*~ + fi(x~,u*)

a set

given by

Bi (xi) ÷ r i (xi,ui)

i[ X±

~i+l = {Xi+l : Xi+l =

and let ~Bi+ 1 (xi+l) F r o m the c o n s t r u c t i o n

,

X i g Xi}

+ fi(xi,ul)

be the upper boundary of

Xi+ I .

we h a v e

x* x* UBi+1(i÷i) = ~Bi+1(i+I) Xi+ 1 E

UBi+ l(xi+ I) ~ ~Bi+ 1 (Xi+ 1)

ci+ I

{xi+ I

NOW, from the assumptions

xi+ I -xi÷q(xi,ul),xicx there exist

~*i+1 (Xi+l) to UBi+I (Xi+l) relations are satisfied n*i÷I(i+I) x*

+

ki+1

at

= UBi+

a

this and previous

~*i+I (xi+[) is a support to the point x*i+l" Then

~}

support

x~+ I , i.e. the following

x* I (i+1)

"*i÷i (xi+1) ÷ ki+1 -> uBi÷1 (xi+1) Comparing

Ci+ 1

expressions ~Bi+ 1 (xi+l)

xi+ I £ Xi+ I we conclude over

el+ 1

that at

66

i+i(i+i)

~*

+ ki+ 1 = ~Bi+l(

X*

i + l ) = UBi(X~)

X*

i+i (xi+1) + k i + I ~ ~ i + l [ x i + l )

~ UBi(x i)

+ ri{x*~i , - i,"*~ + ri(xi,u[)

Xi+ 1 6 Ci+ 1 , X i 6 X.1 which we can write UB i(x*) + r i(x*,u*) x.l 6 X.l

UB i(xi) + r i(x i,u*) From the above expressions we conclude that the function

P(xi)

= UBi(xi)

+ ri(xi,u*)

X$1

maximizes

- T~+I [Xi + fi(xi,ul) ]

over X i . Now, from the assumptions fi (xi'u*) , r i[xi,u* ) and UBi(xi) are differentiable at the point x*z " Moreover UBi+ I (Xi+l) is also continuously differentiable in a small open neighbourhood of xi+ 1. so that a continuously differentiable support ~*+1(Xi+l) exists. Then P [xi) is d i f f e r e n t i a b l e and ~P(X*) ~X i

~UB i (X*)

=

8x i

* 8r i (x*,ui)

+

X* ~Tt*+l(i+l) [

~x i

~Xi+ 1

[ I+

~fi [X *i 'u*) ] ~X i

But from Theorem 216

i [xl) ~x i

(xl) 8x i

which after insertion to the previous formula and after rearranging terms leads to

8xL{ri (xi ,ui ) - u~+l [fi (xi,u*)] }

=

~X.1

x i = x*l Now from the definition of Hamiltonian we get

(iii).

J =0

67

222a.

We s u m m a r i z e

n o w the a s s u m p t i o n s

we h a v e m a d e d u r i n g

(i)

no a d d i t i o n a l

(ii)

the n u m b e r s constant class

(iii)

proof

ki+ I

for all

Corollary.

Assume

i = 0,1,-.-,N-1 assume

that

that

222C.

that

of

x?

1

fi[xi'u[ 1 '

X

=

respect

to

xi ;

differentiable 1

in a

; then

uBi+ICx*

8X i

is a simple

the d e f i n i t i o n

in a small o p e n

with

in

* C i+i) qCx 'ui)

uBiCx

~Xi+ 1

consequence

~X.1

of T h e o r e m

of the H a m i l t o n i a n

§ 214,

and

216.

We g i v e an e x a m p l e cannot

and

is c o n t i n u o u s l y

~Xi+ 1

222(iii), Theorem

r i[x i,u~)

, are d i f f e r e n t i a b l e

The proof

and o n l y the

considered,

i = I ,2 - - - ,N

8UBi+ I x*

~X i

Proof.

has b e e n

differentiable

x*

small o p e n n e i g h b o u r h o o d

i i)

which

are:

of s u p p o r t s

~k+l(Xi+l)

supports

supports

of

UBi[xi)

~r. x*,u*

supports These

f r o m the d e f i n i t i o n

continuously

neighbourhood

222b.

222.

assumption,

of s t r o n g

~*~Xi)z

concerning

of T h e o r e m

be dropped.

showing

that the a s s u m p t i o n

Let us c o n s i d e r

r i x i , u i I = U 2i

Xo = 0

filxi'ui)

u,

= Xi + Ui

1

a 2-stage

6 [0,11

+

X1 = U0

~2 = ?

l

problem

i = 0,1

is

X 2 = 2X 1 + U I = 2U 0 + U I

x~ 6 Int X.

U o , U 1 6 [0,1]

1

with

68

We

have

the

xI E

the

sets

of

reachable

[0,1]

optimal

x2 6

states

[0,3]

control

o

u~--

~2 1

! I

~2

2

-

U~

=

x2 > 1

and

the

upper

X2 > 1

boundaries

2 x2

uB1(xI) : x21

UB2(~)

0<x2
=

I - i)2 I +~(X 2

Let We

us

take

! I

x2

= 0.5 .

Then

u~

= 0 ,

u~

I <X2<3

= 0.5 ,

x~

= 0 .

have

~ri (xi'ui) = ~x.

~fi(xi'ui) =

0

3x.

1

1

i

=

0,1

1

and 8 U B I (0)

3 U B 2 (0.5) = 0

=

~x i

Now

the

is e q u a l

left to

I

~xi+ I

hand -I

side and

of

the

Theorem right

222b e q u a t i o n

hand

side

to

for +I .

i = I

69

223.

The usual

linear M a x i m u m

Theorem.

If linear

of d i f f e r e n t i a b i l i t y

supports

U*l

(ii)

8H i ~x~,u i,pi+i ) T ~Pi+l

(iv)

maximizes

exists

then under assumptions

H i ~x* ,u i ,pi+l * )

=

~Hi(x~,ui,pi+ ," , i) ~x. l

Hi[x*'ui'P*+lJ

is:

and a s s u m p t i o n s

(i)

(iii)

Principle

-

=

x *i*1

of T h e o r e m over

222

U.I r

'

- Pl

+ PT+I x*i+i

has the saddle-point

(ul,p ÷I) Proof.

We easily recognize

Theorem

222(i),(iv)

to be a special

and

(iii) to be a special

ly,

(ii) can be o b t a i n e d

§ 219a.

(i) to be a special

case of

case of T h e o r e m

case of T h e o r e m

222(ii)

222(iii).

by d i f f e r e n t i a t i n g

Last-

the Hamiltonian,

As we have

x* + fi[x*,u * ]j. . ~, = X*i+l then .

Hi[x*'ui'Pi+l)

Now, d i f f e r e n t i a t i n g get

(ii).

= rilx*'u*l

T

- Pi+l

both sides with respect

x*

i+1

to

P~+I

we

70

223b.

In

§ 214

w e have d i s c u s s e d

of two d i f f e r e n t Theorem

222 is a r e s u l t

in § 214 w h i c h Hamiltonian ralization arrive

of the d e f i n i t i o n

have

of H a m i l t o n i a n of the

in § 214a.

definition

version

Hamiltonians.

generalization

as a first

of the second

would

generalizations

of the linear

is a n o n l i n e a r

discussed

to a n o t h e r

222(iii)

two p o s s i b l e

definitions

Using

of H a m i l t o n i a n

of T h e o r e m

222 w h e r e

linear a genewe would

the r e s u l t

the f o r m

=

~X i and in the

~Xi+ 1

linear v e r s i o n

in T h e o r e m

~X.1 223

(x,ui,pi÷1) (ii)

T ~Pi+l

(iii)

~X.

=

X*

i

-

= P[+l

x*

i+1

- P[

1

This

is also

definition

224.

the c l a s s i c a l

of the linear

We n o w i n t e r p r e t

~ .~ _

tion of the p r o b l e m

result

of

other

conditions

UBN[XN,e0,~I,''',~N_I) for given

E

are s a t i s f i e d

In the

formula-

we redefine:

xi+ I - x i = filxi,uil with

for the second

Hamiltonian.

in terms ~

obtained

- ei

unchanged.

We d e f i n e

= UBNIXN,~ 1

and we a s s u m e

that

for the r e d e f i n e d

as the o p t i m a l

the c o n d i t i o n s problem.

value

of § 208

71

225.

Theorem.

Assume

i=1,2,...,N

(i)

the

that

ubi+l(x~,xi+1) an o p e n

~*i + 1 ( x i + 1 )

sequences

conditions

are

is c o n t i n u o u s l y

neighbourhood

i = 1,2,.-.,N

(ii)

for o p t i m a l

following

of

x*i+I

- a support

to

UBN(XN,E )

(iv)

is c o n t i n u o u s l y

neighbourhood

d i m u.

U.

Xi+l'

of

at

at t h e

x*i+1 t

differentiable

in an

E = 0 ,

1

q[x ,uil spect

in

= m = n = dim x.,

1

(v)

in

ubi+1(x*,xi+l)

x*i+I - is d i f f e r e n t i a b l e i = 1,2,-.-,N,

open

differentiable

,

point

(iii)

u~_ I , x~ ,

satisfied:

to

is c o n t i n u o u s l y ui

in an o p e n 1

and

~U i

1

differentiable neighbourhood

is n o n s i n g u l a r ,

of

with

re-

u~

in

i = 0,1,...,N

then x*

,0) 1

Proof.

Let

* x*i+I xi'

be o p t i m a l

states.

two e q u a t i o n s

x*i÷1 o

÷ fi(x 'u )

x*i+1 = x*~ + filx['u~ ) - ~i

Let

us c o n s i d e r

,

72

We c l a i m ~i = 0

that

there

that

is an i n t e r n a l reachable short for

fixed

inverse,

sets

* xi

for a n y fies

of

proves fixed

let

there

first

ei 6 E i

As

part

and

the r e s u l t

u? i

NOW,

let

f?~ 1

of

§ 224.

We

(iv)

give

and

is c o n t i n u o u s

a

(v),

in an o p e n

fi

now the

Ei

fixed

which

map open

equation.

image

of

ui E Di c Ui is an o p e n claim.

x~i

x~+ 1 ,

f[l

second

be t h e

Di

of o u r

of

and

the

which

set.

Then satis-

This

Furthermore

for any

i m a g e of Di is an c Ci+ 1 . Because Di c Ui

say proves

is a c o n t i n u o u s

For

Di .

the

second

function

part

we h a v e

of

moreover

u¢ E~

I"

{6i}k= 1

k = 1,2,...

fi

exists

and

C Ei+i c Xi+. (x~l claim.

of

assumptions

both

Consider

x*i+l

neighbourhood

then our

Then

equation,

the

neighbourhood

u?i t s a y D i c U i , and has a continuous x ~ + 1 - x~ is t h e o n l y p o i n t t h a t

sets.

ei 6 E i

From

function

= u~ .

and

second

open

the

such that

to o p e n

open

for the p r o b l e m

l

x~ ,

fil(x~+ 1 -x~l

fixed

x~

of a p r o o f .

neighbourhood

a small

ue is an i n t e r n a l p o i n t of U i and x~ i i+l p o i n t of X iE+ 1 (x[] , t h e set of s t a t e s

from

sketch

exists

and

be a sequence lim n-~co

that

sk = 0, i

x*~+,

= x.*,. • f~ (x~,u~)

X* i+1

=

that

and

- ~i

ek E E i , u

be a s e q u e n c e

k=l

u{,_ ,, u.*

k = 1,2,..-

X*

and

ri+i(i+i'u~)

is s i t u a t e d

at t h e u p p e r

X*

i+l

boundary

of

sumption

6 i [x~) . xi+~

Then

u~l E D i .

Now,

from

as-

(v)

. + ~kAuk]

k=

~i

_**

i+t

+ x*

i

+ f

i

(x.~,u.~)

+

* + e~A,.,.~] ~f~ (x*,u i u " (u~ - u I) aui

afi (X*'Ui

u

I"

~u. " 1

Au~ = u~k - u* ek o <

< 1

k

(ui -ul]

°

73

Defining

UBN(~N,O,''',o,~k,o,''',O)

from assumption

-- UB~(~N,C k)

we have

(iii)

~ ~f~Cx~,u~ + e~u~U~Cu~u*~ ~UBN(xN, o kEci) -

k = 1,2,-.. ~S i

~U i 0 < 8k < I

F r o m the other side we have N-I

UBN(~.,0)

: uh+1(x~,x,+ ,) ÷ E;

,

,

uB~(~ ,~) : uh+ Ifxl,xi÷ I) ÷ where

ubex+l(x*'xi+l)

Xi+ 1 Ix* )

z

j=i+l

r f~,u*] J

J

N- 1

rj~/x*j,~j; ,~*] j=i+1

is an upper b o u n d a r y

of the set

Then

But

ubi÷ 1(x~,xi+1) *

For given

u~ 1

xi+1

As

1

.

k

we have now

i

u ki 6 D i c U i E

.

= u h [ x*i-1'xi*) + r i(x i,ui)

then

x ki+I 6 Xi+l (x~) "

.

xk

ubi+ 1(xi,xi+1) = ubi÷ I (x~, i÷i) and from a s s u m p t i o n

(i)

So we conclude

that

74

*i) -OB(~,0) ~ , (x~.~ ~ubi+It

.......

=

* ~ ~) uh+ ~ (x*.xi+ * ~) = uh+ ~ (x,.xi+ -

IX* X* + k k i' i+l exAXi+l)

~i+i

, ~uh+ ~(x~,xi÷1

(x ki÷~ x L ~

=

+ ek , k

k k axi+1) ~q(x~.u~ + euAui)

~Xi+ 1

~U i k = 1,2,''"

k = Xi+l

~X~+l

- x~ ~+I

~uk = u~ - u* • I 1

0 < Ok < I -- x --

N o w we h a v e ~UBN(XN'sE. ek~k~ ~ i)

~q(x~ 'Su.i u* ÷ e u~ u ~l)

l

(U k - U*I

1

8ubi+ I (x~,xi+ I*

+ 8kAxi+l)

~ui

~Xi+ 1

(u[ ui) k = 1,2,...

that

is

k k "~UBN(XN, eesi)

k .) ~ f ~x*,u* + e k a u k) aUbz+1(x I 'Xi+ * I + 8kAx x i+l i ~ 1 1 U 1

DE.

8xi+ 1

1

1

Cu~ up o AS

U ki - u ~

fi(x u*÷0 u l ,0

and

l

3UBN(~N, 8e~il k k

~ubi÷ 1(x~, * =

1

NOW,

is n o n s i n g u l a r ,

~u.

then

letting

gk ÷ 0 i

Xi+ I ~xi+ I

we have

+

k

k

%x~Xi+l)

Ax k _ ÷ 0 i+1

8UBN(XN,01

8ubi+ I (x*,xI+ 1 )

~i

~Xi+l

k = 1,2,...

and

o

75

Because

of the assumptions

UBN(XN,0 )

and

sequence

{e~}

ubi+l(x~,x~+t)

into account T h e o r e m

1.

Consider

for any sequence

Now, we complete

the proof

216.

225a. We give some examples

illustrating

Theorem

225.

a problem

max ~ (u0) 2 - (u~) ~] = 0

x I = 2x 0 + u 0

Xo

x 2 = 2x t + u t

9, 2 ;

u0,u I 6 R

K2

That is max [- (u0)2 - ( U t )

2]

x I = u0 x2 = 2u0 + ul

The L a g r a n g i a n

~. =-(Uo)2 and the optimal u~ = ~2 x- 2

is equal to

-

(ul)2

+ X(2U 0 + U! - X 2 )

solution

u~

=

is

t x2

2

x~

B

= ~ x2

Then we have

UB!(xl)

= -(Xl)2

UB2Cx2J = - t C x 2 ) 2

and

aUB i (x[) ax t

4 = -~.

x 2

of

the limit for the special

is equal to the limit

equal to derivatives.

Example

of d i f f e r e n t i a b i l i t y

aUB2 ( ~ 2 ] ax 2

2 -

= -~ x 2

and

taking

76

Now,

our

x I

redefined

problem

= 2x 0 + U 0 - t o

X 2 = 2X 1 + U I -

That

Xo

= 0

u0,u I 6 R

~2 = ~2

E 1

is

max x!

~2

[-(Uol = u0

- ~0

= 2u 0 + u I - 2e 0 - c I

The L a g r a n g i a n

L

is e q u a l

= -[Uo) 2 -

and the optimal

u~

2

E(~2 2

to

[ul)2

+ x[2Uo÷U

solution

-

+

~ -

2%-~

- ~2)

is

÷2%+cl)

x~ -- ~-(x2 + 2% Then

is

u~

;

I

- -

~(x 2 +

2¢ 0 + e l )

cl)

we h a v e

UB l(x 1,~)

= _(Xl

+

UB 2 ix 2 ,E)

t0 )2

= - ~I[ x 2

+

2C 0 + CI )2

and

aUB2(~2,0) aE 0

which

should

previous works.

~UB2 (~2' o)

4 -

=

be c o m p a r e d

page.

This

2

=

- ~ x

with

values

is an e x a m p l e

5 x

on the b o t t o m

where

Theorem

of the

225

77

Example (iv)

2.

may

We

show now

lead

to a w r o n g

max[-[u0 )2-

that

vio,lation

result.

assumption

a problem

[Ul )2]

xI = x0 + x0 + U0

x ~0

=

2 2 x I -- X o + U o

X

of the

Consider

=

0

U0

£ R

Loj

= x1 X2

=

=

~x 2 = x 2I + U 1

That

is

m a x [- [u0) 2 - [Ul)2 ]

x I

u 0

x 2 = 2u 0 + u 1 X22 = U 0 + U 1

The L a g r a n g i a n

is e q u a l

to

L = -(U0 }2 - (U I)2 + l l (2U0 + Ul - ~ ) and the o p t i m a l

solution

-1 -2 U~ = X 2 - X 2

1.

xI

2.

=

xI

=

x 2

xl, 2

-1

=

-1 =

is

X 2

-2

X2

-

2, x 2

-2 =

X 2

-i

-

x 2

+ 1 2 ( U 0 + Ul -~21)

78

Then we have UBI(XI ) = _ix11 )2 : _(x~) 2 : - ~ tI, I 2 2 x l ) I~ 2 - ~(xl)

and

~uBl(x ~) depends

on w h i c h

formula we choose

~uB 2 (x~)

The reason of this strange p h e n o m e n a extended

states

XI

has the following

I X I

wh£ch

=

is that the set of

I :

x I

is a line in a space

2 =

1

x 2 ,

I

form

26R

x I ,x I

2

..fXl,X I] :

2

1

xI

/

79

The f u n c t i o n

U B I {Xll

is d e f i n e d o n l y on the line

and any d i f f e r e n t i a t i o n out of the set

w i t h r e s p e c t to

tion

3.

or

x

leeds

X I , w h i c h has no i n t e r i o r points.

is then a case w h e r e T h e o r e m Example

xI

XI , This

225 does not work.

We show n o w that the v i o l a t i o n

(v) m a y lead to a w r o n g result.

of the a s s u m p -

Consider

a problem

m a x [- [u011 2

I X~

,÷ ,o0,÷ ~Uo~

= X0

x0

=

=

= 0

L<

X2 = X0

X

[.0 l

X 1 =

1

IX 2

1

_--

2

x I + 8u I + 4u I

T h a t is max[-Iu01) 2 - lu~] 2 - Iu~) 2

I~

,u~÷ ~u0~

Ix~ = ~u0'÷ ,~ ~ = ~u~÷ ~u0÷ ,u~ ÷ ~o~

I We have

~x~ u~ and

=

ui

[< ÷,u~j

I

2

I

2

Uo,U0,Ul,U I £ R

80

~fi(xi'ui )

Now

the

set

XI

det

of r e a c h a b l e

states

2 1 x I = 2x I ,

= 0

is

1 2 ] x l,x I E R

LXl] which

is o n c e

more

a line

in a s p a c e

(x~,x~)

2 x 1

1 x 1

/ and the

226.

same phenomena

like

We now return

to o p t i m a l i t y

greater

boundary

upper

by u s i n g

the c o n d i t i o n

Theorem.

Let

such

the

that

V i = { (u i,xi) Then

:

U B i + [(xi+l)

Proof.

From

given

2 occurs.

§ 209.

Given

we can g e n e r a t e below.

transforms xi 6 Xi ,

xi

to

x i + fi(xi'ui)

max [r i(xi,ui) (ui,xi) Ev i

the definition

the

{UBi+I}

be a set of all p a i r s

ui

ui E Ui ,

=

condition

{UBi}

Vi = Ui × Xi control

in E x a m p l e

of the u p p e r

(ui,xi)

xi+ 1 ,

i.e.

= xi+l } "

+ U B ilxi)]

boundary

we have

81 £

]

i-1

UBi+l(Xi+1) = max Z rj(xj,uj) = max [r.(xi,ui) + max ~. r u0qU 0 j=0 ui£uik i u0qU 0 j=0 j(xj,uj)

xi+i-xi

: %

Ixi,ui)

ui_16Ui- 1

ui_16Ui_ I

xi-%

u.6U. !

x1-~0

--

!

1

:

fo(~0'Uo)

%(~o,U0) xi-xi-1 = fi-1 (xi-l'ui-l)

xi-xi-1 = fi-1 (xi-l'Ui-l)

xi+i-xi = fi (xi'ui) max

= max [ri(xi,ui)+UBi(xi) ] = u.EU. 1 i

[ri (xi ,ui) + UBi (xi~]

Xi+l-Xi = fiIxi'ui) 227.

The recursion in § 226 generates {UBi+l} from {UB£} . Normally the principle of Optimality, and then Dynamic Pro~rammin@, is presented in a backward version (to be presented in § 229). Due to the direction of transfer Xi+ I - X i = fiIxi,u£) , where x£ and u i uniquely specify xi+ I , we shall in the backward version only maximize with respect to u i , while in the forward version (§ 226) must treat both

228.

We consider {RUBi} all

x.1

ui

and

the reverse

as a set, from

xi

greater

RUBi(xi)

which

xN

as variables.

can

upper boundary

as a function be

reached)

at stage

(defined

which

i

for

we define

aN

N-I RUBi [xi I = max j=i ~. r.3 [Xj ,Uj )

Here the optimization restricted

by

is with respect

u i £ Ui,...,UN_ 1 6 UN_ 1

to

ui,ui+1,...,UN_ I and

82

Xi+l-Xi Xi+2-Xi+l

XN-XN-I We also define

229.

: fi(xi'ui) = fi+I (Xi+IfUi+l)

= fN-i [XN-I'UN-I )

RUBN(XN)

= 0 .

Theorem. RUB i(xi)

=

max u.6U. 1

{r i(x i,ui) + RUBi+ I [x i + fi(xi'ui)] } 1

Proof. The proof of this theorem is rather well known. We repeat it, however, to make the book self-contained. From the definition 228 we have N-1

N-1

RUBi(xl) : max ~ rj(xj,uj)= max [r.(x.,ui)+ max ~. rj(xj,uj)] = u.6U.1 j=i ui6Ui [ ~ i ui+iEUi+ I j=i+l ui+16Ui+i :

xi+l-xi = fi(xi'ui~

i UN_I6UN_ 1

UN_I6UN_ I

Xi+I-X i = fi(xi,ui)

xi+2-xi+1 = fi+l (xi+l'ui+1)

xi÷2 xi÷l = %÷l(xi÷l,ui÷l) %-~-I %-XN_ I : fN_I(XN_I,UN_I) U. 6U. [rICxl,ull ÷ RUBi+ 1 (xi÷1)] 1 1

=

xi+l-x i : fi(xi,ui) max {ri(xi,uil + RUBi+ 1 Ixi + fi(xl,ui)]} u.6U. 1 l

: fN-l(xN-l'%ll

83

231.

Theorem.

-RUB i(x*) ÷ UB~(~)

: UB~(xl)

-RUB i (xi) + UB~ (~N) t UBi (xi) for all

x.,

for w h i c h

1

the two f u n c t i o n s

are b o t h de-

fined. Proof.

F r o m the a s s u m p t i o n s u~_ 1 ,xj*

quences

208 t h e r e

, j = 1,2,- .-,N

N-I

that

u0EU 0 j=0

•

~0

xl-~o = f0 (%'u0)

ui-IEUi-1 ~,u ~)

x I* -x*0: f o l X

uC,~. 1

z

xi-xi-1 = fi-1 (xi-1 'ui-1)

* * ) xi*-xi-i fi-l(x*i-1'ui-1

ui+IEui+1

xi+l-xi

X* -X* = f. x*,u* i+l .i i( i i)

UN_IEUN_ 1

" XN-XN-I

=

~N-X~_I= Then

se-

m~x ~ 5(xj'uj)

j=0

=

optimal

N-1

x* U* UBN(xN)-- ,. rj(j,j)=

x*

exist

i

=

filxi'ui)

=

fN_I[XN_I,UN_I )

fN_1(~_1,~_1 )

x*

as

is s i t u a t e d

1

at the a b o v e

specified

optimal

trajectory UBN[XN].. =

i-I N-I max ~. rj(xj,uj]__ + m a x ~. u0qU 0 j=0 u i 6 u i j=i

rj (xj,uj) =

:

ui_16Ui_ 1

X*-~0 = f0 (X0'U0)

x~'-xi-1 = fi-1 (xi-l'ui-l~ = UB i (x~) ÷ ~uB i (x*)

UN_1EUN_1

Xi+l-X ~ = fi(x*,u i)

~N-xN_I = fi(XN_I,UN_I)

84

which proves To prove any

the first expression

the second e x p r e s s i o n

Uj_l,X j , j = 1,2,...,N

of the Theorem.

let us observe

satisfying

that for

restrictions

we

have

UBN(XN)

Now,

:

repeating

N-I ~ r (x],u~] j=0 J

the earlier

N-I > ~ r ~) -- j=0 j (xj,uj

argumentation

we conclude

that

{ubi(x*_1,.)}

for

UB~(~] >_ UB i (x~) + RU~ (x~) 231a.

Corollary.

-RUBi(.)

those

for which

xi ,

is a support to RUB i (.)

and

ubi (x*-1 ' ")

are both

defined. Proof.

232.

The c o r o l l a r y

is a simple c o n s e q u e n c e

231, 209,

210 and o b s e r v a t i o n

of § 207.

Theorem.

If and only if

is optimal

both at Proof: x~1

{UBi}

and

then it is situated

{-RUB i + UBN(XN) } . and

Now,

in T h e o r e m both at

231 that if

{UBi}

N O W we only prove the reverse

Let us assume that {-RUB i + UB N IXN)} .

:

it is situated

{-RUBi + U B N ( X N ) } .

It has been already p r o v e d

is optimal

ment.

x$

of Theorems

x~

is situated

both at

That is we have

(x °) ÷

from the d e f i n i t i o n s

of

UB. (-) l

and

RUB.(-) l

and state{UBi}

i-I max E rj (xj,uj) Uo£U 0 j=0

UB i(x 0) =

xl-x0

:

0

° ui_16Ui_ 1

RUBi(x 0) =

max ui6U ±

= f0(x0'Uo )

"

Xi--Xi-i = fi-1 (Xi-1'Ui-l)

N-I E j=i

• •

rj(xj,u9) x

= •

xi+l-x 0i = fi(xO'u£ )

x 0 i

UN-16UN-I

• °

XN-XN-I

= fN-I(XN-I'UN-I)

Let us define the sequences which maximize the first sum {in definition of UB i ) by u~,u 0iw'''tui-1 0 o o ; x0 = x01xlt 0 0i and the sequences which minimize the second • '',xi_1,x 0 0 0 sum (in definition of RUB i ) by Ui,Ui_I,.°',UN_I ; 0 0 I, • "" tXN_irX 0 Then we have xi,xi+ N0 = XN

UB i[x O) =

i-I E r (x~,u~) j=0 J

RUB i Ix0) =

N-I E r (x0,u0) j=i J

and from the equality

UBNIXNI

(*)

= UBi(X0)

N-I = j=0E rj(xj0,ujl0

+ RUBi(x0)

But this means that the sequences o o o Xo,XI,--.,XN_ 1 are optimal. Then as a special case. 233.

u°o , o

X£0

... u o '

"

N-l;

is also optimal

Theorem. . UBi+ I (xi+l) - U B i(x~)

, = r i(x~,ui)

= - R U B i + I ( ix +Wl )

+ RUB i(x*)

86 Proof. and

From Theorem 209

x~•

at

{UBi}. i E

UBi+1[X*i+l) =

x*

i+I

is situated at

{uBi+i}

Then we have

i-I r (x~,u3) = ri(x.~,u~) + r. r (x~,u~) =

j =o

J

j =o

]

= r i(x I,u*) + UB i(x*)

which proves the left-hand equality.

To prove the right-

hand equality let us observe that

uBi+1 (X *i+i? - uBi(x*l = r i(x*,uj* and from Theorem 231

UBi+ 1 ( x*i + I ) = -RUBi+ 1 (x*i + l ) + OBN(XN) oBi (x~) = - R o ~ (x~) + Inserting

UBi(x* )

and

~ (~)

UBi+1(x~+1)

from the two last

expressions to the previous equality we get the required result. 234.

Theorem.

Under assumptions of differentiability

~uB i (x;) ~x.1 Proof.

~ub(xLi,x~) -

~x.1

~i (x~) =

~Xl1

~RUB i(x[) -

~x.1

The two first equalities have already been proved

in Theorem 216.

The last equality is then a simple conse-

quence of Theorem 231 under assumption of differentiability.

87

3.1

Literature

This chapter is based on the report H.F. Ravn: port No.

Upper Boundary Methods.

IMSOR,

Research Re-

10, 1980,

see also R.V.V. Vidal: IMSOR, The early

Notes in Static and Dynamic Optimization.

1981.

(incorrect)

derivation of the Linear Discrete Maximum

Principle is in S. Katz:

Best Operating points for staged systems.

& Eng. Chem. Fundamentals,

Vol.

Ind.

I, NO. 4, 1962, pp.

226-240. L.T. Fan, Ch. S. Wang: Wiley,

The Discrete Maximum Principle.

1964,

and the discussion

around it

F. Horn, R. Jackson:

Discrete Maximum Principle.

Eng. Chem. Fundamentals, pp. 487-488, R. Jackson,

Vol. 4, No.

Ind. &

I, pp.

110-112, No. 4,

On Discrete Analogues

of Pontryagin's

1965. F. Horn:

Maximum Principle.

Int. J. Control,

Vol.

I, No. 4, 1965,

pp. 389-395. M.M. Denn:

Discrete Maximum Principle.

Fundamentals,

Vol.

Ind. & Eng. Chem.

4, No. 2, 1965, pp. 240.

The correct derivation of the Linear Discrete Maximum Principle with convexity assumption

is in

88 H. Halkin:

Optimal Control

ference Equations.

In:

Control Systems, Vol. H. Halkin:

for Systems Described

C.T. Leonder

I.

(ed.):

Academic Press,

in

1964.

A Maximum Principle of the Pontryagin Type for

Systems Described by Nonlinear Difference SIAM J. Control, A.I. Propoi:

Vol. 4, No.

Automation

1965, pp.

1167-1177,

and with directional J.M. Holtzman:

Equations.

I, 1966, pp. 90-111.

The Maximum Principle

Systems.

for Discrete Control

and Remote Control,

Vol.

26, No. 7,

convexity Convexity

Discrete Systems. No.

by Dif-

Advances

and the Maximum Principle

IEEE Trans. Autom. Control,

for

Vol. AC-11,

I, 1966, pp. 30-35.

J.M. Holtzman: Discrete-Time

On the Maximum Principle Systems.

for Nonlinear

IEEE Trans Autom.

Control, Vol.

AC-11, No. 2, 1966, pp. 273-274. J.M. Holtzman,

H. Halkin:

Maximum Principle

Directional

for Discrete Systems.

Convexity and the SIAM J. Control,

Vol. 4, No. 2, 1966, pp. 263-275. M.D. Canon, C.D. Cullum Jr., E. Polak: Control and Mathematical Numerical

algorithms

under assumptions

linear support are discussed S. Katz:

Programming.

McGraw-Hill,

1970.

of the existence of a

in

Best Operating

Points for Staged Systems.

& Eng. Chem. Fundamentals, 226-240.

Theory of Optimal

Vol.

I, No. 4, 1962, pp.

Ind.

89

M.M. Denn, R. Aris:

Green's Functions and Optimal Sys-

tems.

I, II, Ill.

Ind. & Eng. Chem. Fundamentals,

Vol.

4, No.

I, pp. 7-16, No. 2, pp. 213-222, No. 3, pp. 248-257,

1965. M.M. Denn:

Convergence of a Method of Successive Approx-

imations in the Theory of Optimal Processes. Chem. Fundamentals, Vol.

L.T. Fan, C.S. Wang: Wiley,

1964.

4, p. 231,

Ind. & Eng.

1965.

The Discrete Maximum Principle.

CHAPTER

4

COMPUTER ALGORITHM

93

4.1

Introduction

In this chapter we propose an algorithm for the generalized maximum principle with quadratic support.

The algorithm works for

the problems with fixed initial and end points without any constraints on control and states. fy assumptions of Theorem 222

Moreover the problem must satis-

(nonlinear maximum principle)

from

chapter 3.

The chapter is organized as follows. the algorithm is given. scribed.

First a general idea of

After this the exact algorithm is de-

Then results of computations

presented.

for three examples are

At the end of the chapter some remarks are included.

They concern the experience gained from running the examples and possibilities of using this algorithm for more complicated problems.

These are mainly the free-end-point problems and

problems with constraints on control and/or state variables. The computer program for the algorithm was coded in basic FORTRAN IV.

4.2

The examples were run on an IBM 370/168.

Genera!.idea of the al@orithm

TO understand better the idea of the algorithm let us discuss at the beginning intuitively the relation between the multistage and the one-stage problems. stage problem neither the function boundary

ub(x)

the value of value of support

x

Suppose that in the oneg(u)

are known explicitly.

g(u)

for any given

= x

u £ U .

and for any given parameters

~(x)

= xTAx+

bx

H(u,~)

u0

A,b

~(x)

of the quadratic x0

ub(x) .

at which This can be

which maximizes the Hamiltonian

and then calculating the value

is to adjust the parameters support

find

We know the required

we can find a point

this support supports the upper boundary done by finding a point

nor the upper

We can, however,

A

and

b

glu01

= x0 .

Our aim

in such a way that the

supports the upper boundary

ub(x)

at the point

94

.

Once it is done a value

u*

maximizing the Hamiltonian

is a solution to our problem. In the multistage fix-end-point p r o b l e m we know the required value of

XN

in the last stage.

We cannot, however, readily

apply the one-stage approach simply by definition of x = Ixl,x2, .... XNl and u = lu0,ul .... ,UN_ll . This is so because we do not know optimal values of xl,x * * 2, . --,x~_ I and this way we do not know a value of x which is necessary to use the one-stage approach. way.

We can, however, proceed the other

For any support in the last stage

we can compute a point upper boundary

UbN{XN)

xN

~NCXN)

= xN T AN XN + bN XN at which this support supports the

as a solution to the generalized maxi-

m u m principle formulae provided this solution is an optimal solution.

This way we can use the concept of one-stage approach

for the last stage. X0N

For

a

given support

~N~XNI

at which this support supports upper boundary

the value

UbN[XN)

is

computed as a solution to the generalized maximum principle.

Now, the algorithm will consist of two loops which we shall call the higher loop and the lower loop. In the higher loop the parameters of the quadratic support in the last stage of the form

T AN XN = XN

~N[XNI

+

T xN bN

are ad-

justed step by step in order to get the support at the point xN "

The algorithm attempts to find the appropriate support

mainly by changing the values of

bN

shifting this way the sup-

port along the x

axis. The exact way of doing it will be exn plained in the sequel. However, if the rate of convergence of computed values of

xN

to

xN

is too small then the values

JJ on the main diagonal of the matrix A N are increased. In aN the algorithm the rate of convergence is chosen to be 2 i.e. the differences between

xN

and

XN

in each iteration should

be at most half of that in the previous iteration. values

a~ j-

are multiplied by

If not, the

10 .

In the lower loop the solution to generalized maximum principle is searched for.

This loop is executed for every values of pa-

95

rameters

aN

and

bN

fixed in the h i g h e r

level.

loop c o n s i s t at the b e g i n n i n g of c o m p u t a t i o n s i = N-I,N-2,...,2,1,0

ui

The lower

backwards

for

of the f o l l o w i n g v a l u e s

I°

a value

which maximizes

2°

the value of the v e c t o r

the H a m i l t o n i a n

b.

Hilxi,ui,~i+l~

of the s u p p o r t

1

~i [xi 1 = xiT Ai xi + b.l x.I

f r o m the c o n d i t i o n

(xi'ui'i÷i)

Ixi) =

~x.

~x

1

(the v a l u e

b0

1

is not computed)

and then of c o m p u t a t i o n s 3°

forwards

for

i = 1,2,...,N

of

the v a l u e of state from the e q u a t i o n = Xi ÷ f i Cxi,ui~

Xi+l

The values of

ai , i=

1,2,...,N

and b N d u r i n g e x e c u t i o n

of the

lower loop once kept c o n s t a n t unless the rate of d e c r e a s i n g the state v a r i a t i o n s

Ixi- xi_ll

the fact then the values

a~j~

is not sufficient.

on d i a g o n a l s

i = 1,2,...,N-I

are increased.

state v a r i a t i o n s

is a g a i n c h o s e n to be

If this is

of m a t r i c e s

Ai ,

The rate of d e c r e a s i n g 2

of

the

and the values

a~ j 1

are m u l t i p l i e d

by

10 .

The lower loop is e x e c u t e d enough,

Ixi -XiTII

i.e.

tions start again. computed value of IX N

- -

XNI ~ ~ .

calculated

where

£

~ E1.

The h i g h e r xN

The v a l u e

~

T h e n the h i g h e r

loop c o m p u t a stop w h e n the

has to be given,

the v a l u e

in the f o l l o w i n g w a y

[I + 10 e x p ( - Z ) ]

is the h i g h e r

are small

loop c o m p u t a t i o n s

is close e n o u g h to the v a l u e of

in the a l g o r i t h m

ei =

unless the state v a r i a t i o n s

loop step number.

XN

l

cI

say is

96

To start states

the c o m p u t a t i o n s xi ,

of the a l g o r i t h m

i = 1,2,...,N-I

the a l g o r i t h m

initial

values of

T h e y are p r o v i d e d to

f r o m the f o r m u l a

÷g(xN-x 0)

x i-- x 0 The

the i n i t i a l

, are n e e d e d .

values

of

a~ j ,

i = 1,2,...,N

are set to

I

and

l

the

initial

value

of

bN

is

computed

from

the

formula

b N = -2A N x N L e t us n o w p r o c e e d adjusting

to p r e s e n t a t i o n

parameters

drop the subscript

Let us s u p p o s e maximizing

N

indicating

at a p o i n t

the last

we h a v e

x0 .

of f o r m u l a e

the e x p r e s s i o n s

that given a support

~(x)

for

s i m p l e r we

stage.

= x TAx

found that

The d e r i v a t i v e

+ bT x

after

it s u p p o r t s of this

the up-

s u p p o r t at

is

8u(x0) 8x T

= 2Ax 0 + b

N o w we l o o k for a n e w v a l u e o f support

~*(x)

= x TAx

b , say

+ b *T x

~*

(~)

For differentiable equivalent daries

p r e t this c o n d i t i o n small

for an optimal

~u(x°)

upper boundaries

x0

that

~x

to the r e q u i r e m e n t

at the p o i n t s

b* ,

satisfies

~x

Now,

and m o t i v a t i o n

~o m a k e

the H a m i l t o n i a n

per boundary x0

bN .

and

the a b o v e c o n d i t i o n

t h a t the d e r i v a t i v e s x

are equal.

as an e f f e c t of a l i n e a r

is

of u p p e r boun-

W e can t h e n

inter-

approximation

of the

upper boundary.

substituting

for d e r i v a t i v e s

of

~(x)

and

~*(x)

w e have

97

2 A x + b*

= 2Ax 0 + b

that is

b* -- b + 2ACx ° - ~ ) This leads

to a r e c u r r e n t

relation

for i t e r a t i n g

__ b ( ~ )

+ 2A(x ~) -~)

b~+1~ The above

relation

dependently tinuously

a better

diagonal

values

correction the s u p p o r t

of

of b .

A .

Each change a jump

upper boundary

be equal,

of

the c o r r e c t e d

~(£)

This is the c o n d i t i o n

and in-

of t w i c e

con-

A

needs

increases

an a p p r o p r i a t e

in the v a l u e

of

will

Assuming

b*

occur.

x

at w h i c h

the d e r i v a t i v e s

and the n e w v a l u e s

~

that of the at the

i.e.

2~x (Z) + ~(Z)

This gives

A

the a l g o r i t h m

of

of o p t i m a l

the old v a l u e s

x (£) s h o u l d

(1969)

assumption

functions.

Otherwise

supports

with

by H e s t e n e s

both under

rate of c o n v e r g e n c e

is in a v i c i n i t y

supports point

first p r o p o s e d (1969)

differentiable

To force

b (£+I)

was

by P o w e l l

b

value

= 2Ax (£) + b (Z)

of

~(£)

= b (£) + 2 ( A - ~ ) x

proposed

earlier

as

(£)

by F l e t c h e r

(1975).

98 4.3

The a l g o r i t h m

Before

starting

define

the f u n c t i o n s

the p r e s e n t a t i o n which

of the e x a c t

w i l l be u s e d

( ) x "T A x. + b T x. ~i xi = l l • • 1 Hi (xi'ui'~i+1)

Now,

the a l g o r i t h m

I°

Read

2°

Set

= r i (xi'ui)

These

let us

are:

A. = d i a g [ a II 22 nn] 1 i wai '''" wai - Ki+1 [xi + fi (xi'ui)]

is

N,n,m,x0,xN,e

aNJJ xr(0)._

algorithm

in it.

,

:= I

x(0):= i

j = 1,2,...,n

X0 + i

r bN := bN

-

- x0)

i = 1,2,...,N

:= -2aN XN

£=0 k=0

HIGHER 3°

Set

K

LOOP

:= N

4°

Set

K[

:= K.:x = ~

i = 1,2,...,N-I

£ := £+1 a jj 1

:= I

j = 1,2,...,n LOWER

5°

Set

k

:= k+1

E 1 := ~:(1 + 10e-'%)

LOOP

i = 1,2,...,N

99

6°

i = N-1,N-2,...,I,0

For

(i)

u (k) i

from

(ii)

5

from

max ui

8°

For

i = 1,2,...,N

rui'~i

~i

+i j

i ~ 0

~x i

compute (k) X0

(i)

Xi(k)

(k) + fi Ix(k~ %1(k)~ := X i-I -I i- ' i-1 j

(ii)

if

i # N

C • :=

Ix . i(k) _ x (ik - 1 ) I

(iii)

if

i = N

cN

Ix (k) - x ( k - l ) N

For

i = 1,2,...,N

then

9°

H i l x ~ k-l)

~H, m 8x i

m

7°

compute

For

Jji

:= I

:=

and

else

i = 1,2,...,N-1

j = 1,2,...,n Jji

I

if

:= 0

K i := m a x

c3i

J

10 °

Set

x r := x l

11 °

For if

12 °

For if

13 °

For

i = 1,2,

,N

l

i = 1,2,...,N-I K i ~ K[/2

go to

i = 1,2,...,N-I Jj£

= 1

then

i = 1,2,...,N-I

15 °

and

j = 1,2,...,n

a~ j := 10a~ j

set

K[ i

:= K i

:= X 0

c~ ~ K i / 4

100

14 °

Go

15 °

For

16 °

If

to

5°

i = 1,2,...,N-1

max i

K i > ~1

set

go to

go

If

c N < KN/2

18 °

Set

bN

19 °

For

j = 1,2,...,n

if

JjN

to

:= b r N

= 1

then

aJJN

:= 1 0 a J J

(ii)

bj

:= b j + 2 1 a ~ J J - a

Go

to

21 o

b Nr

:= b N

LOOP

21 °

(i)

20 °

:= K i

5°

HIGHER 17 °

Kr i

jjN ) N xj

3°

aNjj

= aj j

j = 1,2,...,n

KN

:= m a x

9 22 °

bN

:= b N + 2 A N I x N - x N )

23 °

If

24 °

STOP

KN > ~

go

to

4°

c$.

101

a[=a£~1 2°

bt~ = b, " -2a, X, k L

4°

0 0

ai=1 , i * N i = ~+I

r . . . . . . . . . . .

,. . . . .

50[

I. . . . . . .

¢1 1 ==¢{1+ Ok+lk

h,

<

exp|-~))

f [b£+ t.a~.ai.l .ut) i

x, = x l - , * f [ u s ) c~=lx~-x[I c.- Ix.-~.l

7°

,t*N

1

8o I

Ci >" K~/4 ..* Jl " 1 ~r

90 I

K1 = cl ' I * N I x~ = x~

1°° i YES

'5°I 1

L

.

0

.

.

~

)

YES

f£

.

I°

o,°si.,0.1,,I

I

K[=K~ . i * N

6

.

NO

.

.

.

i KI=K i . I*N

13° I

I

_~OW~R__LOOP

.

1 7 o ~ - ' Y E S

®

a~ = aN ( NO

21 °

KN = CN YES

i b. - b. * 2aN{% - ~.)

22°

I

I aN ~ 10.a N

• • a.b = = ¢! = ~.K = J =

Fig. 4.1

counter In higher loop counter in lower loop support constants stopcriteria in higher loop stopcriteria in lower loop changes in 5 indicates too slow convergence

The algorithm.

102

4.4

Examples

Example

I

This is the example there Example

I.

which was used in chapter

The p r o b l e m

3 § 225a,

called

is

1

max Z - [u£) Uo,U I i=O subject

to i = 0,1

xi+ 1 = 2x i + U i x0

0

=

22 = c

The optimal

solution was shown to be 2

1

w i t h the optimal

criterion

2

value

R* = - ! c 2 5 If we want to use the g e n e r a l i z e d ratic

support

then the g e n e r a l i z e d

Hi(xi,ui,ai+ ~,bi+l)

The value

u

maximum principle Hamiltonian

is

= _(ui ) 2 _ a i + i ( 2 x i + u i ) 2 _ b i + i ( 2 x i + u i )

which maximizes

it is equal

1

0 bi+l + 4ai+l xi Ui = 2[I + ai+l) Now,

w i t h quad-

from the equation

8HI

~)~1

~x 1

~)x 1

to

i = 1,0

103

We o b t a i n

b I = 2b 2 + 4a 2 u I + 2(4a 2 - a l ) x I

The r e s u l t s ges.

of c o m p u t a t i o n

They correspond

the algorithm. xN

obtained

If the a b s o l u t e

from computations

t h a n ~ t h e n the a l g o r i t h m forms of aii tables

are shown

to 3 v a l u e s

i0 m o r e

calculations

on the n e x t pa-

value

of the d i f f e r e n c e

a n d the g i v e n v a l u e

stops

its n o r m a l

of the h i g h e r

f r o m the last n o r m a l the values

in t a b l e s

of c w h i c h w a s u s e d to stop

computation

between

xN is less

calculation

a n d per-

loop o n l y w i t h v a l u e s of the loops.

of the 10 last c a l c u l a t i o n s

In the

are s e p a r a t e d

by

line.

In the c o m p u t a t i o n

the v a l u e s

x0 = 0 and x2 = c = 5 w e r e

chosen.

104

Table

1

Higher loop step numbet

CPU

Number of lower loop steps

time

0.07

Final a11 in a step

Initial

xo=O

s.

a12

values

U

u0

i

of s t a t e s

X

i

R

x2

2.500

2.500

e=0.1

I

4

10

1

I .591

0.938

1 .591

4.119

-3.410

2

2

I

I

1.758

O.795

I .758

4.311

-3.723

3

1

I

10

1.665

0.786

I .665

4.115

-3.389

4

5

100

10

1.765

I .185

1.765

4.716

-4.521

5

5

100

10

I .846

1.165

1.846

4.858

-4.766

6

4

10

100

1.828

I .366

1.828

5.021

-5.206

.................i...

7

1

10

100

1.875

I .176

1.875

4.927

-4.901

8

I

10

100

I .908

I .118

1.908

4.934

-4.891

9

I

10

100

I .932

I .086

I .932

4.950

-4.911

10

1

10

100

1.950

I .063

1.950

4.963

-4.932 -4.948

11

1

10

100

1.963

I .047

1.963

4.972

12

1

10

100

1.972

1 .035

1.972

4.979

-4.961

13

I

10

100

1.980

I .026

I .980

4.985

-4.971 -4.978

14

1

10

100

1.985

I .019

I .985

4.989

15

1

10

100

I .989

I .014

I .989

4.992

-4.984

16

1

10

100

1.992

1 .010

I .992

4.994

-4.988

Optimal

values

-5

105 Table 2 Higher loop step number

C P U time 0.19 s. Number of lower loop steps

Final all in each step

a I2

u0

uI

xI

x2

2.500

5.000

R

.......... L .................

Initial values of states

x0=0 1

4

2

6

10

3

5

100

10

I

ie

=

0.01

1.591

0.938

1.591

4.119

-3.410

I

1.819

0.900

1.819

4.537

-4.117

10

1.809

0.876

1.809

4.494

-4.040

4

5

100

10

1.848

1.041

1.848

4.737

-4.498

5

5

100

100

1.862

0.990

1.862

4.714

-4.447

6

5

100

100

1.891

1.076

1.891

4.859

-4 •735

7

5

100

100

1.921

1.088

1.921

4.929

-4.872

8

5

100

1000

1.938

1.048

1.938

4.925

-4.856

9

5

100

1000

1.955

1.053

1.955

4.962

-4.929

10

5

100

104

1.965

1.030

1.965

4.960

-4.922

11

5

100

104

1.975

1.031

1.975

4.980

-4.961

12

5

100

105

1.981

1.018

1.981

4.979

-4.958

13

5

I00

105

1.986

1.017

1.986

4.989

-4.979

14

3

10

106

1.984

1.039

1.984

5.007

-5.016

15

1

10

106

1.988

1.017

1.988

4.994

-4.988

16

I

10

106

1.991

1.011

1.991

4.994

-4.988

17

I

10

106

1.994

1.008

1.994

4.995

-4.991

18

1

10

106

1.995

1.006

1.995

4.996

-4.993

19

I

10

106

1.997

1.004

1.997

4.997

-4,995

2O

I

10

106

1.997

1.003

1.997

4.998

-4.996

21

1

10

106

1.998

1.002

1.998

4.999

-4.997

22

1

10

106

1.999

1.002

1.999

4.999

-4.998

23

I

10

106

1.999

1.001

1.999

4.999

-4.998

24

I

10

106

1.999

1.001

1.999

4.999

-4.999

2.000

1.000

2.000

5.000

-5.000

O p t i m a l values

106

Table 3

C P U time 0.32 s.

!High- N u m er ber Final loop of all in a12 U0 uI step lower each numloop ber steps step x0 = 0 Initial v a l u e s of states 1 6 100 1 1.613 0.888

x1

x2

R

2.500

5.000

e = 0.001

I .613

4.115

-3.392

7

10

1

1 .819

0.910

1.819

4.534

-4.112

5

100

I0

I .805

0.882

I .805

4.492

-4.035

7

1000

I0

I .848

1.036

I .848

4.732

-4.488

7

1000

tO0

1 .862

0.983

I .862

4.707

-4.433

6

7

1000

100

I .894

1 .064

I .894

4.853

-4.721

7

7

1000

I000

1 .911

1 .017

I .911

4.839

-4. 685

8

7

1000

I000

1 .933

1 .053

I .933

4.919

-4.846

3

9

7

1000

1 .053

I .953

4.960

-4.924

5

100

I000 104

1 .953

10

I .964

I .030

1.964

4.957

-4.917

11

5

100

104

I .974

1.031

I .974

4.979

-4.959

12

5

100

105

1.980

1 .018

I .980

4.977

-4.956

13

5

100

105

I .985

1.018

1.985

4.989

-4.978

14

5

100

106

1.989

1.010

I .989

4.988

-4.976

15

5

100

106

I .992

I .010

I .992

4.994

-4.988

16

5

100

I07

I .994

1.006

I .994

4.994

-4.987

17

5

100

I07

I .996

1.006

1.996

4.997

-4.994

18

5

100

I08

I .997

1.0O3

1.997

4.997

-4.993

19

5

100

I08

I .998

1.003

I .998

4.998

-4.997

20

5

100

109

1.998

1 .002

I .998

4.998

-4.996

21

5

100

109

1.999

I .002

I .999

4.999

-4.998

22

3

10

i0 Io

I .999

1.004

1.999

5.001

-5.002 -4.999

23

1

10

I .002

I .999

4.999

1

10

10 'f'u 1010

I .999

24

1.999

I .001

I .999

4.999

-4.999

25

1

10

1010

I .999

1.001

I .999

5.000

-4.999

26

1

10

10 l o

2.000

1.001

2.000

5.000

-4.999

10

10 l o

2.O00

1 .000

2.000

5.000

-5.000

27

1

28

1

10

101o

2.000

I .000

2.000

5.000

-5.000

29

1

I0

101o

2.000

1.000

2.000

5.000

-5.000

30

1

10

101o

2.000

1.000

2.000

5.000

-5.000

2.000

1.000

2.000

5.000

-5.000

2.000

1.000

2.000

5.000

-5.000

2

1

2

5

-5

31

1

10

1010

32

1

10

10 l o

O p t i m a l values

107

Example

2

Consider

the

following

problem

2 max E u 0 ,u I ±=1

r i (ui)

Xi

= Xi-i

+ Ui-I

x0

-- 0 ,

x2

= c

-10(u 0 , i) 2 ro (Uo)

=

u 0 _< -I

0

-I

-10(u o - i) 2 r!(ul)

:

r 0

! \'

lem.

I

-I

X2 = U0 + U1 = C

above

problem

c a n be

The L a g r a n g i a n

L = r 0[u0)

I

I ! uo

(ui)2

AS we h a v e

the

< u0 <

is

solved equal

+ r l(ul)

Uo f

as a s t a t i c

optimization

to

+ l(u 0 +u I -c)

and we h a v e

-20 (u 0 * I) 3L

=

%

+

u o ~ -I -I


<

3u 0

-20(% -1) , ~L 3Ul Uo+U

= 2ul

I = C

+ I

= 0

I
o

I

= 0

prob-

108 Consider

now

I°

U 0 i -1

Then

we h a v e

3 cases.

1

i

u o = ~-gX

X

=

- I

u I = -~

-~-~11+c)

and uo

I

0 = -,~(l+c)

0 = ~(1+c) uI

- 1

N

Now,

when

and

u10

value

c ~ -1 give

is e q u a l

the

the b e s t

-I

The b e s t

controls

for t h i s

The

case

and

criterion

2 + ~ i 0 (I+c) 2 = ~90 (I+c)

2

< u0 < 1 controls

U 0 = +I

for t h i s

case

for

c < 0

U I = -I+c criterion [

are

u 0 = -I

1 for

and

is f u l f i l l e d

u 0 ~ -I

to

R = - ~ y10 (1+c)

2°

assumption

uI = value

is e q u a l

to

(I-c)2

for

c <_ 0

(I+c) 2

for

c >_ 0

R

I+c

c>

0

u~

109

3°

1 ~ U0

This

case

is s i m i l a r

0 = ~--(1-c)

U0

to

1° a n d w e h a v e

+ 1

u 0t = - - ~ - ( 1 - c ) c < I

90 2 R = ~-[11-c1

NOW we

can draw

criterion

values

for all

3 cases

R

I 2°\\I/

I°/

I

20

~

30 c

-I

The upper see t h a t are

boundary

I

for this

it is n o t c o n c a v e .

problem

1

is m a r k e d

The optimal

by thick

controls

line.

for the

We

problem

N

+

1!

O

II

X

I^

N ^

N

!

N

^

I^

+

0

!

I

0

I N

!

N

0

r-

I1

N

I~ .

I-1

0

I"1

"CS

I"1

I-J

fl

II

A

I A

Iv 0

0

N

+ ~

I^

Q

+1; +1;

II r

0

I^

0

-I-

,

',.o I ,""

II

0~"

0

Iv

I

-;

I ~ I ',-L.

0

X

IIII

__...-"I

II

I

Id

112 Now,

if w e w a n t to use the g e n e r a l i z e d

quadratic

support,

we have

H:t ( x t , u l , a 2 , b 2 ) As H a m i l t o n i a n e a s y to m a x i m i z e

m a x i m u m p r i n c i p l e with

H1

= (Ul)2

- a 2 ( x 1 + u l )2 _ b 2 ( x 1 + u l )

is d i f f e r e n t i a b l e ,

then for

it by e q u a t i n g the d e r i v a t i v e

a2 > I to zero.

it is This

gives

0 2a2 xt + b2 ul = 2(a 2 -1) Then we have

8x I

-2a 2 (x t + ul)

- b2

8x! = 2al xl + b l

and f r o m the e q u a t i o n

~H I

8~ I

8X 1

8X I

we obtain

b t = b 2 + 2x l ( a 2 - a t ) + 2a 2 u 1 A t last, the H a m i l t o n i a n

H0(U 0 , x 0 , a l , b l )

and we have

at stage

= r0(u0)

0

is

_ a l ( x 0 + u0 }2

b I (x 0 + % )

113

- 2 0 [ U 0 + 1)

-

- b1

2a l(x O + u O )

u 0 < -1

8H o

au °

-2a1 (X 0

--

+ UO)

- b1

-I < u 0 <

- 2 0 ( U 0 - I) - 2a Ifx 0 + u0)

Now t h e

value

maximizing

H0

1 <_u 0

- bI

is g i v e n

1

by

b I + 2a I x 0 + 20

Uo ! - I

2"(10 + e l ) b I + 2a I x 0 u0 =

-

-1 < u 0 <

2a 1

I

b I + 2a I x 0 - 20

2(10+al)

From g e o m e t r i c a l the a b o v e

1 ~ u0

interpretation

conditions

we

see t h a t

one

and only

one

of

is s a t i s f i e d .

In the c o m p u t a t i o n s

the v a l u e

of the c o m p u t a t i o n s

are

x2 = c = 8 w a s

in t a b l e s

4,

5,

6.

chosen.

The

results

114

Table

4

Higher loop step number

CPU time

Number of lower loop steps

x0 = 0

0.07

s.

Final a11 in each step

Initial

1

3

100

2

3

3

3

4

3

aI

values

R

xI

X2

4.000

8.000

c=0.I

7.421

1.224

8.645

54.510

u0

2

uI

of states

10

1.224

100

10

-0.378

8.852

-0.378

8.474

78.353

100

100

-I .591

10.077

-1.591

8.486

98.048

100

100

-I .909

10.184

-1.909

8.275

95.444

5

I

10

tO0

-1.965

10.209

-1.965

8.244

94.915

6

I

10

1000

-1.990

10.143

-1.990

8.153

93.085

7

I

10

104

-2.003

10.158

-2.003

8.155

93.123

8

1

10

104

-2.006

10.093

-2.006

8.087

91.752

9

1

I0

105

-2.008

10.101

-2.008

8.093

91.874

10

1

10

105

-2.007

10.057

-2.007

8.050

91.000

11

1

10

10 5

-2.005

10.031

-2.005

8.026

90.513

12

1

10

10 5

-2.003

10.016

-2.003

8.013

90.252

13

1

10

10 5

-2.002

10.008

-2.002

8.006

90.117

14

1

10

10 5

-2.001

10.004

-2.001

8.003

90.050

15

1

10

10 5

-2.001

10.002

-2.001

8.001

90.019

16

1

10

10 5

-2.000

10.001

-2.000

8.000

90.005

17

1

10

10 5

-2.000

10.000

-2.000

8.000

90.000

18

1

10

10 5

-2.000

10.000

-2.000

8.000

89.998

19

1

10

10 5

-2.000

10.000

-2.000

8.000

89.998

-2

10

-2

Optimal

values

90

115

Table 5 Higher loop step number

CPU time 0.11 s. Number of lower loop steps

Final a11 in each step

a12

Initial values of states

x0 = 0

xI

x2

4.000

8.000

UI

u0

E =0.01

I

5

1000

10

1.127

7.627

1.127

8.754

58.014

2

5

I000

10

-0.586

9.111

-0.586

8.525

83.003

3

5

1000

100

-1.690

10.262

-1.690

8.572 100.542

4

3

100

100

-1.937

10.217

-1.937

8.28O

95.614

5

3

100

100

-1.994

10.140

-1.994

8.146

92.937

6

3

100

1000

-2.011

10.171

-2.011

8.160

93.219

7

1

10

1000

-2.010

I0.092

-2.010

8.081

91.634

8

1

t0

104

-2.010

10.099

~2.010

8.089

91.796

9

1

10

104

-2.008

10.055

-2.008

8.047

90.939

10

1

10

105

-2.007

I0.059

-2.007

8.052

91.044

105 106 106 107 107 108

-2.005

I0.032

-2.005

8.027

90.539

-2.004 -2.003 -2.003 -2.002 -2.001

10.034

-2.004

8.030

90.603

10.018

-2.003

8.015

90.309

10.020

-2.003

8.017

90.347

10.011

-2.002

8.009

90.178

10.011

-2.001

8.010

90.200

-2.001 -2.001

10.006

-2.001

8.005

90.102

10.003

-2.001

8.002

90.050

-2.000

10.002

-2.000

8.001

90.023

11

1

10

12

1

10

13

1

10

14

1

10

15

1

10

16

1

10

17

1

10

18

1

10

108 108

19

1

10

108

2O

1

10

108

-2.000

10.001

-2.000

8.000

90.010

21

1

10

108

-2.000

10.000

-2.000

8.000

90.004

22

1

10

108

-2.000

10.000

-2.000

8.000

90.001

-2.000

23

1

10

108

10.000

-2.000

8.000

90.000

24

1

10

108

-2.000

10.000

-2.000

8.000

90.000

25

1

10

108

-2.000

10.000

-2.000

8.000

90.000

10

108

-2.000

10.000

-2.000

8.000

90.000

-2

10

-2

26

I

Optimal values

90

116 Table 6

CPU time 0.16 s.

Hiegrh- Number Final loop of a11 in a1 u0 uI step lower each 2 numloop ber steps step x0 = 0 Initial values of states 1 7 164. 10 7.647 1.117

xI

x2

R

4.000

8.000 8.764

= 0.001 58.338

1.117

2

7

104

10

-0.607

9.137

-0.607

8.531

83.494

3

7

104

100

-1.699

10.279

-1.699

8.580

100.779

4

5

1000

100

-1.948

10.237

-1.948

8.289

95.815

5

5

I000

100

-1.999

10.143

-1.999

8.144

92.907

6

3

100

100

-2.005

10.077

-2.005

8.071

91.434

7

1

10

100

-2.005

10.041

-2.005

8.036

90.726

8

I

10

1000

-2.005

10.045

-2.O05

8.040

90.800

9

3

100

1000

-2.003

10.023

-2.003

8.020

90.397

10.013

-2.002

8.011

90.213

10.014

-2.001

8.012

90.238

10

I

10

1000

-2.002

11

I

10

104

-2.002

12

I

10

104

-2.001

10.007

-2.001

8.006

90.122

13

I

10

105

-2.001

10.008

-2.001

8.007

90.137

-2.001

8.004

90.070

14

1

10

105

-2.001

10.004

15

1

10

106

-2.001

10.005

-2,001

8.004

90.079

16

1

10

106

-2.000

10.002

-2.000

8.002

90.040

17

1

10

107

-2.000

10.003

-2.000

8 .OO2

90.045

18

1

10

10.001

-2.000

8.001

90.023

1

10

-2.000

10.001

-2.000

8.001

90.026

20

1

10

-2.000

10.001

-2.000

8.001

90.013

21 22

1 1

10 10

10 ? 8 10 8 10 9 10 109

-2.000

19

-2.000 -2.000

10.001 10.000

-2.OOO -2.000

8.001

90.015

8.ooo

90.008 90.004

23

1

10

109

-2.000

10.000

-2.000

8.000

24

1

10

109

-2.000

10.000

-2.000

8.000

90.002

25

1

10

109

-2.000

10.000

-2.000

8.000

90.001

26

1

10

109

-2.000

10.000

-2.000

8.000

90.000

27

1

10

109

-2.000

10.000

-2.000

8.000

90.000

8.000

90.000

28

1

10

109

-2.000

10.000

-2.000

29

1

10

109

-2.000

10.000

-2.000

8.000

90.000

30

1

10

109

-2.000

10.000

-2.0OO

8.000

90.000

1

10

109

-2.000

10.000

-2.000

8.000

90.000

-2

10

-2

8

90

31

Optimal values

117

Example

3

Let us n o w c o n s i d e r see Happel dynamic

(1958)

programming

sed this p r o b l e m energy

the m u l t i s t a g e

pp.

needed

following

65-67.

by A r i s et al.

as the e x a m p l e

to c o m p r e s s

function

should

compression

This p r o b l e m

a gas

(1960).

of a gas problem,

was

also

We h a v e

1 ° in c h a p t e r

1.

isentropically

solved

using

also d i s c u s -

To m i n i m i z e

in 3 stages

the

the

be m a x i m i z e d

R=-

where

x0 = P0

mediate

pressures

The o p t i m a l second

is an i n i t i a l x3 = P3

intermediate

stages

calculus.

and

pressure, is the

pressures

of c o m p r e s s i o n s

xi ,

xI

and

x2

m a y be found u s i n g

= - ~, = L Xo

~+I| xI J

~R

._ _

x3

~x2

L x~

x2"~+i

-- 0

=

after

rearranging

leads

0

to e q u a t i o n s

x~ ~ = x ~0 x 2 x~ ~ = x~~ x 3

The s o l u t i o n

of this e q u a t i o n

is

z13 x~ = (x0~ x 3)

z/3

= (p~0 p3)

I/3 x~ = Ix 0 x23)

are inter-

final p r e s s u r e .

We h a v e

~x i

which

i = 1,2

~/3 = (p0 p~)

after

first and

the d i f f e r e n t i a l

118

The

second

partial

derivative

3~x 2 2R

are

= - ~

(~-I)Xi£i

+ (~+11

x0

~2 R

J

x2

J

o-i 2 x2

=

~X 1 8X 2

a n d at t h e p o i n t

x2 x 1

X ~+1 I

~xi,x2)

3 2 R*

~2~ 2 x I/3~-4/3-

=

~2 R*

u

=

I/3~-2/3 X3

2 x I/3 2/3 x /3 -4z3

3 2 R* = ~

2

-1/3~-1

1/3~-1

x0

x3

< 0

< 0

> 0

~x I ~x 2

NOW

as

x3 3X 2

then

the matrix

[x~,x~)

and the

The problem

may

3x 2

of

second

function

be

> 0

3x I 3X 2

derivatives takes

formulated

2 max ~. u 0 , u l , u 2 i=0

is n e g a t i v e d e f i n i t e

maximum

at t h i s

as a c o n t r o l

- [u£1 ~

point.

problem

at

119

subject

to

i = 0,1

Xi+ I = X i U i X0 = P0 X3 = P3 From e a r l i e r

calculations

the o p t i m a l

solution

to this p r o b l e m

is

seen to be t/3

i/3

113

t13

with the o p t i m a l

The

usual

Then the

values function

criterion

for

value

, see

~

R*(P3)

Happel

(1958)

, the g r e a t e r

p.

upper

260,

are

boundary,

form

uB( 31 >

x3

0<~<

has the

1 .

120

and is not concave. mum principle

This m e a n s that the c l a s s i c a l

(linear)

maxi-

cannot be applied.

In the same w a y as above we can find

~

(x~) -,---

2

rx~]o~s~ lX 0 ]

w h i c h is also not concave.

A l s o smaller u p p e r b o u n d a r i e s

ub l(x O,x I] =

-[hl ~ [Xo]

ub 2[x l,x 2] =

iX I]

ub 3(x 2,x 3) =

{x2]

are not concave.

To use the g e n e r a l i z e d m a x i m u m p r i n c i p l e w i t h q u a d r a t i c

Hamiltonian

we h a v e

Hi(xi,ui,ai+l,bi+l ) = _(ui)~ - ai+1(xl] 2 (ui) 2 - bi+ I xi ui i = 2,1,0

As the m a x i m u m of this H a m i l t o n i a n the n u m e r i c a l

maximization

was p e r f o r m e d .

s e c t i o n m e t h o d was chosen.

F r o m the e q u a t i o n 5H.

8~.

~x i

~x i

l

we get the r e c u r s i o n

1

for

c a n n o t be found a n a l y t i c a l l y

b. l

To do this the golden

121

bi

In the t a b l e s x3

= b i + I ui

7 and

= 8 , ~ = 0.29

given.

The

caused that of

8 the results and

2 values

slow convergence

by the very

the matrix x0

+ 2xi[ai+l(ui)2-ai]

= 1 ,

x3

= 8

3 2 R*

of t h e

partial

~ = 0.29

3 2 R*

for

and

x0

s = 0.01

c a n be o b s e r v e d .

second

and

of c o m p u t a t i o n e = 0.1

flat maximum

of t h e

i : 2,1

This

function

is p r o b a b l y

R. L e t

derivative

= 1 , are

us n o t i c e

for above

values

is

0.0514

0.0128

0.0128

0.0128

8X I 8X 2 =

32 R*

32 R* 3x 2 3x I

which

means

any c h a n g e

t h a t in the v i c i n i t y of the o p t i m a l s o l u t i o n of t h e v a l u e

IR- R*I/Ix I -x~l

IR- R*l/Ix 1 -x Referring be s e e n of

x3 .

shown

back

that

in t a b l e

are p e r f o r m e d . tion was

produces

and any change

to the

was

formulae

values

x2

verified

9 where

to

I

much

and

for t h e

c a n be m a d e

of o n l y

only

1.3%

(x~,x~) 5% of

of

second bigger

derivatives taking

and

better

of c o n t r o l

of

so w e r e

values

lower the

loop

value

the r e s u l t s

steps of

it c a n

smaller

on a c o m p u t e r

As t h e n u m b e r

equal

of

change

l.

their

This

xI

and

in e a c h

a ,

values

are states itera-

i = 1,2,3 ,

ll

then

they

are neither

and

11 w h i c h

and

N = 10

show the

shown

in t h e

results

respectively.

table

for h i g h e r

9 nor number

in t h e of

tables

stages

10

N = 4

1

I

I

I

I

I

I

1

1

I

1

1

1

I

1

4

5

6

7

8

9

I0

11 1

I

1

1

1

I

I

I

1

I

1

Initial

u J

Optimal values

I

I

I

I

1

3

1

.....

a12

all

~ ,

Final

Final

2

Number of lower loop steps

CPU time 0.14 s.

1

7

I

x0=1

Higher loop step number

Table

u0

I

I

I

I

I

I

I

I

I

I

i

1.444 1.452 1.459 1.465 1.472 1.480 1.487

1.698 1.698 1.696 1.697 1.696 1.695 1.694

3.260 3.244 3.234 3.219 3.202 3.190 3.174

2

3.202

1.437

1.698

3.275

2

3.219

1.430

1.698

3.290

-3.688 -3.687 -3.687 -3.687 -3.686 -3.686 -3.686 -3.685 -3.668

7.994 7.994 7.998 8.004 7.998 7.993 8.004 7.995 8

5.508 5.485 5.461 5.431 5.406 5.377 4

3.244 3.234

3.190 3.174

-3.688

-3.688

5.534

7.987

5.585 5.561

7.975

5.612

-3.687

s = 0.1

R

3.260

7.941

8.000

x3

5.637

5.667

x2

3.275

3.290

3.303

1.421

1.699

3.303

3.317

3.333

xI

1.409

u2

1.700

uI

3.317

values of states

a13

I

I

I

I

1

1

1

1

1

1

3

4

5

6

7

8

9

10

11

12

Optimal

3

2

Number of lower loop steps

, ~,

values

10

10

10

10

10

10

10

10

10

10

10

1

a 12

a 11

10

10

10

10

10

10

10

10

10

10

10

1

Initial

Final

s.

Final

CPU time 0.15

1

8

I

x0 = I

Higher loop step number

Table

I

I

I

I

I

I .451 I .453

I .705 I .706 2

2

I .449

3.228

I .704

3.233

3.239

I .447

I .444

I .702

3.259

I

I .445

I .441

1.701

3.261

I

I .703

I .438

I .70O

3.270

I

I .703

I .437

I .699

3.278

1

3.247

I .436

I .698

3.281

I

3.251

1.435

1.698

I

3.285

u2

1.409

uI

1.700

of states

u0

3.316

I

values

a13

5.513 5.507

3.228

5.520 3.233

3.239

5.530

-3.687 -3.687 -3.687

7.997 7.999 8.0O3

-3.668

-3.687

-3.687 8.003

7.998

3.259

5.535

-3.688 8 .OO6 5.545

3.261

3.247

-3.687 7.996

3.251

-3.687 7.993 5.558

-3.688 8.003 5.568

3.278

5.548

-3.688 8.001 5.571

3.281

3.270

-3.688

-3.687

e = 0.01

R

8.005

7.940

8.000

x3

5.577

5.637

5.667

x2

3.285

3.316

3.333

xI

9

u2

1.192 1.191 1.191

1.238 1.236 1.234 1.233 1.230

3

4

5

6

7

1.224 1.200

10

11

Optimal values 1.200

1.188

1.188

1.226

9

1.189 1.188

1.229 1.228

8

1.190

1.191

1.193

1.240

2

1.200

1.189

1.187

1.186

1.184

1.180

1.178

1.174

1.168

1.158

1.139

1.103

of states

uI

s.

1.194

values

0.11

1.242

Initial

uo

CPU time

1

x0 = I

Higher loop step number

Table

1.200

1.224

1.226

1.228

1.229

1.230

1.233

1.234

1.236

1.238

1.240

1.242

1.243

xI

1.441

1.455

1.457

1.460

1 .462

1 .464

I .468

I .470

1.472

1.476

1.479

1.482

1.487

x2

1.730

1.731

1.729

1.730

1.730

1.728

1.728

1.726

1.719

1.709

1.684

1.635

1.730

x3

-3.163

-3.163

-3.163

-3.163

-3.163

-3.163

-3.163

-3.163

-3.161

-3.160

-3.155

-3.146

E = 0.1

10

u1 U2

1.201

.201

Optimal values

1.201

1.202

.250

.248

1.202

11

.253

9

1.203

1.204

1.205

1.207

1.208

1.209

1.210

10

.256

.254

8

.259

6

7

.262

.261

4

.264

3

5

.266

2

1.212

1.201

1.174

1.174

1.174

1.174

1.174

1.174

1.174

1.174

1.174

1.174

1.174

U3

1.201

1.182

1.179

1.177

1.174

1.170

1.167

1.163

1.157

1.150

1.135

1.109

Initial values of states

I .268

u0

CPU time 0.14 s.

I

x0 = I

Higher loop step number

Table

1.201

I .248

1.250

1.253

I .254

I .256

I .259

1.261

1.262

1.264

I .266

1.268

1.270

x 1

1.442

1.499

1.503

1.506

1.509

1.513

1.517

1.521

1.525

1.528

1.532

1.537

1.540

x 2

1.732

1.760

1.764

1.768

1.772

1.776

1.781

1.785

1.790

1.794

1.798

1.803

1.810

x3

2.080

2.080

2.080

2.080

2.080

2.078

2.078

2.076

2.072

2.062

2.040

1.999

2.080

x4

-4.218

-4.218

-4.218

-4.218

-4.218

-4.218

-4.218

-4.218

-4.217

-4.216

-4.213

-4.206

~ = 0.1

R

01

11

1.254 1.254 1.254 1.254 1.254 1.254 1.254 1.254 1.254

1.339

1.339

1.337

1.335

1.333

1.331

1.329

1.328

1.326

1.508

1.504

1.500

1.497

1.493

1.490

1.487

1.483

1.480

1.200

3

4

5

6

7

8

9

10

11

Optimal values

1.200

1.254

1.340

1.513

2

1.200

1.254

1.341

u2

1.517

s.

1

uI

0.33

u0

C P U time

loop s t e p number

Higher

Table

1.144 1.143 1.142 1.141 1.141 1.141

1.167 1.167 1.166 1.166 1.166 1.166

1.202 1.201 1.201 1.201 1.201 1.200

1.200

1.200

1.200

1.140

1.144

1.167

1.202

1.166

1.144

1.167

1.202

1.200

1.145

1.167

1.203

u5 1.145

u4 1.168

1.203

u3

1.128 1.137 1.142 1.147

1.102 1.101 1.101 1.101 1.102 1,102

1.112 1.112 1.112 1.112 1,112 1.112

1.123 1.124 1.124 1.124 1.124 1.124

1.200

1.121

1.101

1.112

1.124

1.200

1.117

1.100

1.112

1.125

1.200

1.105 1.100

1.112

1.125

1.200

1.133

1.111

1.097 1.100

1.112

1.125

1.088

u9

1.100

u8

1.112

u7

1.126

u6

C~

1.497

1.493

1.490

1.487

1.483

1.480

1.200

6

8

9

10

11

Optimal values

1.500

5

7

1.508

1.504

3

1.513

2

4

1.517

1.520

x0 = 0 Initial values

1

xI

Higher loop step number

1.440

1.962

1.969

1.977

1.984

1.991

1.998

2.006

2.013

2.020

2.027

2.034

2.040

x2

1.729

2.461

2.470

2.479

2.488

2.496

2.505

2.515

2.524

2.533

2.542

2.551

2.560

x3

2.075

2.954

2.964

2.977

2.989

2.999

3.010

3.023

3.034

3.046

3.058

3.069

3.080

x4

2.490

3.444

3.456

3.471

3.485

3.497

3.5%1

3.526

3.540

3.554

3.568

3.584

3.600

x5

2.988

3.926

3.942

3.961

3.978

3.994

4.013

4.033

4.048

4.068

4.086

4.104

4.120

x6

3.587

4.412

4.431

4.452

4.472

4.489

4.508

4.533

4.554

4.576

4.598

4.620

4.640

x7

4.304

4.906

4.927

4.952

4.975

4.994

5.015

5.042

5.066

5.090

5.115

5.139

5.160

x8

5.166

5.406

5.427

5.452

5.478

5.498

5.525

5.551

5.574

5.600

5.627

5.655

5.680

x9

6.200

6.202

6.198

6.198

6.204

6.201

6.195

6.199

6.193

6,186

6.173

6.151

6.200

xl0

-10.543

-10.547

-10.547

-10.547

-10.547

-10.547

-10.547

-10.547

-10.547

-10.547

-10.546

-10.545

~ = 0.1

R

~4

128

4.5

Complementary remarks

The algorithm given in p. 4.3 is only a first proposition for solving the multistage optimization problems using the generalized maximum principle.

The three simple examples presented in

p. 4.4 showed reasonable results of applying it.

There is, how-

ever, still too little experience to draw any conclusions.

Some

new solutions in the algorithm formulation may prove useful in future applications.

Therefore computing more numerical examples

and real cases would be welcomed. Few constants control the way of computation in the algorithm. These are: -

the constant 4 in step 8 ° which decides when the increase of values

-

a~ j 1

the constant 2 in step 11 ° which decides whether the increase of values

-

may happen,

a~ j 1

in lower loop should be performed or not,

the constant 2 in step 17 ° which decides whether the increase of values

a~ j 1

in higher loop should be performed or not.

No bigger investigations of the influence of those paremeters have been made.

From our experience it seems only that forcing

too quick convergence i.e. high values of parameters make the results worse.

Examination of the influence of those parameters

as well as some others as for example multiplication by 10 the values of

a~x j

value

in step 50 could be the subject of further researches.

81

in steps 120 and 19 ° or the way of setting the

As it is seen from the examples the algorithm stops sometimes too early.

3-5 more iterations give much better results.

An

improvement of the stopping condition might be the other subject of research. values.

One idea is to examine the variations in criterion

129

The algorithm may be applied to broader class of problems.

One

important area are the problems with bounds on controls and/or states.

These kind of problems may be solved using the above

algorithm together with the appropriate penalty functions. other class of problems are free-end-point problems. them

a

simple

suming that

modification could be tried.

KN(XN) = 0

i.e.

AN = 0

and

The

To handle

It consists of asbN = 0 .

Then only

the lower loop should be executed.

4.6

Literature

The idea of iterating support parameter

b

N

comes from the pa-

pers dealing with static problems M.R. Hestenes:

Multiplier and Gradient Methods.

4, No. 5, 303-320, M.J.D.

Powell:

JOTA, Vol.

1969.

A Method for Nonlinear Constraints in Mini-

mization Problems. Academic Press,

In:

R. Fletcher

(ed.):

Optimization.

1969, pp. 283-298.

The computational experience of applying the Hestenes/Powell method is discussed in A. Miele, P.E. Moseley~ A.V. Levy, G.M. Coggins:

On the

Method of Multipliers for Mathematical Programming Problems. JOTA, Vol.

10, No. 1, 1-33, 1972.

R. Fletcher: timization.

An Ideal Penalty Function for Constrained OpJ. Inst. Maths. Applics., Vol.

15, 319-342,

1975. The convergence of Hestenes/Powell and associated methods is discussed e.g.

in

130 R.D. Rupp:

On the Combination of the Multiplier Method of

Hestenes and Powell with Newton's Method.

JOTA, Vol. 15,

No. 2, 167-187, 1975. R.T. Rockafellar:

The Multiplier Method of Hestenes and

Powell Applied to Convex Programming.

JOTA, Vol. 12,

555-562, 1973. R.T. Rockafellar:

Solving a Nonlinear-Programming Problem

by Way of a Dual Problem.

Symposia Matematica, Vol. XXVII,

1976. R.T. Rockafellar:

Penalty Method and Augmented Lagran-

glans in Nonlinear Programming. Optimization Techniques. D.P. Bertsekas:

Proc. 5th IFIP Conf. on

Rome 1973.

Springer-Verlag, 1974.

Combined Primal-Dual and Penalty Methods

for Constrained Minimization.

SIAM J. Control, Vol. 13,

No. 3, 521-544, 1975. D.P. Bertsekas:

On Penalty and Multiplier Methods for Con-

strained Minimization.

SIAM J. Control and Optim., Vol.

14, No. 2, 216-235, 1976. D.P. Bertsekas:

Multiplier Methods:

A Survey.

Automati-

ca, Vol. 12, 133-145, 1976. D.P. Bertsekas:

On the Convergence Properties of Second-

Order Multiplier Methods.

JOTA, Vol. 25, No. 2, 443-449,

1978. D.P. Bertsekas:

Penalty and Multiplier Methods.

L.C.W. Dixon, E. Spedicato, G.P. Szeg6 (eds.): Optimization, Theory and Algorithms.

In:

Nonlinear

Birkh~user, Boston

1980, pp. 253-278. D.P. Bertsekas:

Constrained Minimization and Lagrange

Multiplier Methods.

Academic Press (in preparation).

Algorithms for lower level loop (for linear supports) were discussed in

131

S. Katz:

Best Operating

Points for Staged Systems.

Eng. Chem. Fundamentals,

Vol.

L.T. Fan, Ch. S. Wang: Wiley,

I, No.

R. Aris:

I, II, III.

The Discrete Maximum Principle.

Green's Functions

4, No.

I,

in the above paper

and also in Convergence

of a Method of Successive Approxi-

in the Theory of Optimal Processes.

Chem. Fundamentals,

Vol.

The solution of the multistage discussed

Vol.

1965.

of these algorithms was discussed

M.M. Denn: mations

and Optimal Systems.

Ind. & Eng. Chem. Fundamentals,

7-16, No. 2, 213-222, No. 3, 248-257,

by Denn & A n s

Ind. &

1962.

1964.

M.M. Denn,

Convergence

4, 226-240,

4, No. 2, 231-232,

Ind. & Eng. 1965.

compression of a gas problem was

in

J. Happel:

Chemical Process Economics.

R. Aris, R. Bellman,

R. Kalaba:

in Chemical

Engineering.

No. 31, Vol.

56, p. 95

(1958).

Some Optimization

Chem. Engg. (1960).

Wiley

Progr.

Symp.,

Problems Series

CHAPTER

CONCLUSIONS

5

AND FURTHER

RESEARCH

135

In this book we have presented a new approach to optimization of the multistage optimization problems called the upper boundary approach. The classical methods of solving this problem like the m a x i m u m principle method and the dynamic programming method are shown to fit smoothly into the new approach.

Moreover, using this ap-

proach a number of new results have been developed,

among these

a new, generalized version of the discrete m a x i m u m principle is arrived at.

The new version does not require the assumption of

directional convexity,

an assumption which has forbidden the ap-

plication of the classical discrete maximum principle to some practical multistage optimization problems.

An example was

given of a simple technical problem of the multistage compression of a gas; it was shown that it did not have the directional convexity property.

This and two other simple problems were

solved numerically using an algorithm for finding a solution to the generalized maximum principle conditions. This book contains the thorough discussion of the present status of the works on the upper boundary approach.

However, it does

not pretend to complete the research in this area.

On the con-

trary, it is felt that many further works should be done to make this approach a flexible tool for solving a variety of practical problems including not only technical or operational systems but also others like economical, tems.

biological,

ecological,

etc. sys-

It seems to be important that to proceed in that direc-

tion some real life problems should be solved with the theory given in this

book.

This will enable to see the limitations of

the theory and the needs for extending it.

This will also con-

tribute to the verification of the algorithm given in chapter 4 and its possible improvement.

It will moreover show how some

practical difficulties can be overcome when using this approach.

136

Apart from these more practically-oriented further theoretical works can proceed. Many multistage

tions on the boundaries

optimization

some

One of them is an exten-

sion of the generalized maximum principle tions.

investigations

for boundary condi-

problems have optimal

solu-

and the only way to deal with them in

the present theory is to use the penalty

functions

techniques.

The straight application of the generalized maximum principle for active boundaries sumptions

for part

is not possible only because of the as-

(iii) of Theorem 222.

It seems that this as-

sumption might be released when using some kind of complementary slackness conditions. During discussions

on the results obtained an idea has appeared

that the generalized maximum principle could be generalized

even

more by defining a new function of the form

ci (xi ,ui ,~i ÷ I) = r i (xi,u i)

Defining now a saddle-point

to

Ci

as a point

0 0 (x~,ui,~i+l)

satisfying 0 0 0 0 ToO Cifxi'ui'~i+1 ) --< Ci(xi'ui' i+I 1 --< Ci(x0'u0i'~i+17

we could get all the results of Chapter 3 §§ 217-219 and 222-223 from two basic theorems:

I.

(x l,ul,K i*+l)

is a saddle-point

2.

Under assumptions

to

Ci(xi,ui,~i+l)

of differentlability

~c i (xl,ul,~l ÷ i) (i)

(ii)

~X i

i

i

• ~U. I

= 0

i+i

=

0

137

i

(ill)

I)

To do it a suitable definition to the function

~i+I

at the fundamentals functional

=

0

of differentiability

is needed.

of the theory.

This requires

with respect

thorough look

Then an application

of

analysis methods would be probably necessary.

A good deal of work may be done on the theoretical tions of properties convergence

of the numerical algorithm such like e.g.

and its rate.

the multipliers

investiga-

The argumentation

of convergence

method given in the literature

of Chapter 4 could be a good starting point. results with the conditions

cited at the end Combining

these

for convergence of the lower loop

migth give solution to the algorithm convergence

problem.

The upper boundary approach has very intuitionally geometrical

interpretation.

elaboration

of the popular presentation

could be further developed

of

appealing

This property could be used for the of this theory which

for lecturing purposes.

The upper boundary approach can be extended to the continuoustime systems. Ravn

(1980).

The way of doing it has already been presented by This proposition needs mainly a mathematical

treatment. In short, the book documents of the approach,

in specific areas the applicability

and above a number of possible extensions

great interest are stretched.

Thus we think that we have here

shown that the Upper Boundary Approach tackling discrete time systems.

of

is a fruitful tool for