Quantitative and Qualitative Games, Volume 58 (Mathematics in Science and Engineering)

Quantitative and Qualitative Games

This is Volume 58 in MATHEMATICS IN SCIENCE AND ENGINEERING A Series of monographs and textbooks Edited by RICHARD BELLMAN, University of Southern California A complete list of the books in this series appears at the end of this volume.

QUANTITATIVE A N D QUALITATIVE G A M E S AUSTIN BLAQUIERE FACULTY OF SCIENCES, PARIS UNIVERSITY OF PARIS, FRANCE

FRANCOISE GERARD FACULTY OF SCIENCES, PARIS UNIVERSITY OF PARIS, FRANCE

GEORGE LEITMANN DIVISION OF APPLIED MECHANICS UNIVERSITY OF CALIFORNIA BERKELEY, CALIFORNIA

@

ACADEMIC PRESS New York and London

1969

COPYRIGHT @ 1969, BY ACADEMIC PRESS, INC. ALL RIGHTS RESERVED NO PART OF THIS BOOK MAY BE REPRODUCED IN ANY FORM, BY PHOTOSTAT, MICROFILM, RETRIEVAL SYSTEM, OR ANY OTHER MEANS, WITHOUT WRITTEN PERMISSION FROM THE PUBLISHERS.

ACADEMIC PRESS INC.

111 Fifth Avenue, New Y&k, New York 10003

United Kingdom Edition published by ACADEMIC PRESS, INC. (LONDON) LTD.

Berkeley Square House, London W l X 6BA

LIBRARY OF CONGRESS CATALOG CARD NUMBER: 74-84246 AMS 1968 SUBJECTCLASSIFICATION 9075

PRINTED IN THE UNITED STATES OF AMERICA

Preface

This book, based on lectures given by A. Blaquikre at the Institute Henri Poincark, is a record of joint research in the field of two-person games. Having arrived at a reasonably comprehensive theory for a wide class of such games, we present it here in the hope that it will serve as the basis for further research and for application in the physical, biological, and social sciences. Thus, this volume is addressed to those interested in game theory for its own sake, as well as to those concerned with conflict situations in technology, economics, psychology, indeed, in any dynamical system. With a view toward increasing the scope of the theory, the treatment proceeds from the general to the particular. Results are obtained for systems whose dynamics are defined by a rather large class of mappings. Further results are derived then by restricting the dynamics to those described by ordinary differential equations (differential games) and by difference equations (multistage games). In Chapter 1, the notions of game and play are introduced, and the problem to be discussed is stated. Certain basic concepts, such as rules, objectives, and strategies, are defined. Chapters 2-4 are devoted to quantitative two-person, zero-sum games with perfect information of the state called games of degree by Rufus Isaacs. These chapters are essentially extensions of the geometric theory of optimal processes developed earlier by A. BlaquiBre and G. Leitmann.7 Differential as well as multi-stage games are treated. Chapters 5-7 deal with qualitative two-player games, called games of t

See Refs. 1-6.

vi

PREFACE

kind by Isaacs. In this type of game the players do not strive to minimize and maximize, respectively, a numerical payoff; rather they have conflicting aims in that they endeavor to terminate play on different target sets. In Chapter 5 additional geometric notions are introduced, such as the map of the game, and optimality for a player at a given state is defined. While the definitions and arguments are different from those employed in earlier chapters, the geometric results exhibit a strong similarity to the ones found in quantitative games. This similarity serves as a guide for the development of the theory. In Chapter 6, the theory is applied to differential games. The arguments used follow closely those employed in Chapter 3. While the framework of the theory of quantitative and qualitative games appears to be essentially the same, the similarities appear to be more formal than real. In Chapter 7, an attempt is made to establish a connection between these two types of games. We believe that geometric notions are well suited for a discussion of dynamical system theory, such as that of the games treated in this book. Geometric notions are intuitively appealing and, at the same time, capable of providing a modicum of rigor. It is our hope that the reader will come to share this point of view and that he will not be discouraged by the many assumptions introduced in the development of the theory. The reader whose interest lies primarily in the application of the theory may elect to forego some of the details and accept many of the arguments on an intuitive basis. It has been our aim to state all assumptions explicitly rather than to leave it to the reader to discover them (or not). The complications which have necessitated many of these assumptions appear to be inherent in the theory. In any event, this book is but a brief introduction to a geometric theory of games, and many questions remain unanswered. We should like to take this opportunity to thank the Office of Naval Research and the Delegation Generale A la Recherche Scientifique et Technique for supporting the research that has resulted in this book. A. BLAQUIBRE F. GBRARD G. LEITMANN

Paris and Berkeley September 1969

Contents

PREFACE

V

NOTATION AND TERMINOLOGY

I. Games and Plays 1.1 1.2 1.3 1.4 1.5 1.6 1.7

11.

Problem Statement and Assumptions Rules of the Game Trajectories and Paths Play, Terminating Play Playable Strategy Pairs Joining of Trajectories and Describing Curves Objectives of the Game, Qualitative and Quantitative Games

9

Some Geometric Aspects of Quantitative Games 2.1 2.2 2.3 2.4 2.5 2.6 2.7

Transfer of State and Performance Index Optimality Game and Isovaiue Surfaces Paths in Augmented State Space Joining of Paths Some Properties of Game Surfaces Uniqueness of the Value of the Game

11 12 13 15 17 17 20

111. Differential Quantitative Games 3.1 3.2 3.3 3.4

The State Equations and Strategies Transfer Time Rules of the Game Target e vii

21 22 23 23

Viii

CONTENTS

3.5 Integral Performance Index 3.6 A Variational Equation and Its Adjoint 3.7 Joining of Paths 3.8 Two Families of Paths 3.9 Regular Interior Points of a Game Surface 3.10 Transformation of a Tangent Plane 3.1 1 Regular Optimal Paths 3.12 Transversality Condition 3.13 A Min-Max Principle, Regular Case 3.14 Discontinuity Manifolds 3.15 Regular Portion of an Optimal Path 3.16 Jump Condition 3.;7 Constraints 3.18 A Min-Max Principle with Jump Condition 3.19 Example 3.1 3.20 Example 3.2 3.21 Example 3.3 3.22 Sufficiency Conditions

24 25 29 30 32 36 37 39 41 42 43 46 49 52 53 58 61 67

IV. Multistage Quantitative Games 4.1 4.2 4.3 4.4

4.5 4.6 4.7 4.8 4.9 4.10

4.1 1 4.12 4.13 4.14 4.15 4.16 4.17 4.18 4.19 4.20

V.

Problem Statement State Equations, Strategies, and Target e Describing Curves and Paths in State Space Cost of Transfer Augmented State Space and Paths in E"" Some Properties of a Game Surface Sets fir and fiE 10-Directional Convexity Regular Points of a Surface &(C) Regular Optimal Paths Variational Difference Equation Linear Transformation and Inverse Transformation Some Properties of Linear Transformations A k and Transformation of a Tangent Plane Adjoint Equations Gradient and Adjoint Vector Transversality Condition A Min-Max Principle Constraints Example 4.1

'B

71 72 74 74 75 77 78 80 81 83 84 86 87 88 90 92 93 93 94 96

Some Geometric Aspects of Qualitative Games 5.1 Map of a Game 5.2 Optimal Strategies in a Qualitative Game, Sets of the Game

103 104

ix

CONTENTS

5.3 Some Properties of Sets of the Game 5.4 Surface of the Game 5.5 A Similarity between Qualitative and Quantitative Games 5.6 Problem Statement

VI.

106 107 110 111

Differential Qualitative Games 6.1 Rules and Objectives of the Game 6.2 A Basic Property 6.3 A Variational Equation and Its Adjoint 6.4 A Property of Boundaries of Sets of the Game 6.5 A Property of the Surface of the Game 6.6 Two Families of Paths 6.7 A Local Min-Max Condition 6.8 Gradient and Adjoint Vector 6.9 Transversality Condition 6.10 A Min-Max Principle 6.11 Autonomous Games with Time-Independent Targets 6.12 Constraints 6.13 Semipermeable Surfaces 6.14 A Property of the Gradient along a Path in a Semipermeable Surface 6.15 Transversality Condition for a Semipermeable Surface 6.16 A Min-Max Principle along a Path in a Semipermeable Surface 6.17 Example 6.1 6.18 Example 6.2

113 114 114 116 118 119 123 125 126 131 132 134 135 139 140 142 143 145

VII. A Connection between Qualitative and Quantitative Games 7.1 7.2 7.3 7.4

Problem Statement, Definitions of Games 1, 2, and 3 Local Saddle-Point Condition, Game 4 Surfaces of the Game A Connection between Paths in Games 1 and 4

150 153 157 159

Appendix

163

References

165

SUBJECTINDEX

169

ASSUMPTIONS, COROLLARIES, LEMMAS,THEOREMS INDEX

172

This page intentionally left blank

Notat ion and Terminology

Unless otherwise indicated mathematical symbols will be those in common use. Additional terminology will be introduced as needed. Vectors will be denoted by lower case letters. Subscripts will be used to denote the components of a vector; superscripts will be used to distinguish vectors. x = ( x , , x 2 , . . . x,) is an element of En.No special notation is used to distinguish between x E En and its column form; that is, unless otherwise specified, all vectors will be column vectors. The row form of x is xT = ( x , x 2 . . . x , ) . The Euclidean length of a vector x will be denoted by IIx11. The gradient of a function @: x @ ( x ) at a point xj will be denoted by grad @(xj). The transpose of a matrix M will be denoted by MT.The topological closure of a setoR in a topological space and the interior of R by R. will be denoted by The term domain will mean an open connected set of the appropriate space. A scalar-valued function defined on a domain will be said to be of class Ckif the function and its first k partial derivatives are continuous on the domain. A vector-valued function will be said to be of class Ce if each of its components is of class Ck. The notation V v 3 u will stand for: -+

a,

“for all v there exists u (which depends on v)” and 3 v V u will stand for: “there exists u (independent of v ) for all v” We shall write Definition whenever we define a new concept, and not for the specification of concepts introduced previously. We shall write Assumption whenever we introduce a new assumption of basic importance for the development of the theory. xi

This page intentionally left blank

Quantitative and Qualitative Games

This page intentionally left blank

I

Games and Plays

1.1 PROBLEM STATEMENT AND ASSUMPTIONS

We shall be interested in the behavior of a set of “persons,” called the players, each of whom strives to modify the state of a system in a most efficacious manner according to his own criterion. If there is only one player in the set, the theory reduces to the one of optimal control. In the following discussion we shall consider only the case of two players J , and JE whose interests are conflicting or at least different. We shall assume that the state of the system or, as we shall say, the state of the game, at any instant of time can be characterized by the values of a finite number of real variables xl, x 2 , . . . , x,, and that these values are known to both players. Hence we may think of the state as a point in an n-dimensional Euclidean space En, termed the state space of the game. In state space we select a rectangular coordinate system so that we may define a point by the state vector x = ( x l , x2, . . . , x,). We shall assume that the evolution in time of the state of the game can be influenced at all times by the two players. For example, as we shall see later, the behavior of the state may be described by a set of differential equations dxv/dt =f v ( x 1 ,

. .

x2,

*

7

x n , ~ 1 u2, ,

. . *

9

ur, ~ 1 ~,

. ..

2 ,

9

vs),

v = 1 , 2 , . . . , n (1.1)

where ul,u2, . . . , u,. and vl, v 2 , . . . ,v, are control variables which define vectors u = (ul, u 2 , . . . ,ur) and u = (vl, v2,. . . , v,) in r-dimensional and s-dimensional Euclidean spaces E’ and E’, respectively. Players J , and J E make their decisions through choosing the values of

2

I

GAMES A N D PLAYS

control variables u and v , respectively, at each instant of time. Later we shall discuss the case in which these choices are governed by functions of x,p , and e , which J,, and J,: select from two prescribed sets of functions. 1.2 RULES OF THE GAME

We shall assume that x belongs to the fixed subset G of En. Let there be given two prescribed sets R, and R E whose members will be denoted by r,. and r*;, respectively. Let T be the collection of all subsets of [O, a)containing 0. Let F be the set of all functions x: T, 4 G for all T, E T. Hereafter we shall w e simply X: T + X = X(T), T € T, to indicate a function x E F. Let .F be the collection of all nonempty subsets of F, and let y:

G X R,x

RE+g

be a mapping that associates with each point xi E G and with each pair (rl,, r,.;) E R , x R , an element y(xi,rIJrr E ) of .F, such that

x(0)

= xi

vx

E

y ( x i , r p , re)

Finally, let there be given two sets of points in G, namely 81. and 8,. We shall call these sets the targets of J , , and J E , respectively. Definition 1.1. A p i n e is defined by its rules and its objectives. The rules of the game consist of (i) the set G (ii) the sets R l , and R , (iii) the mapping y. The objectives will be defined later. Definition 1.2. Members rIJof RI,and rR of RE are strategies of players J , , and J,, respectively. Members (r,, r E ) of R , x R , are strategy pairs.

EXAMPLE 1 . 1 . Consider the following rules of a game: Let G be a domain of ETL.Let there be given the differential equation

dx/dr = f ( . ~ LI, , V) where u = (ul, u2, . . . , ur), u = (vl, u 2 , . . . , vs),.f=

(1.2)

(fi,fi,. . . ,fn>.

1.3

3

TRAJECTORIES AND PATHS

Let the choices of control variables u and u be governed by functions e: x - e ( x ) , x E G, respectively; that is, r , = p and rE = e being chosen from prescribed sets of functions R, and R E , respectively, let Eq. (1.2) be replaced by

p : x-p(x),

dXldT

=f ( X ,

p ( x ) , 4x1)

(1.3)

Equation (1.3) is said to have a solution if there exists a function x : T + X ( T ) defined and continuous on an interual A T , such that X ( T ) E G V T E A T , and dX(T)ldT = . f ( X ( T > > p(x(.)), e(x(.>>> is satisfied almost everywhere in AT. Now, for each initial point xi E G there may exist a solution of Eq. (1.3), x : T - x ( T ) , int [0, co) such that x(0) = xi. Let [0, T ~ be ) its interval of definition. We shall call x a maximal solution if X ( T ) is not defined for T = T ~ c‘r , if’imr-r3,r
S(0) = xs,

xqA lim x ( T ) . 7-rr

r
Then T , ~will be called the extremefuture value of T for the maximal solution and denoted by T+. Next let us define y ( x i , r p , r E ) for all x i E G, all r l , E R , and all rE ER,, as follows: (i) If Eq. (1.3) has no solution x : T + X ( T ) in [0, co) such that x(0) = xi,then we let y ( x i , r,, Y E ) be the point xi itself, that is the function x whose set of definition is T = 0 such that x ( 0 ) = x i ; (ii) otherwise we let ?(xi, r,, r E ) be the set of all maximal solutions x: T X ( T ) of Eq. (1.3) in [0, co), such that x ( 0 ) = x i . Hence y is defined. The rules of the game consist of G, the sets of functions p and e , and mapping y. ---f

1.3 TRAJECTORIES AND PATHS

Definition 1.3. Given the rules, for given strategies r,, and rE chosen by players J , and J , from sets R, and R E , respectively, and for a given state T h a t is, on some interval A T E [0, co).

4

I

GAMES AND PLAYS

xi E G called the initial state, a curve r x ( x i , r,, r E ) in G defined by a function x E y ( x i , r p , r E ) will be called a describing curve. Of course, there may be more than one describing curve for given initial state xi and given strategies r p and Y E . We shall also admit the possibility that rX(xi,rI>,r,) is a single point.

X'

FIG. 1.1. Path

r i fin

state space, terminating play.

Definition 1.4. Given any describing curve rx(.xz, r,, rE) which emanates from .y2and is defined by function x on T,, and given r ) E T,, the image of [0, T,] under the graph ((7, x): x = x ( T ) } will be called a trajectory and denoted by Y7j. Its initialpoint is xz = x(O), its endpoint is x j = x(rj) and its supporting curve is rx(xi,r,, r E ) . We shall say that Fzi is generated by (r,, r,). possibility that r j = 0.

Definition 1.5.

A trajectory

Fij

We shall also admit the

defined by function x on [0, r j ]n T,,

ri # 0, will be called apath and denoted by rii,if

x ( r ) t#

u OB;

V r E [0, r j ) n T,

(1.4)

I t follows from that definition that no point of a path, with the possible exception of its end point, belongs to 8, or 8,. If ri = 0, rijis said to be a null path, and will be denoted by TO.

1.5

5

PLAYABLE STRATEGY PAIRS

1.4 PLAY, TERMINATING PLAY

Definition 1.6. A play is the evolution of state x , described by x: T + x = x(T), T E [o, Tj] along a path rrii. A terminating play is a play whose end state x f = x ( T ~ belongs ) to 8, u OE. It terminates at T = T ~ The . corresponding path will be called a terminating path.

We shall say that a strategy pair (r,, r E ) generates a terminating play from initial state xi if there exists a function x E y ( x i , r p , Y E ) that defines a terminating path. 1.5 PLAYABLE STRATEGY PAIRS

In the following discussion, we shall consider two cases, namely. Case (a): neither 6, nor 8, is empty; and Case ( 6 ) : 8, or 8 E is empty, but not both. For instance, we shall suppose that 8, # 0 and 8, = 0. Definition 1.7. In Case ( a ) we shall say that (r,, rE) is a weaklyplayablef strategy pair f o r player J , at point x i , if it generates a terminating play from initial state xi, and if the corresponding end state x f belongs to 8,; and we shall say that (r,, rE) is a weaklyplayablef strategypair for player J E at point x i , if it generates a terminating play from initial state x i , and if the corresponding end state xf belongs to BE. In Case (b) the definition of a weakly playable7 strategy pair for Jp at point xi will be the same as in Case (a); and we shall say that (r,, r E ) is a weakly playable? strategy pair for J , at point xi, if it is not playable for J , at that point.

We shall let F p ( x i ) and Y E ( x i )denote the sets of all weakly playable strategy pairs for J , and J E , respectively, at point x i . It follows from Definition 1.7 that in Case (a) (r,, such that

YE) EF p ( x i ) 3

3x E y ( x i , rI,, r?;)

r x ( x i ,r p , Y E ) n 0, #

( r p , r E ) E Y E ( x Z ) 3~

t Or, more simply, “playable.”

o

E y(xi,r,,,

rE)

6

I

GAMES AND PLAYS

X'

(c)

(d )

FIG. 1.2. Paths generated by a playable strategy pair for (a) J1, at xz; (b) J , at (d) JE at xz in a single target game.

x z ; ( c ) both J , , and J E a t xz;

1.5

7

PLAYABLE STRATEGY P A I R S

Definition 1.8. Zn Case (a) we shall say that a strategy pair (r,, r,) is strongly playable for J , at point x i , if (i) it is not playable for J , at x i ; and (ii) r x ( x i , r p , r E ) n 0, # o V x E y(xi, r p , Y E ) Likewise a strategy pair (r,, rE) will be said to be strongly playable for JE at point x i , if (i)’ it is not playable for J,, at xi; and (ii)’ r x ( x i , r p , r,) n 8, f: o V x E y ( x t , rl+ r d Xi

9P

FIG. 1.3. Paths generated by a strongly playable strategy pair for J , at point x ~ .

In Case ( b ) a strategy pair (r,, r E )will be said to be strongly playable for J , at point x i , if

rX(xi, r p , r E ) n 8,

# @

Vx E y ( x i , r p , Y E )

There is no difference between the concepts of weak playability and strong playability for J E . Of course, in Case (b), playability for JE at point xi means that no describing curve rx(xi,rz,, r E ) intersects 8, and so this concept is sufficiently strong by itself. Note that, in every case, strong playability implies weak playability. We shall let YP*(xi) and Y E * ( x i ) denote the sets of all strongly playable pairs for J , and J E , respectively, at point xi.

8

1

GAMES AND PLAYS

1.6 JOINING OF TRAJECTORIES AND DESCRIBING CURVES

Definition 1.9. Let FL3be a trajectory emanating from xiand ending at xi; and let rX(xj,F,, F E ) be a describing curve emanating from xi. We shall say that ri7and rx(xj,E,, i E )can be joined if

rii u rx(xi,i,,

iE)

is a describing curve emanating from xi, that is if there exist a strategy

X’

FIG. 1.4. Joining of trajectories.

=

u I’Jk is a trajectory generated by

( P P , PE).

pair (p,,, pE) E R , x R , and a function

r,t(xi,

pE) =

6 E y ( x i , pp,

pe), such that

rij u rx(xj,F,,

Definition 1.10. We shall say that trajectories if 1’’j u Ffkis a trajectory Fik.

ri7and

I’jk

can be joined

One can readily verify that if two pathst 7rii and 7rik can be joined, then u &li is a path vik. Unless otherwise indicated we shall adopt

7rii

i. Recall that a path is a trajectory. However, the converse need not be true.

1.7

OBJECTIVES OF THE

GAME, QUALITATIVE

AND QUANTITATIVE GAMES

9

Assumption 1.1. Every trajectory rij,generated by (rl+ rE) and ending at xi, can be joined with describing curve r x ( x j , i , , i,), no matter what i, E R,, ,P E RE and x E y ( x j , F,, iE). Furthermore, if i, = r p , there exists a pp that is equal to r,; and if i, = r E , there exists a p, that is equal to r,. As a direct consequence of Assumption 1.1, all trajectories rzJ and can be joined; and (i) if rz3 and P k are generated by ( r P , r,) and (rf,, ie),respectively, then r2?u lrJk is generated by a strategy pair ( r p , p E ) ; (11) if Yz3 and are generated by (r,, rB,)and (i1+r E ) ,respectively, then rz3 u is generated by a strategy pair ( p p , r,). 1.7 OBJECTIVES OF THE GAME, QUALITATIVE AND QUANTITATIVE GAMES

Definition 1.11. The objectives of the game are the targets, 8,, and 8, in Case (a), and 8, or OE in Case (b), and two prescriptions, one for player J , and the other for player JE. These prescriptions depend on the type of game considered. (They are given below for the two cases studied in this book.) 1. A qualitative game (or game of kindt) is a game in which the criterion is qualitative. In Case (a), for any given xi E G, the prescription for player J , is to use a strategy rI, E R, such that (rlJ,re) generates a terminating play with end state x f € 8,, no matter what the strategy rE E RE of JE. We shall say that his prescription is to transfer the state to 8,. The prescription for player J E is to transfer the state to OK. Of course, one or the other player, or both, may be unable to control the play in such a manner that the condition which has been prescribed for him is satisfied. Definition 1.12. In a qualitative game, we shall say that a player is a winner if he has transferred the state of the game to his own target, despite the efforts of his-opponent. -i According to the terminology of Isaacs.

10

1

GAMES AND PLAYS

Both players can win if the intersection of 8, and 8, is not empty. In Case (b), O1, # a,0, = m (or 8, # 0 ,8, = 0)the prescription for player J,, (or J,) is to transfer the state to 8, (or 8,), and the prescription for his opponent, JE (or J,,), is to prevent this event. If J,, (or JF:) transfers the state of the game to his target despite the efforts of his opponent, he is the winner. According to Definition 1.12, the player who wishes to prevent the state from reaching the target cannot win since he has n o target of his own.

2. A quantitative game (or game of degree?) is a game in which (i) there is given a single target$ 8 = 8, = 8,; and (ii) there is given a functional which assigns a unique real number to each transfer effected by a strategy pair along a path. Such a functional is called a performance inrie.u, and the number which it assigns to a transfer is called the value of the functional, or the cost. In zero-sum, two player games, the prescription for one of the players is to maximize the cost of transfer from an initial state to the target, while the prescription for the other is to minimize it. In such games there is no winner, since the outcome of the game is a compromise between the players. 7 According

to the terminology of Isaacs. $ This case is Case (a) with 01, = 0,.

II

Some Geometric Aspects o j Quantitative Games

2.1 TRANSFER OF STATE AND PERFORMANCE INDEX

As we have seen in Chapter I , given an initial state xi in G, the rules of the game, the strategies r p and rE chosen by players J,. and .IE from prescribed sets R, and RE,respectively, and a function x E ?(xi,rI,, ra2), the state of the game moves along describing curve rX(xi, r,, r E ) . Here we shall consider the paths which are generated by all strategy pairs. In particular, we shall be concerned with transferring the state along 9 path from an initial state xi to any one in prescribed set of states 0 in G. In general, some paths will correspond to a transfer of the state from xi to some terminal point xf in 8, while others will correspond to a transfer from xi to a terminal point which does not belong to 8. Definition 2.1. Next let us adopt a functional which assigns a unique real number to each transfer effected by a strategy pair along a path. Such a functional will be called a performance index, and the number which it assigns to a transfer from initial point xi to end point xs,along a path risin G , will be called the cost of the transfer.

We shall admit a performance index such that the cost of transfer from xito xsalong path risdepends on the generating strategies r p and rR. We shall denote the cost of such a transfer by V(xi, xs;r I , , r E , ris). Now we introduce Assumption 2.1. The cost which is associated with a null path no is zero; that is Y ( x i , x i ; r p , rE, TO) = 0 V r , E R,, V r , E R, (2.1)

12 2.2

I1

SOME GEOMETRIC ASPECTS OF QUANTITATIVE GAMES

OPTIMALITY

I t will be convenient to consider a quantitative game as a game with two targets O,, and O,, for which 8,. = 0, = 8

Then, for any initial state xi, a strategy pair (rl+ Y E ) which is playable for

J,, at point xi is also playable for J E at that point. Accordingly, in that case, we shall say that (rI+ re) is a playable strategy pair at point xi,

without specifying the player who is concerned, and we shall let ~F(xi) denote the set of all playable pairs ( r p , re) at point xi. We shall let V ( x i ,0 ; rz,, r E ) denote the cost of transfer from xi to 8 along a path rifgenerated by ( r p , r E ) E .F(xi). Since, for given xi and (r,,, rTJ € . F ( x i ) , rif need not be unique, V ( x i ,8; r p , r,) need not be unique. Along nif we have V ( x i , 8 ; rl,, rE) = %'-(xi, xf; r,,, r E , .riff>

(2.2)

Definition 2.2. Let there be given a fixed set X c G . We shall say that a strategy pair (rl,*, r E * ) is optimal on X , if (i) it is playable at all initial points xi E X ; and (ii) V ( x i , 8; rz,*, rB;*) is defined for all xi E X ; that is, if there is more than one path starting at point x i , generated by ( r p * , rE*) and terminating on 8, then all the values of V(xi, 8; rI,*, r E * ) are equal; (iii) the saddle-point condition V ( s i , 8; rI,*, rI.:)

< V(xi,0 ; r p * , r R * ) Q

V ( x i , 8 ; r p , r E * ) (2.3)

holds for all x i E X and for all strategy pairs ( r p * , r E ) E Y ( x 2 ) and

(rt+ rrc*)E 7(xi).

We emphasize that (2.3) must hold for all values of V ( x i , 8; rp*, r E ) and all values of V ( x i ,0 ; r P , rh;*). Definition 2.3. Since V(x', 8 ; rp*, rE*) is defined for all xi s' V*(xi),where V*(xi) L V ( x i , 8 ; rp*, r E * )

E

X, V * :

(2.4)

is a (single-valued) function on X . We shall call V * ( x i ) the Value of the game for play starting at xi.

If (i)-(iii) are satisfied for all xi

E

X , then, by Assumption 2.1, (i)-(iii)

2.3

13

GAME AND ISOVALUE SURFACES

are satisfied for all xi

E

X U 6; and, of course, for xi E 6 we have V*(XZ) =

Let X *

0.

A X u 6.

2.3 GAME AND ISOVALUE SURFACES

+

Let us introduce another variable, xo, and consider an (n 1)-dimensional Euclidean space En+l of points x , where x = (xo, x) = (x,,, x,, . . . , x,) is the vector which defines a point relative to a rectangular coordinate system in En+l, the augmented state space. Definition 2.4. Since V * : xz+ V*(xi)is defined on X * , we can define a game surface X(C) in X* A X* x {xo} by

C(C)

A {x:

xo

+ V * ( x ) = C}

(2-5)

where C is a constant parameter. The intersection of E(C) with X * will be called an isovalue surface7 S(C)

& {x:

V * ( x ) = C}

(2.6)

Surface S(C) is the locus of all initial states for which the Value of the game is the same, namely C ; hence the name isovalue surface. As the value of parameter C is varied, Eqs. (2.5) and (2.6) define one-parameter families of surfaces {C(C)) and {S(C)). For each given C, Eq. (2.5) defines a single-sheeted surface in %*; that is, X(C) is a set of points which are in one-to-one correspondence with the points of X * . Consider now two game surfaces C(C,) and X(C2), corresponding to two different parameter values C, and C,, respectively. Let xol and xO2 denote the values of xo on C(C,) and C(C2), respectively, for the same state. Then it follows from (2.5) that xol - x o 2 =

c, - c,

vx E X ”

Consequently, the members of the one-parameter family {C(C)> may be deduced from one another by translation parallel to the xo-axis. Furthermore, these surfaces are ordered along the xo-axis in the same way as the parameter value C ; that is, the “higher” a surface in the family the f Note that S(C) may be an empty set.

14

11

SOME GEOMETRIC ASPECTS OF QUANTITATIVE GAMES

greater is the parameter value C. Thus one and only one game surface passes through a given point x in X*. Now let us introduce some nomenclature. Definition 2.5. A given game surface C ( C ) separates the set %* into two disjoint sets. We shall denote these sets by A / C ( C ) [“above” C(C)] and B/C(C) [“below C(C)”], respectively. For a game surface X(C)

Frci. 2.1. Game and isovalue surfaces.

corresponding to parameter value C , we have A/C(C) & ( x :

B/C(C) 4 { x :

> c - Y*(x)> xo < c - V * ( x ) } xo

(2.7)

(2.8)

A point x E A/C(C) will be called an A-point relative to X(C), and a point x E B/C(C) a B-point relative to C(C).

2.4

I

15

PATHS I N AUGMENTED STATE SPACE

\\

I

"

' \ \ I

II

XE

S/C(C)

I I

FIG.2.2. A-point and B-point relative to C(C).

Hence, the equation of the game surface through a point xi E X *is

xo + V * ( x ) = xgi

and

+ V*(xi) = c

(2.9)

> V*(xi)- V*(x)} - xgi < V*(xi)- V*(x))

A / C ( C ) = { x : xo - xgi

(2.10)

B/C(C) = ( x : x,

(2.11)

2.4 PATHS IN AUGMENTED STATE SPACE

Definition 2.6.

Paths n i s ( C )in 3 & G x {x,} are defined by

niS(C)& { x ' :

xgk

+ V ( Xxs; " , r p , rE, nks)= C , rkS c nis} (2.12)

where rris is a path in G from xi to xsgenerated by strategies r , rE E RE,and C is a constant parameter.

E

R, and

Thus, path nisis the projection on G of paths nis(C)in 9. Let 0 A 8 x {xo} be the target in augmented state space. One can

16

SOME GEOMETRIC ASPECTS OF QUANTITATIVE GAMES

I1

X’

Xi

FIG.2.3. Path in augniented state space.

easily verify that the above definition of a path in 3 agrees with the general definition of a path given in Section 1.3. By varying the value of parameter C in (2.12) one generates a oneparameter family of paths {IIis(C)}. This family belongs to an x,,cylindrical surface whose intersection with G is path ris. Definition 2.7. Three families of paths emanating from points in %*, (n;i,(C)},{np(C)},and (II;&(C)>, are defined by TI$(C) = { x k :

n ” , ” ~A) { x k :

xp + ~ ( x ” x, f ; rIJ*,TI.,.,

= C, r;;c

n-2) (2.13)

x:

+ y ( x k , xm; r I J ,rR;* , r R ) = C, rp c r p > km

(2.14)

nrE(C)= { x k : x:, + Y ( x k ,xQ;rlJ*,rP*, r:yI;) A

1 j = C , n-PE c r nq PE

(2.15)

2.6

17

SOME PROPERTIES OF GAME SURFACES

Paths T$, T:? and T;% are paths in G, emanating from points xi,xz,and xn of X * , generated by the strategy pairs (rp*, rE), (rl>,rE*)and (rp*,rE*) and ending at points x3,9" and xQ,respectively. Definition 2.8. Since (rE,*,r$;*) is playable at all points xn E X * , there exists a path n;f,(C) which reaches 0 at point x f . It is called an optimal path. Hereafter we shall denote an optimal path by n*(C). Definition 2.9. If (rI,*, rE) and (rp, rE*) are playable strategy pairs at points xz and xl,respectively, there also exist paths n$(C) and ng(C) which reach @.t They are called P-optimal and E-optimal paths, respectively. Hereafter we shall write simply lI,(C) and IT,(C) to indicate these paths. 2.5

JOINING OF PATHS

Let ni3be a path emanating from xi,generated by ( r p , rE), and ending at x3; and let xis be a path emanating from xi,generated by (Ep, i E ) and , ending at xs. Tn addition to Assumption 1.1, according to which rii u xis is a path riS generated by ( p r , p6), we shall introduce Assumption 2.2

+ V ( x j ,xs; E,,

F,,

= Y ( d ,xs; p p ,

PE,

Y ( x i , x3; rl,, r E , TP)

nlS) T i S )

(2.16)

We shall call this assumption an additivity property of the cost. 2.6 SOME PROPERTIES OF GAME SURFACES

In preparation for a fundamental theorem, let us now prove Lemma 2.1. No point of a path rIp(C') which emanates from x i is an A-point relative to the game surface through x i . iOf course, superscript f means that the terminal point of the path belongs to 0. However the terminal point need not be the same for paths IIYfC), rIL!(C), and IIFL (C).

18

11

SOME GEOMETRIC ASPECTS OF QUANTITATIVE GAMES

\

\

Frci. 2.4. Paths

I

I

I

I

n;(C’), n;(C”), n$B(C”) and

\

game surface through xz.

Lemma 2.2. No point of a path n$(C”)which emanates from x i is a B-point relative to the game surface through xi.

Let x 3 = (so3,xj) be a point of a path rlg(C’) generated by (rI,*, r E ) , where . K ~is its projection on G. Let r i j c r Ii,sbe the path in G emanating from s’,generated by ( r p * , r E ) and ending at xj, which has the same supporting curve as T:. First note that, since the family of game surfaces is defined on X * , xj cannot be an A-point relative to any member of the family if xj 6X * . Hence let us suppose that xj E X*. By condition (i) in the definition of an optimal strategy pair, there exists a path ryE which emanates from xj, is generated by ( r p * , r E * ) and reaches 8 at point sf. From Assumption 1.1 it follows directly that T$ U rjpfEis a path r);. Let ( r p * , pE) be the strategy pair that generates this path. Path r;: reaches 8 at point x f , and associated with it is a value of the cost of transfer ~ ( x ’ ,6 ; r p * , pE) = V(x2,

x’; rp*, p E ,

rg)

(2.17)

2.6

19

SOME PROPERTIES OF GAME SURFACES

In view of the saddle-point condition we see that V ( x 7 ,8; rp*, p E )

< V ( x Z 0;,

V*(xz)

rp*, YE*)

(2.18)

From the additivity property of cost (2.16) we have Y ( x z ,x f ;rI.*, pE,

TI?)

where

= Y ( x z ,x3;rp*, rE, +)

+ V ( x 3 ,x f ;r,,*, rE*, n;fE)

V ( x ’ , xf;rI,*, rE*, + ),

n

= V*(x’)

(2.19) (2.20)

From (2.17), (2.18), (2.19) and (2.20) we obtain Y ( x ’ , x’; rf,*, r E ,

+> V*(x’) Q

*(x3)

Then it follows from (2.13) and from the additivity property of cost that, along path IIY(C’), and hence

x,,’ - x: X”’

= Y ( x z ,x 3 ; rp*, rE, rg)

- xo” < V*(XZ) - V*(X?)

(2.21)

Finally, it follows from (2.21), (2.9) and (2.10) that x 3 I S not an A-point relative to the game surface through x’. Lemma 2.2 can be established by similar arguments. Lemmas 2.1 and 2.2 have the following corollaries. Corollary 2.1. All points in %* of a path nyE(C“’)whicfz emanates from x’ belong to the game surface through x’.

This corollary is a direct consequence of Lemmas 2.1 and 2.2 together with the definitions of A- and B-points relative to a game surface. Corollary 2.2. A path II$(C’) whose initial point is a B-point relative to C(C) has no A-point relative to C(C) and, of course, no point in C(C). Corollary 2.3. A path ng(C”) whose initial point is an A-point relative to C(C) has no B-point relative to C(C) and, of course, no point in C(C).

These corollaries are direct consequences of Lemmas 2.1 and 2.2, respectively, together with the translation property of game surfaces.

20 2.7

11

SOME GEOMETRIC ASPECTS OF QUANTITATIVE GAMES

UNIQUENESS OF THE VALUE OF THE GAME

Suppose there exist two optimal strategy pairs ( r l , * , rF;*)and (tp*,t E * ) ; that is, both satisfy conditions (i)-(iii) of Definition 2.2. Upon applying these conditions we arrive at once at Lemma 2.3. If and i j

XI

E

(r,>*,

then

X * , both (rl.*, rfi;*) and (FI,*, FK*) are optimal on X * , E .F(x') and

( F P * , r g * ) E F(.Y'),

V ( s ' , 8 ; r I , * ,rR;*) = V ( Y , 0;

FE*)

I n other \c~orrls,the Value of the game V *( s ' ) is independent of the choice of optimal straregy pair. We also have Corollary 2.4. If both (rl,*, r E * ) and (tl,*,Fx*) are optimal on X * , antlif(rp*, i,.*) and (F,,*, Y E * ) areplayable for allxi E X , then ( r I > *tE*) , and ( F T , * , rr:.*)are also optimal on X * .

I II

DifSerentiaI Quantitative Games

3.1 THE STATE EQUATIONS AND STRATEGIES

Thus far we have not specified the rules of the game. In this chapter we shall restrict the analysis to differential quantitative games. Let there be given the set of differential equations d.uJdt =fi.(x1,

x2,

. . . , x,

. . . , a,, v = 1,2, . . . ,n U l , u2,

01, 0 2 ,

.- .

I

Us),

(3.1)

termed state equations. Hereafter, x, = t will denote the time variable. It follows that f,(x1,

x,, . . . ,x n , U l , u2, . . . , u,,

01, u 2 ,

. . . , v,)

=1

We shall assume that state x = (sl, .uZ, . . . , x), belongs to the domain G of En, that vectors u = ( u l , u 2 , . . . , u,) and u = (ul, v2,. . . , v,) belong to the domains U and V of E'and E", respectively, and that the functionsf,, v = 1 , 2 , . . . ,I I - 1, are of class C1 on G x U x V. We shall consider strategies for players Jr, and J E to be functions of x , p : s--p(x) and e : x+e(x), x EG, respectively, belonging to prescribed classes of fu1ictions, and such that

p ( x )E K , ( X ) e(.x) E K,(x)

cu c

V

where K I L ( xand ) K,,(.u) are given subsets of E' and E", respectively, which may depend on s. These sets of strategies will be denoted by 9,and Y E respectively. ,

22

111

DIFFERENTIAL QUANTITATIVE GAMES

Of course, we can also use the notation of Chapter 1; that is,

Rp = yp, rp = p ,

R, = rF = e

The change in notation is introduced because we wish to emphasize that the sets of strategies are now sets of functions. Here we need not specify the classes of functions p and e . However, we shall require that Y,, and Y , satisfy Assumption 3.1. Whatever p‘, p” E Y;, and e’, e” E Y E ,and whatever s3E G, functions p”’: .Y 4 p’”(x) and e’”: x + e”’(x), x E G , where

p’(s)

/I”’(= .)

e’”(x) = e‘(x)

for x ,

< x,?

are strategies; that is, p”’ E Y,, and e’” E 9,. 3.2 TRANSFER TIME

Definition 3.1. In the following discussion we shall use a variable T , called transfer time, which is a function of time t , T : I + T ( t ) on an interval that depends on G, defined by two conditions; namely, dT/dt = 1 and, for xi E G, .(ti) = 0, where xni = ti.

FIG.3. I . Transfer time

T.

3.4

TARGET

23

0

3.3 RULES OF THE GAME

Equation (3.1) can be rewritten as (Ix/dT = f ( X ,

U , V)

(3.2)

For given strategies p and e , Eq. (3.2) is replaced by W d T = f ( x , p ( 4 , e(x),

(3.3)

x = x ( T ) , T E [0, T ~ ) is , a solution of Eq. (3.3), satisfying If x: T x i = x(O), then u and u are given by functions of T , u, and v; namely, ---f

u:

T + 1/

V:

T

-+

= u(T),

u = v(T), 7

E

u(T) = p(x(T)) V(T)

= e(x(-r))

(3.4)

10, T s )

There may be more than one solution of Eq. (3.3) that satisfies x(0) = x*. Of course, if p and e are C1 functions of x in a neighborhood of xi,it follows that there is a unique solution x of (3.3) satisfying x(0) = xi on an interval AT containing T = 0. But if xi is a point of discontinuity of p or e , or of both p and e , there may be more than one solution of (3.3) that emanates from xi. Furthermore, a solution that is unique in a neighborhood of the initial time may bifurcate at some later time, if the trajectory reaches a point of discontinuity of p or e , or of both. Also, there may be no solution of Eq. (3.3) in [0, 00) that satisfies x(0) = xi. The rules of the game consist of G, Y,, Y E and , mapping y defined as in Example 1.1 of Section 1.2. 3.4 TARGET 0

Assumption 3.2. Target 0 is closed and its boundary7

BAGncompO is defined by the equation 0(x) = 0

(3.5)

where function 0 is of class C' and grad 0 ( x ) # 0 on a domain containing

ae.

t Cornp 8 means the complement of 8 in G . 6 and comp 0 are the closures of 8 and comp 0, respectively, in the topology induced by En on G (see Appendix).

24

111

DIFFERENTIAL QUANTITATIVE GAMES

3.5 INTEGRAL PERFORMANCE INDEX

Now we shall introduce a functional which assigns a cost to a transfer of the state. Let fo: (s,u , v) 4 f o ( x ,u , v ) be a prescribed function of class C’ on G x U x V . I n addition to Assumption 3.1, we now introduce Assumption 3.3. The sets of strategies SP,, and Y Eare such that, for all G, for allp E Y,,,for all e E Y E and , for all paths risin G emanating from sz,generated by ( p , e), represented by x: T 4.Y = x ( T ) , T E [0, T 5 ] . the integral

s1 E

where

U(T)

= ~ ( x ( T ) )V ,( T ) = e(x(T)),

is defined.

Now let us consider a functional or performance index in integral form

T

1 ’(Y, u‘; p, e, r”) =

f0(x(7),u(T),v(T))di-

(3.6)

where Y’ = ~ ( 7 , ) . If T , = 0, then d sis a null path no, and Y-(.xz, Y’; p , e , no)= 0 ‘ifp E 9’,, Ve E 9, Hence, Assumption 2.1 is satisfied. If sSE 8, according to our earlier notation, we have T , = T~ and

T

Let us introduce a variable so and a function xo: [O, T ~ ] such , that

T

4

(3.7)

xo = x~(T),

E

(3.9) where C is any given constant. If T , # 0 we have dXu,/dT =fo(X, U , 0) for all T E (0, T ~ ) . Equations (3.2) and (3.10) constitute a set of n which we shall write in vector form dx/dT = f ( x ,

U,

v)

(3.10)

+ 1 scalar equations (3.11)

3.6

25

A VARIATIONAL EQUATION AND ITS ADJOINT

3.6 A VARIATIONAL EQUATION AND ITS ADJOINT

Next we introduce Assumption 3.4. There exists a strategy pair (p*, e*) that is optimal on a set X c G.

By (3.7), ( p * , e*) is optimal on X * X U 8. Consider a non-null optimal path II*(C) that emanates from xi E T * ,X * A X * x (x,}, is

Xf

n*u i I

I I

1

I

I I

I

I

I

I

I I

I I

I

I

FIG.3.2. Path n*(C)and its projection

X*

on G .

26

III

DIFFERENTIAL QUANTITATIVE GAMES

generated by ( p * , e*), is given by x * : T + x = x * ( T ) on [0,T ~ ] ,and reaches 0 2 0 x {x,,} at xf = x * ( T ~ ) . In other words, x * is a solution of d x / h = f(x,

(3.12)

p * ( x ) , e*(x))

Let v* be the projection of rI*(C) on G. Definition 3.2. If there exists a neighborhood of T * on which p* and e*, are of class C1, we shall associate with Eq. (3.12) the variational equation -= (17~ (

I

~

(af -+--+-af ap*

ax

all ax

afa.') aU ax

1

r

(3.13)

X=X*(T)

where v = 0,1,. cc = 0, 1 , .

ax

. . ,n ..,n

v = 0 , 1 , . . . ,n v . = 1,2, . . . , r v = 0,1, . . . , n a = 1 , 2) . . . )7.

av evaluated for u

= p*(x), u = e s ( x ) ;

ap. n ax -

ax

3P"*(X)

[

ax,

1

and v = 1,2, . . . , r v. = 0, 1 , . . . , I2 v = 1,2, . . . , s a = 0, 1, . . . , ri

For given initial condition q = ri at T = 0, the solution q: = T ( T ) of Eq. (3.13) is unique and continuous on [0, T ~ ] . Furthermore, ?(T), T E [O, Tp], is nonzero provided q i is nonzero. T + yi

3.6

27

A VARIATIONAL EQUATION AND ITS ADJOINT

Definition 3.3. The equation adjoint to Eq. (3.13) is (3.14)

where

For given initial condition 1. = 2 at T = 0, the solution A: T+ 3, = A ( T ) of Eq. (3.14) is unique and continuous on [0, Furthermore, A(T) is nonzero provided Az is nonzero. Equations (3.13) and (3.14) can be developed as follows:

T ~ ] .

and

v = 0 , 1 , 2 ) . . . ,n

It follows from (3.13) and (3.14) that (d/dT)(A(T)

so that

A(T)

-

Y-J(T) =

If the initial vectors 2

-

r1(T))

constant

= A(0)

p.qizo

=0

V T E [0, 7-53

(3.17)

and qi = q(0) are such that with

))3bi/)))$)I

f. 0

that is, if lbiis perpendicular to qi,then it follows from (3.17) that A(T) * q ( T )

=0

VT E

[o, 751

(3.18)

and since IIA(T)II llq(~)ll# 0, A(T) is perpendicular to ?(T) for all 7- E [O, 751.

28

111

DIFFERENTIAL QUANTITATIVE GAMES

Definition 3.4. Equation (3.13) defines a linear transformation .d(O,T ) of 7 j i such that ?(TI = d @ > T ) 7 ' (3.19)

Since Eq. (3.13) is linear and homogeneous, this transformation is nonsingular; that is, the inverse transformation ,d-'(O, T ) such that 7' = . d - l ( O , i5

detincd for all

T

E

T)?(T)

[O, T r ] .

F K . 3 . 3 . Transform of plane P(x*(O))due to . d ( O ,

7)

Next, let P ( x ' l ( 0 ) ) be an n-dimensional plane containing point x*(O) of T l i (C), and let P ( x * ( T ) )be its transform due to the linear transformat i o n d ( 0 , T); namely, f(X*(T))

== c d ( 0 , T ) P ( X * ( O ) ) = { x * ( T )

+ riz ~ P ( x " ( 0 ) ) ;

+d(0,

T)?7':

X*(o)

It follow\ from (3.18) that, if I' IS perpendicular to P ( x F ( 0 ) ) then , X(T) perpendicular to P ( x * ( T ) )for all T E [o, T r ]

ic

3.7

29

JOINING OF PATHS

3.7 JOINING OF PATHS

We shall now prove that Assumption 1.1 is satisfied. Consider a trajectory rij in G that emanates from xi, is generated by , ends at point ( p , e ) , is given by x: T + x = X ( T ) on [0, T ~ ] and xj = X ( T j ) .

Consider also describing curve Pp(xj,fi, 6) that emanates from xj and is given by S E y ( x j , $, 6). Of course, x ( T ~= ) S(0) = xj. If P, or rn(xi, 6,S), or both, reduces to a single point, Assumption 1.1 is trivially satisfied. Accordingly, let us suppose that neither rii nor r t ( x j , j , 6) reduces to a single point. Then, x is a solution of

dxld7 with initial condition x(0)

=f(.y,

= xi;

&/dT

(3.20)

p(x>,e(x>>

and S is a maximal solution of

=f(X,

(3.21)

f i ( x ) , e"(X))

with initial condition S(0) = xj. Now consider the (n-1)-dimensional plane P ( x i ) in En which contains point xi and is perpendicular to the time axis, namely the plane whose equation is x,, = xni. This plane separates E n into two half-spaces, the closed half-space D, {x: x, Q }x: and the open half-space D2 & {x:

x,

> X,j}.

From Assumption 3.1 it follows that functions e": x + e"(x), x E G, where

P(.4

=y

(4

e"(x) = e(x)

1

I

P(x) = fi(x) f?(.Y)

= @(X)

p:

for

XED, nG

for

,YE

x +P(x)

and

D, nG

are strategies. Moreover, if the players employ the strategy pair (jj,e"), the state equation becomes (3.22) d X / d T = f ( 9 , F(X), e"(x)) There exists a maximal solution 2:

T +x

= 2 ( ~of) Eq. (3.22) which

30

111

DIFFERENTIAL QUANTITATIVE GAMES

satisfies the initial condition %(O) = x i , namely the solution 2 such that a(T)

= X(T)

%(T)

=f (~T

~ )

for

T

E

for

T

E [ T ~ T~ ,

[0, T

~ ]

+

T+]

where T+ is the extreme future value of T associated with solution f. Solution 2 generates describing curve r Z ( x i p", , e"), and we have q x i ,

Furthermore, and

p, e") = rij u i ~ ~ p( , ~s)j , p=p*p=p e^= e a Z = e

Since these arguments apply whatever the trajectory and the describing curve I'%(xj,6,s), Assumption 1.1 is satisfied. One can readily verify that Assumption 2.2 is also satisfied. Hence, Lemmas 2.1 and 2.2 apply to differential games for which Assumption 3.1 is satisfied. 3.8 TWO FAMILIES OF PATHS

Again consider a non-null optimal path n*(C) that emanates from EF*, is generated by ('*, e*), and reaches @ at x f . Let T* be its projection on G. Next we introduce xi

Assumption 3.5. All points of T * , with the possible exception of its terminal point. are interior points of X * . Assumption 3.6. For all x t T * , and for all u E K,(x) and u E K,(x), there exist strategies cx, t 9,and @, E Y Ewhich are of class C' on some neighborhood of x in G, and which satisfy the conditions cx,(x) = u and D,(x) = 21, respectively. Let x 3 2 (xo3,x3)be any point of lJ*(C), different from terminal point x f . From Assumption 3.5 it follows that x3 is an interior point of %*. Let us also consider paths rI$(C') and n??(C"), which emanate from

3.8

31

TWO FAMILIES OF PATHS

x 3 and belonz to %*; namely,

II:(C')

{ x k : xgk

+ V ( x k ,x2;aZlre*, T:) = C', &; c (3.23) + V ( x k ,x"; p * , T:?) = C", T;? c n ; F } (3.24) T$}

IIi?(C'') A { x ' : X: where aZ9and BZ1are strategies satisfying Assumption 3.6 at point x3 E T * . Let BZ9,

a,,(x3) = t l b

E Ku(x3)

= ub

E Kv(X3)

PZ,(XJ)

where ub and ub are given vectors. According to Corollary 2.1, n*(C)belongs to the game surface through x ' ; hence it follows from the definitions of a game surface and of an optimal path that the game surface through x z is C ( C ) .

J

F"

FIG.3.4. Paths IIg(C') and nim(C").

32

DIFFERENTIAL QUANTITATIVE GAMES

111

Consequently, x’ E

X(C)

Now it followc from Lemmas 2.1 and 2.2 that ( I ) no point of I [$(C’)is a B-point relative to C(C); and ( 1 1 ) no point of I I ~ ~ z ( C is”an ) A-point relative to C(C). Finally, since II$(C’) and ng(C”)belong to X * , we have N

y X ( C ) A ( A / X ( C ) )U X(C)

(3.25)

N

Bz(c)

(3.26)

11 ;
Il;:’”(C”) c O / C ( C ) where

(B/X(C)) uZ;(C)

Next we shall consider two families of paths emanating from points I1 *(C), x1 4 0 ; namely, 1. the set {ll;l(C’)] and 2. the set { LI;’:”(C”)], generated by strategies Y,, and /?r,, respectively, for all zib E KLL(x3) and all E K , (1’). x1 E

C’j

3.9 REGULAR lNTEHIOR POINTS OF A GAME SURFACE

Before discussing olher properties of paths ll$(C’) and n>Y(C”), let us return t o the definition of game surface C(C) and recall its equation q x )

&

so

+ V * ( s ) = c,

sE

x*

(3.27)

Then it follows from the definition of A / X ( C ) and B / Z ( C ) that (i) a t every point x of A/C(C), q x >

>c

and (ii) at every point x of B/C(C),

+

@(x)

+


Let x’ hb;x and xj 0 ‘ ’ x denote points of paths n$(C‘) arid Ilj:.(C”), respectively; and let [ and [ be functions

2:

T

4

=

[(T)

[:

7

+ h‘’x

=

[(T)

where t(0) = <(O) = 0 at point on and /j7,,respectively.

xi,

and the intervals of definition depend

3.9

33

REGULAR INTERIOR POINTS OF A GAME SURFACE

From (3.25) and (3.26), and from (i) and (ii) above, we can conclude that 1. at every point x j dF:x of path IIg(C’),

+

q x j

and 2. at every point

xj

+P X ) >c

(3.28)

+ SL’x of path rI;Y(C”), + 6”x) < c

(3.29)

@(xj

Next we introduce

Assumption 3.7. At every point sj of rr* there exists a neighborhood A(xj) in G of 2 , on which p * and e* are of class C’. Assumption 3.8.7 At every point x j of rZ*(C), x j 4 0 , there exists a cP(x), neighborhood A(x3)in %* of x j , such that the function 0: x x E A(xi), is of class C1. --f

In view of Assumptions 3.6 and 3.7 we have

PROPERTY 3.1. Whatever x j E II*(C), x j 6 0 , there exist non-null paths TI;j(C’) and IIiY(C”),and an interval [0, €1, E > 0, on which the functions E and 5 are continuous. Then, as a consequence of Assumption 3.8 we have PROPERTY 3.2. Every point x of game surface C(C), in a neighborhood in ?Z“* of x j , can be represented by x =

xj

+ €719 + a(.)

where lime+, [\lo(.)ll/c] = 0 and qj belongs to an n-dimensional plane, the tangent plane of the surface at x j . Definition 3.5. If a point x j of C(C) possesses a neighborhood that belongs to X * , it will be called an inferior point of C(C). A point x j of C(C) at which Property 3.2 holds will be called a regular point of C(C). An optimal path ll*(C) all of whose points, with the possible exception of the terminal one, are regular interior points of C(C) will be called a regular optimal path.

t One can verify that Assumption n*

is not tangent to 0.

3.8 is a consequence of Assumptions 3.5 and 3.7 if

34

111

DIFFERENTIAL QUANTITATIVE GAMES

Note that Assumption 3.8 is stronger than the assumption of regularity ; of course, according to Assumption 3.8 together with (3.27), the partial derivatives

. .-a v * f X ) ax, ax, are defined and continuous, and hence

a~.v *(x)

(3.30) is defined and continuous at every point x ’ , x 3 4 0 , of II*(C). However, at a regular point of C(C), some of the partial derivatives of V*(x)may be not defined, since the tangent plane of Z(C) at that point may be parallel to the x,-axis. Now let us return to path n$(C’), defined in Section 3.8, that emanates from x’ E n*(C), x3 4 0 , and is generated by strategy pair (a,,, e*). We shall assume that IIg(C’) is not a null path.? It follows from Property 3.1 that /lbExll -0

as

7-0

Since x 3 E C(C), and accordingly @ ( x 3 ) = C , it follows from (3.28) and from Assumption 3.8 that grad @ ( x 3 ) . d K x where

Upon dividing (3.31) by

T,

+ o(l16Exll) 2 0

which is positive for a K x # 0, we get

7

Then allowing

T + 0,

r

and consequently

(B”x/T)

we obtain

(3.31)

+

f ( x j , u’, e*(x?))

grad < P ( x 3 ) . f ( x 3 , ub, e*(x3))2 0

(3.32)

for all u” t KIC(x3), and for all x 3 E II*(C) with the possible exception of the terminal point of II*(C). Such non-null paths exist according to Assumptions 3.6 and 3.7.

3.9

35

REGULAR INTERIOR POINTS OF A GAME SURFACE

il XO

I ‘

\

FIG.3.5. Regular interior points of a game surface.

By invoking similar arguments for a non-null path IlF(C‘’) that emanates from point xi E ll*(C), xi $ 0 , and is generated by strategy pair ( p * , B,j), we obtain grad @(xj) . f ( x i , p * ( x j ) ,ub) Q 0

(3.33)

for all ub E &(xi), and for all xi E rI*(C) with the possible exception of the terminal point of II*(C). Moreover, since n*(C)c E(C),we have grad @(xi) * f(xj,p*(xi), e*(xi)) = 0

(3.34)

for all xi which belong to H*(C), with the possible exception of the terminal point of II*(C). Let n(xj) A grad @(xj), xi 4 0 It follows from (3.30) that no(xj) = 1

(3.35)

where no(xi) is the x,-component of n(xi). Note that n(xi) is normal to the

36

111

DIFFERENTIAL QUANTITATIVE GAMES

game surface X(C) and directed into region A / X ( C ) at point xi of X ( C ) ; that is, there exists a positive number 0 such that point xi myxi) belongs to A / X ( C )for all 6 on ( 0 , 0).

+

3.10 TRANSFORMATION OF A TANGENT PLANE

Again consider regular optimal path 1 I" ( C ) ,represented by x " : 7

T E

+X*(7),

[o, 751.

Let x i = ~ " ( 0and ) xj 2 x * ( T ? ) , T~ E [0, T~). Now let us consider a neighborhood in C(C) of point x i , denoted by A ( x i ) , such that A(X')

where

A

x(0)

and 1 . lim c +0

2.

yli

[y] =

x = x(0) E C(C)}

{x:

L x*(0)

+ Eq7 +

(3.36)

O(E)

0

E P,(x*(O)), the tangent plane of

A(xz), and hence of C ( C ) , at

x"(0).

We shall be interested in a transformation of A(x2) by means of state equation (3.12), that is in a(Xj) = {X:

X

=

X(7j),

Tj E

[o, 71))

where x : T -+x(7), 7 E [O, Tj]represents a trajectory riiwhose supporting curve is a solution of (3.12) with initial condition given by (3.36). It follows from Assumption 3.7 that, to each initial point x(0) E A ( x i ) ,

there corresponds a unique trajectory rij provided that I E ~ is sufficiently small. Moreover, since T? E [0, T ~ ) x, * ( T ) does not belong to 0 for all T E [o, Tj]. Since 0 is closed, X ( T ) does not belong to 0 for all T E [O, ~ $ 1 , which implies that trajectory rij is a path, no matter what the initial point is sufficiently small. x(0) E A(xi), provided that From the dependence on initial conditions of the solution x : T + x ( T ) , T E [0, T 3 ] ,it follows that X(7j)

=

X*(Tj)

fEqi

+ O(T,,

€)

where ~ / o ( TE ) ~I I /,~ tends to zero uniformly for all 7j as E -+ 0, and $ 4 ?(T~)where q: 7 q = ?(T) is the solution of variational equation ---f

3.1 1

37

REGULAR OPTIMAL PATHS

(3.13) on [0, T r ] , with initial condition ~ ( 0=) q i ; that is, rl(Tj)

=

d ( 0 , Tj)yli

It follows from Corollary 2.1 that X(Tj) = X*(Tj)

+ €7' +

E

ff(Tj, €)

where I ~ O ( T ~ , e)II/c tends to zero uniformly for all Since (3.37) is satisfied for all x(0) E A(xi), A(xj)

c(c)

T~

as

E +

(3.37)

0.

= C(C)

Since rli E P,.(x*(O)), and since d ( 0 , T ~ is) a linear nonsingular transformation, it follows that q ( ~ ?belongs ) to a n n-dimensional plane which contains point x*(T~).In view of (3.37) this plane is the tangent plane of C ( C ) ,P..(x*(T?)), at x * ( T ~ ) . Furthermore, the orientation of P c ( ~ * ( ~ j ) ) varies continuously with T ~ . 3.11 REGULAR OPTIMAL PATHS

Now consider the solution A: T + 1. = h ( ~ )T ,E [0, T j ] , of adjoint equation (3.14) subject to the initial condition A(0) = li. If we choose li perpendicular to the tangent plane of C(C) at point x*(O), that is, if A2 has the same supporting line as n(x*(O)), then, as a consequence of (3.18) together with the conclusion of Section 3.10, A(T,) is perpendicular to the tangent plane of C(C) at point x * ( T ~ ) ; namely, its supporting line is that of n(x*(Ti)). I t follows from Assumption 3.8, together with the definition of (D and (3.35), that p: T~ + p ( ~ ~where ), p ( ~= ~ n(x*(Tj)), ) is a nonzero continuous vector function of T~ on [o, T j ) . Furthermore, A : T~ + A ( T ~is) a nonzero continuous vector function of T ? on [0, T ~ ] .Hence, if we choose il" in the same direction as n(x*(O)), A ( T ~is) codirectional with n(x*(Tj)) for all T~ E [0, T ~ ) . These properties of A ( T ~ ) together , with (3.32), (3.33) and (3.34), result in

-

>0 f(X',p*(Xj), U b ) < 0

A ( T ~ )f(xj, ub, e*(xi)) h(Tj) *

A ( T ~* )f(xj, p*(x'), e*(sj)) VTj E

[o, T r ) ,

=

I

(3.38)

where

xj = x * ( T ~ ) (3.39)

0

VllbE

K,,(.X'),

(3.40) VVbE K?>(.Y')

38

DIFFERENTIAL QUANTITATIVE GAMES

111

Let x * : T + x = x*(T), T E [O, T f ] , denote the function that represents T*.? Since f is of class C1 on G x U x V , and p* and e* are of class C1 on a neighborhood of T*T, and x * : T ~ - + x * ( T ~ as ) well as A : T~+A(T~) are continuous functions of T~ on [o, 7 5 1 , it follows that

H: where H(Tj)

T~ -+

H(T~)

.

= A ( 7 j ) f(x*(Tj),p*(x*(Tj)),

e*(x*(Tj>))

is a continuous function of T~ on [O, T f ] . Hence (3.40) holds at T~ = T~ and xi = xf. One can prove that (3.38) and (3.39) also hold at T~ = T ~ .From (3.38) and (3.39) it follows that

A(Tj) .f(X*(Tj), %f(x*(Tj)>, e*(x*(Tj)))

>0 <0

A(Tj> * f(x*(Tj), p*(x*(~j)),P z f ( ~ * ( ~ j ) ) )

(3.41) (3.42)

and PZf are strategies that satisfy Assumption 3.6 at point with M,f(xf) = ub and P=t(xf) = vb. Then allowing T~ T,, we obtain by continuity

where

~ , f

x f = x*(T,),

---f

-

>0 v b )< 0

A(T,) f(x*(Tf), ub, e*(x*(T,>)) x(Tf)

f(x*(Tf), p*(x*(Tf)),

Clearly, these inequalities are satisfied for all ub E K , ( x f ) and all vb E K,(xf). Hence, conditions (3.38), (3.39) and (3.40) hold for all T~ E [0, T f ] , for all uh E K,(.uj) and for all v b E K , ( x i ) . These relations can be rewritten as

-

min A(T) f(x,

I I E K,,( J )

11,

e * ( x ) ) = max A(T) f(x, p*(x), u )

-

h-,>( s)

i>E

-

= A(T) f(x, p*(x), e*(x)),

A ( T ) f(x, p*(.u), e*(x))

=0

V T E [0, T f ]

(3.43) (3.44)

where x = x * ( T ) and x = x*(T). Since the functionsf,,, Y = 0, 1, , . . , IZ, and the strategies p* and e* do not contain xg in their arguments, the right-hand side of the adjoint equation which is associated with the zeroth component of R is zero. It

7 Recall that

m* is

the projection of II*(C) on X *

3.12

39

TRANSVERSALITY CONDITION

follows that dAo/dT = 0, and hence A,(T) = constant (3.45) on [O, Tf). Furthermore, since 2 is codirectional with n ( x i ) , and since no(xi) = 1, A,(T)

on [0, T ~ ) . Since A: holds for all T E [0, T,].

T+

A

= constant > 0 = A(T)

(3.46)

is continuous on [0, T j ] , (3.46)

3.12 TRANSVERSALITY CONDITION

We shall now investigate conditions which must hold at the point where optimal path n*(C) reaches 0. The intersection of game surface C(C) with 0 is the set

0 n C(C) = {x: @ ( x ) = C,

x E O}

(3.47)

Because of (3.7), the condition x E O implies that V * ( x ) = 0 which, in turn, implies that @(x) A xo V*(x) = Xo Hence, 0 n C(C) = { x : x0 = C, x E O} (3.48)

+

That is, the intersection of C(C) with 0 coincides with the intersection of 0 and the plane Po, perpendicular to x,-axis, defined by xo = C. In other words, 0 n C(C) may be deduced from O by translation parallel to the xo-axis. Now let us consider again the optimal path n *(C), represented by x*: T X * ( T ) , T E [0, T ~ ]that , reaches 0 n C(C) at terminal point x f . Since A(T) is codirectional with grad @(x*(T))for all T E [0, T ~ )and , since A(T) and grad (D(x*(T)) possess zeroth components A,(T) and 1 , respectively, A(T) = A,(T) grad (D(x*(T)) VT E [0,T ~ ) ---f

Since the adjoint equation is homogeneous of degree one in A, and since A,(T) = constant > 0, we may set A,(T) = 1. Hence, A(T) = grad @ ( x * ( T ) )

VT E [0,7,)

40

I11

DIFFERENTIAL QUANTITATIVE GAMES

FIG.3.6. Transversality condition

Finally, since h ( ~is) continuous on [0,T/],grad @ ( x * ( T ) ) tends to a limit 17' as T -> T , , and h(T,)

=

Thus X(C) possesses a well defined tangent plane, P , ( x f ) , at point x f , namely the plane through x f which is perpendicular to izz.? Since, according to Assumption 3.2, 80 possesses a unique ( n - 1)-dimensional tangent plane, P,,(.u), at every point s of 80, so that the tangent plane, PXn c, (x),of 8(4 n C(C) is defined for all x E 8 0 n C(C), Pl;,,(xf)

= P,(x')

Since A('/) = d ,A(T,) is normal to P , ( x f ) , and hence to P x n e , ( x f ) . This orthogonality condition is called the tvansuevsa/ifj>condition. It This i s a consequence of the uniqueness of optimal path 11* ( C )

3.13

41

A M I N - M A X P R I N C I P L E , KEGULAR CASE

can be expressed as follows. Let 7 = (q0, q,, P , , , ( x f ) ; then A(7,j . q = 0

. . . , q n ) be a vector in (3.49)

Since xo = C and .Y E P,(xf) for points of PXnc-,(xf),and since a0 is defined by the equation B(x) = 0, it follows that 70

=0

(3.50)

3.13 A MIN-MAX PRINCIPLE, REGULAR CASE

Summarizing the results of the preceding sections, we arrive at a theorem of fundamental importance for the discussion of optimal strategies in differential quantitative games. Letting

*%?(Ib, x, u , u ) we have

A A. f(x, u , u )

Theorem 3.1. I f n*(C) is an optimal path, generated by strategies p* and e*, represented by x * : r + x = x * ( T ) , T E [0, r r ] , rf # 0 , and satisfying Assumptions 3.5-3.8, then there exists a nonzero continuous solution A: T A = A@), T E [0, rf],of adjoint equation (3.14) such that

-

8EKl'(X*(T))

- max ~ ( V Tx*(T), ) , p * ( x * ( ~ )u) ), L

=

(b)

ex,(x* ( r )

)

~ ( A ( T ) ,x*(T>,

p*(x*(T>>, e*(x*(T)>)

for all

T

E

[0, T p ]

-@(WT),x*(T), P*(X*(T)),e*(x*(T)))= 0

where x*(T) is the-projection of x * ( T ) on X * . Furthermore, if Assumption 3.2 is satis-ed, then vector (A,(T), & ( T ) , . . . , A,(T)) is normal to the terminal (d) manifold boundary 8 at T = T ~ .

42

111

DIFFERENTIAL QUANTITATIVE GAMES

Note that state equation (3.12) and adjoint equation (3.14) may be written as c_ i ~, a,n(i,, X,

21, V)

a,.

dT

nr., - - _-

a,x(J, x,

ax,

(IT

-2

7-1

u, u)

aM(lL, x, u , u ) +,*(x) au,

3%

where u = p * ( x ) , ZI = e*(x), x = x * ( T ) and = A ( T ) for T E [ 0 , T f ] . By using (3.49) in (a) and (b) of Theorem 3.1 we obtain the functional equation of dynamic programming:

3.14 DISCONTINUITY MANIFOLDS

Now let us suppose that X is domain, and consider Definition 3.6. We shall say that {XI, X,, composition of X , if

. . . , X K } constitutes a de-

(i) X,, CJ = I , 2 , . . . , K < a,is a domain of E n , and (ii) X , n X , = o k # I CJ= 1,2, . . . , K (iii) X , G X (iv) X c

xv c G ;

R

U X,

8=il

As a direct consequence of these conditions,

x = Uf=, x,.

Next we introduce an assumption that is weaker than Assumption 3.7, namely,

3.15

REGULAR PORTION OF AN OPTIMAL PATH

43

Assumption 3.9. X is a domain possessing a decomposition {XI, X,, . . . , X,) such that functions p', e* agree on each X , with functions-say p_",e*-of class C1 on a domain R, 3 F,, and on each nonempty Mkl A x k f3 k # 1, with functions-say p k l , ekl-of class C1 on a domain

xL,

RkL

Mkl*

FIG. 3.7. A decomposition of X ; discontinuity manifolds in En.

Furthermore, we shall make Assumption 3.10. A discontinuity manifold M,, is an (n-1)-dimensional surface which can be represented by m(x) = 0

where function m is of class C1and grad m(x) # 0 on a domain containing MkP

3.15 REGULAR PORTION OF AN OPTIMAL PATH

Definition 3.7. where

{C,(C), C,(C), . . . , C,(C)) is a decomposition of C(C),

+

6 = 1 , 2 , . . . ,K C,(C) A (x: O(X) & xo V*(x) = c, x E X,} And now we make an assumption that is weaker than Assumption 3.8, namely

44

Ill

DIFFERENTIAL QUANTITATIVE GAMES

Assumption 3.11. Function V* is continuous on X and agrees on each X,, 0 = 1 , 2 , . . . , K , with a function-say V'-twice continuously differentiable on R,. As a consequence of Assumption 3.1 1,

(3.53)

is defined for all x €3,& Xu x ( x , ] , and all points of C,(C), 0 = 1 , 2 , . . . , K, are regular interior points of Xo(C). Here we shall relax the assumptions of Theorem 3.1 ; that is, we shall be concerned with an optimal path n*(C), generated by strategy pair ( p * , e*), satisfying Assumptions 3.5 and 3.6, and we shall derive conditions (a)-(c) of Theorem 3.1 for points of a regular portion of n*(C), namely n * ( C ) n X,(C). By the same arguments as those employed in Section 3.8, it follows from Lemmas 2.1 and 2.2, and Corollary 2.1, that grad @(x) . f(x, p*(x), v ) Q 0

(3.54)

grad cD(x). f(x, u , e * ( x ) ) 2 0

(3.55)

grad @(x)

*

f(x, p * ( x ) , e * ( x ) ) = 0

(3.56)

for all x t II* ( C ) n X,(C), for all u E K , ( x ) and for all u E K,(x). Since ( p * , e*) is optimal on X,(3.56) holds for all x E C,(C), O =

1 , 2, . . . , K ,

and for all C ; that is, (3.56) is an identity in x on X u ,5 = 1 , 2 , . . . , K. Let us rewrite (3.56) as (3.57)

By differentiating (3.57) with respect to sa, p = 0, 1, . . . ,n, we obtain

3.15

45

REGULAR PORTION O F AN OPTIMAL PATH

Now, along a portion of optimal path n*(C)lying on C,(C), we have

From (3.58) and (3.59) we obtain

By developing form, we obtain

for x = x * ( T ) Letting

a),(.,

p * ( x ) , e*(x))/a.ua, and writing (3.60) in vector

E X,(C).

A ( T ) = grad @ ( x ) ,

x

= x*(T) E

&(C)

(3.62)

and substituting in (3.54), (3.55) and (3.56), we obtain

<0 .%'(A(T), x * ( T ) , u , e*(x*(T)))> 0 CY?

(A(T),

X*(T), P * ( X * ( T ) ) ,

u)

X (A(T), x * ( T ) , p * ( x * ( ~ ) )e*(x*(T))) , =0

(3.63) (3.64) (3.65)

Of course, conditions (3.63) and (3.64) must hold for all u E K,(x*(T)) and for all u E K,(x*(T)). Furthermore, it follows from (3.53) and (3.62) that &(T)

1

(3.66)

Clearly, conditions (3.63)-(3.66) are identical with conditions (a)-(c) of Theorem 3.1 ; that is, the above method is another way of deriving Theorem 3.1, since it obviously applies to regular optimal paths on Z(C).f We shall let the reader verify that relations (3.63)-(3.66) can also be obtained by invoking arguments similar to those of the previous sections; that is, by considering transfer of the tangent plane of C(C) along a portion of TI *(C)lying on Cu(C).

t

However, here we require that @(x) is twice continuously differentiable.

46

It1

DIFFERENTIAL QUANTITATIVE GAMES

3.16 JUMP CONDITION

Definition 3.8. We shall say that optimal path II*(C) crosses M,, x {x,,} at point x' = x*(T,), if there exists E > 0 such that x * ( T ) E X,(C)

for all

T E

x * ( T ) E C,(C)

for all

T

(T, -

E (T,,

E , T,);

and

+ c);

and

T,

Ak, LA

grad m(xc) - f ( x C pk(xc)), , ek(xc))# 0 grad m(xc). f ( x C ,pz(xc),ez(xc))# 0 We shall now investigate conditions which must hold at a point xc = x*(T,) where optimal path Il*(C) crosses Akz.

Since p * , e* agree on X , with functions p k , ek, respectively, which are of class C' on a domain R , 3 and since A : T + il = A(T),

xk,

7-

IS

a solution of

E

(Tr

-

€2

7-A

A ( T ) tends to a limit A(T, - 0) as T + T,, T < 7,. Since A ( T ) = grad @ ( x * ( T ) ) for all T E (T, - E , T , ) , limr+Te,T
x,,

+

grad @ ( x * ( T ) ) = grad @"(x*(T))

for all

grad @ ( x * ( T ) ) = grad @ ' ( x * ( T ) )

for all

T T

E

(7,

E ( T ~ T, ,

and since grad @,(xC) and grad @'(xC) are defined, we have lim grad @(x*(T)) = grad @"(xc) T+TC 7i Tr

lim grad @ ( X * ( T ) ) = grad @'(xr) TT ',

r>Tc

-

E , 7,)

+

E)

3.16

47

JUMP CONDITION

Letting

grad @(x")- L lim grad @(x*(T)) T+TC T
grad @(xc)+ 2 lim grad @(x*(T)) We have ) ~grad O k ( x c )= A(T, - 0) grad @ ( x ~ =

+

grad @(xc)+ = grad @'(xc) = A ( T ~ 0 ) Thus, z k ( C ) and c , ( C ) possess well defined tangent planes, P x k ( x c ) and P z t ( x c ) , respectively, at point xc, namely the planes through xc which are perpendicular to A(T, - 0) and A(T, 0), respectively. Moreover, according to Assumption 3.10., Mkz possesses a unique (n-1)-dimensional tangent plane, P M ( x ) , at every point x of Mkz, and hence the tangent plane Pcn.A(xc)of Jkllkz n C(C) is defined at point xc. And so we conclude that

+

P ~ ~ & ( E X ~q () x C )n pzZ(xc)

+

Since A(T, - 0), A(T, 0) and grad m ( x c ) t are normal to the (n-1)dimensional plane P, ,-,&(xC), these vectors are linearly dependent; that is, there exist constants el, c2, c3, not all of which are zero, such that clA(~,- 0 )

In view of (3.66),

+ c,A(T, + Oj + c3 grad m ( x c ) = 0

hO(Tc- 0 ) = ho(Tc

(3.67)

+ 0) = 1

Furthermore, since the zeroth component of grad m(xc) is zero, we have c1 = - c 2 ; and since grad m ( x c ) # 0, c1 = -cz

ts 0

Thus, (3.67) can be rewritten as

+ + (c3/cl) grad m(xc) = 0

A(T, - 0) - A ( T ~ 0)

(3.68)

Equation (3.68) is the jump condition at a point where optimal path

II * ( C )crosses discontinuity manifold d k z . Of course, m(xc) = m(xc).

48

111

DIFFERENTIAL QUANTITATIVE GAMES

FIG.3.8. Optimal path crossing a discontinuity manifold in E n i l ; jump condition.

Letting f-

= (fop,fip,

. . . , f,,J

4

= hm ~ ( x * ( T ) ~ , * ( x * ( T )e*(x*(T))) ), T-TC T
f+ = (jo+, f,+, . . . ,fn+)

A Iim f(x*(T>, p*(x*(T)),e*(x(T)>) T+rc T>Tc

we deduce from (3.65) and (3.68) that

c n

c3 _ _

CI

-

.I;+ + , I

1

h(Tc

- OV"+

am

-

Jo-

+ 2 w, + 0)f"-

where partial derivatives am/ax,, v

v=l

i:ax,

2 ax, -f"+

v=l

t=l

x = xc.

n

=

1, 2,

(3.69)

@fv-

. . . ,n, are evaluated at

3.17

49

CONSTRAINTS

3.17 CONSTRAINTS

Let us consider constraint sets K,(x) and KV(..;), respectively, given by Q),(x, U )

<0

y,(x,u)
a = 1, 2 , . . . , k

(3.70)

a = 1 , 2 ) . . . ,I

where functions y , and y, are of class C’ on G x U and G x V , respectively. At a point x = x * ( T ) , T E [O, T f ] , of the projection on of an optimal path n *(C), some or all of inequality constraints (3.70) may be equalities. Suppose that

x*

. . , k‘

p,(x,p*(x)) = 0

a = 1, 2 , .

y,(x, e*(x)) = 0

a = 1 , 2 , . . . , I‘


(3.71)

We shall suppose also that if k > r or I > s, at most r of the cp,(.~, u ) or s of the ym(x,u) vanish at any point of G x C/ and G x V , respectively, and furthermore that the matrices a = 1,2, ..., kf o = 1, 2 , . . . , r c(= 1,2, . . . , l? 0=1,2, ...,s

have maximum rank at x

= x * ( T ) , u = ~ * ( x * ( T ) u) ,= e*(x*(T)).

Now, at a point x E n * ( C )where condition (a) of Theorem 3.1 applies, the Lagrange multiplier rule? gives

(3.72)

t

Kuhn-Tucker conditions.

50

DIFFERENTIAL QUANTITATIVE GAMES

I11

where a . X / a ~ a, X / a v , ??', pT, and vT are row vectors:

n A 'r = (A" A, . . . a,) 'r A ,u = (pl ,u, . . . p7,.0 . . . 0) v ' ~= (vl v 2 . . . vl,0 . . . 0)

and

["":;,:

ay, a ay A ["f; av

where pa 2 0 where v, Q 0 cI =

7

11)]

L1

G =

~ 2 , . .. , iz 1,2, ..., r

1,2, ..., 1 o-= 1 , 2 , . . . , s cI=

All the partial derivatives are evaluated at u

= p * ( x * ( ~ ) ) , u = e*(x*(T)),

x = x*(T)

From (3.70) and (3.71) we see that y , ( x , p * ( x ) ) , w. = 1, 2 , . . . , k', and ym(x,e*(x)), CC = 1, 2, . . . ,l ', have a stationary maximum for X = X*(T)

$ 0.

Hence we may write7

-+z 2 agiuaPp* ax, au, ax,

2 3P.a

n=l

a=l

li=l

-

I -aye + I I - -ay, = o aeg* ax, a=lP=laup ax, 11

n

-0

CC=

1 , 2 , . . . , k' (3.73)

u = 1 , 2 , . . . , 1'

(r

at u = p * ( x * ( ~ ) )u, = e * ( X * ( T ) ) , x = x * ( T ) # x * ( T ~ ) Equations (3.73) may be written in vector form

(3.74)

t This result holds for T E [0, T,), the half open interval since all but the terminal point are assumed to be interior points of X .

3.17

51

CONSTRAINTS

where

9 ax p

[a~)$;;u)]

cc = 1 , 2 , . . . , k 0 = 0,1,. . . , n

. . ,z = 0, 1 , . . . , n

cc = i , 2 , .

ax

(r

evaluated at u = ~ * ( x * ( T ) u) ,= e*(x*(T)), x = x*(T). From (3.72) and (3.74) we obtain f aP* A T a--=

au ax Tafae* x--= aU ax

or equivalently

Tap,

FC-

ax

-

ax

[

(3.75)

where p and v are column vectors:

ru=

0

Finally, in view of (3.75) adjoint equation (3.14) may be written as (3.76) for 7 E [0, T ~ ) .BY continuity of af /ax , a y / a x , ay/ax on some E > 0, Eq. (3.76) holds on [0, T f ] .

[Tr

-

E , Tr]

for

52

111

DIFFERENTIAL QUANTITATIVE GAMES

If constraints (3.70) are independent of x , the adjoint equation reduces

to (3.77) 3.18 A MIN-MAX PRINCIPLE WITH JUMP CONDITION

At last we arrive at Theorem 3.2. If there exists a strategy pair (p*, e*) optimal on X E G , and a decomposition {Xl, X,, . . . , Xlc}, K < 03, of X such that Assumptions 3.9 and 3.11 are satisjied; i f * ( C ) is an optimal path generated by ( p * , e*), represented by x * : T + x = x * ( T ) , T E [0, T r ] , satisfying Assumption 3.5; and if the constraint sets are given by (3.70); then, on every interval [ T ~T, p ] E [O, r f ]such that x * ( T ) €5, & X,, X {x,),

n

c E (1, 2 ,

... ,K},

for r E ( T ~ T, ~ )there , exists a nonzero continuous solution A: of acljoint equation (3.76), such that

-

max

(b)

-j

1 = A(T)

X ( A ( r ) , x*(T), ~ * ( x * ( T ) )v,)

1 EA-dX*(T))

=

T

. P ( ~ ( Tx *)( T, ) , p*(x*(r)), e*(x*(T)))

for all

T

E

[T,,

Tp]

.w(h(r),X*(T), p*(x*(.)), e*(x*(.>>>= 0

(c) A,(T) = constant > 0 where x*(T> is the projection of x * ( T ) on X * .

If Assumption

E

G E (1, 2, r f ) ,then

3.2 is satis$ed, and fi there exist

> 0 , such that x * ( T ) €3,for all T E ( r f -

E,

. . . , K ) and

(d) vector (A,(r), A,(T), . . . , A,(T)) is normal to the terminal manifold 6 at T = r f . Furthermore, i f KI *(C) crosses a discontinuity manifold d k l

at

T

Mkl

x {xo:

= r C and , if Assumption 3.10 is satisjied, then (e) jump condition (3.68) is satisjed at x * ( r , ) .

3.19

EXAMPLE

3.1

53

Conditions (a)-(c) of Theorem 3.2 are direct consequences of (3.61)(3.65). Condition (d) follows from Assumption 3.11 according to which the , with tangent plane of c,(C) is well defined at x * ( T ~ )together lim grad (I, (x*(T))= A(T,)

r’rf

r
Condition (e) was derived in Sec. 3.16. 3.19 EXAMPLE 3.1

Here we shall consider a game described by

with constraints

+

K T L :u12 uz2 Q K,,: 101 Q I

where IV is a positive constant, w Target 8 is defined by

e A (x

=

1v2

(3.79)

< 1.

x2,x3)

x12

+

x22

G

~

2

(3.80)

)

We shall consider the time-optimal problem; namely, J, seeks to minimize while J, seeks to maximize the time of transfer from an initial point to the target. In this problem G = E3, and we shall suppose that there exists a strategy pair (p*, e * ) , p * A (p,*,p,*), that is optimal on a subset X * of G . As in the previous sections, we shall represent a path in G by a function x: T -+ x = x ( T ) , and we shall denote by x , ( T ) , u = 1 , 2 , 3, the components of vector X ( T ) on orthogonal axes in ER. A path in G , generated by a strategy pair that satisfies the necessary conditions for optimality, will be denoted by T * and represented by x*: T x = x * ( T ) , although ~ we cannot state that it is optimal without further investigation. This path is only a “candidate” for optimality. 1 = A(T) will denote a solution of the adjoint As before, A: T equation on some interval, and A,(T), u = 0, 1, 2, 3, will denote the --f

--j

t In Secs. 3.19-3.21

we shall be concerned only with non-null paths

T*.

54

111

DIFFERENTIAL QUANTITATIVE GAMES

components of vector A(T) on orthogonal axes in the augmented state space E4. To apply the necessary conditions we have need of the &?-function; here (3.81) X ( A , x, u , u ) = 1 l 1 U 1 12u2 l , U A,

+

+

+

U =

1, 2, 3

(3.82)

a = 1,2, 3

(3.83)

+

The adjoint equations are

dI=/dT = 0 so that A,(T)

= constant

From the transversality condition we obtain Al(Tf) = CXl*(Tf) AZ(Tf)

= CXZ*(Tf),

MTf)

=0

(3.84)

where, in view of (3.80), c2 =

4-h Z 2 ( ~ , ) ] / R 2

[A?(T,)

(3.85)

By (3.83) and (3.84) we have Al(T) G &(T)

CX1*(Tf)

EE C X z * ( T f )

(3.86)

k,(T) 55 0

From condition (a) of Theorem 3.2 we obtain

v*(T) = sgn A1(7) = sgn C X ~ * ( T ~ )

where u1*(T) = pl*(x*(T)), u ~ * ( T& ) P ~ * ( X * ( T ) ) ,v*(T) A e*(x*(T)), and so, by condition (b) of Theorem 3.2, (3.88)

3.19

EXAMPLE

55

3.1

From ( 3 . 8 9 , (3.86) and (3.88) we deduce

I 1,(7f 1I

JA,2(Tf) Hence,

+

-~ - IXl*(.f)l

<

(3.89)

R

A,2(Tf)

1x11 < RW

(3.90)

must be satisfied on the useablepart of the target, that is on the subset of the target which can be reached by a path n*. The projection on the xr-x2 plane of the useable part of 8 is shown on Fig. 3.9. It is P'Q' U P"Q", where

P'Q' g {(x,, x 2 ) : x12+ x2'

< Rw, 1x11 < Rw,

= R2, 1x11

P"Q" 2 {(x,, x2): xI2+ x2' = R2,

> 01 xz < 0) x2

Along a path n* that reaches 8 at point x f = x*(T,) Eqs. (3.78) may be rewritten as

(3.91)

dx, dr

=

1

Upon integration of (3.91) we find that a path n* that reaches 8 at point xf = x*(T,) is a straight line whose projection on the x1-x2 plane is given by x2 = ax, b (3.92) where

+

a -R ICI b = - a sgn CX,*(T~) w c

A straight line given by Eq. (3.92) in the xl-x2 plane intersects the x,-axis at point P : x1 = R / w , or at point Q : x1 = - R / w , depending

56

111

DIFFERENTIAL QUANTITATIVE GAMES

on the sign of x1*(rf). One can readily verify that P and Q belong to the tangents to the circle given by xI2 x22= R2 at the limit points P ‘ , P“ and Q’, Q”, respectively, of its useable part,? as shown on Fig. 3.9. Furthermore, from the second of Eqs. (3.91) it follows that a path TI* can reach 0 only if c > 0. Of course, a point of the useable part of 8 for which xz > 0, can be reached by a path TI* only if dx,/dT < 0; and a point for which x2 < 0 can be reached only if dx2/dr > 0. This is clear on Fig. 3.9 where the projections on the xrx2 plane of the useable part of 8 and of a family of paths are shown. This condition implies the choice of the positive root in (3.85); namely, c/lcl = 1. In summary, given any point of the useable part of 0, a path TI* that reaches 0 at that point is obtained by integrating Eqs. (3.91) backward. Its projection on the x1-x2 plane is given by (3.92), with c/lcl = 1. However, we must stop the integration of Eq. (3.91) at the point where the solution reaches the x2-x3 plane. This can be explained as follows: Through a given initial point xi = ( x l i , x Z i ,x32) that belongs to both X * n comp 0 and the ~ 2 - ~plane, 3 there pass two paths, one for hl(7) > 0 and the other for hl(7) < 0. They correspond to the same value of r f , as is seen readily by integrating the second of Eqs. (3.91); namely,

+

Xz*(Tf)[l

+ (bc/R)~fl=

X;

(3.93)

and since x2*(7,) is the same for both paths, 7f is the same. If, as we have said before, strategy pair ( p * , e*) is optimal on X * , then there exists an optimal path T * that emanates from each point xi E X * , and the value of rf computed along this path is the Value of the game for play starting at x i . Now, if xi belongs to both X * n comp 8 and the x2-x3 plane, then for the same optimal strategy pair there exist two optimal paths emanating from x’. More precisely, there is no uniqueness of the solution of the state equations

in an open ball with center x’, no matter how small this ball is. But that implies that there does not exist a neighborhood A(xi) of xi on which p* and e * are of class C’. Hence, the portion of the x Z - X ~ plane that belongs to X * n comp 19 is a discontinuity manifold for the strategy pair ( p * , e*).

t

Here, we mean by “useable part” the projection of the useable part of 0 on the

x1-x2 plane.

3.19

EXAMPLE

3.1

57

In view of the uniqueness of

a t points x * ( T ) , T # T ~which , d o not belong t o a discontinuity manifold, and of the continuity of A along non-null paths m* which d o not intersect a discontinuity manifold, we can conclude that paths with A,(T) > 0 emanate from initial states with xl0 > 0, while those with A,(T) < 0 issue from initial states with xl0 < 0. Finally, we see that the projection of X * on the xl-xz plane is the union of regions Q'P'S', Q"P"S" and the projection of target 8, as shown on Fig. 3.9.

FIG.3.9. Example 3.1. Projection on thexl-x, plane of the target, useable part of the target, discontinuity manifold and paths [8: projection on the xl-xz plane of 01.

58

DIFFERENTIAL QUANTITATIVE GAMES

111

3.20 EXAMPLE 3.2

Next let us consider a game described by

with constraints (3.95) Target 8 is defined by 6

A {x = (XI,

Xp,

x3):

x2

< O}

(3.96)

The performance index associated with a path r i S represented , by 7 --t X = X ( T ) , 7 E [o, T s ] , is

X:

("(1 - x , ( T ) x ~ ( T d-r ))

The &"-function is

(3.97)

JO

X(l.,x, u , u ) = 1 - XIX2

+ &(u + u ) +

13

(3.98)

Since the constraints are independent of the state, the adjoint equations are (3.99) so that

A3(7)

G

constant

(3.1 00)

From the transversality condition we obtain Al(Tf)

= h3(7f)= 0

(3.101)

so that, in view of (3.100), 0

&(T)

(3.102)

From condition (a) of Theorem 3.2 we obtain u*(T) = -1

>0 <0 A2(T) > 0 A&) < 0.

if

A,(T) &(T)

U*(T)

=0

if

V*(T)

=0

if

v*(T) = - 1

if

(3.103)

3.20

EXAMPLE

3.2

59

and so, by condition (b), I - X**(T)X2*(7) - &(T)

Along a path rewritten as

=0

(3.104)

that reaches 8 at point xf = x*(T,) Eqs. (3.94) may be

x*

dr

cir

d-r

(3.105)

x2

\ 7;

M

1

77-77 777-

'777;

FIG.3.10. Example 3.2. Projection on the x1-x2 plane of target 0, paths and discontinuity manifold. projection on the x1-x2 plane of M , 6: projection on the x1-x2 plane of 0.1

[G:

Upon integration of (3.105) we find that a path x* that emanates from x20 > 0, and that reaches 8 at point x f = x*(T,), is a straight line whose projection on the x1-x2 plane is given by

xo = (xl0,xZ0,x,O),

Xl*(T)

= XI'

Xz*(T)

= Tf - 7

(3.106)

By (3.104) the sign of A,(r) switches on the surface

M

g (x = (Xi,x2, x3):

1 - XlXZ = 0 )

(3.107)

60

DIFFERENTIAL QUANTITATIVE GAMES

111

Consequently, the strategy pair that satisfies the necessary conditions for optimality is given by

p*(x) = 0 e*(x) = - 1 p*(x) = - 1 e*(x) = 0

i

1 - x1x2< 0

for

(3.108) for

>0

1 - xlx,

Thus, A4 is a discontinuity manifold. Paths

emanating from points

7 ~ *

xo = (xlo,xZo,x,O), xlo > 0, xZo> 0, cross M . The jump condition (3.68) becomes h(T,

- 0)

=

h(T,

+ 0)

(3.109)

since c3/c1 = 0. Hence, A is continuous at the point where a path n* crosses M . The Value of the game for play starting at point x is

Note that @(x) is independent of the x3-coordinate. A surface @(x) = C is shown on Fig. 3.1 1 in xo-x1-x2 space. Finally,

A,(T) =

a v *(x*(T))= 1 - X,*(.)X2*(.) ax,

(3.111)

These expressions may also be obtained by integrating Eqs. (3.99).

3.21

EXAMPLE

61

3.3

XI

/

FIG. 3.11. Example 3.2. Projection on the

.Y~-X~-X~ space

of game surface Z(C).

[G:projection on the x1-x2 plane of M, 6: projection on the x1-x2plane of 0.1 3.21 EXAMPLE 3.3

Next let us consider a game described by d x 1

-= MI&

dr

_ dx2 - U 2 J G dr

- x=, I d dr where w is a positive constant.

+ -2( u + 1) W

+ -(v W

2

- 1)

(3.1 12)

62

111

DIFFERENTIAL QUANTITATIVE GAMES

The constraints are (3.113) Target 8 is the closed half-space

0

L {x = (xl,

x2,

x3): x1

< O}

(3.1 14)

The payoff is time to termination. In this problem G = E3. Let us note that there is no optimal path that emanates from an initial state in the domain 2 {x = (xl, x2,x3): x, < 0 , x1 > 0 ) . Here, the $‘‘-function is

.x (A,x, u , v)

=

1

+

Jx,(AIUl

+

AZU,)

+ ($it’)(Ai(o + 1) + &(v

- I))

+

13

(3.115)

Since constraints (3.113) are independent of the state, the adjoint equations along a path T * , generated by a strategy pair ( p * , e*) satisfying the necessary conditions for optimality, are

-d l1 =o d7

_ _- _ _ _21 1-l1 - a-2 u2 d?L2

dr

24x2

2Jx2

(3.116)

d& _ -0

dT

From the transversality conditions we have h2(rt) = h3(7f) =

so that, by (3.1 16),

0

h,(T) !!0

(3.1 17) (3.118)

From condition (a) of Theorem 3.2 we obtain U1*(T)

= -

AdT)

Jx12(r)+ hz2(r) (3.119)

3.21

EXAMPLE

63

3.3

and, by condition (b),

(i) A,(T)

+ A,(T) > 0;

then Eqs. (3.112) may be written

(3.121)

and (ii) A,(T)

+ A2(7) < 0;

then Eqs. (3.112) may be written

_ dX, --JG

A, (7)

dAi2(T)+

(17 dX, _ -

dT

-&

h2(T)

A2(7)

dAi'(7) +

(3.122)

-w &"(T)

dx3 - 1 dT

For Case (ii), it follows from (3.1 17) and (3.120) at and

T

= -rf that

MTf) < 0

which is in contradiction with the fact that non-null paths which reach 6 belong to the half-space x1 2 0. Hence, Al(Tf) Thus Case (ii) cannot apply at

T

=T

>0 ~ .

(3.123)

64

DIFFERENTIAL QUANTITATIVE GAMES

111

Condition (3.123) has two consequences. First, it follows from (3.1 17), (3.120), and (3.123) that (Jxz*(Tf) -

Lt.)Al(Tf)

>0

Hence, the subset of the target that can be reached by a path T * , that is the useable part of the target, must satisfy the condition A-2

> bt.2

(3.124)

Secondly, if there exist a strategy pair ( p * , e*) that satisfies the necessary conditions for optimality and a path X * that reaches 0 at point x * ( T ~ ) , then, in view of the continuity of h along every non-null portion of n* that does not intersect a discontinuity manifold, there exists an interval of time ( T ~- T,, T ~ ] , T, > 0, on which A,(T) and A,(T) A,(T) are strictly positive. So, let us consider Case (i) first; that is, let T E ( T ~ T,, Tf], and accordingly V ( T ) = 1 in Eq. (3.1 19). Equation (3.120) becomes

+

+

For

T

=T

1 - J X , * ( T ) JAi2(T) A;(T) we deduce from (3.125) that

+ M’hl(T)= 0

(3.125)

~ ,

A,(Tf) =

fJX,*(Tf)

- bL’Ip1

(3.126)

and so by the first of Eqs. (3.1 16), x,(T)

= constant = [Jx,*(Tr)

- wl-1

(3.127)

By substituting expression (3.127) for A,(T) in Eq. (3.125) one readily obtains (3.128) Equation (3.128) requires that (3.129) which implies the choice of the negative root in (3.128). For, if we choose the positive root, then dx,/cf.r < 0 in the second of Eqs. (3.121), which implies that x ~ * ( Tis ) a decreasing function Of T ; this, in turn, implies that x,*(T)

>x~+(T~)

for

T

<

Tf.

3.21

EXAMPLE

3.3

65

But that contradicts (3.129). Hence we have

By substituting (3.127) and (3.130) in Eqs. (3.121) we obtain

(3.131)

Next, let obtains

T'

T~

- T . By integrating Eq. (3.131) backward one

provided that T' Q T ~ X , * ( T , ) . From (3.127) and (3.130) we have

+

Hence, the sign of A,(T) A,(T) becomes negative (in the backward integration of Eqs. 3.131) when x,*(T) = Xz*(Tf)/2, that is for T' Tf - T = Jxz*(Tf)T/2. By substituting i n Eqs. (3.132) one obtains the parametric equations of a parabola &f; namely, xl=

where

(1

3

- + - s - w - J J S ,57 2

x 2 = -S 2

66

111

DIFFERENTIAL QUANTITATIVE GAMES

By eliminating s between these equations we see that the sign of A,(T) A,(T) becomes negative on the surface x1 -

+

(i+

t)x,

+

M

.

T

-

J Jx2~

+

=O

In the region where A,(T) A,(T) < 0 the control vector v*(T), given by the third of Eqs. (3.119), is v*(T) = -1. The trajectory equations can be deduced from Eqs. (3.122) by arguments similar to the ones employed in the case treated above. We shall let the reader verify that the paths thus obtained, by piecing together the part of the solution corresponding to v * ( T ) = + I and the one corresponding to v * ( T ) = - 1, cross M . Let x r = x * ( T , ) be a point where a path T* crosses M . Since v*(T) has a discontinuity at T = T ~ v,* ( T ) = e*(x*(T)), and x* is a continuous function of T on [0, T,], there does not exist a neighborhood A(xc) of xC on which e*(x) is of class C1. Otherwise v* would be continuous at T = T ~ Hence, . M is a discontinuity manifold. The projection A? of A4 on the s - x 2 plane is shown on Fig. 3.12. X2

W2

FIG.3.12. Example 3.3. Projection on the x1-x2 plane of target 0, paths and discontinuity manifold.

3.22

67

SUFFICIENCY CONDITIONS

3.22 SUFFICIENCY CONDITIONS

In this section we shall deduce a sufficiency theorem for optimal strategies. Before doing so, let us introduce some definitions. Suppose that D A {XI, X,, . . . , XI<} constitutes a decomposition of X , and consider the sets K

MAX-UX,

(3.134)

a=l

and where X,, X,, . . . ,X , E D. It is quite evident that Mk can be constructed so that and, in fact, M =

u M,

L< m

(3.137)

k=l

Definition 3.9. Given a decomposition D of X , we shall say that strategy pair ( p , e) is well behaved at xi E X relative to D , if ( p , e ) generates a terminating path rif represented by x: T -+ x = x ( T ) , T E [0, T,], x(0) = xi,such that (i) X ( T ) E X for all T E [0, T ~ ) ; (ii) for all T’ E [O, -rJ] there exists an interval [T,, T b ] such that T’ E [T,, T,] and X ( T ) EX, E D for all T E (T,, T,) or X ( T ) E M kG M for a11 T E (T,, T ~ ) ; (iii) Assumption 3.3 if fulfilled.

Now we are ready to prove Theorem 3.3. Strategy pair ( p * , e*), with corresponding solution x* : T + x = x * ( T ) , T E [0,T ~ * ] , x*(O) = x i , is optimal with respect to all well-behaved (at xi E X relative to the decomposition D below) strategy pairs ( p , e ) , with corresponding solutions x: T + x = x ( T ) , T E [0, -rJ], x(0) = x i , i f it is playable at all xi E X and i f there exist a decomposition

68

DIFFERENTIAL QUANTITATIVE GAMES

111

D of X and a function V * :

x V * ( x ) which (i) is continuous on X ; (ii) is of class C1 on each X , E D ; (iii) on every non-empty M , c_ M agrees with a function-say that is of class C' on a domain containing M,; and (iv) V * ( X )= 0f o r a21 x E 8 ; ---f

Vk*-

such that

+ grad V * ( X * ( T ) -) ~ ( x * ( T~) *, ( x * ( T )e*(x*(~)))] ), d~ = 0, vxi

E

x;

(b)

f o ( ~ , p * ( x u) ),

(c)

vx E X - e, f&, u , e*(x)) vx E x - e,

+ grad V * ( 4 . f ( x , p * ( x ) ,

U)

Q 0,

vuE K , ( ~ ) ;

+ grad V * ( x )- f < x ,u , e*(x)) > 0 , vuE x , ( X ) ;

where V * ( X )= V,*(X) yx E Mk. Consider a path rifthat is generated by a well-behaved strategy pair

( p , e). Since ( p , e ) is well-behaved, interval [0, -r5] contains a finite or at most a countable number of open intervals (0, T ~ ) (, T ~T,~ )., . . , ( T + ~ , = T ~ such ) that X ( T ) belongs either to an X , during any one such open interval. Thus, integral T~

I(x2)

'J

If

grad V * ( x ) . dx

E

D or to an M ,

cM

(3.138)

is composed of a finite or at most countable number of integrals; namely, by (i) and (ii) of the theorem, ilgrad V * ( x ) . d x

+ t

IT12

grad V *(x)

*

dx

+

*

..

This condition is implied by (b) and (c).

+I

+l.f

grad V * ( x ) . dx (3.139)

3.22

69

SUFFICIENCY CONDITIONS

where X(T) E nil c X(T) E

2’c

ndf

for

T E

nif

for

T

X(T) E niV--lvfc nif

(0, T J

E ( T ~ T, ~ )

for

E( T ~ - ~ ,

and where, according to (iii) of the theorem, V * ( x ) = V,*(x) if x E Mk. Then it follows from (i) and (iv) of the theorem that Z(Xi)

= -V*(Xi)

(3.140)

Consequently, integral Z(xi) is invariant with the choice of ( p , e). In view of this conclusion, together with condition (a) of the theorem and state equations (3.1), we have

Next let us consider a well-behaved strategy pair ( p * , e) generating a P-optimal path m$ represented by x: T + x = x(T), T E [0,T f ] , x(0) = x i q and form the cost difference

Jo

Now add the following null-term to the right-hand side of (3.142):

J:,*fgrad V * ( x ) . dx + V*(x*j

Since

J KP

if

(3.143)

grad V * ( x ) . d x =Jorfgrad V * ( X ( T ) )- ~ ( x ( T )~*(x(T)), , e(x(7)))

dT (3.144)

70

111

it follows that

A=

I”

[f( 0 X ( T h P*(X(T>)?

DIFFERENTIAL QUANTITATIVE GAMES

e(x(.)>>

+

(3.145) grad V*(X(T))- f ( x ( ~p*(x(~)>, ), e(x(~)))ldT Now, either T$ is a null path so that A = 0; or T$ is a non-null path so that X ( T ) E X - 19 for all T E [0, T ~ and, ) as a consequence of condition (b) of the theorem,

AGO

(3.146)

Similarly, by considering an E-optimal path and invoking (c) of the theorem, one can show that A 0 for a well-behaved ( p , e*). This concludes the proof of Theorem 3.3. It should be noted that Isaacs’ “Verification Theorem” (Ref. 16) constitutes a special case of Theorem 3.3. We shall leave it to the reader to verify that the strategies deduced in Examples 3.1-3.3 satisfy the conditions of Theorem 3.3 and hence are optimal in the sense of the theorem.

IV

Multistage Quantitative Games

4.1 PROBLEM STATEMENT

In this chapter we shall consider a multistage quantitative game, that is a quantitative game whose state is characterized by n - 1 real numbers xi, x z , . . . , x,-,, and by the stage k of the process, where k E (0, 1 , . . . ,K } It is convenient to think of the state of the game as a point

x

n =

( x l , xz, . . . , x,)

E

En

where x, E (0, 1 , . . . , K } . For x = xk A (xl, xz,. . . , xn-,, k ) we say that the state is at the kth stage. The state changes, that is, x may take on different values in En,as x, varies from 0 to K. The evolution of the state is governed by a set of difference equations; in particular, we shall suppose that the change of xk to xk+l depends on x k and some parameters, the control variables u,, uz, . . . , u, and u,, u 2 , . . . , us, which the players J , and J,, respectively, choose at each stage of play. Let u A (u,, u2, . . . , us)and u A (ul, u 2 , . . . , us). We shall assume that u E U and u E V , where U and V are fixed domains of E' and Es, respectively. With each change or transfer of the state we shall associate a real number, the cost. It is our purpose to discuss some geometric aspects of optimal play, and we shall illustrate some consequences of geometric properties by presenting a derivation of necessary conditions for a restricted class of problems. We shall assume, without loss of generality, that the multistage game which we are discussing now is the discrete version of a differential game which has been quantized, and that the stage k is the time variable. Of course, all the results will hold if k is not the time.

72

IV

2

1

0

I.:

k

MULTISTAGE QUANTITATIVE GAMES

K - l

,

FIG.4.1. The state space and the sets E,, E l , . . . ,Ex.

Since x, takes on integer values on [0, K ] , we shall consider

Ek

{x:

(4.1)

x, = k }

Let 9 be a fixed domain of E n , such that Gk 9 n Ek # o for k = 0 , 1 , ..., K . In what follows we shall assume that state x belongs to the set G 4 Uk Gk, k E (031,. - . K } . 3

4.2 STATE EQUATIONS, STRATEGIES AND TARGET

e

We shall consider the state of the game as a function of stage k ; namely on { i , i + 1, . . . ,j } , O < i < j < K , x(k)= ( x , ( k ) , . . . , X ~ - ~ ( ~ Ck). ) , If i # j we shall require that

x: k - + x k = x ( k )

x(k for k = i, i

+ 1) - x(k) = f k ( x ( k ) ,U W , v(k))

+ 1 , . . . ,j

- 1 , with u and v being functions of k ,

U: k + u = u(k),

V:

k-tu =~ ( k )

(4.2)

4.2

STATE EQUATIONS, STRATEGIES AND TARGET

0

1

2

k

73

8

K-1

K

x FIG.4.2. The sets X , and G,; 8 = X,.

where k belongs to {i, i + 1,

. . . ,j

- l}, and

We shall assume that the functions f:, Y = 1 , 2 , . . . , n - 1 , are of class C' on .9 x U x V , for k = 0, 1 , . . . , K - 1. Of course, since x,,, = k , we havef,,(x, u, u ) = 1. Let there be a fixed domain R G 9 such that X , 4R n E, # 0 for k = 0, 1 , . . . ,K. Let X A U , X , , k ~ ( 01 , . . . ,K - 1). We shall consider strategies to be functions of x, p : x - p ( x ) and e : x + e ( x ) , x E X , belonging to prescribed classes, such that

where K,(x) and.K,(x) are given subsets of (Iand V , respectively, which may depend on x. We shall be interested in transferring the state of the game from an initial state x ( i ) = xi to any one in prescribed set of states 8 2 X , by means of a strategy pair ( p , e ) , where u(k) = p ( x ( k ) ) and v(k) = e ( x ( k ) ) .

74 4.3

IV

MULTISTAGE QUANTITATIVE GAMES

DESCRlBlNG CURVES AND PATHS IN STATE SPACE

For given strategy pair ( p , e) and an initial state x(i) = xi in G, Eq. (4.2) defines a unique sequence of states

+

r = ( ~ ( i )x(i , + I), . . . , x(i + d ) ]

+

where i c/ is the largest value of stage k for which x(i d ) is defined. Of course, i cl K. if x(i) = x i does not belong to A ', or if i = K , or both, x(i 1 ) is not defined; that is, d = 0 and 1' reduces to the single point x i . We shallcall function x: k x7, = x(k) E r, k E {i, i I , . . . ,i d } the maximalsolution of Eq. (4.2) for given initial state x i and given strategy pair ( p , e). Of course, the value of d is defined by x i , p and e. For given xi and strategy pair ( p , e), there is a unique maximal solution. Replacing k by the new parameter T = k - i, the maximal solution x becomes 31: X k = x(T), T E (0,1 , . . . , d }

+ <

+

-

+

+

7--f

+

where X ( T ) = X ( T i). Then we let y ( x i , p , e ) 2 x. Since y ( x i , p , e ) is defined for all .Y' E G and for all strategy pairs (p, e), mapping y is defined. The rules of the game consist of G, the sets of strategies and y. Clearly, r is the describing curve emanating from x' and generated by ( p , e). One can readily verify that Assumption 1.1 is satisfied. From the definition of a path it follows that any subsequence r i j of r, namely .'

= {x(i), x(i

+ I), . . . , x(j)}

j

=

i

+ I,

I E (0, 1 , . . . , dJ

is a path. A point x(k), k E ( 0 , 1 , . . . , K ) , is a null path no. 4.4

COST OF TRANSFER

Next let us consider the transfer of the state from a point x ( i ) = x' to a point x(j) = x', along a path r T T 2 9generated by a strategy pair ( p , e). With each such transfer we shall associate a cost Y"(x',x'; p , e , ~ ~ 7 ) . We shall suppose that (i) the cost of transfer from xk to xk+l along a non-null path rzJ is given by fOWk), v(kN (4.3)

m,

4.5

AUGMENTED STATE SPACE A N D PATHS IN

75

En+'

where

u(k) = p ( x ( k ) ) ,

v(k) = e(x(k))

and the functions fok are of class C1 on 9 x U x V for k = 0, 1 , . . . , K - 1; (ii) the cost of transfer from x i to x j , j = i I, along r i j is given by

+

ifl-1

9 - ( x 2 , x i ;p , e, z-")

A k=L 2

hk(x(k),u(k), v(k))

(4.4)

(iii) the cost of transfer associated with a null path is zero; that is, V ( x k ,x k , p , e , no)= 0

(4.5)

for all k E (0, 1, . . . , K } and for all strategy pairs ( p , e). Hence Assumption 2.1 is satisfied. It is clear that Assumption 2.2 is also satisfied. 4.5 AUGMENTED STATE SPACE AND PATHS IN En+l

Next we introduce

Assumption 4.1.

There exists a strategy pair ( p * , e*) that is optimal on X .

By ( 4 3 , strategy pair ( p * , e*) is optimal on X * k ~ { O , l. ,. . , K } .

A X u 8 = U kX,,

Now let us consider points x = (x0, xl,. . . , x n ) = (xo,X)

-

E En+l

where 4En x {xo} is the augmented state space, and let xo: k x0 = xo(k),k E {i, i 1, . . . , j } , 0 i <j K , be a function satisfying difference equation

+

xo(k

<

+ 1) = x,(k) +fok(X(k),

<

u@), 4 k ) )

f o r k = i, i + 1 , . . . , j - 1. Equations (4.2) and (4.6) constitute a set of n equations which we shall write in vector form

where x : k

-

+

(4.6)

+ 1 scalar difference

~ ( k 1) - ~ ( k=) f k ( x ( k )u(k), , ~(k)) xk = x ( k ) on {i, i

+ 1, . . . , j } ,x k A (xo,xl,

(4.7)

. . . ,x

~ - k~) , ,

76

1V

MULTISTAGE QUANTITATIVE GAMES

x = (xo,xl,.. . , xnPl,xn), and f+,

u , 0)

2 (fAc(x, u , u),fi"x, u , u ) , . . . ,fnk-l(X,

u , u ) , 1)

The definition of paths in En+l will be the same as that in Chapter 2. Now, from the definition of cost it follows that the motion of state x along a path in E"+l is governed by Eq. (4.7), and (i) if points x ( k ) and x ( k 1) belong to a path n$(C) we have

+

+ 1 ) - x ( k ) = f"x(k), u*(k),v(k)) (4.8) (ii) if points x ( k ) and x(k + 1) belong to a path nl,m(C)we have ~ ( +k I ) - ~ ( k=) f ' ( X ( k ) , ~ ( k )v*(k)) , (4.9) x(k

FIG.4.3. Game surface and path in En+'

4.6

(iii) if points x ( k ) and x ( k

x(k where

+ 1) belong to a path n;lk(C) we have

+ 1) - x ( k ) = f k ( x ( k ) ,u*(k),v*(k))

u*(k) = p*(x(k)), 4.6

77

SOME PROPERTIES OF A GAME SURFACE

(4.10)

v*(k) = e*(x(k))

SOME PROPERTIES OF A GAME SURFACE

The definition of a game surface will be the same as that in Chapter 2. Furthermore, we shall define a surface C,(C), k E (0,1 , . . . , K } , by

Definition 4.1. where &,

C,(c)

k E, x {xo}.

A Z(c) n 6,

(4.1 1)

A surface C,(C), k E (0, 1, . . . ,K } , is a single sheeted surface in X , & X , x {xo}; that is, it is a set of points which are in one-to-one correspondence with the points of X,.

Definition 4.2. A surface C,(C), k E (0,1, . . . , K } , separates X, into two disjoint sets, namely A/C,(C) and B/C,(C), defined in A/C,(C)

(A /X (C )) n gk

B/&(C)

A (B/C(C))n 8,

The equation of C,(C) is deduced from (2.5). It is @(x)

A xo + V * ( x ) = c,

xE

x,

(4.12)

where V * ( x ) is the Value of the game for play starting at point x; and, accordingly,

C,(C) = {x = (xo, x): x EX,, Q(x) = C }

(4.13)

> C} @(x) < C }

(4.14)

A/C,(C) = (x = (xo,x):

x E X,,

B/C,(C) = {x = (xo, x): x E X , ,

aqx)

(4.15)

As the value of parameter C is varied, Eq. (4.12) defines a one-parameter family of surfaces, namely {X,(C)}.

78

MULTISTAGE QUANTITATIVE GAMES

IV

Definition 4.3. For a given C, we shall say that xk E C,(C) is an interior point of C,(C), if there exists an n-dimensional ball A(xk) with center xk such that A(xL) = Fk.

Since R is open in E”, X , 2 R n Ek is open in E, and%, & X , x {xo> is open in E, x {x,}. Hence, every point of Z,(C) is an interior point of Xk(C),k = 0 , 1,. . . ,K.

N o w consider non-null optimal path lI*(C) which starts at point

X* 2 x* x

x2 = (so, xi) €$*,

is generated by ( p * , e * ) , and is given by x*: k

+ x’

= x*(k),

k

E

{i, i

{x,},

+ 1 , . . . ,K } ,

x * = (x,*, x*) = (x,*, XI*, . . . , x,*).

We shall prove first that x * ( j ) E%*

V j E {i, i

+ 1 , . . . ,K }

(4.16)

Indeed, by the definition of an optimal path, n*(C) reaches 0 at stage K , and hence x * ( K ) E 0 c %*. If we suppose that x * ( j ) 4 X* for j < K , then p * ( x * ( j ) ) , e*(x*(j)) are not defined, and hence x * ( k ) is not defined f o r k > j . Since x * ( k ) $ B f o r k Q j , we arrive at a contradiction, and so (4.16) is established. It follows from (4.16) that n*(c)c X* (4.17) Finally, it follows from (4.17) and from Corollary 2.1 that n*(C) belongs to the game surface through x i . Of course, according t o the definition of n*(C)and of a game surface, the game surface through xi is

C(C). Now let us consider non-null paths n$(C‘)and nF(C’’)which emanate from x * ( j ) , j < K , and which are generated by strategy pairs ( p , e*) and ( p * , e), respectively. Let us represent IIG(C’) and ni,m(C”)by xp;: k

and x,,: k respectively.

+ xk-

= x,(k),

+xk =

x,(k),

+ 1 , . . . ,l } , k E { j , j + I , . . . , m}, k

E

{j,j

4.7

SETS f i p AND

79

QE

As a consequence of Lemmas 2.1 and 2.2, lIiY(Cn)has no A-point and

IIg (C') has no B-point relative to C(C).

+

+

Since x * ( j ) belongs to %* and j < K , points x p ( j 1) and x,(j 1) of paths II;Y(C") and nj2(C'),respectively, are defined. From the properties of these paths, that BIS from Lemmas 2.1 and 2.2, we deduce that

Next let us define sets Q2,(j K - 1,by Definition 4.4.

+

a,(j 1) a E ( j-I- I )

{xp(j {xdj

+

(4.18) (4.19) 1,. .. ,

for all strategy pairs ( p * , e ) } for all strategy pairs ( p , e*)}

(4.20) (4.21)

+ 1) and Q,(j + I ) , j

+ 1) + 1)

FIG.4.4. Sets fi, and SZ,.

= i, i

80

IV

MULTISTAGE QUANTITATIVE GAMES

From (4.8) and (4.9) we have Q,(j

Q,(j

+ 1) = { x ( j + 1):

x ( j + 1) = x * ( j ) + f j ( x * ( j ) ,u * ( j ) , v ( j ) ) V v ( j ) E K,(x*(j))} (4.22) + 1) = { x ( j + 1): x ( j + 1) = x * ( j )

+ f j ( x * ( j ) , u ( j > ,v * ( j ) )

V u ( j >E K t 6 ( x * ( j ) ) } (4.23)

and from (4.18) and (4.19), sincej is arbitrary, it follows that

+ Qfi;(j+ 1) n (B/Xj+l(c)) = o

i . l I 1 ( j 1) n ( A / ~ ~ + ~= ( Co) )

and for al1jr.z {i, i

+ I , . . . ,K x*(j

-

+ 1)

E

(4.24) (4.25)

I}. Of course, Q,(j

+ I ) nQ,(j + 1)

4.8 x,-DIRECTIONAL CONVEXITY

Definition 4.5. A set S in En+l is directionally convex with respect to vector ivo, or equivalently it is x,+-directionally convex, if for all xl, x2 E S and all Y E [0, I ] there exists a K , --CO < K 0, such that X1

+

Y(X2

- X’)

+

<

KWo E

s

where wo = (1,0, . . . , 0) is a unit vector in the x,-direction. S is x,--directionally convex, if it is directionally convex with respect to --M’o = (-l,O,. . . ,O) .

I

En

K S O FIG.4.5. x,+-directional convexity.

wo

4.9

REGULAR POINTS OF A SURFACE

ck(c)

81

Clearly, directional convexity is weaker than convexity, since a convex set in En+l is directionally convex with respect to ivo and with respect to any other unit vector in En+l, whereas the converse need not be true. Next we introduce Assumption 4.2. Set Q,(j+

1) is x,+-directionally convex, and set - 1.

Q P ( j + 1 ) is x,--directionally convex, f o r j = i, i + 1, . . . , K 4.9

REGULAR POINTS OF A SURFACE c,(C)

Definition 4.6. A point x’ of a surface Z;,(C) will be called a regular point of C,(C), if there exists an n-dimensional ball A(x,) in X,,with

center x k , such that every point

Gk of C,(C) in A(x,)

can be represented by

2 = Xk + E q k + O ( € )

where lime-, [Ilo(c)ll/~]= 0 and q k belongs to an (n - I)-dimensional subspace of Ek x {x,}, the tangent plane of C,(C) at x,.

+

Now let us consider point x * ( j 1) of optimal path n*(C) defined in Section 4.7. As shown in Section 4.7, path II*(C) belongs to E(C), and hence point x * ( j 1) belongs to Xj+l(C).t If grad O(xj+l) is defined in a neighborhood of point x * ( j I), where x3+l - (xo, xl,.. . , ~ ~ - ~1) and , j

+

then x * ( j Let

where

+

+

+ 1 ) is a regular point of Z,+,(C). x r ( j + 1) = x * ( j + 1) + hpx x E ( j + 1) = x * ( j + 1) + s’SC A f 3 ( X * ( j ) ,u * ( j ) , v(j)) - f ’ ( x * ( j ) , u * ( j ) , v*(j)) dEx A f’(x*( j ) , u ( j ) , v*( j ) ) - f 3 ( x * (j ) , u*( j ) , v*(j)) dPx

(4.26)

(4.27) Since x * ( j + 1 ) and x P ( j 1) belong to Q I , ( j l ) , and since Q p ( j 1) is assumed to be x,--directionally convex, then, for all v E [0,1],

+

+

+

t Since all points of C,+,(C) are interior points of X,kl(C),point x * ( j interior point of C7+l(C).

+ 1) is an

82

IV

here exists a

K,

0Q

< +a,such that

K

X*(j

MULTISTAGE QUANTITATIVE GAMES

+ 1) + Y S F X +

KWo

ap(j

E

+ 1)

+ 1) n (A/X,+l(C))= 0 ,we have + 1) + Y 6"X + A/C,+,(C)

Since Q,(j

X*(j

KWo$

which, in turn, implies that

+ 1) + v 6"x $ A/C,+,(C) for all v E [0, I ] (4.28) + 1) is an interior point of Cj+l(C),there exists > 0 such

x*(j

Since x * ( j that, for all E t ( 0 , o),

(T

x*(j

+ 1) +

E

6'x

E

Xk

Then (4.28) implies that

+ 1) +

N

L?/x,+l(c) A ( B / C ,,l(C)> u q+l(c> x*(j

N

E dPX

(4.29)

E

where BP,,,(C) J t follows from (4.29) with (4.13) and (4.15) that @(X*(j

+ 1) +

E

6"x) Q

c

(4.30)

I f grad @(xJ+l) is defined in a neighborhood point x * ( j can be rewritten as @(x*(j

+ 1)) + grad @(x*(j + 1)) -

E

+

6x'

O(E)

+ l), (4.30)

QC

where lim6+o [o(E)/E] = 0. 1) belongs to C,+,(C),andaccordingly cD(x*(j Since x * ( . j (4.31) becomes

+

grad O(x*( j Dividing by

E

E

8"x

+

O(E)

Q0

+ 1)) = C , (4.32)

which is positive, we deduce from (4.32) that

-

grad @(x*(J Then, upon letting that is

+ 1)) -

(4.31)

E

+ 1)) - 6"x +

Q0

0, we obtain

grad @(x*(j

+ 1))

O(E)/E

+ 1)) - BPx < 0

<

[ P ( X * ( j ) , u * ( j ) , v(jN - P ( x * ( j ) ,u * ( j > ,v*(j))l 0 (4.33) This relation holds for all v(j) E K,(x*(j)). grad W x * ( j

*

4.10

83

REGULAR OPTIMAL PATHS

Finally, (4.33) can be rewritten as grad @ ( x * ( j

+ 1)) - f T x * ( j ) ,u * ( j ) , v(J)) S grad @ ( x * ( j + 1)) - f Y x * ( j ) , u * ( j ) , v*(j)) V V ( j )E K,(x*(j)) (4.34)

+

By similar arguments based on the fact that Q2,(j 1) is x,+-directionally convex and that Q , ( j 1) n (5/Xj+1(C))= ia , one obtains

+

grad @ ( x * ( j

+ 1)) - f Y x * ( j ) ,Nj),v * ( j ) ) 2 grad @ ( x * ( j + 1))

*

f i ( x * ( j ) ,u * ( j ) , v * ( j ) ) V 4 j ) E K , ( x * ( j ) ) (4.35)

4.10 REGULAR OPTIMAL PATHS

Definition 4.7. An optimal path n *(C) given by x * : k xk = x*(k), k E {i, i 1 , . . . , K } , is called a regular optimal path, if x * ( k ) is a regular point of C,(C) for all k E {it i 1, . . . , K - I}.

+

---f

+

Recall that all points of Ck(C)are interior points of C,(C), and hence that x * ( k ) is an interior point of Ck(C),k = i, i 1, . . . , K. Let x*: k + x k = x*(k), k E {i, i 1, . . . , K } , represent the projection T * of II*(C) on X*. Since Xk is open in E,, x * ( k ) is an interior point of X,, k = i, i 1 , . . . ,K . Next we introduce

+

+

+

+

Assumption 4.3. At every point x*(k), k = i, i 1 , . . . ,K - 1, there exists a neighborhood A(x*(k)) in Xk of x*(k) on which p * and e* are of class CI.

Since strategy pair ( p * , e*) is optimal o n X , it is optimal on A(x*(k)), k = i , i + 1 , . . . , K - 1. From (4.4) and Definition 2.3 we have A'-1

V*(xk) = zfOy(x(v),p*(x(v)),e*(x(v))) v=k

V x k E A(x*(k))

where x: v + x v = ~ ( vis) the solution of difference equation

+

x(v 1) - x(v> = f ' ( x ( v ) , p*(x(v)), e*(x(v))) with initial condition x(k) = x k , defined on { k , k 1, . . . , K } .

+

(4.36)

84

MULTISTAGE QUANTITATIVE GAMES

IV

Then we have

+

Lemma 4.1. A t points x * ( k ) , k = i, i 1, . . . , K, of optimal path n*(C)on game surface C(C) there exists a neighborhood in E , on which grad @ ( x k )is de$ned,t where

For k = i, i 4 1, . . . , K - 1, Lemma 4.1 is a direct consequence of Assumption 4.3, together with (4.36) and with the fact that the functions A”,v = k , k I , . . . ,K - 1 , are of class C‘ on 9 x U x V . For k = K , it follows from (4.5) that Y * ( x K )= 0 for all x K E 8, and hence grad @ ( x K ) = (1, 0, . . . , 0 )

+

4.1 1

VARIATIONAL DIFFERENCE EQUATION

Again let us consider points x * ( k ) , k path n*(C),and let us introduce Assumption 4.4.

i

+ 1 , . . . ,K , of optimal

Matrix M k given by Mk&

where

= i,

[I +

I 1

dfk(xk,p*(xk),e*(xk)) dX

xk=X*(k)

afk

d f k ( x k p*(xk), , e*(xk)) d f k afkap* -- - - + ---+-dx dx ax au

+

ax

afTcae* av ax

and I is the ( n 1) x (n x 1) unit matrix, is defined and nonsingular for k = i , i + 1 , . . . , K - 1. u)]

afk a - _-

ax

. . . )n

v = 1 , 2 , ..., r v. = 0,1,. . . ,n

ax ae* ax

v,v.=O,l,

ae,*(xk)

v=l,2,

...,s

-=[T vI . = O , I , ...,n

f Recall that x k 4(x,,xl,. . . ,~ , - ~ , k ) .

4.1 1

85

VARIATIONAL DIFFERENCE EQUATION

evaluated at xk = x * ( k ) , u = p * ( x * ( k ) ) ,ZI = e * ( x * ( k ) ) ,k = i, i K - 1. Now let us consider the bound vector

+ 1 , . .. ,

qk (qok,q I k , .. . , qnk)E En+l at point 2. Let q: k q k = q(k), k E { i , i + 1, . . . , K } , be defined in the following manner: -+

1. Let qz = q ( i ) be given at the initial point x * ( i ) of IT*(C). Then

2

x(i) = x*(i)

is a point i n a neighborhood in

+q(i)+

O(E)

X iof x * ( i ) , where lime+,,

[ ~ o ( E ) ~ / E= ]

0.

2. Let 2 be transformed by the state equation (4.7) with the same strategy pair ( p * , e*) that generates n*(C). The state equation can be written x(k

+ 1) - x ( k ) = f k ( x ( k ) , p * ( x ( k ) )e,* ( x ( k ) ) )

(4.37)

f o r k = i , i + 1, . . . , K - 1 One readily deduces from the state equation that x(i

+ I) = x*(i + 1)

+r[l+ that is x(i

d f 2 ( x 2p,* ( x i ) , e*(x2)) dx xt =

X*

(i)

+ 1) = x*(i + 1) + e M i q ( i ) +

O(E)

(4.39)

Letting

we have x(i

+ 1) = x*(i + 1) + q ( i + 1) +

O(E)

(4.40)

Likewise, if x ( k ) = x*(k)

+q ( k )+

O(E)

(4.41)

86

IV

MULTISTAGE QUANTITATIVE GAMES

$+I

A

then x(k

+ 1) =

X*(k

+ 1)

Letting q(k

we have

x(k

+ I) =

Mkq(k)

+ 1) = X * ( k + 1) + q ( k + 1 ) + 4 t )

(4.42)

and so

f o r k = i , i + 1, . . . , K - 1 . Equation (4.43) is the uariafional diference equation. Note that.fnk(x, t i , 2 ) ) = I , k = i, i I , . . . ,K - I , and hence

+

rll,(k

It follows that

+ 1) - q,(k)

(4.44)

=0

+ 1) = . . . = q A K )

(4.45)

q,(4 = qll(i

4.12 LINEAR TRANSFORMATION AND INVERSE

TRANSFORMATION

For given q(i) = q' we have q(k Let

+ 1) = M"(k)

= M'(MLp'q(k

q(k

*

. . M"(9

1.

Since matrices M h ,k = i, i matrices d h ,k = i, i 1,. linear transformarion

+

= MkMk-'

- M h M k - 1 . . . MZ

&h

fork = i , i + 1,... , K -

- 1))

+ 1 , . . . , K - 1 , are assumed nonsingular, . . ,K

+ 1) = e d k q ( i ) ,

- 1 , are nonsingular.

k = i, i

+ 1 , . . . ,K - 1

Hence the (4.46)

4.13

SOME PROPERTIES OF LINEAR TRANSFORMATIONS d kAND

gk

87

is nonsingular; that is, the inverse of dkexists, and we have q(i) = (d")-'q(k If q ( K ) = 71" is given, then

+ 1)

q(i) = ( d - - l q ( K )

and

q(k) = dk-'q( j ) = dk-'(LpylIC-'

where Let

&-l

& I.

.@

&k-1

)-lq(K)

(dK-')-l

where @ k (dK--l)-l. Then we have the inverse linear transformation q ( k ) = @q(K)

f o r k = i, i

(4.47)

+ 1 , . . . ,K - 1.

4.13 SOME PROPERTIES OF LINEAR TRANSFORMATIONS d k AND dc

<

Now let P ( x * ( i ) ) be a given y-dimensional hyperplane, 1 y Q n, containing point x * ( i ) of n*(C). We shall be interested in the transform of plane P ( x * ( i ) )due to linear transformation d L - - l , namely, P ( x * ( k ) )= d L - l P ( x * ( i ) )

A {x*(k) + d k - l q t :

x*(i)

+ 7%EP(X*(i))}

Lemma 4.2. The transform P(x*(k)), due to linear transformation dk-l, of plane P ( x * ( i ) ) containing point x * ( i ) of optimal path n *(C) has these properties: (a) P ( x * ( k ) ) is defined f o r k = i , i I , . . . ,K ; (b) P ( x * ( k ) ) is a plane of the same dimension as P(x*(i)).

+

Since P ( x * ( i ) )is a y-dimensional plane, there exists a basis in P ( x * ( i ) ) , that is y linearly independent vectors e,(i), v = 1 , 2 , . . . , y , such that any nonzero vector 7%in P ( x * ( i ) )is of the form 71' = q(i) =

Y

2 c,e,(i) 'I

v-1

where the c, are constants, not all of which are zero.

88

MULTISTAGE QUANTITATIVE GAMES

1V

Let e,(i), v = I , 2, . . . , y , be transformed along ll*(C) by equation (4.46); then v = 1, 2,. . . ,y e,(k) = d k - I e v ( i ) Since the vectors ev(i),Y = 1 , 2 , . . . , y , are linearly independent and is nonsingular, the vectors e,(k), v = I , 2, . . . ,y, are also linearly independent. Any vector qk of P(x *(k ))is o f the form

.&-I

is linear, and since dk--l q k = 2 c,dk-ley( i) = 2 c,e,(k) Y

"= 1

V=l

+

It follows that P(x *(k )), k = i, i 1 , . . . ,K , is defined and its dimension is y , and so Lemma 4.2 is established. By similar arguments, using the properties of inverse transformation 9Yk,one can prove Lemma 4.3. The trarisfbrm P(x*(k)), due to linear transformation g h ,of a plane P(x *(K)) containing point x * ( K ) of optimal path n*(C) has these properties: (a) P ( x* ( k ))is dejned f o r k = i, i 1 , . . . ,K ; ( b ) P ( x* ( k ))is a plane of the same dimension as P ( x * ( K ) ) .

+

4.14 TRANSFORMATION OF A TANGENT PLANE

Since, according to Lemma 4.1, grad @ ( x h ) is defined at all points 1,. . . K - 1, n*(C) is a regular optimal path on X(C). Consider a neighborhood in X,(C) of x*(i),T namely, x * ( k ) , k = i. i

+

.

h ( x * ( i ) )2 {xz: where x ( i ) = x *(i)

and

x1 =

x(i) E C,(C)}

+ €72 +

O(€)

1 . limc+o [llo(.~ll/~l = 0. E P,. ( x * ( i ) ) , the tangent plane of Xz(C) at x*(i), which is an 2. ( n - 1)-dimensional plane in E , x {so}. -I

t

Recall that x * ( i ) is an interior point of XL(C).

4.14

89

TRANSFORMATION OF A TANGENT PLANE

We shall be interested in a transformation of A(x*(i)) from stage i to stage k by means of state equation (4.37); that is, with a point xi = (xo,xi) of A ( x * ( i ) ) ,we shall associate optimal path n*(C) which emanates from xi and which is generated by the same strategy pair ( p * , e*) that generates II*(C) emanating from x * ( i ) . Then the transform of A(x*(i)) is A

A ( x * ( k ) ) A { x k = x ( k ) : ~ ( kE )fI*(C)

Vxi E A(x*(i)))

From (4.42) we have

A(x*(k))= {x" = ~ ( k ) :~ ( k=) x * ( k )

+ cq(k) + O(<)]

where q is the solution of variational equation (4.43) with initial condition q(i) = $; that is, q(k) = d k - l q i

\

1

I

;

1 x*( i) FIG.4.6. Transformation of tangent plane.

90

MULTISTAGE QUANTITATIVE GAMES

IV

*

Since condition (4.17) applies to all optimal paths, n*(C) c .%* for all xz E A ( x * ( i ) ) . I t follows from Corollary 2.1 that

x(k) = x*(k) where Hence,

+ q ( k )+

C,(C)

4 6 )

[llo(~)Il/~] =0

cD(x*(k)

+ q ( k )+

4 E ) )

=

c

(4.48)

Since x * ( k ) belongs to C,(C), and accordingly @ ( x * ( k ) )= C , (4.48) can be rewritten as

+

+ o(llq(k) + o(~)ll)= 0

grad @ , ( x * ( k ) ) ( ~ q ( k ) o ( E ) )

(4.49)

Equation (4.49) can be written more simply as E

-

grad @ ( x * ( k ) ) q ( k )

where limc+o [ o ( E ) / E ] = 0 Upon dividing by E and allowing

E

-+

+

O(E)

=0

0 we obtain

grad @ ( x * ( k ) ) q ( k ) = 0

(4.50)

Furthermore, (4.45) and the fact that q2 belongs to E , X {x0} imply that q ( k ) belongs to E, x {xo}. Since y ~ 'E P,- ( x * ( i ) )and since d k - l is a linear nonsingular transformation, conditioL'(4.50) leads to Lemma 4.4. If Assumptions 4.3 and 4.4 are satisfied, then the transform, due to linear transformation dh-l, of the tangent plune of C , ( C ) at x * ( i ) is the tangentplane P r , ( x * ( k ) ) of CJC) at x * ( k ) ; that is,

P,- k ( x * ( k ) ) = d L - l P x , ( x * ( i ) ) f o r 4.15

k = i, i

+ 1,.

. . ,K

ADJOINT EQUATIONS

+

1, . Next let us introduce A(k) E E"f1, k = i, i the equation adjoint to variational equation (4.43),

A(k

e*(xk)) + 1) - h(k) = - [dfk(x*, p*(xk), dx

f o r k = i , i + l , . . . , K - 1.

~

~

, K , satisfying

Xk=X*(k)

(4.51)

4.15

91

ADJOINT EQUATIONS

This equation can be rewritten as

[ + df”(x”,

p*(x”), e*(x”))

A(k) = I

dX

f o r k = i , i + l ,...,K - 1 . Since matrices

are assumed to be nonsingular, so are matrices

[I

+

df‘“(x”,p*(x”), e*(x”>) dx

Thus their inverses exist; that is

+

+

f o r k = i, i 1, . . . , K - 1. Hence, vectors A(k), k = i, i I , . . . ,K , are defined for any given A(i) or A ( K ) . It follows from Eq. (4.52) and variational equation (4.43) that

+ 1) = . . . = ~ A ~ ( K ) ~ (4.53) ~ ( K )

+

n

71

n

x A v ( i ) q v ( i )= zhy(i l)qv(i v-0

”=O

or equivalently

-

A(i) q(i) = A(i

”.=O

+ I ) - q(i + I ) = . . - = A(K) - q(K)

(4.54)

Since x,, and x , do not occur in the argument of f”(x”,p*(x”)), e*(x”)), k = 0 , I , . . . ,K - 1, Eq. (4.51) implies that

Ao(k for k = i, i

+ 1) - A,(k) = 0 ,

+ I , . . . ,K - 1.

+ 1) - A,(k)

=0

It follows that

Ao(i) = Ao(i A,(i) = A,(i

A,(k

+ 1) = . . . = Ao(K) + 1) = . . . = A,(K)

(4.55) (4.56)

From (4.54) we can draw the conclusion that, if initial vectors ilz= A(i) and qa = q(i) are chosen such that J.f.qi=o

with

llAzII II qill Z 0

92

IV

MULTISTAGE QUANTITATIVE GAMES

That is, if 3.' is perpendicular to rl', then

A(/<)- q ( k ) = 0 k=i,i+ I, ...,K (4.57) Coiisequently A(/<) is perpendicular to q ( k ) at all points of rl *(C).t Also. if P ( x * ( i ) ) is an ( n - I)-dimensional plane containing point x * ( i ) of rI*(C), and if R1 = A(;) is perpendicular to that plane, then A(k) is perpendicular to the transform P ( x * ( k ) ) = -n'"-'P(x*(i))

+

(4.58)

k = i, i 1, . . . , K . of P ( x * ( i ) ) due to linear transformation db--], Finally, if 3.' = A(;) is chosen perpendicular to the tangent plane P,.( x * ( i ) ) of X t ( C )at x * ( i ) , then it follows from (4.57) and Lemma 4.4 &at A(/<) is perpendicular to the tangent plane P X k ( x " ( k ) ) of x k ( C ) at x * ( k ) , k = i, i I , . . . , K.

+

GRADIENT AND ADJOINT VECTOR

4.16

The s,,-component and the x,-component of A(k) are constants. Thus, if we choose A,(;) = 1 and A,(i) = 0, we have A,(k) = 1 and A,(k) = 0 for k = i, i + 1 , . . . , K. Furthermore, if we choose A(i) perpendicular to P , . ( x * ( i ) ) . then, as pointed out above, A(k) is perpendicular to P,. ( x * ( k ) ) for k = i, i I , . . . , K. Clearly, this choice of A(;) implies tcJt A(;) = grad (D(x') at point x i = x * ( i ) . Since both grad ( D ( x k )at point xli = x * ( / c ) and A(k) ( i ) belong to E, x {x0$ ( i i ) have zeroth component equal to 1 , ( i i i ) are perpendicular to P X k ( x * ( k ) ) for /i = i , i I , . . . , K , it follows that

+

-I

+

for k = i , i A(k

+ 1)

A(/<)= grad (l)(xk) at point XI;= x * ( k ) + 1. . . . , K . And so we deduce from (4.34) that *

f " X * ( k ) , U * ( k ) , v(k))

< A(k + 1)

*

f " ( x * ( k ) , u * ( k ) , v*(k))

Vv(k) t K , ( x * ( k ) )

for k = i, i

A(k

(4.59)

(4.60)

+ 1, . . . , K - 1 , and from (4.35) that

+ 1) - f / " x * ( k ) , u ( k ) , v * ( k ) ) 2 A(k + 1) Vu(k)

-i-Of course, jih(k)l;llr)(h),!# 0.

E K,,(x*(k))

*

f " ( x * ( k ) , U*(k), v*(k))

(4.61)

4.18

A M[N-MAX PRINCIPLE

fork

=

i, i

+ 1,. .., K -

93

1, where

x*(k) A (xo*(k),x*(k)) E II *(C)

and

u*(k) = p*(x*(k)),

v * ( k ) = e*(x*(k))

4.17 TRANSVERSALITY CONDITION

At the terminal point x * ( K ) of optimal path n*(C)

-

Vq(K)E P X K ( x * ( K ) )

A(K) q ( K )= 0

@ti s

The intersection of game surface C(C) with

o n C ( C ) = {x:

Q(X) =

c, .x E el

Because of (4.9, the condition x E 8 implies that @(x)

Hence,

0 n C(c)

s,

+ V * ( x )=

= {x:

,yo =

.X"

c, x E Oj

Consequently, 0 n C(C) may be deduced from 8 by translation parallel to the x,-axis; and since 0 A XI< R n EIi, P x K ( x * ( K )is) an (n - 1)dimensional plane perpendicular to the x,-axis. It follows that (4.62) A,(K) = . . . = Anpl(K) = A,(K) = 0 at the terminal point of ll *(C). This is the transversality condition. 4.18 A MIN-MAX PRINCIPLE

Upon introducing the Hamiltonian Xk+'(A(k

+ I), x ( k ) ,~ ( k )v(k)) , A A(k + 1) - f " ( x ( k )u(k), , v(k))

+ 1 , . . . , K - I , the results of the preceding sections can be

for k = i , i embodied in

Theorem 4.1.i Let n*(C) be an optimal path generated by strategies p * , e* and represented by x* ; k xk = x * ( k ) ,and let rr* be its projection ---f

t

A

0 = 0 x {xo}. and sufficient conditions, see Reference 33.

3 For necessary

94

IV

MULTISTAGE QUANTITATIVE GAMES

+

on X * represented by x*: k + xk = x * ( k ) , k E ( i , i 1,. . . ,K ) . If Assumptions 4.1-4.4 are satisjed, then there exist nonzero vectors A(k), k = i, i 1 , . . . , K which sarisf,, aLljoint equation (4.51), such that

+

(a)

min

+ l), x * ( k ) , u , e*(x*(k))) - max X k + ' ( A ( k + l), x * ( k ) , p*(x*(k)), v ) = c%"k+l(A(k+ l ) , x*(k), p*(x*(k)),e*(x*(k)))

u E l i l r ( x * ( k )1

Xk+'(A(k

uEli7(x*(k))

f o r k = i , i + 1, (b)

(c) (dl

...,K - 1 A,(i) = A,(i A,(i) = A,(i

+ 1) = . . . = A,,(K) > 0 + I) = = A,(K) = 0 * * *

Al(K) = . . . = A,-,(K) = A,(K) = 0

4.19 CONSTRAINTS

Now let us consider constraint sets K?L(xk) and Ku(xk),respectively, given by a = I , 2 , . . . ,I y$(xk, u ) 0 (4.63) a = I , 2 , . . . , rn y$(xk, v) 0

< <

where functions y," and y$ are of class C1 on G, x U and G, x V , respectively. Suppose that V $ ( , K ~ , ~ * ( X ~= ))0

a = I , 2, . . . ,I' Q 1

y / ( x k ,e*(xk)) = 0

CI

=

I , 2 , . . . ,m' Q rn

at x" = x*(k) We shall suppose also that if I > Y and m > s, at most r of the q%(xk,u) and s of the y / ; ( x k , v) vanish at any point of G, x U and G, x V , respectively, and furthermore that the matrices

. . , Fn 1,2, . . . , s

= 1,2,.

o=

have maximum rank at u = p*( xk) ,v = e*(xk),xk = x * ( k ) .

4.19

95

CONSTRAINTS

As a consequence of condition (a) of Theorem 4.1, the Lagrange multiplier rule gives A"(k 1)-a f k p T ( kaqk )y=0 A"(k

+ au + au + 1 ) - + v'yk)- aVk = 0 av av afk

(4.64)

where AT, pT,and vT are row vectors

A (A, A,. . . A,) pT A (plp 2 . . . pl. 0 . . . 0)

'A

v T - (vl v 2 . . . v,.

0 . . . 0)

where pi 0 where v, Q 0

respectively, and

av

b = 1 , 2 , . . . ,s evaluated at u = p * ( x k ) ,v = e*(xk), x k = x*(k). Since q,"(x",p*(xk)), a = 1 , 2 , . . . ,l', and y,k(xk,e*(xk)), a = 1 , 2, . . . , m', have an extremum for x k = x * ( k ) , we may write

(4.65) where a = ~ 2 , . .. , i

ax

b

= 0, I , .

. . ,n

a = 1 , 2 , . . . ,m ax 0 = 0 , 1 , . . . ,n evaluated at u = p * ( x k ) ,u = e*(xk), xk = x*(k) From (4.64) and (4.65) we obtain

AT(k AT(k

ap * = pT(k)avk + 1) au ax ax afk

ae* aYk + 1) afk - - = vT(k) av ax ax

96

1V

MULTISTAGE QUANTITATIVE GAMES

or equivalently

(4.66)

where p and v are column vectors

\I=

Finally, by (4.66) adjoint equation (4.51) may be written as A (k

+ I)

-

A(/<)

= -(

g p k

+ I)

-

(yr (;77'(k) - p(k) -

-

k = i , i + l , ..., K - 1

(4.67)

If the constraints are independent of state x. the adjoint equation reduces to A(k+ I)-A(k)=-

A(k+l)

k = i , i + l , ...,K - 1

(4.68) 4.20

EXAMPLE 4.1

Let us consider a multistage same governed by

x,(k x,(k x,(k

+ 1) - x,(k) = x,@)

+ 1) - x,(k) = u(k) + v(k) + I ) - x3(k) = 1

(4.69)

4.20

EXAMPLE

97

4.1

Let the cost be given by

xO(k

+ I) - xO(k) = x,(k)x,(k) + ~ ' ( k )- ~ ' ( k )

(4.70)

and the control constraints by

K , = K,

= El

A path in augmented state-space E 4 , generated by a strategy pair that satisfies the necessary conditions for optimality, will be represented by x * : k + xk = x * ( k ) , k E {i, i 1 , . . . ,K } , where x * A (xo*, xl*,xz*, x,*), and the corresponding controls will be denoted by u*: k + u = u * ( k ) , v*: k 1' = v*(k). I n view of condition (c) of Theorem 4.1, the ,#-function is

+

---f

*Ic'+l

= x,(k)x,(k)

+ U2(k)- v2(k) + A,(k + I)x,(k)

+ Az(k + l)(u(k) + v(k))

(4.71)

and the adjoint equations for Al and A, are A,(k

A,(k

+ 1) - A l p ) = -x,*(k) + 1) - A,@) = -x,*(k)

- Al(k

+ 1)

(4.72)

f o r k = i , i + 1 , . . . , K - 1. From condition (a) of Theorem 4.1 we obtain

2u*(k) -2v*(k) which implies that

+ A,(k + 1) = 0 + Az(k + 1) = 0

u*(k)

=

-v*(k)

(4.73) (4.74)

Condition (d) of Theorem 4.1 yields

Al(K) = A,(K)

=0

(4.75)

Let

X0*(K) = x? = c X1*(K) = xpX,*(K) = x p Now we deduce from (4.69)-(4.75) that at stage K - 1 X'*(K - 1) = X,*(K) = xfiX1*(K - I ) = X1*(K) - X'*(K - I ) X0*(K - I ) = c - XZK(XIK - x?)

(4.76)

= XITC - X ' K

98

IV

MULTISTAGE QUANTITATIVE GAMES

and

A,(K - 1) = x Z K A,(K - 1 ) = x? - x z K

at stage K - 2

x,*(K - 2) = x2*(K - 1 ) = x Z K x,*(K - 2 ) = x,*(K - 1) - x Z K = x I K - 2xzK X,*(K - 2)

=

=

and

-x?(x1K

- 2x,")

c - XF(2X,K

+

A,(K - 2) = x , ~ xZk' = 2x2" AZ(K - 2 ) = xll' - 2xZK x,"

+

+ xo*(K - 1)

- 3x2K)

+ xI1' - x Z K = 2 ( x l K - x Z K )

More generally, one can prove readily by recursive arguments that at stage K

- m (0 Q m Q K - i) X~*(K - m) = xpK x,*(K - m ) = x l K - mxzK xo*(K - m ) = C - m x z K { x p- t ( m

and

+ 1)xzK}

(4.77)

A,(K - m ) = mxZK A2(K - m ) = mixl" - x?)

(4.78)

Hence, from (4.73) and the second of Eqs. (4.78), we have

u*(K - m ) = - v * ( K - m ) = -[$(m - l)](xlK - x Z K ) (4.79) m = 1,2,. . .,K - i The Value of the game for play starting at stage k is

V * ( x * ( k ) )= x o * ( K ) - xo*(k) = ( K - ~ ) X , " { X , ~ - $ ( K - k

+1

) ~ ~ ~ )

Now let

~,*(k= ) x;,

X1*(k) =,:x

x , * ( k ) = xZk

From (4.77) we deduce

x t = xZ1',

X :

=

xlK - ( K - k)XzK

and hence, the Value of the game for play starting at stage k may be written as (4.80) V*(x*(k))= ( K - k ) x , ' { ~ , ~ g(K - k - l ) ~ , " }

+

4.20

EXAMPLE

4.1

99

It follows that the equation of surface C,(C) is

+ ( K - k)X,”{X,’ + &(K- k - 1)xZk}= C

(4.8 1) k = 0 , 1 , 2 ,...,K Surface C’(C) is shown on Figs. 4.7-4.9 for k f. K , K - 1, k = K - 1, and k = K , respectively, in xo-xl-xz space. The sets Q2,,(K- m) and Q E ( K - m ) , m = 0 , 1 , . . . ,K - i - 1 are defined by xO’

QI,(K - rn) = { x ( K - m ) : x ( K - rn) = x * ( K - m - 1) fK-m-l(x*(K - m - I),u*(K - rn - l),v(K - m - 1)) VV(K - rn - 1) E K , } Q E ( K - rn) = { x ( K - m): x ( K - rn) = x*(K - m - 1) +fKprn-l(x*(K- m - l ) , u ( K - m - l),v*(K - rn - 1)) Vu(K - m - 1) E Ku},

+

FIG.4.7. Example 4.1. Surface &(C), k # K , K - 1 .

100

IV

MULTISTAGE QUANTITATIVE GAMES

---__

A1

FIG.4.8. Example 4.1. Surface x K - - I ( C ) .

For instance, let us consider Q,,(K - m). In view of Eqs. (4.69) and (4.70), Q,,(K - m) is the set of points

x ( K - m ) = (x,(K - m ) , x,(K -m), x2(K - m ) , K - m) such that

+

x0(K - m) = xo*(K - m - 1) xl*(K - m - l)x,*(K - m - 1) + u * ~ ( K- m - 1) - v2(K - m - 1) (4.82)

(4.83) + xz*(K - m - I) x,(K - m) = x,*(K - m - I ) + u*(K - m - 1) + v(K - m - 1)

x,(K - m) = x,*(K - in - 1)

(4.84)

4.20

EXAMPLE

101

4.1

FIG.4.9. Example 4.1. Surface c , ( C ) .X,(C) is a plane perpendicular to the x, axis.

Equations (4.82)-(4.84) together with (4.77) and (4.79) imply that

+ l)Xz"(X1" - + 2)xZ") + [ X , K - (rn + I)X,"]X," + :m"xl" - XzK)Z - v2(K - rn - 1)

x,(K - m ) = C - ( m

~ ( W I

x,(K - m ) = x,*(K - m ) = x l K - mxzzi - $ m ( x l K - x;') v(K - m - 1) xZ(K - HI) = or equivalently

+

(4.85) (4.86) (4.87)

xO(K - m) = xO*(K - m)

+ [ i m ( x F - x Z K )- V(K - m - l ) ] [ i m ( x l K - x Z K )+ V(K - m - I)] (4.88)

x,(K - m) = X,*(K - m) xZ(K - m ) = x,*(K - m ) - im(xI") :x

+ v ( K - W I - 1)

(4.89) (4.90)

102

IV

MULTISTAGE QUANTITATIVE GAMES

\

\ FIG.4.10. Example 4.1. with the xo-x1 plane.

Intersections of the sets Cl,,(K - rn) and R,(K - m)

From (4.88)-(4.90), by eliminating v(K - m - I ) between (4.88) and (4.90), we obtain the equations defining Q,,(K - m),namely x o ( K - m ) - xo*(K - m ) = [x,*(K

x

- m ) - X, ( K

- m ) - m(xl" - xZK)I

- m ) - x,*(K - m)l x l ( K - m) = x,*(K - rn)

[X,(K

(4.91) (4.92)

Similarly, one can see that Qp2(K - m ) is defined by x0(K - WZ)- x ~ ( K m ) = [ x p ( K- m) - x,*(K - m ) - m(xlK - xZK)]

x [x,(K - m ) - X,*(K - m ) ] xl(K - m) = x,*(K - m)

(4.93) (4.94)

These sets are parabolic cylinders parallel to the x,-axis. Their intersections with the xo-x1plane are sketched on Fig. 4.10. It is readily seen that Assumption 4.2 is satisfied.

V

Some Geometric Aspects of Qualitative Games

5.1 MAP OF A GAME

Given any strategy rE E R E , let us associate with it a set of points A(r,) in G ; namely,

2 {x':

A(r,)

3r,

ER

(rI,, rE) E Y I , ( x ' ) }

,,

Similarly, with every strategy r p E R,,,let us associate a set B ( r p ) ; namely, B ( r E ) {x': 3r, E R E , (rI,, rpJ E Y E ( x ' ) } Then let us define the following sets: A

Clearly, we have

n

A(rE),

B

TEERE

A n

r I.€ Rp

B(rI,)

A = (x2: V r E E RE 3, E R,, (rl,, r E ) E Y - , , ( x ' ) } B = {xz: V r , E R , 3r, E R E , (r,,, Y E ) E Y e ( x ' ) }

It follows that? comp A = {x': comp B = {x':

3r, 3r,,

R, E R,,

E

Vr, Vr,

R,,, (r,,, r R ) 6F p ( x a ) } E R E , (r,,, r E ) $ Y E ( x Z ) }

E

Definition 5.1. Sets A , B, comp A , comp B, and the intersections A n B and comp A n comp B , constitute the map of the game.

7 Hereafter comp S will mean the complement of set S in G.

Also, the boundary aS

of a set S will be defined by aS 2 s n cornp S, where S and comp S a r e the closures of S and comp S, respectively, in the topology induced by E" on G (see Appendix). Note that 8s c G.

104

V

SOME GEOMETRIC ASPECTS OF QUALITATIVE GAMES

5.2 OPTIMAL STRATEGIES IN A QUALITATIVE GAME, SETS OF THE GAME

Definition 5.2. In a qualitative game, we shall say that r,,*(xz) E R , is an optimal strategy for player J , at point xz, if ( r p * ( x i ) ,r,) is strongly playable for J1, at that point, no matter what rB;ER,; that is ( r l J * ( s L r) K , ) E FI,*(xz)

Vr,

E

RE

(5.1)

Likewise we shall say that rE*(xz)E R , is an optimal strategy for player JI, at point x', if (r,,, r,*(Y)) is strongly playable for J , at that point, no matter what r p E R p ; that is

( r l , , rE*(x2))E F E * ( x z )

FIG.5.1.

Vr,,

E

R,,

(5.2)

Map of a game.

Recall that strong playability for J I , (or J E ) at xt implies weak playability for J p (or J R ) at that point. Moreover, the concepts of strongly and weakly playable pair ( r p , rt;) for J E (or J,) at xzcoincide if 8, = 0 (or 19,~= a),or if n 8, = 0'and (rl,, r E ) generates a unique describing curve from s'. Returning to the definition of the map of a game and to the definition of a playable strategy pair, we see that (5.1) implies that .Y' E

A n comp B

(5.3)

5.2

105

OPTIMAL STRATEGIES IN A QUALITATIVE GAME

This follows from the fact that ( r p * ( x Z )Y, E ) E T P * ( X ' ) and hence (5.1) implies that x' On the other hand, since

3

(rP*(xz),rp;) $ r E ( x z )

E comp B.

(rI,*(xz),r E ) E F p * ( x z )3 ( r I , * ( x Z )r ,E ) E 9-,>(xZ) and since (rI,*(x'),r,) E F,,(x') V r , E R , implies that V r , E R, 3 r , E R , [namely, r,, = r p * ( x Z ) ]such that (r,, r E ) E F I , ( x ' ) , it follows that xi E A . By similar arguments one can prove that (5.2) implies that x z E B n comp A Definition 5.3. S, S,

A (xi:

(5.4)

3rp*(xi))

4 (xi: 3r,*(xi))

Sets S , and S , will be called the sets of the game. Let us note first that (8, n 8,) n ( S I , u S,) = o. If 8, n 8, = o and e p # o then 8, E S , ; and if 8,, n B E = o and 8, # o then OE SE. If target O,, (or 0,) is empty, the corresponding sets S , (or S,) may be empty or not. Furthermore, it follows from the remarks of Sec. 5.2 that S, G A

comp B

and

S E s BncompA Hence,

s,, n SE = 0

Definition 5.4. Let there be given a fixed set X , pp* E R , is optimal on X,, if Vxi. E X ,

Vr,

E

RE,

s S,.

(pI,*, rBJ E F P * ( x ' )

Likewise, let there be given a fixed set X , G S,. pE* E RE is optimal on X,, i f V x i E X E Vr,, E R,,

We shall say that (5.8)

We shall say that

(r,, pR*) E F e * ( x z )

(5.9)

106

V

SOME GEOMETRIC ASPECTS OF QUALITATIVE GAMES

In other words a stratesy for player J p that is optimal on X, is optimal at all points x2 E X I , ; that is, 3rz,*(xz) such that rI,*(xz)= pp* for all x' E X , . Likewise, a strategy for player J , that is optimal on X E is optimal at all points x2E X,; that is, 3rBT*(x2)such that rE*(xi) = pE* for all x7 E Xp:. 5.3 SOME PROPERTIES OF SETS OF THE GAME

Definition 5.5. A 9-path is a path that emanates from x' E S , and is generated by ( r p * ( Y ) , rl+:), r E E R,<;similarly, an 8-path is a path that

FIG.5.2. Sets of the game, B and &-paths.

emanates from i L E S , and is generated by (r,,, r E * ( 2 ) ) , r p E R,. 9 and 8-paths that reach targets O,, and OP;, respectively, will be called 9-optimal and €-optimal paths, respectively. Now, let us prove Lemma 5.1. No point of a 9'-path belongs to comp S , ; Lemma 5.2. N o point of an 8-path belongs to comp S E .

Consider path T;, emanating from xi E S p , generated by strategy pair ( r p * ( x z ) , r,), and ending at point xs; and suppose that there exists a point X J in T*$n comp S p .

5.4

107

SURFACE OF THE GAME

Let #& c r$ be the path that emanates from x i , ends at xi, and is given by the same function that defines T$ between xi and x3. It follows from the definition of SI, that comp S ,

= {x:

Vr,,

3r,,

(r,,, r E ) r$ F,*(x)}

and hence there exists P, E R E such that (rz,*(xz), PE) r$ F E . * ( x i ) Then it follows from the definition of a strongly playable pair for J , a t xj, that either l?;(xj, r P * ( x z ) ,i E )n 8,, = 0 (i) 3 8 E y ( x j , r,,*(xi), PB2), or I';(xf, r I J * ( x z )P,,) n 8,, # lir (ii) Vx E y ( x j , r,*(x"), i E ) , and there exists 2 E y ( x i , r J , * ( x i ) , i E )that generates from xi a terminating path whose end state belongs to 8,. In cases (i) and (ii), it follows from Assumption 1.1 that there exist p, E R , and a function E y ( x i , r,,*(xi), p,) such that

r&xi,rf,*(xi), p E ) = r$

u 17;(xf, rp*(xi),)F,

Consequently, in Case (i)

r@, rI,*(xi), p,)

n8 , =

o

and in Case (ii) generates from xi a terminating path whose end state belongs to 8,. Both cases contradict the fact that rr*(xi) is an optimal strategy for Jp at point xi. Hence Lemma 5.1 is established. Lemma 5.2 can be proved by invoking similar arguments. 5.4

SURFACE OF THE GAME

Definition 5.6. In a qualitative game, the set 8, defined by

E 4 as, n as, will be termed the surface of the game. Recall that

as, 4 s, n comp s,, as, A S,. n comp S ,

(5.10)

108

SOME GEOMETRIC ASPECTS OF QUALITATIVE GAMES

V

where comp S , S = S , or S = S E , means the complement of S in G. Of course, E may be empty. One can readily establish Lemma 5.3. E is the intersection of the closures of Sr and SE; namely, -

-

E = S , n S,

(5.1 1)

FIG.5.3. Game surface E.

From (5.10) we have -

E = ( s ,n comp sJ,)n (3, n cornp s,) = S,,

n S, n comp s,>n comp S ,

Since S , n S, = 0 ,it follows that Sl,

Hence,

S, Sp -

c comp S , c compS, c

cornpS,

S , G compSP

and accordingly,

(5.12)

5.4

109

SURFACE OF THE GAME

Finally, by substituting in (5.12), we obtain (5.1 1) which proves Lemma 5.3. Another property of 5 is embodied in Lemma 5.4. namely,

B belongs to the intersection of the boundaries of A and B;

5

s aA n aB

(5.13)

From (5.5) and (5.6) we have S, S,

and hence,

-

S, -

S,

Then

c A, s B, -

sA, s B,

S, G comp B

S,

c

comp A

F, c comp B -

S, _c comp A

(5.14) (5.15) Relations (5.14) and (5.15), together with (5.11), establish Lemma 5.4. We have also -

Lemma 5.5. I ~ S ,= comp S , and S, = comp s,, tlien

5 = a A = aB

(5.16)

From (5.5) and (5.6) one can readily deduce that comp A comp B Hence,

s comp S,, s comp S,,

comp A c comp S, ~~

_______ comp B E comp

S,,

B c comp S, A s comp S ,

B

E

comp S,

s comp S ,

It follows that aA Li

2 n comp A c comp S,

n comp S,

(5.17)

aB 2

Z n comp B s comp S,

n comp S ,

(5.18)

110

V

SOME GEOMETRIC ASPECTS OF QUALITATIVE GAMES

By substituting (5.17) and (5.18) in (5.14) and (5.15), respectively, we obtain S I , n S , c 3.4 c comp S,, n comp S, -

-

S I >n S ,

s,,

c aB G

____

comp S , n comp S ,

-

Finally, if = comp S, and S , = comp S,, we have aA = aB = comp S I , n comp S , and so =

S,, n 3,

Of course, if S , = comp S,, satisfied and E = aA = aB.

-

gI, n S,

=

= aA = aB.

the assumptions of Lemma 5.5 are

5.5 A SIMILARITY BETWEEN QUALITATIVE AND QUANTITATIVE GAMES

It is readily seen that Lemmas 5.1 and 5.2 exhibit strong similarities to Corollaries 2.2 and 2.3, respectively, provided we invoke the correspondence between S, and B/X(C)

s,

and

A/X(C)

Also, in differential qualitative games, under certain conditions, properties of E will appear to be similar to those of a C-surface. However, there are differences between these corresponding sets. Some of them are listed below:

(i) E may be empty, whereas C(C) cannot be empty. (ii) E is unique, but there is a one-parameter family of C ( C ) . (iii) In one target qualitative games,? S, or S, may be empty, whereas B/C(C)and A/C(C)are never empty. (iv) S,, and S, are unique, whereas there is a one-parameter family of B/X(C) and A/C(C). These differences play a role in some of the derivations of the next chapter. Many of the results which have been established for regular points of a C-surface will be seen to apply for regular points of E,provided we replace C(C) by E,A / C ( C ) by S E , and B/C(C)by S,.

7 Either 0,

=

a

or 0, = 0.

5.6

PROBLEM STATEMENT

5.6

PROBLEM STATEMENT

111

Now we are ready to state the problem of qualitative games. We shall consider separately two target and one target games, that is Case (a) and Case (b). Let u s first list the different situations that players J, and J E can meet in these two cases. Case (a) el, # ia and 6, # 0 a.1 If xz E S , and if J,, plays optimally, then J,, is sure to win, no matter what the strategy of J E . a.2 If xzE S , and if J E plays optimally, then J E is sure to win, no matter what the strategy of J,. a.3 If xz$ S , U S,, then there is no optimal strategy at that point for J p and for J E .

Case (b) with 0, # Iz( and 6, =

@

b.1 If xz E S p and if J , plays optimally, then J , is sure to win, 110 matter what the strategy of J E . h.2 If xzE SE and if J E plays optimally, then J E can prevent J, from winning. 6.3 If xz $ S , U S,, then there is no optimal strategy at that point for J p and for J E .

Recall that S, n S, = o in Cases (a) and (b). Hence, Cases a.1 and a.2 on one hand, and b.1 and b.2 on the other hand, exclude each other. In Cases a.1, a.2, b.1, and b.2, there is an optimal strategy defined for J p or J E at the initial state x2. In Cases a.1, a.2, and b.1, a winner exists, and this can be assured at the outset of play. Hence, at the beginning of play the following questions arise: 1. Does xz E S p U s, O r does Xz 6sp u SE? 2. If xzE S , u S,, does x2E S p or does xZE S,? That IS, is there potentially a winner, and who is he? 3. If xzE S p , that is if Jp is potentially the winner, what strategy is an optimal one for him? And if X' E S,, what strategy is an optimal one for JE? That is, J p in one case, and J , in the other case, must be able to determine an optimal strategy for himself.

112

V

SOME GEOMETRIC ASPECTS OF QUALITATIVE GAMES

I n general these questions are difficult to answer. However, questions 1 and 2 are simplified if the assumptions of Lemma 5.5 are satisfied, that is if S , = comp S , and S , = comp S,. Then S , and S, have the same boundary, and the first problem reduces to one of determining this boundary, namely the game surface E. Accordingly, in the next chapter, we shall focus our attention on further properties of which hold for differential games, with a view to constructing E. Towards that end the remarks of Section 5.5 will be useful.

-

V I Differential Qualitative Games

6.1 RULES AND OBJECTIVES OF THE GAME

As in Chapter 3, we shall call x = (xl, x2, . . . , x,) the state vector of the game in n-dimensional Euclidean space En,where x, = t will denote the time variable. Let there be given the equation dx/dT = f ( x , u, u)

(6.1) where f = (fi,f2,. . . ,f,),and control variables u = (q,u2, . . . , u,) and v = (ul, u 2 , . . . , vs) are vectors of Euclidean spaces E' and Ex, respectively. Again we shall assume that x, u , and u belong to fixed domains G , U , and V of En,E', and Es, respectively, and that the function f is of class C1 on G x U x V. We shall consider strategies for players J p and JE to be functions of x, p : x -p(x) and e : x + e(x), x E G, respectively, belonging to prescribed classes of functions, and such that:

PW

E KUW

cu

e(x)

E &(x)

E

V

where KU(x)and K,(x) are given subsets of E' and E", respectively. As indicated by the notation, these subsets may depend on x. As in Chapter 3, we shall denote by YI,and Y Ethe sets of strategies for J p and J E , respectively, and we shall require that 9,and Y Esatisfy Assumption 3.1. . Mapping y will be defined as in Example 1.1 of Section 1.2. Targets 8, and 8, will be prescribed dosed? sets in G . We shall suppose that 8, n OE = 0.

t In the topology induced by En on G (see Appendix).

114

VI

DIFFERENTIAL QUALITATIVE GAMES

I n two target games we shall suppose that 8, # @ and 8, # 0 ,and in one target games that 8, f; 0 and 8, = @. For these two cases, the prescriptions for players J,, and J E were specified in Chapter 1. Some of the following arguments will apply to both cases. 6.2 A BASIC PROPERTY

Let us note first that if x E a x , where X i s a subset of G, then for every neighborhood A(x) in G of x

A(x) n X # .a For, if we suppose that A(x) n X = 0 , then X $ X

which implies that x$XncompXA

ax

But this is contrary to our assumption. Unless otherwise indicated we shall let xo denote a point of G which does not belong to 8,, v flE, but belongs to the boundary as, of S, (or as, of S,) and possesses PROPERTY 6.1. For each given point xo E as, (or xo E as,) there exist a neighborhood A(xo) i n G of xo and a strategy p:o (or e$), such that pzo (or e$) is optimal on A(xo) n S,, (or A(xo) n S,). We shall let {xo}denote the set of all points xo which possess Property 6.1. 6.3 A VARIATIONAL EQUATION AND ITS ADJOINT

Consider a point xo ~ z . 7If there exists a neighborhood of xo on which p:o and e,*oare of class C1, then there exists a non-null path mDe, emanating from xo,generated by strategy pair (p:,,, ezo). Let it be represented by x : T x ( T ) , T E [o, T ~ ] . Function x is a solution of

-

with initial condition x(0) = xo.

t

a

Recall that 3 =

as, n as .,

6.3

115

A VARIATIONAL EQUATION AND ITS ADJOINT

If there exists a neighborhood of r Don e whichpz. and e$ are of class C1, we shall associate with Eq. (6.2) the variational equation

and

with p = p,*.,

v = 1,2, . . . , r

ax

M =

ax

v = l , 2 , ..., s a=1,2, ..., n

1,2, . . . , n

with e = ezo; and

By definition, the equation adjoint to Eq. (6.3) is 2.

where

116

VI

DIFFERENTIAL QUALITATIVE GAMES

For given initial conditions q = qo and 1= Ao at T = 0, the solutions T q = q ( ~ )and A: T -+ 1 = A(T) of Eq. (6.3) and Eq. (6.4), respectively, are unique and continuous on [0, T,]. Furthermore, T ( T ) and A(T) are nonzero provided that qo and A0, respectively, are nonzero.

q:

---f

6.4 A PROPERTY OF BOUNDARIES OF SETS OF THE GAME

We shall now prove Lemma 6.1. Let X: T + x(T), T E [o, T,], represent a path rrp, emanating from point xo = x(O), xo E as,, generated by strategypair (p$, e), e E 9,. If at every point of m p , with the possible exception of its end point, there exists a neighborhood on which p$, and e are of class C 1 , then rrp c 3,.

If rD is a null path, Lemma 6.1 is trivial. Hence, let us suppose that not a null path. First of all, let us note that, since O p and OE are closed, and since X ( T ) # O,, U OE for all T E [0, T J , there exists at every point x ( T ) , r pis

TAX', d o b , e ) FIG.6.1. Proof of Lemma 6.1.

6.4 T E

117

A PROPERTY OF BOUNDARIES OF SETS OF THE GAME

[0, T,), a neighborhood A(x(T)) in G of

A(x(T))n (e, u Let x‘ = x(T’), T’

E

X(T)

e,)

=

such that

o

[0, T * ) , and let us suppose that x ( T ’ ) $ S,; that is, 0

n

-

x ( T ’ ) E comp S ,

Let A(x(T’)) be a neighborhood in G of x(T’) such that 0

A(x(T’)) c comp S p

, Let A(xo)be a neighborhood of xo such thatp$ is optimal on A(xo)n,S and consider a point of A(xo) A S,, namely,?

xi

A XO +

ETO

E A(x0)

n s,

Now let us consider describing curve r;(xi,pzo, e) that emanates from xi, is generated by strategy pair (p20,e), and is given by j z E y(xz,~:~,e). Since at every point of nP,with the possible exception of its terminal point, there exists a neighborhood on which p$ and e are of class C1, then, for I E ~ sufficiently small, ii. is a solution of trajectory equation d4d.r =f(x, P,*.(x),4x1) With initial condition 2(0) = xi. Furthermore, for I E ~ sufficiently small, there exists a point of I’;(xi,pzo,e) corresponding to T = T‘, namely X’

9n ( T ’ )

= X’

+ €7’ +

O ( T ’ , €)

where 110(7‘, €)II/€ = 0, 7’ = q ( ~ ’ ) and , q: solution of linear equation

T-+

7 = q ( ~ is ) the

defined on [0, T s ] , subject to the condition q(0) = qo. Since at every point x ( T ) , T E [0, T,), there exists a neighborhood A(x(T)) in G such that A(x(T)) n (e, u e,) = 0

t If there exists no point other than xo in A(xo) n S, then xo E S,. Lemma 6.1 reduces to Lemma 5.1.

In that case,

118

VI

DIFFERENTIAL QUALITATIVE GAMES

then, for 161 sufficiently small, %(T) 60, u 8 , for all T E [O, T ' ] ; and hence, trajectory given by 2 : T -+ %(T), T E [O, T ' ] , is a path, say ; p . Furthermore, for 161 sufficiently small,

rii

0

xi

A

n ii(~') E A(x(T'))c comp S, c comp

Sp

(6.5)

Sincepittois optimal on A(xo) n S,, it is optimal at point x i , and hence 9'-path; namely, ; = 7p (6.6) P Y

GD is a

But (6.5) and (6.6) contradict Lemma 5.1, and hence X(T') E

Since (6.7) holds for all

to, Tsl,

T'

E

[O,

sp

T ~ ) ,and

4 7 s ) E

(6.7) since x is continuous on

g,

And so Lemma 6.1 is established. By similar arguments one can also prove Lemma 6.2. Let x: T x(T), T E [O, rs],represent a path re,emanating frompoint xo = x(O), xo E as,, generated by strategypair ( p , ezo),pE Y P . If at every point of r e ,with the possibility exception of its end point, there ., exists a neighborhood on which p and ezoare of class C1, then rec 3 --f

6.5 A PROPERTY OF THE SURFACE OF THE GAME

As a direct consequence of Lemmas 6.1 and 6.2 we have Corollary 6.1. Let x : T + x ( T ) , T E [o, T ~ ] represent , a path r P eema, nating from point xo = x(O), xo E E,generated by strategy pair (pzo,e$). If at euery point of p D e ,with rhe possible exception of its end point, there exists a neighborhood on which p,*. and ezo are of class C1then vpec E. From the definition of E it follows that xo belongs to both As a consequence of Lemma 6.1, lrne

= s,,

"ge

c

and, by Lemma 6.2,

S ,

as,

and

as,.

6.6

119

TWO FAMILIES OF PATHS

Hence, sP

'ITpe

sE

Finally, (6.8) and (5.1 1) imply that rwe c

(6.8)

-

3

We see that Lemmas 6.1, 6.2, and Corollary 6.1 exhibit strong similarities to Lemmas 2.1, 2.2 and Corollary 2.1, respectively. They enforce the remarks of Section 5.5. It appears now that, provided the assumptions of these lemmas and corollaries are satisfied, surface E has limiting properties similar to the ones of a surface Z(C); that is, surface Z is the common boundary of two regions which contain all paths rl,and r e ,respectively, emanating from {xo} c E. Definition 6.1. A path rl,,emanating from a point xo E by a strategy pair ( p z ~ezo), , will be called a boundurypath.

E,generated

6.6 TWO FAMILIES OF PATHS

Let us consider a point xo E Z, and assume that there exist strategies

pzo and ezo which are of class C' on some neighborhood of xo.Then there exists a non-null path r p eemanating , from xo, generated by (pzo,ef), x = x(T), T and given by x: T Next we introduce --f

E

[O,

T ~ ] .

Assumption 6.1. For all x E rl,,,, and for all u E K,(x) and v E K J x ) , there exist strategies M, E 9,and PZ E Y Ewhich are of class C1 on some neighborhood of x,and which satisfy the conditions M,(x) = u and P,(x) = u , respectively. Assumption 6.2. At every point x of roe, there exists a neighborhood A(x) on which p20 and ezo are of class Cl.

As a consequence of Corollary 6.1 it follows that rrpe c E. Let x' = x(T'), T' E [0, T s ] , be a point of r p ethat does not belong to u OE. Of course, x' E Z. Now consider paths rrw' and re'emanating from x' = x(T'), generated by strategy pairs (&o, &) and (M,., e:o),

ep

120

DIFFERENTIAL QUALITATIVE GAMES

VI

FIG.6.2. Proof of the basic property of paths T,,’.

respectively, where az.(x’) = U ” E K,(x’) Px.(x’) = U b E K,(x‘)

and ub, v b are given vectors. Let US suppose that rrp’ and re‘belong to a neighborhood A(x’) on which pHo, pX,and K ~ , ,e:o are of class C1. We shall prove that rrp’ c 3, and

= s.,

ref

These properties are trivial in the case of null paths; consequently let us suppose that rD’and nd’ are non-null paths. Such paths exist since p$, /Izr and uxr,e,*oare of class C1on A(x‘), and since x’ $ 8 , u eE. For instance, consider path rrp’ and let it be represented by y: T + YtT>, 7 E

Let

X”

[O,

.,I.

= Y(T”),

7’’

E

[0, T,.), and let us suppose that 0

x”

n

E comp

Sp

X”

4 3,;

that is,

6.6

121

TWO FAMILIES OF PATHS

Let A(x”)be a neighborhood in G of Y(T”) such that 0

n

A(x”) c comp S, n A(x’) Let A(xo)be a neighborhood of xo such that p,*. is optimal on A(x0) n S,, and let

xi 2

+

XO

ETO E

A(x0) n S,

By arguments similar to those employed in the proof of Lemma 6.1, one can readily verify that, for I E ~ sufficiently small, there exists a path that emanates from xi, is generated by (p:~,e$,),and ends a t point xi at T = T’. since^:^ is optimal at point xi, that path is a &path, ~ 2 Let it be represented by 1: T + 1 ( ~T)E, [0, 7‘1. Point xj L ?(T’) is given by X’

+ €7’ + O(T’,

= X‘

€)

where limG+o( ~ o ( Te)11/e ’, = 0 , 7’ = q(~’), and q: T + 1;1 = q ( ~ ) ,T E [0, T,] is the solution of variational equation (6.3) subject to the condition q(0) = ?lo. It follows from Lemma 5.1 that and, for

xj E s,

(€1 sufficiently small, X’ E

S , n A(x’)

For 1.1 sufficiently small, there also exists a path dTjk that emanates from xj, is generated by (p;., @,.), ends at point xk at T = T”, and belongs to A(x’). Let it be represented by 9 : T + ?(T), T E [0,T”]. Since xj is arbitrarily close to xo, xj is arbitrarily close to x ’ , and since pE0 and b,, are of class C1on A(x’), xk L ?(T”) is arbitrarily close to x”; that is, for I E ~ sufficiently small,

xkE A(x”) and hence 0

n

xkE comp S p , In this manner we have generated

.

122

-

DIFFERENTIAL QUALITATIVE GAMES

VI

which can be represented by the function <: T <(T), where <(TI = for T E [ 0 , T’) <(T)

=

Y(T

- T’)

for

T

E

[T’,T’

+

T E

[ O , T’

+

T”],

T”]

Employing arguments similar to those used in Section 3.8, one can readily verify that T% u nik is a path that emanates from xi and is generated by strategy pair (p:o, g), where E(x) = e*(x)

for all x E G such that x,

< x‘,

g(x) = /3=.(x)

for all x

2 x,I

EG

such that x,

That is, T$ u dkis a 8 - p a t h , say ~ 2 . However, (6.9) contradicts Lemma 5.1, and hence X” E

s,

FIG.6.3. Paths T,,‘and T~‘.

(6.10)

6.7

Since (6.10) holds for all

to,

123

A LOCAL MIN-MAX CONDITION 7’’ E

[O,

T~),and

~rly

Hence we have proved that

Y(Tr> 6

3,

rpt=

s,

since y is continuous on

(6.1 1)

By similar arguments one can establish that re’

= s,

(6.12)

Finally, if T’ = 7, and x‘ = ~(7,) E 8,, V 8,, then rp‘ and re’are null paths for which (6.11) and (6.12) are obviously valid. Next we shall consider two families of paths, namely 1. the set {rp’]of all paths rp‘ emanating from points x’ E rDe; and 2. the set {ra’)of all paths re‘emanating from points x’ E r pe. 6.7 A LOCAL MIN-MAX CONDITION

Hereafter we shall utilize the notation and the assumptions of Section 6.6. In addition, we shall introduce

Assumption 6.3. Game surface Z can be defined by the equation g(x> = 0 and at each point x‘ = x ( T ‘ ) ,x‘ $ BE, v 8,, of a path rDe starting at xo E 9, there exists a neighborhood A ( x ‘ ) in G of x’ on which the function g is of class C1and grad g ( x ) is defined and different from zero. Furthermore at every point x of A(x’) n S p , g(x) G 0 (6.13) and at every point x of A ( x ’ ) n S ,,

g(x) 2 0

(6.14)

Again, consider a non-null path re‘emanating from x‘,x’4 8, U 8,, generated by strategy pair (az,, e:o).t Let x‘ dex denote points of re’in the neighborhood A ( x ’ ) , and let t be a function 5 : 7 P x = l(7)

+

---f

where E(0) = 0 at point

t We assume that r,’

X I ,

defined on an interval that depends on A(x’).

is of class C1on A(x’) and that re’c A(x’).

124

DIFFERENTIAL QUALITATIVE GAMES

VI

It follows from (6.12) and (6.14) that

g(x' and I(6ex(1+ 0 as

7+

+ 6"x)2 0

0; hence, since

-

grad g(x') 6"x

where

6 and accordingly g(x') = 0,

+ o(JJ(SexlJ) 20

(6.15)

o( I/d"xI()/ ~ ( P11 x= 0

!im

It6 x11-0

Upon dividing (6.15) by

E

XI

7,which

is positive for P x # 0 , we obtain

- +

grad g ( x ' ) -

I16"x II) ~

7

7

Then allowing 7 -+ 0, and consequently leads to

dex/7 - t f ( x ' , u', e,$(x'))

VX', X I

where

ub = a,.(x')

-

grad g(x') f ( x ' , ub,ez0(x'))2 0 E T g e , XI $ 8 , U 8, VUbE K,(X')

XI

Invoking similar arguments for a path rB'emanating from $8, LJ 8,, generated by strategy pair (p$, @%,), results in grad d x ' ) ..f(X', P,*.(X'>, vb) Q 0 VX', XI E T B e , X' $ 8 p V 8 , VVbE K,(X') Moreover, since rPe c

(6.16)

erg*,

XI

(6.17)

E,

-

grad g(x') f (x', P~x'),e$ (x')) = 0 VX', X I E r P e ,X I 4 eP v OE

(6.18)

Conditions (6.16)-(6.18) are embodied in

Lemma 6.3. Let rDe be a boundary path. If Assumptions 6.1-6.3 are which does not belong to 8, u OE, satisfied, then at every point x of rBE min n(x) . f ( x , u, e,*o(x)) U€K"(Z)

(6.19)

6.8

125

GRADIENT AND ADJOINT VECTOR

where n(x) & grad g(x), xo is the initial point of rD6, and (p& e:o) is the strategy pair that generates rge. 6.8 GRADIENT AND ADJOINT VECTOR

Consider a path rDe that emanates from xoE E, is generated by (pz0,e$) and is given by function x: T x = x ( T ) , T E [o, T,]. Let A(xo) be a neighborhood in G of xo such that& and eo: are optimal on A(x0) n S , and A(x0) n S,, respectively, and such that -j

A(XO) n

(e, u e,)

= 0.

Let go denote a point of A(xo) n Z.? Since grad g(xo) is defined, E is such that in arbitrarily small ball A'(xo), with center xo, there exist n - 1 points f which may be represented by 20

= xo

+ EqO +

-

(6.20)

O(E)

where limcA0I~o(E)II/E = 0 and grad g(xo) qo = 0, and for which I E ~ # 0 and the n - 1 vectors qo are linearly independent. Now, with each point 2 O let us associate a path GDethat emanates from it and is generated by ( p : ~ejo). , The points of such paths, say i',at T = T ' , T' > 0 , provided that they exist, are obtained by integrating Eq. (6.2) subject to the initial condition x = io at T = 0. Provided that x' = x ( T ' ) does not belong to 8, U O,, that Assumption 6.2 is satisfied along r g eand , that 1.1 is sufficiently small, points 2' exist and are given by 2' = x' €71' O(E)

+

+

with q' = q ( ~ ' ) where , q is the solution of variational equation (6.3) subject to the initial condition q(0) = qo. Corollary 6.1 implies that 2' = x' E q ' O(E) E z Furthermore,

+

g(2)

- g(x')

= grad g(x')

+

- (€7' + o(E)) + o(llq' + o(~)ll)

or, more simply, g(2')

- g(x')

= E grad g(x'). q'

+

O(E)

where lime-o O ( E ) / E = 0.

t Clearly, i obelongs to the set {xa}of all points for which Property 6.1 is satisfied.

126

VI

Moreover, since x' and 2' belong to E

grad g(x')

Upon dividing (6.21) by

E,

DIFFERENTIAL QUALITATIVE GAMES

E,g(x')

- q' +

and letting

U(E)

= g(2') = 0, and hence

(6.21)

=0

E + 0,

we arrive at (6.22)

grad g(x') * q' = 0

This is true for n - 1 linearly independent vectors q' and for all transfer times 7' E [0, T,). Condition (6.22) can be written as grad g(x(7)) q(7) = 0

7

E

LO,

7,)

(6.23)

Now consider the solution A: 7' il = A(T), 7 E [0, T ~ ] , of adjoint equation (6.4) for given initial condition A(0) = lo. This vector has the property that (CEld7)(A(7) * q(7)) = 0 and so, if we choose ilo perpendicular to the tangent plane of E at point xo, that is if lohas the same supporting line as grad g(xo), then A(7) is perpendicular to the tangent plane of at point X(T) for all 7 for which x(7) E r P ewith , the possible exception of the terminal point of ripe; that is, its supporting line is that of grad g(x(7)). Function A is a nonzero continuous vector function, and so, if we choose A(7) in the same direction as gradg(x(7)) at 7 = 0, it remains codirectional with grad g(x(7)) for all T for which x(7) E 7rD,,, with the possible exception of the terminal point of r P e .

=

6.9 TRANSVERSALITY CONDITION

Again consider a path r P e that emanates from xo E E,is generated by ef) and is given by function x. Let A(xo) be a neighborhood in G of xosuch thatp,*n and e,*oare optimal on A(xo) n Sz> and A(xo) n S,, respectively, and such that, at every point of A(xo) n S,, Q0 (6.24) ( ~ $ 0 ,

and at every point of A(xo) n S,, g(x>

>0

(6.25)

6.9

127

TRANSVERSALITY CONDITION

In addition to Assumption 6.3, we shall make

; ,

Assumption 6.4. A(xo) n and A(xo) n 3, (or A(xo) n S , and 0 A(xo) n S,) constitute apartition of A(xo). Here we shall study the case for which the end point of r g enamely , x f 2 x(T,), belongs either to 8, or to 8,. Next we introduce Assumption 6.5. The boundary of the target to which xf belongs, namely aO,, & 8, n comp 8, or ae, & G, n comp 8,

is defined by the equation

q X )= o

(6.26)

where 8 = 8, or 8 = 8, respectively. Furthermore, there exists a neighborhood A(xf) of xf on which the function 8 is of class C1 and grad 8(x) is defined and different from zero. Let Rf denote a point of A(xf) n 8, (or of A(xf) n 88,). Since grad O(xf) is defined, ae,, (or a@), is such that in an arbitrarily small ball A'(xf), with center x f , there exist n - 1 points Rf which may be represented by if= Xf Eqf O(E) (6.27)

+

+

-

where limc+o IIu(E)II/E = 0, and grad 8 ( x f ) qf = 0, and for which E is strictly positive and the n - 1 vectors qf are linearly independent. Moreover, to each point i ffor which E is positive there corresponds a point in A'(xf) n 88, (or A(xf) n aB,) for which E is negative and qf is the same. For instance, suppose that x f E 8,. Let lo be perpendicular to the tangent plane of E at point xo and in the same direction as grad g(x(0)). Let Af A(T,), where h is the solution of adjoint equation (6.4) with A(0) = 20. We shall prove that y qf = 0 (6.28)

.

for all qf in (6.27), and hence that Lf is collinear with grad O(xf). Let us suppose that V q f z 0 for some q f ; that is, ;If is not perpendicular to the tangent plane of 8,, at point x f .

-

128

VI

DIFFERENTIAL QUALITATIVE GAMES

\ FIG.6.4. Proof of the transversality condition.

If

If

K . qf > 0, we let

P = xf +

Af . q f < 0, we let

+ a(.)

with

E

>0

+

with

E

<0

if= xf + qf

O(E)

If Assumption 6.2 is satisfied along r P ethere , exists? a solution %: T -+ x = ?(T), 7 E [O, T ~ ] ,of trajectory equation (6.2) that satisfies the end condition n(7,) = 2f.

and there exists a point xi = %(O) such that xi = xo

t For I tl

sufficiently small.

+

EqO

+

O(E)

6.9

129

TRANSVERSALITY CONDITION

with qo = q(O), where q is the solution of variational equation (6.3) with end condition qf = q ( ~ ~ ) . We have g(xi) = g(xO E q O 44) and, since g(xo) = 0,

+ + g(xi) = grad g(xo)- (qO + 44)+ o ( l l q o + 4 ) i l ) = E grad g(xo) qo

+

O(E)

where lim~+oO ( E ) / E = 0. For 1.1 sufficiently small, the sign of the righthand side is the same as the sign of E grad g(xo)* qo; that is, g(xi) has the same sign as 4 0 T O . Since A(T) T ( T ) = constant along rSe, for I E ~ sufficiently small, g(xi) has the same sign as €Af qf; that is,

-

-

Also, for 1.1 sufficiently small,

g(xi) > 0 xi E A(xo)

It follows from (6.24), (6.25), (6.29), and Assumption 6.4 that

xi E SE n A(x0)

FIG.6.5. Transversality condition.

(6.29)

130

VI

DIFFERENTIAL QUALITATIVE GAMES

Since 8, n 8fi2= 0 ,for I E ~ sufficiently small there does not exist E [0, ~~1such that %(T') E O E ; and since n(0) = xi and j 2 ( ~ ~=) x f E e,, the path & that emanates from x i , is generated by ( ~ 2 0 ,ezo) and is given by function 2 reaches 8,. That is in contradiction with Lemma 5.2, since 2 is an &-path and hence must belong to S E ; that is, it cannot reach 8, since Or. c S,. Thus we have established that Af qf = 0 for all qf in (6.27). Now let us assume that target 8, is defined by O(x) Q 0. We shall prove that 7'

-

Y = Kgrad O(xf),

K = constant

Suppose that /If = Kgrad O(xf) with K

>0

(6.28)'

< 0, and let

q f A - gradO(xf)

Then

af . > o

Furthermore there exists a

> 0 such that for all E , 0 < < a , E

xf

+ e q f E 8,

If Assumption 6.2 is satisfied along r g ethen , for E sufficiently small there exists a solution 2 : 7 + x = %(T), T E [O, T f ] , of trajectory equation (6.2) with end condition

n(7,)

= Xf

+Eqf

and there exists a point x' = n(0) such that x'

= xo

+ EqO +

O(E)

with qo = q(O), where q is the solution of variational equation (6.3) with end condition qf = q ( ~ ~ ) . We have

-

+

+

o(E)) = E grad g(xo) qo 4-O(E) g(x') = g(xo where limG+oO ( E ) / E = 0. For E sufficiently small, the sign of the righthand side is the same as the sign of E grad g(xo) qo, and hence g(xz) has the same sign as e l o qo. Since E > 0, since A(7) T ( T ) = constant along rDe and since ilf q f > 0, it follows that g(x') > 0 for E sufficiently small.

-

-

-

-

6.10

131

A MIN-MAX PRINCIPLE

Hence, for

E

sufficiently small, Xi E

s~ n h(X*)

Then, by the same arguments as those employed above, one obtains a contradiction with the fact that Xf

+ Erf E O p G s,

Finally, since K # 0, we conclude that K If x f E 8, and 8, is defined by

e(x)

> 0.

>0

one can again prove in similar fashion that

I f = Kgrad 8 ( x f ) ,

K

>0

6.10 A MIN-MAX PRINCIPLE

Now let

&?(A,

x, u , u)

L il . f ( X , u , u )

From Lemma 6.3 and from (6.28) one can deduce

x= Theorem 6.1. Let rrpe be the boundary path represented by x: T x ( T ) , T E [0, T,], rS# 0. If Assumptions 6.1-6.3 are satisfied, then there exists a nonzero continuous solution A : T + il = A(T), T E [0,T , ] , of adjoint equation (6.4) such that --f

(a)

min

&?(A, x , u , e f ( x ) ) = max &?(A, vEKy(x)

u€Ku(x)

x , p:o(x), u)

= # ( I , x , P ~ x )e X , x))

(b)

&?(A, x , P$(x), e>(x)) = o

f o r all T E [0, T J , where xo is the initial point of ripe, and (pzo,e,*.) is the strategy pair that generates ripe. Furthermore, i f 8, 8, = 0 ,if rrpe reaches 8, (or 8,) at point x f , if Assumptions 6.4 and 6.5 are satisfied, and if 8, (or 8,) is de$ned by 8,(x) 0 (or 8 E ( x ) O), then

<

(c)

>

I f = Kgrad 8 ( x f ) ,

where 8 = 8, or 8,.

K

>0

132

VI

DIFFERENTIAL QUALITATIVE GAMES

If xs X(TJdoes not belong to O p u ,O then Theorem 6.1 is a trivial consequence of Lemma 6.3 together with the conclusions of Section 6.8. If xs E Or U OE, we shall consider a value of T , say T‘ E [0, T ~ ) and , a point x’ = x(T’)E r g eat, which relations (a) and (b) above hold; namely, min A(T’)*f(x’, U , e:O(x’)) = max A(+) *f(x’, pzo(x’), U) UEK

I,

( x’)

vEKJx’)

= A(+) .f(x’, pEo(x’), e,*o(x‘))

and

.f(x‘, p*(x’), ezo(x’)) = o

~(7’)

Since f is of class C1 on G x U x V , and pz0 and e,*. are of class C1 on a neighborhood of rSer and x: T‘ + x’ = x(T’) as well as A: T’ + 1‘ = A(T’) are continuous functions of T’ on [o, T s ] , it follows that H , defined by H : T’ +H(T’) where

WT’)= VT‘) .f(x(~’),p*(x(~’)),eXT’))) is a continuous function of T’ on [0, T s ] . Hence, relation (b) holds at r = T , and xs X ( T J It is clear that relation (a) also holds at T = ‘rS,since A(Ts)

and

.f(X(Ts),

Ub,

e3(x(%>)> 20

A(T,) .f(X(Ts), P3(X(.,), can be deduced from

and

A(T’)- ~ ’ ( x ( T ‘~,.(X(T’>), ), e3(x(r’)))

Ub)

Q0

> 0,

azs(xs)= ub

/?Z’(X~) = Ub A(T’) ’f(X(T’), p;O(x(T’)), @z’(X(T’>)) Q 0, by continuity, for every given ub E Ku(xs)and every given v b E K,(xs). 6.11 AUTONOMOUS GAMES WITH TIME-INDEPENDENT

TARGETS

An autonomous game is one in which the function f,in the right-hand side of the state equation (6.1), does not contain the variable x, = t explicitly. This situation can arise only if p and e d o not contain x, in their arguments; that is, if they are functions of x l , . . . , x , - ~ only.

6.11

AUTONOMOUS GAMES WITH TIME-INDEPENDENT TARGETS

133

Let Y(T)& grad g(x(7)). From (6.23) we obtaint

or, taking account of (6.3),

or equivalently,

Now recall thatfn(x, u, v) = 1 , and so af,/ax, = 0 for a = 1 , . . . , n. It follows that the last row of the matrix

apz0

(aj- + - -af+ - -

ax

au ax

afae;or

av ax

is zero. Moreover, since

for v = 1, . . . ,n in an autonomous game, it follows that the last column of this matrix is also zero. On the other hand, if targets OP and 8, are time-independent7 that is, if OP and 8, in two target problems, and 8, or 8, in single-target problems,§ are x,-cylindrical sets in En,then the map of the game is composed of x,-cylindrical sets in En. It follows that S,, S , and E are x,-cylindrical sets in En. In this case, g is a function of x l , . . . , xnP1 only, which implies that the nth component of vector y(7) vanishes; namely, y(.) = (yl(.)7 Y2(7)?. . . Yn-l(T)> Since (6.30) is satisfied by n - 1 linearly independent vectors q(7) at point x = x(T), we arrive at 9

Y(4

t Here we require that g is twice continuously differentiable.

$ Here, p = p,*o and e = e,*o, andp, and e, are the vth components of vectorsp and e , respectively. 9 Of course, E may be empty; and, in one target games with 0, # 0 and 0, = S, may be empty. @,

134

VI

That is, function y:

T

+

y ( ~ )T,

E

DIFFERENTIAL QUALITATIVE GAMES

[ O , T,], where

Y ( 7 ) = grad g ( x ( 7 ) )

is a solution of adjoint equation (6.4). Finally, according to the definition of A(T), we have? (6.31)

A ( T ) = k grad g(x(T)) where k is a positive constant. 6.12 CONSTRAINTS

Let us consider constraint sets K,(x) and K,,(x), respectively, given by y a ( x ,u)

< 0,

Ya(X,D)
a = 1 , 2 , . . . ,k

(6.32)

cc=1,2 ) . . . , 1

where functions y a and y, are of class C1 on G x U and G x V, respectively. At a point x’ = x ( T ’ ) , T’ E [0, T s ] , of a boundary path mDe emanating from xoE E,some or all of inequality constraints (6.32) may be equalities. Suppose that

< <

tc = 1, 2, . . . , k’ k ya(x’,p;o(x’)) = 0, cc = 1, 2 , . . . , 1’ 1 ya(x‘, ezo(x’)) = 0, We shall suppose also that if k > Y or E > s, at most r of the ya(x, u) or s of .the ya(x,u ) vanish at any point of G x U and G x V , respectively, and furthermore that the matrices

1 , 2 , . . . , k‘ cr = 1 , 2 , . . . , r

tc=

[”’$;

41

u = 1 , 2 , . . . , 1’ a = l , 2 , ...,s

have maximum rank at u = p z o ( x ) , D = e*,.(x), x = x*(k). By arguments similar to those employed in Chapter 3, one can verify that adjoint equation (6.4) may be written (6.33)

t At all points X ( T )

where grad g ( X ( T ) ) is defined.

6.13

135

SEMIPERMEABLE SURFACES

where

1

a p A ap,,(x,u) -ax= [ ax,

v = ~ 2 , . .. , k u = i , 2 , ..., n

ax

v = 192, * * , 1 u = l , 2 , ...,n

-

evaluated at x = x ( T ) , u = ~ ~ o ( x ( T ) )u, = e$(x(T)); and Lagrange multipliers p and v are k- and I- component vectors, respectively, given by

1 with p i 2 0, vi < 0.

lu=

Thus, if constraints (6.32) are independent of x, adjoint equation (6.4) reduces to daldr = -(aflax)Ta (6.34) 6.13 SEMIPERMEABLE SURFACES

We shall now complement Theorem 6.1 by

Theorem 6.2. I f g: x --t g(x), x E G, is such that grad g(x) is dejned and is diflerent from zero f o r all x E G , and fi there exist strategies p and e" such that (a)

min n(x) * f (x, u , t?(x)) = max n(x) . f ( x , p(x), u ) vCK,(x)

UEKU(Z)

= n(x> *f ( x ,j w , 44)

and (b)

n(x) *f(x,P(x), W ) )= 0

(6.35) (6.36)

136

VI

DIFFERENTIAL QUALITATIVE GAMES

f o r all x E G , where n(x) & grad g(x); and fi there exists a constant C such that g(x) < c vx~e, [or g(x) > C VXE~,]

and fi the surface %(C) dejned by %(C)

is not empty; then

V(C) c comp S,

[or Furthermore, $8,

(6.37)

{ x : g(x) = C }

g ( C ) c comp S,]

= 0 and. of course, 8 p # 0 , then %(C) C

s,.

Let xi E W(C) and consider describing curve r x ( x i , p ,Z), x E y ( x * , p ,i?), where p is any strategy for J,. If r x ( x 2 , pZ) , is a single point, then obviously rx(xi,p, Z) n 8, = 0 [or

r x ( x t , p ,2) n 8, =

01

Therefore let us suppose that r x ( x i , p ,Z) is given by x : T E [0, T ~ ) where , T , # 0. As a consequence of (6.35), grad g W > ) . f ( X ( ' ) > P ( X ( d ) ,

Z(x(4))

z0

T -+

x = x(T),

(6.38)

for all T E [0, T J ; and since the left-hand side of (6.38) is the time derivative of g ( X ( T ) ) , namely dg(x(T))/dT,for all T E (0, T ~ ) it, follows that

g:7- -g,(+ where g,(T) g(x(T)),is a non-decreasing function of Since xi = x(0) belongs to g ( C ) , and if g(x) < C for all x

E 8,,

T

on [O, T ~ ) .

then

rx(xi,p, e") n 8, for all x E y ( x i , p ,e") and for a l l p E 9,. Since (6.39) holds for all xi E %(C),

=0

(6.39)

6.13

137

SEMIPERMEABLE SURFACES

Consequently,

%(C) c comp A

which implies that %(C) c comp S , By similar arguments, one can prove that %(C) c comp SE if g(x) > C for all x E 8,. Hence, Theorem 6.2 is established. Now, if OE = m and, of course, 8, # 0, Equation (6.39) implies that

( p , e") E q g x i )

vp E Y

Hence

vxi E %(C)

p

%(c) sE

Definition 6.2. A surface %(C)defined by (6.37) is called a semipermeable surface,? if grad g ( x ) is defined and is different from zero for all x E G , and if there exist strategies and e" such that Conditions (a) and (b) of Theorem 6.2 are satisfied. We shall complement Theorem 6.2 by

Theorem 6.3. I f g,: x g,(x), v E I, where I is a set of indices, are functions dejned on G , such that grad g,(x) is deJined and is diflerent from zero f o r all x E G and f o r all v E I , and if there exists a strategy e" such that --f

n,(4

-f(X,

u,

tw)> 0

grad g,(x); and ifthere exist

f o r all u E K,(x), x E G , v E I , where n,(x) constants C,, v E I , such that 9 E #

where 9 E

g,(x) >, c,, v

A

= (x:

then 9 E

Furthermore,

if 0,

8,n9E=

and

0

0

E I>

c comp Sp

= 0 and, of course, 8, 9E

c sE

t According to the terminology of Isaacs.

# 0 , then

138

VI

DIFFERENTIAL QUALITATIVE GAMES

Let x z E g Eand consider describing curve r x ( x z , p ,e"), x E y ( x 2 , p ,&), where p is any strategy for J,. If r x ( x z , p e") , is a single point, then, obviously, r x ( x z , p &) , n 8,, = 0. So let us suppose that F x ( x z , p ,e") is given by x : T + x = x ( T ) , T E [0, T,), where T , # 0. We have grad g,(x(.)) .f(X('),P(X(4), &(x(.))) for all

T

E

E

>0

YE

1

(0, T ~ )from , which it follows immediately that

r x ( x z , p , e")

and hence for all x

YE1

[0, 7,); and hence

T E

dg,(x(T))/dT for all

>0

C 9

E

r x ( x z , p ,z) n e p = y ( x z , p ,Z), for all p

E

Y, and for all x'

( p , e") y p * ( x z ) and so

rn

v p E9 ,

E g E .Consequently,

v x zE 9

E

BE c comp S,,

Now, if 8, =

~5

and, of course, 8,, # 0 , then

(p, VpEYp

VX'EgE

Hence

9 E = SE which proves Theorem 6.3. By similar arguments as in the proof of Theorem 6.3 one can establish

If

Theorem 6.4. gy: x g,(x), v E I, where I is a set of indices, are functions dejined on G , such that grad g,(x) is defined and is different from zero for all x E G and for all v E I, and if there exists a strategy p" such that ---f

n,(x)

for all v

E

K,(x),x

E G,

-f(X,P"(X>,

4Q0

v E I , where nv(x)

grad g,(x);

and

if there

6.14

THE GRADIENT ; SEMIPERMEABLE SURFACE PATH

exist constants C,, v E I , such that

# 0 and 8, n g P= 0 ,where

$3,

g p Li {x: g,(x) Q

then

139

c,,v E r>

g P= comp S ,

Furthermore, if8, = 0 and, of course 8 , # 0 ,then g P

= SP

6.14 A PROPERTY OF THE GRADIENT ALONG A PATH IN A SEMIPERMEABLE SURFACE

Consider a non-null path 75. emanating from xi E % ( C ) , generated by strategy pair (p, Z) and represented by x: T -+ x = x ( T ) , T E [0, T j ] . As a consequence of (6.36) dg(x(T))= grad g(x(7)) ' ~ ( x ( TP(x(T)), ), ?(X(T))) = 0 dT

and so g ( x ( 7 ) ) = C for all Now, we introduce

T E

(6.40)

[0, T i ] . Hence, ii c %'(C).

Assumption 6.6. Strategiesp and Z are of class C1on a neighborhood of 71. Relation (6.36) is an identity in x on G . Let us rewrite it as

i ax, *)'.(X,

P(X),

qx)) =0

(6.41)

a=1

By differentiating (6.41) with respect to x p , /I= 1 , 2 , . . . ,n , we obtain?

a=l

ax, ax,

fa(&

p ( x ) ,q x ) )

Now, along a non-null path ii lying on %'(C),

(a,c,,) 5a " p O f , ( x , P ( x ) , ax, a=lax,ax,

--

dr

=

Z(x)),

8 = 1, 2, . . . , n

t Here we require that g is twice continuously differentiable.

(6.43)

140

VI

DIFFERENTIAL QUALITATIVE GAMES

Equations (6.42) and (6.43) lead to

By developing afa(x,p"(x),Z(x))/ax, and writing (6.44) in vector form, we obtain d (6.45) d-r

for x = X(T) E + If constraint sets K,(x) and K,(x) are given by (6.32), one can verify, by arguments similar to those used in Chapter 3, that adjoint equation (6.45) may be written as

(6.46) where ;1 = n(x(-r)), and p and v have the same meaning as in Sec. 6.12. If constraints (6.32) are independent of x, adjoint equation (6.45) reduces to

(6.47) Clearly, there are similarities between the properties of semipermeable surfaces and the properties of game surface E. One can easily find examples in which E is imbedded in a family of semipermeable surfaces. 6.15 TRANSVERSALITY CONDITION FOR A SEMIPERMEABLE.

SURFACE

Let g : x - g ( x ) be a function defined on G , such that gradg(x) is defined and is different from zero for all x E G , and let us suppose that there exist strategiesp" and e" such that Conditions (a) and (b) of Theorem 6.2 are satisfied. Furthermore, let us suppose that max,,8,g(x) is defined, and let M & max,.op g(x) g ( M ) A { x: g( x) = M } is a semipermeable surface. Clearly, we have n z 0

-

ww ep

6.15

TRANSVERSALITY CONDITION FOR A SEMIPERMEABLE SURFACE

141

Let 3, E V(M) n 8,. Since g ( f z , ) = M and grad g(4,) Z 0, 3 , is not an interior point of 8,; that is, 2, E 88,. Let E ae, 4, €7

+ +

+ + +

where 2, 7 belongs to the tangent plane T(fp) of 38, through f,; and limc+o I]o(E)]]/E = 0. Since 2, €7 O(E) E 8, we have g(fp

+ €7 + 4~)) = g(P,) + €7 . grad g(2,) + 4lk7 + 4e)II) Q M

and since g(f,) = M , we have

€7 * grad g(2,)

Dividing by (i) for

E, E

---f

E

7 * grad gG,)

<0

7 * grad g(2-P)

>0

<0

Hence

4E)II)

Q0

0, we obtain

>0

and (ii) for

and allowing E

+4lE7 +

-

7 grad g(2,) = 0

+

Since this equation holds for all 2, q E T(2,) it follows that grad g(f,) is perpendicular to T(P,). Hence, 88, and % ( M ) have the same tangent plane at point 2,. It follows that grad g(fp) = K , grad O,(f,)

<

Furthermore, if target 8, is defined by e,(x) 0, constant K, is positive. By similar arguments one can prove that, if min,,,b g(x) is defined, then where

grad g ( f E ) = KE grad O,(f,) g E E U(m) n 8,

V(m)2 {x: g ( x ) = m} m L min g ( x ) ZWd,

KE

> 0 if target 8,

is defined by eE(x)

>0

142

VI

DJFFERENTIAL QUALITATIVE GAMES

Surface % ( M ) separates G into two disjoint sets, namely

B 2 {x: g(x)

and

8'

4 {x:

g(x)

>M) <M)

Likewise %(m) separates G into two disjoint sets, namely

B 4{x: g(x) < m>

and

8' A {x: g(x)

> m)

And, from Theorems 6.3 and 6.4 Q c comp S,

and

B c comp S,.

Furthermore (i) if 8E = ia and, indeed 8, # 0 ,then & c S,; and then 9 c S13. (ii) if Br. = ia and, indeed eE # 0 , 6.16 A MIN-MAX PRINCIPLE ALONG A PATH IN A SEMI-

PERMEABLE SURFACE

From the discussion of Sections 6.14-6.15 one can deduce at once Theorem 6.5. If.? is a non-null path emanating from a point xi of a semipermeable surface V ( C )& {x: g(x) = C } , generated by strategy pair (jj,e") and represented by x: T -+ x = x ( T ) , r E [0,T J , and if Assumption 6.6 is satisfied, then there exists a nonzero continuous solution A: T + A(T), T E [0, rs],of adjoint equation dil

such that

6.17

EXAMPLE

143

6.1

Furthermore ifAssumption 6.5 is satisjied and ife, is dejined byOp(x) and

(i)

M

4max g(x) is defined; seep

V(M); (iii) n- reaches 8 , at point x p at r f p , (ii) xi

then

<0

E

> 0.

(c) h(7fP)= K p grad fJ,(x,), K p

Likewise, ifAssumption 6.5 is satisfied, and ifeE is deJned by eE(x) 2 0, and (i)' m = min g(x) is defined; XCeE

(ii)' xi E V ( m ) ; (iii)' reaches 8, at point x E at T

+

then

(d) h(TfE)= Kx grad e E ( X E ) , KE

/ ~ ,

> 0.

6.17 EXAMPLE 6.1

Let us consider a one target qualitative game described by

+

dx,/dr = u u dx21d7 = -2 dx,/dr = 1

with constraints given by and

e,

lul Q 1

= n { x = ( x l , x2, x3):

(6.48)

Ivl Q 1

(6.49)

qX) 2 x12 + x22 - ~2

< 01

(6.50)

Let us suppose that there exists a non-null boundary path ripe, emanating from xo E E, generated by strategy pair (pzo,e 4 ) , and represented by x: T -+ x = x ( T ) , T E [0, r,], which reaches at point x(T,), and that Assumptions 6.2-6.4 are satisfied.? The %-function is given by

ep

%(A, x , u, v) = I1(U

+ v) - 21, + 1,

(6.51)

t Assumption 6.1 is satisfied since the constraints are given by (6.49), and Assumption 6.5 is also satisfied since target 0, is given by (6.50).

144

VI

DIFFERENTIAL QUALITATIVE GAMES

The adjoint equations are so that

diluldT = 0,

a = 1,2,3 a = 1,2,3

A u ( ~= ) constant,

(6.52)

(6.53)

From condition (c) of Theorem 6.1 we have Xl(Tf)

= CXl(Tf)

h z ( T f ) = Cxz(.f) A3(7f)

C

(6.54)

=0

where, in view of (6.50), C2

>0

+

= R-'[Al'(T,)

&'(Tf)]

(6.55)

Then it follows from ( 6 . 5 3 ) and (6.54) that Al(4

= a1(7,)

= CX2(Tf)

A3(T)

(6.56)

0

In view of condition (a) of Theorem 6.1, U(T)

)'(V

= -sgn

xl(Tf)

= sgn XI(7,)

(6.57)

and so by condition (b), &(T)

3

0

which implies that X2(Tf)

(6.58)

=0

Along r p e(6.48) , may be rewritten as (6.59) Upon integration of (6.59) we find that a non-null boundary path rp , that reaches O,, at point x ( T ~is) a straight line whose projection on the xl-x2plane is given by xl(-r) X1(~f)= fR (6.60) X2(T) = -2(T - Tf)

6.18

EXAMPLE

145

6.2

The projection on the x1-x2 plane of the two families of possible boundary paths thus obtained is sketched on Fig. 6.6. One can prove that the closed shaded area of Fig. 6.6 is the projection of a set of E3 that contains S,, whereas the open unshaded area is the projection of a set of IP contained in S,. Indeed, S , and S, depend on 9,and 9,.

FIG.6.6. Example 6.1. Projection on the xl-x, plane of the two families of possible boundary paths; projection of a domain of E 3 that contains set S,. (op: projection of target Op.) 6.18

EXAMPLE 6.2

Next let us consider a one target qualitative game described by Eqs. (3.78), with constraints given by (3.79), and target 0, defined by

e,

2 tX = (xl,x2, xs): qX)A

x12

+

x22

- ~2 Q 01

(6.61)

Again let us suppose that there exists a non-null boundary path 7rpe emanating from-xOE S , generated by strategy pair (pzo,ez0), wherepz~= @zo,,p,*,),and represented by x: 7 -+ x = x(7), 7 E [0, T ~ ] which , reaches O p at the point x(T~),and that Assumptions 6.2-6.4 are satisfied.?

t Assumption 6.1 is satisfied since the constraints are given by (3.79), and Assumption 6.5 is also satisfied since target 0, is given by (6.61).

146

VI

The X-function is given by

S ( A , x, 2.4, u) = A1241

DIFFERENTIAL QUALITATIVE GAMES

+

AZ2.42

+ 11u +

A3

(6.62)

The adjoint equations are so that

dAJd7 = 0,

a = 1,2,3

(6.63)

).(,A

cc = 1 , 2 , 3

(6.64)

= constant,

From condition (c) of Theorem 6.1 we have &(7f)

= cx1(7f) = cxZ(7j)

A3(7j)

=0

A1(7j)

where, in view of (6.61), C2

= R-'[h12(Tf)

C

>0

+ AzZ((.rf)]

(6.65)

(6.66)

Then it follows from (6.64) and (6.65) that A1(7)

CX~(T~)

A2(7)

cxz(~f)

A3(7)

=0

From condition (a) of Theorem 6.1 we obtain

(6.67)

6.18

EXAMPLE

6.2

147

Consequently, (6.71) at a point where a non-null "oundary path Along rrge, (3.78) may be rewritten as

rrpe

reaches

D.

(6.72)

Upon integration of (6.72) we find that a non-null boundary path T~~ that reaches op at point x(T,) is a straight line whose projection on the xl-x2 plane is given by (6.73) x 2 = ax, b where

+

a =A b

Xd7-f)

-

~ ~ ( 7 (Rlw) ~ )

= f=

sgn xl(Tf)

-(R/w)a sgn X1(Tf) = f

W

41 - w2

d1R

w2

A straight line given by Eq. (6.73) intersects the xl-axis at point P: x1 = R/w, or at point Q : x1 = -R/w, depending on the sign of xl(Tf). Of course, it is tangent to the circle given by x12 xZ2= R2. The arguments above yield four families of possible boundary paths whose projections on the x1-x2plane belong to the four half-rays P'M' , P"M", Q'N', Q"N" sketched on Fig. 6.7. Let us consider the case for which a < 0 and b > 0, namely

+

a=-

W

,- - = b

G-2

In that case X1(Tf)

= Rw - I

R JI

- w2

148

VI

DIFFERENTIAL QUALITATIVE GAMES

and hence,

Consider a plane V,(C) given by We have

g ( x ) A x 2 - ax, - b = C g(x)

and


with

C

>0

v x ~ e ,

n ( x ) 2 grad g ( x ) = (-a, 1 , 0 ) = [ ( R c J l - w2))-l]X(~) Conditions (a) and (b) of Theorem 6.2 are satisfied with p(X)

= U(T) =

z(X)

= V(T) = 1

(-W2,

-WJl

- W')

and hence, according to Theorem 6.2, U,(C) is a semipermeable surface such that Vl(C) c comp S , for all C>0

By similar arguments one can see that planes U2(C),U3(C),and g4(C) given by x2-axl-b=C a>O,b
x 2 - ax, - b = C

a

> 0,b > 0, C >

respectively, are semipermeable surfaces such that V2(C) E comp S , V3(C) E comp S ,

V4(C)E comp S ,

I

for all

C <0

for all

C

>0

0

6.18

EXAMPLE

149

6.2 \

x2

/\

M"

N"

FIG.6.7. Example 6.2. Projection on the x1-x2 plane of a domain of E3that contains set S,. (&: projection of target ep).

One can prove that the closed shaded area of Fig. 6.7 is the projection of a set of F that contains S,, whereas the open unshaded area is the projection of a set of E3 contained in S,. Indeed, S , and S, depend on 9 p and 9 ,. We recommend that the reader compare the treatment of this example to the one of Example 3.1.

7 A Connection between Qualitative and Quantitative Games

7.1 PROBLEM STATEMENT, DEFINITIONS OF GAMES 1, 2, AND 3

Definition 7.1. As in Chapter 6, let us consider a qualitative game in E n with state equation dx/dT = f ( ~ U,, U ) (7.1)

where x L2 ( x l , x,, . . . , xn). The definition of the game, the notation and the assumptions will be the same as those in Chapter 6 , except for the fact that we shall now focus our attention on the case where 8, # 0 and OE = 0.Such a game will be termed Game 1. Now let us consider a new variable x,, and the equation (7.2)

dX,/dT = f o ( x , u, v )

wheref, is of class C1 on G x U x V . We shall require that the sets of strategies 9,and 9,satisfy Assumption 3.3 and, as in Chapter 3, we shall write V(xi, xs; p , e, T?‘) f

I”

&(x(T), u(T),v(T))dT

where x: T x = x ( T ) , T E [ O , T , ] , represents a path risin G, x(0) = x * , X(TJ = xs, and U ( T ) = ~ ( x ( T ) V ) (, T ) = e(x(T)). Equations (7.1) and (7.2) constitute a set of n 1 scalar equations which we shall write in vector form ---f

+

dx/dT = f(x,

U,U )

(7.3)

7.1

PROBLEM STATEMENT, DEFINITIONS OF GAMES

where x

1, 2,

AND

3

151

(x,, xl,. . . , xn), and f(x, % 4 2 ( f o ( x ,u , ~ ) , f i ( X , u, v ) ,

Let En+l 2 En x {x,}, 9 A

o,(c) L {x:

. . . ,fn(x, a, u)). G x {x,}, 0 , A 8 , x {x,} and

x =

(x,, x),

E

e,,

xo

< c}

(7.4)

where C is a given constant. As the value of parameter C i s varied, Eq. (7.4) defines a one-parameter family of targets, namely {O,(C)}. 9 and 0, are x,-cylindrical sets whose projections on E nare G and O,, respectively. Definition 7.2. The set 9,the prescribed sets of strategies 9,and Y E , the state equation (7.3) and (i) Or in one case; (ii) 0,(C) in the other case; and the usual prescriptions for Jp and J E , define two single-target qualitative games in En+l.The games corresponding to (i) and (ii) will be termed Game 2 and Game 3 , respectively.

Note that the rules are the same for Game 2 and Game 3. Let xi be a point of 9,and xi its projection on G . We shall let II” denote a path in 9 from xi to xs, and risa path in G from xi to xs. Let IIif be a path in 9 that emanates from x i , is generated by ( p , e ) , and reaches 0, at point x f . Its projection on G is a path rifthat emanates from xi,is generated by ( p , e ) , and reaches 0, at xf,the projection on G of X f . Conversely, with each path rifin G that emanates from xi,is generated by ( p , e ) , and reaches O p at point xf, there is associated a path I I i f in 9 that emanates from xi,is generated by ( p , e ) , and reaches 0 , at point xf. As before, let rif be represented by function x : T + X = x ( T ) , T E [0, T ~ ] then ; to rifthere corresponds rIif

(

= x:

x = (x,

x),

x = x(7),

xo = xd + L f o ( x ( t ) , u(E), ~(6))d5,

where In other words,

7rif

u(7) = p ( x ( 7 ) ) , V(7) = e ( x ( 4 ) is the projection on G of nif.

7

E

LO, ~~1

152

vii

CONNECTION BETWEEN QUALITATIVE AND QUANTITATIVE GAMES

I

I I

@P

I

I I

X-f

I

L E"

FIG.7.1. Targets 0,,

a,,

and O,(C); paths

and

+f.

It follows that, if ( p , e ) is a playable pair (or a strongly playable pair) for Jp a t point x2 in Game 2, it is a playable pair (or a strongly playable pair) for J p at point x2 in Game 1, and conversely. We shall let T p ( x ' ) and Y I , ( x 2 )denote the sets of playable pairs for J p at point xz in Game 1 and at point xzin Game 2, respectively. In Game 3 , we shall let T p ( x e ; C ) denote the set of playable pairs for Jp at point x'. The sets of strongly playable pairs will be denoted by F p * ( x ' ) , Y p * ( x z ) , and Y p * ( x a ;C ) in Games 1, 2, and 3 , respectively. Likewise, if ( p , e ) is a playable pair for JE at point X~ in Game 2, it is a playable pair for J E at point x2in Game 1, and conversely. We shall let Y E ( x * )and F E ( x 2 )denote the sets of playable pairs for JE at point x' in Game 1 and at point x2in Game 2, respectively. We shall let F E ( x e ;C ) denote the set of playable pairs for J E at point xzin Game 3 , and Y E * ( x ' ) , Y E * ( x Z ) Y, E * ( x 2 ;C )the corresponding sets of strongly playable pairs in Games 1, 2, and 3 , respectively.

7.2

LOCAL SADDLE-POINT CONDITION, GAME

153

4

7.2 LOCAL SADDLE-POINT CONDITION, GAME 4

If (p,e)

E Tp(xi),we

shall write (7.6)

where x: T -+ x = x ( T ) , r E [0, T f ] , represents a path rifthat emanates from xi,is generated by (p, e) and reaches 8, at xf. The supporting curve of that path is r x ( x i , p ,e). Since, for given xi and (p,e)fYp(xi), rif need not be unique, V(xi,8,; p, e) need not be unique. We shalllet Vx(xi, 8,; p, e) designate the value of V(xi,O P ; p, e) defined by function x. Let xibe a point of S,, and 1etp;f be an optimal strategy for J p at point xi in Game 1 ; that is, Then

(j&

Ve E Y E

e) E Tp*(xi)

rx(xi,P : ~ ,e) n 8 , #

vx

0

E

y(xi,&, e),

(7.7) Ve E Y E

and, for x E y(xi,p& e), each rx(xi,p:g,e) is the supporting curve of one and only one &optimal path in G . Consider a member r$ of this set of paths. It follows from Lemma 5.1 that r$ c S,. The corresponding path IIif in S, x {xo}that emanates from xz = (xt,xi)and is given by (7.5) with

u ( 4 = p*,i(x(d),

v ( 4 = e(x(T))

reaches 0, at point x f = (xof, xf). By (7.5) and (7.6),

..

xgf = xi

+ vX(xi,8,;

p$, e)

It follows from the definition of O,(C) that

xof< c =+Xf

E 0,(C)

(7.9)

As a direct consequence of (7.7)-(7.9), the set of points xi,xi E S,, defined by

x6

+ Vx(xi,8,;

&, e)


has the following property :

Vx

E y(xi, pzi, e),

Ve E

9, (7.10)

154

CONNECTION BETWEEN QUALITATIVE AND QUANTITATIVE GAMES

VII

At every point

X'

of this set Ve

(&, e) E ,Fp*(xi;C )

E

9,

Hence, at every point X' of this set, p:, is an optimal strategy for player .Ip at that point in Game 3. Next we shall let S,(C) denote the set of all points x a at which there exists an optimal strategy for player J p in Game 3. Of course, the set defined by Eq. (7.10) is a subset of S,,(C). Definition 7.3. We shall say that maxe,9E V(xi,8,; defined, if there exists M such that

p$, e), xi E S,, is

(i) ~'(x', Or; p& e) S M vx E y(xi, p : ~ ,e), Ve E Y E and (ii) 3; E Y E , 38 E y(xi, &, C) such that V;(xi,8,; &, g) = M Then

A

max V ( x t 8,; p$, e ) = M BE.SP.?

Now we introduce Assumption 7.1. maxe,9E V(xi,8,; pzt, e) is defined for all xi E S,.

Then the condition

xgi Q C - max V ( x i , 8,; esYz

&, e),

xi E S ,

(7.11)

implies (7.10). Hence, the set of points defined by (7.11), namely a

R ( C ) = {x': xi = , : x (

.

x'), xa E S,, xd < C - max V(xi, 8,; p,*, e ) ] .

&€YE

(7.12)

belongs to S,(C): Next we introduce

NC)

Assumption 7.2. There exists V ( x i , 8,;

for all xi E S,, for all ( p , ?,i)

t Which may depend on x i .

(7.13)

SAC)

such that

E

&, e )

(7.14)

E F s , ( x i ) and for all values of

V ( x i ,OP;p,CXi).

p , zsi)

max V ( x i ,8,; eeYB

7.2

LOCAL SADDLE-POINT CONDITION, GAME

4

155

Let m(CZt) be the set of all strategies p such that ( p , Czi) Clearly, p:, E w ( C % i ) for all C*i.

E

Fp(xi).

Definition 7.4. We shall say that minpem(e,i)V(xi,0,; p , Czi), xi E S,, is defined, if there exists m such that (i)'

V(xi, 8 , ; p , Cz,)

>m

for a l l p E pz(C,i) and for all values of V(xi,8,; p , Czi); and (ii)' there exists E W(2%i) and a value of V(xi,8,; fi, CZi) that is equal to m. m. Then minpem(iza) V(xi,O F ; p , Cxi) It follows from (7.14) that condition (i)' is satisfied at all points xi E S,. In view of (7.14) and the definition of maxEYE V(xi, 0,; pzt, e ) , ~ ( x ' ,0,; p ,

> max V(xi, 8,; ecYE

~ % i )

p 3 , e) > V(xi,0,; p z f , e )

for all xi E S,, for all p E w ( C z i ) , for all e E Y E and , for all values of V(xi,1 9 ~p ;, C,i) and V(xi,8,;p:t, e ) ; hence,

v(xi,8,; p>,

cZi)> max V(xi, 0,; eeY,

pzf, e)

> V(xi, 8,; p 3 ,

for all xi E S , and for all values of V(xi, 0 p ; p 3 , Css). We conclude that minDem(zzi) V(xi,8,; p , C,i) and V(xi,0,; p,*I,PZi) are defined for all xi E S,. In other words, we reach the following conclusions: 1. If there is more than one path in S, starting at point xi,generated by (p:;, Czi), and terminating on f J P ,then all the values of V ( x i , 0,; pzi, Z,i) are equal; and

These results are embodied in Lemma 7.1. Let p,*,be an optimal strategy for player J p at point xi E Sp in Game I . If maxeEYEV(xi, 0 , ; p 3 , e) is dejned for all xi E S,, and i f there exists Csi E 9,such that

> max V(xi, 0,;

V(xi, 8,; p , 2%;)

eEYE

pzi, e )

156

CONNECTION BETWEEN QUALITATIVE AND QUANTITATIVE GAMES

VII

for all x 2 E S p ,f o r all ( p , ZZt) E F,(xz) andfor all values of V(x*,6 ,; then

p , Ex.),

( I ) V ( X ' ,81,; p,*t,2,") is dejined f o r all xz E S,;

and

(ill v/(xZ, 0,; p:, el Q 1/(xZ,6,; P,*., p,,) Q v(xz, 6,; P , 2,a)

f o r all e E Y Ef,o r all p E d ( f ? , f*o)r, all values of V(x', 0,;p,$, e ) and V(xz,O,,;p, exC),and f o r all xz E S,; and (iii) min v(xZ,oI,; p , ),2

p d ~ , z )

= max ~ ( x '8 ,;, eE .YE

p,:, e) = ~(x',8 ,;

* -

p,., ezs)

Condition (ii) is a local saddle-point condition. Now, if there exists p* E 9,that is optimal on S, in Game 1, if maxPEy, V ( x i , BP;p*, e) is defined for all xi E S , , and if there exists 2 E 9,such that V ( x i , Or; p , 2)

> max V(xi,6,; ee.YE

p*, e)

for all (p,2) E FI,(xi),for all values of V(xi,8 ,; p , 2) and for all xi E S,, then (ii) is identical with the saddle-point condition given in Chapter 2; namely, it becomes

V(xi,O,,;p*, e ) Q V(x2, 6,;p*,

2)

< v(xi,OP;p,2)

for all xi E S,, for all ( p , 2) E Y P ( x i )and for all values of V(x1,BP;p*, e ) and V(xi,0,; p, 2). From the point of view of quantitative games,

vw, 8,; p , 4 ,

(p, e) EFp(xi),

is the cost functional associated with a terminating path in S , that emanates from x i , is generated by strategy pair ( p , e), and reaches target 6 ,. Definition 7.5. The quantitative zero-sum, two-player game which has the same rules as qualitative Game 1, and whose objectives are defined in terms of 8,;t and V(xi,O,;p, e), will be called Game 4.

t That is, 0 = Op.

7.3

157

SURFACES OF THE GAME

In Game 4,p* and P are optimal strategies on S , for players Jp and J E , respectively. 7.3 SURFACES OF THE GAME

Let us consider points xi = (x;, x i ) such that xb = c - max

eeYE

or equivalently X :

+ v(x~,8,;

v(x',;0,

p,*i, e), xi E S,

pzi, Czi) = C , xi E S,

(7.15)

Definition 7.6. Equation (7.15) is the equation of a set of points xi in S , x {xo); that is, (7,15) defines a surface which we shall call ageneralized surface of the game k(C) in Game 4.

This surface has properties which are similar to those of a surface of the game Z(C). When functions pz, and Czi do not depend on xi E S,, then i ( C ) and C(C) coincide for all xi E Sp. Surface % ( C )depends on parameter C . Variable:x is the cost variable. Since V(xi, 8,;p;i, Ezi) does not contain the cost variable, Eq. (7.15) defines a one parameter family of surfaces, {i(C)}, such that any two surfaces of the family can be deduced from one another by a translation parallel to the cost axis. Now consider a path IIifthat emanates from xiE S, x {x,}, is generated by (p, Zzi) where p E W(C%i), and reaches 0, at point x f = (x,f, xf). We have

xgf =

.:

+

qXi,

ep; p , zz{)

for allp E w ( P z i ) and for all values of V(xi,8 , ; p , Psi). From (7.16) and from the fact that

ep; pzi, czi) = max v(xi,ep; p:i, esYE

~(x',

we deduce that xd

> C - V(xi, 8,; xgi

+

el,

* -

psi, e,i) V(X' 8,; p, Czi)

>C

for allp E w ( C z i ) and for all values of V(xi,O,;p, Esi).

(7.17)

158

VII

CONNECTION BETWEEN QUALITATIVE AND QUANTITATIVE GAMES

FIG.7.2. Generalized surface of the game in Game 4.

*

Hence, at every point x2 of the set A / X ( C )defined by A/i;(C) L2 (xZ:

x2 = (xgz,

XZ),

xz E

s,, X :

> c - v ( ~8,;( , &, a,.)}

vp

E

(7.18) there exists a strategy e , namely gZt, such that whatever the strategy p , ( p , CX.) is a playable pair for JE at that point in Game 3 ; that is, ( p , a,)

E F ~ ( X ZC ; )

Y,,

VX'

E

A/2(C)

(7.19)

Since player JE has no target of his own, it follows from the definition of a playable pair that relation (7.19) coincides with E Y,,

(7.20) Equation (7.19), and hence Eq. (7.20), is a consequence of (7.17) if p E w(Cxt),and it is trivial ifp $ w(exz). ( p , Pxt) E TE*(x*; C)

Vp

Vx2E A&C)

7.4

A CONNECTION BETWEEN PATHS IN GAMES

1

AND

159

4

Next we shall let S,(C) denote the set of all points xz at which there exists an optimal strategy for player JE in Game 3. It follows from (7.20) that A / k

= %(C)

(7.21)

Similarly, consider B/$(C) 22 { x i : xi = (x;,

.

.

x”.

xi E sp, X :

< C - V(xi, 8,;

p:i,

c2i)}

(7.22)

Since V(xi, Op;pzi,C 2 i ) = maxeEYEV(xi, OP;p,*i,e ) , B/?(C)

= R(C)

where R ( C ) is the set defined by (7.12). Then it follows from (7.13) that &C) = SP(C) (7.23) Furthermore,

A$(C) Hence,

~& ( A / Z ( C ) )u 2 ( C ) 5 A / i ; ( C ) c S E ( C ) ~-

B 3 ( C ) A (B/T(C>)u Z(C) 5 B / Z ( C )c S,(C) Z(C) = A ~ Z ( Cn ) B ~ Z ( Cc) _ _ _ _

~

_

_

s,(c) n s,(c)

_

(7.24)

But E(C) = S,(C) n S,(C) is the surface of the game in qualitative Game 3. Consequently, Z(C)C B(C) (7.25) 7.4 A CONNECTION BETWEEN PATHS IN GAMES 1 AND 4

Now we introduce Assumption 7.3. Strategies p:i and t?%i do not depend on xi E S p . We shall denote these strategies by p* and 6. Then, as pointed out before, (i) ( p * , e) is an optimal strategy pair on S , in Game 4; and (ii) Z(C) and C(C) coincide for all x i E S,. Let my be a &optimal path in Game 1 , emanating from xi E S p and generated by strategy pair (p*, 6). By Lemma 5.1, mg

= SP

160

VII

CONNECTION BETWEEN QUALITATIVE A N D QUANTITATIVE GAMES

Let II*(C) be an optimal path in Game 4, generated by strategy pair (p*, d ) , whose projection on G is T* = T ~ From . Corollary 2.1, with %* = s, x {xo},n*(c)= C(C). And so, finally, we arrive at Theorem 7.1. If& is an optimal srrategy for player Jp at point xi E S , in Game 1; ifthere exist a cost V ( Y , 8,; p$, e ) satisfying Assumption 7.1 and a strategy dzt E 9,satisfying Assumption 1.2; fi Assumption 7.3 is satisfled, that is, if^:^ = p* and Fzt = Z do not depend on xz; then every 9-optimalpatlz xB in Game 1 , emanating from x a andgenerated by strategy pair (p*, d ) , is the projection on S , of an optimal path fI*(C) in Game 4 , generated by strategy pair (p*, e*), e+ = i?.

FIG.7.3. Connection between paths in Games 1 and 4. Game 1 is Example 6.2; Game 4 is Example 3.1 ; C(C) is game surface in the xo-xl-xz space, for Example 3.1. Sections of C(C) by planes parallel to x1-x2 are shown on this figure.

7.4

A CONNECTION BETWEEN PATHS IN GAMES

1

AND

4

161

Of course, there may be more than one quantitative Game 4 associated with a given qualitative Game 1 . Theorem 7.1 is illustrated by Examples 6.2 and 3.1. We shall let the reader verify that if Example 6.2 is Game 1, then Example 3.1 is a Game 4 associated with it. The corresponding game surface X ( C ) is pictured on Fig. 7.3 in the xo-x1-x2space. This figure should be compared to Figs. 3.9 and 6.7.

This page intentionally left blank

Appendix

Let G be a domain in En;that is, an open connected set in the topology of En. The topology induced by En on G (i.t.) is defined by the following

Axiom. An open set in the i.t. is a subset of G which is open in E" A subset of G is closed in the i.t. if its complement in G is open. It follows that 1'. G i s closed in the i.t. 2'. 0 is closed in the i.t. Let S be a subset of G . Point x is an interior point of S in the i.t. if S a n d there exists an open set in the i t . which contains x. The interior of S in the i.t. is S A {x: x is an interior point of S } It follows that

x

E

Sc

s

Let X be a subset of G, and let Y = comp X where comp X means the complement of X in G . The closure of X in the i.t. is L comp

x

f

164

QUANTITATIVE A N D QUALITATIVE GAMES

It follows that X G

x n

comp X = comp A'

comp Y = comp Y The boundary of X in the i.t. is

ax2XncompX We have

ax = X n comp X i u ax = i u (X n comp i ) = (2 u X) n (iu comp i ) =X

It follows that

comp

n ( X u comp X )

(iu 3x1= comp x u (comp i n comp comp i ) = comp X u ( X n comp X )

Furthermore, we have Hence Hence

XncompX=

o

comp ( X u ax)= comp X

x=xuax

165

REFERENCES

REFERENCES On the Geometry of Optimal Processes 1. A. BLAQUIERE AND G . LEITMANN, On the Geometry of Optimal Processes, Parts I, TI, 111, Univ. California, Berkeley, IER Repts. AM-64-10, AM-65-11, AM-66-1. 2. G. LEITMANN, Some Geometrical Aspects of Optimal Processes, SZAM J. Control 3, 53 (1965). 3. A. BLAQUIERE, Further Investigation into the Geometry of Optimal Processes, SZAMJ. Control 3, 19 (1965). 4. A. BLAQUIERE AND G. LEITMANN, Some Geometric Aspects of Optimal Processes, Part I : Problems with Control Constraints, Proc. Congr. Automatique Thioorique, Paris 1965. Dunod, Paris, 1967. 5. K. V. SAUNDERS AND G. LEITMANN, Some Geometric Aspects of Optimal Processes, Part 11: Problems with State Constraints, Proc. Congr. Automatique Thiorique, Paris 1965. Dunod, Paris, 1967. 6. A. BLAQUIERE AND G. LEITMANN, On the Geometry of Optimal Processes, in “Topics in Optimization” (G. Leitmann, ed.), pp. 265-371. Academic Press, New York, 1967. 7. H. HALKIN,The Principle of Optimal Evolution, in “Nonlinear Differential Equations and Nonlinear Mechanics” (J. P. LaSalle and S. Lefschetz, eds.). Academic Press, New York, 1963. 8. H. HALKIN,Mathematical Foundations of System Optimization, in “Topics in Optimization” (G. Leitmann, ed.), pp. 198-260. Academic Press, New York, 1967. 9. E. ROXIN,A Geometric Interpretation of Pontryagin’s Maximum Principle, in “Nonlinear Differential Equations and Nonlinear Mechanics” (J. P. LaSalle and S. Lefschetz, eds.). Academic Press, New York, 1963. 10. R. E. BELLMAN, “Dynamic Programming.” Princeton Univ. Press, Princeton, New Jersey, 1957. 11. G. LEITMANN, “An Introduction to Optimal Control.” McGraw-Hill, New York, 1966. 12. A. BLAQUIERE AND G. LEITMANN, Further Geometric Aspects of Optimal Processes: Multiple-Stage Dynamic Systems, in “Mathematical Theory of Control” (A. V. Balakrishnan and L. W. Neustadt, eds.). Academic Press, New York, 1967.

On The Theory of Games 13. J. VON NEUMANN AND 0. MORGENSTERN, “Theory of Games and Economic Behavior.” Princeton Univ. Press, Princeton, New Jersey, 1953. 14. J. MCKINSEY, “An Introduction to the Theory of Games.” McGraw-Hill, New York, 1952. 15. L. ZADEH,Optimality and Nonscalar-valued Performance Criteria, IEEE Trans. Automatic Control AC-8, 59 (1963). 16. R. ISAACS, “Differential Games.” Wiley, New York, 1965. 17. L. D. BERKOVITZ, A Variational Approach to Differential Games, in “Advances in Game Theory,” pp. 127-174. Princeton Univ. Press, Princeton, New Jersey, 1964.

166

QUANTITATIVE AND QUALITATIVE GAMES

18. L. D. BERKOVITZ AND W. H. FLEMING, On Differential Games with Integral Payoff, in “Contributions to the Theory of Games 111,” pp. 413-435. Princeton Univ. Press, Princeton, New Jersey, 1957. Necessary Conditions for Optimal Strategies in a Class of 19. L. D. BERKOVITZ, Differential Games and Control Problems, SIAM J . Control 5, 1 (1967). 20. D. L. KELENDZHERIDZE, A Pursuit Problem, in “The Mathematical Theory of Optimal Processes.” Wiley (Interscience), New York, 1962. 21. L. S. PONTRYAGIN, On Some Differential Games, SIAM J. Control 3, 49 (1965). 22. L. S. PONTRYAGIN, On the Theory of Differential Games, Uspehi Mat. Nauk 21, 219 (1966). 23. E. F. MISHCHENKO AND L. S. PONTRYAGIN, Linear Differential Games, Dokl. Akad. Nauk SSSR 174, 27 (1967) [English transl.]: Soviet Math. Dokl. 8, 585 (1967). 24. Y. C. Ho, A. E. BRYSON, AND S. BARON, Differential Games and Optimal PursuitEvasion Strategies, IEEE Trans. Automatic Control AC-10,385 (1965). 25. S. BARON,Differential Games and Optimal Pursuit-Evasion Strategies. Doctoral dissertation in Applied Mathematics, Harvard University, Cambridge, Massachusetts 1966. 26. E. N. SIMAKOVA, Differential Games (a survey paper), Automat. i Telemeh. 27, 161 (1966). 27. I. G. SARMA AND R. K. RAGADE, Some Considerations in Formulating Optimal Control Problems as Differential Games, Internat. J . Control 4, 265 (1966). 28. A. KAUFMANN, “Graphs, Dynamic Programming, and Finite Games.” Academic Press, New York, 1967. 29. G. LEITMANN AND G. MON, Some Geometric Aspects of Differential Games, 1.Astronaut. Sci. 14, 56 (1967). 30. G . LEITMANN A ND G. MON,On a Class of Differential Games, in “Proceedings of the Colloq. Advanced Problems and Methods for Space Flight Optimization, University of Liege, 1967” (B. Fraeijs de Veubeke, ed.) Pergamon Press, Oxford, 1968. 31. G . MON, Differential Game Theory: A Geometric Approach. Unpublished doctoral dissertation, Univ. California, Berkeley, 1967. 32. A . BLAQUIERE AND G. LEITMANN, “Jeux Quantitatifs, MCmorial des Sciences MathCmatiques.” Gauthier-Villars, Paris, 1969. 33. G. WANGAND G. LEITMANN, Necessary and sufficient Conditions for TwoPerson, Zero-Sum Multistage Games, J. Optim. Theory Appl. (1969, in press). 34. A. BLAQUIERE A N D G. LEITMANN, Multistage Quantitative Games, in “Proceedings of 2nd Hawaii International Conference on System Sciences, 1969,” p. 579. Western Periodicals Co., North Hollywood, California. AND N. GANI,Multistage Quantitative Games with Unprescribed 35. A. BLAQUIERE Terminal Stage, Comp. Rend. Acad. Sci. Paris, 268, 428 (1969). 36. A. BLAQUIERE A N D G. LEITMANN, Multistage Quantitative Games with Unprescribed Terminal Stage, Internat. J . Non-Linear Mech. (1969, in press).

On Qualitative Games 37. A. BLAQUIERE AND F. GERARD, “On the Geometry of Optimal Strategies in TwoPerson Games of Kind.” Mehanika, Moscow, 1968. 38. A. BLAQUIERE A N D F. GERARD, On the Geometry of Optimal Strategies in TwoPerson Games of Kind, J. Computer System Sci. Vol. 2, No 3, October 1968.

REFERENCES

167

39. F. GERARD, Theorie gkomktrique des jeux diffkrentiels qualitatifs a deux joueurs. Doctoral dissertation, Facultt des Sciences de Paris, 1968.

On Diferential Equations and System Theory 40. E. A. CODDINGTON AND N. LEVINSON, “Theory of Ordinary Differential Equations.” McGraw-Hill, New York, 1955. AND R. CONTI,“Non-Linear Differential Equations.” Pergamon 41. G. SANSONE Press, Oxford, 1964. 42. W. A. PORTER,“Modern Foundations of Systems Engineering.’’ Macmillan, New York, 1966.

This page intentionally left blank

Subject Index

A A point, 14, 17, 77, 79 Additivity of cost, 17, 19 Adjoint equation, 25, 27, 37-39, 41, 42, 52-54, 58, 62, 114, 115, 126, 131, 134, 135, 140, 142, 144, 146 difference equation, 90, 96, 97 Adjoint vector, 27, 92, 125 Autonomous game, 132

B B point, 14, 18, 77, 79 Boundary path, 119, 124, 131, 134, 143, 145, 147 C Constraint, 49, 52, 53, 58, 62, 94, 97, 134, 143, 145 Control variables, I , 23, 71, 97, 113 Convexity, 8 1 x, directional, 80 Cost, 11, 12, 18, 24, 58, 71, 74, 97, 156, 157 additivity of, 17, 19 Crossing of discontinuity manifold, 46, 60, 66

D Decomposition of domain, 42, 52, 67 of Z(C), 43 Describing curve, 4, 8, 29, 74, 104, 117, 136, 138

Differential games qualitative, 113 quantitative, 21 Discontinuity manifold, 42, 43, 47, 48, 52, 56, 60, 66 Dynamic programming, 42

E E optimal, 17, 70 & path, 106 optimal, 106 Extreme future time 3, 30 G Game, 1 autonomous, 132 game 1, 150, 155, 156, 159, 160 game 2, 150 game 3, 150, 154, 158, 159 game 4, 153, 156, 159, 160 map of, 103 multistage, 71 objectives of, 2, 9, 113 qualitative, 9, 103, 110, 113, 150 example of, 143, 145 quantitative, 9, 21, 71, 110, 150 example of differen~ial,53, 58, 61 of multistage, 96 rules of, 2, 3, 23, 74, 113, 151, 156 set of, 104, 105, 106, 116 value of, 12, 20, 56, 60, 77 zero sum, 10 surface in qualitative game, 107, 108, 112, 118, 123, 159 in quantitative game, 13, 17, 31, 32, 39, 76-78, 84, 93, 157

169

170

SUBJECT INDEX

H

0

%-function in differential qualitative games, 131, 143, 146 in quantitative games 41, 52, 543 583 62 multistage, 93, 97 Hamiltonian, see N-function

Objectives, 2, 9, 113, 156 Optimal pair in quantitative game, 12, 20, 25.. 44.. 52. 53, 67, 75, 83, 93, 145,

497

I Index, see also Cost performance, 11, 24, 58, 74 Jnduced topology, 163 Interior point, 32, 35, 44, 78, 83 lsovalue surface. 13

J Joining of paths, 8, 17, 29 of trajectories, 8 describing curves and, 8 Jump condition, 46, 47, 48, 60 min-max principle with, 52

K Kuhn-Tucker conditions, 49, 95, 135

I, Lagrange multiplier rule, 49, 95, 135 Linear transformation, 28, 37, 86, 87

M Manifold discontinuity, 42, 43, 47, 48, 52, 56, 60, 66 Map of game, 103 Maximal solution, 3, 29, 74 Min-max principle in differential qualitative games, 131 in differential quantitative games, 41 with jump condition, 52 local condition, 123 in multistage quantitative games, 93 along path in semipermeable surface, 142 Multiplier, Lagrange, rule, 49, 95, 135 Multistage quantitative game, 71

1<7

1 2 1 )

1co ' 4 ,

Optimal path, 17, 25, 30, 33, 36, 37, 39, 41, 43, 44, 46, 47, 78, 81, 83, 84, 87. 88, 93, 160 regular, 33, 36, 37, 41, 83, 88 Optimal strategy in qualitative game, 104, 105, 114, 153, 154, 155, 159, 160

P P optimal, 17, 69

I'

path, 106, 118, 121 optimal, 106, 153, 159, 160 Pair optimal, 12, 20, 25, 44, 52, 53, 67, 75, 83, 93, 145, 157, 159 playable, 5, 7, 12, 104, 107, 152, 158 strategy, 2, 5 , 7, 12, 34, 73, 79, 97, 104, 122, 125, 131, 139, 142, 143, 145, 152, 156, 158 well behaved, 67 Path, 3, 4, 8, 36, 53, 74, 119, 150, 159 in augmented state space, 15, 16, 31, 75, 76, 97, 151 boundary, 119, 124, 131, 134, 143, 147 E-optimal, 17, 70 I , 106 &-optimal, 106 joining of, 8, 17, 29 null, 4, 24, 75, 123 optimal, 17, 25, 30, 33, 36, 37, 39, 41, 43, 44, 46, 47, 78, 81, 83, 84, 87, 88, 93, 160 P-optimal, 17, 69 p, 106, 118, 121 P-optimal, 106, 153, 159, 160 in semipermeable surface, 139, 142 terminating, 5, 156 Performance index, 11, 24, 58, 74 Play, 1, 5, 56, 60, 71, 77, 98 terminating, 5, 9, 156 Playable strategy pair, 5, 7, 12, 104, 107, 152, 158 Player, 1, 21, 71, 104, 111, 113, 154, 155, 157

171

SUBJECT INDEX

Principle, Min-Max, 41, 52, 93, 123, 131, 142 Programming, dynamic, 42

Q Qualitative game, 9, 103, 110, 113, 150 differential, 113, 150 Quantitative game, 9, 21, 71, 110, 150 differential, 2 1 multistage, 71 R

Regular point, 32, 35, 44, 81, 1 I0 Regular optimal path, 33, 36, 37, 41, 83, 88 portion of, 43 Rules, 2, 3, 23, 74, 113, 151, 156

S Saddle-point, 12, 19, 156 local, condition, 153, 156 Semipermeable surface, 135, 137, 139, 140, 142, 148 Set of game, 104, 105, 106, 116 Qp, 78, 81, 99, 102 It,, 78, 83, 99, 102 Stage, 71, 78, 89, 97, 98 State equation, 21, 36, 42, 72, 85, 89, 132, 150 space, I , 72, 74 augmented, 13, 15, 16, 31, 54, 75 transfer of, 11 vector, 1, 71, 113 Strategy, 2, 21, 41, 72, 113, 139 optimal, in qualitative game, 104, 105, 114, 153, 154, 155, 159, 160 Strategy pair, 2, 5, 7, 12, 34, 73, 79, 97, 104, 122, 125, 131, 139, 142, 143, 145, 152, 156, 158 optimal, in quantitative game, 12, 20, 25, 44, 52, 53, 67, 75, 83, 93, 145, 157, 159 playable, 5, 7, 12, 104, 107, 152, 158 well behaved, 67 Strongly playable, 7, 104, 107, 152 Sufficiency conditions, 67, 93

Supporting curve, 4, 36, 153 Surface Game in qualitative game, 107, 108, 112, 118, 123, 159 in quantitative game, 13, 17, 31, 32, 39, 76, 77, 78, 84, 93, 157 isovalue, 13 Z,(C), 77, 81,99, 100, 101 semipermeable, 135, 137, 139, 140, 142, 148

T Tangent plane, 36, 40, 45, 47, 81, 88, 141 Target, 2, 23, 53, 58, 62, 72, 113, 127, 143, 145, 151, 156 in augmented state space, 15 time independent, 132, 133 useable part of, 55, 58, 64 Time-optimal, 53, 62 Trajectory, 3, 4, 8, 29, 36, 118 Transfer time, 22 Transversality condition, 39, 40 for differential qualitative games, 126 for quantitative games differential, 54, 58, 62 multistage, 93 for semipermeable surface, 140

U Useable part of target, 55, 58, 64

V Value of game, 12, 20, 56, 60, 77, 98 Variational difference equation, 84, 86, 91 Variational equation, 25, 26, 36, 114, 115, 129 Verification theorem, 70 W Weakly playable, 5, 104, 152 Well-behaved, 67 Winner, 9, 111

X X,-directional convexity, 80

List of Assumptions 1.1 2.1 2.2 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.1 1

4.1 4.2 4.3 4.4 6.1 6.2 6.3 6.4 6.5 6.6 7.1 7.2 7.3

9 11 17 22 23 24 25 30 30 33 33 43 43 44

75 81 83 84 119 119 123 127 127 139 154 154 159

List of Corollaries 2.1 2.2 2.3

19 19 19

2.4 6.1

20 118

List of Lemmas 2.1 2.2 2.3 4.1 4.2 4.3 4.4 5.1

5.2 5.3 5.4 5.5 6.1 6.2 6.3 7. I

17 18 20 84 87 88 90 106

106 108 109 109 116 118 124 155

List of Theorems 3.1 3.2 3.3 4.1 6.1

41 52 67 93 131

6.2 6.3 6.4 6.5 7.1

172

135 137 138 142 160

Mathematics in Science and Engineering ~~~~~

A Series of Monographs and Textbooks E d i t e d by

RICHARD BELLMAN, University o f Southern California

1. T. Y. Thomas. Concepts from Tensor Analysis and Differential Geometry. Second Edition. 1965 2. T. Y. Thomas. Plastic Flow and Fracture in Solids. 1961 3. R. Ark. The Optimal Design of Chemical Reactors: A Study in Dynamic Programming. 1961 4. J. LaSalle and S. Lefschetz. Stability by by Liapunov's Direct Method with Applications. 1961 5. G. Leitmann (ed.). Optimization Techniques: With Applications t o Aerospace Systems. 1962 6. R. Bellman and K. L. Cooke. DifferentialDifference Equations. 1963 7. F. A. Haight. Mathematical Theories of Traffic Flow. 1963

8. F. V. Atkinson. Discrete and Continuous Boundary Problems. 1964 9. A. Jeffrey and T. Taniuti. Non-Linear Wave Propagation: With Applications to Physics and Magnetohydrodynamics. 1964

19. J. Aczel. Lectures on Functional Equations and Their Applications. 1966 20. R. E. Murphy. Adaptive Processes in Economic Systems. 1965 21. S. E. Dreyfus. Dynamic Programming and the Calculus of Variations. 1965 22. A. A. Fel'dbaum. Optimal Control Systems. 1965 23. A. Halanay. Differential Equations: Stability. Oscillations, Time Lags. 1966 24. M. N. Oguztoreli. Time-Lag Control Systems. 1966 25. D. Sworder. Optimal Adaptive Control Systems. 1966 26. M. Ash. Optimal Shutdown Control of Nuclear Reactors. 1966 27. D. N. Chorafas. Control System Functions and Programming Approaches (In Two Volumes). 1966 28. N. P. Erugin. Linear Systems of Ordinary Differential Equations. 1966

29. S. Marcus. Algebraic Linguistics; Analytical Models. 1967

10. J. T. Tou. Optimum Design of Digital Control Systems. 1963.

30. A. M. Liapunov. Stability of Motion.

11. H. Flanders. Differential Forms: With Applications t o the Physical Sciences. 1963

31. G. Leitmann (ed.). Topics in Optimization. 1967

12. S. M. Roberts. Dynamic Programming in Chemical Engineering and Process Control.

32. M. Aoki. Optimization of Stochastic Systems. 1967

1964

33. H. J. Kushner. Stochastic Stability and control. 1967

13. S. Lefschetz. Stability of Nonlinear Control Systems. 1965 14. D. N. Chorafas. Systems and Simulation.

1965 15. A. A. Pervozvanskii. Random Processes in Nonlinear Control Systems. 1965 16. M. C. Pease, 111. Methods of Matrix Algebra. 1965 17. V. E. Benes. Mathematical Theory of Connecting Networks and Telephone Traffic.

1965 18. W. F. Ames. Nonlinear Partial Differential Equations in Engineering. 1965

1966

34. M. Urabe. Nonlinear Autonomous Oscillations. 1967 35. F. Calogero. Variable Phase Approach t o Potential Scattering. 1967 36. A. Kaufrnann. Graphs, Dynamic Programming, and Finite Games. 1967 37. A. Kaufmann and R. Cruon. Dynamic Programming: Sequential Scientific Management. 1967

38. J. H. Ahlberg, E. N. Nilson, and J. L.

Walsh. The Theory of Splines and Their Applications. 1967

39. Y. Sawaragi, Y. Sunahara. and T. Nakamizo. Statistical Decision Theory in Adaptive Control Systems. 1967

In preparation R. Bellman. Methods of Nonlinear Analysis, Volume I

40. R. Bellman. Introduction t o the Mathematical Theory of Control Processes Volume I. 1967 (Volumes II and Ill in preparation)

R. Bellman, K. L. Cooke. and J. A. Lockett. Algorithms, Graphs, and Computers

41. E. S. Lee. Quasilinearization and Invariant Imbedding. 1968

A. H. Jazwinski. Stochastic Processes and Filtering Theory

42. W. Ames. Nonlinear Ordinary Differential Equations in Transport Processes. 1968

S. R. McReynolds and P. Dyer. The Computation and Theory of Optimal Control

43. W. Miller, Jr. Lie Theory and Special Functions. 1968

J. M. Mendel and K. S. Fu. Adaptive, Learning. and Pattern Recognition Systems: Theory and Applications

44. P. B. Bailey, L. F. Shampine. and P. E. Waltman. Nonlinear Two Point Boundary Value Problems. 1968.

G. Rosen. Formulations of Classical and Quantum Dynamical Theory

45. Iu. P. Petrov. Variational Methods in Optimum Control Theory. 1968 46. 0. A. Ladyzhenskaya and N. N. Ural’tseva. Linear and Quasilinear Elliptic Equations. 1968

47. A. Kaufmann and R. Faure. Introduction to Operations Research. 1968 48. C. A. Swanson. Comparison and Oscillation Theory of Linear Differential Equations.

1968 49. R. Hermann. Differential Geometry and the Calculus of Variations. 1968 50. N. K. Jaiswal. Priority Queues. 1968 51. H. Nikaido. Convex Structures and Economic Theory. 1968

52. K. S . Fu. Sequential Methods in Pattern Recognition and Machine Learning. 1968 53. Y. L. Luke. The Special Functions and Their Approximations (In Two Volumes). 1969 54. R. P. Gilbert. Function Theoretic Methods in Partial Differential Equations. 1969

55. V. Lakshrnikantham and S . Leela. Differential and Integral Inequalities (In Two Volumes.) 1969 56. S . H. Hermes and J. P. LaSalle. Functional Analysis and Time Optimal Control.

1969. 57. M. Iri. Network Flow, Transportation, and Scheduling: Theory and Algorithms. 1969 58. A. Blaquiere, F. Gerard, and G. Leitmann. Quantitative and Qualitative Games.

1969 59. P. Falb and J. De Jong. Successive Approximation Methods in Control and Oscillation Theory. 1969

E. J. Beltrami. Methods of Nonlinear Analysis and Optimization H. H. Happ. The Theory of Network Diakoptics