New Trends in Nonlinear Dynamics and Control, and their Applications (Lecture Notes in Control and Information Sciences)

Lecture Notes in Control and Information Sciences Editors: M. Thoma · M. Morari 295 Springer Berlin Heidelberg NewYo...

Author: Wei Kang | Mingqing Xiao | Carlos Borges

45 downloads 794 Views 3MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

Lecture Notes in Control and Information Sciences Editors: M. Thoma · M. Morari

295

Springer Berlin Heidelberg NewYork Hong Kong London Milan Paris Tokyo

Wei Kang Mingqing Xiao Carlos Borges (Eds.)

New Trends in Nonlinear Dynamics and Control, and their Applications With 45 Figures

13

Series Advisory Board A. Bensoussan · P. Fleming · M.J. Grimble · P. Kokotovic · A.B. Kurzhanski · H. Kwakernaak · J.N. Tsitsiklis

Editors Prof. Wei Kang Prof. Carlos Borges Naval Postgraduate School Dept. of Mathematics 93943 Monterey, CA USA Prof. Mingqing Xiao Southern Illinois University Dept. of Mathematics 62901-4408 Carbondale, IL USA

ISSN 0170-8643 ISBN 3-540-40474-0

Springer-Verlag Berlin Heidelberg New York

Cataloging-in-Publication Data applied for A catalog record for this book is available from the Library of Congress. Bibliographic information published by Die Deutsche Bibliothek Die Deutsche Bibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data is available in the Internet at . This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in other ways, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution under German Copyright Law. Springer-Verlag Berlin Heidelberg New York a member of BertelsmannSpringer Science + Business Media GmbH http://www.springer.de © Springer-Verlag Berlin Heidelberg 2003 Printed in Germany The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Data conversion by the authors. Final processing by PTP-Berlin Protago-TeX-Production GmbH, Berlin Cover-Design: design & production GmbH, Heidelberg Printed on acid-free paper 62/3020Yu - 5 4 3 2 1 0

Preface

The concept for this volume originated at the Symposium on New Trends in Nonlinear Dynamics and Control, and their Applications. The symposium was held October 18-19, 2002, at the Naval Postgraduate School in Monterey, California and was organized in conjunction with the 60th birthday of Professor Arthur J. Krener, a pioneer in nonlinear control theory. The symposium provided a wonderful opportunity for control theorists to review major developments in nonlinear control theory from the past, to discuss new research trends for the future, to meet with old friends, and to share the success and experience of the community with many young researchers who are just entering the ﬁeld. In the process of organizing this international symposium we realized that a volume on the most recent trends in nonlinear dynamics and control would be both timely and valuable to the research community at large. Years of research eﬀort have revealed much about the nature of the complex phenomena of nonlinear dynamics and the performance of nonlinear control systems. We solicited a wide range of papers for this volume from a variety of leading researchers in the ﬁeld, some of the authors also participated in the symposium and others did not. The papers focus on recent trends in nonlinear control research related to bifurcations, behavior analysis, and nonlinear optimization. The contributions to this volume reﬂect both the mathematical foundations and the engineering applications of nonlinear control theory. All of the papers that appear in this volume underwent a strict review and we would like to take this opportunity to thank all of the contributors and the referees for their careful work. We would also like to thank the Air Force Oﬃce of Scientiﬁc Research and the National Science Foundation for their ﬁnancial support for this volume. Finally, we would like to exercise our prerogative and thank many of the people involved with the symposium at this time. In particular, we would like to thank Jhoie Passadilla and Bea Champaco, the staﬀ of the Department of Applied Mathematics of the Naval Postgraduate School, for their support in organizing the symposium. Furthermore, we extend our special thanks to

VI

Preface

CAPT Frank Petho, USN, whose dedication to the core mission of the Naval Postgraduate School allowed him to cut through the bureaucratic layers. Without his vision and support the symposium might never have happened. Most importantly, we would like to express our deepest gratitude to the Air Force Oﬃce of Scientiﬁc Research and the National Science Foundation, for the ﬁnancial support which made the symposium possible.

Monterey, California, Early Spring, 2003

Wei Kang MingQing Xiao Carlos Borges

Contents

Part I Bifurcation and Normal Form Observability Normal Forms J-P. Barbot, I. Belmouhoub, L. Boutat-Baddas . . . . . . . . . . . . . . . . . . .

3

Bifurcations of Control Systems: A View from Control Flows Fritz Colonius, Wolfgang Kliemann . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Practical Stabilization of Systems with a Fold Control Bifurcation Boumediene Hamzi, Arthur J. Krener . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Feedback Control of Border Collision Bifurcations Munther A. Hassouneh, Eyad H. Abed . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Symmetries and Minimal Flat Outputs of Nonlinear Control Systems W. Respondek . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Normal Forms of Multi-input Nonlinear Control Systems with Controllable Linearization Issa Amadou Tall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Control of Hopf Bifurcations for Inﬁnite-Dimensional Nonlinear Systems MingQing Xiao, Wei Kang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

Part II System Behavior and Estimation On the Steady-State Behavior of Forced Nonlinear Systems C.I. Byrnes, D.S. Gilliam, A. Isidori, J. Ramsey . . . . . . . . . . . . . . . . . . 119

VIII

Contents

Gyroscopic Forces and Collision Avoidance with Convex Obstacles Dong Eui Chang, Jerrold E. Marsden . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Stabilization via Polynomial Lyapunov Function Daizhan Cheng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 Simulating a Motorcycle Driver Ruggero Frezza, Alessandro Beghi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 The Convergence of the Minimum Energy Estimator Arthur J. Krener . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 On Absolute Stability of Convergence for Nonlinear Neural Network Models Mauro Di Marco, Mauro Forti, Alberto Tesi . . . . . . . . . . . . . . . . . . . . . . 209 A Novel Design Approach to Flatness-Based Feedback Boundary Control of Nonlinear Reaction-Diﬀusion Systems with Distributed Parameters Thomas Meurer, Michael Zeitz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 Time-Varying Output Feedback Control of a Family of Uncertain Nonlinear Systems Chunjiang Qian, Wei Lin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 Stability of Nonlinear Hybrid Systems G. Yin, Q. Zhang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Part III Nonlinear Optimal Control The Uncertain Generalized Moment Problem with Complexity Constraint Christopher I. Byrnes, Anders Lindquist . . . . . . . . . . . . . . . . . . . . . . . . . 267 Optimal Control and Monotone Smoothing Splines Magnus Egerstedt, Clyde Martin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Towards a Sampled-Data Theory for Nonlinear Model Predictive Control Rolf Findeisen, Lars Imsland, Frank Allg¨ ower, Bjarne Foss . . . . . . . . . 295 High-Order Maximal Principles Matthias Kawski . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313

Contents

IX

Legendre Pseudospectral Approximations of Optimal Control Problems I. Michael Ross, Fariba Fahroo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 Minimax Nonlinear Control under Stochastic Uncertainty Constraints Cheng Tang, Tamer Ba¸sar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343

Observability Normal Forms J-P. Barbot, I. Belmouhoub, and L. Boutat-Baddas Equine Commando des Syst`emes (ECS), ENSEA, 6 Av. du Ponceau, 95014 Cergy-Pontoise Cedex, France, [email protected]

1 Introduction One of the ﬁrst deﬁnitions and characterizations of nonlinear observability was given in the well known paper of R. Hermann and A.J. Krener [18], where the concept of local weak observability was introduced and the observability rank condition was given. In [18], observability and controllability were studied with the same tools as those of diﬀerential geometry ([33]). Similarly to the linear case, some direct links between observability and controllability may be found. After this pioneering paper many works on nonlinear observability followed [41, 6]... One important fact, pointed out in the eighties, was the loss of observability due to an inappropriate input. Consequently, the characterization of appropriate input (universal input) with respect to nonlinear observability ([12]) was an important challenge. Since that time, much research has been done on the design of nonlinear observers. From our point of view, one of the ﬁrst signiﬁcant theoretical and practical contributions to the subject was the linearization by output injection proposed by A.J Krener and A. Isidori [30] for a single output system and by A.J Krener and W. Respondek [31] for the multi output case (see also X. Xia and W. Gao [45]). From these works and some other ones, dealing with structural analysis [24, 37, 13, 40, 20, 5] an important literature on nonlinear observer design followed. Diﬀerent techniques were studied: High gain [13, 25]..., Backstepping [23, 39]..., Extended Luenberger [7]..., Lyapunov approach [44]..., Sliding mode [42, 11, 46, 36, 3]..., Numerical diﬀerentiator [10]... and many other approaches. Some observer designs use partially or totally the notion of detectability. This concept will be used and highlighted in this paper in the context of observability bifurcation (see also the paper of A.J Krener and M.Q Xiao [32]). But what is the observability bifurcation? Roughly speaking this is the loss of the linear observability property at one point or on a submanifold. It is important to recall that the classical notion of bifurcation is dedicated to W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 3–17, 2003.

c Springer-Verlag Berlin Heidelberg 2003

4

J-P. Barbot, I. Belmouhoub, and L. Boutat-Baddas

stability properties. So, H. Poincar´e [38] introduced the normal form in order to analyze the stability bifurcation. The main idea behind this concept is to highlight the inﬂuences of the dominant terms with respect to a considered local property (stability, controllability, observability). Moreover, each normal form characterizes one and only one equivalent class. So, the structural properties of the normal form are also the same as those of each system in the corresponding equivalence class. Thus, if the linear part of the normal form has no eigenvalue on the imaginary axis, the system behavior is locally given by this linear part. If some eigenvalues are on the imaginary axis, the linear approximation does not characterize the local behavior and then higher order terms must be considered. In [27] A.J Krener has introduced the concepts of approximate feedback linearization and approximated integrability (see also [17] around a manifold). After that, W. Kang and A.J Krener introduced in [22] the deﬁnition of a normal form with respect to the controllability property, for this, they introduced a new equivalence relation. This relation is composed of a homogenous diﬀeomorphism, as in the classical mathematical context, and of a homogenous regular feedback. After that many authors worked on the subject of controllability bifurcation [21, 28, 29, 43, 15, 2, 14, 16]... In this paper, a new class of homogeneous transformations, by diﬀeomorphism and output injection, is used in order to study the observability bifurcation and deﬁne an observability normal form (in continuous and discrete time). The usefulness of this theoretical approach is highlighted with two examples of chaotic system synchronization. As a matter of fact, it was shown in [35] by H. Nijmeijer and I. Mareels that the synchronization problem may be rewritten and understood as an observer design problem. In the ﬁrst example, the sliding mode observer eﬃciency with respect to the observability bifurcation is highlighted ([9]). In the second example, a special structure of discrete time observer dedicated to discrete time system with observability bifurcation is recalled ([4]). The paper ends with a conclusion and some perspectives.

2 Some Recalls on Observability Let us consider the following system: x˙ = f (x); n

y = h(x) n

(1) n

m

where vector ﬁelds f : IR → IR and h : IR → IR are assumed to be smooth with f (0) = 0 and h(0) = 0. The observability problem arises as follows : can we estimate the current state x(t) from past observations y(s), s ≤ t , without measuring all state variables? An algorithm that solves this problem is called an observer. Motivated by the consideration that it is always possible to cancel all independent parts constituted only by the input and the output in the estimated error, the observer linearization problem was born. Is it possible to ﬁnd in a neighborhood U of 0 in IR n a change of state coordinates z = θ(x) such that dynamic (1) is linear driven by non linear output injection:

Observability Normal Forms

z˙ = Az − β(y).

5

(2)

where β : IR m → IR n is a smooth vector ﬁeld. Note that the output injection term β(y) is cancelled in the observation error dynamic for system (2). The diﬀeomorphism θ must satisfy the ﬁrst-order partial diﬀerential equation: ∂θ (x)f (x) = Aθ(x) − β(h(x)). ∂x

(3)

In [30] A.Krener and A. Isidori showed that equation (3) has a solution if and only if the following two conditions are satisﬁed: i) the codistribution span dh, dLf h, ..., dLn−1 h is of rank n at 0, f ii) τ, adkf τ = 0 for all k = 1, 3, ..., 2n − 1 where τ is the unique solution T T T T vector ﬁelds of (dh) , (dLf h) , ...., (dLn−1 h)T , τ = [0, 0, ....1] f

3 Observability Normal Form In this paper for the lack of space, we only give the normal form for a system with a linear unobservable mode in both continuous and discrete time case. 3.1 Continuous Time Case Let us consider a nonlinear Single Input Single Output (SISO) system: ξ˙ = f (ξ) + g(ξ)u;

y = Cξ

(4)

where, vector ﬁelds f, g : U ⊂ IR n −→ IR n are assumed to be real analytic, such that f (0) = 0. Setting: A = ∂f ∂ξ (0) and B = g(0) around the equilibrium point ξe = 0, the system can be rewritten in the following form: z˙ = Az + Bu + f [2] (z) + g [1] (z)u + O[3] (z, u) ;

y = Cz

(5)

T T [2] [2] [1] [1] where: f [2] (z) = f1 (z) , ..., fn (z) and g [1] (z) = g1 (z) , ..., gn (z) [2]

[1]

with for all 1 ≤ i ≤ n, fi (z) and gi (z) are respectively homogeneous polynomials of degree 2, respectively 1 in z. Deﬁnition 1. i) The component f [2] (z) + g [1] (z)u is the quadratic part of system (5). ii) Consider a second system: x˙ = Ax + Bu + f¯[2] (x) + g¯[1] (x)u + O[3] (x, u);

y = Cx

(6)

6

J-P. Barbot, I. Belmouhoub, and L. Boutat-Baddas

We say that system (5) whose quadratic part is f [2] (z) + g [1] (z)u, is Quadratically Equivalent Modulo an Output Injection to System (6) whose quadratic part is f¯[2] (x) + g¯[1] (x)u, if there exists an output injection: β [2] (y) + γ [1] (y)u

(7)

and a diﬀeomorphism of the form: x = z − Φ[2] (z)

(8) which carries f [2] (z) + g [1] (z)u to f¯[2] (x) + g¯[1] (x)u + β [2] (y) + γ [1] (y)u . T T [2] [2] [2] [2] Where Φ[2] (z) = Φ1 (z) , ......, Φn (z) , β [2] (y) = β1 (y) , ......, βn (y) [2]

[2]

and for all 1 ≤ i ≤ n, Φi (z) and β1 (y) are homogeneous polynomial in z T [1] [1] with respectively in y of degree two, and γ [1] (y) = γ1 (y) , ......, γn (y) [1]

γi (y) is a homogeneous polynomial of degree one in y. iii) If f¯[2] (x) = 0 and g¯[1] (x) = 0 we say that system (5) is quadratically linearizable modulo an output injection. Remark 1. If ∂f (0), C has one unobservable real mode then one can trans∂x form system (4) to the following form:   z˜˙ = Aobs z˜ + Bobs u + f˜[2] (z) + g˜[1] (z)u + O[3] (z, u) n−1 [2] [1] (9) z˙ = αn zn + i=1 αi zi + bn u + fn (z) + gn (z)u + O[3] (z, u)  n y = z1 = Cobs z˜ with:

z˜ = [ z1 · · · · · · zn−1 ]T , 

Aobs

a1  a2   =  ...   an−2 an−1

z = [˜ z T , zn ]T ,

   ... 0 b1 0 ...   ..    .   ..  , Bobs =  . 0   .   .  .  0 ... 0 1  bn−1 0 ... ... 0

10 01 . 0 ..

Remark 2. Throughout the paper, we deal with systems in form (9). Moreover, the output is always taken equal to the ﬁrst state component. Consequently, [2] the diﬀeomorphism (x = z − Φ[2] (z)) is such that Φ1 (z) = 0. Proposition 1. [8] System (5) is QEMOI to system (6), if and only if the following two homological equations are satisﬁed: i) AΦ[2] (z) − ii)

[2] − ∂Φ ∂z B

∂Φ[2] ∂z Az

=f

[2]

(z) − f [2] (z) + β [2] (z1 ) (10)

= g¯ (z) − g (z) + γ (z1 ) [1]

[1]

[1]

Observability Normal Forms

where

∂Φ[2] ∂z Az

:=

[2]

∂Φ1 (z) ∂Φ[2] n (z) Az, ......, ∂z Az ∂z

T

7

[2]

and

∂Φi (z) ∂z

is the Jacobian

[2]

matrix of Φi (z) for all 1 ≤ i ≤ n. Using proposition 1 and remark 1 we show the following theorem which gives the normal form for nonlinear systems with one linear unobservable mode. Theorem 1. There is a quadratic diﬀeomorphism and an output injection which transform system (9) in the following normal form:  n x˙ 1 = a1 x1 + x2 + b1 u + i=2 k1i xi u + O[3] (x, u)     .. .   . = ..   n  [3]   x˙ n−2 = an−2 x1 + xn−1 + bn−2 nu + i=2 k(n−2)i xi u + O (x, u) (11) x˙ n−1 = an−1 x1 + bn−1 u + j≥i=2 hij xi xj + h1n x1 xn n   [3]  + k x u + O (x, u)  i=2 (n−1)i i  n−1 n−1  [2] [2]   x ˙ = α x + i=1 αi xi + bn u + αn Φn (x) + i=1 αi Φi (x) n n n   [2]  [2] ∂Φ n ˜ + fn (x) + i=2 kni xi u + O[3] (x, u) − ∂ x˜n Aobs x For the proof see [8]. Remark 3. 1) If for some index i ∈ [1, n] we have hin xi = 0 then we can recover, at least locally, all state components. 2) If we have some kin = 0 then with an appropriate choice of input u (universal input [12]) we can have quadratic observability. 3) Thus, the local quadratic observability is principally given by the dynamic x˙ n−1 . In the case where conditions 1) and 2) are not veriﬁed, then we can use coeﬃcient αn to study the detectability propriety. Then, we have three cases: a) if αn < 0 then the state xn is detectable, b) if αn > 0 then xn unstable, and consequently undetectable, c) if αn = 0 we can use the center manifold theory in order to analyze stability or instability of xn and consequently its detectability or undetectability. Remembering the well known Poincar´e-Dulac theorem we have: [2]

Remark 4. If Φn (x) check the following equation: αn Φ[2] n (x) +

n−1 i=1

[2]

αi Φi (x) =

[2]

∂Φn Aobs x − fn[2] (x) + βn[2] (x1 ) ∂x

(12)

then quadratic terms in x˙ n are cancelled. Which in general is not the case for arbitrary αn and ai . Nevertheless, this condition is less restrictive than the [2] usual one thanks to the output injection βn (x1 ).

8

J-P. Barbot, I. Belmouhoub, and L. Boutat-Baddas

3.2 Discrete Time Case Now, let us consider a discrete time nonlinear SISO system: ξ + = f (ξ, u);

y = Cξ

(13)

where, ξ is the state of the system and ξ (respectively ξ + ) denote ξ(k) (respectively ξ(k + 1)). The vector ﬁelds f : U ⊂ IR n+1 −→ IR n and the function h : M ⊂ IR n −→ IR are assumed to be real analytic, such that f (0, 0) = 0. As for the continuous time case, we only give the observability normal form for a system with one linear unobservable mode. We apply, as usual, a second order Taylor expansion around the equilibrium point. Thus the system is rewritten as : + z = Az + Bu + F [2] (z) + g [1] (z)u + γ [0] u2 + O3 (z, u) (14) y = Cz T [2] [2] and where: F [2] (z) = F1 (z) , · · · , Fn (z) T [1] [1] and g [1] (z) = g1 (z) , · · · , gn (z) . From (14), we give with A =

∂f ∂x (0, 0),

B=

∂f ∂u (0, 0)

Deﬁnition 2. The system: z + = Az + Bu + F [2] (z) + g [1] (z)u + γ [0] u2 + O3 (z, u);

y = Cz

is said to be quadratically equivalent to the system:  +  x = Ax + Bu + F¯ [2] (x) + g¯[1] (x)u + γ¯ [0] u2 +β [2] (y) + α[1] (y)u + τ [0] u2 + O3 (x, u)  y = Cx modulo an output injection:

β [2] (y) + α[1] (y)u + τ [0] u2

if there exists a diﬀeomorphism of the form:

x = z − Φ[2] (z)

(15)

(16)

(17) (18)

which transforms the quadratic part of (15) into the one of (16). Remark 5. The output injection (17) is diﬀerent from the one deﬁned in (7) for the continuous-time case. This is due to the fact that the vector ﬁeld composition does not preserve the linearity in “u”; So we are obliged to consider the term τ [0] u2 in (17). In the next proposition, we give necessary and suﬃcient conditions for observability QEMOI:

Observability Normal Forms

9

Proposition 2. System (15) is QEMOI to system (16), if and only if there exist (Φ[2] , β [2] , α[1] , γ [0] ) which satisfy the following homological equations: i) F [2] (x) − F¯ [2] (x) = Φ[2] (Ax) − AΦ[2] (x) + β [2] (x1 ) [2] (Ax, B) + α[1] (x1 ) ii) g [1] (x) − g¯[1] (x) = Φ [0] [0] [2] iii) γ − γ¯ = Φ (B) + τ [0] [2] (Ax, B) = (Ax)T φ Bu + (Bu)T φ Ax, and φ is a vector of square where Φ symmetric matrix such that : φ =

2 [2] 1 ∂ Φ (x) 2 ∂x∂xT .

Now, in order to apply our study to a system with one unobservable mode, let us consider the system (4) where the pair (A, C) has one unobservable mode. Then there is a linear change of coordinates (z = T ξ) and a Taylor expansion which transforms the system (4) into the following form:  +  z˜ = Aobs z˜ + Bobs u + F˜ [2] (z) + g˜[1] (z)u + γ˜ [0] u2 + O3 n−1 [2] [1] [0] (19) z + = ηzn + i=1 λi zi + bn u + Fn (z) + gn (z)u + γn u2 + O3  n y = Cobs z Remark 6. The normal form which follows is structurally diﬀerent from the controllability discrete-time normal form, given in ([15], [14]), in the last state dynamics x+ n . For the observability analysis the main structural information is not in the x+ n dynamics but in the previous state evolution + [2] [1] xi for n − 1 ≥ i ≥ 1 . The terms λi xi , bn u, Fn (x), gn (x)u are only important in the case of detectability analysis when η = ±1. The quadratic normal form associated with the system (19) is given in the following theorem (see [4] for the proof). Theorem 2. The normal form with respect to the quadratic equivalence modulo an output injection of the system (19) is:  + n x1 = a1 x1 + x2 + b1 u + i=2 k1i xi u     .. . . = .. n  u + i=2 k(n−2)i xi u x+  n−2 = an−2 x1 + xn−1 + bn−2  n  + n xn−1 = an−1 x1 + bn−1 u + j>i=1 hij xi xj + hnn x2n + i=2 ki(n−1) xi u Moreover, for the one dimension linear unobservable dynamic, by setting: R = AT φn A¯ − η φn ; and by considering the condition: ∃ (i, j) ∈ I such that Ri,j = 0 ; with I ⊆ {1, . . . , n}2

(20)

• If η , ai and λi (∀ n − 1 ≥ i ≥ 1) are such that, ∃ φn , (20) holds, the dynamic is: n n−1 x+ = ηx + λ x +b u+ l x x + k x (21) i i n ij i j ni i u n n i=1

(i,j)∈ I, j=1

i=2

10

J-P. Barbot, I. Belmouhoub, and L. Boutat-Baddas

• And if η, ai and λi (∀ (n − 1) ≥ i ≥ 1) are such that for no φn , (20) is veriﬁed, the dynamic is: n−1 n (22) x+ n = ηxn + i=1 λi xi +bn u+ ( i=2 kni xi ) u Remark 7. • Thanks to the quadratic term kn(n−1) xn u in the normal form described above, it is possible with a well chosen input u, to restore observability. • In the normal form, let us consider more closely the observability singularity’s (here we consider system without input) by isolating the terms in xn which appear in the (n − 1)-th line, as follows: n n−1 (23) j>i=1 hij xi xj + ( i=1 hin xi ) xn ; n we can deduce the manifold of local unobservability: Sn = hin xi = 0 . i=1

4 Unknown Input Observer In many works, the observer design for a system with unknown input was studied [46, 19]... and numerous relevant applications of such approaches were given. In this paper we propose to ﬁnd a new application domain for the unknown input observer design. More precisely, we propose a new type of secure data transmission based on chaotic synchronization. For this we have to recall and give some particular concepts of an observer for a system with unknown input. Roughly speaking, in a linear context, the problem of observer design for a system with unknown input is solved as follows: Assume an observable system with two outputs and one unknown input such that at least one derivative of the output is a function of the unknown input (i.e C1 G or C2 G diﬀerent from zero), x˙ = Ax + Bu + Gω;

y1 = C1 x;

y2 = C2 x

Then to design an observer, we have to choose a new output as a composition of the both original ones, ynew = φ(y1 , y2 ) and ﬁnd observation error dynamics which are orthogonal to the unknown input vector. Unfortunately, this kind of design can not be applied to system with only one output (the case considered in this paper). Nevertheless, it is possible with a step by step procedure to design an observer for such a system. Obviously, there are some restrictive conditions on the system to solve this problem (see [46, 36]). Now, let us consider the nonlinear analytic system: x˙ = f (x) + g(x)u;

y = h(x)

(24)

where vector ﬁelds f and g : IR n → IR n and h : IR n → IR m are assumed to be smooth with f (0) = 0 and h(0) = 0. Now, we can give a particular

Observability Normal Forms

11

constraint in order to solve this problem. The unknown input observer design is solvable locally around x = 0 for system (24) if : • span{dh, dLf h, ..., dLn−1 h} is of rank n at x = 0, f T T T • (dh)T (dLf h)T · · · (dLn−1 g = 0 · · · 0 (observability math) f ching condition) with “ ” means a non null term. Sketch of proof: Setting z1 = h, z2 = Lf h,..., zn = Ln−1 h, we have f z˙1 = z2 ; z˙2 = z3 , ..., z˙n−1 = zn z˙n = f˜(z) + g˜(z)u

(25)

Then under classical boundary assumptions, it is possible for the system (25) to design a step by step sliding mode observer such that we recover in ﬁnite time all state components and the unknown input. Remark 8. In discrete-time the observability matching condition is : T T g = 0 ··· 0 • (dh)T (df oh)T · · · (dfon−1 oh)T

where o denotes the usual composition function and foj denotes the function f composed j times.

5 Synchronization of Chaotic Systems Now we propose new encoding algorithm based on chaotic system synchronization but for which we have also an observability bifurcation. Moreover in both the continuous case and the discrete time case the message is included in the system structure and the observability matching condition is required. 5.1 Continuous-Time Transmission: Chua Circuit Here we just give an illustrative example, so let us consider the well known Chua circuit with a variable inductor (see ﬁgure 1). The circuit contains linear resistors (R, R0 ), a single nonlinear resistor (f (v1 )), and three linear energystorage elements: a variable inductor (L) and two capacitors (C1 , C2 ). The state equations for the circuit are as follows: x˙ 1 x˙ 2 x˙ 3 x˙ 4 ∆

∆

∆

1) = C−1 (x1 − x2 ) + f (x C1 1R x3 = C21R (x1 − x2 ) + C 2 = −x4 (x2 + R0 x3 ) =σ

∆

∆

(26)

∆

1 with: y = x1 = v1 , x2 = v2 , x3 = i3 , x4 = L(t) , x = (x1 , x2 , x3 , x4 )T and f (x1 ) = Gb x1 + 0.5(Ga − Gb )(|x1 + E| − |x1 − E|).

12

J-P. Barbot, I. Belmouhoub, and L. Boutat-Baddas

Fig. 1. Chua Circuit with inductance variation.

Moreover x1 is the output and x4 = L1 is the only state component directly inﬂuenced by σ an unknown bounded function. The variation of L is the information to pass on the receiver. Moreover, we assume that there exist K1 4 and K2 such that |x4 | < K1 and | dx dt | < K2 , this means that the information signal and its variation are bounded. This system has one unobservable real mode and using the linear change of coordinates z1 = x1 , z2 = Cx21R + Cx12R , z3 = C1xC32 R and z4 = x4 we obtain: z˙1 z˙2 z˙3 z˙4

f (x1 ) 1 +C2 ) = −(C C 1 C 2 R z1 + z2 + C 1 f (x1 ) = z3 + C 1 C 2 R z1 z4 = C − zC2 z24 − R0 z3 z4 2 2R =σ

(27)

Equations (27) are in observability normal form [8] with α = 0 and resonant terms h22 = h23 = 0, h14 = C 12 R , h24 = C12 and h34 = −R0 . 2 Moreover the system veriﬁed the observability matching condition [46, 1] f (x1 ) 1) T with respect to σ and as non smooth output injection ( f (x C1 , C1 C2 R , 0, 0) . From the normal form (27) we conclude that the observability singularity manifold is M0 = z Cz21R − Cz22 − R0 z3 = 0 , and taking into account this 2 singularity we can design an observer. Therefore, it is possible to design the following step by step sliding mode observer (here given in the original coordinate for the sake of compactness): x ˆ2 −y dˆ x1 1 = −f (y) + λ1 sign(y − x ˆ1 ) dt C1 R y−˜ x2 dˆ x2 1 x3 + E1 λ2 sign(˜ x2 − x ˆ2 ) dt = C2 R +ˆ (28) dˆ x3 = x ˆ (−˜ x − R x ˜ ) + E λ sign(˜ x − x ˆ ) 4 2 0 3 2 3 3 3 dt dˆ x4 x4 − x ˆ4 ) dt = E3 λ4 sign(˜ with the following conditions: if x ˆ1 = x1 then E1 = 1 else E1 = 0, similarly if [ˆ x2 = x ˜2 and E1 = 1] then x3 = x ˜3 and E2 = 1] then E3 = 1 else E2 = 1 else E2 = 0 and ﬁnally if [ˆ E3 = 0. Moreover, in order to take into account the observability singularity manifold M0 respectively (˜ x2 + R0 x ˜3 = 0), we set Es = 1 if x ˜ 2 + R0 x ˜3 = 0 else Es = 0. And by deﬁnition we take:

Observability Normal Forms

x ˜2 = x ˆ2 + E1 C1 Rλ1 sign(y − x ˆ1 ) x ˜3 = x ˆ3 + E2 C2 λ2 sign(˜ x2 − xˆ2 ) s x ˜4 = x ˆ4 − (˜x2 +R0Ex˜33E−1+E λ3 sign(˜ x3 − x ˆ3 ) S ))

13

(29)

Then the observation error dynamics (e = x − x ˆ) are: de1 dt de2 dt de3 dt de4 dt

= Ce12R − λ1 sign(x1 − x ˆ1 ) = Ce32 − λ2 sign(x2 − x ˆ2 ) = −(x2 + R0 x3 )e4 − λ3 sign(x3 − x ˆ3 ) = σ − Es λ4 sign(˜ x4 − x ˆ4 )

(30)

The proof of observation error convergence is in [9]. Remark 9. In practice we add some low pass ﬁlter on the auxiliary components x ˜i and we set Ei = 1 for i ∈ {1, 2, 3}, not exactly when we are on the sliding manifold but when we are close enough. Similarly, Es = 0 when we are close to the singularity, not only when we are on it. In order to illustrate the eﬃciency of the method we chose to transmit the following message: 0.1 sin (100t) . The message was introduced in the Chua circuit as following: L (t) = L + 0.1L sin (100t) with: L = 18.8mH.

Fig. 2. x4 , x ˆ4 , Es and the singularity (x2 + R0 x3 )

In ﬁgure (2), if we set Es = 0 on a big neighborhood of the singularity manifold (x2 + R0 x3 ), we lose for a long time the information on x4 . We notice that the convergence of the state x ˆ4 of the observer, towards x4 of the system of origin (26) depends on the choice of Es (see both ﬁrst curves of the ﬁgure (2)). In order to have good convergence it is necessary to take Es = 0 on a very small neighborhood of the singularity manifold (x2 + R0 x3 ), as we notice it on both last curves of the ﬁgure (2). But in any case, these simulations conﬁrm that the resonant terms (−x4 x2 − R0 x4 x3 ) = 0 allow us to obtain the message.

14

J-P. Barbot, I. Belmouhoub, and L. Boutat-Baddas

5.2 Discrete-Time Transmission: Burgers Map As information such as it appears nowadays is more and more digitized, treated, exchanged, by “thinking” organs; thus we think that is of ﬁrst importance to study systems in discrete time. Hereafter, we study the following discrete time chaotic system, known under the name of Burgers Map [26]: + x1 = (1 + a) x1 + x1 x2 (31) 2 x+ 2 = (1 − b)x2 − x1 where a and b are two real parameters. We assume that we can measure the state x1 , so we have y = x1 as output of the system. This system is the normal form of: + z1 = (1 + a) z1 + z1 z2 (32) z2+ = (1 − b) (z2 + bz1 z2 ) − z1 2 obtained, modulo (−y 2 ), by applying him the change of coordinates x = z − [2] [2] Φ[2] (z) ; where the diﬀeomorphism Φ[2] is as: Φ1 (z) = 0 and Φ2 (z) = z1 z2 . Encryption: Now let us consider the Burgers map, and let us note that m represents the message and only y = x1 the output, is transmitted to the receiver via a public channel. Then, the transmitter will have the form: + x1 = (1 + a) x1 + x1 x2 (33) 2 x+ 2 = (1 − b)x2 − x1 + m The key of this secure communication consists in the knowledge of the parameters a and b. The fact that the message should be the last information to reach the system constitutes a necessary and suﬃcient condition to recover the message by the construction of a suitable observer. It is what we call the observability “Matching Condition”. Decryption: Now, to decrypt the message we construct the observer: + x ˆ1 = (1 + b) y + y x ˆ2 (34) 2 x ˆ+ = (1 − a)ˆ x − y 2 2 The observer design consists of reconstructing all the linearly unobservable states (i.e. x2 ) in the observer, with the knowledge of y. Reconstruction of x2 : For the sake of causality, we extract x2 from e1 , at e1 the iteration (k − 1); which we approximate by x ˜− ˜− 2 , so: x 2 = y − ,for y = 0. Consequently, when y = 0 this leads to a singularity. However, we overcome this problem, by forcing x ˜2 to take the last remembered value when y = 0. Correction of x ˆ− ˆ2 by x ˜2 in the 2 : By correction, we mean to replace x −− −− 2 ˆ− = (1 − b)˜ x − (y ) . prediction equation of x ˆ2 , then we have: x 2 2C − − Reconstruction of the message m: We have x− = x ˜ = (1 − b)˜ x− − 2 2 2 (y − − )2 + m− − . It is now possible to extract m with 2 delays from e˜2 as:

Observability Normal Forms

15

−− e˜− ˜− ˆ− . Which means that e˜2 (k − 1) = m(k − 2). So 2 = x 2 − x 2C = m we have to wait two steps (these correspond to the necessary steps to the synchronization). The two studied examples consolidate our reﬂection that the observability normal form, allows one to simplify the structural analysis of dynamical systems, while preserving their structural properties. Hence, thanks to the resonant terms, we were able to recover the observability in the receiver and do a suitable observer design, with eventually, bifurcations of observability; and properly reconstruct the conﬁdential message.

References 1. Barbot J-P, Boukhobza T, Djemai M (1996) Sliding mode observer for triangular input form, in Proceedings of the 35th CDC, Kobe, Japan 2. Barbot J-P, Monaco S, Normand-Cyrot D (1997) Quadratic forms and approximate feedback linearization in discrete time, International Journal of Control, 67:567–586 3. Bartolini G, Pisano A, Usai E (200) First and second derivative estimation by sliding mode technique, Int. J. of Signal Processing, 4:167–176 4. Belmouhoub I, Djemai M, Barbot J-P, Observability quadratic normal forms for discrete-time systems, submitted for publication. 5. Besan¸con G (1999) A viewpoint on observability and observer design for nonlinear system, Lecture Notes in Cont. & Inf. Sci 244, Springer, pp. 1–22 6. Bestle D, Zeitz M (1983) Canonical form observer design for nonlinear time varying systems, International Journal of Control, 38:429–431 7. Birk J, Zeitz M (1988) Extended Luenberger observers for nonlinear multivariable systems International Journal of Control, 47:1823–1836 8. Boutat-Baddas L, Boutat D, Barbot J-P, Tauleigne R (2001) Quadratic Observability normal form, in Proc. of the 41th IEEE CDC 9. Boutat-Baddas L, Barbot J-P, Boutat D, Tauleigne R (202) Observability bifurcation versus observing bifurcations, in Proc. IFAC World Congress 10. Diop S, Grizzle JW, Morral PE, Stefanoupoulou AG (1993) Interpolation and numerical diﬀerentiation for observer design, in Proc. of IEEE-ACC, 1329–1335 11. Drakunov S, Utkin V (1995) Sliding mode observer: Tutorial, in Proc. of the IEEE CDC 12. Gauthier J-P, Bornard G (1981) Observability for any u(t) of a class of bilinear systems, IEEE Transactions on Automatic Control, 26:922–926 13. Gauthier J-P, Hammouri H, Othman S (1992) A simple observer for nonlinear systems: application to bioreactors IEEE Trans. on Automat. Contr., 37:875– 880 14. Gu G, Sparks A, Kang W (1998) Bifurcation analysis and control for model via the projection method, in Proc. of 1998 ACC, pp. 3617–3621 15. Hamzi B, Barbot JP, Monaco S, Normand-Cyrot D (2001) Nonlinear discretetime control of systems with a Naimar-Sacker bifurcation, Systems & Control Letters 44:245–258, 16. Hamzi B, Kang W, Barbot JP (2000) On the control of bifurcations, in Proc. of the 39th IEEE CDC

16

J-P. Barbot, I. Belmouhoub, and L. Boutat-Baddas

17. Hauser J, Xu Z (1993) An approximate Frobenius theorem, in Proc. of IFAC World Congress (Sydney), 8:43–46 18. Hermann R, Krener AJ (1977) Nonlinear contollability and observability, IEEE Transactions on Automatic Control, 22:728–740 19. Hou M, Muller PC (1992) Design of observers for linear systems with unknow inputs, IEEE Transactions on Automatic Control, 37:871–875 20. Hou M, Busawon K, Saif M (2000) Observer design based on triangular form generated by injective map, IEEE Trans. on Automat. Contr., 45:1350–1355 21. Kang W (1998) Bifurcation and normal form of nonlinear control system: Part I and II, SIAM J. of Control and Optimization, 36:193–232 22. Kang W, Krener AJ (1992) Extended quadratic controller normal form and dynamic state feedback linearization of nonlinear systems, SIAM J. of Control and Optimization, 30:1319–1337 23. Kang W, Krener AJ Nonlinear observer design, a backstepping approach, personal communication 24. Keller H (1987) Nonlinear Observer design by transformation into a generalzed observer canonical form International Journal of Control, 46:1915–1930 25. Khalil HK (1999) High-gain observers in nonlinear feedback control, Lecture Notes in Cont & Inf Sciences 244, Springer, pp. 249–268 26. Korsch HJ, Jodl HJ (1998) Chaos. A Program Collection for the PC, Springer, 2nd edition 27. Krener AJ (1983) Approximate linearization by state feedback and coordinate change, Systems & Control letters, 5:181–185 28. Krener AJ (1998) Feedback Linearization Mathematical Control Theory, in JBailleul and J-C. Willems Eds, pp. 66–98, Math. Cont. Theory, Springer 29. Krener AJ, Li L (2002) Normal forms and bifurcations of discrete time nonlinear control systems, SIAM J. of Control and Optimization, 40:1697–1723 30. Krener AJ, Isidori A (1983) Linearization by output injection and nonlinear observer, Systems & Control Letters, 3:47–52 31. Krener AJ, Respondek W (1985) Nonlinear observer with linearizable error dynamics , SIAM J. of Control and Optimization, 23:197—216 32. Krener AJ, Xiao MQ (2002) Observers for linearly unobservable nonlinear systems, Systems & Control Letters, to appear. 33. Lobry C (1970) Contˆ olabilit´e des syst`emes non lin´eaires, SIAM J. of Control, pp. 573-605, 1970 34. Nijmeijer H (1981) Observability of a class of nonlinear systems: a geometric approach, Richerche di Automatica, 12:50–68 35. Nijmeijer H, Mareels IMY (1997) An observer looks at synchronization, IEEE Trans. on Circuits and Systems-1: Fundamental Theory and Applications, 44:882–891 36. Perruquetti W, Barbot JP (2002) Sliding Mode control in Engineering, M. Dekker 37. Plestan F, and Glumineau A (1997) Linearization by generalized input-output injection, Systems & Control Letters, 31:115–128 38. Poincar´e H (1899) Les M´ethodes nouvelles de la m´ecanique c´eleste, Gauthier Villard, 1899 R´eedition 1987, biblioth`eque scientiﬁque A. Blanchard. 39. Robertsson A, Johansson R (1999) Observer backstepping for a class of nonminimum-phase systems, Proc. of the 38th IEEE CDC 40. Rudolph J, Zeitz M (1994) A Block triangular nonlinear observer normal form, Systems & Control Letters, 23

Observability Normal Forms

17

41. Sussmann HJ (1979) Single input observability of continuous time systems, Math. Systems Theory, 12:371–393 42. Slotine JJ, Hedrick JK, Misawa EA (1987) On sliding observer for nonlinear systems, ASME J.D.S.M.C, 109:245–252 43. Tall IA, Respondek W (2001) Normal forms and invariants of nonlinear singleinput systems with noncontrollable linearization, in IFAC NOLCOS 44. Tsinias J (1989) Observer design for nonlinear systems, Systems & Control Letters, 13:135–142 45. Xia X, Gao W (1989) Nonlinear observer design by observer error linearization, SIAM J. of Control and Opt., 27:199–213 46. Xiong Y, Saif M (2001) Sliding mode observer for nonlinear uncertain systems, IEEE Transactions on Automatic Control, 46:2012–2017

Bifurcations of Control Systems: A View from Control Flows Fritz Colonius1 and Wolfgang Kliemann2 1 2

Institut f¨ ur Mathematik, Universit¨ at Augsburg, 86135 Augsburg, Germany, [email protected] Department of Mathematics, Iowa State University, Ames IA 50011, U.S.A, [email protected]

1 Introduction The purpose of this paper is to discuss bifurcation problems for control systems viewing them as dynamical systems, i.e., as control ﬂows. Here open loop control systems are considered as skew product ﬂows where the shift along the control functions is part of the dynamics. Basic results from this point of view are contained in the monograph [8]. In the present paper we survey recent results on bifurcation problems-some new results are included and a number of open problems is indicated. Pertinent results from [8] are cited if necessary for an understanding. We consider control systems in Rd of the form x(t) ˙ = f (α, x(t), u(t)), u ∈ U = {u : R → Rm , u(t) ∈ U for t ∈ R},

(1)

where the control range U is a subset of Rm and α ∈ A ⊂ R denotes a bifurcation parameter. For simplicity we assume that for every initial condition x(0) = x0 ∈ Rd and every admissible control function u a unique global solution ϕα (t, x0 , u), t ∈ R, exists. If the dependence on α is irrelevant, we suppress α in the notation. As for diﬀerential equations, it is relevant to discuss qualitative changes in the system behavior when α is varied. Such problems have found much interest in recent years; see e.g. the contributions in this volume or in [7]. The bifurcation theory developed here concerns open loop control system. Based on the concept of the associated control ﬂow, changes in the controllability behavior come into focus. It turns out that the diﬀerence between controllability and chain controllability (which allows for arbitrarily small jumps) is decisive for our analysis. Since we discuss open loop systems with restricted control values, feedback transformations will not be allowed; this is in contrast to classical concepts of normal forms in control theory. In particular, this W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 19–35, 2003.

c Springer-Verlag Berlin Heidelberg 2003

20

F. Colonius and W. Kliemann

is a notable diﬀerence to the bifurcation and normal form theory developed recently by A. Krener and W. Kang; see, in particular, Kang [21]. The contents are as follows: In Section 2 we introduce our framework. Section 3 discusses bifurcation from a singular point, i.e., a point in the state space that remains ﬁxed under all controls; here also an approach to normal forms is discussed. Section 4 treats bifurcations from invariant control sets. Notation: For A, B ⊂ Rd the distance of A to B is dist(A, B) = sup inf |a − b| a∈A b∈B

and the Hausdorﬀ distance is dH (A, B) = max (dist(A, B), dist(B, A)). The topological closure and interior of a set A are denoted by clA and intA, respectively.

2 Control Flows System (1) may be viewed as a family of ordinary diﬀerential equations indexed by u ∈ U. Since they are non-autonomous, they do not deﬁne a ﬂow or dynamical system. In the theory of non-autonomous diﬀerential equations there is the classical device to embed such an equation into a ﬂow by considering all time shifts of the right hand side and to include the possible right hand sides into the state. In the context of uniformly continuous time dependence this goes back to Bebutov [3]; more recently, such constructions have been used extensively by R. Johnson and others in the context of non-autonomous (linear) control systems (e.g. Johnson and Nerurkar [19]); see also Gr¨ une [16] for a diﬀerent discussion emphasizing numerical aspects. Here, however, we will stick to autonomous control systems and only consider the time-dependence stemming from the control functions. Introduce the shift (θt u) (τ ) = u(t + τ ), τ ∈ R, on the set of control functions. One immediately sees that the map Φ : (t, u, x0 ) → (θt u, ϕ(t, x0 , u)) deﬁnes a ﬂow Φ on U × Rd : Abbreviating Φt = Φ(t, ·, ·), one has Φ0 = id and Φt+s = Φt ◦ Φs . Since the state space is inﬁnite-dimensional, additional topological requirements are needed. We require that U is contained in L∞ (R, Rm ). This gives a reasonable framework, if U ⊂ Rm is compact and convex and the system is control aﬃne, i.e., f (α, x, u) = f0 (α, x) +

m i=1

ui fi (α, x).

(2)

Bifurcations of Control Systems: A View from Control Flows

21

Then the ﬂow Φ is continuous and U becomes a compact metric space in the weak∗ topology of L∞ (R, Rm ) (cp. [8, Lemma 4.2.1 and Lemma 4.3.2]). We refer to system (1) with right hand side given by (2) for some α ∈ A as system (2)α ; similarly, we denote objects corresponding to this system by a superﬁx α. We assume that 0 ∈ intU . Throughout this paper we remain in this framework. Remark 1. The class of systems can be extended, if-instead of the shift along control functions-the shift along the corresponding time dependent vector ﬁelds is considered; cp. [10] for a brief exposition. A control set D is a maximal controlled invariant set such that D ⊂ clO+ (x) for all x ∈ D.

(3)

Here O+ (x) = {ϕ(t, x, u), u ∈ U and t ≥ 0} denotes the reachable set from x. A control set C is called an invariant control set if clC = clO+ (x) for all x ∈ C. Often we will assume that local accessibility holds, i.e., that the + − small time reachable sets in forward and backward time O≤T (x) and O≤T (x), respectively, have nonvoid interiors for all x and all T > 0. Then intD ⊂ O+ (x) for all x ∈ D. Local accessibility holds, if dim LA{f0 +

m

ui fi , (ui ) ∈ U }(x) = d for all x ∈ Rd .

(4)

i=1

We also recall that a chain control set E is a maximal subset of the state space such that for all x ∈ E there is u ∈ U with ϕ(t, x, u) ∈ E for all t ∈ R and for every two elements x, y ∈ E and all ε, T > 0 there are x0 = x, x1 , ..., xn = y in Rd , u0 , ..., un ∈ U and T0 , ..., Tn−1 > T with d(ϕ(Ti , xi , ui ), xi+1 ) < ε. Every control set with nonvoid interior is contained in a chain control set; chain control sets are closed if they are bounded. Compact chain control sets E uniquely correspond to compact chain recurrent components of the control ﬂow via E = {(u, x) ∈ U × Rd , ϕ(t, x, u) ∈ E for all t ∈ R}. Control sets D with nonvoid interior uniquely correspond to maximal topologically transitive sets (such that the projection to Rd has nonvoid interior) via D = cl{(u, x) ∈ U × Rd , ϕ(t, x, u) ∈ intD for all t ∈ R}. It turns out that for parameter-dependent systems, control sets and chain control sets have complementary semicontinuity properties; see [9]. Theorem 1. Consider parameter-dependent system (2) and ﬁx α0 ∈ A. (i) Let Dα0 be a control set with compact closure of (2)α0 , and assume that

22

F. Colonius and W. Kliemann

system (2)α0 satisﬁes accessibility rank condition (4) on cl Dα0 . Then for α near α0 there are unique control sets Dα of (2)α such that the map α −→ cl Dα is lower semicontinuous at α = α0 . (ii) Let K ⊂ Rd be compact and suppose that for α near α0 the chain control sets E α with E α ∩ K = ∅ of (2)α are contained in K. Then α → E α is upper semicontinuous at α0 in the sense that there are αk → α0 and lim sup E α = x ∈ M, αk ⊂ E α0 , x ∈ E αk with xαk → x α→α0 where the union is taken over the chain control sets E α0 ⊂ K of (2)α0 . (iii) Let Dα0 be a control set of (2)α0 with α0 ∈ A, and assume that system (2)α0 satisﬁes accessibility rank condition (4) on clDα0 . Let E α be the chain control set containing the control set Dα (given by (i)) and assume that clDα0 = E α0 . Then the control sets Dα depend continuously on α at α = α0 , lim cl Dα = lim E α = cl Dα0 = E α0 .

α→α0

α→α0

Remark 2. Gayer [14] shows that (i) in the previous theorem remains true if (instead of α–dependence) the control range depends lower semicontinuously on a real parameter ρ. Thus a chain control set which is the closure of a control set with nonvoid interior depends continuously on parameters. This equivalence of controllability and chain controllability may be interpreted as a structural stability property of control systems. Hence it is important to study when chain control sets coincide with the closures of control sets. In order to allow for diﬀerent maximal amplitudes of the inputs, we consider admissible controls in U ρ = {u ∈ L∞ (R, Rm ), u(t) ∈ ρ · U }, ρ ≥ 0. It is easily seen that the corresponding trajectories coincide with the trajectories ϕρ (t, x, u) of x(t) ˙ = f ρ (x(t), u(t)) = f (x(t), ρu(t)), u ∈ U. Clearly, this is a special case of a parameter-dependent control system as considered above. The maximal chain transitive sets Ei0 of the uncontrolled system are contained in chain control sets Eiρ of the ρ-system for every ρ > 0. Their lifts Eiρ are the maximal chain transitive sets of the corresponding control ﬂows Φρ . Every chain transitive set for small positive ρ > 0 is of this form with a unique Ei0 , i = 1, ..., m (see [8]). For larger ρ-values, there may exist further maximal chain transitive sets E ρ containing no chain transitive set of the unperturbed system. An easy example is obtained by looking at systems where for some ρ0 > 0 a saddle node bifurcation occurs in x˙ = f (x, ρ). A more intricate example is [8], Example 4.7.8. Observe that for larger ρ-values the chain control sets may intersect and hence coincide. From Theorem 1 we obtain that the maps

Bifurcations of Control Systems: A View from Control Flows

ρ → clDρ and ρ → E ρ

23

(5)

are left and right continuous, respectively. We call (u, x) ∈ U × Rd an inner pair, if there is T > 0 with ϕ(T, x, u) ∈ intO+ (x). The following ρ-inner pair condition will be relevant: For all ρ , ρ ∈ [ρ∗ , ρ∗ ) with ρ > ρ and all chain control sets E ρ every (u, x) ∈ E ρ is an inner pair for (2)ρ .

(6)

By [8, Corollary 4.1.12] the ρ-inner pair condition and local accessibility imply that for increasing families of control sets Dρ and chain control sets E ρ with Dρ ⊂ E ρ the number of discontinuity points of (5) is at most countable; they coincide for both maps and at common continuity points clDρ = E ρ . The number of discontinuity points may be dense (without the inner pair condition, there may be “large” discontinuities which persist for all ρ > 0). Remark 3. The inner-pair condition (6) may appear unduly strong. However it is easily veriﬁed for small ρ > 0 if the unperturbed system has a controllable linearization (more information is given in [8], Chapter 4.) For general ρ > 0 the inner pair condition holds, e.g., for coupled oscillators if the number of controls is equal to the degrees of freedom; for this result and more general conditions see Gayer [14].) Next we show that the number of discontinuity points with a lower bound on the discontinuity size is ﬁnite in every compact ρ-interval. Thus, from a practical point of view, only ﬁnitely many may be relevant. Lemma 1. Consider families of increasing control sets Dρ and chain control sets E ρ with Dρ ⊂ E ρ . (i) Let ρ0 ≥ 0 and assume that E ρ0 ⊂ clDρ for ρ > ρ0 . For every ε > 0 there is δ > 0 such that for all ρ > ρ0 ρ − ρ0 < δ implies dH (clDρ , E ρ ) < ε. (ii) Let ρ0 > 0 and assume that E ρ ⊂ clDρ0 for ρ < ρ0 . For every ε > 0 there is δ > 0 such that for all ρ < ρ0 ρ0 − ρ < δ implies dH (clDρ , E ρ ) < ε. Proof. (i) Since for all ρ the inclusion clDρ ⊂ E ρ holds, one only has to show that dist(E ρ , clDρ ) = sup{d(x, clDρ ), x ∈ E ρ ) < ε. Let ε > 0. By right continuity of ρ → E ρ there is δ > 0 such that dist(E ρ , E ρ0 ) < ε for all ρ with δ > ρ − ρ0 > 0 and we know that E ρ0 ⊂ clDρ ⊂ E ρ . Thus

24

F. Colonius and W. Kliemann

dist(E ρ , clDρ ) = sup{d(x, clDρ ), x ∈ E ρ } ≤ sup{d(x, E ρ0 ), x ∈ E ρ } = dist(E ρ , E ρ0 ) < ε. (ii) Similarly, left continuity of ρ → clDρ yields for ε > 0 a number δ > 0 such that dH (clDρ0 , clDρ ) = dist(clDρ0 , clDρ ) < ε for all ρ with −δ < ρ − ρ0 < 0 and we know that E ρ ⊂ clDρ0 . Thus dist(E ρ , clDρ ) = sup{d(x, clDρ ), x ∈ E ρ } ≤ sup{d(x, clDρ ), x ∈ clDρ0 } = dist(clDρ0 , clDρ ) < ε. Proposition 1. Suppose that the ρ-inner pair condition (6) holds on an interval [ρ∗ , ρ∗ ] ⊂ [0, ∞). Then for every ε > 0 there are only ﬁnitely many points ρ ∈ [ρ∗ , ρ∗ ] where dH (clDρ , E ρ ) ≥ ε. Proof. The inner pair condition guarantees that for all ρ < ρ in [ρ∗ , ρ∗ ] one has E ρ ⊂ clDρ ; see [8, Section 4.8]. Let ε > 0. By the preceding lemma one ﬁnds for every point ρ0 ∈ [ρ∗ , ρ∗ ] a number δ > 0 such that for all ρ = ρ0 in U (ρ0 ) := [ρ∗ , ρ∗ ] ∩(ρ0 − δ, ρ0 + δ) dH (clDρ , E ρ ) < ε. Now compactness implies that [ρ∗ , ρ∗ ] is covered by ﬁnitely many of these sets U (ρ0 ). Only their midpoints ρ0 may have d(clDρ0 , E ρ0 ) ≥ ε. Remark 4. The same arguments show that the reachable sets enjoy the same properties, if their closures are compact and the ρ-inner pair condition holds everywhere.

3 Bifurcation from a Singular Point In this section we discuss bifurcation of control sets and chain control sets from a singular point x0 . Here the linearized system is a (homogeneous) bilinear control system; the associated control ﬂow is a linear skew product ﬂow over the base of control functions. Assume that x0 ∈ Rd remains ﬁxed for all α and for all controls, i.e., fi (α, x0 ) = 0 for i = 0, 1, ..., m. Then the system linearized with respect to x is m ui (t)Ai y(t), u ∈ U, y(t) ˙ = A0 + i=1

(7)

(8)

Bifurcations of Control Systems: A View from Control Flows

25

where Ai := Dfi (x0 ). The solutions are y(t, y 0 , u) = D2 ϕ(t, x0 , u)(y 0 ) and the associated linearized control ﬂow has the form TΦt (x0 ) : U×Rd → U×Rd , (u, y) → (θt u, y(t, y 0 , u)). Clearly this ﬂow is linear in the ﬁbers {u} × Rd , since it corresponds to linear homogeneous diﬀerential equations. The singular point is a trivial control set which is invariant. It need not be a chain control set. The bifurcation from this control set can be analyzed using the Lyapunov exponents of the linearized system which are given by 1 λ(u, y 0 ) = lim supt→∞ log y(t, y 0 , u) . t

(9)

We note that there are a number of closely related notions for this generalization of the real parts of eigenvalues to nonautonomous linear diﬀerential equations. Basic results are given by Johnson, Palmer, and Sell [20]; see [8, Section 5.5] for some additional information. The following result due to Gr¨ unvogel [17] shows that control sets near the singular point are determined by the Lyapunov exponents; note that for periodic controls, the Lyapunov exponents are just the Floquet exponents. Theorem 2. Consider the control-aﬃne systems (2) with a singular point x0 ∈ Rd satisfying (7) and assume that the accessibility rank condition (4) holds for all x = x0 . Furthermore assume that (i) there are periodic control functions us and uh such that for us the linearized system is exponentially stable, i.e., the corresponding Lyapunov exponents satisfy 0 > λs1 > ... > λsd , and for uh the corresponding Lyapunov exponents satisfy λh1 ≥ ... ≥ λhk > 0 > λhk+1 > ... > λhd ; (ii) all pairs (uh , x) ∈ U × Rd with x = x0 are strong inner pairs, i.e., ϕ(t, x, uh ) ∈ intO+ (x) for all t > 0. Then there exists a control set D with nonvoid interior such that x0 ∈ ∂D. Using this result one observes in a number of control systems, e.g., in the Duﬃng-Van der Pol oscillator [17], that for some α-values the singular point x0 is exponentially stable for all controls, hence there are no control sets near x0 . Then, for increasing α-values, control sets Dα occur with x0 ∈ ∂Dα . For some upper α-value, they move away from x0 . Remark 5. Assumption (i) in Theorem 2 is in particular satisﬁed, if 0 is in the interior of the highest Floquet spectral interval (cp. [8]) and the corresponding subbundle is one-dimensional.

26

F. Colonius and W. Kliemann

Remark 6. Gr¨ unvogel [17] also shows that there are no control sets in a neighborhood of the origin if zero is not in the interior of the Morse spectrum of the linearized system (8). This also follows from a Hartman-Grobman Theorem for skew product ﬂows; see Bronstein/Kopanskii [4]. One has to take into account that the spectral condition implies hyperbolicity, since the base space U is chain recurrent. Then the use of appropriate cut-oﬀ functions yields the desired local version. Remark 7. Using averaging techniques, Grammel/Shi [15] considered the stability behavior and the Lyapunov spectrum of bilinear control systems perturbed by a fast subsystem. Remark 8. A number of questions remains open in this context: Is the control set containing the singular point x0 on its boundary invariant? (certainly it is not closed.) Can one also prove a bifurcation of control sets if other spectral intervals (instead of the highest interval) contain 0? What happens if the corresponding subbundles are higher dimensional? Is the consideration of periodic controls necessary? We see that a characteristic of bifurcations from a singular point is that there is (at least) one transitional state. Here the control set {x0 } has already split into two or more control sets which, however, still form a single chain control set. The bifurcation is complete when also the chain control set has split. This should be compared with L. Arnold’s bifurcation pattern for stochastic systems [1]; see in particular the discussion in Johnson, Kloeden, and Pavani [18]. The Hartman-Grobman Theorem alluded to in Remark 6 gives a topological conjugacy result. As for diﬀerential equations, a natural next step is to classify the bifurcation behavior by introducing normal forms of nonlinear systems based on smooth conjugacies. Since we discuss open loop systems with restricted control values, feedback transformations will not be allowed (thus this is diﬀerent from classical concepts of normal forms in control theory). The admissible transformations have to depend continuously on the control functions u in the base space U of the skew product ﬂow. This makes it possible (see [11]) to use methods from normal forms of nonautonomous diﬀerential equations (Siegmund [25]). Then conjugacies eliminate all nonresonant terms in the Taylor expansion without changing the other terms up to the same order. We note that there is also related work in the theory of random dynamical systems which can be considered as skew product ﬂows with an invariant measure on the base space; compare L. Arnold [1]. We consider the control aﬃne system (2) and assume that f0 , . . . , fm are C k vector ﬁelds for some k ≥ 2. Then the associated control ﬂow Φ is, for ﬁxed u ∈ U, k times continuously diﬀerentiable with respect to x. Our notion of conjugacies which, naturally, depend on u is speciﬁed in the following deﬁnition.

Bifurcations of Control Systems: A View from Control Flows

27

Deﬁnition 1. Let ϕ : R × U × Rd → Rd and ψ : R × U × Rd → Rd be two control systems of the form (2) with common singular point x0 . Then ϕ and ψ are said to be C k conjugate if there exists a bundle mapping U × Rd (u, x) → (u, H(x, u)) ∈ U × Rd which preserves the zero section U × {0}, such that (i) x → H(x, u) is a local C k diﬀeomorphism (near x0 = 0) for each ﬁxed u ∈ U (with inverse denoted by y → H(y, u)−1 ), (ii) (u, x) → H(x, u) and (u, y) → H(y, u)−1 are continuous, (iii) for all t ∈ R, x ∈ Rd and u ∈ U the conjugacy ψ(t, u, H(x, u)) = H(θt u, ϕ(t, x, u)) holds. Next we discuss the Taylor expansions and the terms which are to be eliminated by conjugacies. We rewrite system (2) in the form x˙ = A(u(t))x(t) + F (x(t), u(t)),

(10)

where the nonlinearity is given by F (x(t), u(t)) = f0 (x(t)) − A0 x(t) +

m

ui (t)(fi (x(t)) − Ai x(t)).

i=1

In the following we assume that the linearized system is in block diagonal form and that the nonlinearity is C k -bounded. More precisely we assume A = diag(A(1) , . . . , A(n) ) with A(i) : U → Rdi ×di , d1 + · · · + dn = d and Dxi F (x0 , u) ≤ M for i = 1, . . . , k, u ∈ U, with a constant M > 0. The block diagonalization of the linearized system into the matrices A(i) corresponds to a decomposition of Rd into di -dimensional subspaces. Corresponding to the block diagonal structure of A one can write x = (x(1) , . . . , x(n) ) ∈ Rd and F = (F (1) , . . . , F (n) ) with the component functions F (i) : Rd × U → Rdi . For a multi-index = (1 , . . . , n ) ∈ N0 let || = 1 + · · · + n denote the order and deﬁne Dx F = Dx1(1) · · · Dxn(n) F

and x = x(1) ·!" · · x(1)# · · · x(n) ·!" · · x(n)# . 1 -times

n -times

Now we can expand F (·, u(t)) into a Taylor series at x0 F (x, u(t)) =

∈Nn 0 : 2≤||≤k

1 D F (x0 , u(t)) · (x − x0 ) + o(x − x0 k ) , ! x

28

F. Colonius and W. Kliemann

where ! = 1 ! · · · n !. For simplicity we assume without loss of generality that x0 = 0. We are looking for a condition which ensures the existence of a C k conjugacy which eliminates the j-th component Dx F (j) (0, u(t)) · x of a summand in the Taylor expansion of F . Let Φ = diag(Φ(1) , . . . , Φ(n) ) denote the solution of the linearized system (8), i.e., Φ(i) (t, u)y (i) solves the control system y˙ (i) (t) = A(i) (u(t))y (i) (t)

in Rdi , u ∈ U,

with Φ(i) (0, u)y (i) = y (i) . The nonresonance condition will be based on exponential dichotomies: we associate to each Φ(i) an interval Λi = [ai , bi ] such that for every ε > 0 there is K > 0 with Φ(i) (s, u)−1 ≤ Ke−(ai −ε)s and Φ(i) (s, u) ≤ Ke(bi +ε)s

(11)

for s ≥ 0, u ∈ U. Remark 9. Condition (11) holds if we deﬁne, for the Lyapunov exponents as in (9), ai = inf{λ(u, y (i) ), (u, y (i) ) ∈ U × Rdi }, bi = sup{λ(u, y (i) ), (u, y (i) ) ∈ U × Rdi }.

Then [ai , bi ] contains the dynamical spectrum of the corresponding system on Rdi and the assertion follows from its properties; see [24] or [8, Section 5.4]. Next we state the normal form theorem for control systems at a singular point. It shows that nonresonant terms in the Taylor expansion can be eliminated without changing the other coeﬃcients up to the same order. The proof of this theorem is given in [11], where also some ﬁrst examples are indicated. For compact sets K1 , K2 ⊂ R and integers 1 , 2 ∈ Z we deﬁne the compact set 1 K1 + 2 K2 := {1 k1 + 2 k2 : ki ∈ Ki } and we write K1 < K2 iﬀ max K1 < min K2 and similarly for K1 > K2 . Theorem 3. Consider a class of C k control aﬃne systems (2) satisfying the assumptions above. Suppose that to each block an interval Λi is associated with property (11). Let = (1 , . . . , n ) ∈ Nn0 be a multi-index of order 2 ≤ || ≤ k and assume that for some j the nonresonance condition Λj <

n i=1

i Λi

or

Λj >

n

i Λi

i=1

holds. Then there exists a C k conjugacy between (10) and a system x˙ = A(u(t))x(t) + G(x(t), u(t))

(12)

which eliminates the j-th Taylor component !1 Dx F (j) (0, u(t)) · x belonging to the multi index and leaves ﬁxed all other Taylor coeﬃcients up to order ||,

Bifurcations of Control Systems: A View from Control Flows

29

Remark 10. This result shows that-under a non-resonance condition-some terms can be eliminated without changing the other terms up to the same order. We would like to stress that higher order terms may be changed; no analysis of terms of arbitrarily high order appears feasible in this context. Remark 11. The theorem above leads to the problem to ﬁnd a complete catalogue of systems of order k without terms which can be eliminated. Such an analysis must be made for every control range (i.e. for every base ﬂow). It is an interesting question, when diﬀerent control ranges may lead to the same normal forms. Remark 12. The Lyapunov exponents generalize the real part of eigenvalues. The imaginary parts of eigenvalues determine the rotational behavior and hence are also of relevance in describing the bifurcation behavior. For stochastic equations (where an ergodic invariant measure on the base space is given), Arnold/San Martin [2] and Ruﬃno [23] have discussed a corresponding rotation number. Another concept is used by Johnson and others for Hamiltonian skew product ﬂows; in particular, the latter is used for a generalization [13] of a theorem due to Yakubovich who analyzed linear quadratic optimal control problems for periodic systems.

4 Bifurcation from Invariant Control Sets The previous section dealt with bifurcation of control sets from a singular point. Other singular scenarios where bifurcation phenomena occur are totally unexplored. The present section concentrates on the regular situation where local accessibility holds. Bifurcations will be considered for an invariant object in the state space (not for the more general case of invariant objects for the lifted system in U × M ). A ﬁrst question concerns the role of hyperbolicity in this regular context. In the theory of chaotic dynamical systems a classical tool is Bowen’s shadowing lemma. It allows one to ﬁnd close to (ε, T )-chains trajectories of a hyperbolic diﬀerential equation or discrete dynamical system. In the context of control ﬂows, a generalization has been given in Colonius/Du [5]. The required hyperbolicity condition refers to the linearized system given by x˙ = f (x, u), y˙ = D1 f (x, u)y, u ∈ U ρ .

(13)

Theorem 4. Suppose that the uncontrolled system x˙ = f (x, 0) is hyperbolic on a compact chain transitive component M and assume local accessibility for all ρ > 0. Furthermore assume that the chain control set E ρ containing M has nonvoid interior. Then for ρ > 0, small enough, E ρ = clDρ for a control set Dρ with nonvoid interior. Remark 13. Since control ﬂows are special skew product ﬂows, it may appear natural to ask if a shadowing lemma for general skew product ﬂows can be

30

F. Colonius and W. Kliemann

used in this context. However, closing the gap between chain controllability and controllability also requires to close gaps in the base space. Here, in general, hyperbolicity which only refers to the ﬁbers cannot be used. Thus the shadowing lemma for general discrete-time skew product ﬂows by Meyer and Sell [22] can not be used since it excludes jumps in the base space. In another direction one can analyze the behavior near a hyperbolic equilibrium of the uncontrolled system. Then a natural question is if the local uniqueness of the equilibrium of the uncontrolled system transfers to the controlled system. A positive answer has been given in Remark 6 for the case of a singular equilibrium. The following result from Colonius/Spadini [12] gives an analogous result in the regular situation. For the formulation we need the notion of local control sets: For a subset N ⊂ Rd denote + ON (x) = {ϕ(T, x, u), T > 0, u ∈ U and ϕ(t, x, u) ∈ N for all 0 ≤ t ≤ T }. A subset D ⊂ Rd is called a local control set, if there exists a neighborhood + (x) for all x ∈ D and D is maximal with this N of clD such that D ⊂ clON property. Consider a parameter-dependent family of control systems x(t) ˙ = f (α, x(t), u(t)), u(t) ∈ ρU,

(14)

where α ∈ R, ρ > 0 and U ⊂ Rm is bounded, convex and contains the origin in its interior. We consider the behavior near an equilibrium of the uncontrolled system with α = α0 for small control range. Theorem 5. Let f : R × Rd × Rm → Rd be a continuous map which is C 1 with respect to the last two variables. Consider a continuous family of equilibria xα ∈ Rd such that f (α, xα , 0) = 0 and assume that the pair of matrices D2 f (α0 , xα0 , 0), D3 f (α0 , xα0 , 0) is controllable and D2 f (α0 , xα0 , 0) is hyperbolic. Then there exist ε0 > 0, ρ0 > 0 and δ0 > 0 such that, for all |α − α0 | < ε0 and all 0 < ρ < ρ0 , the ball B(xα0 , δ0 ) contains exactly one local control set for (14) with parameter value α. Without hyperbolicity this claim is false. We proceed to a partial generalization of Gr¨ unvogel’s theorem, Theorem 2. Since we discuss bifurcation from a nontrivial invariant set, here the direction of the unstable manifold will be important (it must be directed outward). For an invariant control set C with nonvoid interior and compact closure we denote the lift of C by C, C = cl{(u, x) ∈ U × Rd , ϕ(t, x, u) ∈ intC for all t ∈ R}. The linearized ﬂow over C is obtained by restricting attention to solutions of (13) with (u, x, y) ∈ C × Rd . The corresponding Lyapunov exponents are 1 λ(u, x, y) = lim supt→∞ log |D2 ϕ(t, x, u)y(t)| . t

Bifurcations of Control Systems: A View from Control Flows

31

For x ∈ ∂C deﬁne the outer cone in x for C by there are β, λ0 > 0 such that for all z ∈ Rd with . Kx C = y ∈ Rd , /C |z − y| < β and all 0 < λ < λ0 one has x + λz ∈ Theorem 6. Assume that the system is locally accessible and let C be an invariant control set with nonvoid interior and compact closure. Assume that there exists a compact invariant subset J ⊂ C with the following properties: (i) The unstable part of J is nontrivial, i.e., there is a subbundle decomposition of the vector bundle J × Rd into subbundles V − = 0 and V + , which are invariant under the linearized ﬂow and exponentially separated, such that J × Rd = V + ⊕ V − , all Lyapunov exponents attained in V − are positive, and there are constants γ, c > 0 with TΦt (u, x, y − ) < c exp(γt) TΦt (u, x, y + ) for (u, x, y ± ) ∈ V ± , (ii) There is (u, x, y − ) ∈ V − such that (u, x) ∈ J and x ∈ ∂C and y − ∈ Kx C, the outer cone in x for C. Then the invariant control set C is a proper subset of the chain control set E containing it. Proof. We will construct a chain controllable set which has nontrivial intersections with C and the complement of C. This implies the assertion. Consider the point x as speciﬁed in the second assumption. Since x is in the boundary of C, there are v ∈ U, τ > 0, and a neighborhood N of x such that for all z ∈ N one has ϕ(τ, z, v) ∈ int C. By an appropriately general version of the Unstable Manifold Theorem, see, e.g., [8, Section 6.4], our assumptions imply, that the set J has a nontrivial unstable manifold W − , which is Lipschitz close to V − . In particular, for (u, x, y − ) ∈ V − as speciﬁed in the assumptions, there is x− ∈ N ∩ (Rd \ C) with x− ∈ W − (u, x). Thus d(ϕ(t, x− , u), ϕ(t, x, u)) → 0 for t → −∞. Since (u, x) ∈ J ∩ C it follows that ϕ(t, x, u) ∈ C for all t ∈ R. By compactness of C, there are z ∈ C and tk → −∞ with ϕ(tk , x, u) → z for k → ∞, Now ﬁx ε > 0 and T > 0. We will construct a controlled (ε, T )-chain connecting x− and C. Start in x0 := x− and deﬁne u0 ∈ U as the concatenation of v with any control which keeps the trajectory in C up to time T and has the property that for some T0 > T one has ε d(ϕ(T0 , x− , u0 ), z) < . 2 There is τ > T such that ε d(ϕ(−τ, x− , u), z) < . 2 Thus we deﬁne x1 := ϕ(−τ, x− , u), u1 := u(−τ + ·), and T1 := τ . This yields the desired (ε, T )- chain from x− to x− hitting C.

32

F. Colonius and W. Kliemann

The result above is only a ﬁrst step, since it does not answer the question, if the gap between the control set D and the chain control set E ⊃ D is due to the presence of another control set sitting in E. A partial answer is given in the following result from [10] which shows when the loss of invariance is due to the merger with a variant control set. We need some preparations. Let K ⊂ Rd be compact and invariant. An attractor for the control ﬂow Φ is a compact invariant set A ⊂ U × K that admits a neighborhood N such that A = ω(N ) = {(u, x) ∈ U × K, there are tk → ∞ and (uk , xk ) ∈ N with Φ(tk , uk , xk ) → (u, x)}. Deﬁne for chain recurrent components E, E [E, E ] = {(u, x) ∈ U × K, ω ∗ (u, x) ⊂ E and ω(u, x) ⊂ E }; here ω ∗ (u, x) denotes the limit set for t → −∞. The structure of attractors and their relation to chain control sets is described in the following proposition. Proposition 2. Assume that for every ρ > 0 every chain recurrent component E ρ contains a chain recurrent component Ei0 of the unperturbed system. Then there is ρ0 > 0 such that for all ρ with ρ0 > ρ > 0 the attractors Aρ of the ρ-system are given by ρ ρ Ej , Ek Aρ = i,j∈J

where the allowed index sets J coincide with those for ρ = 0. The chain recurrent components Eiρ depend upper semicontinuously on ρ and converge for ρ → 0 toward U ×Ei0 ; all Ejρ are diﬀerent and they have attractor neighborhoods of the form U × B with B ⊂ K. For a set I ⊂ K the invariant domain of attraction is if C ⊂ clO+ (x) is an invariant inv . A (I) = x ∈ K, control set, then C ⊂ I

(15)

The invariant domain of attraction is closed and invariant. For simplicity we assume that all control sets are in the interior of K. By local accessibility, all invariant control sets have nonvoid interiors. We will assume that for all ρ with ρ1 > ρ > 0 the chain control sets Eiρ are the closures of control sets Diρ with nonvoid interior; observe that some of the control sets in the attractor must be invariant, since every point can be steered into an invariant control set. Then Eiρ = clDiρ implies that also the lifts coincide, i.e., Eiρ = Diρ . It follows that the attractors are given by ρ ρ Di , Dj . (16) Aρ = i,j∈J

We will analyze the case where for ρ = ρ1 the set Aρ1 has lost the attractor property.

Bifurcations of Control Systems: A View from Control Flows

33

Theorem 7. Consider the control system (2) in Rd and assume that K = cl intK ⊂ Rd is a compact and connected set which is invariant for the system with input range given by ρ1 U with ρ1 > 0. Assume that the following strong invariance conditions describing the behavior near the boundary of K are satisﬁed: (i) For all x ∈ L there is εx > 0 with d(ϕ(t, x, u), ∂K) ≥ εx for all u ∈ U and t ≥ 0. (ii) There is ε0 > 0 such that for all x ∈ clL and u ∈ U y = lim ϕ(tk , x, u) ∈ L for tk → ∞ implies d(y, ∂K) ≥ ε0 . k→∞

(17)

Consider the invariant sets in U ρ × K ρ ρ Iρ = Di , Dj , i,j∈J

and assume that they are attractors for ρ < ρ1 and that the projection I ρ1 to Rd of I ρ1 intersects the boundary of its invariant domain of attraction deﬁned in (15), i.e., I ρ1 ∩ ∂Ainv (I ρ1 ) = ∅. Then every attractor containing I ρ1 contains a lifted variant control set Dρ1 with Dρ1 ∩ I ρ1 = ∅. This theorem shows that the loss of the attraction property due to increased input ranges is connected with the merger of the attractor with a variant control set Dρ1 . Connections to Input-to-State Stability are discussed in [10]. Remark 14. The abstract Hartman-Grobman Theorem from [4] can also be applied to the system over an invariant control set. Here, for the linearized system, the base space is the lift of the invariant control set. However, for parameter-dependent systems, this entails that the base space changes with the parameter. Hence it does not appear feasible to obtain results which yield conjugacy for small parameter changes. Here, presumably, normal hyperbolicity assumptions are required. Remark 15. Consider a family of increasing control set Dρ corresponding to increasing control ranges. Assume that they are invariant for ρ ≤ ρ0 and variant for ρ > ρ0 Then Gayer [14] has shown that the map ρ → clDρ has a discontinuity at ρ0 . This is a consequence of his careful analysis of the diﬀerent parts of the boundary of control sets. This also allows him to describe in detail the merging of an invariant control set with a variant control set. Finally we remark that the intuitive idea of a slowly varying bifurcation parameter is made more precise, if the bifurcation parameter actually is subject to slow variations. This leads to concepts of dynamic bifurcations. The fate of control sets for frozen parameters under slow parameter variations is characterized in Colonius/Fabbri [6].

34

F. Colonius and W. Kliemann

References 1. L. Arnold, Random Dynamical Systems, Springer-Verlag, 1998. 2. L. Arnold and L. S. Martin, A multiplicative ergodic theorem for rotation numbers, J. Dynamics Diﬀ. Equations, 1 (1989), pp. 95–119. 3. M. V. Bebutov, Dynamical systems in the space of continuous functions, Dokl. Akad. Nauk SSSR, 27 (1940), pp. 904–906. in Russian. 4. I. U. Bronstein and A. Y. Kopanskii, Smooth Invariant Manifolds and Normal Forms, World Scientiﬁc, 1994. 5. F. Colonius and W. Du, Hyperbolic control sets and chain control sets, J. Dynamical and Control Systems, 7 (2001), pp. 49–59. 6. F. Colonius and R. Fabbri, Controllability for systems with slowly varying parameters, ESAIM: Control, Optimisation and Calculus of Variations, 9 (2003), pp. 207–216. 7. F. Colonius and L. Gr¨ une, eds., Dynamics, Bifurcations and Control, SpringerVerlag, 2002. 8. F. Colonius and W. Kliemann, The Dynamics of Control, Birkh¨ auser, 2000. 9. , Mergers of control sets, in Proceedings of the Fourtheenth International Symposium on the Mathematical Theory of Networks and Systems (MTNS), Perpignan, France, June 19–23 2000, A. El Jai and M. Fliess, eds., 2000. 10. , Limits of input-to-state stability, Systems Control Lett., (2003). to appear. 11. F. Colonius and S. Siegmund, Normal forms for control systems at singular points, J. Dynamics Diﬀ. Equations, (2003). to appear. 12. F. Colonius and M. Spadini, Uniqueness of local control sets, 2002. submitted to J. Dynamical and Control Systems. 13. R. Fabbri, R. Johnson, and C. N´ unez, On the Yakubovich frequency theorem for linear non-autonomous processes, Discrete and Continuous Dynamical Systems, (2003). to appear. 14. T. Gayer, Controlled and perturbed systems under parameter variation, 2003. Dissertation, Universit¨ at Augsburg, Augsburg, Germany. 15. G. Grammel and P. Shi, On the asymptotics of the Lyapunov spectrum under singular perturbations, IEEE Trans. Aut. Control, 45 (2000), pp. 565–568. 16. L. Gr¨ une, Asymptotic Behavior of Dynamical and Control Systems under Perturbation and Discretization, Springer-Verlag, 2002. 17. S. Gr¨ unvogel, Lyapunov exponents and control sets, J. Diﬀ. Equations, 187 (2003), pp. 201–225. 18. R. Johnson, P. Kloeden, and R. Pavani, Two-step transition in nonautonomous bifurcations: An explanation, Stochastic and Dynamics, 2 (2002), pp. 67–92. 19. R. Johnson and M. Nerurkar, Controllability, Stabilization, and the Regulator Problem for Random Diﬀerential Systems, vol. 136, No. 646 of Memoirs of the AMS, Amer. Math. Soc., 1998. 20. R. A. Johnson, K. J. Palmer, and G. R. Sell, Ergodic properties of linear dynamical systems, SIAM J. Math. Anal., 18 (1987), pp. 1–33. 21. W. Kang, Bifurcation and normal form of nonlinear control systems – part I and part II, SIAM J. Control Optim., 36 (1998), pp. 193–212 and 213–232. 22. K. R. Meyer and G. R. Sell, Melnikov transforms, Bernoulli bundles and almost periodic perturbations, Trans. Amer. Math. Soc., 129 (1989), pp. 63–105. 23. P. Ruﬃno, Rotation numbers for stochastic dynamical systems, Stochastics and Stochastics Reports, 60 (1997), pp. 289–318.

Bifurcations of Control Systems: A View from Control Flows

35

24. R. J. Sacker and G. R. Sell, A spectral theory for linear diﬀerential systems, J. Diﬀ. Equations, 27 (1978), pp. 320–358. 25. S. Siegmund, Normal forms for nonautonomous diﬀerential equations, J. Diﬀ. Equations, 14 (2002), pp. 243–258.

Practical Stabilization of Systems with a Fold Control Bifurcation Boumediene Hamzi1,2 and Arthur J. Krener2 1 2

INRIA, Domaine de Voluceau, Rocquencourt, BP 105, 78153 Le Chesnay Cedex, France, [email protected] Department of Mathematics, University of California, One Shields Avenue, Davis, CA 95616, USA, [email protected]

1 Introduction Nonlinear parameterized dynamical systems exhibit complicated performance around bifurcation points. As the parameter of a system is varied, changes may occur in the qualitative structure of its solutions around an equilibrium point. Usually, this happens when some eigenvalues of the linearized system cross the imaginary axis as the parameter changes [7]. For control systems, a change of some control properties may occur around an equilibrium point, when there is a lack of linear stabilizability at this point. This is called a control bifurcation [21]. A control bifurcation occurs also for unparameterized control systems. In this case, it is the control that plays the parameter’s role in parameterized dynamical systems. The use of feedback to stabilize a system with bifurcation has been studied by several authors, and some fundamental results can be found in [1], [2], [3], [6], [16], [8], [17],[11],[12], [10], the Ph.D. theses [13], [21], [22] and the references therein. When the uncontrollable modes are on the imaginary axis, asymptotic stabilization of the solution is possible under certain conditions, but when the uncontrollable modes have a positive real part, asymptotic stabilization is impossible to obtain by smooth feedback [4]. In this paper, we show that by combining center manifold techniques with the normal forms approach, it is possible to practically stabilize systems with a fold control bifurcation [21], i.e. those with one slightly unstable uncontrollable mode. The methodology is based on using a class C 0 feedback to obtain a bird foot bifurcation ([20]) in the dynamics of the closed loop system on the center manifold. Systems with a fold control bifurcation appear in applications. For example, in [25] a fold bifurcation appears at the point of passage from minimum phase to nonminimum phase. W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 37–48, 2003.

c Springer-Verlag Berlin Heidelberg 2003

38

B. Hamzi and A.J. Krener

The paper is divided as follows : in Section §1, we introduce deﬁnitions of ε−Practical Stability, ε−Practical Stabilizability and Practical Stabilizability ; then, in Section §2, we show that a continuous but non diﬀerentiable control law permits the practical stabilization of systems with a fold control bifurcation.

2 Practical Stability and Practical Stabilizability Practical stability was introduced in [23] and is deﬁned as the convergence of the solution of a diﬀerential equation to a neighborhood of the origin. In this section, we propose deﬁnitions for practical stability and practical stabilizability. Let us ﬁrst deﬁne class K, K∞ and KL functions. Deﬁnition 1. [18, deﬁnitions 3.3, 3.4] •

A continuous function α : [0, a) → [0, ∞) is said to belong to class K if it is strictly increasing and α(0) = 0. It is said to belong to class K∞ if a = ∞ and limr→∞ α(r) = ∞. • A continuous function β : [0, a) × [0, ∞) → [0, ∞) is said to belong to class KL if, for each ﬁxed s, the mapping β(r, s) belongs to class K with respect to r; and, for each ﬁxed r, the mapping β(r, s) is decreasing with respect to s and limr→∞ β(r, s) = 0. Let D ⊂ Rn be an open set containing the closed ball Bε of radius ε centered at the origin. Let f : D → Rn a continuous function such that f (0) = 0. Consider the system x˙ = f (x). Deﬁnition 2. (ε−Practical Stability) The origin is said to be locally ε−practically stable, if there exists an open set D containing the closed ball Bε , a class KL function ζ and a positive constant δ = δ(ε), such that for any initial condition x(0) with ||x(0)|| < δ, the solution x(t) of (2) exists and satisﬁes dBε (x(t)) ≤ ζ(dBε (x(0)), t),

∀t ≥ 0,

with dBε (x(t)) = inf ρ∈Bε d(x(t), ρ), the usual point to set distance. Now consider the controlled system x˙ = f (x, v), with f : D × U → Rn , f (0, 0) = 0, and U ⊂ Rm a domain that contains the origin.

Practical Stabilization of Systems with a Fold Control Bifurcation

39

Deﬁnition 3. (ε−Practical Stabilizability) The system (2) is said to be locally ε−practically stabilizable around the origin, if there exists a control law v = kε (x), such that the origin of the closed-loop system x˙ = f (x, kε (x)) is locally ε−practically stable. Deﬁnition 4. (Practical Stabilizability) The system (2) is said to be locally practically stabilizable around the origin, if it is locally ε−practically stabilizable for every ε > 0. If, in the preceding deﬁnitions, D = Rn , then the corresponding properties of ε−practical stability, ε−practical stabilizability and practical stabilizability are global, and the adverb “locally” is omitted. Now, let us reformulate the local ε−practical stability in the Lyapunov framework. Let V be a function V : D → R+ , such that V is smooth on D \ Bε , and satisﬁes x ∈ D =⇒ α1 (dBε (x(t))) ≤ V (x) ≤ α2 (dBε (x(t))), with α1 and α2 class K functions. Such a function is called a Lyapunov function with respect to Bε , if there exists a class K function α3 such that V˙ (x) = Lf (x) V (x) ≤ −α3 (dBε (x)), for x ∈ D \ Bε . Proposition 1. The origin of system (2) is ε−practically stable if and only if there exists a Lyapunov function with respect to Bε . Proof. In [24], the authors gave stability results for systems with respect to a closed/compact, invariant set A. In particular, the deﬁnition of asymptotic stability and Lyapunov function with respect to A were given. In the case where1 A = Bε , asymptotic stability with respect to Bε reduces to our deﬁnition of ε−practical stability (deﬁnition 2). The proof of our proposition is obtained by applying a local version of [24, Theorem 2.9]. If D = Rn and α1 , α2 are a class K∞ functions, the origin is globally ε−practically stable. Remark : When ε = 0, we recover the classical deﬁnitions of local and global asymptotic stability.

3 Systems with a Fold Control Bifurcation In this section, we apply the ideas of the preceding section to the system (2) when its linearization is uncontrollable at an equilibrium, which we take to 1

Bε is a nonempty, compact and invariant set. It is invariant since V˙ is negative on its boundary ; so, a solution starting in Bε remains in it.

40

B. Hamzi and A.J. Krener

be the origin. Suppose that m = 1 and that the linearization of the system (2) at the origin is (A, B) with A=

∂f ∂f (0, 0), B = (0, 0), ∂x ∂v

and rank([B AB A2 B · · · An−1 B]) = n − 1. According to the assumption in (3), the linear system is uncontrollable. Suppose that the uncontrollable mode λ satisﬁes the following assumption Assumption : The uncontrollable mode is λ ∈ R≥0 . Let us denote as ΣU , the system (2) under the above assumption. This system exhibits a fold control bifurcation when λ > 0, and, generically, a transcontrollable bifurcation when λ = 0 (see [21]). From linear control theory [14], we know that there exist a linear change of coordinates and a linear feedback that put the system ΣU in the following form z¯˙ 1 = λ¯ z1 + O(¯ z1 , z¯2 , u ¯)2 , ¯ + O(¯ z1 , z¯2 , u ¯)2 , z¯˙ 2 = A2 z¯2 + B2 u with z¯1 ∈ R, z¯2 ∈ R(n−1)×1 , A2 ∈ R(n−1)×(n−1) and B2 ∈ R(n−1)×1 . The matrices A2 and B2 are given by     0 1 0 ··· 0 0 0 0 1 ··· 0 0         A2 =  ... ... ... . . . ...  , B2 =  ...  .     0 0 0 ··· 1 0 0 0 0 ··· 0 1 To simplify the quadratic part, we use the following quadratic transformations in order to transform the system to its quadratic normal form * + * + z1 z¯ = 1 − φ[2] (¯ z1 , z¯2 ), (1) z2 z¯2 u=u ¯ − α[2] (¯ z1 , z¯2 , u ¯).

(2)

The normal form is given in the following theorem Theorem 1. [21, Theorem 2.1] For the system ΣU whose linear part is of the form (3), there exist a quadratic change of coordinates (1) and feedback (2) which transform the system to

Practical Stabilization of Systems with a Fold Control Bifurcation

z˙1 = λz1 + βz12 + γz1 z2,1 + z˙2 = A2 z2 + B2 u +

n−1

n

41

2 δj z2,j + O(z1 , z2 , u)3 ,

j=1

n

i=1 j=i+2

2 θij z2,j ei2 + O(z1 , z2 , u)3 ,

with β, γ, δj , θij are constant coeﬃcients, z2,n = u, and ei2 is the ith − unit vector in the z2 −space. Let us consider the piecewise linear feedback u = K1 (z1 )z1 + K2 z2 + O(z1 , z2 )2 , with

¯ k1 , z1 ≥ 0, K1 (z1 ) = ˜ k1 , z1 < 0.

We wish to stabilize the system around the bifurcation point. The controllable part can be made asymptotically stable by choosing K2 such that Property P: The matrix A¯2 = A2 + B2 K2 is Hurwitz. Under the feedback (3), the system ΣU has n−1 eigenvalues with negative real parts, and one eigenvalue with positive real part, the uncontrollable mode λ. Nevertheless, if we view the system ΣU as being parameterized by λ, and by considering λ as an extra-state, satisfying the equation λ˙ = 0, the system ΣU under the feedback (3) possesses two eigenvalues with zero real part and n − 1 eigenvalues in the left half plane. Theorem 2. Consider the closed-loop system (1)-(3), then there exists a center manifold deﬁned by z2 = Π(z1 , λ) whose linear part is determined by the feedback (3). Proof. By considering λ as an extra state, the linear part of the dynamics (1)-(3) is given by λ˙ = 0, z˙1 = O(λ, z1 , z2 )2 , z˙2 = B2 K1 (z1 )z1 + A¯2 z2 + O(z1 , z2 )2 . λz1 is now considered as a second order term. Let Σk¯1 (resp. Σk˜1 ) be the system (3) when K1 (z1 ) = k¯1 (resp. K1 (z1 ) = ˜ k1 ) for all z1 . Since the system Σk¯1 (resp. Σk˜1 ) is smooth, and possesses two eigenvalues on the imaginary axis and n − 2 eigenvalues in the open left half plane ; then, from the center manifold theorem, in a neighborhood of the c , c ). origin, Σk¯1 (resp. Σk˜1 ) has a center manifold W (resp. W For Σk¯1 , the center manifold is represented by z2 = Π(λ, z1 ), for λ and z1 suﬃciently small. Its equation is

42

B. Hamzi and A.J. Krener

z˙2 = A2 Π(λ, z1 ) + B2 (k¯1 z1 + K2 Π(λ, z1 )) + O(z1 , z2 )2 , ∂ Π(λ, z1 ) = z˙1 = O(z1 , z2 )2 . ∂ z1 Since λ˙ = 0 and λz1 is a second order term in the enlarged space (λ, z1 , z2 ), then there is no linear term in λ in the linear part of the center manifold. [1] Hence, the linear part of the center manifold is of the form z2 = Π z1 , and [1] its i−th component is z2,i = Π i z1 , for i = 1, · · · , n − 1. Using (3) we obtain [1]

[1]

¯

1 that Π 1 = − kk2,1 and Π i = 0, for 2 ≤ i ≤ n − 1.

z1 ). Similarly for Σk˜1 , the center manifold is represented by z2 = Π(λ, [1] - z1 , whose components are deﬁned by Its linear part is given by z2 = Π - [1] = − k˜1 and Π - [1] = 0, for 2 ≤ i ≤ n − 1. Π 1 i k2,1 Since A¯2 has no eigenvalues on the imaginary axis, and k2,1 is the product of all the eigenvalues of A¯2 , then k2,1 = 0. c , c intersect along the line z1 = 0. Indeed, The center manifolds W and W k k ∂ Π(λ,z ) 1) since λ˙ = 0, then ∂ λk 1 |λ=0,z1 =0 = 0 and ∂ Π(λ,z |λ=0,z1 =0 = 0, for k ≥ 1. ∂ λk So, Π(λ, z1 )|z1 =0 = 0 and Π(λ, z1 )|z1 =0 = 0, for all λ. c Hence, if we slice them along the line z1 = 0 and then glue the part of W , c for which z1 < 0, along this line, we for which z1 > 0 with the part of W deduce that in an open neighborhood of the origin, D, the piecewise smooth system (3) has a piecewise smooth center manifold Wc . The linear part of the center manifold Wc is represented by z2 = Π [1] z1 . The i − th component of z2 , z2,i , is given by [1]

z2,i = Πi (z1 )z1 , with [1]

Π1 (z1 ) = −

K1 (z1 ) [1] and Πi (z1 ) = 0, for i ≥ 2. k2,1

Using (1) and (3), the reduced dynamics on the center manifold is given by

. z˙1 =

[1]

λz1 + Φ(Π 1 )z12 + O(z13 ), z1 ≥ 0, - [1] )z 2 + O(z 3 ), z1 < 0, λz1 + Φ(Π 1 1 1

with Φ the function deﬁned by Φ(X) = β + γX + δ1 X 2 . The following theorem shows that the origin of the system (3) can be made practically stable, for small λ > 0, and asymptotically stable if λ = 0. Theorem 3. Consider system (1) with γ 2 − 4βδ1 > 0, then, the piecewise linear feedback (3) practically stabilizes the system around the origin for small λ > 0, and locally asymptotically stabilizes the system when λ = 0.

Practical Stabilization of Systems with a Fold Control Bifurcation

43

Proof. See appendix.

[1] - [1] such that Φ(Π [1] ) = −Φ(Π - [1] ) = Φ0 , the dynaIf we choose Π 1 and Π 1 1 1 mics (3) will be of the form

z˙1 = µz1 − Φ0 |z1 |z1 + O(z13 ),

(3)

with µ ∈ R a parameter. The equation (3) is the normal form of the bird foot bifurcation, introduced by Krener in [20]. If Φ0 > 0, the equation (3) corresponds to a supercritical bird foot bifurcation. For µ < 0, there is one equilibrium at z1 = 0 which is exponentially stable. For µ > 0, there are two exponentially stable equilibria at z1 = ± Φµ0 , and one exponentially unstable equilibrium at z1 = 0. For µ = 0, there is one equilibrium at z1 = 0 which is asymptotically stable but not exponentially stable. If Φ0 < 0, the equation (3) is an example of subcritical bird foot bifurcation. For µ < 0, there is one equilibrium at z1 = 0 which is exponentially stable and two exponentially unstable equilibria at z1 = ± Φµ0 . For µ > 0, there is one exponentially unstable equilibrium at z1 = 0. For µ = 0, there is one equilibrium at z1 = 0 which is unstable. Notice that both normal forms are invariant under the transformation z1 → −z1 and so the bifurcation diagrams can be obtained by reﬂecting the upper or lower half of the bifurcation diagram of a transcritical bifurcation. In both cases the bifurcation diagrams look like the foot of a bird. In the λ − z1 plane, the dynamics (3) are in the form (3) with Φ0 > 0. A supercritical birdfoot bifurcation appears at (λ, z1 ) = (0, 0). For λ > 0, we have 3 equilibrium points : the origin and ±ε (corresponding to the solutions of z˙1 = 0). The origin is unstable for λ > 0, and the two other equilibrium points are stable (cf. Figure 1). The practical stabilization of the system is made possible by making the two new equilibrium points suﬃciently close to [1] - [1] ) suﬃciently large. the origin, i.e. by choosing Φ(Π 1 ) and Φ(Π 1 If a quadratic feedback was used instead of (3), i.e. ¯ 1 z1 + K2 z2 + z T Qf b z1 + O(z 3 ), u=K 1 1 we can prove that the closed loop dynamics has a center manifold. Moreover, ¯ 1 , the reduced dynamics on the center manifold by appropriately choosing K will have the form z˙1 = λz1 − Φ1 z13 + O(z14 ), with Φ1 > 0, by appropriately choosing Qf b . The equation (3) is the normal form of a system exhibiting a supercritical pitchfork bifurcation. By using a similar analysis as above, we deduce that the /solution of the reduced dynamics converges to the equilibrium points ε = ± Φλ1 , and that the closed-loop system (1)-(3) is practically stabilizable.

44

B. Hamzi and A.J. Krener

The reason of the choice of a piecewise linear feedback instead of a quadratic feedback is that it is preferable to have a supercritical bird foot bifurcation than a supercritical pitchfork bifurcation. This is due to the fact that the sta√ ble equilibria in a system with a bird foot bifurcation grow like µ not like µ as in the pitchfork bifurcation2 , and that the bird foot bifurcation is robust to small quadratic perturbations, while these transform the pitchfork bifurcation to a transcritical one.

Fig. 1.

4 Appendix Proof of Theorem 3 Consider the Lyapunov function V (z1 ) = ε2 −

λ . [1] ) Φ(Π 1

1 2 z , and let ε1 − λ[1] and Φ(Π 1 ) 2 1

Then, from (3), we have . V˙ =

[1]

Φ(Π 1 )(z1 − ε1 )z12 + O(z14 ), z1 ≥ 0, - [1] )(z1 − ε2 )z 2 + O(z 4 ), z1 < 0, Φ(Π 1 1 1

• Practical Stabilization for λ > 0 [1] - [1] such that Φ(Π [1] ) < 0 and Φ(Π - [1] ) > 0, we By choosing3 Π 1 and Π 1 1 1 get ε1 > 0 and ε2 < 0. This choice is always possible since Φ is a second order polynomial whose discriminant, γ 2 − 4βδ1 , is positive ; so, Φ takes both positive and negative values. In this case, V˙ < 0 for z1 > ε1 and z1 < ε2 , and V˙ = 0 for z1 = ε1 or z1 = ε2 . 2 3

Let us recall that the normal for of a pitchfork bifurcation is z˙1 = µz1 − Φ1 z13 , with µ ∈ R the parameter. ¯1 and k ˜1 of the feedback (3) linked This choice is made by ﬁxing the parameters k [1] [1] through (3). to Π 1 and Π 1

Practical Stabilization of Systems with a Fold Control Bifurcation

45

[1] - [1] In the following, and without loss of generality, we choose Π 1 and Π 1 [1] [1] - ), so ε1 = −ε2 ε, with 0 ≤ ε ≤ r, and r is the such that Φ(Π 1 ) = −Φ(Π 1 radius of Br , the largest closed ball contained in D. Let Ω1 and Ω2 be two sets deﬁned by Ω1 =]ε, +r] and Ω2 = [−r, −ε[. If z1 (0) ∈ Ω1 ∪ Ω2 , and since V˙ < 0 on Ω1 ∪ Ω2 , then, from (2) and (2), V˙ satisﬁes

V˙ ≤ −α3 (||z1 ||) ≤ −α3 (α2−1 (V )). Since α2 and α3 are a class K functions, then α3 (α2−1 ) is also a class K function. Hence, using the comparison principle in [24, lemma 4.4], there exists a class KL function η such that V (z1 (t)) ≤ η(V (z1 (0), t)). The sets Ω 1 = [0, ε] and Ω 2 = [−ε, 0] have the property that when a solution enters either set, it remains in it. This is due to the fact that V˙ is negative deﬁnite on the boundary of these two sets. For the same reason, if z1 (0) ∈ Ω 1 (resp. z1 (0) ∈ Ω 2 ), then z1 (t) ∈ Ω 1 (resp. z1 (t) ∈ Ω 2 ), for t ≥ 0. Let T (ε) be the ﬁrst time such that the solution enters Ω 1 ∪ Ω 2 = Bε . Using (2) and (4), we get that for 0 ≤ t ≤ T (ε), ε ≤ ||z1 (t)|| ≤ α1−1 (V (z1 (t)) ≤ α1−1 (η(V (z1 (0), t))) ζ(z1 (0), t). The function ζ is a class KL function, since α1 is a class K function and η a class KL function. Since ζ is a class KL function, then T (ε) is ﬁnite. Hence, z1 (t) ∈ Ω 1 ∪ Ω 2 , for t ≥ T (ε). Hence, for z1 ∈ Br , the solution satisﬁes dBε (z1 (t)) ≤ ζ(dBε (z1 (0)), t). So, in Br , the origin is locally ε−practically stable. Now, consider the whole closed-loop dynamics z˙1 = λz1 + βz12 + γz1 z2,1 + z˙2 = B2 K1 z1 + A¯2 z2 +

n−1

2 δi z2,i + O(z1 , z2 )3 ,

i=1 n−1 n−1 i=1 j=i+2

2 θij z2,j ei2 + O(z1 , z2 )3 .

Let w1 = z1 , w2 = z2 − Π [1] z1 , and w = (w1 , w2 )T . Then, the closed-loop dynamics is given by . [1] ¯1 (w1 , w2 ), for w1 ≥ 0, λw1 + Φ(Π 1 )w12 + N w˙ 1 = [1] 2 ˜ ˜ λw1 + Φ(Π1 )w1 + N1 (w1 , w2 ), for w1 < 0.

46

B. Hamzi and A.J. Krener

w˙ 2 =

¯2 (w1 , w2 ), for w1 ≥ 0, A¯2 w2 + N ˜2 (w1 , w2 ), for w1 < 0. A¯2 w2 + N

Let Ni (w1 , w2 ) =

¯i (w1 , w2 ), N ˜i (w1 , w2 ), N [1]

w1 ≥ 0, for i = 1, 2, w1 < 0, n−1 2 + i=1 δi w2,i and N2 (w1 , w2 ) =

with N1 (w1 , w2 ) = (γ + 2δ1 Π1 )w1 w2,1 n−1 n−1 j 2 i i=1 j=i+2 θi w2,j e2 . i Since Ni (w1 , 0) = 0 and ∂N ∂w2 (0, 0) = 0 (i = 1, 2), then in the domain ||w||2 < σ, N1 and N2 satisfy Ni (w1 , w2 ) ≤ κi ||w2 ||,

i = 1, 2,

where κ1 and κ2 can be arbitrarily small by making σ suﬃciently small. Since A¯2 is Hurwitz, there exists a unique P such that AT2 P + P A2 = −I. Let V be the following composite Lyapunov function V(w1 , w2 ) =

1 2 w + w2T P w2 . 2 1

The derivative of V along the trajectories of the system is given by ˙ 1 , w2 ) = π(w1 ) + w1 N1 (w1 , w2 ) + wT (A¯T P + P A¯2 )w2 + 2wT P N2 (w1 , w2 ), V(w 2 2 2 [1] - [1] )w1 )w2 with π(w1 ) = (λ + Φ(Π 1 )w1 )w12 for w1 ≥ 0 and π(w1 ) = (λ + Φ(Π 1 1 for w1 < 0. For w1 ∈ Ω1 ∪ Ω2 , then π(w1 ) ≤ −α3 (||w1 ||) according to (4). Hence

˙ 1 , w2 ) < −α3 (||w1 ||) + w1 N1 (w1 , w2 ) + w2T (P A¯2 + P A¯2 )w2 + 2w2T P N2 (w1 , w2 ), V(w ≤ −(−κ1 ν + 1 + 2κ2 λmax (P ))||w2 || + κ2 ||w2 || − κ2 ||w2 ||, ≤ −(−κ1 ν − κ2 + 1 − 2κ2 λmax (P ))||w2 ||,

with ν = max{w1 :w1 ∈Ω1 ∪Ω2 } ||w1 ||. By choosing κ1 and κ2 such that κ1 ν + κ2 (1 + 2λmax (P )) < 1, then ˙ 1 , w2 ) < 0. V(w ˙ 1 , w2 ) < 0. So, there exists a class KL Hence, for w1 ∈ Ω1 ∪ Ω2 , V(w function η¯ such that ||w(t)|| ≤ η¯(||w(0)||, t). When w1 ∈ Ω 1 ∪ Ω 2 , and by considering w1 as an input of the system w˙ 2 = A¯2 w2 + N2 (w1 , w2 ),

Practical Stabilization of Systems with a Fold Control Bifurcation

47

we deduce that ||w2 || is bounded, since A¯2 is Hurwitz. Hence, for w1 ∈ Ω 1 ∪Ω 2 , there exists ε¯ such that ||w(t)|| ≤ ε¯. From (4)-(4) we obtain dBε¯ (w(t)) ≤ η¯(dBε¯ (w(0)), t). So the origin of the whole dynamics is locally ε¯−practically stable. • Asymptotic Stabilization for λ = 0 In this case, generically, we have a transcontrollable bifurcation [17, 21]. Since ε1 = ε2 = 0, the sets Ω 1 and Ω 2 reduce to the origin. Hence, the origin of the reduced closed-loop system is asymptotically stable, since the solution ¯1 ∪ Ω ¯2 = {0}. We deduce that the origin of the whole closed-loop converges to Ω dynamics is asymptotically stable by applying the center manifold theorem [5].

References 1. Abed, E. H. and Fu J.-H. (1986). Local Feedback stabilization and bifurcation control, part I. Hopf Bifurcation, Systems and Control Letters, 7, 11–17. 2. Abed, E. H. and Fu J.-H. (1987). Local Feedback stabilization and bifurcation control, part II. Stationary Bifurcation, Systems and Control Letters, 8, 467– 473. 3. Aeyels, D. (1985). Stabilization of a class of nonlinear systems by a smooth feedback control, Systems and Control Letters, 5, 289–294. 4. Brockett, R. (1983). Asymptotic Stability and Feedback Stabilization, In R. W. Brockett, R. Milman and H. Sussman Eds., Diﬀerential Geometric Control Theory, Birkhauser. 5. Carr, J. (1981). Application of Centre Manifold Theory, Springer. 6. Colonius, F., and W. Kliemann (1995). Controllability and stabilization of onedimensional systems near bifurcation points,Systems and Control Letters, 24, 87–95. 7. Guckenheimer, J. and P. Holmes (1983). Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, Springer. 8. Gu, G., X. Chen, A. G. Sparks and S. S. Banda (1999). Bifurcation Stabilization with Local Output Feedback, Siam J. Control and Optimization, 37, 934–956. 9. Hahn, W. (1967). Stability of Motion, Springer. 10. Hamzi, B., J.-P. Barbot, S. Monaco, and D. Normand-Cyrot (2001). Nonlinear Discrete-Time Control of Systems with a Naimark-Sacker Bifurcation, Systems and Control Letters, 44, 245–258. 11. Hamzi, B., W. Kang and J.-P. Barbot (2003). Analysis and Control of Hopf bifurcations, Siam J. Control and Optimization, to appear. 12. Hamzi, B. and W. Kang (2003). Resonant Terms and Bifurcations of Nonlinear Control Systems with One Uncontrollable Mode, Systems and Control Letters, to appear.

48

B. Hamzi and A.J. Krener

13. Hamzi, B. (2001). Analyse et commande des syst`emes non lin´eaires non commandables en premi`ere approximation dans le cadre de la th´ eorie des bifurcations, Ph.D. Thesis, University of Paris XI-Orsay, France. 14. Kailath, T. (1980). Linear Systems, Prentice-Hall. 15. Kang, W. and A.J. Krener (1992). Extended Quadratic Controller Normal Form and Dynamic State Feedback Linearization of Nonlinear Systems, Siam J. Control and Optimization, 30, 1319–1337. 16. Kang, W. (1998). Bifurcation and Normal Form of Nonlinear Control Systemspart I/II, Siam J. Control and Optimization, 36, 193–212/213–232. 17. Kang, W. (2000). Bifurcation Control via State Feedback for Systems with a Single Uncontrollable Mode, Siam J. Control and Optimization, 38, 1428–1452. 18. Khalil, H.K. (1996). Nonlinear Systems, Prentice-Hall. 19. Krener, A. J. (1984). Approximate linearization by state feedback and coordinate change, Systems and Control Letters, 5, 181–185. 20. Krener, A. J. (1995). The Feedbacks which Soften the Primary Bifurcation of MG 3, PRET Working Paper D95-9-11, 181-185. 21. Krener, A.J., W. Kang, and D.E. Chang (2001). Control Bifurcations, accepted for publication in IEEE trans. on Automatic Control. 22. Krener, A.J. and L. Li (2002). Normal Forms and Bifurcations of Discrete Time Nonlinear Control Systems, SIAM J. on Control and Optimization, 40, 1697– 1723. 23. Lakshmikantham,V., S. Leela, and A.A. Martynyuk (1990). Practical stability of nonlinear systems, World Scientiﬁc. 24. Lin, Y., Y. Wang, and E. Sontag (1996). A smooth converse Lyapunov theorem for robust stability, Siam J. Control and Optimization, 34, 124–160. 25. Szederk´enyi, G., N. R. Kristensen, K. M. Hangos and S. Bay Jorgensen (2002). Nonlinear analysis and control of a continuous fermentation process, Computers and Chemical Engineering, 26, 659–670.

Feedback Control of Border Collision Bifurcations Munther A. Hassouneh and Eyad H. Abed Department of Electrical and Computer Engineering, and the Institute for Systems Research, University of Maryland, College Park, MD 20742 USA, {munther, abed}@eng.umd.edu,

—Dedicated to Professor Arthur J. Krener on the occasion of his 60th birthday Summary. The feedback control of border collision bifurcations is considered. These bifurcations can occur when a ﬁxed point of a piecewise smooth system crosses the border between two regions of smooth operation. The goal of the control eﬀort in this work is to modify the bifurcation so that the bifurcated steady state is locally attracting. In this way, the system’s local behavior is ensured to remain stable and close to the original operating condition. Linear and piecewise linear feedbacks are used since the system linearization on the two sides of the border generically determines the type and stability properties of any border collision bifurcation. A two-dimensional example on quenching of heart arrhythmia is used to illustrate the ideas.

1 Introduction The purpose of this paper is to study the feedback control of border collision bifurcations (BCBs) in piecewise smooth (PWS) maps. The goal of the control eﬀort in this work is to modify the bifurcation so that the bifurcated steady state is locally attracting. In this way, the system’s local behavior is ensured to remain stable and close to the original operating condition. Another contribution of the paper is to summarize some available results on BCBs in more detail than exists in the literature, and to supply additional results that are useful for control design. Continuous piecewise-smooth dynamical systems have been found to undergo special bifurcations along the borders between regions of smooth dynamics. These have been named border collision bifurcations by Nusse and Yorke [14], and had been studied in the Russian literature under the name C-bifurcations by Feigin [9]. Di Bernardo, Feigin, Hogan and Homer [3] introduced Feigin’s results in the Western literature. W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 49–64, 2003.

c Springer-Verlag Berlin Heidelberg 2003

50

M.A. Hassouneh and E.H. Abed

Border collision bifurcations include bifurcations that are reminiscent of the classical bifurcations in smooth systems such as fold and period doubling bifurcations. Despite this resemblance, the classiﬁcation of border collision bifurcations is far from complete, and certainly very preliminary in comparison to the results available in the smooth case. The classiﬁcation is complete only for one-dimensional discrete-time systems [15, 2]. Concerning two-dimensional piecewise smooth maps, Banerjee and Grebogi [1] propose a classiﬁcation for a class of two-dimensional maps undergoing border-collision by exploiting a normal form. One result on BCB for two-dimensional maps that has been mentioned but not carefully proved in the literature has recently been proved by the authors in joint work with H. Nusse [13]. This result asserts local uniqueness and stability of ﬁxed points in the case of real eigenvalues in (−1, 1) on both sides of the border. For higher dimensional systems, currently the known results are limited to several general observations. It should be noted that work such as that in this paper focusing on maps has implications for switched continuous-time systems as well. Maps provide a concise representation that facilitates the investigation of system behavior and control design. They are also the natural models for many applications. Even for a continuous piecewise smooth system, a control design derived using the map representation can be translated to a continuous controller either analytically or numerically. There is little past work on control of BCBs; we are aware of the papers by Di Bernardo [4], Di Bernardo and Chen [5] and our work [11, 12]. The present paper summarizes some results in our manuscripts [11, 12] which go further than [4, 5] by doing a systematic feedback design approach and by using a more detailed classiﬁcation of BCBs. In [11, 12], we consider design of feedbacks that achieve safe BCBs for one-dimensional and two-dimensional discrete-time systems. This could entail feedback on either side of the border or on both sides. Suﬃcient conditions for stabilizing control gains are found analytically. This paper is organized as follows. In Sect. 2, we summarize results on BCBs in one-dimensional maps and discuss the available results in twodimensional PWS maps. In Sect. 3, we develop feedback control laws for BCBs in one-dimensional maps. In Sect. 4, the results are applied to a model that has been used in studies of cardiac arrhythmia. In Sect. 5, we collect concluding remarks and mention some problems for future research.

2 Background on Border Collision Bifurcations In this section, relevant results on BCBs are recalled (including one whose proof has just been reported [13]). We begin with results on BCBs in onedimensional (1-D) maps followed by the result on BCBs in two-dimensional (2-D) maps proved in [13]. Since the 1-D case is well understood, we are able to give a detailed description of the possible scenarios in this case. The discussion

Feedback Control of Border Collision Bifurcations

51

of the 2-D case is more brief in that the focus is only on stating the needed result from [13]. 2.1 BCBs in One-Dimensional PWS Maps The presentation below on BCBs in one-dimensional maps closely follows [15, 2], with only cosmetic modiﬁcations. See [15, 2] for more details. Consider the 1-D PWS map xk+1 = f (xk , µ)

(1)

where x ∈ , µ is a scalar bifurcation parameter, and f (x, µ) takes the form g(x, µ), x ≤ xb f (x, µ) = (2) h(x, µ), x ≥ xb Since the system is one-dimensional, the border is just the point xb . The map f : × → is assumed to be PWS: f depends smoothly on x everywhere except at xb , where it is continuous in x. It is also assumed that f depends smoothly on µ everywhere. Denote by RL and RR the two regions in state space separated by the border: RL := {x : x ≤ xb } and RR := {x : x ≥ xb }. Let x0 (µ) be a path of ﬁxed points of f ; this path depends continuously on µ. Suppose also that the ﬁxed point hits the boundary at a critical parameter value µb : x0 (µb ) = xb . Below, conditions are recalled for the occurrence of various types of BCBs from xb for µ near µb . The normal form for the PWS map (1) at a ﬁxed point on the border is a piecewise aﬃne approximation of the map in the neighborhood of the border point xb , in scaled coordinates [15, 2, 3]. The state and parameter transformations occurring in the derivation of the normal form are needed in applying control results derived for systems in normal form to a system in original coordinates. In the interest of brevity, these transformations are not recalled here. The 1-D normal form is [2] axk + µ, xk ≤ 0 xk+1 = G1 (xk , µ) = (3) bxk + µ, xk ≥ 0 b) b) where a = limx→x− ∂f (x,µ and b = limx→x+ ∂f (x,µ . Suppose that |a| = ∂x ∂x b b 1 and |b| = 1. The normal form map G1 (·, ·) can be used to study local bifurcations of the original map f (·, ·) [15, 2]. Note that the original map (2) is not necessarily piecewise aﬃne. Denote by x∗R and x∗L the ﬁxed points of the system near the border to the right (x > xb ) and left (x < xb ) of the border, respectively. Then in the µ µ and x∗L = 1−a . For the ﬁxed point x∗R to actually normal form (3), x∗R = 1−b µ occur, we need 1−b ≥ 0 which is satisﬁed if and only if µ > 0 and b < 1 or

52

M.A. Hassouneh and E.H. Abed

µ µ < 0 and b > 1. Similarly, for x∗L to actually occur, we need 1−a ≤ 0 which is satisﬁed if and only if µ < 0 and a < 1 or µ > 0 and a > 1. Various combinations of the parameters a and b lead to diﬀerent kinds of bifurcation behavior as µ is varied. Since the map G1 is invariant under the transformation x → −x, µ → −µ, a b, it suﬃces to consider only the case a ≥ b. The possible bifurcation scenarios are summarized in Fig. 1. Sample bifurcation diagrams for border collision pair bifurcation (similar to saddle node bifurcation in smooth maps) and period doubling BCB in PWS 1-D maps are depicted in Figs. 2 and 3, respectively. In these ﬁgures, a solid line represents a stable ﬁxed point whereas a dashed line represents an unstable ﬁxed point. All the results pertain to system (3). Detailed descriptions of these bifurcations can be found in [11].

b

−1

+1

a=b

B3

b= a a+1

B1

A2

B2

+1

C3

ab =−1 a ab =1

A2

C1

A1

C2

−1

B2

C1 C3 C2

B1

ab =−1

b = −a a−1 B3

Fig. 1. The partitioning of the parameter space into regions with the same qualitative phenomena. The numbers on diﬀerent regions refer to various bifurcation scenarios (associated parameter ranges are clear from the ﬁgure). Scenario A1: Persistence of stable ﬁxed points (nonbifurcation), Scenario A2: Persistence of unstable ﬁxed points, Scenario B1: Merging and annihilation of stable and unstable ﬁxed points, Scenario B2: Merging and annihilation of two unstable ﬁxed points plus chaos, Scenario B3: Merging and annihilation of two unstable ﬁxed points, Scenario C1: Supercritical border collision period doubling, Scenario C2: Subcritical border collision period doubling, Scenario C3: Emergence of periodic or chaotic attractor from stable ﬁxed point.

The following results give detailed statements relating stability of the ﬁxed point at criticality with the nature of the BCB that occurs. These results, though not diﬃcult to obtain, haven’t previously been stated explicitly in this detail.

Feedback Control of Border Collision Bifurcations

53

(a)

x 0 0 (b)

Growing chaotic attractor for µ>0

x 0 0 (c)

x 0

Unstable chaotic orbit for µ>0

0

µ

Fig. 2. Bifurcation diagrams for Scenarios B1-B3. (a) A typical bifurcation diagram for Scenario B1. (b) A typical bifurcation diagram for Scenario B2. (c) A typical bifurcation diagram for Scenario B3.

x

(a)

stable period-2 orbit

0

0

µ

unstable period-2 orbit

x

(b)

0

0

µ

Fig. 3. Typical bifurcation diagrams for Scenarios C1 and C2. (a) Supercritical period doubling border collision (Scenario C1, b < −1 < a < 1 and −1 < ab < 1) (b) Subcritical period doubling border collision (Scenario C2, b < −1 < a < 0 and ab > 1).

Proposition 1 The origin of (3) at µ = 0 is asymptotically stable if and only if any of (i)-(iii) below holds (i) −1 < a < 1 and −1 < b < 1 (ii) {0 < a < 1 and b < −1} or {0 1} or {−1 1} (v) {−1 < a < 0, b < −1 and ab > 1} or {−1 1} (vi) |a| > 1 and |b| > 1.

54

M.A. Hassouneh and E.H. Abed

Proof (cases (i)-(iii)): Consider the piecewise quadratic Lyapunov function p1 x2k , xk ≤ 0 V (xk ) = (4) p2 x2k , xk > 0 where p1 > 0 and p2 > 0. Clearly, V is positive deﬁnite. To show asymptotic stability of the origin of (3) at criticality (µ = 0), we need to show that the forward diﬀerence ∆V := V (xk+1 ) − V (xk ) is negative deﬁnite along the trajectories of (3) for all xk = 0. There are two cases: Case 1: xk < 0 p1 (x2k+1 − x2k ), xk+1 < 0 ∆V = p2 x2k+1 − p1 x2k , xk+1 > 0 xk+1 < 0 p1 x2k (a2 − 1), (5) = x2k (p2 a2 − p1 ), xk+1 > 0 Case 2: xk > 0

∆V = =

p2 (x2k+1 − x2k ), p1 x2k+1 − p2 x2k , p2 x2k (b2 − 1), x2k (p1 b2 − p2 ),

xk+1 > 0 xk+1 < 0 xk+1 > 0 xk+1 < 0

(6)

It remains to show that ∆V < 0 for all xk = 0 in (i)-(iii). (i) −1 < a < 1 and −1 0. From (5), it follows that ∆V = px2k (a2 −1) < 0 and from (6) it follows that ∆V = px2k (b2 −1) < 0. Thus ∆V < 0 ∀xk = 0. (ii) 0 < a < 1 and b < −1 (the proof for the symmetric case 0 0 then xk+1 = bxk < 0. From (6), ∆V = x2k (p1 b2 − p2 ) < 0 if and only if p2 > p1 b2 > 0. Thus, choosing p1 > 0 and p2 > p1 b2 results in a positive deﬁnite V and a negative deﬁnite ∆V . (iii) −1 < a < 0, b < −1 and ab < 1 (the proof for the symmetric case −1 0. From (5), ∆V = x2k (p2 a2 −p1 ) < 0 if and only if p1 > p2 a2 . Since b < −1, if xk > 0 then xk+1 = bxk < 0. From (6), ∆V = x2k (p1 b2 − p2 ) < 0 if and only if p1 < pb22 . Thus, p1 and p2 must be chosen such that p2 a2 < p1 < pb22 . Clearly, any p2 > 0 works. For p1 > 0 to exist, we need b12 > a2 which is satisﬁed since ab < 1 by hypothesis. Proof (cases (iv)-(vi)): It suﬃces to show that no matter how close the initial condition is to the origin, the trajectory of (3) diverges. (iv) −1 < a < 1 and b > 1 (the proof for the symmetric case −1 1 is similar and therefore omitted): Let x0 = > 0. Then, x1 = b, x2 = b2 and xk = bk . As k → ∞, xk → ∞ no matter how small is.

Feedback Control of Border Collision Bifurcations

55

(v) −1 < a < 0, b < −1 and ab > 1 (the proof for the symmetric case −1 1 is similar and therefore omitted): Let x0 = > 0. It is straightforward to show that x2k = (ab)k . Since ab > 1, x2k → ∞ as k → ∞, for any ﬁxed . (vi) This is an easy exercise and is omitted. The assertions of the next proposition follow from relating the stability of the ﬁxed point at criticality as given in Proposition 1 with the ensuing bifurcation for diﬀerent regions in the (a, b) parameter space as shown in Fig. 1. Proposition 2 1) If the ﬁxed point of system (3) is asymptotically stable at criticality (i.e., at µ = 0), then the border collision bifurcation is supercritical in the sense that no bifurcated orbits occur on the side of the border where the nominal ﬁxed point is stable and the bifurcated solution on the unstable side is attracting. 2) If the ﬁxed point of system (3) is unstable at criticality, then the border collision bifurcation is subcritical in the sense that no bifurcated orbits occur on the side of the border where the nominal ﬁxed point is unstable and the bifurcated solution on the stable side is repelling. 2.2 BCBs in Two-Dimensional PWS Maps Consider a two-dimensional PWS map that involves only two regions of smooth behavior: fA (x, y, µ), (x, y) ∈ RA f (x, y, µ) = (7) fB (x, y, µ), (x, y) ∈ RB Here, µ is the bifurcation parameter and RA and RB are regions of smooth behavior. Since the system is two-dimensional, the border is a curve separating the two regions of smooth behavior and is given by x = h(y, µ). The map f : 2 × → 2 is assumed to be PWS: f depends smoothly on (x, y) everywhere except at the border where it is continuous in (x, y). It is also assumed that f depends smoothly on µ everywhere, and the Jacobian elements are ﬁnite at both sides of the border. Let (x0 (µ), y0 (µ)) be a path of ﬁxed points of f ; this path depends continuously on µ. Suppose also that the ﬁxed point hits the border at a critical parameter value µb . It has been shown [14, 1] that a normal form for the two-dimensional PWS system (7) in the neighborhood of a ﬁxed point on the border takes the form

56

M.A. Hassouneh and E.H. Abed

 τA     −δA    !"  xk+1 = G2 (xk , yk , µ) = JA τB yk+1     −δ  B   !"  JB

1 0 1 0

# #

xk yk xk yk

1 + µ, xk ≤ 0 0

+

1 µ, xk ≥ 0 0

(8)

where τA is the trace and δB is the determinant of the limiting Jacobian matrix JA of the system at a ﬁxed point in RA as it approaches the border. Similarly, τB is the trace and δB is the determinant of the Jacobian matrix JB of the system evaluated at a ﬁxed point in RB near the border. System (8) undergoes a variety of border collision bifurcations depending on the values of the parameters τA , δA , τB and δB . As mentioned previously, only a few results are available on BCBs in twodimensional systems. Next, we state one result that will be needed in the control design of our paper [12], as well as in Sect. 4, in which we consider control of a 2-D model of cardiac arrhythmia. Proposition 3 [13] (Suﬃcient Condition for Nonbifurcation in 2-D PWS Maps) If the eigenvalues of the Jacobian matrices on both sides of the border of a two dimensional PWS map are real in (−1, 1), then a locally unique and stable ﬁxed point on one side of the border leads to a locally unique and stable ﬁxed point on the other side of the border as µ is increased (decreased) through zero.

3 Feedback Control of Border Collision Bifurcations In this section, control of BCBs in PWS maps of dimension one and two is considered. The fact that the normal form for BCBs contains only linear terms in the state leads us to seek linear feedback controllers to modify the system’s bifurcation characteristics. The linear feedback can either be applied on one side of the border and not the other, or on both sides of the border. Both approaches are considered below. The issue of which approach to take and with what constraints is a delicate one. There are practical advantages to applying a feedback on only one side of the border, say the stable side. However, this requires knowledge of where the border lies, which is not necessarily the case in practice. The purpose of pursuing stabilizing feedback acting on both sides of the border is to ensure robustness with respect to model uncertainty. This is done below by investigating the use of simultaneous stabilization as an option — that is, controls are sought that function in exactly the same way on both sides of the border, while stabilizing the system’s behavior. Not surprisingly, the conditions for existence of simultaneously stabilizing controls are more restrictive than for the existence of one sided controls.

Feedback Control of Border Collision Bifurcations

57

Due to space limitations, we only discuss static feedback and do not include details for the 2-D case. Results on the 2-D case and on washout ﬁlter-aided feedback (a form of dynamic feedback) can be found in [12] and [11]. It is important to emphasize that although the control results are based on the normal form, the results can be easily applied to general PWS maps by keeping track of the associated transformations to and from normal form. 3.1 Control of BCB in 1-D Maps Using Static Feedback Consider the one-dimensional normal form (3) for a BCB, repeated here for convenience: axk + µ, xk ≤ 0 xk+1 = (9) bxk + µ, xk ≥ 0 Below, the control schemes described above are considered for the system (9), with a control signal u included in the dynamics as appropriate. Method 1: Control Applied on One Side of Border. In the ﬁrst control scheme, the feedback control is applied only on one side of the border. Suppose that the system is operating at a stable ﬁxed point on one side of the border, locally as the parameter approaches its critical value. Without loss of generality, assume this region of stable operation is {x : x < 0}— that is, assume −1 < a < 1. Since the control is applied only on one side of the border, the linear feedback can be applied either on the unstable side or the stable side of the border. Method (1a): Linear Feedback Applied in Unstable Side of the Border. Suppose that the ﬁxed point is stable if x∗ ∈ − and unstable if x∗ ∈ + . Applying additive linear state feedback only for x ∈ + leads to the closedloop system xk ≤ 0 axk + µ, (10) xk+1 = bxk + µ + uk , xk ≥ 0 uk = γxk

(11)

The following proposition asserts stabilizability of the border collision bifurcation with this type of control policy. Proposition 4 Suppose that the ﬁxed point of (9) is stable in − for µ < 0 (i.e., |a| < 1) and unstable in + for µ > 0 (i.e., b < −1) or the ﬁxed point does not exist for µ > 0 (i.e., b > 1). Then there is a stabilizing linear feedback on the right side of the border. That is, a linear feedback exists resulting in a stable ﬁxed point to the left and right of the border (i.e., achieving Scenario

58

M.A. Hassouneh and E.H. Abed

A1 of Fig. 1). Indeed, precisely those linear feedbacks uk = γxk with gain γ satisfying −1 − b < γ < 1 − b

(12)

are stabilizing. Method (1b): Linear Feedback Applied in Stable Side of the Border. For a linear feedback applied on the stable side of the border to be eﬀective in ensuring an acceptable bifurcation, it turns out that one must assume that the open-loop system supports an unstable ﬁxed point on the right side of the border. This is tantamount to assuming b < −1. Of course, the assumption −1 < a < 1 is still in force. Now, applying additive linear feedback in the x < 0 region yields the closed-loop system axk + µ + uk , xk ≤ 0 xk+1 = (13) bxk + µ, xk ≥ 0 uk = γxk

(14)

Note that such a control scheme does not stabilize the unstable ﬁxed point on the right side of the border for µ > 0. This is because the control has no direct eﬀect on the system for x > 0. All is not lost, however. The next proposition asserts that such a control scheme may be used to stabilize the system to a period-2 solution for µ > 0. Proposition 5 Suppose that the ﬁxed point of (9) is stable in − and is unstable in + (i.e., |a| < 1 and b < −1). Then there is a linear feedback that when applied to the left of the border (i) maintains a stable ﬁxed point to the left of the border for µ < 0, and (ii) produces a stable period-2 orbit to the right of the border for µ > 0 (i.e., the feedback achieves Scenario C1 of Fig. 1). Indeed, precisely those linear feedbacks uk = γxk with gain γ satisfying 1 1 −a<γ <− −a b b

(15)

are stabilizing. Proof: The closed-loop system is given by (a + γ)xk + µ, xk ≤ 0 xk+1 = bxk + µ, xk ≥ 0 The ﬁxed point on the left of the border for µ < 0 remains stable if and only if |a + γ| < 1 ⇐⇒ −1−a<γ <1−a (16) The ﬁxed point on the right of the border for µ > 0 remains unstable since the control is applied only in the x < 0 region. The closed-loop system bifurcates to a period-2 orbit as µ is increased through zero if the ﬁxed point of the

Feedback Control of Border Collision Bifurcations

59

second return map xk+2 for µ > 0, which form a period-2 orbit for the ﬁrst return map, is stable. That is, if 1 1 −a<γ <− −a (17) |(a + γ)b| < 1 ⇐⇒ b b Combining conditions (16) and (17) yields 1 1 max − a, −1 − a < γ < min − − a, 1 − a b b Since b < −1 < a, condition (18) is equivalent to completes the proof.

1 b

(18)

− a < γ < − 1b − a which

Method 2: Simultaneous Stabilization. In this method, the same linear feedback control is applied additively in both the x < 0 and x > 0 regions. This leads to the closed-loop system axk + µ + uk , xk ≤ 0 xk+1 = (19) bxk + µ + uk , xk ≥ 0 uk = γxk

(20)

The result is given in the following proposition the proof of which is straightforward. Proposition 6 The ﬁxed points of the closed-loop system (19)-(20) on both sides of the border can be simultaneously stabilized using linear feedback control uk = γxk if and only if |a − b| < 2

(21)

Indeed, precisely those linear feedbacks uk = γxk with gain γ satisfying −1 − b < γ < 1 − a

(22)

are stabilizing. Next, the case in which |a − b| ≥ 2 is considered. Recall that, because of symmetry, a − b ≥ 2 can be assumed to hold. The next proposition asserts that in this case a simultaneous linear feedback control exists that ensures the border collision bifurcation is from a stable ﬁxed point to a stable period-2 solution (i.e., the feedback achieves Scenario C1, supercritical border collision period doubling). Proposition 7 Suppose a − b ≥ 2. Then, there is a simultaneous control law that renders the BCB in the system (19)-(20) a supercritical border collision period doubling (Scenario C1 of Fig. 1). To achieve this, the control gain must be chosen to satisfy −1 < γ + a < 1

and

− 1 < (γ + a)(γ + b) < 1

(23)

A speciﬁc class of such control gains is γ = −a + , with suﬃciently small.

60

M.A. Hassouneh and E.H. Abed

Note that if the system is known on the stable side but is uncertain on the unstable side (with b < −1) the conclusion of Prop. 7 still applies. This has important implications for robustly stabilizing the system. 3.2 Control of BCB in 2-D Maps Using Static Feedback For the control of BCBs in two dimensional maps, similar ideas are used as in the case of controlling BCBs in 1-D maps. All the control laws are developed based on the map linearizations as the ﬁxed point is approached on both sides of the border. We do not assume the system to be in normal form. This alleviates the need to include state transformations in the design of control laws except for the transformation setting the border to lie on the y-axis. Details on control of BCBs in two-dimensional maps can be found in [12]. The control ideas are illustrated in the next section using a two-dimensional cardiac arrhythmia example.

4 Case Study: Quenching of Alternans in a Cardiac Conduction Model In this section, we consider the cardiac conduction model of [16]. The model incorporates physiological concepts of recovery, facilitation and fatigue. It is formulated as a two-dimensional PWS map. Two factors determine the atrioventricular (AV) nodal conduction time: the time interval from the atrial activation to the activation of the Bundle of His and the history of activation of the node. The model predicts a variety of experimentally observed complex rhythms of nodal conduction. In particular, alternans, in which there is an alternation in conduction time from beat to beat, are associated with period doubling bifurcation in the theoretical model. The authors of [16] ﬁrst deﬁne the atrial His interval, A, to be that between cardiac impulse excitation of the lower interatrial septum to the Bundle of His. (See [16] for deﬁnitions.) The model is An+1 = f (An , Rn , Hn ) (24) Rn+1 where

0 Hn 1  Amin + Rn+1 + (201 − 0.7An )e− τrec   , for An ≤ 130 (A +Hn )  − n − τHn  τf at f at R e + γe n f (An , Rn , Hn ) = 0 Hn 1  Amin + Rn+1 + (500 − 3.0An )e− τrec    , for An > 130 (A +Hn )  − n − Hn τf at Rn e + γe τf at

with R0 = γexp(−H0 /τf at ). Here, H0 is the initial H interval and the parameters Amin , τf at , γ and τrec are positive constants. The variable Hn represents

Feedback Control of Border Collision Bifurcations

61

the interval between bundle of His activation and the subsequent activation (the AV nodal recovery time) and is usually taken as the bifurcation parameter. The quantity Rn represents a drift in the nodal conduction time, and is sometimes taken to be constant. In this paper, we take Rn to be a variable as in [16]. Note that the map f is piecewise smooth and is continuous at the border Ab := 130ms. Next, the following parameter values are assumed (borrowed from [16]): τrec = 70ms, τf at = 30000ms, Amin = 33ms, γ = 0.3ms. 4.1 Analysis of the Border Collision Bifurcation Numerical simulations show that the map (24) undergoes a supercritical period doubling bifurcation as the bifurcation parameter S := Hn is decreased through a critical value (see Fig. 4). We show that this bifurcation is a supercritical period doubling BCB which occurs as the ﬁxed point of the map hits the border Ab = 130. ∗ Let the ﬁxed points of the map (24) be given by (A∗− (S), R− (S)) for An < ∗ ∗ Ab and (A+ (S), R+ (S)) for An > Ab . Under normal conditions, the ﬁxed ∗ point (A∗− (S), R− (S)) is stable and it loses stability as S is decreased through ∗ a critical value S = Sb where A∗− = Ab . Denote by Rb the value of R− at criticality (S = Sb ). Denote by JL the Jacobian of f for An < Ab , JR the Jacobian of f for An > Ab and (α1 , α2 )T the derivative of f with respect to the bifurcation parameter. For the assumed parameter values, Sb = 56.9078ms, Rb = 48.2108ms and −0.31208 0.99379 −1.33223 0.99379 JL = , JR = −0.001597 0.99379 −0.001597 0.99379 α1 −0.69861 and = −0.001607 α2 The eigenvalues of JL are λL1 = −0.3109, λL2 = 0.9926 (τL = 0.6817, δL = −0.3086) and those of JR are λR1 = −1.3315, λR2 = 0.9931 (τR = −0.3384 and δR = −1.3224). Note that there is a discontinuous jump in the eigenvalues at the border collision bifurcation. The stability of the period-2 orbit with one point in RL := {An : An ≤ Ab } and the other in RR := {An : An > Ab } is determined, by the eigenvalues of JLR := JL JR . These eigenvalues are λLR1 = 0.4135 and λLR2 = 0.9867. This implies that a stable period-2 orbit is born after the border collision. The occurrence of a supercritical period doubling BCB can be also seen from the bifurcation diagram depicted in Fig. 4 (a). 4.2 Feedback Control of the Border Collision Period Doubling Bifurcation For the cardiac conduction model, the control is usually applied as a perturbation to the bifurcation parameter S [8, 7]. The state An has been used in the

62

M.A. Hassouneh and E.H. Abed 180

160

160

140

An

130

An

140 130

120 100

120 80

100 60 40

Rn

Rn

80

60

40 40

20

45

50

55

60

65

0 0

500

S

(a)

1000

1500

2000

2500

3000

Beat number n

(b)

Fig. 4. (a) Joint bifurcation diagram for An and for Rn for (24) with S as bifurcation parameter and τrec = 70ms, τf at = 30000ms, Amin = 33ms and γ = 0.3ms (b) Iterations of map showing the alternation in An as a result of a supercritical period doubling BCB (the parameter values are the same as in (a) with S = 45ms.)

feedback loop by other researchers who developed control laws for this model (e.g., [6, 7]). We use the same measured signal in our feedback design. Below, two control methods are used to quench the period doubling bifurcation, replacing the period doubled orbit by a stable ﬁxed point. Feedback applied on the unstable side is considered ﬁrst, followed by simultaneous control. Static Feedback Applied on Unstable Side. It is straightforward to calculate the Jacobians of the closed loop system with linear state feedback un = γ1 (An − Ab ) + γ2 (Rn − Rb ) applied on the unstable side only (An > Ab ) as a perturbation to the bifurcation parameter. The calculated Jacobian for An > Ab involves the control gains (γ1 , γ2 ). By Prop. 3, choosing the control gains such that the eigenvalues are stable and real guarantees that the unstable ﬁxed point is stabilized (i.e., alternan quenching is achieved). The details are omitted, but can be found in our recent work [10]. Figure 5 (a) shows the bifurcation diagram of the controlled system with (γ1 , γ2 ) = (−1, 0). Note that by setting γ2 = 0, only An is used in the feedback. In practice, the conduction time of the nth beat An , can be measured. Simultaneous Feedback Control. Next, a feedback control un = γ1 (An − Ab ) + γ2 (Rn − Rb ) is applied on both sides of the border. The gains are designed to satisfy the assumptions of Prop. 3 for the closed-loop system. Figure 5 (b) shows the bifurcation diagram of the controlled system with (γ1 , γ2 ) = (−1, 0). Figure 6 (a) shows the eﬀectiveness of the control in quenching the period-2 orbit and simultaneously stabilizing the unstable ﬁxed point. The robustness of the control law to noise is demonstrated in Fig. 6 (b).

Feedback Control of Border Collision Bifurcations 140

63

140

138

135

136 134

130

An

An

132

125

130 128

120 126 124

115

122 120 45

50

55

60

110 45

65

50

55

60

65

S

S

(a)

(b)

Fig. 5. (a) Bifurcation diagram of the controlled system using linear state feedback applied in the unstable region with control gains (γ1 , γ2 ) = (−1, 0). (b) Bifurcation diagram of the controlled system using simultaneous linear state feedback with control gains (γ1 , γ2 ) = (−1, 0). 145

145

140

140

135

135

130

130

125

An

125

A

n

120 115

120 115

off

on

off

off

on

110

105

105

100

100

95 0

500

1000

1500

2000

2500

Beat number n

(a)

on

off

on

110

3000

95 0

500

on

off

1000

1500

on

off

2000

2500

3000

Beat number n

(b)

Fig. 6. Iterations of map. Simultaneous linear state feedback control applied at beat number n = 500. The control is switched oﬀ and on every 500 beats to show the eﬀectiveness of the controller ( S = 48ms and (γ1 , γ2 ) = (−1, 0)). (a) without noise (b) with zero mean, σ = 0.5ms white Gaussian noise added to S.

5 Concluding Remarks Feedback control of border collision bifurcations has been studied and applied to a model of cardiac arrhythmia. It was pointed out that the basic theory of BCBs is incomplete and needs further development in order for control problems for higher dimensional systems to be adequately addressed. Among the many open problems of interest are the following: a detailed classiﬁcation of BCBs in nonscalar maps; bifurcation formulas for BCBs; order reduction principles for BCBs; exchange of stability / stability of critical systems; relation of critical system dynamics to multiple bifurcating attractor phenomenon; Lyapunov function analysis; and Lyapunov-based control design as a means of circumventing the need for detailed BCB classiﬁcation.

64

M.A. Hassouneh and E.H. Abed

Acknowledgments. The authors are grateful to Soumitro Banerjee, Helena Nusse and Priya Ranjan for helpful discussions. This work was supported in part by the National Science Foundation under Grants ECS-01-15160 and ANI-02-19162.

References 1. Banerjee S and Grebogi C (1999), “Border collision bifurcations in twodimensional piecewise smooth maps,” Physical Review E 59 (4): 4052–4061. 2. Banerjee S, Karthik MS, Yuan GH and Yorke JA (2000), “Bifurcations in onedimensional piecewise smooth maps- theory and applications in switching systems,” IEEE Transactions on Circuits and Systems-I 47 (3): 389–394. 3. Di Bernardo M, Feigin MI, Hogan SJ and Homer ME (1999), “Local analysis of C-bifurcations in n-dimensional piecewise smooth dynamical systems,” Chaos, Solitons and Fractals 10 (11): 1881–1908. 4. Di Bernardo M (2000), “Controlling switching systems: a bifurcation approach,” IEEE International Symposium on Circuits and Systems 2: 377–380. 5. Di Bernardo M and Chen G (2000), “Controlling bifurcations in nonsmooth dynamical systems,” In: Chen G and Dong X (eds), Controlling Chaos and Bifurcations in Engineering Systems, ch. 18, pp. 391–412, Boca Raton, FL: CRC Press. 6. Brandt ME, Shih HT and Chen G (1997), “Linear time-dely feedback control of a pathological rhythm in a cardiac conduction model,” Physical Review E 56: R1334–R1337. 7. Chen D, Wang HO and Chin W (1998), “Suppressing cardiac alternans: Analysis and control of a border-collision bifurcation in a cardiac conduction model,” IEEE International Symposium on Circuits and Systems 3: 635–638. 8. Christini DJ and Collins JJ (1996), “Using chaos control and tracking to suppress a pathological nonchaotic rhythms in cardiac model,” Physical Review E 53: R49–R51. 9. Feigin MI (1970), “Doubling of the oscillation period with C-bifurcations in piecewise continuous systems,” Prikladnaya Matematika Mechanika 34: 861– 869. 10. Hassouneh MA and Abed EH, “Border collision bifurcation control of cardiac alternans,” International Journal of Bifurcation and Chaos, to appear. 11. Hassouneh MA and Abed EH (2002), “Feedback control of border collision bifurcations in one dimensional piecewise smooth maps,” Tech. Report 200226, Inst. Syst. Res., University of Maryland, College Park. 12. Hassouneh MA, Abed EH and Banerjee S (2002), “Feedback control of border collision bifurcations in two dimensional discrete time systems,” Tech. Report 2002-36, Inst. Syst. Res., University of Maryland, College Park. 13. Hassouneh MA, Nusse HE and Abed EH, in preparation. 14. Nusse HE and Yorke JA (1992), “Border-collision bifurcations including ‘period two to period three’ for piecewise smooth maps,” Physica D 57: 39–57. 15. Nusse HE and Yorke JA (1995), “Border-collision bifurcations for piecewise smooth one-dimensional maps,” Int. J. Bifurcation and Chaos 5: 189–207. 16. Sun J, Amellal F, Glass L and Billette J (1995), “Alternans and period-doubling bifurcations in atrioventricular nodal conduction,” Journal Theoretical Biology 173: 79–91.

Symmetries and Minimal Flat Outputs of Nonlinear Control Systems W. Respondek Laboratoire de Math´ematiques, INSA de Rouen, Pl. Emile Blondel, 76131 Mont Saint Aignan, France, [email protected]

1 Introduction Consider a nonlinear control system of the form Π : x˙ = F (x, u), where x ∈ X, a smooth n-dimensional manifold and u ∈ U , a smooth mdimensional manifold. To the system Π we associate its ﬁeld of admissible velocities F(x) = {F (x, u) : u ∈ U } ⊂ Tx X. We will say that a diﬀeomorphism of the state space X is a symmetry of Π if it preserves the ﬁeld of admissible velocities. The notion of ﬂatness for the system Π, introduced by Fliess, L´evine, Rouchon and Martin [6] formalizes dynamic feedback linearizability of Π by (dynamically) invertible and endogenous feedback. Roughly speaking, Π is ﬂat if we can ﬁnd m (which is the number of controls) functions, depending on the state x, the control u and its time-derivatives, whose successive derivatives, with respect to the dynamics of Π, determine all components of the state x and of the control u. The aim of this paper is to discuss the following general observation relating the notion of symmetries with that of ﬂatness: A system is ﬂat if and only if it admits a suﬃciently large inﬁnite dimensional group of symmetries. We will make this statement rigorous in the three following cases: - single-input systems; - feedback linearizable systems; - contact systems.

Dedicated to Professor Arthur J. Krener on the occasion of his 60th birthday.

W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 65–86, 2003.

c Springer-Verlag Berlin Heidelberg 2003

66

W. Respondek

By suﬃciently large, we will mean an inﬁnite dimensional group parameterized by m arbitrary functions of m variables, where m is the number of controls. In the ﬁrst part of the paper we will deal with single-input systems that are not ﬂat, which, as is well known [4], [30], are just systems that are not linearizable via a static feedback. For such systems, the group of symmetries is very small. Indeed, as proved by Tall and the author, for an analytic system, which is not feedback linearizable and whose ﬁrst order approximation around an equilibrium is controllable, the group of stationary symmetries (that is, preserving the given equilibrium) contains at most two elements (see [34]) and the group of non stationary symmetries consists of at most two 1-parameter families (see [33]). This surprising result follows from the canonical form obtained for singleinput systems by Tall and the author [40]. This form completes a normal form obtained by Kang and Krener [19], [18] who proposed to apply to control systems a method developed by Poincar´e for dynamical systems, see e.g., [1]. In the second part of the paper we will deal with feedback linearizable systems. We will show that in this case the group of local symmetries is parameterized by m arbitrary functions of m variables, where m is the number of controls. Moreover, we will prove that any local symmetry is a composition of one linearizing diﬀeomorphism with the inverse of another linearizing diﬀeomorphism. In the third part of the paper, we deal with control systems subject to nonholonomic kinematic constraints. We study the class of systems that are feedback equivalent to the canonical contact system on J n (R, Rm ), that is, to the canonical contact system for curves. We will describe their symmetries; it turns out that the geometry of such systems, as shown by Pasillas-L´epine and the author [27], [28], is given by a ﬂag of distributions, whose each member contains an involutive subdistribution of corank one. This implies that the geometry of contact systems for curves resembles that of feedback linearizable systems and, as a consequence, the picture of symmetries is analogous. Finally, in the fourth part of the paper we will establish relations between ﬂatness and symmetries for two classes of systems: feedback linearizable systems and for systems equivalent to the canonical contact system for curves. We introduce the notion of minimal ﬂat outputs and give two main results of the paper, Theorems 6 and 7, which say that for those two classes of systems the minimal ﬂat outputs determine local symmetries and vice versa. Moreover, for each of those classes of systems, minimal ﬂat outputs have a clear geometric interpretation: they are functions whose diﬀerentials annihilate a certain involutive distribution which describes the geometry of the considered class of systems. The paper is organized as follows. We introduce the notion of a symmetry of a control system in Section 2 and in Section 3 we discuss our results with Tall on symmetries of single-input systems. In particular, we recall our canonical form in Subsection 3.1 and describe symmetries of the canonical form in

Symmetries and Minimal Flat Outputs of Nonlinear Control Systems

67

Subsection 3.2. We discuss symmetries of feedback linearizable systems in Section 4 starting by symmetries of linear systems. Section 5 is devoted to contact systems for curves and their symmetries. We deﬁne this class of systems, give a geometric characterization of systems equivalent to the canonical contact system for curves, deﬁne transformations bringing a given system to that form, and, ﬁnally, we describe symmetries of the canonical contact system for curves and symmetries of systems equivalent to that form. Section 6 contains main results of the paper, namely two theorems relating symmetries and minimal ﬂat outputs for feedback linearizable systems and systems equivalent to the canonical contact system for curves.

2 Symmetries In this section we will introduce the notion of symmetries of nonlinear control systems (see also [11], [15], [34], [37]). Let us consider the system Π : x˙ = F (x, u), where x ∈ X, a smooth n-dimensional manifold and u ∈ U , a smooth mdimensional manifold. The map F : X × U −→ T X is assumed to be smooth with respect to (x, u) and for any value u ∈ U of the control parameter, F deﬁnes a smooth vector ﬁeld Fu on X, where Fu (·) = F (·, u). Consider the ﬁeld of admissible velocities F associated to the system Π and deﬁned as F(x) = {Fu (x) : u ∈ U } ⊂ Tx X. We say that a diﬀeomorphism σ : X −→ X is a symmetry of Π if it preserves the ﬁeld of admissible velocities F, that is, σ∗ F = F. Recall that for any vector ﬁeld f on X and any diﬀeomorphism y = ψ(x) of X, we put (ψ∗ f )(y) = Dψ(ψ −1 (y)) · f (ψ −1 (y)). ˜0, A local symmetry at p ∈ X is a local diﬀeomorphism σ of X0 onto X ˜ where X0 and X0 are, respectively, neighborhoods of p and σ(p), such that (σ∗ F)(q) = F(q) ˜0. for any q ∈ X A local symmetry σ at p is called a stationary symmetry if σ(p) = p and a nonstationary symmetry if σ(p) = p. Let us consider a single-input control aﬃne system Σ : x˙ = f (x) + g(x)u,

68

W. Respondek

where x ∈ X, u ∈ U = R and f and g are smooth vector ﬁelds on X. The ﬁeld of admissible velocities for the system Σ is the following ﬁeld of aﬃne lines: A(x) = {f (x) + ug(x) : u ∈ R} ⊂ Tx X. A speciﬁcation of the above deﬁnition says that a diﬀeomorphism σ : X −→ X is a symmetry of Σ if it preserves the aﬃne line ﬁeld A (in other words, the aﬃne distribution A of rank 1), that is, if σ∗ A = A. We will call p ∈ X to be an equilibrium point of Σ if 0 ∈ A(p). For any equilibrium point p there exists a unique u ˜ ∈ R such that f˜(p) = 0, where ˜ f (p) = f (p) + u ˜g(p). By the linear approximation of Σ at an equilibrium p ˜ we will mean the pair (F, G), where F = ∂∂xf (p) and G = g(p). We will say that Σ is an odd system at p ∈ X if it admits a stationary symmetry at p, denoted by σ − , such that ∂σ − (p) = −Id. ∂x

3 Symmetries of Single-Input Nonlinearizable Systems In this section we deal with single-input control aﬃne systems of the form Σ : x˙ = f (x) + g(x)u, where x ∈ X, u ∈ R. Our analysis will be local so we can assume that X = Rn . Throughout this section we will assume that the point p around which we work is an equilibrium, that is f (p) = 0 and, moreover, that g(p) = 0. It is known (see [4], [30]) that for single input systems the notions of ﬂatness (see Section 6 for a precise deﬁnition) and feedback linearizability (see Section 4.2) coincide. We will prove that if Σ is not feedback linearizable, i.e., not ﬂat, then the group of local symmetries of Σ around an equilibrium p ∈ Rn is very small. More precisely, the following result of Tall and the author [33], [34] says that if Σ is analytic, then it admits at most two 1-parameter families of local symmetries. We will say that σc , where c ∈ (−, ) ⊂ R, is a nontrivial 1-parameter analytic family of local symmetries if each σc is a local analytic symmetry, σc1 = σc2 if c1 = c2 , and σc (x) is jointly analytic with respect to (x, c). Theorem 1. Assume that the system Σ is analytic, the linear approximation (F, G) of Σ at an equilibrium point p is controllable and that Σ is not locally feedback linearizable at p. Assume, moreover, that the local feedback transformation bringing Σ into its canonical form ΣCF , deﬁned in the next section, is analytic at p. Then there exists a local analytic diﬀeomorphism φ : X0 → Rn , where X0 is a neighborhood of p, with the following properties.

Symmetries and Minimal Flat Outputs of Nonlinear Control Systems

69

(i) If σ is a local analytic stationary symmetry of Σ at p, then either σ = Id or φ ◦ σ ◦ φ−1 = −Id. (ii) If σ is a local analytic nonstationary symmetry of Σ at p, then φ ◦ σ ◦ φ−1 = Tc , where c ∈ R and Tc is either the translation Tc = (x1 + c, x2 , . . . , xn ) or Tc is replaced by Tc− = Tc ◦ (−Id) = (−x1 + c, −x2 , . . . , −xn ). (iii) If σc , c ∈ (−, ), is a nontrivial 1-parameter analytic family of local symmetries of Σ at p, then φ ◦ σc ◦ φ−1 = Tc , where Tc is as above, for c ∈ (−, ). If we drop the assumption that Σ is equivalent to its canonical form ΣCF by an analytic feedback transformation, then items (i) and (iii) remain valid with the local analytic diﬀeomorphisms φ being replaced by a formal diﬀeomorphism. 3.1 Canonical Form of Single-Input Systems The nature of (stationary and nonstationary) symmetries of single-input systems is very transparent when we bring the system into its canonical form. Moreover, our proof of the above theorem, see [34], is also based on transforming the formal power series Σ ∞ of the system into its canonical form ∞ ∞ ΣCF . For these two reasons we will recall our canonical form ΣCF , which was ∞ obtained in [40] (see also [39]) and which completes the normal form ΣN F of Kang and Krener [18] and [19]. Assume that we work around p = 0 ∈ Rn . Together with the system Σ, consider its Taylor series expansion at 0 ∈ Rn : Σ ∞ : ξ˙ = F ξ + Gu +

∞

(f [m] (ξ) + g [m−1] (ξ)u),

m=2

where F = ∂f ∂ξ (0) and G = g(0). Consider the Taylor series expansion Γ ∞ ∞ x = m=1 φ[m] (ξ) Γ∞ : ∞ u = m=1 (α[m] (ξ) + β [m−1] (ξ)v), of a feedback transformation Γ Γ :

x = φ(ξ) u = α(ξ) + β(ξ)v.

70

W. Respondek

Following Kang and Krener [18] and [19] (who have adapted for control systems Poincar´e’s method developed for dynamical systems, e.g., [1]), analyze the action of the transformation formal power series Γ ∞ on the system formal power series Σ ∞ step by step. This means to analyze how the homogenous part of degree m of Γ ∞ acts on the homogeneous part of degree m of Σ ∞ . Let the ﬁrst homogeneous term of Σ ∞ , which cannot be annihilated by feedback, be of degree m0 . As proved by Krener [20], the degree m0 is given by the largest integer such that, for 1 ≤ k ≤ n − 1, all distributions Dk = span {g, . . . , adk−1 g} are involutive modulo terms of order m0 − 2. f Denote x ¯i = (x1 , . . . , xi ). The following canonical form was obtained by Tall and the author in [40] (see also [39]): Theorem 2. The system Σ ∞ is equivalent by a formal feedback Γ ∞ to a system of the form ∞ ΣCF : x˙ = Ax + Bv +

where, for any m ≥ m0 ,

[m] f¯j (x) =

∞

f¯[m] (x),

m=m0

 n 2 [m−2]   xi Pj,i (¯ xi ), 1 ≤ j ≤ n − 2    

i=j+2

(1) n − 1 ≤ j ≤ n;

0,

additionally, we have [m ] ∂ m0 f¯j ∗ 0 (x) i

n−s ∂xi11 · · · ∂xn−s

= ±1

and, moreover, for any m ≥ m0 + 1, [m] ∂ m0 f¯j ∗ (x) i

n−s ∂xi11 · · · ∂xn−s

(x1 , 0, . . . , 0) = 0.

The integers j ∗ and (i1 , . . . , in−s ), where i1 + · · · + in−s = m0 , are uniquely determined by f¯[m0 ] and are deﬁned in [40]. Kang [18] proved that by a successive application of homogeneous feedback transformations we can bring all homogeneous terms f [m] for m ≥ m0 to the above ”triangular” form (1) and simultaneously get g [m] = 0 for m ≥ 1. A result of this normalization is the Kang normal form. It turns out that each homogeneous term f [m] of the Kang normal form is unique under the action of homogeneous feedback transformation of degree m but the Kang normal form is not unique under the action of the full feedback group consisting of transformations Γ ∞ , see [18], [40]. In fact, for each degree of homogeneity m there exists a 1-dimensional subgroup of the group of feedback homogeneous

Symmetries and Minimal Flat Outputs of Nonlinear Control Systems

71

transformations of degree m that preserve the ”triangular structure” of the Kang normal form and can be used to normalize higher order terms. The above canonical form is a result of a normalization coming from the action of this small group. It deserves its name: we proved in [40] that two single-input systems are equivalent under a formal feedback if and only if their canonical forms coincide. If the feedback transformation bringing an analytic system Σ into its canonical form is analytic (and not only formal), then we will denote the corresponding analytic canonical form by ΣCF . 3.2 Symmetries of the Canonical Form Symmetries take a very simple form if we bring the system into its canonical form. Indeed, we have the following result obtained by Tall and the author (see [34] and [33] for proofs and details): Proposition 1. Assume that the system Σ is analytic, the linear approximation (F, G) of Σ at an equilibrium point p is controllable and Σ is not locally feedback linearizable at p. Assume, moreover, that the local feedback transformation, bringing Σ into its canonical form ΣCF , is analytic at p. (i) Σ admits a nontrivial local stationary symmetry if and only if the drift ∞ f¯(x) = Ax + f¯[m] (x) of the canonical form Σ ∞ satisﬁes CF

m=m0

f¯(x) = −f¯(−x), that is, the system is odd. (ii) Σ admits a nontrivial local nonstationary symmetry if and only if the ∞ drift f¯(x) of the canonical form ΣCF satisﬁes f¯(x) = f¯(Tc (x)), that is f¯ is periodic with respect to x1 . (iii) Σ admits a nontrivial local 1-parameter family of symmetries if and ∞ only if the drift f¯(x) of the canonical form ΣCF satisﬁes f¯(x) = f¯(x2 , . . . , xn ). The above result describes all symmetries around an equilibrium of any single-input nonlinear system that is not feedback linearizable and whose ﬁrst order approximation at the equilibrium is controllable. If we drop the assumption that Σ is equivalent to its canonical form ΣCF by an analytic feedback transformation, then the ”only if” statements in items (i) and (iii) remain valid while in the ”if” statements we have to replace local symmetries by formal symmetries, that is, by formal diﬀeomorphisms which preserve the ﬁeld of admissible velocities, see [33] and [35].

72

W. Respondek

4 Symmetries of Feedback Linearizable Systems In the previous section we proved that the group of symmetries of feedback nonlinearizable systems around an equilibrium is very small provided that the linear approximation at the equilibrium is controllable. A natural question is thus what are symmetries of feedback linearizable systems? In this section we will show that symmetries of such systems form an inﬁnite dimensional group parameterized by m arbitrary functions of m variables, where m is the number of controls. We will describe symmetries of linear systems in Brunovsk´ y canonical form and then of feedback linearizable systems. For simplicity we will deal with systems with all controllability indices equal, the general case is treated in [32]. Another description of symmetries of linear systems in Brunovsk´ y canonical form was given by Gardner et al in [8] and [9]. 4.1 Symmetries of Linear Systems Consider a linear control system in the Brunovsk´ y canonical form with all controllability indices equal, say to n + 1,

Λ:

x˙ 0 = x1 .. .

x˙ n−1 = xn x˙ n = v,

on RN , where dim v = m, dim xj = m, N = (n + 1)m. Put π 0 (x) = x0 . For any diﬀeomorphism µ of Rm we deﬁne µ0 : RN → Rm by µ0 = µ ◦ π 0 . Proposition 2. Consider the linear system Λ in the Brunovsk´y canonical form. (i) For any diﬀeomorphism µ of Rm , the map  0  µ LAx µ0    λµ =  .   ..  LnAx µ0 is a symmetry of Λ. (ii) Conversely, if σ is a symmetry of Λ, then σ = λµ , for some diﬀeomorphism µ of Rm .

Symmetries and Minimal Flat Outputs of Nonlinear Control Systems

73

Notice that µ0 is a map from RN into Rm depending on the variables x0 only. The transformation λµ : RN → RN is deﬁned by successive diﬀerentiating this map with respect to the drift Ax. Item (i) claims that such a transformation is always a symmetry of the linear system Λ (in particular, a diﬀeomorphism) while item (ii) claims that all symmetries of linear systems are always of this form. Remark 1. Clearly, an analogous result holds for local symmetries, that is, if µ is a local diﬀeomorphism of Rm , then the corresponding λµ is a local symmetry of Λ and, conversely, any local symmetry of Λ is of the form λµ for some local diﬀeomorphism µ. This local version of the above result will allow us to describe in the next section all local symmetries of feedback linearizable systems. 4.2 Symmetries of Feedback Linearizable Systems Consider a control-aﬃne system of the form Σ : ξ˙ = f (ξ) +

m

gi (ξ)ui ,

i=1

where ξ ∈ Ξ, an N -dimensional manifold, and f and gi for 1 ≤ i ≤ m are C ∞ -vector ﬁelds on Ξ. We will say that Σ is feedback equivalent (or feedback linearizable) to a linear system of the form Λ : x˙ = Ax + Bv, if there exists a feedback transformation of the form Γ :

x = Φ(ξ) u = α(ξ) + β(ξ)v,

with β(ξ) invertible, transforming Σ into Λ. We say that Σ is locally feedback linearizable at ξ0 if Φ is a local diﬀeomorphism at ξ0 and α and β are deﬁned locally around ξ0 . Deﬁne the following distributions: G0 = span {g1 , . . . , gm } Gj+1 = Gj + [f, Gj ] . It is well known (see, e.g., [13], [16], [25]) that Σ is, locally at ξ0 , feedback equivalent to a linear system Λ, with all controllability indices equal to n + 1, if and only if the distributions Gj are involutive and of constant rank (j + 1)m for 0 ≤ j ≤ n.

74

W. Respondek

For any map ϕ : Ξ0 → Rm , where Ξ0 is a neighborhood of ξ0 , put   ϕ Lf ϕ   Φϕ =  ..  .  .  Lnf ϕ

Note that Φϕ is a map from Ξ0 in RN . If ϕ = (ϕ1 , . . . , ϕm ) is chosen such that ⊥

(Gn−1 ) = span {dϕ} = span {dϕ1 , . . . , dϕm } , then it is well known (see, e.g., [13], [25]) that Φϕ is a local diﬀeomorphism of an open neighborhood Ξϕ of ξ0 onto Xϕ = Φϕ (Ξϕ ), an open neighborhood of x0 = Φϕ (ξ0 ), and gives local linearizing coordinates for Σ in Ξϕ . To keep the notation coherent, we will denote by ξ, with various indices, points of Ξϕ , by x, with various indices, points of Xϕ = Φϕ (Ξϕ ) ⊂ RN , and by y, with various indices, points of π 0 (Xϕ ) ⊂ Rm , where π 0 is the projection π 0 (x) = x0 . Combining this result with Proposition 2, we get the following complete description of local symmetries of feedback linearizable systems with equal controllability indices. The notation Dif f (Rm ; y0 , y˜0 ) will stand for the family of all local diﬀeomorphisms of Rm at y0 transforming y0 into y˜0 (more precisely, all diﬀeomorphisms germs with the base point y0 and its image y˜0 ). Theorem 3. Let the system § be locally feedback linearizable at ξ0 with equal controllability indices. Fix ϕ : Ξ0 → Rm such that (Gn−1 )⊥ = span {dϕ} = span {dϕ1 , . . . , dϕm }. (i) Let µ ∈ Dif f (Rm ; y0 , y˜0 ), where y0 = π 0 (x0 ) and y˜0 = π 0 (λµ (x0 )), such that λµ (x0 ) ∈ Xϕ . Then σµ,ϕ = Φ−1 ϕ ◦ λµ ◦ Φϕ is a local symmetry of Σ at ξ0 . (ii) Conversely, if σ is a local symmetry of Σ at ξ0 , such that σ(ξo ) ∈ Ξϕ , then there exits µ ∈ Dif f (Rm ; y0 , y˜0 ), where y0 = π 0 (x0 ), y˜0 = π 0 (˜ x0 ), x ˜0 = Φϕ (σ(ξ0 )) such that σ = σµ,ϕ . −1 Moreover, σµ,ϕ = Φ−1 ϕ ◦ λµ ◦ Φϕ = Φϕ ◦ Φµ◦ϕ .

The structure of symmetries of feedback linearizable systems is thus summarized by the following diagram. Item (i) states that composing a linearizing transformation Φϕ with a symmetry λµ of the linear equivalent Λ of Σ and with the inverse Φ−1 ϕ we get a symmetry of Σ, provided that the image x ˜0 = λµ (x0 ) belongs to Xϕ (otherwise the composition is not deﬁned). Item (ii) asserts that any local symmetry of a feedback linearizable system is of this form. Moreover, any local symmetry can be expressed as a composition of one linearizing transformation

Symmetries and Minimal Flat Outputs of Nonlinear Control Systems σµ,ϕ

(Σ, ξ0 )

75

- (Σ, ξ˜0 )

@

@

Φϕ

? (Λ, x0 )

@ Φµ◦ϕ @ @ R @ λµ

Φϕ

?

- (Λ, x˜0 )

with the inverse of another linearizing transformation. Indeed, observe that for any ﬁxed ϕ, the map Φµ◦ϕ , for µ ∈ Dif f (Rm ; y0 , y˜0 ), gives a linearizing diﬀeomorphism and taking all µ ∈ Dif f (Rm ; y0 , y˜0 ) for all y˜0 ∈ π 0 (Xϕ ), the corresponding maps Φµ◦ϕ provide all linearizing transformations around ξ0 .

5 Symmetries of Contact Systems for Curves In this section we will show that for contact systems for curves, which form a class of systems with nonholonomic kinematic constraints, the group of local symmetries exhibits a structure very similar to that of symmetries of feedback linearizable systems. In Section 5.1 we discuss very brieﬂy systems with nonholonomic kinematic constraints, in Section 5.2 we will deﬁne the canonical contact system for curves and in Section 5.3 we will give a geometric characterization of those contact systems. Then we will show in Section 5.4 how to transform a system to the canonical contact system for curves. Finally, we will describe in Section 5.5 symmetries of the canonical contact system for curves and in Section 5.6 symmetries of their equivalents. 5.1 Systems with Nonholonomic Constraints Consider a mechanical system with nonholonomic kinematic constraints on a smooth N -dimensional manifold Ξ, called the conﬁguration space, and subject to a set of constraints of the form J(ξ)ξ˙ = 0, where ξ(t) ∈ Ξ and J is an s × N smooth matrix of full rank s representing s constraints put on the velocities of the system. The rows of J deﬁne, in the ξ-coordinate system, s everywhere independent diﬀerential 1-forms. Denote by I the codistribution spanned by those 1-forms and by D the distribution annihilated by them, that is D⊥ = I. It is clear that all trajectories of the system, subject to the constraints imposed by J, are just all trajectories of the control system

76

W. Respondek

ξ˙ =

k

gi (ξ)ui ,

i=1

where k = N − s, the controls ui ∈ R, and g1 , . . . , gk are smooth vector ﬁelds on Ξ that locally span D, which will be denoted by D = span {g1 , . . . , gk }. ˜ deﬁned on two manifolds Ξ and Ξ, ˜ respectively, Two distributions D and D are equivalent if there exists a smooth diﬀeomorphism ϕ between Ξ and Ξ˜ such that ˜ p), (φ∗ D)(˜ p) = D(˜ ˜ It is easy to see that two distributions D and D, ˜ for each point p˜ in Ξ. associated to two control systems k gi (ξ) ui ∆ : ξ˙ =

and

i=1

k ˙ ˜ u ∆˜ : ξ˜ = g˜i (ξ) ˜i , i=1

are locally equivalent if and only if the corresponding control systems ∆ and ∆˜ are locally feedback equivalent. 5.2 Canonical Contact System for Curves A nonholonomic control system, with m + 1 controls, x˙ =

m

gi (x) ui ,

i=0

where x ∈ RN , the controls ui for 0 ≤ i ≤ m take values in R, and g0 , . . . , gm are smooth vector ﬁelds on RN , is called the canonical contact system for curves (or the canonical contact system on J n (R, Rm ), or the Cartan distribution on J n (R, Rm )) if g1 =

∂ ∂xn 1

g0 =

∂ ∂x00

, . . . , gm = ∂x∂nm m n−1 j+1 ∂ + xi ∂xj , i=1 j=0

i

where N = (n + 1)m + 1 and (x00 , x01 , . . . , x0m , x11 , . . . , x1m , . . . , xn1 , . . . , xnm ) are coordinates on J n (R, Rm ) ∼ = RN . Equivalently, the canonical contact system for curves is a control system with k = m + 1 controls, whose state space is RN , where N = (n + 1)m + 1, given by x˙ 00 = u0 x˙ 01 = x11 u0 · · · x˙ 0m = x1m u0 .. .. . . x˙ n−1 = xn1 u0 x˙ n−1 = xnm u0 m 1 x˙ n1 = u1 x˙ nm = um ,

Symmetries and Minimal Flat Outputs of Nonlinear Control Systems

77

whose trajectories are thus subject to s = nm nonholonomic constraints − xn1 dx00 = 0 dx01 − x11 dx00 = 0 , · · · , dxn−1 1 .. .

− xnm dx00 = 0. dx0m − x1m dx00 = 0 , · · · , dxn−1 m When m = 1, that is in the case of two controls, the canonical contact system for curves gives the celebrated Goursat normal form (characterized originally by von Weber in 1898 and studied extensively since then), known in the control theory as chained form: x˙ 00 = u0 x˙ 0 = x1 u0 .. . x˙ n−1 = xn u0 x˙ n = u1 . Goursat normal form is the canonical contact system on J n (R, R), that is the canonical contact system for curves with values in R; there is only one chain of integrators. Many control systems with nonholonomic mechanical constraints are equivalent to the canonical contact system. Concrete examples of such systems are: n-trailer, see e.g., [23], [14], [17], [26], [36], [41], the nonholonomic manipulator [38], and multi-steered trailer systems [28], [31], [42], [43] for which m ≥ 2. The interest of control systems that are equivalent to the canonical contact system for curves follows from the fact that for a system that has been transformed to that form, important control theory problems like stabilization (using time-varying and discontinuous feedback) and motion planning can be solved. Canonical contact systems for curves are ﬂat (we will discuss this issue in Section 6). Moreover, they exhibit, as we show in the next section, a beautiful geometry. 5.3 Characterization of the Canonical Contact System for Curves The equivalence problem of characterizing distributions (or control-linear systems in other language) that are (locally) transformable to the canonical contact form goes back to Pfaﬀ [29] who stated and solved it for contact systems on J 1 (R, R) in 1814. For contact systems on J 1 (Rk , R), that is, 1jets of R-valued maps, the ﬁnal answer was obtained by Darboux in 1882 in his famous theorem [5]. The equivalence to the canonical contact system on J n (R, R), that is, arbitrary jets of R-valued curves, was solved by von Weber [44] in 1898, with important contributions made then by Cartan [3], Goursat [10], Kumpera and Ruiz [22], and Murray [24] (we do not study in this paper the issue of singularities, which has recently attracted a lot of attention). Finally, the equivalence problem for the canonical contact systems on

78

W. Respondek

J 1 (Rk , Rm ), that is, for 1-jets of arbitrary maps, called also Bryant normal form, was solved by Bryant [2]. In this paper we will be interested in systems that are equivalent to the canonical contact system on J n (R, Rm ) for m ≥ 2, that is, n-jets of curves in Rm (which are studied in [27] and [28]; for related work, see also [21]). For the case m = 1, that is, the Goursat normal form, see, e.g., [26]. In order to give our characterization, we need the two following notions. The derived ﬂag of a distribution D is deﬁned by the relations D(0) = D

and

D(i+1) = D(i) + [D(i) , D(i) ],

for i ≥ 1.

The Lie ﬂag of a distribution D is deﬁned by the relations D0 = D

and

Di+1 = Di + [D0 , Di ],

for i ≥ 1.

The distribution D is said to be regular at a point p if all the elements Di , for i ≥ 0, of its Lie ﬂag are of constant rank in a small enough neighborhood of p. The following result characterizing the canonical contact system for curves was obtained by Pasillas-L´epine and the author in [27] and [28]. Theorem 4. Let m ≥ 2. The control system m ∆ : ξ˙ = gi (ξ) ui ,

(2)

i=0

deﬁned on an open subset Ξ of Rm(n+1)+1 , is feedback equivalent, in a small enough neighborhood of any point p in X, to the canonical contact system on J n (R, Rm ) if and only if the distribution D = span {g0 , . . . , gm } satisﬁes the three following conditions for 0 ≤ i ≤ n: (i) Each element D(i) of the derived ﬂag has constant rank (i + 1)m + 1; (ii) Each D(i) contains an involutive subdistribution Li ⊂ D(i) that has constant corank one in D(i) ; (iii) The point p is a regular point for the distribution D; or, equivalently, the three following conditions: (i)’ D(n) = T Ξ; (ii)’ D(n−1) is of constant rank nm + 1 and contains an involutive subdistribution Ln−1 that has constant corank one in D(n−1) ; (iii)’ D(0) (p) is not contained in Ln−1 (p). Recall that a characteristic vector ﬁeld of a distribution D is a vector ﬁeld g that belongs to D and satisﬁes [g, D] ⊂ D. The characteristic distribution of D is the distribution spanned by all its characteristic vector ﬁelds. We denote by Ci the characteristic distribution of D(i) . The geometry of a distribution D equivalent to the canonical contact system for curves can thus be summarized by the following incidence relations.

Symmetries and Minimal Flat Outputs of Nonlinear Control Systems

D(0) ∪ L0 C1 ∩ D(1)

⊂ D(1) ∪ ⊂ L1 ⊂ C2 ∩ ⊂ D(2)

79

⊂ · · · ⊂ D(n−2) ⊂ D(n−1) ⊂ D(n) = T Ξ ∪ ∪ ⊂ · · · ⊂ Ln−2 ⊂ Ln−1 ⊂ · · · ⊂ Cn−1 ∩ ⊂ · · · ⊂ D(n−1)

The whole geometry of the problem is encoded in condition (ii). It is checkable via calculating the Engel rank of each distribution D(i) , see [27] for details. A comparison of the above the conditions (i)-(iii) and (i)’-(iii)’ implies that a lot of geometry of the problem is encoded just in the existence of one involutive subdistribution Ln−1 , which is unique if conditions (i)’-(iii)’ of Theorem 4 are satisﬁed and m ≥ 2. Firstly, Ln−1 “knows” about the existence of all involutive subdistributions Li = Ci+1 whose existence is claimed by (ii). Secondly, Ln−1 “knows” about the regularity of all intersections of D(i) with Ci+2 implying that p is regular and that there are no singularities in the problem (for the issue of singularities see [27] and [28]). 5.4 Transforming into the Canonical Contact System for Curves In this Section we recall a construction of diﬀeomorphisms, given in [28] and [31], that bring a system satisfying Theorem 4 to the canonical contact system for curves. Since the distribution Ln−1 is involutive and of corank m+1 (it is of corank one in D(n−1) so of corank m + 1 in T Ξ), we can choose smooth functions ψ = (ψ0 , ψ1 , . . . , ψm ), deﬁned in a neighborhood Ξ0 of ξ0 , such that (Ln−1 )⊥ = span {dψ} = span {dψ0 , dψ1 , . . . , dψm }. By condition (iii)’ there exists a vector ﬁeld g0 ∈ D(0) such that g0 (ξ0 ) ∈ / Ln−1 (ξ0 ). By a suitable rearranging the functions ψi , we can suppose that Lg0 ψ0 (ξ0 ) = 0. Put x00 = ψ00 = ψ0 and for 1 ≤ i ≤ m deﬁne x0i = ψi0 = ψi and xji = ψij =

Lg0 ψij−1 , Lg0 ψ00

j T for 1 ≤ j ≤ n. Deﬁne ψ j = (ψ1j , . . . , ψm ) . For any map ψ : Ξ0 → Rm+1 put  0 ψ0  ψ0   1   Ψψ =  ψ  .  ..   . 

ψn

80

W. Respondek

Proposition 3. Consider the control system ∆ with m ≥ 2 given by (2). The map Ψψ is a local diﬀeomorphism of a neighborhood Ξψ of ξ0 and brings ∆ into the canonical contact system for curves. Notice that the structure of Ψψ resembles that of Φϕ . Indeed, we choose ψi ’s whose diﬀerentials annihilate Ln−1 (like we choose ϕi ’s whose diﬀerentials annihilate Gn−1 ). The diﬀerence is that when constructing φji for j ≥ 1, we keep diﬀerentiating with respect to the drift f while in order to construct ψij we have to normalize the Lie derivative with respect to g0 by Lg0 ψ00 . The reason being that in the linearization problem we deal with control aﬃne systems and the drift f is given by the system. In the problem of transforming to the canonical contact system for curves, a vector ﬁeld g0 ∈ D(0) , satisfying / Ln−1 , is not unique and it is a part of the problem to ﬁnd the ”right” g0 ∈ vector ﬁeld g0 . Now we will describe all symmetries of systems that can be transformed into the canonical contact system for curves. The structure of the remaining part of the section will be the same as that of Section 4 devoted to symmetries of feedback linearizable systems. We start in Section 5.5 with symmetries of the canonical contact system for curves, which from now on will be denoted by CCS n (1, m), and then in Section 5.6 we will describe symmetries of systems that are equivalent to CCS n (1, m). 5.5 Symmetries of the Canonical Contact System for Curves Consider the canonical contact system CCS n (1, m) : x˙ =

m

gi (x)ui

i=0

on J n (R, Rm ), equipped with coordinates x = (x00 , x01 , . . . , x0m , x11 , . . . , xnm ). Denote the projections π00 (x) = x00 and π 0 (x) = (x01 , . . . , x0m ). Let ν = (ν00 , ν1 , . . . , νm ) : Rm+1 → Rm+1 be a diﬀeomorphism of Rm+1 such that Lg0 ν00 = 0. Denote ν 0 = (ν1 , . . . , νm ). Put λ0ν = ν 0 ◦ (π00 , π 0 ) and λ0ν,0 = ν00 ◦ (π00 , π 0 ). For any 1 ≤ j ≤ n, deﬁne λjν =

Lg0 λj−1 ν . Lg0 λ0ν,0

Proposition 4. Consider CCS n (1, m), that is, the canonical contact system on J n (R, Rm ). (i) For any diﬀeomorphism ν of Rm+1 as above, the map  0  λν,0  λ0ν    λν =  .   ..  λnν is a symmetry of the canonical contact system for curves CCS n (1, m).

Symmetries and Minimal Flat Outputs of Nonlinear Control Systems

81

(ii) Conversely, if σ is a symmetry of CCS n (1, m), then σ = λν for some diﬀeomorphism ν of Rm+1 . This description resembles that of symmetries of linear control systems. The diﬀerence is the presence of the term Lg0 λ0ν,0 in the deﬁnition of λjν . Notice that x ˜00 = λ0ν,0 (x) deﬁnes the new independent variable x ˜00 . It follows that the 0 drift g0 is multiplied by Lg0 λν,0 . Now in order that λν deﬁnes a symmetry, all components of the drift must be multiplied by the same function, which explains the presence of the function Lg0 λ0ν,0 in the denominator. Clearly, taking local diﬀeomorphisms ν of Rm+1 we get local symmetries of the canonical contact system CCS n (1, m). 5.6 Symmetries of Systems Equivalent to the Canonical Contact System Let Ψψ be a local diﬀeomorphism of a neighborhood Ξψ of ξ0 onto the neighborhood Xψ = Ψψ (Ξψ ) of x0 = Ψψ (ξ0 ), described by Proposition 3. We will denote by ξ, with various indices, points of Ξψ , by x, with various indices, points of Xψ = Ψψ (Ξψ ) ⊂ RN , and by y, with various indices, points of (π00 , π 0 )(Xψ ) ⊂ Rm+1 , where π00 and π 0 are the projections deﬁned in the previous section. Combining Proposition 3 with 4 we get the following description of local symmetries of systems locally equivalent to CCS n (1, m): Theorem 5. Let the control system ∆ be locally feedback equivalent at ξ0 to the canonical contact system on J n (R, Rm ) for m ≥ 2. Fix ψ : Ξ0 → Rm+1 such that (Ln−1 )⊥ = span {dψ0 , . . . , dψm } = span {dψ}. (i) Let ν ∈ Dif f (Rm+1 ; y0 , y˜0 ), where y0 = (π00 , π 0 )(x0 ) and y˜0 = (π00 , π 0 )(λν (x0 )), such that λν (x0 ) ∈ Xψ . Then σν,ψ = Ψψ−1 ◦ λν ◦ Ψψ is a local symmetry of ∆ at ξ0 . (ii) Conversely, if σ is a local symmetry of ∆ at ξ0 , such that σ(ξ0 ) ∈ Ξψ , then there exits ν ∈ Dif f (Rm+1 ; y0 , y˜0 ), where y0 = (π00 , π 0 )(x0 ) and y˜0 = (π00 , π 0 )(˜ x0 ), x ˜0 = Ψψ (σ(ξ0 )) such that σ = σν,ψ . Moreover, σν,ψ = Ψψ−1 ◦ λν ◦ Ψψ = Ψψ−1 ◦ Ψν◦ψ . The structure of local symmetries of systems equivalent to the canonical contact system for curves CCS n (1, m), denoted shortly by CS, is thus summarized by the following diagram.

82

W. Respondek σν,ψ

(∆, ξ0 )

- (∆, ξ˜0 )

@

@

Ψψ

? (CS, x0 )

@ Ψν◦ψ @ @ R @ λν

Ψψ

?

- (CS, x˜0 )

Item (i) states that composing a normalizing transformation Ψψ with a symmetry λν of CCS n (1, m) and with the inverse Ψψ−1 we get a symmetry of ∆. Moreover, any local symmetry can be expressed as a composition of one normalizing transformation with the inverse of another normalizing transformation. Indeed, observe that for any ﬁxed ψ, the map Ψν◦ψ , for ν ∈ Dif f (Rm+1 ; y0 , y˜0 ), gives a normalizing diﬀeomorphism and taking all ν ∈ Dif f (Rm+1 ; y0 , y˜0 ) for all y˜0 ∈ (π00 , π 0 )(Xψ ), the corresponding maps Ψµ◦ψ provide all normalizing transformations around ξ0 , that is, transformations bringing locally ∆ into CCS n (1, m). Above, we have assumed that m ≥ 2. Symmetries of systems equivalent to the Goursat normal form, i.e., the case m = 1, are studied in [26].

6 Symmetries and Minimal Flat Outputs The smooth nonlinear control system Π : x˙ = F (x, u), where x ∈ X, an n-dimensional manifold and u ∈ U , an m-dimensional ma(q) nifold, is ﬂat at p = (x0 , u0 , u˙ 0 , . . . , u0 ) if there exist a neighborhood O of p ˙ . . . , u(r) ), called ﬂat and smooth functions φ1 , . . . , φm , where φi = φi (x, u, u, (r) outputs, deﬁned in a neighborhood of (x0 , u0 , u˙ 0 , . . . , u0 ), such that x = γ(φ, . . . , φ(q) ) u = δ(φ, . . . , φ(q) ) for some smooth maps γ and δ, along any (x(t), u(t), u(t), ˙ . . . , u(q) (t)) ∈ O. The concept of ﬂatness was introduced by Fliess, L´evine, Rouchon and Martin [6] (see also [7], [14], [30]), and it formalizes dynamic feedback linearizability of Π by (dynamically) invertible and endogenous feedback z˙ = g(x, z, v) u = ψ(x, z, v).

Symmetries and Minimal Flat Outputs of Nonlinear Control Systems

83

It is well known that feedback linearizable systems are ﬂat. In fact, the notion of ﬂatness is clearly invariant under invertible static feedback and thus it is enough to notice that for the linear system Λ the functions φ1 = x01 , . . . , φm = x0m are ﬂat outputs. Also systems equivalent to the canonical contact system for curves CCS n (1, m) are ﬂat. Because of invariance under static feedback, it is enough to consider CCS n (1, m) and to observe, by choosing as ﬂat outputs φ0 = x00 , φ1 = x01 , . . . , φm = x0m , that it is ﬂat at any (x0 , u0,0 , u0,1 , . . . , u0,m ) such that u0,0 = 0, where u0,i stands for the i-th component of the nominal control u0 . We send the reader to [28] and [31] for invariant description of the control value to be excluded and for relations with dynamic feedback decoupling. Let φ1 . . . , φm be ﬂat outputs. It can be proved (see [32]) that then there exist integers k1 , . . . , km such that (j)

span {dx1 , . . . , dxn , du1 , . . . , dum } ⊂ span {dφi , 1 ≤ i ≤ m, 0 ≤ j ≤ ki } and if also (j)

span {dx1 , . . . , dxn , du1 , . . . , dum } ⊂ span {dφi , 1 ≤ i ≤ m, 0 ≤ j ≤ li }, . . , km ) will be called the then ki ≤ li , for 1 ≤ i ≤ m. The m-tuple (k1 , . m diﬀerential m-weight of φ = (φ1 . . . , φm ) and k = i=1 ki will be called the diﬀerential weight of φ. (q)

Deﬁnition 1. Flat outputs of Π at p = (x0 , u0 , . . . , u0 ) are called minimal if their diﬀerential weight is the lowest among all ﬂat outputs of Π at p. 6.1 Symmetries and Flat Outputs The two following theorems describe relations between symmetries and minimal ﬂat outputs for, respectively, feedback linearizable systems and for those equivalent to the canonical contact system for curves. By Sym(Σ, ξ0 ) (resp. Sym(∆, ξ0 )) we will mean the local group of all local symmetries σ of Σ (resp. ∆) at ξ0 such that σ(ξ0 ) ∈ Ξϕ (resp. at ξ0 such that σ(ξ0 ) ∈ Ξψ ), where Ξϕ (resp. Ξψ ) is the domain of the diﬀeomorphism Φϕ (resp. Ψψ ). In the statement below we use the notation of Theorems 3 and 5, in particular, y0 = π 0 (x0 ) and y˜0 = π 0 (λµ (x0 )) (resp. y0 = (π00 , π 0 )(x0 ) and y˜0 = (π00 , π 0 )(λν (x0 ))). Theorem 6. Let the control-aﬃne system Σ be feedback linearizable at ξ0 ∈ Ξ. The following conditions are equivalent. (i) (Gn−1 )⊥ = span {dϕ1 , . . . , dϕm } around ξ0 . (ii) ϕ1 , . . . , ϕm are minimal ﬂat outputs of Σ at ξ0 . (iii) Sym(Σ, ξ0 ) = {σµ,ϕ : µ ∈ Dif f (Rm ; y0 , y˜0 )}. Theorem 7. Let the control-linear system ∆ be feedback equivalent at ξ0 ∈ Ξ to the canonical contact system for curves CCS n (1, m), with m ≥ 2. The following conditions are equivalent.

84

W. Respondek

(i) (Ln−1 )⊥ = span {dψ0 , . . . , dψm } around ξ0 . (ii) ψ0 , . . . , ψm are minimal ﬂat outputs of ∆ at ξ0 . (iii) Sym(∆, ξ0 ) = {σν,ψ : ν ∈ Dif f (Rm+1 ; y0 , y˜0 )}.

7 Conclusions The aim of this paper was to examine relations between two important notions of control systems, those of ﬂatness and symmetries. We have illustrated our general observation that ﬂatness correspond to a big inﬁnite dimensional group of symmetries for three classes of systems. We have shown that systems of the ﬁrst class, that is single-input systems that are not ﬂat (equivalently, feedback non linearizable) admit very few local symmetries; namely, around equilibria with controllable linearization they admit at most two 1-parameter families of local symmetries. Then we have shown that two classes of ﬂat systems admit large inﬁnite dimensional group of local symmetries. Namely, static feedback linearizable systems and systems that are static feedback equivalent to the canonical contact system for curves admit inﬁnite dimensional groups of local symmetries parameterized by as many arbitrary functions as is the number of controls. Moreover, for the two last classes, minimal ﬂat outputs determine symmetries and vice versa. Acknowledgements. The author would like to thank William PasillasL´epine and Issa A. Tall for helpful discussions.

References 1. Arnold VI (1988) Geometrical Methods in the Theory of Ordinary Diﬀerential Equations, Second Edition, Springer-Verlag. 2. Bryant R (1979) Some aspects of the local and global theory of Pfaﬃan systems. Ph.D. thesis, University of North Carolina, Chapel Hill. 3. Cartan E (1914) Sur l’´equivalence absolue de certains syst`emes d’´equations diﬀ´erentielles et sur certaines familles de courbes. Bulletin de la Soci´et´e Math´ematique de France, 42:12–48. Œuvres compl`etes, Part. II, Vol. 2, Gauthiers-Villars, Paris. 4. Charlet B, L´evine J, Marino R (1989) On dynamic feedback linearization, Systems and Control Letters, 13:143–151. 5. Darboux G (1882) Sur le probl`eme de Pfaﬀ., Bulletin des Sciences math´ematiques, 2(6):14–36,49–68. 6. Fliess M, L´evine J, Martin P, Rouchon P (1995) Flatness and defect of nonlinear systems: Introductory theory and examples, International Journal of Control, 61:1327–1361. 7. Fliess M, L´evine J, Martin P, Rouchon P (1999) A Lie-B¨ acklund approach to equivalence and ﬂatness of nonlinear systems, IEEE Trans. Automat. Control, 44:922–937.

Symmetries and Minimal Flat Outputs of Nonlinear Control Systems

85

8. Gardner RB, Shadwick WF (1990) Symmetry and the implementation of feedback linearization, Syst. Contr. Lett, 15:25–33. 9. Gardner RB, Shadwick WF, Wilkens GR (1989) Feedback equivalence and symmetries of Brunovsk´ y normal forms, Contemporary Mathematics, 97:115– 130. 10. Goursat E (1905) Sur le probl`eme de Monge, Bulletin de la Soci´et´e Math´ematique de France, 33:201–210. 11. Grizzle JW, Marcus SI (1985) The structure of nonlinear systems possessing symmetries, IEEE Trans. Automat. Control, 30:248–258. 12. Hunt LR, Su R (1981) Linear equivalents of nonlinear time varying systems, In: Proc. MTNS, Santa Monica, CA, pp. 119–123. 13. Isidori A (1995) Nonlinear Control Systems, Third edition, Springer-Verlag, London. 14. Jakubczyk B (1993) Invariants of dynamic feedback and free systems. In Proc. of the European Control Conference EEC’93 , Groningen, pp. 1510–1513. 15. Jakubczyk B (1998) Symmetries of nonlinear control systems and their symbols, Canadian Math. Conf. Proceed., 25:183–198. 16. Jakubczyk B, Respondek W (1980) On linearization of control systems, Bull. Acad. Polon. Sci. Ser. Math., 28:517–522. 17. Jean F (1996) The car with n trailers: Characterization of the singular conﬁgurations, ESAIM Control, Optimisation, and Calculus of Variations, 1:241–266. 18. Kang W (1996) Extended controller form and invariants of nonlinear control systems with single input, J. of Mathem. Systems, Estimation and Control, 6:27–51. 19. Kang W, Krener AJ (1992) Extended quadratic controller normal form and dynamic feedback linearization of nonlinear systems, SIAM J. Control and Optim., 30:1319–1337. 20. Krener AJ (1984) Approximate linearization by state feedback and coordinate change, Systems and Control Letters, 5:181–185. 21. Kumpera A, Rubin JL (2002) Multi-ﬂag systems and ordinary diﬀerential equations, Nagoya Math. J., 1666:1–27. 22. Kumpera A, Ruiz C (1982) Sur l’´equivalence locale des syst`emes de Pfaﬀ en drapeau. In F. Gherardelli, editor, Monge-Amp`ere equations and related topics, Instituto Nazionale di Alta Matematica Francesco Severi, Rome, pp. 201–247. 23. Laumond JP (1991) Controllability of a multibody robot, IEEE Trans. Robotics and Automation, 9:755–763. 24. Murray R (1994) Nilpotent bases for a class of nonintegrable distributions with applications to trajectory generation for nonholonomic systems, Mathematics of Control, Signals, and Systems, 7:58–75. 25. Nijmeijer H, van der Schaft AJ (1990) Nonlinear Dynamical Control Systems, Springer-Verlag, New York. 26. Pasillas-L´epine W, Respondek W (2001) On the geometry of Goursat structures, ESAIM Control, Optimisation, and Calculus of Variations, 6:119–181. 27. Pasillas-L´epine W, Respondek W (2001) Contact systems and corank one involutive subdistributions, Acta Applicandae Mathematicae, 69:105–128. 28. Pasillas-L´epine W, Respondek W (2001) Canonical Contact Systems for Curves: A Survey, In : Contemporary Trends in Geometric Control Theory and Applications, A. Anzaldo, B. Bonnard, J.P. Gauthier et F. Monroy (eds), World Scientiﬁc, Singapore, pp. 77–112.

86

W. Respondek

29. J-F. Pfaﬀ JF (1814-1815) Methodus generalis, aequationes diﬀerentiarum partialum, nec non aequationes diﬀerentiales vulgares, utrasque primi ordinis, inter quotcunque variabiles, completi integrandi. Abhandlungen der K¨ oniglichPreußischen Akademie der Wissenschaften zu Berlin, Mathematische Klasse, pp. 76–136. 30. Pomet JB (1995) A diﬀerential geometric setting for dynamic equivalence and dynamic linearization, In Geometry in Nonlinear Control and Diﬀerential Inclusions, B. Jakubczyk, W. Respondek, and T. Rzezuchowski (eds.), vol. 32, Banach Center Publications, Warszawa, pp. 319–339. 31. Respondek W (2001) Transforming nonholonomic control system into the canonical contact form, Proc. of the 40th IEEE Conf. on Decision and Control, Orlando, Florida, pp. 1781–1786. 32. Respondek W (2003) Symmetries and minimal ﬂat outputs: linearizable, contact, and extended Goursat systems, in preparation. 33. Respondek W, Tall IA (2001) How many symmetries does admit a nonlinear single-input control system around an equilibrium?, Proc. of the 40th IEEE Conf. on Decision and Control, Orlando, Florida, pp. 1795–1800. 34. Respondek W, Tall IA (2002) Nonlinearizable single-input control systems do not admit stationary symmetries, Systems and Control Letters, 46:1–16. 35. Respondek W, Tall IA (2003) Symmetries of single-input control systems around equilibria, submitted for publication. 36. Rouchon P, Fliess M, L´evine J, Martin P (1993) Flatness and motion planning: the car with n trailers, Proc. of the European Control Conference ECC’93, Groningen, pp. 1518–1522. 37. van der Schaft AJ (1987) Symmetries in optimal control, SIAM J. Control Optim., 25:245–259. 38. Sørdalen O, Nakamura Y, Chung W (1996) Design and control of a nonholonomic manipulator, Proc. of Ecole d’´et´e d’automatique de l’ENSIEG, Grenoble. 39. Tall IA, Respondek W (2000) Normal forms, canonical forms, and invariants of single-input control systems under feedback, Proc. of the 39th IEEE Conf. on Decision and Control, Sydney, pp. 1625–1630. 40. Tall IA, Respondek W (2003) Feedback classiﬁcation of nonlinear single-input control systems with controllable linearization: normal forms, canonical forms, and invariants, SIAM J. Control and Optim., 41:1498–1531. 41. Tilbury D, Murray R, Sastry S (1995) Trajectory Generation for the N-trailer Problem Using Goursat Normal Form, IEEE Trans. on Automat. Contr., 40:802–819. 42. Tilbury D, Sastry S (1995) The multi-steering n-trailer system: A case study of Goursat normal forms and prolongations, International Journal of Robust and Nonlinear Control, 5(4):343–364. 43. Tilbury D, Sørdalen O, Bushnell L, Sastry S (1995) A multi-steering trailer system: Conversion into chained form using dynamic feedback, IEEE Trans. on Robotics and Automation, 11. 44. von Weber E (1898) Zur Invariantentheorie der Systeme Pfaﬀ’scher Gleichungen. Berichte Verhandlungen der K¨ oniglich S¨ achsischen Gesellshaft der Wissenshaften Mathematisch-Physikalische Klasse, Leipzig, 50:207–229.

Normal Forms of Multi-input Nonlinear Control Systems with Controllable Linearization Issa Amadou Tall1,2 1 2

Department of Mathematics, University of California, One Shields Avenue, Davis 95616, CA, [email protected] Department of Mathematics and Informatics, University Cheikh Anta Diop, Dakar, Senegal

Summary. We study, step by step, the feedback group action on Multi-input nonlinear control systems with controllable linearization. We construct a normal form which generalizes the results obtained in the single-input case. We illustrate our results by studying the prototype of a Planar Vertical TakeOﬀ and Landing aircraft (PVTOL).

1 Introduction We consider the problem of transforming the nonlinear control system Π : ζ˙ = f (ζ, u),

ζ(·) ∈ Rn

u(·) = (u1 (·), · · · , up (·))t ∈ Rp

by a feedback transformation of the form Υ :

x = φ(ζ) u = γ(x, v)

to a simpler form. The transformation Υ brings Π to the system ˜ : x˙ = f˜(x, v) , Π whose dynamics are given by 2 ∂φ 2 f˜(x, v) = . · f (ζ, γ(ζ, v))2 ∂ζ ζ=φ−1 (x)

Research partially supported by AFOSR F49620-01-1-0202

W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 87–100, 2003.

c Springer-Verlag Berlin Heidelberg 2003

88

I.A. Tall

We will follow a very fruitful approach proposed by Kang and Krener [3, 4, 5]. Their idea, which is closely related with the classical Poincar´e technique for linearization of dynamical systems (see e.g. [1]), is to analyze the system Π and the feedback transformation Υ step by step and, as a consequence, to ˜ also step by step. produce a simpler equivalent system Π Although this method produces formal normal forms, the theory developed by Kang and Krener has proved to be very useful in analyzing structural properties of nonlinear control systems. It has been used to study bifurcations of nonlinear systems [6, 7], has led to a complete description of symmetries around equilibrium [11, 20], and allowed the characterization of systems equivalent to feedforward forms [18, 19, 21]. The feedback classiﬁcation of single-input nonlinear control systems via this method is almost complete (see [3, 5, 9, 12, 14, 15, 16]), and the aim of this paper is to deal with the multi-input nonlinear control systems. Preliminary results on the quadratic normal form of systems with multiple inputs were derived in [2], in which it is assume that the system is linearly controllable and the controllability indices equal each other. Preliminary results for two-input control systems, with controllable mode, has been recently obtained by Tall and Respondek [17] and this paper gives a generalization of existing results to multi-input systems with controllable mode. The case of uncontrollable mode is already obtained and will be addressed in another paper. In this paper we propose a normal form for multi-input nonlinear control systems with controllable linearization. The normal form presented here generalizes, in the case of multi-input systems with controllable mode, those obtained in the single-input case [3, 12, 14, 15], and in the two-input case [17]. The paper is organized as following: Section 2 deals with basic deﬁnitions. In Section 3, we construct a normal form for multi-input nonlinear control systems with controllable linearization. We illustrate our results by studying the prototype of a planar vertical takeoﬀ and landing aircraft. In Section 4, we give sketches of proofs of our results. For the details of proofs and the invariants, we send the reader to the complete version [13].

2 Notations and Deﬁnitions All objects, that is, functions, maps, vector ﬁelds, control systems, etc., are considered in a neighborhood of 0 ∈ Rn and assumed to be C ∞ -smooth. For a smooth R-valued function h, deﬁned in a neighborhood of 0 ∈ Rn , we denote by h(x) = h (x) + h (x) + h (x) + · · · = [0]

[1]

[2]

∞

h[m] (x)

m=0

its Taylor series expansion at 0 ∈ Rn , where h[m] (x) stands for a homogeneous polynomial of degree m.

Normal Forms of Multi-input Nonlinear Control Systems

89

Similarly, throughout the paper, for a map φ of an open subset of Rn into Rn (resp. for a vector ﬁeld f on an open subset of Rn ), we will denote by φ[m] (resp. by f [m] ) the term of degree m of its Taylor series expansion [m] [m] at 0 ∈ Rn , that is, each component φj of φ[m] (resp. fj of f [m] ) is a homogeneous polynomial of degree m. We will denote by H [m] (x) the space of homogenous polynomials of degree m of the variables x1 , . . . , xn and by H ≥m (x) the space of formal power series of the variables x1 , . . . , xn starting from terms of degree m. Similarly, we will denote by R[m] (x) the space of homogeneous vector ﬁelds whose components are in H [m] (x) and by R≥m (x) the space of formal vector ﬁelds power series whose components are in H ≥m (x). We consider nonlinear control systems, with multi-input, of the form Π : ζ˙ = f (ζ, u),

ζ(·) ∈ Rn , u(·) = (u1 (·), · · · , up (·))t ∈ Rp

around the equilibrium point (0, 0) ∈ Rn × Rp , that is, f (0, 0) = 0, and we denote by Π [1] : ζ˙ = F ζ + Gu = F ζ + G1 u1 + · · · + Gp up , its linearization at this point, where F =

∂f ∂f ∂f (0, 0), G1 = (0, 0), · · · , Gp = (0, 0) . ∂ζ ∂u1 ∂up

We will assume that G1 ∧ · · · ∧ Gp = 0, and the linearization is controllable, that is 4 3 span F i Gk : 0 ≤ i ≤ n − 1, 1 ≤ k ≤ p = Rn . Let (r1 , · · · , rp ), 1 ≤ r1 ≤ · · · ≤ rp = r, be the largest, in the lexicographic ordering, p-tuple of positive integers, with r1 + · · · + rp = n, such that 4 3 (1) span F i Gk : 0 ≤ i ≤ rk − 1, 1 ≤ k ≤ p = Rn . With the p-tuple (r1 , · · · , rp ) we associate the p-tuple (d1 , · · · , dp ) of nonnegative integers, 0 = dp ≤ · · · ≤ d1 ≤ r − 1, such that r1 + d1 = · · · = rp + dp = r. Our aim is to give a normal form of feedback classiﬁcation of such systems under invertible feedback transformations of the form Υ :

x = φ(ζ) u = γ(ζ, v) ,

where φ(0) = 0 and γ(0, 0) = 0. Let us consider the Taylor series expansion Π ∞ of the system Π, given by Π ∞ : ζ˙ = F ζ + Gu +

∞ m=2

f [m] (ζ, u)

(2)

90

I.A. Tall

and the Taylor series expansion Υ ∞ of the feedback transformation Υ , given by

Υ

∞

x = φ(ζ) = T ζ + :

∞ m=2

φ[m] (ζ)

u = γ(ζ, v) = Kζ + Lv +

(3)

∞ m=2

γ [m] (ζ, v) .

Throughout the paper, in particular in formulas (2) and (3), the homogeneity of f [m] and γ [m] will be taken with respect to the variables (ζ, u)t and (ζ, v)t respectively. We will use an approach proposed by Kang and Krener [3, 4, 5], (see also [15]), which consists of applying the feedback transformation Υ ∞ step by step. We ﬁrst notice that, because of the controllability assumption (1), there always exists a linear feedback transformation Υ1 :

x = Tζ u = Kζ + Lv

bringing the linear part Π [1] : ζ˙ = F ζ + Gu = F ζ + G1 u1 + · · · + Gp up into the Brunovsk´ y canonical form [1]

ΠCF : x˙ = Ax + Bv = Ax + B1 v1 + · · · + Bp vp , where A = diag(A1 , · · · , Ap ), B = (B1 , · · · , Bp ) = diag(b1 , · · · , bp ), that is,     A1 · · · 0 b1 · · · 0     A =  ... . . . ...  , B =  ... . . . ...  (4) 0 · · · Ap

n×n

0 · · · bp

n×p

y single-input canonical forms of dimensions rk , for with (Ak , bk ) in Brunovsk´ any 1 ≤ k ≤ p. Then we study, successively for m ≥ 2, the action of the homogeneous feedback transformations Υm :

x = ζ + φ[m] (ζ) u = v + γ [m] (ζ, v)

(5)

on the homogeneous systems Π [m] : ζ˙ = Aζ + Bu + f [m] (ζ, u) .

(6)

Let us consider another homogeneous system ˜ [m] : x˙ = Ax + Bv + f˜[m] (x, v) . Π

(7)

Normal Forms of Multi-input Nonlinear Control Systems

91

Deﬁnition 1. We say that the homogeneous system Π [m] , given by (6), is ˜ [m] , given by (7), if there feedback equivalent to the homogeneous system Π exist a homogeneous feedback transformation Υ m , of the form (5), which ˜ [m] modulo terms in R≥m+1 (x, v). brings the system Π [m] into the system Π The starting point is the following result generalizing that proved by Kang [3]. Proposition 1. The homogeneous feedback transformation Υ m , deﬁned by (5), brings the homogeneous system Π [m] , given by (6), into the homogeneous sy˜ [m] , given by (7), if and only if the following relation stem Π [Ax + Bv, φ[m] ] + Bγ [m] (x, v) = f˜[m] (x, v) − f [m] (x, v)

(8)

holds. In the formulae (8), we deﬁne the Lie bracket of g and h by ∂h ∂g (x, u) · g(x, u) − (x, u) · h(x, u) . g(x, u), h(x, u) = ∂x ∂x

3 Main Results In this section we will establish our main results. We will give, in Subsection 3.1 below, the normal forms we obtain for general control systems. In Subsection 3.2, we will express those results for control aﬃne systems. In the aﬃne case, when the distribution generated by the control vector ﬁelds is involutive, these vector ﬁelds are normalized and the non removable nonlinearities grouped in the drift. When this distribution is not involutive, one of the control vector ﬁelds is normalized and the non removable nonlinearities grouped in the drift and the remaining control vector ﬁelds. Finally, in Subsection 3.3, we will study an example of the prototype of a Planar Vertical TakeOﬀ and Landing aircraft. 3.1 Non Aﬃne Case Let 1 ≤ s ≤ p. We denote by xs = (xs,ds +1 , · · · , xs,r )t ,

xs,r+1 = vs

and we set x ¯s,i = (xs,ds +1 , · · · , xs,i )t for any ds + 1 ≤ i ≤ r + 1. For any 1 ≤ s ≤ t ≤ p, and any ds + 1 ≤ i ≤ r + 1, we also denote by s πt,i (x) = (¯ x1,i , · · · , x ¯s,i , x ¯s+1,i−1 , · · · , x ¯t−1,i−1 , x ¯t,i , x ¯t+1,i−1 , · · · , x ¯p,i−1 )t ,

where, along the paper, we will take x ¯s,i to be empty for any 0 ≤ i ≤ ds . Our main result for multi-input nonlinear control systems with controllable linearization is as following.

92

I.A. Tall

Theorem 1. The control system Π ∞ , deﬁned by (2), is feedback equivalent, by a formal feedback transformation Υ ∞ of the form (3), to the normal form ∞ ˙ = Ax + Bv + ΠN F : x

∞

f¯[m] (x, v) ,

m=2

where for any m ≥ 2, we have f¯[m] (x, v) =

p

r−1

k=1 j=dk +1

∂ k[m] , f¯j (x, v) ∂xk,j

(9)

with, k[m] f¯j (x, v) =

r+1

1≤s≤t≤p i=j+2

+

k[m−2]

s xs,i xt,i Pj,i,s,t (πt,i (x))

r+1

1≤s
(10) k[m−2] s xs,i xt,i−1 Qj,i,s,t (πt,i−1 (x))

for any 1 ≤ k ≤ p and any dk + 1 ≤ j ≤ r − 1. k[m−2] k[m−2] Above, the functions Pj,i,s,t and Qj,i,s,t stand for homogeneous polynok[m−2]

k[m−2]

mials of degree m − 2 of the indicated variables; Pj,i,s,t equal zero for 1 ≤ i ≤ ds (resp. 1 ≤ i ≤ ds + 1 ).

(resp. Qj,i,s,t ) being

Notice that when p = 1, that is, if we deal with single-input control systems, k[m−2] then the homogeneous polynomials Qj,i,s,t are zero, and thus the normal form reduces to Kang normal form [3] given by [m] f¯j (x, v) =

r+1 i=j+2

k[m−2]

1 x21,i Pj,i,1,1 (π1,i (x)) =

r+1 i=j+2

k[m−2]

x21,i Pj,i

(¯ x1,i ) .

Since the homogeneous feedback transformations Υ m leave invariant the terms of degree less than m, Theorem 1 follows from a successive application of Theorem 2 below. Theorem 2. The homogeneous control system Π [m] , deﬁned by (6), is feedback equivalent, by a homogeneous feedback transformation Υ m of the form (5), to the normal form [m] ΠN F : x˙ = Ax + Bv + f¯[m] (x, v) ,

where for any m ≥ 2, the vector ﬁeld f¯[m] (x, v) is given by (9)-(10). k[m−2]

The homogeneous polynomials Pj,i,s,t

k[m−2]

and Qj,i,s,t [m] ΠN F

deﬁned in (10) are the

homogeneous invariants of the normal form under homogeneous feedback transformations Υ m . For explicit formulas of homogeneous invariants of the system Π [m] under feedback transformations Υ m , we send the reader to the complete version [13].

Normal Forms of Multi-input Nonlinear Control Systems

93

3.2 Aﬃne Case In this section we will study, as a particular case, control aﬃne systems, that is, systems Π of the form Σ : ζ˙ = f (ζ) + g(ζ)u = f (ζ) + g1 (ζ)u1 + · · · + gp (ζ)up , ζ ∈ Rn , u ∈ Rp , and feedback transformations Υ of the form Γ :

x = φ(ζ) u = α(ζ) + β(ζ)v ,

where, for any ζ ∈ Rn , we have α(ζ) = (α1 (ζ), · · · , αp (ζ))t ∈ Rp ,

with α(0) = 0 ,

and β(ζ) ∈ Gl(p, R) a p×p invertible matrix, and v = (v1 , · · · , vp )t . All entries of α and β are smooth functions. The linearly controllable assumption (1) implies span adjf gi (0), 0 ≤ j ≤ ri − 1, 1 ≤ i ≤ p = Rn , where the integers 1 ≤ r1 ≤ · · · ≤ rp , satisfying r1 + · · · + rp = n, are controllability indices of the controllable linear part. In particular, the distribution G = span {g1 , · · · , gp }, spanned by the vector ﬁelds g1 , · · · , gp , is of constant rank p in a neighborhood of 0 ∈ Rn . We study the action of the Taylor series expansion Γ ∞ of the transformation Γ , given by

Γ

∞

x = φ(ζ) = T ζ + :

∞ m=2

φ[m] (ζ)

u = α(ζ) + β(ζ)v = Kζ + Lv + on the Taylor series expansion Σ

∞

Σ ∞ : ζ˙ = Aζ + Bu +

∞ m=2

α[m] (ζ) + β [m−1] (ζ)v

(11)

of the system Σ, given by ∞

f [m] (ζ) + g [m−1] (ζ)u

,

(12)

m=2

where we assume the linear part to be already in the Brunovsk´ y canonical form (A, B) deﬁned by (4). A result, similar to that of Theorem 2, could be deduced for each homogeneous part of the formal system Σ ∞ , whose application step by step, leads to the theorem below. Due to page limitation, this result is not presented here and could be found in the complete version [13]. However, Theorem 3 could also be seen as a corollary of Theorem 1.

94

I.A. Tall

Theorem 3. The formal system Σ ∞ , deﬁned by (12), is feedback equivalent, by a formal feedback transformation Γ ∞ of the form (11), to the normal form ∞ ΣN ˙ = Ax + Bv + F : x

∞ m=2

[m−1] [m−1] f¯[m] (x) + g¯1 (x)v1 + · · · + g¯p−1 (x)vp−1

,

[m−1] where for any m ≥ 2, the vector ﬁeld f¯[m] (x) and the vector ﬁelds g¯s (x), for 1 ≤ s ≤ p − 1, are given by

f¯[m] (x) =

p

r−1

k=1 j=dk +1

g¯s[m−1] (x) =

p

r−1

∂ k[m] , f¯j (x) ∂xk,j k[m−1]

g¯s,j

(x)

k=1 j=dk +1

∂ , ∂xk,j

with k[m] f¯j (x) =

r

1≤s≤t≤p i=j+2

+

k[m−2]

s xs,i xt,i Pj,i,s,t (πt,i (x))

r

k[m−2]

1≤s
s xs,i xt,i−1 Qj,i,s,t (πt,i−1 (x)) ,

and k[m−1]

g¯s,j

(x) =

p t=s+1

k[m−2]

s xt,r Qj,r+1,s,t (πt,r (x))

for any 1 ≤ k ≤ p, and any dk + 1 ≤ j ≤ r − 1. Moreover, if the formal distribution 5 . ∞ ∞ [m−1] ∞ [m−1] g1 , · · · , Bp + gp G = span B1 + m=2

is involutive, then the homogeneous polynomials

m=2

k[m−2] Qj,r+1,s,t

are equal zero.

3.3 Example In this subsection we will illustrate our results by considering the physical example described by the prototype of a Planar Vertical TakeOﬀ and Landing aircraft (PVTOL). The equations of motion of the PVTOL (see [10]) are given by x ¨ = − sin θu1 + ε2 cos θu2 , y¨ = cos θu1 + ε2 sin θu2 − 1, θ¨ = u2 ,

Normal Forms of Multi-input Nonlinear Control Systems

95

where (x, y) denotes the position of the center mass of the aircraft, θ the angle of the aircraft relative to the x-axis, ” − 1” the gravitational acceleration and ε = 0 the (small) coeﬃcient giving the coupling between the rolling moment and the lateral acceleration of the aircraft. The control inputs u1 and u2 are the thrust (directed out the bottom of the aircraft) and the rolling moment. We introduce the variables ζ1,1 ζ2,1 ζ2,3 w1

= y, = x, = θ, = u1 − 1,

ζ1,2 ζ2,2 ζ2,4 w2

= y, ˙ = x, ˙ ˙ = θ, = u2 .

The equations of motion of the PVTOL become ζ˙1,1 ζ˙1,2 ζ˙2,1 ζ˙2,2 ζ˙2,3 ζ˙2,4

= ζ1,2 = cos ζ2,3 w1 + ε2 sin ζ2,3 w2 + cos ζ2,3 − 1 = ζ2,2 = − sin ζ2,3 w1 + ε2 cos ζ2,3 w2 − sin ζ2,3 = ζ2,4 = w2 .

(13)

The equilibria of the system is deﬁned by e e e e e e (ζ1,1 , ζ1,2 , ζ2,1 , ζ2,2 , ζ2,3 , ζ2,4 , w1e , w2e )t = (c, 0, 0, 0, 0, 0, 0, 0)t ,

where c is any constant. The linearization of the system (13) around the equilibria is given by ζ˙1,1 ζ˙1,2 ζ˙2,1 ζ˙2,2 ζ˙2,3 ζ˙2,4

= ζ1,2 = w1 = ζ2,2 = −ζ2,3 + ε2 w2 = ζ2,4 = w2 .

It is easy to see that the linear system is controllable with controllability indices r1 = 2 and r2 = 4, and hence d1 = 2 and d2 = 0. The change of coordinates given by 6 ζ2,3 dt x1,3 = ζ1,1 − ε2 cos t 0 ε2 ζ2,4 x1,4 = ζ1,2 + ζ1,2 tan ζ2,3 − cos ζ2,3 x2,1 = ζ2,1 x2,2 = ζ2,2 x2,3 = − tan ζ2,3 x2,4 = −ζ2,4 (1 + tan2 ζ2,3 ) = x˙ 2,3

96

I.A. Tall

followed by the feedback w1 =

v1 1 − ε2 v2 tan ζ2,1 + −1 cos ζ2,1 cos ζ2,1

2 tan ζ2,3 (1 + tan2 ζ2,3 ) v2 = x˙ 2,4 = −w2 (1 + tan2 ζ2,3 ) − 2ζ2,4

takes the system into the following one x˙ 1,3 x˙ 1,4 x˙ 2,1 x˙ 2,2 x˙ 2,3 x˙ 2,4

= x1,4 = v1 = x2,2 + x1,4 x2,3 = x2,3 − x1,4 x2,4 + ε2 (1 − x22,3 )x22,4 = x2,4 = v˜2 .

This system is in normal form (compare with Theorem 3), with 2[0]

2[0]

2[0]

2[2]

Q1,4,1,2 (x) = 1, P2,4,1,2 (x) = −1, P2,4,2,2 (x) = ε2 , P2,4,2,2 (x) = −ε2 x22,3 .

4 Proofs of Main Results The aim of this section is to give sketches of the proofs of Theorems 2 and 3. The proof of Theorem 2 is facilitated by the remark stated below. We ﬁrst notice that the homogeneous system Π [m] , deﬁned by (6), can be decomposed into p homogeneous subsystems Π k[m] , for 1 ≤ k ≤ p, given by Π k[m] : ζ˙k = Ak ζk + bk uk + f k[m] (ζ, u) , where ζ = (ζ1 , · · · , ζp )t , with ζk = (ζk,dk +1 , · · · , ζk,r )t . Moreover, the transformation (5) can also be viewed as a composition of p homogeneous transformations

Υ km

xk x : i uk ui

= ζk + φk[m] (ζ) = ζi = vk + γ k[m] (ζ, v) = vi

if i = k if i = k

for 1 ≤ k ≤ p. Remark. For any 1 ≤ k ≤ p and any l = k, the homogeneous feedback transformation Υ km leaves invariant the homogeneous subsystem Π l[m] . It thus suﬃces to study, for a ﬁxed 1 ≤ k ≤ p, the action of the homogeneous feedback transformation Υ km on the homogeneous subsystem Π k[m] .

Normal Forms of Multi-input Nonlinear Control Systems

97

4.1 Sketch of Proof of Theorem 2 Let us consider the homogeneous system Π [m] , given by (6), and assume, for the simplicity of the proof, that the controllability indices are equal, that is, r1 = · · · = rp = r. The details and the case of non equal controllability indices are given in [13]. Using the remark above, we study the action of the homogeneous feedback transformation Υ km on the homogeneous subsystem k[m] ζ˙k,1 = ζk,2 + f1 (ζ, u) .. .

Π k[m] :

(14)

k[m] ζ˙k,r−1 = ζk,r + fr−1 (ζ, u) ζ˙k,r = uk .

The proof is constructive based on an inductive argument. At each step we construct the transformation which annihilates some undesirable terms. k[m] For any 1 ≤ j ≤ r − 1, we represent the homogeneous polynomial fj uniquely as follows k[m]

fj

r+1

(ζ, v) =

k[m−2]

1≤s≤t≤p i=j+2

+

s ζs,i ζt,i Pj,i,s,t (πt,i (ζ))

r+1

1≤s
+

r+1

1≤t≤p i=j+2

k[m−2]

t ζs,i ζt,i−1 Qj,i,s,t (πt,i−1 (ζ))

k[m−1]

ζt,i Rj,i,t

k[m]

p t (πt,i−1 (ζ)) + Sj,j+1 (πp,j+1 (ζ)) .

The aim is to annihilate all terms of the form

r+1

1≤t≤p i=j+2

k[m−1]

ζt,i Rj,i,t

k[m]

p t (πt,i−1 (ζ)) + Sj,j+1 (πp,j+1 (ζ))

while keeping the structure of the other terms (not the terms itself) invariant. The change of variables given by 6 ζt,r k[m−1] t yk,j = ζk,j − Rj,r+1,t (πt,r (ζ))dζt,r , for 1 ≤ j ≤ r − 1 1≤t≤p

0

k[m]

p (ζ)) yk,r = ζk,r + Sˆr+1 (πp,r wk = y˙ k,r ,

allows us to annihilate the homogeneous terms k[m−1] t ζt,r+1 Rj,r+1,t (πt,r (ζ)) . 1≤t≤p

98

I.A. Tall

We then apply the feedback 6 yt,r−1 k[m−1] t zk,j = yk,j − Rj,r,t (πt,r−1 (y))dyt,r−1 , for 1 ≤ j ≤ r − 2 1≤t≤p

0

p zk,r−1 = yk,r−1 + Sˆrk[m] (πp,r−1 (y)) zk,r = z˙k,r−1

vk = z˙k,r , to annihilate the terms of the form k[m−1] t yt,r Rj,r,t (πt,r−1 (y)) . 1≤t≤p

Each of these feedback transformations allows us to annihilate undesirable terms and modify the other terms. What is important is that the second transformation doesn’t modify the terms

k[m−2]

s ys,r+1 yt,r+1 Pj,r+1,s,t (πt,r+1 (y)) +

1≤s≤t≤p

k[m−2]

t ys,r+1 yt,r Qj,r+1,s,t (πt,r (y))

1≤s
since it only depends on the variables ys,1 , · · · , ys,r , for 1 ≤ s ≤ p. We keep applying those transformations and after we get rid to the homogeneous terms of the form r+1

k[m−1]

ζt,i Rj,i,t

t (πt,i−1 (ζ)) ,

1≤t≤p i=l+2

for some l ≥ j + 1, we then apply a feedback transformation of the form yk,j = ζk,j −

6 1≤t≤p

ζt,l

0

k[m−1]

t Rj,l+1,t (πt,l (ζ))dζt,l ,

k[m] p (ζ)) yk,l = ζk,l + Sˆl,l+1 (πp,l yk,j+1 = y˙ k,j vk = y˙ k,r

for 1 ≤ j ≤ l − 1

for l ≤ j ≤ r − 1

to annihilate the homogeneous terms

k[m−1]

t ζt,l+1 Rj,l+1,t (πt,l (ζ)) .

1≤t≤p k[m]

p (ζ)) present in the j th -component are At the end, the terms Sj,j+1 (πp,j+1 linearizable. This ends the sketch of the proof.

Normal Forms of Multi-input Nonlinear Control Systems

99

4.2 Sketch of Proof of Theorem 3 Let us consider a homogeneous system Σ [m] : ζ˙ = Aζ + Bu + f˜[m] (ζ, u) = Aζ + Bu + f [m] (ζ) + g [m−1] (ζ)u . Thus, Theorem 2 implies the existence of a feedback transformation Υm :

x = ζ + φ[m] (ζ) u = v + γ [m] (ζ, v)

which takes the system Σ [m] into the normal form [m] ΣN F : x˙ = Ax + Bv + f¯[m] (x, v) ,

where the vector ﬁeld f¯[m] (x, v) is given by (9)-(10). Following Proposition 8, we have f¯[m] (x, v) = f˜[m] (x, v) + Ax + Bv, φ[m] (x) + Bγ [m] (x, v) . For any 1 ≤ s ≤ t ≤ p, we get by diﬀerentiation ∂ 2 f¯[m] ∂ 2 f˜[m] ∂ 2 γ [m] ∂ 2 γ [m] (x, v) = (x, v) + B (x, v) = B (x, v) ∂vs ∂vt ∂vs ∂vt ∂vs ∂vt ∂vs ∂vt since f˜[m] is aﬃne with respect to the control. It thus follows that k[m]

∂ 2 f¯j (x, v) = 0 ∂vs ∂vt for any dk + 1 ≤ j ≤ r − 1. k[m−2] This is equivalent of saying that the homogeneous polynomials Pj,r+1,s,t are equal to zero for 1 ≤ k ≤ p and dk + 1 ≤ j ≤ r which means that the transformation takes the system into its normal form. A successive application of this result for m = 2, 3, · · · leads to Theorem 3.

References 1. Arnold VI (1988) Geometrical Methods in the Theory of Ordinary Diﬀerential Equations, Second Edition, Springer-Verlag, 1988. 2. Kang W (1991) Extended controller normal form, invariants and dynamic feedback linearization of nonlinear control systems, Ph.D. dissertation, University of California at Davis. 3. Kang W (1994) Extended controller form and invariants of nonlinear control systems with single input, J. of Mathem. System, Estimation and Control, 4, pp. 253–256.

100

I.A. Tall

4. Kang W (1995) Quadratic normal forms of nonlinear control systems with uncontrollable linearization, Proc. 34th CDC, New Orleans. 5. Kang W, Krener AJ (1992) Extended quadratic controller normal form and dynamic feedback linearization of nonlinear systems, SIAM J. Control and Optim., 30, pp. 1319–1337. 6. Kang W (1998) Bifurcation and normal form of nonlinear control systems − parts I and II, SIAM J. Control and Optimization, 36, 193–212 and 213–232. 7. Kang W (2000) Bifurcation control via state feedback for systems with a single uncontrollable mode, SIAM J. Control and Optimization, 38, 1428–1452. 8. Krener AJ (1984) Approximate linearization by state feedback and coordinate change, Systems and Control Letters, 5, pp. 181–185. 9. Krener AJ, Normal forms of nonlinear control systems with uncontrollable mode, preprint. 10. Sastry S (1999) Nonlinear Systems: Analysis, Stability, and Control, SpringerVerlag. 11. Respondek W, Tall AI (2001) How many symmetries does admit a nonlinear single-input control system around equilibrium, in Proc. of the 40th CDC, pp. 1795–1800, Florida. 12. Tall IA (2000) Classiﬁcation par bouclage des syst`emes de contrˆ oles non lin´eaires mono-entr´ee: formes normales, formes canoniques, invariants et sym´etries. Ph. D. Thesis, INSA de Rouen, France. 13. Tall IA, Normal forms, dual normal forms, and invariants for multi-input nonlinear control systems, in preparation. 14. Tall IA, Respondek W (2003) Feedback classiﬁcation of nonlinear single-input control systems with controllable linearization: normal forms, canonical forms, and invariants, SIAM Journal on Control and Optimization, 41(5) pp. 1498– 1531. 15. Tall IA, Respondek W (2000) Normal forms, canonical forms, and invariants of single single-input control systems under feedback, Proc. 39th CDC, Sydney, pp. 1625–1630. 16. Tall IA, Respondek W (2001) Normal forms and invariants of nonlinear singleinput systems with noncontrollable linearization, NOLCOS’01, Petersburg, Russia. 17. Tall IA, Respondek W (2002) Normal forms of two-inputs nonlinear control systems, submitted to Proc. 41th CDC, Las Vegas, USA, pp. 2732–2737. 18. Tall IA, Respondek W, Feedback equivalence to a strict feedforward form for nonlinear single-input systems, to appear in International Journal of Control. 19. Tall IA, Respondek W (2001) Transforming a single-input nonlinear system to a strict feedforward form via feedback, Nonlinear Control in the Year 2000, A. Isidori, F. Lamnabhi-Lagarrigue, and W. Respondek, (eds.), Springer-Verlag, 2, pp. 527–542, London, England. 20. Tall IA, Respondek W (2002) Nonlinearizable analytic single-input control systems with controllable linearization do not admit stationary symmetries, Systems and Control Letters, 46 (1), pp. 1–16. 21. Tall IA, Respondek W (2002) Feedback equivalence to feedforward form for nonlinear single-input systems, Dynamics, Bifurcations and Control, F. Colonius and L. Grune (eds.), LNCIS, 273, pp. 269–286, Springer-Verlag, Berlin Heidelberg.

Control of Hopf Bifurcations for Inﬁnite-Dimensional Nonlinear Systems MingQing Xiao1 and Wei Kang2 1 2

Southern Illinois University, Carbondale, IL 62901, USA, [email protected] Naval Postgraduate School, Monterey, CA 93943, USA, [email protected]

— Dedicated to Professor Arthur J. Krener on the occasion of his 60th birthday. Summary. This paper addresses the problem of local feedback stabilization of Hopf bifurcations for inﬁnite dimensional systems. The systems are nonlinear, in which the linearization has a simple conjugate pair of eigenvalues on the imaginary axis. All other eigenvalues of the system lie strictly in the left half-plane. Using the integral averaging method, the normal form of nonlinear systems in polar coordinates is derived. Based on the normal form, several nonlinear control laws are derived to stabilize or destabilize the periodic solutions generated by the Hopf bifurcation.

1 Introduction The study of changes in the qualitative structure of the ﬂow of a diﬀerential equation as parameters are varied is called bifurcation theory. At a given parameter value, a diﬀerential equation is said to have stable orbit structure if the qualitative structure of the ﬂow does not change for suﬃciently small variations of the parameter. A parameter value for which the ﬂow does not have stable orbit structure is called a bifurcation value, and the equation is said to be at a bifurcation point. Bifurcation phenomena are an inherent behavior of nonlinear systems with great physical interest (for example, see [8], [16], [18], [19], [3], [18]). Hopf bifurcations are found in many complex dynamical systems in scientiﬁc and industrial research, such as in the ﬂutter dynamics of an aircraft wing, in the oscillation of nonlinear circuits, in the surge and stall of aircraft engines, and in the voltage collapse of power transmission networks. For example, a supercritical Hopf bifurcation describes the ﬁrst stage of transition to turbulence in

The research and both authors are supported in part by AFRL.

W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 101–116, 2003.

c Springer-Verlag Berlin Heidelberg 2003

102

M. Xiao and W. Kang

ﬂuids as postulated by Landau [17], and a subcritical Hopf bifurcation occurs for ﬂows that exhibit an immediate transition behavior [18]. For the supercritical Hopf bifurcation, as the parameter passes a critical value, its stable equilibrium is replaced by a stable limit cycle of small amplitude. Thus the state of the system stays in a neighborhood of the equilibrium. It exhibits a soft or noncatastrophic stability loss. For the subcritical Hopf bifurcation, the region of attraction of the equilibrium point is bounded by the unstable cycle, which shrinks as the parameter approaches its critical value. The domain of attraction disappears and the equilibrium becomes unstable if the parameter passes through the critical value. This phenomenon leads to a sharp or catastrophic loss of stability. It is important and critical to understand how to control such bifurcations, by either completely suppressing the bifurcation or utilizing it beneﬁcially for some non-traditional, time- or energy-critical, and safety-guaranteed applications. The feedback control of the Hopf bifurcation for ﬁnite-dimensional nonlinear systems was studied by Abed and Fu [1]. In [2], stationary bifurcations are studied for systems with a linearization that possesses a simple zero eigenvalue. Liaw and Abed studied the active control of compressors by bifurcation control [20]. Colonius and Kliemann studied the stabilization of one-dimensional control systems [6]. Using the normal form of nonlinear systems, Kang developed the theory and algorithms of controller design for systems with one or two uncontrollable modes, including both stationary and Hopf bifurcations ([12], [13], [14] and [11]). Gu et al studied the stabilization of bifurcations using output feedback [10]. Wang and Murray studied the feedback stabilization of steady-state and Hopf bifurcations with a multi-input case [23]. Zaslavsky investigated the feedback stabilization of connected nonlinear oscillators with uncontrollable linearization [24]. The existing methods of bifurcation control are based on the calculation of the ﬁrst Lyapunov coeﬃcient. The ﬁrst Lyapunov coeﬃcient is always computable in the ﬁnite-dimensional case since it depends on the solutions of a set of algebraic equations, which are always solvable (see e.g. [16] or [4]). However, for inﬁnite-dimensional systems, the computation of the ﬁrst Lyapunov coeﬃcient may require the solution of corresponding boundary-value problems (for example, see section 5.6 of [16]), which are cumbersome and even formidable in some cases. In this paper, we develop a bifurcation control method for inﬁnite-dimensional nonlinear systems by using a so called integral averaging method, in which a direct connection between the control feedback and the stability of the periodic solutions is established, while the solution of corresponding boundary-value problems is not required.

2 Preliminaries Let X be a Hilbert space with inner product < ·, · > and norm · . We consider a nonlinear evolution control system of the form

Control of Hopf Bifurcations for Inﬁnite-Dimensional Nonlinear Systems

dx = A(µ)x + fµ (x, u(x)), dt

103

(1)

where A(µ) is the generator of an analytic linear semigroup Tµ (t) on X for each µ in a neighborhood of the origin, U is an open set of X, u : X → U is a control function, and fµ : D(A(µ)) × U → X is a smooth function with fµ (0, u(0)) = 0andfµ (0, u(0)) = 0 (fµ is the Fr´echet derivative of fµ ). Without loss of generality (e.g. see [4]), we may make the following assumptions for µ ∈ (−δ, δ) with some positive number δ > 0 1. X = X1 ⊕ X2 , where X1 is two-dimensional and X2 is closed. 2. X1 is A(0)-invariant. If A1 (µ) is the restriction of A(µ) to X1 , then * + α(µ) −ω(µ) . A1 (µ) = ω(µ) α(µ)

(2)

3. Let Sµ (t) be the restriction of Tµ (t) to X2 . We assume that X2 is Sµ (t)invariant at µ = 0. We also assume that there exist positive constants M and b so that Sµ (t) ≤ M e−bt .

(3)

Remark 1. Let P be the projection from X = X1 ⊕X2 into X1 . Then Assumption 3 implies that the inﬁnitesimal generator A2 (µ) of the semigroup Sµ (t) in X2 is the restriction of the operator (Id − P )A(µ) on X2 , where Id stands for the identity mapping from X to X, and the spectrum of A2 (µ) stays in the left-half plane a uniform positive distance away from the imaginary axis for small µ. Lemma 1. Let P be the projection from X = X1 ⊕ X2 into X1 . Then, there exist linear and bounded operators E(µ) : D(A(µ)) ∩ X2 → X1 H(µ) : D(A(µ)) ∩ X1 → X2 such that for any x = x1 + x2 ∈ D(A(µ)), where x1 ∈ X1 and x2 ∈ X2 , we have the following equalities, P A(µ)x = A1 (µ)x1 + µE(µ)x2 (Id − P )A(µ)x = µH(µ)x1 + (Id − P )A(µ)x2 . Proof. Let x1 ∈ X1 , x2 ∈ X2 and x1 + x2 = x ∈ D(A(µ)). Then P A(µ)x = P A(µ)(x1 + x2 ) = P A(µ)x1 + P A(µ)x2 and (Id − P )A(µ)x = (Id − P )A(µ)x1 + (Id − P )A(µ)x2 .

(4) (5)

104

M. Xiao and W. Kang

According to the deﬁnition of P , and the assumptions 2 and 3, the operator P A satisﬁes P A(µ)x2 = 0, (Id − P )A(µ)x1 = 0 at µ = 0. Therefore, we can deﬁne the operators E(µ) and H(µ) as follows, µE(µ) := the restriction of P A(µ) on X2 µH(µ) := the restriction of (Id − P )A(µ) on X1 . Notice that Range(E(µ)) ⊆ X1 , thus E(µ) is a compact operator. It must also be bounded. Similarly, because the domain D(H(µ)) has ﬁnite dimension, H(µ) is compact and bounded. 2 Let f1 (x1 , x2 , µ) := P fµ (x1 + x2 , u(x1 + x2 )) f2 (x1 , x2 , µ) := (Id − P )fµ (x1 + x2 , u(x1 + x2 ))

(6)

A2 (µ) := the restriction of (Id − P )A(µ) on X2 .

(7)

and

According to Lemma 1, the system deﬁned by (1) has the following form, x˙ 1 = A(µ)x1 + µE(µ)x2 + f1 (x1 , x2 , µ) x˙ 2 = µH(µ)x1 + A2 (µ)x2 + f2 (x1 , x2 , µ).

(8)

Remark 2. In the sections that follow, nonlinear feedbacks of quadratic and cubic functions of the following forms are used to control the Hopf bifurcation. For quadratic feedback, u(x1 ) = p11 x211 + p12 x11 x12 + p21 x12 x11 + p22 x222 := [p11 , p12 , p21 , p22 ]x21 ,

(9)

where pij is a constant for 1 ≤ i, j ≤ 2. For cubic feedback control, u(x1 ) = p11 x311 + p12 x211 x12 + p13 x12 x211 + p21 x212 x11 +p22 x212 x11 + p23 x322 := [p11 , p12 , p13 , p21 , p22 , p23 ]x31 .

(10)

2.1 Integral Averaging Method In this section, we introduce the integral averaging method for inﬁnitedimensional nonlinear systems around the point of the Hopf bifurcation. In [5], Chow and Mallet-Paret developed a framework for the integral averaging method for the Hopf bifurcation of inﬁnite-dimensional systems based on a ﬁnite-dimensional version of the averaging method. Here we complete all details which are needed for our control design in section 3.

Control of Hopf Bifurcations for Inﬁnite-Dimensional Nonlinear Systems

105

Since f1 , f2 are suﬃciently smooth, we can expand (8) as follows: x˙ 1 = B0 (x2 , µ) + B1 (x2 , µ)x1 + B2 (x2 , µ)x21 + . . . , x˙ 2 = Γ0 (x1 , µ) + Γ1 (x1 , µ)x2 + Γ2 (x1 , µ)x22 + . . . .

(11)

where Bi (x2 , µ) and Γi (x1 , µ) are symmetric i-linear operators [15]. According to (8) and Taylor expansions, we may assume B0 (x2 , µ) B1 (x2 , µ) Γ0 (x1 , µ) Γ1 (x1 , µ)

= µE(µ)x2 + F (x2 , µ)x22 = A1 (µ) + G(x2 , µ)x2 = µH(µ)x1 + J(x1 , µ)x21 = A2 (µ) + N (x, µ)x1

(12)

where F (·, µ), J(·, µ), G(·, µ) and N (·, µ) are symmetric operators on appropriate spaces. We shall see that the operators Bj , J, and G determine the stability of the Hopf bifurcation. For further discussion, we denote + * g (x , µ)x2 g12 (x2 , µ)x2 (13) G(x2 , µ)x2 := 11 2 g21 (x2 , µ)x2 g22 (x2 , µ)x2 where gij (·, µ) : X2 → R is an linear operator, and + * 1 b (x , x , µ) Bj (x2 , µ)xj1 := 2j 1 2 j = 3, 4, . . . bj (x1 , x2 , µ), Next, the integral averaging is applied to the system in polar coordinates x1 = (r cos θ, r sin θ). System (11) is transformed into the following form, r˙ = [µE1 (θ, µ)x2 + F1 (θ, x2 , µ)x22 ] + r[α(µ) + G2 (θ, x2 , µ)x2 ] + r2 C3 (θ, x2 , µ) +r3 C4 (θ, x2 , µ) + . . . , ˙θ = 1 [µE ˆ1 (θ, µ)x2 + Fˆ1 (θ, x2 , µ)x2 ] + [ω(µ) + G ˆ 2 (θ, x2 , µ)x2 ] + rD3 (θ, x2 , µ) 2 r 2 +r D4 (θ, x2 , µ) + . . . , x˙ 2 = Γ0 (r, θ, µ) + Γ1 (r, θ, µ)x2 + Γ2 (r, θ, µ)x22 + Γ3 (r, θ, µ)x32 + . . . ˆ1 , F1 , Fˆ1 , G1 , G ˆ 1 are operators which are computed from E, F, G where E1 , E on appropriate spaces and G2 , Cj , Dj , j ≥ 3 are real-valued functions which have expressions G2 (θ, x2 , µ) = cos2 θg11 (x2 , µ) + sin θ cos θg12 (x2 , µ) + sin θ cos θg21 (x2 , µ) + sin2 θg22 (x2 , µ). and

*

(14)

+ * + cos θ cos θ 2 Cj (θ, x2 , µ) = , x2 , µ) + (sin θ)bj−1 ( , x2 , µ) * sin θ + * sin θ + (15) cos θ cos θ Dj (θ, x2 , µ) = (cos θ)b2j−1 ( , x2 , µ) − (sin θ)b1j−1 ( , x2 , µ) sin θ sin θ (cos θ)b1j−1 (

106

M. Xiao and W. Kang

for j = 3, 4, · · · . To further simplify the system, we scale the above equations by r → r,

x2 → x2 ,

µ → µ

and deﬁne α1 (µ) := −1 α(µ), then r˙ = [α1 r + r2 C3 (θ, x2 , µ) + µE1 (θ, µ) + F1 (θ, x2 , µ)x22 +rG2 (θ, x2 , µ)x2 ] + 2 r3 C4 (θ, x2 , µ) + O(3 ),

(16)

and let ω(0) = ω0 , noting ω(µ) = ω0 + µω (0) + O(2 ), then ˆ1 (θ, µ)x2 + 1 Fˆ1 (θ, x2 , µ)x2 θ˙ = ω0 + [µω (0) + rD3 (θ, x2 , µ) + µr E 2 r (17) ˆ 2 (θ, x2 , µ)x2 ] + O(2 ), +G and x˙ 2 = A2 (µ)x2 + [µH(µ)x1 + J(x1 , µ)x21 + N (x1 , µ)x1 x2 +Γ2 (x1 , µ)x22 ] + O(2 ).

(18)

Now, we deﬁne some functions for the simpliﬁcation of the system in polar coordinates. Let 6 r2 θ φ1 (r, θ, µ, ) := − C3 (s, 0, µ)ds, (19) ω0 0 and let w(r, θ, µ, ) be an element of the dual space of D (A2 (µ))∩X2 satisfying the evolution equation rG2 (θ, 0, µ) +

∂w (r, θ, µ, )ω0 + w(r, θ, µ, )A2 (µ) = 0. ∂θ

(20)

If we deﬁne w(r, θ, µ, ) = rw(θ, ˆ µ, ), then w ˆ satisﬁes G2 (θ, 0, µ) +

∂w ˆ (θ, µ, )ω0 + w(θ, ˆ µ, )A2 (µ) = 0. ∂θ

(21)

which is also an evolution equation on the dual space of D(A(µ) ∩ X2 ). We shall show that (21) is solvable in next section. Now we consider the coordinate transformation r := r + φ1 (r, θ, µ, ) + w(r, θ, µ, )x2 + 2 φ2 (r, θ, µ, )

(22)

where φ2 is a smooth functional that will be speciﬁed later. The transformation (22) has inverse r = r − φ1 (r, θ, µ, ) + O(x2 ) + O(2 ). Notice that

(23)

Control of Hopf Bifurcations for Inﬁnite-Dimensional Nonlinear Systems

107

∂φ1 dr ∂φ1 dθ dφ1 = + dt ∂r dt ∂θ dt ∂φ1 2 ∂φ1 α1 (µ)r + r2 C3 + = (ω0 + µω (0) + rD3 ) + O(2 x2 ) ∂r ∂θ +O(x2 2 ),

and

d ∂w dr ∂w dθ dx2 (wx2 ) = x2 + x2 + w dt ∂r dt ∂θ dt dt ∂w ω0 x2 + w A2 (µ)x2 + (µHx1 + Jx21 ) + O(2 x2 ) = ∂θ +O(x2 2 ),

and 2

dφ2 ∂φ2 dr ∂φ2 dθ ∂φ2 = 2 + 2 = 2 ω0 + O(3 ). dt ∂r dt dθ dt dθ

Now substituting (22) into (16) yields 1 r˙ = [α1 (µ)r + r2 C3 ] + [µE1 + rG2 ]x2 + 2 r3 C4 + 2 ∂φ ∂r (r, θ, µ, )[α1 (µ)r ∂φ1 ∂w 2 +r C3 ] ∂θ (r, θ, µ, )[ω0 + µω (0) + rD3 ] + ∂θ (r, θ, µ, )ω0 x2 +w(r, θ, µ, )A2 (µ)x2 + 2 w(r, θ, µ, c)[µH(µ)x1 + J(x1 , µ)x21 ] 2 2 3 2 +2 ∂φ ∂θ (r, θ, µ, )ω0 + O(x2 ) + O( x2 ) + O( ) ∂φ1 ∂w 2 = [α1 r + r C3 + ∂θ (r, θ, µ, )ω0 ] + [µE1 + rG2 + ∂θ (r, θ, µ, )ω0 2 1 +w(r, θ, µ, )A2 (µ)]x2 + 2 [r3 C4 + ∂φ ∂r (r, θ, µ, )(α1 r + r C3 ) ∂φ1 + ∂θ (r, θ, µ, )(α1 ω (0) + rD3 ) + w(r, θ, µ, )(µHx1 + Jx21 ) ∂φ21 2 −φ1 (r, θ, µ, )(α1 + 2rC3 + ∂r∂θ (r, θ, µ, )ω0 ) + ∂φ ∂θ (r, θ, µ, )ω0 ] 2 2 3 +O(x2 ) + O( x2 ) + O( ) (24)

where C3 , E1 , G2 , · · · are evaluated at (θ, 0, µ). According to (19) and (20), the coeﬃcient of in (24) can be simpliﬁed because the functions φ1 and w satisfy r2 C3 (θ, 0, µ) +

∂φ1 (r, θ, µ, )ω0 = 0, ∂θ

and rG2 +

∂w (r, θ, µ, )ω0 + w(r, θ, µ, )A2 (µ) = 0. ∂θ

To simplify the coeﬃcient of 2 in equation (24), we deﬁne Φ(r, θ, µ, ) by the following equation,

108

M. Xiao and W. Kang

∂φ1 3 1 r3 C4 + ∂φ rD3 ) ∂r (r, θ, µ, )(α1 r + r C3 ) + ∂θ (r, θ, µ, )(α1 ω (0) + ∂φ21 2 +w(r, θ, µ, )(µHx1 + Jx1 ) − φ1 (r, θ, µ, )(α1 + 2rC3 + ∂r∂θ (r, θ, µ, )ω0 ) 2 + ∂φ ∂θ (r, θ, µ, )ω0 2 := Φ(r, θ, µ, )ω0 + ∂φ ∂θ (r, θ, µ, )ω0 . (25)

Let 6 φ2 (r, θ, µ, ) := −

θ

Φ(r, θ, µ, )dθ + 0

θ 2π

6

2π

Φ(r, θ, µ, )dθ. 0

Then the right side of (25) equals 6 2π 1 Φ(r, θ, µ, )dθ. 2π 0 Notice that O(α1 ) = O(µ). A direct calculation yields * + 6 2π 6 r3 2π 1 1 C4 (s, 0, µ) − Φ(r, θ, µ, )dθ = C3 (s, 0, µ)D3 (s, 0, µ) ds 2π 0 2π 0 ω0 6 2π 6 6 1 r2 α1 2π θ + w(r, θ, µ, )J(0, 0)x21 dθ + C3 (s, 0, µ)dsdθ. 2π 0 2πω0 0 0 Deﬁne κ := κ1 + κ2 and + 6 2π * 1 1 C4 (θ, 0, 0) − C3 (θ, 0, 0)D3 (θ, 0, 0) dθ κ1 := 2π 0 ω0 6 2π 1 κ2 := w(θ, ˆ 0, 0)J(0, 0)(cos θ, sin θ)2 dθ, 2π 0

(26) (27)

and χ :=

1 2πω0

6

2π

6

θ

C3 (s, 0, 0)dsdθ, 0

(28)

0

where C3 , C4 , D3 are deﬁned by (15), and w ˆ is the solution of (21) which will be discussed in next section. Then the coeﬃcient of 2 in (24) equals 6 2π Φ(r, θ, µ, )dθ = µα (0)χr2 + κr3 . 0

To summarize, the system deﬁned by (16),(17), and (18) is transformed into the following averaged system under the transformation (22), r˙ = α(µ)r + 2 µα (0)χr2 + 2 κr3 + O(3 ), θ˙ = ω0 + O(),

(30)

x˙ 2 = A2 (µ)x2 + O(),

(31)

(29)

Control of Hopf Bifurcations for Inﬁnite-Dimensional Nonlinear Systems

because of x2 = o(). If we let µ = , (29)-(31) become r˙ = 2 α (0)r + κr3 + O(3 ), θ˙ = ω0 + O(), x˙ 2 = A2 (µ)x2 + O().

109

(32) (33) (34)

This leads to the inﬁnite-dimensional version of Poincar´e-Andronov-Hopf theorem: Theorem 1. The bifurcation curve for periodic orbits of (1) under the new coordinates is approximately µ=−

κ 2 r + O(r3 ) α (0)

(35)

as r → 0. The periodic orbit is orbitally asymptotically stable if κα (0) < 0, and unstable if κα (0) > 0. 2.2 The Existence of w(θ, ˆ µ, ) Lemma 2. The operator equation (21) admits a unique solution that is 2πperiodic in θ. The solution can be expressed in the following form w(θ, ˆ µ, ) = −

∞ n=−∞

µ g2n ◦ (A2 (µ) + inω0 )−1 einθ

(36)

µ where g2n represents the nth coeﬃcient in the Fourier expansion of G2 (θ, 0, µ).

Proof. Notice that the operator equation (21) is in the dual space of D(A2 (µ)). Let x2 ∈ D(A2 (µ)), then G2 (θ, 0, µ)x2 is in the space L2 (0, 2π). Thus it has Fourier series expansion: G2 (θ, 0, µ)x2 :=

∞ n=−∞

where

n=∞ n=−∞

µ g2n (x2 )einθ ,

(37)

µ |g2n (x2 )|2 < ∞. Since G2 (θ, 0, µ) is linear and bounded,

g2n : D(A2 ) → C, is also a linear bounded functional. Deﬁne w(θ, ˆ µ, ) by a Fourier series, w(θ, ˆ µ, )(x2 ) =

∞ n=−∞

Substituting w ˆ into (21) yields

w ˆn (µ, )(x2 )einθ .

(38)

110

M. Xiao and W. Kang µ w ˆn (µ, ) ◦ (inω0 + A2 )x2 = −g2n (x2 ).

(39)

According to Hille-Yosida Theorem [21] and Assumption 3, for any complex number with Re(λ) > −b, the operator (λI − A2 )−1 is bounded on X2 . Furthermore, (λI − A2 )−1 L(X2 ) ≤

M . Reλ + b

(40)

In particular, when λ = −inω0 , we have (inω0 + A2 )−1 L(X2 ) ≤

M . b

(41)

Hence for any ξ ∈ X2 , µ |w ˆn (µ, )(ξ)| = |g2n ◦ (inω0 + A2 (µ))−1 ξ| µ ≤ g2n ◦ (inω0 + A2 (µ))−1 L(X2 ) ξX2 M µ ≤ g2n L(X2 ,R) ξX2 . b

(42)

This inequality implies that the Fourier series (38) converges. It represents the solution of (21). 2

3 Feedback Control of Hopf Bifurcations Recall that we consider the system: dx = A(µ)x + fµ (x, u), dt where fµ : D(A(µ)) × U → X is a smooth function in both variables x and u. Without loss of generality we assume fµ (x, u) = fµ (x) + gµ (x, u)u

(43)

with fµ (0) = 0, and dfµ (0) = 0. In this section, we focus on how to change the value of the stability indicator κ using state feedbacks. If u = 0, the indicator is denoted by κ. With a feedback u = u(x), the indicator of the closed-loop system is denoted by κ ¯. Theorem 2. Consider the system (43). Let x1 = (x11 , x12 ). Suppose at µ = 0 T T 00 P gµ (0, 0) = a b =

(44)

Control of Hopf Bifurcations for Inﬁnite-Dimensional Nonlinear Systems

111

where P represents the projection mapping from X = X1 ⊕ X2 to X2 . Then the following feedback control u(x1 ) = q1 x311 + q2 x312 .

(45)

is able to change the stability of the periodic solutions of the Hopf bifurcation. Furthermore, the relationship between the control law (45) and the stability indicator κ is deﬁned by 3 κ ¯ = κ + (aq1 + bq2 ), 8 where κ and κ ¯ are the stability indicators for the original system with u = 0 and the closed-loop system with the feedback (45), respectively. Proof. At µ = 0, denote P gµ (0, 0) =

* + a . b

With the cubic feedback control (44), we have * + a P gµ (x1 , u(x1 ))u(x1 ) = (q1 x311 + q2 x312 ) + O(x1 )4 . b

(46)

(47)

From equation (15) a direct calculation shows C¯4 (θ, 0, 0) = C4 (θ, 0, 0) + (a cos θ + b sin θ)(q1 cos3 θ + q2 sin3 θ) C¯3 (θ, 0, 0) = C3 (θ, 0, 0) ¯ 3 (θ, 0, 0) = D3 (θ, 0, 0), D where C3 , C4 , and D3 are given by (15). Thus we have 6 2π 1 κ ¯ 1 = κ1 + (a cos θ + b sin θ)(q1 cos3 θ + q2 sin3 θ)dθ 2π 0 3 = κ1 + (aq1 + bq2 ). 8

(48)

(49)

Now we compute κ ¯ 2 . Note that if x2 = 0 we have ˜ 1 )x2 (I − P )gµ (x, u(x1 ))u(x1 ) = (I − P )gµ (x1 , u(x1 ))u(x1 ) = J(x 1

(50)

where J˜ is a function of x1 . Because u(x1 ) is cubic in x1 , we know that ˜ ¯ 0) = J(0, 0). Moreover, the feedback control J(0) = 0 . Hence we have J(0, (44) does not change G2 . Thus κ ¯ 2 = κ2 . Therefore we arrive at 3 κ ¯ = κ1 + κ2 + (aq1 + bq2 ). 8

(51)

Since a and b are not zero simultaneously, the sign of κ can be determined by 2 suitable choices of q1 and q2 .

112

M. Xiao and W. Kang

Remark 3. Assumption (44) implies that the pair (A1 (0), P g0 (0, 0)) is controllable. In Theorem 3, we will address the case in which the critical eigenvalues are not controllable. Remark 4. Let H3 be the linear vector space consisting of 3th order homogeneous polynomials of x1 . Each p ∈ H3 represents a cubic feedback. The closed-loop system admits a normal form (29). Deﬁne F, a mapping from H3 to , as follows F(p) = κ ¯. Then Theorem 2 implies that F is surjective, provided (44) holds. Therefore, one can achieve any given value of κ through a cubic feedback. Note that the cubic feedback control (45) cannot alter the value κ2 , which is determined by higher order modes. Instead, the feedback changes the value of κ by changing the value of κ1 . If the critical eigenvalues are not controllable, then the feedback must be quadratic in order to control the stability of the Hopf bifurcation. Corollary 1. Consider the system (43). Suppose that * + 0 P g0 (0, 0) = . 0

(52)

Suppose that a feedback u(x1 ) contains cubic and higher degree terms only. Then, in a neighborhood of the origin, the feedback does not change the stability of the periodic solutions of (43). Proof. The assumption (52) implies that C3 , D3 , C4 cannot be changed by the feedback u(x1 ) of cubic and higher degrees. Thus the feedback leaves κ1 invariant. From the proof of Theorem 2, we know that the feedback control ˆ Thus κ2 is not changed u(x1 ) does not change the functions G2 , J and w. either. According to the averaged normal form (29), the value of κ determines the stability of the periodic solutions generated by the Hopf bifurcation. Therefore, the stability of (43) is invariant under the nonlinear feedback u(x1 ) of cubic and higher degrees. 2 In order to control the stability of the Hopf bifurcation with uncontrollable critical modes, we must use quadratic feedback because the cubic and higher degree terms in a feedback have no inﬂuence on the stability. Let x1 = (x11 , x12 ). Recall the general form of a quadratic feedback is u(x1 ) = p11 x211 + p12 x11 x12 + p21 x12 x11 + p22 x222 := [p11 , p12 , p21 , p22 ]x21 . According to the discussion in section 2, the above quadratic feedback control changes the values of both κ1 and κ2 . However, under certain assumptions

Control of Hopf Bifurcations for Inﬁnite-Dimensional Nonlinear Systems

113

given in Theorem 3, the control changes κ2 , but not κ1 . In this case, an explicit formula is derived to compute the coeﬃcients in a nonlinear feedback that achieves a desired value of κ. Let (I − P )g0 (0, 0) := K where gµ is deﬁned in (43) and K : U → X2 is a bounded operator. Theorem 3. Consider the system (43). Suppose P gµ (x1 + x2 , u)u = β2 (x2 , u)x21 + β3 (x2 , u)x31 + · · · .

(53)

1. Under the feedback (9), the value of κ ¯ for the closed-loop system satisﬁes the following equations. κ ¯ = κ1 + κ ¯2 ,

(54)

and 1 κ ¯ 2 = κ2 + 2π

6

2π

w(θ, ˆ 0, 0)K[p11 , p12 , p21 , p22 ](cos θ, sin θ)2 dθ, (55)

0

where pij ∈ R, i, j = 1, 2. 2. In the case of either G2 (·, 0, 0) = 0 or K = 0, a nonlinear feedback control of quadratic and higher degrees is not able to change the stability of the Hopf bifurcation. Proof. 1. The condition (53) implies that a quadratic feedback does not change the terms C3 , C4 , and D3 deﬁned by (15). Thus κ1 deﬁned by (26) is invariant under the feedback (9). On the other hand, the feedback (9) changes the coeﬃcient J in (12). The new coeﬃcient J¯ has the following expression, ¯ 1 , µ) = J(x1 , µ)x2 + K[p11 , p12 , p21 , p22 ]x2 . J(x 1 1 Meanwhile, the feedback (9) is not able to change G2 . Therefore, the solution of (21) is not changed. Now conclusions (54) and (55) follow from equation (27). 2. Condition (53) implies that the feedback (9) cannot change G2 . Suppose G2 (·, 0, 0) = 0. From (21), w(θ, ˆ 0, 0) = 0 in the closed-loop system. Therefore, κ ¯ 2 = κ2 under the feedback (9). If K = 0, then obviously k¯2 = k2 from (55). 2 We next consider a general case for the quadratic feedback control. Recall that we assume fµ (x, u) = fµ (x) + gµ (x, u)u and

* + a P g0 (0, 0) := , b

(I − P )g0 (0, 0) := K.

(56)

114

M. Xiao and W. Kang

Notice that gµ is smooth and u is quadratic in x1 = (x11 , x12 ), thus there exists a symmetric 4-linear operator β(µ) : X14 → X1 and a 2 × 2 constant matrix a = (aij ) such that +* + * + * x11 a a11 a12 P gµ (x1 , u(x1 ))u(x1 ) = + [p11 , p12 , p21 , p22 ]x21 b a21 a22 x12 +β(µ)x41 ,(57) where u(x1 ) is a feedback in the form of (9). Theorem 4. Given a feedback (9). Then we have κ ¯ 1 = κ1 +

1 2π

6

2π

f (a, θ)f (p, θ) −

1 C3 (θ, 0, 0)(−a sin θ + b cos θ)f (p, θ) ω0

0 +D3 (θ, 0, 0)(a cos θ + b sin θ)f (p, θ) + (a cos θ + b sin θ)(−a sin θ + b cos θ)f 2 (p, θ) dθ

and 1 κ ¯ 2 = κ2 + 2π

6

2π

w(θ, ˆ 0, 0)Kf (p, θ)dθ 0

where f (p, θ) := [p11 , p12 , p21 , p22 ](cos θ, sin θ)2 f (a, θ) := [a11 , a12 , a21 , a22 ](cos θ, sin θ)2 and C3 , D3 are given in (15). Proof. Let us put x1 into the following format: x˙ 1 = P gµ (x, u)u + B0 (x2 , µ) + B1 (x2 , µ)x1 + B2 (x2 , µ)x21 + . . . .

(58)

We use the same notations as before for C3 , C4 , D3 , G2 , J when u ≡ 0. The ¯ 3, G ¯ 2 , J¯ of the closed-loop system under the feedback satisfy the new C¯3 , C¯4 , D following equations, C¯3 (θ, 0, 0) = C3 (θ, 0, 0) + [a cos θ + b sin θ] f (p, θ) ¯ 3 (θ, 0, 0) = D3 (θ, 0, 0) − [a sin θ − b cos θ] f (p, θ) D C¯4 (θ, 0, 0) = C4 (θ, 0, 0) + f (a, θ)f (p, θ) ¯ 2 (θ, 0, 0) = G2 (θ, 0, 0) G ¯ 0) = J(0, 0) + K[p11 , p12 , p21 , p22 ]. J(0, Now the proof is straightforward using the formulae for κ1 and κ2 .

2

Control of Hopf Bifurcations for Inﬁnite-Dimensional Nonlinear Systems

115

4 Summary We now summarize the main results of this paper. Consider the system deﬁned by (1) with (43). Let the feedback u = u(x1 ) be a function of x1 . 1. Cubic feedbacks are able to change κ1 , but not κ2 . The necessary and suﬃcient condition for the stabilization under a cubic feedback is * + 0 P g0 (0, 0) = . (59) 0 2. If the system satisﬁes (53), then only a quadratic feedback is able to change the stability of the system. In this case, the control changes the value of κ2 only; 3. In general, quadratic feedbacks could make changes to both κ1 and κ2 ; 4. Any nonlinear feedback other than the quadratic and cubic feedback does not change the stability of the Hopf bifurcation of (43).

References 1. Abed E., Fu J. (1986) Local feedback stabilization and bifurcation control, I. Hopf bifurcation. Systems & Control Letters, 7, pp. 11–17. 2. Abed E., Fu J. (1987) Local feedback stabilization and bifurcation control, II. Stationary bifurcation. Systems & Control Letters, 8, pp. 467–473. 3. Aronson D. G. (1977) The asymptotic speed of propagation of a simple epidemic. NSF-CBMS Regional Conf. Nonlinear Diﬀusion Equations, Univ. Houston, Houston, Tex., 1976, pp. 1–23. Res. Notes Math., No. 14, Pitman, London. 4. Carr J. (1981) Applications of Centre Manifold Theory. Springer-Verlag. 5. Chow S., Mallet-Paret J. (1977) Integral averaging and bifurcation. Journal of Diﬀerential Equations, 26, pp. 112–159. 6. Colonius F., Kliemann W. (1995) Controllability and stabilization of one dimensional systems near bifurcation points. Systems and Control Letters, 24, 87–95. 7. Crandall M., Rabinowitz P. (1977) The Hopf bifurcation theorem in inﬁnite dimensions. Arch. Rat. Mech. Anal., 67, pp. 53–72. 8. Crandall M., Rabinowitz P. (1980) Mathematical theory of bifurcation. In: Bifurcation Phenomena in Mathematical Physics and Related Topics, edited by C. Bardos and D. Bessis. Reidel: Dordrecht. 9. Iooss G., Joseph D. D. (1980) Elementary Stability and Bifurcation Theory. Springer, New York. 10. Gu G., Chen X., Sparks A., Banda S. (1999) Bifurcation stabilization with local output feedback linearization of nonlinear systems. SIAM J. Control and Optimization, 30, 934–956. 11. Hamzi B., Kang W., Barbot J.-P., Analysis and Control of Hopf Bifurcations, SIAM J. on Control and Optimization, to appear. 12. Kang W. (1998) Bifurcation and Normal Form of Nonlinear Control Systems, Part I. SIAM J. Control and Optimization, 1(36), pp. 193–212.

116

M. Xiao and W. Kang

13. Kang W. (1998) Bifurcation and Normal Form of Nonlinear Control Systems, Part II. SIAM J. Control and Optimization, 1(36), pp. 213–232. 14. Kang W. (2000) Bifurcation Control via State Feedback for Systems with a Single Uncontrollable Mode. SIAM J. Control and Optimization, 38, 1428– 1452. 15. Kato T. (1995) Perturbation Theory for Linear Operators. Berlin : Springer. 16. Kuznetsov Y. A. (1998) Elements of Applied Bifurcation Theory. Springer, New York. 17. Landau L., Lifshitz E. (1959) Fluid Mechanics. Pergamon, Oxford. 18. Marsden J. E., McCracken M. F. (1976) The Hopf Bifurcation and Its Applications. Springer-Verlag. 19. McCaughan F. (1990) Bifurcation analysis of axial ﬂow compressor stability. SIAM J. of Appl. Math. 50, 1232–1253. 20. Liaw D. C., Abed E. H. (1996) Active control of compressor stall inception: a bifurcation theoretic approach. Automatica, Vol. 32, 109–115. 21. Pazy A. (1983) Semigroup of Linear Operators and Applications to Partial Diﬀerential Equations. Springer-Verlag. 22. Xiao M., Ba¸sar T. (2001) Center manifold of the viscous Moore-Greitzer PDE model. SIAM Journal of Applied Mathematics, Vol. 61, No. 3, pp. 855–869. 23. Wang Y., Murray R. M. (1999) Feedback stabilization of steady-state feedback and Hopf bifurcations. Proc. of the 38th IEEE CDC, pp. 2431–2437. 24. Zaslavsky B. (1996) Feedback stabilization of connected nonlinear oscillators with uncontrollable linearization. Systems Control Lett., 27, no. 3, 181–185.

On the Steady-State Behavior of Forced Nonlinear Systems C.I. Byrnes1 , D.S. Gilliam2 , A. Isidori3 , and J. Ramsey4 1 2 3 4

Department of Systems Science and Mathematics, Washington University, St. Louis, MO 63130, [email protected] Department of Mathematics and Statistics, Texas Tech University, Lubbock, TX 79409, [email protected] Dipartimento di Informatica e Sistemistica, Universit` a di Roma “La Sapienza”, 00184 Rome, ITALY, [email protected] Boeing, [email protected]

This paper is dedicated to Art Krener – a great researcher, a great teacher and a great friend

1 Introduction The purpose of this paper is to discuss certain aspects of the asymptotic behavior of ﬁnite-dimensional nonlinear dynamical systems modeled by equations of the form x˙ = f (x, w)

(1)

in which x ∈ Rn and w ∈ Rr is an input generated by some ﬁxed autonomous system w˙ = s(w) .

(2)

The initial conditions x(0) and w(0) of (1) and (2) are allowed to range over some ﬁxed sets X and W . There are several reasons why the analysis of the asymptotic behavior of systems of this kind is important. Indeed, every periodic function is an output of a system (2), so that the study of the asymptotic behavior of (1) – (2) includes the classical problem of determining existence (and possibly

Research supported in part by AFOSR, The Boeing Corporation, Institut MittagLeﬄer, and ONR

W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 119–143, 2003.

c Springer-Verlag Berlin Heidelberg 2003

120

C.I. Byrnes et al.

uniqueness) of forced oscillations in a nonlinear system. On the other hand, in control theory, an analysis of this kind arises when a system of the form x˙ = f (x, w, u)

(3)

is given, with w still generated by a system of the form (2), and a feedback control law u = u(x, w) is sought to the purpose of steering to 0 a prescribed “regulated output” e = h(x, w) ,

(4)

while keeping all trajectories bounded. In this case, in particular, the interest is in the analysis and design of a system of the form (1) – (2) in which trajectories are bounded and asymptotically approach the set K = {(x, w) : h(x, w) = 0} .

(5)

A problem of this kind is commonly known as a problem of output regulation or as the generalized servomechanism problem ([2, 6, 7, 8, 13]).

2 Notations and Basic Concepts Consider an autonomous ordinary diﬀerential equation x˙ = f (x)

(6)

with x ∈ Rn , t ∈ R, and let φ : (t, x) → φ(t, x) deﬁne its ﬂow [10]. Suppose the ﬂow is forward complete. The ω-limit set of a subset B ⊂ Rn , written ω(B), is the totality of all points x ∈ Rn for which there exists a sequence of pairs (xk , tk ), with xk ∈ B and tk → ∞ as k → ∞, such that lim φ(tk , xk ) = x .

k→∞

In case B = {x0 } the set thus deﬁned, ω(x0 ), is precisely the ω-limit set, as deﬁned by G.D.Birkhoﬀ, of the point x0 . With a given set B, is it is also convenient to associate the set ψ(B) = ω(x0 ) x0 ∈B

i.e. the union of the ω-limits set of all points of B. Clearly, by deﬁnition ψ(B) ⊂ ω(B) , but the equality may not hold.

On the Steady-State Behavior of Forced Nonlinear Systems

121

G.D.Birkhoﬀ has shown that, if φ(t, x0 ) is bounded in positive time, the set ω(x0 ) is non-empty, compact, invariant, and lim dist(φ(t, x0 ), ω(x0 )) = 0 .

t→∞

More generally, recall that a set A is said to uniformly attract1 a set B under the ﬂow of (6) if for every ε > 0 there exists a time t¯ such that dist(φ(t, x), A) ≤ ε,

for all t ≥ t¯ and for all x ∈ B.

With the above deﬁnitions we immediately obtain the following lemma. Lemma 1. If B is a nonempty bounded set for which there is a compact set J which uniformly attracts B (thus, in particular, if B is any nonempty bounded set whose positive orbit has a bounded closure), then ω(B) is nonempty, compact, invariant and uniformly attracts B.

3 The Steady-State Behavior One of the main concerns, if not the main concern, in the analysis and design of control systems, is the ability of inﬂuencing or shaping the response of a given system to assigned external inputs. This can be achieved either by open-loop or by closed-loop control, the latter being almost always the solution of choice in the presence of uncertainties, aﬀecting the control systems itself as well as the external inputs to which the response has to be shaped ([1]-[4]). Among various possible criteria by means of which responses can be analyzed and classiﬁed, a classical viewpoint, dating back to the origins of control theory, is that based on the separation between “steady-state” and “transient” responses; the former being viewed as the unique response (if any) to which any actual response conforms to as time increases, while the latter is deﬁned as the diﬀerence between actual response and steady-state one. There are several well-known strong arguments in support of the important role played by the idea of a steady-state response in system analysis and design. On one hand, in a large number of cases it is actually required, as a design speciﬁcation, that the response of a system asymptotically converge to a prescribed function of time. This is for instance the case in the so-called “set-point control” problem (which includes the problem of asymptotic stabilization to an equilibrium as a special case), where the response of a controlled system is required to asymptotically converge to a ﬁxed (but otherwise arbitrary or unpredictable) 1

Note that, in [10], the property which follows is simply written as lim dist(φ(t, B), A) = 0, with the understanding that t→∞

dist(B, A) := sup dist(x, A) = sup inf dist(x, y) . x∈B

x∈B y∈A

122

C.I. Byrnes et al.

value, and it is also for instance the case when the response of a system is required to asymptotically track (or reject) a prescribed periodically varying trajectory (or disturbance). On the other hand, as is well-known in linear system theory, the ability to analyze and shape the steady-state response to sinusoidally-varying inputs also provides a powerful tool for the analysis and, to a some extent, for the design of the transient behavior. Traditionally, the idea of a separation between steady-state and transient response stems from the observation that, in any ﬁnite-dimensional timeinvariant linear system, (i) the forced response to an input which is a polynomial or exponential function of time normally includes a term which is a polynomial (of degree not exceeding that of the forcing input) or an exponential function (with an exponent whose rate of change is the same as that of the forcing input) of time, and (ii) if the unforced system itself is asymptotically stable, this term is the unique function of time to which the actual response converges as the initial time tends to −∞ (regardless of what the state of the system at the initial time is). In particular, a fundamental property on which a good part of classical network analysis is based is the fact that, in any ﬁnitedimensional time-invariant asymptotically stable (single-input) linear system x˙ = Ax + bu forced by the harmonic input u(t) = u0 cos(ωt), there is a unique initial condition x0 which generates a periodic trajectory of period T = 2π/ω, and this trajectory is the unique trajectory to which any other trajectory converges as the initial time t0 tends to −∞. As a matter of fact, using the variation of parameters formula, it can be immediately checked that the integral formula x0 = (I − eAT )−1

6

T

eA(T −t) b cos(ωt)dt u0

0

provides the unique initial condition x0 from which a forced periodic trajectory, of period T = 2π/ω, is generated. There are various ways in which this elementary result can be extended to more general situations. For example, if a nonlinear system x˙ = f (x, u)

(7)

has a locally exponentially stable equilibrium at (x, u) = (0, 0), i.e. if f (0, 0) = 0, then existence, uniqueness and asymptotic stability of a periodic response forced by the harmonic input u(t) = u0 cos(ωt), for small |u0 |, can be determined via center manifold theory, as explained in more detail in Sect. 4. In particular, it can be proven that, under these hypotheses, for small |u0 | and small x(t0 ) the forced response of the system always converges, as t0 → −∞, to a periodic response generated from a uniquely determined initial state x0 . Even though we have motivated the interest in the notion of steady state response in the context of problems of analysis and design for control systems,

On the Steady-State Behavior of Forced Nonlinear Systems

123

it should be observed here that the principle inspiring this notion, at least in the case of sinusoidally varying or periodic inputs, is the same principle which is behind the investigation of forced oscillations in nonlinear systems, a classical problem with its origin in celestial mechanics. In this respect, however, it must be stressed that for a nonlinear system such as (7), forced by the harmonic input u(t) = u0 cos(ωt), the situation is far more complex than those outlined above, with the possibility of one, or several, forced oscillations with varying stability characteristics occurring. In addition, the fundamental harmonic of these periodic responses may agree with the frequency of the forcing term (harmonic oscillations), or with integer multiples or divisors of the forcing frequency (higher harmonic, or subharmonic, oscillations). Despite a vast literature on nonlinear oscillations, only for second order systems is there much known about existence and stability of forced oscillation and, in particular, which of these kinds of periodic responses might be asymptotically stable. In the above, the steady-state responses of time-invariant systems were intuitively viewed as the limits of the actual responses as the initial time t0 tends to −∞. This intuitive concept appears to be conveniently captured in the notion of ω-limit set of a set, used in the theory of dissipative dynamical systems by J.K.Hale and other authors and summarized in Sect. 2. More speciﬁcally, consider again the composite system (1) – (2), namely the system x˙ = f (x, w) w˙ = s(w) ,

(8)

which will be seen as a cascade connection of a driven system (1) and a driving system (2). Suppose that the forward orbit of a bounded set X × W of initial conditions has a bounded closure. Then (see Lemma 1) the set SSL = ω(X × W ) is a well-deﬁned nonempty compact invariant set, which uniformly attracts X × W . It is very natural to consider as “steady-state behavior” of system (8), or – what is the same – as “steady-state behavior” of system (1) under the family of inputs generated by (2), the behavior of the restriction of (8) to the invariant set SSL. The set in question will be henceforth referred to as the steady state locus of (8) and the restriction of (8) to the invariant set SSL as the steady-state behavior of (8). In the sequel, we will provide a number of examples illustrating the concept steady state locus in various diﬀerent situations and to discuss some of its properties.

124

C.I. Byrnes et al.

4 Some Examples 4.1 Finite-Dimensional Linear Systems Consider a linear time-invariant system x˙ = Ax + Bu y = Cx + Du

(9)

with state x ∈ Rn , input u ∈ Rm , output y ∈ Rp , forced by the input u(t) = u0 cos(ωt)

(10)

in which u0 a ﬁxed vector and ω is a ﬁxed frequency. Writing u(t) = P w(t), with P = u0 0 and w(t) solution of 0 ω w˙ = Sw = w (11) −ω 0 with initial condition w(0) = (1 0)T , the forced response x(t) of (9), from any initial state x(0) = x0 , to the input (10) is identical to the response x(t) of the (augmented) autonomous system x˙ = Ax + BP w w˙ = Sw from the initial condition x(0) = x0 ,

(12)

1 w(0) = . 0

To compute the response in question, various elementary methods are available. In what follows, we choose a geometric viewpoint, which is more suited to the analysis of broader classes of examples, presented in the next sections. Assume that all the eigenvalues of the matrix A have negative real part. Since S has purely imaginary eigenvalues, Cn+2 can be decomposed into the direct sum of two subspaces, invariant for (12), In×n Π VA = , VS = , 02×n I2×2 in which Π is the unique solution of the Sylvester’s equation AΠ + BP = ΠS. By construction x ˜ = x − Πw satisﬁes x ˜˙ = A˜ x and therefore ˜(t) = lim [x(t) − Πw(t)] = 0. lim x

t→∞

t→∞

(13)

On the Steady-State Behavior of Forced Nonlinear Systems

125

On the other hand, since the subspace VS is invariant for (12), if x0 = Πw0 the integral curve of x(t), w(t) of (12) passing through (x0 , w0 ) at time t = 0 is such that x(t) = Πw(t) for all t. This curve is therefore a closed curve and x(t), which is given by cos(ωt) sin(ωt) x(t) = Xw(t) = Π w(0) , − sin(ωt) cos(ωt) is a periodic solution of period T = 2π/ω. We can in this way conclude that, for any compact set of the form W = {w ∈ R2 : w ≤ r} and any compact set X ⊂ Rn , the steady-state locus of (12) is the set SSL = {(x, w) ∈ Rn × R2 : x = Πw, w ≤ r} . Note that Π can easily be computed in the following way. Rewrite (13) in the form 0 ω Π = AΠ + Bu0 1 0 . −ω 0 Split Π as Π = Π1 Π2 and multiply both sides on the right by the vector (1 i)T to obtain Π1 + iΠ2 = (iωI − A)−1 Bu0 . 4.2 Finite-Dimensional Bilinear Systems We consider now the problem of determining the steady state response, to the forcing input (10), of an arbitrary single-input single-output ﬁnite-dimensional nonlinear system having an input-output map characterized by a Volterra series consisting only of a ﬁnite number of terms. To this end, it suﬃces to show how the response can be determined in the special case of a Volterra series consisting of one term only, that is the case in which this map is convolution integral of the form 6 t 6 τ1 6 τk−1 y(t) = ··· w(t, τ1 , . . . , τk )u(τ1 ) . . . u(τk )dτ1 . . . dτk . (14) 0

0

0

Since our method of determining the steady-state behavior is based on the use of state space models, we ﬁrst recall an important result about the existence of ﬁnite dimensional realizations for an input-output map of the form (14). Proposition 1. The following are equivalent (i) the input-output map (14) has a ﬁnite dimensional nonlinear realization, (ii) the input-output map (14) has a ﬁnite dimensional bilinear realization, (iii) there exist matrices A1 , A2 , . . . Ak , N12 , . . . , Nk−1,k , C1 and Bk such that

126

C.I. Byrnes et al.

w(t, τ1 , . . . , τk ) = C1 eA1 (t−τ1 ) N12 eA2 (τ1 −τ2 ) N23 · · · Nk−1,k eAk (τk−1 −τk ) Bk . (15) In particular, from the matrices indicated in condition (iii) it is possible to construct a bilinear realization of the map (14), which has the form x˙ 1 = A1 x1 + N12 x2 u x˙ 2 = A2 x2 + N23 x3 u .. .

(16)

x˙ k−1 = Ak−1 xk−1 + Nk−1,k xk u x˙ k = Ak xk + Bk u y = C1 x1 . The realization in question is possibly non-minimal, but this is not an issue so far as the calculation of the steady-state response is concerned. For convenience, set     A1 x1 + N12 x2 u x1    x2  A2 x2 + N23 x3 u        ..  . .. F (x, u) =  (17) x =  . ,      Ak−1 xk−1 + Nk−1,k xk u xk−1  xk Ak xk + Bk u and H(x) = C1 x1 , with x ∈ Rn , which makes it possible to rewrite system (16) can be in the form x˙ = F (x, u) y = H(x) . Viewing the input (10) as u(t) = P w(t), with w(t) generated by an exosystem of the form (11), we determine in what follows the structure of the steady state locus of the composite system x˙ = F (x, P w) w˙ = Sw ,

(18)

for initial conditions ranging on a set X×W . To this end, we need the following preliminary result (see [13]). Lemma 2. Let A be an n × n matrix having all eigenvalues with nonzero real part and let S be as in (11). Let P denote the set of all homogeneous polynomials of degree p in w1 , w2 , with coeﬃcients in R. For any q(w) ∈ P n , the equation ∂π(w) Sw = Aπ(w) + q(w) ∂w has a unique solution π(w), which is an element of P n .

(19)

On the Steady-State Behavior of Forced Nonlinear Systems

127

Using this property it is possible to prove the following result (see [13]). Proposition 2. Let F (x, u) be as in (17) and S as in (11). Assume that all matrices A1 , A2 , . . . Ak have eigenvalues with negative real part. Then the equation ∂π(w) Sw = F (π(w), P w), ∂w

π(0) = 0

(20)

has a globally deﬁned solution π(w), whose entries are polynomials, in w1 , w2 of degree not exceeding k. By construction, the set {(x, w) : x = π(w)} where π(w) is the solution of (20), is a globally deﬁned invariant set for the system (18). Therefore, if x0 = π(w0 ) the integral curve x(t), w(t) of (18) passing through (x0 , w0 ) at time t = 0 is such that x(t) = π(w(t)) for all t. This curve is then a closed curve and x(t) is a periodic solution of period T = 2π/ω. Moreover it easy to prove the this set is globally attractive and, in particular, that for any pair (x0 , w0 ), the solution x(t) of (18) passing through (x0 , w0 ) at time t = 0 satisﬁes lim [x(t) − π(w(t))] = 0 .

t→∞

(21)

We can in this way conclude that, for any compact set of the form W = {w ∈ R2 : w ≤ r} and any compact set X ⊂ Rn , the steady-state locus of (18) is the set SSL = {(x, w) ∈ Rn × R2 : x = π(w), w ≤ r}} , 4.3 Finite Dimensional Non-linear Systems Consider now a nonlinear system modeled by equations of the form x˙ = f (x, u)

(22)

with state x ∈ Rn and input u ∈ Rm , in which f (x, u) is a C k function, k ≥ 2, of its arguments with f (0, 0) = 0. Let the input function u(t) be as in (10). Therefore, any integral curve of (22) can be seen as the x-component of an integral curve of the autonomous system x˙ = f (x, P w) w˙ = Sw .

(23)

Suppose that the equilibrium x = 0 of x˙ = f (x, 0) is locally exponentially stable. If this is the case, it is well known that for any ε > 0 there exist numbers δ1 > 0 and δ2 > 0 such that, for any x0 , w0 ∈ {x : x ≤ δ1 } × {w : w ≤ δ2 }

128

C.I. Byrnes et al.

the solution x(t), w(t) of (23) satisfying x(0), w(0) = x0 , w0 satisﬁes x(t) ≤ ε,

w(t) ≤ δ2

and therefore the equilibrium (x, w) of (23) is stable in the sense of Lyapunov [9]. It is also known that system (23) has two complementary invariant manifolds through the equilibrium point (x, w) = (0, 0): a stable manifold and a (locally deﬁned) center manifold. The stable manifold is the set of all points (x, 0) such that x belongs to the basin of attraction of the equilibrium x = 0 of x˙ = f (x, 0). The center manifold, on the other hand, can be expressed as the graph of a C k−1 mapping x = π(w) deﬁned on some neighborhood of w = 0, for instance a ball of small radius r centered at w = 0. This mapping by deﬁnition satisﬁes ∂π Sw = f (π(w), P w) ∂w

(24)

and π(0) = 0. Let x(t), w(t) be the integral curve of (23) passing through (x0 , w0 ) at time t = 0. Since the equilibrium (x, w) = (0, 0) of (23) is stable in the sense of Lyapunov and the center manifold in question is locally exponentially attractive, it can be concluded, for r as above, that there exists positive numbers δ, α, λ such that w0 ≤ r x0 − π(w0 ) ≤ δ

⇒

x(t) − π(w(t)) ≤ αe−λt x0 − π(w0 )

for all t ≥ 0.

In particular, if x0 = π(w0 ), the integral curve of (23) is a closed curve and x(t) is a periodic solution, of period 2π/ω, of x˙ = f (x, u0 cos(ωt)) . We can in this way conclude (as in [13]) that, for any compact set of the form W = {w ∈ R2 : w ≤ r} and any compact set of the form X = {x ∈ Rn : x − π(w) ≤ δ}, the steady-state locus of (23) is the set SSL = {(x, w) ∈ Rn × R2 : x = π(w), w ≤ r} .

5 On the Structure of the Steady-State Locus In Section 3, for a system of the form (8), with initial conditions in a set X × W the forward orbit of which was assumed to be bounded, we have suggested to deﬁne the steady-state behavior as the restriction of the system in question to the invariant set ω(X × W ). In this section, we analyze some properties of this set. This will be done under the additional assumption that

On the Steady-State Behavior of Forced Nonlinear Systems

129

the set W of admissible initial conditions w(0) for (2) is a compact invariant subset of Rr and that W = ψ(W ) , i.e. that any point of W is in the ω-limit set of some (possibly diﬀerent) point of W . This assumption will be henceforth referred to as property of Poisson stability. In the present context, the assumption that ψ(W ) = W is quite reasonable. This assumption reﬂects, to some extent, the interest in restricting the class of forcing inputs for (1) to inputs which possess some form of persistency in time. These are in fact the only inputs which seems reasonable to consider in the analysis of a steady-state behavior. As a matter of fact, if it is assumed, without much loss of generality, that the set W of admissible initial conditions for (2) is a closed invariant set, it follows that ψ(W ) ⊂ W . To say that ψ(W ) is exactly equal to W is simply to say that no initial condition for (2) is trivial from the point of view of the steady-state behavior, because this initial condition is assumed to be a point in the ω-limit set of some trajectory. In particular, it is immediate to check that this assumption is fulﬁlled when any point in W is a recurrent point, i.e. whenever each point in W belongs to its own ω-limit set, as occurs when the exosystem is the classical harmonic oscillator. Lemma 3. Suppose ψ(W ) = W . Then, for every w ¯ ∈ W there is an x ¯∈X such that (¯ x, w) ¯ ∈ ω(X × W ). Proof. To prove the Lemma, let φ(t, x, w) :=

φx (t, x, w) φw (t, w)

denote the ﬂow of (8). Pick w ¯ ∈ W . By hypothesis there exists w ∈ W and a sequence of times tk , with tk → ∞ as k → ∞, such that ¯ lim φw (tk , w) = w

k→∞

For any x ∈ X, consider now the sequence {φx (tk , x, w)}. Since φx (tk , x, w) is bounded by assumption, there exists a subsequence θk , with tk → ∞ as k → ∞, such that the sequence φx (θk , x, w) converges to some x ¯. By deﬁnition (¯ x, w) ¯ is a point in ω(X × W ). $ Thus, if the exosystem is Poisson stable, the steady-state locus is the graph of a (possibly set-valued ) map, deﬁned on the whole set W . Since the notion of steady-state locus has been introduced in order to formally deﬁne the steadystate response of a nonlinear system to families of forcing inputs, such as those generated by the exosystem (2), there is an obvious interest in considering the special case in which the steady-state locus is the graph of a single-valued map

130

C.I. Byrnes et al.

π : W → Rn w → π(w) . In this case, in fact, each forcing input produces one and only one steady-state response in (1). More precisely, for every w0 ∈ W , there is one and only one x0 in Rn , namely x0 = π(w0 ), with the property that the response of (8) from the initial condition (x0 , w0 ) remains in the steady-state locus for all times. In this case, we will say that the response x(t) = π(w(t)) = π(φw (t, w0 )) is the steady-state response of (1) to the input (2). In the examples of Sect. 4, the steady-state locus is the graph of a map, and a (unique) steady-state response can be deﬁned. On the other hand, in many of the examples in Sect. 6 which follows, multiple steady-state behaviors are possible, the convergence of the actual response to a speciﬁc one being inﬂuenced by the initial condition of the driven system.

6 More Examples Example 1. Consider the system x˙ = −(3w2 + 3wx + x2 )x + y y˙ = ax − y

(25)

in which a > 0 is a ﬁxed number and w a constant input generated by the exosystem w˙ = 0 .

(26)

For any ﬁxed w, all trajectories of system (25) are ultimately bounded. In fact, consider the positive deﬁnite function V (x, y) = for which

y2 x2 + , 2 2

−(3w2 + 3wx + x2 ) 1 x ˙ V (x, y) = x y , a −1 y

is negative (for nonzero (x, y)) if 3w2 + 3wx + x2 > a .

(27)

If 3w2 > 4a, (27) holds for all x and therefore the equilibrium (x, y) = (0, 0) is globally asymptotically stable. If 3w2 ≤ 4a, system (25) has two additional equilibria, namely the two points (xw− , axw− ), (xw+ , axw+ ) in which xw− and xw+ are the two real roots of 3w2 + 3wx + x2 = a. Note, in particular, that if 3ω 2 = a, one of these two

On the Steady-State Behavior of Forced Nonlinear Systems 1

1

1

0.5

0.5

0.5

0

0

0

0.5

0.5

0.5

1

1

1

1.5

1.5

1.5

2 2

1.5

1

0.5

0

0.5

2 2

1

1.5

1

1 w= √ 2 3

0.5

1

1

0.5

0.5

0

0

0.5

0.5

1

1

1.5

1.5

2 2

0

0.5

2 2

1

1.5

1 w= √ 3

1.5

1

0.5

0

0.5

1

2 2

2 w= √ 3

1

0.5

0

131

0.5

1

3 w= √ 2 3

1.5

1

0.5

0

0.5

1

5 w= √ 2 3

Fig. 1. Steady State Locus

equilibria coincides with the equilibrium at (0, 0), while if 3ω 2 = 4a, these two (nonzero) equilibria coincide. If 3w2 ≤ 4a, (27) holds for all (x, w) but those such for which x ∈ [xw− , xw+ ]. Set now Ωc = {(x, y) : V (x, y) ≤ c} and, for any w pick any c > 0 such that {(xw− , axw− )} ∪ {(xw+ , axw+ )} ⊂ int(Ωc ) . By construction, V˙ (x, y) < 0 on the boundary on Ωc and at all points of R2 \ Ωc . Thus, all trajectories, in ﬁnite time, enter the compact set Ωc , which is positively invariant. Moreover, by Bendixson’s criterion, it is possible to deduce that there are no closed orbits entirely contained in Ωc because ∂ ∂ (−(3w2 + 3wx + x2 )x + y) + (ax − ε−1 y) = −3(x + w)2 − 1 < 0 ∂x ∂y at each point of Ωc . From this analysis it is easy to conclude what follows. For any pair of compact sets X = {(x, y) : max{|x|, |y| ≤ r}

W = {w : |w| ≤ r} ,

the positive orbit of X × W is bounded. Moreover, for large r, if 3w2 > 4a, the set

132

C.I. Byrnes et al.

2 y 0 -2 2

2 0

0 -2

w

-2 x

Fig. 2. Steady State Locus

SSLw = ω(X × W ) ∪ (R2 × {w}) , i.e. the intersection of ω(X × W ) with the plane R2 × {w} reduces to just one point, namely the point (0, 0, w). On the other hand, if 3w2 ≤ 4a, the set SSLw is a 1-dimensional manifold with boundary, diﬀeomorphic to a closed interval of R. Diﬀerent shapes of these sets, for various values of w, are shown in Fig. 1 and a collection of these curves are depicted as a surface in Fig. 2. Example 2. Consider now the system x˙ = y

x2 x4 y2 1 + + −w y˙ = x − x3 − y − + 2 4 2 4

(28)

in which w is a constant input generated by the exosystem (26). For any ﬁxed w, this system has three equilibria, at (x, y) = (0, 0) and (x, y) = (±1, 0). We show now that, for any ﬁxed w, all trajectories of system (28) are ultimately bounded. In fact, consider the positive semi-deﬁnite function V (x, y) = −

x2 x4 y2 1 + + + 2 4 2 4

which is zero only at the two equilibria (x, y) = (±1, 0) and such that, for any c > 0, the sets Ωc = {(x, y) : V (x, y) ≤ c} are bounded. Note that V˙ (x, y) = −y 2 (V (x, y) − w) . If w ≤ 0, V˙ (x, y) ≤ 0 for all (x, y) and therefore, by LaSalle’s invariance principle, all trajectories which start in R2 converge to the largest invariant set contained in the locus where y = 0, which only consists of the union of the three equilibria. If w > 0, V˙ (x, y) ≤ 0 for all (x, y) in the set {(x, y : V (x, y) ≥ w}. Thus, again by LaSalle invariance principle, all trajectories which start in the set

On the Steady-State Behavior of Forced Nonlinear Systems

w=−

w=

1 8

1 4

w=

1 8

w=

1 2

133

Fig. 3. Steady State Locus

{(x, y : V (x, y) ≥ w} converge to the largest invariant set contained in the locus where either y = 0 or V (x, y) = w. Since the locus V (x, y) = w, the boundary of Ωw , is itself invariant and the two equilibria (x, y) = (±1, 0) are in Ωw , it is concluded that all trajectories which start in R2 \ Ωw converge either to the boundary of Ωw or to the equilibrium (x, y) = (0, 0). On the other hand, the boundary of Ωw , for 0 < w < 1/4 consists of two disjoint close curves while for 1/4 ≥ w it consists of a single closed curve (a “ﬁgure eight” for w = 1/4). From this analysis it is easy to conclude what follows. For any pair of compact sets X = {(x, y) : max{|x|, |y| ≤ r}

W = {w : |w| ≤ r} ,

the positive orbit of X × W is bounded. Moreover, for large r, if w ≤ 0, the set SSLw = ω(X × W ) ∪ (R2 × {w}) , i.e. the intersection of ω(X × W ) with the plane R2 × {w} is a 1-dimensional manifold with boundary, diﬀeomorphic to a closed interval of R. If 0 < w <

134

C.I. Byrnes et al.

1/4, the set SSLw is the union of a 1-dimensional manifolds diﬀeomorphic to R and of two disjoint 2-dimensional manifold with boundary, each one diﬀeomorphic to a closed disc. If 1/4 ≤ w, the set SSLw is a 2-dimensional manifold with boundary, diﬀeomorphic to a closed disc for 1/4 < w, or to a “ﬁlled ﬁgure eight” for w = 1/4. Diﬀerent shapes of these sets, for various values of w, are shown in Fig. 3. Example 3. Consider the system x˙ = −x3 + u

(29)

driven by an input u = w1 generated by the harmonic exosystem (11), in which for convenience we set ω = 1. To establish boundedness of trajectories, observe that the positive deﬁnite function V (x) = x2 satisﬁes V˙ = −2x4 + 2xw1 ≤ −2|x|(|x|3 − |w1 |) from which it is concluded that system (29) is input-to-state stable (see [11]; in fact, V˙ < 0 whenever |x| > |w1 |1/3 ). Hence, since w1 (t) is always bounded, then also x(t) is always bounded. For any A > 0, trajectories of (29) – (11) satisfying w(0) = A evolve on the cylinder CA = {(x, w) : w = A} . Using standard arguments based on the method of Lyapunov it is easy to see that these trajectories in ﬁnite time enter the compact set KA = {(x, w) : |x| ≤ 2A1/3 , w = A} which is positively invariant. Hence, by the Poincar´e-Bendixson Theorem, the ω-limit sets of all such trajectories consist of either equilibria, or closed orbits, or open orbits whose α- and ω-limit sets are equilibria. Equilibria clearly can exist only if w0 = 0, in which case there is a unique equilibrium at x = 0. Suppose there is a closed orbit in KA . Since w(t) is a periodic function of period 2π, a simple argument based on uniqueness of solutions shows that the existence of a closed orbit in KA implies the existence of a nontrivial periodic solution, of period 2π, of the equation x(t) ˙ = −[x(t)]3 + w1 (t) . Let

φ(t, x, w) :=

φx (t, x, w) φw (t, w)

(30)

On the Steady-State Behavior of Forced Nonlinear Systems

135

denote the ﬂow of (29) – (11). Existence of a periodic orbit of period 2π is equivalent to the existence of x0 satisfying x0 = φx (2π, x0 , w0 ) .

(31)

Bearing in mind the fact that dφx (t, x, w) = −[φx (t, x, w)]3 + w1 dt

(32)

take the derivatives of both sides with respect to x, to obtain ∂φ d ∂φx x (t, x, w) = −3[φx (t, x, w)]2 (t, x, w) . dt ∂x ∂x Integration over the interval [0, 2π] yields ∂φx (2π, x, w) = exp ∂x

*6

2π

+ −3[φx (τ, x, w)]2 dτ

0

because ∂φx (0, x, w) = 1 . ∂x Suppose w0 = 0. Since by hypothesis (x0 , w0 ) produces a nontrivial periodic solution of (30), φx (τ, x0 , w0 ) cannot be identically zero, and we deduce from the previous relation that 0<

∂φx (2π, x0 , w0 ) < 1 . ∂x

Hence, any nontrivial periodic solution of (30) is locally exponentially stable. We conclude from this that (see Sect. 7), for any ﬁxed w0 = 0, there is a unique x0 satisfying (31). This equation implicitly deﬁnes a unique function π : R2 → R . This function is smooth on R2 \ {0}, but just continuous at w = 0, where the implicit function theorem cannot be used, because at this point we have ∂φx (2π, 0, 0) = 1. ∂x

(33)

To show that π(w) in not C 1 at w = 0, take the derivatives of both sides of (32) with respect to w1 , to obtain ∂φ d ∂φx x (t, x, w) = −3[φx (t, x, w)]2 (t, x, w) + 1 . dt ∂w1 ∂w1 Integration over the interval [0, 2π] yields

136

C.I. Byrnes et al.

* 6 2π +2 ∂φx ∂φx (2π, x, w) = exp − 3[φx (τ, x, w) dτ ] (0, x, w) ∂w1 ∂w 1 0 * 6 6 2π

+

exp −

0

2π

τ

+

2

3[φx (σ, x, w)] dσ dτ.

Now, ∂φx (0, x, w) = 0 ∂w1 by deﬁnition of φx (t, x, w). Moreover, φx (σ, 0, 0) = 0. Thus, we deduce from the relation above that ∂φx (2π, 0, 0) = 2π. ∂w1

(34)

Now, observe that, if there was a continuously diﬀerentiable map π ˆ (w) satisfying π ˆ (w) = φx (2π, π ˆ (w), w) the following would hold ∂π ˆ ∂φx ∂φx ∂π ˆ (2π, π ˆ (w), w) = + (2π, π ˆ (w), w). ∂w1 ∂x ∂w1 ∂w1

(35)

Evaluating this at (ˆ π (w), w) = (0, 0) and bearing in mind (33), this would yield ∂φx (2π, 0, 0) = 0 ∂w1 which contradicts (34). We have in this way found a characterization identical to that described in Sect. 4.3, in a case, though, in which the equilibrium x = 0 of the driven system is not locally exponentially stable. For any compact set of the form W = {w ∈ R2 : w ≤ r} and any compact set of X = {x ∈ R : |x| ≤ r}, the steady state locus is the a graph of a map SSL = {(x, w) ∈ R × R2 : w = π(w), w ≤ r} , which is depicted in Fig. 4. Example 4. Consider the system x˙ = x − (x + u)3 + u3

(36)

driven by an input u = w1 generated by the harmonic exosystem (11), in which for convenience we set ω = 1.

On the Steady-State Behavior of Forced Nonlinear Systems

137

Fig. 4. Steady State Locus

System (36) is not input-to-state stable, because the equilibrium at (x, w1 ) = 0 is unstable. Nevertheless, trajectories are ultimately bounded. In fact, consider the candidate Lyapunov function V (x) = x2 for which we obtain, after a simple algebra, V˙ = −2V [x2 + 3xw1 + 3w12 − 1].

(37)

Since |x| > 2

⇒

x2 + 3xw1 + 3w12 − 1 > 0

we conclude that V˙ < 0 so long as V > 4. As in the previous example, for any A > 0, trajectories of (36) – (11) satisfying w(0) = A evolve on the cylinder CA and, as shown by means of standard arguments based on the method of Lyapunov, in ﬁnite time enter the compact set KA = {(x, w) : |x| ≤ 3, w = A} which is positively invariant. Hence, by Poincar´e-Bendixson’s Theorem, the ω-limit sets of all such trajectories consist of either equilibria, or closed orbits, or open orbits whose α- and ω-limit sets are equilibria. Equilibria clearly can exist only if w0 = 0, and these are the three points at which x = 0, −1, +1. If w0 = 0, there are only closed orbits in KA , which will be analyzed in the following way. First of all, we observe that the set x = 0 is an invariant set for the system (36) – (11). This set is ﬁlled with closed orbits, namely those of the exosystem (11). One may wish to determine the local nature of these closed orbits, by

138

C.I. Byrnes et al.

looking at the linear approximation of the system. Since x(t) = 0 on these orbits, all that we have to do is to study the local stability properties of the equilibrium x = 0 of the periodically varying system x˙ = −(x + w1 (t))3 + x + w13 (t) . The linear approximation about the equilibrium solution x = 0 is the periodic linear system x˙ δ (t) = −(3w12 (t) − 1)xδ which is exponentially stable (respectively, unstable) if 6 2π (3w12 (τ ) − 1)dτ > 0 (respectively, < 0). 0

Recalling that A = w(0), we7see that the above condition holds if and only if 3A2 π − 2π > 0, i.e. A > 2/3. We conclude therefore that, on 7 the plane x = 0, the closed orbits of (36) – (11) inside the disc of radius A = 2/3 are unstable, while those outside this disc are locally asymptotically stable. To determine existence and nature of the nontrivial closed orbits, i.e. those which do not lay inside the plane x = 0, we proceed as follows. Integrating the diﬀerential equation (37) we obtain + * 6 t [x2 (τ ) + 3x(τ )w1 (τ ) + 3w12 (τ ) − 1]dτ V (0) . V (t) = exp −2 0

As before, let φ(t, x, w) :=

φx (t, x, w) ψw (t, w)

denote the ﬂow of (36) – (11) and observe that, since V (t) = [φt (x, w)]2 the function φx (t, x, w) satisﬁes * 6 t + [x2 (τ ) + 3x(τ )w1 (τ ) + 3w12 (τ ) − 1 dτ ]φx (0, x, w) φx (t, x, w) = exp − 0

where for convenience we have written x(τ ) for φx (τ, x, w), under the sign of integral. In particular, * 6 2π + 2 2 [x (τ ) + 3x(τ )w1 (τ ) + 3w1 (τ ) − 1]dτ x. φx (2π, x, w) = exp − 0

Along a nontrivial closed orbit, φx (2π, x0 , w0 ) = x0 for some nonzero (x0 , w0 ), i.e.

On the Steady-State Behavior of Forced Nonlinear Systems

6

2π

0

139

[x2 (τ ) + 3x(τ )w1 (τ ) + 3w12 (τ ) − 1]dτ = 0,

and hence 6 3

2π

6 x(τ )w1 (τ )dτ = 2π − 3A π − 2

0

2π

x2 (τ )dτ.

(38)

0

Bearing in mind the fact that dφx (t, x, w) = −[[φx (t, x, w)]2 + 3φx (t, x, w)w1 (t) + 3w12 (t) − 1]φx (t, x, w) dt take the derivatives of both sides with respect to x, to obtain d ∂φx ∂φx (t, x, w) = −[3x2 (t) + 6x(t)w1 (t) + 3w12 (t) − 1] (t, x, w). dt ∂x ∂x Integration over the interval [0, 2π] yields 6 2π ∂φx [3x2 (τ ) + 6x(τ )w1 (τ ) + 3w12 (τ ) − 1]dτ ] (2π, x, w) = exp[− ∂x 0 because ∂φx (0, x, w) = 1. ∂x Suppose now that φx (2π, x0 , w0 ) = x0 for some nonzero x0 . Using (38) we obtain 6 2π ∂φx 2 (2π, x0 , w0 ) = exp[−2π + 3A π − x2 (τ )dτ ]. ∂x 0 7 If A < 2/3 we have ∂φ2π (x0 , w0 ) < 1. ∂x 7 We have shown in this way that, for A < 2/3, 0<

φx (2π, x0 , w0 ) = x0

⇒

∂φx (2π, x0 , w0 ) < 1. ∂x

7 2/3, any nontrivial closed orbit is locally and this proves that, if A < 7 exponentially stable. This completes the analysis for the case A < 2/3. Having shown that all trajectories enter the positively invariant set KA , in which there are no equilibria and an unstable closed orbit on the plane x = 0, we conclude that there are two (and, see Sect. 7, only two) nontrivial closed orbits in KA , one occurring in the half space x > 0, the other occurring in the half space x < 0.

140

C.I. Byrnes et al.

From this analysis it is easy to conclude what follows. For any pair of compact sets X = {(x, y) : max{|x|, |y| ≤ r}

W = {w : |w| ≤ r} ,

the positive orbit of X × W is bounded. Moreover, for large r, if A ≥ the set

7

2/3,

SSLA = ω(X × W ) ∪ CA , reduces to just one closed curve, 7 namely the curve {(x, w) : x = 0, w = A}. On the other hand, if A < 2/3, the set SSLA is a 2-dimensional manifold with boundary, diﬀeomorphic to a set I × S1 , in which I a closed interval of R. Diﬀerent shapes of these sets, for various values of A, are shown in Fig. 4.

Fig. 5. Steady State Locus

7 On the Existence and Uniqueness of Periodic Steady-State Responses For the purpose of output regulation, we will research the steady-state response of systems deﬁned by (1)-(2). Since the error should asymptotically vanish, stability properties, as well as in the uniqueness, of the steady-state response are important. For these reasons, we are particularly interested in results, for reasonable classes of systems, which would assert that local stability implies global uniqueness. In this section, we shall illustrate such a result

On the Steady-State Behavior of Forced Nonlinear Systems

141

for forced oscillations in dissipative systems, following the earlier work of Levinson, Pliss, Krasnosel’ski and Hale. Recall, the system x˙ = f (x, t),

f (x, t + T ) = f (x, t)

is dissipative if there exists R > 0 such that limt→∞ x(t; x0 , t0 ) < R. Deﬁne F : Rn → Rn via F (x0 ) = x(T ; x0 ) For a dissipative system there exists a ball H = {x : x < h} and a natural number k(a) such that for k > k(a) F k (H) ⊂ H . so that by Brouwer’s Fixed Point Theorem there exists a periodic orbit. Control theoretic examples of systems which are dissipative abound and include systems having a globally exponentially stable equilibrium, forced with a periodic input. More generally, any input-to-state stable system [11], forced with a periodic input, is dissipative. For any such system, Lyapunov theory applies and there exists a function V satisfying gradV (x), f (x, t) < 0 for x 0 . In this context, we can adapt the work of Krasnosel’ski [12] to rigorously formulate the notion that local stability implies global uniqueness. On the toroidal cylinder Rn × S 1 we have a “Lyapunov can” V −1 (−∞, c]. We are interested in zeros of the “translation ﬁeld” ψ(t, τ, x0 ) = x0 − x(t, τ, x0 ) when τ = 0, t = T . In this setting, Krasnosel’ski’s main observations are: For each s and all x ∈ V −1 (c), f (x, s) is nonsingular and ψ(t, s, x) is nonsingular for 0 ≤ t − s ∞. (ii) f (x, s) and ψ(t, s, x) do not point in the same direction. Therefore, there is a homotopy

(i)

ψ(t, s, x) ∼ −f (x, s) (iii) ψ(t, s, x) ∼ −f (x, s) for t ≥ s whenever ψ(τ, s, x) = 0 for s ≤ τ ≤ t. (iv) Since V˙ < 0 holds, ψ(t, s, x0 ) = 0 for t ≥ s. (v) Therefore ψ(T, 0, x) ∼ −f (x, 0). Thus, for ψ(T, 0, x0 ) = x0 − x(T, 0, x0 ) = x0 − P(x0 ), we have indV −1 (c) (ψ(T, 0, ·)) = indV −1 (c) (−f (·, 0)) and by the Gauss-Bonnet Theorem

142

C.I. Byrnes et al.

indV −1 (c) (−f ) = indV −1 (c) (n) = 1 since V −1 (−∞, c] is contractible. If all the zeros of ψ are hyperbolic, then the local index of ψ near x0 satisﬁes indx0 (ψ) = sign det(I − DP(x0 )) so that by Stokes’ Theorem γ

sign det(I − DP(x0 )) = +1.

periodic

In particular, if x0 is asymptotically stable and hyperbolic sign det(I − DP(x0 )) = 1 and therefore, if each periodic orbit is exponentially orbitally stable #γ · 1 = 1 or #γ = 1 . It then follows that the local exponential stability of each periodic orbit implies global uniqueness. We conjecture that this assertion remains valid when the orbits are critically asymptotically stable. We remark that when uniqueness holds, the steady state-locus will be the graph of a function π(w) and that invariance, where π is smooth, will be characterized by a partial diﬀerential equation, as in center manifold theory. In the local theory, this pde is often used to ﬁnd or approximate π. In general, one would expect that π is a viscosity solution. Finally, we note that the arguments above are reminiscent of averaging and that, indeed, one can use these methods to prove the basic averaging theorems.

References 1. Byrnes C.I., Isidori A. (1984), A Frequency Domain Philosophy for Nonlinear Systems, with Applications to Stabilization and to Adaptive Control, Proc. of 23rd IEEE Conf. on Decision and Control, Las Vegas. 2. Isidori A., Byrnes C.I., (1990), Output regulation of nonlinear systems, IEEE Trans. Aut. Control, AC-35, 131–140. 3. Byrnes C.I., Isidori A. (1991), Asymptotic stabilization of minimum phase nonlinear systems, IEEE Trans. Aut. Control, AC-36, 1122–1137. 4. Byrnes C.I., Isidori A. (2000), Bifurcation analysis of the zero dynamics and the practical stabilization of nonlinear minimum-phase systems, Asian Journal of Control, 4, 171–185. 5. Byrnes C.I., Isidori A., Willems J.C. (1991), Passivity, feedback equivalence, and the global stabilization of minimum phase nonlinear systems, IEEE Trans. Autom. Contr., AC-36, 1228–1240.

On the Steady-State Behavior of Forced Nonlinear Systems

143

6. Davison E.J. (1976), The robust control of a servomechanism problem for linear time-invariant multivariable systems, IEEE Trans. Autom. Control, 21, 25–34. 7. Francis B.A. (1977), The linear multivariable regulator problem, SIAM J. Contr. Optimiz., 14, 486–505. 8. Francis B.A., Wonham W.M. (1976), The internal model principle of control theory, Automatica, 12, 457–465. 9. Hahn W. (1967), Stability of Motion, Springer-Verlag, New York. 10. Hale J.K., Magalh˜ aes L.T., Oliva W.M. (2001) Dynamics in Inﬁnite Dimensions, Springer Verlag (New York). 11. Sontag E.D. (1995), On the input-to-state stability property, European J. Contr., 1, 24–36. 12. Krasnosel’ski˘i, M.A., Zabre˘iko, P.P.,(1984) Geometric Methods of Nonlinear Analysis, Springer-Verlag Berlin Heidelberg New York Tokyo. 13. Byrnes, C.I., Delli Priscoli, F., Isidori, A. , (1997) Output Regulation of Uncertain Nonlinear Systems, Birkh¨ auser, Boston Basel Berlin.

Gyroscopic Forces and Collision Avoidance with Convex Obstacles Dong Eui Chang1 and Jerrold E. Marsden2 1 2

Mechanical & Environmental Engineering, University of California, Santa Barbara, CA 93106-5070, [email protected] Control & Dynamical Systems, California Institute of Technology, Pasadena, CA 91125, [email protected]

Summary. This paper introduces gyroscopic forces as an tool that can be used in addition to the use of potential forces in the study of collision and convex obstacle avoidance. It makes use of the concepts of a detection shell and a safety shell and shows, in an appropriate context, that collisions are avoided, while at the same time guaranteeing that control objectives determined by a potential function are met. In related publications, we reﬁne and extend the method to include ﬂocking and swarming behavior.

1 Introduction Goals of the Paper. The purpose of this paper is to make use of the techniques of controlled Lagrangians given in [3] and references therein—in particular gyroscopic control forces—in the problem of collision and obstacle avoidance. We are also inspired by the work of Wang and Krishnaprasad [9]. An interesting feature of gyroscopic forces is that they do not interfere with any prior use of potential forces, as in the fundamental work on the navigation function method of Rimon and Koditschek [8], that may have been set up for purposes of setting control objectives. In particular, the method avoids the often encountered diﬃculty of purely potential theoretic methods in which unwanted local minima appear. The techniques we develop appear to be eﬃcient and the algorithms provably respect given safety margins. This paper is a preliminary report on the methodology of gyroscopic forces. We will be developing it further in the future in the context of networks of agents, including underwater vehicles and other systems. Of course these agents have nontrivial internal dynamics that need to be taken into account, but our view (consistent with methods developed by Steve Morse—see, for instance, [6]) is that only information concerning a “safety shell” need be transmitted to the vehicle network, and this perhaps only to nearest neighbors, W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 145–159, 2003.

c Springer-Verlag Berlin Heidelberg 2003

146

D.E. Chang and J.E. Marsden

rather than all the detailed state information about each agent. Of course such a hierarchical and networked approach is critical for a strategy of this sort to be scalable. Collision avoidance is of course a key ingredient in coordinated control of vehicles and in particular in the ﬂight control community. We refer to, for example, [5]. Earlier work, such as [7] introduces “vortical” forces that are reminiscent of, but not the same as, gyroscopic forces studied in the present paper. A future goal is to apply the present method to coordinated control of groups of underwater vehicles; see, for instance, [1] and references therein. The present paper was inspired by a Caltech lecture of Elon Rimon and we thank him for very useful conversations about the subject and his interest. While there remain a number of important results which remain to be proved in the present context, we hope that the work described here will be helpful towards incorporating gyroscopic forces more systematically into methods based on potential functions. The techniques are further developed in [4] and applied to the problem of ﬂocking and swarming behavior. Gyroscopic Forces. Gyroscopic forces denote forces which do not do any work. Mathematically, a force Fg deﬁned to be a gyroscopic force if Fg · q˙ = 0 where q˙ is a velocity vector. A general class of gyroscopic force Fg have the form ˙ q˙ Fg = S(q, q)

(1)

where S is a skew symmetric matrix. There are two useful viewpoints on gyroscopic forces in the dynamics of mechanical systems. One is that gyroscopic forces create coupling between diﬀerent degrees of freedom, just like mechanical couplings. The other is that gyroscopic forces rotate the velocity vector just like a magnetic ﬁeld acting on a charged particle. The ﬁrst interpretation regards the matrix S in (1) as an interconnection matrix and the second interpretation considers S as an inﬁnitesimal rotation. In this paper, we will take the second viewpoint and use gyroscopic forces to prevent vehicles from colliding with obstacles or other vehicles. In the future, we will also elaborate on the ﬁrst viewpoint relating the matrix S to the graph of inter-vehicle communication links. The ﬁrst viewpoint was taken in [3] when gyroscopic forces were introduced into the method of controlled Lagrangian; indeed, gyroscopic forces are very useful in stabilization of mechanical systems.

2 Obstacle Avoidance The problem of obstacle avoidance is important in robotics and multivehicle systems. The objective is to design a controller for a robot so that it approaches its target point without colliding with any obstacles during the journey. We will employ potential forces, dissipative forces, and gyroscopic forces. The ﬁrst two forces take care of convergence to the target point and the gyroscopic

Gyroscopic Forces and Collision Avoidance with Convex Obstacles

147

force handles the obstacle avoidance. We will compare our method with the navigation function method which was developed in [8]. For the sake of easy exposition, we address a particular situation where there is only one obstacle in a plane. Since our algorithm uses only local information around the vehicle, the same control law works for multiple obstacles. Obstacle Avoidance by Gyroscopic Forces. Suppose that there is a fully actuated vehicle and an obstacle in the xy-plane. For the purpose of exposition, we assume that the vehicle is a point of unit mass and the obstacle is a unit disk located at the origin. We want to design a feedback control law to (asymptotically) drive the vehicle to a target point qT = (xT , yT ) without colliding with the obstacle. A detection shell, a ball of radius rdet is given to the vehicle such that the vehicle will respond to the obstacle only when the obstacle comes into the detection shell. Safety shells can be readily added to this discussion, as in §3 below; the safety shell itself is designed to not collide with the obstacle. ¨ = u, where q = (x, y) The dynamics of the vehicle are given simply by q and u = (ux , uy ). The control u consists of four parts as follows: u = F p + Fd + Fg + v

(2)

where Fp is a potential force which assigns to the vehicle a potential function with the minimum at the target qT ; Fd is a dissipative force; Fg is a gyroscopic force; and v is an another control force. We set v to zero unless this additional control is needed (as remarked later, it may be useful in near-zero velocity collisions). The three forces, Fp , Fd , and Fg are of the following form: Fp = −∇V (q),

˙ q, ˙ Fd = −D(q, q)

˙ q˙ Fg = S(q, q)

where V is a (potential) function on R2 , the matrix D is symmetric and positive-deﬁnite, and the matrix S is skew-symmetric. We choose the potential function V and the dissipative force Fd as follows: V (q) =

1 q − qT 2 , 2

˙ Fd = −2q.

Before we choose a gyroscopic force, let us introduce some deﬁnitions. Let d(q) = (dx (q), dy (q)) be the vector from the vehicle position, q, to the nearest point in the obstacle. Since the obstacle is convex, the vector d(q) is well deﬁned. Let d(q) = d(q) be the distance between the vehicle and the obstacle. We now choose the following gyroscopic force Fg * + ˙ 0 −ω(q, q) ˙ Fg = q. (3) ˙ ω(q, q) 0 Here, the function ω is deﬁned by

148

D.E. Chang and J.E. Marsden

 πVmax   ˙ ≥ 0] if [d(q) ≤ rdet ] ∧ [d(q)· q˙ > 0] ∧ [det[d(q), q]    d(q) πVmax ˙ = ω(q, q) ˙ < 0] − if [d(q) ≤ rdet ] ∧ [d(q)· q˙ > 0] ∧ [det[d(q), q]    d(q)   0 otherwise (4) where Vmax > 0 is a constant and ∧ denotes the logical “and”. The meaning of the function ω is as follows. The vehicle gets turned by the gyroscopic force only when it detects an obstacle in the detection shell (d(q) ≤ rdet ) and it is heading toward the obstacle (d(q) · q˙ > 0). The role of the gyroscopic force is to rotate the velocity vector (as indicated in (3)). The direction of the rotation ˙ depends on the orientation of the two vectors, (that is, the sign of ω(q, q)) ˙ i.e, the sign of det[d(q), q]. ˙ d(q) and q, The energy E of the vehicle is given by its kinetic plus potential energies: ˙ = E(q, q)

1 ˙ 2 + V (q). q 2

(5)

One checks that the energy is non-increasing in time as follows: d ˙ = q˙ · Fd = −2q ˙ 2 ≤ 0. E(q, q) dt

(6)

We now prove, by contradiction, that the vehicle does not collide with the obstacle at nonzero velocity when the initial energy satisﬁes ˙ E(q(0), q(0)) ≤

1 2 V , 2 max

(7)

where Vmax is the positive constant in (4). Suppose that the vehicle collided ˙ c ) = 0. Take a small with the obstacle at time t = tc < ∞ with velocity q(t ∆t > 0 and consider the dynamics in the time interval, I = [tc − ∆t, t− c ]. ˙ ≥ 0 in I. Then, the Without loss of generality, we may assume det[d(q), q] dynamics are given by * + ˙ −2 −ω(q, q) ¨ q= q˙ − (q − qT ) ˙ ω(q, q) −2 ˙ = πVmax /d(q). One can integrate this ODE for q˙ during I as with ω(q, q) follows: * + − − −2∆t cos θ(tc ) − sin θ(tc ) − ˙ c − ∆t) q˙ (tc ) = e q(t (8) − sin θ(t− c ) cos θ(tc ) * + 6 t− − − c −τ ) cos(θ(tc ) − θ(τ )) − sin(θ(tc ) − θ(τ )) −2(t− c − (q(τ ) − qT ))dτ , e − sin(θ(t− c ) − θ(τ )) cos(θ(tc ) − θ(τ )) tc −∆t where

Gyroscopic Forces and Collision Avoidance with Convex Obstacles

6

6

t

t

˙ ω(q(s), q(s))ds =

θ(t) = tc −∆t

tc −∆t

πVmax ds. d(q(t))

149

(9)

˙ Since the velocity q(t) is continuous on the time interval I, there is a β > 0 ˙ such that q(t) > β in I. We can therefore rewrite (8) as * + − − − −2∆t cos θ(tc ) − sin θ(tc ) ˙ c)=e ˙ c − ∆t) + O(∆t). q(t (10) q(t − sin θ(t− c ) cos θ(tc ) because q(t) is bounded during I and ∆t is very small. By (5), (6), (7) and ˙ V (q) ≥ 0, we have q(t) ≤ Vmax for t ∈ I. So, ∆t ≥

d(q(tc − ∆t)) . Vmax

(11)

Since the trajectory is approaching the obstacle during I, one may assume that d(q(t)) ≤ d(q(tc − ∆t))

(12)

for t ∈ I. It follows from (9), (11) and (12) that θ(t− c )≥π

(13)

Notice that the inequality (13) is independent of ∆t. We can conclude from ˙ (10) and (13) that the velocity vector q(t) rotates more than, say, 3π/4 radians during the interval [tc −∆t, t− ] for a small ∆t > 0. However, since we assumed c that the vehicle collided with the obstacle at t = tc with nonzero velocity, the velocity cannot rotate much during the interval [tc −∆t, t− c ] for a small ∆t > 0 ˙ by the continuity of q(t). We have reached a contradiction and therefore, there is no (ﬁnite-time) collision of the vehicle with the obstacle at nonzero velocity. There are two ways that the vehicle may collide with the obstacle: in a ﬁnite time or in inﬁnite time. As shown above, a ﬁnite-time collision occurs only if ˙ c) = 0 q(t

(14)

where tc is the moment of collision. Let us consider the case where there is a time sequence ti " ∞ such that q(ti ) converges to the obstacle. By (5) and(6), 6 ∞ 1 ˙ )2 dτ ≤ E(0) < ∞. q(τ 2 0 Hence, there exists a time sequence si " ∞ such that ˙ i ) = 0. lim q(s

i→∞

(15)

150

D.E. Chang and J.E. Marsden

This means the vehicle slows down, at least sporadically. The common phenomenon in both of these collision possibilities is that the vehicle slows down as shown in (14) and (15). Let us call both types of collision a zero-velocity collision for the sake of simple terminology. One might introduce an additional adaptive control scheme through v in (2) to approach the target as well as avoid the zero-velocity collision. That is, if one assumes that there is an additional control that maintains a minimum velocity, then these zero velocity collision situations can be avoided. We now discuss the asymptotic convergence of the vehicle to the target in the case that the vehicle does not end up with a zero-velocity collision. Suppose ˙ that the trajectory (q(t), q(t)) satisﬁes (7) and does not end with a zerovelocity collision. Since q(t) is a certain distance away from the obstacle, there exists an open set W ⊂ R2 containing the obstacle such that the trajectory lies in the compact set K := E −1 ([0, E(t = 0)])\(W × R2 ). Then, the trajectory exists for all t ≥ 0. One can adapt the usual version of LaSalle’s invariance principle to show the asymptotic convergence of the trajectory to the target state, (qT , 0), where the energy in (5) is used as a Lyapunov function. Here, we give an alternative proof of convergence. Consider the following function: ˙ = E(q, q) ˙ + dV · q˙ U (q, q) 1 2 = (x˙ + y˙ 2 + (x − xT )2 + (y − yT )2 ) + ((x − xT )x˙ + (y − yT )y) ˙ 2 with 0 < < 1. See the Appendix for the motivation for the above choice of ˙ > 0 on Lyapunov function. One can check that (a) U (qT , 0) = 0 and U (q, q) K\{(qT , 0)}, and (b) (qT , 0) is the only critical point of U on K. Along the trajectory, dU ˙ 2 − q − qT 2 − 2(q − qT ) · q˙ = −(2 − )q dt * + ˙ 0 −ω(q, q) ˙ q. +(q − qT ) ˙ ω(q, q) 0 ˙ is bounded on K, one can ﬁnd > 0 and c > 0 such that Since ω(q, q) dU ≤ −cU ≤ 0 dt on K. It follows that U (t) ≤ U (0)e−ct . This proves that the trajectory asymptotically converges to the target. In summary, we have shown that the vehicle semi-globally converges to the target state without collision with the obstacle except possibly for a zerovelocity collision. Here, the semi-global property comes from the dependence of

Gyroscopic Forces and Collision Avoidance with Convex Obstacles

151

Vmax on the initial condition given in (7). We may avoid zero-velocity collision by adding an adaptive scheme. We expect that the set of initial states ending up with zero-velocity collision is small. Remarks. 1. The choice of Vmax satisfying (7) may be conservative. Recall that the gyroscopic force gets turned on only when an obstacle comes into the detection 2 shell. So, we can choose a new value of Vmax satisfying E(t = td ) ≤ 12 Vmax at the moment, t = td , when the vehicle detects the obstacle. The proof given above is still valid with this update rule of Vmax . In this sense, the above collision avoidance algorithm works globally. Moreover, the same control law works in the existence of multiple obstacles since our control law is feedback and the vehicle only uses the local information in its detection shell. 2. One can easily modify the above control algorithm for convex obstacles. When the obstacle is not convex, one needs to add an adaptive scheme. One conservative way is to make a convex buﬀer shell which contains the nonconvex obstacle and regard this convex shell as an obstacle. However, this entails that the vehicle knows the global shape of the obstacle. In reality, the vehicle may only have a local information of the obstacle. In such a case, one needs to apply a scheme to ﬁnd a convex arc (or, surface) which divides the detection shell so that the vehicle lies on one side and the obstacle on the other side. Then, one regards this convex arc as an obstacle. For obstacles and bodies with sharp corners and ﬂat surfaces or edges, the algorithm also needs to be modiﬁed; this can be done and will appear in forthcoming works of the authors. We illustrate the results of such an algorithm in Figure 1 below. 3. We give an alternative choice of a gyroscopic force, which produces a faster convergence of a vehicle to its target point than that in (3). Assume that the vehicle has detected an obstacle in its detection shell. In such a case, let us deﬁne the function, σqT = σqT (q) as follows: 0 if the obstacle does not lie between the two points, q and qT σqT (q) = 1 otherwise where q is the position of the vehicle and qT is the target point of the vehicle. Roughly speaking, the function σqT checks if the vehicle can directly see the target. The new gyroscopic force, F˜g , is deﬁned by the product of the function σqT and the old gyroscopic function Fg in (3) as follows: F˜g = σqT Fg . The vehicle switches oﬀ the gyroscopic force if the vehicle can directly see the target even when there is an obstacle nearby. Simulation studies show that this new gyroscopic force gives faster convergence to the target. We expect that the gyroscopic force F˜g reduces the possibility of a zero-velocity collision. ˙ 4. If d(q) · q˙ ≈ 0 and there is a measurement error, then the sign of ω(q, q) ˙ for becomes fragile. In such a case, one can choose a constant sign of ω(q, q)

152

D.E. Chang and J.E. Marsden

a period. The reason is as follows. The condition, d(q) · q˙ ≈ 0, means that the velocity is almost perpendicular to the obstacle. Hence, the direction of rotation the velocity does not matter much as long as one keeps rotating it until the measurement of the direction of velocity becomes relevant to the vector d(q) in a certain way. Another possible option is to choose the sign of ω which agrees with the current direction of the potential plus dissipative force, −∇V + Fd . Issues such as this are important for robustness. 5. Above, we chose a particular form of potential function, dissipative force, and gyroscopic force. However, one can modify the above proof for more general from of V , Fd and Fg . We also assumed that the vehicle is a point mass. In reality, it has a volume. In this case, the vehicle is equipped with two shells around it where the inner shell is the safety shell which contains the vehicle and the outer shell is the detection shell. In this case, one must prevent the obstacle from coming into the safety shell. For example, d(q) in (4) should be modiﬁed to the distance from the safety shell to the obstacle. 6. One can extend this control algorithm to three dimensions. In such a ˙ q˙ case, the skew-symmetric matrix S in the gyroscopic force Fg = S(q, q) should be an inﬁnitesimal rotation with the axis in parallel to the vector ˙ × q˙ when d(q, q) ˙ × q˙ = 0. When d(q, q) ˙ × q˙ = 0, one just chooses a d(q, q) preferred rotational direction, as in the planar case. Comparison with the Navigation Function Method. We compare our method with the navigation function method developed in [8]. In the navigation function method, when the vehicle is fully actuated and there are some obstacles, then one ﬁrst designs a potential function which has maxima on the boundary of obstacles and the minimum at the target point where no other local minima are allowed, but saddle points are allowed in dynamics because the stable manifolds of saddle points are measure zero. The control force is the sum of the potential force from the potential function and a dissipative force. Then, the vehicle converges to its target point avoiding collision with obstacles. A caveat in the navigation function method is that the construction of such a potential function depends on the global topology of the conﬁguration space excluding obstacles. In other words, the vehicle must know all the information of obstacles in advance; of course one could also consider developing a more local formulation of the navigation function methodology. Our method diﬀers fundamentally from the navigation function method in which the potential force is used for both convergence and collision-avoidance. Our method employs a potential force only for convergence and uses a gyroscopic force for collision avoidance. We design our potential function without considering the conﬁguration of obstacles, so it is easy to choose a potential function. We only use local information inside the detection shell of the vehicle to execute the gyroscopic force. Hence, we need not know all the information of obstacles in advance. In either method, one must be careful about the (perhaps remote) possibility of zero-velocity collisions. In general, we regard these two methods as complementary to each other.

Gyroscopic Forces and Collision Avoidance with Convex Obstacles

153

Simulation: One Vehicle + Two Obstacles. Consider the case of one vehicle and two obstacles. One obstacle is a disk of radius 1 located at (0, 0) and the other is a disk of radius 2 centered at (5, 0) (left side of Figure 1). 4

6

3 4

target 2

2

1

Obstacle

0

0

Obstacle

-1 -2

-2 -4

-4

-3 -2

0

2

4

6

8

10

-4

-5

-4

-3

-2

-1

0

1

2

3

4

Fig. 1. (Left–Vehicle with two obstacles.) The vehicle starts from (−2, −1) with zero initial velocity, converging to the target point, (8, 3), avoiding any collisions with obstacles. The shaded disk about the vehicle is the detection shell. (Right–avoiding a ﬂat obstacle.) This shows a simulation result for a modiﬁed algorithm suitable for objects with ﬂat surfaces; the vehicle starts at (−3, −1) and the target is at (2, 0).

The vehicle is regarded as a point of unit mass. It starts from (−2, −1) with initial zero velocity and converges to the target point qT = (8, 3). We used the following potential function and dissipative force: V (q) =

1 q − qT 2 , 2

˙ Fd = −2q.

√ We used 10 for Vmax in the gyroscopic force in (3) and (4). The right side of this ﬁgure shows that the general methodology, suitably modiﬁed also works for objects with ﬂat surfaces, and sharp corners and edges. As mentioned previously, this will be explained in detail in future publications.

3 Collision Avoidance: Multi-vehicles We develop a collision-avoidance scheme using gyroscopic forces in the case that there are multiple vehicles in a plane. For the purpose of illustration, we only consider two vehicles and we make a remark on how to extend this to multi-vehicle case. Collision Avoidance between Two Vehicles. Let us consider the situation where there are two vehicles in a plane and there are no other vehicles or

154

D.E. Chang and J.E. Marsden

obstacles. Each vehicle wants to approach its own target point without colliding with the other vehicle. We assume that each vehicle has a ﬁnite volume and two shells around its center. The inner shell is called a safety shell, which completely contains the vehicle. Its radius is denoted by rsaf . The outer shell is a detection shell of thickness rdet . Each vehicle detects the other vehicle only when the other vehicle comes into the detection shell. Here, a collision means the collision between safety shells. Let qi = (xi , yi ) be the position of the i-th vehicle, and qT,i = (xT,i , yT,i ) be the target point of the i-th vehicle, with i = 1, 2. The dynamics of the i-th ¨ i = ui with ui = (uxi , uyi ). The control law consists of vehicle are given by q a potential, a dissipative, and a gyroscopic force: u = −∇V + Fd + Fg . For simplicity, we will only design a controller for vehicle 1 in the following. One can get a controller for vehicle 2 in the similar manner. Choose the following potential function and dissipative force for vehicle 1: V1 (q1 ) =

1 q1 − qT,1 2 , 2

Fd,1 (q1 , q˙ 1 ) = −2q˙ 1 .

Before we choose a gyroscopic force, let us deﬁne a couple of functions. Let d(q1 , q2 ) be the distance between the safety shells of the two vehicles, which is given by d(q1 , q2 ) = q1 − q2 − rsaf,1 − rsaf,2 . Deﬁne ϕ : R2 × R2 → [−π/2, π/2] by the signed angle from v to w if [v · w ≥ 0] ∧ [v · w = 0], ϕ(v, w) = 0 otherwise For example, ϕ((1, 0), (1, 1)) = π/4 and ϕ((1, 1), (1, 0)) = −π/4. Deﬁne χ : R2 × R2 → R by 1 if d(q1 , q2 ) ≤ rdet,1 χ(q1 , q2 ) = 0 otherwise which checks if vehicle 2 is in the detection shell of vehicle 1. For the position vectors q1 and q2 of both vehicles, deﬁne q21 = q2 − q1 , q12 = −q21 . The gyroscopic force Fg,1 of vehicle 1 is given by * + 0 −ω(q1 , q˙ 1 , q2 , q˙ 2 ) Fg,1 = χ(q1 , q2 ) (16) q˙1 ω(q1 , q˙ 1 , q2 , q˙ 2 ) 0 where the function ω is given by ω(q1 , q˙ 1 , q2 , q˙ 2 ) = f (q1 , q˙ 1 , q2 , q˙ 2 )

πVmax d(q1 , q2 )

where Vmax is a ﬁxed positive number and the function f is deﬁned by considering four possible cases, C1–C4, as follows:

Gyroscopic Forces and Collision Avoidance with Convex Obstacles

155

C1. If [q21 · q˙ 1 ≥ 0] ∧ [q21 · q˙ 2 ≥ 0]: vehicle 2 is before and heading away from vehicle 1, then 1 if ϕ(q21 , q˙ 1 ) − ϕ(q21 , q˙ 2 ) ≥ 0 f (q1 , q˙ 1 , q2 , q˙ 2 ) = −1 otherwise C2. If [q21 · q˙ 1 ≥ 0] ∧ [q21 · q˙ 2 < 0]: vehicle 2 is before and heading toward vehicle 1, then 1 if ϕ(q21 , q˙ 1 ) − ϕ(q˙ 2 , q12 ) ≥ 0 f (q1 , q˙ 1 , q2 , q˙ 2 ) = −1 otherwise C3. If [q21 · q˙ 1 < 0] ∧ [q21 · q˙ 2 < 0]: vehicle 2 is behind and heading toward vehicle 1, then 1 if ϕ(q12 , q˙ 1 ) − ϕ(q˙ 12 , q2 ) > 0 f (q1 , q˙ 1 , q2 , q˙ 2 ) = −1 otherwise C4. Otherwise (i.e., vehicle 2 is behind and heading away from vehicle 1), f (q1 , q˙ 1 , q2 , q˙ 2 ) = 0. Notice that vehicle 1 does not turn on the gyroscopic force when vehicle 2 is behind and heading away from vehicle 1 in the detection shell of vehicle 1. The energy of each vehicle is given by Ei (qi , q˙ i ) =

1 q˙ i 2 + Vi (qi ) 2

with i = 1, 2. One can check that each energy function is non-increasing in time. Suppose that the initial state (qi (0), q˙ i (0)) of each vehicle satisﬁes Ei (qi (0), q˙ i (0)) ≤

1 (Vmax,i )2 2

with i = 1, 2. We want to show that it cannot happen that the two vehicles collide with q˙ 1 = 0 or q˙ 2 = 0 at the moment of collision. We prove this by contradiction. Suppose that the two vehicles collided at time t = tc with q˙ 1 (tc ) = 0 or q˙ 2 (tc ) = 0. Without loss of generality, we may assume that q˙ 1 (tc ) = 0. Then, one will reach a contradiction by studying the dynamics of vehicle 1 during the time interval, [tc − ∆t, t− c ] for a small ∆t > 0 just as done for the case of obstacle avoidance in § 2. One can also show semi-global convergence of each vehicle to its target point. The proof is almost identical to that in § 2. The point is that each vehicle has its own Lyapunov function which is independent of that of the other vehicle. In this sense, the control scheme given here is a distributed and decentralized control.

156

D.E. Chang and J.E. Marsden

Remarks. 1. We have not excluded the possibility of zero-velocity collision. Since this happens only at low velocity, one can add an adaptive scheme to handle this. 2. We give another possible choice of the function, ω, in (16). We do this from the viewpoint of vehicle 1 as the same procedure applies to other vehicle. Suppose that vehicle 2 is detected and it does not belong to the case, C4, above. Let v12 = q˙ 1 − q˙ 2 be the velocity of vehicle 1 relative to vehicle 2. We regard vehicle 2 as a ﬁxed obstacle located at q2 and compute ω with the algorithm used for obstacle avoidance in § 2 with this relative velocity v12 . 3. We give an ad-hoc extension of the gyroscopic collision avoidance scheme to the case of multiple vehicles. The situation is the same as that of two-vehicle case except that there are more than two vehicles. Suppose that vehicle A detected n other vehicles in its detection shell where we have already excluded vehicles which are behind and heading away from vehicle A. Let di , i = 1, . . . , n be the distance between the safety shell of vehicle A and that of the i-th vehicle. Let n

qCM =

1 qi n i=1

be the mean position of the n vehicles, and n

q˙ CM =

1 q˙ i n i=1

the mean velocity of the n vehicles. We now design ω in the gyroscopic force Fg,A = (−ω y˙ A , ω x˙ A ) where ω as follows: ω=f

πVmax min{di | i = 1, . . . n}

where one decides the value of f by applying the algorithm used for the two-vehicle case, assuming that there is only one (equivalent) vehicle located at qCM with velocity q˙ CM . One needs to modify this in the case that qCM coincides with the position of vehicle A. The same procedure applies to other vehicles. Simulations show that this ad-hoc scheme works well. 4. If one of the vehicles is “adversarial” then clearly the situation described above needs to be modiﬁed. Simulation: Three Vehicles. Figure 2 shows a simulation of three vehicles. The three vehicles are initially located along the circle of radius 2 and they are 120◦ away from one another. The target point of each vehicle is the opposite point on the circle. The detection radius is 0.5 and the safety shell was not considered for simplicity. We used the control law described above with Vmax = √ 2.5. The simulation result is given in Figure 2 where the shaded disks denote detection shells. One can see that each vehicle switches on the gyroscopic force when it detects other vehicles.

Gyroscopic Forces and Collision Avoidance with Convex Obstacles

157

3

2

1

0

-1

-2

-3

-3

-2

-1

0

1

2

3

Fig. 2. Three vehicles are initially located along the circle, 120◦ away from one another. The target of each vehicle is the opposite point on the circle. The shaded disks are detection shells

Appendix One often uses LaSalle’s theorem to prove asymptotic stability of an equilibrium in a mechanical system using an energy function E. Although LaSalle’s theorem is a powerful stability theorem, it normally gives no information on exponential convergence since E˙ is only semideﬁnite. Here, we show that if the energy has a minimum at an equilibrium of interest and the system is forced by a strong dissipative force (and a gyroscopic force), then the equilibrium is exponentially stable. The idea, called Chetaev’s trick, is to add a small cross term to the energy function to derive a new Lyapunov function. In this appendix, we give an intrinsic explanation to Chetaev’s trick. A preliminary work was done in [2] in a diﬀerent situation. Consider a mechanical system with the Lagrangian ˙ = K(q, q) ˙ − V (q) = L(q, q)

1 mij (q)q˙i q˙j − V (q) 2

with q = (q 1 , . . . , q n ) ∈ Rn , and the external force, F = Fd + Fg , where Fd is a strong dissipation given by ˙ = −Rq q, ˙ Fd (q, q)

R = RT > 0,

(17)

˙ q, ˙ S T = −S. Here and Fg is a gyroscopic force of the form Fg = S(q, q) we assume that the matrix S is bounded in magnitude on a domain we are interested in. In showing exponential stability, the role of gyroscopic force is little when the dissipation is strong, i.e., R = RT > 0. Also, one may allow the ˙ Here, we use Rn as a conﬁguration space matrix R to depend on velocity q.

158

D.E. Chang and J.E. Marsden

for simplicity. However, all arguments will be made in coordinate-independent language in the following. Suppose the energy ˙ = K(q, q) ˙ + V (q) E(q, q) has a nondegenerate minimum at the origin, (0, 0) ∈ Rn × Rn : i.e., dV (0) = 0,

∂2V (0) > 0 ∂q i ∂q j

(18)

˙ Without since the kinetic energy is already positive deﬁnite in the velocity q. loss of generality, we assume that V (0) = 0. Let us review the proof of the asymptotic stability of the origin using LaSalle’s theorem with E as a Lyapunov function. Consider the invariant ˙ Rq ˙ = 0}. Suppose (q(t), q(t)) ˙ subset M of the set {E˙ = −q, is a trajectory ˙ lying in M. Then, q(t) ≡ 0. Substitution of this into the equations of motion yields dV (q(t)) ≡ 0. By (18), the critical point q = 0 of V is isolated. It follows q(t) ≡ 0. Hence, M consists of the origin only. By LaSalle’s theorem, the origin is asymptotically stable. We now devise a trick to show the exponential stability of the origin. Consider the following function U : ˙ = E(q, q) ˙ + dVq q˙ U (q, q) ∂V 1 = mij (q)q˙i q˙j + V (q) + i q˙i 2 ∂q

(19)

with 0 < 1. Notice that the deﬁnition of U in (19) is coordinateindependent. For a suﬃciently small , dU (0, 0) = 0,

D2 U (0, 0) > 0

(20)

˙ Hence, U has a where D2 U is the second derivative of U with respect to (q, q). minimum at the origin. We will use U as a Lyapunov function. One computes dU ˙ = −W (q, q) ˙ (q, q) dt

(21)

where ˙ = Rq, W (q, q) ˙ q ˙ −

∂ 2 V i j ∂V i j k −1 q˙ q˙ + i (−Γjk q˙ q˙ − (m dV )i ) ∂q i ∂q j ∂q

∂V ˙ i) ((m−1 Rq) ˙ i − (m−1 S q) ∂q i ˙ q) ˙ − (∇q˙ dV )q˙ + m−1 (dV, dV ) + m−1 (Rq, ˙ dV ) = R(q, +

˙ dV ) − m−1 (S q,

(22)

Gyroscopic Forces and Collision Avoidance with Convex Obstacles

159

i where ∇ is the Levi-Civita connection of the metric m and Γjk are the Christoﬀel symbols of ∇. One can check that for a suﬃciently small > 0

dW (0, 0) = 0,

D2 W (0, 0) > 0.

(23)

By (20) and (23), there exists c > 0 such that d(W − cU )(0, 0) = 0 and D2 (W − cU )(0, 0) > 0. Therefore, W − cU ≥ 0 in a neighborhood N of the origin. This and (21) imply dU dt ≤ −cU ≤ 0. It follows −ct ˙ ˙ U (q(t), q(t)) ≤ U (q(0), q(0))e

(24)

on N . This proves the exponential stability of the origin. One can also go further by invoking the Morse lemma to ﬁnd a local chart z = (z 1 , . . . , z 2n ) 2n i 2 in which the function U becomes/U (z) = i=1 (z ) . Hence, (24) implies 2n i 2 − 2c t where z = z(t) ≤ z(0)e i=1 (z ) . Acknowledgements. We thank Noah Cowan, Sanjay Lall, Naomi Leonard, Elon Rimon, Clancy Rowley, Shawn Shadden, Claire Tomlin, and Steve Waydo for their interest and valuable comments. Research partially supported by ONR/AOSN-II contract N00014-02-1-0826 through Princeton University.

References 1. Bhatta, P., Leonard, N. (2002) “Stabilization and coordination of underwater gliders”, Proc. 41st IEEE Conference on Decision and Control. 2. Bloch A., Krishnaprasad P.S., Marsden J.E., Ratiu T. (1994) Annals Inst. H. Poincar´e, Anal. Nonlin., 11:37–90. 3. Chang D.E., Bloch A.M., Leonard N.E., Marsden J.E., Woolsey C. (2002) ESAIM: Control, Optimization and Calculus of Variations, 8:393–422. 4. Chang, D.E., Shadden S., Marsden, J.E. and Olfati-Saber, R. (2003), Collision avoidance for multiple agent systems, Proc. CDC (submitted). 5. Hwang I., Tomlin C.(2002) “Multiple aircraft conﬂict resolution under ﬁnite information horizon”, Proceedings of the AACC American Control Conference, Anchorage, May 2002, and “Protocol-based conﬂict resolution for air traﬃc control”, Stanford University Report SUDAAR-762, July, 2002. 6. Jadbabaie A, Lin J, Morse A.S. (2002) “Coordination of groups of mobile autonomous agents using nearest neighbor rules”, Proc. 41st IEEE Conference on Decision and Control, 2953–2958; see also IEEE Trans. Automat. Control (to appear 2003). 7. Koseck´ a, J., Tomlin C., Pappas, G., Sastry, S. (1997) “Generation of conﬂict resolution maneuvers for air traﬃc management”, IROS; see also IEEE Trans. Automat. Control, 1998, 43:509–521. 8. Rimon E., Koditschek D.E. (1992) IEEE Trans. on Robotics and Automation, 8(5):501–518. 9. Wang L.S., Krishnaprasad P.S. (1992) J. Nonlinear Sci., 2:367–415.

Stabilization via Polynomial Lyapunov Function Daizhan Cheng Institute of Systems Science, Chinese Academy of Sciences, Beijing 100080, P.R.China, [email protected]

1 Introduction In this paper we summarize some recent results on local state feedback stabilization of a nonlinear system via a center manifold approach. A systematic design procedure for stabilizing nonlinear systems of nonminimum phase, i.e., with unstable zero dynamics, was presented recently in [11]. The basic idea can be described as follows. We ﬁrst propose some sufﬁcient conditions to assure the approximate stability of a dynamic system. Using these conditions and assuming the zero dynamics has stable and center linear parts, a method is proposed to design controls such that the dynamics on the designed center manifold of the closed-loop system are approximately stable. It is proved that using this method, the ﬁrst variables in each of the integral chains of the linearized part of the system do not aﬀect the approximation degree of the dynamics on the center manifold. So the ﬁrst variables are considered as nominating controls, which can be designed to produce a suitable center manifold. Based on this fact, the concept of injection degree is proposed. According to diﬀerent kinds of injection degrees certain suﬃcient conditions are obtained for the stabilizability of systems of non-minimum phase. To apply this approach a new useful tool, the Lyapunov function with homogeneous derivative (LFHD) has been developed. It assures the approximate stability of a dynamic system. Since the center manifold of a dynamic system is not easily obtainable but its approximation is easily computable, LFHD is particularly useful in center manifold approach. Another important tool is the normal form of nonlinear control systems, which is essential for analyzing the stability of systems. It was shown that [13] the classical normal form is very restrictive because it requires a very strong regularity assumption. In [13] a generalized normal form was proposed. It is based on point relative degree and point decoupling matrix. Since they W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 161–173, 2003.

c Springer-Verlag Berlin Heidelberg 2003

162

D. Cheng

are only deﬁned point-wise, certain regularity requirements can be removed. Moreover, the generalized normal form can be obtained by straightforward computation. So it is convenient in use. Several stabilization results for classical normal forms can be developed to their counterparts for generalized normal forms. The main purpose of this paper is to summarize some recent developments [11], [9], [13], [14] into a general frame. The paper is organized as follows: Section 2 introduces the LFHD. Some suﬃcient conditions are presented in Section 3 to assure the negativity of the homogeneous polynomials. Section 4 cites some related results from the center manifold theory. The new stabilization technique for a class of nonlinear systems of non-minimum phase is introduced in Section 5. Section 6 is devoted to the generalized normal form of nonlinear control systems and its applications. In section 7, the stabilization of systems with diﬀerent types of center manifolds are reviewed brieﬂy.

2 Lyapunov Function with Homogeneous Derivative Consider a nonlinear dynamic system x˙ = f (x),

x ∈ Rn ,

(1)

where f (x) is a smooth vector ﬁeld. (The degree of smoothness is enough to assure the existence of following required derivatives.) Let the Jacobian matrix of f (x) at zero to be Jf (0). It is well known that [15] Lemma 2.1 If Jf (0) is a Hurwitz matrix, then the overall system is asymptotically stable at the origin. Now we may use Taylor expansion on each component fi (x) of f (x), to get the lowest degree (ki ) non-vanishing terms as fi = gi (x) + 0(xki +1 ),

i = 1, · · · , n,

where gi is a homogeneous polynomial of degree ki . Deﬁnition 2.2 Construct a system as x˙ = g(x),

x ∈ Rn ,

(2)

where g = (g1 , · · · , gn ), and gi are deﬁned as above. Then (2) is called the approximate system of system (1). Remark. It is clear that if Jf (0) is Hurwitz, the approximate system of (1) is x˙ = Jf (0)x. As a generalization of Lemma 2.1, it is natural to ask whether the asymptotic stability of (2) assures the asymptotic stability of (1)? In fact, the

Stabilization via Polynomial Lyapunov Function

163

Lyapunov function with homogeneous derivative (LFHD), proposed in [11], answers the question partly. Deﬁnition 2.3 A positive deﬁnite polynomial function V > 0 is called a LFHD of system (2) if its derivative along (2) is a homogeneous polynomial. Deﬁnition 2.4 System (1) is said to be approximately stable, if x˙ = f (x) + r(x) is asymptotically stable, where r(x) is a disturbance satisfying r(x) = (r1 (x), · · · , rn (x))T and ri (x) = 0(xki +1 ). Remark. If system (2) is approximately stable, then system (1) is asymptotically stable. The following result provides a suﬃcient condition for system (2) to be approximately stable. Theorem 2.5 System (2) is approximately stable if there exists a LFHD V (x) > 0 such that the derivative V˙ |(2) < 0. Remark. Assume system (1) is odd leading, i.e., the degrees ki of the components of its approximate system (2) are all odd. Then we can choose an integer m such that 2m ≥ max{k1 , · · · , kn } + 1. Setting 2mi = 2m − ki + 1, a LFHD can be constructed as V (x) =

n

pi (xi )2mi ,

pi > 0, i = 1, · · · , n.

(3)

i=1

This is the most useful LFHD. Example 2.6 Consider the following system . x˙ = x2 tan(5y − 6x) y˙ = y 2 sinh(x3 − y 3 ).

(4)

The approximate system is . x˙ = −6x3 + 5x2 y y˙ = −y 5 + x3 y 2 .

(5)

Choosing V = x4 + 4y 2 , then we have V˙ |(5) = 4x3 (−6x3 + 5x2 y) − 8y 6 + 8x3 y 3 ≤ −24x6 + 20( 56 x6 + 16 y 6 ) − 8y 6 + 4(x6 + y 6 ) 2 6 6 = − 10 3 x − 3 y < 0. So (4) is asymptotically stable at the origin.

164

D. Cheng

3 Negativity of a Homogeneous Polynomial To use the LFHD method it is important to verify whether an even degree polynomial is negative deﬁnite. As proposed in [11] a basic inequality is used to estimate the negativity. 2 2 n n n 2 28 ki 22 k 22 2 ki 2 xi , where k = ki . (6) 2 xi 2 ≤ 2 2 k i=1 i=1 i=1 The proof can be found in [12]. Now for a 2k degree homogeneous polynomial V (x) to be negative deﬁnite a necessary condition is that all of the diagonal terms, x2k i , have negative coeﬃcients. As a suﬃcient condition for negativity we can 1. eliminate negative semi-deﬁnite (non-diagonal) terms, 2. Use (6) to split cross terms into a sum of diagonal terms. Then check if the resulting polynomial is negative deﬁnite. This method has already been used in Example 2.5. The following example demonstrates the details. Example 3.1 Consider V (x) = −x41 + ax31 x2 − x21 x22 − x42 + x1 x2 x23 − x43 . We have V ≤ −x41 + ax31 x2 − x42 + x1 x2 x23 − x43 ≤ −x41 − x42 − x43 + |a|( 34 x41 + 14 x42 ) + 14 x41 + 14 x42 + 12 x43 = − 34 (1 − |a|)x41 − 14 (3 − |a|)x42 − 12 x43 . It is obvious that V (x) is negative deﬁnite as |a| < 1. (A simple algebraic transformation, say x31 x2 = (λx1 )3 · (x2 /λ3 ), can show that as |a| = 1, V is still negative deﬁnite.) Now system (2) can be expressed component-wise as x˙i = aiS xS , i = 1, · · · , n. |S|=ki

where S = (s1 , · · · , sn ) and |S| =

n i=1

si ,

xS =

n 8 i=1

xsi i .

Using the above technique, some suﬃcient conditions for (2) to be approximately stable are provided in [11]. Denote by Qi = {|S| = ki | sj (j = i) are even and aiS < 0}. ∂V It represents the set of negative semi-deﬁnite terms in ∂x gi for any LFHD. So i such terms can be skipped when the negativity is tested. Using the above technique, two suﬃcient conditions for approximate stability of the approximate system (2) (same as for (1)) are presented in [11].

Stabilization via Polynomial Lyapunov Function

165

Theorem 3.2 CRDDP( Cross Row Diagonal Dominating Principle) System (2) (equivalently, (1) )is approximately stable at the origin, if there exists an integer m with 2m > max{k1 , · · · , kn }, such that si + 2m − ki −aidi > |aiS | 2m |S|=ki ,S∈Qi (7) s n i , i = 1, · · · , n. + |ajS | 2m j=1,j=i |S|=kj ,S∈Qj A simpler form, which deals with each row independently, is obtained as Corollary 3.3 DDP(Diagonal Dominating Principle) System (2) (equivalently, (1) )is approximately stable at origin if −aidi > |aiS |, i = 1, · · · , n. (8) |S|=ki ,S∈Qi

4 Center Manifold Approach Stabilization is one of the basic and most challenging tasks in control design. The asymptotic stability and stabilization of nonlinear systems has received tremendous attention. The center manifold approach has been implemented to solve the problem. Some special non-linear controls are designed to stabilize some particular control systems. The method used there is basically a case-bycase study [1], [2]. For control systems in canonical form, assume the center manifold has minimum phase, then a “quasi-linear” feedback can be used to stabilize linearly controllable variables. We refer to [3], [7], [4], [7], [8] for minimum phase method and its applications. Our purpose is to apply the techniques developed in Sections 2-3 to the dynamics on center manifold of closed-loop nonlinear control systems to ﬁnd stabilizing controls without a minimum phase assumption. We cite some related results of the theory of center manifold here [5]. Theorem 4.1 Consider the following system . x˙ = Ax + p(x, z), x ∈ Rn , (9) z˙ = Cz + q(x, z), z ∈ Rm , where Reσ(A) < 0, Reσ(C) = 0, p(x, z) and q(x, z) vanish at zero with their ﬁrst derivatives. Then there exists an m dimensional invariant sub-manifold through the origin, described by S = {(x, z) | x = h(z)}, where h(z) satisﬁes

(10)

166

D. Cheng

∂h(z) (Cz + q(h(z), z)) − Ah(z) − p(h(z), z) = 0. ∂z

(11)

Theorem 4.2 The dynamics on the center manifold are z˙ = Cz + q(h(z), z),

z ∈ Rm .

(12)

System (9) is asymptotically stable, stable, unstable, iﬀ system (12) is asymptotically stable, stable, unstable respectively. Theorem 4.3 Assume there exists a smooth function φ(z), such that ∂φ(z) (Cz + q(φ(z), z)) − Aφ(z) − p(φ(z), z) = 0(zk+1 ). ∂z

(13)

φ(z) − h(z) = 0(zk+1 ).

(14)

Then

For convenience, we call Theorem 4.1, Theorem 4.2, and Theorem 4.3 the existence theorem, equivalence theorem, and approximation theorem respectively.

5 Stabilization of Non-minimum Phase Nonlinear Systems Consider an aﬃne nonlinear control system  m ξ˙ = f (ξ) + g (ξ)u , i i i=1  y = h(ξ), y ∈ Rm .

ξ ∈ Rn

(15)

Assume the decoupling matrix is nonsingular and G = Span{gi } is involutive, then the system can be converted into a feedback equivalent normal form as [7] . x˙ = Ax + Bv x ∈ Rs (16) z˙ = q(z, x), z ∈ Rt . Without loss of generality, we assume (A, B) is of the Brunovskey canonical form. That is, the linear part of (16) can be expressed as  x˙ i1 = xi2     .. . i i   x˙ si −1 = xsi   i x˙ si = vi , xi ∈ Rsi , i = 1, · · · , m.

Stabilization via Polynomial Lyapunov Function

Note that

m i=1

167

si = s and s + t = n. It is well known [7] that if system (16) is

of minimum phase, i.e., z˙ = q(z, 0) is asymptotically stable, then the system is stabilizable by linear state feedback. Now if (16) is of non-minimum phase, [11] proposes a systematic way to stabilize it. The method, called designing center manifold, can be roughly described as follows: We look for a set of controls of the following form vi = ai1 xi1 + · · · + aisi xisi − ai1

t

Pji (z),

i = 1, · · · , m.

(17)

j=2

where Pji (z) are polynomials of degree j, aij are chosen such that the linear part is Hurwitz. Now we use  t  i P (z) j=2 j      i i 0 x = φ (z) =  (18)  , i = 1, · · · , m   . .   . 0 to approximate the center manifold, say x = h(z). Using Theorem 4.3, it is easy to see that if ∂φ q(z, φ(z)) = 0(zr+1 ) ∂z

(19)

then the diﬀerence (approximation error) is φ(z) − h(z) = 0(zr+1 ), where φ(z) = (φ1 (z), · · · , φm (z)). Next, we consider the approximate dynamics on center manifold of the closed-loop system. It is expressed as z˙i = qi (z, φ(z)),

i = 1, · · · , t.

(20)

Denote the approximate system of (20) by z˙i = ηi (z),

i = 1, · · · , t.

(21)

If deg(ηi ) = ki are odd, and let k = max{ki }. Then we have the following main result: Theorem 5.1 Assume there exists φ(z) as described above, such that 1. (19) holds; 2. qi (z, φ(z)) − qi (z, φ(z) + 0(zr+1 )) = 0(zki +1 ),

i = 1, · · · , t.

(22)

168

D. Cheng

3. There exists a LFHD V (z) > 0 such that V˙ |(21) < 0. Then the system (16) is stabilizable by control (17). Remark. 1. In fact, condition 3 assures the approximate stability of (20). Then in the light of condition 1, condition 2 assures the asymptotical stability of the true dynamics on center manifold. 2. Of course, condition 3 can be replaced by saying (21) is approximately stable. We give a simple example to describe the stabilizing process. Example 5.2 Consider the following system  x˙ 1 = x2    x˙ = u 2 (23)  = z1 ln(1 + z1 )x1 + 3z12 z22 z ˙ 1    z˙2 = z2 x1 + z1 z2 x1 . We search for a control having the form u = −x1 − 2x2 + P2 (z) + P3 (z), where deg(P2 ) = 2, deg(P3 ) = 3. Then P2 (z) + P3 (z) . x = φ(z) = 0 A simple computation shows that we can choose P (z) = P2 (z) + P3 (z) as P (z) = −3z22 − 2z13 . The approximate dynamics on the center manifold become . z˙1 = −2z15 + 32 z13 z22 + 0(z6 ) z˙2 = −3z23 + 0(z4 ).

(24)

Now the approximation error is ∂φ(z) g(z, φ(z)) = 0(z4 ). ∂z Then a straightforward computation shows that q1 (z, φ(z) + 0(z4 )) − q1 (z, φ(z)) = 0(z6 ), q2 (z, φ(z) + 0(z4 )) − q2 (z, φ(z)) = 0(z5 ). The condition (22) is obviously satisﬁed. Using CRDDP or DDP, Theorem 3.2 or Corollary 3.3 shows that (24) is approximately stable. Now Theorem 5.1 assures that the control u = −x1 − 2x2 − 3z22 − 2z13 stabilizes the system (23).

Stabilization via Polynomial Lyapunov Function

169

6 Generalized Normal Form From previous section one sees that the normal form is essential in the stabilization via center manifold approach. But to get Byrnes-Isidori normal form (16), the system (15) should be regular in the sense that the relative degree vector is well deﬁned and the decoupling matrix is nonsingular on a neighborhood of the origin. A generalized normal form is proposed in [13], based on the so called point relative degree vector. Unlike the relative degree vector, the point relative degree vector is always well deﬁned. Deﬁnition 6.1 1. For system (15) the point relative degree vector (ρ1 , · · · , ρm ) is deﬁned as . Lg Lkf hi (0) = 0, k < ρi − 1, (25) Lg Lfρi −1 h(0) = 0, i = 1, · · · , m. 2. The essential relative degree vector, ρe = (ρe1 , · · · , ρem ), (the essential ep point relative degree vector, ρep = (ρep 1 , · · · , ρm )) , for the state equation of (15) is deﬁned as the largest one of relative degree (resp. point relative degree), e ρ∗ , for all possible auxiliary outputs, which makes the decoupling matrix, W ρ ep (resp. W ρ ), non-singular. That is, ρ∗e =

m

e

e ρ ρ∗e is nonsingular}; i = max{ρ | W

(26)

i=1

and ρ∗ep =

m i=1

ep

ρ∗ep = max{ρep | W ρ i

is nonsingular}.

(27)

For system (15) we give two fundamental assumptions: H1. The decoupling matrix is invertible at the origin; H2. g1 (0), · · · , gm (0) are linearly independent and Span{g(x)} is involutive near the origin. Then we can seek a generalized normal form as 0 1   0 z˙i = A z i + b u +  + pi (z, w)u, z i ∈ Rρi  i i i   αi (z, w)  (28) i = 1, · · · , m   r  w˙ = q(z, w), w ∈ R     yi = z1i , i = 1, · · · , m, where r+

m i=1

ρi = n, αi (z, w) are scalars, pi (z, w) are ρi ×m matrices, q(z, w) is

a r×1 vector ﬁeld, and (Ai , bi ) are Brunovsky canonical form, and pi (0, 0) = 0.

170

D. Cheng

Comparing (28) with (16), the only diﬀerence between them is that in (16) gi = (0 · · · 0 di (x, z))T , i = 1, · · · , m, and in (28) there exist pi (z, w) which are higher degree input channels. Then we can prove the following: Proposition 6.2 Consider system (15). 1. Assume H1 and H2 and if the system has point relative degree vector ρp = (ρ1 , · · · , ρm ), then there exists a local coordinate frame such that the system can be converted into the generalized normal form (28). 2. Assume H2 and if system (15) has essential point relative degree vector ρep = (ρ1 , · · · , ρm ), then there exists a local coordinate frame such that the system can be converted into the generalized normal state form as the state equation of (28). Remark. Unlike the classical normal form, the essential point relative degree vector is straightforwardly computable, and then so is the generalized normal form. Now for the generalized normal form we have some stabilization results, which are parallel to their counterparts for the classical normal form [13]. Observer system (28), for the case of “minimum phase” we assume H3. αi (0, w)pi (0, w) = 0, i = 1, · · · , m. Then we have Proposition 6.3 Assume H3. For the generalized normal state form (state equation of (28)) if the pseudo-zero dynamics w˙ = q(0, w)

(29)

are asymptotically stable at zero, then (6) is stabilizable by a pseudo-linear state feedback control. For the case of non-minimum phase, using the center manifold approach we have the following Theorem 6.4 For system (28), assume there exist m homogeneous quadratic functions φ(w) = (φ1 (w), · · · , φm (w)) and m homogeneous cubic functions ψ(w) = (ψ1 (w), · · · , ψm (w)) such that the following hold: 1. There exists an integer s > 3, such that 



  L.D. Lq(φ+ψ,0, · · · , 0,w) (φ + ψ) ≥ s; !" #

(30)

ρ−m





  L.D. p(φ + ψ, 0, · · · , 0, w)α(φ + ψ, 0, · · · , 0, w) ≥ s. !" # !" #

ρ −m

ρ −m

(31)

Stabilization via Polynomial Lyapunov Function

2.



171



  L.D. qi (φ + ψ, 0, · · · , 0, w) = Li , !" #

i = 1, · · · , r.

(32)

ρ −m

3. w˙ = q(φ + ψ, 0, · · · , 0, w) !" #

(33)

ρ −m

is L = (L1 , · · · , Lr ) approximately asymptotically stable at the origin, and 4. qi ((φ + ψ + 0(ws ), 0(ws ), · · · , 0(ws ), w) !" #

ρ −m

= qi (φ + ψ, 0, · · · , 0, w) + 0(wLi +1 ), !" #

i = 1, · · · , r.

(34)

ρ −m

Then the overall system (28) is state feedback stabilizable. Remark. 1. In the above theorem we use only quadratic and cubic functions for control design. In fact, higher degree polynomials may also be used. It is not diﬃcult to generalize the above result to the more general case. 2. Again, the LFHD can be used to test the approximate stability of (33).

7 Other Types of Center Manifolds In either classical or generalized normal form, the state equations of linearly uncontrollable variables (z in (16) or w in (28) ) are the key for stabilization. ∂q If their linear parts, say ∂z |(0,0) in (16), have positive real part eigenvalues, the system is not stabilizable. So if the system is stabilizable we have to assume that only the negative and zero real part eigenvalues are allowed. In the above discussion, we didn’t consider negative eigenvalues in the linear part of nonlinear state equations. But it is not diﬃcult to include them. In fact, [11] considered systems with both negative and zero eigenvalues. Next, we assume the linear part of the nonlinear state equations has only zero real part eigenvalues. Then this part of the variables will appear in the center manifold. We simply denote it by C, which means the linear part is Cz. Now according to the form of C, we say that the system has diﬀerent types of center manifolds. The types are deﬁned as follows: Case 1. Zero center: C = 0 Case 2. Multi-fold zero center:

172

D. Cheng



01 0 0 0 1 · · ·   C = 0 0 0  0 0 0 00 0

··· 0 .. .

0



   . 0   ··· 1 ··· 0

(Of course, we may have more than one such Jordan Block.) Case 3. Oscillatory center: 0 a C= . −a 0 In fact, in the above Sections 4-6 only case 1 is considered. The case when the center manifold is of the type of 01 C= 00 was discussed in [9]. It was proved that we are not able to ﬁnd a LFHD to stabilize the dynamics of this kind of center manifold approximately. A generalized LFHD was introduced to stabilize such systems. The basic idea is to add some cross terms to LFHD and keep its positivity. When the multiplicity of zero is greater than two, we still don’t know how to stabilize it systematically. As for the case of an oscillatory center, a standard way is to convert it into a normal dynamical form, which has a Taylor expansion with the lowest degree terms of degree greater than or equal to 3. (We refer to [6] for normal form of dynamics. Do not confuse it with the normal forms of control systems discussed is Sections 5-6.) Some recent results were presented in [10]. The method of LFHD can also be used for estimating the region of attraction of equilibrium points. It can also be used for time-varying systems [14].

8 Conclusion This paper summarized the method of stabilization of nonlinear aﬃne systems via designed center manifold. A few key points in this method are listed as follows: •

•

Consider the normal form of a control system (16), the ﬁrst variables in each integral chain don’t aﬀect the approximation degree of the center manifold. So they can be used as nominate controls to design the required center manifold. Lyapunov function with homogeneous derivative (LFHD) can assure the approximate stability of a dynamic system. This is the key of the method,

Stabilization via Polynomial Lyapunov Function

• •

173

because it is very diﬃcult, if not impossible, to get the precise dynamics on center manifold, but the approximation theorem provides an easy way to get its approximate dynamics. Then the approximate stability of the approximate dynamics assures the stability of the true dynamics on center manifolds. Certain easily veriﬁable conditions, as CRDDP and DDP etc., were developed to check the negativity of the homogeneous derivative of LFHD. The method depends on the normal form of nonlinear control systems. To apply it to more general systems the generalized normal form of nonlinear control systems was introduced. The generalized normal form covers almost all smooth aﬃne nonlinear control systems.

The method has been extended to treat the cases of multi-zero and oscillatory centers. But only some special simple cases can be properly handled [9], [10]. The general cases remain for further investigation.

References 1. Aeyels, D. (1985), Stabilization of a class of non-linear systems by a smooth feedback control, Sys. Contr. Lett. Vol. 5, 289–294. 2. Behtash, S., Dastry D. (1988), Stabilization of non-linear systems with uncontrollable linearization, IEEE Trans. Aut. Contr., Vol. 33, No. 6, 585–590. 3. Byrnes, C.I., Isidori A. (1984), A frequency domain philosophy for non-linear systems, IEEE Conf. Dec. Contr. Vol. 23, 1569–1573. 4. Byrnes, C.I., Isidori A., and Willems J.C. (1991), Passivity, feedback equivalence, and the global stabilization of minimum phase nonlinear systems, IEEE Trans. Aut. Contr. Vol. 36, No. 11, 1228–1240. 5. Carr,J. (1981), Applications of Center Manifold Theory, Springer-Verlag. 6. Guchenheimer, J., Holmes, P. (1983), Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, Springer-Verlag. 7. Isidori,A. (1995), Non-linear Control Systems, 3nd ed., Springer-Verlag. 8. Nijmeijer, H., van der Schaft A.J. (1990), Non-linear Dynamical Control Systems, Springer-Verlag. 9. Cheng,D. (2000), Stabilization of a class of nonlinear non-minimum phase systems, Asian J. Control, Vol. 2, No. 2, 132–139. 10. Cheng,D., Spurgeon S. (2000), Stabilization of nonlinear systems with oscillatory center, Proc. 19th CCC, Hong Kong, Vol. 1, 91–96. 11. Cheng, D., Martin C. (2001), Stabilization of nonlinear systems via designed center manifold, IEEE Trans. Aut. Contr., Vol. 46, No. 9, 1372–1383. 12. Cheng, D. (2002), Matrix and Polynomial Approach to Dynamic Control Systems, Science Press, Beijing. 13. Cheng, D., Zhang L. (2003), Generalized Normal Form and Stabilization of Nonlinear Systems, Internal J. Control, (to appear) 14. Dong, Y., Cheng D., Qin H., New Applications of Lyapunov Function with Homogeneous Derivative, IEE Proc. Contr. Theory Appl., (accepted) 15. Willems, J.L. (1970), Stability Theory of Dynamical Systems, John Wiley & Sons, New York.

Simulating a Motorcycle Driver Ruggero Frezza and Alessandro Beghi Department of Information Engineering, University of Padova, Italy, {frezza,beghi}@dei.unipd.it

Summary. Controlling a riderless bicycle or motorcycle is a challenging problem because the dynamics are nonlinear and non-minimum phase. Further diﬃculties are introduced if one desires to decouple the control of the longitudinal and lateral dynamics. In this paper, a control strategy is proposed for driving a motorcycle along a lane, tracking a speed proﬁle given as a function of the arc length of the mid lane.

1 Introduction Driving a bicycle or a motorcycle is a diﬃcult task which requires repeated learning trials and a number of bruises. A well known idiom, however, goes like: Once learnt, one never forgets how to ride a bicycle. Underneath this apparently trivial guidance task, there lies a complex control problem, that is of interest both for the physics it involves and the applications. Our speciﬁc motivation in dealing with the control of bicycles and motorcycles comes from the world of virtual prototyping. Nowadays, all major car manufacturers and automotive suppliers employ virtual prototyping softwares to cut time and costs in the development of new models. Virtual vehicles are described by highly detailed mechanical models where all the vehicle components (chassis, suspensions, tires, powertrain, etc.) are represented so that the software can accurately reproduce the very complex behaviour of real vehicles. To be used for testing purposes, the simulation environment has to provide both a means of describing the interaction between the vehicle and the environment (course of road, weather conditions, etc.) and a control system that emulates the driver, so that appropriate test maneuvers can be executed. Commercial software systems are available to accomplish all of these tasks (see e.g. [1], [2]). The same software systems can be employed in the ﬁeld of motorcycle manufacturing as far as the modeling part is concerned. However, a satisfactory virtual motorcycle driver that can be used to test the virtual W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 175–186, 2003.

c Springer-Verlag Berlin Heidelberg 2003

176

R. Frezza and A. Beghi

vehicle on the desired maneuvers has not yet been developed. This is due to the fact that driving a motorcycle is a more demanding task than driving a car, and the diﬃculty obviously lies in keeping the motorcycle upright while following a desired path. These two goals are often in contrast: In order to lean the motorcycle in the turn, one has ﬁrst to steer it the opposite way to generate a centrifugal force that pushes the motorcycle in the right direction. Considering the problem from a control theoretic standpoint, we can say that the motorcycle exhibits a non-minimum phase behaviour. As is known, control design for nonlinear, non-minimum phase systems is a current research topic in the scientiﬁc community (see for instance the recent papers [3, 4, 5, 6, 7].) The study of guidance systems for bicycles and motorcycles has already been considered in the literature. In particular, a very nice exposition on the physics of driving a bicycle is given in [8]. In the control literature, the two approaches that are more closely related to what we propose in this paper are due to Von Wissel [9] and Getz [10] who both wrote their Ph.D. thesis on controlling a riderless bicycle. Getz’s work [10] is about the trajectory tracking problem. The motorcycle is controlled to follow a curve in the plane with a given time parametrization. Von Wissel [9] proposes a solution to the path planning problem which consists in ﬁnding a feasible path avoiding ﬁxed obstacles. The solution consists in the choice of optimal precomputed maneuvers that satisfy the requirements and it is computed by dynamic programming techniques. The control problem we consider here is diﬀerent. The motorcycle must be steered inside a lane of given width, and a desired forward velocity is also assigned as a function of the arc length of the midlane, so that the problem cannot be stated in the standard terms of trajectory tracking. The proposed control strategy is a kinematic path following control law which decouples lateral from longitudinal control. The roll angle is controlled by nonlinear state feedback linearization and a simple proportional, integral and derivative law is adopted for the control of longitudinal velocity. Other marginal control tasks have also been solved in particular situations where the dynamics neglected in the model become relevant and aﬀect the behaviour of the controller. One such situation happens in heavy braking and during gear shifting. The paper is organized as follows. In Section 2 the adopted motorcycle model is presented, and an analysis of the existing approaches to the problem motivates the need for a new control strategy. In Section 3, the concept of feasible trajectories and the proposed control scheme are introduced. Simulation results are given in Section 4, and some concluding remarks are drawn in Section 5.

2 Model and Traditional Control Approaches Characteristically, in control, the ﬁrst tradeoﬀ is in modeling. The model of the motorcycle should be rich enough to capture the most relevant dynamical

Simulating a Motorcycle Driver

177

features of the system for control purposes, but simple enough to be able to derive a control law. In this paper, the model is that of a kinematic cart with an inverted pendulum on top, with its mass concentrated at distance c in the forward direction from the rear axle and at height p from the ground (see Fig. 1). It is, in principle, the same model used by [9] and [10], even if their control goals were diﬀerent.

Fig. 1. The bicycle model.

There are restrictions on how wheeled vehicles can move on a plane, in that they cannot slide sideways. Sideways motion can, however, be achieved by maneuvering with combinations of steering actions and forward/reverse translations. Constraints in the velocity that cannot be integrated into constraints in the conﬁguration of the vehicle are called “non-holonomic” and there is a large literature dealing with the representation and control of such systems [12, 13, 14, 15, 16]. Let X, Y and θ be the position of the contact point of the rear wheel on the ground and the orientation of the bicycle with respect to an inertial reference frame, then, the kinematic model of the bicycle is   X˙ = cos(θ)v      Y˙ = sin(θ)v   tan(ϕ) . θ˙ = v = σv (1) b cos(α)     σ˙ = ν     v˙ = u

178

R. Frezza and A. Beghi

where ϕ and α are the steering and the roll angle respectively and ν and u are the control actions. The roll angle satisﬁes p¨ α = g sin(α) + (1 + pσ sin(α)) cos(α)σv 2 + c cos(α)(σv ˙ + σ v). ˙

(2)

Now consider a road described by its width l and a diﬀerentiable path Γ parameterized by s Γ = {(X(s), Y (s)) s ∈ [s0 , s1 ] ⊂ R} ⊂ C 1 (R2 )

(3)

in the inertial reference frame. The goal is to drive the motorcycle along the road at a velocity vr (s) assigned as a function of the arc length of Γ . Clearly, the problem is diﬀerent than tracking the trajectory Γ (s(t)) where s solves s˙ = vr (s) s(0) = 0

(4)

even if, in case of no error, the vehicle trajectories are coincident. The bicycle trajectory tracking problem has been dealt with by N. Getz [17] who applied internal equilibrium control to solve it. In the approach proposed by Getz, an external control law for trajectory tracking in absence of roll dynamics is ﬁrst determined. Then, an internal control law that linearizes by state feedback and stabilizes the roll dynamics about a desired trajectory αd (t) is found. Afterwards, the internal equilibrium roll angle αe is computed or estimated by solving the roll dynamics (2) imposing α ¨ = 0 and α˙ = 0 and the current value of the external tracking controller. Finally, the internal equilibrium controller is determined by computing an internal control law that tracks the internal equilibrium roll angle along the trajectories that result by the application of the external control law. The problem tackled in this paper is diﬀerent than trajectory tracking and Getz’s controller may not be applied. In the remaining part of this section we derive a new formulation of the model that allows for a better description of the lateral control task. Some sort of perceptive referenced frame as described in Kang et al. [18, 19] is implicit in our derivation. As a ﬁrst step, we change reference frame and write the evolution of the reference path Γ in body frame, i.e., as seen by an observer riding the bicycle. The body frame coordinates (x, y) of a point (X, Y ) are * + * + * + * + x cos(θ) sin(θ) X X0 = − (5) y − sin(θ) cos(θ) Y Y0 where (X0 , Y0 ) is the current position in the inertial frame of the body frame. The point of coordinates (x, y) moves, therefore, with respect to the body frame, with velocity * + * +* + * + x˙ 0 σv x v = − (6) y˙ −σv 0 y 0

Simulating a Motorcycle Driver

179

where the velocity component about the y axis of the body frame is zero due to the non holonomic constraint. Assume that locally, in a neighborhood of the body frame, the reference path Γ may be represented as a function y = γ(x, t) x ∈ [0, L].

(7)

Clearly, in the moving frame, the representation also evolves in time. The local evolution of the reference path Γ as perceived by an observer sitting on the vehicle is obtained combining (7) with (6): ∂γ ∂γ (x, t) = (−σ(t)x + (x, t)(1 − σ(t)γ(x, t)))v(t). ∂t ∂x

(8)

The lateral control goal of following the path Γ consists in choosing bounded and continuous controls such that the trajectory of the origin of the moving frame covers the contour Γ , i.e.: γ(0, t) = 0 ∀t;

{(x(t), y(t)), t ∈ R+ } = Γ.

(9)

Since v appears in each term of the right hand side and s˙ = v, one may write the evolution of γ in terms of the arc length s, decoupling de facto the lateral control task from the longitudinal one. . Now, as in [20], let us introduce the moments ξi (t) = ∂ i−1 γ/∂ti−1 (0, t) for i = 1, 2, . . . , from (8) we can write  ˙  ξ1 = ξ2 (1 − σξ1 )v (10) ξ˙2 = (ξ3 − σ(1 + ξ22 + ξ1 ξ3 ))v .  ˙ ξ3 (t) = . . . This inﬁnite set of ODE’s is locally equivalent, under appropriate hypotheses on the regularity of Γ , to the PDE (8). Observe that if the reference path can be generated by a ﬁnite dimensional system of diﬀerential equations, then the inﬁnite set of ODE’s closes. For instance, if the reference path is composed by a combination of line segments, and circular arcs, the equations close with ξ˙3 (t) = Aj δ(t − tj ) where the δ are Dirac’s δ-functions and the tj are the instants at which the path changes from a line segment to a circular arc or vice versa. The ﬁrst two moments ξ1 and ξ2 code the pose of the bicycle with respect to the reference path. Successive moments are related to the shape of the reference path. The lateral control task consists in regulating to zero the ﬁrst moment ξ1 (t), however, if one imposes ξ1 (t) = 0 for all t ≥ 0, one sees immediately that for v(t) = 0, it is necessary that ξ2 (t) = 0. If one, then, imposes ξ2 (t) = 0 for all t ≥ 0, one obtains the exact tracking controller σ(t) = ξ3 (t).

(11)

180

R. Frezza and A. Beghi

3 Exact Tracking Control and Feasible Trajectories In [20], the stability of a particular MPC for path following in the absence of roll dynamics was shown. There, the strategy consists in choosing on line an optimal feasible trajectory for the vehicle that, in some sense, ﬁts best the local representation of the reference path, as, for instance, a polynomial of order n γc (x, t) =

n

ai (t)xi

(12)

i=2

satisfying the boundary conditions  *

+

* + γc (0, t) 0 = ∂γc 0 ∂x (0, t)

   



γc (L, t) ∂γc  ∂x (L, t)  .. .

∂ n−2 γc ∂xn−2



  =  

γ(L, t)



∂γ  ∂x (L, t) 

.. .

. 

(13)

∂ n−2 γ ∂xn−2

At time t, the control action is chosen to achieve exact tracking of the current feasible trajectory ∂ 2 γc (0, t) = 2a2 (t) ∂x2 = 2a2 (γ(L, t), . . . , ∂ n−2 γ/∂xn−2 (L, t))

σ(t) =

(14)

which is a linear output feedback. The main result of [20] is that the linearization of the closed loop system governing the evolution of the ﬁrst two moments ξ1 and ξ2 is +* + * + * 0 1 ξ1 ξ˙1 = v (15) (n−1) 2 ξ2 − (n−1)(n) − ξ −2 ξ˙2 2 3 L L which is stable. While in the absence of roll dynamics the bicycle can follow any trajectory with smooth curvature bounded in absolute value by σmax = 1/b tan(φmax ), in presence of the roll dynamics this is not true anymore. As a matter of fact, substituting the control law (11) in the roll dynamics (2), the bicycle will most likely fall down. A question that naturally arises is then: Are there trajectories which may be tracked exactly maintaining the roll angle in a region where the tires are working, for example in the interval I = (−2π/3, 2π/3)? Deﬁnition 1. A trajectory γ(t) with t ∈ [0, T ] which may be exactly tracked kinematically is called feasible if along γ the roll dynamics of the bicycle admit a solution with α ∈ I for all t ∈ [0, T ].

Simulating a Motorcycle Driver

181

Do there exist feasible trajectories? The answer is clearly aﬃrmative. Straight lines σ(t) = 0 and circular arcs σ(t) = k are feasible trajectories as long as |k| < σmax because the roll dynamics admit an equilibrium solution in I. The set of feasible trajectories is, however, much richer and its characterization is an interesting vehicle dynamics problem since the ease of handling of a bicycle is related to the set of feasible trajectories. The proposed solution to the lateral control problem is reminiscent of the MPC paradigm [21]. An on-line ﬁnite horizon optimal control problem is solved by selecting, among all feasible trajectories up to the time horizon T , the one that is closest to the reference path. The bicycle is controlled in order to follow for one time step the current optimal feasible trajectory and the whole procedure is iterated. The bicycle will follow the envelope of the locally optimal feasible trajectories. One certainly would like that the resulting bicycle path be “close” in some sense to the reference path Γ which, in general, is not feasible. This is a curve ﬁtting problem. Demonstrating convergence of the control law requires a ﬁrst step showing that given a road of arbitrary width l > 0, with Γ satisfying appropriate bounds on the curvature and on the jerk, there exists a feasible trajectory fully contained within the road margins. However, characterizing feasible trajectories is a very diﬃcult problem, and the design of a control action ν(t) = σ(t) ˙ so that the roll angle stays bounded is still an unanswered question. A simpler approach is that of solving the inverse roll dynamics for σ(t) given a trajectory α(t) that satisﬁes the constraints on α ∈ I. Because of the inertia of the bicycle and its rider, the roll angle typically does not show aggressive dynamics. Therefore, during a short time interval, the roll angle trajectories may be approximated well by smooth functions such as cubic polynomials α(t) = α0 (T − t)3 + α1 (T − t)2 t + α2 (T − t)t2 + α3 t3 . The coeﬃcients αi for i = 0, 1, 2, 3 have a clear meaning:  α0 = α(0)  T3    α1 = T12 dα dt (0) + 3α0 . 1 dα  α2 = T 2 dt (T ) − 3α3    α3 = T13 α(T )

(16)

(17)

The ﬁrst two coeﬃcients are clearly determined by the current state of the bicycle while the remaining two, α2 and α3 are considered optimization parameters. Physical constraints on the roll angle α and its time derivative are easily considered. The problem now has been transformed in that of ﬁnding, among the ∞2 roll angle trajectories parameterized by α2 and α3 the one that corresponds to a feasible trajectory that best ﬁts the reference path. Due to the bounds

182

R. Frezza and A. Beghi

for physically admissible roll angle and roll rate, the search for the optimal trajectory is in a compact set. The adopted search algorithm implements an adaptive scheme to automatically reﬁne the grid and gives good computational performances.

4 Results The developed virtual driver has been used to test a number of diﬀerent motorcycle models in the ADAMS [1] environment, ranging from highperformance racing motorcycles to scooters and bicycles. The set of test maneuvers has been chosen to be rich enough so that the virtual vehicles have been requested to operate in critical situations, such as heavy braking in the middle of a curve. In all the situations the virtual motorcycle driver behaved satisfactorily, succeeding in preserving stability and completing the maneuver. We report here the time evolution of some of the most relevant variables for the maneuver corresponding to driving on the track of Figure 2 at a constant speed of 30 m/s. The chicane consists of straight segments connected by curves with radius equal to 125 m.

Fig. 2. Track used in the simulation. The dot at the beginning of the chicane represents the initial position of the motorcycle, and the arrow shows the direction in which the track is covered.

Simulating a Motorcycle Driver

183

The roll angle α and the steering angle ϕ (i.e., the control input) are reported in Figure 3 and Figure 4, respectively. In particular, in Figure 4 one can clearly see the strong countersteering actions at each curve entrance and exit points. Observe that the steering angle ϕ for this maneuver is the only control input, since the velocity must be kept constant. The small irregularities in the steering angle ϕ(t) are due to the eﬀect of tire slip, which is not modelled but that is taken care of by the overall controller architecture. The motorcycle lateral acceleration is shown in Figure 5. The distance of the motorcycle from the mid lane is shown in Figure 6. It can be seen that the maximum error is on the order of 0.6 m, which is considered a good result for the given maneuver.

45.0

35.0

25.0

Angle (deg)

15.0

5.0 0.0 -5.0

-15.0

-25.0

-35.0

-45.0 0.0

10.0

20.0 Time (sec)

30.0

40.0

Fig. 3. Roll angle α(t).

5 Conclusions A new approach to the control of a riderless motorcycle has been proposed, with the aim of designing a virtual motorcycle driver to be used in connection with virtual prototyping software. At the heart of the approach, that follows the MPC philosophy, lies the idea of ensuring that the designed trajectories, that approximate the assigned reference trajectory, are feasible for the motorcycle. The choice of the “best” feasible approximating trajectory

R. Frezza and A. Beghi 0.6

0.5

0.4

0.3

Angle (deg)

0.2

0.1

0.0

-0.1

-0.2

-0.3

-0.4 0.0

10.0

20.0 Time (sec)

30.0

40.0

30.0

40.0

Fig. 4. Steering angle ϕ(t). 10.0

5.0

Acceleration (meter/sec**2)

184

0.0

-5.0

-10.0 0.0

10.0

20.0 Time (sec)

Fig. 5. Lateral acceleration.

Simulating a Motorcycle Driver

185

1.0

Length (meter)

0.5

0.0

-0.5

-1.0 0.0

10.0

20.0 Time (sec)

30.0

40.0

Fig. 6. Distance from mid lane.

is made by searching among an appropriately parameterized set of curves. The proposed controller has been extensively tested in a virtual prototyping environment, showing good performance, even with respect to robustness. A number of theoretical issues are still open. In particular, a proof of the convergence of the trajectory generating MPC-like scheme to a feasible trajectory is still lacking, and is the object of ongoing research. Acknowledgments. The authors would like to acknowledge the support of Mechanical Dynamics, Inc, and Dr. Ing. Diego Minen and Luca Gasbarro for their invaluable contributions to their understanding of vehicle dynamics.

References 1. Anonymous (1987) ADAMS User’s Guide. Mechanical Dynamics, Inc. 2. Anonymous (1997) VEDYNA User’s Guide. TESIS Dynaware GmbH, M¨ unchen 3. Hongchao Zhao, Degang Chen (1998), IEEE Transactions on Automatic Control, 43(8):1170–1174 4. Huang, J (2000) IEEE Transactions on Automatic Control, 45(3):542–546 5. Devasia, S., Paden, B (1998) IEEE Transactions on Automatic Control, 43(2):283–288

186

R. Frezza and A. Beghi

6. Guemghar,K., Srinivasan, B., Mullhaupt, P., Bonvin, D. (2002) Predictive control of fast unstable and nonminimum-phase nonlinear systems. In: Proceedings of the 2002 American Control Conference, pp. 4764–9 7. Ju Il Lee, In Joong Ha, (2002) IEEE Transactions on Automatic Control.,47(9): 1480–6 8. Fajans, J. (2000) Am. J. Phys., 68(t):654–659 9. von Wissel, D (1996) DAE control of dynamical systems: example of a riderless ´ bicycle. Ph.D. Thesis, Ecole Nationale Sup´erieure des Mines de Paris 10. Getz, N (1995) Dynamic Inversion of Nonlinear Maps with Applications to Nonlinear Control and Robotics. Ph.D. thesis, University of California, Berkely 11. Getz, N (1994) Control of balance for a nonlinear nonhlonomic non-minimum phase model of a bicycle. In: Proceedings of the American Control Conference, Baltimore 12. Rouchon P., Fliess M., L`evine J., Martin P. (1993) Flatness and motion planning: The car with n trailers. In: Proceedings of the European Control Conference, ECC’93,pp. 1518–1522 13. Fliess M., L`evine J., Martin P., Rouchon P. (1995) Design of trajectory stabilizing feedback for driftless ﬂat systems. In: Proceedings of the European Control Conference, ECC’95, pp. 1882–1887 14. Murray R., Sastry S., IEEE Trans. Automatic Control 38(5):700–716 15. Murray R., Li Z., Sastry S. A Mathematical Introduction to Robotic Manipulation. CRC Press Inc. 16. Samson C., Le Borgne M., Espiau B. (1991) Robot Control: The Task Function Approach. Oxford Engineering Science Series, Clarendon Press 17. (1995) Internal equilibrium control of a bicycle. In: Proceedings of the IEEE Conf. on Decision and Control, pp. 4285–4287 18. Kang W., Xi N. (1999) Non-time referenced tracking control with application in unmanned vehicle. In: Proceedings IFAC World Congress of Automatic Control, Beijing, China 19. Kang W., Xi N., Sparks A. (2000) Theory and Applications of formation control in a perceptive referenced frame. In: proceedings of the IEEE Conf. on Decision and Control 20. Frezza R. (1999) Path following for air vehicles in coordinated ﬂight. In: Proceedings of the 1999 IEEE/ASME Conference on Advanced Intelligent mechatronics, Atlanta, Georgia, pp. 884–889 21. Mayne, D.Q., Rawlings, J.B., Rao, C.V., Scokaert, P.O.M. (2000) Automatica, 36(6)789–814

The Convergence of the Minimum Energy Estimator Arthur J. Krener Department of Mathematics, University of California, Davis, CA 95616-8633, USA, [email protected]

Summary. We show that under suitable hypothesis that the minimum energy estimate of the state of a partially observed dynamical system converges to the true state. The main assumption is that the system is uniformly observable for any input. Keywords: Nonlinear Observer, State Estimation, Nonlinear Filtering, Minimum Energy Estimation, High Gain Observers, Extended Kalman Filter, Uniformly Observable for Any Input.

1 Introduction We consider the problem of estimating the current state x(t) ∈ Rn of a nonlinear system x˙ = f (x, u) y = h(x, u) x(0) = x0

(1)

from the past controls u(s) ∈ U ⊂ Rm , 0 ≤ s ≤ t, past observations y(s) ∈ Rp , 0 ≤ s ≤ t and some information about the initial condition x0 . The functions f, h are assumed to be known. We assume that f, h are Lipschitz continuous on Rn and satisfy linear growth conditions |f (x, u) − f (z, u)| ≤ L|x − z| |h(x, u) − h(z, u)| ≤ L|x − z| |f (x, u)| ≤ L(1 + |x|) |h(x, u)| ≤ L(1 + |x|)

(2)

for some L > 0 and all x ∈ Rn and u ∈ U . We also assume that u(s), 0 ≤ s ≤ t is piecewise continuous. Piecewise continuous means continuous from the left

Research supported in part by NSF DMS-0204390 and AFOSR F49620-01-1-0202.

W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 187–208, 2003.

c Springer-Verlag Berlin Heidelberg 2003

188

A.J. Krener

with limits from the right (collor) and with a ﬁnite number of discontinuities in any bounded interval. The symbol | · | denotes the Euclidean norm. The equations (1) model a real system which probably operates over some compact subset of Rn . Therefore we may only need (2) to hold on this compact set as we may be able to extend f, h so that (2) holds on all of Rn . To construct an estimator, we follow an approach introduced by Mortenson [9] and reﬁned by Hijab [5], [6]. To account for possible inaccuracies in the model (1), we add deterministic but unknown noises , x˙ = f (x, u) + g(x)w y = h(x, u) + k(x)v

(3)

where w(t) ∈ Rl , v(t) ∈ Rp are L2 [0, ∞) functions. The driving noise, w(t), represents modeling errors in f and other possible errors in the dynamics. The observation noise, v(t), represents modeling errors in h and other possible errors in the observations. We assume that |g(x) − g(z)| ≤ L|x − z| |k(x) − k(z)| ≤ L|x − z| |g(x)| ≤L |k(x)| ≤ L.

(4)

Note that g(x), k(x) are matrices so |g(x)|, |k(x)| denote the induced Euclidean matrix norms. Deﬁne Γ (x) = g(x)g (x) R(x) = k(x)k (x) and assume that there exist positive constants m1 , m2 such that for all x ∈ Rn , m1 I p×p ≤ R(x) ≤ m2 I p×p .

(5)

In particular this implies that k(x) and R(x) are invertible for all x. The initial condition x0 of (1) is also unknown and viewed as another noise. We are given a function Q0 (x0 ) ≥ 0 which is a measure of the minimal amount of ”energy” in the past that it would take to put the system in state x0 at time 0. We shall assume that Q0 is Lipschitz continuous on every compact subset of Rn . Given the output y(s), 0 ≤ s ≤ t, we deﬁne the minimum discounted ”energy” necessary to reach the state x at time t as 6 1 t −α(t−s) Q(x, t) = inf e−αt Q0 (z(0)) + e |w(s)|2 + |v(s)|2 ds (6) 2 0 where the inﬁmum is over all triples w(·), v(·), z(·) satisfying

The Convergence of the Minimum Energy Estimator

z(s) ˙ = f (z(s), u(s)) + g(z(s))w(s) y(s) = h(z(s), u(s)) + k(z(s))v(s) z(t) = x.

189

(7)

The discount rate is α ≥ 0. Notice that Q(x, t) depends on the past control u(s), 0 ≤ s ≤ t and past output y(s), 0 ≤ s ≤ t. A minimum energy estimate x ˆ(t) of x(t) is a state of minimum discounted energy given the system (3), the initial energy Q0 (z) and the observations y(s), 0 ≤ s ≤ t, x ˆ(t) ∈ arg min Q(x, t). x

(8)

Of course the minimum need not be unique but we assume that there is a piecewise continuous selection x ˆ(t). Clearly Q satisﬁes Q(x, 0) = Q0 (x).

(9)

In the next section we shall show that Q(x, t) is locally Lipschitz continuous and it satisﬁes, in the viscosity sense, the Hamilton Jacobi PDE 0 = αQ(x, t) + Qt (x, t) + Qx (x, t)f (x, u(t)) 1 1 + |Qx (x, t) |2Γ − |y(t) − h(x, u(t))|2R−1 2 2

(10)

where the subscripts x, t, xi , etc. denote partial derivatives and |Qx (x, t)|2Γ = Qx (x, t)Γ (x)Qx (x, t)

|y(t) − h(x, u(t))|2R−1 = (y(t) − h(x, u(t))) R−1 (x)(y(t) − h(x, u(t))). To simplify the notation we have suppressed the arguments of Γ, R−1 on the left but they should be clear from context. In the next section we introduce the concept of a viscosity solution to the Hamilton Jacobi PDE (10) and show that Q(x, t) deﬁned by (6) is one. Section 3 is devoted to the properties of smooth solutions to (10) and its relationship with the extended Kalman ﬁlter [4]. The principal result of this paper is presented in Section 4, that, under suitable hypothesis, any piecewise continuous selection of (8) globally converges to the corresponding trajectory of the noise free system (1) and this convergence is exponential if α > 0. We close with some remarks.

2 Viscosity Solutions The following is a slight modiﬁcation of the standard deﬁnition [2]. Deﬁnition 1. A viscosity solution of the partial diﬀerential equation (10) is a continuous function Q(x, t) which is Lipschitz continuous with respect to x on every compact subset of Rn+1 and such that for each x ∈ Rn , t > 0 the following conditions hold.

190

A.J. Krener

1. If Φ(ξ, τ ) is any C ∞ function such that for ξ, τ near x, t Φ(x, t) − Q(x, t) ≤ e−α(t−τ ) (Φ(ξ, τ ) − Q(ξ, τ )) . then 0 ≥ αΦ(x, t) + Φt (x, t) + Φx (x, t)f (x, u(t)) 1 1 + |Φx (x, t)|2Γ − |y(t) − h(x, u(t))|2R−1 . 2 2 2. If Φ(ξ, τ ) is any C ∞ function such that for ξ, τ near x, t Φ(x, t) − Q(x, t) ≥ e−α(t−τ ) (Φ(ξ, τ ) − Q(ξ, τ )) . then 0 ≤ αΦ(x, t) + Φt (x, t) + Φx (x, t)f (x, u(t)) 1 1 + |Φx (x, t)|2Γ − |y(t) − h(x, u(t))|2R−1 . 2 2 Theorem 1. The function Q(x, t) deﬁned by (6) is a viscosity solution of the Hamilton Jacobi PDE (10) and it satisﬁes the initial condition (9). Proof. Clearly the initial condition is satisﬁed and Q(·, 0) is Lipschitz continuous with respect to x on compact subsets of Rn . We start by showing that Q(·, t) is Lipschitz continuous with respect to x on compacta. Let K be a compact subset of Rn , x ∈ K, T > 0 and 0 ≤ t ≤ T . Now 6 1 t −α(t−s) −αt 0 2 Q(x, t) ≤ e Q (z(0)) + e |y(s) − h(z(s), u(s))|R−1 ds (11) 2 0 where z˙ = f (z, u) z(t) = x. By standard arguments, z(s), 0 ≤ s ≤ t is a continuous function of x ∈ K and the right side of (11) is a continuous functional of z(s), 0 ≤ s ≤ t. Hence the composition is bounded on the compact set K and there exists c large enough so that K ⊂ {x : Q(x, t) ≤ c for all 0 ≤ t ≤ T } . Fix x ∈ K and t ∈ [0, T ], given > 0 we know that there exists w(s) such that Q(x, t) + ≥ e−αt Q0 (z(0)) (12) 6 t 1 e−α(t−s) |w(s)|2 + |y(s) − h(z(s), u(s))|2R−1 ds + 2 0 where

The Convergence of the Minimum Energy Estimator

191

z˙ = f (z, u) + g(z)w z(t) = x. Now 6

t

6

t

eαs |w(s)|2 ds 6 t e−α(t−s) |w(s)|2 ds ≤ eαt

|w(s)|2 ds ≤

0

0

0

≤ 2eαt (c + ). Using the Cauchy Schwarz inequality we also have 6

t

6 |w(s)| ds ≤

1 ds

0

where

12 6

t

0

t

|w(s)|2 ds

12

≤M

0

1 M = 2T eαT (c + ) 2 .

Notice that this bound does not depend on the particular x ∈ K and 0 ≤ t ≤ T , only that w(·) has been chosen so that (13) holds. Let ξ ∈ K, deﬁne ζ(s), 0 ≤ s ≤ t by ζ˙ = f (ζ, u) + g(ζ)w ζ(t) = ξ. where w(·) is the above. Now for 0 ≤ s ≤ t we have 6 t |f (ζ(r), u(r))| + |g(ζ(r))| |w(r)| dr |ζ(s)| ≤ |ζ(t)| + s 6 t L(1 + |ζ(r)| + |w|) dr ≤ |ζ(t)| + s

so using Gronwall’s inequality |ζ(s)| ≤ eLT (|ξ| + LT + LM ) . Since ξ lies in a compact set we conclude that there is a compact set containing ζ(s) for 0 ≤ s ≤ t ≤ T for all ξ ∈ K. Now Q(ξ, t) ≤ e−αt Q0 (ζ(0)) 6 1 t −α(t−s) |w(s)|2 + |y(s) − h(ζ(s), u(s))|2R−1 ds e + 2 0 so

192

A.J. Krener

Q(ξ, t) − Q(x, t) ≤ + e−αt Q0 (ζ(0)) − Q0 (z(0)) 6 1 t −α(t−s) e |y(s) − h(ζ(s), u(s))|2R−1 ds + 2 0 6 1 t −α(t−s) e |y(s) − h(z(s), u(s))|2R−1 ds − 2 0

(13)

Again by Gronwall for 0 ≤ s ≤ t |z(s) − ζ(s)| ≤ e(LT +LM ) |x − ξ|. The trajectories z(s), ζ(s), 0 ≤ s ≤ t lie in a compact set where Q0 and the integrands are Lipschitz continuous so there exists L1 such that Q(ξ, t) − Q(x, t) ≤ + L1 |x − ξ|. But was arbitrary so Q(ξ, t) − Q(x, t) ≤ L1 |x − ξ|. Reversing the roles of (x, t) and (ξ, t) yields the other inequality. We have shown that Q(·, t) is Lipschitz continuous on K for 0 ≤ t ≤ T . Next we show that Q(x, t) is continuous with respect to t > 0 for ﬁxed x ∈ K. Suppose x, τ ∈ K. If τ < t, let w(·) satisfy (13) and deﬁne w(s) ¯ = w(s + t − τ ) ζ˙ = f (ζ, u) + g(ζ)w ¯ ζ(τ ) = x Then ζ(s) = z(s + t − τ ) and Q(x, τ ) ≤ e−ατ Q0 (ζ(0)) 6 1 τ −α(τ −s) 2 |w(s)| ¯ + e + |y(s) − h(ζ(s), u(s))|2R−1 ds 2 0 so Q(x, τ ) − Q(x, t) ≤ + e−ατ Q0 (z(t − τ )) − e−αt Q0 (z(0)) 6 1 t−τ −α(t−s) − e |w(s)|2 ds 2 0 6 1 t−τ −α(t−s) e |y(s) − h(z(s), u(s))|2R−1 ds − 2 0 6 1 t −α(t−s) |y(s + τ − t) − h(z(s), u(s))|2R−1 e + 2 t−τ −|y(s) − h(z(s), u(s))|2R−1 ds

The Convergence of the Minimum Energy Estimator

193

Clearly the quantities

1 2

1 2

9t t−τ

e−ατ Q0 (z(t − τ )) − e−αt Q0 (z(0)), 9 t−τ −α(t−s) e |y(s) − h(z(s), u(s))|2R−1 ds, 0

e−α(t−s) |y(s + τ − t) − h(z(s), u(s))|2R−1 − |y(s) − h(z(s), u(s))|2R−1 ds

all go to zero as t − τ $ 0. Let χt (s) be the characteristic function of [0, t] and T > 0. For 0 ≤ t − τ ≤ T 6 6 1 t−τ 1 t−τ −α(t−s) 2 e |w(s)| ds ≤ |w(s)|2 ds 2 0 2 0 6 1 T χt−τ (s)|w(s)|2 ds ≤ 2 0 which goes to zero as t − τ $ 0 by the Lebesgue dominated convergence theorem so lim Q(x, τ ) − Q(x, t) < . t−τ →0−

If τ > t, let w(·) satisfy (13) and deﬁne 0 if 0 ≤ s < τ − t w(s) ¯ = w(s + t − τ ) if τ − t ≤ s ≤ τ ζ˙ = f (ζ, u) + g(ζ)w ¯ ζ(τ ) = x Then ζ(s) = x(s + t − τ ) and Q(x, τ ) ≤ e−αt Q0 (ζ(0)) 6 1 τ −α(τ −s) 2 |w(s)| ¯ e + |y(s) − h(ζ(s), u(s))|2R−1 ds + 2 0 so Q(x, τ ) − Q(x, t) ≤ + e−ατ Q0 (ζ(0)) − e−αt Q0 (ζ(τ − t)) 6 1 τ −t −α(τ −s) + e |y(s) − h(ζ(s), u(s))|2R−1 ds 2 0 6 1 τ −α(τ −s) + |y(s) − h(ζ(s), u(s))|2R−1 e 2 τ −t −|y(s + t − τ ) − h(ζ(s), u(s))|2R−1 ds This clearly goes to as t − τ " 0 so we conclude that lim

t−τ →0+

Q(x, τ ) − Q(x, t) < .

194

A.J. Krener

But was arbitrary and we can reverse t and τ so lim Q(x, τ ) = Q(x, t).

τ →t

Now by the Lipschitz continuity with respect to x for all x, ξ ∈ K, 0 ≤ τ, t ≤ T |Q(ξ, τ ) − Q(x, t)| ≤ |Q(ξ, τ ) − Q(x, τ )| + |Q(x, τ ) − Q(x, t)| ≤ L1 |ξ − x| + |Q(x, τ ) − Q(x, t)| and this goes to zero as (ξ, τ ) → (x, t). We conclude that Q(x, t) is continuous. Next we show that 1 and 2 of Deﬁnition 1 hold. Let 0 ≤ τ < t then the principle of optimality implies that 6 1 t −α(t−s) −α(t−τ ) 2 2 Q(x, t) = inf e |w(s)| + |v(s)| ds Q(z(τ ), τ ) + e 2 τ where the inﬁmum is over all w(·), v(·), z(·) satisfying on [τ, t] z˙ = f (z, u) + g(z)w y = h(z, u) + k(z)v z(t) = x.

(14)

Let Φ(ξ, τ ) be any C ∞ function such that near x, t Φ(x, t) − Q(x, t) ≤ e−α(t−τ ) (Φ(ξ, τ ) − Q(ξ, τ )) .

(15)

Suppose w(s) = w, a constant, on [τ, t] and let ξ = z(τ ) where v(·), z(·) satisfy (14). For any constant w we have 6 1 t −α(t−s) 2 Q(x, t) ≤ e−α(t−τ ) Q(ξ, τ ) + |w| + |v(s)|2 ds. (16) e 2 τ so adding (15, 16) together yields −α(t−τ )

Φ(x, t) ≤ e

1 Φ(ξ, τ ) + 2

6 τ

t

e−α(t−s) |w|2 + |v(s)|2 ds

Recall that u(t) is continuous from the left. Assume t − τ is small then for any constant w Φ(x, t) ≤ (1 − α(t − τ )) Φ(x − (f (x, u(t)) + g(x)w)(t − τ )) 1 + |w|2 + |y(t) − h(x, u(t))|2R−1 (t − τ ) + o(t − τ ) 2 Φ(x, t) ≤ Φ(x, t) − αΦ(x, t)(t − τ ) −Φt (x, t)(t − τ ) − Φx (x, t)(f (x, u(t)) + g(x)w)(t − τ ) 1 + |w|2 + |y(t) − h(x, u(t))|2R−1 (t − τ ) + o(t − τ ) 2 0 ≥ αΦ(x, t) + Φt (x, t) + Φx (x, t)(f (x, u(t)) + g(x)w) 1 − |w|2 + |y(t) − h(x, u(t))|2R−1 2

The Convergence of the Minimum Energy Estimator

We let

195

w = g (x)Φx (x, t)

to obtain 0 ≥ αΦ(x, t) + Φt (x, t) + Φx (x, t)f (x, u(t)) + 12 |Φx (x, t)|2Γ − 12 |y(t) − h(x, u(t))|2R−1 .

(17)

On the other hand, suppose Φ(x, t) − Q(x, t) ≥ e−α(t−τ ) (Φ(ξ, τ ) − Q(ξ, τ ))

(18)

in some neighborhood of x, t. Given any > 0 and 0 ≤ τ < t there is a w(s) such that Q(x, t) ≥ e−α(t−τ ) Q(ξ, τ ) 6 1 t −α(t−s) |w(s)|2 + |v(s)|2 ds + (t − τ ) e + 2 τ

(19)

where ξ = z(τ ) from (14). Adding (18, 19) together yields for some w(s) 6 1 t −α(t−s) |w(s)|2 + |v(s)|2 ds + (t − τ ), e 2 τ 0 ≥ Φ(x, t) − αΦ(x, t)(t − τ ) −Φt (x, t)(t − τ ) − Φx (x, t)f (x, u(t))(t − τ ) 6 t Φx (x(s), s)g(x(s))w(s) ds − τ 6 1 1 t |w(s)|2 ds + |y(t) − h(x, u(t))|2R−1 (t − τ ) + 2 τ 2 +o(t − τ ) + (t − τ ).

Φ(x, t) ≥ e−α(t−τ ) Φ(ξ, τ ) +

At each s ∈ [τ, t], the minimum of the right side with respect to w(s) occurs at w(s) = g (x(s))Φx (x(s), s) so we obtain 0 ≤ αΦ(x, t) + Φt (x, t) + Φx (x, t)f (x, u(t)) + 12 |Φx (x, t)|2Γ − 12 |y(t) − h(x, u(t))|2R−1 .

(20) 2

Note that we have an initial value problem (9) for the Hamilton Jacobi PDE (10) and this determines the directions of the inequalities (17, 20).

196

A.J. Krener

3 Smooth Solutions In this section we review some known facts about viscosity solutions in general and Q(x, t) in particular. If Q is diﬀerentiable at x, t then it satisﬁes the Hamilton Jacobi PDE (10) in the classical sense [2]. There is at most one viscosity solution to the Hamilton Jacobi PDE (10) [2]. Furthermore [9], [5], if Q is diﬀerentiable at (ˆ x(t), t) then 0 = Qx (ˆ x(t), t).

(21)

If, in addition, x ˆ is diﬀerentiable at t then d Q(ˆ x(t), t) = Qt (ˆ x(t), t) + Qx (ˆ x(t), t)x ˆ˙ (t) dt = Qt (ˆ x(t), t) so this and (10) imply that d 1 Q(ˆ x(t), t) = −αQ(ˆ x, t) + |y(t) − h(ˆ x(t), u(t))|2 . dt 2

(22)

x(t), t) and x ˆ is diﬀerentiable Suppose that Q is C 2 in a neighborhood of (ˆ in a neighborhood of t. We diﬀerentiate (21) with respect to t to obtain 0 = Qxi t (ˆ x(t), t) + Qxi xj (ˆ x(t), t)x ˆ˙ j (t). We are using the convention of summing on repeated indices. We diﬀerentiate the Hamilton Jacobi PDE (10) with respect to xi at x ˆ(t) to obtain 0 = Qtxi (ˆ x(t), t) + Qxj xi (ˆ x(t), t)fj (ˆ x(t)) −1 +hrxi (ˆ x(t), u(t))Rrs (ˆ x(t)) (ys (t) − hr (ˆ x(t), u(t)))

so by the commutativity of mixed partials Qxi xj (ˆ x(t), t)x ˆ˙ j (t) = Qxi xj (ˆ x(t), t)fj (ˆ x(t), u(t)) −1 +hrxi (ˆ x(t), u(t))Rrs (ˆ x(t)) (ys (t) − hr (ˆ x(t), u(t)))

If Qxx (ˆ x(t), t) is invertible, we deﬁne P (t) = Q−1 x(t), t) and obtain an ODE xx (ˆ for x ˆ(t), x ˆ˙ (t) = f (ˆ x(t), u(t)) + P (t)h x (ˆ x(t), u(t))R−1 (ˆ x(t)) (y(t) − h(ˆ x(t), u(t)))(23) Suppose that Γ (x), R(x) are constant, f, h are C 2 in a neighborhood of x ˆ(t) and Q is C 3 in a neighborhood of (ˆ x(t), t) then we diﬀerentiate the PDE ˆ(t) to obtain (10) twice with respect to xi and xj at x

The Convergence of the Minimum Energy Estimator

197

0 = αQxi xj (ˆ x(t), t) + Qxi xj t (ˆ x(t), t) +Qxi xk (ˆ x(t), t)fkxj (ˆ x(t), u(t)) + Qxj xk (ˆ x(t), t)fkxi (ˆ x(t), u(t)) +Qxi xj xk (ˆ x(t), t)fk (ˆ x(t), u(t)) + Qxi xk (ˆ x(t), t)Γkl Qxl xj (ˆ x(t), t) −1 −hrxi (ˆ x(t), u(t))Rrs hsxj (ˆ x(t), u(t)) −1 +hrxi xj (ˆ x(t), u(t))Rrs (ys (t) − hs (ˆ x(t), u(t)))

If we set to zero α, the second partials of f, h and the third partials of Q then we obtain 0 = Qxi xj t (x, t) + Qxi xk (x, t)fkxj (ˆ x(t), u(t)) + Qxj xk (ˆ x(t), t)fkxi (ˆ x(t), u(t)) −1 +Qxi xk (ˆ x(t), t)Γkl Qxl xj (ˆ x(t), t) − hrxi (ˆ x(t), u(t))Rrs hsxj (ˆ x(t), u(t))

and so, if it exists, P (t) satisﬁes x(t), u(t))P (t) + P (t)fx (ˆ x(t), u(t)) P˙ (t) = fx (ˆ +Γ − P (t)h x (ˆ x(t), u(t))R−1 hx (ˆ x(t), u(t))P (t)

(24)

We recognize (23, 24) as the equations of the extended Kalman ﬁlter [4]. Baras, Bensoussan and James [1] have shown that, under suitable assumptions, the extended Kalman ﬁlter converges to the true state provided that the initial error is not too large. Their conditions are quite restrictive and hard to verify. Recently Krener [8] proved the extended Kalman ﬁlter is locally convergent under broad and veriﬁable conditions. There is a typographical error in the proof, the corrected version is available from the web.

4 Convergence In this section we shall prove the main result of this paper, that is, under certain conditions, the minimum energy estimate converges to the true state. Lemma 1. Suppose Q(x, t) is deﬁned by (6) and x ˆ(t) is a piecewise continuous selection of (8). Then for any 0 ≤ τ ≤ t 6 1 t −α(t−s) −α(t−τ ) Q(ˆ x(t), t) = e Q(ˆ x(τ ), τ ) + e |y(s) − h(ˆ x(s), u(s))|2R−1 ds 2 τ Proof. With suﬃcient smoothness the lemma follows from (22). If Q, x ˆ are not smooth we proceed as follows. Let 0 ≤ si−1 < si ≤ t then Q(ˆ x(si ), si ) = inf e−α(si −si−1 ) Q(z(si−1 ), si−1 ) 5 6 1 si −α(si −s) 2 2 + |w(s)| + |v(s)| ds e 2 si−1 where the inﬁmum is over all w(·), v(·), z(·) satisfying

198

A.J. Krener

z˙ = f (z, u) + g(z)w y = h(z, u) + k(z)v z(si ) = x ˆ(si ). If x ˆ(s), u(s) are continuous on [si−1 , si ] then Q(ˆ x(si ), si ) ≥ inf e−α(si −si−1 ) Q(z(si−1 ), si−1 ) . 6 5 1 si −α(si −s) + inf |w(s)|2 + |v(s)|2 ds e 2 si−1 ≥ e−α(si −si−1 ) Q(ˆ x(si−1 ), si−1 ) . 6 5 si 1 −α(si −s) 2 + inf e |v(s)| ds 2 si−1 ≥ e−α(si −si−1 ) Q(ˆ x(si−1 ), si−1 ) 1 x(si ), u(si ))|2R−1 (si − si−1 ) + o(si − si−1 ). + |y(si ) − h(ˆ 2 Since x ˆ(s), u(s) are piecewise continuous on [τ, t], they have only a ﬁnite number of discontinuities. Let τ = s0 < s1 < . . . < sk = t then for most i the above holds so x(τ ), τ ) Q(ˆ x(t), t) ≥ e−α(t−τ ) Q(ˆ 6 t 1 e−α(t−s) |y(s) − h(ˆ x(s), u(s))|2R−1 ds. + 2 τ On the other hand

Q(ˆ x(si ), si ) ≤ inf e−α(si −si−1 ) Q(z(si−1 ), si−1 ) 5 6 1 si −α(si −s) 2 2 + |w(s)| + |v(s)| ds e 2 si−1

for any w(·), v(·), z(·) satisfying z˙ = f (z, u) + g(z)w y = h(z, u) + k(z)v ˆ(si−1 ). z(si−1 ) = x In particular if we set w = 0 and assume x ˆ(s) is continuous on [si−1 , si ] then 6 1 si −α(si −s) x(si−1 ), si−1 ) + e |v(s)|2 ds Q(ˆ x(si ), si ) ≤ e−α(si −si−1 ) Q(ˆ 2 si−1 x(si−1 ), si−1 ) Q(ˆ x(si ), si ) ≤ e−α(si −si−1 ) Q(ˆ 1 x(si−1 ), u(si−1 ))|2R−1 (si − si−1 ) + o(si − si−1 ). + |y(si ) − h(ˆ 2

The Convergence of the Minimum Energy Estimator

199

Therefore since x ˆ(s), u(s) are piecewise continuous on [τ, t], with only a ﬁnite number of discontinuities then Q(ˆ x(t), t) ≤ e−α(t−τ ) Q(ˆ x(τ ), τ ) 6 t 1 e−α(t−s) |y(s) − h(ˆ x(s), u(s))|2R−1 ds. + 2 τ 2 Deﬁnition 2. [3] The system z˙ = f (z, u) + g(z)w y = h(z, u)

(25)

is uniformly observable for any input if there exist coordinates {xij : i = 1, . . . , p, j = 1, . . . , li } where 1 ≤ l1 ≤ . . . ≤ lp and li = n such that in these coordinates the system takes the form yi x˙ i1

= xi1 + hi (u) = xi2 + fi1 (x1 , u) + gi1 (x1 )w .. .

x˙ ij

= xij+1 + fij (xj , u) + gij (xj )w .. .

(26)

x˙ ili −1 = xili + fi,li −1 (xli −1 , u) + gi,li −1 (xli −1 )w x˙ ili = fi,li (xli , u) + gi,li (xli )w for i = 1, . . . , p where xj is deﬁned by xj = (x11 , . . . , x1,j∧l1 , x21 , . . . , xpj ).

(27)

Notice that in xj the indices range over i = 1, . . . , p; k = 1, . . . , j ∧ li = min{j, li } and the coordinates are ordered so that right index moves faster than the left. We also require that each fij , gij be Lipschitz continuous and satisfy growth conditions, there exists an L such that for all x, z ∈ Rn , u ∈ U , |fij (xj , u) − fij (z j , u)| ≤ L|xj − z j | |gij (xj ) − gij (z j )| ≤ L|xj − z j | |fij (xj , u)| ≤ (L + 1)|xj | |gij (xj )| ≤ L.

(28)

A system as above but without inputs is said to be uniformly observable [3].

200

A.J. Krener

Let 

010 0 0 1   Ai =   0 0 0 000

l ×l ... 0 i i n×n  ... 0 A1 0 0    ..  A =  0 ... 0  .   ... 1 0 0 Ap ... 0 p×n C1 0 0   C =  0 ... 0  0 0 Cp 

Ci = 1 0 0 . . . 0

1×li

l ×1 n×1  fi1 (x1 , u) i f¯1 (x, u)     .. .. f¯i (x, u) =  f¯(x, u) =    . . fili (xli , u) f¯p (x, u) 

l ×l n×l  gi1 (x1 ) i g¯1 (x)     .. g¯i (x) =  g¯(x) =  ...   . gili (xli ) g¯p (x)

(29)

¯ h(u) = h1 (u), . . . , hp (u)

(30)

x˙ = Ax + f¯(x, u) + g¯(x)w ¯ y = Cx + h(u)

(31)



then (26) becomes

We recall the high gain observer of Gauthier, Hammouri and Othman [3]. Their estimate x ¯(t) of x(t) given x ¯0 , y(s), 0 ≤ s ≤ t is given by the observer ¯ x ¯˙ = A¯ x + f¯(¯ x, u) + g¯(¯ x)w + S −1 (θ)C (y − C x ¯ − h(u)) x ¯(0) = x ¯0

(32)

where θ > 0 and S(θ) is the solution of A S(θ) + S(θ)A − C C = −θS(θ).

(33)

It is not hard to see that S(θ) is positive deﬁnite for θ > 0 for it satisﬁes the Lyapunov equation θ θ −A − I S(θ) + S(θ) −A − I = −C C 2 2

The Convergence of the Minimum Energy Estimator

201

where C, −A − θ2 I is an observable pair and −A − θ2 I has all eigenvalues equal to − θ2 . Gauthier, Hammouri and Othman [3] showed that when θ is suﬃciently large, p = 1, u=0 and w(·) is L∞ [0, ∞) then |x(t) − x ¯(t)| → 0 exponentially as t → ∞. We shall modify their proof to show when θ is suﬃciently large, p is arbitrary, u(·) is piecewise continuous and w(·) is L2 [0, ∞) then |x(t)− x ¯(t)| → 0 exponentially as t → ∞. The key to both results is the following lemma. We deﬁne |x|2θ = x S(θ)x. Since S(θ) is positive deﬁnite, for each θ > 0, there exists constants M1 (θ), M2 (θ) so that M1 (θ)|x| ≤ |x|θ ≤ M2 (θ)|x|.

(34)

Lemma 2. [3] Suppose g¯ is of the form (29) and satisﬁes the Lipschitz con¯ which is independent of ditions (28). Then there exists a Lipschitz constant L n θ such that for all x, z ∈ R , ¯ − z|θ . |¯ g (x) − g¯(z)|θ ≤ L|x

(35)

Note that g¯(x) is an n×l matrix so |¯ g (x)− g¯(z)|θ denotes the induced operator norm. Proof. It follows from (33) that Sij,rs (θ) = Let C =

1 M12 (1)

Sij,rs (1) (−1)j+s = j+s−1 j+s−1 θ θ

j+s−2 j−1

.

(36)

then |x|2 ≤ C|x|21 .

Let σ = max{|Sij,rs (1)|} then for each constant w ∈ Rl

Sij,rs (1) (¯ gij (x)w − g¯ij (z)w) j+s−1 (¯ grs (x)w − g¯rs (z)w) θ 1 ≤ σL2 |x − z j | |xs − z s | |w|2 . θj+s−1 j

|¯ g (x)w − g¯(z)w|2θ ≤

Deﬁne ξij =

xij , θj

ζij =

zij θj

and ξ j , ζ j as with xj , z j . Then 1 |x − z j | ≤ |ξ j − ζ j | θj j and so

202

A.J. Krener

|¯ g (x)w − g¯(z)w|2θ ≤ σL2 θ

|ξ j − ζ j | |ξ s − ζ s | |w|2

≤ σL2 θn2 |ξ − ζ|2 |w|2 ≤ σL2 θCn2 |ξ − ζ|21 |w|2 ≤ σL2 Cn2 |x − z|2θ |w|2 |¯ g (x) −

≤ σL2 Cn2 |x − z|2θ .

g¯(z)|2θ

2 Notice that for each u ∈ U , f¯(·, u) also satisﬁes the hypothesis of the above lemma so ¯ − z|θ . |f¯(x, u) − f¯(z, u)|θ ≤ L|x Theorem 2. Suppose • the system (25) is uniformly observable for any input so that it can be transformed to (31) which satisﬁes the Lipschitz and growth conditions (28), • u(·) is piecewise continuous, • w(·) is L2 [0, ∞), i.e., 6 ∞

|w(s)|2 ds < ∞

0

• x(t), y(t) are any state and output trajectories generated by system (31) with inputs u(·) and w(·), • θ is suﬃciently large • x ¯(t) is the solution of (32). Then |x(t) − x ¯(t)| → 0 exponentially as t → ∞. Proof. Let x ˜(t) = x(t) − x ¯(t) then d 2 x S(θ)x ˜˙ |˜ x| = 2˜ dt θ x + f¯(x, u) − f¯(¯ x, u) + (¯ g (x) − g¯(¯ x)) w − S −1 (θ)C C x ˜ = 2˜ x S(θ) A˜ ≤ −θ|˜ x|2θ + 2|˜ x|θ |f¯(x, u) − f¯(¯ x, u) + (¯ g (x) − g¯(¯ x)) w|θ 2 ¯ ≤ −θ + 2L(1 + |w|) |˜ x|θ . Deﬁne

6

t

β(t, τ ) = τ

¯ + |w(s)|) ds. −θ + 2L(1

¯ and τ large enough so that We choose θ ≥ 5L 6 τ

∞

|w(s)|2 ds

12

≤ 1.

The Convergence of the Minimum Energy Estimator

203

Then for t − τ ≥ 1 6 ¯ − τ ) + 2L ¯ β(t, τ ) = (−θ + 2L)(t

t

τ

|w(s)| ds

6 ¯ − τ ) + 2L ¯ ≤ (−θ + 2L)(t

12 6

t

1 ds τ 1

6

¯ − τ ) + 2L ¯ (t − τ ) 2 ≤ (−θ + 2L)(t

τ ∞

τ

t

|w(s)| ds 2

|w(s)|2 ds

12

12

¯ − τ) ≤ −L(t By Gronwall’s inequality for 0 ≤ τ < τ + 1 ≤ t |˜ x(t)|2θ ≤ eβ(t,τ ) |˜ x(τ )|2θ ≤ e−L(t−τ ) |˜ x(τ )|2θ ¯

and we conclude that |x(t) − x ¯(t)| → 0 exponentially as t → ∞.

2

Theorem 3. (Main Theorem) Suppose •

the system (1) is uniformly observable for any input and so without loss of generality we can assume that is in the form x˙ = Ax + f¯(x, u) ¯ y = Cx + h(u)

(37)

¯ as above, with A, C, f¯, h • g(x) has been chosen so that (25) is uniformly observable for any input ¯ as above, and WLOG (25) is in the form (31) with A, C, f¯, g¯, h • k(x) has been chosen to satisfy condition (5), • x(t), u(t), y(t) are any state, control and output trajectories generated by the noise free system (37), • Q(x, t) is deﬁned by (6) with α ≥ 0 for the system x˙ = Ax + f¯(x, u) + g¯(x)w ¯ y = Cx + h(u) + k(x)v

(38)

where Q0 (x0 ) ≥ 0 is Lipschitz continuous on compact subsets of Rn , • x ˆ(t) is a piecewise continuous minimizing selection of Q(x, t), (8). Then |x(t)− x ˆ(t)| → 0 as t → ∞. If α > 0 then the convergence is exponential.

Proof. Let x ¯(t) be the solution of the high gain observer (32) with g¯ = 0, driven by u(t), y(t) where x ¯0 = x ˆ0 and the gain is high enough to insure exponential convergence,

204

A.J. Krener

|x(t) − x ¯(t)| → 0 as t → ∞.

(39)

We know that for any T ≥ 0 there exists wT (t) such that the solution zT (t) of z˙T = AzT + f¯(zT , u) + g¯(zT )wT zT (T ) = x ˆ(T ) satisﬁes for 0 ≤ τ ≤ T e−α(T −τ ) Q(zT (τ ), τ ) +

1 2

9T τ

2 ¯ e−α(T −s) |wT (s)|2 + |y(s) − CzT (s) − h(u(s))| ds R−1 ≤

e−αT T +1

+ Q(ˆ x(T ), T )

From Lemma 1 we have for 0 ≤ τ ≤ T , Q(ˆ x(T ), T ) = e−α(T −τ ) Q(ˆ x(τ ), τ ) 6 T 1 2 ¯ + e−α(T −s) |y(s) − C x ˆ(s)) − h(u(s))| R−1 ds. 2 τ By the deﬁnition (6) of Q since x(s), y(s) satisfy the noise free system (37), Q(x(τ ), τ ) ≤ e−ατ Q0 (x(0)), so Q(ˆ x(τ ), τ ) ≤ Q(x(τ ), τ ) ≤ e−ατ Q0 (x(0)). Hence Q(ˆ x(T ), T ) is bounded if α = 0 and goes to zero exponentially as T → ∞ if α > 0. From the deﬁnition (8) of x ˆ(τ ) we have Q(ˆ x(τ ), τ ) ≤ Q(zT (τ ), τ ). From these we conclude that 6 1 T αs 2 ¯ |wT (s)|2 + |y(s) − CzT (s)) − h(u(s))| ds e R−1 2 τ 1 + eατ Q(ˆ x(τ ), τ ) ≤ T +1 6 T 1 2 ¯ + eαs |y(s) − C x ˆ(s)) − h(u(s))| R−1 ds 2 τ 1 + Q0 (x(0)) ≤ T +1 and it follows that 6 ∞ 0

2 ¯ eαs |y(s) − C x ˆ(s)) − h(u(s))| R−1 ds < ∞.

The Convergence of the Minimum Energy Estimator

205

Therefore given any there is a τ large enough so for all T ≥ τ 1 2

6

T

τ

2 ¯ eαs |y(s) − C x ˆ(s)) − h(u(s))| R−1 ds <

and 1 2

6 τ

T

2 ¯ |wT (s)|2 + |y(s) − CzT (s)) − h(u(s))| R−1 ds 1 −ατ 0 0 <e + Q (x ) + . T +1

(40)

Let z¯T (t) be the solution of the following high gain observer for zT (t) driven by u(t), wT (t) and CzT (t), z¯˙ T = A¯ zT + f¯(¯ zT , u) + g¯(¯ zT )wT + S −1 (θ)C (CzT − C z¯T ) 0 z¯T (0) = x ˆ

(41)

then the error z˜T (t) = zT (t) − z¯T (t) satisﬁes z˜˙ T = A˜ zT + f¯(zT ) − f¯(¯ zT , u) + (¯ g (zT ) − g¯(¯ zT )) w − S −1 (θ)C C z˜T Proceeding as in the proof of Theorem 2 we obtain d zT |2θ |˜ zT |2θ ≤ β(t, τ )|˜ dt where

6 βT (t, τ ) =

τ

t

¯ + |wT (s)|) ds. −θ + 2L(1

¯ and τ large enough so that for any T ≥ τ We choose θ ≥ 5L 06 τ

T

1 12 |wT (s)| ds

≤ 1.

2

Then for t − τ ≥ 1 6 ¯ − τ ) + 2L ¯ βT (t, τ ) = (−θ + 2L)(t

τ

t

|wT (s)| ds

6 ¯ − τ ) + 2L ¯ ≤ (−θ + 2L)(t

12 6

t

1 ds τ

¯ − τ ) + 2L ¯ (t − τ ) ≤ (−θ + 2L)(t ¯ − τ) ≤ −L(t

1 2

06 τ

τ T

t

|wT (s)| ds 2

1 12

|wT (s)| ds 2

12

206

A.J. Krener

By Gronwall’s inequality for 0 ≤ τ ≤ T |˜ zT (T )|2θ ≤ eβT (T,τ ) |˜ zT (τ )|2θ ≤ e−L(T −τ ) |˜ zT (τ )|2θ ¯

and we conclude that |zT (T ) − z¯T (T )| → 0 exponentially as T → ∞ hence x ˆ(T ) = zT (T ) → z¯T (T ) exponentially. The last step of the proof is to show x ¯(T ) → z¯T (T ) (exponentially if α > 0). Now d ¯ (¯ x − z¯T ) = A¯ x + f¯(¯ x, u) + S −1 (θ)C (y − C x ¯ − h(u)) dt zT , u) + g¯(¯ zT )wT + S −1 (θ)C (CzT − C z¯T ) − A¯ zT + f¯(¯ = A − S −1 (θ)C C (¯ x − z¯T ) ¯ ¯ ¯ +f (¯ x, u) − f (¯ zT , u) − g¯(¯ zT )wT + S −1 (θ)C (y − CzT − h(u)) so d |¯ x − z¯T |2θ = 2(¯ x − z¯T ) S(θ) A − S −1 (θ)C C (¯ x − z¯T ) dt ¯ zT )wT + S −1 (θ)C (y − CzT − h(u)) +f¯(¯ x, u) − f¯(¯ zT , u) − g¯(¯ ≤ −θ|¯ x − z¯T |2θ + 2|¯ x − z¯T |θ |f¯(¯ x, u) − f¯(¯ zT , u) − g¯(¯ zT )w|θ ¯ +2|¯ x − z¯T | |C y − CzT − h(u) | ¯ x|2 + 2LM2 (θ)|¯ ≤ (−θ + 2L)|˜ x − z¯T |θ |wT | θ ¯ +2|¯ x − z¯T | |C y − CzT − h(u)) |. ¯ Using (28) and (34) we conclude that there is an We have chosen θ ≥ 5L. M3 (θ) > 0 such that ¯ |¯ x − z¯T | |C C (x − zT ) | ≤ M3 (θ)|¯ x − z¯T |θ |y − CzT − h(u)| R−1 Therefore d ¯ x − z¯T |2 + 2LM2 (θ)|¯ |¯ x − z¯T |2θ ≤ (−θ + 2L)|¯ x − z¯T |θ |wT | θ dt ¯ +M3 (θ)|¯ x − z¯T |θ |y − CzT − h(u))| R−1 d ¯ ¯ x − z¯T |θ + LM2 (θ)|wT | + M3 (θ)|y − CzT − h(u)| |¯ x − z¯T |θ ≤ −L|¯ R−1 . dt By Gronwall’s inequality for 0 ≤ τ ≤ t ≤ T

The Convergence of the Minimum Energy Estimator

207

|¯ x(t) − z¯T (t)|θ ≤ e−L(t−τ ) |¯ x(τ ) − z¯T (τ )|θ 6 t ¯ + e−L(t−s) LM2 (θ)|wT | ds τ 6 t ¯ ¯ e−L(t−s) M3 (θ) |y(s) − CzT (s) − h(u(s))| + R−1 ds ¯

τ ¯

≤ e−L(t−τ ) |¯ x(τ ) − z¯T (τ )|θ 6 t 12 6 t 12 ¯ −2Ls 2 +LM2 (θ) e ds |wT | ds 6 +M3 (θ)

τ

t

e−2Ls ds ¯

12 6

τ

τ

t

τ

2 ¯ |y(s) − CzT (s) − h(u(s))| R−1 ds

12

≤ e−L(t−τ ) |¯ x(τ ) − z¯T (τ )|θ 6 ∞ 12 6 t 12 ¯ +LM2 (θ) e−2Ls ds |wT |2 ds ¯

6 +M3 (θ)

τ

∞

¯ −2Ls

e

12 6 ds

τ

τ

τ

t

2 ¯ |y(s) − CzT (s) − h(u(s))| R−1 ds

12

x(τ ) − z¯T (τ )|θ ≤ e−L(t−τ ) |¯ 12 6 t 12 1 2 +LM2 (θ) |wT | ds ¯ 2L τ 12 6 t 12 1 2 ¯ +M3 (θ) |y(s) − Cz (s) − h(u(s))| ds . −1 T R ¯ 2L τ ¯

As before, from (40) we see that given any δ we can choose τ large enough so that for all α ≥ 0 and all τ ≤ t ≤ T we have |¯ x(t) − z¯T (t)|θ ≤ e−L(t−τ ) |¯ x(τ ) − z¯T (τ )|θ + δ ¯

so we conclude that |¯ x(t) − z¯T (t)| → 0 as t → ∞. In particular, |¯ x(T ) − z¯T (T )| → 0 as t → ∞. If α > 0 then (40) implies that for T = 2τ T T ¯T |¯ x(T ) − z¯T (T )|θ ≤ e−L 2 |¯ x( ) − z¯T ( )|θ 2 2 12 12 1 1 −α T2 0 0 + Q 2e +LM2 (θ) (x ) + ¯ T +1 2L 1 2 12 1 1 −α T2 0 0 2e +M3 . + Q (x ) + ¯ T +1 2L Since we have already shown that |¯ x( T2 )− z¯T ( T2 )| → 0 as T → ∞, we conclude that |¯ x(T ) − z¯T (T )| → 0 exponentially as T → ∞. 2

208

A.J. Krener

5 Conclusion We have shown the global convergence of the minimum energy estimate to the true state under suitable assumptions. The proof utilized a high gain observer but it should be emphasized that the minimum energy estimator is not necessarily high gain. It is low gain if the discount rate α is small and the observation noise is substantial, i.e. R(x) is not small relative to Γ (x). It becomes higher gain as the α is increased, Γ (x) is increased or R(x) is decreased. For any size gain, the minimum energy estimator can make instantaneous transitions in the estimate as the location of the minimum of Q(x, t) jumps around. The principal drawback of the minimum energy estimator is that it requires the solution in the viscosity sense of the Hamilton Jacobi PDE (10) that is driven by the observations. This is very challenging numerically in all but the smallest state dimensions and the accuracy of the estimate is limited by the ﬁneness of the spatial and temporal mesh. Krener and Duarte [7] have oﬀered a hybrid approach to this diﬃculty. The solution of (10) is computed on a very coarse grid and this is used to initiate multiple extended Kalman ﬁlters (23) which track the local minima of Q(·, t). The one that best explains the observations is taken as the estimate.

References 1. Baras JS, Bensoussan A, James MR (1988) SIAM J on Applied Mathematics 48:1147–1158 2. Evans LC (1998) Partial Diﬀerential Equations. American Mathematical Society, Providence, RI 3. Gauthier JP, Hammouri H, Othman S (1992) IEEE Trans Auto Contr 37:875– 880 4. Gelb A (1974) Applied Optimal Estimation. MIT Press, Cambridge, MA 5. Hijab O (1980) Minimum Energy Estimation PhD Thesis, University of California, Berkeley, California 6. Hijab O (1984) Annals of Probability, 12:890–902 7. Krener AJ, Duarte A (1996) A hybrid computational approach to nonlinear estimation in Proc. of 35th Conference on Decision and Control, 1815–1819, Kobe, Japan 8. Krener AJ (2002) The convergence of the extended Kalman ﬁlter. In: Rantzer A, Byrnes CI (eds) Directions in Mathematical Systems Theory and Optimization, 173–182. Springer, Berlin Heidelberg New York, also at http://arxiv.org/abs/math.OC/0212255 9. Mortenson RE (1968) J. Optimization Theory and Applications, 2:386–394

On Absolute Stability of Convergence for Nonlinear Neural Network Models Mauro Di Marco1 , Mauro Forti1 , and Alberto Tesi2 1 2

Dipartimento di Ingegneria dell’Informazione, Universit` a di Siena V. Roma 56 - 53100 Siena Italy, {dimarco,forti}@dii.unisi.it Dipartimento di Sistemi e Informatica, Universit` a di Firenze v. S. Marta 3 - 50139 Firenze, Italy, [email protected]

Summary. This paper deals with a class of large-scale nonlinear dynamical systems, namely the additive neural networks. It is well known that convergence of neural network trajectories towards equilibrium points is a fundamental dynamical property, especially in view of the increasing number of applications which involve the solution of signal processing tasks in real time. In particular, an additive neural network is said to be absolutely stable if it is convergent for all parameters and all nonlinear functions belonging to some speciﬁed and well characterized sets, including situations where the network possesses inﬁnite non-isolated equilibrium points. The main result in this paper is that additive neural networks enjoy the property of absolute stability of convergence within the set of diagonal self-inhibition matrices, the set of symmetric neuron interconnection matrices, and the set of sigmoidal piecewise analytic neuron activations. The result is proved by generalizing a method for neural network convergence introduced in a recent paper, which is based on showing that the length of each forward trajectory of the neural network is ﬁnite. The advantages of the result in this paper over previous ones on neural network convergence established by means of LaSalle approach are discussed.

1 Introduction A neural network is a large-scale nonlinear dynamical system obtained by massively interconnecting in feedback a large number of elementary processing units usually called neurons. The network is aimed at mimicking some fundamental mechanisms of biological neural systems, in order to achieve real time processing capabilities useful to tackle pattern recognition and optimization problems [1]. A neural network is said to be convergent, or completely stable, when each trajectory converges towards an equilibrium point (a stationary state). Convergence is a fundamental property for neural network applications to several signal processing tasks [2, 3, 4, 5]. Consider for example the implementation W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 209–220, 2003.

c Springer-Verlag Berlin Heidelberg 2003

210

M. Di Marco, M. Forti, and A. Tesi

of a content addressable memory. The pattern corrupted by noise is provided as the initial condition to the neural network, and the process of retrieval of the uncorrupted pattern, which is represented by some stable equilibrium point of the network, is achieved during the convergent transient towards this equilibrium. It is important to note that a convergent behavior is in general not guaranteed for a neural network. Indeed, oscillatory dynamics, such as limit cycles, and even complex dynamics, such as chaotic attractors, have been observed in simulations and real experiments of some classes of neural networks [2, 3, 6, 7]. A number of methods to investigate convergence of neural network trajectories have been developed in the last two decades [1, 3, 8]. Nevertheless, the problem of convergence is far from being completely solved, since several issues need a much deeper understanding [9]. In particular, one of the most challenging issues in current neural network research is to obtain methods for studying convergence in situations where the nonlinear dynamical system modeling the network is of high order, and, most importantly, it possesses multiple (possibly non-isolated) equilibrium points. Within this context, this paper deals with the class of additive neural networks, which are dynamical systems typically speciﬁed by sets of parameters (e.g., the neuron self-inhibitions, the synaptic neuron interconnections, and the biasing neuron inputs), and sets of nonlinear functions (e.g., the sigmoidal neuron activations) [2, 3]. Speciﬁcally, we are interested in the issue of absolute stability of convergence, which means roughly speaking that convergence holds for all neural network parameters, and all nonlinear neuron activations, belonging to speciﬁed and well characterized sets. This property represents a kind of robustness of convergence with respect to parameter variations and with respect to the actual electronic implementation of the nonlinearities. It is worth to point out that an absolutely stable neural network enjoys the interesting property of being convergent even in situations, which are encountered as the parameters are varied, where the network possesses inﬁnite non-isolated equilibrium points, e.g., when there are entire manifolds of equilibria. The main result in this paper (Theorem 1) is that additive neural networks enjoy the property of absolute stability of convergence with respect to the set of diagonal self-inhibition matrices, the set of symmetric neuron interconnection matrices, and the set of sigmoidal piecewise analytic neuron activations. The class of symmetric neural networks is central in the literature concerning convergence of neural networks [2, 4, 5, 10]. Moreover, any neuron activation referred to in speciﬁc examples is indeed modeled by piecewise analytic functions. To prove Theorem 1, a method introduced in a recent paper [11] to address absolute stability of convergence for the class of standard Cellular Neural Networks [5], see also [12], is generalized. The method is based on proving a fundamental limit theorem for the neural network trajectories, according to which the total length of each forward trajectory is ﬁnite. This in turn

Absolute Stability of Convergence for Neural Networks

211

ensures convergence of each trajectory towards a singleton, independently of the structure of the set of the neural network equilibrium points. The structure of the paper is brieﬂy outlined as follows. In Sect. 2, the piecewise analytic neural network model which is dealt with in the paper is introduced. Section 3 presents the main results on absolute stability of convergence, while Sect. 4 compares the obtained results with some basic existing results in the literature on neural network convergence. Finally, the main conclusions drawn in the paper are reported in Sect. 5. Notation. Rn : real n-space A = [Aij ] ∈ Rn×n : square matrix A : transpose of A α = diag(α1 , · · · , αn ) ∈ Rn×n : diagonal matrix with diagonal entries αi , i = 1, · · · , n x = (x1 , · · · , xn ) ∈ Rn : column vector n 2 1/2 : Euclidean norm of x x2 = i=1 xi ∇V (x) : gradient of V (x) : Rn → R

2 Piecewise Analytic Neural Networks Consider the class of additive neural networks described by the diﬀerential equations x˙ = Ax + T g(x) + I (N) where x = (x1 , · · · , xn ) ∈ Rn is the vector of the neuron states, A ∈ Rn×n is a matrix modeling the neuron self-inhibitions, and T ∈ Rn×n is the neuron interconnection matrix. The diagonal mapping g(x) = (g1 (x1 ), · · · , gn (xn )) : Rn → Rn has components gi (xi ) that model the nonlinear input-output activations of the neurons, whereas I ∈ Rn is a vector of constant neuron inputs. Model (N) includes the popular Hopﬁeld neural networks [4], the emerging paradigm of Cellular Neural Networks [5], and several other neural models frequently employed in the literature. Throughout the paper, some assumptions on the self-inhibition matrix A, the interconnection matrix T , and the activations g, are enforced. In order to state these assumptions, the next deﬁnitions are introduced. Deﬁnition 1 (Diagonal inhibitions DA ). We say that A ∈ DA , if and only if A = diag(−a1 , · · · , −an ) is a diagonal matrix such that ai > 0, i = 1, · · · , n. Deﬁnition 2 (Symmetric interconnections TS ). We say that T ∈ TS , if and only if T is symmetric, i.e., T = T . Deﬁnition 3 (Sigmoidal piecewise analytic activations GA ). We say that g ∈ GA , if and only if, for i = 1, · · · , n, the following conditions hold:

212

M. Di Marco, M. Forti, and A. Tesi

a) gi is bounded on R; b) gi is piecewise analytic on R, i.e., there exist intervals Λij = (λij , λij+1 ) ⊂ R, j = 1, · · · , pi , with λi1 = −∞, λip = ∞, λij+1 > λij , such that gi is analytic in Λij ; moreover, gi ∈ C 0 (R), and gi is strictly increasing on R, i.e., it results ∞ > Mji > dgi (xi )/dxi > 0 for all xi ∈ Λij . Assumption 1 We assume that A ∈ DA , T ∈ TS , and g ∈ GA .

Some comments on the above assumptions are in order. The hypothesis of diagonal self-inhibitions is standard for neural networks [4]. Moreover, the set of piecewise analytic activations g ∈ GA is of interest, as it is witnessed by the fact that the most commonly used sigmoidal functions are analytic, and they belong to GA . Consider for example the popular activation gi (ρ) = (2/π) arctan(λπρ/2) proposed by Hopﬁeld [4] (see Fig. 1(a)), the sigmoidal function gi (ρ) = 1/(1 + e−ρ ), the multilevel activations gi (ρ) = ki /(1 + e−η(ρ−θi ) ) [13], and other activations referred to in the literature. It is also noted that it is common practice in circuit theory to construct a model of a nonlinear function by using piecewise analytic approximations [14] (see Fig. 1(b)-(c)). The assumption of symmetric interconnections, T ∈ TS , is enforced for two main reasons. The ﬁrst one is that such hypothesis is at the core of the most fundamental results on neural network convergence established in the literature via LaSalle approach (cf. Sect. 4). The second one, is that symmetry may be crucial to establish convergence. More speciﬁcally, it is known that when the neuron interconnection matrix is largely non-symmetric, the network can exhibit oscillatory dynamics, such as limit cycles, and even complex dynamics, such as chaotic attractors [2, 3, 7, 15]. More recent investigations have also shown that there are special classes of neural networks which undergo complex bifurcations, leading to the birth of non-convergent dynamics, even close to some neural network with nominal symmetric interconnections [7, 16]. In this paper, we are interested in absolute stability of convergence (see Deﬁnition 5) for the class of neural networks (N) where the self-inhibitions A ∈ DA , the interconnection matrix T ∈ TS , and the neuron activations g ∈ GA . To deﬁne the concept of absolute stability of convergence, let us consider the set of the equilibrium points of (N), which is given by E = {x ∈ Rn : Ax + T g(x) + I = 0}.

(1)

Under the assumptions A ∈ DA and g ∈ GA it can be shown that E is not empty [17]. Furthermore, it is stressed that the equilibria of (N) may be in general non-isolated, as it is shown in Example 1 below. Deﬁnition 4 ([3]). Given some A, T , g, and I, the neural network (N) is said to be convergent if and only if, for any trajectory x(t) of (N), there exists x ˜ ∈ E such that

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

g i ( ρ)

g i ( ρ)

Absolute Stability of Convergence for Neural Networks

0

0

−0.2

−0.2

−0.4

−0.4

−0.6

−0.6

−0.8

−0.8

−1 −3

−2

−1

0

1

ρ

2

−1 −3

3

213

−2

−1

0

ρ

(a)

1

2

3

(b)

1

g i ( ρ)

0.8

0.6

0.4

0.2

0

−3

−2

−1

0

ρ

1

2

3

(c) Fig. 1. Sigmoidal piecewise analytic neuron activations in the class GA . (a) The standard analytic neuron activation gi (ρ) = (2/π) arctan(λπρ/2) proposed by Hopﬁeld (λ = 1); (b) a piecewise analytic approximation of the sigmoidal activation in (a) using polynomials and rational functions (see (4)); (c) sigmoidal piecewise analytic function obtained by piecing together two exponential functions (gi (ρ) = (1/2)eρ−1 for ρ ≤ 1; gi (ρ) = 1 − (1/2)e−3(ρ−1) for ρ > 1).

lim x(t) = x ˜.

t→+∞

The property of convergence, which is frequently referred to in the literature as complete stability, is related to a speciﬁc neural network where the self-inhibitions A, interconnection matrix T , nonlinear activations g, and inputs I, have been ﬁxed. In this paper we are interested in the stronger property of absolute stability of convergence, which is related to an entire set of neural networks, as stated in the next deﬁnition.

214

M. Di Marco, M. Forti, and A. Tesi

Deﬁnition 5. We say that convergence of (N) is absolutely stable within the sets DA , TS , and GA , if and only if (N) is convergent for any neuron selfinhibition matrix A ∈ DA , any neuron interconnection matrix T ∈ TS , any neuron activations g ∈ GA , and any neuron input I ∈ Rn . It is worth to remark that the concept of absolute stability of convergence as in Deﬁnition 5 should not be confused with the property of absolute stability deﬁned in previous work [17, 18]. In fact, in those papers the absolute stability refers to neural networks that possess a unique equilibrium point, while in Deﬁnition 5 we consider more generally neural networks with multiple equilibria, e.g., networks with entire manifolds of equilibrium points.

3 Main Result The main result on absolute stability of convergence of neural network (N) is as follows. Theorem 1. Convergence of (N) is absolutely stable within the sets DA , TS , and GA . This result includes situations where the neural network (N) possesses inﬁnite non-isolated equilibrium points, see Example 1 below. In this respect, it diﬀers substantially from existing results on complete stability of (N) in the literature, as it is discussed in Sect. 4. Example 1. Consider the second-order neural network

where

and

x˙ 1 = −x1 + g1 (x1 ) − g2 (x2 ) x˙ 2 = −x2 − g1 (x1 ) + g2 (x2 )

(2)

       

1 3 ρ , |ρ| ≤ 1; 4 ρ , ρ > 1; g1 (ρ) = ρ + 3    ρ     −ρ + 3 , ρ < −1

(3)

 1  ρ − ρ3 , |ρ| ≤ 1;   4     3ρ , ρ > 1; g2 (ρ) = 3ρ + 1     3ρ   , ρ < −1.  −3ρ + 1

(4)

Absolute Stability of Convergence for Neural Networks

215

The neural network (2) satisﬁes A ∈ DA , T ∈ TS , and g(x) = (g1 (x1 ), g2 (x2 )) ∈ GA . Moreover, it can be easily veriﬁed that the set E of the equilibrium points of (2) contains the set E˜ = {(x1 , x2 ) ∈ R2 : x1 = −x2 ; |x1 | ≤ 1, |x2 | ≤ 1}. Figure 2 reports the trajectories of (2), as obtained with MATLAB, for a number of diﬀerent initial conditions. It is seen that each trajectory converges ˜ in accordance with the towards a unique equilibrium point within the set E, result in Theorem 1.

2

1.5

1

x2

0.5

0

−0.5

−1

−1.5

−2 −2

−1.5

−1

−0.5

0

0.5

1

1.5

2

x1 Fig. 2. Phase portrait of neural network (2). The initial conditions are represented by the symbol ◦. The set E˜ of equilibrium points corresponds to the thick solid segment.

Below, the main steps of the proof of Theorem 1 are outlined. The whole technical details can be found in [19]. 1. Let us consider the Lyapunov function n

1 ai W (x) = − g (x)T g(x) − g (x)I + 2 i=1

6 0

xi

dgi (ρ) ρ dρ dρ

(5)

which has been introduced by Hopﬁeld to study the dynamics of (N) [4]. The following classic result holds [3, 4].

216

M. Di Marco, M. Forti, and A. Tesi

Property 1 Suppose that A ∈ DA , T ∈ TS , and g ∈ GA . Then, W is a ˙ (N) (x) = [∇W (x)] x˙ ≤ 0 strict Lyapunov function for (N), i.e., it results W n n ˙ ˙ (N) (x) < 0 for for all x ∈ R , E = {x ∈ R : W(N) (x) = 0}, and W n x ∈ R \E. Since the trajectories of (N) are bounded on [0, +∞), Property 1 implies, on the basis of LaSalle invariance principle, that the ω-limit set of each trajectory of (N) is contained within the set E of equilibrium points of (N). If in particular E is made up of isolated equilibria, then (N) is convergent, due to the connectedness of the ω-limit set. However, when (N) has uncountably many equilibrium points, it cannot be concluded on the basis of LaSalle approach alone that (N) is convergent. Indeed, there is the risk that a trajectory indeﬁnitely slides over some manifold of equilibria, without converging to a singleton. The previous discussion shows that further argument with respect to LaSalle invariance principle is needed to prove convergence of (N) in the general case of non-isolated equilibrium points. Such an argument, which is provided in points 2) and 3) below, exploits the basic assumption that the neuron nonlinearities g are modeled by piecewise-analytic functions. 2. The existence of a strict Lyapunov function W (see Property 1), together with the hypothesis g ∈ GA (cf. Assumption 1), is exploited in order to establish a basic property of the neural network trajectories. Namely, it is shown that the length of the trajectories of (N) on the time interval [0, +∞) is necessarily ﬁnite. To state this result more formally, let x(t), t ∈ [0, +∞), be some trajectory 9t of (N). For any t > 0, the length of x(t) on [0, t) is given by 0 x(σ) ˙ 2 dσ. The following result can be proved. Theorem 2. Suppose that A ∈ DA , T ∈ TS , and g ∈ GA . Then, any trajectory x(t) of (N) has ﬁnite length on [0, +∞), i.e., 6 t 6 +∞ x(σ) ˙ x(σ) ˙ L= 2 dσ = lim 2 dσ < +∞. 0

t→+∞

0

Once it has been shown that the total length of x(t) is ﬁnite, a standard mathematical argument allows one to conclude the existence of the limit of x(t) as t → +∞, hence convergence of x(t) towards an equilibrium point of (N) (see the proof of Corollary 1 in [11]). 3. The proof of Theorem 2 exploits a fundamental inequality established by L ojasiewicz for analytic functions, which is reported next. Consider a function F (x) : Γ ⊂ Rn → R, which is analytic in the open set Γ , and assume that the set of critical points of F C = {x ∈ Γ : ∇F (x) = 0} is not empty. Then, it possible to give an estimate from below of the norm of the gradient of F , in a neighborhood of a critical point x0 , as stated in the next lemma.

Absolute Stability of Convergence for Neural Networks

217

Lemma 1. 1 Suppose that x0 ∈ C. Then, there exists R > 0 and an exponent θ ∈ (0, 1) such that ∇F (x)2 ≥ |F (x) − F (x0 )|θ for x − x0 < R.

(6)

Lojasiewicz inequality plays a key role to prevent trajectories of (N) from sliding over some manifold of equilibrium points without converging to a singleton. In fact, the use of such an inequality leads to a direct proof that the length of each trajectory of (N) is necessarily ﬁnite, even when the neural network possesses entire manifolds of equilibria. The technical details of the proof of this argument are given in [19]. We conclude this section by noting that the method of proof of Theorem 1 generalizes that previously introduced to analyze absolute stability of the standard Cellular Neural Networks [11], and general additive neural networks with neuron activations modeled by piecewise linear functions [12]. It is worth to note that the property of ﬁniteness of trajectory length for standard Cellular Neural Networks was directly proved in [11] by exploiting the special structure of the piecewise aﬃne vector ﬁelds describing the dynamics of a Cellular Neural Network. Since general piecewise analytic vector ﬁelds are considered for (N), the speciﬁc arguments used in [11] are no longer applicable. As it was noted before, L ojasiewicz inequality is the key additional mathematical device here employed to extend the proof of ﬁniteness of length to the general model (N).

4 Discussion In this section, we brieﬂy compare the result on absolute stability of convergence in Theorem 1 with previous results on convergence of the additive neural network (N) in the literature [2, 3, 4, 10]. Those results require A ∈ DA , and the symmetry of the interconnection matrix (T ∈ TS ), but are applicable in the less restrictive case, with respect to Theorem 1, where the neuron nonlinearities g only need to be Lipschitz continuous sigmoidal functions. Since under such hypotheses (N) possesses a strict Lyapunov function, the quoted results on convergence are obtained as a direct application of LaSalle invariance principle. It is noted, however, that all those results require the additional assumption, with respect to Theorem 1, that the equilibrium points of (N) be isolated. This is due to the fact that, as it was discussed before, LaSalle approach is not suitable to prove convergence in the general case where the equilibrium points are not isolated. Finally, we remark that an additional mathematical device used in some of the quoted papers is Sard’s theorem, which 1

See, e.g., Theorem 4 in [20].

218

M. Di Marco, M. Forti, and A. Tesi

enables to prove that given speciﬁc neuron self-inhibitions A, neuron activations g, and symmetric interconnection matrix T , for almost all neural network inputs I the equilibrium points are isolated (see, e.g., [10, Theorem 4.1]). In summary, the results in [2, 3, 4, 10] are basically results on almost absolute stability of convergence. Furthermore, they leave open the issue of convergence for those parameters where there are non-isolated equilibrium points of (N).2 From the previous discussion it is seen that the result on absolute stability of convergence in Theorem 1 actually refers to a more restrictive class of neural networks with respect to [2, 3, 4, 10]. However, it enjoys the signiﬁcant advantage that there is no technical diﬃculty to verify the hypotheses of Theorem 1. For comparison, we stress the fact that the notion of almost absolute stability of convergence is not simple to deal in practice, due to technical diﬃculties to establish whether or not a given neural network has isolated equilibria. Indeed, Sard’s theorem does not specify the critical set for which there are non-isolated equilibrium points, and there is no general method to ﬁnd all equilibria of (N) for sigmoidal activations. There are also other advantages of the property of absolute stability of convergence as in Theorem 1, with respect to the weaker notion of almost absolute stability of convergence. In fact, there are interesting applications where it is required that the neural network (N) be convergent also when it possesses entire manifolds of equilibrium points. This is the case when the network is used to retrieve a static pattern within a set of uncountably many equilibrium patterns, as in the problems discussed in [2, Sect. I].

5 Conclusion The paper has considered a class of additive neural networks where the neuron self-inhibitions are modeled by diagonal matrices with negative diagonal elements, the neuron interconnection matrix is symmetric, and the neuron activations are modeled by sigmoidal piecewise analytic functions. The main result is that convergence is absolutely stable within this class, i.e., it holds for all parameters and all nonlinear functions deﬁning the considered class, including situations where the neural network possesses inﬁnite non-isolated equilibrium points. The result has been proved through a generalization of a new method that has been recently introduced to analyze convergence of neural networks. The method is based on showing that the length of each forward 2

It is known that a system possessing a strict Lyapunov function and non-isolated equilibria may in general be non-convergent. A classical example is due to Palis and De Melo [21, Example 3, p. 14]. This corresponds to a planar gradient system where all bounded trajectories have an entire circle of equilibrium point as the ω-limit set. Actually, these trajectories show a large-size non-vanishing oscillation in the long-run behavior, without converging to a singleton. It is worth to note that such a kind of non-convergent dynamics would be highly undesirable for the application of neural networks to solve signal processing tasks.

Absolute Stability of Convergence for Neural Networks

219

trajectory of the neural network is necessarily ﬁnite. It has also been pointed out that LaSalle approach would be not suitable to address absolute stability of convergence for the considered class. Indeed, even in the presence of a strict Lyapunov function, LaSalle invariance principle does not imply convergence in the general case of non-isolated equilibrium points. Future work aims at exploring absolute stability of convergence for larger classes of neural networks. A possible case of special interest is that of neural networks where the neuron activations are characterized by a threshold value under which they are zero. In that case, the energy function of the neural network is no more strictly decreasing along the trajectories, and a more general technique than that presented in this paper is required to prove ﬁniteness of trajectory length for each forward trajectory.

References 1. Grossberg S (1988) Nonlinear neural networks: Principles, mechanisms, and architectures. Neural Networks, 1:17–61. 2. Cohen M. A., Grossberg S. (1983) Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Trans. Syst. Man Cyber., 13:815–825. 3. Hirsch M. (1989) Convergent activation dynamics in continuous time networks. Neural Networks, 2:331–349. 4. Hopﬁeld J. J. (1984) Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Nat. Acad. Sci., 81:3088– 3092. 5. Chua L. O., Yang L. (1988) Cellular neural networks: Theory. IEEE Trans. Circuits Syst., 35(10):1257–1272, October. 6. Chua L. O. (1997) CNN: A vision of complexity. Int. J. Bifurcation and Chaos, 7(10):2219–2425, October. 7. Di Marco M., Forti M., Tesi A. (2000) Bifurcations and oscillatory behavior in a class of competitive cellular neural networks. Int. J. Bifurcation and Chaos, 10(6):1267–1293, June. 8. Chua L. O., Wu C. W. (1992) On the universe of stable cellular neural networks. Int. J. Circuit Theory Applicat., 20:497–517. 9. Lemmon M. D., Michel, A. N. (Eds.) (1999) Special section on neural networks in control, identiﬁcation, and decision making. IEEE Trans. Automat. Contr., 44(11):1993–2057, November. 10. Li J. H., Michel A. N., Porod W. (1988) Qualitative analysis and synthesis of a class of neural networks. IEEE Trans. Circuits Syst., 35(8):976–986, August. 11. Forti M., Tesi A. (2001) A new method to analyze complete stability of PWL cellular neural networks. Int. J. Bifurcation and Chaos, 11:655–676, March. 12. Forti M. (2002) Some extensions of a new method to analyze complete stability of neural networks. IEEE Trans. Neural Networks, 13:1230–1238, September. 13. Bang S. H., Sheu B. J., Chang J. C.-F. (1994) Search of optimal solutions in multi-level neural networks. In Proc. ISCAS 1994, IEEE Int. Symp. on Circuits and Systems, volume 6, pages 423–426, London.

220

M. Di Marco, M. Forti, and A. Tesi

14. Chua L. O., Desoer C. A., Kuh E. S. (1987) Linear and Nonlinear Circuits. McGraw-Hill, New York. 15. Chua L. O. (Edt.) (1995) Special issue on nonlinear waves, patterns and spatiotemporal chaos in dynamic arrays. IEEE Trans. Circuits Syst. I, 42(10):557– 823, October. 16. Di Marco M., Forti M., Tesi A. (2002) Existence and characterization of limit cycles in nearly symmetric neural networks. IEEE Trans. Circuits Syst. I, 49(7):979–992, July. 17. Forti M., Tesi A. (1995) New conditions for global stability of neural networks with application to linear and quadratic programming problems. IEEE Trans. Circuits Syst. I , 42:354–366. 18. Liang X.-B., Wang J. (2001) An additive diagonal stability condition for absolute stability of a general class of neural networks. IEEE Trans. Circuits Syst. I, 48:1308–1317, November. 19. Di Marco M., Forti M., Tesi A. (2002) A method to analyze complete stability of analytic neural networks. Technical Report 15, Universit` a di Siena, Siena, Italy. 20. L ojasiewicz S. (1959) Sur le probl`eme de la division. Studia Math., T. XVIII:87– 136. 21. Palis J. De Melo W. (1982) Geometric Theory of Dynamical Systems. SpringerVerlag, Berlin.

A Novel Design Approach to Flatness-Based Feedback Boundary Control of Nonlinear Reaction-Diﬀusion Systems with Distributed Parameters Thomas Meurer and Michael Zeitz Institute of System Dynamics and Control Engineering, University of Stuttgart, Pfaﬀenwaldring 9, 70569 Stuttgart, Germany, {meurer,zeitz}@isr.uni-stuttgart.de

1 Introduction Diﬀerential ﬂatness has proven to be a very powerful approach for analysis and design of open–loop and stabilizing feedback tracking control for nonlinear ﬁnite–dimensional systems [2, 10, 11]. Thereby, ﬂatness can be interpreted as a generalization of controllability or the possibility to determine the inverse model, respectively. The ﬂatness approach has been extended to the design of open–loop boundary control for inﬁnite–dimensional or distributed parameter systems (DPSs) described by partial diﬀerential equations (PDEs). The parameterization of system states and boundary input by a ﬂat output (inverse system) can be obtained for parabolic DPS by assuming a power series expansion of the solution [5, 6, 7, 8]. Applications concern the linear heat conduction equation [5], rather general nonlinear parabolic PDEs describing diﬀusion or heat conduction [8], and nonlinear tubular reactor models [6, 7]. Note, that also the case of time–dependent coeﬃcients is treated [8]. In the scope of industrial applications, ﬂatness–based open–loop boundary control of a solenoid valve modelled by Maxwell’s equations with space–dependent coeﬃcients is addressed in [12]. Nevertheless, the use of open–loop boundary control is rather limited due to disturbances acting on the system or model uncertainties. Hence, a closed–loop strategy is desired, trying to cope with these eﬀects. This was the motivation of the authors for a ﬂatness–based approach to the design of feedback boundary tracking control for linear DPS as illustrated in [9] for the boundary control of a linear heat conduction equation. The results of [9], which allow the exploitation of the full scope of diﬀerential ﬂatness for the feedback boundary control design, are extended in the present contribution W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 221–235, 2003.

c Springer-Verlag Berlin Heidelberg 2003

222

T. Meurer and M. Zeitz

to asymptotic tracking control for nonlinear parabolic DPS. This includes motion planning, inspection of state or input constraints, asymptotic feedback stabilization, and observer design. The proposed ﬂatness–based procedure is demonstrated for the boundary control of a nonlinear parabolic distributed parameter model of a reaction–diﬀusion system. The paper is organized as follows. The control problem is speciﬁed in Section 2 for a rather general scalar nonlinear reaction–diﬀusion PDE. The proof of ﬂatness for the considered DPS and the design of ﬂatness–based open–loop control are treated in Section 3 serving as basis for the boundary feedback control design in Section 4. The performance of open–loop and feedback control is studied in various simulation scenarios in Section 5, followed by some concluding remarks.

2 Problem Formulation Consider a scalar reaction–diﬀusion system modelled by the rather general nonlinear parabolic PDE ∂x(z, t) ∂ 2 x(z, t) ∂x(z, t) = , t > 0, z ∈ (0, 1) (1) + ϕ x(z, t), ∂t ∂z 2 ∂z ∂x (0, t) = u(t), t>0 (2) ∂z ∂x (1, t) = 0, t>0 (3) ∂z x(z, 0) = x0 (z), z ∈ [0, 1] (4) y(t) = x(1, t), t ≥ 0, (5) where state x(z, t), time t, spatial coordinate z, boundary input u(t), output variable y(t), and any model parameter are assumed to be perfectly non–dimensionalized. The nonlinear function ϕ x, ∂x ∂z could describe a temperature–dependent heat source, some kind of chemical reaction following a nonlinear reaction rate [13], or nonlinear convection. In Section 3, some restrictions will be imposed on the class of functions ϕ. The considered control problem concerns the design of a boundary control u for a possibly unstable system (1)–(5), in order to realize the transition between the initial stationary proﬁle x0 (z) = x1S (z) and a ﬁnal stationary proﬁle x2S (z) in ﬁnite time T such that y(t ≥ T ) = x2S (1) in the presence of model errors or exogenous disturbances. For the solution of this control problem, both open–loop and feedback control will be considered. Therefore, inspired by the ﬂatness property of ﬁnite–dimensional nonlinear systems, a ﬂat output will be determined in order to parameterize the system state x(z, t) as well as boundary input u(t).

Flatness-Based Feedback Boundary Control of Reaction-Diﬀusion Systems

223

3 Flatness-Based Motion Planning and Open-Loop Boundary Control The proof of ﬂatness and hence the design of an open–loop control for the considered DPS is based on a power series approach as proposed in [5, 6, 7, 8]. Similarly, motion planning is addressed based on Gevrey functions [3]. 3.1 Flatness of the Nonlinear DPS In order to analyze the ﬂatness of (1)–(5), a formal power series ansatz for the state x(z, t) with yet unknown time–varying coeﬃcients an (t) is used, i.e. x(z, t) =

∞

an (t)(1 − z)n ,

(6)

n=0

which can be formally diﬀerentiated with respect to t and z. The formal power series ansatz (6) considered throughout this contribution diﬀers from the one proposed e.g. in [6, 7, 8], where the ansatz is deﬁned in terms of the scaled arguments a ˜n (t) = n! an (t). However, the formal approach (6) provides signiﬁcantly better numerical conditioning for the implementation and simulation of the boundary feedback control design presented in Section 4. Inserting (6) into PDE (1) yields ∞

0 =ϕ

(1 − z)n [a˙ n (t) − (n + 2)(n + 1)an+2 (t)]

n=0 ∞

n

an (t)(1 − z) , −

n=0

∞

1 n

(n + 1)an+1 (t)(1 − z)

(7) .

n=0

This inﬁnite set of nonlinear equations for the series coeﬃcients an (t), n ≥ 0 only provides a solution by comparing polynomials in (1 − z), if the nonlinear source term is restricted to functions allowing a power series expansion of the following form ∞ ϕ x, ∂x = ϕ¯n (an+1 (t))(1 − z)n , ∂z

(8)

an+1 (t) = [a0 (t), . . . , an+1 (t)],

(9)

n=0

where

and ϕ¯n (an+1 (t)) being smooth with respect to its arguments for any n ∈ N. This condition imposes only weak limitations as will be shown throughout this section. As a result, terms of equal order n in the polynomials (1 − z)n can be arranged in (7)

224

T. Meurer and M. Zeitz ∞

(1 − z)n [a˙ n (t) − (n + 2)(n + 1)an+2 (t) − ϕ¯n (an+1 (t))] = 0,

n=0

which yields a˙ n (t) − (n + 2)(n + 1)an+2 (t) − ϕ¯n (an+1 (t)) = 0 ∀n ∈ N.

(10)

Eqn. (10) can be evaluated recursively by solving for an+2 (t) and considering the boundary condition (3) and the output equation (5) an+2 (t) =

a˙ n (t) − ϕ¯n (an+1 (t)) , (n + 2)(n + 1)

a1 (t) = 0,

a0 (t) = y(t).

(11)

This recursion allows a parameterized solution for the series coeﬃcients an (t) in terms of the controlled variable y(t) and its time–derivatives y (i) (t), i ≥ 1 . n n even (j) 2 an (t) = φn y(t), y(t), ˙ . . . , y (t) , j = n−1 (12) n odd. 2 In general, Eqn. (11) can be eﬃciently evaluated by use of a computer– algebra–system like Mathematica [14]. Under the formal assumption of series convergence, the state x(z, t) at any point (z, t) and the input u(t) = ∂x ∂z (0, t) can be determined by substituting (12) into the formal ansatz (6), i.e. x(z, t) =

∞

φn (yn (t)) (1 − z)n = Φ(y(t), y(t), ˙ . . . ),

(13)

n=0

u(t) = −

∞

nφn (yn (t)) = Ψ (y(t), y(t), ˙ . . . ),

(14)

n=1

where yn (t) = [y(t), . . . , y (n) (t)]. A straightforward comparison of (13), (14) with the deﬁnition of diﬀerential ﬂatness for ﬁnite–dimensional systems [2] illustrates, that the controlled variable y(t) = x(1, t) represents a ﬂat output for the inﬁnite–dimensional system (1)–(4), whereby an inﬁnite number of time–derivatives of the ﬂat output is necessary to parameterize system state and input. Note that Eqn. (8) only imposes a rather weak restriction on ϕ x, ∂x ∂z since many nonlinear functions allow at least an asymptotic power series expansion, which can be evaluated by the formal power series ansatz (6) using for example Cauchy’s product formula. Some technically important examples for ϕ x, ∂x ∂z together with their evaluation using the formal power series ansatz (6) are summarized below: • Model of an isothermal tubular reactor concentration x(z, t) with third 3 order reaction i=1 βi xi and linear convection α ∂x ∂z

Flatness-Based Feedback Boundary Control of Reaction-Diﬀusion Systems

225

∂x 2 3 ϕ(x, ∂x ∂z ) = α ∂z + β1 x + β2 x + β3 x ∞ n −α(n + 1)an+1 (t) + β1 an (t) + β2 = an−p (t)ap (t)+ n=0 p=0 p n β3 an−p (t) ap−q (t)aq (t) (1 − z)n p=0

q=0

(15) • Model of a tubular bioreactor concentration x(z, t) with linear convection x α ∂x ∂z and Monod kinetics β 1+x [1] ∂x x ϕ(x, ∂x ∂z ) = α ∂z + β 1+x m ≈ α ∂x + β (−1)n xn+1 ∂z (16) n=0 ∞ m −α(n + 1)an+1 (t) + β = (−1)p cnp+1 (t) (1 − z)n , n=0

p=0

where cn1 (t) = an (t) and cnp (t) denotes the Cauchy product for xp , p ∈ N x with x from (6). Here, the kinetic function β 1+x is expanded only asym1 ptotically by truncating the series expansion for 1+x at some m ∈ N. 3.2 Convergence of the Formal Solution and Motion Planning In order to obtain an open–loop boundary control as stated in Section 2, a smooth trajectory yd (t) for the ﬂat output y(t) of class C ∞ has to be speciﬁed connecting the two stationary proﬁles x1S (z) and x2S (z) at z = 1. It follows directly that yd (0) = x1S (1) and yd (t ≥ T ) = x2S (1) since the ﬂat output y(t) completely parameterizes the state (13) and hence the stationary proﬁles xS (z) = x(z, t → ∞). The convergence of the inﬁnite series solutions (13) and (14) has to be ensured by an appropriate choice of the desired trajectory yd (t). Due to the broad class of possible nonlinear source terms, convergence has to be proven for each given function ϕ x, ∂x ∂z in (1) allowing an expansion given by (8). Therefore in the remainder of this paper, only the particular function ϕ given by Eqn. (15) will be considered. In [6], it is shown that (13) and (14) converge for β1,3 = 0 in (15), i.e. a second order reaction, with unit radius of convergence, if yd (t) : R → R is a Gevrey function of class γ ≤ 2, i.e. a C ∞ function satisfying (n)

sup |yd (t)| ≤ t∈R

m(n!)γ , Rn

∀n ≥ 0, γ ≤ 2,

(17)

where m and R are constants, and if in addition the following inequality is satisﬁed

226

T. Meurer and M. Zeitz

|α| + |β2 |m + 2R−1 ≤ 2. This clearly illustrates the importance of motion planning since the given inequality combines the physics of the problem given by parameters α and β2 with the estimates on the bounds of the chosen desired trajectory. Following a similar argumentation as in [6], it can be shown, that the open–loop control (14) converges for β1,3 = 0 in (15), i.e. a third order reaction, with unit radius of convergence, if the desired trajectory yd (t) : R → R is a Gevrey function of class γ ≤ 2 and if in addition the following inequality is satisﬁed |α| + |β1 | + |β2 |m + |β3 |m2 + 2R−1 ≤ 2.

(18)

Numerical results indicate that also nonlinearities involving powers of order ≥ 4 in the state x(z, t) allow the derivation of convergent inﬁnite series expansions for system state and boundary control. In the following, similar to [5] and [8], motion planning is based on a so called smooth ’step function’ of Gevrey order γ = 1 + ω1   if t ≤ 0  0 1 if t ≥ T Θω,T (t) = (19) t  θ (τ )dτ ω,T   0T if t ∈ (0, T ) θ (τ )dτ 0

ω,T

where θω,T denotes the ’bump function’   0 θω,T (t) =

 exp

0

−1 ω (1 − Tt ) Tt

1 if t ∈ (0, T ) if t ∈ (0, T ).

(20)

Therefore, a desired trajectory being consistent with both the initial and ﬁnal stationary proﬁle is realized by yd (t) = x1S (1) + (x2S (1) − x1S (1))Θω,T (t),

(21)

such that yd (0) = x1S (1) and yd (t ≥ T ) = x2S (1). As outlined above, the parameters ω and T of (19) have to be chosen appropriately for every desired ﬁnal stationary proﬁle x2S (z) in order to ensure convergence of the series expansions (13), (14) and satisfy possible state and input constraints. This results in the associated open–loop control ud (t) = Ψ (yd (t), y˙ d (t), . . . ) with |ud (t)| < u ¯ similar to results proposed in [6, 7, 8, 12] for other DPS control problems. Note however, that for implementation the open–loop control (14) has to be truncated at a certain integer, since it is impossible to compute an inﬁnite series. Due to this fact and in view of exogenous disturbances, model errors or instability, a closed–loop control strategy is needed.

Flatness-Based Feedback Boundary Control of Reaction-Diﬀusion Systems

227

4 Flatness-Based Feedback Boundary Tracking Control Based on a re–interpretation of the results for the ﬂatness–based open–loop control design for the DPS (1)–(5) as summarized in Section 3, the design approach originally proposed in [9] for linear ﬂatness–based feedback boundary control is extended to the considered nonlinear problem. 4.1 Flat Design Model for Feedback Boundary Control As illustrated in the previous section, appropriate motion planning ensures convergence of the formal solution (6). Therefore, re–consider Eqn. (7), and substitute the abbreviations (8), (9), i.e. ∞

(1 − z)n [a˙ n (t) − (n + 2)(n + 1)an+2 (t) − ϕ¯n (an+1 (t))] = 0.

(22)

n=0

Under the formal assumption of convergence, the series expansion (14) for the open–loop control can be truncated at some upper limit 2N, N ∈ N, which is equivalent to consider only the ﬁrst 2N − 2 terms of summation in (22) a˙ n (t) − (n + 2)(n + 1)an+2 (t) − ϕ¯n (an+1 (t)) = 0,

n ∈ {0, 1, . . . , 2N − 2}. (23)

Diﬀering from the previous section where (23) was solved for an+2 (t) to obtain a recursion formula, interpret (23) as a set of nonlinear ordinary diﬀerential equations (ODEs) for the coeﬃcients an (t), n = 0, 1, . . . , 2N − 2. Note, since (k) a1 (t) = 0 by boundary condition (3) and hence a1 (t) = 0, k ∈ N, the coeﬃcients an (t), n = 1, 3, . . . , 2N −1 can be expressed in terms of an (t), n = 0, 2, . . . , 2N − 2 by successively evaluating (23): a1 (t) = 0, a2n+1 (t) = θ2n+1 (a0 (t), a2 (t), . . . , a2n (t)) ,

n ≥ 1.

(24)

¯n+1 (t) denotes (9) with (24) substituted. Therefore, consiIn the following a dering 2N − 2 terms in (23) results in the following N nonlinear ODEs a˙ 0 (t) a˙ 2 (t)

= 2a2 (t) + ϕ¯0 (¯ a1 (t)) , = 12a4 (t) + ϕ¯2 (¯ a3 (t)) , .. .

(25)

a2N −1 (t)) . a˙ 2N −2 (t) = 2N (2N − 1)a2N (t) + ϕ¯2N −2 (¯ It is essential for the determination of the only unknown coeﬃcient a2N (t) to consider the inﬂow boundary condition (2) together with a truncated formal ansatz (6), i.e.

228

T. Meurer and M. Zeitz 2N −1 ∂x (0, t) ≈ − u(t) = (n + 1)an+1 (t) ∂z n=0

1 ⇒ a2N (t) = − 2N

0 u(t) +

2N −2

1 (n + 1)an+1 (t) ,

(26)

n=0

with a2n+1 (t), n ≥ 0 by (24), such that the boundary input u is introduced into the ODEs (25). As a result, the following state–space representation is T obtained for ζ = [ζ1 , . . . , ζN ] with ζn (t) = a2n−2 (t), n = 1, 2, . . . , N       ϕ¯0 (ζ1 ) 0 2ζ2       0 12ζ3 ϕ¯2 (ζ1 , ζ2 )             . . . .. .. .. ζ˙ =  + +  u(t),        (2N − 2)(2N − 3)ζN  ϕ¯2N −4 (ζ1 , . . . , ζN −1 )  0 ϕ¯2N −2 (ζ) ρ (ζ) −(2N − 1) !" # !" # !" # f 1 (ζ)

f 2 (ϕ(ζ)) ¯

b

(27) where ρ(ζ) can be determined from (26) with (24). This input aﬃne nonlinear system of ﬁnite dimension, allows an interpretation as a generalized controller normal form due to the obtained triangular structure of f 1 (ζ) and f 2 (ϕ(ζ)). ¯ In order to further illustrate this, consider a linear reaction–diﬀusion PDE (1) without convection, i.e. ϕ(x, ∂x ∂z ) = βx and follow the procedure above. This yields [9]     β 2 ... 0 0 0 0    β ... 0 0 0         . . . .. .. .. ζ˙ =  .. ζ +   u(t), .     0   0 . . . β (2N − 2)(2N − 3)  0 0 −2(2N − 1) . . . −(2N − 2)(2N − 1)β −(2N − 1) !" # !" # A

b

where the matrix A shows a characteristic band structure with one–sided coupling except for the last row, concluding the interpretation as a generalized controller normal form with ﬂat output ζ1 (t). The initial condition of (27) follows directly from the assumption of transition between stationary proﬁles x1S (z) → x2S (z). Due to this assumption, every desired trajectory connecting x1S (z) and x2S (z) has to satisfy y (i) (t = 0) = x1S (1)δi,0 , where δi,0 denotes the Kronecker delta. Hence, evaluation of (12) for t = 0 provides ζn (0) = a2n−2 (0) = φ2n−2 (x1S (1), 0, . . . , 0),

n = 1, 2, . . . , N.

(28)

Flatness-Based Feedback Boundary Control of Reaction-Diﬀusion Systems

229

System (27) is ﬂat with ﬂat output y(t) = ζ1 (t) = a0 (t) as can be easily veriﬁed by the argumentation given in Section 3. The derived ﬂat ﬁnite– dimensional system of ODEs serves as a design model for ﬂatness–based feedback boundary control for the nonlinear DPS (1)–(5), as will be outlined in the next section. 4.2 Flatness-Based Feedback Boundary Control with Observer In order to track the ﬂat output y(t) along an appropriately designed desired trajectory yd (t) (see Section 3.2 on motion planning), feedback tracking control [2, 10] is designed based on the ﬂat representation (27), (28). Flat systems are exactly feedback linearizable [2], such that asymptotically stable tracking control can be designed by methods of linear control theory – for details, the reader is referred to [2, 10]. Following the ﬂatness–based feedback control design approach for ﬁnite–dimensional nonlinear systems, a static feedback control law can be obtained (N )

u = ψ(y, y, ˙ . . . , yd

− Λ(e)),

(29)

where Λ(e) with e = e, e, ˙ . . . , e(N −1) can be any type of control, asymptotically stabilizing the tracking error e(t) = y(t) − yd (t). Note, that (29) is derived by successive time–diﬀerentiations of the ﬂat output y(t) = ζ1 (t) up (N ) to N –th order and introduction of a new input v = y (N ) = yd − Λ(e). For the design of Λ(e) consider extended PID–control [4], i.e. 6 Λ(e) = p0

t

e1 (τ )dτ + 0

N

pk ek (t).

(30)

k=1

The parameters pk , k = 0, . . . , N are assumed to be coeﬃcients of a Hurwitz polynomial in order to obtain asymptotically stable tracking error dynamics and can be determined by eigenvalue assignment λN +1 + pN λN + . . . + p1 λ + p0 =

N +1 8

(λ − λk ).

(31)

i=1

Note, that the integral part in the extended PID–control is necessary for robustness purposes [4]. Since full state information is necessary for the implementation of the feedback control (29)–(31), an observer is needed to estimate the non–measured states. This results in the block diagram depicted in Figure 1 for the realization of the proposed ﬂatness–based feedback boundary tracking control with observer for nonlinear parabolic DPS governed by (1)–(5). There, all non– measured states are replaced by their estimated counterpart as indicated by the hat on each aﬀected quantity. For the observer design assume that the

230

T. Meurer and M. Zeitz

$

$

¼

"# $

$

!

% $ & ' Fig. 1. Block diagram of the ﬂatness–based feedback boundary tracking control ¯ d = [yd , y˙ d , . . . , yd(N ) ]. scheme with spatial proﬁle estimation for DPS (1)–(5) with y

ﬂat output, i.e. y(t) = ζ1 (t) = x(1, t) is measured, such that the following nonlinear tracking observer can be designed based on model (27), (28) ˙ ˆ ˆ + f 2 (ϕ( ˆ + bˆ = ζˆ 0 ¯ ζ)) u + l(t) y − ζˆ1 , ζ(0) (32) ζˆ = f 1 (ζ) with suitable initial conditions ζˆ 0 . The time-varying observer gain vector l(t) can be determined based on a linearization of the observer error dynamics along the desired trajectory ζ d (t) which is known due to the ﬂatness property of the design model [11]. Hence, the gain vector l(t) can be designed using the Ackermann–formula for linear time–varying systems [10, 11]. This allows the ˆ k , k = 1, . . . , N , for the observer error assignment of suitable eigenvalues λ dynamics N

λ + pˆN −1 λ

N −1

+ . . . + pˆ1 λ + pˆ0 =

N 8

ˆ k ). (λ − λ

(33)

i=1

In addition to state feedback, the designed observer can be used for spatial proﬁle estimation throughout the transition process. An estimate x ˆ(z, t) of the spatial proﬁle x(z, t) at time t can be obtained by evaluating the power ansatz (6) x ˆ(z, t) =

2N −1

a ˆn (t)(1 − z)n ,

(34)

n=0

where the time–varying coeﬃcients an (t) are replaced by their estimated counterparts a ˆn (t). These coeﬃcients can be determined directly from the observed states since ﬁrst

Flatness-Based Feedback Boundary Control of Reaction-Diﬀusion Systems

a ˆ2n−2 (t) = ζˆn (t),

n = 1, 2, . . . , N

231

(35)

and second using (24) a ˆ1 (t) = 0, a ˆ2n+1 (t) = θ2n+1 (ˆ a0 (t), a ˆ2 (t), . . . , a ˆ2n (t)) ,

n = 1, 2, . . . , N − 1.

(36)

5 Simulation Results In order to illustrate the performance of the proposed ﬂatness–based feedback control scheme, simulation results are shown for the model of an isothermal tubular reactor concentration x(z, t) (non–dimensionalized) with third order reaction governed by ∂ 2 x(z, t) ∂x(z, t) ∂x = + β1 x + β2 x2 + β3 x3 . +α 2 ∂t ∂z ∂z

(37)

For the simulation, PDE (37) with boundary and initial conditions (2)–(4) are discretized using the method–of–lines (MOL) approach with spatial discretization ∆z = 0.01. The initial condition and hence the initial proﬁle are assumed to be equal to zero, i.e x0 (z) = x1S (z) = 0. Various parameter sets {α, β1 , β2 , β3 } will be considered in the simulation scenarios. Note, that the parameter values are assigned with some kind of arbitrariness, but can be adapted to real physical values. The desired trajectory (21) is parameterized by x1S (1) = 0, x2S (1) = 0.5, ω = 1.1, and T = 2.0. Feedback control and observer are designed for N = 6 in the design model (27). In any scenario, the eigenvalues for feedback control (31) and observer (33) are assigned as follows λk = −20 − 2(k − 1), ˆ k = −50 − 2(k − 1), λ

k = 1, 2, . . . , 7,

(38)

k = 1, 2, . . . , 6.

(39)

For comparison purposes, the series expansion of the open–loop boundary control (14) is also truncated at N = 6 for the MOL simulations. 5.1 Stable Operating Point At ﬁrst, simulation results are shown for the parameters α = 0, β1 = −1, β2 = 3, and β3 = −3. For this parameter set, the transition from the initial stationary proﬁle to the ﬁnal stationary proﬁle remains stable. Simulation results for applying the open–loop as well as feedback control are depicted in Figure 2. The transition from the initial proﬁle to the ﬁnal proﬁle is illustrated in Fig. 2, top (left), for the evolution of the controlled state x(z, t) in the (z, t)–domain. The control performance for open–loop and feedback control only diﬀers slightly, whereby zero steady state error is only obtained for the feedback control due to the induced truncation error for the open–loop control.

232

T. Meurer and M. Zeitz

5.2 Model Error In order to study robustness issues, assume the model parameters in the MOL discretized plant diﬀer by 50% from the parameters used for the control design, i.e. α ¯ = 0, β¯1 = −1.5, β¯2 = 4.5, β¯3 = −4.5 in the simulation model and α = 0, β1 = −1, β2 = 3, β3 = −3 for the open–loop and feedback control design. Simulation results for this scenario are depicted in Figure 3, showing the applied boundary controls (left) and the obtained control performance (right). These results clearly illustrate the great control performance of the proposed feedback boundary control strategy reaching zero steady state error. 5.3 Unstable Operating Point Depending on the parameters α and βi , i = 1, 2, 3, the dynamics of (37) might change drastically. Hence by assuming α = 0, β1 = −1, β2 = 3, and β3 = 3, the transition to the desired ﬁnal stationary proﬁle has to be stabilized. This is illustrated in Figure 4 comparing open–loop and feedback boundary control. Due to the instability, the open–loop control is no longer applicable, whereby the ﬂatness–based feedback boundary tracking control with observer clearly stabilizes the system providing excellent control performance. 5.4 Proﬁle Estimation Finally, the applicability of the proposed approach to proﬁle estimation is illustrated by considering the simulation scenarios of Sections 5.2 and 5.3. The estimated proﬁles x ˆ(z, t) at various times t obtained by evaluating (34) with the observer data are compared in Figure 5 to the proﬁles x(z, t) determined by the MOL–simulation of the controlled DPS (37). In any case, the estimated proﬁles match the exact proﬁles almost exactly throughout the transition process.

6 Conclusion This contribution presents an extension of the ﬂatness–based feedback boundary tracking control approach introduced in [9] for linear parabolic DPS to scalar nonlinear parabolic reaction–diﬀusion equations. The approach is based on a re–interpretation of the power–series approach to derive a formal open–loop boundary control by means of a ﬂat output. It provides an approximation of the considered nonlinear inﬁnite–dimensional system by an inherently ﬂat nonlinear ﬁnite–dimensional system, which serves as a design model for feedback boundary control and tracking observer.

Flatness-Based Feedback Boundary Control of Reaction-Diﬀusion Systems

233

−0.1

1

−0.2 −0.3

0.6

−0.4

x

u

0.8

−0.5

0.4

−0.6

0.2

−0.7

0 0

−0.9 0

1

Time t

1 0

Space z

FB OL

−0.8

2 0.5

0.5

1

t

1.5

2

2.5

−3

0.6

1.5

0.5

1

x 10

0.4

e(t)

x(1,t)

0.5

0.3

0 0.2

0 0

−0.5

FB OL Des.

0.1 0.5

1

1.5

t

2

−1 0

2.5

FB OL 0.5

1

t

1.5

2

2.5

Fig. 2. Simulation results for open–loop and feedback controlled stable DPS with α = 0, β1 = −1, β2 = 3, and β3 = −3. Top (left): evolution of x(z, t) in (z, t)–domain for open–loop control; top (right): comparison of open–loop and feedback boundary control; bottom (left): comparison of output y(t) = x(1, t) and desired trajectory yd (t); bottom (right): comparison of tracking error e(t) = y(t) − yd (t). 0.6

0

0.5

−0.2

0.4

x(1,t)

−0.4

u

0.3

−0.6 0.2 −0.8 −1 0

0.5

1

t

1.5

2

FB OL Des.

0.1

FB OL 2.5

0 0

0.5

1

t

1.5

2

2.5

Fig. 3. Simulation results for open–loop and feedback controlled stable DPS with model error described in Sec. 5.2. Left: comparison of open–loop and feedback boundary control; right: comparison of output y(t) = x(1, t) and desired trajectory yd (t).

Simulation studies illustrate the performance and robustness of the feedback control scheme when applied to the original DPS involving model errors and unstable behavior, pointing out the necessity of a closed–loop strategy for

234

T. Meurer and M. Zeitz 0.6

0.6

0.4

0.5

0.2

0.4

x(1,t)

0

u

0.3

−0.2

0.2

−0.4 −0.6 −0.8 0

0.5

1

t

1.5

2

FB OL Des.

0.1

FB OL

0 0

2.5

0.5

1

t

1.5

2

2.5

Fig. 4. Simulation results for open–loop and feedback controlled unstable DPS with α = 0, β1 = −1, β2 = 3, and β3 = 3. Left: comparison of open–loop and feedback control; right: comparison of output y(t) = x(1, t) and desired trajectory yd (t). 0.7

0.7

x(z,tj) ∧

0.6

t =2.0 4

x(z,tj) t4=2.0

0.4

x(z,t)

x(z,t)

0.4

t3=1.0

0.3

t3=1.0

0.3

t2=0.75

t2=0.75

0.2

0.2

0 0

∧

0.5

0.5

0.1

x(z,tj)

0.6

x(z,tj)

t =0.5 1

0.2

0.1 0.4

z

0.6

0.8

1

0 0

t =0.5 1

0.2

0.4

z

0.6

0.8

1

Fig. 5. Comparison of estimated proﬁle x ˆ(z, t) from (34) with the exact proﬁle x(z, t) for simulation scenarios of Section 5.2 (left) and 5.3 (right).

the control of DPS. The proposed approach is completely model based and directly provides design rules for feedforward and feedback control as well as for the observer, establishing an analogy to ﬂatness–based control design for ﬁnite–dimensional systems.

References 1. Delattre C, Dochain D, Winkin, J (2002) Observability analysis of a nonlinear tubular bioreactor. Proceedings of the 15th Int. Symposium on Mathematical Theory of Networks and Systems (MTNS), Notre Dame, USA 2. Fliess M, L´evine J, Martin P, Rouchon P (1995) Internat. J. Control 61:1327– 1361 3. Gevrey M (1918) Annales Scientiﬁques de l’Ecole Normale Superieure 25:129– 190 4. Hagenmeyer V (2001) Nonlinear stability and robustness of an extended PID– control based on diﬀerential ﬂatness. Proceedings (CDROM) 5th IFAC Symposium on Nonlinear Control Systems NOLCOS (St. Petersburg, Russia), 179–184

Flatness-Based Feedback Boundary Control of Reaction-Diﬀusion Systems

235

5. Laroche B, Martin P, Rouchon P (2000). Int. J. Robust Nonlinear Control 10:629–643 6. Lynch AF, Rudolph J (2000b) Flatness–based boundary control of a nonlinear parabolic equation modelling a tubular reactor. In Isidori A, Lamnabhi– Lagarrique F, Respondek W (eds) Nonlinear Control in the Year 2000 Vol. 2. Springer 7. Lynch AF, Rudolph J (2000c) Flatness–based boundary control of coupled nonlinear PDEs modelling a tubular reactor. Proceedings of the International Symposium on Nonlinear Theory and its Applications NOLTA2000 (Dresden, Germany), 641–644 8. Lynch AF, Rudolph J (2002) Internat. J. Control 75(15):1219–1230 9. Meurer T, Zeitz M (2002) submitted 10. Rothfuß R, Rudolph J, Zeitz M (1996) Automatica 32:1433–1439 11. Rothfuß R (1997) Anwendung der ﬂachheitsbasierten Analyse und Regelung nichtlinearer Mehrgr¨ oßensysteme. Fortschr.–Ber. VDI Reihe 8 Nr. 664, VDI Verlag, D¨ usseldorf 12. Rothfuß R, Becker U, Rudolph J (2000) Controlling a solenoid valve – a distributed parameter approach. Proceedings of the 14th Int. Symposium on Mathematical Theory of Networks and Systems (MTNS), Perpignan, France 13. Showalter K (1995) Nonlinear Science Today 4(4):3–10 14. Wolfram S (1999) The Mathematica Book. Wolfram Media & Cambridge University Press, 4th edition

Time-Varying Output Feedback Control of a Family of Uncertain Nonlinear Systems Chunjiang Qian1 and Wei Lin2 1 2

Dept. of Electrical Engineering, The University of Texas at San Antonio, San Antonio, TX 78249, [email protected] Dept. of Electrical Engineering and Computer Science, Case Western Reserve University, Cleveland, OH 44106, [email protected]

Dedicated to our teacher and friend Art Krener on the occasion of his 60th birthday

Summary. This paper addresses the problem of global regulation by time-varying output feedback, for a family of uncertain nonlinear systems dominated by a linearly growing triangular system. The growth rate of the nonlinearities is not known a priori, and therefore the problem is unsolvable by existing results on this subject. By extending the output feedback design method introduced in [17] to the timevarying case, we explicitly construct a linear time-varying output feedback control law that globally regulates all the states of the systems without knowing the growth rate.

1 Introduction and Discussion Control of nonlinear systems by output feedback is one of the fundamental problems in the ﬁeld of nonlinear control. A major diﬃculty in solving the output feedback control problem is the lack of the so-called separation principle for nonlinear systems. As such, the problem is more challenging and diﬃcult than its full-state feedback control counterpart. Over the last two decades, a number of researchers have investigated the output feedback control problem for nonlinear systems and obtained some interesting results; see, for instance, the papers [2, 15, 20, 13, 3, 18, 1, 16] and [7]–[11] to just name a few. Note that most of the aforementioned results only deal with nonlinear systems involving

C. Qian’s work was supported in part by the U.S. NSF grant ECS-0239105, UTSA Faculty Research Award, and Texas Space Grant Consortium. W. Lin’s work was supported in part by the U.S. NSF grants ECS-9875273 and DMS-0203387.

W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 237–250, 2003.

c Springer-Verlag Berlin Heidelberg 2003

238

C. Qian and W. Lin

no uncertainties, while fewer of them consider the output feedback control of uncertain nonlinear systems in which the uncertainties are associated with the measurable states (e.g. the system outputs). When the uncertainties are related to the unmeasurable states, the output control problem becomes much more involved. The essential diﬃculty is that the uncertainties (e.g. unknown parameters or disturbances) associated with the unmeasurable states prevent one designing a conventional observer that often consists of a copy of the uncertain system. Such an observer is of course not implementable due to the presence of the uncertainty. In this paper, we investigate the output feedback control problem for nonlinear systems with uncertainties that are associated with the unmeasurable states. Speciﬁcally, we consider a family of time-varying uncertain systems of the form x˙ 1 = x2 + φ1 (t, x, u) x˙ 2 = x3 + φ2 (t, x, u) .. . x˙ n = u + φn (t, x, u),

y = x1

(1)

where x = (x1 , · · · , xn )T ∈ IRn , u ∈ IR and y ∈ IR are the system state, input and output, respectively. The uncertain mappings φi : IR × IRn × IR → IR, i = 1, · · · , n, are continuous and satisfy the following condition: Assumption 2 that

For i = 1, · · · , n, there is an unknown constant θ ≥ 0 such |φi (t, x, u)| ≤ θ(|x1 | + · · · + |xi |).

(2)

In the case when θ is known, Assumption 2 reduces to the condition introduced in [18], where a linear state feedback control law was designed to globally exponentially stabilize the nonlinear system (1). Under the same condition, we showed recently in [17] that global exponential stabilization of (1) can also be achieved by a time-invariant dynamic output compensator. The linear output feedback controller was constructed using a feedback domination design method that substantially diﬀers from the so-called separation principle. When the parameter θ is unknown, the problem of global regulation of systems (1) has been studied in the literature and there are some results available using full state feedback. For instance, the paper [12] proved that global adaptive regulation of systems (1) is solvable via dynamic state feedback (see Corollary 3.11 and Remark 3.12 in [12]). In [19], a time-varying state feedback control law was proposed to regulate a class of systems (1) under certain growth/structural conditions. By comparison, global regulation of the uncertain system (1) by output feedback is a far more diﬃcult problem that receives much less attention. Only under certain structural assumptions [14, 5] (e.g. (1) is in a output-feedback form) and restrictive conditions on θ (e.g.

Time-Varying Output Feedback for Uncertain Nonlinear Systems

239

the bound of θ is known [14]), adaptive regulation was achieved via output feedback. However, the problem of how to use output feedback to globally regulate a whole family of systems (1) remains largely open and unsolved. The objective of this paper is to prove that under Assumption 2, this open problem can be solved by time-varying output feedback. To be precise, we shall show that there exists a C k (k ≥ 0) time-varying dynamic output compensator of the form ξ˙ = M (t)ξ + N (t)y, u = K(t)ξ,

M : IR → IRn×n , K : IR → IR1×n

N : IR → IRn (3)

such that all the solutions of the closed-loop system (1)-(3) are globally ultimately bounded. Moreover, lim (x(t), ξ(t)) = (0, 0).

t→+∞

It must be pointed out that the nonlinear system (1) satisfying Assumption 2 represents an important class of uncertain systems that are impossible to be dealt with by existing output feedback control schemes such as those in [18, 13, 1, 16]. First, the uncertain system (1) includes a family of linear systems with unknown time-varying parameters θi,j (t)’s as a special case: x˙ 1 = x2 + θ1,1 (t)x1 x˙ 2 = x3 + θ2,1 (t)x1 + θ2,2 (t)x2 .. . x˙ n = u + θn,1 (t)x1 + θn,2 (t)x2 + · · · + θn,n (t)xn y = x1

(4)

where θi,j (t) is a C 0 function of t bounded by an unknown constant θ. To the best of our knowledge, none of the existing output feedback control schemes can achieve global regulation of the time-varying system (4). However, it is easy to verify that Assumption 2 is fulﬁlled for system (4), and therefore global regulation via time-varying output feedback is possible, according to the main result of this paper—Theorem 1 in Section 2. Besides the aforementioned uncertain time-varying linear system, there are also a number of classes of uncertain nonlinear systems satisfying Assumption 2, whose global regulation problem cannot be solved by any output feedback design method in the present literature. For instance, consider the nonlinearly parameterized system x˙ 1 = x2 +

x1 (1 − θ1 x2 )2 + x22

x˙ 2 = u + ln(1 + (x22 )θ2 ) y = x1

(5)

240

C. Qian and W. Lin

where θ1 ∈ IR and θ2 ≥ 1 are unknown constants. It can be veriﬁed that 2 2 2 2 2 2 x1 2 2 2ln(1 + (x22 )θ2 )2 ≤ (2θ2 − 1)|x2 |. 2 and 2 (1 − θ1 x2 )2 + x2 2 ≤ (1 + θ1 )|x1 | 2 Clearly, system (5) satisﬁes Assumption 2 with θ = max{2θ2 − 1, 1 + θ12 }, and hence can be handled by the time-varying output feedback approach proposed in this paper. However, since θ1 and θ2 are not known, all the existing output feedback design methods including those in [18, 13, 1, 16] and [17] are not applicable to the nonlinearly parameterized system (5). In summary, the uncertain system (1) satisfying Assumption 2 encompasses a family of uncertain linear/nonlinear systems whose global state regulation problem is not solvable by any existing design method. In this paper, we propose to solve this nontrivial problem by using time-varying output feedback. In particular, we will further exploit the feedback domination design technique introduced in [17] and combine it with the idea of using a time-varying gain. First, we extend the linear high-gain observer in [7, 3, 6] to a time-varying observer that does not require any information of the uncertainties of the system. We then use a feedback domination design method to construct a controller whose gain is also time-varying. Finally, we prove that a combination of these two designs leads to a solution to the problem of global state regulation via output feedback for the entire family of uncertain systems (1). As already demonstrated in [17], an important feature of our design method is that the precise knowledge of the nonlinearities or uncertainties of the system needs not be known (in [17], only θ in Assumption 2 requires to be known). In the present work, due to the use of time-varying gains, an additional advantage of the proposed time-varying output feedback control scheme is the possibility of controlling the uncertain system (1) without knowing the parameter θ. This is a nice “universal” property that allows one to globally regulate an entirely family of uncertain nonlinear systems using a single time-varying dynamic output compensator.

2 Time-Varying Output Feedback Control In this section, we prove that without knowing the parameters θ in (2), it is still possible to globally regulate a whole family of systems (1) by time-varying output feedback. This is achieved by employing the output feedback design method proposed in [17] combined with the idea of the use of time-varying gains. Theorem 1. Under Assumption 2, there is a time-varying output feedback controller (3) that solves the global state regulation problem for the uncertain nonlinear system (1). Proof. The proof is carried out in the spirit of [17] but quite diﬀerent from [17], due to the design of both time-varying observer and controller. We divide

Time-Varying Output Feedback for Uncertain Nonlinear Systems

241

the proof into two parts. In part 1, we design a linear observer without using the information of uncertain nonlinearities. In order to overcome the diﬃculty caused by the unknown parameter θ, we use a time-varying gain instead of a constant high-gain in [17]. In part 2 we propose a domination-based design method to construct an output controller, achieving global state regulation for the uncertain system (1). In this part, we again design a time-varying gain to take care of the unknown parameters/uncertainties and to guarantee the convergence of the closed-loop system. Part 1 – Design of a Time-Varying Linear Observer: We begin by introducing the time-varying dynamic output compensator x ˆ˙ 1 = x ˆ2 + La1 (x1 − x ˆ1 ) .. . ˙x ˆn + Ln−1 an−1 (x1 − x ˆ1 ) ˆn−1 = x x ˆ˙ n = u + Ln an (x1 − x ˆ1 )

(6)

where L = t + 1 is a time-varying gain, aj > 0, j = 1, · · · , n, are coeﬃcients of the Hurwitz polynomial p(s) = sn + a1 sn−1 + · · · + an−1 s + an , and u = u(t, x ˆ1 , · · · , x ˆn ) is a time-varying feedback control law to be determined later. −ˆ xi Let εi = xLi i−1 , i = 1, · · · , n. Then, it is easy to see that 

φ1 (t, x, u)  1 φ2 (t, x, u) L ε˙ = LAε +  .  ..







0   −ε2 1 L    +  ..  .

   

(7)

−(n − 1) L1 εn

1 Ln−1 φn (t, x, u)

where  ε1  ε2    ε = . ,  ..  

εn



−a1 1 · · ·  .. .. . .  . . A= .  −an−1 0 · · · −an 0 · · ·

 0 ..  . . 1 0

By construction, A is a Hurwitz matrix. Hence, there exists a positive deﬁnite matrix P = P T > 0 satisfying AT P + P A = −I. Choose the Lyapunov function V (ε) = εT P ε. By Assumption 2, it can be shown the existence of an unknown constant Θ ≥ 0 such that

242

C. Qian and W. Lin



φ1 (t, x, u)

  V˙ (ε) ≤ −Lε2 + 2εT P  .  ..

φ2 (t,x,u) L

  (n − 1)  ε2  + 2P L 

φn (t,x,u) Ln−1

1 1 ≤ − [L − c1 ] ε2 + Θε |x1 | + |x2 | + · · · + n−1 |xn | , L L with c1 = 2P (n − 1). Since xi = x ˆi + Li−1 εi , we have 2 2 2 2 2 1 2 2 1 2 2 2 2 ˆi 22 + |εi |, 2 Li−1 xi 2 ≤ 2 Li−1 x

i = 1, · · · , n.

Using this relationship, it is straightforward to show that * + √ 1 1 2 ˙ x2 | + · · · + n−1 |ˆ x1 | + |ˆ xn | V (ε) ≤ −(L − Θ n − c1 )ε + Θε |ˆ L L + * √ x ˆ22 n x ˆ21 x ˆ2n 2 + ≤ − L − Θ n − Θ − c1 ε + Θ . + ··· + 2 2 2L2 2L2n−2 Part 2 – Design of a Time-Varying Feedback Controller: Deﬁne z = (z1 , · · · , zn )T with x ˆ2 x ˆn , · · · , zn = n−1 , L L

(8)

u = −Ln [bn z1 + bn−1 z2 + · · · + b1 zn ],

(9)

ˆ1 , z2 = z1 = x and design the controller as

where sn + b1 sn−1 + · · · + bn−1 s + bn verify that under new coordinates (8) rewritten as    0 a1  a2  1  z 2    z˙ = LBz + Lε1  .  −  . ..  ..  L  an

is a Hurwitz polynomial. It is easy to and controller (9) z-subsystem can be 



0

1

    , B =    0 0 −bn −bn−1 (n − 1)zn

 ··· 0  ..  .  .(10) ··· 1  · · · −b1

With the choice of Lyapunov function U (z) = z T Qz, where positive deﬁnite matrix Q satisﬁes B T Q + QB = −I, the following inequality can be easily proved

Time-Varying Output Feedback for Uncertain Nonlinear Systems



0 z2 .. .

 1  U˙ = −Lz2 + 2Lε1 z T Q[a1 , · · · , an ]T − 2z T Q  L 

243

    

(n − 1)zn ≤ −(L − c3 )z + Lc2 εz L Lc2 ≤− − c3 z2 + 2 ε2 2 2 2

(11)

where c2 = 2Q[a1 , . . . , an ]T and c3 = 2Q[0, 1, . . . , n − 1]T are constants. Construct W (ε, z) = c22 V (ε) + U (z), which is positive deﬁnite and proper Lyapunov function with respect to (7)(10). A simple calculation yields 2 √ ˙ (ε, z) ≤ −c22 L − ( n + n )Θ − c1 ε2 − L − Θc2 − c3 z2 .(12) W 2 2 2 2 From (12), it is clear that all the solutions of the closed-loop system exist and are well-deﬁned on the time interval [0, +∞). In addition, it can be concluded that there is a constant c > 0 such that 2 √ ˙ ≤ −cW, ∀ t ≥ T + 1, T := 2 · max{( n + n )Θ + c1 , Θc2 + c3 }. W 2 2

Hence

W (t) ≤ e−c(t−T −1) W (T + 1),

∀t ≥ T + 1.

This implies that there is a constant c¯ > 0 such that for i = 1, · · · , n, |εi | ≤ e−c(t−T −1)/2 c¯,

|zi | ≤ e−c(t−T −1)/2 c¯.

(13)

This, in turn, implies that |xi (t) − x ˆi (t)| ≤ (t + 1)i−1 e−c(t−T −1)/2 c¯, |ˆ xi (t)| ≤ (t + 1)i−1 e−c(t−T −1)/2 c¯,

∀t ≥ T + 1.

(14)

By the deﬁnition of ξi , it follows from (14) that all the states are ultimately bounded. Moreover, lim x ˆi (t) = 0

t→+∞

and

lim xi (t) = 0,

t→+∞

i = 1, · · · , n.

Remark 1. It must be noticed that the time-varying controller (9) is globally ultimately bounded. In fact, it follows from (9) and (13) that |u(t)| = |(t+1)n [bn z1 +bn−1 z2 +· · ·+b1 zn ]| ≤ (t+1)n e−c(t−T −1)/2 cˆ, t ≥ T +1.

244

C. Qian and W. Lin

This, together with the fact that u(t) is bounded on [0, T +1], implies globally ultimate boundedness of u(t). In addition, it is obvious that the controller converges to zero as time tends to inﬁnity. By same reasoning, it can be seen that the term Li (x1 − x ˆ1 ) in (6) is globally ultimately bounded and convergent to zero, although L itself is not bounded. Remark 2. Note that the gain function L(t) in observer and controller is a time-varying function that is unbounded as time tends to inﬁnity. This will cause implementation diﬃculty in computing the gain L(t) when t is very big. However, as we will see in the simulations of Examples 1 and 3, the trajectories will converge to the zero in quite a short time. Therefore, in real application, we can simply saturate the gain L(t) after a certain amount of time. Remark 3. In contrast to the time-invariant linear observer proposed in [17], in this paper we construct a time-varying linear observer that is substantially diﬀerent from the conventional design (as it does not use a copy of the uncertain system (1)). The time-varying feedback design, together with Assumption 2, enables us to deal with the diﬃculties caused by the uncertainties or nonlinearities of system (1). On the other hand, the use of time-varying gains in constructing both the observer and controller makes it possible to gradually eliminate the eﬀects of the unknown parameter θ. Remark 4. In the proof of Theorem 1, the observer gain L = t + 1 was used to simplify the proof. Notably, the gain L can be chosen as a slower growth √ function of t. For example, it is not diﬃcult to show that by choosing L = m m t + 1 with m ≥ 1, the following time-varying controller x ˆ2 x ˆn n + · · · + b1 n−1 u = −L bn x ˆ1 + bn−1 L L still achieves global regulation of (1), where bi ’s are the same parameters in (9). It should be pointed out that the time-varying feedback design method above can be easily extended to the more general case where θ in Assumption 2 is a time-varying function, not necessarily bounded by an unknown constant. Indeed, a similar argument shows that global regulation of the uncertain system (1) is still possible by time-varying output feedback, as long as θ(t) ≤ c(1 + tm ) for an integer m ≥ 0. Of course, in this case the timevarying gain L(t) used in designing the observer and controller should be a higher-order function of t instead of a linear function of t. Theorem 2. Suppose the time-varying nonlinear system (1) satisﬁes |φi (t, x, u)| ≤ c(1 + tm )(|x1 | + · · · + |xi |),

i = 1, · · · , n,

where c > 0 is an unknown constant. Then, global state regulation of (1) can be achieved by the time-varying output feedback controller (3).

Time-Varying Output Feedback for Uncertain Nonlinear Systems

245

Next we discuss some interesting consequences and applications of Theorem 1. The ﬁrst result shows how an output feedback controller can be used to regulate a family of time-varying linear systems of the form (4). Clearly Assumption 2 holds for system (4). Therefore, we have: Corollary 1. For an uncertain time-varying system (4), there is a timevarying linear dynamic output compensator of the form (3) regulating all the states of (4). Example 1. As discussed in Section 1, system (5) satisﬁes Assumption 2. By Theorem 1, system (5) can be globally regulated by linear time-varying output feedback. Choosing the coeﬃcients a1 = a2 = 2 and b1 = b2 = 2, the time-varying output feedback controller is given by x ˆ˙ 1 = x ˆ2 + 2(t + 1)(y − x ˆ1 ) 2 ˙x ˆ2 = u + 2(t + 1) (y − x ˆ1 ) u = −(t + 1)(2ˆ x2 + 2(t + 1)ˆ x1 ).

(15)

The simulation shown in Fig.1 demonstrates the eﬀectiveness of the output feedback controller (15). 3 2.5

^x 1

2 1.5

x1

1 0.5 0 −0.5

0

1

2

3 Time

4

5

6

1

2

3 Time

4

5

6

4 3 ^x 2

2 1

0 x 2 −1 −2 −3

0

Fig. 1. Transient responses of (5)-(15) with θ1 = 4, θ2 = 5 and (x1 (0), x2 (0), x ˆ1 (0), x ˆ2 (0)) = (1, 2, 3, 4)

246

C. Qian and W. Lin

Example 2. Consider a single-link robot arm system introduced in [4]. The state space model can be put into the following form (see [17]) x˙ 1 = x2 x˙ 2 = x3 −

F2 (t) K mgd x2 − x1 − (cos x1 − 1) J2 J2 J2

x˙ 3 = x4 x˙ 4 = v + y = x1

K2 K F1 (t) x3 − x1 − x4 2 J1 J2 N J2 N J1 (16)

where J1 , J2 , K, N, m, g, d are known parameters and F1 (t) and F2 (t) are viscous friction coeﬃcients that may vary continuously with time. In the case when F1 (t) and F2 (t) are bounded by known constants, the problem of global output feedback stabilization was solved in [17]. However, when F1 (t) and F2 (t) are bounded by unknown constants, the problem of how to use output feedback to globally regulate the states of (16) remains unsolved. Observe that Assumption 2 holds for system (16) because for an unknown θ > 0 2 2 2 2 2 F2 (t) 2 2 F1 (t) 2 2 2 2 2 | cos x1 − 1| ≤ |x1 |, ≤ θ|x x |, x 2 2 J2 2 2 2 J1 4 2 ≤ θ|x4 |. By Theorem 1, it is easy to construct a time-varying dynamic output feedback controller of the form (6)–(9) achieving global regulation of system (16). As shown in [17], one can design a single linear output feedback controller stabilizing simultaneously a family of nonlinear systems (1) under Assumption 2 with θ being a known constant. This kind of universal property is still preserved in the present paper. In fact, Theorem 1 indicates that there is a single time-varying output feedback controller of the form (3) such that an entire family of systems (1) satisfying Assumption 2 can be globally regulated. In addition, the time-varying controller (3) requires only the output instead of the full-state information of the systems. These two features make the controller (3) easy to implement and practically applicable, because only one signal (the output) needs to be measured and only one controller is used for the control of a whole family of uncertain systems. For example, it is easy to see that the time-varying output feedback controller (15) designed for the planar system (5) also simultaneously regulate the following uncertain systems in the plane:    2 2 2 x1 θ )  x˙ 1 = x2 + ln(1+u  x˙ 1 = x2  x˙ 1 = x2 + x1 1+u2 θ 2 θ(t) 2 x˙ 2 = u + d(t) ln(1 + x42 ) x ˙ = u + sin x 2 1+t2  x˙ 2 = u + x2 sin(x2 u)  2  y = x1 , |d(t)| ≤ θ y = x1 , |θ(t)| ≤ θ y = x1 In the remainder of this section, we discuss how Theorem 1 can be generalized to a class of uncertain nonlinear systems that are perturbed by a

Time-Varying Output Feedback for Uncertain Nonlinear Systems

247

time-varying dynamic system. To be speciﬁc, consider the following C 0 timevarying system z˙ = Ψ (t, z) x˙ 1 = x2 + φ1 (t, z, x, u) x˙ 2 = x3 + φ2 (t, z, x, u) .. . x˙ n = u + φn (t, z, x, u) y = x1 ,

(17)

where the z-subsystem is an nonautonomous system that is globally uniformly bounded. Since dynamic behavior of the z-dynamics of (17) cannot aﬀected by the feedback control u, the best control objective that is achievable is to regulate the partial state (x1 , · · · , xn ) of (17). However, due to the presence of the unmeasurable states z and (x2 , · · · , xn ), global regulation of the x-subsystem using output feedback is not a trivial problem. In what follows, we illustrate how Theorem 1 can lead to a solution to the problem. Corollary 2. If the z-subsystem of (17) is globally uniformly bounded with an unknown bound and |φi (t, z, x, u)| ≤ γ(z)(|x1 | + · · · + |xi |),

i = 1, · · · , n,

(18)

where γ(z) is an unknown C 0 function, then there exists a time-varying output feedback controller of the form (3) such that all the states (z, x) are globally ultimately bounded. Moreover lim (x1 (t), · · · , xn (t)) = 0.

t→+∞

Proof. By assumption, z(t) is bounded by an unknown constant θ. This, together with the condition (18), implies that Assumption 2 holds. As a consequence, Corollary 2 follows immediately from Theorem 1. The following example demonstrates an interesting application of Corollary 2. Example 3. Consider the four-dimensional system z˙1 = z2 z˙2 = −z13 x˙ 1 = x2 + z12 sin(x1 ) x˙ 2 = u + ln(1 + |x2 z2 |) y = x1

(19)

Obviously, the z-subsystem is globally stable and hence uniformly bounded, although its bound is not precisely known (depends on the initial state

248

C. Qian and W. Lin

(z1 (0), z2 (0)). In addition, it is easy to verify that |φ1 (z1 , x1 )| ≤ z12 |x1 | and | ln(1 + |x2 z2 |)| ≤ |z2 ||x2 |. As a result, (18) holds. By Corollary 2, one can design a time-varying output feedback controller to regulate (x1 , x2 ) of (19). Due to the universal property of the proposed output feedback control scheme, we simply adopt the output feedback control law (15) for system (19). The simulation results of the closed-loop system (19)-(15) are given in Fig. 2. 3 2.5

^x 1

2 1.5 x1

1 0.5 0 −0.5

0

1

2

3 Time

4

5

6

1

2

3 Time

4

5

6

4 3 2

^x

2

1 0 x2

−1 −2 −3

0

Fig. 2. Transient responses of (19)-(15) with (z1 (0), z2 (0), x1 (0), x2 (0), x ˆ1 (0), x ˆ2 (0)) = (1, 2, 1, 2, 3, 4)

3 Conclusion By integrating the idea of the use of time-varying gains and the output feedback domination design method introduced in [17], we have explicitly constructed in this paper a time-varying output feedback controller that achieves global state regulation, for a family of uncertain nonlinear systems whose output feedback regulation problem has remained unsolved until now. The proposed time-varying controller is linear and can simultaneously regulate a whole family of nonlinear systems bounded by any linearly growing triangular systems with unknown growth rates. It has also demonstrated, by means of examples, that the proposed controller is easy to design and implement.

Time-Varying Output Feedback for Uncertain Nonlinear Systems

249

References 1. Besancon G. (1998) State aﬃne systems and observer-based control, NOLCOS’98. Vol. 2, pp. 399–404. 2. Gauthier J. P., Kupka I. (1992) A separation principle for bilinear systems with dissipative drift, IEEE Trans. Automat. Contr., Vol. 37, pp. 1970–1974. 3. Gauthier J. P., Hammouri H., Othman S. (1992) A simple observer for nonlinear systems, applications to bioreactors, IEEE Trans. Automat. Contr., Vol. 37, pp. 875–880. 4. Isidori A. (1995) Nonlinear Control Systems, 3rd ed., New York: SpringerVerlag. 5. Krsti´c M., Kanellakopoulos I., Kokotovi´c P. K. (1995) Nonlinear and Adaptive Control Design, John Wiley. 6. Khalil H., Esfandiari F. (1995) Semi-global stabilization of a class of nonlinear systems using output feedback, IEEE Trans. Automat. Contr., Vol. 38, pp. 1412–1415. 7. Khalil H. K., Saberi A. (1987) Adaptive stabilization of a class of nonlinear systems using high-gain feedback, IEEE Trans. Automat. Contr., Vol. 32, pp. 875– 880. 8. Krener A. J., Isidori A. (1983) Linearization by output injection and nonlinear observer, Systems & Control Letters, Vol. 3, pp. 7–52. 9. Krener A. J., Respondek W. (1985) Nonlinear observers with linearizable error dynamics, SIAM J. Contr. Optimiz. Vol. 23, pp. 197–216. 10. Lin W. (1995) Input saturation and global stabilization of nonlinear systems via state and output feedback, IEEE Trans. Automat. Contr., Vol. 40, pp. 776–782. 11. Lin W. (1995) Bounded smooth state feedback and a global separation principle for non-aﬃne nonlinear systems, Systems & Control Letters, Vol. 26, pp. 41–53. 12. Lin W., Qian C. (2002) Adaptive control of nonlinearly parameterized systems: the smooth feedback case, IEEE Trans. Automat. Contr., vol. 47, pp. 1249– 1266. A preliminary version of this paper was presented in Proc. of the 40th IEEE CDC, Orlando, FL., 4192–4197 (2001). 13. Marino R., Tomei P (1991) Dynamic output feedback linearization and global stabilization, Systems & Control letters, Vol. 17, pp. 115–121. 14. Marino R., Tomei P. (1993) Global adaptive output feedback control nonlinear systems, Part II: noninear parameterization, IEEE Trans. Automat. Contr., Vol. 38, pp. 33–48. 15. Mazenc F., Praly L., Dayawansa W. D. (1994) Global stabilization by output feedback: examples and counterexamples, Systems & Control letters, Vol. 23, pp. 119–125. 16. Praly L. (2001) Asymptotic stabilization via output feedback for lower triangular systems with output dependent incremental rate, Proc. 40th IEEE CDC, Orlando, FL, pp. 3808–3813. 17. Qian C., Lin W. (2002) Output Feedback Control of a Class of Nonlinear Systems: A Nonseparation Principle Paradigm, IEEE Trans. Automat. Contr., Vol. 47, pp. 1710–1715. 18. Tsinias J. (1991) A theorem on global stabilization of nonlinear systems by linear feedback, System & Control Letters, Vol. 17, pp. 357–362.

250

C. Qian and W. Lin

19. Tsinias J. (2000) Backstepping design for time-varying nonlinear systems with unknown parameters, System & Control Letters, Vol. 39, pp. 219–227. 20. Xia X. H., Gao W. B. (1989) Nonlinear observer design by observer error linearization, SIAM J. Contr. Optimiz., Vol. 27, pp. 199–216.

Stability of Nonlinear Hybrid Systems G. Yin1 and Q. Zhang2 1 2

Wayne State University, Detroit, MI 48202, [email protected] University of Georgia, Athens, GA 30602, [email protected]

Dedicated to Professor Arthur J. Krener on the Occasion of His 60th Birthday Summary. This work is devoted to stability of nonlinear hybrid systems (or nonlinear systems with regime switching). The switching regime is described by a ﬁnitestate Markov chain. Both continuous-time and discrete-time systems are considered. Aiming at reduction of complexity, the system is setup as one with two-time scales which gives rise to a limit system as the jump rate of the underlying Markov chain goes to inﬁnity. Using perturbed Liapunov function methods, the stability of the original system in an appropriate sense is obtained provided that the corresponding limit system is stable.

1 Introduction Stability of nonlinear hybrid systems (or nonlinear systems with regime switching) have drawn much needed attention in recent years. This is because many of such systems arise from various applications in estimation, detection, pattern recognition, signal processing, telecommunications, and manufacturing among others. In this paper, we often refer to such systems as hybrid systems. Here, by hybrid systems, we mean that the systems under consideration are subject to both usual dynamics given by diﬀerential or diﬀerence equations as well as by discrete events represented by jump processes. The resulting systems have the distinct features that instead of one ﬁxed system, one has a number of systems with regime changes modulated by certain jump processes such that among diﬀerent regimes, the dynamics are quite diﬀerent. In [6], piecewise multiple Liapunov functions are used to treat stability of both switching systems and hybrid systems. In [26] (see also [20]), the authors consider invariant sets for hybrid dynamical systems and obtain several stability properties using Liapunov-like functions. Stochastic sequences are considered in [3] where the stability of stochastic recursive sequences is established when the underlying process is not necessarily Markovian; see also [2] for a queueing W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 251–264, 2003.

c Springer-Verlag Berlin Heidelberg 2003

252

G. Yin and Q. Zhang

system model with retrials. Additional recent progress in stability of hybrid systems can also be found in [10, 17, 18], among others. In this paper, we consider both a continuous-time model and a discretetime model. In the continuous-time model, the underlying system is modeled by a diﬀerential equation, while in the discrete-time model, the system under consideration is governed by a diﬀerence equation. Both systems are subject to random switching. We model the modulating jump process by a Markov chain. Recent study of hybrid systems has indicated that such a formulation is more general and appropriate for a wide variety of applications; see, for example, feedback linear systems [5], robust linear control [19], Markov decision problems [15], portfolio selection problems and nearly optimal controls [32]. Due to various modeling considerations, the Markov chain often has a large state space, which makes it diﬃcult to analyze the stability of the underlying systems. To overcome the diﬃculties, by noting the high contrasts of the transition rates and introducing a small parameter ε > 0, one can incorporate two-time-scale Markov chains into the problem under consideration. For some recent developments on two-time-scale (or singularly perturbed) Markovian systems, we refer the reader to [1, 21] among others. Note that the introduction of the small parameter ε > 0 is only a convenient way for the purpose of time-scale separation. In the previous work, by concentrating on the hierarchical approach, we accomplish the reduction of complexity by showing that the underlying system converges to a limit system in which the coeﬃcients of the dynamics are averaged out with respect to the invariant measures of the Markov chain. Then using the optimal controls of the limit systems as a reference, we construct controls for the original systems and demonstrate their asymptotic optimality; see for example, [15, 16, 27]. In the aforementioned work, when we deal with the underlying systems, we have focused on the study of asymptotic properties for ε → 0 and t in a bounded interval for continuous-time systems, and ε → 0 and k → ∞, but εk remains to be bounded for discrete-time systems. As alluded to before, in this paper, we examine the reduction of complexity from a diﬀerent angle, namely, from a stability point of view via a singular perturbation approach. We mainly concern ourselves with the behavior of the systems as ε → 0 and t → ∞ for a continuous-time systems and as ε → 0, k → ∞, and εk → ∞ for a discretetime system. We show that if the limit system (or reduced system) is stable, then the original system is also stable for suﬃcient small ε > 0. The approach we are taking is perturbed Liapunov function method. The Liapunov function of the limit system is used, and perturbations of the Liapunov functions are then constructed. We use the limit system to carry out the needed analysis so it is simpler than treating the original system directly. To achieve the reduction of complexity, the original dynamic systems are compared with the limit systems and then perturbed Liapunov function methods are used to obtain the desired bounds. Our results indicate that one can concentrates on the properties of much simpler limit systems to make inference about the stability of the original more complex systems.

Stability of Nonlinear Hybrid Systems

253

The rest of the paper is arranged as follows. Section 2 presents the results for nonlinear dynamic systems with regime switching in continuous time. Section 3 proceeds with the development of discrete-time Markov modulated systems. The conditions needed are given, whereas the proofs can be found in [4, 29].

2 Continuous-Time Models Let α(·) = {α(t) : t ≥ 0} be a continuous-time Markov chain with state space M = {1, . . . , m} and generator Q = (qij ) ∈ Rm×m satisfying qij ≥ 0 for i = j and j∈M qij = 0 for each i ∈ M. For a real-valued function g(·) deﬁned on M, 6 t g(α(t)) − Qg(·)(α(ς))dς is a martingale, 0

where for each i ∈ M, Qg(·)(i) =

qij g(j) =

j∈M

qij (g(j) − g(i)).

(1)

j=i

A generator Q or equivalently its corresponding Markov chain is said to be irreducible if, the system of equations νQ = 0,

m

νi = 1

(2)

i=1

has a unique solution ν = (ν1 , . . . , νm ) satisfying νi > 0 for i = 1, . . . , m. Such a solution is termed a stationary distribution. We consider the case that the Markov chain has a large state space, i.e., the cardinality |M| is a large number. In various applications, to obtain optimal or nearly optimal controls of hybrid systems involving such a Markov chain, the computation complexity becomes a pressing issue due to the presence of a large number of states. As suggested in [27], the complexity can be reduced by observing that not all of the states are changing at the same rate. Some of them vary rapidly and others change slowly. To take advantage of the hierarchical structure, the states can be divided naturally into several classes. To reﬂect the diﬀerent rates of changes, we introduce a small parameter ε > 0 into the system. The Markov chain becomes one depending on ε, i.e., α(t) = αε (t). Assume that the generator of αε (t) is Qε with Qε =

1- Q + Q, ε

(3)

- and Q are themselves generators. If Q = 0, the generator Qε is where Q changing at a fast pace, and the corresponding Markov chain varies rapidly.

254

G. Yin and Q. Zhang

= 0, the Markov chain The smaller the ε is, the faster the chain varies. If Q is also subject to weak interactions in addition to the rapid variations due to Q/ε. Consider the case that the states of the Markov chain are divisible to a number of classes such that within each class, the transitions take place at a fast pace, whereas among diﬀerent classes, the transitions appear less frequently. Such a formulation can be taken care of by taking advantage of - In this paper, we assume that the structure of Q. - = diag(Q -1 , . . . , Q - l ), Q

(4)

where diag(A1 , . . . , Al ) is a block diagonal matrix having matrix entries A1 , . . . , Al of appropriate dimensions. Let Mi = {si1 , . . . , simi }, i = 1, 2, . . . , l, - i . Then, M = M1 ∪ · · · ∪ Ml . Moreodenote the state spaces corresponding Q ver, we assume that the resulting Markov chain corresponding to each block Qi is recurrent. To proceed, we deﬁne an aggregated process αε (t) by αε (t) = i, when αε (t) ∈ Mi , for each i = 1, . . . , l. That is, we lump all the states in Mi as a “super” state. This idea can be schematically presented as in the ﬁgures. Fig. 1 presents the states in the original Markov chain, and Fig. 2 depicts the aggregated super-states, represented by the circles, after lumping all the states in Mi together.

Fig. 1. Original Markovian States

Note that in general that the aggregated process αε (·) is no longer Markov. However, this will not concern us since it can be shown that the aggregated process converges weakly to a Markov chain whose generator is an average of with respect to the stationary measures. If l, the number the slow generator Q of classes (or groups) satisﬁes l m, then working with the limit systems will substantially reduce the amount of complexity. Let f (·) : Rn × M → Rn , σ(·) : Rn × M → Rn×n , and w(·) be a standard n-dimensional Brownian motion that is independent of αε (·). Let xε (t) be the

Stability of Nonlinear Hybrid Systems

255

Fig. 2. Aggregated Super States

state variable of a switching diﬀusion at time t ≥ 0, governed by the following equation: dxε (t) = f (xε (t), αε (t))dt + σ(xε (t), αε (t))dw xε (0) = x0 , αε (0) = α0 ,

(5)

where f (·) and σ(·) are appropriate functions satisfying suitable conditions. This model is motivated by, for example, the price movements in a stock market, where xε (t) represents the price of a stock, f (x, α) = f0 (x, α)x, σ(x, α) = σ0 (x, α)x, and f0 (x, α) and σ0 (x, α) are the expected appreciation rate and volatility, respectively. The Markov chain describes the market trends and other economic factors. A special form of (5) is that σ(·) ≡ 0. That is a system of nonlinear ordinary diﬀerential equations with regime switching. As ε → 0, the dynamic system is close to an averaged system of switching diﬀusions. Our interest is to ﬁgure out the stability of the dynamic system governed by (5). Deﬁnition 1.. Property (H): For an appropriate function h(·) (either h(·) : Rn × M → Rn or h(·) : Rn × M → Rn×n ), we say h(·) satisﬁes property (H) if for some K > 0 and for each α ∈ M, h(·, α) is continuously diﬀerentiable, and |hx (x, α)x| ≤ K(|x| + 1); h(0, α) = 0 for all α ∈ M. Let V (x, i) be a function. The notation ∂ ι V (x, i) is used in this paper: If ι = 1, ∂V (x, i) is the gradient; if ι = 2, ∂ 2 V (x, i) is the Hessian; if ι > 2, ∂ ι V (x, i) is the usual multi-index notation of mixed partial derivatives. References on matrix calculus and Kronecker products can be found, for instance in [8], among others. Deﬁnition 2.. Property (V): For each i = 1, . . . , l and for some positive integer n0 , there is a function V (·, i) that is n0 -times continuously diﬀerentiable with respect to x, |∂ ι V (x, i)||x|ι ≤ K(V (x, j) + 1) for 1 ≤ ι ≤ n0 − 1, i, j = 1, . . . , l, and |∂ n0 V (x, i)| = O(1) (where for 2 ≤ ι ≤ n0 , ∂ ι V (x, i) denotes the ιth derivative of V (x, i)), V (x, i) → ∞ as |x| → ∞, K1 (|x|n0 + 1) ≤ V (x, i) ≤ K2 (|x|n0 + 1).

256

G. Yin and Q. Zhang

Remark 1.. Note that in [4, 29], a global Lipschitz condition was assumed for h(·, α) in property (H). It turns out that this condition can be relaxed by assuming that the associated limit martingale problem has a unique (in the sense of in distribution) solution for each initial condition. Remark 2.. For the original systems, in order to establish the desired stability, one needs m Liapunov functions. Our results demonstrate that the problem can be simpliﬁed by using l Liapunov functions for the limit system instead. The stability analysis can be done based on a model of much smaller dimension. When l m, the advantage of the singular perturbation approach is particularly pronounced. In the analysis to follow, the function V (x, i) is a Liapunov function for the limit hybrid stochastic diﬀerential equation. The growth and smooth conditions will be satisﬁed if V (x, i) is a polynomial of order n0 or has polynomial growth of order n0 . It follows from this condition that |∂ ι V (x, i)|[|f (x, i)|ι + |σ(x, i)|ι ] ≤ K(V (x, i) + 1). For the continuous-time problem, we use the following assumptions. - i is irreducible. (C1) For each i = 1, . . . , l, Q (C2) The functions f (·) and σ(·) satisfy property (H). Lemma 1.. Assume condition (C1). Then the following assertions hold: (a) The probability distribution vector pε (t) ∈ R1×m with pε (t) = (P (αε (t) = sij ), i = 1, . . . , l, j = 1, . . . , mi ), satisﬁes pε (t) = θ(t)˜ ν + O(ε(t + 1) + e−κ0 t/ε )

(6)

for some κ0 > 0 , and θ(t) = (θ1 (t), . . . , θl (t)) ∈ R1×l satisﬁes dθ(t) = θ(t)Q, θ(0) = p(0)11. dt (b) For the transition probability matrix P ε (t), we have P ε (t) = P (0) (t) + O ε(t + 1) + e−κ0 t/ε ,

(7)

where P (0) (t) = 11Θ(t)˜ ν and dΘ(t) = Θ(t)Q, Θ(0) = I. dt

(8)

(c) The aggregated process αε (·) converges weakly to α(·) as ε → 0, where α(·) is a Markov chain generated by Q.

Stability of Nonlinear Hybrid Systems

257

To proceed, we show that the limit of (xε (·), αε (·)) is given by dx(t) = f (x(t), α(t))dt + σ(x(t), α(t))dw,

(9)

with x(0) = x0 , α(0) = α0 , where for each i ∈ M, f (x, i) =

mi

νji f (x, sij ),

j=1

σ(x, i)σ (x, i) = Ξ(x, i) =

mi

νji Ξ(x, sij ),

j=1

where Ξ(x, sij ) = σ(x, sij )σ (x, sij ) and z denotes the transpose of z for either a matrix or a vector. In fact, associated with (9), there is a martingale problem (see [27, Appendix] and the references therein) with operator 1 LF (x, i) = Fx (x, i) + tr[Fxx (x, i)Ξ(x, i)] + QF (x, i), 2

(10)

for any C 2 functions F (·, i) with compact support. We need another condition. Note that only uniqueness in the sense of in distribution is needed. We do not require the pathwise properties of the solution. Thus the condition is a lot milder than that of the pathwise counter part. Suﬃcient conditions may be established by using a technique such as in [27, Lemma 7.18]. (U) The martingale problem with operator L given by (10) has a unique solution in distribution. Lemma 2.. Assume (C1), (C2) and (U). Then (xε (·), αε (·)), the solution of (5) converges weakly to (x(·), α(·)) as ε → 0 where (x(·), α(·)) is the solution of (9). Remark 3.. This lemma indicates that associated with the original system, there is a limit process, in which, the system is averaged out with respect to the stationary measure. Suppose that the system represents a manufacturing system. As pointed out in, for example, in [23], the management at the higher level of the production decision-making hierarchy can ignore daily ﬂuctuations in machine capacities and/or demand variations by looking at only the “average of the system” to make long-term planning decisions. Remark 4.. Note that in the above lemma, when σ(·) ≡ 0, under (C1) and (C2), the sequence (xε (·), αε (·)) given in (5) converges weakly to (x(·), α(·)) such that x(·) satisﬁes d x(t) = f (x(t), α(t)), x(0) = x0 , α(0) = α0 , dt where f (x, i) =

mi j=1

νji f (x, sij ), for i = 1, 2, . . . , l.

(11)

258

G. Yin and Q. Zhang

9t Remark 5.. The quantity 0 I{αε (s)=sij } ds is known as occupation measure of the Markov chain since it represents the amount of time the chain spends in state sij . It is a useful quantity, for instance, we can rewrite x˙ ε = f (xε (t), αε (t)) in the variational form as ε

x (t) = x0 +

mi 6 l i=1 j=1

0

t

f (xε (s), sij )I{αε (s)=sij } ds,

which often facilitates the require analysis. Lemma 3 indicates that 6 ∞ e−t I{αε (t)=sij } dt 0

can be approximated by that of the aggregated process 6 ∞ e−t νji I{αε (t)=i} dt 0

in the sense that the mean squares error is of the order O(ε) as given in (12). Although there are certain similarities, Lemma 3 is diﬀerent from that of [27, p.170], since in lieu of working with a ﬁnite horizon, inﬁnite time intervals are dealt with. To ensure the integral to be well deﬁned, a discount factor e−t is used in (12). Lemma 3.. Assume (C1). Then for each i = 1, . . . , l and j = 1, . . . , mi , *6 ∞ +2 E e−t (I{αε (t)=sij } − νji I{αε (t)=i} )dt = O(ε), (12) 0

where

νji

denotes the jth component of ν i for i = 1, . . . , l and j = 1, . . . , mi .

Theorem 1.. Assume conditions (C1) and (C2), and suppose that for each i = 1, . . . , l, there is a Liapunov function V (x, i) satisfying property (V), and having continuous mixed partial derivatives up to the order 4 such that LV (x, i) ≤ −γV (x, i), for some γ > 0, 1 LV (x, i) = Vx (x, i)f (x, i) + tr[Vxx (x, i)Ξ(x, i)] + QV (x, ·)(i), 2

(13)

for i ∈ M. Let EV (x0 , α0 ) < ∞. Then EV (xε (t), αε (t)) ≤ e−γt EV (x0 , α0 )(1 + O(ε)) + O(ε), i.e., the original system is stable. Note that QV (x, ·)(i) is the coupling term associated with the switching Markov chain. It is the ith component of the column vector Q(V (x, 1), . . . , V (x, l)) . To illustrate, let us look at a simple example.

Stability of Nonlinear Hybrid Systems

259

Example 1.. Consider the case of a scalar case for simplicity. Suppose that n = 1, σ(·) ≡ 0, and that the system (5) becomes xε(t) ˙ = f (xε (t), αε (t)). - is irreducible. In this Suppose in addition that l = 1. Thus the generator Q ε case, the Markov chain α (·) can be viewed as a fast varying noise process. In the limit, it will be replaced by its stationary measure. Thus the limit becomes x(t) ˙ = f (x(t)),

f (x) =

m

νi f (x, i).

i=1

−1 1 = 0. Then ν = (2/3, 1/3). Let and Q 2 −2 f (x, 1) = −x and f (x, 2) = ln x. Then V (x) = x2 can be used as a Liapunov function for the limit ODE. Theorem 1 implies that the original system is also stable. It is worthwhile to note that Markovian property helps to stabilize the underlying system (see [18]). As a result, it need not be the case that all the individual components of the system be stable. In fact, as seen in the example, f (x, 2) = ln x yielding an unstable component. However, the longrun behavior of the system is dominated by the average with respect to ν. As long as the averaged system is stable, the original system will be stable. This discussion carries over to the case that l > 1. In the multi-class case, it need not be the case that all the components are stable. All we need is that the averaged system with respect to the stationary measure ν i is stable. - = Q = Suppose that Q

3 Discrete-Time Models In this section, we consider the stability of a discrete-time system. We consider a Markov chain αkε with ﬁnite state space M = {1, . . . , m} and transition probability matrix P ε = P + εQ,

(14)

where P is a transition probability matrix of a time-homogeneous Markov chain satisfying P = diag(P 1 , . . . , P l ),

(15)

with each P i , i = 1, . . . , l, being a transition probability matrix of appropriate dimension, and Q = (qij ) is a generator of a continuous-time Markov chain. Such Markov chains are often referred to as nearly completely decomposable chains. Recent studies have focused on asymptotic properties such as asymptotic expansion of probability vectors [28], asymptotic normality of an

260

G. Yin and Q. Zhang

occupation measure [30], and near-optimal controls of hybrid systems [16]). Similar to the continuous-time models, deﬁne an aggregated process αεk by αεk = i, when αkε ∈ Mi , for each i = 1, . . . , l. Deﬁne also interpolated process αε (t) = αεk , for t ∈ [εk, εk + ε). Let f (·) : Rn × M → Rn , σ(·) : Rn × M → Rn×n be appropriate functions satisfying suitable conditions and wk be an external random noise independent of αkε . Let xεk be the state at time k ≥ 0, and √ xεk+1 = xεk + εf (xεk , αkε ) + εσ(xεk , αkε )wk (16) xε0 = x0 , α0ε = α0 . As ε → 0, the dynamic system is close to an averaged system of switching diﬀusion. Again, our interest is to ﬁgure out the stability of the dynamic system governed by (16), which will be carried out using the stability of its limit system. For the discrete-time problems, we assume: (D1) P ε , P , and Pi for i ≤ l are transition probability matrices such that for each i ≤ l, Pi is aperiodic and irreducible, i.e., for each i, the corresponding Markov chain is aperiodic and irreducible. (D2) f (·) and σ(·) satisfy property (H). (D3) {wk } is a sequence of independent and identically distributed random variables with zero mean and Ewk wk = I, the identity matrix. Remark 6.. Condition (D2) implies that f (x, α) and σ(x, α) grow at most linearly. A typical example of the noise {wk } is a sequence of Gaussian random variables. In fact, in our study, we only need a central limit theorem holds for a scaled sequence of the random noise. Thus the independence condition can be relaxed considerably by allowing mixing type noise together with certain moment conditions. Nevertheless, the independence assumption does make the presentation much simpler. For a hybrid LQ problem with regime switching, conditions (D2)–(D3) are readily veriﬁed. The conditions are also veriﬁed for certain problems arising from wireless communications such as CDMA systems. Lemma 4.. Assume (D1) holds. For each i = 1, . . . , l and j = 1, . . . , mi , deﬁne πij = ε

∞ k=0

e−kε [I{αεk =sij } − νji I{αεk ∈Mi } ].

Then E(πij )2 = O(ε), i = 1, . . . , l and j = 1, . . . , mi .

(17)

Stability of Nonlinear Hybrid Systems

261

Lemma 5.. Assume conditions (D1)-(D3), and (U). Then the sequence (xε (·), αε (·)), the solutions of (16), converges weakly to (x(·), α(·)), the solution of (9). Remark 7.. When σ(·) ≡ 0, the sequence (xε (·), αε (·)) converges weakly to (x(·), α(·)), the solution of (11). Remark 8.. Similar to the continuous-time problems, the occupation measures allow us to work with the system with regime switching modulated by the Markov chain eﬀectively. For example, dealing with (16), we can write it as xεk+1 = xεk + ε

mi l i=1 j=1

f (xεk , sij )I{αεk =sij } +

mi l √ ε σ(xεk , sij )I{αεk =sij } . i=1 j=1

It indicates that the Markov chain sojourns in a state sij ∈ M for a random duration, during which the system takes a particular conﬁguration. Then it switches to si1 j1 ∈ M with si1 j1 = sij and the system then takes a new conﬁguration and so on. A discrete-time version of Lemma 1 asserts that the probability distribution and the k-step transition probability matrices can be approximated by means of asymptotic expansions. The leading terms have the interpretation of total probability. That is, they consist of the probabilities of jumps among the l groups and the stationary distribution of a given group. Remark 9.. As observed in Remark 6, if {wk } is a sequence of Gaussian random variables, then the moment condition E|wk |n1 holds for any positive integer n1 < ∞. Moreover, we can treat correlated random variables of mixing type. However, for notational simplicity, we conﬁne ourselves with the current setup. Theorem 2.. Assume that (D1)–(D3) hold, that n1 in property (V) satisﬁes n0 ≥ 3, and that E|wk |n1 < ∞ for some integer n1 ≥ n0 , that LV (x, i) ≤ −γV (x, i) for some γ > 0, where the operator is deﬁned by 1 LV (x, i) = Vx (x, i)f (x, i)+ tr[Vxx (x, i)Ξ(x, i)] + QV (x, ·)(i), 2

(18)

for i ∈ M, and that EV (x0 , α0 ) < ∞. Then EV (xεk+1 , αεk+1 ) ≤ e−εγk EV (x0 , α0 ) + O(ε),

(19)

i.e., the original system is stable. Example 2.. Consider a two-state Markov chain αkε ∈ {1, 2} and a scalar system (16). Suppose that f (x, 1) = −5x, f (x, 2) = 5(x + (ln x/4)), σ(x, 1) = 1/2 1/2 (4x)/(x − 1), and σ(x, 2) = cos x and that P = . Then the limit 3/4 1/4 system is

262

G. Yin and Q. Zhang

1 dx = (−x + ln x)dt + 2

0:

1 3 16x2 2 + cos2 x dw. 5 (x − 1)2 5

The stability of the limit yields that of the original system in the sense as stated in Theorem 2. To proceed, we further exploit the recurrence of the underlying systems. Instead of (V), we will use a modiﬁed condition for the next result. Deﬁnition 3.. Property (V’): For each i = 1, . . . , l, there is a twice continuously diﬀerentiable Liapunov function V (x, i) such that |Vx (x, i)||x| ≤ K(V (x, i) + 1), for each i,

(20)

that minx V (x, i) = 0, that Vx (x, i)f (x, i) ≤ −c0 for some c0 > 0, and that V (x, i) → ∞ as |x| → ∞. For some λ0 > 0 and λ1 > λ0 , deﬁne B0 = {x : V (x, i) ≤ λ0 , i = 1, . . . , l}, B1 = {x : V (x, i) ≤ λ1 , i = 1, . . . , l}. Let τ0 be the ﬁrst exit time from B0 of the process xεk , and τ1 be the ﬁrst return time of the process after τ0 . That is, τ0 = min{k : xεk ∈ B0 }, τ1 = min{k ≥ τ0 : xεk ∈ B0 }. The random time τ1 − τ0 is known as the recurrence time. It is the duration that the process wanders around from the exit of B0 to return to B0 . Theorem 3.. Consider the system (16). Assume (D1)–(D3) and (V’) are satisﬁed. Then for some 0 < c1 < c0 , Eτ0 (τ1 − τ0 ) ≤

Eτ0 V (xετ0 , αετ0 )(1 + O(ε)) + O(ε) , c1

(21)

where Eτ0 denotes the conditional expectation with respect to the σ-algebra Fτ0 . Remark 10.. The above theorem indicates that for suﬃciently small ε > 0, xεk is recurrent in the sense that if xετ0 ∈ B1 − B0 , then the conditional mean recurrence time of τ1 − τ0 has an upper bound [λ1 (1 + O(ε)) + O(ε)]/c1 . In addition to the conditional moment bound, we may also obtain the probability bound of the form for κ > 0, 2 Eτ0 V (xετ0 , αετ0 +1 )(1 + O(ε)) + O(ε) P sup V (xεk , αεk ) ≥ κ2Fτ0 ≤ . κ τ0 ≤k<τ1 Compared with Theorem 2, the growth conditions are much relaxed. The main reason is: We do not need the moment estimates since we can work with truncated Liapunov function.

Stability of Nonlinear Hybrid Systems

263

4 Conclusion Stability of nonlinear switching systems have been considered. Our discussion has been conﬁned to the case that the Markov chains have only recurrent states. It is known that for ﬁnite state Markov chains (either in continuous time or discrete time) can either have all recurrent states or include transient states in addition to recurrent states. The results presented above can be extended to such cases. Detailed development together with verbatim proofs can be found in [4, 29]. For future study, eﬀort may directed to stabilization of nonlinear dynamic systems with regime switching.

References 1. Abbad M, Filar JA, Bielecki TR (1992) Algorithms for singularly perturbed limiting average Markov control problems, IEEE Trans. Automat. Control AC37: 1421–1425. 2. Altman E, Borovkov AA (1997) On the stability of retrial queues, QUESTA, 26: 343–363. 3. Altman E, Hordijk A (1997) Applications of Borovkov’s renovation theory to nonstationary stochastic recursive sequences and their control, Adv. Appl. Probab., 29: 388–413. 4. Badowski G, Yin G (2002) Stability of hybrid dynamic systems containing singularly perturbed random processes, IEEE Trans. Automat. Control, 47: 2021–2032. 5. Blair WP, Sworder DD (1986) Feedback control of a class of linear discrete systems with jump parameters and quadratic cost criteria, Int. J. Control, 21: 833–841. 6. Branicky MS (1998) Multiple Liapunov functions and other analysis tools for switched and hybrid systems, IEEE Trans. Automat. Control, 43: 475–482. 7. Ethier SN, Kurtz TG (1986) Markov Processes: Characterization and Convergence, J. Wiley, New York. 8. Graham A (1981) Kronecker Products and Matrix Calculus with Applications, Ellis Horwood Ltd., Chinchester. 9. Iosifescu M (1980) Finite Markov Processes and Their Applications, Wiley, Chichester. 10. Ji Y, Chizeck HJ (1990) Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control, IEEE Trans. Automat. Control, 35: 777–788. 11. Krishnamurthy V, Wang X, Yin G (2002) Spreading code optimization and adaptation in CDMA via discrete stochastic approximation, preprint. 12. Kushner HJ (1984) Approximation and Weak Convergence Methods for Random Processes, with applications to Stochastic Systems Theory, MIT Press, Cambridge, MA. 13. Kushner HJ, Yin G (1997) Stochastic Approximation Algorithms and Applications, Springer-Verlag, New York. 14. LaSalle JP (1979) The Stability of Dynamical Systems, SIAM, Philadelphia, PA.

264

G. Yin and Q. Zhang

15. Liu RH, Zhang Q, Yin G (2001) Nearly optimal control of singularly perturbed Markov decision processes in discrete time, Appl. Math. Optim., 44: 105–129. 16. Liu RH, Zhang Q, Yin G (2002) Asymptotically optimal controls of hybrid linear quadratic regulators in discrete time, Automatica, 38: 409–419. 17. Mao X (1994) Exponential Stability of Stochastic Diﬀerential Equations, Marcel Dekker, New York. 18. Mao X (1999) Stability of stochastic diﬀerential equations with Markovian switching, Stochastic Process Appl., 79: 45–67. 19. Mariton M, Bertrand P (1985) Robust jump linear quadratic control: A mode stabilizing solution, IEEE Trans. Automat. Control, AC-30: 1145–1147. 20. Michel AN, Hu B (1999) Towards a stability theory of general hybrid dynamical systems, Automatica, 35: 371–384. 21. Pervozvanskii AA, Gaitsgori VG (1988) Theory of Suboptimal Decisions: Decomposition and Aggregation, Kluwer, Dordrecht. 22. Simon HA, Ando A (1961) Aggregation of variables in dynamic systems, Econometrica 29: 111–138. 23. Sethi SP, Zhang Q (1994) Hierarchical Decision Making in Stochastic Manufacturing Systems, Birkh¨ auser, Boston. 24. Solo V, Kong X (1995) Adaptive Signal Processing Algorithms, Prentice-Hall, Englewood Cliﬀs, NJ. 25. Tsai CC (1998) Composite stabilization of singularly perturbed stochastic hybrid systems, Internat. J. Control, 71: 1005–1020. 26. Ye H, Michel AN, Hou L (1998) Stability theory for hybrid dynamical systems, IEEE Trans. Automat. Control, AC-43: 461–474. 27. Yin G, Zhang Q (1998) Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, Springer-Verlag, New York. 28. Yin G, Zhang Q (2000) Singularly perturbed discrete-time Markov chains, SIAM J. Appl. Math. 61: 834–854. 29. Yin G, Zhang Q (2003) Stability of Markov modulated discrete-time dynamic systems, to appear in Automatica. 30. Yin G, Zhang Q, Badowski G (2000) Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient states, Ann. Appl. Probab. 10: 549–572. 31. Yin G, Zhang Q, Badowski G (2003) Discrete-time singularly perturbed Markov chains: Aggregation, occupation measures, and switching diﬀusion limit, Adv. in Appl. Probab. 35: in press. 32. Zhang Q, Yin G (2003) Nearly optimal asset allocation in hybrid stockinvestment models, preprint.

The Uncertain Generalized Moment Problem with Complexity Constraint Christopher I. Byrnes1 and Anders Lindquist2 1 2

Washington University, St. Louis, MO, USA, [email protected] Royal Institute of Technology, Stockholm, Sweden, [email protected]

This paper is dedicated to Arthur Krener – a great researcher, a great teacher and a great friend – on the occasion of his 60th birthday. In this work we study the generalized moment problem with complexity constraints in the case where the actual values of the moments are uncertain. For example, in spectral estimation the moments correspond to estimates of covariance lags computed from a ﬁnite observation record, which inevitably leads to statistical errors, a problem studied earlier by Shankwitz and Georgiou. Our approach is a combination of methods drawn from optimization and the diﬀerentiable approach to geometry and topology. In particular, we give an intrinsic geometric derivation of the Legendre transform and use it to describe convexity properties of the solution to the generalized moment problems as the moments vary over an arbitrary compact convex set of possible values. This is also interpreted in terms of minimizing the Kullback-Leibler divergence for the generalized moment problem.

1 Introduction Let α0 , α1 , · · · , αn be a sequence of C 2 functions deﬁned on some interval I of the real line. Given a suitable sequence of complex numbers, c0 , c1 , · · · , cn , we are interested in moment problems of the form 6 αk (t)Φ(t)dt = ck , k = 0, 1, · · · , n, (1) I

for functions Φ ∈ L1+ (I), where L1+ (I) is the space of positive functions in L1 (I). In fact, suppose that α0 , . . . , αr−1 are real functions and αr , . . . , αn are complex-valued functions whose real and imaginary parts, taken together with

This research was supported in part by grants from AFOSR, VR, Institut MittagLeﬄer, and SBC.

W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 267–278, 2003.

c Springer-Verlag Berlin Heidelberg 2003

268

C.I. Byrnes and A. Lindquist

α0 , . . . , αr−1 are linearly independent over R. This is no restriction since a purely imaginary moment condition can always be reduced to a real one. For simplicity of exposition, we also assume that α0 = 1. Let P be the real vector space that is the sum of the real span of α0 , . . . , αr−1 and the complex span of αr , . . . , αn . Hence, the real dimension of P is 2n − r + 2. If P+ denotes the subset of all functions in P that have a positive real part on I, then P+ is a nonempty, open, convex subset of dimension 2n − r + 2. For this moment problem to have a solution it is clearly necessary that the sequence c0 , c1 , · · · , cn is positive in the sense that c, q := Re

n

qk ck > 0

(2)

k=0

for all (q0 , q1 , · · · , qn ) ∈ Rr × Cn−r+1 such that q :=

n

qk αk ∈ P+ .

(3)

k=0

Indeed, 6 c, q =

Re I

n

qk αk Φdt > 0,

k=0

whenever (3) holds. If C+ denotes the space of positive sequences, then C+ is a nonempty, open, convex subset of dimension 2n − r + 2. In [3] we considered the problem to ﬁnd, for each Ψ in some class G+ , the particular solution Φ to the moment problem (1) minimizing the KullbackLeibler divergence 6 Ψ (t) IΨ (Φ) = dt. (4) Ψ (t) log Φ(t) I Here G+ is the class of functions in L1+ (I) satisfying the normalization condition 6 Ψ (t)dt = 1 (5) I

and the integrability conditions 26 2 2 2 2 αk Ψ dt2 < ∞, 2 Re{q} 2 I

k = 0, 1, . . . , n,

(6)

for all q ∈ P+ . If I is a ﬁnite interval, (6) of course holds for all Ψ ∈ L1+ (I). In fact, Ψ could be regarded as some a priori estimate, and, as was done in [10] for spectral densities, we want to ﬁnd the function Φ that is “closest” to Ψ in the Kullback-Leibler distance and also satisﬁes the moment conditions (1).

The Uncertain Moment Problem with Complexity Constraint

269

This notion of distance arises in many applications, e.g., in coding theory [8] and probability and statistics [13, 11, 9]. Note, however, that Kullback-Leibler divergence is not really a metric, but, if we normalize by taking c0 , c1 , · · · , cn in C+ := {c ∈ C+ | c0 = 1}

(7)

so that Φ satisﬁes (5), the Kullback-Leibler divergence (4) is nonnegative, and it is zero if and only if Φ = Ψ . In [3] we proved that the problem to minimize (4), subject to the moment conditions (1), has a unique solution for each Ψ ∈ G+ and c ∈ C+ and that this solution has the form Φ(t) =

Ψ (t) Re{q(t)}

(8)

for some q ∈ P+ , which can be determined as the unique minimum in P+ of the strictly convex functional 6 (9) JΨ (q) = c, q − Ψ log (Re{q(t)}) dt. I

This ties up to a large body of literature [4, 7, 5, 6, 2, 3, 10] dealing with interpolation problems with complexity constraints. In this paper we consider a modiﬁed optimization problem in which c is allowed to vary in some compact, convex subset C0 of C+ , where C+ ⊂ C+ is given by (7). In fact, the moments c1 , c2 , · · · , cn may not be precisely determined, but only known up to membership in C0 . The problem at hand is then Problem 1. Find a pair (Φ, c) ∈ L1+ (I) × C0 that minimizes the KullbackLeibler divergence (4) subject to the moment conditions (1). We will show that this problem has a unique minimum and that the corresponding c lies in the interior of C0 only if Ψ satisﬁes the moment conditions, in which case the optimal Φ equals Ψ . An important special case of Problem 1 was solved in [15]. In [15] the uncertain covariance extension problem, as a tool for spectral estimation, is noted to have two fundamentally diﬀerent kinds of uncertainty. It is now known [3, 10] that the rational covariance extension problem can be solved by minimizing the Kullback-Leibler divergence (4), where Ψ is an arbitrary positive trigonometric polynomial of degree at most n, and where the functions α0 , α1 , . . . , αn are the trigonometric monomials, i.e, αk (t) = eikt , k = 0, 1, . . . , n. The corresponding moments are then the covariance lags of an underlying process. The uncertainty involving the choice of Ψ is resolved in [15] by choosing Ψ = 1. Then minimizing the Kullback-Leibler divergence is equivalent to ﬁnding the maximum-entropy solution, corresponding to having no a priori

270

C.I. Byrnes and A. Lindquist

information about the estimated process, namely the solution to the trigonometric moment problem maximizing the entropy gain 6 log Φ(t)dt. (10) I

The other fundamental uncertainty in this problem arises from the statistical errors introduced in estimating the covariance lags from a given ﬁnite observation record. This was modeled in [15] by assuming that the true covariance lags are constrained to lie in the polyhedral set + ck ∈ [c− k , ck ],

k = 0, 1, . . . , n.

(11)

In this setting, it is shown that the maximal value of (10) subject to the moment conditions is a strictly convex function on the polytope C0 deﬁned by (11) and hence that there is a unique choice of c ∈ C0 maximizing the entropy gain. As will be shown in this paper, this is a special case of our general solution to Problem 1.

2 Background The problem described above is related to a moment problem with a certain complexity constraint: In [2, 3] we proved that the moment problem (1) with the complexity constraint (8) has a unique solution. More precisely, we proved Theorem 1. For any Ψ ∈ G+ and c ∈ C+ , the function F : P+ → C+ , deﬁned componentwise by 6 Ψ (t) dt, k = 0, 1, . . . , n, (12) αk (t) Fk (q) = Re{q(t)} I is a diﬀeomorphism. In fact, the moment problem (1) with the complexity constraint (8) has a unique solution qˆ ∈ P+ , which is determined by c and Ψ as the unique minimum in P+ of the strictly convex functional (9). Note that JΨ (q) is ﬁnite for all q ∈ P+ . In fact, by Jensen’s inequality, 6 6 6 Ψ dt ≤ − log Ψ log (Re{q(t)}) dt ≤ log Re{q(t)}Ψ dt, I Re{q(t)} I I where both bounds are ﬁnite by (6). (To see this, for the lower bound take k = 0; for the upper bound ﬁrst take q = 1 in (6), and then form the appropriate linear combination.) In this paper we shall give a new proof of Theorem 1 by using methods from convex analysis. As proved in [3], following the same pattern as in [4, 5, 6], the optimization problem of Theorem 1 is the dual problem in the sense of mathematical programming of the constrained optimization problem in the following theorem.

The Uncertain Moment Problem with Complexity Constraint

271

Theorem 2. For any choice of Ψ ∈ G+ , the constrained optimization problem to minimize the Kullback-Leibler divergence (4) over all Φ ∈ L1+ (I) subject to ˆ and it has the form the constraints (1) has unique solution Φ, ˆ= Φ

Ψ , Re{ˆ q}

where qˆ ∈ P+ is the unique minimizer of (9). Moreover, for all Φ ∈ L1+ (I) and q ∈ P+ , −IΨ (Φ) ≤ JΨ (q) − 1

(13)

ˆ with equality if and only if q = qˆ and Φ = Φ.

3 The Uncertain Moment Problem We are now a position to solve Problem 1. We shall need the following deﬁnition [14, p. 251]. Deﬁnition 1. A function f is essentially smooth if 1. int(domf ) is nonempty; 2. f is diﬀerentiable throughout int(domf ); 3. limk→∞ |∇f (x(k) )| = +∞ whenever {x(k) } is a sequence in int(domf ) converging to the boundary of int(domf ). An essentially smooth function such that int(domf ) is convex and f is a strictly convex function on int(domf ) is called a convex function of Legendre type. ˆ of Theorem 2 clearly depends on c, and hence we The optimal point IΨ (Φ) may deﬁne a function ϕ : C+ → R ˆ i.e., which sends c to IΨ (Φ), 6 c →

Ψ (t) log I

Ψ (t) dt. ˆ Φ(t)

(14)

We also write qˆ(c) to emphasize that the unique minimizer qˆ in P+ of the ˆ for the unique minimizer functional (9) depends on c. Similarly, we write Φ(c) of Theorem 2. Then, by Theorem 2, ˆ = Φ(c)

Ψ , Re{ˆ q }(c)

for all c ∈ C+ .

272

C.I. Byrnes and A. Lindquist

Theorem 3. The function ϕ is a convex function of Legendre type. In particular, ϕ is strictly convex, and the problem to minimize ϕ over the compact, convex subset C0 of C+ has a unique solution. The minimizing point cˆ belongs to the interior of C0 only if Ψ satisﬁes the moment conditions (1), in which case qˆ(ˆ c) = 1. The gradient of ϕ is given by ∇ϕ(c) = −ˆ q (c),

(15)

and the Hessian is the inverse of the matrix 6 n Ψ (t) H(c) := αj (t) . 2 αk (t)dt I Re{ˆ q (c)(t)} j,k=0

(16)

The proof of this theorem will be given in Section 5. As an illustration, we can use Newton’s method to solve Problem 1. In fact, suppose that C0 has an nonempty interior. Then, for any c(0) in the interior of C0 , the recursion * + 00 (ν+1) (ν) H(c(ν) )ˆ c = c + λν q (c(ν) ), ν = 0, 1, 2, . . . (17) 0I will converge to cˆ for a suitable choice of {λν } keeping the sequence inside C0 . This algorithm can be implemented in the following way. For ν = 0, 1, 2, . . . , the gradient qˆ(c(ν) ) is determined as the unique minimum in P+ of the strictly convex functional 6 (ν) (ν) (18) JΨ (q) = c , q − Ψ log (Re{q(t)}) dt, I

(ν+1)

is obtained from (17). and then c As an example, consider the special, but important, case that C0 is deﬁned as the polyhedral set of all c = (c0 , c1 , · · · , cn ) ∈ C+ satisfying (11). The Lagrange relaxed problem is then to minimize −

+

L(c, λ , λ ) = ϕ(c) +

n k=0

−

λ (ck −

c− k)

+

n k=0

λ+ (c+ k − ck ),

(19)

+ where λ− k ≥ 0 and λk ≥ 0, k = 0, 1, . . . , n, are Lagrange multipliers. By Theorem 3, the Lagrangian has a unique stationary point that satisﬁes

qˆ(c) = λ+ − λ− .

(20)

By the principle of complementary slackness, a Lagrange multiplier can be positive only when the corresponding constraint is satisﬁed with equality at the optimum. In particular, if all components of qˆ(ˆ c) are nonzero, cˆ must be a corner point of the polyhedral set C0 .

The Uncertain Moment Problem with Complexity Constraint

273

4 A Derivation of the Legendre Transform from a Diﬀerentiable Viewpoint Suppose U is an open subset of RN , which is diﬀeomorphic to RN , and that F is a C 1 map F : U → RN with a Jacobian, Jacq (F ), which is invertible for each q ∈ U . A useful formulation of the Poincar´e Lemma is that Jacq (F ) is symmetric for each q ∈ U if and only if F is the gradient vector, ∇f , for some C 2 function f : U → R, which is unique up to a constant of integration. Remark 1. Here, we mean symmetric when represented as a matrix in the standard basis of RN , i.e., symmetric as an operator with respect to the standard inner product. We interpret the gradient as a column vector using this inner product as well. Alternatively, consider the 1-form ω=

N

Fk dqk ,

k=1

where Fk and qk denote the kth component of F and q, respectively. To say that Jacq (F ) is symmetric for all q ∈ U is to say that dω = 0 on U , and therefore ω = df for an f as above. More generally, N

N qk∗ dqk Fk dqk − qk∗ dqk = df (q) −

k=1

k=1

so that df (q) =

N

qk∗ dqk

⇔

F (q) = q ∗

⇔

∇f (q) = q ∗ .

(21)

k=1

We now specialize to the strictly convex case, i.e., we suppose that U is convex and that Jacq (F ) is positive deﬁnite for all q ∈ U . Alternatively, we could begin our construction with a strictly convex C 2 function f . In this case, we note that (21) is equivalent to inf {f (p) − p, q ∗ } = f (q) − q, q ∗ p

⇔

∇f (q) = q ∗ .

(22)

274

C.I. Byrnes and A. Lindquist

The left hand side of the equivalence (22) deﬁnes a function of q ∗ , which we denote by g(q ∗ ), and which we will soon construct in an intrinsic, geometric fashion. For now, it suﬃces to note that, in light of (21), we obtain the following expression for g: g(q ∗ ) = f ((∇f )−1 (q ∗ )) − q ∗ , (∇f )−1 (q ∗ ).

(23)

In fact, since f is strictly convex, the map F is injective. Since F has an everywhere nonvanishing Jacobian, by the inverse function theorem, F is a diﬀeomorphism between U and V := f (U ), where V is an open subset of RN . Since the inverse of a positive deﬁnite matrix is positive deﬁnite, F −1 has an everywhere nonsingular symmetric Jacobian, Jacq∗ (F −1 ). Therefore, we may apply our general construction to ﬁnd, up to a constant of integration, a unique C 2 function f∗ : V → R satisfying N

[F −1 ]k dqk∗ = df ∗ (qk∗ )

k=1

and, more generally, N

N [F −1 ]k dqk∗ − qk dqk∗ = df ∗ (qk∗ ) − qk dqk∗

k=1

k=1

and consequently df ∗ (q ∗ ) =

N

qk dqk∗

⇔

F −1 (q ∗ ) = q

⇔

∇f ∗ (q ∗ ) = q.

(24)

k=1

Of course, this geometric duality has several corollaries. Fix q0 ∈ U and q0∗ := F (q0 ) ∈ V . Let qˆ be an arbitrary point in U and denote its image, F (ˆ q ), by qˆ∗ . Let γ be any smooth oriented curve starting at q0 and ending at qˆ, and consider γ ∗ := F (γ). We may then compute the following path integral as a function of the upper limit, 6 6 6 N N f ∗ (ˆ q ∗ ) − f ∗ (q0∗ ) = df ∗ (q ∗ ) = [F −1 ]k (q ∗ )dqk∗ = qk dFk . (25) γ∗

γ ∗ k=1

γ k=1

Then, integrating by parts, we obtain f ∗ (ˆ q ∗ ) = f ∗ (q0∗ ) +

N k=1

N 6 2qˆ 2 q k Fk 2 − Fk dqk q0

k=1

6 2qˆ 2 ∗ ∗ = f (q0 ) + q, ∇f 2 − df q0

= ˆ q , ∇f (ˆ q ) − f (ˆ q ) + κ, where

γ

γ

The Uncertain Moment Problem with Complexity Constraint

275

κ := f ∗ (q0 ) − q0 , ∇f (q0 ) + f (q0 ) is a constant of integration for f ∗ , which we may set equal to zero. Therefore, q ∗ ) and qˆ∗ = ∇f (ˆ q ), since qˆ = (∇f )−1 (ˆ f ∗ (ˆ q ∗ ) = (∇f )−1 (ˆ q ∗ ), qˆ∗ − f (∇f )−1 (ˆ q∗ ) , or, recalling that qˆ is arbitrary, f ∗ (q ∗ ) = (∇f )−1 (q ∗ ), q ∗ − f (∇f )−1 (q ∗ ) = −g(q ∗ ). Remark 2. Since our fundamental starting point assumes that F has a symmetric everywhere nonsingular Jacobian, the above analysis extends to strictly concave functions, the only change being that the inﬁma be replaced by suprema. Furthermore, since the Hessian of f ∗ is the inverse of the Hessian of f , it follows that, on any open convex subset of V , f ∗ will be strictly convex (strictly concave) whenever f is strictly convex (strictly concave). Remark 3. These expressions are well-known in convex optimization theory. (See, e.g., [12, 14].) Indeed, since f ∗ = −g, (22) yields f ∗ (q ∗ ) = sup {q ∗ , q − f (q)} , q∈U

(26)

which is referred to as the conjugate function of f . Then, (23) yields f ∗ (q ∗ ) = q ∗ , (∇f )−1 (q ∗ ) − f ((∇f )−1 (q ∗ )),

(27)

which is the Legendre transform of f [12, p. 35]. Remark 4. We have derived the Legendre transform and certain of its properties from a diﬀerentiable viewpoint, because the corresponding functions deﬁned by the moment problem are in fact inﬁnitely diﬀerentiable. In contrast, the trend in modern optimization theory is to assume as little diﬀerentiability as possible. For example, if f is a strictly convex C 1 function, then F is a continuous injection deﬁned on U and is therefore an open mapping by Brouwer’s Theorem on Invariance of Domain. Thus it is a homeomorphm between U and V . Following [14], one can deﬁne the conjugate function via (26) and verify that (27) holds. In particular, the inverse of F is given by a gradient. Far deeper is the situation when f maps U to an open convex set W , and one also wants to show the V = W . Such a global inverse function theorem for strictly convex C 1 functions f is given in the beautiful Theorem 26.5 in [14], under the additional assumption that f is a convex function of Legendre type. Returning to the case of smooth F , a global inverse function theorem can be proved under the condition that F is proper, in which case F is a diﬀeomorphism.

276

C.I. Byrnes and A. Lindquist

5 The Main Theorem Applying the path integration methods of the previous section to the function F in Theorem 1, we obtain the strictly concave C ∞ function f : P+ → R taking the values 6 Ψ (t) log Re{q(t)}dt. (28) f (q) = I

The function f can be extended to the closure of P+ as an extended realvalued function. In particular, F is a diﬀeomorphism between P+ and its open image in C+ . In this setting, JΨ (ˆ q (c)) = f ∗ (c),

(29)

where qˆ(c) is the minimizer of (9) expressed as a function of c. According to Remark 2, f ∗ is strictly concave on any convex subset of F (P+ ), since f is. (See also [14, page 308] for a discussion about properties of the conjugate function f ∗ in the concave setting.) We also note that ˆ IΨ (Φ(c)) = 1 − JΨ (ˆ q (c)),

for all c ∈ C+

(30)

by Theorem 2, and hence, in view of (29), the function ϕ : C+ → R, deﬁned in (14), is given by ϕ(c) = 1 − f ∗ (c),

(31)

and consequently ϕ is a strictly convex function on any convex subset of F (P+ ). We are now prepared to prove our main result. Theorem 4. The function F deﬁned in Theorem 1 is a diﬀeomorphism between P+ and C+ . Moreover, the value function ϕ is a convex function of Legendre type on C+ . Proof. Since the image of F is an open subset in the convex set C+ , it suﬃces to prove that it is also closed. To show this, we show that F is proper, i.e. that for any compact subset K of C+ , the inverse image F −1 (K) is compact in P+ . This will follow from the fact that F is inﬁnite on the boundary of P+ , which in turn follows from the following calculation: 6 ∂f Ψ (t) dt, k = 0, 1, . . . , n. (32) = αk (t) ∂qk Re{q(t)} I Now, t → Re{q(t)} is a smooth, nonnegative function on I. As q tends to the boundary of P+ , this function attains a zero on the interval, and hence, since α0 = 1 and q is C 2 , the integral (32) is divergent at least for k = 0. Therefore, F is a diﬀeomorphism between P+ and C+ .

The Uncertain Moment Problem with Complexity Constraint

277

We have already seen that ϕ is a strictly convex function. Therefore it just remains to show that f is essentially smooth. Clearly, P+ is nonempty and f is diﬀerentiable throughout P+ , so conditions 1 and 2 in Deﬁnition 1 are satisﬁed. On the other hand, condition 3 is equivalent to properness of F , which we have already established above. All that remains to be proven are the identities in Theorem 3. Recalling that the function F : P+ → C+ in Theorem 1 is given by F (q) = ∇f (q),

(33)

the map ∇ϕ : C+ → P+ is the inverse of the diﬀeomorphism −F , i.e., ∇ϕ = −F −1 .

(34)

Therefore, ∇ϕ sends c to −ˆ q (c), which establishes (15). To prove (16), observe that the Hessian ∂2ϕ ∂ qˆ =− 2 ∂c ∂c but, since F (ˆ q ) = c, this is the inverse of 2 ∂F 22 − , ∂q 2q=ˆq which, in view of (12), is precisely (16). Clearly, the strictly convex function ϕ has a unique minimum in the compact set C0 . The minimizing point cˆ belongs c), h = 0 for all h ∈ TcˆC+ , in which case we to the interior of C0 only if ∇ϕ(ˆ must have qˆ(ˆ c) = q0 = 1 by (15). This concludes the proof of Theorem 3. Remark 5. As discussed in Remark 4, one can also deduce this theorem from Theorem 26.5 in Rockafeller [14], which would imply that F is a homeomorphism for a C 1 strictly convex function f . That F is a diﬀeomorphism for a C 2 function f would then follow from the Inverse Function Theorem. An alternative route, as indicated in Remark 4 could be based on Brouwer’s Theorem on Invariance of Domain to prove that F is a homeomorphism. Either proof would of course entail the use of substantial additional machinery not needed in the smooth case. Indeed, this motivated us to develop the self-contained derivation of the Legendre transform and the subsequent proof presented here. Acknowledgment. We would like to thank J. W. Helton for inquiring about the relationship between our earlier diﬀerential geometric proof of Theorem 1 and the Legendre transform, which led to the development of Sections 4 and 5.

278

C.I. Byrnes and A. Lindquist

References 1. Akhiezer, N. I. (1965) The Classical Moment Problem and Some Related Questions in Analysis, Hafner Publishing, New York. 2. Byrnes C.I., Lindquist A., Interior point solutions of variational problems and global inverse function theorems, submitted for publication. 3. Byrnes C.I., Lindquist A. (Dec. 2002) A Convex Optimization Approach to Generalized Moment Problems, in “Control and Modeling of Complex Systems: Cybernetics in the 21st Century: Festschrift in Honor of Hidenori Kimura on the Occasion of his 60th Birthday”, Koichi Hashimoto, Yasuaki Oishi, and Yutaka Yamamoto, Editors, Birkhauser Boston. 4. Byrnes C.I., Gusev S.V., Lindquist A. (1999) A convex optimization approach to the rational covariance extension problem, SIAM J. Control and Optimization 37, 211–229. 5. Byrnes C.I., Georgiou T.T., Lindquist A. (2001) A generalized entropy criterion for Nevanlinna-Pick interpolation with degree constraint, IEEE Trans. Automatic Control AC-46, 822–839. 6. Byrnes C.I., Georgiou T.T.,Lindquist A. (Nov. 2000) A new approach to spectral estimation: A tunable high-resolution spectral estimator, IEEE Trans. on Signal Processing SP-49, 3189–3205. 7. Byrnes C.I., Gusev S.V., Lindquist A. (Dec. 2001) From ﬁnite covariance windows to modeling ﬁlters: A convex optimization approach, SIAM Review 43, 645–675. 8. Cover T.M., Thomas J.A. (1991) Elements of Information Theory, Wiley. 9. Csisz´ ar I. (1975) I-divergence geometry of probability distributions and minimization problems, Ann. Probab. 3, 146–158. 10. Georgiou T.T., Lindquist A., Kullback-Leibler approximation of spectral density functions, IEEE Trans. on Information Theory, to be published. 11. Good I.J. (1963) Maximum entropy for hypothesis formulation, especially for multidimentional contingency tables, Annals Math. Stat. 34, 911–934. 12. Hiriart-Urruty J.B., Lemar´echal C. (1991) Convex Analysis and Minimization Algorithms II, Springer-Verlag. 13. Kullback S. (1968) Information Theory and Statistics, 2nd edition, New York: Dover Books, (1st ed. New York: John Wiley, 1959). 14. Rockafellar R.T. (1970) Convex Analysis, Princeton University Press, Princeton, NJ. 15. Shankwitz C.R., Georgiou T.T. (October 1990) On the maximum entopy method for interval covariance sequences, IEEE Trans. Acoustics, Speech and Signal processing 38, 1815–1817.

Optimal Control and Monotone Smoothing Splines Magnus Egerstedt1 and Clyde Martin2 1 2

School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA, [email protected] Department of Mathematics and Statistics, Texas Tech University, Lubbock, TX 79409, USA, [email protected]

Summary. The solution to the problem of generating curves by driving the output of a particular, nilpotent single-input, single-output linear control system close to given waypoints is analyzed. The curves are furthermore constrained by an inﬁnite dimensional, non-negativity constraint on one of the derivatives of the curve. The main theorem in this paper states that the optimal curve is a piecewise polynomial of known degree, and for the two-dimensional case, this problem is completely solved when the acceleration is controlled directly. The solution is obtained by exploiting a ﬁnite reparameterization of the problem, resulting in a dynamic programming formulation that can be solved analytically.

1 Introduction When interpolating curves through given data points, a demand that arises naturally when the data is noise contaminated, is that the curve should pass close to the interpolation points instead of demanding exact interpolation. This means that outliers will not be given too much attention, which could otherwise potentially corrupt the shape of the curve. In this paper, we investigate this type of interpolation problem from an optimal control point of view, where the interpolation task is reformulated in terms of choosing appropriate control signals in such a way that the output of a given, linear control system deﬁnes the desired interpolation curve. The curve is obtained by minimizing the energy of the control signal, and we, furthermore, deal with the outliersproblem by adding quadratic penalties to the energy cost functional. In this manner, deviations from the data points are penalized in order to produce smooth output curves [5, 14]. The fact that we minimize the energy of the control input, while driving the output of the system close to the interpolation points, tells us that the curves that we are producing belong to a class

W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 279–294, 2003.

c Springer-Verlag Berlin Heidelberg 2003

280

M. Egerstedt and C. Martin

of curves that in the statistics literature is referred to as smoothing splines [14, 15]. However, in many cases, this type of construction is not enough since one sometimes want the curve to exhibit a certain structure, such as monotonicity or convexity. These properties correspond to non-negativity constraints on the ﬁrst and second derivative of the curve respectively, and hence the nonnegative derivative constraint will be the main focus of this paper. Our main theorem, Theorem 2, states that the optimal curve is a piecewise polynomial of known degree, and we will show how the corresponding inﬁnite dimensional constraint (it has to hold for all times) can be reformulated and solved in a ﬁnite setting based on dynamic programming. That piecewise polynomial splines are the solutions to a number of different optimal control problems is a well-known fact [11]. However, in [11], the desired characteristics of the curves were formulated as constraints, while this paper investigates how the introduction of least-square terms in the cost function aﬀects the shape of the curve. Furthermore, in [13] only the optimality conditions (necessary as well as suﬃcient) were studied, but it is in general not straight forward to go from a maximum principle to a numerically tractable algorithm, which is the case in this paper. The problem of monotone interpolation has furthermore been extensively studied in the literature. In [8, 9] the problem of exact interpolation of convex or monotone data points using monotone polynomials is investigated. Questions concerning existence and convergence of such interpolating polynomials have been studied in [4, 7, 12]. Those results are in general not constructive in so far as they can not be readily implemented as a numerical algorithm. In [3], however, a numerical algorithm for producing monotone, interpolating polynomials is developed, even though no guarantees that the monotonicity constraint is respected for all times is given. What is diﬀerent in this paper is ﬁrst of all that we focus exclusively on producing monotone, smoothing curves, i.e. we do not demand exact interpolation. Secondly, we want our solution to be constructive so that it can be implemented as a numerically sound algorithm with guaranteed performance in the case when the curve is generated by a second order system. The main contribution in this paper is thus that we show how concepts, well studied in control theory, such as minimum energy control and dynamic programming, give us the proper tools for shedding some new light on the monotone smoothing splines problem. The outline of this paper is as follows: In Sections 2 and 3, we describe the problem and derive some of the properties that the optimal solution exhibits. We then, in Section 4, show how the problem can be reparameterized as a ﬁnite dimensional dynamic programming problem. In Section 5, we give the exact solution to the monotone interpolation problem when the underlying dynamics is given by a particular second order system.

Optimal Control and Monotone Smoothing Splines

281

2 Problem Description Consider the problem of constructing a curve that passes close to given data points, at the same time as we want the curve to exhibit certain monotonicity properties. In other words, if p(t) is our curve, we want (p(ti ) − αi )2 , i = 1, . . . , m to be qualitatively small. Here, (t1 , α1 ), . . . , (tm , αm ) are the data points, with αi ∈ , i = 1, . . . , m, and 0 < t1 < t2 < . . . < tm ≤ T , for some given terminal time T > 0. We do not only, however, want to keep the interpolation errors small. We also want the curve to vary in a smooth way, as well as p(n) (t) ≥ 0, ∀t ∈ [0, T ],

(1)

for some given, positive integer n. Let 

 0 1 0 ··· 0 0 0 1 ··· 0     A =  ... . . . ...    0 0 0 ··· 1  00 0 · · · 0 0  ..    b=. 0 1 c1 = 1 0 · · · 0 c2 = 0 0 · · · 1 ,

(2)

where A is an n × n-matrix, b is n × 1, and c1 and c2 are 1 × n. Then, by using the standard notation from control theory [2], our problem can be cast as . 6 5 m 1 T 2 1 2 inf (3) u (t)dt + τi (c1 x(ti ) − αi ) , u 2 0 2 i=1 subject to   x˙ = Ax + bu, x(0) = 0 u ∈ L2 [0, T ]  c2 x(t) ≥ 0, ∀t ∈ [0, T ],

(4)

where τi ≥ 0 reﬂects how important it is that the curve passes close to a particular αi ∈ . Here, c1 x(t) takes on the role of p(t), and by our particular choices of A and b in Equation 2, x is a vector of successive derivatives. It is furthermore clear that by keeping the L2 -norm of u small, we get a curve that varies in a smooth way.

282

M. Egerstedt and C. Martin

Now, if x˙ = Ax + bu then x(t) is given by 6 t x(t) = eAt x(0) + eA(t−s) bu(s)ds, 0

which gives that c1 x(ti ) can be expressed as 6 ti c1 eA(ti −t) bu(t)dt c1 x(ti ) = 0

since x(0) = 0. This expression can furthermore be written as 6 T c1 x(ti ) = gi (t)u(t)dt, 0

where we make use of the following linearly independent basis functions: A(t −t) c1 e i b if t ≤ ti gi (t) = i = 1, . . . , m. (5) 0 if t > ti The fact that these functions are linearly independent follows directly from the observation that they vanish at diﬀerent points. Our inﬁmization over u can thus be rewritten as  06 12  m 1 6 T  T 1 inf , (6) u2 (t)dt + τi gi (t)u(t)dt − αi u 2 0  2 i=1 0 which is an expression that only depends on u. Since we want c2 x(t) to be continuous, we let the constraint space be C[0, T ], i.e. the space of continuous functions. In a similar fashion as before, we can express c2 x(t) as 6 t 6 t c2 eA(t−s) bu(s)ds = f (t, s)u(s)ds. c2 x(t) = 0

0

This allows us to form the associated Lagrangian [10] 06 12 6 m T 1 T 2 1 L(u, ν) = u (t)dt + τi gi (t)u(t)dt − αi 2 0 2 i=1 0 6 T6 t − f (t, s)u(s)dsdν(t), 0

(7)

0

where ν ∈ BV [0, T ] (the space of functions of bounded variations, which is the dual space of C[0, T ]). The optimal solution to our original optimization problem is thus found by solving max

inf

0≤ν∈BV [0,T ] u∈L2 [0,T ]

L(u, ν).

(8)

Optimal Control and Monotone Smoothing Splines

283

3 Properties of the Solution ˜ ˜b, c˜), where A˜ is an n × n matrix, ˜b is n × 1, Lemma 1. Given any triple (A, ˜ + ˜bu, x(0) = 0, then the set of controls in L2 [0, T ] and c˜ is 1 × n. If x˙ = Ax that make the solution to the diﬀerential equation satisfy c˜x(t) ≥ 0, ∀t ∈ [0, T ], is a closed, non-empty, and convex set. Proof: We ﬁrst show convexity. Given two ui (t) ∈ L2 [0, T ], i = 1, 2, such that 6 t ˜ c˜eA(t−s)˜bui (s)ds ≥ 0, ∀t ∈ [0, T ], i = 1, 2, 0

then for any λ ∈ [0, 1] we have 6 t ˜ c˜eA(t−s)˜b(λu1 (s) + (1 − λ)u2 (s))ds ≥ 0, ∀t ∈ [0, T ], 0

and convexity thus follows. Now, consider a collection of controls, {ui (t)}∞ i=0 , where each individual control makes the solution to the diﬀerential equation satisfy c˜x(t) ≥ 0 ∀t ∈ [0, T ], and where ui → u ˆ as i → ∞. But, due to the compactness of [0, t], we have that 6 t 6 t ˜ ˜ A(t−s) ˜ c˜e c˜eA(t−s)˜bˆ bui (s)ds = u(s)ds ≥ 0, ∀t ∈ [0, T ]. lim i→∞

0

0 2

The fact that L [0, T ], with the natural norm deﬁned on it, is a Banach space gives us that the limit, u ˆ, still remains in that space. The set of admissible controls is thus closed. Furthermore, since x(0) = 0, we can always let u ≡ 0. This gives that the set of admissible controls is non-empty, which concludes the proof. Lemma 2. The cost functional in Equation 3 is convex in u. The proof of this lemma is trivial since both terms in Equation 3 are quadratic functions of u. Lemmas 1 and 2 are desirable in any optimization problem since they are strong enough to guarantee the existence of a unique optimal solution [10], and we can thus replace inf in Equation 7 with min, which directly allows us to state the following, standard theorem about our optimal control. Theorem 1. There is a unique u0 ∈ L2 [0, T ] that solves the optimal control problem in Equation 3. We omit the proof of this and refer to any textbook on optimization theory for the details. (See for example [10].)

284

M. Egerstedt and C. Martin

Lemma 3. Given the optimal solution u0 . The optimal ν0 ∈ BV [0, T ], ν0 ≥ 0, varies only where c2 x(t) = 0. On intervals where c2 x(t) > 0, ν0 (T ) − ν0 (t) is a non-negative, real constant. Proof: Since ν0 (T )−ν0 (t) ≥ 0, due to the positivity constraint on ν0 , we reduce the value of the Lagrangian in Equation 7 whenever ν0 changes, except when c2 x(t) = 0. But, since ν0 maximizes L(u0 , ν), we only allow ν0 to change when c2 x(t) = 0, and the lemma follows. Now, before we can completely characterize the optimal control solution, one observation to be made is that 6 t c2 x(t) = 0 0 · · · 1 x(t) = u(s)ds, 0

i.e. f (t, s) is in fact equal to 1 in Equation 7. This allows us to rewrite the Lagrangian as 1 L(u, ν) = 2 6 −

6

T

0

6

T

m

1 u (t)dt + τi 2 i=1

06

2

T

12 gi (t)u(t)dt − αi

0

t

u(s)dsdν(t).

0

(9)

0

By integrating the Stieltjes integral in Equation 9 by parts, we can furthermore reduce the Lagrangian to

L(u, ν) =

1 2 6

6

−

T

0 T

u2 (t)dt +

6 T m 1 τi ( gi (t)u(t)dt − αi )2 2 i=1 0

(ν(T ) − ν(t))u(t)dt,

(10)

0

which is a more easily manipulated expression. Deﬁnition 1. Let P P k [0, T ] denote the set of piecewise polynomials of degree k on [0, T ]. Let, furthermore, P k [0, T ] denote the set of polynomials of degree k on that interval. Theorem 2. The control in L2 [0, T ] that minimizes the cost in Equation 3 is in P P n [0, T ]. It furthermore changes from diﬀerent polynomials of degree n only at the interpolation times, ti , i = 1, . . . , m, and at times when c2 x(t) changes from c2 x(t) > 0 to c2 x(t) = 0 and vice versa. Proof: Due to the convexity of the problem, and the existence and uniqueness of the solution, we can obtain the optimal controller by calculating the Fr´echet diﬀerential of L with respect to u, and setting this equal to zero for all increments h ∈ L2 [0, T ].

Optimal Control and Monotone Smoothing Splines

285

By letting Lν (u) = L(u, ν), we get that δLν (u, h) = lim 1 (Lν (u + h) − Lν (u)) = →0 1 6 T 6 T0 m (11) u(t) + τi ( gi (s)u(s)ds − αi )gi (t) − (ν(T ) − ν(t)) h(t)dt. 0

i=1

0

For the expression in Equation 11 to be zero for all h ∈ L2 [0, T ] we need to have that 6 T m τi ( gi (s)u0 (s)ds − αi )gi (t) − (ν(T ) − ν(t)) = 0. u0 (t) + i=1

0

This especially has to be true for ν = ν0 , which gives that 6 T m u0 (t) + τi ( gi (s)u0 (s)ds − αi )gi (t) − Cj = 0, i=1

(12)

0

whenever c2 x0 (t) > 0. Here Cj is a constant. The index j indicates that this constant diﬀers on diﬀerent intervals where c2 x0 (t) > 0. Now, the integral terms in Equation 12 do not depend on t, while gi (t) is in P n [0, ti ] for i = 1, . . . , m. This, combined with the fact that ν0 (T )−ν0 (t) = Cj if x(t) ˙ > 0, directly gives us that the optimal control, u0 (t), has to be in P P n [0, T ]. It obviously changes at the interpolation times, due to the shape of the gi ’s, but it also changes if Cj changes, i.e. it changes if c2 x0 (t) = 0. It should be noted that if c2 x0 (t) ≡ 0 on an interval, ν0 (t) may change on the entire interval, but since c2 x0 (t) ≡ 0 we also have that u0 (t) ≡ 0 on the interior of this interval. But a zero function is, of course, polynomial. Thus we know that our optimal control is in P P n [0, t], and the theorem follows. Corollary 1. If n = 2 then the optimal control is piecewise linear (in P P 1 [0, T ]), with changes from diﬀerent polynomials of degree one at the interpolation times, and at times when c2 x(t) changes from c2 x(t) > 0 to c2 x(t) = 0 and vice versa.

4 Dynamic Programming Based on the general properties of the solution, the idea now is to formulate the monotone interpolation problem as a ﬁnite-dimensional programming problem that can be dealt with eﬃciently. If we drive the system x˙ = Ax + bu, where A and b are deﬁned in Equation 2, between xi and xi+1 on the time interval [ti , ti+1 ], under the constraint c2 x(t) ≥ 0, we see that we must at least have c2 xi ≥ 0 c2 xi+1 ≥ 0 D(xi+1 − xi ) ≥ 0,

(13)

286

where

M. Egerstedt and C. Martin



 1 0 ··· 0 0 0 1 ··· 0 0   D=. . , . . ...   ..  0 0 ··· 1 0

and the inequality in Equation 13 is taken component-wise. We denote the constraints in Equation 13 by D(xi , xi+1 ) ≥ 0. Since the original cost functional in Equation 3 can be divided into one interpolation part and one smoothing part, it seems natural to deﬁne the following optimal value function as   Sˆ (x ) =   i i   min Vi (xi , xi+1 ) + Sˆi+1 (xi+1 ) + τi (c1 xi − αi )2 xi+1 |D(xi ,xi+1 )≥0 (14)     ˆ Sm (xm ) = τm (c1 xm − αm )2 , where Vi (xi , xi+1 ) is the cost for driving the system between xi and xi+1 using a control in P P n [ti , ti+1 ], while keeping c2 x(t) non-negative on the time interval [ti , ti+1 ]. The optimal control problem thus becomes that of ﬁnding Sˆ0 (0), where we let τ0 = 0, while α0 can be any arbitrary number. In light of Theorem 2, this problem is equivalent to the original problem, and if Vi (xi , xi+1 ) could be uniquely determined, it would correspond to ﬁnding the n × m variables x1 , . . . , xm , which is a ﬁnite dimensional reparameterization of the original, inﬁnite dimensional programming problem. For this dynamic programming approach to work, our next task becomes that of determining the function Vi (xi , xi+1 ). Even though that is typically not an easy problem, a software package for computing approximations of such monotone polynomials was developed in [3]. In [8, 9] this problem of exact interpolation, over piecewise polynomials, of convex or monotone data points was furthermore investigated from a theoretical point of view. It is thus our belief that showing that the original problem can formulated as a dynamic programming problem involving exact interpolation is a result that is valuable since it greatly simpliﬁes the structure of the problem. It furthermore transforms it to a form that has been extensively studied in the literature. In the following section, we will show how to solve this dynamic programming problem exactly for a second order system in such a way that the computational burden is kept to a minimum. This work was carried out in detail in [6], and we will, throughout the remainder of this paper, refer to that work for the proofs. Instead we will focus our attention on the diﬀerent steps necessary for constructing optimal, monotone, cubic splines.

Optimal Control and Monotone Smoothing Splines

287

5 Example – Second Order Systems If we change our notation slightly in such a way that our state variable is given by (x, x), ˙ x, x˙ ∈ , the dynamics of the system becomes x ¨ = u. The optimal value function in Equation 14 thus takes on the form ˆ   Si (xi , x˙ i ) =   min Vi (xi , x˙ i , xi+1 , x˙ i+1 ) + Sˆi+1 (xi+1 , x˙ i+1 ) + τi (xi − αi )2 xi+1 ≥xi ,x˙ i+1 ≥0     Sˆm (xm , x˙ m ) = τm (xm − αm )2 . (15) 5.1 Two-Points Interpolation Given the times ti and ti+1 , the positions xi and xi+1 , and the corresponding derivatives x˙ i and x˙ i+1 , the question to be answered, as indicated by Corollary 1, is the following: How do we drive the system between (xi , x˙ i ) and (xi+1 , x˙ i+1 ), with a piecewise linear control input that changes between diﬀerent polynomials of degree one, only when x(t) ˙ = 0, in such a way that x(t) ˙ ≥ 0 ∀t ∈ [ti , ti+1 ], while minimizing the integral over the square of the control input? Without loss of generality, we, for notational purposes, translate the system and rename the variables so that we want to produce a curve, deﬁned on the time interval [0, tF ], between (0, x˙ 0 ) and (xF , x˙ F ). Assumption 3 x˙ 0 , x˙ F ≥ 0, xF > 0, tF > 0. It should be noted that if xF = 0, and either x˙ 0 > 0 or x˙ F > 0, then x(t) ˙ can never be continuous. This case has to be excluded since we already demanded that our constraint space was C[0, T ]. If, furthermore, xF = x˙ 0 = x˙ F = 0 then the optimal control is obviously given by u ≡ 0 on the entire interval. One ﬁrst observation is that the optimal solution to this two-points interpolation problem is to use standard cubic splines if that is possible, i.e. if x(t) ˙ ≥ 0 for all t ∈ [0, tF ]. In this well-studied case [1, 13] we would simply have that x(t) = where

1 3 1 2 at + bt + x˙ 0 t, 6 2

6 a tF (x˙ 0 + x˙ F ) − 2xF = 3 . b tF tF xF − 1/3t2F (2x˙ 0 + x˙ F )

(16)

(17)

288

M. Egerstedt and C. Martin

This solution corresponds to having ν(t) = ν(ti+1 ), for all t ∈ [ti , ti+1 ) in Equation 10, and it gives the total cost 6 tF (x˙ 0 t2F − 3xF tF )(x˙ 0 + x˙ F ) + 3x2F + t2F x˙ 2F I1 = (at + b)2 dt = 4 , (18) t3F 0 where the subscript 1 denotes the fact that only one polynomial of degree one was used to compose the second derivative.

dx0

1

0.8

dx

0.6

0.4

dxF 0.2

tM 0

tF −0.2

−0.4

0

0.2

0.4

0.6

0.8

1

t

Fig. 1. The case where a cubic spline can not be used if the derivative has to be non-negative. Plotted is the derivative that clearly intersects x˙ = 0.

However, not all curves can be produced by such a cubic spline if the curve has to be non-decreasing at all times. Given Assumption 3, the one case where we can not use a cubic spline can be seen in Figure 1, and we, from geometric considerations, get four diﬀerent conditions that all need to hold for the derivative to be negative. These necessary and suﬃcient conditions are (i) a > 0 (ii) b < 0 (iii) x(t ˙ M) < 0 (iv) tM < tF ,

(19)

where a and b are deﬁned in Equation 16, and tM is deﬁned in Figure 1. We can now state the following lemma. Lemma 4. Given Assumption 3, a standard cubic spline can be used to produce monotonously increasing curves if and only if xF ≥ χ(tF , x˙ 0 , x˙ F ) =

7 tF (x˙ 0 + x˙ F − x˙ 0 x˙ F ). 3

(20)

Optimal Control and Monotone Smoothing Splines

289

The proof of this follows from simple algebraic manipulations [6], and we now need to investigate what the optimal curve looks like in the case when we can not use standard, cubic splines. 5.2 Monotone Interpolation Given two points such that xF < χ(tF , x˙ 0 , x˙ F ), how should the interpolating curve be constructed so that the second derivative is piecewise linear, with switches only when x(t) ˙ = 0? One ﬁrst observation is that it is always possible to construct a piecewise polynomial path that consists of three polynomials of degree one that respects the interpolation constraint, and in what follows we will see that such a path also respects the monotonicity constraint. The three interpolating polynomials are given by  if 0 ≤ t < t1  a1 t + b1 if t1 ≤ t < t2 (21) u(t) = 0  a2 (t − t2 ) + b2 if t2 ≤ t ≤ tF , where

a1 b1 a2 b2

t1 x˙ 0 − 2x1 t1 x1 − 2/3t21 x˙ 0

=

6 t31

=

6 (tF −t2 )3

(tF − t2 )x˙ F − 2(xF − x1 ) (tF − t2 )(xF − x1 ) − 1/3(tF − t1 )2 x˙ F

(22)

,

and where x(t1 ) = x(t2 ) = x1 that, together with t1 and t2 , is a parameter that needs to be determined. Assumption 4 x˙ 0 , x˙ F , xF , tF > 0. We need this assumption, which is stronger than Assumption 3, in the following paragraph but it should be noted that if x˙ 0 = 0 or x˙ F = 0 we would then just let the ﬁrst or the third polynomial on the curve be zero. We now state the possibility of such a feasible three polynomial construction. Lemma 5. Given (tF , x˙ 0 , xF , x˙ F ) such that xF < χ(tF , x˙ 0 , x˙ F ), then a feasible, monotone curve will be given by Equation 21 as long as Assumption 4 holds. Furthermore, the optimal t1 , t2 , and x1 are given by  t = 3 xx˙ 10 ,   1 1 t2 = tF − 3 xFx˙−x , F 3/2  x ˙  x1 = 0 xF . 3/2

(x˙ 0

3/2

+x˙ F )

(23)

290

M. Egerstedt and C. Martin

The proof is constructive and is based on showing that with the type of construction given in Equation 21, the optimal choice of t1 , t2 , x1 gives a feasible curve. We refer the reader to [6] for the details, and we can thus construct a feasible path, as seen in Figure 2, by using three polynomials whose second derivatives are linear.

0.2 0.15 x

0.1 0.05 0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5 t

0.6

0.7

0.8

0.9

1

1

dx/dt

0.5

0

−0.5

Fig. 2. The dotted line corresponds to a standard, cubic spline, while the solid line shows the three polynomial construction from Lemma 5. Depicted is the position and the velocity.

Theorem 3 (Monotone Interpolation). Given Assumption 3, the optimal control that drives the path between (0, x˙ 0 ) and (xF , x˙ F ) is given by Equation 16 if xF ≥ χ(tF , x˙ 0 , x˙ F ) and by Equation 21 otherwise. Proof. The ﬁrst part of the theorem is obviously true. If we can construct a standard, cubic spline, then this is optimal. However, what we need to show is that when xF < χ(tF , x˙ 0 , x˙ F ) the path given in Equation 21 is in fact optimal. The cost for using a path given in Equation 21 is 6 I3 =

t1

6 2

(a1 t + b1 ) dt + 0

tF

t2

3/2

(a2 (t − t2 ) + b2 )2 dt =

3/2

4(x˙ F + x˙ 0 )2 , 9xF

where the coeﬃcients are given in Equation 23. We now add another, arbitrary polynomial, as seen in Figure 3, to the path as

Optimal Control and Monotone Smoothing Splines

 a1 t + b1     0 u(t) = a3 (t − t3 ) + b3   0    a2 (t − t2 ) + b2

if if if if if

0 ≤ t < t1 t1 ≤ t < t3 t3 ≤ t < t4 t4 ≤ t < t2 t 2 ≤ t ≤ tF ,

291

(24)

where 0 < t1 ≤ t3 ≤ t4 ≤ t2 < tF . Furthermore, t3 , t4 , and x2 = x(t4 ) (see Figure 3) are chosen arbitrarily while the old variables, t1 , t2 and x1 = x(t1 ), are deﬁned to be optimal with respect to the new, translated end-conditions that the extra polynomials give rise to. After some straight forward calculations, we get that the cost for this new path is 3/2

I5 =

3/2

4(x˙ F + x˙ 0 )2 12(x2 − x1 )2 + , 9(xF − x2 ) (t4 − t3 )3

(25)

where the subscript 5 denotes the fact that we are now using ﬁve polynomials of degree one to compose our second derivate. It can be seen that we minimize I5 if we let x2 = x1 and make t4 − t3 as large as possible. This corresponds to letting t3 = t1 and t4 = t2 , which gives us the old solution from Lemma 5, deﬁned in Equation 21.

1

dx0

0.8

dx

0.6

0.4

dxF 0.2

0

t1 −0.2

0

0.2

t2

t4

t3

0.4

0.6

0.8

tF

1

t

Fig. 3. Two extra polynomials are added to the produced path. Depicted is the derivative of the curve.

5.3 Monotone Smoothing Splines We now have a way of producing the optimal, monotone path between two points, while controlling the acceleration directly. We are thus ready to formulate the transition cost function in Equation 15, Vi (xi , x˙ i , xi+1 , x˙ i+1 ), that

292

M. Egerstedt and C. Martin

deﬁnes the cost for driving the system between (xi , x˙ i ) and (xi+1 , x˙ i+1 ), with minimum energy, while keeping the derivative non-negative. Based on Theorem 3 we, given Assumption 3, have that1 Vi (xi , x˙ i , xi+1 , x˙ i+1 ) =  x˙ (t −t )2 −3(xi+1 −xi )(ti+1 −ti )(x˙ i +x˙ i+1 )+3(xi+1 −xi )2 +(ti+1 −ti )2 x˙ 2i+1  4 i i+1 i   (ti+1 −ti )3         if xi+1 − xi ≥ χ(ti+1 − ti , x˙ i , x˙ i+1 )

(26)

 3/2 3/2  4(x˙ i+1 +x˙ i )2    9(x −x i+1 i)       if xi+1 − xi < χ(ti+1 − ti , x˙ i , x˙ i+1 ), where t0 = x0 = x˙ 0 = 0. If we use this cost in the dynamic programming algorithm, formulated in Equation 15, we get the results displayed in Figures 4–6, which shows that our approach does not only work in theory, but also in practice.

1 0.8

x

0.6 0.4 0.2 0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5 t

0.6

0.7

0.8

0.9

1

2.5 2

dx/dt

1.5 1 0.5 0 −0.5

Fig. 4. Monotone smoothing splines with τi = 1000, i = 1, . . . , 5.

1

If xi+1 − xi = x˙ i = x˙ i+1 = 0 then the optimal control is obviously zero, meaning that Vi (xi , x˙ i , xi+1 , x˙ i+1 ) = 0.

Optimal Control and Monotone Smoothing Splines

293

1 0.8

x

0.6 0.4 0.2 0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5 t

0.6

0.7

0.8

0.9

1

4

dx/dt

3 2 1 0

Fig. 5. Monotone smoothing splines with τ4 = 10τi , i = 4 (with t4 = 0.8), resulting in a diﬀerent curve compared to that in Figure 4 where equal importance is given to all of the waypoints.

1 0.8

x

0.6 0.4 0.2 0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

0.5 t

0.6

0.7

0.8

0.9

1

3

dx/dt

2 1 0 −1

Fig. 6. Smoothing splines, without the monotonicity constraint on the derivative. The curve is produced based on the theory developed in [5, 16].

6 Conclusions In this paper we propose and analyze the optimal solution to the problem of driving a curve close to given waypoints. This is done while the state space of

294

M. Egerstedt and C. Martin

the control system used for generating the curve is constrained by an inﬁnite dimensional non-negativity constraint on one of the derivatives of the curve. This problem is found to support a ﬁnite reparameterization, resulting in a dynamic programming formulation that can be solved analytically for the second order case. Simulation results furthermore support our claim that the proposed solution is not just theoretically sound, but also produces a numerically stable algorithm for computing monotone, smoothing splines.

References 1. Ailon A, Segev R (1988) Driving a linear constant system by piecewise constant control. International Journal of Control 47(3):815–825 2. Brockett RW (1970) Finite dimensional linear systems. John Wiley and Sons, Inc., New York 3. Constantini P (1997) Boundary-valued shape-preserving interpolating splines. ACM Transactions on Mathematical Software 23(2):229–251 4. Darst RB, Sahab S (1983) Approximation of continuous and quasi-continuous functions by monotone functions. Journal of Approximation Theory 38:9–27 5. Egerstedt M, Martin CF (1998) Trajectory planning for linear control systems with generalized splines. Proceedings of the Mathematical Theory of Networks and Systems, Padova, Italy 6. Egerstedt M, Martin CF (2000) Monotone smoothing splines. Proceedings of the Mathematical Theory of Networks and Systems, Perpignan, France 7. Ford WT (1974) On interpolation and approximation by polynomials with monotone derivatives. Journal of Approximation Theory 10:123–130 8. Hornung U (1980) Interpolation by smooth functions under restrictions on the derivatives. Journal of Approximation Theory 28:227–237 9. Iliev GL (1980) Exact estimates for monotone interpolation. Journal of Approximation Theory 28:101–112 10. Luenberger DG (1969) Optimization by vector space methods. John Wiley and Sons, Inc., New York 11. Mangasarian OL, Schumaker LL (1969) Splines via optimal control. Approximation with Special Emphasis on Spline Functions, Schoenberg IJ (Ed.), Academic Press, NY 12. Passow E, Raymon L, Rouler JA (1974) Comotone polynomial approximation. Journal of Approximation Theory 11:221–224 13. Schumaker LL (1981). Spline functions: basic theory. John Wiley and Sons, New York 14. Wahba G (1990) Spline models for observational data. Society for Industrial and Applied Mathematics 15. Wegman EJ, Wright IW (1983) Splines in statistics. Journal of the American Statistical Association 78(382) 16. Zhang Z, Tomlinson J, Martin CF (1997) Splines and linear control theory. Acta Applicandae Mathematicae 49:1–34

Towards a Sampled-Data Theory for Nonlinear Model Predictive Control Rolf Findeisen1 , Lars Imsland2 , Frank Allg¨ ower1 , and Bjarne Foss2 1

2

Institute for Systems Theory in Engineering, University of Stuttgart, Pfaﬀenwaldring 9, D-70550 Stuttgart, Germany, {findeise,allgower}@ist.uni-stuttgart.de Department of Engineering Cybernetics, NTNU, 7491 Trondheim, Norway, {Lars.Imsland,Bjarne.Foss}@itk.ntnu.no

Summary. This paper considers the stability, robustness and output feedback problem for sampled-data nonlinear model predictive control (NMPC). Sampled-data NMPC here refers to the repeated application of input trajectories that are obtained from the solution of an open-loop optimal control problem at discrete sampling instants. Speciﬁcally we show that, under the assumption that the value function is continuous, sampled-data NMPC possesses some inherent robustness properties. The derived robustness results have a series of direct implications. For example, they underpin the intuition that small errors in the optimal input trajectory, e.g. resulting from an approximate numerical solution, can be tolerated. Furthermore, the robustness can be utilized to design observer-based semi-globally stable output feedback NMPC schemes.

1 Introduction Model predictive control (MPC), also known as receding horizon control or moving horizon control, is by now a well established control method. Especially linear MPC, i.e. predictive control for linear systems considering linear constraints, is widely used in industry; mainly since it allows to handle MIMO systems and constraints on states and inputs systematically [38]. Motivated by the success of linear MPC, predictive control of nonlinear systems (NMPC) has gained signiﬁcant interest over the past decade. Various NMPC strategies that lead to stability of the closed-loop have been developed in recent years and key questions such as the eﬃcient solution of the occurring open-loop optimal control problem have been extensively studied (see e.g. [33, 1, 10] for recent reviews). In this paper we are interested in stability, robustness, and output feedback for continuous time NMPC with sampled measurement information; i.e. we W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 295–311, 2003.

c Springer-Verlag Berlin Heidelberg 2003

296

R. Findeisen et al.

consider the stabilization of continuous time systems by repeatedly applying input trajectories that are obtained from the solution of an open-loop optimal control problem at discrete sampling instants. In the following we refer to this problem as sampled-data NMPC. In the ﬁrst part of this paper we brieﬂy review how nominal stability for sampled-data NMPC can be achieved. Based on the nominal stability results we show in Section 4 that, under the assumption that the value function is continuous, the closed-loop using a nominally stable sampled-data NMPC scheme possesses some inherent robustness properties. Some consequences of the result are outlined. Expanding the robustness results obtained in Section 4 to measurement errors, we consider in Section 5 the output feedback problem for sampled-data NMPC. Speciﬁcally we state conditions on the observer error that must be satisﬁed to achieve semi-global practical stability of the closedloop.

2 State Feedback Sampled-Data NMPC We consider the stabilization of time-invariant nonlinear systems of the form x(t) ˙ = f (x(t), u(t)),

x(0) = x0

(1)

subject to the input and state constraints: u(t) ∈ U ⊂ Rm , x(t) ∈ X ⊆ Rn , ∀t ≥ 0. With respect to the vector ﬁeld f : Rn×Rm → Rn we assume that it is locally Lipschitz continuous and satisﬁes f (0, 0) = 0. Furthermore, the set U ⊂ Rm is compact, X ⊆ Rn is connected, and (0, 0) ∈ X ×U. In sampled-data NMPC an open-loop optimal control problem is solved at discrete sampling instants ti based on the current state information x(ti ). The sampling instants ti are given by a partition π of the time axis. Deﬁnition 1. (Partition) Every series π = (ti ), i ∈ N of (ﬁnite) positive real numbers such that t0 = 0, ti < ti+1 and ti → ∞ for i → ∞ is called a partition. Furthermore, let π ¯ := supi∈N (ti+1 −ti ) be the upper diameter of π and π := inf i∈N (ti+1 −ti ) be the lower diameter of π. When the time t and ti occurs in the same setting, ti should be taken as the closest previous sampling instant ti < t. In sampled-data NMPC, the input trajectory applied in between the sampling instants is given by the solution of the following open-loop optimal control problem: min J(x(ti ), u ¯(·)) subject to: u ¯(·)

x ¯˙ (τ ) = f (¯ x(τ ), u ¯(τ )),

x ¯(ti ) = x(ti ) (2a) u ¯(τ ) ∈ U, x ¯(τ ) ∈ X τ ∈ [ti , ti + Tp ] (2b) x ¯(ti + Tp ) ∈ E.

(2c)

Towards a Sampled-Data Theory for Nonlinear Model Predictive Control

297

The bar denotes predicted variables, i.e. x ¯(·) is the solution of (2a) driven by the input u ¯(·) : [ti , ti + Tp ] → U with the initial condition x(ti ). The cost functional J minimized over the control horizon Tp ≥ π ¯ > 0 is given by 6 J(x(ti ), u ¯(·)) :=

ti +Tp

ti

F (¯ x(τ ), u ¯(τ ))dτ + E(¯ x(ti + Tp )),

(3)

where the stage cost F : X × U → X is assumed to be continuous, satisﬁes F (0, 0) = 0, and lower bounded by a class K function1 αF : αF (x) ≤ F (x, u) ∀(x, u) ∈ X × U, where · denotes the Euclidean vector norm. The terminal region constraint E and the terminal penalty term E are often used to enforce stability of the closed-loop [33, 20]. The solution of the optimal control problem (2) is denoted by u ¯ (·; x(ti )). It deﬁnes the open-loop input that is applied to the system until the next sampling instant ti+1 : ¯ (t; x(ti )), u(t; x(ti )) = u

t ∈ [ti , ti+1 ) .

(4)

The control u(t; x(ti )) is a feedback, since it is recalculated at each sampling instant using the new state measurement. Remark 1. The main idea behind predictive control is to solve the optimal control problem for the current state on-line. Thus, no explicit expression for u (t; x(ti )) is obtained. Note that this is not equivalent to the rather diﬃcult task of ﬁnding a solution to the underlying Hamilton Jacobi Bellman PDE, since only the current state is considered. Typically the resulting dynamic optimization problem is solved using the so called direct approach, which has attracted signiﬁcant research interest in recent years (see e.g. [41, 2, 40, 4, 11, 17, 13]). Speciﬁcally, it has been established that an on-line solution is possible for realistically sized problems even with present-day computing power. We denote the solution of (1) starting at time t1 from an initial state x(t1 ), applying an input u : [t1 , t2 ] → Rm by x(τ ; u(·), x(t1 )), τ ∈ [t1 , t2 ]. For clarity of presentation we limit ourselves to input signals that are piecewise continuous and thus refer to an admissible input as: Deﬁnition 2. (Admissible Input) An input u : [0, Tp ] → Rm for a state x0 is called admissible, if it is: a) piecewise continuous, b) u(τ ) ∈ U ∀τ ∈ [0, Tp ], c) x(τ ; u(·), x0 ) ∈ X ∀τ ∈ [0, Tp ], d) x(Tp ; u(·), x0 ) ∈ E. Furthermore, we often refer to the so-called value function: Deﬁnition 3. (Value function) The value function V (x) is deﬁned as the minimal value of the cost for the state x: V (x) = J(¯ u (·; x); x). Various sampled-data NMPC schemes that guarantee stability and ﬁt into the given setup exist [6, 20, 36, 7, 25, 31, 16]. These schemes diﬀer in the way the 1

A continuous function α : [0, ∞) → [0, ∞) is a class K function, if it is strictly increasing and α(0) = 0.

298

R. Findeisen et al.

terminal penalty term E and (if it appears at all) the terminal region E are determined. We do not assume an explicit controllability assumption on the system. Instead, as often done in NMPC, we derive stability under the assumption of initial feasibility of the optimal control problem.

3 Nominal Stability of State Feedback NMPC The following theorem establishes conditions for the convergence of the closedloop states to the origin. It is a slight modiﬁcation of the results given in [20] and [6, 5]. We state it here together with a condensed proof, since it lays the basis for the robustness considerations in Section 4. Theorem 1. (Stability of sampled-data NMPC) Suppose that (a) the terminal region E ⊆ X is closed with 0 ∈ E and that the terminal penalty E(x) ∈ C 1 is positive semi-deﬁnite (b) ∀x ∈ E there exists an (admissible) input uE : [0, π ¯ ] → U such that x(τ ) ∈ E and ∂E ¯] (5) f (x(τ ), uE (τ )) + F (x(τ ), uE (τ )) ≤ 0 ∀τ ∈ [0, π ∂x (c) the NMPC open-loop optimal control problem is feasible for t = 0. Then for the closed-loop system (1), (4) x(t) → 0 for t → ∞, and the region of attraction R consists of the states for which an admissible input exists. Proof. As usual in predictive control the proof consists of two parts: in the ﬁrst part it is established that initial feasibility implies feasibility afterwards. Based on this result it is then shown that the state converges to the origin. Feasibility: Consider any sampling instant ti for which a solution exists (e.g. t0 ). In between ti and ti+1 the optimal input u ¯ (τ ; x(ti )) is implemented. Since no model plant mismatch nor disturbances are present, x(ti+1 ) = ¯ (τ ; x(ti )), x(ti )). Thus, the remaining piece of the optimal input x ¯(ti+1 ; u u ¯ (τ ; x(ti )), τ ∈ [ti+1 , ti + Tp ] satisﬁes the state and input constraints. Fur¯ (τ ; x(ti ))) ∈ E and we know from assumption (b) thermore, x ¯(ti +Tp ; x(ti ), u of the theorem that for all x ∈ E there exists at least one input uE (·) that renders E invariant on [ti + Tp , ti + Tp + π ¯ ]. Picking any such input we obtain as admissible input for any time ti + σ, σ ∈ (0, ti+1 − ti ] u ¯ (τ ; x(ti )), τ ∈ [ti + σ, ti +Tp ] u ˜(τ ; x(ti + σ)) = . (6) uE (τ − ti − Tp ), τ ∈ (ti + Tp , ti +Tp + σ] Speciﬁcally, we have for the next sampling time (σ = ti+1 − ti ) that u ˜(·; x(ti+1 )) is a feasible input, hence feasibility at time ti implies feasibility at ti+1 . Thus, if (2) is feasible for t = 0, it is feasible for all t ≥ 0. Furthermore, if the states for which an admissible input exists converge to the origin, it is clear that the region of attraction R consists of those points.

Towards a Sampled-Data Theory for Nonlinear Model Predictive Control

299

Convergence: We ﬁrst show that the value function is decreasing starting from a sampling instant. Remember that the value of V at for x(ti ) is given by: 6 ti +Tp V (x(ti )) = F (¯ x(τ ; u ¯ (·; x(ti )), x(ti )), u ¯ (τ ; x(ti ))dτ ti

+E(¯ x(ti +Tp ; u ¯ (·; x(ti )), x(ti ))),

¯ (·; x(ti )), x(ti )), and the cost resulting from (6) starting from any x(ti + σ; u σ ∈ (0, ti+1 − ti ], using the input u ˜(τ, x(ti + σ)), is given by: 6 J(x(ti +σ), u ˜(·; x(ti +σ))) =

ti +σ+Tp

F (¯ x(τ ; u ˜(·; x(ti +σ)), x(ti +σ)), u ˜(τ ; x(ti +σ)))dτ

ti +σ

+E(¯ x(ti + σ+Tp ; u ˜(·; x(ti + σ)), x(ti + σ))). Reformulation yields ˜(·; x(ti + σ))) = V (x(ti )) J(x(ti + σ), u 6 ti +σ − F (¯ x(τ ; u ¯ (·; x(ti )), x(ti )), u ¯ (τ ; x(ti )))dτ −E(¯ x(ti +Tp ; u ¯ (·; x(ti )), x(ti ))) ti

6 +

ti +σ+Tp

F (¯ x(τ ; u ˜(·; x(ti + σ)), x(ti + σ)), u ˜(τ ; x(ti + σ)))dτ

ti +Tp

+E(¯ x(ti + σ+Tp ; u ˜(·, x(ti + σ)), x(ti + σ))). Integrating inequality (5) from ti +σ to ti +σ+Tp starting from x(ti + σ) we obtain zero as an upper bound for the last three terms on the right side. Thus, 6 ti +σ J(x(ti + σ), u ˜(·, x(ti + σ)))−V (x(ti )) ≤ − F (¯ x(τ ; u ¯ (·; x(ti ))), u ¯ (τ ; x(ti )))dτ. ti

Since u ˜ is only a feasible but not necessarily the optimal input for x(ti + σ), it follows that 6 ti +σ V (x(ti + σ))−V (x(ti )) ≤− F (¯ x(τ ; u ¯ (·; x(ti )), x(ti )), u ¯ (τ ; x(ti )))dτ, (7) ti

i.e. the value function is decreasing along solution trajectories starting at a sampling instant ti . Especially we have that: 6 ti+1 x(τ ; u ¯ (·; x(ti )), x(ti )), u ¯ (τ ; x(ti )))dτ. V (x(ti+1 ))−V (x(ti )) ≤− F (¯ ti

By assumption, this decrease in the value function is strictly positive for x(ti ) = 0. Since this holds for all sampling instants, convergence holds similarly to [20, 7] by an induction argument and the application of Barbalat’s lemma.

300

R. Findeisen et al.

Various ways to determine a suitable terminal penalty term and terminal region exist. Examples are the use of a control Lyapunov function as terminal penalty E [25] or the use of a local nonlinear or linear control law to determine a suitable terminal penalty E and a terminal region E [36, 7, 6, 9, 31]. Note that Theorem 1 allows to consider the stabilization of systems that can only be stabilized by feedback that is discontinuous in the state [20], e.g. nonholonomic mechanical systems. However, for such systems it is in general rather diﬃcult to determine a suitable terminal region and a terminal penalty term. In the next section, we examine when and under which conditions the nominal NMPC controller is robust against (small) disturbances. The examination is based on the observation that the decrease of the value function in (7) is strictly positive. Since for convergence only a (ﬁnite) decrease in the value function is necessary, one can consider the integral term on the right hand side of (7) as a certain robustness margin.

4 Robustness of State Feedback Sampled-Data NMPC Several NMPC schemes have been proposed that take uncertainties directly in the controller formulation into account. Typically these schemes follow a game-theoretic approach and require the on-line solution of a min-max problem (e.g. [28, 30, 8]). In this section we do not consider the design of a robustly stable NMPC controller. Instead we examine if sampled-data NMPC based on a nominal model possess certain inherent robustness properties with respect to small model uncertainties and disturbances. We note that the results derived show similarities to the discrete time results presented in [39]. However, since we consider the stabilization of a continuous time system applying pieces of open-loop input signals, we also have to take the inter-sampling behavior into account. The results are also related to the robustness properties of discontinuous feedback via sample and hold [26]. However, note that we do not consider a ﬁxed input over the sampling time. Speciﬁcally, we consider that the disturbances aﬀecting the system lead to the following modiﬁed system equation: x˙ = f (x, u) + p(x, u, w)

(8)

where f , x and u are the same as in Section 2, where p : Rn × Rm × Rl → Rn describes the model uncertainty/disturbance, and where w ∈ W ∈ Rl might be an exogenous disturbance acting on the system. It is assumed that p is bounded over the region of interest, R × U × W. With regard to existence of solutions, we make the following assumption: Assumption 5 The system (8) has a continuous solution for any x(0) ∈ R, any piecewise continuous input u(·) : [0, Tp ] → U, and any exogenous disturbance w(·) : [0, Tp ] → W.

Towards a Sampled-Data Theory for Nonlinear Model Predictive Control

301

With respect to the value function V we assume that: Assumption 6 The value function is continuous. Assumption 7 There exists a K function αV such that for all x1 , x2 ∈ R : V (x1 ) − V (x2 ) ≤ αV (x1 − x2 ). In the following Ωc denotes level sets of V contained in R, where c > 0 speciﬁes the level: Ωc = {x ∈ R|V (x) ≤ c}. Given this deﬁnition we furthermore assume that Assumption 8 For all compact sets S ⊂ R there is at least one level set Ωc such that S ⊂ Ωc . In general there is no guarantee that a stabilizing NMPC schemes satisﬁes Assumption 6, especially if state constrains are present. As is well known [34, 20], NMPC can also stabilize systems that cannot be stabilized by feedback that is continuous in the state. Such feedbacks in general also imply a discontinuous value function. Many NMPC schemes, however, satisfy this assumption at least locally around the origin [7, 9, 33]. Furthermore, NMPC schemes that are based on control Lyapunov functions [25] without any constraints on the states and inputs satisfy Assumption 6. 4.1 Stability Deﬁnition and Basic Idea We consider persistent disturbances and the repeated application of open-loop inputs, i.e. we cannot react instantaneously to disturbances. Thus, asymptotic stability cannot be achieved, and the region of attraction R is in general not invariant. As a consequence, we desire in the following only “ultimate boundedness”-results; that the norm of the state after some time becomes small, and that this should hold on inner approximations of R. Furthermore, we want to show that the bound can be made arbitrarily small depending on the bound on the disturbance and the sampling time (practical stability), and that the region of initial conditions where this holds can be made arbitrarily large with respect to R (semiglobal ). In view of Assumption 8 and for simplicity of presentation, we parameterize these regions with level sets. Speciﬁcally we derive bounds that the maximum allowable disturbance and sampling time must fulﬁll such that we converge from any arbitrary level set of initial conditions Ωc0 ⊂ R in ﬁnite time to an arbitrary small (but ﬁxed) set Ωα around the origin without leaving a desired set Ωc ⊂ R with c > c0 , compare Figure 1. The derived results are based on the observation that small disturbances and model uncertainties lead to a (small) diﬀerence between the predicted state x ¯ and the real state x. As will be shown, the inﬂuence of the disturbance on the value function can be bounded by

302

R. Findeisen et al.

R

Ωc

x(0)

Ωα Ωc0

Fig. 1. Set of initial conditions Ωc0 , maximum attainable set Ωc , desired region of convergence Ωα and nominal region of attraction R.

6 ti+1 V (x(ti+1 ))−V (x(ti )) ≤− F (¯ x(τ ; u ¯ (·; x(ti )), x(ti )), u ¯ (τ ; x(ti )))dτ ti

+ (ti+1 − ti , x(ti ), w(·)),

(9)

where corresponds to the disturbance contribution. Thus, if the disturbance contribution (ti+1 ) “scales” with the size of disturbance (it certainly also scales with the sampling time ti+1 − ti ) one can achieve contraction of the level sets, at least at the sampling points. To bound the minimum decrease in the derivations below, we need the following fact: Fact 1 For any c > α > 0 with Ωc ⊂ R, Tp > δ > 0 the lower bound Vmin (c, α, δ) on the value function exists and is non-trivial for all x0 ∈ Ωc /Ωα : 9δ 0 < Vmin (c, α, δ) := minx0 ∈Ωc /Ωα 0 F (¯ x(s; u(·; x0 ), x0 ), u ¯ (s; x0 ))ds < ∞. 4.2 Additive Disturbances Considering the additive disturbance p in (8) we can derive the following Theorem Theorem 2. Given arbitrary level sets Ωα ⊂ Ωc0 ⊂ Ωc ⊂ R. Furthermore, assume that the additive disturbance satisﬁes p(x, u, w) ≤ pmax with pmax Lf x π¯ αV e − 1 ≤ min {c − c0 , Vmin (c, α/4, π), α/2} (10) Lf x where Lf x is the Lipschitz constant of f over Ωc . Then for any x(0) ∈ Ωc0 the closed-loop trajectories under the nominal feedback (4) will not leave the set Ωc , x(ti ) ∈ Ωc0 ∀i ≥ 0, and there exists a ﬁnite time Tα such that x(τ ) ∈ Ωα ∀τ ≥ Tα . Proof. The proof consists of 3 parts. In the ﬁrst part we establish conditions that guarantee that the state does not leave the set Ωc for all x(ti ) ∈ Ωc0 .

Towards a Sampled-Data Theory for Nonlinear Model Predictive Control

303

In the second part we establish conditions such that the states converge in ﬁnite time to the set Ωα/2 . In the last part we derive bounds, such that for all x(ti ) ∈ Ωα/2 the state does not leave the set Ωα . First part (x(ti +τ ) ∈ Ωc ∀x(ti ) ∈ Ωc0 ): We start by comparing the nominal (predicted) trajectory x ¯ and the trajectory of the real state x starting from ¯(ti + τ ) can the same initial state x(ti ) ∈ Ωc0 . First note that x(ti + τ ) and x be written as (skipping the additional arguments the state depends on): 6 ti +τ x(ti + τ ) = x(ti ) + (f (x(s), u(s; x(ti ))) + p(x(s), u(s; x(ti )), w(s)))ds ti ti +τ

6 x ¯(ti + τ ) = x(ti ) +

ti

f (¯ x(s), u(s; x(ti )))ds.

This is certainly possible for all times τ ≥ 0 such that x ¯(ti + τ ) ∈ Ωc and ¯, using the Lipschitz property of f in x x(ti + τ ) ∈ Ωc . Subtracting x from x inside of Ωc (where Lf x is the corresponding Lipschitz constant), applying the triangular inequality and partial integration as well as the Gronwall-Bellman inequality we obtain: pmax Lf x τ e ¯(ti + τ ) ≤ −1 . (11) x(ti + τ ) − x Lf x Furthermore, at least as long as x is in Ωc we have that x(ti + τ )) V (x(ti + τ )) − V (x(ti )) ≤ V (x(ti + τ )) − V (¯ pmax Lf x τ ≤ αV (x(ti + τ ) − x e ¯(ti + τ )) ≤ αV −1 . Lf x Here we used that V (¯ x(ti + τ )) − V (x(ti )) ≤ 0 (see (7)). Thus, if pmax Lf x π¯ e − 1 ≤ c − c0 αV Lf x

(12)

then x(ti + τ ) ∈ Ωc τ ∈ [0, ti+1 − ti ], ∀x(ti ) ∈ Ωc0 . Second part (x(ti ) ∈ Ωc0 and ﬁnite time convergence to Ωα/2 ): Assume that (12) holds. Note that (12) assures that x(ti + τ ) ∈ Ωc , ∀τ ∈ [0, ti+1 − ti ]. Assuming that x(ti ) ∈ Ωα/2 we know that x(ti+1 )) + V (¯ x(ti+1 )) − V (x(ti )) V (x(ti+1 )) − V (x(ti )) = V (x(ti+1 )) − V (¯ pmax Lf x π¯ ≤ αV e − 1 − Vmin (c, α/2, π). Lf x To achieve convergence to the set Ωα/2 in ﬁnite time we need that the right hand side is strictly less than zero. If we require that pmax Lf x π¯ αV e −1 ≤ Vmin (c, α/4, π), Lf x

304

R. Findeisen et al.

then we achieve ﬁnite convergence, since V (x(ti+1 )) − V (x(ti )) ≤ kdec := −Vmin (c, α/2, π)+Vmin (c, α/4, π) < 0 as α/4 < α/2. Thus, for any x(ti ) ∈ Ωc0 we have ﬁnite time convergence to ? the set Ωα/2 for a sampling time tm that >

. We can also conclude that x(ti+1 ) ∈ Ωc0 satisﬁes tm − ti ≤ Tα := c−α/2 kdec for all x(ti ) ∈ Ωc0 . Third part (x(ti+1 ) ∈ Ωα ∀x(ti ) ∈ Ωα/2 ): This is trivially satisﬁed following the arguments in the ﬁrst part of the proof, assuming that pmax Lf x π¯ e − 1 ≤ α/2. αV Lf x If V is locally Lipschitz over all compact subsets of R, it is possible to replace condition (10) by the following more explicit one: pmax ≤

Lf x min {(c − c0 ), Vmin (c, α/4, π), α/2} . LV (eLf x π¯ − 1)

Here LV is the Lipschitz constant of V over Ωc . Remark 2. Calculating the robustness bound is diﬃcult, since in general no explicit expression for Vmin (c, α/4, π) can be found, nor is it in general possible to calculate the necessary Lipschitz constants or to obtain an explicit expression for αV . The result is still of value, since it underpins that small additive disturbances can be tolerated and it can be utilized for the design of output feedback NMPC schemes. 4.3 Input Disturbances/Optimization Errors The results can be easily extended to disturbances that directly act on the input. To do this we have to assume that f is also Lipschitz in u over R × U. One speciﬁc case of such disturbances can for example be errors in the optimal input due to the numerical solution of the optimal control problem. To simplify the presentation we assume that the disturbed input is given by u ¯ (t; x(ti )) + v(t), where v(·) is assumed to be piecewise continuous. Following the ideas in the ﬁrst part of the proof of Theorem 2, we obtain 6 ti +τ ¯(ti + τ ) ≤ Lf x x(s) − x ¯(s)ds + Lf u vmax τ, x(ti + τ ) − x ti

where Lf u is the Lipschitz constant of f (x, u) with respect to u over Ωc × U, and vmax is the maximum input error. Via the Gronwall-Bellman inequality, this gives (11) with pmax exchanged with Lf u vmax . The remainder of the proof stays unchanged, thus we obtain the following result for input disturbances: Theorem 3. Given the level sets Ωα ⊂ Ωc0 ⊂ Ωc ⊂ R and assuming that the additive input disturbance satisﬁes u ≤ umax and that

Towards a Sampled-Data Theory for Nonlinear Model Predictive Control

αV

Lf u vmax Lf x π¯ e −1 Lf x

305

≤ min {(c − c0 ), Vmin (c, α/4, π), α/2} .

(13)

Then for any x(0) ∈ Ωc0 the closed-loop trajectories under the nominal feedback (4) will not leave the set Ωc , x(ti ) ∈ Ωc0 ∀i ≥ 0, and there exists a ﬁnite time Tα such that x(τ ) ∈ Ωα ∀τ ≥ Tα . Assuming that V is locally Lipschitz we can obtain, similarly as for Theorem 2, a more explicit bound: pmax ≤

Lf x min {(c − c0 ), Vmin (c, α/4, π), α/2} . Lf u LV (eLf x π¯ − 1)

(14)

One direct implication of this result is that approximated solutions to the optimal control problem can in principle be tolerated. Such approximated solutions can for example result from the numerical integration of the diﬀerential equations, as considered in [23]. Furthermore, Theorem 3 gives a theoretical foundation for the so called real-time iteration scheme, in which only one Newton step optimization is performed per sampling instant [13]. Note that the result can, similarly to results on robustness properties of discontinuous feedback via sample-and-hold [26], in principle be expanded to other disturbances, e.g. neglected fast actuator dynamics or computational delays.

5 Output-Feedback Sampled-Data NMPC One of the key obstacles for the application of NMPC is that at every sampling instant ti the system state is required for prediction. However, often not all system states are directly accessible. To overcome this problem one typically employs a state observer for the reconstruction of the states. Yet, due to the lack of a general nonlinear separation principle, stability is not guaranteed, even if the state observer and the NMPC controller are both stable. Several researchers have addressed this question. The approach in [12] derives local uniform asymptotic stability of contractive NMPC in combination with a “sampled” state estimator. In [29], see also [39], asymptotic stability results for observer based discrete-time NMPC for “weakly detectable” systems are given. The results allow, in principle, to estimate a (local) region of attraction of the output feedback controller from Lipschitz constants. In [37] an optimization based moving horizon observer combined with a certain NMPC scheme is shown to lead to (semi-global) closed-loop stability. Here we follow and expand the ideas derived in [19, 18, 24], where semiglobal stability results for output-feedback NMPC using high-gain observers are derived. In this section we outline explicit conditions on the observer error, allowing to consider diﬀerent types of observers such as moving horizon observers, sliding mode observers, observers with a linear error dynamics with arbitrary placeable poles, or observers with a ﬁnite time error convergence.

306

R. Findeisen et al.

5.1 Robustness to Estimation Errors We assume that instead of the real system state x(ti ) at every sampling instant only a state estimate x ˆ(ti ) is available. Thus, instead of the optimal feedback (4) the following “disturbed” feedback is applied: ¯ (t; x ˆ(ti )), u(t; x ˆ(ti )) = u

t ∈ [ti , ti+1 ) .

(15)

The estimated state x ˆ(ti ) can be outside the region of attraction R. To avoid feasibility problems we assume that the input is ﬁxed to an arbitrary, bounded value in this case. Similarly to the previous results, we can state that: Theorem 4. Given the level sets Ωα ⊂ Ωc0 ⊂ Ωc ⊂ R. Furthermore, assume ˆ(ti ) ≤ emax , where that the state estimation error satisﬁes x(ti ) − x 1 Lf x π ¯ αV (e emax ) + αV (emax ) ≤ min c − c0 , Vmin (c, α/4, π), α/4 . 2 Then for any x(0) ∈ Ωc0 the closed-loop trajectories with the feedback (4) will not leave the set Ωc , x(ti ) ∈ Ωc0 ∀i ≥ 0, and there exists a ﬁnite time Tα such that x(τ ) ∈ Ωα ∀τ ≥ Tα . Proof. The proof follows the ideas of Theorem 2. First part (x(ti +τ ) ∈ Ωc ∀x(ti ) ∈ Ωc0 ): We consider the diﬀerence in the value function between the initial state x(ti ) ∈ Ωc0 at a sampling time ti and the developing state x(ti +τ ; x(ti ), uxˆ ). For simplicity of notation, uxˆ denotes in the following the optimal input resulting from x ˆ(ti ) and ux the input that correspond to the real state x(ti ). Furthermore, xi = x(ti ) and x ˆi = x ˆ(ti ). By adding and subtracting terms to the diﬀerence in the value function, we obtain the following equality V (x(τ ; xi , uxˆ ))−V (xi ) = V (x(τ ; xi , uxˆ )) − V (x(τ ; x ˆi , uxˆ )) + V (x(τ ; x ˆi , uxˆ )) − V (ˆ xi ) + V (ˆ xi ) − V (xi ). (16) ˆi , uxˆ ) ∈ Ωc ) if One way to ensure that x ˆi ∈ Ωc (this also implies that x(τ ; x xi ∈ Ωc0 is to require that αV (emax ) ≤ c − c0 . Then the last two terms can be bounded using αV , which is also possible for the ﬁrst two terms: xi − xi ) V (x(τ ; xi , uxˆ ))−V (xi ) ≤ αV (eLf x (τ −ti ) ˆ 6 τ − F (x(s; x ˆi , uxˆ ), uxˆ )ds + αV (ˆ xi − xi ) ti

From this it follows (since the contribution of the integral is negative) that if αV (eLf x π¯ emax ) + αV (emax ) ≤ c − c0 (which implies αV (emax ) ≤ c − c0 ) then x(ti + τ ) ∈ Ωc ∀τ ∈ (ti+1 − ti ).

(17)

Towards a Sampled-Data Theory for Nonlinear Model Predictive Control

307

Second part (x(ti ) ∈ Ωc0 and ﬁnite time convergence to Ωα/2 ): We assume that (17) holds and that x(ti ) ∈ Ωc0 . Note that (12) assures that x(ti +τ ) ∈ Ωc , ∀τ ∈ [0, ti+1 − ti ]. Assuming that x(ti ) ∈ / Ωα/2 and that αV (emax ) ≤ α/4 we know that x ˆ(ti ) ∈ / Ωα/4 . Then we obtain from (16) that V (x(τ ; xi , uxˆ ))−V (xi )≤−Vmin (c, α/4, τ −ti )+αV (eLf x π¯ ˆxi −xi )+αV (ˆ xi −xi ) Thus we know that if αV (eLf x π¯ emax ) + αV (emax ) ≤

1 Vmin (c, α/4, π) 2

and αV (emax ) ≤ α/4

that we achieve ﬁnite time convergence from any x(ti ) >∈ Ωc0 to ? the set Ωα/2 c−α/2 for a sampling time tm that satisﬁes tm − ti ≤ Tα := kdec . We can also conclude that x(ti+1 ) ∈ Ωc0 for all x(ti ) ∈ Ωc0 . Third part (x(ti+1 ) ∈ Ωα ∀x(ti ) ∈ Ωα/2 ): This is trivially satisﬁed following the arguments in the ﬁrst part of the proof, assuming that αV (eLf x π¯ emax ) + αV (emax ) ≤ α/2.

As for Theorem 3 and Theorem 2 it is possible to derive an explicit bound on emax assuming that V is locally Lipschitz: 1 1 min (c − c V ), (c, α/4, π), α/4 . (18) emax ≤ 0 min LV (eLf x π¯ + 1) 2 The result allows the design of output feedback NMPC controllers. 5.2 Output Feedback NMPC Theorem 4 lays the basis for the design of observer based output feedback NMPC controllers that achieve semi-global practical stability. Semi-global practical stability here means that for any given three sets Ωα ⊂ Ωc0 ⊂ Ωc ⊂ R there exists observer parameters and an upper bound on the maximum sampling time π ¯ , such that the closed-loop system states will not leave the set Ωc and converge in ﬁnite time to the practical stability region Ωα , where they remain afterwards. Achieving the semi-global practical stability requires that the observer error x(ti ) − x ˆ(ti ) can be made suﬃciently small. Since the required bound of emax directly depends on c − c0 and on α, as well as on the maximum (¯ π) and minimum (π) sampling time, using ﬁxed NMPC controller parameters (in addition to the sampling time) requires that the observer has some sort of a tuning knob to decrease the maximum observer error emax . One possibility for such an observer is a high-gain observer, which allows under certain further restrictions, that the observer error can be suﬃciently

308

R. Findeisen et al.

decreased in a suﬃciently short time by increasing the observer gain. This approach has been exploited in [19, 24] for output feedback stabilization of nonlinear MIMO systems which are uniformly globally observable. We do not go into details here, and refer to [19] for the sampled-data case. We note that it is also possible to consider other observers, which allow a suﬃcient decrease in the observer error. One example are moving horizon observers with contraction constraint [37], where increase of the contraction rate allows to achieve any desired observer error. Other examples are observers with linear error dynamics that allows to place the poles of the error dynamics arbitrarily. Such observers can for example be obtained exploiting certain normal forms and output injection [3, 27]. Another class of suitable observers are observers that achieve a ﬁnite time observer error convergence such as sliding mode observers [14] or the approach presented in [15, 35].

6 Conclusions In this paper we considered the stabilization of nonlinear systems using NMPC with sampled measurement information. In a ﬁrst step we reviewed a generic stability result for sampled-data NMPC. Based on this stability result we considered the inherent robustness properties of sampled-data NMPC. Speciﬁcally we showed that NMPC possesses some inherent robustness properties to additive disturbances in the diﬀerential equations, to input disturbances and to measurement uncertainties, which could for example be caused by the application of a state observer. The robustness to measurement uncertainty derived here can be used to derive output feedback schemes that achieve semiglobal practical stability, that is, for a fast enough sampling frequency and fast enough observer, it recovers up to any desired accuracy the NMPC state feedback region of attraction (semi-global) and steers the state to any (small) compact set containing the origin (practical stability). The price to pay is that the value function must be continuous. In general there is no guarantee that nominally stable NMPC schemes satisfy this assumption, especially if constraints on the states are present, see [21]. Thus, future research has to focus on either relaxing this condition, or to derive conditions under which an NMPC scheme does satisfy this assumption, see for example [22].

References 1. Allg¨ ower F, Badgwell TA, Qin JS, Rawlings JB, Wright SJ (1999) Nonlinear predictive control and moving horizon estimation – An introductory overview. In P.M. Frank, editor, Advances in Control, Highlights of ECC’99, pp. 391–449. Springer.

Towards a Sampled-Data Theory for Nonlinear Model Predictive Control

309

2. Bartlett RA, W¨ achter A, Biegler LT (2000) Active set vs. interior point strategies for model predictive control. In Proc. Amer. Contr. Conf., pp. 4229–4233, Chicago, Il. 3. Bestle D, Zeitz M (1983) Canonical form observer design for non-linear timevariable systems. Int. J. Contr., 38(2):419–431. 4. Biegler L (2000) Eﬃcient solution of dynamic optimization and NMPC problems. In F. Allg¨ ower and A. Zheng, editors, Nonlinear Predictive Control, pp. 219–244. Birkh¨ auser. 5. Chen H (1997) Stability and Robustness Considerations in Nonlinear Model Predictive Control. Fortschr.-Ber. VDI Reihe 8 Nr. 674. VDI Verlag, D¨ usseldorf. 6. Chen H, Allg¨ ower F (1998) Nonlinear model predictive control schemes with guaranteed stability. In R. Berber and C. Kravaris, editors, Nonlinear Model Based Process Control, pp. 465–494. Kluwer Academic Publishers. 7. Chen H, Allg¨ ower F (1998) A quasi-inﬁnite horizon nonlinear model predictive control scheme with guaranteed stability. Automatica, 34(10):1205–1218. 8. Chen H, Scherer CW, Allg¨ ower F (1998) A robust model predictive control scheme for constrained linear systems. In 5th IFAC Symposium on Dynamics and Control of Process Systems, DYCOPS-5, pp. 60–65, Korfu. 9. Chen W, Ballance DJ, O’Reilly J (2000) Model predictive control of nonlinear systems: Computational delay and stability. IEE Proceedings, Part D, 147(4):387–394. 10. De Nicolao G, Magni L, Scattolini R (2000) Stability and robustness of nonlinear receding horizon control. In F. Allg¨ ower and A. Zheng, editors, Nonlinear Predictive Control, pp. 3–23. Birkh¨ auser. 11. de Oliveira NMC, Biegler LT (1995) An extension of Newton-type algorithms for nonlinear process control. Automatica, 31(2):281–286. 12. de Oliveira Kothare S, Morari M (2000) Contractive model predictive control for constrained nonlinear systems. IEEE Trans. Automat. Contr., 45(6):1053– 1071. 13. Diehl M, Findeisen R, Schwarzkopf S, Uslu I, Allg¨ ower F, Bock HG, Schl¨ oder J (2002) An eﬃcient approach for nonlinear model predictive control of largescale systems. Part I: Description of the methodology. Automatisierungstechnik, 12:557–567. 14. Drakunov SV (1992) Sliding-mode Observer Based on Equivalent Control Method. In Proc. 31st IEEE Conf. Decision Contr., pp. 2368–2369, Tucson. 15. Engel R, Kreisselmeier G (2002) A continuous–time observer which converges in ﬁnite time. IEEE Trans. Aut. Control, 47(7):1202–1204. 16. Findeisen R, Allg¨ ower F (2001) The quasi-inﬁnite horizon approach to nonlinear model predictive control. In A. Zinober and D. Owens, editors, Nonlinear and Adaptive Control, Lecture Notes in Control and Information Sciences, pp. 89– 105, Berlin, Springer-Verlag. 17. Findeisen R, Diehl M, Uslu I, Schwarzkopf S, Allg¨ ower F, Bock HG, Schl¨ oder JP, Gilles ED (2002) Computation and performance assesment of nonlinear model predictive control. In Proc. 42th IEEE Conf. Decision Contr., Las Vegas. 18. Findeisen R, Imsland L, Allg¨ ower F, Foss BA (2002) Output feedback nonlinear predictive control - a separation principle approach. In Proceedings of 15th IFAC World Congress, Barcelona, Spain. 19. Findeisen R, Imsland L, Allg¨ ower F, Foss BA (2003) Output feedback stabilization for constrained systems with nonlinear model predictive control. Int. J. of Robust and Nonlinear Control, 13(3-4):211–227.

310

R. Findeisen et al.

20. Fontes FA (2000) A general framework to design stabilizing nonlinear model predictive controllers. Syst. Contr. Lett., 42(2):127–143. 21. Grimm G, Messina MJ, Teel AR, Tuna S (2002) Examples when model predictive control is nonrobust. (preprint) 22. Grimm G, Messina MJ, Teel AR, Tuna S (2002) Model predictive control: For want of a local control Lyapunov function, all is not lost. (preprint) 23. Gr¨ une L, Ne˘si´c D (2003) Optimization based stabilization of sampled-data nonlinear systems via their approximate discrete-time models. To appear in SIAM J. Contr. Optim. 24. Imsland L, Findeisen R, Bullinger E, Allg¨ ower F, Foss BA (2002) A note on stability, robustness and performance of output feedback nonlinear model predictive control. To appear in J. Proc. Contr. 25. Jadbabaie A, Yu J, Hauser J (2001) Unconstrained receding horizon control of nonlinear systems. IEEE Trans. Automat. Contr., 46(5):776–783. 26. Kellett C, Shim H, Teel A (2002) Robustness of discontinuous feedback via sample and hold. In Proc. Amer. Contr. Conf., pp. 3512–3516, Anchorage. 27. Krener AJ, Isidori A (1983) Linearization by output injection and nonlinear observers. Syst. Contr. Lett., 3:47–52. 28. Lall S, Glover K (1994) A game theoretic approach to moving horizon control. In D. Clarke, editor, Advances in Model-Based Predictive Control. Oxford University Press. 29. Magni L, De Nicolao G, Scattolini R (2001) Output feedback and tracking of nonlinear systems with model predictive control. Automatica, 37(10):1601– 1607. 30. Magni L, Nijmeijer H, van der Schaft AJ (2001) A receding-horizon approach to the nonlinear H∞ control problem. Automatica, 37(5):429–435. 31. Magni L, Scattolini R (2002) State-feedback MPC with piecewise constant control for continuous-time systems. In Proc. 42th IEEE Conf. Decision Contr., Las Vegas, USA. 32. Martinsen F, Biegler LT, Foss BA (2002) Application of optimization algorithms to nonlinear MPC. In Proceedings of 15th IFAC World Congress, Barcelona, Spain. 33. Mayne DQ, Rawlings JB, Rao CV, Scokaert POM (2000) Constrained model predictive control: stability and optimality. Automatica, 26(6):789–814. 34. Meadows ES, Henson MA, Eaton JW, Rawlings JB (1995) Receding horizon control and discontinuous state feedback stabilization. Int. J. Contr., 62(5):1217–1229. 35. Menold PH, Findeisen R, Allg¨ ower F (2003) Finite time convergent observers for linear time-varying systems. Submitted to the 11th Mediterranean Conference on Control and Automation MED’03. 36. Michalska H, Mayne DQ (1993) Robust receding horizon control of constrained nonlinear systems. IEEE Trans. Automat. Contr., AC-38(11):1623–1633. 37. Michalska H, Mayne DQ (1995) Moving horizon observers and observer-based control. IEEE Trans. Automat. Contr., 40(6):995–1006. 38. Qin SJ, Badgwell TA (1996) An overview of industrial model predictive control technology. In J.C. Kantor, C.E. Garcia, and B. Carnahan, editors, Fifth International Conference on Chemical Process Control – CPC V, pp. 232–256. American Institute of Chemical Engineers.

Towards a Sampled-Data Theory for Nonlinear Model Predictive Control

311

39. Scokaert POM, Rawlings JB, Meadows ES (1997) Discrete-time stability with perturbations: Application to model predictive control. Automatica, 33(3):463– 470. 40. Tenny MJ, Rawlings JB (2001) Feasible real-time nonlinear model predictive control. In 6th International Conference on Chemical Process Control – CPC VI, AIChE Symposium Series. 41. Wright SJ (1996) Applying new optimization algorithms to model predictive control. In J.C. Kantor, C.E. Garcia, and B. Carnahan, editors, Fifth International Conference on Chemical Process Control – CPC V, pp. 147–155. American Institute of Chemical Engineers.

High-Order Maximal Principles Matthias Kawski Arizona State University, Tempe, AZ 85287, [email protected]

— Dedicated to Arthur Krener on his 60th birthday. —

1 Introduction The High-Order Maximal Principle (HMP) [15] is one of the ﬁrst among A. Krener’s many major contributions to modern control theory. While some precursors had appeared in the literature in earlier years, compare the discussions in [15] and in the abbreviated announcement [14], it was [15] that became the starting point for much research in the next decades, continuing to this day – typical for many of A. Kreners’s path-breaking contributions. Originally formulated in the early 1970s [14], the HMP underwent a laborious and lengthy birth process (due to many technical intricacies) until it ﬁnally appeared as a journal article in 1977 [15]. This long delay is in large part due to the very delicate nature of precise technical conditions that must be imposed on needle variations in order to assure that higher-order approximating cones have the convexity properties that are needed for meaningful tests for optimality. It was only in the last year [6], almost 30 years after the HMP was ﬁrst announced, that a counter example was constructed that authoritatively demonstrated that these highly technical conditions on the families of control variations cannot be eliminated, even for some of the most benign systems. The purpose of this note is to survey some of these recent results and put them into perspective, demonstrating that even three decades after Krener’s original article major discoveries are still made in this research area. In particular, we highlight how these recent results demarcate a deﬁnite boundary beyond which any argumentation along the lines of the Maximum Principle of optimal control cannot possibly work.

This work was partially supported by the National Science Foundation through the grants DMS 00-72369 and DMS and DMS 01-07666.

W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 313–326, 2003.

c Springer-Verlag Berlin Heidelberg 2003

314

M. Kawski

The article is organized as follows: After this brief introduction we ﬁrst describe general features of reachable sets and their boundaries, and some key ideas of the classical approach, followed by a brief review of technical deﬁnitions, in particular families of control variations, approximating cones and the associated high order open mapping principles, and graded structures. The main focus is on convexity properties of approximating cones to the reachable sets. Next we discuss some recent counter examples, and highlight key aspects of the construction, pointing out how their features relate to classical theorems and technical conditions on control variations posed in the past. In the ﬁnal section, we point out some further consequences, speculating about deﬁnite boundaries beyond which the basic paradigm that underlies the Maximum Principle can no longer apply.

2 Reachable Sets and Their Boundaries, Pictorially Much progress has been made in recent years to extend classical optimal control techniques and results to ever more general systems, assuming ever weaker technical hypotheses, compare e.g. [21, 22] for a survey of the current state of the art, including a detailed discussion of “generalized diﬀerential quotients”. However, this note goes into the opposite direction: Rather than looking at the most general systems, we focus on the inherent limitations of the basic paradigm underlying any generalization of the Pontryagin Maximum Principle (PMP) – even in the case of most regular nonlinear systems. Thus, for the sake of clarity, we may restrict our attention to the well-studied class of analytic systems that are aﬃne in the controls x˙ = f (x) +

m

uj (t)gj (x),

x ∈ Rn ,

u ∈ U ⊆ Rm ,

(1)

j=1

initialized at x(0) = 0 ∈ Rn . The controls u(·) are assumed to be measurable and take values in the set of admissible control values U, a convex compact subset of Rm containing 0 in its interior. The vector ﬁelds are f and gj are real-analytic. Solution curves are denoted by x(t, u), or simply by x(t) if the control u is clear from the context. By suitably adding the running cost as an additional state, if necessary, one may rephrase a large class of general optimization as time-optimal control problems, and we restrict our attention to the latter. Departing from the classical calculus of variations, compare e.g. [20], optimal control starts by considering the set of all solution curves generated by the system (1). We refer to the union of their graphs as the funnel F(T ) of reachable sets, also thought of as the disjoint union {t} × R(t) ⊆ Rn+1 (2) F(T ) = 0≤t≤T

High-Order Maximal Principles

315

of copies of the reachable sets R(t), the sets of all points x(t, u) which can be reached in time t from x(0) along solution curves x(t, u) of (1) corresponding to admissible controls u(·). The funnel F(T ) is readily seen to be a closed set for general classes of systems which include the special case of systems (1) considered here. Thus any trajectory x∗ (·) corresponding to an optimal control u∗ (·) for time T must lie on the boundary of the reachable sets R(t) for all times 0 ≤ t ≤ T .

R(T )

Rn xs (·)

xs (T )

x∗ ≡ 0 p(T )

p(·)

t

Fig. 1. Extremal trajectories and the funnel of reachable sets

The Pontryagin Maximum Principle [17] asserts that a necessary condition for a control u∗ : [0, T ] → U to be optimal is that there exists an absolutely continuous, nontrivial section (x∗ , 0) ≡ (x∗ , p∗ ) : [0, T ] → T ∗ Rn that is a solution of the Hamiltonian system x˙ =

∂H ∗ (x, p) ∂p

p˙ = −

∂H ∗ (x, p) ∂x

(3)

def

where H ∗ (x, p) = maxu∈U H(x, p, u) is the pointwise maximized “pre-Hamiltonian” H : T ∗ Rn ×U → R which in turn is deﬁned by H(x, p, u) = p, f (x) + ∗ j uj gj (x). Pictorially, one may think of p (·) as an outward normal vector ∗ ﬁeld along the trajectory x (·), deﬁning at each time t ∈ (0, T ] a supporting hyperplane to (some approximating cones of) the reachable set R(t) at x∗ (t). While, in general, the reachable sets R(t) need not be convex, this picture is nonetheless correct as long as one considers only ﬁrst order approximating cones. The HMP and this note are concerned with the problems that arise when relaxing this restriction by developing notions of higher order variations. Basically, the PMP asserts that a necessary condition for x∗ (T ) to lie on the boundary of R(T ) is that the inner product (dual pairing) of p(T ) with any ﬁrst order variational vector ξ in an approximating cone K1 is nonpositive. The HMP extends this condition by requiring that this also holds for all suitably deﬁned higher order variational vectors ξ ∈ Kk , k ≥ 1 – where, of course, the precise deﬁnition of the cone Kk is the main issue. It is instructive to consider a simple example in order to clarify certain kinds of lack of convexity or smoothness that do not cause any diﬃculty, distinguishing these from more problematic situations. Consider the following

316

M. Kawski

two systems in the plane, which are easily seen to be equivalent via the analytic coordinate change (y1 , y2 ) = (x1 , x2 − x21 ) (with analytic inverse). x˙ 1 = u x˙ 2 = 3x21

y˙ 1 = u y˙ 2 = 3y12 − 2y1 u

x(0) = 0, |u(·)| ≤ 1

(4)

It is easy to see that all time-optimal controls are piecewise constant with at most two pieces. The constant controls u ≡ 1 and u ≡ −1 generate the lower boundaries of the reachable sets s → (s, |s3 |), and s → (s, |s3 | − s2 ), 0 ≤ s ≤ T , respectively, which are also trajectories of the system. The controls with us (t) = ±1 for 0 ≤ t ≤ s and us (t) = ∓1 for s ≤ t ≤ T steer to the curves of endpoints s → (2s − T, 2s3 + (T − 2s)3 ) and s → (2s − T, 2s3 + (T − 2s)3 − (2s − T )2 ), respectively. For T /2 ≤ s ≤ T these form the upper boundaries of the reachable sets, i.e. these controls are optimal. However, if the second piece is longer than the ﬁrst leg, i.e. if T > 2s > 0 then these controls are no longer optimal, and the endpoints of the respective trajectories lie in the interior of the reachable sets.

Fig. 2. Harmless lack of convexity of reachable sets

This simple example illustrates several features: Most importantly, the boundaries are piecewise smooth. It is conceivable that the boundaries of reachable sets of some systems might not have locally ﬁnite stratiﬁcations, and it was only in [2] that the question about their generic subanalyticity was ﬁnally resolved. Outward corners of R(1), like those at the points (±1, 1) and (±1, 0), respectively, clearly pose no problems for characterization via supporting hyperplanes. The point (0, 0) lies on the boundary of the reachable set for either system, yet the interior of the reachable set of the second system contains points in each of the four quadrants that are arbitrarily close to (0, 0), i.e. there is no supporting hyperplane of the reachable set that passes through (0, 0). Since these two systems are equivalent up to a simple polynomial coordinate change (with polynomial inverse), this makes it clear that one should not directly look for supporting hyperplanes of the reachable sets themselves, but rather for supporting hyperplanes of suitably deﬁned approximating cones. Finally, the reachable sets of both systems also exhibit an inward corner at (0, 1/4). However, this is much less problematic than it may appear at

High-Order Maximal Principles

317

ﬁrst sight. The reason is that this corner is eﬀectively due to a foldover of the reachable sets resulting in conjugate points. Indeed, there are two distinct controls u∗ = −u , each switching between the values ±1 and ∓1 at t0 = T /2 that steer to this point. The key observation is that for every control u that is close to u∗ (close to u , respectively) in any reasonable sense (e.g. in the L1 norm), the endpoints of the corresponding trajectory all lie below the curve s → (2s − T, 2s3 + (T − 2s)3 ) (or s → (T − 2s, 2s3 + (T − 2s)3 ), respectively) near s = T /2. Thus one is naturally led to construct local approximating cones to the reachable set by restricting the perturbed controls to be close to the reference control – i.e. in general one does not expect to obtain good conditions by simply working with all endpoints near the reference endpoint. However, the examples presented later in this note will show that even this is not enough as it is still possible to have inward corners, even when restricting all controls to be close in e.g. L1 .

3 Control Variations and Approximating Cones A family of control variations of a reference control u∗ , deﬁned on some interval [0, T ] is a (not necessarily continuous) curve s → us : [0, T ] → U in the space L1

of all admissible controls such that us −→ u0 = u∗ as s $ 0. We say that such a family generates ξ ∈ Rn as an k-th order tangent vector to the reachable set R(T ) at x(T, u∗ ), written ξ ∈ Kk , if x(T, us ) = x(T, u∗ ) + us − u∗ kL1 ξ + o(us − u∗ kL1 ).

(5)

Also, write Kk = {λξ : λ ≥ 0, ξ ∈ k }. Note that this cone generally not only depends on the endpoint x(T, ·) ∈ R(T ) but also on the choice of the reference control u∗ . More speciﬁcally, in the example discussed in the preceding section, there are two distinct second cones, each a half plane, at the same point (0, 1/4) – corresponding to the diﬀerent reference controls u∗ = −u . It is easy to see that the cones {Kk }k≥0 form an increasing sequence: Using a suitably reparameterization of the family {us }s≥0 one ﬁnds that if ξ ∈ Kk then also ξ ∈ Kk+ for all ≥ 0. To be useful as a tool to derive optimality criteria it is highly desirable to reﬁne the notion of admissible families of control variations so that • each cone Kk is convex, and • an open mapping theorem is available for the particular class of cones. The ﬁrst property allows one to work with just a small, ﬁnite, number of families of control variations, and still conclude that the cone is large. In the ideal case (n + 1) well-chosen families of control variations may already suﬃce to conclude that the cone has n-dimensional interior, or even that it is the whole (tangent)-space.

318

M. Kawski

The second desired property makes the cone into an approximating cone: If the cone is the whole space, one wants to be able to conclude that the reference endpoint x∗ (T ) lies in the interior of the reachable set R(T ), and hence u∗ cannot be optimal. Moreover, a desirable property is that even in the case that the cone is not the whole space, it still has a directional approximation property: E.g., for every convex cone C that is contained in the interior of the approximating cone Kk (plus its vertex {0}), there exist some ε > 0 such that the reachable set R(T ) contains the intersection (x∗ (T ) + C) ∩ Bε (x0 ) of the cone C with some open ball. In the case of ﬁrst order approximating cones, i.e. k = 1, well-known properties of ﬁrst order derivatives make it relatively simple to establish the two desired properties, essentially yielding the Pontryagin Maximum Principle. Another noteworthy special case is that of a stationary reference trajectory, i.e. x∗ ≡ 0 and f (0) = 0. In this case it is remarkably simple to construct suitable cones Kk that have both of these properties, and moreover, even explicitly construct controls steering to any desired target in C ∩ Bε (0), starting from only a ﬁnite number of families of control variations, compare [12] (a special case of the general results [8, 9]). What so much simpliﬁes the case of a stationary reference trajectory x∗ ≡ 0 is the ability to concatenate (append) any control us deﬁned on an interval [0, s] to the zero-control on the interval [0, T − s], to obtain a new control u ˜s , now deﬁned on [0, T ], but such that x(T, u ˜s ) = x(s, us ) ∈ R(s) ⊆ R(T ). It is these families of control variations with each us deﬁned (or diﬀering from u∗ ≡ 0) only on the interval [0, s] that prove to be critical in the explicit constructions [12]. In the case of a nonstationary reference trajectory there exists a very rich literature of ever more intricate technical conditions that are imposed on the families of control variations in order to be able to derive both the desired convexity results and approximation properties (open mapping theorems). Krener’s original article [15] was just the most prominent starting point in this series. Other notable approaches include [5, 7, 8, 9, 13], all the way to [21, 22]. Suppose that {us }s≥0 and {vs }s≥0 are two families of control variations of the same reference control u∗ = u0 = v0 , generating the tangent vectors ξ ∈ Kk and η ∈ Kk to R(T ) at x(T, u∗ ), respectively. In order to obtain, for any λ ∈ [0, 1], a new family of control variations {ws }s≥0 that generates (a positive multiple of) (λξ + (1 − λ)η) as a k-th order tangent vector, one would like to combine elements from {us }s≥0 and {vs }s≥0 . Conceptually, the most attractive case is when for suﬃciently small s0 > 0 the support of the functions (us − u∗ ), s < s0 is disjoint from support of the functions (vs − u∗ ), s < s0 , i.e. if Su ∩ Sv = ∅ where Su =

s<s0

{t ∈ [0, T ] : us (t) = u∗ (t)} and Sv =

{t ∈ [0, T ] : vs (t) = u∗ (t)}

s<s0

(6)

High-Order Maximal Principles

319

In this case, one may deﬁne   uα(λ,s) (t) ws (t) = vβ(λ,s) (t)  u∗ (t)

if t ∈ Su if t ∈ Sv else

(7)

for suitably chosen reparameterizations α and β. Relying on continuity of the ﬂows, one may reasonably expect that this new family of control variations might be shown to generate (λξ + (1 − λ)η) as the desired tangent vector. This reasoning is basically sound – but the technical details, depending on the particular setting, can be formidable. For this scheme to be broadly applicable it is natural to work with families of control variations which are supported on very small sets, i.e. in above language that e.g. the measure of Su goes to zero as s0 $ 0. Compare e.g. [16, 17] for the early uses of such needle variations. We formally deﬁne a family {us }s≥0 of the control u∗ : [0, T ] → U to be a family of needle variations if for each s0 > 0 there exist a ﬁnite number N of (s ) (s ) (s ) closed intervals Ii 0 = [ai 0 , bi 0 ] ⊆ [0, T ], i = 1, . . . N such that us (t) = u∗ (t) if s ≤ s0 , t ∈

N i=1

(s0 )

Ii

and

N i=1

(s0 )

bi

(s0 )

− ai

< Cs

(8)

and for some constant C > 0. A further special class of needle variations are those whose support is concentrated at a single point. We deﬁne a family of control variations {us }s ≥ 0 of u∗ , deﬁned on [0, T ] to be a (needle) variation at t0 ∈ [0, T ] if in addition us (t) = u∗ (t) for all t ∈ [0, T ]\[t0 −s, t0 +s]. Note that, with this deﬁnition, the variations employed in [12] for systems with stationary reference trajectory are needle variations at t0 = 0. The small, even asymptotically vanishing, support of families of needle variations much facilitates combining them in order to create new families of needle variations that generate convex combinations of tangent vectors. In order to generate convex combinations of tangent vectors generated by diﬀerent families of needle variations that have disjoint support (for suﬃciently small s0 > 0), and may proceed in the manner outlined above. The only difﬁcult case is when the supports of the diﬀerent families of needle variations, that are to be combined, have nontrivial intersection even for arbitrarily small s0 . A natural strategy is to shift one family of control variations by a small amount in time, which may even go to zero as s goes to zero, and then com(1) pose the families in the aforedescribed manner. For example, if {us }s≥0 and (2) (1) {us }s≥0 are two families of control variations of u∗ such that us (t) = u∗ (t) (2) for all t ∈ [0, T ] \ [t0 , a(s)] and us (t) = u∗ (t) for all t ∈ [0, T ] \ [t0 , b(s)] for some functions a(s), b(s) $ 0, one may deﬁne

320

M. Kawski

 (1)  u (t) if t ∈ [t0 , t0 + a(α(λ, s))]    α(λ,s) (2) λ u (t − a(α(λ, s))) if t ∈ (t0 + a(α(λ, s)), β(λ,s) us (t) =   t0 + a(α(λ, s)) + b(β(λ, s))]   ∗ else u (t)

(9)

where α and β are suitable reparameterizations of the original curves. In general, one expects that a time-shifted family may generate a diﬀerent curve of endpoints in R(T ), but if e.g. α(λ, s) −→ 0 suﬃciently fast as s −→ 0, it is conceivable that it will generate the same tangent vector. In addition, one also has to be very careful as the two (or more) families of control variations may in general have more complicated interactions. However, for a long time the general thinking has been that if enough care is taken carefully spelling out very intricate technical conditions on the admissible families of control variations, then all such eﬀects due to small, but vanishing time shifts and nonlinear interactions can be guaranteed to be of even higher order, so that the new combined family of control variations will indeed generate the desired convex combination. In this spirit, much of the classical literature, e.g. [3, 5, 7, 8, 9, 15, 13, 21, 22] make speciﬁc requirements on the class of admissible control variations – a common one being that each variation must be moveable by a small amount, i.e. any such translation only causing higher order perturbations. Progress made over several decades seemed to that eventually it might be possible to eventually drop all such technical hypotheses as they appeared to be automatically satisﬁed for all known systems. Their explicit statement seemed only required to make speciﬁc proofs work. However, as we shall discuss in the next section, any such hope has to be abandoned as without such conditions one may indeed loose the desired convexity. On the side we may point out, that the typical open mapping theorems adapted to the speciﬁc cones of higher order tangent vectors rely on topological arguments, which require in addition to the above that the convex combination (λ1 ξ1 + . . . + λn+r ξn+r ) of tangent vectors ξ1 , . . . ξn+r ∈ Kk , with (λ1 + . . . + λn+r ) = 1 can be generated by continuously parameterized combinations of the respective families of control variations. This requirement precludes for example the intuitive strategy of always choosing the ordering of the shifted control variations in such a way as to minimize the total sum of all shifts, e.g. not moving the shortest variations at all, etc.

4 Cones of Needle Variations Which Lack Convexity In this section we shall discuss the construction of a simple, yet very carefully crafted counterexample showing that cones of tangent vectors generated by needle variations without additional technical requirements for being moveable may indeed lack convexity. The technical details and full calculations may be found in [6] – here we concentrate on pointing out how this example is constructed, defying prior expectations that were inspired by generalized notions

High-Order Maximal Principles

321

of continuously diﬀerentiable dependence on initial conditions: small perturbations of the control variations should only result in small, higher order eﬀects in the curve of endpoints, and thus not aﬀect the resulting tangent vectors. Following the technical presentation in [6], we start with a simple polynomial system with a stationary reference trajectory x∗ ≡ 0 corresponding to the reference control u∗ ≡ (0, 0). This system with output is easier to analyze, and serves as a preparation for a system with nonstationary reference trajectory that is the desired counter example. x˙ 1 x˙ 2 x˙ 3 x˙ 4 x˙ 5 x˙ 6

= u1 = u2 = x21 = x22 = x4 x21 − x71 = x3 x22 − x72

|u1 (·)| ≤ 1 |u2 (·)| ≤ 1 x(0) = 0 ϕ(x) = (x1 , x2 , x5 , x6 )

(10)

The presence of the positive deﬁnite terms x21 and x22 in the drift vector ﬁeld f obviously causes a lack of controllability. However, if one considers the image of the reachable set under the output map ϕ, eﬀectively projecting out the two uncontrollable directions x3 and x4 , a very intricate picture emerges. We write y = (y1 , y2 , y3 , y4 ) = ϕ(x) The ﬁrst two components y1 and y2 are easily linearly controllable, and we may concentrate on e.g. the slice of the outputreachable set deﬁned by ϕ(R(T )) ∩ {y ∈ R4 : y1 = y2 = 0}. It is easy to see that one can reach all points in the ﬁrst quadrant y3 ≥ 0, y4 ≥ 0 (that are suﬃciently close to the origin) in small time. The standard tool for such arguments are families of dilations which provide a ﬁltered structure on the set of polynomials that corresponds to diﬀerent scalings of the control, and the corresponding iterated integral functionals. For a control u deﬁned on [0, T ] and δ, ε1 , ε2 ∈ [0, 1] consider the three-parameter family of controls uδ,ε1 ,ε2 , also deﬁned on [0, T ] by uδ,ε1 ,ε2 (t) = u∗ (t) = (0, 0) for t ∈ [0, T − δT ] and uδ,ε1 ,ε2 (T − δt) = (ε1 u1 (T − t), ε2 u2 (T − t))

(11)

With this one ﬁnds, for example, that the terms on the right hand side of the ﬁfth component of (10) scale like x4 (T − δt, uδ,ε1 ,ε2 ) · x21 (T − δt, uδ,ε1 ,ε2 ) = δ 5 ε21 ε22 · x21 (T − t, u) · x4 (T − t, u) x71 (T − δt, uδ,ε1 ,ε2 ) = δ 7 ε71 · x71 (T − t, u)

(12)

Thus after one further integration one sees that when letting δ = s $ 0 and either holding ε1 = ε ﬁxed, or letting it go to zero with s, the ﬁrst, positive deﬁnite term, will necessarily dominate the second one – unless u2 ≡ 0. On the other hand, by holding either u2 ≡ 0 or u1 ≡ 0 ﬁxed and, it is possible to reach points of the forms y(T ) = (0, 0, −cs8 , 0) and y(T ) = (0, 0, 0, −cs8 ), respectively, (for some constant c > 0, using controls supported on intervals

322

M. Kawski

of length at most s). From here it is fairly simple to see that one can reach all points of the form (y(T ) = (0, 0, c1 s8 , c2 s8 ) with at least one of c1 ≥ 0 or c2 ≥ 0 (both suﬃciently small) using controls supported on an interval of 8 length at most s. Next one may show that a vector ξ = (0, 0, ξ3 , ξ4 ) ∈ Kϕ lies in the 8-th order approximating cone if and only if ξ3 ≥ 0 or ξ4 ≥ 0. This means that this cone of tangent vectors generated by needle variations at t = T is a union of (diﬀerent, but not complementary) half-spaces, and thus is non-convex. Using concatenations of the simple needle variations at t = T that generate 8 one might the 8-th order tangent vectors (0, 0, −1, 0) and (0, 0, 0, −1) in Kϕ try to generate tangent vectors (0, 0, ξ3 , ξ4 ) to ϕ(R(T ) with both ξ3 < 0 and ξ4 < 0. However, it can be shown [6] that any such eﬀort using linear rescaling of the parameterization and shifting the support of either family by the required amount to make them nonoverlapping, fails. To be precise, for k if and only if all 8 ≤ k < 38/3 it can be shown that ξ = (0, 0, ξ3 , ξ4 ) ∈ Kϕ ξ3 ≥ 0 or ξ4 ≥ 0. Nonetheless, it is still possible to reach points in quadrant by suitably combining nonlinear reparameterizations of the original families of control (3) (4) variations. More speciﬁcally if {us }s≥0 and {us }s≥0 are the original families of control variations, supported each on [T − s, T ], then the key is to consider combinations of the form  0 if a(s) + b(s) < t  (34) (3) us (T − t) = u (T − (t − b(s))) if b(s) < t ≤ a(s) + b(s) (13)  (4) u (T − t) if 0 ≤ t ≤ b(s) and, this is critical, a(s) = o(s5/3 ) < s. However, such nonlinear rescaling changes the order of the tangent vector generated. Here, using control variations that generate (0, 0, −1, 0) and (0, 0, 0, −1) as 8-th order tangent vectors will generate e.g. (0, 0, −1, −1) only as a tangent vector of order k = (40/3). It 40/3

has been shown that indeed Kϕ = R4 and hence this system is small-time locally output controllable (STLOC). y4

6

6

Kϕ

-

y3

y4

6

k

Kϕ

-

y3

y4

6

40

Kϕ3

-

y3

8 ≤ k < 38/3 Fig. 3. Cross-sections of approximating cones of tangent vectors for system (10)

High-Order Maximal Principles

323

This necessary nonlinear rescaling has some profound consequences. In particular, it shows that STLOC is not structurally stable in the desired sense, i.e. such that higher order perturbations w.r.t. a graded structure shall not destroy controllability of a nominal system. In this speciﬁc case, one can show that the perturbed system z˙1 z˙2 z˙3 z˙4 z˙5 z˙6

= u1 = u2 = z12 = z22 = z4 z12 − z17 + z110 + z210 = z3 z22 − z27 + z110 + z210

|u1 (·)| ≤ 1 |u2 (·)| ≤ 1 z(0) = 0 ϕ(z) = (z1 , z2 , z5 , z6 )

(14)

k for is no longer STLOC. Indeed for any k ≥ 8, a vector (0, 0, ξ3 , ξ4 ) lies in Kϕ system (14) if ξ3 ≥ 0 or ξ4 ≥ 0 (for all suﬃciently small times T > 0). In the standard language of research on small-time local controllability, one might say that the terms z4 z12 and z3 z22 are the lowest order ones that provide accessibility, but are potential obstructions to controllability, while the higher order terms z17 and z27 are able to neutralize these potential obstructions, providing controllability of (10). Nonetheless the terms z110 + z210 which by any method of counting are of even higher order, again destroy controllability. This kind of behavior is completely opposite of the key ideas underlying nilpotent approximations (constructed from suitable ﬁltrations of the Lie algebra generated by the system vector ﬁelds and associated families of dilations) that has dominated much of the work on controllability and optimality in the 1980s, see [10] for an authoritative survey. Indeed, one may raise the question whether it might be possible to construct systems that exhibit inﬁnite such chains of higher order terms that alternate to provide and destroy controllability. This is closely related to the open question whether controllability is ﬁnitely determined [1].

One might argue that it is not surprising at all that one can construct such counterexamples, exhibiting nonconvex cones of tangent vectors generated by needle variations, by means of nontrivial output maps that project out those components that otherwise would dominate any such pathological behavior. After all, from [12] it is known that analytic systems with a stationary reference trajectory cannot have such undesirable behavior. Thus we continue by modifying the above system so that it has a nonstationary reference trajectory, yet still is such that the cone of tangent vectors to the reachable set, that is generated by needle variations as deﬁned above, is not convex. Speciﬁcally consider the following system which has two additional control, and an additional drift term that lies in the span of the two additional controlled vector ﬁelds. For emphasis we also performed a simple linear coordinate change in the last two components with c = 0 any nonzero constant.

324

w˙ 1 w˙ 2 w˙ 3 w˙ 4 w˙ 5 w˙ 6

M. Kawski

= u1 = u2 = w12 + (1 + u3 ) = w22 + (1 + u4 ) = c−1 · (w4 w12 − w3 w22 − w17 + w27 ) = w4 w12 + w3 w22 − w17 − w27 + w110 + w210

|u1 (·)| ≤ 1 |u2 (·)| ≤ 1 |u3 (·)| ≤ 1 (15) |u4 (·)| ≤ 1 w(0) = 0 w∗ (t) = ( 0, 0, t, t, 0, 0)

The critical part in this construction is to align the directions x3 and x4 that were projected out in system (10) with the new control vector ﬁelds g3 and g4 and an additional drift term. A key feature is that the velocity w˙ = 0 lies on the boundary of the set of all possible velocities – but only at the initial point w∗ (0) = 0 of the reference trajectory. Consequently, this system inherits some of the nice properties of systems with stationary reference trajectory – but once the system sets along (or near) the reference trajectory it looses much of this controllability.

w6

BBBB 6 BB BB BB BB BB

w6 = |c · w5 | k

K, k ≥ 8

-w5

Fig. 4. Cross-sections of approximating cones of tangent vectors for system (15)

As a result, it is possible to generate each of (0, 0, 0, 0, ±1, |c|) ∈ K8 as an 8 − th-order tangent vectors to the reachable set R(T ) at w∗ (T ) using needle variations of the reference control u∗ ≡ (0, 0, 0, 0) at time t0 = 0, but not by using needle variations at any time t0 > 0. Following similar technical arguments as in system (10), it then can be shown that the intersection of the reachable set R(T ) with the hyperplane {w ∈ R6 : w1 = w2 = 0, w3 = w4 = T } is contained in the union of the two half-spaces deﬁned by w6 ≤ |c|w5 , giving the complete picture.

5 Conclusion In summary, this counter example demonstrates that the highly technical conditions on needle variations (e.g. that they be “moveable” by not-toosmall amounts) are indeed necessary. In other words, we should not expect

High-Order Maximal Principles

325

any future version of a High-order Maximal Principle to be able to eliminate the kind of technical assumptions that we ﬁnd in e.g. Krener’s pioneering work [15]. The larger the class of admissible control variations is in any theory, the larger the cone of tangent vectors will be. And larger cones provide stronger necessary conditions. However, this work shows that the natural candidate of the largest possible cone generated by needle variations (which works so well for stationary reference trajectories) is too large: Due to its lack of convexity it looses its usefulness for any argumentation along the ideas of the PMP and HMP (i.e. based on separating hyperplanes). On the other hand, any theory that places suﬃcient restrictions on the admissible control variations for the cones of tangent vectors to be convex will necessarily unable to certify the optimality or lack of optimality of some controls for even the most benign nonlinear systems, cascades of polynomials. On a diﬀerent level, the perturbations introduced in system (14) raise a more worrisome prospect, namely that controllability (here: small-time local controllability, “STLC”) may be considerably less structurally stable than expected. Much work in the 1980s and early 1990s by many authors was based on an implicit assumption that nilpotent approximating systems constructed from a ﬁltration of the Lie algebra L(f, g1 , . . . gm ) would be able to capture the key controllability properties e.g. [4, 10, 18, 19] The system presented in [11] remained for a long time a very isolated counter example that showed that the theory was still incomplete. The work discussed here casts a much darker shadow on this approach as apparently much higher order perturbations may again destroy the local controllability of the usual nilpotent approximating systems. The question whether controllability, and thus also optimality, of even the most benign analytic aﬃne control systems is ﬁnitely determined, remains open [1]. Another question that remains open is whether reachable of nice, say analytic, systems may have inward cusps (rather than just inward corners). In such cases, the reference trajectory might lie on the boundary of the reachable sets at all times, yet the approximating cones, possibly even by needle variations, are the entire (tangent) spaces.

References 1. Agrachev A (1999) Is it possible to recognize local controllability in a ﬁnite number of diﬀerentiations? In: Blondel V, Sontag E, Vidyasagar M, and Willems J (eds) Open Problems in Mathematical Systems and Control Theory. Springer, Berlin Heidelberg New York 2. Agrachev A, Gauthier J.-P. (2001) Annales de l’Institut Henri Poincar´e; Analyse non-lin´eaire 18:359–382 3. Bianchini R, Stefani G (1984) Int J Cntrl 39:701–714 4. Bianchini R, Stefani G (1990) SIAM J Control Optim 28:903–924

326 5. 6. 7. 8. 9. 10. 11. 12.

13.

14. 15. 16. 17. 18. 19. 20. 21. 22.

M. Kawski Bianchini R (1999) Proc Symposia Pure Math 64:91–101, Birkh¨ auser Bianchini R, Kawski M (2003) SIAM J Control Optim (to appear) Bressan A (1985) SIAM J Control Optim 23:38–48 Frankowska H (1987), J Math Anal Appl 127:172–180 Frankowska H (1989), J Optim Theory Appl 60:277–296 Hermes H (1991), SIAM Review 33:238–264 Kawski M (1988) Bull AMS 18:149–152 Kawski M (1988) An angular open mapping theorem, in: Analysis and Optimization of Systems, Bensoussan A, Lions J L, eds., Lect. Notes in Control and Information Sciences 111:361–371 Springer, Berlin Heidelberg New York Knobloch H (1981), Higher Order Necessary Conditions in Optimal Control Theory, Lect. Notes in Control and Information Sciences 34 Springer, Berlin Heidelberg New York Krener A (1973) The high order maximal principle. In: Geometric Methods in Systems Theory, Mayne D and Brockett R (eds) Reidel, Dordrecht (Holland) Krener A (1977) SIAM J Control Optim 15:256–293 Lee E and Markus L(1967) Foundations of optimal control theory, Wiley, New York Pontryagin L, Boltyanskii V, Gamkrelidze R, Mischenko E (1962) The mathematical theory of optimal processes, Wiley, New York Sussmann H (1983), SIAM J Control Optim 21:686–713 Sussmann H (1987) SIAM J Cntrl Optim 25:158–194 Sussmman H and Willems J (1997) IEEE Control Systems Magazine 17:32–44. Sussmann H (2002) Proc IEEE CDC (to appear) Sussmann H (2003) Set-valued Anal (to appear)

Legendre Pseudospectral Approximations of Optimal Control Problems I. Michael Ross1 and Fariba Fahroo2 1 2

Department of Aeronautics and Astronautics, Code AA/Ro, Naval Postgraduate School, Monterey, CA 93943, [email protected] Department of Applied Mathematics, Code MA/Ff, Naval Postgraduate School, Monterey, CA 93943, [email protected]

Summary. We consider nonlinear optimal control problems with mixed statecontrol constraints. A discretization of the Bolza problem by a Legendre pseudospectral method is considered. It is shown that the operations of discretization and dualization are not commutative. A set of Closure Conditions are introduced to commute these operations. An immediate consequence of this is a Covector Mapping Theorem (CMT) that provides an order-preserving transformation of the Lagrange multipliers associated with the discretized problem to the discrete covectors associated with the optimal control problem. A natural consequence of the CMT is that for pure state-constrained problems, the dual variables can be easily related to the D-form of the Lagrangian of the Hamiltonian. We demonstrate the practical advantage of our results by numerically solving a state-constrained optimal control problem without deriving the necessary conditions. The costates obtained by an application of our CMT show excellent agreement with the exact analytical solution.

1 Introduction Many problems in control theory can be formulated as optimal control problems [5]. From a control engineer’s perspective, it is highly desirable to obtain feedback solutions to complex nonlinear optimal control problems. Although the Hamilton-Jacobi-Bellman (HJB) equations provide a framework for this task, they suﬀer from well-known fundamental problems [1, 3, 5], such as the nonsmoothness of the value function and the “curse of dimensionality”. The alternative framework of the Minimum Principle, while more tractable from a control-theoretic point of view, generates open-loop controls if it can be solved at all. The Minimum-Principle approach is also beset with fundamental numerical problems due to the fact that the costates are adjoint to the state perturbation equations [3]. In other words, the Hamiltonian generates W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 327–342, 2003.

c Springer-Verlag Berlin Heidelberg 2003

328

I.M. Ross and F. Fahroo

a numerically sensitive boundary value problem that may produce such wild trajectories as to exceed the numerical range of the computer [3]. To overcome this diﬃculty, direct methods have been employed to solve complex optimal control problems arising in engineering applications [2]. While the theoretical properties of Eulerian methods are widely studied [5, 12], they are not practical due to their linear (O(h)) convergence rate. On the other hand, collocation methods are practical and widely used [2], but not much can be said about the optimality of the result since these methods do not tie the resulting solutions to either the Minimum Principle or HJB theory. In fact, the popular Hermite-Simpson collocation method and even some Runge-Kutta methods do not converge to the solution of the optimal control problem [10]. This is because an N th -order integration scheme for the diﬀerential equations does not necessarily lead to an N th -order approximation scheme for the dual variables. That is, discretization and dualization do not necessarily commute [14]. By imposing additional conditions on the coeﬃcients of Runge-Kutta schemes, Hager[10] was able to transform the adjoint system of the discretized problem to prove the preservation of the order of approximation. Despite this breakthrough, the controls in such methods converge more slowly than the states or the adjoints. This is because, the controls are implicitly approximated to a lower order of accuracy (typically piecewise linear functions) in the discrete time interval. In this paper, we consider the pseudospectral (PS) discretization of constrained nonlinear optimal control problems with a Bolza cost functional[6, 8, 9]. PS methods diﬀer from many of the traditional discretization methods in the sense that the focus of the approximation is on the tangent bundle than on the diﬀerential equation[15]. In this sense, they most closely resemble ﬁnite element methods but oﬀer a far more impressive convergence rate known as spectral accuracy[17]. For example, for smooth problems, spectral accuracy implies an exponential convergence rate. We show that the discretization of the constrained Bolza problem by an N th -order Legendre PS method does not lead to an N th -order approximation scheme for the dual variables as previously presumed[7, 9]. However, unlike Hager’s Runge-Kutta methods, no conditions on the coeﬃcients of the Legendre polynomials can be imposed to overcome this barrier. Fortunately, a set of simple “closure conditions,” that we introduce in this paper, can be imposed on the discrete primal-dual variables so that a linear diagonal transformation of the constrained Lagrange multipliers of the discrete problem provides a consistent approximation to the discrete covectors of the Bolza problem. This is the Covector Mapping Theorem (CMT). For pure state-constrained control problems, the CMT naturally provides a discrete approximation to the costates associated with the so-called D-form of the Lagrangian of the Hamiltonian[11]. This implies that the order of the state-constraint is not a limiting factor and that the interior point constraint at the junction of the state constraint is not explicitly imposed. More importantly, the jump conditions are automatically approximated as a

Legendre Pseudospectral Approximations of Optimal Control Problems

329

consequence of the CMT. These sets of results oﬀer an enormously practical advantage over other methods and are demonstrated by a numerical example.

2 Problem Formulation We consider the following formulation of an autonomous, mixed state-control constrained Bolza optimal control problem with possibly free initial and terminal times: Problem B Determine the state-control function pair, [τ0 , τf ] τ → {x ∈ RNx , u ∈ RNu } and possibly the “clock times,” τ0 and τf , that minimize the Bolza cost functional, 6 τf J[x(·), u(·), τ0 , τf ] = E(x(τ0 ), x(τf ), τ0 , τf ) + F (x(τ ), u(τ )) dτ (1) τ0

subject to the state dynamics, ˙ ) = f (x(τ ), u(τ )) x(τ

(2)

e x(τ0 ), x(τf ), τ0 , τf = 0

(3)

end-point conditions,

and mixed state-control path constraints, h(x(τ ), u(τ )) ≤ 0

(4)

Assumptions and Notation For the purpose of brevity, we will make some assumptions that are often not necessary in a more abstract setting. It is assumed the functions E : RNx ×RNx ×R×R → R, F : RNx ×RNu → R, f : RNx ×RNu → RNx , e : RNx × RNx ×R×R → RNe , h : RNx ×RNu → RNh are continuously diﬀerentiable with respect to their arguments. It is assumed that a feasible solution, and hence an optimal solution exists in an appropriate Sobolev space, the details of which are ignored. In order to apply the ﬁrst-order optimality conditions, additional assumptions on the constraint set are necessary. Throughout the rest of the paper, such constraint qualiﬁcations are implicitly assumed. The Lagrange multipliers discussed in the rest of this paper are all assumed to be nontrivial and regular. The symbol N(·) with a deﬁning subscript is an element of the h Natural numbers N. Nonnegative orthants are denoted by RN + . The shorthand h[τ ] denotes h(x(τ ), u(τ )). By a somewhat minor abuse of notation, we let hk

330

I.M. Ross and F. Fahroo

denote hN [τk ] = h(xN (τk ), uN (τk )) where the superscript N denotes the N th degree approximation of the relevant variables. The same notation holds for all other variables. Covectors are denoted by column vectors than row vectors to conform with the notion of a gradient as a column vector. Under suitable constraint qualiﬁcations[11], the Minimum Principle isolates possible optimal solutions to Problem B by a search for vector-covector pairs in the primal-dual space. Denoting this as Problem Bλ , it is deﬁned as: Problem Bλ Determine the state-control-covector function 4-tuple, [τ0 , τf ] τ → {x ∈ Ne h , and the clock times RNx , u ∈ RNu , λ ∈ RNx , µ ∈ RN + }, a covector ν ∈ R τ0 and τf that satisfy Eqs.(2)-(4) in addition to the following conditions: ˙ ) = − ∂L[τ ] λ(τ ∂x ∂L =0 ∂u ∂Ee ∂Ee , {λ(τ0 ), λ(τf )} = − ∂x(τ0 ) ∂x(τf ) ∂Ee ∂Ee {H[τ0 ], H[τf ]} = ,− ∂τ0 ∂τf

(5) (6) (7) (8)

where L is the D-form of the Lagrangian of the Hamiltonian deﬁned as[11], L(x, u, λ, µ) = H(x, u, λ) + µT h(x, u)

(9)

where H is the (unminimized) Hamiltonian, H(x, u, λ) = λT f (x, u) + F (x, u)

(10)

h and µ ∈ RN + satisﬁes the complementarity condition,

µT (τ )h[τ ] = 0

∀τ ∈ [τ0 , τf ]

(11)

In the above equations, Ee is deﬁned as Ee (x(τ0 ), x(τf ), τ0 , τf , ν) = E(x(τf ), x(τ0 ), τ0 , τf ) + ν T e(x(τ0 ), x(τf ), τ0 , τf ) (12) If the path constraint, Eq.(4), is independent of the control (i.e. a pure state constraint), then the costate, λ(τ ), must satisfy the jump condition[11], λ− (τe ) = λ+ (τe ) +

∂h ∂x(τe )

T η

(13)

Legendre Pseudospectral Approximations of Optimal Control Problems

331

where η ∈ RNh is a (constant) covector which eﬀectively arises as a result of the implied interior point constraint (with a pure state constraint), h(x(τe )) = 0

(14)

where τe denotes the entry or exit point of the trajectory. The important point to note about the jump condition, Eq.(13), is that it is derived by explicitly imposing the constraint, Eq.(14). This is important from a controltheoretic point of view but as will be apparent from the results to follow in the Legendre pseudospectral method, it is not necessary to explicitly impose this constraint. In fact, the method automatically determines an approximation to the covector jump as part of the solution.

3 The Legendre Pseudospectral Method The Legendre pseudospectral method is based on interpolating functions on Legendre-Gauss-Lobatto (LGL) quadrature nodes[4]. These points which are distributed over the interval [−1, 1] are given by t0 = −1, tN = 1, and for 1 ≤ l ≤ N −1, tl are the zeros of L˙ N , the derivative of the Legendre polynomial of degree N, LN . Using the aﬃne transformation, τ (t) =

(τf − τ0 )t + (τf + τ0 ) 2

(15)

that shifts the LGL nodes from the computational domain t ∈ [−1, 1] to the physical domain τ ∈ [τ0 , τf ], the state and control functions are approximated by N th degree polynomials of the form x(τ (t)) ≈ xN (τ (t)) =

N

xl φl (t)

(16)

ul φl (t)

(17)

l=0

u(τ (t)) ≈ uN (τ (t)) =

N l=0

where, for l = 0, 1, . . . , N φl (t) =

(t2 − 1)L˙ N (t) 1 N (N + 1)LN (tl ) t − tl

are the Lagrange interpolating polynomials of order N . It can be veriﬁed that, 1 if l = k φl (tk ) = δlk = 0 if l = k Hence, it follows that xl = xN (τl ), ul = uN (τl ) where τl = τ (tl ) so that τN ≡ τf . Next, diﬀerentiating Eq. (16) and evaluating it at the node points, tk , results in

332

I.M. Ross and F. Fahroo

x˙ N (τk ) =

N dxN 22 dxN dt 22 2 2 = Dkl xl ≡ dk 2 2 = dτ τ =τk dt dτ tk τf − τ0 τf − τ0

(18)

l=0

where Dkl = φ˙ l (tk ) are entries of the (N + 1) × (N + 1) diﬀerentiation matrix D [4]  L (t ) 1 N k   LN (tl ) . tk −tl k = l       N (N +1)  − 4 k=l=0 D := [Dkl ] := (19)   N (N +1)  k=l=N   4      0 otherwise This facilitates the approximation of the state dynamics to the following algebraic equations N

τf − τ 0 f (xk , uk ) − Dkl xl = 0 2

k = 0, . . . , N

l=0

Approximating the Bolza cost function, Eq.(1), by the Gauss-Lobatto integration rule, we get, J[X N , U N , τ0 , τf ] = E(x0 , xN , τ0 , τf ) +

N τf − τ0 F (xk , uk )wk 2 k=0

where

X N = [x0 ; x1 ; . . . ; xN ],

U N = [u0 ; u1 ; . . . ; uN ]

and wk are the LGL weights given by wk :=

1 2 , N (N + 1) [LN (tk )]2

k = 0, 1, . . . , N

Thus, Problem B is discretized by the following nonlinear programming (NLP) problem: Problem BN Find the (N +1)(Nx +Nu )+2 vector X N P = (X N ; U N ; τ0 , τf ) that minimizes J(X N P ) ≡ J N = E(x0 , xN , τ0 , τf ) +

N τf − τ0 F (xk , uk )wk 2 k=0

subject to

(20)

Legendre Pseudospectral Approximations of Optimal Control Problems

333

N

τf − τ0 f (xk , uk ) − Dkl xl = 0 2

(21)

l=0

e(x0 , xN , τ0 , τf ) = 0 h(xk , uk ) ≤ 0

(22) (23)

for k = 0, . . . , N. Problem B λ can also be discretized in much the same manner. Approximating the costate by the N th degree polynomial, N

λ(τ (t)) ≈ λ (τ (t)) =

N

λl φl (t)

(24)

l=0

and letting ΛN P = [λ0 ; λ1 ; . . . ; λN ; µ0 ; µ1 ; . . . ; µN ; ν 0 ; ν f ], we can discretize Problem B λ as, Problem BλN Find X N P and ΛN P that satisfy Eqs.(21)-(23) in addition to the following nonlinear algebraic relations: N

Dkl λl = −

l=0

∂Lk ∂xk

(25)

∂Lk =0 ∂uk ∂Ee ∂Ee , {λ0 , λN } = − ∂x0 ∂xN ∂Ee ∂Ee {H0 , HN } = ,− ∂τ0 ∂τN µTk hk = 0,

(26) (27) (28)

µk ≥ 0

(29)

for k = 0, . . . , N. Remark 1. In the case of pure state constraints, it is necessary to determine a priori a switching structure and impose the jump conditions for optimality. Assuming a suﬃciently large N , the jump condition can be approximated as, λ(te ) = λ(te+1 ) +

∂h(xe ) ∂xe

T η

(30)

for all points te that are the junction points of the switching structure. This is the indirect Legendre pseudospectral method[8] and represents a discretization of the multi-point boundary value problem. It is obvious that the direct method (Problem B N ) is far simpler to implement than the indirect method.

334

I.M. Ross and F. Fahroo

This is true of any direct/indirect method[2]. However, unlike the indirect method, not much can be said about the optimality or the convergence of the direct method. The theorem of the next section shows how to get the high performance of the indirect method without actually implementing it by way of the signiﬁcantly simpler implementation of the direct method. 3.1 KKT Conditions for Problem BN The Lagrangian for Problem BN , can be written as N - µ) - = J N (X N P ) + ν-T e(x0 , xN , τ0 , τf )+ J (X N P , ν-, λ, N T τf − τ0 T N - i hi (X N P ) )fi (X N P ) − di (X )} + µ λi {( 2 i=0

(31)

-i, µ - i are the KKT multipliers associated with the NLP. Using where ν-, λ Lemma 1 below, the KKT conditions may be written quite succinctly in a certain form described later in this section. Lemma 1. The elements of the Diﬀerentiation Matrix, Dik , and the LGL weights, wi , together satisfy the following properties, wi Dik + wk Dki = 0

i, k = 1, . . . , N − 1

(32)

For N the boundary terms, we have 2w0 D00 = −1, and 2wN DN N = 1. Further, i=0 wi = 2. For a proof of this, please see [9]. -k µ -k Lemma 2. The LGL-weight-normalized multipliers λ wk , wk satisfy the same equations as the discrete costates (Cf. Eq.(25)) at the interior nodes, k = 1, . . . N − 1; i.e., we have N -k µ ∂L λ (xk , uk , , k)+ Dki ∂xk wk wk i=0

0

-i λ wi

1 =0

(33)

Proof: Consider the interior state variables (x1 , . . . xN −1 ). From applying the N KKT condition at the interior nodes to Eq.(31), i.e. ∂J ∂ xk = 0, we have N T τf − τ0 ∂ ∂J N - Ti hi = − fi − di + µ λ i ∂xk i=0 2 ∂xk Since the functions f , h, F are evaluated only at the points ti , we have

(34)

Legendre Pseudospectral Approximations of Optimal Control Problems

335

T N τf − τ0 ∂fk ∂ - T τf − τ0 τf − τ0 T -k + fi + µi hi + Fi wi = λ λ ∂xk i=0 i 2 2 2 ∂xk T τf − τ0 ∂Fk ∂hk -k wk + (35) µ 2 ∂xk ∂xk For the term involving the state derivatives, a more complicated expression is obtained since the diﬀerentiation matrix D relates the diﬀerent components of xk : N N T ∂ -i Dik λ λi di = ∂xk i=0 i=0

(36)

wk Dki , therefore by putting together Eqs. (35)-(36), wi the following is obtained for k = 1, . . . , N − 1 : 0 1 T T N -i λ ∂hk τf − τ0 ∂fk τf − τ0 ∂Fk -k = 0 + wk + Dki µ λk + wk ∂xk 2 ∂xk wi ∂xk 2 i=0 From Lemma 1, Dik = −

(37) Dividing Eq. (37) by wk yields the desired result for k = 1, . . . , N − 1. -k µ -k Lemma 3. The LGL-weight-normalized multipliers λ wk , wk satisfy the discrete ﬁrst-order optimality condition associated with the minimization of the Hamiltonian at all node points: -k µ λ ∂L (xk , uk , , k) = 0 ∂uk wk wk

(38)

Proof: Considering the terms that involve diﬀerentiation with respect to the control variables uk in Eq. (31) yields τf − τ0 2

∂fk ∂uk

T

-k + λ

∂hk ∂uk

T

-k = − µ

∂J N ∂uk

k = 0, . . . , N.

(39)

Since ∂J N = ∂uk

τf − τ0 2

∂Fk wk ∂uk

dividing Eq.(39) by wk , yields the desired result.

(40)

Lemma 4. At the ﬁnal node, the KKT multipliers satisfy the following equation:

336

I.M. Ross and F. Fahroo

0 wN

N -i -N µ -N ∂L λ λ (xN , uN , , )+ DN i ∂xN wN wN wi i=0

1 ≡ cN

(41)

-N ˜e ∂E λ − ≡ cN wN ∂xN

(42)

where E˜e = Ee (x0 , xN , τ0 , τN , ν-) Proof: The following KKT condition holds for the last node:

∂e ∂xN

T

ν- +

τ f − τ0 2

∂fN ∂xN

T

-N − λ

N

-i + DiN λ

i=0

∂hN ∂xN

T

-N = − µ

∂J N ∂xN (43)

Using the relationship DiN = −

wN DN i , wi

i = N

and

2DN N =

1 wN

-N -N = λ and adding 2DN N λ wN to both sides of Eqn (43) and rearranging the terms, the following is obtained: τf − τ 0 2

∂FN ∂xN

τf − τ0 wN + 2

∂fN ∂xN

T

- i ∂hN T λ -N = DN i + µ wi ∂xN i=0

N

- N +wN λ

- N − ∂E − 2DN N λ ∂xN

∂e ∂xN

T

ν-

(44)

or 0 wN

N -N µ -i ∂L λ λ (xN , uN , , N)+ DN i ∂xN wN wN wi i=0

1 =

-N ˜e λ ∂E − ≡ cN . wN ∂xN

Corollary 1. The result for the zeroth node (i.e. initial time condition) can be shown in a similar fashion: 0 1 N -0 -0 µ -i ˜e -0 ∂L λ λ ∂E λ −w0 = (x0 , u0 , , )+ D0i + ∂x0 w0 w0 wi w0 ∂x0 i=0 ≡ c0 - i and ν- satisfy the condition, Lemma 5. The Lagrange multipliers λ

Legendre Pseudospectral Approximations of Optimal Control Problems

337

N -i ˜e ∂E 1 λ =− wi H xi , ui , 2 i=0 wi ∂τN

(45)

N -i ∂E ˜e 1 λ = wi H xi , ui , 2 i=0 wi ∂τ0

(46)

Proof: Applying the KKT condition for the variable, τN , we have, 0 1 N T N λ -T - fi ∂E 1 ∂e T λ Fi wi i i − = − wi Fi + fi ν- = + ∂τN ∂τN 2 2 2 i=0 wi i=0 and hence the ﬁrst part of the lemma. The second part of the lemma follows similarly by considering the variable τ0 . Collecting all these results, and letting -0; λ -1; . . . ; λ -N ; µ - N P = [λ - ;µ - ;... ;µ - ; ν-0 ; ν-f ] Λ 0

N

the dualization of Problem B

1

N

may be cast in terms of Problem BN λ :

Problem BN λ - N P that satisfy Eqs.(21)-(23) in addition to the following Find X N P and Λ nonlinear algebraic relations: -k µ ∂L λ (xk , uk , , k) = 0 ∂uk wk wk

-k ≥ 0 - Tk hk = 0, µ µ 0 1 N -i -k µ ∂L λ λ =0 (xk , uk , , k)+ Dki ∂xk wk wk wi i=0

k = 0, . . . , N

(47)

k = 0, . . . , N

(48)

k = 1, . . . , N − 1

(49)

and N -i -N µ ∂L λ λ cN (xN , uN , , N)+ DN i = ∂xN wN wN wi wN i=0

-N ˜e ∂E λ − = cN wN ∂xN N -i -0 µ c0 λ ∂L λ (x0 , u0 , =− , 0)+ D0i ∂x0 w0 w0 wi w0 i=0 -0 ˜e ∂E λ + = c0 w0 ∂x0 N -i ˜e ∂E 1 λ =− wi H xi , ui , 2 i=0 wi ∂τN N -i ∂E ˜e 1 λ = wi H xi , ui , wi ∂τ0 2 i=0

(50) (51) (52) (53) (54)

(55)

338

I.M. Ross and F. Fahroo

where c0 and cN are arbitrary vectors in RNx . The deliberate formulation of the KKT conditions for Problem BN in the above form facilitates a deﬁnition of Closure Conditions: Deﬁnition 1. Closure Conditions are deﬁned as the set of constraints that must be added to Problem BN λ so that every solution of this restricted problem is equivalent to the solution of Problem BλN From this deﬁnition, the Closure Conditions are obtained by simply matching the equations for Problems BN λ to those of Problem BλN . This results in, c0 = 0 cN = 0

(56) (57)

N -i 1 λ = H0 = HN wi H xi , ui , 2 i=0 wi

(58)

The Closure Conditions facilitate our main theorem: The Covector Mapping Theorem -i, µ - i that are equal to the Theorem 1. There exist Lagrange multipliers λ pseudospectral approximations of the covectors λN (τi ), µN (τi ) at the shifted LGL node τi multiplied by the corresponding LGL weight wi . Further, there exists a ν- that is equal to the constant covector ν. In other words, we can write, λN (τi ) =

-i λ , wi

µN (τi ) =

-i µ , wi

ν- = ν

(59)

3.2 Proof of the Theorem Since a solution, {xi , ui , λi , µi , ν}, to Problem BλN exists (by assumption), it follows that {xi , ui , wi λi , wi µi , ν} solves Problem BN λ while automati-i, µ -i, cally satisfying the Closure Conditions. Conversely, a solution, {xi , ui , λ N λ ν-}, of Problem B that satisﬁes the Closure Conditions provides a solution, -i µ -i λN {xi , ui , λ . wi , wi , ν }, to Problem B Remark 2. A solution of Problem BλN always provides a solution to Problem BN λ ; however, the converse is not true in the absence of the Closure Conditions. Thus, the Closure Conditions guarantee an order-preserving bijective map between the solutions of Problem BN λ and BλN . The commutative diagram depicted in Fig.1 captures the core ideas.

Legendre Pseudospectral Approximations of Optimal Control Problems

339

convergence

Problem B

O

Problem B discretization (indirect)

gap

Problem B

NO

dualization

dualization

ON

Covector Mapping Theorem

convergence

Problem B

Problem B

N

discretization (direct)

Fig. 1. Commutative Diagram for Discretization and Dualization

Remark 3. The Closure Conditions given by c0 = 0 = cN are a simple requirement of the fact that the PS transformed discrete adjoint equations be satisﬁed at the end points in addition to meeting the endpoint transversality conditions. On the other hand, the condition given by Eq.(58) states the constancy of the discrete Hamiltonian in a weak form (see Lemma 1). Remark 4. The Closure Conditions signify the closing of the gap between Problems BN λ and BλN which exist for any given degree of approximation, N . The issue of convergence of Problem B N to Problem B via Problem B λN is discussed in Ref.[13].

4 Numerical Example To illustrate the theory presented in the previous sections, the Breakwell problem[3] is considered: Minimize 6 1 1 2 J= u dt 2 0 subject to the equations of motion x(τ ˙ ) = v(τ ),

v(τ ˙ ) = u(τ )

the boundary conditions x(0) = 0,

x(1) = 0,

v(0) = 1.0,

and the state constraint x(τ ) ≤ = 0.1

v(1) = −1.0

340

I.M. Ross and F. Fahroo

Figures 2 and 3 demonstrate the excellent agreement between the analytical solution[3] and the solution obtained from our Legendre pseudospectral method. The solution was obtained for 50 LGL points with the aid of DIDO[16], a software package that implements our ideas. The cost function obtained is 4.4446 which agrees very well with the analytic optimal result of 4 J = 9 = 4.4444. It is apparent that the optimal switching structure is freeconstrained-free. The costates corresponding to the D-form of the Lagrangian are shown in Figure 4. Note that the method adequately captures the fact that λv should be continuous while λx should have jump discontinuities given by,1 2 + λ− j = 1, 2 τ1 = 3, τ2 = 1 − 3 x (τj ) − λx (τj ) = 92 Figure 4 exhibits a jump discontinuity of 22.2189 which compares very well with the analytical value of 22.2222. 0.1

1 position

0.05

0 velocity

0

0

0.1

0.2

0.3

0.4

0.5 Time

0.6

0.7

0.8

0.9

1

−1

Fig. 2. PS states, x and v. Solid line is analytical.

5 Conclusions A Legendre pseudospectral approximation of the constrained Bolza problem has revealed that there is a loss of information when a dualization is performed after discretization. This information loss can be restored by way of Closure Conditions introduced in this paper. These conditions also facilitate a spectrally accurate way of representing the covectors associated with the continuous problem by way of the Covector Mapping Theorem (CMT). All these results can be succinctly represented by a commutative diagram. The 1

Ignoring the typographical errors, the costates given in Ref.[3] correspond to the P -form[11] and exhibit a jump discontinuity in λv as well.

Legendre Pseudospectral Approximations of Optimal Control Problems

341

1

0

−1

Control, u

−2

−3

−4

−5

−6

−7

0

0.1

0.2

0.3

0.4

0.5 Time

0.6

0.7

0.8

0.9

1

Fig. 3. PS control, u. Solid line is analytical. 50

10

lambdax lambda

v

0

−50

0

0

0.1

0.2

0.3

0.4

0.5 Time

0.6

0.7

0.8

0.9

1

−10

Fig. 4. Costates, λx and λv from CMT. Solid line is analytical.

practical advantage of the CMT is that nonlinear optimal control problems can be solved eﬃciently and accurately without developing the necessary conditions. On the other hand, the optimality of the solution can be checked by using the numerical approximations of the covectors obtained from the CMT. Since these solutions can presently be obtained in a matter of seconds, it appears that the proposed technique can be used for optimal feedback control in the context of a nonlinear model predictive framework.

References 1. Bellman, R. E. (1957). Dynamic Programming. Princeton University Press, Princeton, NJ. 2. Betts, J. T. (1998). Survey of numerical methods for trajectory optimization. Journal of Guidance, Control, and Dynamics, Vol 21, No. 2, 193–207.

342

I.M. Ross and F. Fahroo

3. Bryson, A.E., Ho, Y.C., (1975). Applied Optimal Control. Hemisphere, New York. 4. Canuto, C., Hussaini, M. Y., Quarteroni, A., Zang, T.A. (1988). Spectral Methods in Fluid Dynamics. Springer Verlag, New York. 5. Clarke, F. H., Ledyaev, Yu. S., Stern, R. J., Wolenski, P. R. (1998). Nonsmooth Analysis and Control Theory, Springer-Verlag, New York. 6. Elnagar, J., Kazemi, M. A., Razzaghi, M. (1995). The pseudospectral Legendre method for discretizing optimal control problems. IEEE Transactions on Automatic Control, Vol. 40, No. 10, 1793–1796. 7. Elnagar, J., Razzaghi, M. (1997). A collocation-type method for linear quadratic optimal control problems. Optimal Control Applications and Methods, Vol. 18, 227–235. 8. Fahroo, F., Ross, I. M. (2000). Trajectory optimization by indirect spectral collocation methods. Proc. AIAA/AAS Astrodynamics Specialist Conference. Denver, CO, 123–129. 9. Fahroo, F.,Ross, I. M. (2001). Costate estimation by a Legendre pseudospectral method. Journal of Guidance, Control, and Dynamics, Vol. 24, No. 2, 270–277. 10. Hager, W. W. (2000). Runge-Kutta methods in optimal control and the transformed adjoint system. Numerische Mathematik, Vol. 87, pp. 247–282. 11. Hartl, R. F., Sethi, S. P., Vickson, R. G. (1995). A survey of the maximum principles for optimal control problems with state constraints. SIAM Review, Vol. 37, No. 2, 181–218. 12. Mordukhovich, B. S., Shvartsman, I. (2002). The approximate maximum principle for constrained control systems. Proc. 41st IEEE Conf. on Decision an Control, Las Vegas, NV. 13. Ross, I. M., Fahroo, F. (2001). Convergence of pseudospectral discretizations of optimal control problems. Proc. 40th IEEE Conf. on Decision and Control, Orlando, FL. 14. Ross, I. M., Fahroo, F. (2002). A perspective on methods for trajectory optimization,” Proc. AIAA/AAS Astrodynamics Specialist Conference, Monterey, CA, Invited Paper, AIAA-2002-4727. 15. Ross, I. M., Fahroo, F. (2002). Pseudospectral methods for optimal motion planning of diﬀerentially ﬂat systems, Proc. 41st IEEE Conf. on Decision and Control, Las Vegas, NV. 16. Ross, I. M., Fahroo, F., (2002). User’s manual for DIDO 2002: A MATLAB application package for dynamic optimization,” NPS Technical Report AA-02002, Department of Aeronautics and Astronautics, Naval Postgraduate School, Monterey, CA. 17. Trefethen, L. N., (2000). Spectral Methods in MATLAB, SIAM, Philadelphia, PA.

Minimax Nonlinear Control under Stochastic Uncertainty Constraints Cheng Tang and Tamer Ba¸sar Coordinated Science Laboratory, University of Illinois, 1308 W. Main Street, Urbana, Illinois 61801-2307, USA, {cheng, tbasar}@control.csl.uiuc.edu

Summary. We consider in this paper a class of stochastic nonlinear systems in strict feedback form, where in addition to the standard Wiener process there is a normbounded unknown disturbance driving the system. The bound on the disturbance is in the form of an upper bound on its power in terms of the power of the output. Within this structure, we seek a minimax state-feedback controller, namely one that minimizes over all state-feedback controllers the maximum (over all disturbances satisfying the given bound) of a given class of integral costs, where the choice of the speciﬁc cost function is also part of the design problem as in inverse optimality. We derive the minimax controller by ﬁrst converting the original constrained optimization problem into an unconstrained one (a stochastic diﬀerential game) and then making use of the duality relationship between stochastic games and risksensitive stochastic control. The state-feedback control law obtained is absolutely stabilizing. Moreover, it is both locally optimal and globally inverse optimal, where the ﬁrst feature implies that a linearized version of the controller solves a linear quadratic risk-sensitive control problem, and the second feature says that there exists an appropriate cost function according to which the controller is optimal.

1 Introduction There has been considerable research in the past decade on the subject of optimal control for system regulation and tracking under various types of uncertainty. The types of uncertainty include among many others additive exogenous disturbances, lack of knowledge about the system model, and time varying dynamics. There are two prominent approaches to such problems, namely H ∞ control and game theory. In H ∞ control, the system consists of both a (deterministic) nominal model and a (deterministic) uncertainty model. The design objective is to achieve a certain form of disturbance attenuation with respect to the uncertainty. In the game-theoretic approach, the uncertainty

Research supported in part by NSF through Grant ECS 02-2J48L.

W. Kang et al. (Eds.): New Trends in Nonlinear Dynamics and Control, LNCIS 295, pp. 343–361, 2003.

c Springer-Verlag Berlin Heidelberg 2003

344

C. Tang and T. Ba¸sar

is taken as an adversary whose objective is to counteract whatever action the controller takes, e.g. in a standard zero-sum game, the designer acts as a minimizing player while the uncertainty acts as a maximizing player, thus the name of minimax control. Both approaches have been under intense investigation, with their respective results and connections established clearly in [1]. According to one approach, the uncertainties in the dynamical system are just random noise. The well-known linear-quadratic-Gaussian (LQG) optimal control problem is just one example, where the uncertainty is modeled as exogenous (Gaussian) noise. The case of dynamic uncertainty (with the possibility of non-Gaussian noise) can be formulated as a minimax type optimization problem, such as the robust version of the linear quadratic regulator (LQR) approach to state feedback controller design given in [14, 15]. More generally, a robust version of the LQG technique was discussed in [11, 12, 16, 17], where the concept of an uncertain stochastic system was introduced. Again, the problem is of the minimax type and it involves construction of a controller which minimizes the worst-case performance, with the system uncertainty satisfying a certain stochastic uncertainty constraint. This constraint is formulated in the framework of relative entropy (or Kullback-Leibler divergence measure) by restricting the relative entropy between an uncertainty probability measure related to the distribution of the uncertainty input and the reference probability measure. One advantage of such an uncertainty description is that it allows for stochastic uncertainty inputs to depend dynamically on the uncertainty outputs. In addition, by making use of the duality relationship between a stochastic game and the risk-sensitive stochastic control (RSSC) problem [3, 4, 13], it was possible to synthesize the robust LQG controller from the associated algebraic or diﬀerential Riccati equations (from a certain specially parameterized RSSC problem). In the inﬁnite horizon case of [17], the uncertainty was described by an approximating sequence of martingales, and it was shown that H ∞ norm-bounded uncertainty can be incorporated into the proposed framework by constructing a corresponding sequence of martingales. The controller proposed in such a problem thus guarantees an optimal upper bound on the time-averaged performance of the closed-loop system in the presence of admissible uncertainties. A natural though not immediate extension of such a methodology would be to stochastic nonlinear systems, which is the subject of this paper. Here we consider a particular class of stochastic nonlinear systems in strict-feedback form, where the uncertainty satisﬁes a stochastic integral quadratic constraint. The minimax optimization is formulated in the inﬁnite horizon, and the objective is to seek a state-feedback controller which guarantees an optimal upper bound on the time-averaged performance of the closed-loop system in the presence of admissible uncertainties. As in [17], we consider the time-average properties of the system solutions. Therefore, the admissible uncertainty inputs need not be L2 [0, ∞)−integrable. Additionally, using the newly developed locally optimal design and stochastic backstepping technique [2, 6], we are able

Minimax Nonlinear Control under Stochastic Uncertainty Constraints

345

to impose additional speciﬁcations on the minimax state-feedback controller, namely one that minimizes over all state-feedback controllers the maximum (over all disturbances satisfying a given bound) of a given class of integral costs, where the choice of the speciﬁc cost function is also part of the design problem as in inverse optimality. One of the main contributions of this paper is to generalize the earlier considered linear-quadratic stochastic minimax optimization problem to stochastic nonlinear systems in strict feedback form. The uncertainty input in the system model is expressed as an integral quadratic one, which is equivalent to the relative entropy constraint formulated in [16, 17]. Secondly, by converting the original problem into an unconstrained stochastic diﬀerential game and RSSC problem, we are able to construct a state-feedback control law that is both locally optimal and globally inverse optimal, i.e. in the nonlinear system design, we are able to guarantee a desired performance for the linearized system dynamics, while also ensuring global stability and global inverse optimality for an a posteriori cost functional of the nonlinear closed-loop system. The organization of the paper is as follows. In the next section, the stochastic strict-feedback system and the related inﬁnite-horizon constrained minimax optimization problem formulation is given, with the notions of local optimality and global inverse optimality introduced. In section 3, we illustrate the complete procedure of constructing the minimax state-feedback control law. The paper ends with the concluding remarks of section 4, and two appendices.

2 Problem Formulation Consider the following stochastic system in strict-feedback form: dx1 (t) = [x2 (t) + f1 (x1 (t)) + ξ1 (t)]dt + h 1 dwt .. .. . .

dxn−1 (t) = [xn (t) + fn−1 (x[n−1] (t)) + ξn−1 (t)]dt + h n−1 dwt dxn (t) = [fn (x[n] (t)) + b(x[n] (t))u(t) + ξn (t)]dt + h n dwt

(1)

where x = (x1 ... xn ) ∈ Rn is the state, x[k] = (x1 ... xk ) , 1 ≤ k ≤ n, denotes the subvector from x consisting of its ﬁrst k components, u is the scalar control input, ξ = (ξ1 ... ξn ) ∈ Rn is the unknown disturbance input, and w ∈ Rr is a standard vector Wiener process. The underlying probability space is the triple (Ω, F, P), where the sample space Ω is C([0, ∞), Rr ), and the probability measure P is deﬁned as the standard Wiener measure on C([0, ∞), Rr ). We equip the sample space with a ﬁltration, i.e. a nondecreasing family {Ft , t ≥ 0} of sub-σ-ﬁelds of F : Fs ⊆ Ft ⊆ F for 0 ≤ s < t < ∞, which has been completed by including all sets of probability zero. This ﬁltration can be thought of as the ﬁltration generated by the mapping {Πt : C([0, ∞), Rr ) → Rr , Πt (w(·)) = w(t), t ≥ 0} [3, 17]. Here, the control input u

346

C. Tang and T. Ba¸sar

and the uncertainty input ξ are Ft -adapted with their speciﬁc structure given later in the section. We also assume that the following conditions (on system dynamics) hold. Assumption 1. The functions fi : Ri → R, i = 1, · · · , n, are C ∞ in all their arguments (or simply are smooth functions), with fi (0) = 0, i = 1, · · · , n. The nonlinear function b : Rn → R is C 2 and b(x) > 0, ∀x ∈ Rn . Furthermore, H = (h1 ... hn ) , with HH being positive deﬁnite, i.e. HH > 0. The ﬁrst part of the conditions above is a standard smoothness assumption for this class of nonlinear systems; the condition imposed on fi at x = 0 is to assure that the origin is an equilibrium point of the deterministic (unperturbed) part of the system. Note that system (1) can also be written compactly as dx = [f (x) + G(x)u + ξ]dt + Hdwt where

 x2 + f1 (x[1] )  x3 + f2 (x[2] )    f (x) =  , ..   .





  G(x) =  

fn (x[n] )

0 .. . 0 b(x[n] )

(2)    . 

(3)

Deﬁne the uncertainty output process zi (t) ∈ Rpi , i = 1, ..., L, as zi (t) = Ci x(t) + Di u(t)

(4)

where Ci ∈ Rpi ×n , Di ∈ Rpi ×1 , i = 1, ..., L. To facilitate the exposition, we assume that CiT Di = 0, i = 1, ..., L,

(5)

and denote by Gt the ﬁltration generated by the uncertainty output processes, i.e. Gt = σ{zi (s), 0 ≤ s ≤ t, i = 1, ..., L}. Clearly, Gt ⊂ Ft . Adopting the stochastic uncertainty model in [17], we make the following assumption. Assumption 2. The disturbance input ξ(t) ∈ Rn is a Gt -adapted process with the property that 6 t 1 2 E[exp |ξ(s)| ds ] < ∞ (6) 2 0 for all t > 0, which further satisﬁes the following bounds 6 1 T (|zi (s)|2 − |N 1/2 ξ(s)|2 )ds ≥ 0, P-a.s., i = 1, ..., L, lim inf T →∞ T 0 where N := (HH )−1 .

(7)

Minimax Nonlinear Control under Stochastic Uncertainty Constraints

347

Denote by Γ the set of all processes ξ such that Assumption 2 holds and system (1) admits a unique solution. Note that the nonlinear system (1) together with the above assumptions completely describe an uncertain stochastic system, in the same way as in robust control formulation, where {ξ, u} are taken as input signals and {z, y := x} as the outputs. The mapping from uncertainty output z to the uncertainty (or disturbance) input ξ represents the stochastic uncertainty model adopted by the designer, just as in deterministic H ∞ theory. In its current form, the stochastic nonlinear system (1) is driven by both the uncertainty input ξ and the additive noise input described by the Wiener process wt . This allows for the modeling of both additive disturbances and unmodeled dynamics, which may be the case in many realistic situations, and we interpret ξ as dynamic uncertainty input and w as exogenous uncertainty input. Furthermore, the uncertainty description in the form of stochastic uncertainty constraint (7) allows for the stochastic uncertain input ξ to depend dynamically on the uncertainty output z, which implies that it may depend on the applied control input u, thus giving rise to a constrained minimax optimization problem later in the section. The above assumption imposes the constraint in the form of a bound on the power of disturbance in terms of the power of the uncertainty output, which can be regarded as a generalization of standard uncertainty models used in the literature, e.g. it was shown in [17] that the standard H ∞ normbounded linear time-invariant (LTI) uncertainty would satisfy a similar form of constraint, in which case ξ, z are in fact L2 [0, ∞)-integrable, and it readily leads to (7). In addition, we note that the uncertainty description in (7) is similar to the form of integral quadratic constraint (IQC) in deterministic formulations, and the condition (7) imposes an upper bound on the “energy” of the process ξ [14, 18]. With the given dynamic equation (1) under stochastic uncertainty constraint, the admissible control laws are Ft −adapted state-feedback policies and are chosen as u(t) = µ(t, x(t)), µ ∈ U, where U is the set of all locally Lipschitz continuous state-feedback policies. We now consider the following cost functional 6 T 1 J(µ; ξ) = lim sup E[ q(x(s)) + r(x(s))u(s)2 dt)] (8) T T →∞ 0 where q(·) and r(·) are some positive deﬁnite continuous functions. One of our goals is to obtain a closed-form minimax solution with respect to J(µ; ξ) subject to the stochastic uncertainty constraint (Assumption 2), i.e. a µ∗ ∈ U such that inf sup J(µ; ξ) = sup J(µ∗ ; ξ).

µ∈U ξ∈Γ

ξ∈Γ

(9)

348

C. Tang and T. Ba¸sar

This is a particular type of a stochastic zero-sum game where the controller acts as the minimizing player, and the disturbance input ξ acts as the maximizing player. Note that in this constrained stochastic game, the minimizing player imposes restrictions on the choice of the strategy available to the maximizing player through the constraint. This observation reveals a major diﬀerence between the current formulation and the standard game-type optimization problem related to the worst-case LQG control design. It is well-known that such an optimization problem is associated with a corresponding Hamilton-Jacobi-Isaacs (HJI) equation whose solution yields the optimal value of the game, whenever it exists. Such an endeavor is generally not feasible, however, particularly given the nonlinearity of the system dynamics (1). This infeasibility of solving the HJI equation has motivated the development of the inverse optimality design. Deﬁnition 1. A state-feedback control law µ ∈ U is globally inverse optimal for the constrained minimax problem (9) if it achieves the optimal value with respect to the cost function (8) for some positive-deﬁnite continuous functions q(·) and r(·). Inverse optimality exploits the fact that an HJI equation with given cost function (8) is only a suﬃcient condition for achieving optimality and robustness. In the inverse optimal design, the ﬂexibility in the choices of q(·) and r(·) opens up the possibility of a closed-form solution to the HJI equation within some loose constraints on the desirable performance. More speciﬁcally, the inverse optimal approach enables us to design robust control laws without solving an HJI equation directly, but implicitly from an iterative design process. In addition to global inverse optimality for the nonlinear system, we also wish to achieve a better performance, namely local optimality, for a corresponding linearized system with respect to some nonnegative quadratic functions x Qx and Ru2 in place of q(·) and r(·)u2 in (8). Toward this end, we rewrite system (1) as: ˜ dx = [Ax + f˜(x) + Bu + G(x)u + ξ]dt + Hdwt

(10)

where A = fx (0), B = G(0) := ( 0 · · · 0 b0 ) , with obvious deﬁnitions for the ˜ Denote the linearized versions of x and u by x and perturbation terms f˜, G. u , respectively. Then, the linearized system is given by dx = [Ax + Bu + ξ]dt + Hdwt .

(11)

Note that (A, B) is a controllable pair by the structure of these matrices. Given a nonnegative-deﬁnite Q such that (A, Q) is observable, we consider the following cost functional J (µ; ξ) = lim sup T →∞

6 T 1 E[ (x Qx + Ru2 )dt)]. T 0

(12)

Minimax Nonlinear Control under Stochastic Uncertainty Constraints

349

The associated minimax optimization problem here is inf sup J (µ; ξ) = sup J (µ∗ ; ξ)

µ ∈U ξ∈Γ

ξ∈Γ

(13)

for which we seek to obtain an explicit, closed-form solution. We have the following deﬁnition. Deﬁnition 2. Consider the stochastic nonlinear system (1) with its linearized dynamics (11). A globally inverse optimal control law µ ∈ U is locally optimal if its linearized version u (t) = µ (t, x(t)), µ ∈ U, is optimal with respect to the constrained minimax problem (13), where the cost function is given by (12) with 2 ∂ 2 q(x) 22 r(0) = R, = Q, (14) ∂x2 2x=0 where R > 0 and Q ≥ 0 are a priori ﬁxed. This type of a control design problem was ﬁrst introduced in [6] for deterministic systems, and then extended to stochastic systems under a risksensitive criterion in [2], in both cases without the uncertainty constraint (7). To carry out such a design, normally one ﬁrst solves the linearized optimization problem (13), and then applies a nonlinear coordinate transformation to construct a globally inverse optimal control law, as well as the non-negative cost terms q(·), r(·)u2 , subject to the local optimality condition (14). To obtain the solution to the constrained minimax problem (9) in this paper, we ﬁrst convert it into a formulation in terms of the relative entropy (or the Kullback-Leibler divergence measure). We then show that this constrained optimization problem can be transformed into an unconstrained one, and by making use of the duality relationship between free energy and relative entropy [3, 4], this problem can be solved via a corresponding risk-sensitive stochastic control (RSSC) formulation. Utilizing a stochastic backstepping design as in [2], we obtain a closed-loop state feedback control law that is both locally optimal and globally inverse optimal under the stochastic uncertainty constraint.

3 The Main Result In this section, we present the construction of the state-feedback controller in two steps. First, using Girsanov’s theorem and a duality relationship, we convert the original problem into an associated RSSC formulation. Next, a stochastic backstepping procedure is applied which leads to the construction of the desired control law. Minimax Optimization and the RSSC Problem Note that Assumption 2 ensures that ξ(t) satisﬁes a Novikov type condition [7, 8]. Let Σ = −H [HH ]−1 . For 0 ≤ T < ∞, we deﬁne

350

C. Tang and T. Ba¸sar

6

Q

w (t) = w(t) −

t

Σξ(s)ds, 0

and

6 ζ(t) = exp

t

ξ (s)Σ dw(s) −

0

1 2

6

t

|Σξ(s)|2 ds ,

0

where 0 ≤ t ≤ T . Then, it can be shown that (ζ(t), Ft ) deﬁnes a continuous Ft -adapted martingale with E[ζ(T )] = 1. From Girsanov’s theorem [7, 8], ζ(T ) deﬁnes a probability measure QT on the measurable space (Ω, FT ) by the equation T

QT (A) = E P [1A ζ(T )], ∀A ∈ FT ,

(15)

where the expectation is under the probability measure PT , the restriction of the reference probability measure P to (Ω, FT ). From this deﬁnition, the probability measure QT is absolutely continuous with respect to PT , QT PT . Furthermore, (wtQ , Ft ; 0 ≤ t ≤ T ) is a standard Brownian motion process on (Ω, FT ) and under the probability measure QT . We further note that with wt being the complete coordinate mapping process on Ω, F = B(C[0, ∞), Rr ), and P the associated Wiener measure on (Ω, F), there is a unique probability measure Q with the property that Q(A) = E[1A ζ(T )], ∀A ∈ FT , 0 ≤ T < ∞,

(16)

where the expectation is under the probability measure P. In addition, {wtQ , Ft ; 0 ≤ t < ∞} is a standard Brownian motion process on (Ω, F, Q) (see section 3.5 of [7], [8]). With the additional condition that ζ(t) is uniformly integrable, P and Q would be mutually absolutely continuous. Note that wtQ is thus deﬁned for all t ∈ [0, ∞), and there are corresponding probability measures {QT ; 0 ≤ T < ∞} deﬁned on FT , with the property that when Q is restricted to any FT it agrees with QT , i.e. QT (A) = E[1A ζ(T )], ∀A ∈ FT . Furthermore, the following consistency condition holds: QT (A) = Qt (A), ∀A ∈ Ft , 0 ≤ t ≤ T. Therefore, the family {Q}0≤T <∞ is consistent, and (16) deﬁnes well a ﬁnitely additive set function Q on the algebra ∪0≤T <∞ FT . We also note that (ζ(t), Ft ) is in fact the admissible martingale deﬁned in [17] in terms of the adopted admissible uncertainty model. It is deﬁned via a sequence of approximating martingales. Here, let τn = inf{t ≥ 0 : 9t |ξ(s)|2 ds > n} and ξn (t) = ξ(t)1{t≤τn } . Then, the processes 0 6 t 6 1 t ζn (t) = exp ξn (s)Σ dw(s) − |Σξn (s)|2 ds 2 0 0

Minimax Nonlinear Control under Stochastic Uncertainty Constraints

351

deﬁnes a sequence of approximating martingales, with corresponding probability measures QTn deﬁned on (Ω, FT ) as T

QTn (A) = E P [1A ζn (τn ∧ T )], ∀A ∈ FT . Since τn → ∞ with probability one as n → ∞, then ζn (T ) → ζ(T ) with probability one. This fact together with Assumption 2 implies that QTn ⇒ QT as n → ∞. Using the deﬁnition of the relative entropy (or the KullbackLeibler divergence measure) provided in appendix A, and the following explicit expression (see [3, 4]): T

h(QTn ||PT ) = E Qn [

1 2

6 0

T

T

|Σξn (s)|2 ds] = E Qn [

1 2

6

T ∧τn

|Σξ(s)|2 ds],

(17)

0

the condition for the ﬁniteness of relative entropy can be readily veriﬁed. Lastly, the inequality (7) in Assumption 2 implies the following relationship 6 1 1 QTn T E |zi (t)|2 dt − h(QTn ||PT ) ≥ 0 lim inf T →∞ T 2 0 for i = 1, ..., L, and for all n, which leads to the third condition of admissible martingale deﬁnition in [17]. As pointed out earlier in the section, the standard H ∞ norm-bounded LTI uncertainty would satisfy Assumption 2, in which case ξ, z are in L2 ([0, ∞), P), and the inequality (7) holds. Given this new formulation, we use the expectation E[·] as interpreted under the original probability measure P, whereas E Q [·] to denote the expectation under new probability measure Q. The system dynamics can then be described as dx1 = [x2 + f1 (x1 )]dt + h 1 dwtQ .. .. . .

dxn−1 = [xn + fn−1 (x[n−1] )]dt + h n−1 dwtQ dxn = [fn (x[n] ) + b(x[n] )u]dt + h n dwtQ

(18)

or dx = [f (x) + G(x)u]dt + HdwtQ

(19)

where wtQ is the standard Brownian motion (under probability measure Q), and the nonlinear dynamic system is still in strict feedback form. Similarly as in the previous section, the linearized system is given by dx = [Ax + Bu ]dt + HdwtQ .

(20)

Note that in comparison with (11), it has the same form of stochastic differential equations (SDEs), with the diﬀerence being that we now have the

352

C. Tang and T. Ba¸sar

uncertainty probability measure Q, and that the system is driven by a standard Brownian motion process wtQ . Using the fact that ξ(·) represents admissible uncertainty input associated with the uncertainty probability measures {Q}0≤T <∞ , and using the explicit expression for the relative entropy (17), the stochastic uncertainty constraint of Assumption 2 can also be expressed as follows: Assumption 2 . For the uncertainty probability measure {Q}0≤T <∞ associated with system (18), the following condition holds T 1 1 lim inf [E Q ( T →∞ T 2

6

T

|zi (s)|2 ds) − h(QT ||PT )] ≥ 0, i = 1, ..., L.

(21)

0

Note that h(QT ||PT ) characterizes a measure of the discrepancy between the probability measure Q and the reference probability measure P, both deﬁned on the same probability space (Ω, F). Denote by Ξ the set of probability measures Q such that Assumption 2 holds, and for 0 ≤ T < ∞, QT PT , h(QT ||PT ) < ∞. The elements of Ξ are called admissible probability measures. Note that the relative entropy functional is a convex function in Q, which implies that the set Ξ is convex and non-empty. Therefore, the stochastic system (18) under an admissible probability measure Q corresponds to the uncertain dynamical system (1) with disturbance input ξ. With system (18) and Assumption 2 , we consider the admissible control law µ ∈ U and the cost function 6 1 Q T q(x(s)) + r(x(s))u(s)2 dt)] J (µ) = lim sup E [ T →∞ T 0 Q

(22)

where q(·) and r(·) are the same as in (8). The original minimax problem (9) now becomes a constrained minimax optimization problem (or a stochastic game) with respect to J Q (µ), with the minimax control µ∗ given by sup J Q (µ∗ ) = inf sup J Q (µ).

Q∈Ξ

µ∈U Q∈Ξ

(23)

To obtain a solution to the above constrained optimization problem, we apply the stochastic S-procedure (or the Lagrange multiplier principle) as in [9, 16] to obtain an unconstrained minimax optimization problem. Using Lemma B.1 of appendix B, in terms of system (18), we deﬁne the cost functional JλQ (µ)

6 1 Q T = lim sup E [ (q(x(s)) + r(x(s))u(s)2 T →∞ T 0 (24) L L λi 2 T T |zi (s)| )dt) − ( + λi )h(Q ||P )] 2 i=1 i=1

Minimax Nonlinear Control under Stochastic Uncertainty Constraints

353

where λ = (λ1 ... λL ) ≥ 0 is a given constant vector of Lagrange multipliers, and Q ∈ P, the set of probability measures Q with the properties that QT PT and h(QT ||PT ) < ∞ for 0 ≤ T < ∞. Since λ is a Lagrange multiplier, one can equivalently have an unconstrained minimax optimization problem inf sup JλQ (µ).

µ∈U Q∈P

(25)

Similarly, for the linearized system (20) and Assumption 2 , we consider the admissible control law µ ∈ U and the cost function 6 T 1 x (s)Qx(s) + Ru(s)2 dt)] (26) JQ (µ) = lim sup E Q [ T →∞ T 0 where Q and R are given in (14). There is also a corresponding constrained minimax optimization problem (or a stochastic game) with respect to JQ (µ), with the minimax control µ∗ given by sup JQ (µ∗ ) = inf sup JQ (µ).

Q∈Ξ

µ ∈U Q∈Ξ

(27)

Application of Lemma B.1 yields an unconstrained minimax optimization problem Q inf sup Jλ (µ)

µ ∈U Q∈P

(28)

with the following cost functional 6 T 1 Q Jλ (µ) = lim sup E Q [ (x (s)Qx(s) + Ru(s)2 T →∞ T 0 (29) L L λi |zi (s)|2 )dt) − ( λi )h(QT ||P T )] + 2 i=1 i=1 where λ ≥ 0 and Q ∈ P. Let Vλ denote the optimal value of the constrained minimax problem (25), and deﬁne the set Λ as Λ := {λ : λ ≥ 0 such that Vλ < ∞}. L Let θ = 2/( i=1 λi ), and associated with system (18), deﬁne the following risk-sensitive cost function . 5 6 T θ 2 (λ (µ) = lim sup q˜(t, x(t), u(t); λ)dt)] (30) ln E Q exp[ 2 0 T →∞ θT L where q˜(t, x, u; λ) = q(x)+r(x)u2 + i=1 λ2i |zi |2 . This is a standard RSSC problem for strict-feedback system (18) under probability measure Q. Similarly, for the linearized system (20), there is a corresponding LEQG cost function

354

C. Tang and T. Ba¸sar

2 ln E Q (λ (µ) = lim sup T →∞ θT

.

θ exp[ 2

6

T

5 ˜ x(t), u(t); λ)dt] Q(t,

(31)

0

˜ x, u; λ) = x (t)Qλ x(t) + Rλ u(t), Qλ = Q + L λi C Ci , and where Q(t, i=1 2 i L Rλ = R + i=1 λ2i Di Di . Note that in the above RSSC and LEQG problems, the probability measure Q is assumed to be as given, with wtQ being a standard Brownian motion. The expectation in the cost functional (30) is considered under the ﬁxed probability measure Q. The optimal state-feedback controller µ∗ ∈ U is deﬁned such that (λ (µ∗ ) = inf (λ (µ)

(32)

µ∈U

with the optimal value denoted by (∗λ , while the optimal linearized statefeedback controller µ∗ ∈ U is given by (λ (µ∗ ) = inf (λ (µ)

(33)

µ ∈U

with the optimal value denoted by (∗λ . Analogous to the deﬁnitions given in the previous section, one can also deﬁne local optimality and global inverse optimality with respect to the standard RSSC problem, as in [2]. Utilizing the duality relationship between free energy and relative entropy established in [3, 4], we arrive at the following conclusion. First, we consider the linearized system (11) with stochastic uncertainty input satisfying Assumption 2. Theorem 1. Consider the linear stochastic system (11)-(7) with associated minimax cost function (12), as well as the system (20) with associated risksensitive cost function (31). Suppose that the set Λ is nonempty. Then, the following relationship holds: inf sup J (µ ; ξ) = inf sup JQ (µ ) = inf (∗λ ,

µ ∈U ξ∈Γ

µ ∈U Q∈Ξ

λ∈Λ

where the optimal control law µ∗ is the minimax optimal control policy, sup J (µ∗ ; ξ) = (∗λ∗ .

ξ∈Γ

Furthermore, the closed-loop system is absolutely stable. The proof of the theorem follows from [17] and is given in appendix B. Next, the nonlinear stochastic system (1) is considered, and a similar type conclusion can be arrived at. Theorem 2. Consider the stochastic system (1)-(7) with associated minimax cost function (8), as well as the system (18) with associated risk-sensitive cost function (30). Then, the following hold: (i) Suppose that the set Λ is nonempty, and that λ∗ attains the inﬁmum (in the associated RSSC problem). Let µ∗ be the corresponding optimal control law. Then, the following relationship holds:

Minimax Nonlinear Control under Stochastic Uncertainty Constraints

355

inf sup J(µ; ξ) = inf sup J Q (µ) ≤ sup J(µ∗ ; ξ) ≤ inf (∗λ = inf (λ (µ∗ ).

µ∈U ξ∈Γ

µ∈U Q∈Ξ

λ∈Λ

ξ∈Γ

λ∈Λ

Furthermore, the closed-loop system is absolutely stable. (ii) Conversely, if there exists a minimax optimal controller µ ˜ ∈ U such µ; ξ) < ∞, then the set Λ is nonempty. Moreover, the following that supξ∈Γ J(˜ relationship holds: inf (∗λ = inf (λ (µ∗ ) ≤ sup J(˜ µ; ξ).

λ∈Λ

λ∈Λ

ξ∈Γ

The proof of the theorem follows from [3, 16, 17] and is given in appendix B. In addition, using the above theorems, we also have the following result with regard to the desired local optimality and global inverse optimality of the control policy. Theorem 3. Consider the stochastic system (1)-(7) with linearized dynamics (11), and associated minimax cost functions (8)-(12), as well as the system (18) with linearized dynamics (20), and associated risk-sensitive cost functions (30)-(31). The following conclusions hold: (i) Let the set Λ be nonempty, and the control law µ∗ be locally optimal with respect to the LEQG problem (31). Then, it is locally optimal with respect to the constrained minimax problem (12), i.e. inf sup J (µ ; ξ) = J (µ∗ ; ξ) = inf (λ (µ∗ ).

µ ∈U ξ∈Γ

λ∈Λ

(ii) Let the set Λ be nonempty, and the control law µ∗ be globally inverse optimal with respect to the RSSC problem (30). Then, it is globally inverse sub-optimal with respect to the constrained minimax problem (8), i.e. inf sup J(µ; ξ) ≤ sup J(µ∗ ; ξ) ≤ inf (λ (µ∗ ).

µ∈U ξ∈Γ

ξ∈Γ

λ∈Λ

Furthermore, the closed-loop system is absolutely stable. Locally Optimal Controller Design Using Theorem 3 and the stochastic backstepping procedure from [2], we can now construct a state-feedback controller for the constrained minimax optimization problem of section 2. First, the linearized risk-sensitive design is considered, and then the backstepping procedure is extended to the nonlinear case with global inverse optimality. From standard LEQG theory, since (A, B) is controllable and (A, Qλ ) is observable, there exists a threshold value θ∗ for θ, such that for all θ < θ∗ , the LEQG problem (31) admits a unique solution, given by

356

C. Tang and T. Ba¸sar

µ∗ (x ) = −Rλ−1 B P x

(34)

where P is the minimal positive-deﬁnite solution of the generalized algebraic Riccati equation (GARE): A P + P A − P BRλ−1 B − θHH P + Qλ = 0. (35) Furthermore, the feedback matrix A − Rλ−1 BB P is Hurwitz. For θ > θ∗ , (35) does not admit any nonnegative-deﬁnite solution, and the cost is unbounded. For the linearized risk-sensitive design, we let V (x) = x P x , and apply a coordinate transformation z = Lx based on a Cholesky decomposition of P , P = L ∆L, where ∆ is a diagonal matrix consisting of the positive eigenvalues, δi ’s, of P , and L is a lower triangular matrix. This brings the equation (35) into the form ¯λ = 0 ¯H ¯ ∆ + Q (36) A¯ ∆ + ∆A¯ − ∆ BRλ−1 B − θH where the barred quantities are the corresponding representations in the new ¯ = LH and Q ¯ λ = (L )−1 Qλ L−1 . coordinate system with A¯ = LAL−1 , H This gives rise to the transformed subsystem as    0    ¯ [i] dwt (37) dz[i] = A¯[i] z[i] +  ...  dt + H zi+1 for 1 ≤ i < n, where z[i] = L[i] x[i] . Deﬁning the value function recursively as V¯ (z[i] ) = z[i] ∆[i] z[i] = V¯ (z[i−1] ) + δi zi2 , V¯ (z[0] ) = 0,

one arrives at the following equation ¯ [i] = 0. ¯ [i] H ¯ ∆[i] + Q A¯ [i] ∆[i] + ∆[i] A¯[i] + θ∆[i] H [i] This iterative process yields a formulation of the linearized dynamics in the new coordinate system as ¯ ¯ dz(t) = (Az(t) + Bul (t))dt + Hdw(t), and the optimal linearized control law is given by µ ¯l (z) = −Rλ−1 B ∆z. For the nonlinear system (in strict feedback form), we follow the same backstepping procedure, but with the perturbation terms in (1) brought in. The virtual controls are now used to cancel out the nonlinearities and to cope with the second-order terms resulting from Itˆ o diﬀerentiation. The transformation zi = φ(x[i] ) is now a nonlinear mapping and is given by our choice of the virtual control law at each step. After the ﬁnal step, this construction results in the lower-triangular diﬀeomorphism z = Φ(x). The linear part of ˜ ˜ this diﬀeomorphism is Lx, i.e. z = Lx + Φ(x), where Φ(x) contains only the higher order terms.

Minimax Nonlinear Control under Stochastic Uncertainty Constraints

357

At each step with i = 1, ..., n, let zi = φi (x[i] ) = xi − α ¯ i−1 (z[i−1] ), where α ¯ i−1 (z[i−1] ) = α ¯ [i−1] z[i−1] + α ˆ i−1 (z[i−1] ). Here, the linear term α ¯ [i−1] z[i−1] comes directly from the linearized backstepping design, while the nonlinear transformation term α ˆ [i−1] (·) is yet to be designed. We choose the value fun ∆[i] z[i] = V¯i−1 + δi zi2 , the same as in the linearized design. ction as V¯i = z[i] Then, through the iterative design process, we arrive at the following expression for the z[i] -subsystem by making use of the corresponding results at the previous (i − 1)-th step, i.e.    0    .. ¯ [i] dwt , dz[i] = A¯[i] z[i] + fˆ[i] (z[i] ) +   dt + H . ¯i xi+1 − α where the nonlinear dynamics fˆ[i] is given by ∂α ¯ i−1 ˆ fˆi (z[i] ) = α ˆ i + α[i] Ψˆ[i] (z[i] ) + f˜i (Φ−1 f[i−1] [i] ) − ∂z [i−1]    0 ∂2α ˆ i−1 ¯ ∂α ˆ i−1  ¯  ..  1 ¯ − H[i−1] H[i−1] . A[i−1] z[i−1] +  .  − Tr 2 ∂z[i−1] 2 ∂z[i−1] zi Furthermore, the Itˆ o diﬀerential for V¯i is then given by ¯ [i] H ¯ [i] dwt + Tr[H ¯ ∆[i] ] − θz ∆[i] H ¯ [i] H ¯ ∆[i] z[i] ∆[i] H dV¯i = 2z[i] [i] [i] [i] ¯ ˆ −z[i] Q[i] z[i] + 2zi δi [fi + (xi+1 − α ¯ i )] dt. We select α ˆ i to cancel out the nonlinearities, i.e., ∂α ¯ i−1 ˆ α ˆ i = −α[i] Ψˆ[i] (z[i] ) − f˜i (Φ−1 f[i−1] [i] ) + ∂z    [i−1] 0 ∂2α ˆ i−1 ¯ ∂α ˆ i−1  ¯  ..  1 ¯ + H[i−1] H A[i−1] z[i−1] +  .  + Tr [i−1] . 2 ∂z[i−1] 2 ∂z[i−1] zi With such a choice and by letting zi+1 = xi+1 − α ¯ i = 0, V¯i satisﬁes the following HJI equation θ ¯ [i] H ¯ V¯iz + q¯i (z[i] ) + 1 Tr[V¯izz H ¯ [i] H ¯ ] = Ji , V¯iz A¯[i] z[i] + fˆ[i] (z[i] ) + V¯iz H [i] [i] 4 2 (38) where q¯i (·) is the nonlinear cost term in the associated RSSC problem for the z[i] -subsystem, and Ji is a positive constant, representing the optimal long term average risk-sensitive cost. Note the presence of an additional term

358

C. Tang and T. Ba¸sar

2 ¯ [i−1] H ¯ ˆi (·), which is a result of the Itˆ (1/2)Tr (∂ 2 α ˆ i−1 /∂z[i−1] )H o [i−1] in α diﬀerentiation rule. ∆z[n] = V¯n−1 + δn zn2 , and after the nonlinear transformation With V¯ = z[n] z = φ(x), the original nonlinear dynamics is now described by ¯ + fˆ(z) + Bu)dt + Hdw ¯ t dz = (Az in the new coordinate system, and the Itˆ o diﬀerential of V¯ (z) is given by ¯ t + Tr[H ¯H ¯ ∆] + z ∆(B¯ dV¯ = 2z ∆Hdw r−1 (z)B ¯H ¯ )∆z − z Qz ¯ + 2zn δn [fˆn + u] dt. −θH To achieve global inverse optimality, we select functions r¯(z) and q¯λ (z) through a completion of squares, i.e. ¯ λ z + (¯ q¯(z) = z Q r−1 (z) − Rλ−1 )δn2 zn2 − 2zn δn fˆn (z) and

r¯(z) =

(39)

(Rλ−1 + σ ¯ (z))−1 : σ ¯ (z) ≥ 0 Rλ :σ ¯ (z) < 0

¯ −1 δ −1 η¯ + η¯1 Q ¯ −1 η¯ , and η¯1 , η¯2 are approwhere σ ¯ (z) = 2δn−1 η¯2 − 2¯ q1 Q 1 [n−1] n [n−1] 1 priately deﬁned functions. Then, the state-feedback control law −1 u=µ ¯λ (z) = −R¯λ (z)B ∆z

is globally inverse optimal with respect to the RSSC problem (30). For more details of the stochastic backstepping design, we refer to [2]. Note that the diﬀeomorphism z = Φ(x) is a bijective mapping from the transformed state z to the original state x, and thus x = Φ−1 (z) is bijective. The state-feedback controller u = µλ (x) = µ ¯λ (Φ(x)) applied to system (18) thus achieves both local optimality and global inverse optimality, with cost terms q(·), r(·). By Theorem 3, it is also globally inverse optimal with respect to the constrained minimax problem (8).

4 Conclusion In this paper, we have considered the constrained minimax optimization problem for stochastic nonlinear systems in strict-feedback form. The uncertainty in the system satisﬁes stochastic integral quadratic constraints, or can be expressed as a bound on the relative entropy. Using the Girsanov’s theorem and the duality relationship between stochastic game and RSSC control problem, we have obtained a state-feedback controller that is both locally optimal and globally inverse optimal.

Minimax Nonlinear Control under Stochastic Uncertainty Constraints

359

Appendix A. Relative Entropy This appendix presents the deﬁnition of relative entropy (or the KullbackLeibler divergence measure) as well as the duality relationship between free energy and relative entropy that is exploited in the paper. The results are taken from [17]; for more details and proof, we refer to [3, 7, 17]. Let (Ω, F) be a measurable space, and let P(Ω) be the set of probability measures on (Ω, F). Deﬁnition A.1 Given any two probability measures Q, P ∈ P(Ω), the relative entropy h(Q||P ) of the probability measure Q with respect to the probability measure P is deﬁned as . dQ E Q [log dQ ] if Q P and log ∈ L1 (dQ), dP dP h(Q||P ) = (40) +∞ else. In the above deﬁnition, dQ dP is the Radon-Nikodym derivative of the probability measure Q with respect to the probability measure P [7, 8]. Note that the relative entropy is a convex, lower semicontinuous functional of Q. Furthermore, the following Legendre-type duality relationship holds. Lemma A.2 ([3]) Let P ∈ P(Ω) and ψ : Ω → R be a measurable function, denote E(ψ) = ln(ψP (dω)). (i) For any Q ∈ P(Ω),

6

h(Q||P ) =

sup eψ ∈L1 (Ω,F ,P ), ψ bounded below.

{

ψQ(dω) − E(ψ)};

(41)

(ii) For any ψ bounded below, 6 E(ψ) =

sup

{

h(Q||P )<∞

ψQ(dω) − h(Q||P )}.

(42)

Moreover, if ψeψ ∈ L1 (Ω, F, P ), then the supremum in ( 41) is attained at Q∗ given by eψ dQ∗ =9 ψ . dP e P (dω)

Appendix B. This appendix presents the proofs of Theorem 1 and 2 in the paper. First, some results on the stochastic S-procedure are given, which will be used later in the proofs.

360

C. Tang and T. Ba¸sar

S-Procedure. Consider the following set of real-valued measurable functionals deﬁned on C([0, T ], Rn ) × Rn × Ω → R : g0 (x(·), x0 , ω), g1 (x(·), x0 , ω), . . . , gk (x(·), x0 , ω).

(43)

Associated with these functionals and the system (1), deﬁne the following functional on the set P, the set of probability measures with Q P and h(Q||P ) < ∞, G0 (Q) = E Q [g0 (x(·), x0 , ω)] G1 (Q) = E Q [g1 (x(·), x0 , ω)] − h(Q||P ) .. . Gk (Q) = E Q [gk (x(·), x0 , ω)] − h(Q||P ) where x(·) is the solution to (1) corresponding to the initial condition x(0) = x0 , and Q ∈ P. Lemma B.1 (Stochastic S-Procedure [16]) Suppose that system (1) and the functionals (43) are such that the following condition is satisﬁed: G0 (Q) ≥ 0 for any Q ∈ P such that G1 (Q) ≥ 0, . . . , Gk (Q) ≥ 0. k Then there exist constants τ0 ≥ 0, τ1 ≥ 0, ..., τk ≥ 0, i=0 τi > 0, such that, for any probability measure Q ∈ P, τ0 G0 (Q) ≥

k

τi Gi (Q).

i=0

Furthermore, if there exists a probability measure Q ∈ P such that G1 (Q) > 0, . . . , Gk (Q) > 0, then there exist constants τi , i = 1, ..., L, such that τ0 = 1, and G0 (Q) ≥

k

τi Gi (Q).

i=0

Deﬁnition B.2. The stochastic uncertain system (1) (or (11)) with uncertainty satisfying (7) is said to be absolutely stable if there exist constants c > 0 such that for any admissible uncertainty input ξ(·) ∈ Ξ, 6 T 1 QT lim sup [E ( |x(t)|2 dt) + h(QT ||PT )] ≤ c. (44) T →∞ T 0 Proof of Theorem 1. Given the linear stochastic system (11) with uncertainty satisfying (7), using Lemma B.1, the constrained minimax optimization problem can be converted into an unconstrained optimization problem (27) involving the uncertainty probability measure Q and the Lagrange multiplier λ, i.e.

Minimax Nonlinear Control under Stochastic Uncertainty Constraints

361

sup J (µ ; ξ) = sup JQ (µ )

ξ∈Γ

Q∈Ξ

for any µ ∈ U such that the closed-loop system is absolutely stable. Then, following the proof of Theorem 3 in [17], and by utilizing the duality relationship of Lemma A.2, there is an associated LEQG problem (λ (µ ), with the optimal µ∗ being a linear state-feedback control law. The conclusion of the theorem follows by applying Theorem 4 of [17]. Proof of Theorem 2. The proof of the theorem follows the same line as in proof of Theorem 3 of [17], by making use of the duality relationship of Lemma A.2 as well as the stochastic uncertainty constraint (7) of Assumption 2. Acknowledgment. The ﬁrst author would like to thank Professor Renming Song for his comments and suggestions in the preparation of this paper.

References 1. Ba¸sar T, Bernhard P (1995) H ∞ -Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach. Birkhauser, Boston 2. Ba¸sar T, Tang C (2000) J. Optimization Theory & Applications 105:521–541 3. Dai Pra P, Meneghini L, Runnggaldier R (1996) Math. Control Signals Systems 9:303–326 4. Dupuis P, Ellis R(1997) A Weak Convergence Approach to Theory of Large Deviations. Wiley, New York 5. Dupuis P, James M R, Petersen I R (1998) Proceedings of the IEEE Conference on Decision and Control 3:2365–2370 6. Ezal K, Pan Z, Kokotovic P V (2000) IEEE Trans. on Automatic Control 45:260–271 7. Karatzas I, Shreve S E (1988) Brownian Motion and Stochastic Calculus. Springer-Verlag, New York 8. Oksendal O (1998) Stochastic Diﬀerential Equations: An Introduction with Applications. Springer-Verlag, New York 9. Luenberger D G (1969) Optimization by Vector Space Methods, Wiley, New York 10. Pan Z, Ba¸sar T (1996) SIAM J. Control Optim. 34:1734–1766 11. Petersen I R, James M R (1996) Automatica 32:959-972 12. Petersen I R, James M R, Dupuis P (2000) IEEE Trans. Automat. Control 45:398–412 13. Runolfsson T (1994) IEEE Trans. Automat. Control 39:1551–1563 14. Savkin A V, Petersen I R (1995) Internat. J. Robust Nonlinear Control 5:119– 137 15. Ugrinovskii V A, Petersen I R (1999) SIAM J. Control Optim. 37:1089–1122 16. Ugrinovskii V A, Petersen I R (1999) Math. Control Signals Systems 12:1–23 17. Ugrinovskii V A, Petersen I R (2001) SIAM J. Control Optim. 40:1189–1226 18. Yakubovich V A (1988) Systems Control Lett. 11:221–228

New Trends in Nonlinear Dynamics and Control, and their Applications (Lecture Notes in Control and Information Sciences)

Nonlinear Model Predictive Control: Towards New Challenging Applications (Lecture Notes in Control and Information Sciences)

Singular Control Systems (Lecture Notes in Control and Information Sciences)

New Directions and Applications in Control Theory (Lecture Notes in Control and Information Sciences)

Networked Control Systems (Lecture Notes in Control and Information Sciences)

Ensuring Control Accuracy (Lecture Notes in Control and Information Sciences)

L2-Gain and Passivity Techniques in Nonlinear Control (Lecture Notes in Control and Information Sciences)

Neural Networks in Multidimensional Domains: Fundamentals and New Trends in Modelling and Control (Lecture Notes in Control and Information Sciences)

Feedback Control, Nonlinear Systems, and Complexity (Lecture Notes in Control and Information Sciences)

Assessment and Future Directions of Nonlinear Model Predictive Control (Lecture Notes in Control and Information Sciences)

Dynamics, Bifurcations and Control (Lecture Notes in Control and Information Sciences)

Nonlinear and Adaptive Control: NCN4 2001 (Lecture Notes in Control and Information Sciences)

Nonlinear Control in the Year 2000: Volume 1 (Lecture Notes in Control and Information Sciences)

Linear and Nonlinear Iterative Learning Control (Lecture Notes in Control and Information Sciences) (v. 291)

Nonlinear Analysis and Synthesis Techniques for Aircraft Control (Lecture Notes in Control and Information Sciences)

Nonsmooth Impact Mechanics: Models, Dynamics and Control (Lecture Notes in Control and Information Sciences)

Advances in the Control of Nonlinear Systems (Lecture Notes in Control and Information Sciences)

Nonlinear Control in the Year 2000, Volume 2 (Lecture Notes in Control and Information Sciences)

Iterative Learning Control: Convergence, Robustness and Applications (Lecture Notes in Control and Information Sciences)

General Hybrid Orthogonal Functions and their Applications in Systems and Control (Lecture Notes in Control and Information Sciences)

Adaptive Dual Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

Chaos Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

Bifurcation Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

Nonlinear Observers and Applications (Lecture Notes in Control and Information Sciences)

New Directions in Nonlinear Observer Design (Lecture Notes in Control and Information Sciences)

Nonlinear Control Systems: An Algebraic Setting (Lecture Notes in Control and Information Sciences)

Control of Indefinite Nonlinear Dynamic Systems: Induced Internal Feedback (Lecture Notes in Control and Information Sciences)

Nonlinear Control Systems: An Algebraic Setting (Lecture Notes in Control and Information Sciences, Volume 242)

Nonlinear Control of Wheeled Mobile Robots (Lecture Notes in Control and Information Sciences)

Recent Advances in Control and Optimization of Manufacturing Systems (Lecture Notes in Control and Information Sciences)

Topics in Time Delay Systems: Analysis, Algorithms and Control (Lecture Notes in Control and Information Sciences)

New Trends in Nonlinear Dynamics and Control, and their Applications (Lecture Notes in Control and Information Sciences)

Nonlinear Model Predictive Control: Towards New Challenging Applications (Lecture Notes in Control and Information Sciences)

Singular Control Systems (Lecture Notes in Control and Information Sciences)

New Directions and Applications in Control Theory (Lecture Notes in Control and Information Sciences)

Networked Control Systems (Lecture Notes in Control and Information Sciences)

Ensuring Control Accuracy (Lecture Notes in Control and Information Sciences)

L2-Gain and Passivity Techniques in Nonlinear Control (Lecture Notes in Control and Information Sciences)

Neural Networks in Multidimensional Domains: Fundamentals and New Trends in Modelling and Control (Lecture Notes in Control and Information Sciences)

Feedback Control, Nonlinear Systems, and Complexity (Lecture Notes in Control and Information Sciences)

Assessment and Future Directions of Nonlinear Model Predictive Control (Lecture Notes in Control and Information Sciences)

Dynamics, Bifurcations and Control (Lecture Notes in Control and Information Sciences)

Nonlinear and Adaptive Control: NCN4 2001 (Lecture Notes in Control and Information Sciences)

Nonlinear Control in the Year 2000: Volume 1 (Lecture Notes in Control and Information Sciences)

Linear and Nonlinear Iterative Learning Control (Lecture Notes in Control and Information Sciences) (v. 291)

Nonlinear Analysis and Synthesis Techniques for Aircraft Control (Lecture Notes in Control and Information Sciences)

Nonsmooth Impact Mechanics: Models, Dynamics and Control (Lecture Notes in Control and Information Sciences)

Advances in the Control of Nonlinear Systems (Lecture Notes in Control and Information Sciences)

Nonlinear Control in the Year 2000, Volume 2 (Lecture Notes in Control and Information Sciences)

Iterative Learning Control: Convergence, Robustness and Applications (Lecture Notes in Control and Information Sciences)

General Hybrid Orthogonal Functions and their Applications in Systems and Control (Lecture Notes in Control and Information Sciences)

Adaptive Dual Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

Chaos Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

Bifurcation Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

Nonlinear Observers and Applications (Lecture Notes in Control and Information Sciences)

New Directions in Nonlinear Observer Design (Lecture Notes in Control and Information Sciences)

Nonlinear Control Systems: An Algebraic Setting (Lecture Notes in Control and Information Sciences)

Control of Indefinite Nonlinear Dynamic Systems: Induced Internal Feedback (Lecture Notes in Control and Information Sciences)

Nonlinear Control Systems: An Algebraic Setting (Lecture Notes in Control and Information Sciences, Volume 242)

Nonlinear Control of Wheeled Mobile Robots (Lecture Notes in Control and Information Sciences)

Recent Advances in Control and Optimization of Manufacturing Systems (Lecture Notes in Control and Information Sciences)

Topics in Time Delay Systems: Analysis, Algorithms and Control (Lecture Notes in Control and Information Sciences)

Recommend Documents